# Research that can’t fool itself. > How our market research with AI agents guards against fooling itself: blind controls, the index as it stood, changes declared first, both halves of history. Page: https://adjltechnology.com/notes/research-that-cannot-fool-itself/ Updated: 2026-11-12 Field note We research stock-market ideas working with AI agents, and a person decides at every step. The easy mistake in this work is a result that only looks good because of how it was tested. These are the checks we use against that. Method only, no numbers. - **Written by:** Daniel Laskowski - **Published:** November 12, 2026 - **Reading time:** 4 min People read the top layer, crawlers follow the middle one and machines parse the bottom one. ## A person decides at every gate. The agents write the code, run the studies and review each other’s work. A person sets each question. An agent asks focused questions and writes a design spec, and the workflow requires a person to approve the spec and the plan before any code is written. A person can also choose to shorten that path. A reviewer agent checks the finished work, and a person chooses which of its findings to act on. Nothing an agent finds is adopted on its own say-so. Whether a change goes forward is a person’s call, made after the results are in. ## Code is tested where the answer is known. Before a strategy meets real history, its code is tested on made-up prices where the right trades are known in advance. If the code doesn’t make exactly those trades, it’s wrong, whatever its results look like later. One test cuts the price history off at a given day and checks that nothing decided before that day changes. Code that quietly peeks at the future fails it. The agents run these tests as they build and again in review. ## Every idea faces a blind control. A signal can look good just because the market went up. So each idea is compared with a blind control on the same days. The control keeps the same exits and holding rules, and enters without the signal. If the idea doesn’t beat entering blind, the signal isn’t doing the work. Entries can bunch up on the same bad days for the whole market. The test allows for that, so one bad day isn’t counted as many separate wins. ## The index as it stood on each day. Test a strategy on today’s list of large companies and you’ve quietly picked the winners. The companies that shrank, failed or were bought out aren’t on today’s list. A test that never meets them looks better than real life would have been. So our later studies use the index as it stood on each day, with the companies that later left it still in. A strategy may buy a stock only on days it was really in the index. A separate study measured how much this matters, with companies that had dropped out of the market added back, some of them found by hand. Every test also charges trading costs. ## Changes are written down before the run. The easiest way to fool yourself is to try a lot of tweaks and keep the one that worked. So the candidates are fixed in writing before any data runs. When a recent study tuned the settings of textbook strategies, the full set of settings to try was written down before any test. When another tested changes to the indicator Loep offers, each candidate change was declared in its spec first. Near misses aren’t adopted after the fact. ## A result has to hold in both halves. Some ideas work in one stretch of history and nowhere else. To catch that, the history is split in two, and a result has to hold in both halves. Where a test uses random draws, it has to hold for every random seed too. When settings are tuned, each year of the test runs on settings chosen only from the years before it. The test never borrows from its own future. ## Failures go in the log. Most ideas fail. The findings log says so, in the same plain terms it uses for the ones that hold up. When an earlier conclusion turns out to be wrong, the log records the correction. A log of failures is what makes a success worth reading. It shows the bar every idea had to clear. ## Then it runs on paper. A person chooses what goes forward, from the research’s recommendations, after the testing. Automated paper-trading programs, built with our AI agents, then follow those strategies on paper after every market close. They’re plain rules-based programs. No AI model makes a decision when they run, and no order is ever sent to a broker. The programs run the research’s own code, or check that the published version still matches it. What survives can reach [Loep](https://adjltechnology.com/work/loep/) as an indicator, with its method and its weaknesses written out and its settings withheld. There are no numbers in this note, on purpose. Results are the easiest part of research to fool yourself with, and the method is the part you can judge. The [agentic research](https://adjltechnology.com/work/agentic-research/) case study walks through the whole process. ## Related - [How we research with AI agents](https://adjltechnology.com/work/agentic-research/) - [Loep](https://adjltechnology.com/work/loep/) - [AI Engineering](https://adjltechnology.com/services/ai-engineering/) ## More notes - [What AI crawlers read](https://adjltechnology.com/notes/what-ai-crawlers-read/) - [A build that refuses to overclaim](https://adjltechnology.com/notes/a-build-that-refuses-to-overclaim/) - [All field notes](https://adjltechnology.com/notes/) Start here ## Tell us what’s slow, or what can’t be found. One paragraph is plenty. Tell us what you want built, the step that costs the most, or the page that should rank and doesn’t. If you know why the obvious fix hasn’t worked, say that too. [Start a project](https://adjltechnology.com/start/) [daniel@adjlcapital.com](mailto:daniel@adjlcapital.com) [Email Daniel](mailto:daniel@adjlcapital.com) Daniel reads and answers these himself, usually within a day or two. No sequence, no drip, and nothing sold to anyone. [ADJL Technology](https://adjltechnology.com/) Today’s investment, tomorrow’s legacy. ADJL Technology builds AI systems, websites and apps, and the technical SEO that gets them found. Founded and led by Daniel Laskowski. [daniel@adjlcapital.com](mailto:daniel@adjlcapital.com) Built 2026-11-12 from commit 6afca01. No cookies, no trackers, no third-party scripts. ADJL Technology. Nothing on this site is an offer to sell or a solicitation of any security, and nothing here is investment advice. Descriptions of internal systems are general by design and omit parameters and thresholds.