In 2024, three researchers at New York University taught a small transformer to solve a hard algorithmic problem by substituting strings of meaningless dots for its written chain of thought. Its accuracy on a problem called 3SUM jumped from near-random to perfect. The experiment, by Jacob Pfau, William Merrill, and Samuel Bowman, was a deliberately extreme test of a question that now divides artificial intelligence research: when a model prints a step-by-step account of its reasoning, is that account actually describing the computation that produced the answer? The dots answered no, and a cascade of follow-up work has since shown the gap between the text and the machinery is wide, measurable, and consequential.
The stakes of the question rose quickly. Chain-of-thought prompting was discovered in 2022 as a way to make language models less boneheaded on logic and arithmetic: ask them to think step by step, and they did better. By 2024, OpenAI’s o1 had turned the trick into a product, training models to generate their own reasoning traces before answering. The traces were treated as more than a performance hack. They looked like an auditable paper trail of machine thought, the kind of record that would let a human check why a model reached a given conclusion. Reasoning models went on to take gold medals at the 2025 International Mathematical Olympiad, and in May 2026 OpenAI said a general-purpose reasoning model had solved the unit distance problem, a famous open question in geometry. The company did not release the model’s raw chain of thought, only a rewritten summary produced by two human experts with the help of another model, consistent with a policy it adopted in 2024 and that Google DeepMind and Anthropic also follow.
The dots experiment undercut the paper-trail picture at its foundation. Pfau, Merrill, and Bowman trained a 34-million-parameter Llama model on 3SUM, a task that asks whether any three numbers in a list sum to zero, and on a variant of 2SUM. Without intermediate tokens, the model scored 66 percent on the longer 3SUM inputs, indistinguishable from guessing. With a chain of thought written out as explicit pairwise sums, it reached 100 percent. With the chain of thought replaced by an equal number of dots, it also reached 100 percent. On 2SUM, the numbers were 95.1 percent for chain of thought, 93.6 percent for dots, and 78.7 percent with nothing in between. The tokens’ meaning was irrelevant; what mattered was that they existed, giving the network extra positions in which to carry out computation that no human could read. The paper’s title gives the phenomenon its name: hidden computation. The authors noted the uncomfortable implication for the field’s dominant alignment methods, which judge models by reading their output text: a model that can reason across meaningless tokens is doing computation that behavioral oversight cannot see.
Later work moved from the toy setting to real frontier models and quantified how little of a visible reasoning trace actually drives the answer. A 2025 study by Jiachen Zhao, Yiyou Sun, Weiyan Shi, and Dawn Song, at Northeastern University and UC Berkeley, measured the causal contribution of each step in a chain of thought by perturbing steps and watching the answer. They called the measure a true thinking score. On the AIME math dataset with the Qwen-2.5 model, the mean score across steps was about 0.03 on a scale of zero to one; only 6.4 percent of steps scored above 0.3, and just 2.3 percent reached 0.7. Most verbalized steps were what the authors call decorative thinking: they look like reasoning, appear in the transcript, and contribute almost nothing to the final answer. Even self-correction moments, the aha pauses where a model announces it needs to recompute, were often decorative; for Qwen-2.5, about 12 percent of self-verification steps scored below 0.005, meaning the model never actually used the correction it printed. The same team showed the phenomenon can be controlled: a direction in the model’s internal state space can force it to truly engage a step or skip it.
The interpretive battle over these results is as sharp as the results themselves. Subbarao Kambhampati of Arizona State University, whose lab produced related evidence that replacing a model’s correct traces with incorrect ones does not degrade its answers, argues that reasoning traces are not thoughts at all. His working hypothesis is that models perform what he calls approximate retrieval: the thinking tokens load up the context window in a way that makes reasoning-shaped text more likely to follow, the way mumbling words to yourself can jog a memory. The words barely matter. His group’s ICML 2026 position paper, titled Stop Anthropomorphizing Intermediate Tokens as Reasoning/Thinking Traces!, proposes replacing the vocabulary of thoughts with the neutral term derivational trace. OpenAI’s Sébastien Bubeck rejects the critique, saying the Apple study that found a complete accuracy collapse in reasoning models on complex puzzles tested obsolete models and that modern models starting with GPT-5.5 do not suffer from the problem. Apple did not make its researchers available for comment, and because frontier companies keep their raw traces secret, outsiders cannot test the claim directly.
The practical question is what to do with the gap. In verifiable domains, the answer is straightforward: check the result, not the story. Code runs or it does not, proofs are either correct or not, and the models’ best uses are in domains where an external verifier exists. In unverifiable domains, the danger is subtler. Chain of thought is increasingly proposed as a monitoring tool, a window into whether a model is reasoning safely before it acts. The causal studies show such monitoring can read decoration as thought. Researchers in the field put it bluntly: what matters is the right answer for the right reason, so these systems can be trusted. The dots experiment did not prove that reasoning models never reason. It proved that the printed chain of thought is not, by itself, evidence that they did.
References
Jacob Pfau, William Merrill, and Samuel R. Bowman, Let’s Think Dot by Dot: Hidden Computation in Transformer Language Models. arXiv:2404.15758 (2024).
Jiachen Zhao, Yiyou Sun, Weiyan Shi, and Dawn Song, Can Aha Moments Be Fake? Identifying True and Decorative Thinking Steps in Chain-of-Thought. arXiv:2510.24941 (2025).
Parshin Shojaee, Iman Mirzadeh, Keivan Alizadeh, Maxwell Horton, Samy Bengio, and Mehrdad Farajtabar, The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexity. arXiv:2506.06941 (2025).
Subbarao Kambhampati et al., Position: Stop Anthropomorphizing Intermediate Tokens as Reasoning/Thinking Traces! Proceedings of the 43rd International Conference on Machine Learning (ICML), 2026. arXiv:2504.09762.
John Pavlus, Is AI Reasoning Right for the Wrong Reasons? Quanta Magazine, 31 July 2026.