Stop Anthropomorphizing Intermediate Tokens as Reasoning Traces

Intermediate Tokens are Not 'Thinking' Processes

Intermediate Token Generation (ITG) is a standard method used to improve language model performance on complex reasoning tasks by producing output before the final solution. However, a position paper presented at ICML 2026, "Position: Stop Anthropomorphizing Intermediate Tokens as Reasoning/Thinking Traces!", argues that calling these outputs "reasoning traces" or "thinking traces" is a dangerous anthropomorphism. The authors contend that these terms imply the model is following human-like cognitive steps, which misleads the community about the nature of these models and leads to questionable research.

The Danger of the "Thinking" Metaphor

The core argument is that treating intermediate tokens as an interpretable window into a model's "thought process" is a fundamental misunderstanding of LLM architecture. Because these models generate tokens based on probability, the resulting text is a sequence of predictions, not a conscious derivation of a solution.

One specific example cited is the "aha" moment—where a model might output a token indicating a sudden realization. The authors argue that this is not a sign of an internal state change, as the model's forward pass only differs from the previous one by the inclusion of that token in the context. Interpreting such moments as meaningful cognitive shifts is an unwarranted assumption.

Engineering and Research Implications

Anthropomorphizing these tokens has practical consequences for how AI systems are audited and developed. If intermediate tokens are not a faithful representation of the underlying computation, they cannot serve as reliable audit artifacts.

Moving from Interpretation to Reproducibility

Technical discussion suggests that instead of trying to make a model's internal narration more interpretable, engineers should focus on making the computation more reproducible. This involves recording actual inputs, model versions, configurations, and tool observations to isolate differences between runs, rather than asking a model to explain its "thoughts."

The Disconnect Between Trace and Result

Observations from the community highlight a common failure mode: models often output a "reasoning trace" where they identify a mistake ("Wait, that’s wrong"), yet proceed to make the exact same mistake in the final answer. This disconnect demonstrates that the nominal meaning of the tokens does not always govern the final output.

Community Perspectives and Counterpoints

Discussion among researchers and practitioners reveals a divide in how these tokens should be viewed:

• The Metaphorical View: Some argue that "reasoning tokens" is simply a convenient shorthand for "learned prompt augmentation tokens" and that most serious researchers treat it as a metaphor rather than a literal description of cognition.
• The Functional View: Others suggest that while the tokens aren't "thinking," they act as a form of "digital scratchpad." Generating intermediate text helps the model compute the final answer more effectively by providing a structured context to draw from.
• The RL Influence: Some point out that Reinforcement Learning (RL) rewards exact final answers, which forces models to develop error-correction habits. While this makes the output look like human thinking (e.g., verification and correction steps), it remains a mechanical process of probability generation.
• The Human Parallel: A counter-argument posits that human inner monologues are also not always faithful representations of subconscious data processing, suggesting that the gap between "trace" and "result" is a feature of both artificial and human intelligence.

"If intermediate tokens are not a faithful representation of the computation, then they are a pretty bad audit artifact too. We probably shouldn't be trying to make the model's internal narration more interpretable, but rather the computation around it more reproducible."

Conclusion

The authors of the ICML 2026 paper call for the AI community to abandon the language of "thinking" and "reasoning" when describing intermediate tokens. By treating these traces as black boxes or mechanical augmentations rather than cognitive windows, researchers can avoid the pitfalls of hypermentalizing and develop more robust, grounded methods for evaluating and auditing AI performance.