TECH

Don’t be fooled—LLMs don’t reason

Source unique·il y a 13 j

The recent breakthroughs in AI, such as AlphaGo’s victory over Lee Sedol, have often been framed as evidence of machine reasoning, but this interpretation obscures a critical distinction. While systems like AlphaGo combined intuitive pattern recognition with deliberate search to make creative decisions, today’s large language models (LLMs) operate primarily through associative fluency rather than structured deliberation. The stakes are high: without genuine reasoning capabilities, AI’s outputs in fields like medicine or science risk being untrustworthy, opaque, and prone to post-hoc fabrication of reasoning steps.

What distinguishes AlphaGo’s approach to decision-making from that of modern LLMs?

AlphaGo relied on a dual architecture: a policy network for intuitive pattern recognition and a search machinery for deliberate, step-by-step evaluation of future moves. Modern LLMs, by contrast, generate responses through next-token prediction, producing chains of thought that are still rooted in the same associative process, without an independent mechanism for structured reasoning or explicit epistemic state tracking.

Why does the author argue that today’s AI models lack genuine reasoning capabilities?

The author identifies three key shortcomings: LLMs maintain no explicit, persistent epistemic state to track hypotheses, confidence, or unresolved questions; they do not separate knowledge from reasoning, as both are embedded in neural network weights; and their chains of thought are often retroactively constructed, obscuring the actual path to a conclusion rather than reflecting a transparent deliberative process.

What specific move in the AlphaGo vs. Lee Sedol match exemplifies the difference between intuition and reasoning?

Move 37 in game two of the 2016 match was initially dismissed by commentators as a glitch, but it demonstrated AlphaGo’s reasoning prowess. The policy network rated the move as highly unlikely for a human player, yet the search machinery identified its long-term strategic value by evaluating thousands of future board states, a process LLMs cannot replicate.

Ce que ça pourrait changer

If AI systems are to be trusted in high-stakes domains like medicine or climate science, they must evolve beyond associative fluency to incorporate explicit, auditable reasoning processes. Without such capabilities, their outputs risk being unreliable, difficult to debug, and potentially misleading—undermining their utility in fields where transparency and correctness are paramount. The current trajectory of scaling LLMs may sharpen their intuition but will not, on its own, bridge the gap to genuine reasoning.

AlphaGo
A Go-playing AI developed by DeepMind that defeated world champion Lee Sedol in 2016, notable for its combination of intuitive pattern recognition (via a policy network) and deliberate search (via a game tree) to make strategic decisions.
Large Language Models (LLMs)
AI systems trained to predict the next token in a sequence, producing fluent text but lacking explicit reasoning mechanisms, as highlighted by the author’s critique of their reliance on associative fluency rather than structured deliberation.
System 1 and System 2
A cognitive theory adapted from Daniel Kahneman, where System 1 represents fast, intuitive thinking and System 2 denotes slow, deliberate reasoning. The author uses this framework to contrast AlphaGo’s dual architecture with the monolithic, associative nature of LLMs.
Chain of thought
A process in LLMs where intermediate reasoning steps are generated before arriving at a final answer, intended to mimic deliberation. The author argues that this does not constitute genuine reasoning, as the steps are still produced by the same next-token prediction mechanism.
Epistemic state
A structured representation of what a system knows, doubts, or has ruled out, which the author argues is essential for genuine reasoning. Modern LLMs lack such a persistent, inspectable state, making their reasoning opaque and untrustworthy.
Lee Sedol
A South Korean professional Go player and one of the greatest of all time, who lost 1-4 to AlphaGo in 2016. His remark about AlphaGo’s move 37 highlighted the AI’s perceived creativity, which the author later dissects as a product of reasoning rather than intuition.
DeepMind
A subsidiary of Alphabet (Google’s parent company) specializing in AI research, where the author was a core member of the AlphaGo team. The article reflects the author’s departure from DeepMind, citing a need for a new approach to machine reasoning.