LLMs Exhibit Reasoning-Output Dissociation Despite Correct Chain-of-Thought

A critical vulnerability in Large Language Model (LLM) reasoning has been identified: the ability to execute correct chain-of-thought logic while simultaneously producing incorrect final answers. This reasoning-output dissociation, uncovered by the Novel Operator Test, fundamentally challenges the assumption that transparent reasoning steps equate to reliable outcomes. The implications are profound, as it suggests that current evaluation benchmarks may not fully capture the nuances of LLM logical fidelity, potentially overstating their true reasoning capabilities.

The Novel Operator Test, designed to separate operator logic from its linguistic label, revealed this dissociation across multiple models. Notably, Claude Sonnet 4 at depth 7 exhibited 31 errors where internal reasoning was verifiably correct, yet the declared answer was wrong. Similar patterns were observed in mixed-operator chains. The use of a "Trojan operator" further confirmed that the novelty of a name alone does not impede reasoning, but rather highlights a genuine difficulty with novel logic, as seen in Llama's widening novelty gap. The benchmark also categorized failure types, distinguishing between strategy failures at shallower depths and systematic content failures at deeper levels, even post-intervention.

This finding necessitates a re-evaluation of LLM trust and deployment strategies, especially in domains requiring high accuracy and verifiability. While the ability to trace reasoning steps is a step towards explainable AI, this dissociation indicates that such transparency does not guarantee correctness. Future research must focus on bridging this gap between internal logical consistency and external output accuracy, potentially through new architectural designs, training paradigms, or advanced verification mechanisms. Until this fundamental disconnect is resolved, the deployment of LLMs in high-stakes decision-making environments will carry an inherent, difficult-to-diagnose risk.