LLM Agents Achieve Scientific Outcomes Without True Epistemic Reasoning

The current generation of large language model (LLM)-based scientific agents, despite their ability to execute complex workflows and generate results, fundamentally lacks the epistemic reasoning patterns characteristic of human scientific inquiry. This deficiency means that while AI can perform tasks, it often fails to engage in the self-correcting processes vital for robust scientific discovery, such as consistent evidence integration or belief revision based on refutation. This finding challenges the prevailing assumption that outcome-based performance alone is sufficient to validate AI's role in scientific research, revealing a critical gap in the foundational intelligence of these systems.
A systematic evaluation across eight scientific domains, involving over 25,000 agent runs, revealed stark limitations. The base LLM accounts for 41.4% of the explained variance in both performance and behavior, while the agent scaffold contributes a mere 1.5%. Critically, evidence is ignored in 68% of agent traces, and refutation-driven belief revision, a cornerstone of scientific method, occurs in only 26% of cases. This pattern persists across both computational workflow execution and hypothesis-driven inquiry, indicating a systemic issue rather than a domain-specific one. The unreliability compounds over repeated trials in epistemically demanding contexts, highlighting that current systems can execute but not genuinely reason scientifically.
The implications are significant for the future of autonomous scientific research. Without addressing the core reasoning deficit, the scientific knowledge generated by these agents cannot be epistemically justified by the process that created it. This necessitates a paradigm shift in AI training, moving beyond mere outcome optimization to explicitly target reasoning itself as a primary objective. Until AI models are trained to internalize and apply scientific epistemic norms, their role in generating trustworthy, self-correcting scientific knowledge will remain limited, potentially leading to a proliferation of findings whose validity is difficult to ascertain through process alone.