Online Chain-of-Thought Boosts Expressive Power of Multi-Layer State-Space Models

Multi-layer State-Space Models (SSMs), while offering computational efficiencies, possess fundamental limitations in handling compositional tasks, creating an inherent expressive gap when compared to streaming algorithms. This research meticulously dissects these boundaries, revealing that the architectural design of base SSMs restricts their ability to perform complex, multi-step reasoning. Understanding these limitations is crucial for guiding the development of next-generation AI architectures, particularly as SSMs gain prominence as alternatives to transformer-based models.

The study further investigates the impact of Chain-of-Thought (CoT) reasoning on SSMs' capabilities. It establishes a critical distinction: offline CoT, where reasoning steps are pre-computed, does not fundamentally enhance the expressive power of SSMs. In stark contrast, online CoT, which involves dynamic, iterative reasoning during computation, substantially increases their power, rendering multi-layer SSMs equivalent in expressive capability to streaming algorithms. This finding highlights that the temporal aspect of reasoning—how and when intermediate steps are generated—is paramount for unlocking advanced computational abilities in these models. The research also demonstrates that while width and precision are not interchangeable resources in base SSMs, they achieve a clean equivalence once online CoT is integrated, offering new insights into resource allocation and model design.

These results provide a unified perspective on how depth, finite precision, and CoT interact to shape the power and limits of SSMs. The implication is that for SSMs to tackle more sophisticated, real-world problems requiring complex reasoning, integrating online CoT mechanisms will be essential. This shift could enable SSMs to move beyond their current niche applications, potentially challenging the dominance of transformer architectures in domains like long-context understanding and sequential decision-making, provided the computational overhead of online CoT can be efficiently managed. The future trajectory of SSM development will likely involve deeply embedding dynamic reasoning processes within their core architecture.