The Fundamental Limitations of State-Space Models for Tracking Sequential State

State-space models (SSMs) are not inherently more expressive than transformers for solving sequential state-tracking problems, despite their recurrent formulation. Like transformers, SSMs are limited to the complexity class TC0 and cannot express solutions to NC1-hard problems such as permutation composition, which are essential for tasks like tracking chess moves, evaluating code, or tracking entities in a narrative.