Researchers have identified a fundamental vulnerability in how large language models handle multi-step reasoning tasks: while individual steps may achieve high accuracy in isolation, chaining those steps together produces near-random outputs. In a paper titled 'Long-Horizon State Tracking in LLMs: Executing MD5 through a Deep Sequence of Dependent Tool Calls,' published on arXiv (2609.00012v1), the authors demonstrate this cascading failure by tasking LLMs with executing the MD5 cryptographic algorithm through sequential tool calls. The finding exposes why long-horizon task evaluation remains rare in LLM benchmarking—the mathematical reality of error accumulation makes real-world multi-step autonomy far more fragile than headline accuracy numbers suggest.
The mechanics of this failure are straightforward but damning. When an LLM achieves 95% accuracy on individual steps, end-to-end performance across 20 dependent steps collapses to approximately 36% (0.95^20). Extended to 50 steps—common in complex planning scenarios—success rates plummet below 1%. The MD5 case study illustrates this concretely: the algorithm requires precise state tracking across 64 sequential rounds of bitwise operations, with each round depending entirely on the previous round's output. Any error propagates forward, corrupting all subsequent computations. The research team found that while state-of-the-art LLMs could execute individual MD5 operations correctly, maintaining accuracy across the full sequence proved nearly impossible without external verification mechanisms.
This research highlights a critical gap between how the field measures LLM capability and how these systems actually perform in deployment. The problem isn't that existing benchmarks are too hard—it's that they've largely ignored the category of tasks where small errors compound exponentially. Some proposed solutions include external memory augmentation, hierarchical task decomposition, and intermediate verification checkpoints, though the paper emphasizes these remain Band-Aids on a deeper architectural problem. The irony is uncomfortable: measuring per-step accuracy while ignoring cascade failures represents a benchmarking flaw the field should have designed against years ago, yet it continues shaping how we evaluate models deployed for real-world autonomous reasoning.