Decoded• May 1, 2026• 37:52Interview

Beyond Bigger Models: Recursion As The Next Scaling Law In AI

From Decoded

Francois Chauvard•Visiting Partner, YC

Executive Summary

Standard feed-forward transformers (LLMs) have inherent architectural limitations for complex, multi-step reasoning, as they cannot perform algorithms requiring more sequential steps than the model has layers.
Two recent papers on Hierarchical Reasoning Models (HRM) and Tiny Recursive Models (TRM) reintroduce recursion at inference time, enabling small models (7-27M parameters) to outperform massive LLMs on specific reasoning benchmarks like ARC-Prize.
The key innovation is a stable training method, inspired by Deep Equilibrium Models, that backpropagates through a single, truncated recursive step, overcoming the historical instability of Recurrent Neural Networks (RNNs).
The future of AI may involve hybrid systems that combine the vast knowledge and powerful latent representations of large models with the efficient, recursive reasoning capabilities of smaller models like TRM.

12 quotes

Concerns Raised

Recursive models like HRM and TRM are currently task-specific and not general-purpose.
Training LLMs to reason via Chain of Thought requires human-labeled traces, which are not always available for novel problems.
Backpropagation through time remains a fundamental challenge for deep recurrent models, despite recent workarounds.

Opportunities Identified

Combining the recursive reasoning of TRM-like models with the broad knowledge of large-scale LLMs.
Using large models to create powerful latent representations and then applying smaller recursive models to reason within that space.
Further exploration of the "outer refinement loop" and truncated backpropagation as powerful new training techniques.

Key Themes

The Limits of Feed-Forward Transformers

Standard transformer-based LLMs process inputs in a single, parallel forward pass. This architecture makes them unable to perform algorithms that require more sequential steps than the model has layers, limiting their ability for "latent reasoning" on complex problems like sorting an unsorted list.

This highlights a fundamental architectural constraint of the dominant AI paradigm, suggesting that simply scaling up current models may not be sufficient to achieve more advanced, general reasoning capabilities.

The Re-emergence of Recursion in AI

The episode discusses how Hierarchical Reasoning Models (HRM) and Tiny Recursive Models (TRM) successfully reintroduce recursion, a concept central to older Recurrent Neural Networks (RNNs). Unlike LLMs' "Chain of Thought" which reasons in the discrete token space, these models perform recursive operations in a continuous latent space, enabling more powerful and efficient reasoning.

This signals a potential shift in AI architecture, moving beyond pure scale to incorporate more complex computational structures inspired by classical computer science and older neural network designs.

Efficiency and Specialization vs. Scale and Generality

The discussion highlights a stark contrast between tiny, specialized models (TRM is 7M parameters) that excel at specific reasoning tasks and massive, general-purpose LLMs (trillions of parameters) that possess broad knowledge but are less efficient at these tasks. The recursive models must be trained per-task, unlike generalist LLMs.

This raises important questions about the future of AI development: will progress come from building ever-larger generalist models, or from a hybrid approach that uses smaller, specialized "reasoning modules" for specific tasks?

Overcoming Historical Training Challenges

A major reason for the decline of RNNs was the difficulty of training them using backpropagation through time (BPTT), which led to vanishing or exploding gradients. The new models (HRM/TRM) introduce a novel training technique that only backpropagates through a single, truncated recursive step, making the training process stable and effective.

This innovation in training methodology is crucial, as it unlocks the power of recursive architectures without the prohibitive instability that plagued earlier attempts, opening new avenues for research.

The Future is Hybrid

The speakers predict that the most powerful future AI systems will be hybrids. They will leverage large models to create rich, semantically meaningful latent representations of the world, and then use small, efficient recursive models like TRM to perform complex reasoning within that latent space.

This provides a compelling vision for the next generation of AI, suggesting a modular architecture that combines the strengths of both large-scale knowledge models and compact, powerful reasoning engines.

Get started free

Topics

AI Recursion Hierarchical Reasoning Models (HRM)Tiny Recursive Models (TRM)Recurrent Neural Networks (RNN)Transformer Architecture Large Language Models (LLM)Reasoning Limitations Backpropagation Through Time (BPTT)Chain of Thought (CoT)Latent Space Reasoning ARC-Prize Model Scaling Computational Efficiency Deep Equilibrium Models (DEQ)Algorithmic Reasoning

Processed May 1, 2026 yt-dlp + mlx-whisper + Gemini

You're reading a preview

Get started free →