The King Is Dead - What Will Replace the Transformer and What the Large Reasoning Models Era Looks Like

Ben Rotenberg, an AI consultant specializing in enterprise technology adoption, opened with the question preoccupying everyone in the field: what will replace ChatGPT? Before answering, he invites us to understand the architecture behind all the major tools and why the current technological "king" is reaching its limits.

What is the Transformer's built-in limitation?

The Transformer architecture powering ChatGPT, Claude, and Gemini operates like a person reading a thousand-page book: to understand the current word, it must go back and process every page from the beginning. This makes it thorough and powerful, but also consistently slower and more expensive as context grows longer. Beyond that, the Transformer excels at detecting language patterns but is limited in genuine multi-step reasoning.

What does Mamba offer?

Mamba, commercially adopted by AI21 Labs, is based on an approach called State Space Models. Instead of re-reading everything, the model maintains a compressed running state that updates with each new word, like a reader holding a continuously refreshed summary in memory rather than returning to page one each time. This allows it to handle enormous contexts of millions of words without losing what actually matters.

What does Sapient Intelligence's HRM offer?

The Hierarchical Reasoning Model, developed by Sapient Intelligence, is not just trying to be faster. It is trying to think differently. The inspiration comes from the human brain and the model described by Daniel Kahneman in Thinking Fast and Slow: a fast shallow system alongside a slow thorough one. The two layers of HRM work together to enable deep latent reasoning, meaning actual computation within the neural network, rather than generating words to simulate thinking as Chain-of-Thought does.

The results are striking: HRM with only 27 million parameters solved complex Sudoku puzzles and massive mazes almost perfectly after learning from just 1,000 examples. Far larger models failed entirely. It also outperformed models many times its size on the ARC-AGI benchmark, which is considered a measure of general intelligence capabilities.

What does this mean for the future of AI?

Rotenberg emphasizes this is not a ChatGPT upgrade but a generational shift. We are on the verge of transitioning from "Large Language Models" to "Large Reasoning Models." The limitations troubling us today may look in retrospect like minor birth pangs of a young technology. What this means for OpenAI's lead and the future of work, Rotenberg admits he does not know, but it is clear we are living through a moment whose full significance cannot be grasped in real time.