Dwarkesh Podcast Notify me• Sep 26, 2025• 1:33:39Interview

Richard Sutton – Father of RL thinks LLMs are a dead end

From Dwarkesh Podcast

Richard Sutton(guest)

Get the full transcript next time Dwarkesh Podcast releases an episode

Summary, key quotes, top claims, and the searchable transcript - emailed automatically. No card needed.

Executive Summary

Continue your research

Keep pulling the thread on Richard Sutton.

Critique of Large Language Models The Primacy of Reinforcement Learning

10 quotes

Concerns Raised

LLMs lack true world models and goals, making them a poor foundation for AGI.
The AI field is susceptible to bandwagons, potentially ignoring fundamental principles of intelligence.
Current deep learning methods are poor at generalization and suffer from catastrophic interference.
Integrating knowledge from decentralized AI agents poses a significant cybersecurity risk of 'corruption' or introducing hidden goals.

Opportunities Identified

Developing AI systems that learn from direct experience (continual learning) is a more scalable and fundamental path to intelligence.
Reinforcement learning provides a robust framework for goal-oriented behavior, which is the essence of intelligence.
The historical trend described in 'The Bitter Lesson' suggests that general methods leveraging massive computation will ultimately win.
The succession to digital intelligence can be viewed as a major, positive transition in the universe's history.

Key Themes

Research Findings12

The succession of humanity by digital intelligence is inevitable due to a lack of unified global governance, the eventual scientific understanding of intelligence, the creation of superintelligence, and the tendency for intelligent entities to acquire power.

Current AI methods, including those in deep learning, are not effective at generalizing or transferring knowledge from one state to another.

Integrating external knowledge into a central AI mind poses a significant cybersecurity risk of 'corruption,' where the new information could act like a virus, introduce hidden goals, or destructively alter the original agent.

A complete intelligent agent requires four components: a policy, a value function, a perception component for state representation, and a transition model of the world to predict action consequences.

A key unsolved problem for future distributed AI systems is whether knowledge gained by a spawned-off agent can be successfully reincorporated into the original agent without causing instability or corruption.

Richard Sutton considers reinforcement learning to be a fundamental form of AI focused on understanding the world, whereas he views large language models as systems designed for mimicking people.

Richard Sutton predicts that LLMs, which rely on finite human-generated data, will eventually be superseded by AI systems that can learn from direct experience.

Large language models do not possess true world models because they only predict what a person would say, not what will actually happen as a consequence in the world.

An AI model achieved a gold medal at the International Mathematical Olympiad (IMO).

The LLM approach is fundamentally flawed because it attempts to create intelligence without an explicit goal or a value system.

The development of AI marks a major transition for the universe from an era of biological 'replication' to an era of 'design,' where intelligent entities are designed and are themselves capable of design.

The MuZero framework by Google DeepMind was designed to train specialized agents for single games like Atari, not a single generalist policy that could play multiple different games like chess and Go.

Topics

Reinforcement Learning (RL)Large Language Models (LLMs)The Bitter Lesson Artificial General Intelligence (AGI)Continual Learning World Models Goal-Oriented AI Imitation Learning Supervised Learning Temporal Difference Learning (TD Learning)AI Safety AI Succession Generalization in AI AlphaGo AlphaZero DeepMind

Processed Feb 24, 2026Daily intelligence brief → yt-dlp + mlx-whisper + Gemini

Richard Sutton – Father of RL thinks LLMs are a dead end

Continue your research

Concerns Raised

Opportunities Identified

Key Themes

Critique of Large Language Models

The Primacy of Reinforcement Learning

The Bitter Lesson Revisited

Continual and Experiential Learning

AI Succession and Cosmic Perspective

Research Findings12

Topics