Andrei Karpathy provides a sober and deeply technical perspective on the state of AI, arguing that the development of truly capable AI agents is a decade-long endeavor, not an imminent breakthrough.
He critiques the current hype cycle, particularly around reinforcement learning, which he describes as a "terrible" and inefficient paradigm, famously coining the phrase "sucking supervision through a straw." Karpathy introduces the concept of building AI "ghosts" (trained on internet data) versus biological "animals" (products of evolution), and advocates for a research direction focused on isolating a model's "cognitive core" from its vast, and often distracting, memorized knowledge.
He also offers a realistic assessment of AI coding assistants, noting they excel at boilerplate but fail at the novel, intellectually-intense tasks required for frontier AI research.
8 quotes
Concerns Raised
The AI industry is over-predicting the short-term capabilities of AI agents.
Reinforcement learning is an inefficient and noisy paradigm for training intelligent systems.
LLMs are too good at memorization, which may hinder the development of true general intelligence.
Training on synthetic data leads to "model collapse" where output diversity is lost.
Using LLMs as reward models is unreliable as they are easily 'gamed' with adversarial examples.
Opportunities Identified
Developing new learning paradigms beyond imitation and reinforcement learning.
Isolating a model's "cognitive core" from its memorized knowledge to create smaller, more efficient models.
Improving data quality for pre-training, as current internet datasets are described as "total garbage".
Creating mechanisms for models to distill experiences into weights, analogous to human sleep.