The primary paradigm for software development has irrevocably shifted to 'agentic engineering,' where human value lies in high-level direction and delegation to AI agents, a change that occurred around December 2023.
Current AI models are more like 'ghosts' imitating flawed human data than true intelligences, and their capabilities are 'jagged,' excelling only in easily verifiable domains like code and math while stagnating elsewhere.
Reinforcement learning, as commonly practiced, is a 'terrible' and highly inefficient learning method ('sucking supervision through a straw') that was a misstep in the path to AGI.
The most promising path to AGI is recursive self-improvement, which could be accelerated by isolating a 'cognitive core' of intelligence from the 'slop' of memorized knowledge that currently bloats LLMs.
The next decade will be defined by the maturation of AI agents, which will eventually require physical interfaces (sensors, actuators) to continue learning by running experiments in the real world.
▶The Agentic Revolution in Software Development
Karpathy argues that a fundamental shift in software development occurred around December 2023, moving from manual coding to 'agentic engineering'. He conceptualizes this as 'Software 3.0', where developers primarily prompt and delegate to AI agents, making much of the existing human-centric documentation and infrastructure obsolete. He believes this new paradigm will create engineers far more productive than the traditional '10x' benchmark.
Investors should scrutinize companies' engineering cultures and hiring practices, as Karpathy suggests current methods are outdated and fail to identify or leverage the skills required for agentic engineering, creating a potential competitive disadvantage.
▶Critique of Current AI Paradigms and Data
Karpathy is critical of several core tenets of modern AI development. He describes reinforcement learning as a 'terrible' and inefficient paradigm, characterizes pre-training data from the internet as 'total garbage', and warns that training on synthetic data leads to a 'silent collapse' in output diversity. He likens current models to 'ghosts' imitating human data rather than 'animals' learning through an evolutionary process.
This critique suggests that true breakthroughs may not come from simply scaling current methods, but from foundational research into new learning paradigms, potentially creating opportunities for startups that can solve the data quality or reinforcement learning efficiency problems.
▶The Pursuit of Recursive Self-Improvement and the 'Cognitive Core'
Karpathy believes the ultimate goal of all frontier AI labs is recursive self-improvement, where LLMs are used to improve the next generation of LLMs. He proposes a research direction to accelerate this by isolating a model's 'cognitive core'—its problem-solving algorithms—by stripping away the vast, memorized knowledge from pre-training, which he views as a potential hindrance.
Analysts should monitor for research that moves beyond simple knowledge regurgitation towards models that demonstrate novel problem-solving, as this aligns with Karpathy's view of a more efficient path to AGI and could signal a significant technological leap.
▶Pragmatic Open-Source Contributions and PhilosophyMay 2026
Despite his work at frontier labs, Karpathy is a significant contributor to open-source AI, releasing educational tools like nanoGPT, MicroGPT, and NanoChat, as well as the 'auto-research' agent. He holds the view that the current dynamic, where open-source models lag 6-8 months behind the frontier, is a healthy setup for the industry. His projects aim to demystify complex AI concepts, such as showing an LLM can be trained with just 200 lines of Python.
Karpathy's actions and philosophy suggest a belief in the power of a distributed, open community to innovate, even envisioning a 'swarm of agents' outperforming centralized labs. This highlights the enduring importance of the open-source ecosystem as both a check on and a catalyst for the entire AI industry.