Keep pulling the thread on Andrej Karpathy.
The AI field does not currently have a single, clean algorithm that can be unleashed upon the world to learn automatically from experience alone.
Pretraining on internet documents is the AI industry's practical solution to the cold start problem, analogous to the role of evolution in biology.
It is possible that Large Language Models will diverge from animal-like intelligence and become permanently different but world-altering, analogous to planes versus birds.
Richard Sutton's essay "The Bitter Lesson" has become a foundational text within frontier Large Language Model research circles.
Richard Sutton is not sure that Large Language Models are "bitter lesson pilled" because they are trained on finite, human-generated data.
Richard Sutton's "child machine" concept, inspired by Alan Turing, envisions a system that learns through dynamic interaction with the world without a large pretraining stage or supervised finetuning.
Richard Sutton views pretraining, even when used as an initialization for reinforcement learning, as being 'tainted with human bias' and 'fundamentally off course'.
AlphaZero, which was not trained on human games of Go, defeated AlphaGo, which was initialized using human games.
The success of AlphaZero in learning the game of Go is difficult to generalize to real-world problems because Go is a simple, closed environment.
Animals observe demonstrations but are not directly supervised with actions in the way supervised learning operates, as they are never 'teleoperated' in a training mode.
Recent AI research is exploring memory, such as the use of `CLAUDE.md` files with the Claude model, as a mechanism for test-time learning that uses text as a substrate instead of model weights.
Richard Sutton's proposed explanation for learning long-horizon sparse rewards, which relies on temporal difference learning and reward discounting, is not compelling.