Keep pulling the thread on Jared Kaplan.
Scaling laws exist for the pre-training phase of AI, showing that model performance predictably improves with increases in compute, dataset size, and neural network size.
The length of tasks that AI models can perform is doubling roughly every 7 months, according to research by the organization Meter.
Over the next few years, AI models may be able to perform tasks that take humans days, weeks, months, or years to complete.
Anthropic expects to release a model superior to Claude 4 in less than 12 months.
In the past year, Y Combinator has observed a shift from startups selling AI as a 'copilot' requiring human approval to selling AI as a replacement for full workflows.
The 'holy grail' in scaling law research is to discover a method that improves the slope of the scaling law, providing a compounding advantage in capability for a given amount of compute.
When scaling laws appear to fail, it is more likely due to a flaw in the AI training process, such as a wrong architecture or a training bottleneck, than a failure of the laws themselves.
The AI industry is currently achieving 3x to 10x gains per year in efficiency through a combination of algorithmic improvements, compute scaling, and inference optimization.
The existence of precise scaling laws over many orders of magnitude since 2019 provides conviction that AI will continue to get smarter in a predictable way.
Scaling laws also exist for the reinforcement learning phase of AI training, showing predictable improvements in model skill.
The fundamental driver of AI progress is the discovery of a simple method to systematically improve AI by scaling up compute in pre-training and reinforcement learning, not that researchers have suddenly become smarter.
To achieve human-level AI, models need to be trained with relevant organizational knowledge, memory for long-horizon tasks, and better oversight for nuanced tasks.