Keep pulling the thread on Dan Klein.
The core principle of traditional software engineering is modularity, which is in direct tension with the core principle of modern machine learning, which is end-to-end optimization.
The development of large language models is beginning to hit diminishing returns due to data walls and compute limits, causing their progress to follow an S-curve rather than an exponential one.
Post-training techniques like Reinforcement Learning from Human Feedback (RLHF) can increase the frequency of hallucinations because models learn to produce outputs that humans prefer, which are not always factual.
The fluency and confidence of LLMs like ChatGPT remove the surface cues, or "code smells," that humans traditionally use to detect potentially unreliable information, making it harder to notice errors.
In the self-driving car industry, companies like Waymo and Zoox are increasingly adopting end-to-end training methods as they get closer to production.
The mission of Dan Klein's company, Scaled Cognition, is to build AI technologies that are incapable of lying.
The rapid progress of AI in coding and mathematics is primarily due to the verifiability of the outputs, such as passing unit tests or formal verification by tools like Lean.
The practice of using a chain of AI models to check each other for errors is often ineffective because the models' errors tend to be strongly correlated.
Scaled Cognition's APT-1 model is architected to treat information and actions as first-order objects, unlike traditional models that operate on tokens which lack inherent semantics until assembled.
Scaled Cognition trains its models using verifiable reinforcement learning on simulated, generated data.
A computational linguistics study reconstructing the Proto-Austronesian language from hundreds of modern languages provided strong evidence for the functional load hypothesis, a pattern not visible in smaller-scale studies.
An Anthropic research paper demonstrated that in their neural network, the representations for Chinese words and their English equivalents are stored in the same location.