What does the next training paradigm look like?

Dario Amodei stated that AI model performance can degrade when a model trained on a small context length is then used at a long context length, according to a quote cited by Dwarkesh Patel.

Dwarkesh Patel interprets Dario Amodei's comments to mean that performance gains from short-horizon Reinforcement Learning (RL) training do not necessarily generalize to long-horizon tasks.

The CursorTab model performs online learning by predicting which code edits will be accepted by the user, an objective it processes for over 400 million requests per day.

The on-policy self-distillation (OP-SD) technique for model training does not require an outer-loop verifiable reward signal.

On-policy self-distillation (OPSD) provides a denser supervision signal for model training compared to naive Reinforcement Learning (RL).

For the purpose of continual learning, on-policy self-distillation (OPSD) is a superior method to supervised fine-tuning (SFT).

The EfficientZero model could likely defeat a novice human player in a previously unseen Atari game if both were given two hours to play against a simulator.

For each step taken in an actual game, the EfficientZero model runs dozens of simulated games internally.

Dwarkesh Patel speculates about a future AI feature, which he calls "/dream", that would use large amounts of compute to build and train against simulated environments based on real-world observations, contrasting with current, less compute-intensive features in models like Codex, Cursor, or Claude.

Training AI agents on live websites like Amazon by running thousands of parallel checkout attempts is not feasible because the company, led by Andy Jassy, would detect and block the bots.

Creating clones of applications like Slack and Gmail for AI training is currently a labor-intensive and unscalable method.