TwiML AI Podcast• Sep 16, 2025• 57:56Interview

Is It Time to Rethink LLM Pre-Training? [Aditi Raghunathan] - 747

From TwiML AI Podcast

Aditi Raghunathan•Assistant Professor of Computer Science, Carnegie Mellon University

Executive Summary

Overtraining large language models on excessive data can paradoxically make them worse for downstream fine-tuning, a counter-intuitive finding that challenges the 'more data is always better' paradigm.
The standard next-token prediction objective fundamentally limits LLM creativity and long-range planning, as it encourages local, greedy decisions rather than global coherence.
New training techniques like "memorization sinks" offer a path toward more controllable and editable models by intentionally localizing specific information (e.g., facts, private data) into designated neurons that can be selectively ignored or updated.
Alternative training methods, such as multi-token prediction and diffusion-like processes, show promise for overcoming the creative limitations of current models by forcing them to plan entire sequences at once.

9 quotes

Concerns Raised

Overtraining models on excessive data degrades their fine-tuning potential.
The next-token prediction paradigm fundamentally limits LLM creativity and long-range planning.
Benchmark performance is an increasingly poor proxy for real-world model utility and adaptability.
Retrieval-Augmented Generation (RAG) is not a complete solution for keeping models updated, as they often fail to override parametric knowledge.

Opportunities Identified

"Memorization sinks" can enable targeted information removal and updates, improving model control and safety.
Multi-token prediction and diffusion-like training can unlock greater creativity and structured generation.
Disentangling factual knowledge from reasoning abilities could make models more robust and easier to maintain.
Developing better evaluation metrics focused on adaptability could guide the creation of more useful models.

Key Themes

The Perils of Overtraining

The discussion highlights a critical and non-obvious failure mode of LLMs: training a model on too much data relative to its size can degrade its adaptability. This overtraining makes the model more rigid and significantly harder to fine-tune for new tasks, even as its benchmark performance on the original data continues to improve.

This is a crucial insight for practitioners selecting base models. It suggests that the 'best' model on a leaderboard is not necessarily the best starting point for customization, and that earlier training checkpoints may be more valuable.

Fundamental Limits of Next-Token Prediction

The core training objective for most LLMs, next-token prediction, is identified as a primary cause for their lack of true creativity and global planning. This auto-regressive process encourages models to make locally optimal choices, struggling with tasks that require structured, novel, or globally coherent outputs, such as generating new jokes or complex problem solutions.

This challenges the scalability hypothesis that simply making models bigger will solve all problems. It points to a need for fundamental architectural and training objective innovations to achieve more advanced reasoning and creativity.

Engineering Controllability with Memorization Sinks

A novel technique called "memorization sinks" is proposed to gain more granular control over what a model learns and remembers. By training the model to store specific information (e.g., facts, PII) in designated, isolated neurons, it becomes possible to edit, update, or forget that information without a full retrain, addressing key challenges in privacy and model maintenance.

This provides a concrete engineering approach to making LLMs safer, more reliable, and easier to update. It moves beyond post-hoc fixes towards building controllability into the model's architecture from the start.

Beyond Benchmarks: The Gap Between Performance and Utility

A recurring point is the growing divergence between high benchmark scores and the practical user experience, particularly regarding model adaptability and reliability. Models can be optimized to excel on static tests but fail when deployed in dynamic environments or when users attempt to customize them, indicating that current evaluation methods are inadequate.

This highlights the need for new evaluation metrics that capture a model's adaptability, robustness, and fine-tuning potential, which are more indicative of its real-world value than static benchmark scores alone.

Alternative Training for Enhanced Creativity

To overcome the limitations of next-token prediction, the research explores alternative training objectives like multi-token prediction and diffusion-based methods. These approaches force the model to generate entire sequences simultaneously, encouraging better global planning and the ability to produce more structured and diverse outputs, which is key for creative tasks.

This research points toward the next generation of model architectures that may move beyond simple auto-regression, potentially unlocking new capabilities in areas like scientific discovery, theorem proving, and creative content generation.

Get started free

Topics

LLM Limitations Next-Token Prediction Model Fine-Tuning Overtraining Model Adaptability Creativity in AI Memorization Sinks Model Control AI Safety Interpretability Multi-Token Prediction Diffusion Models Retrieval-Augmented Generation (RAG)Benchmark Performance Transformer Architecture

Processed Apr 2, 2026 yt-dlp + mlx-whisper + Gemini