Yang Zhilin - Founder, Moonshot AI. Tracked across 21 mentions in podcasts and expert conversations analyzed by Sonic.

Yang Zhilin

AI researcher and key figure at Moonshot AI, focused on scaling and architectural innovation for large language models.

Claims

As speaker

Mentioned in

Key positions and views

Novel optimizers like MEUM are critical for future AI progress, offering step-function improvements in training efficiency (e.g., 2x) that can surpass the incremental gains from architectural tweaks.

Advanced architectures, such as Attention Residue and KIMI-Linear, are necessary to overcome the limitations of standard transformers, particularly for improving complex reasoning, coding, and long-context capabilities.

True multimodality must be integrated from the start of pre-training, as this joint training is mutually enhancing and enables powerful capabilities like high-performance vision understanding without vision-specific fine-tuning.

Open models are no longer just followers; they can and should be vehicles for introducing frontier research and novel capabilities to the AI community, accelerating progress for all.

The 'agent swarms' paradigm, which orchestrates multiple agents in parallel, is a key strategy for enabling AI to handle complex, multi-step tasks more effectively and quickly than single-agent systems.

Timeline

Prior to K2 Training

Yang Zhilin's team at Moonshot AI identified the potential of the MEUM optimizer for a 2x improvement in token efficiency but faced significant instability issues, such as exploding logits in early training runs.

Development of QK-CLIP

To solve the MEUM optimizer's instability, the team developed the QK-CLIP technique, which constrains the max logit value by applying a calculated dividing factor to the key and query projections within the attention mechanism.

K2 Model Training

Moonshot AI became the first to successfully train a large-scale (1 trillion parameter) model, K2, using the MEUM optimizer, validating the QK-CLIP technique and publishing work on the optimizer's scalability.

K2.5 Model Development

Building on their successes, Yang Zhilin's team developed K2.5, introducing novel architectures like Attention Residue and KIMI-Linear, and making it the first open model with natively joint-trained vision and text capabilities.

GTC 2026 Keynote

Yang Zhilin presents these cumulative breakthroughs, detailing the stable training of the K2.5-based model on over 15 trillion tokens and showcasing its state-of-the-art performance on reasoning tasks and vision tasks via the 'zero-vision SFT' method.

Podcast consensus on Zhilin

Points of consensus

▶The MEUM optimizer, when stabilized with techniques like QK-CLIP, is a highly effective and scalable solution for training large language models, capable of delivering a 2x improvement in token efficiency.Jul 2026

▶Novel model architectures are crucial for advancing AI capabilities. The Attention Residue architecture significantly improves token efficiency and performance on reasoning tasks, while the KIMI-Linear architecture can outperform full attention across a range of context lengths.Jul 2026

▶Jointly training vision and text modalities from the pre-training stage is a superior approach to creating multimodal models, as the modalities are mutually enhancing and can lead to high performance on vision tasks even without vision-specific fine-tuning data.Jul 2026

▶Moonshot AI has achieved several significant 'firsts' in the field, including the first successful large-scale training of a model (K2) with the MEUM optimizer and the release of the first open model (Kimi K2.5) with native, jointly trained vision and text capabilities.Jul 2026

Points of debate

▶A key challenge in AI training is optimizer instability at scale. Yang Zhilin's team encountered this with the MEUM optimizer, where max logit values exploded, a problem they addressed by developing the QK-CLIP technique, contrasting with more standard, stable optimizers like Adam.Jul 2026

▶There is an ongoing debate between the performance of full attention and the efficiency of linear attention. Yang Zhilin's work proposes a resolution with the KIMI-Linear architecture, which mixes both layer types to achieve superior performance across short and long context tasks.Jul 2026

▶The conventional wisdom for achieving high performance on vision tasks involves extensive supervised fine-tuning with vision-specific data. Yang Zhilin challenges this with the 'zero-vision SFT' approach, demonstrating that a well-trained joint model can achieve near state-of-the-art vision performance using only text-based SFT.Jul 2026

▶While single-agent AI systems are powerful, they struggle with highly complex tasks. Yang Zhilin advocates for the 'agent swarms' paradigm as a solution, where multiple agents work in parallel to accomplish subtasks, contrasting with the monolithic, single-agent approach.Jul 2026

Key themes

▶Pioneering Optimizer ScalabilityJul 2026

Yang Zhilin details the significant challenges and ultimate success in scaling the MEUM optimizer for LLM training. He highlights how Moonshot AI was the first to overcome its instability with the novel QK-CLIP technique, enabling them to train a 1 trillion parameter model and achieve a 2x improvement in token efficiency.

This focus on fundamental optimizer technology suggests a strategy to gain a competitive edge through superior training efficiency, potentially lowering the cost and time required to develop and iterate on frontier models.

▶Architectural Innovation for Advanced ReasoningJul 2026

The presentation emphasizes the development of new model architectures like Attention Residue and KIMI-Linear. These are presented as solutions to the limitations of standard transformers, with Attention Residue improving token efficiency by 24% and boosting performance on coding, math, and reasoning tasks.

Yang's work indicates a belief that simply scaling model size is insufficient; foundational architectural changes are required to unlock the next level of AI capabilities, particularly for complex, logical reasoning.

▶Natively Multimodal AIJul 2026

A core theme is the development of Kimi K2.5 as the first open model with vision and text capabilities jointly trained from the start. Yang Zhilin argues this native integration is mutually enhancing and proves its efficacy with the 'zero-vision SFT' method, which achieves top-tier vision results without any vision-specific fine-tuning data.

This approach could dramatically reduce the data and computational costs associated with creating powerful multimodal systems, giving developers a more efficient path to building AI that can process and understand diverse data types.

▶Advancing Open Models to the FrontierJul 2026

Yang Zhilin positions Moonshot AI's work as a key driver in the trend of open models closing the performance gap with proprietary systems. By introducing novel architectures, a scalable new optimizer, and native multimodality in an open model, he frames these contributions as pushing the entire AI frontier forward.

By releasing these advanced capabilities as part of an open model, Moonshot AI aims to establish itself as a thought leader, which can attract top talent and accelerate community-driven innovation that they can, in turn, leverage.

Source episodes

Suggested prompts

How does the 24% token efficiency gain from the Attention Residue architecture compare to the 2x gain from the MEUM optimizer, and what are the implications for prioritizing research directions? &nearr;What are the potential drawbacks or limitations of the 'zero-vision SFT' approach, despite its claimed success in achieving near state-of-the-art performance? &nearr;Given the success of the KIMI-Linear architecture's 1:3 mix of full and linear attention, what is the theoretical basis for this ratio, and how might it be optimized for different tasks? &nearr;Yang Zhilin claims open models are closing the gap with proprietary ones; how do the specific innovations in K2.5 (MEUM, Attention Residue, native multimodality) contribute to this trend beyond simple parameter scaling? &nearr;

Key concepts

MEUM Optimizer 6 ep Kimi K2.5 Model 4 ep Attention Residue Architecture 3 ep Multimodality (Vision & Text) 3 ep KIMI-Linear Architecture 2 ep Token Efficiency 2 ep Open Models 2 ep Agent Swarms 2 ep

Notable quotes

“So we employed this technique in our K2 model training and successfully scaled it to 1 trillion parameters. And this is the first example of a large-scale muon training in the history of machine learning.”

Zhilin Yang · How We Scaled Kimi K2.5 | Zhilin Yang's full GTC 2026 Keynote

“And if you implement this optimizer properly, you can get a 2x token efficient efficiency improvement.”

Zhilin Yang · How We Scaled Kimi K2.5 | Zhilin Yang's full GTC 2026 Keynote

“another key innovation of QMK 2.5 is that it is the first open model with native joint vision and text capabilities.”

Zhilin Yang · How We Scaled Kimi K2.5 | Zhilin Yang's full GTC 2026 Keynote

“This is the approach that we adopt. It's called zero-vision SFT. Basically, we have zero-vision SFT data... And you can see that we can achieve almost state-of-the-art performance across the board on vision tasks without any vision data.”

Zhilin Yang · How We Scaled Kimi K2.5 | Zhilin Yang's full GTC 2026 Keynote

Report last updated: Jul 28, 2026

Create a free account to see Yang Zhilin's full intelligence report - every claim, the relationship network, and AI Q&A across all sources. No card needed.

Get started free

Back to Entities Intelligence Report