Monte Carlo Tree Search (MCTS) is a superior training method for game AI because it provides a low-variance, stable learning signal for every action, in contrast to naive reinforcement learning [8].
Training models on soft targets, such as the full MCTS visit count distribution, is significantly more information-efficient and effective than training on one-hot labels representing the single best action [6, 17].
The cost of training state-of-the-art AI has decreased dramatically, making it accessible beyond large labs, driven by open-source innovations like Katago and productivity gains from LLM-assisted coding [5, 10].
While Large Language Models are useful assistants for concrete tasks like coding and experiment execution, they currently lack the high-level strategic thinking and problem-identification capabilities required for cutting-edge research [16, 25].
On-policy reinforcement learning is generally preferable due to its stability, although off-policy methods can offer robustness benefits if the replay buffer is carefully managed to avoid learning from irrelevant states [2, 7, 13].
Circa 2016
Describes the AlphaGo Lee system, which was initialized with supervised learning on human games, used tens of thousands of MCTS simulations per move, and included a 'playout' step that was later deemed unnecessary [4, 20, 24].
Post-2016
Notes the shift to tabula rasa learning with systems like AlphaGo Zero, which learned from scratch via self-play without human data and removed the inefficient 'playout' step [11, 24].
2020
Highlights the creation of the open-source project Katago, which achieved a 40x reduction in compute required for training through innovations like transfer learning from 9x9 to 19x19 boards [5, 19].
2021
References a paper on scaling laws for board games, indicating a growing scientific understanding of how to predict the compute needed to solve games of increasing complexity [14].
Present Day
Discusses his own experiments using LLMs like Claude for coding assistance and hyperparameter tuning, concluding they are useful tools but not yet capable of high-level research strategy [10, 16, 25].
▶The Evolution and Democratization of Go AIMay 2026
Zhang traces the lineage of Go-playing AI from DeepMind's costly AlphaGo to the more efficient open-source Katago. He highlights how innovations like Katago's architecture and training methods have reduced the required compute by 40x, bringing the cost down from millions to a few thousand dollars [5, 10].
The rapid decrease in training costs for specialized, superhuman AI suggests that cutting-edge capabilities can quickly become commoditized, shifting the competitive landscape from pure compute access to algorithmic and architectural innovation.
▶Reinforcement Learning Methodologies and Trade-offsMay 2026
Zhang delves into the technical details of reinforcement learning, contrasting the high variance of naive RL with the stability of Monte Carlo Tree Search (MCTS) [8]. He also discusses the stability advantages of on-policy methods versus the potential robustness gains and pitfalls of off-policy training [2, 7, 13].
For analysts, this highlights that the choice of RL algorithm is not a settled matter; it involves critical trade-offs between stability, sample efficiency, and robustness that directly impact model performance and training reliability.
▶The Power of Information-Rich Training SignalsMay 2026
A core principle in Zhang's explanation is the value of using 'soft' targets over 'hard' one-hot labels. He explains that AlphaGo's policy network learns from the entire MCTS visit distribution, which is analogous to model distillation, as it provides more information per sample and leads to better learning [6, 17].
This focus on the quality of the learning signal itself is a key driver of efficiency and performance, suggesting that future AI breakthroughs may come from designing better teaching methods for models, not just bigger models or more data.
▶The Practical Application and Limits of Modern AI Tools
Zhang provides a grounded perspective on the current utility of Large Language Models in research and development. He finds them effective for concrete tasks like generating code for data structures or running experiments, but notes their inability to perform high-level strategic planning or identify core research problems [16, 25].
This indicates a clear boundary in current AI capabilities; while LLMs are powerful productivity enhancers for developers, the crucial tasks of scientific discovery and strategic direction remain firmly in human hands.