Keep pulling the thread on Eric Jang.
The open-source project Katago, created by David Wu from Jane Street in 2020, achieved a 40x reduction in the compute required to train a strong Go bot from scratch.
Training a strong Go AI, which originally cost DeepMind millions of dollars, can now be accomplished for a few thousand dollars of rented compute, partly due to advances in LLM-assisted coding.
A fundamental concept in AlphaGo is the use of a trained value function to evaluate a board state, which radically speeds up the search process by avoiding the need to play out the game tree to its full depth.
The Nash equilibrium policy found by systems like AlphaGo for the game of Go appears to be superhuman, as no human strategy has been able to defeat it.
The core self-improvement loop in AlphaGo involves training the policy network to directly predict the improved, more confident action distribution that results from the MCTS search process.
A 10-layer neural network can amortize and approximate a nearly intractable search problem to a very high fidelity.
The Monte Carlo Tree Search (MCTS) algorithm provides a low-variance learning signal for every action, in contrast to naive reinforcement learning methods which suffer from high variance.
AlphaGo's policy network is trained to imitate the entire MCTS visit count distribution, which is a soft target, rather than just the single best action, which would be a one-hot target.
AlphaGo's training algorithm is highly stable because it is framed as a supervised learning problem on improved labels, which avoids the difficult exploration problem inherent in many naive RL setups.
Most Go practitioners today train against the AI model Katago.
All Go AIs are trained against and resolve games using the Tromp-Taylor rules because they are completely unambiguous for computers.
Eric Jang found that Claude 4.6 was able to generate a reasonable data structure for a Monte Carlo Tree Search implementation.