Keep pulling the thread on Noam Brown.
Noam Brown co-created the first AI system to achieve superhuman-level performance in no-limit Texas Hold'em for both heads-up and multiplayer.
Noam Brown co-created an AI system, Cicero, that can strategically out-negotiate humans using natural language in the board game Diplomacy.
The Libratus AI bot defeated four top heads-up no-limit hold'em poker players by approximating the Nash equilibrium without adapting to or exploiting its opponents.
Over the course of 120,000 hands, the Libratus bot won close to $2 million in notional money against human professionals.
The main algorithmic improvement in Libratus (2017) over the 2015 bot was a shift from a pre-computed lookup table strategy to a real-time, search-based approach.
The Libratus bot developed a strategy of "overbetting" (e.g., betting 10 times the pot), a tactic not commonly used by humans at the time.
If the real-time search component is removed from AlphaZero, its Elo rating drops to around 3,000, which is below top human level.
As of late 2022, no raw neural network without a search component has achieved superhuman performance in the game of Go.
The final training run for the Pluribus AI cost less than $150 on AWS.
The thousand-fold decrease in training cost from Libratus to Pluribus was due to algorithmic improvements, particularly depth-limited search, not just the falling cost of compute.
The AI systems Libratus and Pluribus did not use any neural networks.
Self-play alone is not sufficient for an AI to achieve superhuman performance in the game Diplomacy; it must incorporate human data.