Simons Institute for the Theory of Computing Notify me• Oct 15, 2024• 52:02Interview

Learning to Reason with LLMs

Noam Brown(Researcher, OpenAI, guest)

Get the full transcript next time Simons Institute for the Theory of Computing releases an episode

Summary, key quotes, top claims, and the searchable transcript - emailed automatically. No card needed.

Executive Summary

Continue your research

Keep pulling the thread on Noam Brown.

Inference-Time Compute vs. Training-Time Compute The Power of Search and Planning in AI

11 quotes

Concerns Raised

The computational techniques for advanced reasoning (like O-1) are very expensive, even if they are an alternative to pre-training.
The competitive nature of AI development is leading to less transparency and a departure from academic norms of sharing research details.
Current simple methods for scaling inference, like majority voting, show diminishing returns and do not scale effectively.

Opportunities Identified

Scaling inference-time compute offers a new vector for AI improvement, potentially unlocking massive performance gains from existing models.
O-1's generalizable reasoning ability shows promise across a wide variety of complex domains, from STEM to law (LSAT).
The performance of models like O-1 is on a steep upward trajectory, with significant improvements expected in the near future as the techniques are refined.

Key Themes

Research Findings12

Adding a 30-second search at the end of a poker game improved the AI's performance by an amount equivalent to scaling up the model's parameters by 100,000x.

The 2017 poker bot Libratus defeated four top poker professionals by a margin of 15 big blinds per 100 over 120,000 hands.

The raw neural network of AlphaGo and AlphaGo Zero, without search, has an Elo rating of around 3,000, which is below top human performance.

As of 2024, no neural network has achieved superhuman performance in Go without using a search algorithm at inference time.

Pre-training costs for large language models are exceeding $100 million.

By scaling up inference compute, OpenAI's O-1 model improved its accuracy on the AIME math test from 20% to over 80%.

The full O-1 model achieves over 80% accuracy on the AIME math benchmark when using consensus.

On a competition coding benchmark, GPT-4 achieves the 11th percentile while O-1 achieves the 89th percentile among human competitors.

On the GPQA Diamond benchmark, O-1 scores approximately 78%, surpassing the expert human score of 70%.

OpenAI's O-1 model uses large-scale reinforcement learning to generate an optimized chain of thought before answering.

The ability to scale inference compute means the anticipated wall for AI progress due to astronomical pre-training costs does not exist.

The 2015 poker bot Claudico lost to four top poker professionals by a margin of 9.1 big blinds per 100 in an 80,000-hand competition.

Topics

AI Reasoning Inference-Time Compute Test-Time Compute OpenAI O-1 Chain of Thought (CoT)Reinforcement Learning (RL)Game Theory Poker AI Libratus Pluribus AlphaGo Scaling Laws Computational Cost AIME Benchmark GPQA Benchmark System 2 Thinking Search Algorithms Generator-Verifier Gap

Processed Jul 28, 2026Daily intelligence brief → yt-dlp + mlx-whisper + Gemini

Learning to Reason with LLMs

Continue your research

Concerns Raised

Opportunities Identified

Key Themes

Inference-Time Compute vs. Training-Time Compute

The Power of Search and Planning in AI

O-1 as a Generalizable Reasoning Engine

The Generator-Verifier Gap

Research Findings12

Topics