Keep pulling the thread on Nathan Lambert.
The o1 system is designed by training models on long reasoning chains using reinforcement learning.
Unlike traditional autoregressive language models, OpenAI's o1 system performs an online search for the user at inference time.
The increased spending on inference for the o1 system confirms the existence of new "inference scaling laws."
The performance of OpenAI's o1 model improves with both more reinforcement learning at train-time and more compute at test-time.
ChatGPT will eventually integrate the o1 system and automatically route appropriate user queries to it.
The pricing for o1 applies to the intermediate reasoning tokens generated by the model, which are not shown to the user.
The high inference cost of o1 is likely due to a parallel decoding process where the model generates and rates multiple candidate reasoning steps internally before producing an output.
Based on OpenAI's performance plots, generating a completion with a system like o1 can be 10 to 1000 times more expensive than with existing language models.
The o1 model has been used internally at OpenAI to author pull requests in the company's codebases.
OpenAI's o1 reasoning system is built upon the company's earlier work on Q* and Strawberry.
OpenAI's o1 is a prototype and lacks the clear product-market fit that ChatGPT had at its launch.
The release of OpenAI's o1 is more comparable to the original GPT-3 release than to the ChatGPT release.