Keep pulling the thread on Nathan Lambert.
OpenAI's o1 model is priced at $15 per million input tokens and $60 per million output tokens.
DeepSeek's R1 model is priced at $0.55 per million input tokens and $2.19 per million output tokens.
The price competition for reasoning models is expected to resemble the Mixtral inference price war from 2023.
DeepSeek AI released its first full-fledged reasoning model, R1, on January 20th.
DeepSeek's R1 reasoning language model is MIT-licensed, allowing companies and researchers to build upon and train on its outputs.
The training of DeepSeek's R1 model consists of four stages, starting with supervised finetuning and followed by multiple stages of reinforcement learning and rejection sampling.
DeepSeek R1 Zero is the first open model trained with large-scale reinforcement learning without a preliminary supervised fine-tuning step.
DeepSeek's research suggests that advancing AI intelligence boundaries may require more powerful base models and larger-scale reinforcement learning, as distillation is effective but not sufficient for breakthroughs.
The release of DeepSeek R1 with a permissive license marks the first time since Stable Diffusion's release that the most discussed AI model is openly available.
The R1-Zero model exhibits reliability issues, such as changing languages during reasoning.
DeepSeek's reinforcement learning training for R1-Zero involved thousands of model update steps, a scale significantly larger than the hundreds of steps used in the Tülu 3 work.
DeepSeek uses the Group Relative Policy Optimization (GRPO) algorithm for reinforcement learning, which is a variant of PPO with a different value approximation method.