“According to an analysis by Toby Ord, a one-million-fold increase in total reinforcement learning compute is needed to achieve a performance boost equivalent to a single GPT-level improvement.”