“The compute required to train GPT-1 was 18,000 petaflops, while GPT-4 required approximately 21 billion petaflops.”