“The training cluster for GPT-4 was rumored to consist of 25,000 A100 GPUs, cost approximately $500 million, and consume around 10 megawatts of power.”