“The training cluster for GPT-4 was rumored to consist of 25,000 NVIDIA A100 GPUs, cost approximately $500 million to build, and consumed roughly 10 megawatts of power.”