“NVIDIA GPU clusters have superior interconnects compared to TPUs, making them more suitable for the distributed training of Mixture-of-Experts models.”