“The design of Google's TPU pods, including their torus-shaped interconnects and low-level software collectives, proved to be an ideal foundation for building distributed deep learning systems.”