“The high-performance interconnect between chips on Google's TPUs is well-suited for long-context attention operations and serving sparse models with many experts.”