Pre-training of frontier language models currently involves using tens of thousands of accelerato..., Sonic AI
“Pre-training of frontier language models currently involves using tens of thousands of accelerator chips, such as GPUs, TPUs, or Trainium, for a duration of several months.”