Major innovations in AI algorithms and architectures, such as the Transformer and FlashAttention,..., Sonic AI
“Major innovations in AI algorithms and architectures, such as the Transformer and FlashAttention, are concentrated in labs with abundant GPU resources rather than in compute-poor environments like academia.”