“The dominance of the Transformer architecture in AI was partly due to its high performance on GPUs, in contrast to similarly capable recurrent neural networks which run poorly on them.”