“The Transformer architecture succeeded by adapting and parallelizing techniques originally developed for RNNs and LSTMs, such as gating, for efficient execution on GPUs.”