“The primary challenges in scaling Transformers to longer sequence lengths are speed and memory consumption.”