“Previous methods for optimizing attention, such as sparse and low-rank approximations, have not seen widespread adoption because they typically trade model quality for speed.”