“Flash Attention reduces the memory requirement of the attention mechanism from quadratic O(N^2) to linear O(N) with respect to sequence length N.”