“Flash Attention is 10 to 20 times more memory-efficient than standard attention implementations while being an exact algorithm with no approximation.”