“The core principle of Flash Attention is to exploit the asymmetry between fast, small SRAM and slower, larger HBM on GPUs by designing an I/O-aware algorithm that minimizes data movement.”