SM
Stanford MLSys Seminar
Dan
Summary
Stanford MLSys Seminar is hosted by Dan covering Flash Attention, Transformer Architecture, Attention Mechanism, and Long-Sequence Modeling. Notable guests include Tri Dao (PhD student, Stanford University).
1episodes
36total claims
12topics covered
1 episodes
FlashAttention - Tri Dao
›Jan 12, 2023 · WITH Tri Dao
Flash Attention is an I/O-aware algorithm that significantly improves the efficiency of the Transformer attention mechanism, achieving 1.5-3x speedups and 10-20x memory savings. The core innovation...
Flash AttentionTransformer ArchitectureAttention MechanismLong-Sequence Modeling+13 more