A fundamental mathematical equivalence exists between linear attention (attention without the sof..., Sonic AI
“A fundamental mathematical equivalence exists between linear attention (attention without the softmax function), linear Recurrent Neural Networks, and state-space models (SSMs).”