The KIMI-Linear architecture mixes linear attention layers and full attention layers at a 1:3 rat..., Sonic AI
“The KIMI-Linear architecture mixes linear attention layers and full attention layers at a 1:3 ratio to balance long-context capability with computational efficiency.”