“Attention Residuals (AttnRes) were integrated into the Kimi Linear architecture, a model with 48 billion total parameters and 3 billion activated parameters.”