“A Kimi Linear model (48B / 3B activated, 1.4T tokens) with Attention Residuals (AttnRes) improved its score on the GPQA-Diamond multi-step reasoning benchmark by 7.5 points.”