“In internal experiments by the Qwen team, their linear attention architecture performed better than Sliding Window Attention (SWA).”