“Qwen's 30 billion parameter MoE model, which activates 3 billion parameters per token, approximates the performance of a 14 to 15 billion parameter dense model.”