“For efficient inference with large batch sizes, Mixture-of-Experts models access all experts, but each expert processes only a small fraction of the total batch.”