“The standard and most effective method for mapping a Mixture-of-Experts (MoE) layer onto multiple GPUs is expert parallelism, where different experts are placed on different GPUs.”