“Mixture-of-Experts (MoE) and Switch Transformers are an energy-efficient architectural approach for building larger and more capable language models.”