“In Mixture-of-Experts models, efficiency is achieved by serving large batches of independent requests, which allows a fraction of the batch to be sent through each expert, rather than by avoiding memory retrieval for unused experts.”