“A current limitation of Mixture-of-Experts models is their regular structure, where each expert is the same size and paths through the model merge back together quickly.”