“For efficient serving of large modular models, frequently used experts could be replicated for load balancing, while rarely used experts could be paged out from HBM to slower DRAM.”