Keep pulling the thread on Yang Zhilin.
MoonEP is an Expert Parallelism communication library designed to maintain perfectly balanced token loads across ranks through the use of dynamic redundant experts.
The MoonEP library ensures every rank receives exactly S × K tokens (input tokens per rank multiplied by top-k routes per token), regardless of how skewed the routing is.
In benchmark tests, MoonEP's communication time is consistently lower than that of DeepEP v2 at every level of router imbalance.
MoonEP's communication time remains almost flat as router imbalance increases, whereas DeepEP v2's latency degrades steadily under the same conditions.
During end-to-end training benchmarks, MoonEP's iteration time remains flat at every level of router imbalance.
Training with MoonEP avoids GPU memory fragmentation and Out-of-Memory (OOM) errors at high router imbalance, issues that affect DeepEP v2.
MoonEP utilizes a near-optimal GPU planning kernel for online planning of redundant experts, which has negligible overhead.
MoonEP's use of statically known shapes eliminates the need for per-layer Mixture-of-Experts (MoE) host synchronization.
Support for the Zhenwu PPU in the MoonEP library is currently under review and is planned for a future release.