Moonshot AI developed a distributed implementation of the MEUM optimizer that partitions its stat..., Sonic AI
“Moonshot AI developed a distributed implementation of the MEUM optimizer that partitions its states across the data parallel group for memory efficiency on NVIDIA GPU clusters.”