The ST-MoE-32B model was the first sparse model to achieve state-of-the-art transfer learning per..., Sonic AI
“The ST-MoE-32B model was the first sparse model to achieve state-of-the-art transfer learning performance on tasks including SuperGLUE, ARC Easy, ARC Challenge, XSum, CNN-DM, WebQA, Natural Questions, Winogrande, and ANLI R3.”