“The 269 billion parameter ST-MoE-32B model has a computational cost comparable to a 32 billion parameter dense encoder-decoder Transformer model.”