“Gemini 1.5 Pro achieved superior performance to Gemini 1.0 Ultra while using significantly less compute, partly due to a Mixture of Experts (MoE) architecture.”