The 70B parameter Chinchilla model, trained on 1.4T tokens, outperformed the 280B parameter Gophe..., Sonic AI
“The 70B parameter Chinchilla model, trained on 1.4T tokens, outperformed the 280B parameter Gopher model, which was trained on 300B tokens, under the same compute budget.”