Kaplan et al. (2020) suggested that for a 10x increase in compute, model size should be scaled by..., Sonic AI
“Kaplan et al. (2020) suggested that for a 10x increase in compute, model size should be scaled by approximately 5.5x and the number of training tokens by approximately 1.8x.”