“A model with 100 billion active parameters would be considered optimally trained under Chinchilla scaling laws with approximately 2 trillion training tokens.”