“The total FLOPs (C) for training a Transformer model over D tokens can be approximated as C ≈ 6ND, where N is the number of non-embedding parameters.”