Training a large language model on synthetic data generated by asking it to recreate the internet..., Sonic AI
“Training a large language model on synthetic data generated by asking it to recreate the internet is generally not a good idea and leads to a degradation in model quality.”