A key problem with training on synthetic data from LLMs is that the model outputs are "silently c..., Sonic AI
“A key problem with training on synthetic data from LLMs is that the model outputs are "silently collapsed," meaning they lack diversity and occupy a very small portion of the possible output space.”