The distillation approach to creating synthetic data, where a smaller model is trained on the out..., Sonic AI
“The distillation approach to creating synthetic data, where a smaller model is trained on the outputs of a larger model, cannot produce a model that is more capable than the original teacher model.”