“Factuality tuning improves factuality by fine-tuning a model with DPO on a dataset where preferences are assigned based on truthfulness scores derived from either external references or the model's own confidence.”