“The success of Direct Preference Optimization (DPO) in open source is currently limited to its application with the UltraFeedback dataset, which has been shown to boost performance on benchmarks like AlpacaEval, MT-Bench, and TruthfulQA.”