“Direct Preference Optimization (DPO) is simpler to implement and work with than other RLHF methods, leading to its widespread adoption in open source.”