“GPT-4 is considered more accurate than human labelers for generating preference data, with an estimated 80% accuracy compared to 70% for humans.”