“Large labs like OpenAI likely focus their preference data collection on "good answer vs. good answer" comparisons, having learned to filter out other less useful data types.”