“OpenAI uses an "outer loop optimization" process where they monitor user interactions with models like ChatGPT, mix user prompts with curated data, and then retrain reward models and rerun RLHF to address emergent issues.”