“InstructGPT demonstrated that a moderate amount of fine-tuning can significantly shift model behavior towards better alignment on GPT-3-sized models.”