“The InstructGPT version of a GPT-2-sized model was preferred by human evaluators over the base GPT-3 model, despite being two orders of magnitude smaller in parameter count.”