“Reinforcement learning is a more compute-efficient method for incorporating human feedback signals into language models compared to attempting to achieve the same effect during pre-training.”