“The current method of training Large Language Models (LLMs) by up-weighting every token in a successful trajectory is a conceptually simple form of Reinforcement Learning (RL).”