“Value functions are not widely used in current reinforcement learning applications like RLHF because they do not provide significant benefits for tasks with verifiable rewards.”