“In experiments for both WebGPT and GopherCite, RL training provided limited or no improvement over baselines, particularly when combined with rejection sampling.”