“Offline RL methods are likely being used in industry for RLHF because they simplify the training process and reduce inference costs.”