“An early concrete AI safety result at OpenAI involved teaching a robot to execute a backflip based on human preferences, which was a precursor to the techniques used in ChatGPT.”