“Andrej Karpathy states the significant improvement in chess-playing ability from GPT-3.5 to GPT-4 was a direct result of someone at OpenAI adding a very large amount of chess data to the pre-training set.”