“Andre Karpathy's 'auto-research' agent, running overnight on his NetoChat project, discovered superior hyperparameter tunings, such as for weight decay on value embeddings and Adam betas, that he had missed after two decades of manual research experience.”