Keep pulling the thread on Andrej Karpathy.
The auto research tool allows a human to set a strategic direction and constraints for an AI agent, which then autonomously conducts experiments within those guardrails.
Andrej Karpathy ran 700 machine learning experiments in two days using his auto research tool, which resulted in 20 genuine improvements and an 11% performance speed-up on his task.
Toby Lutke, CEO of Shopify, used an adaptation of the auto research tool to develop a small machine learning model that outperformed models twice its size.
Tools based on Andrej Karpathy's auto research concept can accelerate workflows, reducing a task that would normally take a week to approximately one hour.
Andrej Karpathy released an open-source software tool called auto research.
Andrej Karpathy's auto research tool is comprised of 600 lines of Python code and has received 57,000 stars on GitHub.
In Andrej Karpathy's design for auto research, each experiment takes approximately five minutes, allowing for up to 12 experiments per hour.
The design of the auto research tool, where a human provides a specific objective that an agent must optimize for, is presented as a solution to the agent control problem.
The speaker has developed a modified version of the auto research tool named 'AutoWolf'.
The 'AutoWolf' tool includes an 'escape harness' feature that introduces random behavior to help the system avoid local minima and find more globally optimal solutions.
The cost to run an analysis using the 'AutoWolf' tool is described as trivial, ranging from single dollars to tens of dollars per run.
The organization Exponential View utilizes a 'ladder of reasoning' architecture that includes tools like single-shot expert panels, auto research, and AutoWolf for increasingly complex problems.