The conversation emphasizes that AI development has moved into a phase where the ability to evaluate model performance is the most critical factor for improvement. Evals are described as the "product requirement document" for AI models, defining success and guiding the research and development process.
Mercore's success highlights a massive new market for high-skilled human intelligence to train, fine-tune, and evaluate AI. This moves beyond simple data labeling to involve domain experts like radiologists, financial analysts, and even comedy writers in the post-training and reinforcement learning process.
Mercore is presented as the fastest-growing company in history, achieving a $400-500M revenue run rate in under 1.5 years. Remarkably, this growth was achieved with high capital efficiency, with the company being described as "lifetime profitable" and never having burned money.
The market for AI training data has evolved from a crowdsourcing problem solved by companies like Scale and Surge using low-skilled labor to a sourcing problem for high-caliber professionals. Mercore's model is built on providing elite talent, with median pay rates of $95/hour compared to the ~$30/hour of older models.
Keep pulling the thread on Brendan Foody.