Mercore is leveraging AI to automate talent assessment, while also becoming a key provider of high-skill human experts to top AI labs for model evaluation.
The speaker predicts a massive shift in the data annotation market, away from low-skill crowdsourcing (SFT/RLHF) and towards a rapidly growing, high-value market for expert-led AI evaluations ('evals').
A significant portion of future knowledge work will involve humans creating evaluation frameworks and tests for AI systems, rather than performing the tasks themselves.
The company's strategy is to first dominate the human data market for AI labs, then expand its AI-powered platform to high-volume contractor and full-time hiring markets.
11 quotes
Concerns Raised
The public's primary concern about AI will shift to models being too aligned and lacking creativity.
Companies are misallocating billions on overhyped and less effective SFT/RLHF data.
Human hiring processes are often flawed due to subjective, 'vibes-based' assessments.
Opportunities Identified
The market for high-quality, human-generated AI evaluations is set to grow by orders of magnitude.
Automating candidate assessment with LLMs to build a more efficient and unified global labor market.
Expanding from the AI data niche to disrupt the broader high-volume contractor and full-time hiring markets.
Building custom AI agents to automate specific knowledge work verticals, such as in finance.