Keep pulling the thread on Gabe Pereyra.
According to Winston, Harvey's token consumption has reached 13 trillion tokens.
Law firms cannot rely on a single AI model provider like Anthropic or OpenAI due to conflict of interest risks, such as when representing a competing AI lab.
Harvey's strategy is to open-source general legal AI capabilities while building proprietary infrastructure that enables law firms to own and build their own models on their unique, sensitive data.
Harvey's review product, which can process 100,000 contracts, can cost up to $20,000 for some jobs.
Gabe Pereyra predicts that customers will receive unexpectedly large consumption-based AI bills, citing an anecdote that Uber's CTO used the company's entire annual coding token budget in three months.
Harvey launched LAB (Legal Agent Benchmark), a benchmark for measuring the performance of AI agents on real-world legal tasks.
Harvey has published initial results on its LAB benchmark showing the performance of closed-source models from OpenAI, Anthropic, and DeepMind.
Harvey has fully open-sourced its Legal Agent Benchmark (LAB) to allow the community to use it for agent evaluation.
Initial results from Harvey's legal benchmark show that while Anthropic's models are strong, there are specific areas where OpenAI's models or open-source models perform better.
Harvey's product focus is on organizational productivity for law firms, which differs from the individual productivity tools being built by AI labs like OpenAI and Anthropic.
Gabe Pereyra believes the relationship between application-layer AI companies and foundational model providers will resemble the dynamic between companies like Snowflake and Databricks and the major cloud providers they are built on.
Harvey is developing a product called Shared Spaces, which allows a law firm and its client to collaborate on legal projects and creates a potential environment for training unique models on that relationship's data.