Gabe Pereyra, co-founder of Harvey, was in the first class at Google Brain and later worked at Meta.

Harvey's previous benchmark, BigLawBench, was a chat-based QA dataset that was not fully open-sourced due to the company's lack of bandwidth at the time.

Harvey's strategy is to open-source general legal AI capabilities while building proprietary infrastructure that enables law firms to own and build their own models on their unique, sensitive data.

Harvey's founders envisioned building two companies in parallel: a traditional enterprise seat-based SaaS business and a consumption-based business for advanced AI models.

Simple assistant-type queries on Harvey's platform can cost $20 for a single query.

Harvey is one of the largest consumers of embeddings for some of the AI labs it works with.

Harvey's R&D approach is inspired by DeepMind's top-down, goal-oriented strategy rather than Google Brain's more bottoms-up, researcher-led model.

Baritz Off, a former researcher at Google Brain, is now at OpenAI where he ran the post-training team.

Harvey's product focus is on organizational productivity for law firms, which differs from the individual productivity tools being built by AI labs like OpenAI and Anthropic.

Law firms cannot rely on a single AI model provider like Anthropic or OpenAI due to conflict of interest risks, such as when representing a competing AI lab.

Harvey is developing a product called Shared Spaces, which allows a law firm and its client to collaborate on legal projects and creates a potential environment for training unique models on that rela...

Harvey is moving some of its traffic to open-source models because serving the largest closed frontier models has become too expensive.

Harvey works with a range of inference and "as a service" companies including Base10, Fireworks, Together AI, Applied Compute, Trajectory, and Ngram.

Initial results from Harvey's legal benchmark show that while Anthropic's models are strong, there are specific areas where OpenAI's models or open-source models perform better.

The AI research field has shifted from a culture of open publishing on platforms like arXiv to one where major labs no longer publish their most significant work.

Harvey launched LAB (Legal Agent Benchmark), a benchmark for measuring the performance of AI agents on real-world legal tasks.

Gabe Pereyra predicts that customers will receive unexpectedly large consumption-based AI bills, citing an anecdote that Uber's CTO used the company's entire annual coding token budget in three months...

Harvey's review product, which can process 100,000 contracts, can cost up to $20,000 for some jobs.

Anthropic's Opus model is reportedly 3 times more expensive than OpenAI's latest model while being only 10-20% more performant, creating pricing pressure between providers.

Harvey has fully open-sourced its Legal Agent Benchmark (LAB) to allow the community to use it for agent evaluation.