Aug 11, 2026, 9:00 AM ETConference Presentation, Fireside Chat

How Harvey Built a Research Lab on a Budget | Gabe Pereyra

Harvey operates as an "application layer" company competing against well-funded frontier labs by leveraging the frontier ecosystem rather than building all infrastructure internally.
The company's high-level playbook focuses on building benchmarks, post-training models via "Neolabs" (infrastructure providers), and serving models in production at scale.
Gabe (co-founder and president) argues that application companies can now achieve "frontier intelligence" through post-training, even if they cannot match the general capabilities of frontier pre-training.

The Sensitive Data Constraint: Harvey cannot train on client data (e.g., law firm contracts, emails) due to privilege and sensitivity, necessitating synthetic data generation.
Synthetic Data Strategy: The company utilizes "vibe coding" by domain experts (lawyers) to guide coding models in generating realistic, complex data sets.
- Experts plant specific issues into synthetic "data rooms" (e.g., missing contracts, mismatched clauses) to create ground-truth validation sets.
- Tools like Mercore and Snorkel are used to scale the realism of these synthetic contracts.
Released Data Sets:
- Legal Agent Bench: A taxonomy of tasks associates perform, covering complex drafting and case law research.
- Contracting Data Set: Designed to teach agents negotiation capabilities similar to in-house departments.
- Diligence Data Set: A large RL environment with data rooms up to 80 million tokens, used to research long-context and complex task performance.
Open Source Philosophy: Harvey open-sources benchmarks to validate data quality, encourage community pull requests, and ensure labs benchmark against their standards.
- Elon Musk retweeted one of their datasets, increasing visibility.
- The strategy is to share general synthetic techniques while retaining proprietary data derived from specific client private data.

Role of Neolabs: Partnerships with infrastructure providers (Fireworks, Base10, Trajectory, Applied Compute, Ngram) allow Harvey to bootstrap training with existing recipes and expertise.
- Providers offer specialized optimizations: KV compaction (Base10), enterprise search (Ngram), and specific model training (Trajectory/Nemotron).
Multiple Vendor Strategy: Harvey works with multiple Neolabs simultaneously to diversify bets, manage bandwidth constraints, and access different research philosophies and open-source models (e.g., GLM 5.2, Nemotron, Kim-E3).
Internal Capabilities: The company is increasingly performing post-training internally using APIs like Tinkerer, reducing the need to build custom training infrastructure from scratch.
Goal: To build a "Harvey version of Composer 1," packaging synthetic data work and post-training into a deployable model alongside closed-source options.

Serving Complexity: Harvey serves models across 60 countries with multiple product surfaces, requiring a robust matrix for handling model families, fallbacks, and SLAs.
Pre-Production Evaluation:
- Automated Evals: Uses the Legal Agent Bench to quickly gauge frontier status and domain strengths.
- Human Testing: Conducts side-by-side comparisons and tests critical user journeys for specific product surfaces.
- Decision Factors: Models are deployed based on aggregated signals including cost, latency, and region availability.
Post-Deployment Monitoring:
- A/B Testing: Used for large rollout changes to track engagement.
- Key Metrics: Uptime, token efficiency, and direct user feedback (e.g., "angry customer emails").
Integration Tactics:
- Naive Swapping: Replacing closed-source models with open-source alternatives (e.g., GLM 5.2) in low-complexity tasks like citation generation.
- Model Routing: Dynamically routing queries to open-source models when they are more appropriate or cost-effective.

Hiring Shift: Harvey moved away from competing for expensive frontier researchers (who command $100M+ packages) toward hiring domain experts (lawyers) and researchers who prefer non-lab environments.
Infrastructure Democratization: The availability of managed training and serving APIs (Fireworks, Tinkerer) reduces the need for deep infrastructure expertise, allowing a smaller team to execute complex post-training.
Domain Expertise: Internal lawyers, trained to use coding models, now lead the creation of training rubrics and synthetic data, replacing the need for traditional ML researchers to manually curate data.

Distribution Mismatch: Synthetic data distributions do not yet perfectly match real-world production data; there is a gap between model performance on synthetic tests and actual customer usage.
Context Management: Current models struggle to manage the massive context (80M+ tokens) found in large legal data rooms effectively.
Continual Learning: The end goal is enabling customers to customize models to their specific workflows without exposing private client data to the training pipeline.
Product Strategy Shift: The focus is moving from individual productivity (like Cursor) to organizational productivity (orchestrating teams, resource allocation, and project management across large firms).
Performance Gaps: Despite open-source progress, a performance gap remains in complex, long-context environments compared to frontier capabilities.

Data Generation Mechanics: Experts create rubrics to "plant" specific errors in synthetic data rooms, allowing the model to be graded on whether it catches known issues.
Benchmark Validation: Open-sourcing benchmarks allows labs to verify data quality, though Harvey balances this by retaining proprietary insights from private client data.
Debugging Pipelines: The "gating" of the pipeline (benchmarks -> product deployment) helps isolate failures; models often overfit benchmarks but fail in out-of-distribution production scenarios.
Competition with Generalist Products: Harvey competes by going "hyper-vertical," addressing the complexities of coordinating large teams and client projects rather than just drafting individual documents.