Oct 14, 2026, 10:30 AM ETInterview, Fireside Chat

Can Open Models Keep Up With OpenAI and Anthropic? | Nathan Lambert

Model advancement is defined by three forms of capital: compute, data, and talent.
Frontier labs (OpenAI, Anthropic) currently possess the highest concentration of these resources, enabling them to take the most ambitious bets earliest.
Organizational structure is a critical differentiator; labs must frequently reorganize from the bottom up to optimize model-building efficiency, with Meta cited as a recent successful example.
Chinese labs possess a talent pool comparable to Western labs but are significantly compute-starved.
The average OpenAI researcher is reported to have access to approximately 5,000 H100 equivalents, a scale described as incomprehensible to top Chinese researchers.
Chinese companies primarily adopt open-weight models as a practical commercialization strategy to access the US market, rather than purely altruistic distribution.
Chinese labs are described as having a more serious approach to security due to stricter government enforcement and consequences compared to the US.

The lag time between frontier model releases and high-quality open-source equivalents has narrowed to approximately two to three months.
The Chinese strategy for catching up relies heavily on model distillation to effectively purchase compute-equivalent performance.
Previous post-training was dominated by supervised fine-tuning (SFT); the current paradigm has shifted toward reinforcement learning (RL)-heavy environments.
There is ambiguity in the technical community regarding whether distillation is the primary driver for Chinese model improvements, with some suggesting their methods are under-documented or distinct.
Frontier labs argue that closing off APIs for specific models is necessary to prevent easy distillation of proprietary reasoning traces and to capture higher margins.
Distilling reasoning models is harder than distilling standard LLMs because reasoning traces (thinking tokens) are often hidden behind enterprise agreements or blocked by API protections.
Nathan (the interviewee) suggests that distillation does not guarantee a massive performance leap for open labs due to the difficulty of accessing high-quality "teacher" reasoning traces.
A "fast follower" dynamic exists where open labs leverage the data industry infrastructure built by frontier labs (e.g., specialized datasets) sold publicly after a delay.

Frontier labs operate on a flywheel where massive upfront capital expenditures on compute generate revenue, which funds further exponential scaling.
The open-source ecosystem follows a slower, more fragmented trajectory involving model builders, inference providers, and fine-tuning services.
Open-source models benefit from direct customer feedback loops regarding specific enterprise needs (e.g., private data usage), which is less accessible in closed API models.
The "Near Frontier" tier of open models is expected to emerge, capturing markets like private enterprise data that are less profitable for closed labs to serve directly.
Companies are increasingly moving data collection and RL environment creation in-house to prevent signal leakage and protect intellectual property.
The data industry faces quality issues, including "reward hacking" and low-quality synthetic data, necessitating years of transition to higher ROI data pipelines.
Specialized tasks in math and code are becoming increasingly dominated by frontier models due to the scalability of general reasoning.
Conversely, non-verifiable knowledge work (e.g., biology, healthcare, legal) remains a domain where open fine-tuned models can compete effectively using specific, proprietary datasets.
Inference costs are projected to drop multiplicatively (potentially 2x every eight months), creating a Jevons Paradox scenario where demand may outpace margin preservation.

Trillium is a new nonprofit open lab focusing on releasing open-source environments and tools to demystify Recursive Self-Improvement (RSI).
The initiative aims to build an open stack for SFT, mid-training, and RL to provide transparency on the actual mechanics of RSI.
The lab plans to document specific tasks, configurations, compute usage, and evaluation results to create a verifiable record of progress.
This approach seeks to counter the narrative of a "hidden" frontier by demonstrating that RSI techniques are replicable given sufficient capital and data.
The name "Trillium" is chosen to symbolize spreading nutrients and enabling new growth in the AI industry before the "canopy" of large companies dominates.
Trillium intends to release high-leverage open-source RL environments (e.g., for terminal tasks) that are too costly for most actors to create individually but are critical for research.
There is a concern that LLMs and AI agents may overwhelm academic communities with noise (e.g., fake research results), making genuine contribution harder for PhD students.
The interviewee expresses confidence that RSI will not lead to an immediate "fast takeoff" due to the exponential resource requirements vs. linear performance gains of current autoregressive models.

The AI capability plot does not cause significant alarm, but the lack of preparation in public cybersecurity is a primary concern.
The interviewee believes the current "fear" regarding the frontier breaking away is overstated given the known constraints of scaling laws.
Frontier labs are expected to maintain a lead in serving the cheapest intelligence per watt on high-demand evaluations through infrastructure integration.
The market is expected to have many winners, with frontier labs dominating high-scale general tasks while specialized open models capture niche, high-value domains.
Mathematical domains (like Navier-Stokes) are viewed as the most suitable for the current form of intelligence, whereas other forms of discovery may require new architectures.
There is a consensus that knowledge spreads rapidly in Silicon Valley (e.g., through informal networks), making true trade secrets regarding model methods difficult to maintain long-term.
The industry is shifting toward agent swarms and coordination systems, a space frontier labs are currently leading but which will eventually be adopted by the open sector.
Security risks are expected to diffuse as closed models reach the frontier, followed by open models, creating a staggered timeline of vulnerability.
The interviewee advocates for increased "eyes on the problem" via open science to help society prepare for the inevitable integration of advanced AI capabilities.