Interview
Inference's Scaling Era | Tuhin Srivastava, Baseten & Corinne Riley, Greylock | RAISE Summit 2026
- The inference market is projected to become one of the world's largest by 2030, with total AI spending expected to reach approximately $5 trillion, though the accuracy of this figure carries uncertainty.
- The app layer is anticipated to generate roughly $2.5 trillion in inference spend based on an assumed 50% gross margin, driven by continued growth as AI adoption accelerates.
- Base 10 is developing an "inference cloud" integrating compute, software, and hooks for reinforcement learning, post-training, evaluations, sandboxes, and routing, currently operating clusters across 20 clouds in 90 regions.
- The performance gap between open-source and closed-source frontier models is expected to narrow further as ecosystem partners invest, with current lags standing at three months or less.
- Open-source models are predicted to deliver 95% to 100% of the intelligence of closed labs at 70% lower prices, with 77% of current code generation already powered by open-source solutions.
- Enterprise adoption of open-source is expected to expand aggressively beyond early startups to large digital natives and Fortune 500 companies, potentially resulting in 90% open-source powered organizations within 12 to 18 months.
- Token volume utilizing open and custom models among sophisticated customers is forecast to reach between 40% and 50% within one year, with a lower bound possibility of 10%, varying by company task and post-training familiarity.
- Significant token spend will emerge from infinite generation scenarios, particularly in the biology and pharma sectors, as the provider ecosystem supporting open-source model production continues to expand.
- Base 10 has begun significant investments this year in software for post-training and techniques for personalizing and specializing models to support this shift.
- NVIDIA has publicly committed to investing in the Nemotron open-source model, and open source is expected to remain viable for enterprises despite regulatory questions, supported by a blueprint similar to the open web.
- Uncertainties include the precision of the $5 trillion 2030 spend estimate and the specific percentage of open-source usage, which will vary by company rather than reaching a uniform level across the industry.