Interview, Fireside Chat
Inference Will Be $10TRN Market: What No One Sees | Software Investors are F***ed | Naveen Rao
- Energy as the Primary Constraint: Naveen Rao predicts the industry will run out of excess energy capacity within a few years; building one gigawatt costs approximately $50 billion, making it more complex and expensive than building a semiconductor fab ($20–30 billion).
- Grid Capacity vs. Demand: The U.S. grid operates at ~900 gigawatts with a peak capacity of 1.3 terawatts; while current data center growth has relied on this 44% excess capacity, Rao argues the U.S. is years behind China, which has built six times more capacity over the last two decades.
- Infrastructure Investment Outlook: Building 40 gigawatts of new capacity in a single year is deemed impossible under current conditions, with a realistic target closer to 10 gigawatts (valued at roughly $500 billion), indicating a significant backlog in infrastructure.
- Two-Tiered Energy Pricing: Data centers are already paying a premium for electricity ("behind-the-meter" deals) compared to the general grid, as the high-margin nature of AI compute allows providers to absorb 30–40% higher energy costs.
- Shift to Energy-First Hardware Design: Rao asserts that physical efficiency gains (Moore's Law/Dennard scaling) have largely stalled, shifting the focus to co-designing models and architecture to optimize for energy rather than raw speed.
- Model Efficiency Gains: Significant efficiency improvements remain possible through "co-evolution" of model and physical layers (e.g., exploiting sparsity), which standard GPUs cannot fully utilize due to memory architecture limitations.
- Hardware Specialization: The compute landscape is moving toward specialized, workload-specific chips (e.g., Cerebras, Groq) rather than horizontal commoditization; Rao believes the hardware stack will not commoditize because it remains the most valuable layer for converting energy to intelligence.
- Inference Market Valuation: The inference market is projected to reach a $10 trillion valuation within 10 years, driven by infinite demand for intelligence in an "adversarial" global environment.
- Investment Advice for Software Investors: Pure software/SaaS investors are described as being in a precarious position ("kind of fucked") because the historical SaaS margin model is ending; future value will accrue to hardware and energy conversion capabilities rather than software applications.
- Commoditization Timeline: While open-source models may become commoditized within six months (similar to older software), high-margin value will remain concentrated in frontier models for specific, complex tasks, creating a market structure similar to the mobile phone industry (20% of models capture 80% of margins).
- RL vs. Pre-training: Reinforcement Learning (RL) is identified as a critical, albeit inefficient, paradigm for the next few years because it allows models to solve tasks via verifiable rewards where pre-training data is insufficient.
- Post-Transformer Architecture: Rao agrees with Yann LeCun that the current transformer architecture is inefficient for reasoning and that the industry will eventually move to a "post-transformer" world with more direct, biologically inspired reasoning mechanisms.
- Verification as the New Bottleneck: As AI production becomes cheaper and faster, the primary industry bottleneck shifts from generating artifacts to verifying their correctness; owning the verification layer is considered key to controlling the automation stack.
- Robotics Investment Timeline: Rao disagrees with the view that robotics is five+ years away, predicting working prototypes for specific physical tasks within 2–3 years and a major market emergence in 5–7 years.
- Robotics vs. Agentic Coding: The physical robotics market is expected to be 100 times larger than agentic coding because physical problem-solving in the real world represents an unexplored frontier for AI intelligence.
- Founding Principles: Successful founders in the AI era must possess a strong future vision rather than just execution skills; the "big bet" mindset required for deep tech (e.g., curing cancer) requires ignoring traditional market validation metrics in favor of technical progress metrics.
- Strategic Vertical Integration: If unconstrained by capital, Rao would build custom manufacturing and fabs to hyper-optimize the silicon layer for intelligence delivery, rather than relying on commodity hardware.
- Overrated Sectors: Customer support AI and agentic coding interfaces (excluding model providers) are cited as overrated and non-differentiated investment targets due to low defensibility and easy commoditization.
- Future Market Share: Rao predicts 80% of token traffic will flow through open models in three years, though closed/frontier models will capture the majority of financial spend for high-stakes tasks.
- Rejection of AI Doomism: Rao explicitly dismisses the narrative that AI will destroy humanity, arguing instead that AI is the necessary tool to unlock progress in physics and scientific discovery previously beyond human reach.