Dylan Patel — The single biggest bottleneck to scaling AI compute

Hyperscaler & Lab CapEx Deployment Timelines

• $600B Combined Forecast: The "Big Four" (Amazon, Meta, Google, Microsoft) have a combined forecasted CapEx of $600 billion for the current year.
  • This capital expenditure does not represent immediate 50 gigawatts of deployed compute.
  • Significant portions are allocated to long-lead items like turbine deposits for 2028–2029 and data center construction for 2027.
• Incremental Capacity Reality: Only approximately 20 gigawatts of incremental compute capacity is expected to come online in the US this year.
  • A portion of this year's spending was actually incurred in the prior year.
  • Contracts often cover future setup rather than immediate hardware delivery.
• Lab Raises vs. Needs: OpenAI raised $110 billion and Anthropic raised $30 billion.
  • These sums are sufficient to cover compute rental costs for the year at an estimated $10–13 billion per gigawatt.
  • This coverage excludes the substantial revenue these labs will generate during the year.
• Anthropic's Compute Gap:
  • Current Anthropic capacity is estimated at 1.5–2.5 gigawatts.
  • To sustain projected revenue growth (adding $60 billion over 10 months), Anthropic requires an additional 4 gigawatts of inference capacity.
  • This implies a total requirement of roughly 5 gigawatts by year-end to maintain current R&D fleets.

The Compute Acquisition Conundrum

• OpenAI's Aggressive Strategy: OpenAI secured compute from a diverse range of providers, including CoreWeave, Oracle, SoftBank Energy, and nScale.
  • They signed five-year deals for the vast majority of their capacity.
  • They prioritized locking in capacity early to avoid future bottlenecks.
• Anthropic's Conservative Lag: Anthropic previously adopted a "principled" conservative stance to avoid bankruptcy risks.
  • This resulted in a disadvantage when facing sudden, massive demand spikes.
  • They now face a "conundrum" requiring last-minute acquisitions.
• Cost of Last-Minute Capacity:
  • Acquiring compute in a pinch requires accessing "neoclouds" or spot markets with shorter-term contracts.
  • This often involves paying 50% markups via revenue share models (e.g., serving through AWS Bedrock or Google Vertex).
  • Short-term spot prices for H100s have surged, with some labs signing deals at $2.40/hour.
  • These rates imply gross margins of over 35% for cloud providers, significantly higher than long-term contract margins.

Depreciation Cycles & GPU Value

• Rejection of Short Cycles: The assumption that GPUs depreciate within 3–4 years (as argued by Michael Burry) is challenged.
  • The actual depreciation cycle may be closer to 5 years or longer.
  • Older chips (e.g., A100s) may see reuse as 7nm fabs become necessary for mass production in 2030.
• Value Drivers Beyond Flops: The value of a GPU is not solely determined by its raw FLOPS performance but by:
  • Model Efficiency: Newer models (e.g., GPT-5.4) are cheaper to run than predecessors (GPT-4) despite being more capable, due to sparsity and architectural advances.
  • Token Yield: An H100 can serve significantly more tokens of a high-quality model (GPT-5.4) than an older model (GPT-4).
  • Utility vs. Price: As long as the AI "takeoff" continues, the utility of existing chips remains high because the alternative (newer chips) is supply-constrained.
• Arkin-Allen Effect: As fixed costs for compute rise, the relative price difference between "best" and "average" models shrinks.
  • High fixed costs push providers to choose the highest-quality models to maximize the utility of the expensive compute.
  • This concentrates market share and revenue on the top-tier models (e.g., Opus, GPT-5).

Semiconductor Supply Chain Bottlenecks

• The EUV Tool Constraint: The ultimate bottleneck by 2028–2030 is ASML's EUV (Extreme Ultraviolet) lithography tools.
  • Current production is ~70 tools/year, rising to ~80 next year and ~100 by the end of the decade.
  • It takes approximately 3.5 EUV tools to produce the wafers required for 1 gigawatt of AI data center capacity.
  • Even with 700 EUV tools by 2030, the theoretical max is ~200 gigawatts of AI chip capacity, assuming 100% allocation to AI.
• Supply Chain Rigidity:
  • The supply chain for EUV tools is not just ASML; it involves complex, artisanal components (e.g., Zeiss optics, Trumpf/Synerion sources).
  • Expansion is slow due to the need for highly specialized labor (e.g., <1,000 Zeiss engineers) and multi-year build times.
  • "Production hell" historically takes 5–7 years to move from lab functionality to high-volume manufacturing.
• 3nm & N2 Node Shifts:
  • TSMC is prioritizing AI customers (NVIDIA, Google, Amazon) over Apple for 3nm and N2 capacity.
  • Apple, historically the first customer for new nodes, will likely become a smaller percentage of TSMC's revenue and volume.
  • Huawei is noted as having the potential to exceed NVIDIA if it had access to 3nm+ process technology, given its strong software, networking, and talent base.

Memory Constraints

• HBM vs. DDR Trade-offs: The industry cannot simply switch to commodity DDR DRAM to solve memory shortages.
  • Bandwidth is the Bottleneck: AI training and inference are constrained by memory bandwidth (moving weights and KV cache), not just capacity.
  • HBM Advantage: An HBM4 stack provides ~2.5 TB/s bandwidth in a 13mm footprint, whereas DDR5 provides ~64–128 GB/s in the same footprint.
  • Wasted Compute: Switching to DDR would leave GPU FLOPS idle waiting for data.
• Impact on Consumer Electronics:
  • 30% of hyperscaler CapEx in 2026 is allocated to memory.
  • Smartphone volumes are expected to halve in the low/mid-range segment due to memory price inflation (DDR prices tripling).
  • This will likely increase the cost of an iPhone by ~$150–$250 as manufacturers pass on costs.
• Fab Construction Lag:
  • Memory vendors delayed building new fabs during the 2023 downturn.
  • New fabs take ~2 years to build; meaningful capacity additions will not arrive until late 2027–2028.

Power & Infrastructure

• Power is Not the Primary Bottleneck: While critical, power generation can scale to hundreds of gigawatts by 2030.
  • Potential sources include gas turbines, aeroderivatives, reciprocating engines, ship engines, and fuel cells.
  • "Behind-the-meter" generation (building power on-site) is becoming a major strategy due to interconnection queue delays.
  • Unlocking 20% of the US grid's peak capacity (currently unused) could add ~200 gigawatts of data center power.
• Labor as a Constraint:
  • Building 200 gigawatts of power generation requires hundreds of thousands of specialized electricians and construction workers.
  • Solutions involve modularizing data centers (shipping pre-integrated racks) to reduce on-site labor needs.
• Space Data Centers Skepticism:
  • Space data centers do not solve the fundamental bottleneck: semiconductor chip manufacturing.
  • Deployment Delays: Testing and shipping GPUs to space adds 6+ months of delay, which is critical in a fast-scaling industry.
  • Networking Challenges: Inter-satellite laser links are more expensive and less reliable than terrestrial fiber/optical transceivers.
  • Thermal Physics: Higher power densities (required for efficiency) are difficult to manage in the vacuum of space compared to Earth's liquid/immersion cooling.

Strategic Implications: US vs. China

• US Advantage: The US and its allies (TSMC, ASML, Samsung) hold the majority of advanced semiconductor capacity.
  • The US is building the infrastructure (data centers, memory, logic) to generate massive ROIC, potentially outpacing China's economic growth.
• China's Path:
  • China lacks a fully indigenized EUV supply chain; they currently rely on ASML DUV tools.
  • While China aims for a fully domestic 3nm/7nm ecosystem by 2030, mass production will lag behind the West due to "production hell."
  • If AI takeoff timelines are "fast" (<2028), the US wins. If timelines are "slow" (>2030), China may catch up via scale and DUV/multi-patterning.
• The "Distillation" Barrier: As US models become significantly more capable, the ability to distill knowledge from US models into Chinese models decreases, potentially widening the performance gap.

Specific Architecture & Scaling

• Scale-Up Topologies:
  • NVIDIA: Uses "all-to-all" networking within a rack (72 GPUs), enabling terabyte-scale bandwidth but limiting scale-up domain size.
  • Google: Uses a "torus" topology (8,000+ TPUs) where chips connect to only 6 neighbors, enabling massive scale but introducing routing latency.
  • Amazon: Moving toward a hybrid "dragonfly" topology to balance scale and latency.
• Model Size & RL Efficiency:
  • Smaller models (e.g., 1 trillion parameters) are often preferred over massive ones (5 trillion+) for Reinforcement Learning (RL).
  • Faster RL feedback loops on smaller models allow researchers to iterate ideas faster, compounding efficiency gains more rapidly than waiting for massive models to converge.