Modal CEO: Has Open-Source LLM Inference Peaked? | What Happens to the Cost of Compute

Modal Financial & Valuation Milestones

• Achieved $1 billion in Annual Recurring Revenue (ARR) as of the interview.
• Closed a $750 million Series D round led by Excel, valuing the company at $15.75 billion.
• Projected to reach $4 billion in revenue within the next 12 months, representing a 50% growth rate over the prior 12-month period.
• Reported profitability on a P&L basis, though cash flow remains negative due to heavy capital expenditure on GPU capacity and infrastructure.
• Sandbox product line now generates a $300 million run rate, constituting roughly one-third of total revenue.

Market Outlook & Inference Economics

• Eric Vissio predicts the inference market will expand to a $100 billion opportunity, likening its scale to the cloud computing revolution.
• Forecasted 5x to 20x growth for both open-source and frontier model inference markets over the next 12 months.
• Believes a market pullback of 20–30% is inevitable but views it as a temporary dip in a massive long-term trend.
• Estimates that 90% gross margins are obsolete in the current AI stack due to multi-layer vendor economics, predicting sustainable long-term gross margins for platforms like Modal between 40% and 60%.
• Anticipates a "specialization" trend where the inference market fragments into distinct products for low-latency, batch, and post-training tasks rather than a single monolithic provider.

Operational Strategy & Infrastructure

• Controls 0.1% of the world's compute but aggregates demand across thousands of customers to smooth variable workloads, enabling efficient capacity planning.
• Builds proprietary infrastructure (file system, container runtime, scheduler) to avoid reliance on Kubernetes or Docker, allowing for rapid scaling of millions of sandboxes per minute.
• Purchases GPUs based on 3–4 month lead times using a linear or exponential extrapolation of customer growth rates (30–40% month-over-month).
• Currently employs 150 software engineers, with an estimated monthly spend of $500,000 on coding agents, aiming to increase agent utilization by 10x.
• Launching new product lines including training services, batch processing, and reinforcement learning tools to expand the platform beyond inference.

Talent Acquisition & Competitive Landscape

• Faces intense competition for engineering talent, with new graduates receiving offers of over $20 million total compensation from major AI labs.
• Modal competes by offering equity packages with significant upside and establishing a precedent for employee secondaries (up to 10% of equity) to provide liquidity.
• Targets 95th percentile compensation but relies on equity potential rather than matching peak cash salaries offered by competitors.
• Identifies "brand" and "community movement" as critical defensibility factors, noting strong community engagement on social media compared to competitors like Base10 and Fireworks.
• Views "eating glass" founder culture as counterintuitive; emphasizes enjoyment of daily work and a 20–50 year horizon rather than short-term exit strategies.

Strategic Philosophy & Founder Insights

• Vissio (age 36/37 at founding) believes the ideal founder age is 32, balancing experience with the ability to navigate people management and hiring.
• Rejects the notion of early-stage "moats," citing Ben Thompson's "gingerbread man" strategy: survival depends on speed, team quality, and brand, not static barriers.
• Advocates for "product-led growth" where developer experience and visual identity drive adoption, contrasting with traditional enterprise sales motions.
• Critiques the "two-on-ten" seed model (raising small amounts to build for 18 months), labeling it outdated in the current high-growth AI era.
• Expresses skepticism regarding Neo-clouds (e.g., CoreWeave, Nebius) moving up the stack, citing the difficulty of replicating five years of complex infrastructure engineering while managing core financial constraints.

Broader Industry Perspectives

• Predicts Europe will struggle to produce competitive open-source models due to difficulties in aggregating capital and human talent at scale, despite high individual human capital.
• Believes CUDA is overstated as a moat; NVIDIA's true advantage is its ecosystem, kernel optimizations, and first-class support for open-source engines.
• Foresees hardware prices (GPUs) eventually falling due to gravity (supply catching up to demand), though current shortages may persist for 1–5 years.
• Supports the strategy of enterprises owning their own intelligence via fine-tuned open models for consistency and control, while acknowledging this is not necessary for all use cases.
• Identifies drug discovery for chronic conditions as a high-conviction application for AI, citing personal motivation and rapid progress in AI-aided design despite regulatory hurdles.