Interview, Fireside Chat
Modal CEO: Has Open-Source LLM Inference Peaked? | What Happens to the Cost of Compute
- The inference market is anticipated to experience a short-term pullback and a 20% to 30% stock market correction, with hardware prices potentially rising for one to five years before declining long-term, though the long-term opportunity is expected to expand 10X to 100X.
- Financial projections include reaching a $4 billion valuation or higher, achieving 40% to 60% long-term gross margins, and driving the training product to a $100 million run rate within six to 12 months.
- Specific product milestones target a $300 million run rate for the sandbox product by summer 2025, while revenue is projected to grow 50% over the next 12 months.
- The company plans to scale GPU capacity by 30% to 40% monthly to reach two to three times current capacity within three to four months, aiming to offer 100 GPUs on short notice without requiring long-term customer commitments.
- Strategic growth involves expanding into adjacent products such as continuous integration, agent harnesses, and model training, while targeting a 5X, 10X, or 20X growth rate for both frontier and open source LLM inference markets over the next year.
- Operational risks include near-term capital constraints due to interest rates and data center construction delays, as well as potential difficulties for companies reliant on continual equity infusion during a venture capital pullback.
- Market dynamics predict fragmentation into specialized areas like low latency and fine-tuning, a potential plateauing of open source inference due to cheap frontier products, and a resurgence of US frontier providers as enterprises avoid Chinese models.
- Regional analysis suggests Europe may struggle to aggregate capital and human capital compared to the US, while open source models are expected to shift toward licensing models to fund training.
- The organization intends to maintain aggressive hiring and compensation strategies to attract top talent, including new graduates, alongside plans to expand FDR and sales rep roles to enhance customer experience.
- Long-term vision includes operating the company for 20 to 50 years to build a $100 billion specialized entity, defying the 90% software margins of a decade ago in favor of sustainable gross margins.
- Defensibility relies on the team, speed of product decision-making, and brand value, with expectations that software engineering demand will remain high for 10 to 20 years due to increased efficiency and cloud spending on coding agents.