Sequoia Capital39 min
Fireworks Founder Lin Qiao on the Power of Small Models to Democratize AI Use Cases
Lin Qiao, Sonya Huang, Pat Grady, Lynn Tiao
Interview, Fireside Chat
Fireworks Mission & Origins
- Founded in 2022 to compress industry time-to-market from an estimated five years down to five weeks or five days.
- Founded by former Meta PyTorch leaders (Lynn Diao and others) to address the gap between research model creation and production-scale inference.
- The company operates as a SaaS platform focused on high-quality tuning and low-latency inference using a "small model stack."
Meta PyTorch Migration History
- Meta originally maintained three distinct frameworks: Caffe2 (mobile), Onyx (server-side), and PyTorch (research).
- The decision was made to consolidate into a single stack by retaining PyTorch's frontend while rebuilding the backend from scratch.
- The migration, initially estimated as a six-month project, evolved into a five-year undertaking requiring a complete rebuild of the inference and training stack.
- Upon completion, the unified stack supported over 5 trillion daily inferences at Meta.
Strategic Bet on PyTorch
- Fireworks chose to be PyTorch-native because it dominates the "top of the funnel" for model creation by researchers.
- The "funnel effect" dictates that models created in PyTorch naturally flow downstream to production, making framework conversion difficult and inefficient for customers.
- The company avoids being framework-agnostic to avoid distraction, focusing entirely on optimizing the PyTorch ecosystem.
Technical Architecture & Optimization
- Fireworks differentiates itself as a system rather than a library, automating complex optimizations (e.g., auto-tuning, quantization, and caching) behind a simple API.
- Performance is achieved through handwritten CUDA kernels, distributed inference across nodes, and disaggregated inference (chopping models across GPUs).
- The platform implements semantic caching to avoid re-computation for recurring application workload patterns.
- The system utilizes function calling to act as a router for accessing both public APIs and private enterprise APIs, effectively creating a "Mixture of Experts" architecture over hundreds of specialized models.
Market Trends & Customer Segmentation
- The customer journey typically begins with startups using OpenAI for heavy experimentation, followed by a migration to Fireworks for production scaling, low latency, and cost reduction.
- Customer acquisition targets a specific progression: startups first, then digital-native enterprises, and finally traditional enterprises (e.g., healthcare, banking).
- A shift in decision-making is observed where CTOs are replacing CIOs/CISOs as the primary buyers, driven by the need for innovation-led business transformation.
- Traditional enterprises are adopting GenAI due to a fear of obsolescence and the realization that foundation models allow them to fine-tune rather than train from scratch.
Open Source vs. Closed Source Dynamics
- Prediction: Model quality for open-source and closed-source models of the same size (e.g., 7B–100B parameters) will converge within a few years.
- Differentiation will shift from raw model capability to customization and fine-tuning for specific enterprise workloads.
- Open-source models (e.g., Llama 3) offer a richer ecosystem for researchers to build upon, enabling a "thousand flowers" approach to solving diverse enterprise problems.
- Fireworks believes small, specialized models are more effective for specific tasks than monolithic general models.
Fine-Tuning & Product Design
- Fine-tuning is identified as a complex workflow involving data labeling, algorithm selection (e.g., SFT, DPO), and hyperparameter tuning, which Fireworks aims to automate.
- The company distinguishes between "failure cases" and "product design failures," requiring product managers to define system behaviors for edge cases (e.g., auto-generation in tables).
- Off-the-shelf models perform well in domains like coding and translation, but fine-tuning is required for domains requiring specific business logic (e.g., insurance summarization templates).
Competitive Landscape & Future Outlook
- Nvidia: Competition is expected to emerge soon due to market economics and industry pressure against monopolies, particularly as the AI model space stabilizes.
- OpenAI: Fireworks does not view OpenAI as a direct threat because OpenAI's mission is AGI, whereas Fireworks focuses on enterprise-specific customization and cost efficiency for smaller models.
- Market Asymptote: While model capabilities may plateau in applicability, innovation in model architecture is expected to continue, necessitating continued optimization.
- Agents: Lynn Diao expresses high bullishness on the performance of AI agents in the coming year.
Rapid Fire Insights
- Favorite AI App: Fathom (used by sales teams for meeting summarization and training).
- 2024 Model Prediction: Quality will continue to increase with new models released weekly; specific winners are hard to predict, but overall improvement is certain.
- Most Admired Entity: Meta, for its commitment to open-sourcing Llama models and decentralizing power from hyperscalers.