Fireworks Founder Lin Qiao on the Power of Small Models to Democratize AI Use Cases

Fireworks Mission & Origins

• Founded in 2022 to compress industry time-to-market from an estimated five years down to five weeks or five days.
• Founded by former Meta PyTorch leaders (Lynn Diao and others) to address the gap between research model creation and production-scale inference.
• The company operates as a SaaS platform focused on high-quality tuning and low-latency inference using a "small model stack."

Meta PyTorch Migration History

• Meta originally maintained three distinct frameworks: Caffe2 (mobile), Onyx (server-side), and PyTorch (research).
• The decision was made to consolidate into a single stack by retaining PyTorch's frontend while rebuilding the backend from scratch.
• The migration, initially estimated as a six-month project, evolved into a five-year undertaking requiring a complete rebuild of the inference and training stack.
• Upon completion, the unified stack supported over 5 trillion daily inferences at Meta.

Strategic Bet on PyTorch

• Fireworks chose to be PyTorch-native because it dominates the "top of the funnel" for model creation by researchers.
• The "funnel effect" dictates that models created in PyTorch naturally flow downstream to production, making framework conversion difficult and inefficient for customers.
• The company avoids being framework-agnostic to avoid distraction, focusing entirely on optimizing the PyTorch ecosystem.

Technical Architecture & Optimization

• Fireworks differentiates itself as a system rather than a library, automating complex optimizations (e.g., auto-tuning, quantization, and caching) behind a simple API.
• Performance is achieved through handwritten CUDA kernels, distributed inference across nodes, and disaggregated inference (chopping models across GPUs).
• The platform implements semantic caching to avoid re-computation for recurring application workload patterns.
• The system utilizes function calling to act as a router for accessing both public APIs and private enterprise APIs, effectively creating a "Mixture of Experts" architecture over hundreds of specialized models.

Market Trends & Customer Segmentation

• The customer journey typically begins with startups using OpenAI for heavy experimentation, followed by a migration to Fireworks for production scaling, low latency, and cost reduction.
• Customer acquisition targets a specific progression: startups first, then digital-native enterprises, and finally traditional enterprises (e.g., healthcare, banking).
• A shift in decision-making is observed where CTOs are replacing CIOs/CISOs as the primary buyers, driven by the need for innovation-led business transformation.
• Traditional enterprises are adopting GenAI due to a fear of obsolescence and the realization that foundation models allow them to fine-tune rather than train from scratch.

Open Source vs. Closed Source Dynamics

• Prediction: Model quality for open-source and closed-source models of the same size (e.g., 7B–100B parameters) will converge within a few years.
• Differentiation will shift from raw model capability to customization and fine-tuning for specific enterprise workloads.
• Open-source models (e.g., Llama 3) offer a richer ecosystem for researchers to build upon, enabling a "thousand flowers" approach to solving diverse enterprise problems.
• Fireworks believes small, specialized models are more effective for specific tasks than monolithic general models.

Fine-Tuning & Product Design

• Fine-tuning is identified as a complex workflow involving data labeling, algorithm selection (e.g., SFT, DPO), and hyperparameter tuning, which Fireworks aims to automate.
• The company distinguishes between "failure cases" and "product design failures," requiring product managers to define system behaviors for edge cases (e.g., auto-generation in tables).
• Off-the-shelf models perform well in domains like coding and translation, but fine-tuning is required for domains requiring specific business logic (e.g., insurance summarization templates).

Competitive Landscape & Future Outlook

• Nvidia: Competition is expected to emerge soon due to market economics and industry pressure against monopolies, particularly as the AI model space stabilizes.
• OpenAI: Fireworks does not view OpenAI as a direct threat because OpenAI's mission is AGI, whereas Fireworks focuses on enterprise-specific customization and cost efficiency for smaller models.
• Market Asymptote: While model capabilities may plateau in applicability, innovation in model architecture is expected to continue, necessitating continued optimization.
• Agents: Lynn Diao expresses high bullishness on the performance of AI agents in the coming year.

Rapid Fire Insights

• Favorite AI App: Fathom (used by sales teams for meeting summarization and training).
• 2024 Model Prediction: Quality will continue to increase with new models released weekly; specific winners are hard to predict, but overall improvement is certain.
• Most Admired Entity: Meta, for its commitment to open-sourcing Llama models and decentralizing power from hyperscalers.