Interview, Fireside Chat
Fireworks Founder Lin Qiao on the Power of Small Models to Democratize AI Use Cases
- Fireworks projects a timeline of two to five years to compress industry AI deployment time-to-market from five years to five weeks or five days.
- A convergence in quality between open source and closed source models is predicted for sizes ranging from seven billion to seventy billion parameters, and potentially up to one hundred billion parameters.
- AI model quality is expected to continue improving throughout 2024, driven by weekly new model releases and rapid iteration cycles.
- Many specific AI applications are forecast to achieve best-in-class performance in 2024 due to the accelerated pace of model development.
- AI agents are anticipated to see significant success and adoption this year rather than experiencing the disappointments often associated with emerging technologies.
- A new model architecture within the General AI space is expected to emerge soon, indicating the current innovation curve is overdue for a shift.
- Competition for NVIDIA is predicted to arrive soon, fueled by economic incentives and industry pressure against monopoly in general-purpose GPU and specific AI model segments.
- Meta's strategy of open-sourcing foundation models is expected to persist, continuing to push boundaries and shrink quality differences with proprietary alternatives.
- A next-generation function calling model featuring multiple breakthroughs will be released, accompanied by developer-focused demos and examples.
- A new product is planned to automate fine-tuning workflow complexity, including data labeling, algorithm selection, and hyperparameter tuning, to reduce technical overhead for application developers.
- Market engagement is expected to shift toward CTOs rather than CIOs as business transformation becomes increasingly innovation-driven.
- Open source models are predicted to enable a "thousand flowers" ecosystem of thousands of small, specialized models that outperform single proprietary solutions for specific enterprise problems.
- While general model applicability may stabilize or plateau in the short term, heavy customization toward specific use cases will become the dominant strategic direction.
- Open source models are likely to surpass closed source models in certain domains, challenging the assumption that proprietary models must always maintain a time-lag lead.
- Returns to scale on frontier models are expected to slow down, shifting the competitive focus toward optimization, tuning, and the application of mature capabilities.
- OpenAI's AGI mission is viewed as potentially limiting deep customization for specific enterprise problems, creating a strategic opening for competitors focused on workload-specific tuning.
- A "mixture of experts" architecture will be built to access hundreds of smaller, agile experts curated by Fireworks rather than relying on a few large models.
- The majority of these "experts" are expected to be hosted directly on Fireworks's own platform for curation and service, reducing reliance on external repositories like Hugging Face or AWS.
- Simple API access to the totality of knowledge is expected to be achieved through a function-calling layer that routes requests to both public and private APIs.
- The industry is expected to move away from training models from scratch toward fine-tuning on high-quality small datasets, making advanced AI technology affordable for everyone.