May 12, 2025, 3:38 PM ETConference Presentation, Panel

Will Open Source AI Overtake Closed Models? Ft. Olama, Fireworks and Open Router

Ollama (Jeff Morgan): Founded by Jeff Morgan; provides the primary interface for rapid deployment of open-source LLMs.
Fireworks (Dima): Founded by Dima (alongside Lynn); specializes in high-speed, large-scale model serving and fine-tuning for generative AI.
OpenRouter (Alex Atala): Serves as a routing marketplace to optimize pricing, performance, and uptime across multiple model providers, including deep integration with Fireworks.

Decentralization of Innovation: Human genius is positioned as a global resource; centralizing it behind closed model labs is deemed a higher-risk, lower-leverage bet.
Future of Development: Young ML practitioners will likely prioritize open-source development over closed labs due to lower barriers to entry.
Practical Deployment (Consumer): Open weights allow distribution on non-data center hardware, effectively granting users ownership even if full training data/papers remain closed.
Enterprise Ownership: Enterprises require full ownership of fine-tuned or distilled models containing proprietary data to ensure security and customization rights.
Technological Necessity: Regulatory bodies may regulate applications but should not ban the underlying technology, which acts as a general-purpose utility similar to electricity.

Current Usage Share: Open-source models currently account for approximately 20% to 30% of total inference tokens, often serving as a fallback for specific enterprise constraints.
Fine-Tuning Lifecycle: A temporary "weak moment" for fine-tuning is predicted within the next year as foundation models improve via Reinforcement Learning (RL), reducing the marginal utility of customization.
Long-Term Shift: The open-source wave is expected to mirror historical software trends, where transparency and customizability eventually drive majority adoption once businesses begin experimenting at scale.

Shifting Leadership: The "stalwart" role for open-source frontier models rotates between players (e.g., Meta/Llama, Mistral, DeepSeek, Alibaba/Qwen) rather than remaining static.
Strategic Motivation: Model leaders are open-sourcing not merely for community distribution but to fuel their own consumer app ecosystems (e.g., Meta's social network, Chinese models' consumer apps).
DeepSeek Success Factors:
- Engineering Efficiency: Small teams achieved superior integration of research and engineering compared to larger entities.
- Reasoning & UX: Introduced the first high-performing open-source reasoning model with transparent "thought" outputs, contrasting with the "black box" nature of competitors like 01.
- Infrastructure Strain: Initial inability to handle inference load forced the broader ecosystem (including Fireworks) to scale and decentralized providers to adopt the model.
Post-Processing: Providers like Perplexity and Fireworks performed fine-tuning to remove distinct behavioral artifacts and optimize for specific business use cases (e.g., removing political bias).

Llama 4 Outlook: Meta possesses the necessary compute, talent, and high-level commitment to potentially leapfrog competitors, though current benchmarking difficulties persist.
Model Architecture: Llama 4 is expected to leverage mixture-of-experts architectures similar to previous successful open models.
Evaluation Evolution: The industry is shifting away from pre-training-only metrics to post-training RL evaluation to mitigate "reward hacking" and better assess model capabilities.
Distillation Strategy: DeepSeek's successful release of both massive and distilled models highlighted the value of offering accessible, cheaper tiers alongside frontier versions.

General Consensus: All three panelists predict a 50/50 split between open and closed-source inference tokens in five years.
Closed-Source Structure: Closed source will likely remain dominated by a few leading players (e.g., ~25% of traffic per major provider).
Open-Source Structure: Open source will be fragmented across a diverse ecosystem of model families, functions, and customizations rather than a single dominant model.
Capital Efficiency: The shift toward post-training RL and customization allows open source to achieve high performance with less capital investment than the trillion-dollar data centers required for pre-training.
Critical Variable: The success of decentralized inference providers is a key determinant; without sustainable decentralization, closed source may retain a majority share due to economic incentives and infrastructure barriers.