Conference Presentation, Panel
RAISE Summit 2025: The AI Evolution Open Source, Fast Inference, and the Agentic Revolution
- Market Shift: The AI sector is moving from a focus on large language models to "inference and production," with agentic AI being deployed into production faster than foundational models due to easier testing and validation.
- Strategic Narrative: The industry is characterized by a "total war" for superintelligence, with major players racing to build full stacks encompassing compute, models, and agents, rather than focusing solely on chatbots.
- Open Source vs. Proprietary:
- Proprietary models currently hold a performance lead of approximately 6 to 12 months over open source alternatives.
- There are roughly 4 million open-source models available, though the open-source ecosystem is described as a "long game" strategy.
- Hugging Face notes that top-down knowledge diffusion is rapid, allowing open-source actors to quickly copy and iterate on frontier capabilities (e.g., DeepSeek reasoning models).
- Enterprise Adoption Drivers:
- Data Privacy: Enterprises increasingly favor open source to train on private data while retaining perpetual ownership and avoiding disclosure to model vendors.
- Cost Efficiency: Open source models offer significant cost advantages and allow for the use of custom chips, lower latency, and control over deployment location.
- Moat Dynamics: While models are viewed as increasingly commoditized, defensible moats are identified in chip manufacturing and full-stack integration (control of both model and compute layers).
- Hardware and Inference:
- Efficiency Constraints: The primary bottleneck for scaling agentic systems is the trade-off between model speed and system efficiency, particularly regarding the power and energy required for large inference fleets.
- NVIDIA's Position: While NVIDIA remains the gold standard, its dominance is challenged by the need for specialized lanes in high-performance, high-security, and energy-efficient inference use cases.
- Democratization Strategy: Startups like Sanoma Nova (referred to as "Xamanova" in transcript) aim to reduce the power footprint from 140kW racks to 10kW racks to enable on-premise deployment.
- Price Metric: "Price per token" has become the critical metric guiding the industry, driving a shift toward optimizing inference fleets rather than just raw training performance.
- Model Size Matrix:
- Large Models (1-10T parameters): Mostly closed-source, used for frontier capabilities.
- Mid-Range (Sub-trillion parameters): Open-source models (e.g., DeepSeek) are rapidly closing the gap with frontier performance.
- Small Models (3-4B parameters): A rising trend for edge and robotics; these models, when trained efficiently, can handle data wrangling and basic reasoning without requiring constant connectivity to large data centers.
- Agentic AI Outlook:
- Consumer Side: A wave of personal AI assistants is anticipated, with expectations for agents to handle complex tasks like taxes and accounting within the next 1-2 years.
- Enterprise Side: Every SaaS company faces an existential choice: convert into an agentic application or face extinction, though implementation requires significant infrastructure overhaul.
- Democratization: New managed services (e.g., "Samba Managed") are being launched to allow data centers to enter the AI market without requiring deep expertise, reducing deployment time from two years to 60 days.
- Identified Risks:
- Energy Crisis: The massive energy requirements for AI infrastructure (gigawatt-scale data centers) pose a significant supply chain and operational risk.
- Data Security: Future risks include high-profile data breaches involving private training data, which could severely damage trust in the ecosystem.
- Innovation Stagnation: A potential slowdown in the rate of improvement at the frontier could bottleneck the entire industry's growth trajectory.
- Valuation Concerns: While not the primary concern, the market shows worry over the lack of diverse exits, with an over-reliance on massive acquisitions (e.g., "aque hire") rather than independent IPOs.
- Social Impact: The pace of AI advancement is outstripping government regulation and civil society adaptation, creating significant societal friction.