Fireside Chat, Interview
From Cloud to Edge: AI Gets Personal
- Core Prediction: On-device and smaller-parameter generative AI models (text, image, voice, video) will become the dominant paradigm over the next 12–18 months.
- Hardware Enablement:
- Smartphone compute power today rivals computers from 10–20 years ago, driven by Moore's Law.
- Models in the 2B to 8B parameter range possess sufficient capability for robust on-device generation of text, images, and audio.
- Diffusion models are inherently smaller and more capable than their text-based predecessors of similar eras.
- Technological Enablers:
- Distillation: New tooling allows powerful large models to be distilled into smaller parameter sizes while retaining core capabilities.
- Real-time Latency: On-device processing eliminates network latency, enabling instant responses for chatbots, filters, and voice agents (critical for apps like Instagram, TikTok, Uber, and Lyft).
- Key Advantages:
- User Experience: Delivers "sleek" and immediate interactions without waiting for cloud round-trips.
- Privacy: Local processing of sensitive data (e.g., meeting notes, private conversations) increases user trust and adoption rates.
- Efficiency: Reduces the need to route complex visual and audio processing through multiple server layers.
- Emerging Applications:
- Real-time Voice Agents: 11Labs and similar players are focusing on low-latency, fluent AI companions and support agents.
- Augmented Reality (AR) & 3D Interaction:
- Reimagining the camera as an input/output device to project virtual furniture, wallpaper, and surfaces onto physical rooms.
- Enabling AI to interact with the physical world rather than just capturing it.
- Economic Implications:
- Infrastructure Costs: On-device inference does not substantially reduce overall infrastructure costs, as cloud inference prices are already dropping significantly.
- Developer Economics: Shifts the focus to tooling, iteration speed, and the challenge of managing update cycles on diverse hardware.
- Hybrid Architectures: Teams must adopt holistic strategies, balancing cloud flexibility with on-device constraints.
- Winners and Stakeholders:
- Hardware Manufacturers: Chipmakers, camera sensor vendors, and device manufacturers (phones, wearables like Apple Watch and Fitbit) are seeing heightened interest.
- Model Developers: Focused on proliferating adoption across diverse device setups.
- Supply Chain: Long-term impact expected across the entire hardware and software supply chain.
- 2025 Outlook:
- Foundation model technology is considered mature, with infrastructure ready for mass deployment.
- Primary investment thesis focuses on "mixed reality" experiences where generative video and 3D models enhance real-world perception via cameras and microphones.