Aidan Gomez: What No One Understands About Foundation Models | E1191

Market & Model Trends

• There is effectively no market for "last year's" models; technological obsolescence renders previous generations useless compared to current iterations.
• The industry is moving toward a dual-structure landscape featuring both large, horizontal models for prototyping and smaller, verticalized models for specific use cases.
• Scaling by simply increasing compute remains the most reliable, albeit "dumbest," strategy for immediate model improvement, though it is highly inefficient.
• Sustaining linear intelligence gains requires exponential increases in compute, creating significant economic constraints as costs rise.
• The market will likely see consolidation within three years, with smaller model builders potentially becoming subsidiaries of major cloud providers (Microsoft, Amazon, Google).
• Becoming a subsidiary of a cloud provider is flagged as a dangerous business strategy that compromises long-term independence and value proposition.

Technical Innovations & Research

• Major gains in recent AI performance are driven more by data innovation (quality, scraping algorithms, synthetic data) than raw model scaling.
• Synthetic data generation is becoming a critical market where models create scalable datasets to train smaller, more efficient models, potentially displacing human annotation.
• Current models lack true reasoning and problem-solving capabilities because public internet data rarely demonstrates the "work" or reasoning process behind conclusions.
• Future advancements will likely focus on "curriculum learning" and reasoning frameworks where models can fail, learn, and retry, similar to how humans learn.
• Retrieval Augmented Generation (RAG) is identified as a game-changer for reducing hallucinations by allowing models to query private knowledge bases and cite sources.
• The speaker has shifted their belief from prioritizing scale to recognizing that data quality is the primary determinant of model performance.

Business Strategy & Economics

• The business of selling raw models is trending toward a low-margin, commoditized race to the bottom due to price dumping and open-source free models.
• Value accumulation is shifting to the chip layer (high margins for NVIDIA, Google, AMD) and the application layer, rather than the model layer itself.
• Enterprises are moving from experimental "proof of concept" budgets to urgent production deployment, driven by fear of being left behind.
• The primary use case for enterprise adoption is workforce augmentation (e.g., Co-pilots) rather than full automation or replacement.
• Cohere raises approximately $1 billion total, with the speaker noting that raising large rounds is more complex than seed funding, though their valuation strategy prioritizes independence over rapid acquisition.
• Enterprise adoption blockers remain centered on trust and security, specifically concerns regarding data privacy and intellectual property leakage.
• Cohere addresses enterprise trust via private deployments (on-prem/VPC) to ensure customer data is never processed externally.

Interface & Consumer Trends

• Voice interaction is identified as a magical, compelling interface capable of conveying emotion and inflection, though text chat remains a viable primary interface for many tasks.
• GUIs are not dying but will coexist with chat interfaces; the future involves selecting the optimal interface (click, voice, or text) for specific workflows.
• Agentic behavior (models that can execute long-term tasks autonomously) is viewed as the next major productivity transformation, but building these agents is structurally disadvantaged for model consumers.
• Robotics is predicted to see major breakthroughs in the next 5–10 years due to the maturity of foundation models acting as dynamic planners.
• Despite fears of replacement, the speaker argues AI will augment rather than replace human workers, as human trust and accountability remain essential for high-stakes interactions.

Geopolitical & Ecosystem Observations

• London and the UK are highlighted as tech optimism hubs with strong engineering talent, contrasting with a broader European culture that is often hostile to tech and prone to regulation.
• The speaker expresses alignment with the optimistic views of Yann LeCun rather than the "doomsday" perspective of Jeff Hinton regarding AI's future impact.
• There is a belief that the current market underrates the speed of AI advancement and the potential for rapid capability gains in the next 12–24 months.
• Gaming in youth is correlated with successful founding due to the development of resilience, optimism, and the understanding that failure is a path to progress.
• The ultimate goal for AI is to drive global productivity growth, which is seen as the most critical solution to stagnating wealth and social turmoil.