Interview, Fireside Chat
Aidan Gomez: What No One Understands About Foundation Models | E1191
- The market will evolve into a landscape containing both specialized vertical models and general horizontal "rule it all" models, with an anticipated culling and consolidation occurring within three years where model builders are acquired by major cloud providers.
- Continued reliance on scaling models is expected to face economic constraints, as sustaining linear gains in intelligence requires compute to double, though cheaper, smaller models derived from distillation will persist alongside the trend of "race to the bottom" pricing for pure model vendors.
- Value accumulation is projected to occur at the chip and application layers, while the model layer risks becoming a low-margin commodity business due to factors like price dumping by OpenAI and the availability of free open-source alternatives.
- Supply chain constraints regarding GPUs are predicted to ease, with training platforms from Google, AMD, and Tranium becoming viable alternatives to NVIDIA, though model progression will likely outpace data center build-out.
- User interaction models are expected to shift toward voice interfaces and AI agents, with GPT-5 anticipated within the next three to six months, while the text chat interface is viewed as insufficient for all use cases.
- Productivity gains of approximately 5% are forecast to have a seismic impact on economies and healthcare, whereas a lack of technological growth could lead to social turmoil, with general-purpose humanoid robotics expected to see major breakthroughs within five to ten years.
- The speaker notes that older models will rapidly become obsolete, incremental intelligence gains will become increasingly inefficient and costly to achieve, and the cost per flop will continue to drop quickly.
- Strategic risks include the danger of companies becoming subsidiaries of cloud providers and the difficulty of operating with tight margins if a business relies solely on scaling or selling models without vertical integration.
- Future infrastructure plans may involve companies like Cohere building their own data centers if procuring from providers becomes more expensive, reflecting a broader shift in the industry's structural dynamics.