Jun 12, 2024, 10:03 AM ETFireside Chat, Interview, Conference Presentation

Alex Wang: Why Data Not Compute is the Bottleneck to Foundation Model Performance | E1164

Alex believes advanced AI, potentially surpassing nuclear weapons, could serve as the ultimate military asset if deployed by authoritarian regimes like China or Russia.
China's centralized industrial policy and ability to execute aggressive state-directed strategies present a significant competitive threat, potentially allowing them to surpass US capabilities in the near term.
The discussion highlights a geopolitical dichotomy where the most cutting-edge AI systems may need to remain closed for security reasons, while less advanced versions can remain open to foster economic growth.
US leadership in AI is viewed as critical to preventing hostile conquests, necessitating a strategic approach to ensure the US does not fall behind in the race for AGI.

The industry has entered a phase of diminishing returns on compute, with massive increases in NVIDIA revenue and GPU spending not yielding proportionally "jaw-dropping" model improvements since GPT-4 in late 2022.
Current stagnation is attributed to hitting a "data wall," where models have exhausted nearly all accessible, high-quality internet data (pre-training on the open web).
Existing internet data excels at "emulating" the web but fails to capture the complex, non-codified reasoning chains required for advanced agent tasks (e.g., fraud analysis, complex problem-solving).
Future progress requires "frontier data," defined as complex reasoning chains, agentic workflows, and high-level expert interactions that do not currently exist in the public domain.

Enterprise Data: Massive amounts of valuable data exist within enterprises (e.g., JP Morgan holds 150 petabytes of proprietary data vs. GPT-4's <1 petabyte), but this data is largely locked away due to confidentiality concerns.
Production Mechanisms: The industry must shift from a scarcity mindset to a production mindset, utilizing a hybrid "human-in-the-loop" approach where algorithms generate data and experts validate/correct it (similar to safety drivers in autonomous vehicles).
New Roles: The creation of "AI trainers" or "contributors" is identified as a high-leverage career path, where human experts (scientists, mathematicians) codify their reasoning to permanently improve AI capabilities.
Longitudinal Data: New data sources will emerge from longitudinal collection of human activities, including workplace process mining and consumer wearables, though these require careful structuring and anonymization.
Competitive Moats: Data is expected to become the primary durable competitive advantage for model providers, more so than algorithms or compute, as data rights and proprietary datasets become scarce resources.

Commoditization: Foundation models will likely become commoditized over the next decade, driving up the cost of training to tens or hundreds of billions of dollars, necessitating consolidation under nation-states or hyperscalers (Google, Amazon, Microsoft).
Value Capture: While models may become commodities, value is predicted to accrue in the services, infrastructure, and customization layers above the model stack (e.g., on-premises solutions for sensitive data).
Pricing Models: The industry is shifting away from per-seat licensing toward consumption-based pricing to align with the value generated by AI agents and software, rather than just human headcount.
On-Premise Trend: Large enterprises are expected to increasingly adopt on-premise or open-source models (like Llama) to prevent sensitive proprietary data from being used to train competitor systems.
Software Evolution: The "end of software" narrative suggests a transition from centralized walled-garden SaaS to a decentralized universe of customized, purpose-built applications for specific enterprise problems.

Hiring Philosophy: Scale AI prioritizes "Navy SEALs" over a larger "Navy," with founder Alex Kawamoto approving 25-30% of hires to maintain an exceptionally high bar for talent density.
Hypergrowth Correction: The company learned that hyper-growth in headcount dilutes talent density; they have since stabilized team size while increasing revenue per employee.
PR Strategy: The company advocates for "no PR," favoring direct channels (podcasts, founder-led communication) over traditional media to avoid sensationalist click-driven narratives that can damage company reputation.
Misconceptions: The primary misconception is that "compute is the only constraint to AGI," whereas the speaker argues that data abundance and quality are the true limiting factors.
Future Vision: The speaker hopes to avoid a "boom and bust" cycle similar to the autonomous vehicle sector by managing expectations and ensuring promises remain aligned with technical reality.

Data Regulation: Current restrictive regulations (e.g., HIPAA, EU data laws) may stifle innovation; the speaker advocates for "pro-data" policies that enable safe, anonymized data pooling for industry advancement.
Sector-Specific Pools: Proposed data pooling initiatives include safety data in aerospace and fraud/compliance data in finance to accelerate industry-wide progress.
Balancing Act: Society must navigate a balance between liberal democracy values and the need for aggressive data access to secure technological and military leadership.