newsfilter
← Interviews
Lex Fridman29 min

Exponential Progress of AI: Moore's Law, Bitter Lesson, and the Future of Computation

Rich Sutton

Lecture, Presentation

The author argues that historical AI progress relies on exponential computational growth rather than human-designed expertise, yet current research prioritizes incremental, non-scalable methods over approaches capable of leveraging future compute surges. Potential drivers for this scaling include distributed IoT networks, specialized ASICs, and algorithmic breakthroughs in self-supervised learning, alongside speculative frontiers like quantum and neuromorphic computing. The essay concludes that the industry must shift toward evaluating methods based on their 5-to-20-year scalability to harness these emerging exponential gains.