Interview, Fireside Chat
Moore's Law is Not Dead (Jim Keller) | AI Podcast Clips
- Moore's Law historically defined as doubling transistor count every two years, though the speaker's operational model defines it as doubling computer performance every 2–3 years.
- Current "shrink factor" is approximately 0.6 every two years, deviating from the original 0.5 target while maintaining the exponential trend via constant chip area efficiency.
- The speaker dismisses predictions of Moore's Law's demise, noting a 50-year history of similar false prophecies and attributing the law's longevity to a "cascade of diminishing return curves" where thousands of individual innovations overlap to sustain exponential growth.
- Modern transistors are currently ~1,000 atoms in width (down from ~120), with fundamental physics limits estimated around 2–10 atoms.
- Manufacturing methods are evolving to place atoms in single atomic layers, though current atom-by-atom placement remains impractical for large-scale computing.
- The innovation stack sustaining Moore's Law spans thousands of technologies across optics, chemistry, physics, material science, metallurgy, and equipment.
- The speaker projects a roadmap capable of achieving a 100x shrink factor over the next 10–20 years, with current progress reaching a 15x shrink.
- Architectural strategy must account for the influx of transistors by relying on abstraction layers to manage complexity, as human cognitive capacity and team sizes (optimal between 10–100) do not scale linearly with transistor counts.
- Software refactoring is required because algorithmic complexity (e.g., O(n²)) can negate hardware speedups if code is not adapted for new transistor densities.
- Computational mathematics has evolved hierarchically from simple equations (A = B + C) to vector units, matrix operations, and now topological data analysis used in AI.
- The speaker debates the nature of AI computation: while some view neural network training as a search for optimal weights, the speaker argues it involves "endless projections" and layer-specific feature extraction (e.g., detecting "pointy ears") rather than simple search.
- Despite high-level algorithmic shifts, the fundamental hardware building blocks remain binary logic gates (AND, OR) performing basic arithmetic (add, subtract, multiply), though emerging fields like analog computing and quantum computing aim to alter this paradigm.
- The speaker asserts that every order of magnitude increase in computation capability fundamentally changes the "kind" of computation performed, citing the difference between an ant and an anthill as an analogy for the transition from quantity to quality.
- New semiconductor fabrication techniques, such as nanowires replacing traditional gate structures, are identified as immediate pathways to sustain shrinking and performance gains for the next decade.
- Bell's Law is referenced to illustrate how Moore's Law-driven hardware advances previously triggered distinct computing eras (mainframes, PCs, mobile, 5G), suggesting the next 10x growth will generate unpredictable, novel computational forms.
- The speaker predicts that AI will increasingly produce results where the underlying mathematical function is unknown or "not completely mathematically describable," shifting from guessing functions to using machine learning to find patterns in massive datasets without human intuition.