newsfilter
← Interviews
Dwarkesh Patel2h 14m

How GPT, Claude, and Gemini are actually trained and served – Reiner Pope

Reiner Pope, Ilya

Interview, Lecture

John Mueller Jr. discusses the technical and economic drivers behind AI inference architectures, detailing how startups like Maddox optimize for memory bandwidth bottlenecks and latency bounds in sparse Mixture of Experts models. The analysis highlights that frontier models are currently overtrained by a factor of 100x relative to scaling laws, a phenomenon that dictates current API pricing structures for context length and caching tiers. Finally, Mueller explains how industry scaling is shifting toward larger single-rack domains to maximize expert parallelism while utilizing reversible network techniques to mitigate training memory constraints.