Latest Interviews
Showing 1–2 of 2 transcripts.
Clear all filters- Sequoia Capital42 min
Meta’s Joe Spisak on Llama 3.1 405B and the Democratization of Frontier Models | Training Data
Joe Spisak, Stephanie Zhan, Sonya Huang, Amy
Meta has open-sourced its Llama 3.1 405B model with a permissive commercial license to maximize ecosystem adoption and challenge closed-source competitors while serving as a teacher for distilling smaller, on-device variants. The release prioritizes expanded multilingual support, extended context windows, and state-of-the-art tool use, leveraging a massive 16,000-GPU training run that rivals or surpasses current frontier models from other companies. This strategic move shifts industry value from base model architecture to proprietary data and application layers, encouraging startups to build fine-tuned solutions on open foundations rather than investing in expensive pre-training.
- Sequoia Capital37 min
Making AI accessible with Andrej Karpathy and Stephanie Zhan
Andrej Karpathy, Stephanie Zhan, Brian Halligan, Alex, Sam, Peter, Michael
Andrej Karpathy outlines a future where the Large Language Model serves as a central CPU for a new "LLM OS," treating text, images, and audio as interchangeable peripherals within a decentralized startup ecosystem. He emphasizes that while massive scale drives current capabilities, the industry must overcome significant engineering and energy inefficiency barriers by adopting new hardware architectures and shifting from imitation learning toward self-correcting reinforcement loops. Drawing on lessons from Elon Musk’s management style, Karpathy advises founders to prioritize high-performance products, maintain technical rigor against organizational bloat, and foster a "coral reef" of vertical-specific applications rather than relying on monolithic corporate dominance.