Lecture, Conference Presentation
Andrej Karpathy: Software Is Changing (Again)
- Software is predicted to undergo fundamental transformation into "software 3.0," driven by large language models (LLMs) enabling programmable neural networks, with a massive volume of code expected to be written and rewritten as capabilities migrate from earlier versions.
- The industry is characterized by rapid deep tech development and centralizing R&D secrets within LLM labs, creating a "tech tree" that is growing quickly while reliance on these models increases to the point where global functionality may degrade if state-of-the-art systems fail.
- The personal computing revolution for LLMs has not yet materialized, with unclear future forms and current experimental efforts like using Mac Minis for batch inference, requiring the invention of new operational paradigms.
- Technology diffusion is expected to reverse its traditional pattern, initiating with consumers rather than governments or corporations, while the "vibe coding" phenomenon serves as an entry point for development, allowing rapid custom app creation without prior language training.
- A long-term shift in autonomy is forecasted over the next decade, with the industry moving gradually along a spectrum from non-autonomous to "partially autonomous" systems, countering premature claims that 2025 will define the era of agents.
- Documentation and data accessibility are transitioning toward formats legible to LLMs, with early adoption by entities like Vercel and Stripe, alongside a predicted need for direct agent-friendly interfaces like
llms.txtand tools to handle a long tail of non-adapting software. - Current LLM capabilities exhibit specific limitations, including "jagged intelligence" prone to logical errors such as incorrect number comparisons and "anterograde amnesia" preventing native long-term learning or context consolidation over time.