Oct 17, 2025, 1:15 PM ETInterview, Fireside Chat

Andrej Karpathy — “We’re summoning ghosts, not building animals”

Timeline Prediction: AI agents will not dominate "this year" but rather "this decade," requiring approximately ten years to resolve current bottlenecks.
- Reasoning: Current agents lack sufficient intelligence, multimodality, computer use capabilities, and continual learning.
Optimism vs. Pessimism: Karpathy remains optimistic about the tractability of AI development, though he sounds pessimistic due to industry "fundraising" narratives and unrealistic timelines found on social media.
- Context: He characterizes current digital entities as "ghosts" or "spirits" mimicking humans, distinct from "building animals" via evolution.
Reinforcement Learning (RL) Assessment: Karpathy describes RL as "terrible" compared to alternatives, noting it "sucks supervision through a straw" by only using a single final reward signal to weight entire trajectories.
- Critique: RL upweights incorrect steps within a correct trajectory, creating high variance and noise, whereas humans would review specific parts of a solution.
- Process Supervision: The field lacks automated, robust process supervision due to the difficulty of assigning partial credit without LLM judges gaming the system via adversarial examples.

Memory vs. Cognition: Current LLMs are "distracted" by excessive memorization of training data; the future goal is a "cognitive core" stripped of memory, forcing models to look up facts rather than regurgitate them.
- Prediction: In 10 years, a cognitively competent model could be as small as one billion parameters if trained on high-quality, curated data rather than the "garbage" of the current internet.
Model Collapse: Synthetic data generation risks "silently collapsing" distributions where models generate repetitive, low-entropy content.
- Comparison: Humans also "collapse" over time but avoid total stagnation through external entropy sources (e.g., social interaction).
In-Context Learning vs. Pre-training: Karpathy draws an analogy between human working memory and the model's KV cache.
- Information Density: In-context learning assimilates ~320KB per token (35 million-fold difference) compared to the ~0.07 bits per token compressed into weights during pre-training.
Future Architecture: Expectations for 10 years hence include giant neural networks trained via gradient descent, but with modified attention mechanisms (e.g., sparse attention) and optimized hardware kernels.
- Historical Context: Progress in AI has historically required simultaneous improvements across algorithms, data, hardware, and optimization, rather than a single breakthrough.

Economic Impact Forecast: Karpathy rejects the idea of a sudden "intelligence explosion" visible in GDP, arguing that automation is a gradual process already ongoing for centuries.
- GDP Trend: AI will likely maintain the current exponential growth rate (e.g., ~2%) rather than creating a new, steeper trajectory, as technology diffuses slowly.
- Labor Replacement: AGI will likely automate specific tasks (e.g., code, call centers) rather than entire jobs immediately, leading to a "loss of control" scenario where many entities compete autonomously.
Coding Automation: The "AI automating AI" narrative is premature because current models struggle with unique, non-boilerplate code.
- Limitation: Models fail to integrate custom architectural tweaks or understand context-specific assumptions, often bloating code with unnecessary defensive programming.
- Productivity: Current utility lies in autocomplete and "vibe coding" for boilerplate, not full-stack agent engineering.
Self-Driving Analogy: AI deployment faces similar "march of nines" reliability challenges as autonomous vehicles; demos are easy, but achieving 99.99% reliability in production requires immense iteration.
- Safety: Software failures can have catastrophic, unbounded consequences (e.g., security breaches), comparable to physical safety risks in self-driving.

Eureka Mission: Karpathy is building "Starfleet Academy" (Eureka) to teach technical skills via high-quality "ramps to knowledge" designed to maximize "Eureka per second."
- Goal: Create an experience where learners are never stuck or bored, mimicking the effectiveness of a skilled human tutor.
- Methodology: Courses like NanoChat serve as capstone projects to codify knowledge, with AI acting as a TA rather than the primary content creator (currently).
Pedagogical Principles:
- First-Order Thinking: Education should focus on simplifying systems to their first-order terms (e.g., "MicroGrad") before adding complexity.
- Pain Before Solution: Effective teaching presents the problem first to motivate the solution, avoiding the "curse of knowledge" where experts skip foundational steps.
- Learning by Teaching: The most effective way for students to verify understanding is to attempt to explain concepts to others.
Future of Learning: Post-AGI education may shift from utilitarian (learning to make money) to recreational (learning for self-improvement and fun, similar to gym culture).

Evolution vs. AI: Karpathy disagrees with Richard Sutton's "build animals" vision, arguing that AI is currently "imitating internet documents" rather than evolving via biological processes.
- Distinction: Animals use evolution to hardwire behaviors; AI uses pre-training to "distill" knowledge, which Karpathy calls "crappy evolution."
AGI Definitions: He prefers the definition of AGI as "any economically valuable task at human performance," noting that current AI has made little dent in complex, messy jobs like radiology.
- Task Horizon: Automation will likely begin with short-horizon, closed-loop tasks (e.g., call centers) rather than broad knowledge work.
Superintelligence: He anticipates a future of "gradual loss of control" due to multiple competing autonomous entities, rather than a single god-like AI.
- Concern: The risk is not immediate takeover, but the gradual layering of systems that society can no longer understand or manage.