Scaling and the Road to Human-Level AI | Anthropic Co-founder Jared Kaplan

Core Thesis: Scaling Laws Drive Predictable AI Progress

• Contemporary AI models improve through two distinct, scalable training phases:
  • Pre-training: Models learn to predict the next word in text and multimodal data, uncovering correlations within vast corpora.
  • Reinforcement Learning (RL): Models are optimized via human or AI feedback to prioritize helpful, honest, and harmless behaviors.
• Precise scaling laws exist for both phases, demonstrating that performance improves predictably as compute, data size, and model parameters increase.
  • These trends hold across many orders of magnitude, suggesting future intelligence growth will be linear and continuous.
  • Scaling laws provide a "dumb" but powerful tool to diagnose training failures rather than assuming the trend itself is broken.

Capabilities and Time Horizons

• AI capabilities are evaluated along two axes:
  • Flexibility (Y-axis): Moving from narrow tasks (e.g., AlphaGo) to broad multimodal systems capable of interacting with text, images, and eventually robotics.
  • Task Horizon (X-axis): The duration of tasks an AI can complete independently, which is doubling roughly every seven months.
• Current trajectories suggest AI will soon handle tasks spanning days, weeks, or years, potentially replicating the work of entire human organizations or scientific communities.
• Organizational knowledge, memory, and nuanced oversight are identified as the remaining critical ingredients for achieving broadly human-level AI.
  • Memory: Models must store and retrieve information across long context windows to maintain progress on multi-day tasks (a key feature of Claude 4).
  • Oversight: Developing AI capable of generating nuanced reward signals (e.g., for creativity or taste) rather than just crisp, binary correctness.

Strategic Recommendations for Builders

• Build on the Frontier: Construct products that currently "don't quite work" because rapid model improvements (e.g., from Claude 4 to 5) will likely render them functional within months.
• Use AI to Integrate AI: Leverage AI tools to accelerate the integration of AI into existing workflows and business processes.
• Target Greenfield Areas: Look beyond software engineering for rapid adoption in fields involving data interaction and high skill requirements, such as finance (Excel/analysis) and law.
• Focus on "70% Right" Use Cases: Prioritize applications where high reliability is not strictly required (70-80% accuracy) to explore the full capabilities of current models, rather than waiting for 99.9% perfection.

Dialogue Highlights: Claude 4 and Future Architecture

• Claude 4 Improvements:
  • Enhanced coding agent capabilities with reduced "eagerness" (fewer unnecessary try-except blocks) and improved supervision to follow instructions precisely.
  • Introduced persistent memory features, allowing models to save and retrieve context across multiple sessions to complete long-horizon tasks.
• Human-AI Collaboration Models:
  • Near-term: Human-in-the-loop workflows where AI handles execution while humans act as managers to sanity-check work.
  • Long-term: Full automation of workflows where AI acts as an end-to-end replacement, particularly in domains requiring broad knowledge synthesis (e.g., drug discovery, biology) rather than deep, narrow expertise.
• Efficiency and Compute:
  • Current focus remains on unlocking frontier capabilities rather than immediate efficiency gains (e.g., FP4 precision), driven by the Jevons Paradox where increased capability drives increased demand.
  • Algorithmic efficiency gains of 3x to 10x per year are expected as the field matures.

Theoretical Context: Physics Background in AI Research

• Kaplan's physics background influenced AI research by prioritizing macro-trends and precise mathematical formulations over vague claims.
  • Quantifying Trends: Asking "dumb" questions (e.g., is the growth exponential or a power law?) to establish precise scaling laws.
  • Interpretability: Viewing AI interpretability as similar to neuroscience, leveraging the ability to measure every parameter in AI models to reverse-engineer "brain" features.
• Self-Correction Mechanism: Long-horizon task scaling is likely driven by incremental improvements in a model's ability to self-correct, which exponentially extends task duration.
• Training Data Sources: Future long-horizon training will utilize a hybrid approach, combining human-generated tasks with AI-generated tasks to scale supervision efficiently.