Why I don’t think AGI is right around the corner

Current State of AI Limitations (As of July 2025)

• LLMs currently lack fundamental capabilities required to produce "normal human-like labor," specifically the ability to improve over time through practice.
• Unlike human employees who build context and adjust to feedback incrementally, LLMs operate on a fixed baseline with no mechanism for continuous learning.
• Current workflows relying on LLMs are restricted to simple, self-contained, short-horizon "language in, language out" tasks.
• Attempts to teach models via prompts or static fine-tuning yield results that do not match the iterative improvement seen in human employees.
• The "Continual Learning" bottleneck prevents LLMs from accumulating tacit knowledge or building up context across long sessions.
• While models can improve during a single session (e.g., co-writing an essay), this learning is ephemeral and lost once the session ends.
• Proposals to mitigate memory loss via rolling context windows (e.g., Cloud Code) are viewed as brittle outside text-based domains like software engineering.

Disagreements with Industry Forecasts

• Rejection of Sholto Douglas and Trenton Bricken's forecast: The speaker disagrees with the view that AI progress can stall yet still automate white-collar jobs within five years due to data abundance alone.
• Economic Impact Assessment: Even if AI progress halts today, the speaker predicts less than 25% of white-collar employment will be eliminated, contrary to the "death center" expectation.
• Reasoning on Automation: Automation of specific subtasks is possible, but the inability to build context prevents AI from functioning as actual, autonomous employees.
• Skepticism regarding 2026 Computer Agents: The speaker doubts the prediction that reliable, end-to-end computer use agents (e.g., handling taxes autonomously) will exist by the end of next year.
  • Constraint 1 (Horizon Length): Agentic tasks require long rollouts (e.g., two hours) to verify success, significantly slowing progress compared to single-turn tasks.
  • Constraint 2 (Data Scarcity): There is no large pre-training corpus of multimodal computer-use data comparable to the internet's text data used for NLP.
  • Constraint 3 (Engineering Difficulty): Even seemingly simple algorithmic breakthroughs (e.g., DeepSeek R1 training procedure) took years to debug and implement after initial concepts.
• Comparison to Historical Progress: The speaker notes that while GPT-3 enabled "cool demos," it took four years (GPT-2 to GPT-4) to reach practical utility, suggesting similar delays for computer use.

Forward-Looking Timelines (50-50 Probability Bets)

• 2028: An AI capable of handling a week-long, end-to-end tax filing process for a small business, including chasing receipts, emailing stakeholders, and filing IRS forms, comparable to a competent general manager.
  • The speaker compares this milestone to the transition from GPT-2 to GPT-4 in language processing.
• 2032: The emergence of AI capable of "on-the-job" learning that matches human depth, specifically regarding tacit understanding of preferences, workflows, and audience resonance (e.g., AI video editors).
• Long-term (Next Decades): A "broadly deployed intelligence explosion" is expected once continual learning is solved, allowing models to amalgamate learnings across all copies without requiring a software-only singularity.
• 2028 ASI Risk: The speaker maintains a 50-50 bet on the possibility of an aligned or misaligned Artificial Super Intelligence (ASI) occurring by 2028.

Structural Shifts in AI Progress

• End of Scaling-Only Growth: Training compute on frontier systems has grown >4x annually for a decade but cannot sustain this pace beyond the 2030s due to chip, power, and GDP constraints.
• Algorithmic Necessity: Post-2030 progress will depend almost entirely on algorithmic improvements, with the "low-hanging fruit" of the deep learning paradigm already harvested.
• Probability Distribution: AI progress is characterized as log-normal; while the probability of AGI per year decreases after 2030, the cumulative probability of a "truly crazy outcome" remains high if timelines are extended.
• Market Reality: The speaker expects broken, early versions of continual learning or reinforcement learning from human feedback (RLHF) to be released gradually rather than via a single announcement of a solved problem.

Narrative Context

The speaker distinguishes their position from both extreme optimism and pessimism, acknowledging the "spiky" nature of current models while arguing that the lack of continuous learning fundamentally limits their economic transformation potential in the short term. The timeline predictions are framed as probabilistic bets rather than certainties, reflecting the log-normal nature of technological breakthroughs where progress can stall for years or accelerate rapidly once a bottleneck is cracked.