Interview, Fireside Chat, Conference Presentation
AI researchers debate how close we are to recursive self-improvement
- By 2036, scenarios where generalization remains elusive or continual learning proves impossible may prevent the "true spark" of general capabilities despite benchmark dominance, unless a "discontinuity" is discovered in current self-attention RL scaling paradigms.
- AI is expected to dominate human R&D and cross the human ELO score within the next few years, driven by compute scaling, though this trajectory could be altered by dramatic regulation rather than technical asymptotes.
- Research will likely see AI spend a comparable amount of compute on analysis and theory building as on experiments, potentially offering a 10x speedup for well-specified objectives, while open-ended science requiring paradigm shifts remains a hurdle.
- As code production automates, tools for testing will become critical to ease verification bottlenecks, with specific adoption like "Antithesis" by Jane Street anticipated starting in early 2025.
- Frontier models may reach 100b to 200b active parameters by 2030, but inference efficiency priorities could cause a plateau in active parameters as the bottleneck shifts to environments, with post-training progress relying on synthetic data as pre-training yields diminishing returns.
- A productivity uplift of 10x for AI researchers is predicted within two years, accelerating AI progress toward an ASI capable of dominating top human experts across every field in five to ten years, potentially sooner for code and math.
- Full generality for white-collar work as a drop-in remote worker may take around three years, whereas basic tasks could be completed in one year, with the "last human job" remaining in defining objectives and alignment.
- Current LLM training methods may fail to discover necessary discontinuities to avoid asymptotic curves, potentially requiring the abandonment of gradient descent and neural nets, while the "dumb after a month" cycle could bottleneck explosive growth.
- Companies will likely target incentives for AI progress in simulations and general science, following a linear increase in ELO, with frontiers labs potentially losing their advantage if they cannot incentivize capabilities in smaller models or if real-world deployment matters more than benchmarks.
- Generalization from tricky narrow tasks to realistic tasks may depend on model size, while distillation is expected to fight centralizing forces by allowing capabilities learned through RL to be distilled from small data amounts.
- Sample efficiency improvements will make learning from deployment a larger part of training, eventually enabling EIs capable of automating AR and D through a combination of human feedback and practice environments.