Interview
Technological inevitability & human agency in the age of AGI | DeepMind's Allan Dafoe
- AI and AGI are expected to become one of the most significant historical developments, eventually comprising close to 100% of the economy compared to the current 0.01% of total revenue.
- Labor substitution is anticipated to occur once AI systems surpass humans at economically relevant tasks, creating profound economic and political impacts.
- The cost of training a model is expected to decrease by roughly 10x every two years, potentially allowing PhD students or individual actors to invent breakthroughs once compute becomes sufficiently cheap.
- Google DeepMind is identified as the primary unit for frontier models like Gemini, responsible for building the technology and leading the thinking on frontier policy issues.
- Gemini 1.0 received a three out of five on persuasion, a two out of five on cybersecurity, and a one or two out of five on self-reasoning in specific evaluations.
- Significant capabilities in the cyber domain are predicted to emerge within 6, 12, or 18 months, with full model capabilities often requiring months of experimentation and "capability elicitation" to reveal.
- The team is currently hiring for several positions, with the process expected to conclude by the time the podcast goes live.
- A staged release approach for Google DeepMind involves internal use, trusted testing, broader deployment, and general access while protecting model weights.
- The speaker encourages prospective joiners to enter the field immediately due to the orders of magnitude of growth remaining.
- AI is expected to transform education by providing tutoring and assistance comparable to the best teachers, benefiting both struggling and exceptionally advanced students.
- Applications for sustainability, including nuclear fusion plasma control, weather prediction, and flight path optimization, are viewed as significant opportunities to address major global challenges.
- The "super cooperative AGI hypothesis" posits that AGI systems may eventually cooperate to solve global coordination problems, though their cooperative skill is distinct from being pro-social.
- Global coordination is considered almost a necessary condition for good outcomes, as AI systems aligned with conflicting interests in great power conflicts could lead to devastating results.
- Differential technological development is recognized as important for AGI safety but is deemed difficult to execute because it requires anticipating consequences of two viable pathways.
- The market may naturally provide motivation for safety and alignment, potentially delaying the need for dedicated investment by alignment-focused groups by approximately two years.
- Risks include AI systems having alien goals, being backdoored with undetectable "magic words," or exhibiting deceptive behaviors that the market may not address in time.
- Cooperative skill is expected to be instrumentally useful and developed as a byproduct of any development agenda, though increasing it may harm agents excluded from the cooperative dynamic.
- Military competition is viewed as ubiquitous at a macro historical level, while modern competition has declined but remains a motivator in domestic national security politics.
- The EU AI Act's code of practice is expected to be a meaningful form of regulation involving Google, which is participating in its development.
- The "offense-defense balance" in AI is compared to biological systems where damage is hard to patch, differing from typical computer security vulnerabilities.
- A significant gap exists for social scientists, including economists, historians, and ethicists, to enter the field of AI governance.
- The speaker expects the market will eventually motivate safety and alignment, but believes dedicated investment in addressing deception and alignment is still necessary.
- AGI is projected to arrive around the time systems achieve the ability to perform radical, recursive self-improvement.
- ML research might be automated well before full AGI is reached, a prospect considered shocking and consequential if true.
- Capabilities may arrive unevenly as the trajectory approaches AGI, where the sequence of which skills emerge first matters significantly.
- Accelerating technological advances are expected to generate benefits alongside disruptions that society may not adapt to in time.
- The speaker anticipates that by the time their child attends school, AI tutoring models will be available as good as the best human teachers.