#1 - Miles Brundage on the world's desperate need for AI strategists and policy experts

Interview Overview

• Speaker: Myles Brundage, Research Fellow at the University of Oxford's Future of Humanity Institute (FHI) and PhD candidate at Arizona State University.
• Primary Focus: AI policy, security risks from bad actors, and the mechanics of coordination to prevent AI arms races.
• Professional Background: Former energy policy analyst; transitioned to AI policy due to its neglected status and high potential impact.

Current Research and Risk Landscape

• Focus Areas:
  • Bad Actors: Analyzing risks posed by state actors, terrorists, corporations, and individual rogue actors.
  • Near-Term Applications:
    • Cybersecurity (automated detection and vulnerability production).
    • Information operations (automated fake news generation).
    • Physical harm via autonomous drones and weapons (e.g., weaponization of consumer drones by ISIS).
  • Uncertainty Management: Prioritizing understanding "what is technologically possible" over predicting specific future timelines for accidents or malicious acts.
  • Diffusion of Capability: Anticipating a steady increase in the baseline skill level required to cause harm, lowering the barrier for individual actors to access dangerous AI tools.

The Coordination Challenge and Arms Races

• Core Problem: A misalignment of incentives between individual actors (companies or nations) and global safety.
  • Competitive pressure creates an incentive to "skim" safety measures to gain economic, military, or intelligence advantages.
  • Significant tradeoffs may exist between raw performance (hardware, data, sensors) and safety constraints.
• Proposed Solutions:
  • Incentive-Compatible Mechanisms: Developing frameworks where adhering to safety protocols is a prerequisite for accessing cutting-edge computing power or breakthrough technologies.
  • Collaboration vs. National Security: Warning that framing AI primarily as a national security issue may exacerbate arms race dynamics; favors international, collaborative approaches over unilateral acceleration.
  • Precedents: Potential for coordination among the small number of high-concentration actors (major tech firms and nations) regarding "capability caution" and mutual vetting of safety procedures.

Policy, Governance, and Current Progress

• Regulatory Stance:
  • Near-Term: Governments should hire in-house AI experts to manage labor impacts and crisis response ("low-hanging fruit").
  • Long-Term: Caution against rushing to lead AI development if it intensifies global competition; advocacy for positive, collaborative proposals rather than purely defensive national strategies.
• Recent Developments:
  • Asilomar Principles: A consensus view developed to avoid arms races and weaponization, though currently at the "high-level principle" stage rather than actionable policy.
  • Research Milestones:
    • "Racing to the Precipice" (Armstrong et al.): Detailed the stark risks of AI arms races.
    • Asilomar AI Principles: Established shared values for scientists.
    • Bostrom & Flynn: Outlined policy desiderata for machine superintelligence.
• Current Gap: Transitioning from identifying problems and principles to developing concrete, actionable models and formal proposals.

Career Advice and Entry Points

• Key Roles and Organizations:
  • Tech Industry: Google DeepMind (specifically their open policy researcher position), OpenAI, and other labs where direct exposure to technical development informs policy.
  • Academia: FHI, Center for the Study of Existential Risk, Leverhulme Center for Future of Intelligence, and Tech Policy Lab (University of Washington).
  • Government/Policy: Congressional staff (AI Caucus), AAAS Fellowships, and think tanks (e.g., Brookings).
• Skill Requirements:
  • Interdisciplinary Approach: Success requires bridging technical AI knowledge with social science (politics, economics) or policy expertise.
  • Complementary Roles:
    • Synthesizers: Individuals who can distill literature on topics like authoritarian surveillance without needing deep technical math skills.
    • Technical Experts: Individuals capable of modeling game theory issues related to arms races.
• Entry Strategies:
  • Networking: Attend major AI conferences (NeurIPS, ICML, IJCAI) and policy-focused conferences (We Robot, Governance of Emerging Technologies).
  • Education: Advanced degrees (PhD, Master's) are beneficial but not strictly mandatory; many organizations are open to visitors and collaborative research.
  • Portfolio Approach: Consider combining AI policy work with broader global improvement efforts (e.g., strengthening government quality) if direct AI impact is uncertain.
• Risk of Inaction:
  • Acknowledges the risk of "discrediting the cause" through alarmism but argues the greater risk is failing to contribute to a critical problem.
  • Recommends maintaining a measured tone and seeking expert consensus before taking strong public stances.

Future Outlook

• Field Maturity: AI policy is currently analogous to AI safety five years ago; the field is moving from "nebulous" problem framing to specific, formal research agendas and concrete white papers.
• Growth Trajectory: Expect a rapid expansion of hiring and opportunities as the need for specialized policy teams grows within industry and government.
• Call to Action: Urges experts to apply their specific backgrounds to this "neglected area," emphasizing that the field is in flux and values opportunism and adaptability.