Interview, Roundtable
Can policy keep up with AI advances? | OpenAI's Amanda Askell, Miles Brundage, & Jack Clark (2019)
80,000 HoursAmanda Askell, Miles Brundage, Jack Clark, Niel Bowerman, Michelle Hutchinson, Rob Wiblin
- OpenAI's Strategic Publication Norms: OpenAI released a research paper and a small version of its GPT-2 language model while withholding the full dataset and large model weights to balance scientific transparency with mitigation of malicious use risks.
- Explicit Safety Experiment: The decision to limit the release of the text-generation model serves as a "responsible experiment" to establish new norms for publication in AI, aiming to prevent the normalization of unrestricted access to powerful dual-use technologies.
- Malicious Use Threats: The speakers identify specific near-term threats including autonomous drone warfare (lowering the cost and difficulty of asymmetric attacks), large-scale disinformation via text generation, and the harassment of individuals via deepfakes.
- AI and Political Systems: AI is viewed as a tool that may accelerate authoritarian control through scalable surveillance and speech manipulation, potentially creating a competitive advantage for centralized states over liberal democracies in managing their political systems.
- The "Short-Term" vs. "Long-Term" Distinction: The panel argues that the distinction between immediate AI safety issues and long-term AGI alignment is often overblown, positing that the mechanisms for coordination and safety governance required now are structurally similar to those needed in the future.
- Avoiding Arms Races: Miles Brundage and Amanda Askell suggest that development races do not necessarily have to be adversarial; game theory indicates that shared upsides and shared downsides to unsafe development can incentivize collaborative races rather than "races to the bottom" on safety.
- Coordination Mechanisms: OpenAI proposes developing institutional, software (e.g., interpretability), and hardware-based mechanisms to build trust and verify safety between competing organizations and governments.
- Military Engagement: Jack Clark argues the AI community must avoid stigmatizing engagement with militaries, advocating instead for collaboration to ensure that if militaries develop autonomous systems, they are engineered with safety constraints.
- Career Entry Strategies:
- Newsletter Creation: Writing policy-specific newsletters is identified as a high-leverage activity for building credibility, calibrating thinking, and networking with policymakers and organizations.
- Project Management: Project manager roles are highlighted as accessible stepping stones for transitioning into AI policy, allowing individuals to learn the field while managing internal processes like safety reviews.
- Technical Translation: The ability to translate complex technical trajectories into coherent narratives for non-technical policymakers is deemed a critical skill, filling the gap between technical experts and traditional policy lobbyists.
- Government vs. Private Sector: While OpenAI offers a unique, mission-driven environment, there is a strong push for talent to enter government agencies (particularly the US Department of Defense and intelligence community) where there is a lack of AI-literate staff and a high potential for immediate policy impact.
- Required Expertise: Successful AI policy professionals benefit from "interactional expertise" (the ability to converse fluently with technical experts) and the capacity to synthesize information across multiple domains (economics, game theory, law, and technical AI).
- OpenAI Hiring: OpenAI is currently hiring for research assistant roles (for career switchers with significant experience in other domains) and research scientist roles, with project manager roles planned for the near future.
- Future Conference: The panel anticipates the eventual formation of a dedicated AI policy conference to centralize community discussion, currently fragmented across general AI and policy events.
- Long-term Optimism: Despite safety concerns, the speakers express optimism about AI's potential to solve grand challenges like global poverty and disease, provided that robust safety and coordination mechanisms are established.