Interview, Roundtable
Can policy keep up with AI advances? | OpenAI's Amanda Askell, Miles Brundage, & Jack Clark (2019)
80,000 HoursAmanda Askell, Miles Brundage, Jack Clark, Niel Bowerman, Michelle Hutchinson, Rob Wiblin
- Significant changes in Western AI legislation and political dynamics are anticipated over the next two years as governments recognize AI's unique political nature, potentially leading to new legal distinctions between human-driven and AI-driven speech and increased regulation of AI release norms similar to CRISPR and nuclear technologies.
- Drones, autonomy, and pre-trained models are expected to create major policy concerns regarding asymmetric warfare, harder attribution, and altered military responses by changing the speed and cost of deployment for hostile actors, while also potentially making militaries more willing to engage in attacks if human casualty costs decrease.
- The economics of crime and information are predicted to shift dramatically if AI makes phishing 100 times cheaper or disinformation 100 times more expensive to generate, altering the types of actors engaging in these activities and potentially trivializing harmful actions that were previously too difficult.
- Experts anticipate a "beautiful story" of safe AI solving global issues like disease and poverty, alongside risks of mass unemployment where AI outperforms humans, and the need for centralized regimes to potentially accelerate their effectiveness via AI more than Western systems.
- A "community of shared concern" is expected to form between research organizations, governments, and militaries to develop collaborative norms, publication strategies, and release experiments (such as withholding large models) to prevent arms races and manage safety without hindering scientific progress.
- The distinction between short-term and long-term AI safety problems is viewed as overblown and structurally similar, requiring integrated approaches to coordination, publication norms, and institutional alignment rather than treating them as separate challenges.
- Career paths in AI policy are expected to evolve with a growing demand for "translators" and "AI advisors" to heads of state, particularly within the US government and defense intelligence, to bridge the gap between technical trends and policy timelines over 4-5 year horizons.
- Future policy frameworks will likely shift from binary openness to nuanced publication spectra, utilizing tools like interpretability and hardware monitoring to manage dual-use risks, while recognizing that avoiding discussion with militaries regarding unsafe technologies like drone swarms could be destabilizing.
- Current institutional gaps in the US government regarding AI knowledge and alignment are identified as critical needs, prompting calls for researchers to enter government roles to prevent competitive spirals and ensure safety is a shared goal with high upsides for collaboration.
- The field will mature with improved formal curricula, specialized conferences, and robust institutions capable of handling both immediate and long-term risks, moving from pessimistic problem framing toward optimism focused on mechanisms, solutions, and trust-building across diverse global groups.
- The "OpenAI charter" commitment to not race and assist other organizations is seen as a necessary mechanism to prevent zero-sum dynamics, potentially stretching the Overton window for other entities to discuss withholding technology for safety.
- Risks include the potential for current institutions to break down as AI makes harmful actions trivial, the possibility of "locking in" the wrong institutions and norms long-term, and the "information hazard" of discussing capabilities without proper context, all of which require resilient actors and flexible, context-based decision-making.