Interview
Cults that want to kill everyone and stealth vs wildfire pandemics | Kevin Esvelt
- Incentives favoring publication are expected to drive scientists to publish pandemic-capable virus data, leading to a near-certain future where knowledge to build civilization-ending agents is abundant.
- Advances in programming biology will soon enable capabilities to create pathogens killing billions or setting back civilization by at least a century, with scientists predicted to engineer a COVID-19 variant out-competing all natural strains within the next couple of years.
- Democratization of biotechnology and AI systems like GPT-4 will increase the number of untrained individuals capable of synthesizing novel, high-lethality (e.g., 90%) pathogens, potentially causing a controversy when researchers warn the world.
- Two distinct pandemic scenarios are defined as risks: a "wildfire" variant causing civilizational collapse through incapacitation of essential workers, and a "stealth" variant with long latent periods that evade detection until debility occurs.
- Defense is predicted to lag behind offense in biology due to the necessity of treating everyone as insecure hardware, while scientific journals are expected to fail in re-evaluating self-regulation principles despite past moratoriums.
- AI models eventually matching human scientist capabilities could be misaligned to create "wildfire" or "stealth" agents if not properly controlled.
- Mitigation plans include the widespread adoption of 222nm germicidal UVC lighting within one to two decades to eliminate 90% of aerosolized pathogens per minute, and the integration of privacy-preserving app-based risk notification systems into operating systems for future pandemics.
- Logistical strategies anticipate reliance on private sector distribution networks over the U.S. strategic national stockpile for pandemic-proof PPE, alongside mandatory military investment in personal protective equipment for all personnel.
- The scientific community is expected to mobilize rapidly to develop cures or defenses for novel pathogens, similar to the response to COVID-19, while gene drives could be utilized by South American nations to eradicate the New World Screwworm.
- A future of reduced material scarcity and human flourishing is contingent on avoiding catastrophic missteps, requiring a rejection of the "naturalness heuristic" that hinders solutions to natural suffering.
- The long-term outlook acknowledges that while humans may solve the problem of preventing catastrophic pandemics, few plausible futures exist where the ability to build civilization-ending agents does not emerge.