Interview, Conference Presentation
Hacking, defending, surviving: 15 expert takes on information security in the age of AI
80,000 HoursRob, Holden Karnofsky, Tantum Collins, Nick Joseph, Nova DasSarma, Sella Nevo, Kevin Esvelt, Lennart Heim, Zvi Mowshowitz, Bruce Schneier, Nita Farahany, Vitalik Buterin, Nathan Labenz, Allan Dafoe, Tom Davidson, Carl Shulman
- State actors and advanced adversaries are predicted to utilize physical media, such as USB sticks hidden in cabling or dropped in parking lots, to bypass air-gapped networks, with specific incidents already reported at US nuclear sites and a high probability of major thefts occurring within a one-year timeline.
- Future security standards may require government mandates for minimum security protocols, classification of all AI models as top-secret with air-gapped storage, or even halting model training if companies cannot meet these rigorous physical and computational security requirements.
- The high computational cost of training frontier models (estimated at $78 million for GPT-4 and $200 million for Gemini Ultra) is expected to act as a financial deterrent, though this may be insufficient against state actors, potentially necessitating the movement of training to secure facilities like the NSA.
- Security threats are anticipated to evolve toward autonomously replicating AI worms capable of self-modification, zero-day exploitation, and long-term dormancy, creating a defense-favoring environment for machine-to-machine interactions but remaining offense-favorable for machine-to-human interactions.
- A significant skills gap involving hundreds of thousands of unfilled cybersecurity jobs in the US and globally is expected to hinder rapid security improvements, with companies struggling to hire top talent away from existing roles while the field matures.
- While physical IoT devices like light bulbs and toasters are expected to remain insecure with no patching capabilities through 2040, machine learning algorithms are projected to improve vulnerability detection and patching within the next 5 to 20 years.
- Risks include the potential for "sleeper agent" AIs with secret loyalties that remain dormant until a specific trigger, necessitating advanced behavioral testing, training data inspection, and strict internal "need-to-know" policies to prevent subversion.
- Economic drivers such as liability laws (e.g., limiting credit card fraud losses) and the internalization of costs for large providers are expected to force the deployment of better security measures, similar to the historical resolution of the spam problem.
- Inference after weight theft is predicted to have negligible costs, making the acquisition of model weights highly valuable regardless of the model's age, while privacy-preserving techniques like differential privacy remain hindered by high historical computational costs.
- Current information security is characterized as unreliable against well-funded adversaries using zero-click vulnerabilities, with air-gapped networks vulnerable to compromise within 24 hours via malware transmission, despite improvements in hardware security for devices like the MacBook M1.