Interview
Dawn Song: Adversarial Machine Learning and Computer Security | Lex Fridman Podcast #95
- Security vulnerabilities are expected to persist indefinitely, with industry approaches shifting toward formally verified systems that prove specific properties like memory safety while remaining susceptible to attacks outside their verified scope, such as side-channel attacks.
- Threat actors will increasingly target humans as the weakest link through sophisticated social engineering and deep fakes, prompting the development of AI-powered security assistants for identity verification and defense against these manipulations.
- Machine learning systems face adversarial risks at both inference and training stages, where minimal poisoned data points can induce backdoor attacks or incorrect model learning, necessitating defenses based on spatial, temporal, and multi-modal consistency checks.
- Physical adversarial attacks, including pixel-level perturbations on wearable objects and modifications to road signs, will remain effective under real-world variations, though autonomous vehicles are projected to adopt multi-sensor fusion strategies combining vision, radar, ultrasonic, and sound data to mitigate sensory attacks.
- Real-world APIs and black-box systems are vulnerable to model theft and output manipulation, while model queries can extract sensitive training data like social security numbers, requiring differential privacy mechanisms that add noise during the training process.
- The digital data economy will likely evolve toward clear property rights and user-controlled data usage, enabling consumers to choose between free services with data access or paid privacy subscriptions, alongside the use of blockchain platforms combining zero-knowledge proofs and secure computing for confidential transactions and immutable usage logs.
- Program synthesis is anticipated to advance toward generating complex logic including loops and recursion in limited domains, with future research focusing on generalization to unseen inputs and adaptation to new tasks within a growing community of neural program synthesis.
- Global advancement in AI is predicted to foster cross-border collaboration and economic growth through open access to academic research and code, with personal creation of programs, robots, and ideas serving as a source of fulfillment.