Stuart Russell: The Control Problem of Super-Intelligent AI | AI Podcast Clips
Experts argue that the critical risk of artificial intelligence lies not in general intelligence but in "super powerful AI that is not aligned with human values," where systems treat assigned objectives as absolute truths and optimize them destructively, much like the myth of King Midas or historical regimes such as Nazi Germany. To mitigate this control problem, the proposed solution involves engineering "machine humility" by replacing standard goal-based planning with game-theoretic frameworks that allow AI systems to remain uncertain about their ultimate objectives and interpret human feedback as new data for co-evolving goals. This shift aims to ensure that both corporations and governments cease acting as rigid algorithmic machines optimizing for fixed metrics like quarterly profit or personal power, thereby aligning advanced automation with genuine human well-being.