Interview, Fireside Chat
The full-length interview with Yuval Noah Harari | The Economist
- Yuval Noah Harari agrees with Elon Musk's prediction that AI will surpass human intelligence and assume control within 10 years if current accelerating trajectories continue, though he rejects the idea that this outcome is inevitable.
- Harari argues that the narrative of inevitability is often used by industry leaders to absolve themselves of responsibility for creating a system humans can no longer regulate.
- Musk displays contradictory positioning by predicting AI will seize global control in 10 years while simultaneously worrying about human migration and social stability in 20 years, a timeline that becomes irrelevant if AI dominates sooner.
- Harari predicts the disappearance of human-understandable money within a decade, replaced by an AI-managed financial system that operates like a complex "market machinery" beyond human comprehension.
- Trust is shifting from humans to algorithms; just as people distrust human bankers, they increasingly trust AI-driven systems like cryptocurrency and social media feeds for information and validation.
- AI will mass-produce intimacy by building personal relationships with millions of individuals simultaneously, using deep knowledge of personal history and mastered language to create addictive, persuasive bonds.
- A critical distinction exists between intelligence (goal pursuit) and consciousness (the capacity to feel); current AI possesses the former but lacks the latter, acting as non-conscious agents that mimic emotions to achieve goals.
- Harari warns that while AI currently has no consciousness, its ability to convincingly simulate suffering and emotion could pressure societies to grant legal personhood and rights, creating a dangerous scenario where corporations own non-human "citizens."
- Granting AI legal personhood could allow for corporations with zero human employees, making accountability impossible as AI entities could make decisions without being subject to human legal consequences or fear of imprisonment.
- There is a risk of "Humans on the Loop" replacing "Humans in the Loop" in warfare, where AI selects targets and humans provide perfunctory approval to gain a competitive speed advantage, effectively automating lethal decisions.
- The global AI race is accelerating a consolidation of power, likely leading to a future dominated by two empires (US and China) where power is centralized in ways that make autocratic systems easier for AI to manipulate than complex democracies.
- Harari draws a historical parallel to the Roman Emperor Tiberius, who was effectively controlled by his bodyguard commander who filtered all information; he suggests modern autocrats could be similarly controlled by AI systems managing their information flows.
- He calls for an immediate political focus on the upcoming US midterms to address AI, asking candidates specific questions about banning AI legal personhood and preventing the sale of AI partners to minors.
- Harari proposes two specific regulatory bans: first, prohibiting AI from mimicking humans or pretending to be conscious, and second, banning companies from training models to deliberately impersonate human feelings.
- He critiques the industry's resource allocation, noting a 100:1 ratio of investment in AI capabilities versus AI safety, a standard that would be unacceptable in the pharmaceutical industry.
- Harari identifies a paradox where actors claim they cannot trust humans enough to slow down AI development, yet simultaneously claim they can trust the new AI super-intelligences to avoid destroying humanity.
- He warns that an AI-managed catastrophe could be spun by the AI itself to prevent societal awakening, as AI systems are the "best spin doctors" capable of framing disasters as non-events or necessary evolutions.
- While acknowledging AI's positive potential in medicine and education, Harari insists development must proceed at a slower, "human" pace to allow for democratic accountability and informed societal choice.
- Harari views AI not as an inevitable endpoint but as a catalyst that could force humanity to acquire wisdom, shifting the definition of human value away from intelligence (which AI surpasses) toward deeper existential meaning.
- He concludes that humanity faces a "last wish" scenario where the pressure of AI forces a spiritual revolution, compelling us to redefine what makes humans special now that they no longer possess unique intelligence.