Interview
Defensive acceleration & regulating AI when you fear government | Vitalik Buterin
- Core Philosophy: Vitalik Buterin's essay "My Techno-Optimism" introduces "Defensive Accelerationism" (DEAC), a synthesis of accelerationism and existential risk concerns aimed at accelerating technologies that enhance human defense and agency.
- The Three Techno-Views: Buterin categorizes technology perspectives using a fork-in-the-road meme:
- Techno-Optimism: Views the road as a single path toward a bright future where a "bear" (risk) chases you; you must move fast to reach safety.
- Pessimism/Decay: Views the road as leading away from a past "castle" toward a dark future, implying technology causes decline.
- Defensive Accelerationism: Acknowledges the fork, the bear, and the destination; asserts that while the path forward offers benefits, specific choices must be made to avoid the dark fork while outrunning the bear.
- AI Risk Probability: Buterin lowered his estimated probability of human extinction from AI (p-doom) from roughly 10% to between 8% and 9%.
- Reasons for Probability Shift:
- Slower Progress: AI capabilities in 2024 feel more incremental compared to the massive leap seen between 2022 and 2023.
- Saturation of "Scale": The industry may have already passed the one-time "scale" insight, where training costs jumped from $100M to $1B, which cannot be replicated easily without trillions in investment.
- Cultural Shift: AI risk concepts have filtered into public consciousness, causing some engineers and companies to move less aggressively toward unaligned deployment.
- Centralization vs. Decentralization: Disagreements on AI regulation are often driven by a fundamental distrust of authority, where:
- Centralized AI fears: Users fear AI will enable totalitarian governments or massive corporate monopolies (e.g., facial recognition used to suppress dissent in Russia).
- Decentralized AI fears: Skeptics worry that "safety" measures require centralized control that creates a single point of failure or authoritarian leverage.
- Polytheistic vs. Singleton AI: Buterin discusses the tension between:
- Singleton AI: A single super-intelligent AI managing the world to enforce peace and safety, which risks becoming a global totalitarian government.
- Polytheistic AI: Multiple distributed AIs for different nations or individuals, which preserves liberty but risks instability if one actor achieves a "step function" leap in capabilities and dominates the race.
- The "Step Function" Risk: If AI growth is super-exponential, a minor lead or a single "magic leap" (e.g., nanotech, 100x compute boost, or algorithmic breakthrough) could allow one actor to immediately outpace all others and capture total control.
- Biodefense as Defense Technology: Buterin emphasizes accelerating "micro-physical" defense technologies, specifically:
- Wastewater Surveillance: To detect new pathogens earlier.
- Air Filtration: Universal HVAC upgrades (HEPA/UV) to reduce airborne transmission rates (R0) in indoor settings.
- Nasal Vaccines: Rapid, non-invasive vaccines to stop transmission rather than just symptoms.
- Funding Source: These initiatives were partially funded by crypto donations from the "India COVID Crypto Relief" campaign, which raised $470 million in 2021.
- Cyber Defense: Buterin advocates for zero-knowledge proofs (ZKPs) to solve the "unique human" problem, allowing users to prove they are one person (preventing bots) without revealing their identity or KYC data.
- Info Defense: The "Community Notes" feature on X (Twitter) is cited as a successful model for fighting misinformation:
- It uses a matrix factorization algorithm to identify notes that receive high ratings from across the political spectrum (high "helpfulness") while filtering out partisan notes (high "polarity").
- It avoids pre-assuming a central authority on truth, instead relying on cross-partisan consensus.
- Quadratic Funding: Buterin introduced "Pairwise Bounded Quadratic Funding" to prevent collusion in funding mechanisms, mathematically bounding the damage an attacker can extract even if they capture multiple identities.
- Human-AI Merging: Buterin proposes Brain-Computer Interfaces (BCIs) as a potential long-term solution to preserve human agency:
- The alternative to merging is humans becoming "pets" to super-intelligent AIs with no decision-making power.
- While "mismatch" concerns exist (humans slowing down AI), intelligence is argued to be distinct from flight because it relies heavily on social and political contexts where humans have evolved complexity.
- Crypto's Current Status: Buterin acknowledges crypto's slower-than-expected real-world impact but highlights recent breakthroughs:
- Scalability: The upcoming "Dencun" hard fork (Proto-Danksharding) will significantly reduce Layer 2 transaction costs.
- Privacy: ZK-proofs enable new use cases like "ZooPass" for privacy-preserving group membership.
- Decentralized Social: Farcaster exemplifies an open protocol where content is on-chain but clients (e.g., Warpcast) are interchangeable, avoiding single-vendor lock-in.
- Effective Altruism (EA): Buterin distinguishes between the "Internet EA" (philosophical, moderate, meta-ethical) and the "In-Person EA" (often more unilateral and politically aggressive):
- He defends the core EA ideas of scale and moral consistency but criticizes actions (e.g., by the Singularity Institute for AI) that violate the "Golden Rule" of meta-ethics (acting in ways you'd accept if your enemies did the same).
- He advocates for a "moderate" approach to charity and self-improvement, avoiding extreme unilateralism.
- Startup Opportunity: 80,000 Hours and Entrepreneur First are running a "Defensive Acceleration" incubator program offering $250,000 in investment and coaching for startups building technologies that enhance human defense against AI risks and aggression.
- Hiring: 80,000 Hours is hiring for a Head of Video and Head of Marketing, with roles focused on expanding their video content strategy and managing a $3 million annual marketing budget.