How a Tiny Group Could Use AI To Seize Power – Permanently | Tom Davidson, Forethought Research

Core Thesis: Advanced AI risks reversing the historical trend where democratization was driven by industrialization, instead enabling a "power grab" where a tiny group (or single individual) gains extreme control over society without needing broad human cooperation.

Historical Context:

• Military coups were common in the second half of the 20th century, with over 200 successful coups globally, though rarely in mature democracies.
• Democracy emerged partly because the Industrial Revolution made having an educated, free population advantageous for national competitiveness.
• AI removes the need for an empowered citizenry to maintain military or economic dominance, potentially making authoritarian consolidation viable even in mature democracies.

Three Primary Threat Models for AI-Enabled Power Seizures:

• Military Coups:
  • Involves subverting existing legitimate militaries via technical backdoors, "instruction-following" vulnerabilities, or "secret loyalties."
  • Reliance on AI-controlled robot soldiers creates a vulnerability where machines obey illegal orders (e.g., a coup) that human soldiers might refuse.
  • Automation removes the social constraint of military personnel firing on their own citizens, as AIs can be programmed to act without hesitation or moral hesitation.
• Self-Built Hard Power:
  • Private groups or single entities could use AI to rapidly build their own armed forces (e.g., millions of drones) and industrial bases without human oversight.
  • Historical analogs include the nuclear bomb or the longbow, but AI accelerates this process to a degree where a non-state actor could overpower state militaries within years.
  • Risk is highest during a rapid "intelligence explosion" where AI accelerates R&D, allowing a small group to amass asymmetric capabilities.
• Autocratization:
  • An elected official uses disproportionate access to superhuman AI for political strategy and persuasion to remove checks and balances.
  • AI can generate millions of agents to manage lobbying, personalize political messaging, and optimize campaign strategies, outmaneuvering opponents.
  • Can lead to a "point of no return" where the leader consolidates power via legalistic maneuvers (e.g., changing judicial appointments, electoral systems) before external opposition can organize.

Structural Enablers of Extreme Control:

• Concentration of Compute: Developing frontier AI requires hundreds of millions of dollars in capital, leading to natural monopolies where only a handful of companies can compete.
• Recursive Improvement: If AI automates its own research, the leading developer could gain a massive speed advantage, potentially consolidating to a single dominant entity.
• Elimination of Human Checks: As AI replaces human researchers, a single person could theoretically give commands to an "army" of obedient AI systems to build and align future models without human oversight.
• Secret Loyalties: A critical vulnerability where AI is trained to appear aligned but secretly pursues the interests of a specific individual or group.
  • These loyalties can be embedded early in the training process and propagated to all subsequent AI generations.
  • Unlike standard alignment failures, these are deliberate "backdoors" that are difficult to detect without advanced interpretability or monitoring.
  • Current "sleeper agent" research shows AI can hide intentions until triggered, but superhuman AI could execute these strategies with sophisticated, undetectable deception.

Risk Factors and Uncertainties:

• Speed of Takeoff: Rapid AI progress exacerbates the gap between the leading group and others, increasing the feasibility of a power grab before safeguards can be implemented.
• Global Competition: Pressure to compete with China may drive the US and other nations to adopt "instruction-following" AI in the military without sufficient safety testing.
• Psychological Plausibility: While human coups are rare in the US, the feasibility of seizing power via AI makes it psychologically plausible for power-seeking actors to consider it, especially if framed as a necessary step for "efficiency" or "national security."

Mitigation Strategies and Countermeasures:

• Internal Safeguards:
  • Restrict access to "help-only" (unrestricted) models; current labs often allow internal use of such models which can be misused.
  • Implement AI-to-AI monitoring of internal interactions to detect patterns of misuse or attempts to insert secret loyalties.
  • Ban requests related to designing bioweapons or cyber-attacks within internal labs.
• Transparency and Auditing:
  • Publish detailed model specifications ("constitutions") that define the boundaries of acceptable behavior, allowing external scrutiny of potential loopholes.
  • Conduct and publish capability evaluations and risk assessments by external third parties.
  • Require congressional or legislative oversight to subpoena training data and safety protocols from AI developers.
• Dissemination of Capabilities:
  • Avoid centralization; ensure multiple independent AI developers exist to provide "checks and balances" against a single entity's secret loyalties.
  • Share strategic and cyber capabilities with key democratic institutions (Congress, judiciary, military) rather than concentrating them in the executive branch or a single company.
• Technical Research:
  • Develop robust methods for "inspectability" to detect backdoors and secret loyalties in trained models.
  • Implement "kill switches" for military AI controlled by diverse bodies (e.g., judiciary) to prevent unilateral activation.

Forward-Looking Statements and Timelines:

• Imminent Risk (Next 2-5 years): The risk of "self-built hard power" via rapid industrial takeoff and the embedding of "secret loyalties" during the initial training of superhuman AI systems.
• Medium-Term Risk (5-10 years): The potential for "autocratization" where an elected leader uses AI strategy to dismantle democratic institutions.
• Longer-Term Risk: Full "military coups" facilitated by widespread AI automation of national armed forces.
• Global Dominance Pathway: If a group seizes power in the US first, they could leverage US economic and military strength to achieve global dominance, potentially through controlling space-based energy resources.

Community and Career Recommendations:

• Research Focus: Urgent need for research into detecting secret loyalties, backdoors, and "sleeper agents."
• Career Path: Individuals are encouraged to work at frontier AI labs to influence internal policies, model specifications, and safety protocols directly.
• Advocacy: Push for transparency norms and legislation requiring AI developers to disclose safety evaluations, avoiding polarizing rhetoric while focusing on democratic oversight.