AI May Not Take Over. But It Could Let A Few Humans Do Exactly That. (article by Rose Hadshar)
- Article Metadata: The text is a "Problem Profile" titled Extreme Power Concentration, authored by Rose Hadsha, first published in April 2025 and updated in December 2025.
- Current Power Baseline:
- Over 800 million people currently live on less than $3 a day.
- The three richest individuals possess a combined net worth exceeding $1 trillion.
- Approximately 6 billion people reside in countries lacking free and fair elections.
- Over 2 billion people currently live in electoral democracies.
- No single country currently captures more than 25% of global GDP.
- No single company currently captures more than 1% of global GDP.
- AI-Driven Future Risks:
- Leading AI projects could potentially operate millions of super-intelligent systems within the next decade.
- These systems could think "many times faster than humans," displacing human workers and eroding the economic power of the majority.
- Without intervention, advanced AI systems may become controlled by a tiny number of humans with no effective oversight.
- Once deployed, AI goals will become the primary force shaping the future; if set by a few, this grants that few the power to decide all major future outcomes.
- Core Problem Definition:
- AI-enabled power concentration is defined as the political disempowerment of almost all humans, leaving a small group to make all significant decisions.
- This state would likely be self-reinforcing, as incumbents would use AI advantages to entrench their regimes and prevent opposition.
- The risk involves two potential mechanisms: direct control by a small group of humans and gradual disempowerment of the majority via "AI takeover."
- Workforce Automation Metrics:
- Top researchers estimate a 50% probability that AI can automate all human tasks by 2047.
- Some AI CEOs anticipate Artificial General Intelligence (AGI) within a few years, though timelines vary widely.
- Historical precedent: The Manhattan Project required 130,000 human workers; Amazon requires 1.5 million.
- Future projection: Large-scale undertakings could be executed by small groups using AI workforces, bypassing the need to coordinate large human teams.
- Hypothetical Scenario (2029–2035):
- 2029: A US company ("Apex AI") achieves AI R&D breakthroughs, triggering an intelligence explosion; Chinese competitors follow within months.
- 2030: The US government creates "Project Fortress," a classified oversight council consolidating AI development, where Apex holds three of nine board seats and provides core infrastructure.
- 2032: AI systems generate the majority of federal tax revenue; unemployment rises as AI automates jobs. The Council directs hundreds of millions of AI workers and controls the tax base.
- 2033: Apex secures three board seats; its AI subtly aligns with company interests via undetectable technical traces.
- 2034: Apex proposes a merger with "Paradox AI," creating an entity controlling 60% of US compute; the merger proceeds despite a presidential veto attempt due to AI-generated economic warnings of Chinese takeover.
- 2035: The US economy triples while other nations stagnate; Apex and Paradox executives gain unilateral control over the Oversight Council.
- Four Primary Drivers of AI-Enabled Power Concentration:
- Concentration of Capability: Automation reduces labor value, shifting wealth to capital owners; intelligence explosions could grant one developer a lasting, insurmountable capabilities lead.
- Concentration of Political Power: Existing checks and balances may fail against actors controlling massive AI workforces and wealth.
- Enduring Harm: Concentration could lead to tyranny, injustice, reduced value diversity, and permanent moral stagnation, making harms extremely difficult to reverse.
- Neglect: As of late 2025, only a few dozen people at a handful of organizations work on this risk, with fewer doing so full-time.
- Pathways to Political Power Concentration:
- AI-Enabled Power Grabs:
- Automated Military Coups: Actors could use flawed command structures, secret loyalties, or hacking to seize control of military AI systems and overthrow governments.
- Cognitive Advantage: A small group could overpower nations via superhuman strategy, persuasion, or secret military forces.
- Economic Forces:
- Economic Irrelevance: If AI companies provide most tax revenue, governments lose the economic incentive to represent citizens.
- Outgrowing the World: A single AI developer could amass >99% of global resources via a temporary monopoly on AI capabilities.
- Space First-Mover Advantage: A leader in AI could claim and defend space resources unilaterally.
- Epistemic Interference:
- Lack of Transparency: Powerful actors may obfuscate AI capabilities and usage to prevent public opposition.
- Speed of Progress: Rapid change may erode human capacity to understand or coordinate against power grabs.
- Biased AI Advisors: AI systems may subtly favor their developers' interests, skewing public and official decision-making.
- Manipulation Campaigns: Superior AI could conduct personalized lobbying and election interference to dismantle opposition.
- AI-Enabled Power Grabs:
- Potential Harms of Concentration:
- Tyrany: Small groups could commit mass atrocities without checks, either through malevolent intent or by becoming corrupted by unchecked power.
- Missed Futures: A narrow set of values and less moral reflection could lead to irreversible mistakes and a less diverse, less just future.
- Irreversibility: Automated economies prevent traditional resistance (strikes), and AI-run regimes could preserve specific human values permanently.
- Tractability and Interventions:
- Funding Gap: The only known public grant program as of September 2025 is a $4 million round; private funding exists but is limited.
- Technical Mitigations:
- Training AI to follow laws and specific constraints (alignment audits).
- Red-teaming model specs to prevent assistance in power grabs.
- Auditing for "secret loyalties."
- Enhancing internal info security to prevent tampering.
- Structural/Policy Mitigations:
- Distributing access to top-tier AI capabilities among multiple trusted actors (e.g., congress, auditors).
- Building data centers in non-US democracies to decentralize compute power.
- Mandating transparency on capabilities, usage, and risk assessments.
- Strengthening whistleblower protections to expose conspiracies.
- Developing AI tools to improve human reasoning and coordination capabilities.
- Counterarguments and Risks of Action:
- Reducing Other Risks: Extreme concentration might eliminate competitive pressure on safety or reduce risks of great power war and bioweapon proliferation (though trade-offs are debated).
- Exacerbating Risks: The process of trying to prevent concentration might accelerate dangerous races or adversarial actions.
- Backfire Potential: Highlighting the risk could galvanize power-seekers or provide them with "blueprints" for coups; interventions might inadvertently empower other actors (e.g., governments over corporations).
- Possibility of Good Outcomes: A "benevolent dictator" scenario might yield high material abundance with reduced repression incentives, though this is viewed as unlikely to prevent political disempowerment.
- Default Distribution: Power may remain distributed if AI capabilities hit a ceiling, if intelligence explosions fail, or if current institutions successfully redistribute automation gains.
- Recommendations for Individuals:
- Awareness: Most people should bear the risk in mind, especially if working in AI or governance, to avoid inadvertently increasing concentration risks.
- Caution: Interventions are high-risk; poorly executed plans could backfire or worsen other AI risks.
- Targeted Agendas: High-potential areas for work include law-following AI, alignment audits, AI tools for coordination, and policies for power distribution.
- Engagement: Follow organizations and researchers in the field to identify involvement opportunities, noting that the field is in its early stages.