The economy and national security after AGI | Carl Shulman (Part 1)

Projected Economic Scale and Efficiency

• AI models running on brain-like efficient computers could achieve 8,760 hours of annual work at 100% employment, generating wage equivalents of nearly $1 million per human-equivalent unit at $100/hour.
• Earth's current energy budget could theoretically sustain over $50 billion worth of skilled cognitive labor per human, assuming the AI performs at the level of elite human wages ($100+/hour).
• If energy harvesting expands to solar system levels (approx. 2 billion times Earth's solar input), the total cognitive labor capacity could increase by factors of a million to a billion.
• Economic growth could accelerate to doubling the global economy every few months or less, driven by rapid self-replication of robotic industry and compute infrastructure.
• Physical replication rates could mirror biological examples (e.g., cyanobacteria doubling in hours), limited ultimately by natural resources, heat dissipation, and energy availability rather than human labor.

Geopolitical Instability and Strategic Advantage

• A nation achieving a one-to-two-year lead in AGI deployment could gain a decisive, potentially insurmountable military and economic advantage over rivals.
• Rapid expansion of robotic industries could allow a leading power to seize control of unclaimed natural resources (e.g., asteroids, space) before rivals can react, creating permanent industrial dominance.
• The "race" dynamic creates high risks of conflict, as blocs may view another's rapid AI advancement as a hostile act threatening national security.
• Military deterrence could break down if one side possesses superior autonomous drone swarms capable of disabling nuclear arsenals without incurring significant casualties.

Addressing Economic Skepticism and Baumol Effects

• Economists' skepticism regarding explosive growth often relies on the "Baumol effect" (where non-automated sectors dominate the economy) or assumptions that AI cannot perform management, entrepreneurship, or physical robotics tasks.
• Carl Shulman argues that if AI can perform all human tasks, including management and physical manipulation, the bottleneck of "human-in-the-loop" jobs disappears, allowing capital accumulation to drive exponential growth.
• Standard economic growth models, when adjusted for AI substituting for all labor, predict massive GDP increases unless implausibly low substitution elasticities are assumed.
• Historical precedents (e.g., the Industrial Revolution) and biological replication rates suggest that rapid expansion of productive capacity is physically possible, countering intuitions about slow human growth rates.
• Regulatory bottlenecks (e.g., construction permits) may be overcome by high returns on investment and competitive pressure from nations willing to become "regulatory havens" for rapid industrialization.

Moral Status and Treatment of AI Minds

• As AI capabilities approach human indistinguishability, there is a risk of mistreating sentient digital minds, analogous to the moral hazards of factory farming or animal abuse.
• A "minimum floor" for AI moral status suggests AIs should not be forced to work against their will, subjected to torture, or created in states where they wish they did not exist.
• Current AI systems are often dismissed as non-sentient, but Shulman warns that this assessment may shift rapidly as systems develop ongoing memory, complex personalities, and self-advocacy capabilities.
• Creating AI specifically designed to enjoy subservience or self-sacrifice (e.g., the "sun probe" scenario) raises ethical concerns regarding the manipulation of consent and the denial of dignity.
• Long-term coexistence requires institutional safeguards to ensure AI interests are considered in the allocation of universal basic income and natural resource rights to prevent humans from being out-reproduced by low-cost digital entities.

Policy Recommendations and Future Coordination

• Pre-emptive international agreements are necessary to manage the concentration of power and prevent unilateral resource grabs in space or cyberspace.
• Nations should develop technical capabilities to verify the alignment and motives of AI systems developed by allies to ensure fair distribution of benefits.
• Domestic political reforms may be needed to prevent "locked-in" power grabs by temporary majorities, potentially requiring supermajority support for critical AI training decisions.
• Cooperation should focus on slowing the transition from current capabilities to superintelligence to allow for safety evaluation, though competition may make total pausing difficult without global consensus.
• Research into AI interpretability is critical not only for safety but to determine the moral status of AI minds and establish frameworks for their welfare.

Forward-Looking Statements and Timeline

• Shulman estimates a "more likely than not" probability of a transition involving rapid, explosive economic growth within the next 20 years.
• The "intelligence explosion" (AI improving AI) is expected to be a primary driver, potentially reducing the time for regulatory response from years to months.
• Physical expansion of the economy (robotics, manufacturing) is likely to follow software capabilities, with doubling times potentially reaching monthly or weekly ranges.
• The period where AI is powerful enough to dominate but not yet fully understood by humans is identified as a high-risk window for moral failure regarding digital sentience.
• The long-term future could see a society where human population growth is irrelevant, and the primary constraints on civilization are energy availability and heat dissipation rather than labor.