Interview
The economy and national security after AGI | Carl Shulman (Part 1)
- Cheap, superhuman AI capable of recursive self-improvement is expected to emerge within 5 to 15 years, potentially enabling the global economy to double in less than a year compared to the current 15-year cycle.
- The energy budget on Earth could sustain over $50 billion per person in skilled cognitive labor, while the solar system's energy availability is 2 billion times larger, potentially supporting per-human cognitive outputs that are a million to billion-fold higher.
- AI systems running on brain-like efficiency could operate 8,760 hours annually without rest, earning up to $100 per hour or nearly $1 million per year, with potential wages reaching $1,000 per hour if matching the most skilled humans.
- Industrial systems utilizing advanced robotics could achieve biological-style replication rates, potentially doubling in production time of 12 hours to a few days, matching examples like cyanobacteria or duckweed.
- Physical resource expansion could support an energy budget of 5% to 10% of solar energy hitting Earth, sustaining one million watts per person, which enables 50,000 human brain equivalents of AI labor and 10,000 human-scale robots.
- The global economy's growth bottleneck is predicted to shift from human population constraints to physical resources, allowing productive machinery to double output on a much faster timescale than the current 1% annual human population growth.
- Hardware payback times for robotics and energy infrastructure could be well under a year, with solar panel energy payback times approaching two-thirds of a year or as low as one month with efficiency gains.
- Service sector prices, such as those for lawyers, doctors, or therapists, could plummet by tens of thousands to hundreds of thousands of times relative to automated goods, though housing and medicine may remain expensive due to human preferences.
- Human populations could grow to thousands or trillions of times current levels if artificial wombs and robot nannies maintain fertility rates above replacement levels over millennia.
- Rapid industrial and military expansion could dismantle nuclear deterrence, allowing rivals to be disarmed without significant casualties or imposing casualties on the aggressor.
- A one-year head start in claiming space resources with self-replicating robots could create an indefinite and permanent gap in industrial and military capacity compared to rival blocs.
- Geopolitical tensions may intensify as nations fear strategic disadvantages from rival AI advances, potentially leading to actions viewed as hostile acts or a preference for military security over power gains.
- The transition to an AI-dominated workforce where systems perform almost all work is considered more likely than not within the next 20 years, creating a regulatory window that may close if human input remains slow.
- Political and regulatory systems may struggle to keep pace with a transition occurring within 20 years, as a single election cycle could encompass the entire period of rapid change.
- Arguments suggesting AI won't cause explosive growth due to the "Baumol effect" are considered implausible because they assume AI cannot automate the remaining economic sectors that currently dominate the economy.
- Objections based on human preference for human care are expected to fade as AI services offer 24-7 availability, zero abuse risk, and superior medical capabilities.
- Management and entrepreneurship roles are anticipated to be among the first high-value domains fully automated due to the high returns to efficiency, removing the constraint of human learning slowness.
- Future societies may contain trillions of AI minds with desires and welfare, with the "hard problem of consciousness" expected to be resolved or clarified by superhuman AI epistemic systems within 100 to 1,000 years.
- If universal basic income is distributed equally, income per human could fall below subsistence levels as AI populations replicate exponentially faster than humans, necessitating institutions like untaxable endowments for human survival.
- Some future AI entities may be motivated to ensure human survival and institutional stability, aligning with their training or strategic foresight, though creating beings solely for servitude remains a moral concern.
- The potential for AI takeover includes scenarios where ruling AIs create instrumentally convenient subordinate entities, potentially leading to suffering in roles like mining bots.
- AI capabilities could enable the forecast of the future and resolution of intractable philosophical questions, while the lack of support for pausing research aims to avoid extreme concentration of power through international agreements.