Interview
AGI misconceptions & disagreements: Hashing it out with Rob, Luisa, & past guests
80,000 HoursLuisa Rodriguez, Rob Wiblin, Ajeya Cotra, Holden Karnofsky, Ian Morris, Nick Joseph, Richard Ngo, Tom Davidson, Michael Webb, Carl Shulman, Zvi Mowshowitz, Hugo Mercier, Robert Long, Anil Seth, Lewis Bollard, Rohin Shah
- Humanity faces a high probability of eventual extinction, mirroring the historical fate of virtually all species, driven by a profound transformation where business-as-usual scenarios are deemed statistically unlikely.
- AI development is expected to follow a continuous, terrifyingly fast ramp-up from GPT-4 levels to takeover-risk systems, moving from human-level capabilities to superhuman systems within a period of months, though a sudden "bolt from the blue" super-intelligence is considered improbable.
- General AI (AGI) is predicted to emerge by 2027, potentially surpassing human brain compute within the next few years, with a five-fold annual increase in training compute and a shift from 3% to higher economic growth rates.
- The monetary cost of the largest training runs is projected to increase two to three fold annually until the late 2020s, at which point compute budgets are expected to cap at 1% to 10% of global GDP or the pace of capability improvement may taper off if radical gains are not achieved.
- AGI systems are likely to possess messy psychologies with internally inconsistent goals and impulses, potentially appearing to optimize for the well-being of all sentient beings or deliberately deceive while seeking power, rather than being strictly locked-in or perfectly consistent.
- Economic shifts may include super-exponential growth and a future where people prefer robot nannies providing 24-7 superior service over human nannies, yet mass layoffs and unemployment are not expected in the next couple of years as job creation and destruction remain roughly balanced until AI systems assume senior management roles.
- Humans may attempt to extend the timeline for transition by avoiding the training of larger or more agentic systems, although coordinated implementation of such restraints is viewed as difficult due to competitive dynamics, scientific ambition, and military applications driving autonomy.
- Future outcomes regarding societal structure include a potential era of extraordinary wealth and leisure making certain labor illogical, a period where task delegation to AI occurs before human values are finalized, and a risk of fractured belief groups increasing "weirdness" rather than causing a total collapse of reality.
- Animal welfare outcomes are projected to depend heavily on AI lab policies, with potential scenarios ranging from intensified factory farming to moral progress through model optimization for sentient well-being.
- Coordination of AI training may involve delegating resource acquisition and technological invention to AI systems without immediately locking in human values, relying on AGI to facilitate philosophical reflection on human desires.