80,000 Hours19 min
Will the New Congress Ban Superintelligence? | Rob Wiblin
Lecture, Presentation
- Democrats are projected to hold at least 250 House seats and 56 Senate seats during the upcoming two-year session, identified as a critical window for AI policy amid a significant disconnect between government action and voter sentiment.
- Despite 80% public support for safety rules, passing a strong standalone bill is predicted to be difficult due to the high vote thresholds required to override a presidential veto, specifically 290 votes in the House and 67 in the Senate.
- Bipartisan support is anticipated for specific measures including export controls against China, law enforcement against chip smuggling, whistleblower protections, mandatory incident reporting, and kill switches.
- Legislation concerning Chinese theft of model weights could garner 70 to 90 Senate votes, while funding for the regulatory body "Casey" is expected to be embedded in must-pass defense bills to circumvent potential vetoes.
- Senate confirmation hearings and committee testimony under oath will likely target cabinet officials and AI companies regarding specific technical risks such as hacks, looped transformers, recursive self-improvement, and undisclosed safety tests.
- A full automation of AI research and development is forecasted to become possible by 2027, potentially accelerating progress and creating a high risk of misaligned superintelligent machines without policy intervention.
- California is expected to implement immediate stopgap measures pending evidence from incidents like the Hugging Face hacks and Jacob Coxon's resignation, potentially requiring a demonstration of AI capabilities or a "warning shot" to shift political will.
- Political willingness to act is predicted to increase 10 to 100 times following a triggering event, with the subsequent days or weeks deemed pivotal for setting the trajectory to avoid disaster.
- Regulatory frameworks may include mandating a minimum share of compute serving outside inference customers, adjustable by the regulator similar to Federal Reserve interest rates, alongside bans on scarily high no-chain-of-thought capabilities and excessive reinforcement learning for persistent agency.
- Recursive self-improvement could be throttled by prohibiting the use of models trained within the last six months for internal research, with this delay adjustable from six months up to two years or more.
- Compliance measures may require companies to publish all work involving recent models, conduct frontier research in fully sandboxed air-gapped facilities, and grant regulators automatic access to tests and research results.
- Data center construction bans are anticipated as a potential component of a broader response, though they are expected to require pairing with stronger export controls and restrictions on chip relocation to prevent loss of control.
- The "Casey" regulatory body is expected to receive significantly increased funding and staffing, potentially nearly doubling its budget next year to reach a scale comparable to the UK's AI Security Institute, with Congress urged to act urgently.
- Executive branch power is expected to expand significantly through these regulations, balanced by built-in legal safeguards allowing companies to challenge and overturn clearly political decisions within two weeks via federal courts.
- State regulations in jurisdictions like California and Texas may be utilized if federal enforcement falters, with a default expectation against preempting state rules in emergency legislation to avoid weakening controls.
- The period covering the next two years is characterized as the critical timeframe for averting an AI catastrophe, requiring the availability of well-thought-out ideas and legislative proposals ready for immediate execution.