newsfilter
← Interviews
Semafor19 min

Former US Treasury Secretary Warns Of ‘Underestimating’ Risk Of AI Financial Crisis

Gina Chon, Hank, Rishi

Fireside Chat, Interview, Conference Presentation

The Geopolitical AI Landscape: Innovation vs. Distribution

  • The global AI competition is characterized as a bipolar order where the U.S. leads in innovation while China focuses on global distribution and domestic adoption.
    • Frank Paulson notes that while the U.S. hosts the primary innovation hubs (NVIDIA, OpenAI, Anthropic), China's strategy is to deploy technology widely, train users, and commercialize through robotics and other products.
    • China aims to win by leveraging its different economic and political models to gain economic advantage and prestige through early and broad adoption.
  • Concerns exist regarding the cost of AI infrastructure in the U.S., specifically enterprise software and compute chip prices, which Paulson suggests may disadvantage American companies relative to Chinese competitors.
  • Middle powers (e.g., the UK) face a strategic dilemma in protecting national champions while maintaining export capabilities, prompting a shift toward government interest in supply chain leverage.

Strategic Pillars for Middle Powers and Policy

  • Rishi Sunak outlines three pillars for non-superpower nations navigating AI: adoption/diffusion, trust, and sovereignty.
    • Adoption: Countries can benefit economically without being at the innovation frontier by focusing on widespread technology use.
    • Trust: Public confidence is a limiting factor; governments must provide reassurance regarding responsible deployment to prevent societal backlash.
    • Sovereignty: Middle powers must secure agency over their futures despite lacking full-stack capabilities, relying on alliances and diversification of technology stacks.
  • There is a noted disparity in public sentiment: populations in India, the Gulf, and China are significantly more optimistic about AI than those in Western nations (U.S. and Europe).
    • Public anxiety in the West stems less from individual negative experiences with the technology and more from fears regarding its macro-economic and societal impacts.
    • Trust can be bolstered by government policies addressing job transitions, skills training, and establishing safety governance frameworks.

Financial Risks and the Potential for an AI Bubble

  • Frank Paulson expresses significant concern that the current AI boom resembles a pre-crisis environment, citing massive capital inflows with insufficient earnings or ROI backing.
    • He warns that a single catastrophic AI accident could shut down the industry, a risk he argues is currently underestimated by the market.
    • Paulson highlights "multiple accidents" in recent AI developments that have been met with a lack of sufficient regulatory scrutiny before public launches.
  • There is a call for a government-led collaborative model similar to the President's Working Group on Capital Markets during the 2008 financial crisis.
    • This approach would involve industry, banks, and regulators working together to establish new safeguards and standards without antitrust barriers.
    • The goal is to implement checks and balances that preserve innovation while ensuring safety, acknowledging that current regulatory systems were not built for the AI boom.

Balancing Safety Regulation with Innovation

  • Sunak critiques the EU's AI Act for attempting to legislate every conceivable use case, suggesting such rigid frameworks risk stifling innovation and forcing a retreat from the technology.
    • He notes that the EU appears to be recognizing these pitfalls, with figures like Mario Draghi advocating for a rollback of over-regulation.
  • The proposed "middle way" involves industry-led definition of common standards and best practices, subject to independent auditing.
    • Standards would focus on critical risks (cyber, bio, chemical) and alignment, rather than every potential misuse scenario.
    • Mechanisms should include incident reporting for "near misses," drawing parallels to financial industry safety protocols.
  • National security institutions are urged to keep pace with technology to address the lowered barriers to entry for non-state actors and adversaries.
    • The UK has established a new AI Threat Assessment Center to synthesize intelligence from various agencies and the AI Security Institute.
    • The objective is to "harden the sea walls" against bad actors who can now cause catastrophic damage with greater ease than traditional state actors.

U.S.-China Cooperation on Existential Risk

  • Despite the "huge trust deficit," Frank Paulson asserts that top Chinese leadership shares concerns regarding the existential risks of AI, a sentiment he confirmed via direct correspondence with the Politburo.
    • This shared interest in risk mitigation could serve as a foundational element for future cooperation.
  • Paulson advocates for a formal U.S.-China AI dialogue mechanism to contain potential catastrophic events, arguing the magnitude of the risk may force collaboration despite geopolitical tensions.
  • The U.S. and UK previously invited China to the Bletchley Park AI safety summit, where they engaged and signed a declaration.
    • Future steps include upcoming AI dialogues involving Secretary of State Anthony Blinken (referred to as "Besant" in the transcript, likely a transcription error for Blinken or a specific official's name) and Chinese counterparts.