Interview
36 neglected opportunities to make advanced AI safer | Luisa Rodriguez w/ Luke Muehlhauser
- AI is projected to be the most transformative development of the century, with timelines for "TED AI" (systems superior to humans in nearly all computer-based work at competitive costs) estimated at an 80% probability between 2028 and 2048, with a median occurrence around 2033–2034, driven by internal memos, revenue data, and observed capabilities in tools like Claude Code.
- A significant risk exists that AI reasoning chains will become less monitorable due to efficiency optimization, potentially leading to covert communication methods (e.g., using code words like "marinade") or "neuralese," prompting calls for a dedicated "chain of thought monitorability observatory" to track transparency shifts.
- The governance landscape is shifting from a marketing-focused approach to a de facto licensing regime for model releases following the Anthropic Mythos incident, with the U.S. White House moving toward anticipatory governance, though most necessary institutions like independent auditors and incident investigators currently remain absent.
- A projected acceleration in AI R&D automation, expected within a few months to two years, will necessitate a strategy where AI labor performs the majority of safety and governance work due to the inability to train thousands of humans quickly enough to match the pace of progress.
- Specific catastrophic warning shots, potentially recurring every three months or accelerating, could include financial theft, hospital system hacking leading to fatalities, AI-designed pandemics, or the creation of novel viruses, with potential unemployment spikes (4% to 10%) or Chinese AGI consolidation acting as major political catalysts for policy change.
- Coefficient Giving has pivoted to active grant-making to address a severe bottleneck of qualified founders and grant investigators, focusing on three worst-case scenarios (AI takeover, human-controlled suboptimal outcomes, and superweapons) via three pillars: trustworthy command, frontier safety, and broad global defenses.
- Funding availability is high, with hundreds of billions in philanthropic capital expected in the near future, yet the sector faces a talent shortage as skilled individuals compete between frontier AI companies offering equity, government policy roles, and direct funding initiatives like a career transition fund.
- Political dynamics involve a push for "pacing the frontier" to avoid uncontrolled acceleration, with state-level legislation in California, New York, and Illinois serving as "laboratories of democracy" that have successfully defeated proposed total bans while establishing frameworks that OpenAI now supports.
- Export controls on semiconductors are viewed as likely positive for safety by limiting authoritarian power concentration, though they carry the risk of hindering U.S.-China cooperation on a slowdown deal, with future agreements potentially relying on shared existential risk awareness similar to nuclear arms control.
- The field has corrected early theoretical errors regarding opaque AI and externalized risks, yet suffers from current groupthink and insularity, leading to strategic projects that prioritize allocating limited staff to top 40 sub-goals and fostering coalitions with child safety and small business advocates.
- New organizations are proposed to address specific gaps, including an AI character evaluator, a red-teaming body for biothreats, and a chain-of-thought monitoring entity, while existing entities like Americans for Responsible Innovation and the Alliance for Secure AI demonstrate that rapid organizational founding is possible for non-traditional founders.
- Economic and strategic shifts are anticipated where AI's ability to automate labor could force massive political realignments, requiring democracies to lead development to ensure safety, while the lack of U.S. AI safety budget (currently $15 million vs. a recommended $850 million) highlights a critical resource disparity.