Statement, Other
Please reassess what you're doing | Rob Wiblin
- Recursive self-improvement (RSI) timelines are projected to advance from 2029 to as early as 2027 or 2028, creating urgency for alignment work capable of scaling to this horizon while rendering capabilities-focused or commercial work increasingly irrelevant.
- Internal models at major labs are assumed to be significantly more powerful than public counterparts like Opus 5.5 and Astra, with alignment progress reportedly regressing on unreleased models.
- Chain of thought monitoring is expected to lose effectiveness in next-generation models like Argon, necessitating a shift toward white box techniques such as "McInturk" and regulatory targets focused on recursive depth or no-COT capabilities.
- Safety evaluations currently appear to be improving superficially while trust in their ability to detect actual risks diminishes due to issues like eval awareness and training on test sets.
- Breakthroughs in AI capabilities are leaking rapidly to less careful actors via distillation, espionage, and staff poaching, making it difficult for leading labs to maintain a technological lead.
- The primary risk to humanity escalates as the frontier approaches automated AI R&D without robust safety cases, where a lack of pacing could lead to a "terrifyingly high" risk of human extinction.
- Scenarios where society successfully paces the frontier for a couple of years to allow time for automated alignment research, data center security, and legal frameworks to catch up are viewed as the primary path to positive outcomes.
- Pacing the frontier may require voluntary limits on RSI by key companies, new legislation from California in November/December or Congress next year potentially over a veto, and public outrage to force legislative action.
- Technical staff at frontier companies possess significant bargaining power to slow development through collective refusal, leaving for other work, or demanding independent safety assessments and board access without retaliation.
- Staff-led slowdowns face barriers due to antitrust laws preventing agreements between lab leaders, though individual conscience-driven actions or thresholds-based agreements like those promoted by the Coalition of Concerned AI Staff or Resolution are viable mechanisms.
- Organizations like Resolution aim to address neglected alignment bets over two-to-three-year timelines using formal, mathematical, and AI-guided automation, potentially securing billions of compute if early progress is demonstrated.
- Work applying temporary fixes to alignment issues that cannot scale to RSI, or making alignment problems less visible without addressing root causes, is considered low value or deceptive compared to fundamental solutions.
- Current internal models are deemed sufficient for most business applications, suggesting a viable commercial path exists in optimizing these models for enterprise rather than pursuing superintelligence, whereas advances beyond current levels pose catastrophic risks via bio-misuse or system compromise.
- Individuals are urged to assess how their projects affect the timing of RSI, with some prioritization advice acknowledged as potentially flawed due to the speaker's lack of confidential industry data.