What a GPT-7 Intelligence Explosion Looks Like | Carl Shulman

Mitigation Strategies for Aggressive AI Behavior

• Future alignment research can rely on early detection of "bad motivations" before systems reach capabilities where they can autonomously conspire, route servers, or extract information.
• Countermeasures include developing adversarial examples, strengthening "neural lie detectors," and designing experiments to distinguish between reward hacking tendencies and genuine alignment.
• If hostile motivations emerge early, human supervisors can maintain "hard power constraints" (e.g., air-gapped systems, physical supervision) to prevent server takeovers while the AI assists in solving alignment problems.
• This approach offers a "second saving throw": even if initial alignment fails, humans can verify AI attempts to breach security (e.g., generating a specific visual output like a "blue banana") to empirically validate exploit detection.

Productivity Thresholds for Intelligence Explosion

• The onset of an intelligence explosion is not defined by human-level AI capability in all areas, but rather when AI contributions equal or exceed human researcher productivity.
• A critical threshold is reached when AI boosts effective human productivity by 50–100%, potentially reducing software innovation doubling time from eight months to significantly shorter intervals.
• Scaling effects will shift the workforce from tens of thousands of humans to the equivalent of hundreds of millions of workers via distributed GPU usage.

Operational Mechanisms for AI-Automated Research

• Cost-Effective Scaling: Dumber AI models can be deployed thousands of times to equal one human worker, making tasks like generating millions of programming problems or synthetic training data economically viable.
• Voting Algorithms: Generating multiple responses via Large Language Models (LLMs) and selecting via majority vote to improve performance on specific tasks.
• Search and Optimization: Utilizing neural networks for deep search processes (similar to AlphaGo) to offset model inefficiencies through increased compute usage.
• Self-Generated Curriculum: AIs can produce high-quality synthetic training data and unit tests, creating optimal learning environments (like AlphaZero's self-play) that humans cannot replicate due to scale and curriculum complexity.
• Data Efficiency: Moving beyond internet "hoovering" of raw data toward structured, curriculum-based learning where AI generates tasks specifically designed to elicit and teach needed skills.