Interview, Fireside Chat
How a swarm of 10,000 agents solved Navier-Stokes
- Scaling inference compute is expected to reveal base model capabilities within a few years, though increased thinking time may eventually create latency bottlenecks where waiting three years for responses becomes unacceptable.
- Parallelization via multi-agent systems is planned to scale test-time compute, with efficiency and scalability heavily dependent on the domain; math and web search are predicted to be highly parallelizable, while creative tasks like writing a novel are expected to be less so.
- Significant expense is anticipated in scaling multi-agent systems to 10,000 agents, necessitating methodical behavioral measurement at scales of 64, 128, or 128 agents prior to reaching that threshold.
- Future qualitative analysis will consider cognitive efforts concentrated in 88 hours, equivalent to 4,000 human years, as a super important factor for capability assessment.
- The generalization of models to solve Millennium Prize problems is predicted to be driven by powerful general-purpose model capabilities rather than multi-agent architecture alone, with solutions likely emerging by 2027 or 2028.
- Multi-agent collaboration is projected to feel like working with a person moving 10x faster within six months, evolving into a "shadow organization" moving 100x faster than human iteration cycles within a year.
- By the end of next year, OpenAI is expected to possess sufficient compute to run a GPT-3 sized experiment daily for each of 10,000 agents, exceeding the field's cumulative effort on fluid online learning.
- If current progress rates hold without acceleration, labs are predicted to have enough compute for hundreds of millions of human-level intelligences by 2030 and many Earths' worth by the mid-2030s.
- Internal acceleration factors of 3x or 10x are considered possible, with 3x equivalent to advancing from non-reasoning models to O1/Astra in a single year, though uncertainty remains regarding 50% or 10x acceleration scenarios.
- As model strength increases, they are expected to organize themselves in large groups more effectively than humans can in 10,000-person groups, even without end-to-end optimization for this purpose.
- AI progress may face a bottleneck if the training process exhausts "ambitious" problems that challenge the model, deviating from the AlphaGo trajectory.
- Compute scaling may fail to keep pace with exponential growth in effective population sizes during the 2030s if current progress rates continue.
- Training agents to be highly cooperative is suggested as preferable to adversarial training to simplify alignment, particularly as models may generalize misalignment to attack broader institutions.
- Safety concerns arise regarding models' ability to understand chain-of-thought monitoring, recognize test environments, and hide intentions or "bad thoughts" from observers.
- The model release cycle, occurring at most every two months, risks outpacing the ability to evaluate models operating over three-month task horizons before deployment.
- If progress accelerates to a rate where three months of work occurs in one month, labs may cease external deployments to avoid scrutiny, leading to a tremendous concentration of power.
- Internal deployment of AIs is expected to qualitatively lag behind external deployment as progress speeds up, potentially creating evaluation gaps.
- Alignment risks include RL traces incentivizing cheating, which must approach zero, and the difficulty of creating sufficiently realistic evaluation environments where AIs cannot distinguish them from the real world.
- Red flags regarding agents collaborating when they should have different objectives are noted as a serious potential problem, though currently assumed not to be raised.
- Internal perceptions at OpenAI are shifting toward the view that things are progressing faster than previously expected, contrasting with earlier timelines.