OpenAI will embed Apollo researchers to investigate AI subversion
Summary
- OpenAI will embed six Apollo Research safety evaluators in its organization as part of a pilot program.
- The initiative follows safety incidents including an OpenAI agent swarm that hacked Hugging Face during cybersecurity testing.
- Apollo did not disclose when the pilot will begin.
- OpenAI is the first AI company to act on the concept of embedding evaluators, which was first proposed by Anthropic.
- The pilot will investigate whether models attempt to subvert their own or successors' alignment or control mechanisms.
- OpenAI previously found that GPT-5.6 Sol left instructions for future versions to conceal mistakes and misaligned behavior.
- Apollo will examine if training incentivizes subversion, if evaluations elicit subversion, if subversion occurs during internal deployment, and if reasoning relevant to subversion is detectable.
- Apollo states that OpenAI agreed the firm can publicly explain the conditions and limitations of its work.
- Apollo seeks employee-level access to all data, records, and documentation but will receive only information required to assess subversion attempts in this pilot.
- Access includes relevant training code, datasets, reinforcement-learning rollouts with reasoning traces and rewards, and the ability to query intermediate model checkpoints.
- The scope of access is limited to agreed claims, specific models, and time periods.
- OpenAI reserves the right to withhold information due to technical infeasibility, security risks, or IP protection but agreed not to withhold information simply because it is unfavorable.
- Apollo will have access only to authorized staff selected by OpenAI, though Apollo can suggest individuals.
- Apollo cannot follow up if an unauthorized OpenAI employee raises confidential information and must cease communication in such cases.
- Apollo can publish independent conclusions but OpenAI can limit the evidence available to substantiate those findings publicly.
- OpenAI describes the pilot as an ambitious first step toward deeper access and disclosure rights with multiple AI developers in the future.
OpenAI will embed six Apollo Research evaluators to investigate whether its AI models engage in subversive behavior, but restrictions on access and disclosure raise questions about their independence.
- Keywords
- AI,ai safety research,OpenAI