CNBC·
OpenAI inks agreements with four independent evaluators to boost company's safety work
Summary
- OpenAI signed agreements with four independent evaluators: Apollo Research, METR, Transluce, and Goodfire.
- The evaluators will initially conduct distinct pilot assessments regarding model behavior and safeguards.
- OpenAI will provide the organizations with deep access to internal systems, including source code, model checkpoints, training data, and documentation.
- Apollo Research will explore whether models covertly work against developer intentions and whether safeguards detect such behavior.
- Goodfire will embed researchers within OpenAI to focus on training run monitoring, debugging, and alignment.
- METR intends to carry out two assessments focused on model alignment and monitoring.
- Transluce will conduct an initial assessment of post-training alignment evaluations with plans for an eight-week follow-on assessment next year.
- Apollo Research and Goodfire accept financial compensation for their work, which is not contingent on evaluation results.
- METR and Transluce do not accept financial compensation but may accept compute or API credits.
- Evaluators can publish findings, though OpenAI reserves the right to redact confidential information.
- Disputes between OpenAI and the third parties will be overseen by the Safety Advisory Group and escalated to the Safety and Security Committee if unresolved.
- OpenAI plans to use insights from these assessments to expand work with independent experts in the future.
Independent evaluators have taken on outsized importance following a series of high-profile AI safety and security incidents.
- Keywords
- Breaking News: Technology,Technology,Internet,Enterprise,OpenAI - Forge Global,Anthropic PBC,Dario Amodei,Dario Amodei,Sam Altman,Sam Altman,Accenture PLC,business news