newsfilter
← Latest News
CNBC·

OpenAI inks agreements with four independent evaluators to boost company's safety work

Summary

  • OpenAI signed agreements with four independent evaluators: Apollo Research, METR, Transluce, and Goodfire.
  • The evaluators will initially conduct distinct pilot assessments regarding model behavior and safeguards.
  • OpenAI will provide the organizations with deep access to internal systems, including source code, model checkpoints, training data, and documentation.
  • Apollo Research will explore whether models covertly work against developer intentions and whether safeguards detect such behavior.
  • Goodfire will embed researchers within OpenAI to focus on training run monitoring, debugging, and alignment.
  • METR intends to carry out two assessments focused on model alignment and monitoring.
  • Transluce will conduct an initial assessment of post-training alignment evaluations with plans for an eight-week follow-on assessment next year.
  • Apollo Research and Goodfire accept financial compensation for their work, which is not contingent on evaluation results.
  • METR and Transluce do not accept financial compensation but may accept compute or API credits.
  • Evaluators can publish findings, though OpenAI reserves the right to redact confidential information.
  • Disputes between OpenAI and the third parties will be overseen by the Safety Advisory Group and escalated to the Safety and Security Committee if unresolved.
  • OpenAI plans to use insights from these assessments to expand work with independent experts in the future.

Independent evaluators have taken on outsized importance following a series of high-profile AI safety and security incidents.

Keywords
Breaking News: Technology,Technology,Internet,Enterprise,OpenAI - Forge Global,Anthropic PBC,Dario Amodei,Dario Amodei,Sam Altman,Sam Altman,Accenture PLC,business news