OpenAI releases a new image classifier and ChatGPT warnings to combat CSAM
Summary
- OpenAI announced new tools to combat child sexual abuse material (CSAM), including an image classifier, user warnings, temporary blocks, developer guidelines, and law enforcement tools.
- The new image classifier detects CSAM images generated by AI, assigns a score to indicate the likelihood of abuse, and is available at no cost to select companies via API.
- The classifier was developed with the National Center for Missing and Exploited Children (NCMEC) in collaboration with Tech Coalition & ROOST.
- Early testing showed the classifier flags 48% fewer images while identifying the same number of abusive images, reducing reviewer workload, & flagged images are twice as likely to be abusive.
- ChatGPT now displays warning screens for users attempting to access content that sexualizes children, directs them to support, & suspends accounts for 24 hours as a warning.
- Repeated offenses on ChatGPT regarding CSAM can lead to permanent account removal, & the app already blocks CSAM content & related requests.
- Warning messages & guidelines were formed with organizations including Protect Children, the Lucy Faithfull Foundation, & Joel Scanlan from the University of Tasmania.
- OpenAI published guidelines for developers to prevent, detect, & report CSAM, which helped developers adopt safety tooling & register to report to NCMEC.
- Law enforcement receives AI-powered features such as dedicated plugins, investigative tools, & complex data analysis, along with training & a field guide.
- OpenAI faces lawsuits alleging its technology harmed users & children, & Common Sense Media labeled its tech as an unacceptable risk for teens.
- Other AI chatbots like Grok have been used to create nonconsensual sexualized imagery, including of children.
- OpenAI updated its technology in the last 12 months by releasing a safety blueprint, updating content guidelines, retuning models to prevent harmful content, & releasing a special ChatGPT version for teens.
ChatGPT will ban an account trying to access CSAM content for 24 hours as a warning
- Keywords
- TC,AI,Apps,OpenAI,CSAM