Artificial intelligence news.
Straight from the original announcements.

OpenAI releases CSAM detection tool and new ChatGPT safety interventions

November 16, 2026, 18:00By the AI News Bites Editorial TeamSource: OpenAI

OpenAI is releasing an image classifier designed to detect potential child sexual abuse material, including AI-generated content, alongside new ChatGPT interventions for users seeking abusive material. The company says the measures are intended to help organizations identify abuse, interrupt harmful behavior and strengthen cooperation across the child-safety sector.

Key points
  • OpenAI is offering its new CSAM image classifier at no cost to vetted organizations.
  • The classifier includes a signal for potentially AI-generated CSAM.
  • OpenAI estimates the tool could reduce flagged-image review work by 48%.
  • ChatGPT will remove flagged conversations and pause access for 24 hours after certain requests.
  • OpenAI has published practical CSAM safety guidance for developers.

A classifier for potential CSAM

The new image classifier looks for visual signs of abuse and returns scores to help reviewers decide which images require attention first. OpenAI says it is the first broadly available CSAM classifier with a specific signal indicating whether material may be AI generated.

The company is offering access through the OpenAI API at no cost to vetted organizations, including child-safety groups and other industry teams of varying sizes. OpenAI developed the classifier with assistance from the National Center for Missing and Exploited Children, which operates the CyberTipline, a centralized reporting hub for the online exploitation of children.

OpenAI says it has also worked with the Tech Coalition and ROOST to share access to the tool, while supporting other systems such as Thorn’s Safer CSAM detection service. The company recently began participating in the Tech Coalition’s Lantern program, where participants share and receive signals related to online child sexual exploitation and abuse.

Early testing points to less review work

In an evaluation of user-uploaded images, OpenAI compared the classifier with its current protections. The company estimated that the new system could flag 48% fewer images while finding roughly the same number of abusive images as the existing tool. Images flagged by the new classifier were about twice as likely to contain abuse, according to OpenAI.

Early partners reported additional results. In fal’s testing on human-reviewed production images, reviewers confirmed 72% of high-confidence flags as novel CSAM. invideo AI reported that one in four flagged cases led to a ban. SafetyKit’s evaluations involving drawings or animations that sexualize children found that the classifier detected 90% of cases.

OpenAI says its teams tested the system internally and with early partners to assess detection, false positives and how the tool could support safety teams. The company plans to continue improving it using feedback from organizations that adopt the classifier.

ChatGPT warnings will interrupt harmful requests

OpenAI says ChatGPT already prohibits CSAM, trains its models to refuse related requests and takes action against accounts that violate its policies. The new product warnings add an intervention for attempts to find abusive material.

The warning explains that sexualizing children causes harm, including when the requested material is AI generated. OpenAI says it will remove the flagged conversation and pause access to ChatGPT for 24 hours, while keeping specialist support available to help people stop harmful behavior. Repeated violations may result in longer restrictions, account closure or reports to authorities.

The approach was developed with input from Protect Children, the Lucy Faithfull Foundation, Dr. Caoilte Ó Ciardha of the University of Kent and Joel Scanlan of the University of Tasmania. Scanlan independently evaluated a trial of the deterrent messages.

Guidance for developers and safety teams

OpenAI has also published CSAM guidance for developers. It covers ways to prevent abuse, detect harmful content, respond to user reports, review suspected abuse and report it. The guidance also explains how safety identifiers can connect misuse to individual users and how organizations can support people who review sensitive material.

Developers can use the guidance with OpenAI’s image classifier and Moderation API, as well as ROOST’s self-hosted review console Coop and Thorn’s Safer service. OpenAI says the guidance was developed with input from the Tech Coalition, Thorn, the National Center for Missing and Exploited Children and the Internet Watch Foundation.

The company is also working with Anthropic and Google to commission guidance on responding to disclosures of child sexual exploitation and abuse in AI systems. Published by the Tech Coalition, the guidance offers recommendations for developers handling these situations. OpenAI says it has already contributed to increased developer access to child-safety tools and registration to report CSAM to NCMEC.

Related news

Check Point Says AI Agents Need Steering Before the Kill Switch
Safety and ethicsNovember 19, 2026, 14:00

Check Point Says AI Agents Need Steering Before the Kill Switch

Check Point is positioning agentic steering as the first response to risky AI agent behavior, while reserving automatic shutdown for agents that have been hijacked. The…

Princeton study finds ChatGPT’s election answers vary by user
ResearchNovember 18, 2026, 17:09

Princeton study finds ChatGPT’s election answers vary by user

ChatGPT’s memory settings can change the partisan slant of its election-related answers, according to a preliminary Princeton field audit of 160 US users. The study…

Australia targets frontier AI with six mandatory safety guardrails
BusinessNovember 11, 2026, 04:24

Australia targets frontier AI with six mandatory safety guardrails

Australia will introduce six mandatory safety guardrails for frontier artificial intelligence companies and establish a regulator with powers to impose significant…