AI-generated child sex abuse images, videos rising at alarming levels, warn internet watchdogs
The Internet Watch Foundation warns that the use of artificial intelligence to create child sexual abuse material is increasing at an alarming rate, with synthetic images and videos becoming significantly more sophisticated and photorealistic. The organization says the problem has reached a potential “tipping point” as distinguishing AI generated material from images involving real children becomes increasingly difficult.
A major change is where this material is appearing. AI generated CSAM was previously associated more heavily with hidden areas of the internet such as the Dark Web, but the IWF says it is now increasingly being distributed through the open web and publicly accessible online platforms. This increases both its visibility and the potential scale of distribution.
The figures cited by the IWF illustrate the acceleration. During a six month period, the organization acted on 74 reports involving AI generated child sexual abuse imagery, already exceeding the 70 reports recorded during the entire previous 12 months.
The growing realism of synthetic content creates an additional challenge for investigators. Analysts may have difficulty determining whether an image depicts an identifiable child who is currently at risk or is entirely AI generated. This can divert investigative resources and complicate efforts to prioritize genuine victims requiring immediate intervention.
The technology also remains connected to real world abuse. According to IWF interim chief executive Derek Ray Hill, AI systems producing this material can be trained using images of real child abuse victims. Synthetic CSAM therefore does not necessarily exist independently from actual exploitation, even when the final generated image depicts a fictional or altered individual.
The material identified by investigators ranges from highly realistic deepfake videos to manipulated sexual images created from photographs of clothed children. More than half of the flagged content was hosted on servers located in Russia and the United States, with additional material hosted in Japan and the Netherlands.
The report also connects the issue with the broader rise of sextortion and online sexual exploitation. Young users can be manipulated into sharing intimate images and subsequently blackmailed. Meta has responded with measures for Instagram including automatically blurring suspected nude images in direct messages and warning users before they view them. These protections are enabled by default for teenage accounts.
Overall, the IWF’s warning highlights how generative AI is changing the scale and complexity of online child sexual exploitation. Greater realism, easier creation, distribution on the open web and the use of real victim material to generate synthetic content are making detection, victim identification and removal increasingly difficult for authorities and online safety organizations.
