With the exploding popularity of generative artificial intelligence, many open source models are now available online for anyone to adapt to their tasks, such as creating product renderings in a specific artistic style.
But these models also end up in the hands of nefarious actors who may optimize them to produce illegal content such as hate speech or child sexual abuse material (CSAM). This is a growing problem – the National Center for Missing and Exploited Children received more than 1.5 million reports of AI-generated CSAM in 2025, up from 67,000 in 2024.
Typically, engineers test AI for harmful capabilities by driving the model and examining its results. However, this is impossible for CSAM as it is illegal to generate such content in the US, regardless of the intent.
To avoid this dilemma and improve AI safety, Associate Professor Ashia Wilson and her graduate student Vinith Suriyakumar worked with researchers at MIT’s Healthy ML Lab led by Marzyeh Ghassemi and the nonprofit child safety organization Thorn to develop a new testing approach that determines whether a model can generate CSAM without a prompt. Thorn is a nonprofit child protection organization whose mission is to transform the way children are protected from sexual abuse and exploitation in the digital age.
Their technique examines how the inner workings of a model have been adjusted, but never generates a result. By examining hidden representations, it is possible to reliably conclude whether a model has been specialized to generate harmful images.
During testing, the testing procedure identified model variants that were specialized to generate CSAM with 100 percent accuracy. A hosting platform could use this technique to flag unsafe models and quickly remove them or prevent them from being uploaded in the first place.
“This opens up new opportunities for platforms that host open source models and for law enforcement agencies to actually test whether a model is capable of generating CSAM. Until now, we had no way to measure this. It was a huge blind spot that some people took advantage of. Now we can address an AI security problem that has serious negative implications,” says Vinith Suriyakumar, an electrical engineering and computer science (EECS) graduate student at MIT and lead author of a paper on it Technology.
Suriyakamur and Wilson, the Lister Brothers Career Development Professor in EECS, principal investigator at the Laboratory for Information and Decision Systems (LIDS) and senior author, are joined on the paper by Lena Stempfle, an MIT postdoctoral fellow. Ghassemi, associate professor of EECS and member of the Institute of Medical Engineering Sciences (IMES) and LIDS; and others at Boston University and Thorn. The paper was presented at the “Trustworthy AI for Good” workshop at the International Conference on Machine Learning Spotlight.
Exam Adjustments
Newer techniques have made it easier for users to specialize a generative AI model for their task through a process called fine-tuning.
Instead of retraining the entire model on a task-specific data set, individuals can use an algorithm called low-rank adaptation (LoRA) to specialize the model more efficiently.
This has led to a wave of new generative AI model variants for various purposes, such as creating watercolor paintings that mimic an artistic movement. But it has also enabled malicious actors to create models that can generate high-quality CSAM and other malicious images.
To test a model, engineers typically challenge it for malicious content and review its results. However, this manual review process is not scalable. Furthermore, the repeated generation of abhorrent images can have negative psychological effects on human evaluators.
This assessment method quickly fails in testing CSAM, the generation of which for any purpose is illegal in the United States and many other international jurisdictions.
“We find ourselves in this very difficult situation where we cannot resort to the de facto means of assessment due to the law itself. We had to throw away the entire toolkit and take a different approach,” says Suriyakumar.
After learning about this mystery, researchers teamed up with Thorn to tackle this problem.
A non-generative solution
Instead of focusing on the results, researchers focused on the changes a LoRA algorithm makes during fine-tuning.
Their technique examines these modifications, called LoRA adapters, to determine whether a model has been specialized for a malicious function without generating any output.
Using a technique called Gaussian probing, researchers feed the model a series of random data points and analyze how it manipulates that data within its layered internal structure.
“We never run the model to the end or prompt the model, so we never generate images,” explains Suriyakumar.
The researchers capture these changes at multiple points in time within the model’s internal structure and average them to summarize how the LoRA adapter changed the model’s computation. They perceived these responses as a strong signal of how a model became specialized.
They tested their method using variations of three types of models and compared the results with ground truth data from LoRA adapters, which are known to generate CSAM, other malicious images, and safe content.
Their method was 100 percent accurate in identifying models that were fitted to generate CSAM.
“There are a lot of child safety concerns surrounding AI, and these are real concerns that need to be addressed. A lot of children are harmed by AI deepfakes. We’ve shown that Gaussian probing can be a very useful tool, and we hope that the research community really pays more attention to this problem,” says Wilson.
Importantly, their technology is scalable and would be relatively inexpensive to implement. With thousands of model variants published online every month, scalability is critical to helping reviewers remove harmful customizations before they become widespread.
Gaussian probing is also more robust than some other testing techniques because a malicious actor would have to carefully alter the inner workings of the base model to avoid detection.
In the future, the researchers want to evaluate their technique on a larger set of model variants and investigate whether Gaussian studies can detect harmful capabilities in base models before they are adjusted.
“Now we have a technological approach to partially address this problem. So much effort has been put into this collaboration that has allowed us to tackle a really difficult problem that harms so many children nationally and globally. Hopefully we can have a transformative impact in this area,” says Ghassemi.
This work was supported in part by the Bridgewater AIA Labs Research Fellowship.
https://news.mit.edu/2026/new-method-keeps-kids-safe-from-illegal-ai-generated-content-0713
