alt Hacker NewsAI doesn't know what either porn or children are. It finds correlations between aspects of inputs and the labels porn and children. Even if you did develop an advanced enough AI that could develop a good enough idea of what porn and children are, how would you ever verify that it is indeed capable of recognizing child porn without plugging in samples for it to flag?
Replies
So it is able to correlate an image as porn and also correlate an image as containing children. Seems like it should be able to apply an AND operation to this result and identify new images that are not part of the data set.
AI doesn’t understand context either — it can’t tell the difference between an innocent photo of a baby in a bathtub with a parent, a telehealth photo, or something malicious. Google is using AI in addition to hashing, and both systems can get it wrong. With AI you’re always dealing with confidence levels, not certainty. No model in history has ever had 100% confidence on anything.
A scanning system will never be perfect. But there is a better approach: what the FTC now requires Pornhub to do. Before an image is uploaded, the platform scans it; if it’s flagged as CSAM, it simply never enters the system. Platforms can set a low confidence threshold and block the upload entirely. If that creates too many false positives, you add an appeals process.
The key difference here is that upload-scanning stops distribution before it starts.
What Google is doing is scanning private cloud storage after upload and then destroying accounts when their AI misfires. That doesn’t prevent distribution — it just creates collateral damage.
It also floods NCMEC with automated false reports. Millions of photos get flagged, but only a tiny fraction lead to actual prosecutions. The system as it exists today isn’t working for platforms, law enforcement, or innocent users caught in the blast radius.
LLMs don't "know" anything. But as you say, they can identify correlations between content "porn" and a target image; between content labeled "children" and a target image. If a target image scores high in both, then it can flag child porn, all without being trained on CSAM.