A lot of perl cluthiching in this article. Just remove the csam identified and reupload the dataset
back
5 comments
I think, over a dataset of nearly 6 billion images, it's just impossible to ensure that it contains no child pornography.
Like trying to ensure all my kitchen surfaces are 100% free of germs. Not possible, or the amount of effort and disruption caused by the cleaning will make it impossible to get on with my life. Nasty stuff exists out there. We don't want too much of it, though. That's my opinion about it.
You have to ask yourself not “how impossible” it is , rather how much effort was put into mitigating this. The answer is probably a nominal amount but obviously not enough
I'll bet OpenAI and Google are also contaminated, but that we'll never know and it'll never be cleaned.
With open source efforts, we can eventually reach a state of public certification. It can be fully vetted, unlike closed source models.
The paper says that almost 500 of these were identified by MD5. That indicates that about 0 effort was put into mitigation.
How does that logic work? Unless your definition of "mitigation" is "modifying CSAM before including it in the dataset"?
Like, "identified by MD5" doesn't mean that they didn't attempt to remove images! A lot of CSAM out there is not yet on those lists, and a lot of it is being added as we speak, so if LAION-5B was filtered using one list of MD5 hashes, that doesn't mean it doesn't contain CSAM that's on a different list (or a newer version thereof).
And you have this problem no matter whether LAION actually took any steps to remove CSAM before publishing the dataset or not. Unfortunately, any sufficiently large such dataset will contain some bad content (how we should deal with this is a different question that I sadly don't have an answer for).
If it's all new, I guess that's one thing. But in general, if LAION wanted to know if this content was in the collection, they could have queried the same database. Some new heuristics turned up others, but I mean the basic search could have been done.
So why all the worry about re-uploading a clean dataset because someone "could do a diff" to find the bad images? Could they not use the same md5 search to find the same?
You're not allowed to have the MD5 hashes these researchers used to find these images, so it's not possible to reproduce this result. These hash databases and APIs are tightly controlled, and only law enforcement, major tech companies and top researchers are allowed to use them.
i guess the argument is that these images were a needle in a haystack and now you have given the public a supermagnet to go find it.
like right now, if someone tries to download old laion and find those images manually, it would probably take them a very very long time. if they had the diff, they would find them in a few minutes.
i dont think its a good argument, but its an argument. the researchers found them without a diff so likely someone else can too. so theres not much reason to leave them in that i can tell.
Yes, I get that. But that's not what the researchers did. They used a database of know hashes. Can bad actors not do the same?
I don't think it's easy to get access to the PhotoDNA database. And it would be a service that people query against, so they'd have an audit trail.
md5 hashes are not a reliable indication of anything, since producing images that have collisions can be done in realtime.
https://www.exploit-db.com/docs/english/46047-md5-collision-...
That's if you have control of both images. With a given MD5 hash, you can't make another file with that hash.
Still not an argument for ever using them as reliable evidence. You have to trust whoever made the original hash to not have generated it with a duplicate, innocuous image providing a collision. These hashes are not verifiable as they appear to be parts of secret databases, so you are trusting the authority of that database not to do something like make innocuous colliding images for the sake of violating your personal freedoms.
As the article points out, the problem is that by doing so you create a map for people to find CSAM just by diffing the datasets.
Good. Federal agencies can deal with that. Go shut down those websites and prosecute the maintainers. Seems like a freebie.
Get the dataset minus the illegal data back. It's essential to maintain parity with closed source efforts.
I'd bet that OpenAI also has this in their dataset too. And nobody will ever even know. Open source is the only AI that can be knowingly scrubbed and cleaned.
These "illegal content" moral panics have ruined Internet freedom, they are now ruining AI as well. Such moral panics have been used as an excuse for absolutely heinous mass surveillance of the entire population.
The surveillance is for possible future crimes and is more debatable. We're talking about crimes we know have already happened. What would the argument even be for not addressing those.
Ruining AI? You don’t think what’s ruining AI is ingesting despicable content in it and then use this data set to train thousand of other tools with this despicable content? In what world don’t you think it’s not a problem?
It is a problem, just as crime in general in society. But our efforts to stamp it out completely could cause more harm than good. The price of liberty, is unfortunately crime. Otherwise if we go too far in preventing crime, we will be living in a totalitarian state.
if crime is done to me personally, it takes away my liberty. individuals have rights, which is the foundational concept of the Enlightenment and all modern society.
there is no panic here. every reasonable person agrees this is a problem, and it will take very little effort to fix it. these AI companies are not hurting for money. every single one of them is valued in the billions of dollars. they can spare "0.5 ppm" of their funding to go do what almost everyone agrees needs to be done.
I don't think that there actually is a low effort fix. Any sort of hashing or filtering strategy is inevitably going to miss some things. It's likely not possible to have humans manually review every image and even if you could, then you'd have the problem of traumatizing the people doing the review.
I'm not saying that more shouldn't be done, but that doesn't mean that the solution is easy or that a perfect solution even exists.
I think this is the correct response. Have federal agencies remove the content itself, instead of worrying about identifiable dead links.
The primary obstacle to this is that a relatively easily identifiable diff would then be accessible for anyone motivated to identify the material.
How was the data collected in the first place? Why was the data considered useful for training if no one had evaluated it?
The point of various unsupervised and 'self-supervised' training methods is the ability to use unlabeled data, which enables you to use much more data than it's feasible for people to evaluate. Some crude automated filtering (e.g. deduplication) is still helpful, but removing the bottleneck of having a person look at every single item can enable you to use thousand or million times larger datasets, which is what's required to give models a 'general understanding' of a wide domain, by taking an (automated) look at literally everything that is available.
It's useful because it includes most of the web. In enormous datasets like this, introspection and filtering is done with statistical/ML techniques. This was done with both LAION (obviously not enough to filter CSAM out) and the models trained on it - statistical analysis, 3rd party model classification and captioning, aesthetic scores, perceptual hashes etc. Only the filtered part of the dataset makes it into the actual training set, as LAION has notoriously poor quality.
Likely what will happen but because it's a strict liability crime it will warrant as complete a review of all data before re uploading to ensure there's nothing left
That's just completely ridiculous, I can't believe what the world's come to nowadays over such moral panics.
There's likely images of animal and human torture and mutilation in there. And there are people that fantasize about doing such things all the time. Nothing done about that, though.