Item — HN Reader

Indeed ^_^, but what I meant to ask is why did the possibility of AI crawlers fetching your site trigger such a big response? Especially when there's services/tools like Cloudflare and Anubis that solve that.

Some people are worried about the volume, but at worst your site is down for a while. Unless you're on serverless, but in that case you would've had to be worried about hackers before AI crawlers existed.

If it's IP theft, if someone's motivated enough to get past Cloudflare/Anubis, you would've had to worry about them before AI crawlers existed too.

1 comments

I very much don't want my stuff to be used in training genAI models, and if even a single crawler finds its way to being able to get my data then it's game over because there's no way to get it excluded after the fact.

> Especially when there's services/tools like Cloudflare and Anubis that solve that.

They don't solve it. They offer some help, but it's far from good enough.

The exact reason why I don't want my stuff included isn't really relevant to the topic, honestly ("I don't like it" is plenty sufficient reason, after all), but I will say there are multiple reasons. The main issue is that if it happens, there's no way to correct for that.

IP violation is one of them, but not the main one -- but it does illustrate the point: in the case of more traditional IP "theft", there is technically a way to resolve the problem after it happens. Not so with genAI crawlers.

I see, would love to ask more questions but I get the impression you'd like to stop here. Thanks for the conversation!

I try not to get into the reasons I don't want my stuff to be ingested by genAI if that's not the topic, because it inevitably results in the conversation derailing as people argue with my reasons. But they aren't a big secret.

I don't want my stuff to get used to train genAI because I think that genAI is, on the whole, a negative for society. Not because of anything inherent in the technology itself, but because of the way genAI companies are employing and marketing it. On top of that, I resent that genAI companies have abused my trust by slurping everything up to train the models without asking or even so much as a "how do you do" (this is a different than an IP law argument).

I also think that genAI companies are increasingly abusive and manipulative and don't want to support them in any way. And the fact that they often argue about how dangerous the technology is and the great risk to humanity it presents, but at the same time are eager to develop and sell it, indicates an extreme level of ethical bankruptcy.

I think it's reasonable. I also find it frustrating that you don't get credit for the content you create. It's stopped me from writing more about virtual offices, for example.

And if you really want to prevent any chance of that happening, the only way is to take it down unfortunately.