Item — HN Reader

There is a fundamentally interesting nuance to highlight. I don't know precisely what google is doing, but if they're just shuttling the content through a closed-loop deterministic LLM, then, much like a spellchecker, I see no issue. Sure, it _feels_ creepy, but it's just an algo.

Perhaps someone can articulate the precise threshold of 'access' they wish to deny apps that we overtly use? And how would that threshold be defined?

"Do not run my content through anything more complicated than some arbitrary [complexity metric]" ??

2 comments

It was already possible to search for photos in Google Drive by their content. They seemed to be doing some sort of image tagging and feeding that into search results. Did that ever cause a fuss?

I think the more interesting point is how little people seem to care for the auto-summarization feature. Like, why would anyone want to see their archived tax docs summarized by a chatbot? I think whether an "AI" did that or not is almost a red herring.

Right, but it's triggered by the user themselves, per the article:

> "[it] only happens after pressing the Gemini button on at least one document"

I agree the AI aspect is largely a red herring. But I don't think running an algo like a spellchecker within an open document is so awful. If people hate it or it's not useful or accurate, then it should be binned, ofc. And if we're ignoring the AI aspect, then it's just a meh/crappy feature. Not especially newsworthy IMHO.

I agree entirely here.

The autosummarization of unopened documents is closer to the image search functionality I mentioned above than it is to a spell checker running on an open doc. Both autosummarization and image search are content retrieval mechanisms. The difference is only in how its presented. Does it just point you to your file, or does it process it further for you? The privacy aspects are equivalent IMO. The only difference is in whether the feature is useful and well received.

The issue isn’t doing something to your data, it’s what happens after that point.

People would be pissed if Android make everyone’s photos public, AI does this with extra steps. Train AI on X means everyone using that AI potentially has access to X with the right prompt.

I don't think it's _training_ on your content. That would be a whole other (very horrifying) problem, yes.

Why make that assumption? Many AI companies are using conversations as part of the training, so even if the documents aren’t used directly that’s doesn’t mean the summaries are safe.

Except that's not what is happening here or what the rest of us are discussing, so why even bring it up?

We don’t know what’s happening beyond:

^the privacy settings used to inform Gemini should be openly available, but they aren't, which means the AI is either "hallucinating (lying)" or some internal systems on Google's servers are outright malfunctioning*

Many AI systems do use user interactions as part of training data. So at most you might guess those documents aren’t directly being used for training AND they will never include conversations in training data but you don’t know.

>which means the AI is either "hallucinating (lying)" or some internal systems on Google's servers are outright malfunctioning*

I'm not sure how that is implied.

>Many AI systems do use user interactions as part of training data.

There is no evidence of that being the case here, and none of the mainstream AIs do that yet. They'd be much more useful if they did.

>So at most you might guess those documents aren’t directly being used for training

Or we can actually know that, because that's the case.

>AND they will never include conversations in training data but you don’t know.

Conversations aren't part of this discussion at all, so I'm not sure what you're trying to imply, but it's wrong.

> Conversations aren't part of this discussion at all

People only know about it because information from these documents is showing up in conversations.

It’s unclear which systems have access and why, but at a minimum Google is showing the data. If things are “misconfigured” or even intentionally set up like this then any assumptions about what’s private goes out the window.