Item — HN Reader

I honestly don't know what the hell Google was thinking. I know shoving AI into everything is the rage these days, but this is simply an even more insane way to demolish the credibility of your own product than the AI overviews in Google Search.

Glad they came to their senses quickly.

1 comments

I like the AI overviews.

That's how they get you. They are trained to make you like them. They are not trained to convey true information.

Yes, they are at best misleading and often just present dangerously incomplete or misdirecting information. The more you know about a topic the more appalling the summary.

I check variations of this search every month and it still fails at a reasonably high rate https://imgur.com/a/tKSQuA3

That's interesting. "mums good for cats" and a "mums bad for cats" gives the expected response, it's just "okay" being parsed funny. I've reported it to someone at Google so it'll get fixed... eventually.

Replacing “okay” with “safe” also fails

https://imgur.com/a/PgCzK1H

Kind of unbelievable 4 years into this hype cycle we still have basic issues like answering the inverse of users question

It's also very non-deterministic. Not a 100% failure rate, seems to go up&down with time, and varies depending on plant/animal combination

Can you elaborate? Are they not toxic? Are they toxic, but the chemical analysis wrong?

If you asked your wife "is it ok if I did X" and she said "YES, its not ok to do that.." would that be normal behavior?

Ah I see now. Maybe my brain translated the question implicitly to "Are mums poisonous to cats?". I'm tempted to say it's a minor quibble, but on the other hand, there are probably a good deal of people who may stop reading after "Yes"

Right and someone in a hurry quickly checking their phone may read the affirmative response as an OK.

It's worse on mobile with large text in my example - https://imgur.com/a/wvNspMT

Imagine your parents asking about a drug combination using an old iPhone SE with large text enabled.

"is zazzafraz and banza ok"

Yes zazzafraz (long scientific name 1) and banza

(long scientifgic name 2) are toxic in combination

I find them irritating because they will link a source entirely unrelated to (or worse, directly contradicting) the fact they are claiming. The Google AI summaries might be the most blatantly hallucinating model widely deployed right now.

I’ve had them abruptly try to turn mathematical theorems into cooking recipes before cutting off mid-sentence in a panic.

YMMV, but nearly every one I read has a glaringly incorrect comment. I frequently research off-trail mountaineering routes and the incorrect bullshit it confidently spouts at the top of a list of results (results which very much counter the AI overview) is liable to end up getting someone hurt, lost, or worse.

I also see this failure mode where it inverts the question and answer the affirmative to the inverted question (therefore the opposite of correct answer).

If you read too quickly or are on a mobile device, the clarifying information may be below the line break.

https://imgur.com/a/tKSQuA3

on a mobile device (iPhone Pro Max) with Large Text enabled you can see how the clarifying text starts to roll over to the next line:

https://imgur.com/a/wvNspMT

Imagine a senior on an older smaller phone with large text enabled, easily missing the Opposite Day answer.

I suppose I could think of worse use cases like: give me a recipe for a sketchy peptide shake, but not by much.

Not every technology needs to exist simply because it’s possible.