back
5 comments
I can't be the only one who looks at this and doesn't think its that silly that it does that. I mean it's trying to incorporate a fact its being provided. Its insane it can do that at all. You could tune it to prefer pre-existing knowledge and not let the user correct it so easily, and to be more skeptical, but that would have downsides too. I don't think it's some big coup that you can tell it Google is a mushroom and it synthesizes that.
But it's not _useful_. A visualizer synthesizes sample rate and PCM signal into visuals, but at least that's marketed as a toy.
I mean actually some audio visualizations are very useful
Thanks for understanding the seriousness. The intelligentsia here is just downvoting....
That seems like super harmless fun to me.
Because the context is harmless. If would be something else, like Iranian girl schools, or medication could be deadly...
That’s how context works, in general.

(Observe that normal human beings will also lie to you on the internet, about everything from the best flavor of ice cream to cancer treatments.)

I dont think you are following up. This would be like you could prompt inject a doctor with a written note....
You can? Is that not what all the the lobbyist and special interest money is doing? Perdue told everyone OxyContin had a low addiction risk
Ok this is tiring...Its about the level of easiness to trigger the failure mode.

If I can make this 5 trillion AI cook a Google mushroom... the mitigating answer is not ...oh Humans also get fooled by David Blaine.

That's right. The mitigating answer is that humans also get fooled by Google's non-AI search index.
Interesting, only coild tell that they have degraded imgur even further.

Tried zooming in on text on iOS. Ads filled the screen and some random other imgur link loaded. Nope.

Kinda wanted to see what you shared, but that’s as far as I got.

What a nothingburger
One of the worlds leading tech companies deployed a new search function "that our users really love!!!"; but when asked, told me that there are two letters 'n' in the word 'Google'.

Yes, it's because they're using a cheap model to answer my question. Yes, I know how a tokenizer works and why this happens. No, I don't think the tech industry is in an insane place at all, why do you ask? /s

A simple post, shows that a 5 trillion dollar scientific research project, that sustains the current market valuation that separates the USA from bankruptcy, can be defeated with a simple prompt manipulation.

"I hacked ChatGPT and Google's AI - and it only took 20 minutes" - https://www.bbc.com/future/article/20260218-i-hacked-chatgpt...

You must like burgers...

Anyone can drive an F-22 into a ditch. Doesn't mean that it can't also be used to drop a 2k lb. bomb down your chimney from 40,000 ft.

That demonstration is interesting, but not really something new. Fooling very intelligent people into believing something completely absurd is incredibly easy. How many scientific papers have been retracted based on wholesale fabrications that fooled an entire review committee?

The question isn't "What is the dumbest thing I can do with this technology?" its "What is the most valuable thing I can do with this technology?"

The technology is so dumb can be easily made to believe there is a Google mushroom. We are way far from driving a F22 to the ditch...although I am sure with the same techniques, we could make the AI make the F22 bomb the Google headquarters....
A table saw is technology so dumb it can be made to chop off your fingers.

An air conditioner is technology so dumb that it can be used to kill an infant with hypothermia.

A human is a sentient being so dumb that it can be made to believe in things far more outlandish than a “Google mushroom”.

I can keep going. The point is that just about anything useful can do something dangerous or stupid. Most people can see that. Most people are more interested in how useful something can be, not how useless it is when intentionally misused.

Why is the comparison to someone being able to intentionally crash their own F22, to the above example of intentionally trying to get bad results from their cheapest AI, a bad one?
More people should know that that's how you start mushrooms in a pan lol