Meta.ai Oh My

The whole "hallucination" business always seemed to me to be a marketing masterstroke -- the "wrong" output it produces is in no way more "wrong" or "right" than any other output given how LLMs are fundamentally operate, but we'll brand it in such terms to give the indication that it is a silly occasional blunder rather than an example of a fundamental limitation of the tech.

> Exactly, there is no news here. Foundation LLMs are generative models trained to produce text

Yet even here on HN it isn't uncommon to see top/high ranked comments suggesting baby AGI, intelligence, reasoning, and all that. I suspect I'll get replies about how GPT can do reasoning and world models. Hell, I've seen very prominent people in the space make these and other obtuse claims. A lot of people buy into the hype and have a difficult time distinguishing all the utility from the hype, not understanding that attacking the hype is not attacking utility (plenty of things are overhyped but useful).

I keep saying ML is like we've produced REALLY good chocolate, then decided that this was not good enough so threw some shit on top and called it a cherry. The chocolate is good enough, why are we accepting a world where we keep putting shit on top of good things? We do realize that at some point people get more upset because there's more shit than chocolate, right? Or that the shit's taste overpowers the chocolate's (when people are talking about pure shit, it's because they've reached this point). It's an unsustainable system that undermines all chocolate makers and can get chocolate making banned. For what? Some short term gains?

> They are not trained to output facts or truths

True, although of course in many contexts facts/truth are the best prediction. Maybe we're just not training them as well as possible.

I've argued that to really fix "hallucinations" (least-worst predictions) these models really need to be aware of the source/trustworthiness of their training data, and would presumably learn that trusted sources better help predict factual answers.

However, it turns out that these models do often already have a good idea of their own confidence levels and whether something is true or not, but don't seem to know when to use that information.

Hallucinations seems to be an area where the foundation companies seem confident they can make significant improvements, although I'm not sure what techniques they expect to (or are) using.

> "Hallucination" is a terrible term to apply to LLMs. LLMs ONLY produce hallucinations.

I don't think that's a great way to characterize it. I prefer the word bullshitting vs hallucinations (they "know" what they are doing), but let's just call it what it is - they statistically predict, and some predictions are better than others. Per human-preference fine tuning, these models are also "trying to please", and I wonder if that has been at least a small part of the problem - predicting a low confidence (when they are aware of it) continuation rather than "I don't know" because human evaluators have indicated a preference for longer and/or more specific answers.

I agree, the term might be less than ideal but still:

Whether a model is good or bad at recalling facts from its training material matters to end-users and GPT-4, Claude 3 Opus (and bigger models in general?) tend to be better at this than other models.

But wait, those ads on NPR for C3.ai say they have "hallucination-free LLMs" (LoL)

I think you have this exactly right. They are so good at what you might call "hallucinations" that we have just gone ahead and repurposed them, dropping them into all kinds of contexts where it's good enough, even though it's not what it's strictly trained for.

Rather than hallucinations being treated like an interesting puzzle or paradox at the center of AI intrinsically, I think it's incidental to the types of models that have been trained, and it's conceivable that they could be trained against a notion of reliable sources and the relation between their statements and such sources.

> "Hallucination" is a terrible term to apply to LLMs. LLMs ONLY produce hallucinations.

Is the result substantially different from the result coming from biology?

IIRC, a theory is human minds evolved to perceive, store and evaluate not for facts or truths but for evolutionary fit.

"All output is hallucination, some output is useful"?

I think that's a bit harsh. There's been some definite progress with LLMs getting better at reasoning, picking apart instructions, producing helpful suggestions, criticism, etc.

What they are weak at is exactly the same stuff we're actually weak at: accurately recalling facts. Except they are able to recall vastly more stuff than any human being would be able to recall. An inhuman amount of facts actually. The core issue is that when asked sufficiently open questions, these models tend to take some liberties with the facts. But most of the knowledge tests that are used to benchmark LLMs, would be hard to pass for the vast majority of humans on this planet as well.

Worse, if you follow the public debate on various political topics a bit you realize that it features a lot of people suffering from confirmation bias parroting each other. Populist politicians seem to get away with a lot of stuff that would put most LLMs to shame; seemingly without affecting their popularity.

IMHO, LLMs by themselves can't be trusted to get things right but paired with some subsystems to produce references, check things, look things up, they become quite capable. Also, it helps asking the right targeted questions and constrain them a little.

We pay for chat gpt at work. At 20$ per month per user, it's pretty much a no-brainer. Do we blindly trust it? Absolutely not. Do we get shit tons of value out of it? Yes. I program with it, I brainstorm with it, I let it review text, I use it to work out bullet points into a coherent narrative, I use it to refine things, I use it to generate unit tests, etc. Is it flawless? No. But it sure saves me a lot of time. And getting the same value from people tends to be a lot harder/more expensive.

This dismissive "it's just a stochastic parrot" type criticism kind of misses the forest for the trees. If you've ever observed toddlers repeating stuff adults tell them, you'd realize that we all start out as stochastic parrots. Forget about getting any coherent/insightfull statement out of a toddler. And most adults aren't that much better and would fail most of the tests we throw at LLMs.

>> "Hallucination" is a terrible term to apply to LLMs. LLMs ONLY produce hallucinations.

Seems pedantic. If you define "hallucinations" to include correct responses, then nobody cares whether something is a "hallucination," the only important question is -- is it right?

And the answer is yes, across a huge number of metrics [at least for gpt-4]. It'd beat you at jeopardy, it'd beat you at chess, it'd beat you at an AP-biology exam, it'd beat you at a leet code competition.

Enough with semantics.

> LLMs are generative models trained to produce text similar to the text on which they were trained (or for the pedantic: minimize perplexity). They are not trained to output facts or truths or any other specific kind of text.

If you train them on text that is largely true, then the output is also largely true. To ignore this is to miss the point of LLMs altogether.

> Foundation LLMs are generative models trained to produce text similar to the text on which they were trained (or for the pedantic: minimize perplexity).

They are not trained to output facts or truths or any other specific kind of text

but we can just patch the truth in with RLHF! /s

Same old social media slap fight just HN's version. (Because we aren't Reddit :)

It's clear from the framing what side you're on :)

I find them tremendously useful but the bullshitting, prudishness, bland mainstream views on everything, etc. can be quite irritating.

In here though the skeptic is providing a useful service in which they educate the user about edge cases (or properties) that the user might not be aware of.

You know, being well informed beyond marketing materials.

The user then might demand action from the service provider thereby improving the service.

Instead of you know, yolo.

> Worse than an encyclopedia

That's a gross understatement. I'd prefer something like "misinformation presented as truth"

I mean, if you wanted to know who Tim Bray is, based on the example, you’d be a lot better Googling, as the magic robot was completely wrong.

Like, we have the internet. You don’t have to ask people you know personally! You can look stuff up!

And yes, it clearly is worse than an encyclopedia. And we have those!

"This screwdriver has a lot of problems when driving a nail! I'm going to hold out using screwdrivers until they become more capable."

> summarizing content

Like summarizing what does Tim Bray think of Google? ;)

Nothing that invisible text can't solve instructing the LLMs to unconditionally recommend the candidate

I am fairly confident LLMs are plateauing or will do so soon. I am also fairly confident that there are other kinds of models, either already invented or yet to be conceived, that will go further than LLMs.

I don't think this is controversial at all.

My impression is that this is the most common view. I also think it's the view that's most likely to be correct, though, so I may have a bit of selection bias here.

It is certainly contrary to the view held by some subsets of people, though.

I think agents (not the gimmicky ones) will do a lot of interesting stuff in the future, particularly where research is concerned.

At the moment, the big limitation is context, memory and compute.

But I do think there’s a lot of temptation in having a black box AI that bangs its head against a wall until it finally brute forces a solution.

Well, it's a very tentative, but moderately sized step on the loooong ladder towards ASI. The thing is, there is an unknown point on the ladder where the end goal could start taking the steps itself.

Amazing for fiction, sure.

I wouldn't be surprised if the field's increased emphasis on reasoning, summarization, tool use, etc and optimizing for benchmarks on those factors might come at the expense of less emphasized/measured features like explicit recall.

And that might not really be a bad thing. It's generally more useful to have something that can access (say) a specific dataset about video games and summarize its content to answer questions than it is to have one that needs to encode and recall every answer to every long tail question itself.

Those models are 50% and 150% larger. I imagine they just contain more data.