Item — HN Reader

I thought it was well known that GPT-3 is pretty good at producing incoherent bullshit. No surprise here.

Take this for example:

> At the party, I poured myself a glass of lemonade, but it turned out to be too sour, so I added a little sugar. I didn’t see a spoon handy, so I stirred it with a cigarette. But that turned out to be a bad idea because it kept falling on the floor. That’s when he decided to start the Cremation Association of North America, which has become a major cremation provider with 145 locations.

What?

2 comments

GPT doesn't have an 'understanding' class or a 'reasoning' function or whatever. It's a really well put together piece of statistics and sentences like these show it doesn't really have a concept of 'making sense'. You can use your much more advanced human brain to visibly see where it put in random variables (cigarette) and where it borrowed pieces of sentences (but it turned out to be too sour). You can see it made no connection between those two things that wasn't based on pure probability, and got it wrong anyway.

I'm not trying to be reductive, i like the model, it's just good to know the limitations of the tools you are using and to remember that it's not an independent thinker.

The cigarette and the too-sour part were all part of the prompt and were written by a human. I guess somebody had better tell them the bad news...

I think you're drawing an inappropriate conclusion.

Imagine someone offered you a million dollars per contiguous accurate word you produce in a continuation of some text they randomly drew from the internet. You were allowed to have a team of experts help you and have a month to construct your best response (but not to go look up the text itself).

If the text they gave you was some weird non-sequitur thing are you going to make your continuation an earnest reply? No. People don't usually write that way. Instead, you might decide that the source material was a children's story or a joke site and continue in kind. You want the millions of dollars, after all.

So when prompted with some "natural reasoning" question without any other context-- involving a farcical situation with a bathing suit and a court appearance of course you are going to continue it with a story about wearing a bathing suit to court. Because, duh.

This is the task GPT3 is accomplishing. But instead of merely being incentivized by millions of dollars, GPT3 was built from the ground up to exist for only this purpose and it does it reasonably well.

Would we assume that you didn't understand the world, didn't have a concept of 'making sense', or were only capable of madlibs because when asked to accurately continue text from the internet and prompted with a bit of narrative about a bathing suit and a court you recognized the genre? No, we would not. Quite the opposite.

As far as I can tell GPT3 understands the world better than some MIT Technology Review writers and HN posters.

>It's a really well put together piece of statistics

But why think "statistics" precludes it from having genuine understanding to some degree. After all, there is a statistical description the human brain but that doesn't seem to preclude understanding.

I keep asking this whenever I see dismissive responses of this sort, and I never get a reply.

Statistics doesn’t preclude understanding, but statistics are definitely not enough. For example, uncertainties/probabilities/statistics is original to whether the model incorporates causal/reasoning structure. Any tractable amount of data with the former can’t approximate an ounce of the latter. All breakages will be attributed to “distribution shifts” of the underlying statistical distribution, or other pretty words we can come up with... but that basically makes purely statistical approaches “stupid”.

Note that humans have multiple modes of cognition, one of them is very much a stupid, statistical approach that works most of the time.

Kahneman and Tversky came up with questions meant to probe that in humans:

> If a baseball bat and a ball cost a total of $1.10, and the bat costs $1 more than the ball, then how much does the ball cost?

Most people arrive at $0.10 quickly (system 1), but you can't arrive at the right answer if you go by your gut, you have to shift into a different mode of cognition (system 2) in order to compute the correct answer.

>Any tractable amount of data with the former [statistics] can’t approximate an ounce of the latter [causal/reasoning structure].

I don't know why you think this is true. If statistically B follows A to a high degree, then a sufficiently advanced statistical model will represent "A then B" in some manner. In a predictive language model, at some point the best way to model a text corpus that indirectly references the "A then B" causal structure is to just model that structure and reference it as needed.

Because if you have a working concept of time, space, and modes of transport, you are aware that a a person has been driving for 2 hours, you can easily deduce the handful of possible towns they might arrive at. Indeed we have software that does that.

The statistical model will die to combinatorial explosion between billions of possible combinations of locations, times, and modes of transport. In various literature in 2 hours you might have travelled across town, across continents, or to the moon. Statistical approach to such problems is dumb.

But this isn't pointing to a fundamental limitation of statistical models, only a limitation of the text corpus. If you had a billion pages of text written about some town and the text included descriptions of travel distances and locations, the model should eventually develop a good representation of the town and relative locations. But of course without such a seed of spatial information, it will just make up plausible data. A human would behave similarly when forced to write a story while lacking critical information.

>Statistical approach to such problems is dumb.

Well, expecting your model to extract a spatial representation of the world from text is a dumb approach indeed. We interact with the spatial information much more directly. But our ability to navigate is fundamentally just a process of capturing regularities in our sensory input.

The statistical argument has limitations, for instance, when there are more pieces of data to record than there are atoms in the universe. Then it falls firmly into impossible category.

> But our ability to navigate is fundamentally just a process of capturing regularities in our sensory input.

i don't think this is true at all, many animals have dedicated 'hardware' for navigation that can sense magnetic fields, etc. We seem to be born with spatial awareness that is far beyond what GPT will ever be capable of.

> I don't know why you think this is true. If statistically B follows A to a high degree, then a sufficiently advanced statistical model will represent "A then B" in some manner.

Yes, but suppose A implied B only if C were true. And in the training corpus C were always true (hence learned A=>B) but in the test corpus suppose C is not true, then the learned statistical rule is wrong.

The problem is that to cover all these possibilities will require not a billion documents but something like exp(billion) documents (exaggerating, but to give you a feel for combinatorial explosions) as @ClumsyPilot explains.

And problem is the real world always keeps “changing” (Eg: whether C is valid or not) compared to the examples you trained on. This is referred to in polite company as “distribution shift” where your underlying statistical distribution keeps evolving.

But this isn't an issue for statistically modelling causal relationships specifically, this is a core problem of modelling causal relationships at all. The fact that GPT-3 is sensitive to the real world changing, or to having insufficient information to form a universally accurate model says nothing interesting about GPT-3.

Aargh, autocorrect and expired edit window. original -> orthogonal

> But why think "statistics" precludes it from having genuine understanding to some degree. After all, there is a statistical description the human brain but that doesn't seem to preclude understanding.

It's a matter of scale. Adult human brains aren't just trained for 18 years with a static dataset - they've got hundreds of millions of years of mammalian evolution and fault tolerance built into them. Our brains self-reconfigure in response to external stimuli as we age, to the point where we can (in rare cases) lose significant fractions without becoming a vegetable. The biochemistry of a single neuron is likely oodles more complex than the most complex AI we've made.

It's like going from an analysis of a small family to an analysis of a civilization: at some point the sample size crosses a threshold and emergent phenomena start to dominate the system. Yes, it's basically all just statistics (ignoring quantum hocus pocus and the supernatural) but we still don't understand how to go from statistics to intelligence.

I'm not seeing an argument here. Mammalian evolution is great. So is backprop. They're both methods to efficiently search the state-space of parameters. If your argument is scale, you have to argue how scale precludes a sufficiently general learning algorithm and enough computation from capturing the underlying structure.

>The biochemistry of a single neuron is likely oodles more complex than the most complex AI we've made.

There's a lot of biochemistry in service to supporting the neurons existence exclusive from its functional properties for the brain. In terms of a neurons I/O mapping, deep learning has that covered[1]

>Yes, it's basically all just statistics... but we still don't understand how to go from statistics to intelligence.

But this assumes intelligence isn't an emergent phenomenon of sufficiently general learning. GPT-3 suggests this might be the case.

[1] https://www.biorxiv.org/content/10.1101/613141v1.full.pdf

GPT-3 is a language engine, not a reasoning machine. What is understanding, neurologically? At what point can we measure whether an organism or piece of technology is able to be aware of concepts?

Does a honeybee (arguably the most intelligent insect) understand that pollen is a necessary component of honey? Or that it is using geometry to convey directions using angles of the sun? Why should we expect a piece of technology with magnitudes less sophistication to be able to mimic higher order lifeforms?

Putting aside these as of yet unanswerable questions it seems to me that, interestingly, the less commonly used definitions of "understand" may increasingly fit the bill with an inverse relation to common use.

From Merriam Webster: 2 : to accept as a fact or truth or regard as plausible without utter certainty 3 : to interpret in one of a number of possible ways 4 : to supply in thought as though expressed // "to be married" is commonly understood after the word engaged

There's a wonderful interview where Richard Feynman interrupts an interviewer to explain how "why questions" have layers of understanding

https://fs.blog/2012/01/richard-feynman-on-why-questions/

> Of course, it’s an excellent question. But the problem, you see, when you ask why something happens, how does a person answer why something happens? For example, Aunt Minnie is in the hospital. Why? Because she went out, slipped on the ice, and broke her hip. That satisfies people. It satisfies, but it wouldn’t satisfy someone who came from another planet and knew nothing about why when you break your hip do you go to the hospital. How do you get to the hospital when the hip is broken? Well, because her husband, seeing that her hip was broken, called the hospital up and sent somebody to get her. All that is understood by people. And when you explain a why, you have to be in some framework that you allow something to be true. Otherwise, you’re perpetually asking why. Why did the husband call up the hospital? Because the husband is interested in his wife’s welfare. Not always, some husbands aren’t interested in their wives’ welfare when they’re drunk, and they’re angry.

>Why should we expect a piece of technology with magnitudes less sophistication to be able to mimic higher order lifeforms?

Honeybees apparently have 1 million neurons compared to GPT-3's 175 billion parameters. Granted, there isn't a 1 to 1 correspondence between biological neuron and parameter. And considering much of the honeybees neuron's would be directed towards biological function and control mechanisms, whereas all of GPT-3's parameters are directed towards text prediction, the relevant expressive power of GPT-3 is plausibly much larger.

I believe honeybees have closer to a million neurons, not 100 billion - according to this wikipedia list [1] - but perhaps it makes more sense to count the total number of neurons in the entire colony.

[1] https://en.wikipedia.org/wiki/List_of_animals_by_number_of_n...

Ha you're right. I was quickly scanning a google result and misread it to read 100 billion. Edited. That number should have struck me as unreasonable :)

It's an inherently limited model of the human brain. It pretends biology and electrochemistry aren't playing some important role 'statistics' cannot. It's GPT which has to do more legwork to be compared to a human brain, not the other way around.

>It pretends biology and electrochemistry aren't playing some important role 'statistics' cannot.

But the issue here isn't about modelling the human, it's about learning a model that captures understanding. To assume that human or animal biology is necessary for understanding is a big leap that isn't supported by the evidence. For one, the relevance of biology and electrochemistry appears to be to influence the function of neurons in various ways. So there's no reason to think that this influence cannot be captured by a model driven by statistical relationships.

Sounds like the way dreams are usually structured. Seemingly coherent overarching structure with completely arbitrary zig-zags at the micro level.