Item — HN Reader

I have a hard time understanding how GPT works and how it's so good at convercing.

From what I understand, GPT works by predicting the next token based on the previous right?

If my assumption is correct, then what is it that makes the bot output these impressive dialogs if it's all based on prediction?

6 comments

Most of what human brain does is prediction of what comes next. For example, as you're reading each word in this sentence, your brain is trying to predict the next word you might see (a word or a phrase or even a whole sentence that's likely to come next). When you don't see what you expected you get surprised.

> Most of what human brain does is prediction of what comes next.

This is categorically false.

Are you challenging the widely accepted theory in neuroscience? OK, let’s hear your arguments.

You made the claim.

"GPT works by predicting the next token based on the previous"

Technically true, but a very limited description in my view.

Any computer program can be said to do the same thing. A program has a set of instructions and an internal state based on how those instructions processed previous inputs. It generates an output and a new internal state based on the next input.

Breaking it down further, a computer is just a large collection of very simple NAND gates with two binary inputs and a binary output.

Your brain can be said to consist of a large collection of simple atoms that behave according to simple rules that can be simulated in a large enough computer given enough time.

These descriptions are also technically true, but they are so limited that you wouldn't expect a computer or a brain to be as capable as they are, right?

GPT can clearly emulate a CPU when asked to, so I think it is obvious that it is Turing complete (?), with the caveat that it can only execute 32k instructions before it has to stop.

I think (?) one could say that GPT can be viewed as an extremely large state machine that reacts to inputs based on the current state and previous inputs.

If that is true, then that might be a mental model of GPT that makes you a lot less surprised about its reasoning capabilities?

I was initially also dismissive about it in the same way (just mechanically predicting tokens), but I've had to change my mind when seeing how it can reason about given scenarios.

> Turing complete

This cannot possibly be true. The architecture is feedforward, which implies it answers each query in an amount of time bounded by some function. By the time-hierarchy theorem there exist computable functions with arbitrarily high asymptotic complexity.

I won't comment on anything else, but I'd like to share I had your exact same reaction: I still think it's way overhyped but it's not really defensible to say it's just a Markov chain.

> The architecture is feedforward, which implies it answers each query in an amount of time bounded by some function.

Are you talking about each step where it produces a new output token?

If so, sure. But how is that different from any CPU that executes an instruction?

If you mean in general, doesn't it have a feedback loop where its own previous outputs affect future outputs?

You could ask it to emulate this Basic program:

  10 PRINT "Hello"
  20 GOTO 10

And it would in principle output "Hello" forever (but in practice segfault after 32k tokens), right?

(I just tried something like this in ChatGPT. It did emulate a few iterations which it output, but then it performed a halting analysis and told me "As there is no condition to exit the loop, the program will continue to run in an infinite loop until it is manually interrupted." - not sure who's point that proves :-)

> By the time-hierarchy theorem there exist computable functions with arbitrarily high asymptotic complexity.

You're clearly more versed in the theory than I am with just an average CS level understanding. But isn't that the same as "with the caveat that it can only execute 32k instructions before it has to stop"?

I'm not sure what the limits of the current GPTs are. It's far from clear that they'll evolve into AGI, we might still see a new AI winter in the future. But at the same time it looks like we've stepped up to a completely new level of capabilities.

The feedforward nature does mean it's not computationally complete for a single iteration.

But a single iteration only generates a single token, which gets fed back in to the next iteration. If you let it loop forever, generating unlimited tokens, then the theoretical computational complexity is unbounded.

At least theoretically. In practice, it's not going to split computation over that many tokens, and you run into token limits.

Well, okay, but you're running into the same problem with memory, and I could invoke the space variant of the theorem.

You could say that an infinite family of GPT-like models with increasing context size collectively form a Turing-complete computational model and I would have no objections, but you're stretching the definition a bit...

In the real world nothing has infinite memory, so no computer would be turing complete. Therefore this requirement is ignored.

Hundreds of millions of parameters, hundreds of gigabytes of RAM and languages with a vocabulary of only 10^4 words mean it can produced incredibly nuanced text. It is impressive.

That's absolutely right, it just predicts the next token. One of the discoveries that led to GPT was the concept that "token prediction is universal" in the sense that all other natural language tasks are a sub-task of token prediction. For example translating between one language and another is just predicting what would continue after you say something then say "and here it is in french: ".

There are levels to token production from generating complete jibberish, to generating very shallow nonsense sentences, to generating gramatically coherent sentences that dont really say anything, .. and so on. They've pushed the depth of its token prediction to a new level that makes it more impressive than anything previous.

This is partly true, but the prediction isn't based on the previous token. It is based on the tokens generated, in the context of having parsed trillions of tokens and calculated the relative importance of those tokens in the context of the tokens around it.

So the matrix of weights not only encodes probability of the next work, but also encodes which word (amongst a choice of equally probable words) makes the most sense in the token stream.

That's my very shallow understanding of the attention mechanism anyway.

GPT also uses embeddings. It converts each token into a vector that captures the meaning and context of the word. Related tokens are close by in this large vector space.

The way I understand it is that the best way to predict the next words in a sentence is to understand the underlying reality. Like some kind of compression.