Item — HN Reader

It’s unfortunate - he’s an author, it would have been fine to stick with an authors perspective that LLMs can’t write (which is true), as well as the copyright stuff (which I don’t agree with but he certainly has standing to give an opinion on).

But he’s made the error of trying to come at it from a technical perspective, when he clearly knows nothing about that side of things, which discredits the rest.

2 comments

How does a tiny error, in an aside and unrelated to the greater point, made at near the end discredit the rest of it?

A lot of people will stop reading or become completely distracted when encountering something so blatantly wrong, and it also indicates the author is willing to say such things. I see some people downplaying it, that’s fine, it’s clearly not helping the author make a case. Even im this discussion, the current top comment and a long thread are solely about that error (granted that’s not too unusual here)

> But he’s made the error of trying to come at it from a technical perspective

Huh? That quote appears to be the entirety of the "technical perspective" of the post and is an aside from his larger points that you have blessed as "fine". Literally nothing in the rest of the post relies on that incorrect statement.

Let me quibble with what is discredited here, given the entirety of your point is built upon an error.

Here's another technical point:

> They're word-association mechanisms with no embodiment and no way to associate the text vectors they manipulate with real-world phenomena.

This is wrong too. RLVR grounds foundational models in reality.

Aren't RLVR signals typically based off formal systems not natural phenomena? The formal sciences are certainly useful for producing tools used in natural science, but it's not entirely clear they alone are sufficient to associate text to natural (real-world) phenomena. Honestly, the pretraining and RLHF are probably more tied to the world than RLVR, for all that RLVR might be useful (maybe even more useful) for making the model perform better in certain tasks (such as working with formally defined systems).

Yeah, I should have said RL, not RLVR. The point is RL interacts with external world, while autoregressive pretraining is limited to the passive ingestion, and RLHF relies on a model of human preferences that has no access to truth sources besides the limited training data it was built upon.

If you want a more concrete example, then LLMs are also trained on visual data these days, which means they do have access to the world in an important way. This directly contradicts the blogpost's claim that LLMs have

> no way to associate the text vectors they manipulate with real-world phenomena.

Historically, that LLMs were text-only used to be a major argument for why they "lack access to meaning", see the Stochastic Parrot paper and the Octopus paper that it references. But even the authors of those papers have (grudgingly) conceded that the argument no longer holds due to multimodality.