Item — HN Reader

I hate that framing because it glosses over what makes AI distinct from other technology. Pretty much anything can be used as a tool, that’s not an interesting aspect of AI. However standard tools do not take decisions on behalf of the user, they do not apply judgement. They come with assumptions and constraints that make some decisions more likely to be made. But the whole thing about LLMs is that they apply judgement and take decisions. When you use an LLM you delegate both aspects to it with very little overview or control.

I don’t think it’s “just a tool”, it’s something that directly compete with the role of humans in a system, by design

1 comments

If you go back in time and imagine yourself in 1950, would you have said that about computers in general? Yes their decisions are the result of pre programmed heuristics, but to the operator that can feel as opaque as AI. In fact in the early days of computing AI was used to refer to a broader swath of what we now call heuristics and machine learning.

Is a very sophisticated computer driven adaptive control system making decisions for the user? Arguably yes. If your car has a bunch of assist tech you are not driving. You are commanding a nonlinear control system.

AI today is quantitatively far beyond these systems but internally it is still just a giant decision table. It’s just a differentiable table that can be automatically programmed by back propagation at what would be unimaginable scale for manual implementation.

As soon as you put a machine between you and the target action that is more complex than what you can hold in your head and that has multiple layers or feedback loops, you are ceding fine grained direct control in favor of a simpler abstract control surface.

I will change my mind a little if I start seeing evidence of genuine volition, but as of now I suspect that’s a property of life not intelligence. One interesting result of AI when viewed as an experiment is: I think it proves that life and intelligence are separate phenomena and that consciousness and intelligence may actually be unrelated.

I have the hypothesis that consciousness (and therefore true volition) is a property of life. You have to be metabolic and tied in some intimate way to thermodynamics and the arrow of time. Solid state electronics doesn’t have this property. It’s unrelated to intelligence. A bacterium may be conscious.

LLMs don’t have a will of their own and will (very likely) never have one, they aren’t an entity or anything that can have a sense of self. They are a compressed dataset with a next token prediction engine. From the token prediction + complexity of the dataset + reinforcement learning + whatever other techniques AI folks are using we are able to derive judgement and decision making (of course the LLMs don’t judge or decide anything, they have no free will of any sort, it’s purely derived from the patterns within the dataset).

Though I personally don’t find the topic of consciousness interesting, it’s not something we are able to define and is distracting from all the other aspects of LLMs that should be discussed or evaluated for what they are.

My fridge has a will to keep the food at a certain temperature.

The difference is that with software before the rise of giant machine-learning models, the software is derived entirely from human-understandable source, and often a full software stack can be built on a single computer, as anyone who ever ran Gentoo knows. While the source of any non-trivial software stack is too large for any single human to understand in full, a single human can understand any part of it. That's not true of model weights. And my understanding is that even the smallest open-weights models that are remotely in the same class as LLMs require large computing clusters with powerful GPUs to train. That puts training a model, or retraining one from its true source materials (the training data and processes), in an entirely different class than rebuilding Gentoo, even with GNOME, Firefox, LibreOffice, etc., even if the training data and processes are available, which they're usually not. So, we should treat these models as a different kind of thing than the tools that we developers have worked with until now.

>If you go back in time and imagine yourself in 1950, would you have said that about computers in general?

I'm not the parent, but personally, no, I don't believe so.

I always appreciate the "it's like past changes" thought when things change, but in my opinion, this one simply isn't. It's notably different both quantitatively and, critically, qualitatively, than all previous increments in computers/software. This is much more lateral than punch cards -> assembly -> compilers -> frameworks.

Also, regardless of that thought, the "it's like past changes" argument relies on a false implication: That all changes that survive are "good" in the broad sense. Many changes often persist simply because they are inevitable - not because they are good. They have both good and bad parts, and it's subjective as to which side it leans towards.

I agree, and you wrote that better than I could have. Though it’s difficult to say why exactly I feel that way