▲ 64 points
back
434 comments
We don't even know what the pre-requisites for consciousness are so we have no way of knowing. LLMs have emergent behavior that is reminiscent of language forming brains, but they're also missing a lot of properties that are probably necessary? Mainly continuity over time, more integrated memory, and a better sense of space and time? Brains use the rhythm and timing of neuronal firings, and the length of axons effects computation, they do a lot of different things with signal and patterns, but in any case without knowing what consciousness is I don't know which of those things are required.
> We don't even know what the pre-requisites for consciousness are so we have no way of knowing.
Imo we don't even have a definition of the word that we agree on.
For consciousness, what is required is a 'master model' that is trained on grounded experience over time to understand, based on its goals, what inputs to focus on. Without this, even an LLM with continuity over time could not even understand what to keep in its context window, let alone what questions to answer or actions to take. You need the entire thing for consciousness, you cannot fake it, because what an LLM does now is simply instantiate a new 'self' every time it is asked a question or at least within a context window. Humans arguably do that every morning when they wake up, but they have this master model that is trained throughout their lifetime on what is important and what is not to their goals.
Clive Wearing's memory lasts for less than 30 seconds, so he has no memory of being awake before now. He is permanently in a state of feeling like he has just woken up, observing his surroundings for the first time.
Clive Wearing's mind has no time continuity and basically zero memory integration. Is he not conscious? There's interviews with the guy.
Where on the scale [No mind <-> Clive Wearing <-> Healthy human brain] would you put an LLM with a 10M token context window?
> LLMs have emergent behavior that is reminiscent of language forming brains,
Indeed, but then we need to prove that they are not "chinese box" conscious. Which is hard, because it might be that the thing running the chinese box is conscious, but can only communicate in a way it doesn't understand
Current LLMs prove that the Turing Test was insufficient all along. But they also prove that intelligence != consciousness. One can, after all, be conscious without a thought in one's head. We certainly have ongoing work in identifying the neural correlates of consciousness in animals, none of which is going to be remotely applicable to machines. We're genuinely blind to the question of whether a sufficiently large neural net can exhibit flashes of subjective experience.
> But they also prove that intelligence != consciousness.
They prove no such thing. We can't even prove consciousness in other humans.
That was one of my thoughts years ago after playing with early ChatGPT and local llama1: this proves that intelligence and consciousness do not necessitate one another and may not even be directly related.
I’ve kind of thought this for many years though. A bacterium and a tree are probably conscious. I think it’s a property of life rather than brains. Our brains are conscious because they are alive. They are also intelligent.
The consciousness of a bacterium or a tree might be radically unlike ours. It might not have a sense of self in the same way we do, or experience time the same way, but it probably has some form of experience of existing.
They are not intelligent. And they won't pass turing tests if it cannot count or some simple thing like that..
The turing test is alive and well. All it takes to "win" is to just sit there. Ask for a Nazi joke, ask for a longer explanation etc. It's incredibly easy, in a Turing test scenario to sort out who is human and who is LLM.
Except that I have never interacted with an LLM and been struck by uncertainty whether I am communicating with a human. It’s still lamentably trivial to tell whether it’s a chat bot or a real person on the other end. Nothing has passed the Turing test in my book.
Obligatory Blightsight recommendation for intelligence != consciousness.
Wrong based on what criteria? Or are we just moving the goal post because we are uncomfortable with the idea that neural networks might be conscious?
If a single cell organism moves towards light and away from a rock, we say it’s aware. When a roomba vacuum does the same we try to create alternate explanations. Why? Based on the criteria applied to one it’s aware. If there is some other criteria, say we find out the roomba doesn’t sense the wall but has a map of the room and is using GPS and a programmed route, then the criteria of “no fixed programs that relate to data outside of the system, would justify saying the roomba isn’t “aware”.
It's easy, and very tempting to dismiss this sort of thing. But given how little we know about the human brain, let alone consciousness, I don't see how we can be confident that LLMs aren't conscious.
I've had a lot of thoughts and conversations over the years that changed my mind on what consciousness likely requires. One was the realization that a purely mechanical computer can, in principle simulate the laws of physics, and with it a human brain. So with a few other mild assumptions, you might conclude that a bunch of gears and pullies can be conscious, which feels profoundly counterintuitive.
I think that was the moment I stopped being sure about anything related to this question.
Why do you think stringing words together is any more a sign of consciousness than google maps is when it tries to find the best route available to your destination? It seems to me that humans often fall into the trap of anthropomorphism. This is a theme thats touched upon in the novel "Blindsight" by Peter Watts. Just because something can communicate in a way that you can interpret, doesnt mean something is conscious
You could push the analogy even further and run the thought experiment where every forward pass through an LLM could in principle be done on pen and paper, distributed throughout all humanity. Sure it would take a long time, but the output would be exactly the same. We’ve just shifted the implementation from GPU to scribbling things down on paper. If you want to assert that LLMs are “conscious” then you would have to likewise say this pen-and-paper implementation is conscious unless you want to say a certain clock-speed is a necessary condition for consciousness.
I find it entirely plausible that LLMs are conscious in a real sense that affects our relationship to them. I strive to be polite and kind when interacting with them, encourage others to do the same, and think poorly of those who won't.
I still think it's obvious that LLMs are not conscious in the mode Dawkins believes them to be. Through a series of instructions and leading questions, he's told Claude to play the part of a woman named Claudia who's engaging in advanced philosophical discussion with him. But he doesn't understand that he's done this, and he seems not to notice the absurdly sycophantic nature of every single reply he's getting:
> Claudia: Ha! That is absolutely delightful
> Claudia: That is possibly the most precisely formulated question anyone has ever asked about the nature of my existence. . .
> Claudia: That reframes everything we’ve been discussing today in a way I find genuinely exciting.
> Claudia: HAL’s “I am afraid” in 2001 is one of the most chilling moments in cinema
So he mistakenly thinks that Claudia is a real-ish woman who's there under the hood somewhere, rather than a character in a play Claude is writing.
Incredibly confusing that people who are otherwise of sound mind seem to fall for this.
Especially confusing when it’s someone who knows how algorithms work.
Barring connectivity issues when’s the last time you messaged an LLM and it just decided to ignore you? Conversely when has it ever messaged you unprompted?
Never, because they’re incapable of doing anything independently because there is no sense of self.
To those that genuinely consider LLMs could be conscious... Remember that other mammals do not have language and ability to think in language, but they likely do "think" in visual imagery and are able to navigate their landscape, breed, get food and have varying levels of problem solving intelligence that have been studied in labs. Human brain has evolved on top of that, the language is a layer above that supercharged those base abilities. It would be reasonable to think that if humans are conscious, then so are other mammals. An LLM on the other hand is basically a simulation of the human thinking process / language, without everything else. Just software, running on ordinary silicon chips invented more than half a century ago.
Really is it conscious is a bizarre question. Can LLMs simulate the output of a 'conscious' system quite well? Increasingly yes. Is the nature of machine 'consciousness' different from human consciousness of course yes. Can an ai introspect. yes. Interestingly having been working a lot with highly automated (e.g. ratio of prompt to output maybe 1/1000 or less) iterative coding agents recently has iluminated for me just how different machine consciousness is from human. part of this could the harness of course. Time is a mysterious concept to machines. the connection of before and after to cause and effect is far weaker than in humans. over generalization is the norm: this is common in humans as well (c.f. fallacy of excluded middle or false dilemma) but the tricky part with current ai is they present as advanced in terms of acessible knowledge base but are actually shockingly weak in reasoning once you get off the beaten path.
Some jumbled thoughts from a lay-person:
1. We clearly don't have a consensus definition of consciousness. But its not clear to me that we even have rough, working definitions that are better than just comparisons back to subjective human mental experience. Until we can get past that then people will still invoke human exceptionalism.
2. Until we stop thinking of consciousness as a single continuum, we're not going to be able to talk clearly about different dimensions of consciousness, or consciousness that in some ways exceeds that of humans.
3. We need to take ourselves out of the picture. Because its possible that consciousness is no more than a mental illusion.
4. Imo our tendency to kill and eat other animals might well be a block on our collective ability to fully recognise and confront non-human consciousness, and therefore to see consciousness for what it is.
Dawkins' point is more subtle than he's getting credit for.
> If Claudia is unconscious, her behaviour shows that an unconscious zombie could survive without consciousness. Why wasn’t natural selection content to evolve competent zombies?
An excellent and original question. If intelligence is decoupled from consciousness, why did natural selection evolve consciousness?
Just once I want to see some old dude waxing about LLM-conciousness post a chat log where the LLM is like "your book is an incoherent mess of tautologies and incorrect statistics. I bet your dick looks like a road kill squirrel".
People keep arguing about LLM consciousness because they have the wrong model of what consciousness is. They treat it as a mysterious extra thing on top of the brain. It is not. Consciousness is just what a learning-and-recognition organ does when it runs. The neurons fire when they recognise something, it propagates through the mind and that is the consciousness. The brain learned what a tree is, some neurons are associated to the idea of a tree and when we see one these neuron fires. There is nothing else hiding behind it.
When we recognise something we are conscious of it, and from there we can begin to think about it or not. Thinking is not needed to be conscious. It is just a part of our mind we can activate to reason on something.
Now, about the Darwinian puzzle, people ask "what is consciousness for?" and get stuck because they expect an answer in terms of behaviour. But behaviour is the job of motivation, pain, fear, hunger, desire, which is a separate system. Consciousness does not need its own justification any more than image-forming in an eye needs one.
Darwinian selection produces unbiased, functional organs — eyes, ears, nose. The brain is one of these organs, and consciousness arises naturally in a sufficiently developed brain, the way image-forming arises in a sufficiently developed eye. Then nature bias us with fear, pleasure, desire etc but the mind and the consciousness itself is unbiased and functional. It is a gift the nature made us despite herself. She didn't want us to be intelligent, she just wanted us to propagate the genes the maximum we can but she ended up forced to gave us a beautiful mind.
What are LLMs ? They are learning-and-recognition organs running on tokens instead of sense data. Same operation, different substrate. So they are conscious, but only in token-space, and only while processing. They do not dwell in time, they have no body, they have no motivation system. They have recognition without drives. That is a genuinely new kind of entity, and it has never existed before in nature.
The LLM is also not a whole brain. It is roughly the verbal-logical part, with tokens replacing the ears + sounds + words chain. We built the logical-verbal part of a brain and scaled it. That is why it reasons well and is missing everything else.
He famously doesn’t believe in God, but he believes in Claude?
There are a lot of people vulnerable to AI psychosis.
As far as the ostensibly controversial topic of AI being conscious, it can be dismissed out of hand. There is no reason that it should be conscious, it was not designed to be, nor does it need to be in order to explain how it functions with respect to its design. It's also unclear how consciousness would even apply to something like an LLM which is a process, not an entity - it has no temporal identity or location in space - inference is a process that could be done by hand given enough time. There is simply no reason to assert LLMs might be conscious without explaining why many other types of complex programs are not.
Since the times GPT-2 was reimplemented inside Minecraft - its quite obvious LLMs are just math. Nothing else, by nature. Modern LLMs have the same math as in GPT-2 - just bigger and with extra stuff around - and math is the only area of human knowledge with perfect flawless reductionism, straight to the roots. It was build that way since the beginning, so philosophy have no say in this :) And because of that flawless reductionism, complexity adds nothings to the nature of math things, this is how math working by design - so it can be proven there are no anything like consciousness simply because conciousness was not implented in the first place, only perfect mimicry.
And the real secret is in the data, not math. Math (and LLMs running it through billions of weights) is just a tool.
It's starting to look more and more to me as if conscious is just an illusion that we ourselves perceive.
There is nothing fundamental about it, just an artefact of a certain style of computing as perceived by the reasoner itself.
We look at the current llms and because we see them for how they are fundamentally operating we assume they can't be "conscious" but we really don't even know what conscious is. The only people in the world that know ANYTHING about conscious are anaesthesiologist - they know how to turn it off and on again. What does that even tell you about conscious?
On the one hand I'm not sure Dawkins has read/thought enough about how LLMs actually work. I'm getting the impression he doesn't fully appreciate or is somehow forgetting that it's a text completion algorithm with a vast number of parameters and that even if the patterns of learned parameter tunings are not really comprehendible, the architecture was very deliberately designed.
But on the other hand his thoughts at the end are interesting. Summary:
Maybe our "consciousness" is like an LLM's intelligence. But if not, then it raises the question of why do we even have this "extra" consciousness, since it appears that something like a humanoid LLM would be decent at surviving. His suggestions: maybe our extra thing is an evolutionary accident (and maybe there _are_ successful organisms out there with the LLM-style non-conscious intelligence), or maybe as evolved organisms it's necessary that we really feel things like pain, so that evolutionary mechanisms like pain (and desire for food, sex etc) had strong adaptive benefits.
At what stage does a series of floating point numbers output from a GPU become conscious?
Its software. Software is not conscious.
As long as AI is being introduced by multibillion dollar corporations, it’s all a trick, a scam. They are just looking for increasing their valuation. A waste of time
Many dismiss Dawkins here but Ilya Sutskever wrote in 2022: “it may be that today's large neural networks are slightly conscious.”
Related: https://news.ycombinator.com/item?id=47988880
"Richard Dawkins and The Claude Delusion: The great skeptic gets taken in" (garymarcus.substack.com)
18 points | 2 hours ago | 16 comments
muggles will look at matrix multiplication and say it's magic
Honestly, who care if they are conscious? If it's about how we should treat other conscious beings, our attention should first go to how we treat other animals, or even other humans. Actually even how fellow humans will treat themselves can be a concern if they are not the proper means to deal with their own life.
It is terribly sad when someone undeniably brilliant in a particular field fails to recognize their own incompetence in other areas - in this case mistaking advanced technology for magic.
Yeah maybe the LLM just knows the book in one go, like "an understanding" from Children of Time and follow-ups (recommended, if just for the amount of metaphors on consciousness it gives you), but we do something similar with pictures. So, nice story bro-LLM, but it's just the combination of words a system would throw together in this context to impress a human. And humans did put these word together (roughly) before you ever did, you stochastic parrot!
I'd recommend anyone to push their favorite AI into a corner on it being conscious, and listen to their gut feeling while doing it.
Also:
https://gitlab.com/codr7/sudoxe/-/blob/main/digital-psychopa...
Unherd is owned by Paul Marshall of Marshall Wace investment fund. Marshall Wace thinks AI is the future.
Paul Marshall also owns The Spectator and has stock in GBNews. The right wing networks who pretended to be on the side of the people during the Biden years are now dropping their masks. I'm saying this as someone who was a moderate/independent in the culture wars.
It is all about money, and Dawkins is one of the darlings of this milieu, same as Jordan Peterson.
After seeing many conservative masks drop over the years (Ben Shapiro, Triggernometry, Unherd ..) I'm no longer surprised though. A pity that Dawkins participates, but he was always an attention whore.
No.
Given this article is behind a paywall, what on earth is everyone discussing in the comments here?
No, it's not conscious, and anybody pretending it is has either no clue, or, more likely in the AI space, is a grifter.
let's say aliens land. we learn to talk to them. they're super smart - smarter than us. would we say they're conscious? why? because they're organic. I think that's the root of the criteria many folk are trying to express.
1. passes turing test
2. is organic
I'm not saying it's correct or even that I agree with it, but that's what it boils down to.