▲ 56 points
back
137 comments
I personally think this is a loaded concept. This is what I think:
For the technology we have now (LLM)s there does not and will not ever exist a test that can perfectly differentiate between an LLM and any human.
We will always have LLMs that will pass the test and we will have humans that will fail. The reason for this is two fold and contradictory.
The first reason is that consciousness is just a loaded concept. It's some arbitrary category with fuzzy boundaries. One persons definition of consciousness includes language another persons definition of consciousness includes logical thought. It's a made up collection of "features" that's it, no more or no less. It's a very arbitrary set of features too... mashed together for no rhyme or reason.
The second reason is that LLMs already meet the criteria for what most people technically define as consciousness. The resistance we see nowadays is simply moving the goal posts. Five years ago people were in agreement that the turing test was a really good test. We've surpassed that and changed our definitions adding more stringent criteria for sentience. AI already meets the criteria for sentience the way we defined it from 1950-2020. Thus any test to measure sentience will always be a moving target.
The authors of the paper seem to define what they mean by consciousness quite precisely. They mean phenomenal consciousness:
>The central goal of any C-test is to determine whether a target system has subjective and qualitative experience – often called ‘phenomenal consciousness’.
Phenomenal consciousness is, as far as I can tell, binary. You either have subjective experience or you don't. Even if you are subjectively aware of almost nothing, you still have subjective experience. For example, when you wake up in the morning your awareness might be quite limited for the first few seconds - let's say you have your eyes closed and the only thing you are aware of is the warmness of the blanket. But still, you are aware. You have subjective experience. Whereas in deep sleep you do not.
Phenomenal consciousness is thus pretty well-defined. It's just that we have zero understanding of how physical systems could possibly give rise to it, and it might be in principle impossible to ever figure that out. It is possible that physical systems do not in fact give rise to consciousness, although obviously the presence vs absence of consciousness in humans is at least to some extent correlated with certain measurable physical states.
I doubt that any actual test for consciousness is in principle possible, but the authors of this paper do at least clearly define what they mean when they refer to consciousness. They mean the presence of subjective experience, not intelligence or language or logical thought or having a model of oneself or any other things.
> LLMs already meet the criteria for what most people technically define as consciousness.
I strongly dispute this. Try "talking" to an LLM over time. It has no memory, no agency, no emotions, no desires, no pain, no sense of self.
I think cats, dogs and certainly primates are far more conscious than these stochastic parrots. It do agree it's frightening how many people are fooled by a fancy autocomplete.
> It has no memory, no agency, no emotions, no desires, no pain, no sense of self.
It does have permanent memory, that's what training creates. It doesn't have agency. Whether it has desires, pain or sense of self is completely unknown, unless of course you have some mechanistic model of these phenomena such that we can see its presence in the human brain and see that LLMs do not have a comparable process.
Of course you don't have such a model because it doesn't exist at this time. The problem is not that people have been fooled by stochastic parrots, it's that you've fooled yourself into thinking we understand more about the human mind than we actually do, and also possibly that humans are far more complex and special than might actually be the case, eg. that we are not basically stochastic parrots ourselves.
Turing test was never meant as a test for consciousness. Intelligence — even AGI - that’s one thing. But “feelingness” or sentience— that’s a totally different matter.
The Argument from Consciousness
This argument is very well expressed in Professor Jefferson's Lister Oration for 1949, from which I quote. “Not until a machine can write a sonnet or compose a concerto because of thoughts and emotions felt, and not by the chance fall of symbols, could we agree that machine equals brain—that is, not only write it but know that it had written it. No mechanism could feel (and not merely artificially signal, an easy contrivance) pleasure at its successes, grief when its valves fuse, be warmed by flattery, be made miserable by its mistakes, be charmed by sex, be angry or depressed when it cannot get what it wants.”
This argument appears to be a denial of the validity of our test. According to the most extreme form of this view the only way by which one could be sure that a machine thinks is to be the machine and to feel oneself thinking. One could then describe these feelings to the world, but of course no one would be justified in taking any notice. Likewise according to this view the only way to know that a man thinks is to be that particular man. It is in fact the solipsist point of view. It may be the most logical view to hold but it makes communication of ideas difficult. A is liable to believe ‘A thinks but B does not’ whilst B believes ‘B thinks but A does not’. Instead of arguing continually over this point it is usual to have the polite convention that everyone thinks.
--
I.—COMPUTING MACHINERY AND INTELLIGENCE
A. M. TURING
Mind, Volume LIX, Issue 236, October 1950, Pages 433–460, https://doi.org/10.1093/mind/LIX.236.433
Hell, we don't even have a good definition for intelligence that is accepted across different fields.
The problem I see is we keep looking for supersets of all these features added together, and as the article points out, something that humans have....
>This rationale is particularly powerful if human experience is limited to a small, and perhaps idiosyncratic, region in the space of possible states of consciousness, as may be the case.
There will be a large part of humanity that will want consciousness/sentience as something that humans have and other things don't (you already see this often with the 'has a soul' argument). This will set us up to have a large blind in to machine systems behavior in self organizing systems, and is a shared concern among those that see AI as a potential existential risk.
Because you say consciousness is a loaded concept I will try to give you a description of what it actually is. I do agree that many people think different things when they talk about consciousness.
It's your inner experience. Oh well, you can probably read this in a philosophy textbook. But what I think it really is, can only be explained with a thought experiment.
Lets say you go into a pitch black sensory deprivation tank with salt water, so you can float in the water. Doctors give you injection that makes you forget about who you are, what is your name. You also forget about how to talk, you know no concept of the languge. You simply temporarily forget about you and what makes you really YOU in this world. The cemical in your blood also deprives you of any physical feeling, you don't feel any water around you. All there is to do is to gaze into darkness. And still...you are conscious, you have the inner experience. There is something like to be you, even in that state.
This particular study isn't limited to LLMs.
It is about how to test consciousness in a wide variety of things. We have no tests.
People may think their dog is conscious, but we can't test that. We can't even test if other people are conscious.
What about invertebrates? Bacteria? Rocks, atoms, AI, your laptop etc? While somebody might say "yes, no, definetly not, yes, maybe" ... still, no tests.
And consciousness in this particular proposal deals with whether or not it "‘feels like’ something to be the target system".
>The second reason is that LLMs already meet the criteria for what most people technically define as consciousness.
An LLM is an inert pile of data. You could perhaps argue that the process of inference exhibits consciousness, but the LLM itself certainly doesn't. There's simply no there there.
I think a lot of Computer Science people need to go read Kant
>Five years ago people were in agreement that the turing test was a really good test
The limitations of the Turing test have been discussed for much longer than that. Ironically its dependence on human judgement is arguably a weak point.
An LLM, in its perfect form is basically a p-zombie. There is no consciousness there. In its current form, its not much more than a really big and fast spreadsheet :)
> The first reason is that consciousness is just a loaded concept. It's some arbitrary category with fuzzy boundaries
It is right now. We might one day devise a mechanistic explanation for consciousness though, in which case any system that follows that mechanistic process would be conscious.
> AI already meets the criteria for sentience the way we defined it from 1950-2020.
If that is the case why hasn't Kurzweil already won the bet? https://longbets.org/1/
I think you're confusing intelligence with consciousness
Well, you can use some tricks - ie. hook it up to your brain, let it learn for a while and tell us how it feels when we flip the switch to OFF.
I've noticed that conversations about "consciousness" tend to go in circles because the participants are using different definitions of the word without realizing it.
Some people use the word "conscious" almost interchangeably with terms like "intelligent", "creative", or "responds to stimuli". Then people start saying things like LLMs are conscious because they pass the turing test.
However, others (including the authors of this paper and myself) use the term "consciousness" to refer to something much more specific: the inner experience of perceiving the world.
Here's a game you can play: describe the color red.
You can give examples of things that are red (that other people will agree with). You can say that red is what happens when light of a certain wavelength enters your eyeball. You can even try saying things like "red is a warm color", grouping it with other colors and associating it with the sensation of temperature.
But it is not possible to convey to another person how the color red appears to you. Red is completely internal experience.
I can hook a light sensor up to an arduino and it can tell me that an apple is red and that grass is not red. But almost no one would conclude that the arduino is internally "experiencing" the color red like they themselves do.
While the paper is using this more precise definition of consciousness, it seems to be trying to set up a framework for "detecting" consciousness by comparing external observations of the thing in question to external observations of adult human beings, who are widely considered by other adult human beings to be conscious entities [1]. I don't see how this approach could ever produce meaningful results because consciousness is entirely an internal experience.
[1] There is a philosophical idea that a person can only ever be sure of their own consciousness; everyone else could be mindless machines and you have no way of knowing (https://en.wikipedia.org/wiki/Solipsism). Also related is the dead internet theory (https://en.wikipedia.org/wiki/Dead_Internet_theory).
>But it is not possible to convey to another person how the color red appears to you. Red is completely internal experience.
Let's say in the future we're able to engineer brains. Let's say we take a person and figure out how their brain fires/operates when it perceives a color and we manipulate another person's brain to mimic the firing. Finally, let's say we're able to show, in the end, that the two people have equivalent internal (neural) responses to the color. We've then "conveyed" one person's experience of perceiving the color to another. Why not?
We don't fully understand our biology and our brain, but at the same time we speculate that our experience somehow can't be manipulated scientifically? Why?
I think the interesting discussion here is as you're putting it, consciousness, the subjective experience of living and feeling. These are not requirements for intelligence or any physical process, and yet it is an indisputable fact that it exists.
The only conclusion I can make is that there is indeed a non physical reality.
> There is a philosophical idea that a person can only ever be sure of their own consciousness; everyone else could be mindless machines and you have no way of knowing
A while back I realised there must be at least two: me, and the first person who talked or wrote about it such that I could encounter the meme.
In principle all the philosophers might be stochastic parrots/P-zombies from that first source, but the first had to be there.
(And to pick my own nit: technically they didn't have to exist, infinite monkeys on a typewriter and/or Boltzmann brain).
That is exactly correct.
I would only add that we attribute consciousness to our fellow humans, because we perceive them to be creatures like us from what we can observe about their physical bodies and behaviors being similar to ours.
With AI, it is much less intuitive to assume creations we know to have arise from very different origins than ourselves have the same kind of interior experiences we do. Even if the surface behavior is the same.
I'm genuinely not certain how your definition of consciousness is distinct and different from 'responds to stimuli'.
I'm pretty sure that the subjective experience of colors is mostly due to a combination of the overlapping ranges of wavelengths our eye's cones respond to (how similar different colors appear to us), and associative recall ("grass green").
https://en.wikipedia.org/wiki/Cone_cell
Note that subjective perception of color is only loosely related to the actual frequencies of light involved.
Try loading the image of these "red" strawberries into GIMP/Photoshop, and use the color picker to see what color they really are - grey.
https://petapixel.com/2017/03/01/photo-no-red-pixels-fascina...
This is the problem with the whole debate
Nobody has ever actually defined an empirical and falsifiable set of hypotheses about how to define “consciousness”
Half of the field is exactly this, and why the link in question exists
It’s an incoherent question
The solipisist can't find reason to form agreements with others. Others are mindless in his view.
He can't define consciousness in terms of what we agree, there's nobody to agree with.
So the game of describing the color red to others, cannot be played to any meaningful end. Red is red to the solipsist.
Coming up with your own interpretation of consciousness is an ability truly conscious people have.
It can never be completely agreed upon in a philosophical conversation without dogma or compromise.
Both solipsism and total agreement, cannot be truthfully used as philosohical tools to contain consciousness.
> There is persisting uncertainty about when consciousness arises in human development, when it is lost due to neurological disorders and brain injury,
Not just when it arises during development or when it gets reduced due to disorders and brain injury, but also at it fluctuates under a range of other known (and possibly unknown) states such as when you're under general anesthesia[1].
Besides the when, there's also kind/degree/nature, such as during meditation, sleep, influence under substances, etc.
Exciting field.
At the core, we are pure awareness devoid of any object (thoughts). We can direct this awareness which we call the art of attention. We're pure awareness and we decide to attend something.
However, to attend and receive information from the physical plane, we need appropriate instruments. Brains and its organs of senses are those instruments. If these instruments have fault, obviously we as awareness don't receive enough information and it seems our consciousness is reduced.
In waking state, we receive info from physical senses. In dream state, these senses are in suspended mode but mind is active in imagining the experiences in a virtual world and hence awareness has an instrument.
In deep sleep, even mind is in a state of rest and hence awareness has almost no active instrument. Still, we awareness does exist and thus we know that we had a good or bad sleep when we wake up.
Arguing that an AI chatbot is conscious is like ancient romans asking "Is a really big fire the same as lightning?" They both burn down big trees and can kill people. Thus, they must be the same! A time traveller would tell them they don't understand lightning and they won't for 2000 years and trying to understand it through philosophizing is only going to some sort of random conjectures like the best way to avoid lightning is to burn down all the nearby trees as big fires need wood to burn, therefore lightning needs trees to strike.
Does this imply the discussion and growth related to learning these facts are useless?
They did understand it's different, didn't they?
I mean, by the time of their empire, several Greek already understood it. I have no idea about how widespread that knowledge was, but every time I look into science spread on those times, I discover the answer is "widely".
It's becoming clear that with all the brain and consciousness theories out there, the proof will be in the pudding. By this I mean, can any particular theory be used to create a human adult level conscious machine. My bet is on the late Gerald Edelman's Extended Theory of Neuronal Group Selection. The lead group in robotics based on this theory is the Neurorobotics Lab at UC at Irvine. Dr. Edelman distinguished between primary consciousness, which came first in evolution, and that humans share with other conscious animals, and higher order consciousness, which came to only humans with the acquisition of language. A machine with only primary consciousness will probably have to come first.
What I find special about the TNGS is the Darwin series of automata created at the Neurosciences Institute by Dr. Edelman and his colleagues in the 1990's and 2000's. These machines perform in the real world, not in a restricted simulated world, and display convincing physical behavior indicative of higher psychological functions necessary for consciousness, such as perceptual categorization, memory, and learning. They are based on realistic models of the parts of the biological brain that the theory claims subserve these functions. The extended TNGS allows for the emergence of consciousness based only on further evolutionary development of the brain areas responsible for these functions, in a parsimonious way. No other research I've encountered is anywhere near as convincing.
I post because on almost every video and article about the brain and consciousness that I encounter, the attitude seems to be that we still know next to nothing about how the brain and consciousness work; that there's lots of data but no unifying theory. I believe the extended TNGS is that theory. My motivation is to keep that theory in front of the public. And obviously, I consider it the route to a truly conscious machine, primary and higher-order.
My advice to people who want to create a conscious machine is to seriously ground themselves in the extended TNGS and the Darwin automata first, and proceed from there, by applying to Jeff Krichmar's lab at UC Irvine, possibly. Dr. Edelman's roadmap to a conscious machine is at https://arxiv.org/abs/2105.10461
I find it amusing that they launch into a discussion of the "urgent need" for consciousness tests without ever stopping to define what they mean by consciousness!
The word seems to be heavily overloaded and refer to a bunch of unrelated phenomena, with most people disagreeing over exactly what it means, or more often just breezing past any definition of what it means and proceeding to argue about it regardless.
I find this comment amusing, as a core premise of the article is trying to figure out how to define and measure consciousness and the problems that entails
Consciousness & intelligence are orthogonal. It’s highly plausible we achieve superintelligence before we have conscious machines.
That said, understanding consciousness is not necessarily a prerequisite for engineering it. It may very much end up being an emergent phenomenon tied to sensory integration & processing, in which case it ends up self-assembling under the right circumstances. Exciting times…
So my thermostat is 'intelligent', just not 'conscious'?
If consciousness can't even be precisely defined, imagine devising a "test" for it.
Laudable effort, but I don't see any progress resulting from it.
The tests themselves are what bring precision to the notion of consciousness. Think of a medical example in which doctors are meant to determine whether or not a patient receives anaesthesia—surely a more effective empirical toolkit would be valuable in that situation, no?
There seems to be a tendency with the discourse around consciousness to slip into vague philosophizing and thus run into the hard problem. I feel like if it doesn't make sense to think of it that way then don't think of it that way.
> appear to
Funny thing is you could absolutely say the same thing about every other human on the planet. There is just no way for you to confirm that I’m having a subjective experience. I might be acting like I am. And so might everyone else. The only one you can be absolutely sure of is yourself.
We all just sort of assume that other people have consciousness but it’s impossible to know for sure.
For someone interested in deep questions about consciousness.
[1] A 25 year wager between a Philosopher and Neuroscientist where Philosopher won (Chalmers vs Koch)
[2] A nice debate between Kastrup and Koch, where it looks like Koch has abandoned his physicalist beliefs
[1] https://www.scientificamerican.com/article/a-25-year-old-bet...
I have a tangential question about non-human consciousness.
It is almost impossible for a human to perceive color red as colour blue. But this won't be a problem for a futuristic humanoid. It would just be some kind of relabelling or a latent space transformation. Now imagine a AI virus that causes such reassignment of distinct colour values say in self driving scenario causing traffic accidents. Have people thought about it ?
Robots can operate at the level of spectra (spectrophotometry) rather than color (projections of spectra, colorimetry). The mapping from the spectral power distribution to color co-ordinates for a given species is well understood and not arbitrary. The mapping from those co-ordinates (colors) to their names is also not arbitrary.
The only issue I can think of that would cause confusion, which has nothing to do with AI, is metamerism, which is when different spectra are perceived to be the same color, because information gets lost when the infinite-dimensional spectrum is projected down to three dimensional color. In practice it is not a problem; most people are not even aware of the phenomenon.
Not really true.
You are asking that humans perceived red as blue, but I can imagine red things as blue.
But if you did not perceive things as red, how would you know to relabel them as blue.
The robot in this case would undergo the same operation when relabeling. It first has to perceive red, relabel to blue. No major difference unless you swap at hardware level which would be analogous to genetically engineering our eye cones to fire at altered wavelengths
>It is almost impossible for a human to perceive color red as colour blue.
Eh, you mean like humans seeing the color green as the color red? Not sure why red/blue matters at all in this case as stoplights are red/green, one of the highest risk of human failure color combinations!
Furthermore direct color encoding of safety controls tends to be coupled with shape or placement. Stop signs are octagons. Stop lights have a directional component.
If you're worried about an AI virus, I'd be far more worried about it just directly crashing the car, then induced confusion crashing the car.
Relation between consciousness and intelligence seems to be a problem for everyone including the authors of the article.
Hmm... Peter Watts, Blindsight.
The novel explores themes of identity, consciousness, free will, artificial intelligence, neurology, and game theory as well as evolution and biology. [Wikipedia]
Especially interesting, and contains a long list of literature.
> Relation between consciousness and intelligence seems to be a problem for everyone including the authors of the article.
Which parts of the paper gave you the impression the authors struggled with this?
FWIW, Anil Seth from the list of authors has been a strong opponent of the Panpsychist view (consciousness is a fundamental property of matter, and everything is to some degree conscious) recently been popularized by David Chalmers et al.
I'm not sure what to make of this comment. Can you elaborate on why you felt this was important to point out?