▲ 180 points
back
141 comments
I believe that training a system to understand the electrical signals that define a movement is significantly different from a system that understands thought.
I work in neurotech, I don't believe that the electrical signals of the brain define thought or memory.
When humans understood hydro-dynamics, we applied that understanding to the body and thought we had it all figured out. The heart pumped blood, which brought nutients to the organs, etc etc.
When humans discovered electricity, we slapped ourselves on the forehead and exclaimed "of course!! it's electric" and we have now applied that understanding on top of our previous understanding.
But we still don't know what consciousness or thought is, and the idea that it is a bunch of electrical impulses is not quite proven.
There are electrical firing of neurons, absolutely, but do they directly define thought?
I'm happy to say we don't know, and that "mind-reading" devices are yet un-proven.
A few start-ups are doing things like showing people images while reading brain activity and then trying to understand what areas of the brain "light-up" on certain images, but I think this path will prove to be fruitless in understanding thought and how the mind works.
Agree completely. The brain is so incredibly complex that we've barely scratched the surface. It's not just neurons, which are very complex and vary wildly in genetics between them - it's hundreds of other helper cells all interacting with each other in sometimes bizarre ways.
To try to boil down it all to any simple signal is just never going to work. If we want to map consciousness it's going to be as complex as simulating it ourselves, creating something as dense and detailed as a real brain.
We know that the brain is a structure that works through electrochemical reactions. Synapses transmit signals sent by axons to neurons. We can test this. We can measure it. There's nothing else going on that we can describe using known science.
Ah, we might say, maybe there is an unknown science - we did not know about so much before, like electricity, like X-Rays, like quantum physics, and then we did, and the World changed.
The difference is that we observed something that science could not explain, and then we found the new science that explained it, and a new science was born.
It's pretty clear to me - but you may know more - that we can explain all brain activity through known science. It might be hard to think of us as nothing more than a bunch of electrochemical reactions in a real-world reinforcement learning system, but that's what we are: there's no gap that needs new science, is there?
> There are electrical firing of neurons, absolutely, but do they directly define thought?
Well, surgeons and researchers have shown that electrical stimulation of certain brain regions, can induce "perception" during procedures. They can make a patient have the conscious experience of certain smells, for instance.
It's not conclusive proof of anything, but I wouldn't bet against us getting closer to the mark, than we were when we only considered hydro-dynamics as the model.
> I don't believe that the electrical signals of the brain define thought or memory.
Yes and no. It'll be something like a JPEG file. You can have a JPEG file that contains an image of a cat. But give that file to someone who has no clue about JPEG encoding and the file looks like random noise. They'll take 100 years to figure out it's an image of a cat.
Actually it's like if you take an electron beam prober to one of the NVidia AI GPU chips while it's figuring out whether it likes Wordsworth poetry.
You say you don’t believe something is true and then say you don’t know, but I’ll disagree with “electrical signals don’t “define” (encode) thoughts.
To be clear, of course it’s true that our thoughts are more than just electrical activity. The brain is a system. However, it seems clear that thoughts are at least partially encoded in electrical activity.
What you mentioned those startups will find fruitless, that’s already been done for years in a research setting. It may not be a successful business model, but it’s already been demonstrated.
There are fMRI studies and electrical measurement studies. You could argue fMRI decoding of images is not electrical activity which is true, but a bunch or work shows they are strongly correlated.
For electrical activity alone we’re already decoding information like words, so it’s hard to claim electrical activity doesn’t define thoughts.
Maybe you mean to say, doesn’t define all the content of our thoughts which is a much different claim.
For me it's like attaching wires to CPU and trying to decipher what youtube video is being playing right now.
Absolutely not possible.
One thing is probably true: You have to train on the individual person, and it’s not transferable to a different person. Similar to how when taking an LLM and training on the fluctuations of its neural network to “read its thoughts”, the training results won’t transfer to interpreting the semantic contents of the network activity of a different LLM.
So you probably can’t build a universal mind-reading device.
This sounds logical and convincing.
At the same time, it should also be easy to falsify.
Has there been an experimental setup like this tested? If I’m not mistaken it should falsify your claim.
Train a decoder on rich neural recordings, then test it on entirely new thoughts chosen under blinded conditions.
If it can still recover the precise unseen content from signals alone, the claim that electrical activity is insufficient is overturned.
seems like trying to take a single pixel signal (so to speak) and interpolate entire image out of it.
I thought it was pretty established by now that it is likely that other parts of the body participate in both memory and thought, a fully distributed system?
Does it make sense to think of thoughts, consciousness etc. as an emergent property of the neuronal activity in our brains?
This is silly. It's the sum of electrical and chemical network activity in the brain. There's nothing else it can be. We've got a good enough handle on physics to know that it's not some weird quantum thing, it's not picking up radio signals from some other dimension, and it's not some sort of spirit or mystical phlogiston.
Your mind is the state of your brain as it processes information. It's a computer, in the sense that anything that processes information is a computer. It's not much like silicon chips or the synthetic computers we build, as far as specific implementation details go.
There's no scientific evidence that anything more is needed to explain everything the mind and brain does. Electrical and chemical signaling activity is sufficient. We can induce emotions, sights, sounds, smells, memories, moods, pleasure, pain, and anything you can experience through targeted stimulation of neurons in the brain. The scale of our experiments has been gross, only able to read and write from large numbers of neurons, but all the evidence is consistent.
There's not a single rigorously documented phenomenon, experiment, or any data in existence that suggests anything more than electrical and chemical signaling is needed to explain the full and wonderful and awe-inspiring phenomenon of the human mind.
It's the brain. We are self constructing software running on 2lb chunks of fancy electric meat stored in a bone vat with a sophisticated network of sensors and actuators in a wonderful biomechanical mobility platform that empowers us to interact with the world.
It explains consciousness, intelligence, qualia, and every other facet and nuance of the phenomena of mind - there's no need to tack on other explanations. It'd be like insisting that gasoline also requires the rage of fire spirits in order to ignite and power combustion engines - once you get to the point of understanding chemical combustion and expansion of gases and transfer of force, you don't need the fire spirits. They don't bring anything to the table. The scientific explanation is sufficient.
Neocortical networks, with thalamic and hippocampal system integrations, are sufficient to explain the entirety of human experience, in principle. We don't need fire spirits animating cortical stacks, or phlogiston or ether or spirit.
Could spirit exist as a distinct, separate phenomenon? Sure. It's not intrinsic to subjective experience, consciousness, and biological intelligence, though, and we should use tools of rational thinking when approaching these subjects, because a whole lot of pseudo-scientific BS gets passed as legitimate scientific and philosophical discourse without having any firm grounding in reality.
We are brains in bone vats - nothing says otherwise. Unless or until there's evidence to the contrary, let that be enough.
From some dystopic device log:
[alert] Pre-thought match blacklist: 7f314541-abad-4df0-b22b-daa6003bdd43
[debug] Perceived injustice, from authority, in-person
[info] Resolution path: eaa6a1ea-a9aa-42dd-b9c6-2ec40aa6b943
[debug] Generate positive vague memory of past encounter
This reminds me of "Upgrade" – a sci-fi movie about a paralized man who gets an AI brain implant, which can move his body for him. It's pretty decent.
Can you imagine having chatgpt in your brain to constantly police wrongthink? Would save the British media a job.
When it infers illicit intent, it "corrects" you by biasing the output: a misclick here, a poisoned verb there... phantom intention drift™ injected into your parietal lobe milliseconds before your conscious even boots.
You should make text based game
Split brain experiments show that a person rationalizes and accommodates their own behavior even when "they" didn't choose to perform an action[1]. I wonder if ML-based implants which extrapolate behavior from CNS signals may actually drive behavior that a person wouldn't intrinsically choose, yet the person accommodates that behavior as coming from their own free will.
[1]: "The interpreter" https://en.wikipedia.org/wiki/Left-brain_interpreter
Split brain experiments have been called into question.[0]
[0]: https://www.sciencedaily.com/releases/2017/01/170125093823.h...
AI following the Libet ([0]1983) paper about preconscious thought apparently preceding 'voluntary' acts (which really elevated the question of what 'freewill' means).
The prima facie case for free will* is that it feels free. If you can predict the action before the feeling it removes that argument (unless you want to invoke time travel as an option)
*one of the predominant characterisations of free will, anyway. I'm a compatiblist, so I have no issue with caused feelings of decision making being in conflict with free will. I also have a variation of Tourette's, so I have a different perception of doing things wilfully when compared to most people. It's really hard to describe how sometimes you can't tell if something was tic or not.
That it precedes voluntary acts tells us that most of what we do are not conscious. Which has been known for over a century, maybe millenia.
(opinion stolen from some Chomsky video)
Well, what does freewill mean to scientists?
> is it time to worry?
Shouldn't the device be the judge of that?
I think the real danger lies in how many will accept that output as the unadulterated unmistakable truth for actions, for judgment. Talk about a sinister device.
You don’t need a sinister device. This is essentially how propaganda works.
Ok does anyone else’s mind just immediately go to “The Minority Report” is soon going to no longer be just a sci fi dystopia?
For me, it immediately made me think of Psycho-Pass.
It’s a cyberpunk anime where society uses a system called the Sibyl System to constantly scan people’s mental states and “crime potential” (their Psycho-Pass).
People can be arrested before they’ve done anything - just because the system picked up certain signals from them.
Very, very interesting idea
yes, first thing i thought of. although i'm quite confident it's still outside the scope of our lifetimes, i do worry for future generations
Rather than the Karpathy thing about in class essays for everything, maybe random selections of students will be asked to head to the school fMRI machine and be asked to remember the details of writing their essay homework away from school.
fMRI machines are not cheap, nor plentiful.
If, one day someone can make a small, cheap device that can do the job of a fMRI it would be more world changing than you can imagine. If you had easy access to realtime data about what is going on in your brain, there is evidence to suggest that you can learn to influence the data and literally change your own mind.
It's interesting that the path from 'decide to do something' to performing the action is hundreds of ms long. It's also interesting that grabbing the data early in the process and acting on it can perform the action before the conscious 'self' understands fully that the action will take place. It's just another reminder that the 'you' that you consider to be running the show is really just a thin translation layer on top of an ocean of instinct, emotion, and hormones that is the real 'you'.
I rather prefer the holistic take that we are our whole selves and not just the part that reflects on what we do or the part that reacts to external and internal material stimuli. We know we can change the instincts, emotions, and hormones when they conflict with what we know by reflection to be just and good. To put it another way, we know that we can do things "without thinking" that are either just or unjust and by reflection can achieve some level of mastery over the direction of our impetuses.
> That learning enabled the system to detect her intention to play hundreds of milliseconds before she consciously attempted to do so, says trial leader Richard Andersen, a neuroscientist at the California Institute of Technology in Pasadena.
Sounds like Libet's Delay and all that. Conscious awareness is just a documentary covering something that has been decided some half a second ago.
Unlike the vast sea of the subconscious, we can try to take direct control of technology. But we don’t. So we are left to fret about what technology will do to us (meaning: what people will power will use it for).
The Amish take control of technology like that. If it doesn't pass review it's forbidden. It's a pretty good idea.
Consider that heroin used to be legal. We might have a dozen such technologies that we know are hazardous but have yet to kill enough people to force the issue.
I wonder how much this experience is similar to the Alien Hand Syndrome, where people experience that part of their body, usually a hand, act on their own.
Install one of these on every citizen!
I guess I will start paying attention when it can predict word choice in my internal monologue.
Of course, for shallow people in need of validation, doing exactly that is the point of sycophancy. "I have a great idea, I will ask the AI.." followed immediately by "What an insightful question, this gets to the heart of the matter"
I find the take a quirk in how the state of the art assistive technology works is reason for privacy fear mongering to be tired, unimaginative, and typical of today's journalism that cares more for clicks than reporting fact.
It's a very interesting quirk of a immensely useful device for those that need it, but it's not an ethical dilemma.
I for one am sick and tired of these so-called ethicists who's only work appear to be so stir up outrage over nothing holding back medicinal progress.
Similar disingenuous articles appeared when stem-cell research was new, and still do from time to time. Saving lives and improving life for the least fortunate is not an ethical dilemma, it's an unequivocally good thing.
Quit the concern trolling nature.com, you're supposed to be better than that
You don't need a device to do this.
Maybe skulls will need a faraday cage.
20-year old me would’ve been amazed at how much closer technology is getting to Ghost in the Shell. 30-year old me is firmly in the “nope, no thanks” camp.
I love seeing the advancements still, don’t get me wrong, but in the current data, advertising, and attention economies under Capitalism? No fucking way that shit is ending up in my head.
Seems like they are really jumping to conclusions here.
> Smith’s BCI system, implanted as part of a clinical trial, trained on her brain signals as she imagined playing the keyboard. That learning enabled the system to detect her intention to play hundreds of milliseconds before she consciously attempted to do so
There are some serious problems lurking in the narrative here.
Let's look at it this way: they trained a statistical model on all of the brain patterns that happen when the patient performs a specific task. Next, the model was presented with the same brain pattern. When would you expect the model to complete the pattern? As soon as it recognizes the pattern, of course!
> That learning enabled the system to detect her intention to play hundreds of milliseconds before she consciously attempted to do so
There are two overconfident assumptions at play here:
1. Researchers can accurately measure the moment she "consciously attempted" to perform the pretrained task.
2. Whatever brain patterns that happened before this arbitrary moment are relevant to the patient's intention.
There's supposed to be a contradiction here: The first assumption is correct, and the second assumption is also correct. Therefore, the second assumption does not invalidate the first assumption. How? Because the circumstances of the second assumption are a special thing called "precognition"... Tautological nonsense.
Not only do these assumptions blatantly contradict each other, they are totally irrelevant to the model itself. The BCI system was trained on her brain signals during the entirety of her performance. It did not model "her intention" as anything distinct from the rest of the session. It modeled the performance. How can we know that when the patient begins a totally different task, that the model won't just "play the piano" like it was trained to? Oh wait, we do know:
> But there was a twist. For Smith, it seemed as if the piano played itself. “It felt like the keys just automatically hit themselves without me thinking about it,” she said at the time. “It just seemed like it knew the tune, and it just did it on its own.”
So the model is not responding to her intention. That's supposed to support your hypothesis how?
---
These are exactly the kind of narrative problems I expect to find any "AI" research buried in. How did we get here? I'll give you a hint:
> Along the way, he says, AI will continue to improve decoding capabilities and change how these systems serve their users.
This is the fundamental miscommunication. Statistical models are not decoders. Decoding is a symbolic task. The entire point of a statistical model is to overcome the limitations of symbolic logic by not doing symbolic logic.
By failing to recognize this distinction, the narrative leads us right to all the familiar tropes:
LLMs are able to perform logical deduction. They solve riddles, math problems, and find bugs in your code. Until they don't, that is. When an LLM performs any of these tasks wrong, that's simply a case of "hallucination". The more practice it gets, the fewer instances of hallucination, right? We are just hitting the current "limitation".
This entire story is predicated on the premise that statistical models somehow perform symbolic logic. They don't. The only thing a statistical model does is hallucinate. So how can it finish your math homework? It's seen enough examples to statistically stumble into the right answer. That's it. No logic, just weighted chance.
Correlation is not causation. Statistical relevance is not symbolic logic. If we fail to recognize the latter distinction, we are doomed to be ignorant of the former.