▲ 41 points
back
31 comments
I really wish we could talk about new kinds of hardware without the comparison to the human brain. SNN models capture some characteristics of biological neurons ... but those attributes aren't at all specific to humans.
Representing the way information is represented during computation (via spike rate-coding or whatever), using far less power, changing the interaction between computation and memory -- all of these can be exciting and interesting without pretending that it's closely related to human brains.
I do think there's some value in recognizing when people create something directly inspired by looking at how the brain works and trying to mimic it with hardware.
Whether or not that's what's happening here is another matter.
... but the thing being mimicked is just as true of a mouse brain or (I think?) any vertebrate brain.
Then the article title should probably be "mimicking biological brains"!
Oh well.
Apologies for the quack philosophy, but isn't it just that we as human observers of the world can't help but identify our reflection in everything we see or do?
or, conversely, a quack marketing approach: the more you can relate a product to an existing successful product, the more people will think it's a good idea.
According to this table [1] the 1.15B artificial neurons they're claiming is just at little below the estimate for raccoon dogs and Tanimbar corella / cockatoo. Even if it's a technically impressive system, "inspired by the racoon dog brain" doesn't make for as good a press release.
[1] https://en.wikipedia.org/wiki/List_of_animals_by_number_of_n...
Well, even then, it's massive progress that we've reached the realm of mammals. Only 2 more orders of magnitude to go.
It's not even close to a mammal. It might have 1bn artificial neurons, while a cat has 700m biological neurons. But
a) individual mammal neurons are far more computationally complex than any existing artificial neuron, even in SNNs.
b) a cat has 10 trillion synapses compared to the 100 billion artificial synapses in Intel's computer. So even ignoring how simple artificial neurons are, mammal brains have far more topological complexity.
But semiconductors operate so much faster than neurons that it is hard to be confident that AI hadware has not already caught up to cat brains.
>> all of these can be exciting and interesting without pretending that it's closely related to human brains
Why do you say they are not closely related? They are certainly an analogy to parts of the human brain. They are inspired by the way real neurons work at some level. But using the word "pretending" you sound as if you think these similarities are pure fiction. Why is that?
I have three basic problems with using "brain" and especially "human brain":
1) Nobody is even bothering to explain why this is like a human brain rather than 1M spider brains or 1B nematode brains - I suspect 1M spider brains is probably closer to reality, and I doubt that adds up to one human. The choice of "human" is a marketing gimmick.
2) Even in nematode brains, each neuron is very different, in a way that can't be explained by a small array of floating-point numbers. These SNN architectures are too homogenous and simple to explain the actual behavior of individual neurons. (See https://www.quantamagazine.org/how-computationally-complex-i... which discusses standard ANNs, but I strongly doubt a single layer of SNN neurons can actually emulate an individual rat neuron.)
3) Epigenetic effects are clearly important for thinking and memory formation, yet totally ignored by these architectures: https://www.smithsonianmag.com/smart-news/scientists-say-the...
Because it’s nowhere close to the scale of the human brain, calling it a mouse brain would be closer. People have 100x the neurons and 10,000x the synaptic connections and wildly different architecture, so it’s really got nothing in more in common with a human brain over some other kind of brain.
Remove the human brain part and try to explain why this is news to a layperson.
I'm not an expert in these systems, and their announcement materials don't make a bunch of stuff clear, but my attempt at summarizing for a layperson would be:
- Current computers represent values as patterns of binary values. Analog computers sometimes represent values as e.g. voltages within a range. SNNs represent values with the timing of "spike" events. You could think of this like if you and your friends arranged a scheme to communicate using only the original Yo app, just by varying timing and frequency.
- Current computers move values back and forth between memory and the CPU, and has very little room to hold data "close" to the CPU. Suppose your e-reader can only have one doc page open at a time; if you need to figure something out by closely comparing values between several docs, you might spend more time opening/closing/finding/paging through docs than actually reading them or thinking. These systems put (some) memory to be held by the same physical components which are doing the computations, so they aren't moving stuff back and forth in the same way.
- However, current computers are quiet general in what they can do. In exchange for speed an efficiency, these systems are not general purpose. For example, b/c they have 1.15B "neurons" and 128B "synapses", any value can only (directly) influence on average 112 other values. You could think of this a bit like if Excel only let your formula reference a capped number of cells.
- But for research problems which fit these constraints, this system can be extremely fast and use less power than a normal computer would.
If I'm LARPing as a layperson then you've lost me in the first sentence.
Human capacity is the benchmark for capacity comparison for every thing humans care about on the planet
For example even though the impulse-striking power or multi spectral vision system of the mantis shrimp is much more capable than human systems, it is not a benchmark for any of our systems
Therefore we’re going to continue to evaluate everything. On human action until we create something that surpasses it.
Comparing it to the brain, I agree is silly
comparing them to human capabilities however, (which is effectively what people are saying when they compared to the brain, even though they don’t know that) is very reasonable
this will never happen. why? because humans are outrageously self-centeredly narcissistic. so narcissistic that we invented religions where god is just an omnipotent/omnipowerful version of a man, and then we go "god created man in his image" to justify it.
humans can't help but be obsessed with other humans, the same way every species is hardwired in a specific way to relate to their own species.
the idea of any computer system "brain" not being compared and contrasted to our own is against our very narcissistic nature.
“In neuromorphic computing, however, a "spike input" — a set of discrete electrical signals — is fed into the spiking neural networks (SNNs), represented by the processors. Where software-based neural networks are a collection of machine learning algorithms arranged to mimic the human brain, SNNs are a physical embodiment of how that information is transmitted. It allows for parallel processing and spike outputs are measured following calculations.
Like the brain, Hala Point and the Loihi 2 processors use these SNNs, where different nodes are connected and information is processed at different layers, similar to neurons in the brain.”
My impression is that there hadn't been much success with SNN's yet.
You are one though.
It's a bit cargo cult-y to claim that synthetic SNNs and human brains are equivalent just because they share some similarities.
Isn't a human brain a spiking neural network? And a successful one at that? I didn't say what we can build now is identical to a human brain (...why would anyone say that?), I said we have proof that the concept is viable, because a spiking neural network is literally the thing reading these words right now.
Having checked a mirror, I stand by my initial assessment.
My point was that humans are arguably quite successful, their success is due to their neural networks, and their neural networks are spiking (in contrast to the ones used in LLMs).
The spiking might be important, or it might not. (One way to find out!) But we certainly have proof that it can be made to work!
Last time I checked SNNs still don’t work as well as traditional NNs. Why are they working on hardware before getting software to work?
Traditional NNs didn't work very well either, until we were able to run them at massive scales. That's why they didn't see much adoption or use until the late 2000s/early 2010s. SNNs are much more difficult to run at scale, but that difficulty is solved with analogue computers[1][2]. We don't have details, but I'd bet this is just Intel's crack at a modern analogue processor, with some design concessions specifically for running SNNs.
Loihi is a digital chip.
SNNs are behind, but not that far behind, especially after recent astrocyte-related advancements, and if implemented correctly in hardware they consume a fraction of power. SNNs don't get nearly the same amount of attention as ordinary networks largely because they lack efficient hardware to run on (just like ordinary NNs until the advent of GPGPU).
It's not exactly new hardware though, it's just a cluster of 1152 existing Intel Loihi 2 neuromorphic chips.
Which “recent astrocyte-related advancements”?