▲ 126 points
back
109 comments
The article doesn't note this, but wafer scale integration is a very old idea. We discussed it at Inmos back in the day, since often the systems we built essentially consisted of many CPU die sliced out of the wafer, bonded into a package, then tiled onto a PCB[1]. But there are...issues: cooling for one. Iann Barron joked that you could make a toaster from two WSI wafers running full-tilt.
That is super cool. I wrote to SGS-Thomson when I was in highschool and they sent me what felt like a refrigerator box of manuals for the 400 through the 9000. Huge transputer fan. The simplicity was striking, and that you could construct a system that could handle huge numbers of threads communicating across the network with almost no "system software" in the way was mind blowing.
Most famously, Gene Amdahl started https://en.wikipedia.org/wiki/Trilogy_Systems in the 80s to explore this idea.
The calculus changes when you don't have to dice the wafer, packaging, etc. I'd say that we clock things now at the highest speed that we can safely remove heat, so these wafer scale chips, we trade frequency for area and need/should clock them much slower.
At large production runs, the wafer in a Cerebras is 20k each for a system costing millions and the primary engineering feat is still cooling. I'd love to see a WSI system utilizing NTV (near threshold voltage) logic.
https://semiengineering.com/near-threshold-computing-2/
Another interesting design pattern that has arisen is that Cerebras, Esperanto, Tenstorrent, and InspireSemi are all mesh networks using message passing.
What kinds of things did you work on at Inmos?
>> But there are...issues: cooling for one
> That is super cool.
...so apparently not... ;)
I worked on board level designs and interfaces and memory subsystems among other things.
The traditional computer chip will have a power draw proportional to frequency and the square of voltage, which itself must be increased to control delay and raise frequency:
P ∝ C×V²×f
One difficulty would be that modern technological nodes have high leakage and dissipate power even when they are not switching, making it more advantageous to clock aggressively, finish the computation as fast as possible and cut the power on that entire circuit for the remainder of the timer slot ("race to idle"), as opposed to reducing the frequency and prolonging the "on" phase.
But that's a deliberate design choice, knowing the chip will be cut out and fitted with a substantial thermal solution. Wafer level power draw it's definitely something you can control at the design stage.
I wonder when we'll convert to plasma computing, as in the entire instruction set and operators running as a waveform in a condensed cloud of plasma where the voltage outs at specific points equal the computational result of the inputs?
If we could do that then we could run terahertz frequencies.
Yes, I was going to say I thought this had been looked at with the transputer.
I also recall Clive Sinclair suggesting this approach in the late 80s (can't recall if that was somehow related to the transputer or was completely separate). I believe his idea was that the faulty CPUs that naturally exist due to wafer defects would be cut off from the main group (I could have misremembered but I think it may have been via some kind of self test process).
It looks like they were close to launching a product! https://qlwiki.qlforum.co.uk/doku.php?id=qlwiki:sinclair_waf...
RAM, not CPU, but IIRC he was talking about CPUs too.
and later there was a prototype storage product:: https://www.computinghistory.org.uk/det/3043/Anamartic-Wafer...
There’s been quite a few. I can’t remember the first one I found which was also heavily analog. Here’s another:
https://www.kip.uni-heidelberg.de/vision/previous-projects/f...
The HICANN chip is really interesting, never seen or heard of it before!
Not exactly a wafer but once I tried to create a very powerful LED bulb by combining LEDs of multiple 10W LED bulbs close together and yep, it got very hard to cool it down. I wasn't expecting to get that hot but later once I gave it some thought, I said silly me of course it would be hard to cool it down. When you put together multiple elements that heat-up, the radiator-to-heater ratio quickly deteriorates.
And now when I've red the title, the heat was the first thing that came to my mind.
Similarly cobweb: https://openaccess.city.ac.uk/id/eprint/28487/1/Anderson%20t...
I'm not well versed in this subject but could you solve the heat issue by running the cores at low voltage and clock speeds?
I was going to say ... pretty sure I saw this on TV in the 1980's.
> you could make a toaster from two WSI wafers running full-tilt.
I'd totally eat compute toast, where can I get such a toaster?
That sounds great... until you get a defect on your networking block.
Then what? You've got a heterogenous network with tons of "this core to this core is not like the others" exceptions (latency, bandwidth, etc).
I know chip-to-chip/memory interconnects burn a ton of power, but fabbing discrete "biggest chip we can get with decent yield" still seems a solid tradeoff in the reality of < 100% yields.
Does anyone have a link or search phrases on how this is currently handled for high-chiplet counts? E.g. interconnection routing architectures that are still reasonable with random manufacturing-time failing links
I assume it's probably an 1800 core wafer and they just cut fuse off the 200 defective cores. Some redundancy is likely just built-in based on the expectations of process reliability.
Probably multiple networking blocks, too, and you'd use less demanding process features on the things that can't be duplicated. In fact you could probably even have FPGA-style soft programmable fabric interconnects to work around process failures.
This is on a 22nm process, which is what Intel Haswell [1] (4th gen core) was using 10 years ago. The latest gen chips are now 7 and 5nm [2], and it seems a lot of the innovation in chip manufacturing is about shrinking this size.
How much is being done to improve yields of these older process sizes, maybe using the improvements done for smaller sizes? Logically it must be possible to have 100% yield on wafers at a certain process size -- but what size is that?
[1] https://en.wikipedia.org/wiki/Haswell_(microarchitecture)
[2] https://en.wikipedia.org/wiki/Microprocessor_chronology#2020...
Core to core latency is already very heterogeneous on existing CPUs. Fusing off a few links isn't the end of the world.
Now that Cerebrus has proven it works, I would love to have an x86 / ARM / NVidia do this. And for best results, onboard one of the memory maker as well. Cerebrus seems to have underestimated memory requirement of LLM. So imagine, 16 H200 GPU along with single digit TB HBM memory stitched together on a single substrate wafer. It seems doable with correct technology.
Go for it China. You are in good track here.
> 16 H200 GPU along with single digit TB HBM memory stitched together on a single substrate wafer
How on earth would you cool this?
>"The latter has only been managed by Cerebras so far, but it looks like Chinese developers are looking towards them as well."
Cerebra's wafer has 850,000 cores which totally dwarves 1600 cores on Chinese wafer. I did read though that Cerebra cores optimized for tensor ops. Does Chinese version have more universal cores or it just way smaller clone of Cerebra?
There were experiments with wafer scale FPGAs in the 1990s. The idea was that being programmable, the final chip could be programmed to route around defects. Lasers were also used to eliminated defective cells.
The concept is interesting.
I guess it will have to be able to route around broken cores?
I've seen other projects that market themselves as "wafer scale". https://www.cerebras.net/product-chip/
> I guess it will have to be able to route around broken cores?
Yeah, but you'll also have to route around broken routes, and that starts to get a bit too much chicken-and-egg problem for me.
I guess you could design something akin to error correction codes, meaning you're resilient to X failures. Ex: a 64-bit bus could have physically Single Error Correction, Double Error Detection, which IIRC would be 72 physical wires.
That means any wire can completely fail, but you still have a 64-bit bus (indeed, the 8x error-correction wires could all fail and you'd still have a 64-bit bus).
------------
At some point, it makes more sense to cut the chips out, test them for reliability. Then cut the router out, and test those for reliability, and then finally glue them together.
On the other hand, doing it all on one wafer has cost savings / manufacturing simplicity. The math is likely difficult for optimizing over costs, production speeds, and so forth.
There are techniques to disable nodes that fail testing, so it shouldn't be a problem (within reason).
The article talks about chiplets. I presume the wafer will still be cut into distinct chips? I thought there were thermal (and yield) reasons to not making chips that are too large.
My take was that they currently have a working chiplet design and are looking to move to a wafer-scale (i.e. not cut up) design. Thermal/power/yield are all issues and the design has to take all of that into account. Cerebras has done it for their NN processors, so it has been done before.
They can use these for their high speed trains!
What about trains requires this level of processing power???
What kind of wattage is expected for this? And how is heat management done?
How the hell are they gonna cool a cpu like that?
Big block of copper. Water pumped through channels. The total heat output isnt all that big comparred to other water-cooled processes. A car engine cylinder pumps out more heat across a similar area. You will just need pumps and fans bigger than the toy parts used in normal pc cooling.
Been hearing it for 30 years.
Finally?
1.7%
What do you mean?
I would have said 1.3.
Those who try may get 1.7%
Those who don't may forever get 0%
> China planning 1600-core chips that use an entire wafer – 'wafer-scale' designs
... and they will cool it by pouring water on it. /s
Pretty much. This is the way Cerebrus does it: https://web.archive.org/web/20230812020202/https://www.youtu...
"China" is "planning" to do this...?
A better title might be: "Researchers in China studying 1600-core chip".
Ok, we've replaced China with some Chinese researchers in the title above.
Software determines whether good hardware succeeds or fails. China has yet to build successful software ecosystems on top of its hardware innovations.
> China has yet to build successful software ecosystems on top of its hardware innovations.
I'm not sure your belief is grounded in reality. I'd go as far as to assert that if China was able to research and develop these chips, both their design and production processes, they certainly are not leaving software as an afterthought.
Nevertheless, even entertaining your fantasy, once these chips are out and people like you and me are able to take these toys out to play with them, you'll soon get software that does something interesting and useful. Software is hardly the hard part, or even costlier.
I'm surprised by the cluelessness of most replies here, given that this is HN.
Hardware is only as useful as the software that can run on it. Radically new hardware requires rewrites of certain layers of that software. Ain't nobody got time for that, unless they can be assured that there will be a large number of companies and customers who need software to run on the new hardware.
Wait what, China has by significant margin the second largest software ecosystem in the world, and in VC terms is comparable to the US.