Item — HN Reader

Currently there's a big problem in GPGPU computing - high latency until the computation results are available. That can be tens of milliseconds. This significantly limits type of tasks you can efficiently offload to GPU. I understand AMD's hUMA/HSA is supposed to address this problem.

But there's another problem: currently CPU memory buses are connected to two or more DDR3 memory channels. And DDR3 doesn't simply have sufficient bandwidth for high performance graphics and GPGPU computing, especially when shared with CPU.

Intel Haswell will have CPU and GPU on-package together with shared 128MB of eDRAM 64 GBps "L4 cache". I believe that should enable low latency high performance memory sharing.

I don't understand AMD's bandwidth story. Does the GPU share one memory controller with CPU and have another private one, for example connected to GDDR5? I don't see how hUMA could work efficiently over PCIe bus either, so I guess hUMA is about APU + CPU only.

How does AMD provide the bandwidth?

3 comments

> DDR3 doesn't simply have sufficient bandwidth for high performance graphics and GPGPU computing, especially when shared with CPU

This is an overgeneralization. Real-time 3d rendering is bandwidth hungry but there are plenty of cache-friendly or latency-bound codes in areas that can benefit from GPGPU.

Though one problem is that we don't yet have a lot of popular GPGPU apps on the mass market, and the ones that we do have exploit the strengths and weaknesses of the old school GPUs (= high bandwidth, limited communication needs with CPU)

AMD currently uses DDR3 for their APUs but the next generation will support DDR3 or GDDR5. The PS4 for example uses an APU with GDDR5 memory.

I think this hUMA/HSA is a value proposition and not meant for high-end graphics or GPGPU, which can easily stream through GBs of data very quickly (high end cards have > 4 GB on card). Even Haswell strikes me as a value product; everything great until your problem doesn't fit into your cache.

The transparent memory hierarchy is still quite expensive, and there are lots of performance benefits, at least at the high end, to managing it yourself.

>I think this hUMA/HSA is a value proposition and not meant for high-end graphics or GPGPU, which can easily stream through GBs of data very quickly (high end cards have > 4 GB on card).

I think the use of it in gaming consoles goes against that a bit. I also don't see why you're combining the architecture with the specific hardware. You're thinking x86 CPU with an embedded-class GPU in it. What happens if they build a high end GPU with a CPU in it? Nothing stops them from putting 4GB of DRAM on the same package as that like they do with GPUs (or did historically with SRAM on Slot-based CPUs). You can certainly imagine a market for both, the later would just be the expensive / high end model of the former.

And if you really, actually need a dedicated GPU for some highly specialized workload, I imagine you'll still be able to buy one. But now you're talking about the market where people currently put four GPUs into one box, which is not exactly mass market.

There are a lot of performance benefits to managing the memory hierarchy yourself. But I think that the demise of the Cell demonstrated that not enough people are willing to do it to justify it as architectural decision.

In the video game market sure, but in the HPC market, CUDA rules.

I was talking about for a single chip. There was a lot of interest in Cell in the HPC market, but it also died.

For applications that can tolerate the latency, offloading computation to the GPUs is such a multiplier in performance that they have to put up with manually managing the hierarchy - but even there, it's a rather coarse memory management.