It surprises me that this works well enough to be useful. I would have thought that network latency, being orders of magnitude higher than memory latency, would be a huge problem. Latency Numbers Everyone Should Know: https://static.googleusercontent.com/media/sre.google/en//st...
back
4 comments
I'd be surprised if this works for anything latency sensitive over anything more than a LAN.
Even just the time it takes speed of light between NY and LA (410^6m/310^8m/s=1/75s) is roughly how long a 60 fps frame is (1/60s). Add OS serializing the frame from the GPU onto the network card, network switching of those packets, and you're starting to really feel that latency.
There are people out there gaming at 30fps with their TV set to Super Duper Image Processing Mode 500ms Latency Edition. Though I suppose these are realistically already served by the cloud gaming offerings.
Yeah, I've tried with shadow.tech and you can feel the latency. There's enough throughput to get a quality video stream through but there's enough latency to feel annoyed. I only play sandbox games though so I imagine it'd be worse with something competitive.
The datacenter is probably not thousands, but hundreds of kilometers away so there is room to deliver 60fps. I was surprised how well GeForce Now works.
With 5G clouds now offers to deploy services to the small datacenters near 5G customers. In some cases less than 1 mile away.
Excuse me but does my 5G cloud have GPUs to spare right now, I could really use some shade.
I don’t understand what 5G has to do with this.
For gaming, this is obviously a no-go. But for bunch of AI/ML related workloads, it might make perfect sense.
GPUs usually run on big command buffers that are generated in a streaming fashion and then submitted at specific points, so it's theoretically possible that a game could hit 60fps this way. You'd just be eating extra latency between command buffer submission and actual rendering.
Here is a video of DOOM 2016 running at 60 fps. https://discord.com/channels/755570806397993111/755570806397...
Not so sure about no-go. The amount of GPU latency in modern AAA titles already approaches 20+ms in the most egregious cases.
Unless there is a need to evict all gpu memory on every frame, I think it is feasible to game on GPUs that live across a very fast LAN.
Fast ethernet is getting cheaper than ever - you can easily get 10gb on consumer gear or even 20 and used hardware on I believe 40 or maybe 100 is getting pretty affordable.
In my experience 40G gear can often be had for cheaper than 10G. I have a pair of Mellanox 40G Infiniband cards that cost me about $20 each on ebay and could be turned in to Ethernet cards with a few commands.
Output screencast could be encoded right in GPU pipeline
About 10 years ago I found set operations in ruby were slower then set operations in redis. So I shipped all my data over the network - let redis sort into a sorted set and then crunched my data in redis - retrieving it again over the network in its reduced form… I think it makes sense that for vector operations a remote gpu could be pretty cool. Now if we can get this working from MacBooks to Linux gpus I’d be pretty stoked
PCI-Express 16x 4.0 has 31,5 GByte/s. The fastest fiber ETH has 50 GB/s. So it "could" be useful if you have datacenter grade equipment ;)
Those aren't latency numbers though, they're throughput.
Since we have full visibility into the pipeline our (Juice labs) Weissman scores are off the charts!
Let me help you.
1) Take off the glass
2) Use a drill, make the connect wider
3) now bundle 9 glass and put in hole
4) now bandwidth is more wide
Fasterness!
Video and tensors could be compressed before transmission.