Item — HN Reader

Help: I can wrap my head around WebGPU for in browser & on device 3D, video, and ML like face or hand tracking, etc.

I struggle to imagine the use cases where WebGPU runs on the edge / servers & makes more sense than other ways to access GPUs. Anyone have a good layman’s example use case?

4 comments

1. You want or have a “serverless” function. 2. You want to add some sort of ML work to that serverless function. 3. The serverless function runs on a maimed browser environment (v8)

Specific examples… you have an API that takes a picture returns if there’s a hotdog in the picture. Or an API to generate images from customer input. Etc.

Why is this better or why does it make more sense? If you’re already using the above environment for your functions it makes sense to keep using it. In this way, you don’t have to manager or provision GPUs at all.

Yeah, but why edge for that? It makes sense for something like Stadia (rip) so i guess that's one example, but is the user experience for hotdog-or-not really improved that much by the extra 50 Ms or whatever of latency that edge gives you over closest core?

Yeaaaaa I don't think most people really need that extra 50ms. I think CloudFlare is trying to build out a developer product, and needs to build all features to catch the long-tail of work. If you're using CF and suddenly you need a trendy GPU, its better for CF to sell it to you before you open a AWS account.

That’s a fair question and I think it can be asked more generally: “I struggle to imagine the use cases where X runs on the edge / servers & makes more sense than other ways to access X.” You are balancing the cost and speed of X in different locations, the cost and speed of accessing X over the network, and whether you have existing infrastructure deployed there already.

Cloudflare Workers and other edge services tend to make sense if you want more power than the client (e.g. phone) can provide, low latency to the service, and easy-to-scale deployments.

They have pretty cool examples here: https://ai.cloudflare.com/?gpu

Text Generation Speech Recognition Image Classification Text Classification Text Embedding Translation

The demos in this page are using Workers AI, not WebGPU APIs. However, yes, you can perfectly run an ML inference runtime that takes advantage of GPUs using just Wasm and WebGPU APIs. That's exactly what we have here:

https://github.com/cloudflare/workers-wonnx

Serverless single shot inference