▲ 66 points
back
13 comments
Hey Raph, hoping you might see this: Almost all (Rust) game engines seem to focus entirely on sprites (or more generally, texture-based primitives) for rendering. I've been experimenting with a toy game project based heavily around more complex 2D primitives [1], such as splines or Bézier curves that might benefit from some vertex-based shader shenanigans [2]. But apparently this is something 2D game engines just don't support. Since you have a lot of experience with curves, Rust and GPUs I was wondering if you can think of specific reasons this path seems to be unsupported?
[1] My goal is to implement realistic train track geometries, including transition curves, using Euler spirals or perhaps a simpler curve.
[2] For example, I'd like to try tessellating curves via shaders as required, based on zoom level etc.
I would say the most likely reason is that 2D graphics is just plain hard. Jasper's classic essay[1] goes into more detail.
That said, I'd really love to see more stuff along the lines of what you describe. I think Vello could render, for example, something that visually looks like Mini Metro or Mini Motorways. I also of course would be a massive fan of using Euler spirals - the only curve that's truly, really simpler is quadratic Béziers, and making those look good in your application would be hard. I'm about to do a lot of Euler spiral math on GPU as part of the stroke rework, which you may find interesting.
Strokes are easier for GPUs than fills, because you don't have to worry about the winding number issues (non-convex paths, paths with holes, etc). There are two recent papers, Polar Stroking[2] and Converting stroked primitives to filled primitives[3].
Also check out Freya Holmér's excellent Shapes library[4] (easily integrated with Unity).
Tesselating curves via shaders is the future (through mesh shaders in particular), and I've got a colleague working on that, but portable infrastructure seems to me just as thorny a problem as compute. I am very biased, but if it were me I'd try to render the 2D stuff with a 2D renderer like Vello.
Best of luck with your project, and I'd be interested in following the progress. Feel free to stop by the Zulip[5] and ask such questions there, as well.
[1]: https://blog.mecheye.net/2019/05/why-is-2d-graphics-is-harde...
[2]: https://arxiv.org/abs/2007.00308
[3]: http://w3.impa.br/~diego/projects/Neh20/
[4]: https://assetstore.unity.com/packages/tools/particles-effect...
In the ML section you mentioned Kompute and MediaPipe. Have you seen IREE? It has a Vulkan-like compute-only HAL. https://github.com/iree-org/iree
Oh yes, and I probably should have mentioned that as well. I've been talking to the developers but haven't checked in lately. I'll take another look, thanks.
> WebGPU 1.0 is basically a “least common denominator” of current GPUs. This has advantages and disadvantages. It means that there are fewer choices (and fewer permutations), and that code written in WGSL can run on all modern GPUs. The downside is that there are a number of features that most (but not all) current GPUs have that can speed things further, and those features are not available.
This is really the issue with 3D Web APIs.
WebGL 2.0 give us something like a Playstation 3 (released in 2005).
WebGPU 1.0 will give us the state of the art in 3D API as they were in 2014.
Meanwhile streaming and native will give us the APIs as they are available today.
I was experimenting with using Vello for a project just the other day. Drawing over ~5000 rectangles it seems to result in all the rectangles disappearing from the screen but that aside it was a great experience.
It is way faster than Pathfinder, and Macroquad is faster than Pathfinder. Pathfinder was overall a good experience except it wasn't that easy to get started but once things were set up, it was pleasant especially the API with text alignment etc.
For some reason Macroquad seemed to have the best frame rate with all those rectangles (OHLC candles if anyone is wondering -- 3-4k of them being dragged around). Vello looked like it most definitely would beat Macroquad but instead of dropping the framerate it looks like it drops the rectangles.
It's possible you ran into problems with large pathtags, fixed by [235], but that hasn't been merged yet. With that and some other tweaks (we still have to manually adjust buffer sizes, but that will soon be dynamic again as well), we are able to render the paris-30k test file, which has ~50k paths and almost a million path segments.
We will also soon be special-casing rectangles. Might be worth rerunning your experiment in a few weeks when all this has landed.
In any case, glad for your interest!
I am always amazed at how little of what is written in these reports I have access to referents to enable me to understand. I always read to the end anyway, and come away feeling like I have learned something, albeit unclear on what.
From what I can see this is comparing shader languages weighing pros and cons.
To use an analogy. Think of Vulkan, Metal, DirectX 12 (DX12) as different database engines. They run compute/render shaders which are incompatible dialects of each other (like Oracle SQL != Postgres SQL != MySQL). WGPU tries to be the ANSI SQL.
This an oversimplification ofc. Today most shaders compile to an intermediate form Spir-V which then targets different backends and can lead to some funny translation.
> GLSL/HLSL/MSL -> SPIRV -> WGSL -> SPIRV -> GLSL/HLSL/MSL
It's nice to see more collaboration.
With the apparent interest in bevy, is there any chance that you will try to integrate the UI project xilem with bevy? Seems like it could be a great fit, and if it happens before the bevy project chooses a direction the two project could probably both benefit. Bevy for the great xilem architecture and xilem from the big community of developers that bevy has
I'd love to see work along these lines, and hope to talk with Cart soon. However, I'm stretched thin as it is, so certainly won't be leading any such integration work. My main goals for 2023 are to get Vello in real usable shape, and there's quite a lot to be done there (the roadmap is in draft and will be published soon, probably today).
They already have a demo of integrating bevy with vello, the renderer that Xilem uses.
Yes, that is why I brought it up