back
24 comments
Thanks for writing these updates, Raph - back in 2019, your early piet-gpu blog posts helped to reignite my interest in 2D rendering. I recently learned that you also invented Cairo's trapezoidal-coverage algorithm, which was my original introduction to anti-aliased 2D rendering ten years ago!

Over the last few weeks, this interest has finally spiralled into an attempt to develop a renderer of my own. I've come up with a design which seems promising, but I do have one question for which I haven't yet found a good answer:

It seems to be common wisdom that 16xMSAA has unacceptable quality for 2D rendering. In practice, though, I'm struggling to find any test-cases where I can differentiate 16xMSAA from 256-shade analytical AA with the naked eye. The only exception has been fine strokes (e.g. downscaled text, or the whiskers on this Ghostscript tiger [0]), which tend to "shimmer" or "sparkle" as they move. Does MSAA have any other common failure modes that I should be aware of?

[0]: https://threejs.org/examples/webgl_loader_svg.html

Thanks for your response! I write these blog posts for you and others to understand the technology better.

I didn't invent the Cairo trapezoidal algorithm, I think that was mostly Carl Worth, maybe Keith Packard, but I certainly did do one of the earliest analytical area algorithms in the free software space, which I know inspired other work. It's been quite a ride, I went back and looked at my early work[1], and thought it was impressive that I could render the tiger in 1.5s. Now I'm seeing times in the 300µs range on discrete graphics cards.

16xMSAA is pretty good, and I would say is definitely good enough for most vector art. I consider it not quite good enough for text scaling, and anything less than 16x not acceptable. Desktop cards shouldn't have much trouble with it, but mobile devices might not be capable of it or able to deliver adequate performance.

Of course the upside of MSAA is that it's much easier to solve conflation artifacts, so I imagine we'll end up with some variant of it in additional to the analytical AA; and the latter will continue to be used for almost all glyph rendering.

[1]: https://www.levien.com/svg/

Raph’s work has been immensely interesting for me over the past few months as I’ve become interested in 2d rendering. He has an immense amount of references littered throughout his different blogs and repos, and it has been a wonderful experience (trying) to fully understand the simpler parts of modern/next-gen 2d rendering.

I’m excited to see results of innovation in this area, but the fact that he (and others like pcwalton, nical, etc) are making things like incremental updates and blog posts with supporting images and pseudo code available to us is just fantastic.

Thank you, Raph. I’ve had so much fun exploring your work on font-rs, and on Piet. I hope one day I’ll be able to move past tinkering and actually pitch in and help, thanks to your fantastic write ups and docs.

I was really confused about this, at first thinking it was related to the Piet programming language (https://esolangs.org/wiki/Piet) which is also comprised of 2D drawings. I kept thinking this was some clever implementation that compiles Piet code on the GPU instead.

Took me a while to realise that this was just a name clash! Doh.

Same here actually, I was imagining a mosaic of little Piet programs all run in parallel on a GPU. The time I spent with Piet was pretty rewarding. I did a project where I took my company's phone screen interview question and completed it in esoteric languages. Piet was the most fun of the challenges, it was pretty hard to wrap my mind around rotating the stack to keep track of variables, but when I got it right it was pretty cool.
the name piet comes from this project https://github.com/linebender/piet

it's an abstraction of common 2d vector graphics libs.

That project and the programming language were both named after the famous artist, Piet Mondrian.

https://en.m.wikipedia.org/wiki/Piet_Mondrian

My post wasn’t to complain about the name collision though. I don’t see an issue with the name being shared like that as they’re occupy similar but distinct domains. I just thought I’d share my confusion in case it entertained anyone.

I really hope to help on piet-gpu. I previously tried to understand the code, but I gave up. I will restart from the blog articles this time.
Thanks for your interest! If you want to dig into the project, another good resource is the #gpu stream on xi.zulipchat.com. That has some more in-depth discussions as well.

I'm openly embracing the fact that piet-gpu is advanced research into not just 2D rendering, but parallel algorithms and also GPU infrastructure. The fact that it's doing advanced compute portably is not something a lot of other projects can claim. Unfortunately, it adds constraints and makes the code more difficult than it would be if it were targeting a single GPU.

So, feel free to ask questions, and I'll do my best to answer them. Education is part of what I'm trying to do, and I'd like to put together more resources for understanding both piet-gpu and also compute shader programming more generally.

Thank you very much!
Could somebody help me understand what exactly this is? The description tells about 2D rendering, and it seems that it sits on top of, for example, Vulkan, so my best guess it is a 2D rendering library to be used in games or other applications (as opposed to, say, a compositor, hardware driver, OpengL implementation, Vulkan implementation, or whatever else it could be).

Background: This actually seems related to a pet project of mine, tinkering with graphics rendering on an FPGA. An application-side 2D renderer would be very interesting to me to better understand the application side of it, when building the hardware side. Especially when it is a renderer that uses a presumably modern approach of using shaders for everything, making the hardware side insanely flexible, and because there are already plenty of other renderers to understand a static rendering pipeline.

it's vector graphics on gpu. traditionally vector graphics is done on cpu and cached on a texture.

this project seeks to draw vector graphics directly using gpu.

I think you are still assuming some context to understand your comment that I'm still missing. By "directly using gpu" you probably mean without Vulkan and without any operating-system-level graphics driver, right? I.e. writing to GPU registers directly.

Does this project assume any other operating system level infrastructure? If so, which OS -- Linux, Windows, BSD, ...?

When modern GPU graphics are used, it's assumed that an API layer is also used. This allows the drivers to implement the API specification within approximations appropriate to the hardware. The OS, drivers and API all collaborate to provide a consistent abstraction similar to a file system or sockets; a header file is included to use the API, and the initialization process creates a GPU context, which allows binding of resources etc. There are some examples of direct register programming, but nearly all come from gaming consoles with fixed hardware and tight performance budgets.

Vulkan is simply the lowest-level approach currently available among the major APIs. Piet is an example of a rendering engine that stands on top of Vulkan: it still needs application code to be a complete renderer, but it provides higher level abstractions that speak in the desired domain language(shapes, vector data, etc.).

directly using gpu == using vulkan shader

this paper explains it well

https://w3.impa.br/~diego/projects/GanEtAl14/

Or the OP's earlier blog post

https://raphlinus.github.io/rust/graphics/gpu/2019/05/08/mod...

Pretty much all of the rendering pipeline runs on the GPU, in compute shaders. In most 2D renderers, the CPU is doing a lot of the work (tesselating vector shapes into triangle meshes, computing bounding boxes, etc). In piet-gpu, the CPU just encodes a high-level, declarative description of the scene (semantically similar to SVG) in an efficient binary format, and the GPU does the rest.

Regarding hardware support, it needs a GPU capable of compute shaders (which rules out a bunch of the older or very low end mobile chips). There is a hardware abstraction layer which currently supports Vulkan, Direct3D12, and Metal, but I believe D3D11 could be added, as well as other lower level interfaces (I took a look at deko3d and was tempted to use that as a way to port to Nintendo Switch).

A Intel GPU might be behind Vulkan, WebGL2, DX12, D3D11, WebGPU, OpenGL4, GLES3.2,or Metal ... There are a similar nr of current GPU hw architectures as current GPU APIs, is there still a portability benefit from the API abstractions or are they just zero or negative sum competition?
I think this is designed to be a 2d rendering library (similar to skia or cairo) which forms the basis on which an application GUI toolkit could be built. I'm pretty sure it's designed to work cross-platform (although I'm not sure if the current prototype does), although it probably relies on an OS of some sort being available.
Piet is the backend for the Druid UI toolkit. It is indeed cross-platform. The SVG backend looks like it would work without an OS, but I'm not sure the value of a GUI toolkit without an OS anyway :)

https://github.com/linebender/piet#backends

https://github.com/linebender/druid/tree/master/druid-shell/...

A GPU toolkit without an OS API could be useful for WASM rendering, and a rendering library without a GUI is useful for headless rendering (matplotlib graphing, video generation/compositing, meme generators, etc.).
I find it hard to understand the real world implications of this. As I understand it this is a research project aimed to improve future 2D rendering.

Would this somehow help improving Chrome's rendering performance? Right now Chrome has massive performance issues with rendering very large SVGs (zoomed in SVGs) and I am convinced that clipping could dramatically improve this.

Real-world implication is that you can render extremely complicated flat vector graphics in real time at full resolution, and use GPU rather than CPU resources. Current CPU-based vector graphics rendering makes quality compromises for performance (rendering is often subtly incorrect, e.g. adjacent solid-color regions get rendered with hairline gaps of the background showing through, lines have poor antialiasing, or compositing is done in the wrong color space), gets bogged down as drawing complexity increases (which can lead creators to compromise their artistic vision for performance), takes CPU that might be needed for other tasks, and sometimes cannot handle display refresh rates so you end up with rendering delays, janky scrolling, etc.
Do you have an example site that shows this problem?
It is quite interesting that after reading a single paragraphs I know I am reading Ralph Linus without knowing anything else.