Item — HN Reader

“If you want to build a ship, don’t drum up the men to gather wood, divide the work, and give orders. Instead, teach them to yearn for the vast and endless sea.” — Antoine de Saint-Exupéry

I will one hundred percent agree with you that my approach is not a good one if you want a pretty good GUI in a reasonable amount of time. For that, an incremental approach, especially adapting some existing successful design, would be better. But that is not in fact my goal, it is to yearn for the sea.

I am, quite deliberately, spending a lot of risk points. In addition to using a language which may (see elsethread) not be a good fit for expressing UI at all, I'm building a GPU 2D renderer from scratch, also using compute shader techniques which have not been proven, and I am designing a reactive architecture that is not just a simple adaptation of React. Any of these could fail, as have some of my previous attempts. But I think they will be interesting failures, in that we'll learn something, and if all the pieces do come together, it will be a UI toolkit capable of performance completely untouchable by the existing state of the art. In turn, I'm interested in how that could open up new creative possibilities constrained by current implementations.

So I'm pretty comfortable with my approach. And hey, next time you're in the Bay Area (or perhaps when I'm in your neck of the woods), lemme buy you beer and we can talk about why it gives you satisfaction to dump on other people's work.

6 comments

To be frank, I think that the kind of work you are doing is necessary.

In my opinion GUI is not yet something I would consider to be a "solved problem". Both from the API perspective and the rendering side, and compute shaders are indeed extremely promising and could be something close to an end-game in this space.

This is why I read your articles.

And I have absolutely zero interest in dumping on your work, but I have the feeling that a large part of this work is going to waste because of what I consider to be flaws in the ways you are approaching the problems, or maybe the way you advertise your approach, I can sense how discouraging it can be for other devs.

What, specifically do you think the flaws are? I think if you're going to make such criticism, it helps to give actionable specifics (i.e. constructive criticism). How is the approach discouraging to other devs?

I think that people who manage to achieve "impossible" tasks tend to be overly optimistic (and naïve) at first, consciously or not, this is a good trick to fuel their own motivation and that of others.

On the contrary, knowing/talking too much about the difficulties can quickly kill the fun and motivation.

He should talk about how great the end-goal will be and why it is important.

Would you rather end-users continue to get stuck with applications that ignore important things like accessibility, that can block some people from getting or keeping a job, so as not to kill some developers' motivation? I'm glad Raph is talking about the difficulties so they (hopefully) won't be ignored or clumsily bolted on afterward this time.

Edit to add:

"Those who cannot learn from history are doomed to repeat it." --George Santayana

I think the time for recklessly moving fast and breaking things in software, without taking into account known complexity and avoiding the mistakes of the past, is over. Our impact on the world, and the resulting responsibility, is simply too great for that.

Last part might be worth an edit. I saw his comment as constructive well meant criticism rather than callous, thoughtless, or mean spirited.

Also love the work you do and enjoy following it. Your treatises on Oklab are the #1 place I send programmers of all skill levels to understand color.

As another anecdatum, I thought the last part was beautifully done and completely deserved.

The dismissive complaint was not without merit, but wow did it assume that the commenter's use cases are all that matter and all that need be considered. The world of UIs is much, much, much bigger than that, and awash in unsolved or badly-solved problems that matter a lot to many people.

I agree with the complaint, fwiw, when applied to an important subset of the design space. I even think it's useful to try to understand where the limits of that subset are, and why.

But saying that exploration is pointless because we're all happy living on this here big island and there's nowhere else that could possibly be better so why bother looking, it's all good except we still don't know why people keep dropping dead, but that's an acceptable drawback to what is otherwise a paradise on Earth—hang on a sec while I scrape off these leeches, they're so silly sometimes—and the people who think otherwise are just malcontents who ought to be out catching fish for the rest of us to enjoy.

I think another great counterpoint to dogmatic incrementalism is Gary Bernhardt's classic talk "A Whole New World": https://www.destroyallsoftware.com/talks/a-whole-new-world

>So I'm pretty comfortable with my approach. And hey, next time you're in the Bay Area (or perhaps when I'm in your neck of the woods), lemme buy you beer and we can talk about why it gives you satisfaction to dump on other people's work.

I love this response - stealing it :)

>it will be a UI toolkit capable of performance completely untouchable by the existing state of the art

Why exactly? What are the current state of the art UI toolkits leaving on the table performance-wise?

I'll have a lot more to say about this as I gather quantitative performance data. But it's a good question. I expect the big wins are: fast 2D (vector with blends and so on) rendering with compute shaders, multithreaded creation of expensive resources like image decompression and text layout, pushing incremental reactivity all the way from app logic to GPU (as opposed to needlessly redoing work), and of course just using a fast, non-GC language.

What do you mean by “pushing incremental reactivity all the way from app logic to GPU (as opposed to needlessly redoing work)”?

From a computation perspective, UI is fundamentally an incremental computation engine. Most elements are not changing from frame to frame, so you can either recompute and re-render, or be smarter about only propagating deltas. I'd like to propagate those deltas all the way to the GPU, so you reuse lots of things from the previous frame if they haven't been invalidated. I'll be writing about this in considerably more detail; stay tuned.

Wait, are you talking about deltas as in damage rects?

If I recall correctly, damage rects are a raster-stage optimization that rerenders components only if they have changed ('dirtied') since the previous render. You could extend this concept of 'damage rect' to the next layer down, like React has pursued with the vdom/dom split. If you continue extending the analogy through every layer (raster, layout, scene graph, component hierarchy, data model, ...) all the way down to and including the application interaction model ("user has pressed the E key", "received IO result", ...) and then statically project every interaction model event back up through all the layers to derive a rerender box, then I think you could say damage rects are kinda like deltas. (I think this explanation is mostly reasonable.)

Yeah, I get it. Although in React's case the whole vDOM diffing thing is self-inflicted - it is only necessary because they insist on the UI being a "pure" function of state. Turns out that the only way to do this with an imperative API is some kind of tree diffing, so that you don't recreate expensive stuff from zero each frame.

'infogulch has it right. A damage region is a way of saying "this region of pixels hasn't changed," but you can also say that for a scene graph, attributes of widgets, layout, the view tree, and other things. Ideally you press a key and the CPU does only a tiny amount of work figuring out what changed, followed by the GPU doing a tiny amount of work rerendering the changed pixels.

> it will be a UI toolkit capable of performance completely untouchable by the existing state of the art

Could you give an example of an application where the bottleneck is UI code? In my experience the bottleneck is always either disk or network. Not trying to bash you, genuinely curious.

There's plenty of apps that we developers use daily that fall into that. Apps like Teams, Slack and Jira can feel incredibly slow to a lot of people, even with everything already in memory and not waiting for anything from the network or disk. Typing messages, changing tabs, going to read a notification... Facebook sometimes takes a second to show characters you typed in a reply, and the characters often show up out of order (also zero network activity during it). A lot of complex WYSIWYG editors are also incredibly heavy, while simpler ones aren't.

Sure, that might be technically a bandwidth problem: in this case probably RAM. It is solvable with faster computers/faster RAM. But it's slower or at least the same speed as it was in the past with slower machines. Since hardware got better, it has got to be something different in the software side.

And it's not even about difficult things like Unicode Glyphs and Emojis, which are common canned responses when anyone says that software "is slower than N years ago". Those things are handled by the OS, not by Jira. And there are super fast apps that make use of them.

I don't have access to such corporate software at the moment, but can you verify in devtools where the delay is actually coming from in the apps you mention? Leaping to "the stack is bad it needs to be rewritten in rust" is extreme, especially when apps like VS Code are generally regarded as non-slothy [1]. Or at least not more slothy than the domain requires [2].

I'd wager the developers are poorly incentivized - if something satisfies the ticket and looks fast enough on their tiny test bed with state of the art hardware and great network (perhaps even localhost), it ships. On the other hand I don't consider something good enough to ship until it is fast enough on testbeds orders of magnitude larger than what I expect a "normal" workflow would include. The process of going from "works" to "fast enough for people using 100x larger inputs than I expect" almost never involves "rewrite in rust"[3], but instead "cache cleverly", "debounce discretely", and if all else fails "sit down and ponder on novel algorithms and data structures". These are all operations that are just as easy, if not easier, in high level languages as compared to rust.

[1]: Full disclosure I was paid to write vscode for a period. When VS Code is slow, and it 100% is at times, the root cause is almost always an extension blocking progress for some dumb reason. This absolutely blows, but isn't a problem rust would solve - indeed extensions can already invoke rust.

[2]: inb4 "but sublime!": Running experiments, I've found sublime to in fact be slower than a fresh VS Code install at working with very large files. Of course when you have extensions trying to do dumb stuff with the big files, VS Code can get worthlessly slow. Again not a problem rust would solve. Try it: make a 5M line file, click it open it in Sublime, then in VS Code. On my machine VS Code opens it well before Sublime can.

[3]: Yes there are some times when rewriting in rust is appropriate, for instance VS Code's search is ripgrep - but rust isn't handling the UI at all, it's running in a separate thread doing what it does best (multithreaded systems programming), while the main renderer thread is doing what it does best (rendering). This is the way forward for the truly "inner loop" code, IMO.

> I don't have access to such corporate software at the moment, but can you verify in devtools where the delay is actually coming from in the apps you mention?

I already did. Like I said, that happens when there's zero network activity (the apps are a bit sluggish even with wifi deactivated) but also zero disk activity as well (according to Apple's tools), as caches are warm.

Sure, network in the apps I mentioned is also incredibly slow (mostly because of lots of requests and redundant data), but the real slow part is the interface itself. Dragging anything, typing text, clicking buttons, popups. Everything takes more time than a native app from 15-20 years ago.

Devtools show it's death by a thousand cuts. Thousands of sub-milisecond javascript functions, thousands of unnecessary re-renders. That happens in almost every operation. Even popup menus take a long time to show up, despite not really doing anything before such as loading data. This is in both Teams and Jira, btw, Slack is not as bad. Interestingly, Teams works faster when opened inside Safari rather than in Electron, but not by much.

> Leaping to "the stack is bad it needs to be rewritten in rust" is extreme

Thankfully I said nothing of the sort... You mention Rust a couple times more, so I guess this is something of a pet peeve to you, which I'll ignore since it has nothing to with my message. I was only answering to your query, I'm not interested in making arguments because I have no dog in this race. Like I said, other apps (even those written in the web platform) are faster than the examples I gave.

> I'd wager the developers are poorly incentivized - if something satisfies the ticket and looks fast enough on their tiny test bed with state of the art hardware and great network (perhaps even localhost), it ships.

In this case it's also not about being fast on localhost or having 100x more data, although this is definitely a safe bet on most products. It's slow even on the base case, even without doing anything remote, or having almost no data.

If you want me to wager on why this happens: developers work in a way they can retain their sanity. If there's weekly changes of scope, they'll program defensively in a way that allow quick changes. So there's no room for macro performance optimisations. The architecture is optimized for change, not performance. I know there are weekly stupid changes in Jira/Teams because I see bugs and little test features coming and going every single fucking week, frequently disrupting my workflow. VSCode on the other hand is a developer tool, and performance and familiarity seems to take precedence over quick stupid features. Why? VSCode is a dev tool, so developers indeed know better. In normal products developers are unable to fight back the asshole product manager or product owner changing their mind every other week.

This post is about moving the GUI layer to Rust, away from the standard HTML/JS/CSS used in the apps you mention. That’s why I brought it up.

My claim is that moving to Rust is useless without solving the unnecessary rerenders, and once those are solved moving to Rust would be pointless. So that only real problem is fixing bad coding practices, which has nothing to do with the underlying language. In fact using a lower level language is likely to make that much more difficult.

Got it. I’m just making it clear that I’m only replying to your question (“Could you give an example of an application where the bottleneck is UI code?”) rather than arguing for language X or Y.

I actually agree with you.

I guess I’m kinda tired of people trying to lure me into arguments I don’t want to have.

The fact that our modern computers still often don't feel amazingly fast is a perennial topic of griping on forums like this one. I'm sure the UI stack has something to do with that.