▲ 457 points
back
103 comments
Great technical article as always, and apologies for not responding to the meat of the article, but I tend to look at declarative UI with some degree of skepticism. Raph, do you think this general approach can eventually be scaled up to develop complex and highly interactive applications such as DAWs, video editors, and graphic design software? At least based on my experience with SwiftUI, I find the paradigm easy and satisfying to get started with, and great for prototyping and widgets, but quickly run into roadblocks whenever trying to build creative software that's not just a view around a database. Or do you see this as just another tool in the UI toolbox, and not necessarily mutually exclusive with imperative and immediate mode UI? Curious to hear your thoughts!
I think your skepticism is well placed, and I think you are asking the right question.
Here is why I'm hopeful. SwiftUI is as you say wonderful, but is very much a closed ecosystem. You can't really implement your own custom views, rather you can assemble the premade ones in various (cool and interesting!) ways. If you wanted grid layout before iOS 14, you were on your own.
By contrast, in what I'm building, everything is open-ended, and you are invited to build fully custom versions of every piece of the system - change propagation, async resource loading, layout, drawing, animation, everything.
So yes, I am hopeful this approach will give good results for especially those highly intensive applications you mention. And if not, we'll learn something why not. I'm looking forward to trying!
Ok I'm not Raph but, FWIW, my 3d sculpting app is written using SwiftUI (https://sculptura.app), which is not a view around a database (in fact it was previously UIKit... SwiftUI is so much better). I recently implemented a piano-roll editor in SwiftUI: https://github.com/AudioKit/AudioKitUI/tree/main/Sources/Aud.... I have little doubt a DAW, at least, could be implemented in SwiftUI easier than with UIKit or AppKit.
Have you thought about backing the persistent widget tree with native widgets as the final layer? Or, at least supporting it as a render target. That is, UI/NSViews on Apple platforms, DOM nodes on web, GTK objects on Linux, etc. Essentially using them as the final “render tree”.
SwiftUI takes this approach, as ComponentKit did before it. The framework still does most everything (state updates, layout, animation, etc), but the host compositor handles the actual rendering.
This approach has a few advantages in my opinion. These widgets integrate nicely with the host accessibility system, for one. They also can come with built-in styling to make them “fit into” the target platform.
There are also performance benefits to leveraging the system compositor – Firefox switched to have platform native layers on macOS and it helped immensely with battery life.
It is worth noting that SwiftUI and ComponentKit components don’t have 1:1 mappings with native widgets – they both perform flattening (i.e. for drawing paths and layout only nodes) to optimize their performance.
I think most importantly, though, it allows for rewriting code incrementally as that approach naturally supports bidirectional embedding. Having rewritten the Shortcuts app editor from UIKit into ComponentKit into SwiftUI (maybe one of the larger projects written in SwiftUI?), incremental rewriting is crucial for existing software projects.
The other concern is mobile – it just is not feasible to rewrite the text input stack on mobile, for example, so being able to use native text editing views would very much be necessary.
This all looks fantastic!
I've thought about it. Your "etc" in the first paragraph is doing a lot of work, especially on Windows. The idea that there is a "native" widget set is increasingly a fiction. I agree though that on mac and iOS it makes quite a bit of sense.
A somewhat galaxy-brain approach to this is to make views generic over Cx, and add "factory" methods to Cx for creating button, stack, slider, etc.
All that said, my personal feeling is that while this would give promising early results for creating simple property-sheet like UI, it will be extremely difficult to make polished, truly native-feeling UI in it, as ultimately the seams will show. The first 90% will go well, but the second 90% will be painful.
I don't want to discourage people from trying it though!
Personal opinion but I think the old RAD approach in tools like Delphi where the components are laid out visually using the IDE but having the option of generating the components programmatically during runtime including attaching/detaching event handlers etc with state handled application-wide or within the context of the "form" is a much faster way to develop than to use the declarative approach in tools like Flutter where state management is really complex.
Also, one could develop custom components very easily in Delphi with custom draw events which would draw just the component.
Really cool work. My impression after reading the article is that it’s significantly inspired by SwiftUI, but without the magic annotations (@State, @Binding, @EnvironmentObject, @StateObject, etc.). It will be interesting to see how Rust handles the fully-statically-typed view tree, which has been really pushing the limits of the Swift compiler.
Question for the author: perhaps I missed it, but how do you plan to handle view trees that change based on state (ie SwiftUI’s IfElseView + viewbuilder)?
Good catch! Yes, the plan is to implement _ConditionalContent. It would look something like this:
if_view(bool_predicate, || view1(...then...), || view2(...else...))
I'm generally pretty hopeful about the ability of the Rust compiler to handle big complex types, but it is a risk. There other projects out there that also stress it, and the compiler team is pretty serious about making this work well.
From the first paragraph:
> ... Architectures that work well in other languages generally don’t adapt well to Rust, mostly because they rely on shared mutable state and that is not idiomatic Rust, to put it mildly. ...
The author doesn't mention Redux (the architecture), which is surprising. There are three principles[1]:
1. The global state of your application is stored in an object tree within a single store.
2. The only way to change the state is to emit an action, an object describing what happened.
3. Changes are made with pure functions.
In other words, the components of an application never mutate the state tree directly. Rather, they emit actions which re-generates the state tree without mutation.
This style of state management is compatible with Rust's ownership model.[2] The emphasis on pure functions (that clone state rather than mutate) means that it's not necessary for your application to alias mutable references, which I'm guessing underlies the "generally don't adapt well to Rust" part of the claim.
[1] https://redux.js.org/understanding/thinking-in-redux/three-p...
The other responses have this right. I think the Redux pattern is similar enough to Elm that I didn't feel a need to make a finer distinction.
I'll also say this: the tools that Rust provides for reasoning about mutation are powerful and principled. A central philosophy of Rust is that mutation isn't the problem, it's shared mutable state. If you believe that philosophy (and I do), then restricting yourself to pure functions over clonable state feels like tying one hand behind your back. I hope I've made the case that providing finer grained access to mutable app state is an approach at least worth exploring.
I think Redux is similar enough to Elm that it's not worth mentioning separately. Redux plays fast and loose with types whereas Elm uses types strictly; that's probably why Elm is mentioned more often in the context of Rust.
He does mention Redux in passing if you expand "Advanced topic: comparison with Elm"
Apart from the architectural similarity, Elm also predates both Redux and Flux, even being referenced in Redux's Prior Art page [1].
[1]: https://redux.js.org/understanding/history-and-design/prior-...
redux architecture is just a global scan(). more or less what the elm architecture is as well.
it's... it's beautiful... wishing Ralph good luck! his blog posts have taught me sooo much in the past. i particularly enjoyed the disclaimer at the end, a lot of these Rust projects aren't production ready but learning about their respective architectures is a great way to passively consume 'academic' paradigms/concepts. if anything, this is what keeps me interested in Rust! i don't have a formal background in CS, so new projects and their inspiration serve as a great gateway into more rigorous study.
edit: the potential for Python bindings is very interesting, it seems to me that Python and Rust are developing a sweet kinship :,)
Very interesting! This somehow seems convergent with the model-level incrementalization approach that incr_dom [1] and its successor bonsai [2] are using. Have you had a chance to compare these?
[1]: https://github.com/janestreet/incr_dom | https://www.youtube.com/watch?v=R3xX37RGJKE
[2]: https://github.com/janestreet/bonsai/blob/master/docs/blogs/...
I would say that they were even greater inspirations for the Druid architecture that predated this latest work - we were hoping that a lot of the incremental/reactive patterns could be expressed using combinators over immutable data structures. That didn't work out as well as hoped; I think it's possible to build things that way, but it's also pretty hard going and the community never really reached critical mass.
A semi-explicit goal of this work (that somehow didn't make it into the blog post) is that developing for this architecture, both building the UI components and using them, should be a lot more fun than before. Of course, that's a slippery goal to quantify. We'll just have to see how it goes, but I'm hopeful.
> The problem is that it requires shared mutable access to that state, which is clunky at best in Rust (it requires interior mutability).
I don't see the problem with using interior mutability (Rc<RefCell<T>> or Arc<Mutex<T>>)
With Slint [1], we just embrace it, and rely on interior mutability for the shared state, and that works well.
I tried using interior mutability in rui [1] but the clunkiness appeared in having to call clone on the Rc/Arc too often. I'd have a few clones before moving into a closure, like this:
let text = self.text.clone();
focus(move |has_focus| {
let text = text.clone();
state(TextEditorState::new(), move |state| {
let text = text.clone();
let text2 = text.clone();
let cursor = state.with(|s| s.cursor);
let state2 = state.clone();
focus(move |has_focus| {
state(TextEditorState::new, move |state, cx| {
let cursor = cx[state].cursor;
canvas(move |cx, rect, vger| {
Seems like a large part of the complexity is enabling the creation and use of reusable UI components that work in a variety of UI hierarchies and modify a variety of backend models (or app states), in a type-safe way. Is that right or are there other problems attempting to be solved?
The simple way to do this is a callback system. Why is that not appropriate for Rust? Does it require custom ownership dynamics that the borrow checker does not support?
Callbacks are often isopmorphic to a big ball of mutable state, which Rust makes very painful.
It’s like the classic Joe Armstrong quite about getting the whole jungle when all you wanted is the banana.
You can do it, and if you check out the Gtk/QT bindings you’ll see the boilerplate it introduces. Not insurmountable but many people are interested in figuring out if there’s a better way.
Borrow checking gets a lot more complicated with closures that live past their scope. It’s usually much more frustrating than it’s worth.
Animation is usually the biggest pain point with frameworks like this -- and it usually feels like a complete afterthought for framework designers. Of course you always have the simple CSS-style stuff (here's a list of properties you can attach transitions to) but as soon as you get into anything more complicated it all falls apart.
> Animation is usually the biggest pain point with frameworks like this -- and it usually feels like a complete afterthought for framework designers.
It tends to be. The only (popular) JS framework where it's a first class concept is with Svelte. There's a basic set of transitions and the framework handles the post-out-transition removal (which is annoying in a lot of frameworks) but there's also custom CSS animations where a JS-defined curve is rendered into CSS [1], semi-automated FLIP animations for non-transitioning nodes [2], and deferred transitions where the outro of one element is sync'd with the intro of another [3]. There's a few other things like springs and whatnot but I consider the package as a whole to be a significant advancement in the state of the art.
[1] https://svelte.dev/tutorial/custom-css-transitions [2] https://svelte.dev/tutorial/animate [3] https://svelte.dev/tutorial/deferred-transitions
The only other framework I know of that's tackled it in core is Inferno but I haven't read through their implementation in detail.
CSS transitions are fairly restricted, but animation in CSS is substantially more capable than transitions if you’re using the animations API instead: https://developer.mozilla.org/en-US/docs/Web/CSS/CSS_Animati...
CSS is pretty flexible. What kind of animations would you expect to be difficult to express?
How important are animations in a UI? I typically find them annoying (because you have to wait for them).
Really interesting post.
One thing I think might be problematic for you would be fallibility in the `Adapt` callback process. i.e. if it would not be sound to change the state of a child due to some emerging app state inconsistency or similar.
To put it differently, I understand your architecture outputs a new _statically typed_ tree every time through kind of recursive state mutation. However, being statically typed, it is clear from the start of the operation what the end complete static type will be (to the compiler at least). If the transformation of one of the children is not possible (although the rest might be fine), what do you do?
The obvious way would be to make each conversion fallible, but this might be a pain to use/propagate. Otherwise, you might have state that all operations involved in state mutation must be infallible.
Anyway just my 2c. I might have misunderstood the mechanism, though.
Thank you for all your impressive work!
Very insightful post. Although my work is mainly about audio not GUI, I still learn a lot from your idea. For my audio work, I also use declarative style and diff algorithm for updating the audio graph. But when rewriting it, I found sending messages is also very convenient:
https://github.com/chaosprint/glicol/tree/main/rs/synth
This might be interesting for you as I found that you also have a synth project (https://github.com/raphlinus/synthesizer-io). Admittedly, there is still some way to go for this audio lib. Will further study this post when I got more spare time.
What are your thoughts on Sycamore? It uses Svelte-style compile-time reactivity.
To be honest, I haven't looked too deeply into Sycamore. A lot of the reactive machinery looks pretty similar to Dioxus (threading a context scope, using explicit observable objects for change propagation). I think it's worth comparing more carefully, and would be more than happy to link a writeup to such a comparison.
Author Raphlinus has an incredible history of great, superb technical posts here, many which have spawned great discussions[1].
Referenced in this article are: Xi-Editor, discussed in Xi-Editor Retrospective[2] and Druid, discussed in Rust 2021: GUI[3], which follows closely after Principled Reactive UI[4] (describing a prototype for Druid called Crochet).
> I have long believed that it is possible to find an architecture for UI well suited to implementation in Rust, but my previous attempts (including the current Druid architecture) have all been flawed.
These have all been very very good & technical posts on what toolkits really support & enable ui (amid other great technical topics too). It's delightful seeing such an ongoing continuation, an evolcing refinememt of ideas & self-review from someone of such expert caliber.
Rarely do we get such an intimate view into what the real hunt for rightness is, see how we ever hunt for perfection. Personally I believe that hunt for better is one of the undertold aspects of hackerdom, a less visible less knowable reciprocal to fast-and-dirty. Tapping & enabling this creative, knowledge & intellect based capability is a core spring from where greatness emerges.
> I have studied a range of other Rust UI projects and don’t feel that any of those have suitable architecture either.
It's also notable how widely raphlinus travels to get the best persepctive available. This post begins with a a vast field survey of other ui libraries & their origins. I lack the energy to search down & link HN discussions on each of these, for there are many! But seeing how everyone else is doing, looking wide & far to explore their peer's attempts, is also a notable characteristic I admire here.
[1] https://news.ycombinator.com/from?site=raphlinus.github.io
[2] https://raphlinus.github.io/xi/2020/06/27/xi-retrospective.h... https://news.ycombinator.com/item?id=23663878 (538 points, 26 months ago, 157 points)
[3] https://raphlinus.github.io/rust/druid/2020/09/28/rust-2021.... https://news.ycombinator.com/item?id=24631611 (374 points, 31 months ago, 244 comments
[4] https://raphlinus.github.io/rust/druid/2020/09/25/principled... https://news.ycombinator.com/item?id=24599560 (234 points, 31 months ago, 97 comments)
Would be great to eventually have different backends for this, like Java had where the app looked native in Windows, Gtk, etc. One could even think of having a TUI backend besides an HTML or Canvas backend.
That would be restricting of course when one needs features available in only one backend and developers could opt-in for more control and require a certain backend like Gtk or not require but detect the backend and get extra features.
I've never persevered with immediate-mode UIs deeply enough to get to the point I needed to solve this, given I mostly deal with complex nested UIs and in my (possibly limited/incomplete?) experience, immediate-like UIs don't really work well for that type of complex and dynamic setup, but it sounds like the Widget tree persists in this model (at least more than the other trees), but it's not clear if it's update-able as well? (there is mention of it being rebuild-able, so I guess so?).
I wonder what happens regarding state (say, selection state, or visibility/enabled state) in the case where you might want to allow the user to re-arrange entire UI components (say the user draging a nested tab/pane from one window of the app to another, and docking it into another different hierarchy): would the trees have to be completely re-built (I guess sub treelets could still be kept?) along with re-building the id paths. Would that mean diffing is hard/impossible in some cases to transfer across a large re-build of these trees?
First question is easy: yes, the widget tree can be updated. That's generally done by diffing data stored in view nodes, but in fact the View trait is open-ended and you can implement the rebuild method to do anything you like. And yes, selection state lives in widgets and it is absolutely a goal to have that persist.
The second question is more challenging (see the "advanced topic" under identity for a little more background). View id's cannot be re-parented, in other words when a parent relationship is expressed in an id path (by virtue of having the child id follow the parent in a path), that relationship cannot be changed. However, widget ids and view ids are not necessarily the same, though they can be. I think what's needed for your use case is a level of indirection so the view id paths remain stable, but the widget id structure relationships can be changed. I haven't worked out all the details, but think it can be done, and if it's done right it wouldn't require any rebuilding of widget subtrees.
Obviously Raph knows what he's doing, so this isn't a critique, but a question:
What's the reason for replicating the "React idea" outside the web browser?
The way I understand the motivation behind React is that it tries to work around the too high level and too rigid DOM by mapping one way to describe an UI (the React API) to another (the DOM), for the only reason that there's no realistic way to bypass the DOM (except doing everything yourself - including fundamentals like text rendering - via a 2D- or WebGL canvas).
But if you don't have something as rigid as the DOM as lowest layer to begin with, what's the point of building a React-style system with 'tree diffing' against data that persists across frames? Is it really worth the complexity managing intermediate data that sits between the UI description that (most likely) needs to be updated each frame, and the low-level rendering instruction stream - which most likely also needs to be generated each frame?
Or is it all about Rust's language restrictions?
I am currently designing an UI framework in rust and I took a similar approach. It is intended for medical monitoring systems. I should be able zo open source it at some point. But for now, all I can say is that this approach seems the way to go.
Some of your previous explorations were motivated (IIRC) by UIs that had (potentially) many thousands of objects for interactive visualizations of datasets. Is that still a motivating example for this evolved approach?
Big fan of your work, Raph.
One small typo:
> {anonymous function of type FnMut(u32) -> ()}
It looks like the param type should be `&mut u32`. And in that simple case the whole thing could probably just be `fn(&mut u32)` since the closure doesn't capture any locals.
I think the answer to this is no, but I'll ask anyway.
Is there any concern with the allocation for the view/widget tree every cycle? I know it's retained mode so the most important resources (graphics) are cached between cycles, but I'm wondering if the trees ought to be allocated out of arenas or something so that the allocating/freeing every cycle isn't undue performance penalty.
(I'm not sure what existing regained frameworks do.)
As someone who has written some 10,000 lines of Rust, if this comma is intentional this is just frankly unreadable:
adapt(
|data: &mut Arc<Parent>, thunk| {
let mut child = data.child.clone();
thunk.call(&mut child), // <-- HERE
if !Arc::ptr_eq(&data.child, &child) {
Arc::make_mut(data).child = child;
}
},
child_view(...) // which has Arc<Child> as its app data type
)
Related and possibly helpful:
* How Glimmer.js implemented autotracking: https://www.pzuraq.com/blog/how-autotracking-works
* Discussion specifically on Lamport Clocks and autotracking: https://v5.chriskrycho.com/journal/autotracking-elegant-dx-v...
Is this inspired by adaptron by any chance? If no, do you have an opinion on said work?
What happens if two child components need mutable access to the same data? Say, a group of dropdowns to filter/sort a table plus a pie chart with clickable slices that also change the filtering.
kudos for explaining the meaning of the name, alot of projects just puck a random name and call it a day.
> The name “Xilem” is derived from xylem, a type of transport tissue in vascular plants, including trees. The word is spelled with an “i” in several languages including Romanian and Malay, and is a reference to xi-editor, a starting place for explorations into UI in Rust (now on hold).
Another UI framework that re-renders everything each cycle? Yeah, it's gonna be awesome for todo lists (as long as you don't have more than ~100 todo items). Meanwhile high-performance apps will still be written in retained-mode UI toolkits.
Apologies for the noob question, but why reinvent the wheel when we have HTML/DOM/CSS?