So... separating front-end from back-end in and of itself is not necessarily a problem, but using async to do it adds complexity that is not worth it. Originally, I was hoping that a CRDT or similar mechanisms could be used to smooth out all such problems in a general way (including letting plugins contribute their annotations without causing jank on the main thread), but I discovered that such a collaboration layer cannot hide the underlying explosion in the potential state space.
I wrote about this in some detail: https://raphlinus.github.io/xi/2020/06/27/xi-retrospective.h...