Are Immutable Linux Distros right for you?

There’s a lot of comments in here about desktops, but IMO why even discuss Linux on the desktop… 99.9999% of Linux deployments are not Arch installs on old Thinkpads. Immutable distros *are* becoming a de-facto standard for server deployments, IoT devices, etc. They improve security, enable easy rollbacks, validation of a single non-moving target for systems/hardware developers…

There’s also been a ton of very advanced development in the space. You can now take bootable containers and use them to reimage machines and perform upgrades. Extend your operating system using a Dockerfile as you would your app images:

https://github.com/containers/bootc

Another opinion: immutability is required to guarantee software integrity, but there is no need to make whole system or "apps" immutable units. NixOS also consists of "immutable units", but its "granularity" is similar to packages of traditional Linux distros, each unit (Nix store item) representing single program, library or config file. This provides a better tradeoff, allowing to change system relatively easily (much easier than in immutable distros described here, and in many cases as easy as in traditional Linux distros) while having advantages of immutability.

I totally see the advantages of immutable distros, particularly in a professional or cloud environment. Even as a hobbist, I feel tempted to use immutable distros if it were not because of:

- Learning. Figuring out how to migrate a setup even to the most mainstream-like immutable distro (fedora silverblue) can take a while, and to niche distros like talos even longer. However, a k8s-friendly setup with low customization requirements would help to speed up the migration (but it requires more powerful machines).

- Long term support. Regular distros like Debian and AlmaLinux offer free 5 and 10 year support cycles which means maintenance can be done every 1 or 2 years. On the other hand, immutable distros would require much more frequent maintenance, once every 6 months. A weekend every 6 months is a sizeable part of my time budget for hobbies.

One aspect in which immutables distros have improved a lot is in resource usage. They used to require significantly more disk space and have slightly higher minimum requirements than regular distros, but that doesn't seem to be the case anymore.

I'm not seeing any discussion about disk space when using immutable distros. I was running nix for a while and generally loved it. I know I can run nix-gc to clean up unused components. But, when I'm using docker I'm constantly running out of disk space. Again, I know how to use docker system prune, but it's an annoyance.

The discussion in the article talks about using containers and flatpak and snap and all those things bundle dependencies and really swell the disk usage requirements. Is there a good solution other than owning a massive SSD?

It isn't as big a problem for servers which don't change as often and where you need instant rollbacks, but I'm using immutable (or atomic distros like nixos) on my laptop and having trouble.

It makes me think I'm not using these systems correctly.

After a couple years of running manjaro I ended up switching to bazzite, a fedora silverblue based distro. For the past years, I stopped being a tinkerer, and started turning on my personal laptop less and less. But when I did, I'd find that doing an update would break things, and lead to hours of figuring out what broke, or why an update wouldn't install. It was so incredibly frustrating. My personal circumstances just changed so that I don't have the time to spend on those shenanigans anymore. I looked at Nixos for a long time, but the steep learning curve always held me back. And a fedora atomic desktop started to look pretty good, but it took me to get so fed up with the not being able to do an update after a couple months without things breaking again that I got over the fact that I would probably need to switch to GNOME or KDE to run a well supported atomic desktop. I got over that and settled on bazzite with gnome, because it's promise of setting up my hardware for casual gaming without effort. I've changed a couple months ago and honestly, it's made Linux fun for me again. The things I don't want to have to tinker with, the ui, desktop, software, it all just works and seems very stable. Software is installed with flatpaks, appimages, or in distrobox. If I want to tinker, I do what I always used to do; use docker (podman and distrobox on fedora). Its been an absolute pleasure so far, with hardly a learning curve for me (based on previous experience and practices I suppose). Highly recommend.

Been running Kinoite for a good bit (~1 year). I'm a bit over it. Love the idea of immutability, but rebooting every time I get a new system image via rpm-ostree, which is often, is tiresome. Of course, I could update less frequently; alas, habits formed from years of using rolling releases.

I switched to EndeavourOS. Between flatpak and brew and mise, I have relatively well sandboxed applications. This gives me most of the benefits of the immutable OSes, although nowhere near as rigorous, obviously. For a technologist, though, it's fine.

I just went all-in on Bluefin DX. It’s my first time using Linux where almost everything worked out of the box, even my 4070. Had to disable Bluetooth to get suspend working, but otherwise, this is the year of Linux on the desktop for me.

Isn't NixOS immutable? If so, surprised it wasn't mentioned.

For me: no.

I want full control over my system. Immutability means leaving part of that to the OS developer. Definitely don't want that. Even though it's ostensibly better for security (though it's only really making one step in the kill chain harder, which is establishing persistence).

I run Fedora Kinoite full time on my primary machine, and it's great. Obviously a bit of a learning curve, but if your workflow can be achieved using Flatpaks and Toolbox, it's fine. You can (and I do) layer packages but I have only 3 or so I need to layer (asusctl, supergfxctl and asusctl-rog-gui).

My only real gripe is that Firefox still ships as an rpm in the base image. I understand that they want to include a working web browser at all costs, and I don't think they can distribute the Flatpak version with the base image, but it's annoying that I have to mess with the image (removing Firefox) to then re-install the (more up to date) Flatpak.

I've been on Manjaro (arch based) for the past four years. It's mostly been fine but I've had to recover it from a botched Grub update once (an update randomly self destructed its configuration), which wasn't fun. But after four years it's in good shape, everything works, I run the latest kernel, etc. I have zero reason to wipe its installation and reinstall it again. Most other Linux distributions never lasted four years until I found a need to reinstall them or install some newer version.

And it's Linux so regardless of the distribution you'll be dealing with some amount of weird shit on regular basis. Has been true since I cycled home with a stack of slackware floppies almost thirty years ago. There's always configuration files to fiddle with, weird shit to install, etc.

But an immutable base OS makes a lot of sense and it's not mutually exclusive with that being updated regularly. Containerization is the norm for a lot of server side stuff. Effectively, I've been using immutable server operating systems for almost a decade. It's fine. All the stuff I care about runs in a container. And that container can run on anything that can run containers. Which is literally almost anything these days. I generally don't care much about the base OS aside from just running my containers hassle free on a server.

Containerization would make sense for a lot of end user software as well. IMHO things like flatpak and snap would be fine if they weren't so anal/flaky about "security". Because they are protecting a mutable OS from the evil foreign software. Running a bit of software that needs a GPU isn't a security problem, it's the main FFing reason I'm using the computer at all. Or own a GPU. This needs to be easy, not hard. And it shouldn't need a lot of manual overrides.

If I run a browser or things like Dartable, I usually have no reason to run them in crippled/unaccelerated mode. Sorry that's not a thing. It's the main reason I bypass flatpak on Manjaro for both packages. And I bypass PAC as well because I trust Firefox to have a good release process. So, I use the tar ball and it self updates without unnecessary delay. Which considering a lot of its updates are about security is exactly what I want.

Same with development tools. I use vs code and intellij. Both can self update. I have no need for a third party package manager second guessing those updates or dragging their heels getting those updates to me.

Yes. I am already running and Aeon desktop base system with gnu guix for the userland.

It's great.

I was about to ask why openSUSE Aeon when the normal Tumbleweed supports immutable mode where / is mounted read only, when I realized that they actually removed it in https://bugzilla.opensuse.org/show_bug.cgi?id=1221742

But I'll share my experience: I think an immutable / really is the way forward. Just the ability to roll back and boot using an older snapshot is great: I have had an update break the boot, but I have the option of running a single command to roll back while I investigate the issue. At the time the issue happened I was busy with life and I simply rolled back and used that version for three months before I had time to investigate.

Strictly speaking this does not require the current / to be mounted read only, but merely requires periodic bootable snapshots be taken and these are available to be used as a read-only /.

I worked a while with Silverblue, it is great, but they should use Distrobox instead of Toolbox. In Distrobox one can also encapsulate the home folder and one can export a link to a software running in a box to the outer system. The last one is pleasant for example with VS Code, which will only work properly when installed in a box.

I am using Fedora Kinoite for a year now. It is finally the stable desktop Linux experience I was looking for. The limitations people are constantly talking about really don't seem like a big deal to me. For everything not available as a Flatpak there is distrobox and there is always layering as an escape hatch.

Is this how embedded folks make sure that a device starts with exactly the same installation every time a machine is booted?

I wonder why embedded products like Nvidia Jetson do not come with an immutable Linux (and instead are based on Ubuntu which updates itself on every opportunity via apt and snap and whatnot).

OpenWRT is pretty much the oldest still running (and popular) with UCI. There's the classic nvram ones, but those are hardly manageable manually.

I've been using immutable distros for a couple years right now (Silverblue and openSuse MicroOS/Aeon now), and all I can say is they're much, much better than "normal" distros.

Containerized apps are nice, containers for development is nice, but you can have those in "normal" distros with a lil work setting things up.

The real killer feature is you can have a bleeding edge system with zero fear of breakage.

My first immutable distro was Illumos-based SmartOS. Everything the system needs is read from a read-only USB stick and run from RAM. I wish more distros worked that way. A recent submission on here gives me hope: https://news.ycombinator.com/item?id=42428722

I suppose TinyCore Linux in its default configuration also counts.

i don’t feel like immutable distros are ready for prime time because there are still some really big limitations with flatpaks that will take time to resolve. using a secondary drive on steam is still painful and works inconsistently. 1password can’t talk to firefox to unlock it. applications like steam can’t share rpc status with discord

Configuration as code has come as long way too along with these immutable OSs. For example I do not miss messing with preseed or kickstart files (I preferred working with kickstart files). Ignition / butane I find is much easier to work with and is a core part of configuring the OS.

I like immutable distros. What i do not like is that developers and maintainers do not give possibilities to admins and powerusers to build a immutable core themselves. This removes choice and learning experiences for the customer/user/admin.

Maybe this will finally change.

I don't use Snap on my Ubuntu Desktop systems because I don't like apps secretly updating without my awareness and also for the immense amount of additional disk space used by Snap.

Having said that, no, I don't see any usage of immutable Linux in my future.

Are there any immutable distros that cleanly divide system/desktop and end-user programs, with only the former being immutable and the latter being business as usual for desktop Linux? So the kernel, drivers, and KDE/GNOME would be fall into the immutable “core”, but apps like Firefox, Krita, and Anki would be in a mutable space managed by a traditional package manager like apt.

Just wondering because it’s really just the system itself and my desktop environment that I find the benefits of immutability most pertinent, whereas it’s something of a bad fit for applications with the woes flatpak and friends bring for desktop integration and such.

If you also need determinism and full source bootstrapping (you care about supply chain security) check out https://codeberg.org/stagex/stagex

I don't really understand the exact problem that immutable distros solve. Seems like it's some vague "instability" in normal distros?

> An immutable Linux distribution has its core system locked as read-only. This ensures the base operating system remains untouched during normal use, protecting it from accidental changes, unauthorized modifications and corruption.

So, in other words, I'm using an immutable system already! (Windows 11)

I’m honestly surprised Immutable distros is so controversial. I get why people choose not to do it, but I don’t know why I see so much hate towards them in a lot of Linux communities.

SteamOS is immutable and incredibly successful. macOS (not Linux of course) is also immutable and very successful.

As long as the OS’s have a concept of overlays, an immutable system rarely gives up much in the way of flexibility either.

They are not for me, but I am glad they exist.

I've been meaning to fully commit to GNU Guix one of these days, now that Plasma has fully landed. I've tried Fedora Kinoite in the past, but I can't handle Plasma without Oxygen. I know that Kinoite has some kind of a way to force packages to be installed into the base system, but it kinda feels like it defeats the purpose.

A new breed of distros for sure but how immutable is it, really? What I'm interested in knowing is the mechanisms and techniques in place for making sure no one can change any core components of the system. It's just like randomness. At first, it sounds super secure but we all know nothing is truly random

I went to check and see if proxmox had any immutability proposed for it yet, and I came across this: https://github.com/ashos/ashos#proxmox

I’m not quite sure what’s going on here yet, but seems interesting

You don't need a whole new distro

    $ apt search btrfs apt
    Sorting... Done
    Full Text Search... Done
    apt-btrfs-snapshot/noble,noble 3.5.7 all
      Automatically create snapshot on apt operations

Not sure how NixOS didn't make it to the list.

I am now running EndlessOS for a while and i love it: Its a bit like going back to the home computer days when the OS was residing in a ROM and you didn't really have to care.

Somehow I managed to answer no to all the questions…

10k lines quick browsed the top 500 meanwhile all everyone cares about 2024 in the year of reducing costs is arm64 not distro flavor.

This just sounds like a problem solved a long time ago in the embedded space for using squashFS for the bootable Linux image.

The term "stability" should not be used outside of the major Linux distributions such as Debian and Fedora. For a distribution to be stable over the long term it needs a large enough community, a stable governance model, and a reasonable build system where one maintainer cannot take unilateral action without it being discovered.

A cute name and a university student somewhere does not constitute stability, no matter how good the intentions. It's not a bad thing, but you have to know what you get yourself into. Most of the distributions listed in the article belong to the latter category.

Immutable systems are great for embedded, network equipment, appliances and industrial applications, and specialized distributions for those applications have largely been immutable for a long time already. Nobody really wants an immutable system for their main desktop, because working is all about mutating state. You write may write documents, save bookmarks, install plugins, or try new software. Those are the things really immutable systems like kiosks wants to disallow.

So in order to make for usable software these desktops generally split your system into a mutable user part and an immutable system part. That's basically how unix-like desktops have worked since forever. Stuff in /bin and /sbin is only changed by the package manager. So the fit is quite good, but it also means it really isn't as useful as it's made out to be. That's why most people don't use them.

The use case is mostly for rolling back updates, not really running from readonly filesystems or preventing change in other ways, but most distributions already do that. You can roll back updates with both dnf and apt. It's not perfect and doesn't always work, but mainly from a lack of testing. With snapshots it's pretty much infallible though.

My recommendation if you really want something that "just works" is to install one of the major and time tested distributions. Pick Debian if you don't know what to choose. And then learn how to use it. Anything these tiny experimental distributions offer, such as running off read only filesystems or rebuilding it for your brand of cpu, or testing a new desktop environment, is likely possible in Debian too. With the added benefit of it being around in 20 years. And the core distribution is less likely to break in some way because some maintainer found inspiration for something. As long as you don't run untrusted stuff as root, stay out of the system files, and generally let the package manager do its job, you're going to be fine.

What I would like to see a desktop distribution work on is basically the same things as 20 years ago which still isn't really done outside some exploratory work (probably because it's actually hard):

- Packages on a user level where it is easy to install new stuff without touching the system area. More tricky in practice than in theory because of state changes to configuration files, saved file formats etc. But some should be easier than others.

- Desktop software service accounts, just like we do for server software. Mostly relevant for larger packages such as Firefox, Libre Office, movie players.

- Integration with popular third party package managers from the language ecosystems. Most language packages are anemic. All the powers that a package manager gives, reporting, listing untracked files, listing changes, rolling back updates, should be available for them by integrating directly with them. Package definitions should be able to be imported without manual work.

- Package managers should have at least some knowledge of an application's access patterns to help with application confinement. Still today things like selinux policies are packaged are separate entities and managed with external tools, which brings a lot of complexity since all possible configurations must be supported there. A package manager knows more about the system and could handle these files. Confining desktop software is a usability problem more than a technical one, but it is clear that desktop environments needs something to build on to make it practical.