back
212 comments
This part seems fairly editorial:

>Debian Policy still cites the FHS, even though the FHS has gone unmaintained for more than a decade.

What ongoing maintenance would a file system standard require? A successful standard of that type would have to remain static unless there was a serious issue to address. Regular changes are what the standard was intended to combat in the first place.

>The specification was not so much finished as abandoned after FHS 3.0 was released...

OK.

>...though there is a slow-moving effort to revive and revise the standard as FHS 4.0, it has not yet produced any results.

So it is not abandoned then. A slow moving process is exactly what you would want for the maintenance of a file system standard.

>Meanwhile, in the absence of a current standard, systemd has spun off its file-hierarchy documentation to the Linux Userspace API (UAPI) Group as a specification. LWN covered that development in August, related to Fedora's search for an FHS successor.

Ah. Systemd/Fedora want a standard that they can directly control without interference from others.

Author here: It was abandoned. I linked to one of the former maintainers who said as much. The current effort is by a few people who asked the LF to take out over, and have (so far) done little after an initial flurry of activity. That, too, is covered in the other article I wrote about the FHS recently.

Prior to the group who started an update effort, it had not been touched in about a decade. That’s not slow-moving: that’s abandoned.

>> Debian Policy still cites the FHS, even though the FHS has gone unmaintained for more than a decade.

> What ongoing maintenance would a file system standard require? A successful standard of that type would have to remain static unless there was a serious issue to address. Regular changes are what the standard was intended to combat in the first place.

It's 2025, anything that wants to be considered modern (and everything should want that), needs to be undergoing constant change and delivering regular "improvements."

>>...though there is a slow-moving effort to revive and revise the standard as FHS 4.0, it has not yet produced any results.

> So it is not abandoned then. A slow moving process is exactly what you would want for the maintenance of a file system standard.

The FHS people to get off their butts. There's no excuse for that pace now that we have such well-developed AI assistants. They should be pushing quarterly updates at a minimum, and a breaking change at least every year or two. It's been obvious for decades that "etc" is in urgent need of renaming to "config", "home" to "user", and "usr" to "Program Files" to keep up with modern UX trends.

Counterpoint: Modern distro’s needs have evolved past the FHS in some cases, and everybody deviates from it slightly but incompatibly.

A standard does no good if it does not reflect reality. I think it is a worthwhile effort to try to bring it back in line with actual real world usage.

The /usr merge is an example of something that a modern FHS might reflect.
I'm very happy that the "independent standard" facade of the UAPI group fell, and its actions are now directly attributed to systemd's interests.
> What ongoing maintenance would a file system standard require?

adaption to _a lot_ of subtle changes to requirements

- very different security related requirements today

- very different performance related requirements/characteristics

- very different need for various edge cases

and lastly adapt based on what turned out to work well and what didn't

so some examples not already mentioned in the article

- /boot -- dead or at least differently used if you use efistub booting

- /etc/X11 -- half dead on wayland

- /etc/xml, /etc/sgml -- dead, should IMHO never have existed

- also why was /etc/{X11,xml,sgml} every explicit part of the standard when the spec for `/etc` already implies them as long as e.g X11 is used ??

- `/media` -- dead/half dead depending on distro, replaced by `/run/media/{username}/{mount}`

- `/sbin` -- "controversial"; frequent reoccurring discussions that it isn't needed anymore, didn't work out as intended etc. It was useful for very old style thin clients as `/sbin` was in storage but `/bin` was mounted. And there are still some edge cases where it can makes sense today but most fall under "workaround for a different kind of problem which is better fixed properly".

- `/tmp` -- "controversial", long history of security issues, `/tmp` dir per program fixes the security issues (e.g. systemd service PrivateTmp option) but requires having a concept of "programs" instead of just "running processes" (e.g. by systemd services or flatpack programs). Also `tmpfiles.d` can help here.

- `/usr/libexec` -- dead, nice idea but introduces unneeded complexity and can be very misleading in combination swith suid and similar

- `/usr/sbin` see `/sbin`

- `/usr/share/{color,dict,man,misc,ppd,sgml,xml}` -- should never have been in the standard they are implied by the definition of `/usr/share`; at least sqml,xml are dead. dict was for spell check/auto completion, except that neither works anymore like dict expects

- `/var/account` -- to specific to some subset of partially dead programs, shouldn't be in the standard

- `/var/crash` -- distro specific mess

- `/var/games` -- basically dead/security mess, I mean 99% of games today are user per-user installed (e.g. Steam) and even for such which are packed any variable download data is per user, making it shared creates a permission/security mess

- `/var/lock` -- as mentioned there are better technical solutions by now, e.g. using `flock` instead of "presence of file" and some other techniques. Tend to also avoid issues of crashed programs not cleaning up "lock files" leading to dead locks and needing manual intervention.

- `/var/mail` assumes a quite outdated form of managing mail which is quite specific to the mailing program, as it's very program specific it IMHO shouldn't be in the standard

- various legacy program specific, non "generic" file system requirements e.g. that `/usr/lib/sendmail` must exist and be a link to a sendmail compatible program and similar.

also missing parts:

- `/run/user/{uid}`

- `/var/run/user/{uid}`

- `/proc`

- `/sys`

- user side versions (e.g. from the XDG spec which is also somewhat in a zombie state from my personal experience with it , e.g. .config, .local/{bin,share})

- references to light weight sandboxing, e.g. per-program /temp etc.

- factory reset stuff (`/usr/share/factory`) needed for having a uniform way for devices sold with Linux and device specific distro customization(e.g. steam deck)

so yes, it's quite outdated

Well, it probably depends on which software's concern will be implementing a policy to prevent users from having permission to fill critical directories and prevent the system from operating normally, which is discussed in the article. Which is also a coordination problem because the most common user of disk is software itself, I think.

FHS seems to specifically imbue the user with the responsibility and consequences of filling up the disk.

A more neutral phrasing would be.

> Debian Policy still cites the FHS, and FHS has remained static for over a decade.

> The FHS 3.0 is clearly reaching the end of its useful life, if not actually expired.

Interesting take.

I think that the FHS is still extremely helpful for packagers, sysadmins and others so they won't stomp on each other's feet constantly. It helps set expectations and prevents unnecessary surprises.

Just the fact that one particular FHS rule might be outdated or even harmful doesn't mean that the FHS as a whole has outlived its usefulness.

Standards are a double edged sword though. They are great for getting everyone to agree to the "most correct" answer. But they also freeze evolution in place. What happens when your standard doesn't support contemporary use cases? What if it's at direct odds with, say, modern security practices?

FHS hasn't changed in years. Since then, sandboxing, containers, novel package schemes, and more are the zeitgeist. What does the FHS say about them?

yes but also no

every distro has defined their own new file system layout standard

sure they all started out with the common ancestor of FHS 3.0, but diverged since then in various degrees

and some modern competing standards try to fix it (mainly UAPI Group)

(And yes some people will go one and one about how UAPI is just a way for systemd to force their ideas on others, but if you don't update a standard for 10+[1] years and aren't okay with others taking over this work either, idk. how you can complain for them making their own standard).

[1]: It's more like 20 years, but 10 years ago the Linux Fundation took over it's ownership.

Debian systemd maintainer Luca Boccassi has recently pushed through and dismissed several problematic and undesired breakages as "niche cases" in a way I personally find antithetical to what I expect from Debian.

I hope they have a change of mind in their approach.

This is usual systemd maintainer behavior. Poetterings "I don't consider it much of a problem" is legend: https://github.com/systemd/systemd/issues/5644
There is Devuan, if you want to Debian but without systemd. I suspect though that "natively" non-systemd distros will be more consistent, personally I've found happiness with Guix.
Can anyone tell why systemd developers run fast and loose with what they believe and bully everyone with a stick made out of their ideas?
In this case, I think the upstream maintainer's response -- "Upstream systemd will do X, distros who want to are free to do Y" -- is legitimate. Consider the reverse: If systemd requires a writable /run/lock, then distros who want to be more safe won't really be able to (or will have to implement a much more intrusive patch).

Looking from the outside, it looks more that this is a failure of the Debian systemd package maintainer to follow Debian's rules. (Though since I'm not a part of that community, I recognize that there may be cultural expectations I'm not aware of.)

That has been their method since the beginning; why would they change from a tactic which works for them?

The central problem with systemd is that they don't want to let you go about your business, they want you to conform to their rule.

Linux is 34 years old, and some of the Unix-ism borrowed are even older. There is genuine cruft that has downsides. Different relative priorities of backwards comparability, maintainability and the various issues the legacy issues cause are reasonable.

Systemd basically arose out of a frustration at the legacy issues so the whole project exists as a modernizing effort. No wonder they consider backwards compatibility low priority.

Because that's what they've always done, and it continues to work for them?

Systemd doesn't work for me, but it has taken over most Linux distributions, so clearly it's got something people want that I don't understand. That was the case for PulseAudio too.

Did you read the same article?

   * Their is an option for the old behavior.
   * It is a security issue and better solutions to replace exist.
   * FHS isn't maintained.
I think everyone involved would prefer updates to the applications, which fix the issue. Debian opted - for now - for reliability for its users, which fits in their mission statement. On Arch /run/lock is only writeable for the superusers, which improves security. As user I value reliability and security and that legacy tools remain usable (sometimes by default, sometimes by a switch).
well the primary author does work for Microsoft

a company that considers "consent" to be a dirty word

Probably because everyone lets them keep getting away with it.

Any time there's systemd criticism there's always a quick rebuttal "But it was too hard writing anything in any other init system before so stop complaining".

So there's enough pass being given from the start to Systemd and the developers because it always has been forced upon us.

In this case I assume their "fear" is that unprivileged users can exhaust resources (inodes, filesystem space) in an important tmpfs breaking the system. The proper solution for backward compatibility would probably be something like make /run/lock its own mountpoint, but they fixed it in their system (Fedora) so now it's no longer their problem. Just be thankful their software is portable to such strange niche operating systems like Debian. /s
First before venting I'll say this: thanks to stuff like hypervisors, VMs, non-systemd distros, minimal immutable Linux distro like Talos (made to run Kubernetes with a minimum number of executable) and OCI containers where, by definition, PID 1 is not systemd, there's thankfully a real way out of systemd, even on a Linux stack.

And I think more people should look into being, once again, 100% systemd-free.

> Can anyone tell why systemd developers run fast and loose with what they believe and bully everyone with a stick made out of their ideas?

Because the goal is to take control of Linux. That's why systemd is PID1. That's why Poettering works for Microsoft.

The real question is: why was that ultra-convoluted xz backdoor attempt only working on Linux systems that did have systemd? People shall try to wag the dog saying "but it's because this and that made is so that xz was loaded by OpenSSH, it's got nothing to do with systemd". It's got everything to do with systemd.

And the other question is: how many backdoors are operational, today, on systems that have systemd?

Systemd is Microsoft-level bloat, running as PID 1, spreading its tentacles everywhere in Linux distros, definitely on purpose.

Poettering is moreover an insufferable bully, as can be seen once again.

From TFA:

> So what do you recommend how to go on from here? Change Debian policy (as asked in #1111839), revert the change in systemd, find a Debian wide solution or let every package maintainer implement their own solution?

I suggest Debian just drops systemd once and for all. Debian can still be made systemd-free but it's a hassle. Just make Debian systemd free once again.

Meanwhile you'll find me running systemd-less distros on VMs and running containers giving the PID 1 finger to systemd.

I can't wait to switch my Proxmox to FreeBSD's bhyve hypervisor (need to find the time to do it).

But most of all: I cannot wait for the day a systemd-less hypervisor Linux like Proxmox comes out.

It's coming and people who write stuff like: "Don't use Docker, use systemd this and systemd that" are misguided.

systemd is to me the antithesis of what Linux stands for.

I hope Debian gets pissed enough at some point to fully drop systemd.

P.S: one of my machine runs this: https://www.devuan.org/ and honestly it's totally fine. So yup: power to all those running systemd-less distros, BSDs, etc.

I still cannot understand how it came about that everyone adopted systemd, despite it being a showcase of the most selfish, reckless and irresponsible maintainership ever. And it's not like it's something new, it's been like that for what, 10, 15 years now?
Another conflict: in Debian Trixie (current stable release), when you run `systemctl status …` you’re given the rather scary warnings:

    State: degraded
    …
    Tainted: unmerged-bin
This is because Debian has a separate /usr/bin and /usr/sbin. There are no plans to either back off this warning by the systemd maintainers, or to patch it out by the Debian maintainers, and also no plans to merge /usr/sbin and /usr/bin.

So we’re left with the absurd situation that Debian users are being told the default clean install is ‘degraded’ and ‘tainted’ by its main init system!

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1085370

https://github.com/systemd/systemd/issues/35438

If this is the case, I am surprised no one has patched this anti-feature out of the source. Both sides made that suggestion directly in the bug reports you linked. I expect it should be positively trivial to implement a patch, but someone must step up and do the work.
The last paragraph in the article states how Fedora uses lockdev(3) to somehow address the issue. It's not clear to me how a library that exposes a standardized management interface to these locks enables non-root users to create them without a world writable lock directory. Is it that lockdev(3) allows the system to use a different directory?

Secondly, it's suggested that the /run/lock directory and/or the solution of acquiring a lock for a shared resources is dated -- but, no mention of the "modern" way to address this? Is it implied that systemd offers a solution to this? What is it?

PS lockdev has such a poorly written manpage.

They should've never accepted systemd into Debian, or other submarines to monopolize knowledge around RH/IBM and steal attention away from original software packages (gnome, wayland, podman, hell even rpm, etc.)
It's been a very long time since I heard about uucico (Unix-to-Unix Copy-In Copy-Out program, part of the UUCP suite). Glad to see it's still being shipped! I wonder if any network uses it.
I've been running it the last year or two to get e-mail to a vintage DOS BBS that had a UUCP package. I was pleasantly surprised it was out of the box usable on both CentOS and Debian, and Postfix still ships with example UUCP email config.
Non-clickbait title -- Debian Technical Committee overrides /run/lock permission change
More-clickbaity title -- Debian Technical Committee tells its doofus maintainers to stop worshipping Poettering so much
and also might be just temporary

like overriding it now makes a lot of sense, there needs to be grace periods etc.

but we live in a world where OSes have to become increasingly more resilient to misbehaving programs (mainly user programs, or "server programs" you can mostly isolate with services, service accounts/users etc.). And with continuous increases in both supply chain attacks and crappy AI code this will only get worse.

And as such quotas/usage limits of a temp fs being shared between all user space programs like lvm2 and dmraid is kinda a bad idea.

and for such robustness there aren't that many ways around this change, basically the alternatives are:

- make /var/lock root only and break a very small number of programs which neither use flock nor follow the XDG spec (XDG_RUNTIME_DIR is where your user scoped locks go, like e.g. for wayland or pipewire)

- change lvm2, dmraid, alsa(the low level parts) and a bunch of other things your could say are core OS components to use a different root only lock dir. Which is a lot of work and a lot of breaking changes, much more then the first approach.)

- use a "magic" virtual file system which presents a single unified view of /var/lock, but under the hood magically separates them into different tempfs with different quotas (e.g. based on used id the file gets remapped to /run/user/{uid}, except roots gets a special folder and I guess another folder for "everything else"???) That looks like a lot of complexity to support a very small number of program doing something in a very (20+ years) outdated way. But similar tricks do exist in systemd (e.g. PrivateTemp).

kinda only the first option makes sense

but it's not that it needs to be done "NOW", like in a year would be fine too, but in 5 years probably not

Why is that non-clickbait? Honestly "Debian Technical Committee overrides systemd /run/lock permission change" might be a better title than either, I don't know whether the thing or the actors are more interesting here. But you can only say so much in a title.
Yeah. I expected better from LWN.
The "somebody might mailicuously exhaust memory via /run” argument seems silly IMHO... you can trivially put a hard limit on its size via the tmpfs mount option. I guess if you bend over backwards that's still a DOS in that new files can't be created in /run once it is full, but come on...

There is support for quotas in tmpfs! /me runs and hides under desk to avoid fruit being thrown at me

The problem is /var/lock is still used for various system root components.

Which includes services like lvm2, dmraid and audio drivers.

So you need at least a different /var/lock for "root" and every one else.

So you now can either fix all the root tooling to use a different lock folder. Which would break a lot of things.

Or you break a very small number of very old tools which (mostly) decided to neither use use flock (from 1996) nor follow the XDG spec (XDF_RUNTIME_DIR) from 2003.

So kinda obvious choice what way to go with ;), it just should be coordinated better and communicated better.

And yes you want quotas on tempfs on XDF_RUNTIME_DIR, without question. At least for a system more robust against misbehaving programs.

To be clear a lot of this security concerns have been irrelevant/ignored with thread models from the 2000th. But times (sadly) have changes and you shouldn't just blindly trust user space programs anymore. Even if we ignore malicious programs just thing about all the bugs AI will sneak in. And honestly I'm worried that Desktop Linux will fail to adopt to this changes. Server Linux is clearly adopting, but also more in the non-gnu Linux ecosystem while the more FSF/GNU parts of Linux seem mostly stuck in a past which doesn't look like it will have a future :/. This sucks without question but I just don't see a future without a lot of supply chain attack and AI coding induced crazy misbehavior of user space programs.

PS: Even if you just want a steam deck with the same degree of robustness as console (i.e. a game going rogue will have a hard time to hang the console fully not matter what it does, i.e. you can always press the menu button and close/kill it) you need this kind of subtle changes. And many other.

Wow, talk about making a mountain of a mole hill.

Letting upstream systemd single-handedly define what directories exist with what modes in your distro has never been the intended Modus Operandi.

Debian has a huge selection of packages available for it and clearly is going to have more headaches when it comes to preserving compatibility with all that software.

This is a trivial matter for Debian to handle appropriately, while systemd stays focused on its current priorities. I'm surprised this is being talked about at all outside the appropriate mailing lists... slow week for linux news?

Is anyone surprised that the systemd folks basically said "fuck you, we'll do what we want to" to everyone else?
> Any process could write as much as it wanted to /run, which

> could effectively DoS the system by exhausting space or inodes

Does ulimit and quotas sufficiently address this concern?

Somehow I feel that if all the time that has been invested in debating and discussing this had been spent on patching the affected apps, the problem would be properly solved.

I mean, yeah, I get it, systemd bad, democracy good, but these world-writable lock folders are actually a huge pain, and adding some shim code to upgrade to a more secure solution seems achievable?

As always, it will just take another decade until debian has figured it out.
Debian discussions make political discussions seem quick and fast acting by comparison

> He said that he uses cu ""almost constantly for interacting with embedded serial consoles on devices a USB connection away from my laptop""

Whyyyyyyyyyyyyyyy

There are a million better ways of doing this.