Supporting Linux kernel development in Rust

> I'm concerned about the gradual move from GCC to LLVM. The lack of copyleft protections on LLVM means that it's much more dependent on corporate sponsorship, and means that there's a risk that major improvements to LLVM compilers will only become available as proprietary products.

As someone who works within LLVM professionally, I don't think this is particularly likely -- compilers are massive and complicated beasts, and the "moat" for proprietary improvements is small: they're either niche and therefore not of interest to the majority of programmers, or they're sufficiently general and easiest to maintain by releasing back upstream (where Apple, Google, and Microsoft will pay a small army of compiler engineers to keep them working).

Your concern is the one that kept GCC from stabilizing its various intermediate representations for decades, which is why virtually all program analysis research happens in LLVM these days.

Edit: To elaborate on the above: neither Apple, nor Google, nor Microsoft wants to individually maintain LLVM. Microsoft appears (to this outsider) to be actively looking to replace (parts of) MSVC/cl with the LLVM ecosystem, because they're tired on maintaining their own optimizing compiler.

That makes me sad as well, but I also think it was largely self-inflicted. GCC took much too long to realize that people needed a common code-generation framework, and be willing to support that and work towards it. LLVM was designed from the start to be such a framework.

I don't see how, personally. There are tons of thriving open source non-copyleft projects. And there are tons of GPL violators who go on business as usual. I think perhaps copyleft licenses are overattributed to the success of such projects.

I think GCC’s resistance to modularisation and the resulting rise of LLVM has proven beyond doubt that the greater risk is that by putting up barriers to corporate contribution there will not be any major improvements in first place.

I am even more concerned about LLVM monoculture.

LLVM is just a proprietary-able version of GCC. And thus compiler authors mostly focus on benchmarks while not caring about other metrics such as compilation speed, correctness and codebase quality / ease of contribution. There appears to be some tribal knowledge requirement to add a new target.

With all the hype and apple funding LLVM gets, it could have been better.

A GCC back-end for Rust is incredibly important for FLOSS. Even if the world doesn't switch from C to Rust overnight, Rust creates a negative halo effect for GCC, casting doubt on GCC's future relevance. Rust support in GCC would be a huge win for both projects.

Can you give an example of a non-copyleft open source product where the major improvement is proprietary?

Yes, we are short of getting back into the days of Shareware and PD, and then the anti-GPL crowd will be happy with the outcome.

As commercial software user, I don't have much issue with it, after all I have been coding since the early 80's.

And in spite of it and my occasional Linux rants, I am thankful for GPL, because without it there wouldn't be a UNIX to install at home to get my university work from DG/UX done without having to spend one hour traveling into the campus and fighting for a terminal.

Without Linux + GNU based userspace, the commercial UNIXes would all be around.

GCC changed from GPLv2 to GPLv3 which is a stupid license for a compiler (Thanks FSF). I'm happy that LLVM exists, happy that Microsoft, Apple, Linux, and BSD's can use it without that GPLv3 BS.

That's a front end?

How would that make things any easier? The kernel supports being built with clang now, and there’s no issue linking an object built with another compiler into a module either.

The point of this LPC session was how to incrementally introduce Rust in the existing Linux kernel, with all its existing drivers and syscalls and similar. A rewrite would indeed be a daunting and problematic proposition.

There are kernels written in Rust, such as Redox, but those are separate projects, and that's not what this conference session or article are talking about.

Standing reminder: the Rust project is not a fan of rabid Rust over-evangelism (e.g. "Rewrite It In Rust", "Rust Evangelism Strike Force"), and sees it as damaging and unhelpful. We discourage that kind of thing wherever we see it. We're much more in favor of a measured, cautious approach.

I was thinking about something similar recently - if the hardware drivers could somehow be abstracted from the rest of the linux kernel (as if...), then suddenly a ton of experimental kernels could have widespread hardware availability.

It would probably shake up the OS ecosystem to no small degree, but I imagine the fresh ideas that could be experimented with by non-experts would be hugely beneficial. Isn't that the big argument against so many new projects? "It looks cool, but the hardware support isn't there."

FreeBSD might be a better target though, since that's specifically designed to be compatible with everything.

ESX once ran this way, it had native drivers and Linux drivers, where it would run Linux as a vm and have it handle drivers for unsupported hardware. theres some more documentation on Wikipedia[1], it later switched to using vmklinux as a shim for those drivers.

[1] https://en.wikipedia.org/wiki/VMware_ESXi#Architecture

This conference talk considered a re-write out-of-scope; they were only discussing how new code could be written in Rust.

Interesting idea, though!

L4Linux was (is?) a similar project: https://l4linux.org/overview.shtml

The idea of that project was to turn Linux into a user mode application run by a microkernel; rewriting that microkernel in Rust would essentially give you what you're suggesting.

this is a very old idea. ndiswrapper did this for network drivers back in 2003, and freebsd implements shim layers for linux graphics drivers, linux applications, some solaris kernel modules (dtrace, firewall, others?). the main problem is that nowadays, modern simple hardware has mostly standardized on a fairly small set of APIs (e.g. SATA/SCSI/NVMe for the majority of disks, UVC for the majority of cameras), and modern complex hardware requires complex shims. see: the amount of work put into Linux graphics kernel APIs.

> "One of the challenges to building an entirely new kernel is the vast amount of hardware support in the form of drivers [..]"

This reminds me of an interview with Linus Torvalds from many many years ago, when he was still very young. The interviewer asked him if he was afraid of getting replaced by someone young and hungry. Torvalds shrugged it of with something along the lines of: Nah, no one likes to do driver development.

I might be wrong, but isn't there already a precedent for something similar in the form of the XNU kernel? From a cursory glance, it is basically a BSD and a Mach kernel running side by side.

I'm not sure why you'd need a VM for this. It should be much easier to implement bridges for major kernel apis like character devices, as I understand some BSDs already do?

Xenomai uses this model, I believe. It is written in C, but I don’t see why you couldn’t do something similar with Rust.

in some ways, this is how Service Console works in vmware ESX: https://en.wikipedia.org/wiki/VMware_ESXi

Yup - that got a very brief mention in the article as "Google is working on automated C++ conversions"; there's also https://github.com/google/autocxx which builds on top of it.

C++ is a richer language than C and has more information about ownership (e.g., if you return a std::string you know how ownership works; if you have two arguments that are a char * and a length it's much less clear), but additional annotations in the kernel's C headers to convey this level of information in a machine-parseable way would be awesome.

This is mostly just a random tidbit but one of the fascinating things to me about Swift <-> ObjC interop is that it's not just that Swift was modified for ObjC, but ObjC was modified for Swift as well. In general it works pretty well and I wonder how much of that is because Apple can modify ObjC whenever it wants.

Not even remotely as difficult as Swift. Rust can call into C at any time. It is ABI compatible and has no VM

Do you have some kind of official blog for this? I'm interested in BPF and currently learning Rust, would love to learn more about this.

It does, problem is i am in IST Timezone, Will that work for the team ?

The vast majority of the text in that post is inaccurate and hyperbolic. Linux is not "forcing adoption of HDCP" (it's just something Linux has a driver for), Rust does not require internet access to use (and the trademark comments are not particularly accurate either), the Kernel Self Protection Project is alive and well and making new improvements in every new version of the kernel (see Kees Cook's regular blog posts for details), systemd is not spelled "SystemD", gettext has no dependency on Java (and the link in that post explains that). And in general, decisions about what technologies to use are made by those who show up and do the work to build viable solutions, not by people who snark.

It's a rant by an obscure one-developer distribution that formerly used the Linux kernel. Some people are never happy no matter what you do, and if you make the mistake of treating their rants as useful feedback, you get a list of 30 more demands before they'd consider gracing your little project with the honor of their all-important usage again.

If I'm understanding correctly, the only reason for not using Rust is... Rust is trademarked?

The licensing complaints about Rust apply equally to Python, which they package without objection: https://www.hyperbola.info/packages/extra/x86_64/python/

I don’t see how that conplaint is at all meaningful to kernel development.

It’s typical GNU/Pendantry.

The kernel will not be shipping a rust compiler, patched or otherwise.

Once again the GPL camp takes and relicenses a project so that further improvements they make can't be used by the original authors.

Shame really.

Maybe a GNU Rust compiler without the trademark is in order? The state of KSPP is not nice as well. I wish Ada was more popular. From what I know, it has much of the security guarantees rust has and it has been battle tested.