In a nutshell: It's fine to use LLMs to answer questions, analyze, distill, refine, check, suggest, review. But not to create.
back
5 comments
Yes, and I think the reasons set out in TFA are interesting, which on my reading are:
1) Code is the least interesting bit in some ways - they’re trying to build not only rust but a community of experts around rust which is able to maintain and improve it in the long term and the authors of prs need to understand the code deeply and show a willingness to be part of that long run. Llm-generated code even if it looks polished doesn’t indicate a human author who deeply understands it and is willing to be part of that long term maintenance.
2) They don’t have (and don’t want to enforce) a community consensus that llms are good or llms are bad. They just want some rules written down so everyone knows where they stand and people of either camp (and none) can get on with it.
Seems a pretty reasonable approach to me although I don’t really use rust much so don’t have a vested interest.
No codebase worth being deeply invested in is deeply understood by anyone, including the original authors. The eternal maintenance argument is tiresome. If there is a problem in need of fixing, you figure it out, you do not depend on some author being available, and LLMs are exceptionally helpful in this regard as well. Then, if the LLM has trouble figuring out, so will another human. It is a code smell.
I disagree. For sure there doesn't exist anyone out there who knows every detail of Rust (or any similarly complex project) because that's simply not possible, but domain expertise is a very real thing and simply invaluable to the long term success of any project.
From my own work experience, the "dinosaurs" are the most valuable people in the whole org, because oftentimes understanding the code isn't nearly sufficient to make good long term choices. You very often need to know the historical context of why certain things were made in order to not shoot blindly in the dark with random changes.
Strongly disagree. The obvious counterexample is in the foss world is ffmpeg, which I would submit is deeply understood by Fabrice Bellard. As a sibling points out, there are also various worthwhile single author projects.
Outside the world of foss there are for example multiple games that are highly respected that were all entirely or largely developed by a single developer. Rimworld, Songs of Syx, Dwarf Fortress, Ostranauts and many others. In a proprietary context I have myself deeply understood codebases and worked with others who have also.
That being said, you are arguing against a straw man that neither I nor the rust article propose. They are not saying maintainers need to understand all the code they are saying the person submitting a pr needs to deeply understand that pr. Why? Because they want to build a community who deeply understand various aspects of rust so when things need maintenance there are experts around who even if they don’t know a particular piece are invested in the success of the project and know enough to get up the curve and fix whatever needs to be fixed.
I have a bunch of single-author projects that I wrote myself that I no longer have a deep understanding of. None are near as large as your references.
Fabrice Bellard is obviously a superhuman, but I expect even he relies on re-analyzing code he previously wrote and has subsequently forgot. This doesn't take him long because he knows his own conventions and proclivities. But I'm confident he doesn't "deeply understand" the entire code base. He has a shallow understanding of the entire code base. He has a deep understanding of crucial parts of it. And he has the ability to quickly re-establish a deep understanding of any piece.
> The obvious counterexample is in the foss world is ffmpeg, which I would submit is deeply understood by Fabrice Bellard
Bellard hasn't worked on FFMpeg since 2003.
There are all kinds of projects out there that have had significant impact on the world, which were written and maintained over the course of years by a single, highly capable developer. The way people talk about code nowadays, just because they have more powerful authoring tools, is frankly disgraceful. Imagine if modern construction techniques had become a reason to stop caring about the natural beauty of wood grain.
People are reading this as a slight against single authors, but it is a slight against the belief that anyone deeply understands complex code they wrote even a week ago. It is just a bullshit belief. Yes, you build an intuition about it, but you need to constantly refresh your understanding. That is just how the brain works.
Now, if you have program that is highly complex, written by a single author who is not you, hinging on the ongoing goodwill and capability of that single author, that is a liability you do not want to have, operationally. Again, that is just how it is.
Creation is still allowed:
>Pre-arranged, non-critical, high-quality, well-tested, and well-reviewed code changes that are originally created by an LLM are allowed, with disclosure.
It's technically an experiment, but it's definitely good to see they're keeping an open mind about this.
> Experiment: LLM-created code changes intended for review
> We leave space open to experiment with LLMs to inform future policies.
> This experiment is meant to inform future non-experimental policy, not to serve as the perpetual LLM usage policy.
Hmm, that doesn't sound too bad. So you basically get to use a tool to help with the tedious parts, without being allowed to use the same tool to pass off said tedium to others.
This closes the gap. A key insight. A load bearing difference.
Now, is writing code considered tedium, for a project like Rust?
This implies that "create" is the hard part that we should not offload to machines.
But IME all of the other things are the hard part.
So basically it's fine to get the LLM to think for you and give you the answers as long as you understand Rust syntax well enough to type it all out yourself?
This also seems like it will produce worse code, because it's preventing the LLM from doing the easy/boring part that it's good at.
They explicitly say that "create" is not the hard or valuable part. I suggest you actually read their policy instead of just the headline and then reacting to comments.
Sounds like the beginning of the end of rust to me
AI is a jet engine - Banning is for creation is like saying "we require you to go by ship/car/propeller-plane"
Yes, a policy that says AI slop PRs will not be tolerated. But I've had AI track down real hard to find bugs in several projects and provide correct fixes. I've also had AI build new features and new products. The project that ignores this acceleration will eventually fall behind the ones that embrace it.
I love how your analogy actually works with the real world. You can't fly a jet plane until you've got all the various licenses and logged flying hours in prop planes first.
"AI is a jet engine "
and rust just wants to ensure you are a capable pilot that can actually steer a plane before stealing everyone else's attention
Why does a language need higher developer velocity? Why is acceleration needed?
Acceleration makes sense for end user applications, where you're allowed to break whatever you see fit, but I don't see how it helps with core infrastructure like a programming language, where every mistake is multiplied a million fold.
Exactly this. There's plenty of boring software out there where writing the actual code is a chore. That's where AI works pretty well, as long as the people at the controls understand and can explain the problem and non-functionals.
For projects like Rust, coding velocity is not the bottleneck, speed / productivity is not a priority.
you're free to fork and improve it with your favorite harness. Go get 'em tiger.
I wouldn't trust the majority of AI bros with a bicycle, yet alone a jet engine.