▲ 96 points
back
85 comments
> The prompt used to create the code should also be provided. The LLM-generated code should be clearly marked.
I have a feeling the people who write these haven't really used LLMs for programming because even just playing around with them will make it obvious that this makes no sense - especially if you try to use something local based that lets you rewrite the discussion at will, including any code the LLM generated. E.g. sometimes when trying to get Devstral make something for me, i let it generate whatever (sometimes buggy/not working) code it comes up with[0] and then i start editing its response to fix the bug so that further instructions are under the assumption it generated the correct code from the get go instead of trying to convince it[0] to fix the code it generated. In such a scenario there is no clear separation between LLM-generated code and manually written code nor any specific "prompt" (unless you count all snapshots of the entire discussion every time one hits the "submit" button as a series of prompts, which technically is what the LLM using as a prompt instead of what the user types, but i doubt this was what the author had in mind).
And all that without taking into account what someone commented in the article about code not even done in a single session but with plans, restarting from scratch, summarizing, etc (and there are tools to automate these too and those can use a variety of prompts by themselves that the end user isn't even aware of).
TBH i think if FSF wants to "consider LLMs" they should begin by gaining some real experience using them first - and bringing people with such experience on board to explain things for them.
[0] i do not like anthropomorphizing LLMs, but i cannot think of another description for that :-P
Agreed, it's almost like requiring that code always come with full transcripts of all the meetings where the team discussed the next steps.
> I have a feeling the people who write these haven't really used LLMs for programming because even just playing around with them will make it obvious that this makes no sense
This is one problem with LLM generated code. It is very greenfield. There’s no correct or even good way to do it. Because it’s a little bit unbounded in possible approaches and quality of output.
I’ve tried tracking prompt history in many permutations as a means to documenting and making rollbacks more possible. I hasn’t felt like that's the right way to think about it.
What you're describing isn't any different from a branch of commits between two people practicing a form of continuous integration where they commit whatever they have (whether it breaks the build or not, or is buggy, etc.), capped off by a merge commit when it's finally in the finished state.
Eh, i do not think these are comparable, unless you really stretch the idea of what is a "commit", who makes it and you consider all sorts of destructive modifications of branch history and commits normal.
Huh?
Hal and Dave work together. Hal is going home at 6:00 PM, but before it's time to leave, Dave tells Hal to go ahead and start working on some new feature. At 5:50 PM, Hal hits Cmd+Q, saving whatever unfinished work there is and no matter what state it's in and commits it to a new development branch with the commit message "Start on $X" followed by a copy of the explanation of that Dave first gave Hal about what they needed to do. Then Hal pushes that commit upstream for Dave and leaves. At 6:00 PM Dave, still at the office, runs git-pull, spends a little time fixing up several issues with the code Hal wrote, then commits the result and pushes it to the development branch of the shared repo. Dave's changes mainly focus on getting the project to build again and making sure some or all of the existing tests pass. Dave then writes an email to Hal about this progress. At 8:30 PM Hal reads Dave's email about what Dave fixed and what Hal should do now. Hal then runs git-pull and writes some more code, pushing the result to the development branch before watching a movie and going to bed. Around midnight, Dave runs git-pull, fixes some more problems with the code that Hal wrote, and then pushes that to the repo. The next day at the office, they resume their work together following this pattern, where Hal writes the bulk of the code followed by Dave fixing it up and/or providing instruction for Hal about how to proceed. When they're done, one of them switches to the main branch with `git checkout main` and runs `git merge $OUR_DEVELOPMENT_BRANCH_NAME`.
Which part of this entails "destructive modifications of branch history"?
> There is also, of course, the question of copyright infringements in code produced by LLMs, usually in the form of training data leaking into the model's output
Well yes, LLMs like Claude Code are merely a "copyright violation as a service". Everyone is so focused on the next new "AI" feature but we haven't actually resolved the issue of all model providers using stolen code to train their models and their lack of transparency on sourced training data.
Copyright violation is not stealing, and training is not copyright violation (it's already been ruled as fair use, multiple times).
I think the concerning problem is when the LLM reproduces some copyrighted code verbatim, and the user doesn't even stand a chance to know it.
Yes, but that's not what the grandparent comment was talking about.
If I’m the grandparent comment, it was a big part of what I mean. Stolen/Unknown content goes in for training, verbatim or very close “inspired by” code comes out and there is no way to verify the source - “violation as a service”.
When that happens, it's because the code was trivial enough to be compressed to a minuscule handful of bits... either because it literally is trivial, or because it's common enough to have become part of our shared lexicon.
As a society, we don't benefit from copyright maximalism, despite how trendy it is around here all of a sudden. See also Oracle v. Google.
Quake's sqrt approximation is not trivial and is not common.
[1] https://www.reddit.com/r/programming/comments/oc9qj1/copilot...
(Shrug) It's a math trick, documented by Abrash among others and very heavily discussed on forums such as this one. And it didn't originate in the Quake codebase. Like much IEEE754 hackery, it goes back to the father of IEEE754 himself, William Kahan.
Nobody benefits from a law that says that LLMs can't regurgitate the Quake sqrt() approximation. If that's what the law actually says, which it isn't.
> it's already been ruled as fair use, multiple times
most countries don't have a concept of fair use
but they nearly all have copyright law
That fact in itself is a worse injustice than anything the LLM companies are doing. At the very least, it should be open to use in reporting, parody, and critique. Having no concept of such fair-use is oppressive and stifling.
Fair use is not the only way to allow critique and/or parody.
Why should we care about conflicting IP laws in other countries? Most of them have no effective means of extraterritorial enforcement.
presumably you want to be able to export your services to the 95% of the world's population lives outside of the reach of the united states regime
if not there's plenty of competitors that will
One can train model with copyrighted code as it is fair use, fair enough.
Are there any rulings about use of code generated by model trained on copyrighted code?
I believe distinction is clear.
Yeah, copyright infringement isn't stealing, copyright shouldn't even exist to begin with.
I just think it's especially asinine how corporations are perfectly willing to launder copyrighted works via LLMs when it's profitable to do so. We have to perpetually pay them for their works and if we break their little software locks it's felony contempt of business model, but they get to train their AIs on our works and reproduce them infinitely and with total impunity without paying us a cent.
It's that "rules for thee but not for me" nonsense that makes me reach such extreme logical conclusions that I feel empathy for terrorists.
Your views are contradictory. Copyright shouldn't exist, but the businesses infringing on it are the bad ones?
>We have to perpetually pay them for their works and if we break their little software locks it's felony contempt of business model
You don't have to pay them, or break their restrictions.
>but they get to train their AIs on our works and reproduce them infinitely and with total impunity without paying us a cent.
You don't need to allow this either. Unfortunately open-source code is necessarily public.
>It's that "rules for thee but not for me" nonsense that makes me reach such extreme logical conclusions that I feel empathy for terrorists.
The way LLMs use code is fundamentally different from wholesale copying. If someone read your code and paraphrased it and tweaked it, it would be a completely new work not subject to the original copyright. At least it would be really hard to get a court to regard it as an infringement. This is like what LLMs do.
> Your views are contradictory.
How is it contradictory? Tell it to the corporations who defend copyright for you and public domain fair use for themselves. If they were honest, they'd abolish copyright straight up instead of creating this idiotic caste system.
> Copyright shouldn't exist, but the businesses infringing on it are the bad ones?
Yes. Copyright shouldn't exist to begin with, but since it does, one would expect the corporations to work within the legal framework they themselves created and lobbied so heavily for. One would expect them to reap the consequences of their actions and be bound by the exact same limitations they seek to impose on us.
It is absolutely asinine to watch them make trillions of dollars by breaking their own rules while simultaneously pretendinf that nothing is happening and insisting that you mortal citizen must still abide by the same rules they are breaking.
The sheer dishonesty of it makes me sick to my core.
> If someone read your code and paraphrased it and tweaked it, it would be a completely new work not subject to the original copyright.
Derivative work.
I was once told that corporate programmers are warned by legal not to even read AGPLv3 source code, lest it subconsciously infect their thought processes and the final result. This is also the reason we have clean room reverse engineering where one team produces documentation and another uses it to reimplement the thing. Isolating minds from the copyrighted inputs is the whole point of it. All of this is risk management meant to disallow even the mere possibility that a derivative work was created in the process.
There is absolutely no reason to believe LLMs are any different. They are literally trained on copyrighted inputs. Either they're violating copyrights or we're being oppressed by these copyright monopolists who say we can't do stuff we should be able to do. Both cannot be true at the same time.
> At least it would be really hard to get a court to regard it as an infringement.
It's extremely hard to get a court to do anything. As in tens of thousands if not hundreds of thousands of dollars difficult. Nothing is decided until actual judges start deciding things, and to get to that point you need to actually go through the legal system, and to do that you need to pay expensive lawyers lots of money. It's the reason people instantly fold the second legal action is threatened, doesn't matter if they're right. Corporations have money to burn, we don't.
And that's assuming that courts are presided by honest human beings who believe in law and reason instead of political activist judges or straight up corrupt judges who can be lobbied by industry.
>I was once told that corporate programmers are warned by legal not to even read AGPLv3 source code, lest it subconsciously infect their thought processes and the final result.
There are different views out there about this. If you literally just copy a piece of code and make stupid changes, it might be a derivative work. But this is not guaranteed. There are times when there is one idiomatic way to do a thing, so your code will necessarily be similar to other code in the world. That type of code is not copyrightable, even if it appears in a larger work that is copyrightable. A small amount of bog standard code resembling something in another project is not in and of itself evidence of infringement.
Corporations would rather not have to deal with unnecessarily similar code or deliberate copyright or patent infringement. So they generally tell you not to look at anything else.
You should read over the criteria for something to be copyrightable: https://guides.lib.umich.edu/copyrightbasics/copyrightabilit...
The biggest issue is that any individual part of a copyrighted work may not be copyrightable. If you dissect a large copyrighted work, it probably contains many uncopyrightable structures. For example, in a book, most phrases and grammatical structures are not copyrightable. The style is not copyrightable. In code, common boilerplate is probably not copyrightable. Please don't make your stuff bizarre to add to its originality though. We need to be able to read and understand it lol.
>There is absolutely no reason to believe LLMs are any different. They are literally trained on copyrighted inputs. Either they're violating copyrights or we're being oppressed by these copyright monopolists who say we can't do stuff we should be able to do. Both cannot be true at the same time.
People who learn to program by reading open-source code are also trained on copyrighted inputs. Copyright may need some rethinking to cope with the reality of LLMs. Unless it is proven that AI is spitting out unique and copyrightable blocks of code from other projects, it really isn't infringing. People do this type of shit all the time. Have you ever looked at Stack Overflow and copied a couple of lines from it? You probably infringed on someone's copyright.
Ultimately, even if you prove copyright infringement happened, you have basically no recourse unless you also prove damages. Since open-source code is public and given away for free, the only possible damage is generally in being deprived of contributions that might have resulted from direct usage of your code. But direct integration of your entire project might have been highly unlikely anyway. Like it or not, people can be inspired by your work in a way that can't be proven without their direct confession.
> copyright shouldn't even exist to begin with.
It sounds like you're confusing copyright with patents.
Secrets? Just leak them, it only has to happen once. Guilds? Revoke their privileges and protections, and there's nothing they can do about it.
Absolutely an improvement. Information wants to be free. Stop criminalizing it and people will find a way to free it. And once it's out there it's over, there is no containing it.
People want to be paid for their work. If you don't let them, they won't do the work. "Information" does not have a mind of its own.
Even when the idea of a thing is "out there" there is a lot of grunt work and special stuff that needs to be implemented to get the best outcomes. Nobody owes you that work for free. Regardless of what GPL copers say, it is very hard to make money with software without enforcing some access restrictions and IP. Open source is great when it works, but it does not work for most things nor is it at the leading edge for most things.
Not really, only a handful of authorities have weighed on that and most of them in a country where model providers literally buy themselves policy and judges.
Wasn't copyleft essentially intended to be "copyright violation as a service"? I.e. making it impossible for an individual working with copyleft code to use copyright to assert control over the code?
Copyleft requires strong copyright protections. Without a license, you have no rights at all to use the code. If you want to use the code, because it's copyrighted, you have abide by the terms of the license.
> Without a license, you have no rights at all to use the code.
Exactly - that is what copyright was set up for (whereas before copyright, people could copy freely), and then copyleft comes and says that everyone can use and modify the code, and the only thing that the license prohibits is the ability of authors of derived works to apply their own copyright. In this way (known as being "viral"), copyleft uses the legal mechanism of copyright to essentially bring things to how they were before copyright.
If AI code is non-copyrightable, does this not achieve the aims of copyleft by different means?
I'm playing a bit fast and loose here but there's a solid idea at the heart of this statement - I'm just on the cusp of going to bed so wanted to post a placeholder until tomorrow. The gist is "what do copyleft licences aim to achieve as an end goal - and what would non-copyrightable code mean in that broader context?"
A GPL model trained purely on GPL based code could work.
It looks like the FSF is going to sit this one out like the SaaS revolution, to which they reacted late with the AGPL but did not push it. They are not working on a new license and Siewicz is already low-key pushing in favor of LLMs:
"Many years ago, he said, photographs were not generally seen as being copyrightable. That changed over time as people figured out what could be done with that technology and the creativity it enabled. Photography may be a good analogy for LLMs, he suggested."
I have zero trust in the FSF since they backstabbed Stallman.
EDIT: Criticizing anything from LWN, be it Debian, Linux or FSF related, results in instant downvotes. LWN is not a critical publication and just lionizes whoever has a title and bloviates on a mailing list or at a conference.
I have no idea how to criticize them because I have no idea what to say about LLMs irt the GPL, other than that Free Software should try its best to legally protect itself from LLMs being trained on its code.
I've always been in favor of the GPLs being pushed as proprietary, restrictive licenses, and being as aggressive in enforcement as any other restrictive license. GPL'd software is public property. The association with Open Source, "Creative Commons" and "Public Domain" code is nothing but a handicap; proprietary code can take advantage of all permissively licensed code without pretending that it shares anything in terms of philosophy, and without sharing back unless it finds it strategically advantageous.
> They are not working on a new license and Siewicz is already low-key pushing in favor of LLMs
I just have no idea what I would put in a new license, or what it means to be "in favor" of LLMs. Are Free Software supporters just supposed to not use them, ever? Even if they're only trained on permissively licensed code? Do you think that it means that people are pushing to allow LLMs to train on GPL-licensed software?
I just don't understand what you're trying to say. I also have zero trust in the FSF over Stallman, simply because I don't hear people who speak like Stallman at the FSF i.e. I think his vision was pushed out along with his voice. But I do not understand what you're getting at.
More or less what you said in your last paragraph: Stallman also reacted late to the web revolution, but at least he was passionate. That passion seems gone.
I don't see any sense of urgency in the reported discussion or any will to fight against large corporations. The quoted parts in the article do not seem very prepared, there are a lot of maybes, no clear stance and no overarching vision that LLMs must be fought for software freedom.
"I have zero trust in the FSF since they backstabbed Stallman."
The controversial line might have also been that one.
Sure, but remember that the Stallman situation started with a highly clumsy Minsky/Epstein mail on an MIT mailing list. The Epstein coverup was bipartisan and now all tech companies are ostensibly on Trump's side and even finance his ballroom.
Are there any protests or demands for the cancellation of Trump, Clinton, Wexner, Black, Barak?
I have not seen any. The cancel tech people only go after those who they perceive as weak.
Millions of people have been trying for a decade to
The tech people have never tried to cancel Bill Clinton or Ehud Barak. They were vocal about Trump when it was politically expedient during Trump's first term and from 2020-2024.
They are entirely silent since January 2025.
Cancellation of Stallman was the low point of that period, at least within tech, but it also made quite a lot of people aware that this monster of a practice must be resisted, or it will devour everyone unchecked. (Or, at least, anyone.)
You're forgetting the "second cancellation", where people brought legitimate (and often long-standing) criticisms against Richard Stallman. Cancelling a philosopher for having bad takes on age of consent, but otherwise drawing the line between "rape" and "not rape" in a sensible place, is not a good idea; but removing a community leader for a long history of applied misogyny is much more appropriate.
Yes. 100% agree.
1. Understand that code that has been wholly or partly LLM generated is tainted - it has (in at least some part) been created neither by humans nor by a deterministic, verifiable, process. Any representations to its quality are therefore void.
2. Ban tainted code.
Consider code that (in the old days) had been copy pasted from elsewhere. Is that any better than LLM generated code? Why yes - to make it work a human had to comb through it, tweaking as necessary, and if they did not then stylistic cues make the copy pasta quite evident. LLMs effectively originate and disguise copy pasta (including mimicking house styles), making it harder/impossible to validate the code without stepping through every single statement. The process can no longer be validated, so the output has to be. Which does not scale.
It depends on the nature of the code and codebase.
There have been many occasions when working in a very verbose enterprise-y codebase where I know exactly what needs to happen, and the LLM just types it out. I carefully review all 100 lines of code and verify that it is very nearly exactly what I would have typed myself.
> A member of the audience pointed out that the line between LLMs and assistive (accessibility) technology can be blurry, and that any outright ban of the former can end up blocking developers needing assistive technology, which nobody wants to do.
This is because LLMs are a type of assistive technology, usually for those with mental disabilities. It's a shame that mental disabilities are still seen as less important than physical disabilities. If one takes them seriously, one would realize that banning LLMs is inherently ableist. Just make sure that the developer takes accountability for the submitted code.