back
93 comments
The author currently misses the issue that all decent LLMs that have been released have been created by companies rather than opensource communities.

Sure, a lot of models have been released under permissive licenses, but.

Thats like releasing shareware. The special source for making _new_ LLMs comes from the dataset, and training clusters. None of which are cheap or easily run/financed by the community.

More concretely, an ML mode is only "open source" if both the training code and the training data are freely available.

Otherwise, it's not possible for the community to reproduce the model.

That's a bit like saying FOSS is open source only if a copy of the programmers is applied with the code.

Everything that has a source, has another source that has produced that source.

The algorithms behind creating LLMs are all published papers for all to read, the libraries (like TensorFlow) are themselves FOSS projects, and the data... is the open web for the most part.

The Wikipedia dump alone is more than enough to get a very decent LLM shaped up.

How an LLM is produced IS NO SECRET. It's just that to produce it you need millions (or for the more sophisticated ones: billions) in data center fees / power / GPU to train the model. So if the training scripts were included, you still can't make a LLama model yourself at home.

I think open source advocates bring some understandable but wrong intuitions to how LLMs are being distributed.

In the world of hand-coded software, binaries are hard to work with, source is easy to work with, and compilation is cheap.

In the analogous world of LLM training, model weights are easy to work with, having training data does not let you reliably change model behavior, and "compilation" (training) is insanely expensive.

So, if your goal is agency to create tools for your own purposes, 9/10 researchers would rather work from a trained foundation model than the source data. The foundation models are of course released by companies because they cost $10s of millions to train -- but releasing them enables a thriving community of research, building adjacent frameworks, and specialized models to be created by much less powerful actors.

I've never understood the dogmatism around FOSS, but I've felt I understand the ideals. Those ideals are so much better served by releasing weights than by leaving LLMs only available through commercial APIs.

(author here): I am actually more focused on training corpus and tooling around foundational models, and the possibility of run these models on resources you control. An analogy would be the CPU and the opensource software running on top. I know people want the entire toolchain to be opensource, but I think the real value lies in that "top-layer".
First incarnations of Unix were paid by corporations and universities.
Thus GNU... And Linux.

20+ years later, look who is still around.

and you had the source, documentation and all the bits needed to build it from scratch.

llama is a binary blob. Very capable, and its great that it got leaked. But its not a win for opensource. Its an accident of licensing. FB's lawyers would never have let it be a proper open source license, the PR and IP risks were way too high.

It just so happens that the leak meant that the PR risk went away, and knee capping openAI is a good thing for meta, and thus worth the IP risk.

Yes and no. The models themselves (with permissive licenses) might be like freeware, rather than "open source" or "free software". But they are a big enabler in allowing people to build other F/OSS on top -- eg. an experimental Gnome/KDE addon that allows voice control is now a real possibility. Most practical FOSS (other recalcitrant parts of the GNU community) already builds on top of all kinds of proprietary tools from computer hardware to closed operating systems like MacOS/Windows. LLMs are just the latest in the list of blobs -- developers & users can evaluate whether the power/opacity tradeoff is acceptable.
I don't think the comparison applies. Shareware can't really be modified in any way. However a foundational model is meant to be modified, fine-tuned, augmented, etc...

I agree calling it fully open source is a stretch but it's not the same as shareware.

Has anyone tried to take a distributed training framework and make it _really_ distributed?
Eh, I don't know; that's perhaps like giving credit to HP and Dell and perhaps Apple for Linux?
I have a feeling this article was partly written by a GPT. This part

> For LLMs to be effective, they require a few things:

And then that list are just a dead giveaway.

I'm not nearly as cynical about GPT as many are (in fact I use them myself), but I think text written by it should be marked as such.

edit: The author expressed that it was indeed not written using any GPT.

(author here): it wasn't. This is how I write, and probably an example of ChatGPT flagging neurodivergent/non-native speakers as AI-generated.
I experimented a bit with AI written blog posts on here, I'm actually pretty happy with the result:

https://write.as/mnmlmnl/

I push my bash history, attach a couple files, paste some doc and web references, and then lightly edit the result. Beats not writing the blog post.

The tricky part is not getting fooled by the fact that it looks decent, and really put in the effort to think each sentence over to catch the misleading "fringe hallucinations" (as I call them, not full on hallucination, but mildly misrepresenting the actual intent/meaning).

> I think text written by it should be marked as such

Good luck with that. It’s a tool.

What makes it a dead giveaway?
I think your comment was written by a GPT, does that make it true?

Please refrain from making baseless accusations.

I still haven't found a use for LLMs. The propensity of them to get it wrong, and being fancy statistics machines based on existing data, just feels like a cheap trick. It's still a fancy Markov chain, it just has guard rails.

There just isn't a place for them in my life.

I'm not swigging the Kool-Aid yet either, but I think you're being a little dismissive. I've found ChatGPT useful for some sorts of questions that can only really be posed in natural language, particularly grammatical oddities that come up as I learn Spanish.

Probably the single most impressive result I've had so far was with "What's that weird thing on the top of a P-38's engine nacelle that looks like a recessed sideways wheel?" Got it in one. I can't imagine getting anywhere with a conventional search engine there.

I definitely wouldn't rely on it for anything professional, though.

They're still extremely effective as a fuzzy search engine and translator (e.g., I can write out a function in plain English and get something that would have taken me a minute in 5 seconds).

Additionally (more importantly), the most magical part is still the architecture: there seems to be no end to the expressivity of transformers as long as you can pour in more compute. This can be extremely powerful and lead to more generalizable reasoning given the right learning objective (e.g., pose tasks as an RL Markov Decision Processes where state=text, actions=tokens, rewards=performance of generated code/math/language).

If you look at the technical discussion from the perspective of individuals and companies creating sophisticated autocomplete, the whole fad seems ridiculous.
Currently working on a pandas personal project (which I have only a little bit of experience with from before) and it’s easily 10x better than googling. I ask it a question to plot something simple and it spits code out which very rarely doesn’t work. I’d easily pay $20 for this… but it’s the free model that does it.
Other than coding assistant, ChatGPT is awesome at answering to everyday questions where you previously had to find the answer in ad-bloated content farms.

Things like "how to do this", "how does this work", questions about medicine, DIY, etc …

And contrary to the content farm internet, it allows you to have a real conversation to give you more précise information.

Is it 100% accurate ? I think not. But it’s trained on nearly all the books about all the topics so I put a lot more trust on this than on SEO optimized content farms.

The best way to chat with it is to think that you are talking to someone who have read and remembered everything with an endless knowledge of everything but who acts as an faillible human when you speak with it.

Here's one example: Today I used GPT-4 to write documentation for a Neovim plugin: https://github.com/Robitx/gp.nvim/pull/72/files

It may be a fancy Markov chain, but it seems to have understood the code it documented.

Like pretty much every other thing knowledge product, The only real value I see in them is in terms of helping humans think better. You can kind of tell them to "interview" or "question you" and they can spark your creativity.
This is kind of wishful thinking. I'm not saying you're wrong, but there's no real way to prove you're right. Sometimes open source wins, but not every time. The whole machine learning field is still too young to have a clear answer.
(author here): I am currently writing a book about programming with LLMs, I have absolutely put my money where my mouth is over the last year, and there is not doubt in my mind that we will see incredible tools in 2024.

Already the emergent tools and frameworks are impressive, and the fact that you can make them yours by adding a couple of prompting lines and really tailor them to your codebase is the killer factor.

My tooling ( https://github.com/go-go-golems/geppetto ) sucks ass UI wise, yet I get an incredible value out of it. It's hard to quantify as a 10X, because my code architecture has changed to accomodate the models.

In some ways, the trick to coding with LLMs is to... not have them produce code, but intermediate DSL representations. There's much more to it, thus the book.

Can you name some examples where open source won?

In the past I wanted to believe this can be the future, where open source will somehow win (at least in some parts). What I see is that even the biggest projects are mere tools in the hands of the big corporations. Linux, Postgres, etc. All great! But have been assimilated. I cannot really consider them a win.

It seems to me that it goes back and forth - it also seems to me that the advancements in LLMs will go a similar route.

I mean, I just the last hour setup the deepseek-coder-6.7B-instruct-GGUF model on my computer and tried it out with llama.cpp. It is maybe free ChatGTP level.

If I could have run this when I was a student it would have been a real killer app. It is kinda impressive that I can run it on a crappy laptop.

I some way this is also really scary, job wise. I would not recommend anyone getting into the programmer fields after the last years progress in these tools.

But at least they will not be locked up to "OpenAI"'s servers ...

I wonder if there could be a community effort doing model training, like the protein folding? If 30 000 computers ran some model training while not being used, would that be enough to get anywhere?

Not getting into programming because anyone can use an LLM to do programming does not suddenly fill the productivity needs in an absolute sense. People can do more with less which also means we will quickly raise the demand for the new things we need to do.
You must have a very different definition of ‘crappy laptop’ than I do.
> LLMs are ultimately democratic

LLMs reflect what they have been trained on, so the question is what they are trained on. In this line of thinking, at best they represent the groupthink, and at worst they are biased one way or another.

Which is not to be taken negatively, but is something important to keep in mind, though.

I don't think the author is saying the content LLMs produce is "democratic".

They're just saying that they're democratic in the sense that anyone can run them and they have a strong open-source community behind them.

> I build "brushes" for refactoring my code every day.

Anyone know what the author means by brushes?

Also any recommendations for non-Copilot IDE LLM integrations? I've tried a couple but they felt far behind Copilot in terms of quality and smooth IDE integration.

https://githubnext.com/projects/code-brushes/

You select some code and apply a brush and it'll use a preset prompt to modify it: "Make Readable", "Add Types", "Document" are some of the options. Copilot has a pretty poor implementation, and mostly they just butcher your code - but they seem very powerful, especially with custom brushes.

Things like "replace this PHP5 class with a new PHP8 readonly dataclass", "Convert this react components to have imageLinks as prop srcsets". Nothing fantastic, but damn this stuff is tedious to do manually. Now I can just batch process my files et voila.
the dependency of LLMs is very clear, without opensource, there is no LLMs, zero! not just LLMs, almost all AI/Machine-learning training are using linux, Slurm, python, i work in the HPC/ML field, include hardward/software/administration, i know this firsthand.
The skepticism in the comments makes me even more bullish for this revolution.
>I can't really see a future where big tech can keep up with a motivated community (and motivated it is, as any look at social media or tech websites will show you)

As far as I can tell, many of these "motivated communities" organize on Discord (search is garbage, poor integrations, login-gated and invite-only in many cases), and also Twitter/X (now login-gated, API locked down). And beyond AI there are communities and lots of valuable content on Reddit (also locking down as hard as they practically can). I will be 0% surprised if Github locks down their platform, too. It's too soon to tell how much it will pay off, but almost as soon as ChatGPT came out, the obvious strategy of every company with user-generated content is for all the valuable communications, insights, and social graphs of their communities to be proprietary training data for the proprietary LLMs of the future.

I also share their idea about reallocation of programming resource away from big corporations.

There is so much bullshit empire building in our industry that serves only to waste talent. Small teams working on problems that need tech but can’t normally afford it should hopefully deliver something akin to a productivity shot in the arm across industries.

I like the term surveillance capitalism.

We need to move quickly, because the “moat” that big tech monopolies will seek to create will be regulatory, decelerationist, and authoritarian.

I treat usage of LLMs in coding as an admission of incompetence, and this post has only reinforced that view.
LLMs are the death of open source. Labor might be free, but training resources seldom are. They won't be able to keep up.
That people actually entertain these ideas is SO WEIRD to me. The last sentence is just obviously false, precisely because the models are software that anyone can trade around. LoRA's, in other words. The fact that original models "might" -- and even that's a big might -- be hard to produce from scratch, so what? Operating systems are hard to produce from scratch, but anyone can grab Linux and do anything.
They can only remix what already exists, not create new things, no?