> So why do we care from where LLMs learn?
Because humans aren't computers and the similarities between the two, other than the overuse of the word "learning" in the computer's case, are nonexistant?
back
1 comments
Are you really asserting that these models aren't learning? What definition of learning are you using?
Don't know if they are, and don't really care either and I don't care to anthropomorphize circuitry to the extent that AI proponents tend to, especially.
Humans and Computers are 2 wholly separate entities, and there's 0 reason for us to conflate the two. I don't care if another human looks at my code and straight up copies/pastes it, I care very much if an entity backed by a megacorp like Micro$oft does the same, en-masse, and sells it for profit, however.
Okay, so the scale at which they sale their service is a good argument that this is different from a human learning.
However, on the other hand we also have the scale at which they learn, which kind of makes every individual source line of code they learn from pretty unimportant. Learning at this scale is statistical process, and in most cases individual source snippets diminish in the aggregation of millions of others.
Or to put it the other way round, the actual value lies in the effort of collecting the samples, training the models, creating the software required for the whole process, putting everything into a good product and selling it. Again, in my mind, the importance of every individual source repo is too small at this scale to care about their license.
The idea that individual source snippets at this scale diminish in aggregation, is undercut by the fact that OpenAI and MSFT are both selling enterprise-flavoured versions of GPT, and the one thing they promise is that enterprise data will not be used to further train GPT.
That is a fear for companies because the individual source snippets and the knowledge "learned" from them is seen as a competitive advantage of which the sources are an integral part - and I think this is a fair point from their side. However then the exact same argument should apply in favour of paying the artists, writers, coders etc whose work has been used to train these models.
So it sounds like they are trying to have their cake and eat it too.
Hmm. You sure this is the same thing? I would say it’s more about confidentiality than about value.
Because what companies want to hide are usually secrets, that are available to (nearly) no one outside of the company. It’s about preventing accidental disclosure.
What AIs are trained on, on the other hand, is publicly available data.
To be clear: what could leak accidentally would have value of course. But here it’s about the single important fact that gets public although it shouldn’t, vs. the billions of pieces from which the trained AI emerges.
It's really not different in scale. Imagine for a moment how much storage space it would take to store the sensory data that any two year old has experienced. That would absolutely dwarf the text-based world the largest of LLMs have experienced.
If you don't care, why are you confidently asserting things you're not even interested in examining? It just drowns out useful comments.
Do humans really read terabytes of C code to learn C?
Humans look at a few examples and extrapolate…
But that also exists in the AI world. It’s called „fine tuning“: a LLM trained on a big general dataset can learn special knowledge with little effort.
I’d guess it’s exactly the same with humans: a human that received good general education can quickly learn specific things like C.
Humans have experienced an amount of data that absolutely dwarfs the amount of data even the largest of LLMs have seen. And they've got billions of years of evolution to build on to boot
You're straying away. Let's talk about learning C.
Also humans didn't evolve in billion of years.
The process of evolution "from scratch", i.e. from single-celled organisms took billions of years.
This is all relevant because humans aren't born as random chemical soup. We come with pre-trained weights from billions of years of evolution, and fine-tune that with enormous amounts of sensory data for years. Only after that incredibly complex and time-consuming process does a person have the ability to learn from a few examples.
An LLM can generalize from a few examples on a new language that you invent yourself and isn't in the training set. Go ahead and try it.
I can't even convince it to put the parameters in a function call in the correct order, despite repeatedly asking.