Genuinely curious if anyone has ideas how an LLM provider could moat AI. They feel interchangeable like bandwidth providers and seem like it will be a never ending game of leapfrog. I thought perhaps we’d end up with a small set of few top players just based on the scale of investment but now I’ve also seen impressive gains from much smaller companies & models, too.
back
15 comments
1) Push for regulations around data provenance. If you train on anything, you have to prove that you had the rights to train on it. This would kill all but the largest providers in the USA, though China would still pose a problem. You could work around that bit by making businesses and consumers liable for usage of models that don't have proof of provenance.
2) If you had some secret algorithm that substantially outperformed everyone, you could win if you prevented leakage. This runs into the issue that two people can keep a secret, but three cannot. Eventually it'll leak.
3) Keep costs exceptionally low, sell at cost (or for free), and flood the market with that, which you use to enhance other revenue streams and make it unprofitable for other companies to compete and unappealing as a target for investors. To do this, you have to be a large company with existing revenue streams, highly efficient infrastructure, piles of money to burn while competitors burn through their smaller piles of money, and the ability to get something of value from giving something out for free.
> China would still pose a problem
Not if regulation prohibits LLMs from China, which isn't that far fetched to be honest.
I think LLM will turn into a commodity product and if you want to dominate with a commodity product, you need to provide twice the value at half the cost. Open AI will need a breakthrough in reliability and/or inference cost to really create a moat.
If US regulation prohibits China from training LLMs? How?
If you mean trying to stop GPUs getting to China, US already has tried that with specific GPU models, but China still gets them.
Seems hard/impossible to do. Even if US and CCP were trying to stop Chinese citizens and companies doing LLM stuff
Not prohibit training, but make it illegal for US companies to embed, use or distribute LLMs created by China. Basically the general consumer will need to go through the effort of use the LLM that they want, which we know means only a small fraction of people.
That assumes that only the US is interested in using LLMs commercially, which isn't really true. Even if you can get America to sanction Chinese LLM use, you aren't even going to American allies to go along with that, let alone everyone else.
China's biggest challenge ATM is that they do not yet economically produce the GPUs and RAM needed to train and use big models. They are still behind in semiconductors, maybe 10 or 20 years (they can make fast chips, they can make cheap chips, they can't yet make fast cheap chips).
Canada would be reluctant. So would Europe, South Asia and so forth. The biggest hurdle for China is the CCP. It is one thing to use physical products from China but relying on the CCP for knowledge may be a step too far for many nations.
Making something illegal isn't gonna work if there's clearly value in doing it, and isn't actually harmful. Regulatory unfairness is easily discerned.
Look at how competitive chinese EVs are, and no amount of tarriffs are gonna stop them from dominating the market - even if americans prevent their own market from being dominated, all of their allies will not be able to stop their own.
I'm pretty sure 3) is Meta's strategy currently
In the early days of Google, people believed there could be absolutely no moat in search because competition was just "one click away" and even Google believed this and deeply internalized this into their culture of technological dominance as the only path to survival.
At the beginning of ride sharing, people believed there was absolutely no geographical moat and all riders were just one cheaper ride from switching so better capitalized incumbents could just win a new area by showering the city with discounts. It took Uber billions of dollars to figure out the moats were actually nigh insurmountable as a challenger brand in many countries.
Honestly, with AI, I just instinctively reach for ChatGPT and haven't even bothered trying with any of the others because the results I get from OAI are "good enough". If enough other people are like me, OAI gets order of magnitudes more query volume than the other general purpose LLMs and they can use that data to tweak their algorithms better than anyone else.
Also, current LLMs, the long term user experience is pretty similar to the first time user experience but that seems set to change in the next few generations. I want my LLM over time to understand the style I prefer to be communicated in, learn what media I'm consuming so it knows which references I understand vs those I don't, etc. Getting a brand new LLM familiar enough to me to feel like a long established LLM might be an arduous enough task that people rarely switch.
>LLM familiar enough to me.....
The problem with ChartGPT is that that dont own any platform. Which means out of the 3 Billion Android + Chrome OS User, and ~1.5B iOS + Mac. They have zero. There only partner is Microsoft with 1.5B Window PC. Considering a lot of people only do work on Windows PC I would argue that personalisation comes from Smartphone more so than PC. Which means Apple and Google holds the key.
It is really unbelievable how much money companies will spend to avoid talking to and thoroughly understanding their users. OpenAI could probably learn a lot from interviewing 200 random people every 6 months and seeing what they use and why, but my guess is they would consider that frivolous.
UXR is a thing that all large companies invest in
what makes you think that they don’t?
There's still one thing missing here: the browser. I do not agree with Gruber's analogy that the LLM is the browser. The interface to the LLM is the browser. We have seen some attempts at creating good browsers for LLM but we do not have NetScape, IE/Edge, Chrome, FF, Brave yet. Once we do, you would very easily be jumping between these models, and even letting the browser pick a model for you based on the type of question.
Also companies will be (and are) bundling these subscriptions for you, like Raycast AI, where you pay one monthly sum and get access to «all major models».
> The interface to the LLM is the browser
That is one of the reason why ChatGpt has a desktop App, so that users can directly interact with it and give access to users files/Apps as well.
But it isn't a browser, because it only interfaces to one single LLM. You need to have multiple models there (like visiting websites).
They all work the same and each has its own pro's and con's for each model they launch. Even the API's are generic. It's a bit more difficult to lock in 3rd party partners using your API if your API literally is English. It's going to be a race to the bottom where the value in LLMs is the underlying value of the GPU-time they run on plus a few % upmark.
Three ideas. All tried, none worked yet.
First, get government regulation on your side. OpenAI has already looked for this, including Sam Altman testifying to Congress about the dangers of AI, but didn't get the regulations that they wanted.
Second, put the cost of competing out of reach. Build a large enough and good enough model that nobody else can afford to build a competitor. Unfortunately a few big competitors keep on spending similar sums. And much cheaper sums are good enough for many purposes.
Third, get a new idea that isn't public. For instance one on how to better handle complex goal directed behavior. OpenAI has been trying, but have failed to come up with the right bright idea.
One view is that it’s not first mover, but first arriver advantage. Whoever gets to AgI (the fabled city of gold or silver?, Ag pun intended) will achieve exponential gains from that point forward, and that serves as the moat in the limit. So you can think of it as buying a delayed moat, with an option price equivalent to the investment required until you get to that point in time. Either you believe in that view or you don’t. It’s more of an emotional / philosophical investment thesis, with a low probability of occurrence, but with a massive expected value.Meanwhile, consumers and the world benefit.
What if the AGI takes an entire data center to process a few tokens per second. Is the still a first-arriver advantage? Seems like the first to make it cheaper than an equivalent-cost employee (fully loaded incl hiring and training) will begin to see advantage.
What if the next one to get there produces a similar service for 5% less? Race to the bottom.
And would AI that is tied to some interface that provides lock-in even be qualified to be called general? I have trouble pointing my finger on it, but AGI and lock-in causes a strong dissonance in my brain. Would AGI perhaps strictly imply commodity? (assuming that more than one supplier exists)
Depending on how powerful your model is, a few tokens per second per data center would still be extraordinarily valuable. It's not out of the realm of possibility that a next generation super intelligence could be trained with a couple hundred lines of pytorch. If that's the case, a couple tokens per second per data center is a steal.
Good point. It’s 2 conditions and both have to be true :
- Arrive first
- Use that first arrival to innovate with your new AGI pet / overlord to stay exponentially ahead
Exponential gains from AGI requires recursive self improvement and the compute headroom to realize them. It's unclear if current LLM architectures make either of those possible.
People need to stop talking about "exponential" gains; these models don't even have the ability to improve themselves, let alone at this or that rate. And who wants them to be able to train themselves while being connected to the Internet anyway? I sure don't. All it takes for major disruption is superhuman ability at subhuman prices.
What does AGI even mean in this case? If progress toward more capable and more cost-effective agents is incremental, I don't see a defensible moat. (You can maintain a moat given continued outpaced investment, but following remains more cost-effective)
Since we're talking about the economic impact here, AGI(X) could be defined as being able to do X% of white collar jobs independently with about as much oversight as a human worker would need.
The exponential gains would come from increasing penetration into existing labor forces and industrial applications. The first arriver would have an advantage in being the first to be profitably and practically applicable to whatever domain it's used in.
Why would the gains be exponential? Assume that X "first arrival" develops a model with a certain rnd investment, and Y arrives next year with investment that's an order of magnitude less costly by following, and there's a simple enough switchover for customers. That's what's meant by no defensible moat; a counterexample is Google up to 2022 where for more than a decade nothing else came close in value prop. Maybe X now has an even better model with more investment, but Y is good enough and can charge way less even if their models are less cost-effective.
> ... Google up to 2022 where for more than a decade nothing else came close in value prop. Maybe X now has an even better model with more investment ...
I was very confused at this point because I haven't really seen X as a competitor to Google's ad business, at least not in investment and value prop... Then I saw you were using X as a variable...
> The exponential gains would come from increasing penetration into existing labor forces and industrial applications
Only if they are much cheaper than the equivalent work done by humans, but likely the first AGI will be way more expensive than humans.
Yes, "first to arrive at AGI" could indeed become a moat, if OpenAI can get there before the clock runs out. In fact, that's what's driving all the spending.
None of that would matter if they could find the holy grail though.
Every time a new model comes out I ask it to locate El Dorado or Shangri-La for me. That’s my criteria for AGI/ASI.
Alas I am still without my mythical city of gold.
Somebody is going to write Wizard of Oz for all this and I’m for it.
Regulatory capture. Persuade governments that AI is so dangerous that it must be regulated, then help write those regulations to ensure no one else can follow you to the ladder.
That's half the point of OpenAI's game of pretending each new thing they make is too dangerous to release. It's half directed at investors to build hype, half at government officials to build fear.
If Elon Musk's pals regulate away OpenAI because they declared their technology to be too dangerous that would be an ironic turn.
One way to build a technical moat is to build services which encourage lock-in, and therefore make it hard to switch to a competitor. Some of OpenAI's product releases help facilitate that: the Assistant API creates persistent threads that cannot be exported, and their File Search APIs build vector stores that cannot be exported.
Create value at a higher layer and depend on tight integration to generate revenue and stickiness at both layers. See: Windows + MS Office.
They don’t need to make a moat for AI, they need to make a moat for the OpenAI business, which they have a lot of flexibility to refactor and shape.
Genuinely curious if anyone has ideas how an LLM provider could moat AI.
Patents. OpenAI already has a head start in the game of filing patents with obvious (to everybody except USPTO examiners), hard-to-avoid claims. E.g.: https://patents.google.com/patent/US12008341B2
> curious if anyone has ideas how an LLM provider could moat AI
By knowing a lot about me, like the details of my relationships, my interests, my work. The LLM would then be able to be better function than the other LLMs. OpenAI already made steps in that direction by learning facts about you.
By offering services only possible by integrating with other industries, like restaurants, banks, etc... This take years to do, and other companies will take years to catch up, especially if you setup exclusivity clauses. There's lots of ways to slow down your competitors when you are the first to do something.
It is better to leave this up to the «LLM browser», than the LLM. Both because of privacy and portability.
Create closed source models that are much better than the other ones, don't publish your tricks to obtaining your results, and then lock them down in a box that can't be tampered with? I hope it doesn't go that way.
Alternatively, a model that takes a year and the output of a nuclear power plant to train (and then you can tell them about your tricks, since they aren't very reproducible).
An algorithmic breakthrough IMHO. If someone finds out either how to get 10x performance per parameter or how to have a model actually model real causality (to some degree) they will have a moat.
Also, I suspect that the next breakthrough will be kept under wraps and no papers will be published explaining it.
The people working on it will still be allowed to move companies and people talk to each other informally. These Chinese groups working on this with far fewer GPUs appear to be getting 10x results tbh. Maybe they have more GPUs than claimed but we’ll see.
DeepSeek’s recent progress with V3 being a case in point which reportedly only cost $5.5M.
You'd have to have either a breakthrough algorithm (that you keep secret forever) or proprietary training data.
Spend money developing proprietary approaches to ML.
Legislation, if you can’t compete on merit then regulate.