back
96 comments
I think this whole premise is ignoring the point that most decently sized companies want to and eventually will be running their own LLM workloads. Currently use cases are limited by scope and imagination, and predominantly focused on cost efficiency. Once a use case becomes a top line revenue driver budgets will become essentially only limited by ROI. There are some inference workloads that are unacceptable to send to the frontier labs for privacy reasons. Most of it will come from firms who want to productionalize their own fine-tunes. In any case, the market for inference is less than 1% of what it will be in 5-10 years. This whole notion that GPUs will be sitting idle en masse is ridiculous. People will just start running GPU databases if it becomes cost efficient.
but this is just an absolute fantasy, what, exactly, will this compute be doing? Our lives are already deeply entwined with technology and use barely any compute. How could our lives become 100x more dependent on compute?

You can be thrilled by this exciting technology and all the possibilities it brings without thinking it is going to require this huge capital investment in GPUs. You could radically change billions of lives with a few dozen GPUs.

Push everybody to buy hardware they don’t need to constantly run agents for pretty basic tasks, such as processing your daily emails and sending you little summaries notifications. So, the most inefficient software tools ever produced, but that keeps the whole industry alive. That’s pretty much the vision Jensen Huang is selling to ensure NVIDIA continues to grow
I hope we can quickly escape these tasks and the focus on coding. It’s all very superficial. There are a ton of serious AI workloads imaginable and I haven’t seen anyone pushing those frontiers, at least not publicly. Nvidia had a shiny keynote about their digital twin of the world for solving large problems but i guess that one died already.
Ditch the servers.

Do and sync over client to client.

Keep data local again.

Only use cloud for backups of local client encrypted blobs of vectors:data

If you get rid of a lot of the suspect semantics hallucinated up over decades of software development it's not hard to see the geometry of an electronic snowflake. All the language just obfuscates the elegance. Crude meat suit grunts and clicks.

Streamline it all to management of geometric states and access control and put the semantics on the presentation layer. What if we don't need python and go and ruby anymore? Made sense in a pre-gpu everywhere reality. Could just be high school stats classes to generate sets of values. Let go of the obscure linguistic chants.

The data centers are just to serve surveillance purposes. Obfuscated behind politically correct memes of creating jobs.

Chip away at the monolith and atomize the topology

TIL about GPU databases. I did a bit if research but only found this one: https://github.com/bakks/virginian

What are the advantages of a GPU database?

TIL too, but probably what GPUs are good at: optimizing for throughput while hiding latency. Probably better suited for OLAP than OLTP.
Is anyone really running GPU databases in prod at scale? Or are you assuming that the GPU compute from a crash will become so cheap that it makes this niche scale?
> most decently sized companies want to and eventually will be running their own LLM workloads

And probably some decently sized states too. Not only commercial actors are up to the job.

Is it just me or is this comment extremely difficult to parse? Besides the lack of any links/evidence the assertions are full of jargon "decently sized", "scope and imagination", "productionalize their own fine-tunes", "This whole notion... is ridiculous". It is currently at the top of HN and I hope that is just because it's a Sunday afternoon in many places.
Im not sure what this phrase even means. Does it mean that "70% of the revenue made by companies selling tokens comes from openAI and anthropic"? If so, how does it follow that "the AI industry doesn't exist"? For every other industry, the "size of the industry" is given by the total revenue, not by the percentage of the top 2.

These people don't know what they're talking about.

I'm with you on Zitron being a bit challenged in the knowing what he's talking about department.

I think he's implying the money in AI is all spending from OpenAI+Anthropic who get it from investors. But those two have annualized revenue of about $26bn and $74bn or about $100bn total which presumably represents actual customer demand for the product.

That's $100bn revenue on about $1tn capex so far which isn't enough to make it profitable in itself but the things growing like crazy so you'd expect that. $100bn is about 0.08% of world GDP so if AI customer demand grows to 1% of GDP that's up 12x from here.

Microsoft and others report their stake in Open AI as net income. Open AI and others revenue comes in large part from large customers like Microsoft. Nvidia makes large investments into Open AI, SpaceX, and core weave and in return they buy Nvidia hardware. It is all very circular. And at the same time it appears these companies have near zero moat. I think it is foolish to not question how big this business really is. Seemingly no one knows.

Surely AI is a thing that is here to stay, but I am not at all convinced that the big frontier models are going to be able to return their investment and if it becomes apparent they cannot, then you are almost certainly going to see a massive correction.

> Open AI and others revenue comes in large part from large customers like Microsoft.

Could you provide a citation for this?

You can like the character or not, but there is a trend I’m following ( heavily vested in NVIDIA, so tongue in cheek when I say this ) that might be highly align with Zitron. Looking at the moves from NVIDIA ( Groq )and AMD ( Taalas ) which are pure inference plays. I believe this shows that the impetus to train a better-bigger model might be coming to a level of maturity that might merit a serious threat to the frontier labs.

For frontier labs, their fund-train-new model play might not be as effective, and a shift of spent of compute cost moving away from training to inference might be a tell-tell sign of the LLM as we know it plateauing out as scale is just not as effective. Open models might also be placing a major pressure on meeting then revenue targets need to sustain the model, lots of customer hosting their own inference to mitigate costs.

If you only move the needle just slightly in the direction of inference, frontier labs will soon loose their alphas. Becoming just another SaaS for inference might not be as attractive unless you are Google/MSF ( IMHO ).

Should this pan out, it could be a scenario where the NeoClouds could soon loose their biggest customers, so I tend to agree with that aspect of Zitron’s view.

Thoughts?

I’m not a heavy user, but use them routinely to do dumb translations and refactors etc. For my money the models are mostly indistinguishable at achieving long term objectives. Every time a new one drops I’m a part of a number of communities that go insane trying to “max” on it, waiting for resets etc. I just don’t get it, and if it turns out my view it rational, then it seems like training new models will become a thing of the past at some point. It’s just not economical other than for maxing hype.
Inference time compute is the new scaling.
Ed wrote a post back in 2024 where he claimed OpenAI would fail in 2 years if they didn’t do the, seemingly impossible at that time, things like raising more money than any company before etc.

And OpenAI did all of that and is still alive today. Would be good to know this context because if you are out there boldly making doomer predictions month after month then you should also rate your previous ones.

Because one day Ed will be right and he’ll go around and take a victory lap while ignoring he has basically not been right before.

Timing is always next to impossible to predict. Open AI has managed to continue to find investors and as long as they continue to do that they will be around no matter how much money they lose.
> Because one day Ed will be right

Bold prediction.

Sounds like his prediction in 2024 was spot on.
So the core of your comment is...."the bubble has not burst yet, therefore there is no unsustainable bubble" ?
Core of my comment is 1/ being wrong on timing is being wrong 2/ There is no clear definition of bubble burst. Public markets were down 10% in July and many stocks like SNDK fell 50% -- is this a bubble burst?
I don't know much about this guy other than every time I see him on here he has an axe to grind about AI
You could start a new business selling grindstones to the axe grinders. Like selling pickaxes in a gold rush.
Wasn't it always seen as a "winner takes all" business?
There is no "all", that's the point. There is no pot of gold at the end of the AI rainbow.
There is some gold, but it’s a fairly small amount compared to the ATM discussed by AI companies (basically software generation, software security, audio transcripts, translation, image processing, image generation, search, etc). So it’s not like there won’t be an AI industry in the future but it won’t be absolutely everywhere the way the AI boosters are projecting. Inference will be a low margin industry, training will be capex constrained and likely low margin too. And some services on top will have decent margins. But nothing like the datacenters full of PHDs Amodei and Altman are dreaming about.

Just like any other technology

There's a lot of money changing hands, at least that's a spoil isn't it? And if the music stops, someone is going to end up holding it.
Well a lot of money is being spent, but is a lot of money being made?

With circular deals it’s hard to tell.

Most tech revolutions end up making a fair bit of money for successful businesses.
> Wasn't it always seen as a "winner takes all" business?

No. It was always a pyramid scheme marketed as a "winner takes all" business. But there's no winning. The only winners are those who cash out before it collapses. This is literally Web3 2.0.

AI isn't Web3.
If there was one winner then that would make a lot of people very angry.
Ai revenue or datacenter/compute revenue? There's a lot of circular financing right now and there's a pretty obvious bubble for sure but I haven't been able to figure out what exactly this guy's argument is after seeing it a few times recently. Like yeah most ai datacenter spend is those two companies - that's pretty standard monopoly (or monopsony for the cloud providers) dynamics. If you're saying some major percentage of all ai related spending is openai and anthropic spending money on compute - well that's not exactly right because Google etc are also spending money on building datacenters (that's not ai spend in this definition? WTF is ai revenue exactly?) - and that's just pretty much indicating it's a frothy market with two unprofitable companies at the center with huge cogs? We knew that already.

Feels like they want to make a clean headline grabbing argument about how "70% of all the spending is actually just these two companies" and are ending up with a really muddled headline that's just like yeah that's how monopoloies and duopolies work. When there's a lot more insidious circular complicated shenanigans going on that gets collapsed by this framing.

The general concern (from him and others) is that the cloud providers are making extremely large capex commitments (borrowing, going cashflow negative) to satisfy demand from a small number of customers who may not be able to pay them.
> to satisfy demand from a small number of customers who may not be able to pay them

Because a lot of isn't real demand, e.g. given away for free or very very cheap.

>> When there's a lot more insidious circular complicated shenanigans going on that gets collapsed by this framing.

All that is discussed in the video, plus those distinctions. And most important, that AI revenue would not exist...if OpenAI and Anthropic would not be funded, by the same Amazon, Google and Microsoft they are buying it from!

We now have several voices saying the same:

"Aswath Damodaran: Big Tech Has No Idea How AI Pays Off" - https://news.ycombinator.com/item?id=49229981

"Why Wall Street Is Ignoring Big Tech's Debt" - https://news.ycombinator.com/item?id=49230630

One interpretation is that so called profitability is one 15 minute phone call between OpenAI and Anthropic to increase prices. If you think that antitrust is the reason that this won't happen, you've stated a speculation. Not a certainty.
there isn't enough demand to raise prices. OpenAI announced price CUTS recently and they are hilariously unprofitable already. Why would they be cutting prices if they have the ability to raise them? outside of this website and the ownership class everyone hates AI
They are pressured by open models to reduce prices, not increase
"Because Goldman does this kind of nonsense."

Hilarious. You can agree or disagree with Zitron but he really has no business talking about Goldman Sachs. His posts are littered with evidence he has no ability to perform the type of financial analysis he thinks he does.

Mind sharing more details?
Sure. Here's an example:

https://www.wheresyoured.at/exclusive-openai-financials/

Zitron wrote:

> Additional factors – including interest income and interest expense – left it with a net loss of $8.84 billion. It then marked $3.74 billion of losses as “net loss attributable to noncontrolling members capital,” leaving the net loss attributable to the company as $5.09 billion.

> It’s unclear what this means, nor how OpenAI reconciled the removal of $3.74 billion in costs. I will not speculate further.

It is very clear what this means, and no speculation is required if you understand basic consolidation accounting, which you would expect someone in his position to understand.

It's not rocket science: when you have a parent company with entities it doesn't wholly own, the slice of losses belonging to the other equity holders is split out as "noncontrolling interests." Nothing is removed or hidden; the total loss is unchanged, it's just allocated to reflect that the parent company doesn't own the whole. Framing it as OpenAI removing costs implied something sketchy and requiring speculation where there's only routine GAAP accounting.

But it's even worse than this. So many of Ed's claims conflate the foundational R&D and capital expenditures these companies are incurring with the unit economics of their businesses. He seems woefully unable to understand that you could sped gobs of money on the former and still have positive gross margins that scale over time with the latter.

One example - he often compares current revenues to capex being spent on future capacity to claim that AI companies aren't profitable. (See this post for example: https://www.wheresyoured.at/am-i-meant-to-be-impressed/ .)

But this ignores that the capex spent to build more capacity is expected to generate additional future revenue. You don't need to recoup your capex immediately. A better approach would be to amortize the capex and compare revenues to that.

Clearly he assumes revenue won't increase enough to recoup this level of capex (and it's very possible it won't) but IMO it's either a miscalculation of how the financing works or a deliberately misleading framing to compare current small revenues to a big scary capex number.

I'm sure the above is simplified by the way, but I am confident that people who work at Goldman understand the relevant details extremely well.