Item — HN Reader

Ah, this from the same David Siegel who said almost 2 yrs ago (in a talk found here: https://youtu.be/0z60xUDo-NI?si=PTDe11-sn2P53qo5&t=420) that the AI data center buildout was premature because:

> Even if the current approaches will continue to scale, this would be as if in the early days of computing, perhaps someone invented a bubble sort for sorting numbers (an n-squared algorithm), and the tech companies at the time decided they were going to build vast data centers to sort numbers and not bother to figure out that there's an n-log-n way of doing it <laughs>

...to which I have to say: yes, definitely! And he's right about open-source AI too.

1 comments

> AI data center buildout was premature

Ask Amodei how he feels about going to spaceman bad for compute that he couldn't find anywhere else in the market.

Yes, there's a present shortage of usable frontier compute, but that doesn't establish that every proposed data center will earn a decent return, or that today’s hardware + model architecture will remain economically competitive, or more to Siegel's point, that algorithmic efficiency could not dramatically reduce compute requirements.

To borrow a real example from a prior boom, railroads were congested during the initial build-out while people simultaneously funded and built too many railroads for future demand. Likewise, Anthropic et al can be compute-starved now while the industry as a whole is overbuilding expensive, depreciating infrastructure.

I'm not sure comparing this one to the "real life" previous ones is worth doing. Digital has a way of scaling that IRL doesn't. You don't get exponetials with humans, but you do with models. Scale has shown time and time again that it works. Despite everything you've read in '25 about walls, lack of x, and so on it still does. And the more you have you find ways of using it even more. RL can use ~ 7:1 ratio of inference : training. That is, you run 7 nodes of inference for every node of training. And it keeps improving. Every algorithmic find on inference (and there've been plenty) translates in improvements. And we're not seeing any signs of this not continuing. And then there's the "zero" method, that hasn't been tried at true scale yet. Bitter lesson and all that.

I'll grant that the scaling laws have held so far, and that the bitter lesson is yet to be proven wrong. However, that doesn't exclude the possibility that the current architecture's S-curve will flatten, and a new one will replace it, which was Siegel's point. At greater layers of abstraction, what is observed is a raw capabilities/intelligence increase, but what form it takes is subject to change. We'll see where it all goes, the only way out is through.