Ask HN: Will programmers write more efficient code during the memory shortage?

Some programmers will write more efficient code. At my $dayjob (one of the big tech companies) we're already planning a major goal next year of optimizing server code to reduce RAM requirements, and this is directly in response to the crunch.

In practice I expect most optimizations will come from "stop doing stupid stuff" and not "use fancy advanced algorithms." But that's a cynical perspective so don't be cynical like me.

I don't believe so do for two reasons:

- everyone assumes their program / website is the only thing running at the machine at a given time, and dev machines are always more powerful than user machines

- it's not really lack of advanced data structures and algorithms that result in the bloat most of the time but the fact that programs and websites are delivered by large teams, there are dozens of submodules that are often loaded even when not needed, and doing it properly is hard to architect to without getting into big complexity and gnarly bugs waiting to happen when someone from other team modifies something and does not know full picture. So it's cheaper to just keep things the way they are to reduce complexity of architecture and fragility.

Programmers will write more efficient algorithms if their employers tell them to trade time-to-market for hardware cost. Previously, it was trade hardware cost for time-to-market.

"Programmers" don't make this decision, the product owner does.

It's definitely possible, but I agree with many other commenters it probably will not happen.

I wrote an encrypted mesh networking library that runs on normal operating systems. A customer asked me if I could make it run on an ESP32 with 520 kiB of RAM. At first this seemed impossible, but it turned out that it was, and not even that hard. While the original library was not memory hungry at all for a desktop CPU, it still wasted space on unnecessarily large buffers. Cutting those out made the library run on an ESP32 while leaving plenty of room for an application.

Also, my first PC was a 200 MHz single-core 32-bit AMD k6 with 32 MiB of RAM. This ran a graphical OS with browsers, word processors, 3D games and so on. Nowadays you can get a CPU with more than that amount of RAM as just built-in cache.

So a good place to start optimizing code would be to actually get a "severely resource constrained" computer and start making your code work on it.

If that happens, we will see it in triple-A games first. If some new titles have significant lower hardware specs than expected.

If buyers can't afford the hardware anymore, the studios need to adjust. It's definitively possible to scale games down a lot. There are a few AAA games that were "dumbed down" for the Switch 1 (Hogwarts, cyberpunk, ...). And that's a really low-spec device.

There are two factors: existing gamers not able to afford upgrading. But also new gamers, that might only be able to afford much lower spec PCs than people who bought 2 years earlier.

Why games? Because there is a clear point where people stop buying games. Minimum hw specs are known before buying.

I don't think we will in the HN snark sense of "this react site or electron app uses way too much memory". Those companies will continue to work in the same way, maybe even less efficiently as people chase modern UIs with extra animations or videos or effects, or with some AI code generation tacking on too many features or tech debt.

In specific sectors I do think we will see more optimization. If you're working on cloud compute or AI training / large scale data processing, there will be a big focus on optimization as prices are very large at that scale and shortages have a bigger effect.

Also in gaming I think the next cycle will be different. Big game studios used to push for the best possible graphics that might require the newest consoles or high end gaming computers, but the next releases might not be as much of an upgrade. The next gen of consoles or graphics cards themselves might be delayed, or be less powerful, or be too expensive and flop, as chip manufacturing companies continue focus on more lucrative markets and leave average consumers behind.

For the people I work with, I'd say no to writing more efficient code, and maybe to using more advanced algorithms. I say maybe only because they'll just ask an LLM to make it more memory efficient. Which in theory will cause the LLM to choose a more advanced DS. But I don't have a lot of faith that the LLM will get said DS right all the time.

I work with dynamic language developers. Perl, React, Python and so on. Most currently don't care about memory usage. Some devs are actively wasteful. Usually in the name of code clarity or speed of delivery.

Until prices hit the large hyperscalers, I don't think most people are going to make significant changes. You might see a small set of open source projects related to self hosting put in an effort, but in general, I don't think so.

Some big-tech orgs (that have their own hardware) will take costs into account, but they already do that. The "optimization" is more likely to be business-optimizations; "this can be slower if it uses less memory", rather than inventing new stuff.

Note that I am excluding any of the big AI labs. They are definitely going to be working to figure out how to use less memory, but that's primarily not related to the direct cost.

No.

Even today it is possible to use languages and programming techniques to be ultra efficient when it comes to memory. It's not that difficult to use them but the world don't care.

Actual example: I have built same utility using Rust, Zig and Haskell (and Go, humorous writeup[0]). Rust binary is 450kb, Haskell binary 30mb. Zig was unfinished but I constrained memory to 512kb just for fun.

Thus it's not about memory per se but more about convenience. Many applications could be scaled by 1-3 degrees of magnitude in memory if they reused techniques but... no one cares.

4K resolution frame is ~40mb uncompressed. Another 10mb is 80k lines of text. Thus fitting in 50mb for 99.8% of applications should be perfectly doable with today's tech.

If the price increase would be per each next 1mb then maaaaaybeeee. But if it's like 100% over GB then I'm sure it won't have any impact.

[0]: https://xlii.space/eng/the-four-language-waltz-a-tale-of-all...

"will they" seems like the least interesting/useful question though. How can we, e.g. raise awareness in users and programmers that this is possible and useful?

I wanna hang out with coders who care, I wanna read articles about caring and how you can achieve more with less code, and I want to see video content that visualizes the sheer insanity of it all. Like those "scale of the universe" videos, except the complexity isn't fascinating, but embarrassing.

Make it uncool, make software bloat a goober thing you people do at most because they "have to", for pointy-haired bosses, or because they're in a rush or under other constraints. Not something that just gets normalized because hey, thinking is hard, caring is hard, so let's all conspire and pretend we're not being deplorable slobs, under the "leadership" of even worse slobs. There is little technical difficulty here, it's mostly social. Ignorance and greed are in cahoots and don't want to get called out, so call them out.

Though I think to start with, you have to not give a shit about the majority, and huddle with people who care and make better things. If the majority comes around, great, if not, still better than standing on the beach, wondering if the ocean will ever spontaneously take this or that shape.

I'll just recognise that I'm old (and a bit grumpy): back in the day, if you didn't write minimally efficient code, it simply wouldn't run or it would be terribly slow.

Young devs have not been exposed to this and they can get away with writing inefficient code most of the time, sometimes all of the time.

And even if there is a RAM shortage, we are still in the Gigabyte age. So I don't think we will go back to cycle and byte counting.

I can't even get some devs to care if their query runs in 3s or 8ms.

> Will programmers write more efficient code during the memory shortage?

If we can insert "some" then Yes.

> use of more advanced algorithms and data structures that use less memory?

I don't think so.

At least at work there is a push to decrease memory usage but the way I've seen it playing out is not using some O(N) data structure instead of O(N lg(N)) per-say but instead replacing `int[]` with `byte[]` or in-lining some fields to remove some indirection costs.

It is a really simple matter of incentives.

If your backlog is full of feature requests you will add features.

If your backlog is full of tasks to reduce memory usage, speed up areas etc., then that is what you will do.

Most programmers will have no choice whatsoever.

Algorithms and data structures aren't the problem, bundling Chromium to make a chat app is the problem.

I think Rust's rise in popularity will probably lead to some benefits.

Games will probably get more efficient but they're easier to scale down to the memory that's available.

I expect mobile OS and maybe mobile app developers to do so.

Google has already downgraded memory for their upcoming Pixel 11 while at the same time imposing local running models as a first-class feature. Both decreasing the memory pool and increasing the demands on it.

The key is that they own the full stack (hardware and software) and can demand the OS be more efficient, along with perhaps forcing that goal on app developers as well via tightened background limits etc.

> advanced algorithms and data structures

This isn't where memory goes or real-world speed comes from. For most applications, it's abstractions like React running in electron that hurt performance. There are also richer resources backing parts of apps--higher resolutions, better quality, higher framerates.

I was thinking about this recently. If you discount web/electron bloat, actually the memory bloat of software isn't hugely terrible.

I still often notice that servers on Linux use <1GB of RAM even with relatively high use. I don't think that's really changed massively in 20 years.

There's hope for the extinction of Electron based apps.

For this incentive to exist, the app needs to be such an obvious memory hog that users start identifying it as the source of the problem.

Even then, a lot is required for most businesses to prioritize this (presumably) temporary issue at the cost of things like: participation in the AI race, other features, bug fixes, new markets etc.

Heck, sometimes software is so inefficient that it costs developer and tester productivity but a fix is not prioritized for years.

Realistically, the impetus here will come from the platform holders, not the individual programmers.

Apple recently released an 8GB MacBook Neo, which bucks the previous trend of increasing RAM in low-end SKUs, and signals that software needs to be prepared to run in that configuration for a long time to come. I expect we'll see similar moves in smartphones if RAM prices stay elevated.

Unless there's an actual financial metric to hit: no.

For most people; computers just suck now.

I work in AAA video games, and I'm often told that consoles hold back games. This is true in the most essential sense: for consoles we are forced by the first-parties to optimise our games so that they function on consoles at a reasonable framerate with not hitches.

Those optimisations help PC players too; and they have more hardware headroom so they can crank things up even higher: without those constraints games would be much heavier and would require you to be on a faster moving hardware cycle.

The same is true when I worked in ecommerce; there was a common verbiage which I don't remember verbatim but the spirit of it was "we lose 10% conversion every 100ms" - so we built our systems to do as much as possible within 100ms (including network RTT).

If people actually have target hardware, and a metric that they're pushed towards, it can happen, people can learn how to navigate the abstractions or do better with the abstractions they have... but I think this wont happen, because in the examples above there was an actual cost to things. Conversion rate is money, not being permitted to launch on console is money.

But you know what modern MBAs see as being money? Time to market & developer salaries. If every company has the same mentality: then it doesn't affect conversation in the same way. Those exogenous constraints simply don't exist in the majority of technology today. It's like a collective action problem.

If there was going to be a collective action against this for financial reasons we would have seen it, shipping laptops with 16G of RAM instead of 4G (or 8G) has a really large headline cost in a company of a few thousand people; but those companies never pushed hard on Teams or Slack.

In my experience advanced algorithms and data structures are almost never needed. You need simple data structures and algorithms and understanding of the environment in which you are running (for example see https://handmade.network).

Unless users complain it’s not going to happen. Somehow SPA (Single-page application) consume memory as much as operating system.

I don't think many will.

But I think they should. Not only for the practical reason, but because it leads to better software in my opinion

Limitations foster creativity. Abundance kills innovation.

In my company: - All computers are refurbished and at least 10 years old. - Cloud deployments use Hetzner or self hosted machines. - All hardware in general is a bit dodgy. Our main CI/CD setup is on a very unreliable network with varying availability and bandwidth

It can be annoying once in a while but it reflects our customers reality. And it catches a lot of usability and performance bugs that we wouldn't otherwise catch

I can only recommend it

You must be joking.

Niklaus Wirth wrote his A Plea for Lean Software in 1995. Pdf at https://people.inf.ethz.ch/wirth/Articles/LeanSoftware.pdf

People did not bother and then the Web made everything worse. We are now so used to layers and layers of abstraction, humongous libraries and frameworks that most do not know anything about how to write "lean" software.

Nothing is going to change in the application/services layer.

I've dumped python for anything that doesn't involve scipy / pytorch in favor of go. So yes, I'm having my llm's output go now which is generally more memory efficient than python.

One can dream, but I doubt it. As for all the reasons I don't have much faith, I'd defer to the other comments.

I will say though, optimizing memory usage is easier than ever. Doing a scan of your codebase with an LLM for _large_ memory gains can probably shave a decent chunk off of any application with ease. You don't have to go down the rabbit hole, but taking the top 3 large things it catches would probably result in notable gains for minimal time usage.

Hell, I've hunted down code that was causing an OOM from a coworker that ran a regex that would have to search the entire string of a base64 payload that was often around 24MB, when it very easily could have been a basic string operation on the first 50 characters or so. I caught it myself, but went back after with an LLM about 8 months ago and it also spotted it with a vague point in the direction of where it could be.

There are probably a lot of small slip ups across a codebase that are simple fixes that add up over time, but we never catch because we're not actively profiling.

8Gb laptops are about the only tech still selling. Exec's better think long and hard about that.

Programmers might write more efficient code if the performance was unacceptable on their own laptops, but even in the not-flush-with-cash companies I've worked all my career, it's been easy for me to always have a dev laptop with twice as much RAM as most consumers have. I'm presently working on a laptop with 64GB, and we don't even use any local AI models. If a PE company is willing to spec dev laptops like this, even in the memory shortage, then I assume the memory pressure will never hit developers.

So, this carefree attitude directly shapes all code that runs client-side (JS + native apps) since the only impact on the company is whatever happens on the dev's laptop. The rest of the impact is "free" since it's paid (in either money or in misery) by customers.

For server side, I also highly doubt it, as being more memory efficient on the server side has always had a benefit to the company who pays the bill. The only things that may change may be the relative cost, but if management comes and says "AWS bill going up, help?" devs will say "OK, we can find ways to save RAM, but the team won't be doing any new features or fixing any customer-facing bugs during that time" and management will say "Okie dokie, we'll just pay the bigger bill then."

I would say that in most cases you dont need fancy algorithms and data structures, you just need to stop treating RAM as a rough notebook. Allocating in a loop, not reusing buffers and pointer chasing are a performance killers and are quite easy to avoid before they are introduced to code. Other than that, just be more thoughtful about network and disk usage and nested loops. Its a sum of stupid things that make your program slow.

One can certainly wish, but I am inclined to believe that RAM use isn’t just about sloppy programmers. I think it is, to a significant degree, about the kinds of problems we solve now that we have more RAM.

And not all exorbitant RAM use is about sloppiness. Sometimes you can trade more RAM use for lower complexity. Bugs and development time were expensive. RAM was not. So sometimes there is calculation rather than sloppiness behind certain types of heavy RAM use.

I might be wearing my tinfoil hat too tight, but I think that market pressure will continue prioritising shipping worse code faster, especially now that with AI it's possible to ship code really fast. The "solution" to expensive hardware will be way less capable hardware and we'll end up with "thin clients" and all compute will happen in centralised "clouds". I hope I'm wrong

That’ll be a minor part of it.

The whole Electron “ship a browser for each app” ideology will die first.

We sill probably see an increase in the use of OS-level memory compression tricks instead - things like zwap on Linux, for instance. Because the trend is usually to cure the symptoms, not the cause, as it is generally easier to set up and good enough. Makers who already have invested on memory-careless tech such as Electron won't rewrite; at best they'll just be replaced by better alternatives.

Machine telemetry data (logs, metrics, traces) are shipped around as highly redundant utf-8 strings. I'm sure there are plenty of other examples of 2+ orders of magnitude improvements that aren't even low hanging fruit, they're basically groundfall. But picking them up may nonetheless require quite a bit of effort, and a major reevaluation of corporate values. Maybe the RAM crunch will be sufficient incentive? Idk.

EDIT: It's also possible that the era of hypergrowth might end. Most of the world's addressable eyeballs are already being monetized. There's no new coal seam to mine. If tech becomes more about doing the same with less, and less about explosively growing at all costs, we'll probably see more efficient systems. And fewer people building them. I think this kind of larger systemic change would be more likely to drive efficiency improvements than the (probably more temporary) RAM crunch.

No. They won't.

Most games are written for the 3% who can afford a luxury gaming PC. That's how the market is working for a while now.

The memory shortage is really for these insane memory requirements for LLMs.

A web browser and the basic mobile app will be fine.

The iPhone 17 Pro has the most RAM and it's only 12GB. Hell the iPhone 16 Pro only had 8 GB. The vast majority of consumer cases don't need it. I doubt Apple and other manufacturers will go beyond that to keep prices down.

My bigger hope (for apple ecosystem) is that macbook neo forces apple to optimize. Devs don't have that scale.

Backwards thinking programmers might (there is currently plenty of memory efficient software; but it's always missing that feature you really need funny how that works). Forward thinking ones will realize we are heading towards the glut of all gluts and buy 4TB to play around with.

In particle physics, processing jobs face a limit of about 2GB/core. This is driven by the realities of "grid computing". The hardware in the computing facilities have approximately 2GB/core and jobs allocate based on number of core. This ratio has been fairly stable for a long time. Some facilities may have a bit more RAM per CPU core but 2GB/core is the general rule of thumb. I hazard a guess that this will not change throughout the (hopefully small number of) years that this memory cost bubble persists. In fact, the increase in cost may lead some facilities to keep older hardware going beyond its EOL/warranty period, prolonging a ratio that may actually be relatively generous compared to today's hardware systems.

As a consequence of this fixed RAM/core ratio, substantial software development effort goes to either making jobs fit in 2GB or if that is not possible then to utilize multithreading. Generally, particle physics processing does not particularly benefit from MT except in this fixed RAM/core situation. Sometimes large memory jobs are needed (inherently or because of bloat that is too costly to improve). When run on the "grid", these jobs must allocate multiple cores just to get "their" memory. If those jobs can use the extra cores, overall throughput does not have to suffer.

That's for conventional software, which still makes up the bulk of the computing. The situation for the growing amount of GPU-accelerated software is different and more varied. One trend can be seen relating to VRAM. Research groups with easy access to big GPUs like A100 write code to fit or exceed the relatively copious VRAM limits of the data-center GPUs, while groups that lack easy access to DC GPUs but have access to more modest "gamer" GPUs write more advanced software that can fit the smaller VRAM. In some cases, they write the software so it can scale the computation, keeping GPU utilization high while staying just under the VRAM limit.

General budget crisis and limited resources in the particle physics field are in part responsible for all of this tailoring of the software to fit the hardware. If better funded, particle physicists could spend more time doing physics and less time squeezing last drops of processing power.