Item — HN Reader

For seriously high performance systems like webservers or exchanges, you need to get the architecture right from the start. You can't rearchitect a part or two or solve design problems by making skilled hires later on. The 80% part is a distraction (it's also factually untrue that 80% of code doesn't matter. Some applications don't have hotspots, the performance impact is basically smeared all across the code. It really depends on the application. Sqlite found major speedups by implementing hundreds of little changes that were each almost undistinguishable from noise).

High performance doesn't mean unreadable. Microoptimizations can make code less readable, true, but they also give you the least amount of speedup (in the best case you can get several times speedup if you found some hotspot that can be vectorized, but that's pretty rare). You can get huge performance improvements by optimizing networking and system calls and doing that doesn't make much of an impact on readability.

3 comments

Aside: this is the deeply unobvious optimisation video "Performance Matters" by Emery Berger: https://www.youtube.com/watch?v=r-TLSBdHe1A - they got a 25% speedup in SQLite in 2019!

Start at 9:30 and watch to 10:40 if you want to jump straight into the entré. Dessert discusses SQLite at 40:25 to 41:25.

Summary: A: Causal analysis is needed to discover opportunities for improvements - they developed a technique and tool. B: Modern CPUs have so many hidden causes of performance variation, that you require special tools to actually measure small performance gains. C: They found a surprising difference between -O2 and -O3 that implied overfitting (false reporting of performance improvement).

I presume these techniques are used in large software companies, but I am guessing many (most?) developers in smaller companies know little about the topic. I still find it unobvious.

As someone who has done performance work for most of my career, I strongly agree with what you've written here. If you have a program that isn't well optimized, it might be true that once you start profiling you can find some hot spots and easy wins, but that doesn't last forever. Earlier in my career I was working on Python web apps, now I write high performance C++ systems, but you can apply this equally to both domains.

Take for example a big and slow Python/Ruby/PHP/whatever web app, something that I'm sure a lot of people on HN have experience with. If you profile one of these apps you're probably going to find that it is slow because it's using an ORM that generates hundreds of SQL queries, the ORM creates tons of intermediate objects which put a lot of stress on the memory allocator and GC, and there will be an endless amount of code that is munging data to take it from one representation (e.g. whatever the ORM returned) to some slightly different representation (whatever the caller actually wanted). There might be a few queries that are particularly expensive, but once you've fixed those you're still going to be left with something big and slow with no obvious path forward to make the code faster. Furthermore the root cause of these problems may not be obvious when looking at profiles because things like memory allocation that are internal to the runtime of your interpreter are typically either not exposed by profiling tools, or if they are it's not clear what action can be taken to improve them.

Likewise if you have a C++ program that does a lot of unnecessary copying and memory allocation, the program is going to be slow because there's a lot of time spent everywhere and there's no one thing to fix. Look at a bottom up profile of a C++ program and see how much time is spent in string constructors, memcpy, etc. and unless you've been thoughtful about this stuff from the start what you find is probably going to be alarming. In fact, since a lot of copy constructors are inlined (including for STL types), with many profiling tools it might not even be obvious that copies are happening at all, since they won't show up in call stacks.

Not every program needs to be high performance, but if you start by writing a lot of code that is slightly inefficient everywhere then it's probably going to be impossible to make things fast if you change your mind down the line.

And not only does the ORM itself have big performance impact, your own code will be designed around the assumptions of this particular ORM, e.g if it is easier to fetch a few row objects from different tables using the ORM and then do some comparison computations in your relatively slow interpreted language rather than writing one fast SQL JOIN the former will almost always be the solution that ends up in production.

ORMs invites you to write bad code, instead of thinking ahead of what queries I need for this specific operation, you just start writing ORM based code aimlessly and try to make it work with the tools available in your language of choice.

And the ORM entities will eventually infect every part of your project and it will almost be impossible to remove them after the fact.

> For seriously high performance systems like webservers or exchanges, you need to get the architecture right from the start.

Agreed - I worked at a company that went through a very painful period of not coping with load. They got there in the end by completely rewriting the core system - lots of people claimed the switch from an interpreted language to a compiled language but the reality was that the rewrite was a completely different architecture.