Item — HN Reader

> Can someone with some more experience explain why this happens? The most basic webserver, like a TCP socket served from a C program, serving some files etc. can take tens of thousands of requests per second, per core, on reasonably recent hardware.

You're grossly overstating the capacity of a server to meet peak load demands, specially when hit with unexpected load spikes. The slashdot effect was a phenomenon observed two decades ago, in a time when the bulk of the web was served as Apache running CGI scripts. The rate of requests any random server could process was around ~300requests per second, and some of the links being hugged to death were literally served from a box under some dude's desk.

1 comments

The Apache web server was popular because it was one of the first MVP in its category. I think it was weird that the solution developed back then was to figure out load balancing among more servers, rather than optimizing the web server itself.

People did both. And you want load balancing for operations/redundancy anyway.

Or because once you get the max requests per sec out of a machine, you can often get a second machine to double rps for much cheaper than a machine that can do double the rps.

In the 20-30 year old time frame, easy to obtain servers are one to two cores, often one core per socket. SMP scalability was not a given; you could certainly run on SMP and get benefits, but depending on your load you might not get anywhere close to 2x the throughput.

A single core webserver serving static files isn't going to be much faster than 1 ms per file, so you're around 1k rps. And that's assuming you files are all cached. If you do dynamic pages, they're more likely to start closer to 10 ms, and can easily be much longer, so 100 rps is a good target.

But, if your pages are sizable, you can easily run out of bandwidth and then everything gets slow and you can hit too much concurrency and things get worse. A T3 was a very good connection back then, and it was 45 mbps. 1000 rps of 50k images is going to fill up your T3.

Load balancing, done right, is very scalable for typical http loads where client sends little and server sends lots. Direct Server Return means the load balancer only sees the traffic from the client, which significantly reduces the effort for the load balancer. Of course, you can't do a lot of fancy things with DSR, but that means your balancers aren't overloaded with fancy things. If your DSR load balancer fails, you can fail over to DNS round robin.

> A single core webserver serving static files isn't going to be much faster than 1 ms per file, so you're around 1k rps. And that's assuming you files are all cached.

What “is” is this? Already in 2005, I was serving images at >3k qps on a single core of my server, without any tuning at all. Now we're 20 years later, with much faster TCP stacks and much faster cores.