The Architecture of Open Source Applications: Nginx (2012)

Frustratingly, the problem with automated DNS-based is that it doesn't work with all domain registrars, which is a showstopper when you are not allowed to change a customer's existing registrar nor move its NS records.

For my self hosted setup I've shifted to Caddy and not bother about setting up crons etc for SSL. It's just so much nicer to have the Reverse Proxy handle that for you automagically.

And the config file is just 2 lines to reverse proxy instead of 20+ equivalent in NGINX.

Nginx is making the same mistakes that dethroned Apache. No one wants to fiddle with a web server. Default to $currentIndustryStandard, make it work, forward to my application.

The Let's Encrypt Certbot does this automatically for nginx and apache as well, just with a bit of extra config: https://certbot-dns-cloudflare.readthedocs.io/en/stable/

>the classic Apache-style paradigm was to have "the server" be generic and explicitly support certain application languages

The first wave was cgi-bin based fork() and exec() a process.

Followed by what you're describing...things like mod_perl and mod_php, where the app code was running inside the apache process.

Then, later, a separate daemon that kept some pool of processes running for apache to open up connections to, like fastcgi.

Why is it strange? Would you rather be concerning yourself with SSL termination and handling files (efficiently thru the kernel) instead of application logic?

I think about it as, there's a bunch of code you want to run to serve a webpage: Business logic, SSL termination, caching, lets-encrypt, load balancing, etc etc. Separately from the question of what you want your web server to do is the question of how you cut up the cake of all that stuff. It makes the most sense to write some of this code (eg caching) in highly optimized native language. And it makes the most sense to write your business logic in whatever language your team is most productive. (Eg python or ruby or whatever).

The question of process architecture (how you cut the cake) has a ton of options! For example, a nodejs process can:

- Run other javascript code by including it via npm. (Directly or in a worker thread)

- Run C code by compiling it to asmjs and including it like any other javascript code

- Run C / Rust / Go code by compiling it to wasm and including it via the Webassembly constructor

- Run C code by compiling it to a napi module, and linking to it via nodejs's native modules API

- Run code in any language in a child process (via spawn / exec) and communicate to it via stdin / stdout pipes

- Run code in a sibling process and communicate via named pipes, IPC or via network requests

There are engineering tradeoffs with all of these options. For example, wasm works in the browser, and isn't CPU architecture specific. But wasm tooling is language specific. And it runs about 3x slower than native modules (which don't work in the browser). Child processes run at native speed and work with any language, but "calling" into the child process is much more complicated than calling a wasm / napi function. Errors behave differently too - an uncaught exception in an npm module will often crash your whole nodejs process. But a segfault in a sibling process will just crash that process. (And its not obvious which one is better!). If you want to, you can include the logic for SSL termination, caching, etc in the same programming language as everything else you're doing - which means writing your business logic in C, or your caching logic in javascript. This has runtime performance vs productivity implications. Rust is great for this - but link times get pretty bad.

In short, there isn't a "right" answer to any of this. Basically all of these options are viable ways to divide up your program. When I was getting started, apache + CGI was the common approach. (Your logic ran as a child process of the web server). Now, application servers have complete HTTP stacks and its common to do interop between your app server and nginx via HTTP. It works pretty well, but there's nothing essential or obvious about that choice either.

> As a relatively young dev, the idea of a "web server" as a standalone binary that serves your application (vs a library that you use to write your own "server") feels strange.

In my eyes, the ideal setup is one that's layered: where you have an ingress that's basically a load balancer that also ensures that you have SSL/TLS certificates, enforces rate limits, perhaps is used for some very basic logging, or can optionally do any URL rewriting that you need. Personally, i think that Caddy (https://caddyserver.com/) is lovely for this, whereas some people prefer something like Traefik (https://traefik.io/), though the older software packages like Nginx (https://nginx.org/en/) or even Apache (https://www.apache.org/) are good too, as long as the pattern itself is in place.

Then, you may additionally have any sorts of middleware that you need, such as a service mesh for service discovery, or providing internal SSL/TLS - personally Docker Swarm (https://docs.docker.com/engine/swarm/) overlay networks have always been enough for me in this regard, though some people enjoy other solutions, such as Hashicorp Consul (https://www.consul.io/), or maybe something intended for Kubernetes or other platforms that you already may be using, like Linkerd (https://linkerd.io/).

Finally, you have your actual application with its server. Personally, i think that the web server should be embedded (for example, embedded Tomcat with Spring Boot) or indeed just be a library that's a part of the application executable, as long as you can update it easily enough by rebuilding the application - containers are good for this, but aren't strictly necessary, since sometimes other forms of automation and packaging are also enough.

The reason why i believe this, is because i've seen plenty of deployments where that just isn't the case:

  - attempts to store certificates within the application, each application server having different requirements for the formats to be used, making management (and automation) of renewal a total nightmare
  - attempts to use external application servers, like a standalone install of Tomcat/TomEE/GlassFish, which are always a pain to manage and integrate with the application, often leading to inconsistent configurations, especially if something like Ansible isn't used
  - no ability to easily do URL rewriting, whenever you realize that you need to do something a bit differently, or figuring out how to do rate limits or anything of the sort becomes needlessly hard, and even then you can't really protect the app from too much traffic if it is the endpoint for all the requests, especially troublesome if it's also used by other internal services
  - no observability, since it becomes harder to tell whether there's a problem with the load overall (e.g. load balancer nodes are not keeping up), or the actual server implementation within the application
  - actually, not having a separate ingress also makes load balancing too hard in many setups, since service meshes simplify this bunches

Thanks! Expanded:

The Architecture of Open Source Applications: Nginx - https://news.ycombinator.com/item?id=10616989 - Nov 2015 (126 comments)

Nginx design details - https://news.ycombinator.com/item?id=4211480 - July 2012 (29 comments)