▲ 122 points

Containerize Go and SQLite with Docker

awstip.com

by hhaluk·4y ago·85 comments·view on hn ↗

85 comments

If you are using Go (which solves most of your dependency problems) and SQLite (which means you don't need to integrate with an external database via service discovery) why do you need Docker at all?

Perhaps because Docker has great stories for deployment. Many of the complexities of deployment are handled for you (writing a systemd unit, managing rollback, etc).

Yeah, but for Docker to handle the complexities of deployment, you first need to handle the complexities of Docker. So OP's question is valid: for most Go apps, all you have to do is compile a binary and copy it to the server - no Docker or other paraphernalia required. Of course that may not be so simple due to various reasons, but it helps to keep that possibility in mind...

Docker isn't a cache-all for containers. Docker has a pretty terrible deployment story at this point and I forsee in a couple years it will probably be considered legacy compared to Podman/K3d/Local K8s/etc

I deploy my app (c# with sqlite, some native dependencies) using custom-made scripts to ubuntu, centos and redhat dedicated servers and it's a major pain to have a separate script for each version of each OS. I'll switch to docker soon so that I have a single target

> If you are using Go (which solves most of your dependency problems) and SQLite (which means you don't need to integrate with an external database via service discovery) why do you need Docker at all?

Dependencies is not really a problem with Docker, nor the thing it is designed to solve. If dependencies was the problem people cared about, everyone would just go with the single statically linked executable/fat JAR and no one would ever be bothered with Docker.

Docker is primarily about containerization, but it's also about ease of packaging and deployment. It's also a deployment format that provides horizontal scaling for free.

Also, the one-database-per-service architecture pattern is quite common, as well as ephemeral databases and local caching, and keep in mind that SQLite also supports in-memory databases.

Okay, I'll bite.

> It's also a deployment format that provides horizontal scaling for free.

Um. Docker does not provide horizontal scaling at all, for that you need orchestration. And those tools are anything but free if your time has any value.

They probably have 100 other apps and deploying this one without docker would make it a snowflake.

Because the deployment environment is or may be a Kubernetes cluster or some other kind of containerized environment. Wrapping up your application in a neat package makes other people's jobs easier - to them, it's a black box container, not a binary they need to install and manage on a server.

It looks great on CV.

And now he can add front page of HN

I think the only reason I still use docker for everything is for automatic restarts without worrying about another tool for whatever language/framework I'm working with

Uniformity perhaps? I for one help manage several workloads that has become quite a lot easier to manage through containers and I'dd rather deal with that than having a few workloads being deployed differently from everything else. Maybe containerizing a go project like this is more work than strictly necessary, it would be a quick 10 line Dockerfile and pretty much done.

In a production environment one would probably deploy using something like Fargate, Kubernetes or Fly.io.

> In a production environment one would probably deploy using something like Fargate, Kubernetes or Fly.io.

Docker swarm mode is pretty good and terribly easy to get up and running in no time.

I have a few small personal projects hosted on Herzner on a couple of Docker swarm mode deployments with 100% uptime in the past two years, and all it took to get that infra up and running is installing Docker on a bare Linux node.

The only downside I'm aware is that inter-node traffic speeds can be relatively low.

> C libraries are required to interact with SQLite

Or: modernc.org/sqlite (plus https://github.com/zombiezen/go-sqlite), "an automatically generated translation of the original C source code of SQLite into Go"

as mentioned yesterday: https://news.ycombinator.com/item?id=29959193#29960726

This article is just an example of someone that encountered a problem and didn't do their due diligence of finding the best solution for their use case.

The discussion of building fully static go binaries for docker has been rehashed a million times on the internet, the modernc.org/sqlite is at version 3+, so nothing in this should be new information for anyone that has an interest in the topic.

I strongly dislike this type of content that just repacks basic existing information and uses it as an excuse to click bait people on to a spammy blog.

This is really interesting. Copying to the scratch container is powerful and I’ve used it a lot, regularly, but occasionally something comes up where I need to use a more fully featured base to support things. One other downside I’ve encountered with Scratch is no shell so doing docker exec or kubectl exec doesn’t work. Does anyone know a good solution to this problem?

The Kubernetes folks' solution to this is the addition of `kubectl debug` (added as `kubectl alpha debug` in Kube 1.18, graduated to `kubectl debug` in Kube 1.20) as an alternative to `kubectl exec`. It takes an existing Pod and lets you attach a new container with whichever image you like, so that your production images don't need debugging tools.

Also, before `kubectl debug`[0] existed, you could always edit a Deployment and add a sidecar container of `alpine` or `busybox` and enable process namespace sharing[1] get some leverage to debug with.

A bunch of other options in the docs as well

[0]: https://kubernetes.io/docs/tasks/debug-application-cluster/d...

[1]: https://kubernetes.io/docs/tasks/configure-pod-container/sha...

We drop a statically compiled BusyBox binary on images like that as "sh". If we need more we can symlink to it in /tmp at debug time (or just call it directly). It strikes a good balance between slim and debuggable.

Do you do this during build time or debug time?

Been using containers and fighting this problem for a long time... I never thought of this simple solution, thank you for sharing it!

In Kubernetes 1.22+, you can use ephemeral containers: https://kubernetes.io/docs/tasks/debug-application-cluster/d...

An emerging solution I've been investigating is using Nix as a build system for docker. The syntax is fairly lightweight and you can build containers that are scratch + coreutils, or whatever else you decide to put in there.

I was originally turned onto this by a post on the repl.it blog about using Nix this way.

You can also do this with other build systems or using weird docker file hacks by hand.

Here's some idea of what the syntax looks like:

https://github.com/NixOS/nixpkgs/blob/master/pkgs/build-supp...

If you run a single process in a Docker container, have it output logs over STDOUT, and have no meaningful way to interact with it other than shutting it down, you don't have much need for a shell. It can't do anything in the container anyhow. By contrast, if you've basically got a full OS in there, then yeah, a shell is really useful.

I would say most of my Go projects involve what you described (logs to stdout, simple interface) but I often find myself needing a shell to debug my docker build, like ensuring that files have ended up in the correct place, and have the correct contents, within the docker container.

If your stack involves running containers on a Linux machine, you can use `nsenter` to use the debugging tools on the host OS to debug the processes within the container.

Distroless works well. Still no shell, but comes with “what’s usually missing in scratch”.

Instead of a container, just statically build the golang program with sqlite. Single binary deployment.

Testing locally, adding the "-linkmode external -extldflags '-static'" flags doesn't bring down the binary size. The heavy lifting is done by '-s -w'. It is a good idea to drop privileges even if there's a small attack surface.

At $work we use the following template. Sharing in case you find it interesting:

    FROM golang:1.17.1-alpine3.14 as builder
    RUN apk update && apk add ca-certificates curl git make tzdata
    RUN adduser -u 5003 --gecos '' --disabled-password --no-create-home mycompany
    COPY . /app
    WORKDIR /app
    RUN make build
    
    FROM scratch
    COPY --from=builder /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/
    COPY --from=builder /app/bin/app-3 /bin/app-3
    COPY --from=builder /etc/passwd /etc/passwd
    USER mycompany
    CMD ["app-3"]

And a very simple makefile:

    help:
     @echo "Please use 'make <target>' where <target> is one of the following:"
     @echo "  test                 to run unit tests."
     @echo "  build                to build the app as a binary."
     @echo "  build-image          to build the app container."
     @echo "  run                  to run the app with go."
    
    run:
     go run -ldflags="$(govvv -flags -pkg) -w -s" ./cmd/app-3/main.go
    
    test:
     go test -v -coverpkg=./... -coverprofile=profile.cov ./...
     go tool cover -func profile.cov
     rm profile.cov
    
    build:
     CGO_ENABLED=0 go build -mod=readonly -a -ldflags="$(govvv -flags -pkg $(go list ./info)) -w -s -linkmode external -extldflags '-static'" -o ./bin/app-3 ./cmd/app-3/*
    
    build-image:
     docker build -t mycompanytown/app-3:latest .

What do you use the SSL certs in the container for? I always have run Docker behind either a reverse proxy like Nginx or in some kind of cloud platform that handles SSL at the perimeter.

Add -ldflags '-s -w' to go build to strip DWARF, symbol table and debug info.