back
220 comments
> Everything after that removes the manifest files and any temporary files downloaded during this command. It's necessary to remove all these files in this command to keep the size of the Docker image to a minimum. Smaller Dockerfiles mean faster deployments.

It isn't explicitly explained, but the reason why it must be in this command and not separated out is because each command in a dockerfile creates a new "layer". Removing the files in another command will work, but it does nothing to decrease the overall image size: as far as whatever filesystem driver you're using is concerned, deleting files from earlier layers is just masking them, whereas deleting them before creating the layer prevents them from ever actually being stored.

If all you want to do is run a series of commands while only creating a single layer, heredocs are probably the simplest / most readable approach:

https://www.docker.com/blog/introduction-to-heredocs-in-dock...

The older “Docker without Docker” blogpost linked from there goes into that, it’s one of the best deep dives into containers (and honestly one of the best pieces of technical writing full stop) I’ve come across. https://fly.io/blog/docker-without-docker/
An terbative to removing files or go through contortions to stuff things in a single layer is to use a builder image and copy the generated artefacts into a clean image:

    FROM foo AS builder

    .. build steps

    FROM foo

    COPY --from=builder generated-file target
(I hope I got that right; on a phone and been a while since I did this from scratch, but you get the overall point)
The new best practice is to use the RUN --mount cache options. Making the removal of intermediate files unnecessary and speeds up the builds too. Surprised to see so few mentions of it.
Yeah this is a very widely misunderstood or unknown thing about docker files. After the nth time explaining it to somebody, I finally threw it into a short video with demo to explain how it worked: https://youtu.be/RP-z4dqRTZA
Self hoisting here, I put this together to make it easier to generate single (extra) layer docker images without needing a docker daemon, capabilities, chroot, etc: https://github.com/andrewbaxter/dinker

Caveat: it doesn't work on Fly.io. They seem to be having some issue with OCI manifests: https://github.com/containers/skopeo/issues/1881 . They're also having issues with new docker versions pushing from CI: https://community.fly.io/t/deploying-to-fly-via-github-actio... ... the timing of this post seems weird.

FWIW the article says

> create a Docker image, also known as an OCI image

I don't think this is quite right. From my investigation, Docker and OCI images are basically content addressed trees, starting with a root manifest that points to other files and their hashes (root -> images -> layers -> layer configs + files). The OCI manifests and configs are separate to Docker manifests and configs and basically Docker will support both side by side.

I wanted to explain layers really bad, Sam Ruby even put comments in there about it, but I stayed away from it for the sake of more clearly explaining how Linux was configured for Rails apps.

It is really weird looking at Dockerfiles for the first time seeing all of the `&% \` bash commands chained together.

Note that the explicit "apt-get clean" is probably redundant:

https://docs.docker.com/develop/develop-images/dockerfile_be...

> Official Debian and Ubuntu images automatically run apt-get clean, so explicit invocation is not required.

Is there some reason not to use multi-stage builds for this? Blogspam: https://sequoia.makes.software/reducing-docker-image-size-pa...
If you want to check your images for some common leftover files in all the layers, I made an app for that: https://github.com/viraptor/cruftspy
Every command in a dockerfile creates a layer, and most Dockerfile builders cache each if that line hasn't changed. Dynamic callouts to run updates or check things on the web won't rerun.
Such a helpful comment. Just in this one note, I learned a few things about Docker that I had no idea about:

1) a layer is essentially a docker image in itself and

2) a layer is as static as the image itself

3) Docker images ship with all layers

Thanks jchw!

If you only care about the final image size, then `docker build --squash` squashes the layers for you as well.
I used to think I hated Docker, but I think what I actually hate is using Docker locally (building/rebuilding/cache-busting images, spinning containers up and down, the extra memory usage on macOS, etc etc). I don't need all that for development, I just want to run my dang code

But I've been really enjoying it as a way of just telling a PaaS "hey here's the compiler/runtime my code needs to run", and then mostly not having to worry about it from there. It means these services don't have to have a single list of blessed languages, while pretty much keeping the same PaaS user experience, which is great

> the extra memory usage on macOS

It's worth noting that the Docker experience is very different across platforms. If you just run Docker on Linux, it's basically no different than just running any other binary on the machine. On macOS and Windows, you have the overhead of a VM and its RAM to contend with at minimum, but in many cases you also have to deal with sending files over the wire or worse, mounting filesystems across the two OSes, dealing with all of the incongruities of their filesystem and VFS layers and the limitations of taking syscalls and making them go over serialized I/O.

Honestly, Docker, Inc. has put entirely too much work into making it decent. It's probably about as good as it can be without improvements in the operating systems that it runs on.

I think this is unfortunate because a lot of the downsides of "Docker" locally are actually just the downsides of running a VM. (BTW, in case it's not apparent, this is the same with WSL2: WSL2 is a pretty good implementation of the Linux-in-a-VM thing, but it's still just that. Managing memory usage is, in particular, a sore spot for WSL2.)

(Obviously, it's not exactly like running binaries directly, due to the many different namespacing and security APIs docker uses to isolate the container from the host system, but it's not meaningfully different. You can also turn these things off at will, too.)

As someone who has not yet adopted to Docker, even though that seems to be what is done on contemporary best-in-class PaaS....

I don't totally understand the maintenance story. On heroku with buildpacks, I don't need to worry about OS-level security patches, patches to anything that was included in the base stack provided by the PaaS, they are responsible for. Which I consider part of the value proposition.

When I switch to using "Docker" containers to specify my environment to the PaaS... if there is a security patch to the OS that I loaded in one of my layers... it's up to me to notice and update my container? Or what? How is this actually handled in practice?

(And I agree with someone else in this thread that this fly.io series of essays on docker is _great_, it's helping me understand docker better whether or not I use fly.io!)

I work a lot on modernizing codebases. Many times it will be codebases without any proper testing, different versions of frameworks and programming languages, etc.

Currently, I'm working in 3 projects at once + my own. Even though it's all PHP, the projects are PHP 8.1 / Symfony 6.2, PHP 8.2 / Symfony 6.2, PHP 8.1 / Laravel 8 and the newest project I joined is PHP 7.4 / Symfony 4. I have to adhere to different standards and different setups; I don't want to switch the language and the yarn or npm versions or the Postgres or MySQL versions and remember each one. Some might use RabbitMQ, another Elasticsearch.

Docker helps me tremendously here.

Additionally, I add a Makefile for each project as well and include at least "make up" (to start the whole app), "make down", "make enter" (to enter the main container I'll be working with), "make test" (usually phpunit) and perhaps "make prepare" (for phpunit, phpstan, php-cs-fixer, database validate) as a final check before adding a new commit.

Locally, I had additional aliases on my shell. "t" for "make test", "up" for "make up".

So now I just have to go to a project folder, write "up" and am usually good to go!

Docker can be a pain sometimes (especially docker for mac!), but in moderately complex apps, it's so nice to be able to just fire up 6 services (even some smaller apps have redis, memcached, pg/mysql, etc) and have a running dev environment.

If someone updates nodejs, just pull in the latest dockerfile. Someone updates to a new postgres version? Same thing. It's so much better than managing native dependencies.

> I don't need all that for development, I just want to run the dang code

Have you ever worked somewhere where in order to run the code locally on your machine, it's a blend of "install RabbitMQ on your machine, connect to MS-SQL in dev, use a local Redis, connect to Cassandra in dev, run 1 service that this service routes through/calls to locally, but then that service will call to 2 services in dev", etc?

If anyone is looking for a complete guide I put together this last month: https://nickjanetakis.com/blog/a-guide-for-running-rails-in-...

It includes running Rails and also Sidekiq, Postgres, Redis, Action Cable and ties in esbuild and Tailwind too. It's all set up to use Hotwire as well. It's managed by Docker Compose. The post also includes a ~1h hour ad-free YouTube video. The example app is open source at https://github.com/nickjj/docker-rails-example and it's optimized for both development and production. No strings attached. The example app has been maintained and deployed a bunch over the years.

Am I the only person who struggles to deploy Rails apps.

It's a super productive framework to develop in, but deploying an actuals Rails apps - after nearly 20 years of existance, still seems way more difficult than it should be.

Maybe it's just me.

I have my own local development Rails setup and template files that could be dropped in any project with minimal changes (mostly around configuring the db connection)

- https://gitlab.com/sdwolfz/docker-projects/-/tree/master/rai...

Haven't spent the time to document it. But the general idea is to have a `make` target that orchestrates everything so `docker-compose` can just spin things up.

I've used this sort of thing for multiple types of projects, not just Rails, it can work with any framework granted you have the right docker images.

For deployment I have something similar, builds upon the same concepts (with ansible instead of make, and focused on multi-server deploys, terraform for setting up the cloud resources), but not open sourced yet.

Maybe I'll get to document it and post my own "Show HN" with this soon.

I do miss the pre-docker days of using capistrano to deploy rails projects. Most deploys would take less than two minutes in the CI server and most of that was tests. The deploys were hot and requests that happened during the deploy weren't interrupted. Now with Docker I'm seeing most deploys take around ten minutes.

The downside of capistrano was that you'd be responsible for patching dependencies outside of the Gemfile and there would be occasional inconsistencies between environments.

Thanks but I already have my own containers and infrastructure for dev and prod Rails.

It's fine for people who are just starting out or want a repeatable environment.

Also, if you want stability and fewer headaches long-term:

- use a RHEL-derived kernel and customize the userland (container or host) quay.io has a good cent 9 stream. Ubuntu isn't used at significant scale for multiple reasons, and migrating over later is a pain.

- consider podman over docker

- use packaging (nix, habitat, or rpms) rather than make install (and use site-wide sccache)

- container management (k8s or nomad)

- configuration management (chef) because you don't always have the luxury of 12factor ephemeral instances based on dockerfiles and need to make changes immediately without throwing away a database cluster or zookeeper ensemble

- Shard configuration and app changes, with a rollback capability

- Have CI/CD for infrastructure that runs before landing

- Monitoring and alerting

- Don't commit directly to production except for emergencies. Require a code review signoff by another engineer. And be able to back out changes.

- Have good, tested backups that aren't replication

- Don't sweat the small stuff, but get the big stuff right that doesn't compound tech debt

This is really cool. Is the Rails server production ready? I was always under the impression you had to run it with Uvicorn or similar, although I haven't been following Rails development recently.
Will it be called "Ruby on Whales"? Joke aside, it's trivial to write your own Dockerfile and this still is what nontrivial apps will do, due to customization.
> RAILS_SERVE_STATIC_FILES - This instructs Rails to not serve static files.

...but... it's set to True....

I've found the "Docker for Rails Developers" book by Rob Isenberg [1] to be a great resource for getting started using Rails and Docker. It's only a couple years out of date at this point but should still be highly relevant for anyone trying to get started. The only issue I've had with Rails and Docker is serving the container on a 1gb d.o. droplet - the node-sass gem is usually a sticking point for a lot of people and believe the 1gb droplet to be just too small to host a dockerized rails app. But the benefits of using docker for Development is still overwhelmingly worth the effort of containerizing things.

It's super cool rails 7.1 is including the Dockerfile by default - not that rails apps need more boilerplate though..

[1]: https://pragprog.com/titles/ridocker/docker-for-rails-develo...

This is long overdue. Rails got very nice updates in the past years to make it easier to handle JS and other assets.

Deploying it was still always a hassle and involved searching for existing Dockerfiles and blog posts to cobble together a working one. At the beginning I always thought I'm doing something wrong as it's supposed to be easy and do everything nicely out of the box.

And dhh apparently agrees (Now at least: https://dhh.dk/posts/30-myth-1-rails-is-hard-to-deploy) as there's now a default Dockerfile and also this project he's working on, this will make things a lot nicer and more polished: https://github.com/rails/mrsk

I have a proposal out for making the deployment story even easier by providing RubyGems a way to describe the packages they depend on to operate properly at https://community.fly.io/t/proposal-declare-docker-images-de...

The idea is I could run a command like `bundle packages --manager=apt` and get a list of all the packages `apt` should install for the gems in my bundle.

Since I know almost nothing about the technicalities and community norms of package management in Linux, macOS, and Windows, I'm hoping to find people who do and care about making the Ruby deployment story even better to give feedback on the proposal.

This is a manual using that image with Redis and Postgres: https://medium.com/@gustavoinzunza/rails-redis-pg-with-docke...
This is a really good news. And to be fun, the easiest way to become mainstream, is to have a guide on how to deploy a Rails application in 1 click, for any cloud hosting service.
I still use Capistrano. In fact, I like Capistrano so much that I have full Load Balancing, Auto-Scaling, and End-to-End encryption enabled for my projects on AWS via the elbas gem.

My primary use case for this is PCI Compliance. While PCI DSS and/or HIPAA do not specifically rule out Docker, the principle of isolation leans heavily twoard the principle that web hosts must be running on a private virtual machine.

This rules out almost all docker based-PaaS (including Fly.io, Render.com, AWS App Runner, and Digital Ocean), as these run your containers on general Docker hosts. In fact, the only PaaS provider that I can find advertising PCI compliance is Heroku, which now charges +$1800/month plus for Heroku Private to achieve it.

I would love to share my configuration with anyone that needs it.

For some more (too many?) in-depth tricks for rails and docker, see also:

"Ruby on Whales: Dockerizing Ruby and Rails development"

https://evilmartians.com/chronicles/ruby-on-whales-docker-fo...

Previously posted to hn - but without any comments.

I use FROM scratch. The app itself is built with a binary packer that is copied in.

Still, I'd much rather use Nix/Guix over Dockerfile.

This is awesome and I'm glad Rails is adding something official here. Unfortunately if you want to use the Dockerfile in development with docker-compose you will need to make some changes/additions. Most notably: only precompile assets if deploying, not during development
Though this is a great step... you still have to edit the Dockerfile if you are using Postgres and Redis.
Deploying with Cloud66 is a lot easier. Just git push.

I can chose my hosting provider and switch each other.

What kills rails for me at the moment is the time it takes to start up. I'd love to be able to use on top of things like cloud run where resources ramp down to zero when there are no requests, but the startup time makes this very difficult.
It's been a while since I've done any Ruby/Rails development, but just curious why they chose to use a Debian/Ubuntu based image in the default Dockerfile instead of an Alpine based image?
As someone who doesn’t know rails at all, what’s the innovation here? Surely Rails has had Dockerfiles written for it before?
How on earth did Rails not have an official Dockerfile until now? Have people been deploying like it's 2010?
I use dokku. Works really well.
But how does this work when talking to dev databases on my machine?