19

Aim: to create the smallest working docker images every time

Current

REPOSITORY          TAG       IMAGE ID            CREATED             SIZE
a-docker-image      latest    x                   42 minutes ago       1.92 GB

Attempt

Adding a cleanup step at the end of the Dockerfile:

#clean
RUN apt-get purge -y wget
RUN rm -r a-build-dir
RUN apt-get purge -y a-package

reduced the image size a little bit:

REPOSITORY          TAG       IMAGE ID            CREATED             SIZE
a-docker-image      latest    y                   2 minutes ago       1.86 GB

Discussion

I have built various docker images. Every time I try to decrease the size of the created image, but I always feel it is too large. I am looking for a script that has already been created by someone on github that removes all superfluous packages from the image so the size of the created image will be as small as possible.

As I said I always try to reduce the size of the image, but I want to apply this consistent so that every image that I create from now on will be as small as possible.

Question

How to create the smallest working docker image every time?

  • 2
    I like this concept. I feel like many of the containers I am using are quite bloated. – wogsland Mar 9 '17 at 1:23
1

There's a variety of techniques involved, with no single solution. You will likely want to do several of the following:


First, optimize your image layers for reuse. Put frequently changing steps later in the Dockerfile to increase the chances that early layers are cached from previous builds. A reused layer will show up as more disk space in a docker image ls, but if you examine the underlying filesystem, only one copy of each layer is ever stored on disk. That means 3 images of 2 GB each, but which only have 50 MB different in the last few layers the build, will only take up 2.1 GB of disk space, even though the listing makes it appear that they are using 6 GB since you are double counting each of the reused layers.

Layer reuse is why you see images with infrequently changing build dependencies install those first before copying in code. See any python example that has a pattern like:

FROM python
WORKDIR /app
COPY requirements.txt .
RUN pip install -r requirements.txt
# note how the code is copied only after the pip install
# since code changes but requirements.txt doesn't
COPY . .
CMD ["gunicorn", "app:app"]

Pick a minimal base image. This is why you see people go from ubuntu to debian:slim (the slim variants are smaller, shipping with fewer tools), or even alpine. This reduces the size of your starting point, and is very helpful if you are constantly pulling new versions of the base image. However, if your base image rarely changes, layer reuse removes much of the advantage of a minimal base image.

The smallest base image you can pick is scratch, which is nothing, no shell or libraries, and is only useful with statically compiled binaries. Otherwise, pick a base image that includes the tools you need without lots of tools you don't need.


Next, any step that changes or deletes a file should be combined with previous steps that create that file. Otherwise the layered filesystem, which uses copy-on-write even on things like a file permission change, will have the original file in a previous layer and the image size will not shrink when you remove files. This is why your rm commands have no effect on resulting disk space. Instead, you can chain the commands, like:

RUN apt-get update \
 && apt-get install -y \
      a-package \
      wget \
 && ... \
 && apt-get purge -y wget \
 && rm -r a-build-dir \
 && apt-get purge -y a-package

Note that overuse of command chaining can slow down your builds since you need to reinstall the same toolset any time a prerequisite changes (e.g. the code being pulled with wget). See multi-stage below for a better alternative.


Any file you create that you do not need in your resulting image should be deleted, in the step that creates it. This includes package caches, logs, man pages, etc. To discover what files are being created in each layer, you can use a tool like wagoodman/dive (which I have not personally vetted and would express caution since it runs with full root access on your host), or you can build your docker images without pruning the intermediate containers and then view the diff with:

# first create and leave containers from any RUN step using options on build
docker image build --rm=false --no-cache -t image_name . 
# review which layers use an unexpectedly large amount of space
docker image history image_name
# list all containers, particularly the exited ones from above
docker container ps -a 
# examine any of those containers
docker container diff ${container_id} 
# ... repeat the diff for other build steps
# then cleanup exited containers
docker container prune

With each of those intermediate containers, the diff will show what files are added, changed, or deleted in that step (these are indicated with an A, C, or D before each filename). What diff is showing is the container specific read/write filesystem, which is any file changed by the container from the image state using copy-on-write.


The best way to reduce image size is eliminating any unneeded components, like compilers, from your shipped image. For that, multi-stage builds let you compile in one stage, and then copy only the resulting artifacts from the build stage to a runtime image that has only the minimum needed to run the application. This avoids the need to optimize any of the build steps since they are not shipped with the resulting image.

FROM debian:9 as build
# still chain update with install to prevent stale cache issues
RUN apt-get update \
 && apt-get install -y \
      a-package \
      wget \
RUN ... # perform any download/compile steps

FROM debian:9-slim as release
COPY --from=build /usr/local/bin/app /usr/local/bin/app
CMD [ "/usr/local/bin/app" ]

Multi-stage is ideal with statically compiled binaries which you can run with scratch as your base image, or transitioning from a compile environment like JDK to a runtime like JRE. This is the easiest way to dramatically reduce your image size while still having fast builds. You may still perform chaining of steps in your release stage if you have steps that change or delete files created in previous steps, but for the most part, the COPY from another stage isolates the release stage from any layer bloat experienced in the earlier build stages.


Note, I do not recommend squashing images since this reduces the size of one image at the expense of eliminating layer reuse. That means future builds of the same image will require more disk and network traffic to send updates. To go back to the first example, squashing may reduce your image from 2 GB to 1 GB, but not 3 images may take up 3 GB instead of the 2.1 GB.

25

A Dockerfile creates a new layer for each of the commands in the file. Since layers are well, layered on top of each other - you cannot remove files that a previous layer added. This is why when you install packages, or download files, or create builds each in a separate command - these are still there in the image, even if in a future layer you removed them.

So if you just change this:

RUN apt-get update -y
RUN apt-get install -y wget a-package
# ...
RUN apt-get purge -y wget
RUN rm -r a-build-dir
RUN apt-get purge -y a-package

To this:

RUN apt-get update -y \
    && apt-get install -y wget a-package \
    && mkdir a-build-dir \
    && wget http://some-site/very-big-source-code.tar.gz \
    && tar xzvf very-big-source-code.tar.gz \
    && do-some-compilation \
    && apt-get purge -y wget \
    && cd .. \
    && rm -rf a-build-dir \
    && apt-get purge -y a-package

You will get a much smaller image.


Another option, is to squash the image after you built it. Q: How does the new docker --squash work?


Yet another option, is to choose a slim base image. For example, images that use Alpine Linux as their base instead of Debian, take just 10-15mb instead of 180-250mb. And this is before adding your own application and data. Many official base images on Docker Hub have an alpine version.

  • 3
    2.37 vs. 1.47 GB – 030 Mar 7 '17 at 22:48
4

Probably not exactly an answer, but worth giving the alternatives.

Chef's habitat has been created with this in mind, creating a package with all needed dependencies without the extraneous distro/base image load you don't wish.

Extracts on what matters here, the container size from this blog post with a simple nodejs app:

michael@ricardo-2:plans_pkg_part_2$ docker images
REPOSITORY           TAG                 IMAGE ID            CREATED             SIZE
mfdii/node-example   latest              36c6568c606b        40 minutes ago      655.9 MB
node                 latest              04c0ca2a8dad        16 hours ago        654.6 MB
mfdii/mytutorialapp  latest              534afd80d74d        2 minutes ago       182.1 MB

mdfii/node-example is a docker image from a classic dockerfile while mfdii/mytutorialapp is the docker image produced with habitat.

If size is your main concern and you're up to take the learning curve of Habitat plans, this could be a solution for you.

0

One could also use dive

docker run --rm -it \
    -v /var/run/docker.sock:/var/run/docker.sock \
    wagoodman/dive:latest <dive arguments...>

to get a report about what waste could be removed from a docker image in order to reduce the size.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.