20

This question already has an answer here:

Why this question?

The reason for posting this Q&A is that sometimes certain software is required to compile software in a docker image. Once compiled, these packages are superfluous and should be removed in order to reduce the image size. In some cases images that were more than 1.5GB where reduced to less than 300MB. The following use case does not represent a real world scenario, but provides an example of this use case.

Attempts to solve the issue

It works to remove a directory when the remove command is defined in the same layer as the creation of the folder:

FROM alpine
RUN mkdir dir && cd dir && wget http://google.com && rm -rf dir

results in:

user@host$ docker build -t dir .
Sending build context to Docker daemon 1.336 MB
Step 1/2 : FROM alpine
 ---> 4a415e366388
Step 2/2 : RUN mkdir dir && cd dir && wget http://google.com && rm -rf dir
 ---> Using cache
 ---> c283805e687f
Successfully built c283805e687f

When the aim is to remove the directory in another layer, either this:

FROM alpine
RUN mkdir dir && cd dir && wget http://google.com
RUN rm -r dir

or this fails:

FROM alpine
RUN mkdir dir && cd dir && wget http://google.com
RUN rm -rf dir

another attempt was using workdir

FROM alpine

RUN mkdir dir
WORKDIR dir
RUN wget http://google.com
WORKDIR /
RUN ls
RUN rm -r dir

building the image resulted in:

Sending build context to Docker daemon 2.048 kB
Step 1/7 : FROM alpine
 ---> 4a415e366388
Step 2/7 : RUN mkdir dir
 ---> Using cache
 ---> aed5c75218cb
Step 3/7 : WORKDIR dir
 ---> Using cache
 ---> 715de09e7e08
Step 4/7 : RUN wget http://google.com
 ---> Using cache
 ---> c07bfb3e1133
Step 5/7 : WORKDIR /
 ---> Using cache
 ---> c6de86d0191a
Step 6/7 : RUN ls
 ---> Using cache
 ---> 9567593cce39
Step 7/7 : RUN rm -r dir
 ---> Running in 7fe7b28294bf
rm: can't remove 'dir': Directory not empty
The command '/bin/sh -c rm -r dir' returned a non-zero code: 1

marked as duplicate by 030, Community Jul 17 at 14:24

This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.

  • You most certainly can have multiple RUN statements in your Dockerfile. – David Ficociello Apr 2 '17 at 3:20
  • Check your Dockerfile. Looks like you copy pasted the same line twice. If you look at step 2 and 3 they are both executing the same code. That's why you're getting the no dir found. You've already deleted it in step 2 – David Ficociello Apr 2 '17 at 3:21
12

mkdir dir && cd dir && wget http://google.com && rm -rf dir doesn't do what you think it does.

Let's break it down:

  1. mkdir dir creates dir.
  2. cd dir changes directory into it.
  3. wget http://google.com downloads google.com inside dir/.
  4. rm -rf dir attempts to delete a directory called dir that's in the current directory.

The problem is that there is not dir in the current directory, because you're already cded into it. And since you've added the -f flag, the command doesn't produce an error. This is one of the reasons why you shouldn't use force flags without being very sure you need them.

Since you didn't use the WORKDIR command, the directory change is discarded at the end of the RUN. Thus, the next line executes in the original working directory, where it tries to mkdir dir but, as we've previously discussed, that directory still exists with the wgetted contents inside it.

  • Sure it's possible to remove directories; you just specified the wrong path. – Xiong Chiamiov Apr 2 '17 at 16:34
  • It's a bit of a strange error, considering that that's precisely what rm -r is intended to address. You can probably add -f in this case, but I would do some further investigation to see what exactly is in that directory (e.g. an ls -A). But you've gone past the original question at this point. – Xiong Chiamiov Apr 2 '17 at 18:59
4

Building on Xiong Chiamiov's answer, which correctly identified the root cause of the problem - the dir reference by relative path when attempting to empty or delete that directory depends on the working directory at the time, which was not correctly set in the cases mentioned in the OP.

So there are 2 solutions available:

  • set the proper working dir prior to executing the dir removal:

    RUN mkdir dir && cd dir && wget http://google.com && cd .. %% rm -rf dir
    
  • use a /dir full path reference to that directory instead. But with care (adjust if needed) if used in cases just similar, but not identical to this one, since this approach assumes the initial working dir was /:

    RUN mkdir dir && cd dir && wget http://google.com && rm -rf /dir
    
  • 1
    Hmm. Why not rm -rf ../dir? That avoids the absolute path reference – jpaugh Apr 3 '17 at 17:28
  • 1
    @jpaugh it might work, but it feels weird (cutting the branch you're sitting on?) – Dan Cornilescu Apr 3 '17 at 18:29
  • That's true; but it works essentially the same way when you specify an absolute path. FWIW, I've done this before, in an interactive shell. ls tends to lose its meaning... ;-) You can even rm -rf ../dir && mkdir ../dir && cd ../dir, which puts you in an new, empty (valid) folder – jpaugh Apr 3 '17 at 18:39
  • @jpaugh You're right, that's why I said it feels - it's just a feeling :) – Dan Cornilescu Apr 3 '17 at 20:06
4

Recent Dockerfile spec allows for multiple FROM statements.

https://docs.docker.com/engine/userguide/eng-image/multistage-build/

Using this approach, you can retain your compilation results to a pure runtime environment.

1

A solution is to remove the not empty directory using find:

Dockerfile

FROM alpine

RUN mkdir dir && cd dir && wget http://google.com && cd / && echo -e "BEFORE\n" && ls && find /dir -delete && echo -e "\nAFTER\n" && ls

Outcome

Sending build context to Docker daemon  2.048kB
Step 1/2 : FROM alpine
 ---> 4a415e366388
Step 2/2 : RUN mkdir dir && cd dir && wget http://google.com && cd / && echo -e "BEFORE\n" && ls && find /dir -delete && echo -e "\nAFTER\n" && ls
 ---> Running in ba93a1742a76
Connecting to google.com (172.217.20.110:80)
Connecting to www.google.nl (172.217.20.99:80)
index.html           100% |*******************************| 11092   0:00:00 ETA

BEFORE

bin
dev
dir
etc
home
lib
media
mnt
proc
root
run
sbin
srv
sys
tmp
usr
var

AFTER

bin
dev
etc
home
lib
media
mnt
proc
root
run
sbin
srv
sys
tmp
usr
var
 ---> 2e3109285a2d
Removing intermediate container ba93a1742a76
Successfully built 2e3109285a2d
  • You have unnecessary stuff in there, I believe you missed the point in Xiong's answer - the problem is not the -rf options to the rm cmd. – Dan Cornilescu Apr 3 '17 at 14:11
  • If I specify the whole path /path/to/dir and run it from / it still fails even with rf. I had to remove all the content from the folder. – 030 Apr 4 '17 at 10:18
  • Are you saying rm -rf doesn't work and one has to manually empty every dir before deleting the dir? Very odd. – Dan Cornilescu Apr 4 '17 at 12:21
  • The rm -rf does not work indeed. I had to use find /path -delete – 030 Apr 6 '17 at 8:07

Not the answer you're looking for? Browse other questions tagged or ask your own question.