34

Why this question?

The reason for posting this Q&A is that sometimes certain software is required to compile software in a docker image. Once compiled, these packages are superfluous and should be removed in order to reduce the image size. In some cases images that were more than 1.5GB where reduced to less than 300MB. The following use case does not represent a real world scenario, but provides an example of this use case.

Attempts to solve the issue

It works to remove a directory when the remove command is defined in the same layer as the creation of the folder:

FROM alpine
RUN mkdir dir && cd dir && wget http://google.com && rm -rf dir

This results in:

user@host$ docker build -t dir .
Sending build context to Docker daemon 1.336 MB
Step 1/2 : FROM alpine
 ---> 4a415e366388
Step 2/2 : RUN mkdir dir && cd dir && wget http://google.com && rm -rf dir
 ---> Using cache
 ---> c283805e687f
Successfully built c283805e687f

When the aim is to remove the directory in another layer, this fails:

FROM alpine
RUN mkdir dir && cd dir && wget http://google.com
RUN rm -r dir

And this fails:

FROM alpine
RUN mkdir dir && cd dir && wget http://google.com
RUN rm -rf dir

Another attempt was using workdir:

FROM alpine

RUN mkdir dir
WORKDIR dir
RUN wget http://google.com
WORKDIR /
RUN ls
RUN rm -r dir

Building the image resulted in:

Sending build context to Docker daemon 2.048 kB
Step 1/7 : FROM alpine
 ---> 4a415e366388
Step 2/7 : RUN mkdir dir
 ---> Using cache
 ---> aed5c75218cb
Step 3/7 : WORKDIR dir
 ---> Using cache
 ---> 715de09e7e08
Step 4/7 : RUN wget http://google.com
 ---> Using cache
 ---> c07bfb3e1133
Step 5/7 : WORKDIR /
 ---> Using cache
 ---> c6de86d0191a
Step 6/7 : RUN ls
 ---> Using cache
 ---> 9567593cce39
Step 7/7 : RUN rm -r dir
 ---> Running in 7fe7b28294bf
rm: can't remove 'dir': Directory not empty
The command '/bin/sh -c rm -r dir' returned a non-zero code: 1
Improve this question
2
  • You most certainly can have multiple RUN statements in your Dockerfile.
    – David Ficociello
    Apr 2, 2017 at 3:20
  • Check your Dockerfile. Looks like you copy pasted the same line twice. If you look at step 2 and 3 they are both executing the same code. That's why you're getting the no dir found. You've already deleted it in step 2
    – David Ficociello
    Apr 2, 2017 at 3:21

4 Answers 4

Reset to default
18

mkdir dir && cd dir && wget http://google.com && rm -rf dir doesn't do what you think it does.

Let's break it down:

  1. mkdir dir creates dir.
  2. cd dir changes directory into it.
  3. wget http://google.com downloads google.com inside dir/.
  4. rm -rf dir attempts to delete a directory called dir that's in the current directory.

The problem is that there is not dir in the current directory, because you're already cded into it. And since you've added the -f flag, the command doesn't produce an error. This is one of the reasons why you shouldn't use force flags without being very sure you need them.

Since you didn't use the WORKDIR command, the directory change is discarded at the end of the RUN. Thus, the next line executes in the original working directory, where it tries to mkdir dir but, as we've previously discussed, that directory still exists with the wgetted contents inside it.

Improve this answer
2
  • Sure it's possible to remove directories; you just specified the wrong path.
    – Xiong Chiamiov
    Apr 2, 2017 at 16:34
  • It's a bit of a strange error, considering that that's precisely what rm -r is intended to address. You can probably add -f in this case, but I would do some further investigation to see what exactly is in that directory (e.g. an ls -A). But you've gone past the original question at this point.
    – Xiong Chiamiov
    Apr 2, 2017 at 18:59
7

Recent Dockerfile spec allows for multiple FROM statements.

https://docs.docker.com/engine/userguide/eng-image/multistage-build/

Using this approach, you can retain your compilation results to a pure runtime environment.

Improve this answer
1
  • My situation is that - due to security reasons I am downloading the packages in the intermediate layer. Then, ADD'ing them to the final image. Followed by using RUN command to install the packages and delete the packages. My goal is to delete the downloaded packages once the RUN command installs them. Is there any way to a) combine the ADD and RUN into one layer; b) remove the ADD layer?
    – variable
    Jun 3, 2020 at 12:19
6

Building on Xiong Chiamiov's answer, which correctly identified the root cause of the problem - the dir reference by relative path when attempting to empty or delete that directory depends on the working directory at the time, which was not correctly set in the cases mentioned in the OP.

So there are 2 solutions available:

  • set the proper working dir prior to executing the dir removal:

      RUN mkdir dir && cd dir && wget http://google.com && cd .. && rm -rf dir

  • use a /dir full path reference to that directory instead. But with care (adjust if needed) if used in cases just similar, but not identical to this one, since this approach assumes the initial working dir was /:

      RUN mkdir dir && cd dir && wget http://google.com && rm -rf /dir
Improve this answer
4
  • 1
    Hmm. Why not rm -rf ../dir? That avoids the absolute path reference
    – jpaugh
    Apr 3, 2017 at 17:28
  • 1
    @jpaugh it might work, but it feels weird (cutting the branch you're sitting on?)
    – Dan Cornilescu
    Apr 3, 2017 at 18:29
  • That's true; but it works essentially the same way when you specify an absolute path. FWIW, I've done this before, in an interactive shell. ls tends to lose its meaning... ;-) You can even rm -rf ../dir && mkdir ../dir && cd ../dir, which puts you in an new, empty (valid) folder
    – jpaugh
    Apr 3, 2017 at 18:39
  • 1
    @jpaugh You're right, that's why I said it feels - it's just a feeling :)
    – Dan Cornilescu
    Apr 3, 2017 at 20:06
1

A solution is to remove the not empty directory using find:

Dockerfile

FROM alpine

RUN mkdir dir && cd dir && wget http://google.com && cd / && echo -e "BEFORE\n" && ls && find /dir -delete && echo -e "\nAFTER\n" && ls

Outcome

Sending build context to Docker daemon  2.048kB
Step 1/2 : FROM alpine
 ---> 4a415e366388
Step 2/2 : RUN mkdir dir && cd dir && wget http://google.com && cd / && echo -e "BEFORE\n" && ls && find /dir -delete && echo -e "\nAFTER\n" && ls
 ---> Running in ba93a1742a76
Connecting to google.com (172.217.20.110:80)
Connecting to www.google.nl (172.217.20.99:80)
index.html           100% |*******************************| 11092   0:00:00 ETA

BEFORE

bin
dev
dir
etc
home
lib
media
mnt
proc
root
run
sbin
srv
sys
tmp
usr
var

AFTER

bin
dev
etc
home
lib
media
mnt
proc
root
run
sbin
srv
sys
tmp
usr
var
 ---> 2e3109285a2d
Removing intermediate container ba93a1742a76
Successfully built 2e3109285a2d
Improve this answer
4
  • You have unnecessary stuff in there, I believe you missed the point in Xiong's answer - the problem is not the -rf options to the rm cmd.
    – Dan Cornilescu
    Apr 3, 2017 at 14:11
  • If I specify the whole path /path/to/dir and run it from / it still fails even with rf. I had to remove all the content from the folder.
    – 030
    Apr 4, 2017 at 10:18
  • Are you saying rm -rf doesn't work and one has to manually empty every dir before deleting the dir? Very odd.
    – Dan Cornilescu
    Apr 4, 2017 at 12:21
  • The rm -rf does not work indeed. I had to use find /path -delete
    – 030
    Apr 6, 2017 at 8:07

Not the answer you're looking for? Browse other questions tagged or ask your own question.