2

When developing code on a very large repository, I would like to run multiple builds/tests in parallel in containers. I need to get my repository or some other large directory into the container. If I mount it read/write, the parallel processes will fight each other and I will need to be able to clean after them, there is also issue with privilege escalations on those files. If I mount it read/only then I cannot run processes that write files inside the directory.

Docker suggest to build a new container with a COPY of the repository, save it as new layer and then start several containers from this new image. Having to make a copy of megabytes of data every time you want to run tests is making it take too long though.

How can I solve this efficiently?

1

Solution to this problem is to create an overlay filesytem over a read-only mount, but if you try to do it directly, overlay will refuse to put upper and work directories on another overlay filesystem. The trick is to create a tmpfs for the upper and work directories like this:

Create a script called run-in-c.sh

#!/bin/bash
NAME=$1
shift
HOSTNAME=dock${NAME}
CONTAINER=dc-${USER}-${NAME}
REPOSITORY=${HOME}/repository
BASEIMAGE=hub.docker.io/my-org/my-base-container
OVERLAY=/mnt/overlay
LOWERDIR=/mnt/lower
UPPERDIR=${OVERLAY}/upper
WORKDIR=${OVERLAY}/work
TARGET=/mnt/repository
# PRIVILEGED="--privileged"
PRIVILEGED="--cap-add SYS_ADMIN"
docker container create --name $CONTAINER $PRIVILEGED --hostname $HOSTNAME --volume ${CO}:${LOWERDIR}:ro $BASEIMAGE
docker container start $CONTAINER
docker container exec $CONTAINER mkdir -p $OVERLAY
docker container exec $CONTAINER mount -t tmpfs tmpfs $OVERLAY
docker container exec $CONTAINER mkdir -p $WORKDIR $UPPERDIR $TARGET
docker container exec $CONTAINER mount -t overlay overlay -o lowerdir=${LOWERDIR},upperdir=${UPPERDIR},workdir=${WORKDIR} $TARGET
docker container exec -it --workdir $TARGET $CONTAINER $*
docker container stop $CONTAINER
docker container rm $CONTAINER

Then you can run commands in the container as:

run-in-c.sh test01 'cd dir && command args'

or simply get interactive shell with:

run-in-c.sh naming-stuff-is-too-hard bash
  • This answer requires you to run as root inside the container as the volume is mounted as root:root into the container and if you executed your commands as user, you would not be able to write files (unless all your directories are world writable). If you try to chown all files in the repository, it would create a copy of each of them in the upper layer, which would be counter productive. If there was some way to make this solution running as a regular user, I would be interested to see it. – Jiri Klouda Apr 13 '18 at 5:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.