In current deployment of a web app we are using svn to deploy a specific revision from a repository. The amount of data transferred is just the diff between the currently deployed revision and the revision being deployed, which is rather small. Deployments go to hundreds of machines, hundreds times a week.

We would like to run the app in a container for extra testing, fast reverts and service separation, but there is a problem. When creating a container, we copy the app in there as one layer, then run the container through all tests, security scanning, etc. But during deployments, the amount of data transferred is the entire layer with the app each time. There has to be some way to minimize the amount of data transferred just to the diff between revisions, that is standard and has some tools around it. It does not seem like we are the only ones trying to solve the problem.

So before we implement some home grown solution, what would you do?

  • 3
    wouldn't it be acceptable to run the svn up inside the container instead of COPY of the full source code? The container layers would then be a bit smaller and you can iteratively move forward with the base image. Commented Nov 26, 2018 at 5:06
  • The app is only a small part of the mono-repo, but yes, we considered some sort of home grown solution where we do a full COPY every week or so and multiple small COPY for revision differences from that base container with the full copy layer. Should we do copy for each update and have multiple layers each day or should we use one big copy from the weekly or daily base? There are questions. But that is something that will take a lot of work to code and maintain and I was hoping someone has already faced similar problems and has a solution somewhere ready to reuse. Commented Nov 26, 2018 at 16:45
  • 2
    Have you read this github.com/moby/moby/issues/21950. Let us better understand why you need this; because its slow or because of expensive bandwidth or both or curiosity only? Are you using docker on target machines already?
    – Kyslik
    Commented Nov 29, 2018 at 11:45
  • Yes, I've read it before I asked, it was one of the pages coming in the searches and we already use overlay driver so we don't hit this issue. I am trying to see if someone implemented the multiple nested containers and the management of the updating of the base containers. If two or three nested levels would be better, how would look the orchestration of the build pipeline through jenkins for example. If someone already solved the issues, maybe has a solution template available somewhere on github, where they already solved the dozens of small issues that usually pop up. Commented Nov 29, 2018 at 17:55