3

I use GitLab's pipelines to describe the deployment process. And, as part of it, I build Docker images. For that purpose, I use Docker-in-Docker to build those images (i.e. download the Docker-in-Docker image and do all the Docker-stuff, inside of it).

I also set to use Docker Buildkit features. And one of them is --cache-from option (which decreases build time). Which seems doesn't work with Docker-in-Docker (at least, for me).

Here's the pipeline's config:

stages:
  - build
  - purge
  - deploy

variables:
  DOCKER_HOST: tcp://localhost:2375
  DOCKER_TLS_CERTDIR: ""
  DOCKER_BUILDKIT: 1
  LATEST: ${CI_REGISTRY_IMAGE}/${CI_PROJECT_NAME}:${CI_COMMIT_REF_SLUG}
  IMAGE_COMMIT_TAG: ${CI_REGISTRY_IMAGE}/${CI_PROJECT_NAME}:${CI_COMMIT_SHORT_SHA}

services:
  - docker:dind

build:
    stage: build
    image: docker:stable
    script:
        - docker login -u gitlab-ci-token -p ${CI_BUILD_TOKEN} ${CI_REGISTRY}
        - docker build --cache-from ${LATEST} -t ${LATEST} -t ${IMAGE_COMMIT_TAG} .
        - docker push ${IMAGE_COMMIT_TAG}
        - docker push ${LATEST}

The log output shows the BuildKit is engaged, but there's nothing about using of 'CACHE'. And, the result of it -- 'build' stage takes much more time.

Please, your suggestions, ideas?

3

Caching of layers with buildkit in an external registry requires an extra step or two depending on how you want to cache your layers. The easy option is to include a build arg that enables the inline cache:

docker build --build-arg BUILDKIT_INLINE_CACHE=1 --cache-from ${LATEST} -t ${LATEST} -t ${IMAGE_COMMIT_TAG} .

Note that the inline cache only caches layers for the target stage that was pushed in the image, so other states in a multi-stage build would need to be built and cached separately, or rebuilt without caching, neither of which is ideal.

You can also cache to a local file, or push the cache to a different registry image rather than inline with the image you pushed. Unfortunately the standard docker build CLI doesn't have access to all the buildkit flags to enable this. Instead, you can install buildkit directly or use buildx which is a CLI plugin for managing buildkit. You can install buildx separately but in current releases it's available with experimental CLI options that can be enabled with export DOCKER_CLI_EXPERIMENTAL=enabled. I believe you'd need to create a container based builder for all of the buildkit options, which can be done with:

docker buildx create --use --driver docker-container --name local ${DOCKER_HOST:-unix:///var/run/docker.sock}

Then you should be able to run something like:

docker buildx build --cache-from ${IMG_CACHE} --cache-to ${IMG_CACHE} -t ${LATEST} -t ${IMAGE_COMMIT_TAG} .

More documentation on the buildkit caching is available at: https://github.com/moby/buildkit#export-cache

More details on buildx is available at: https://github.com/docker/buildx

8
  • 1
    HI, @BMitch! Sorry for such a late response (I was offline up to now) and thank you for your answer. I know about buildx, but it's not applicable, as I don't have any Docker stuff persistent. I have immutable GL-runner :( and that's why I have to use Docker as service. They same reason is about using external cache. – acd Apr 1 '20 at 10:58
  • BTW, when I run the build-stage without DinD (on different GL-runner) and any external cache, Docker uses cache from the built images not its own build-cache. – acd Apr 1 '20 at 11:02
  • @acd an immutable runner would not be able to build an image. Perhaps you mean an ephemeral runner, which should not prevent you from running docker commands. – BMitch Apr 1 '20 at 11:51
  • No-no. "Immutable runner" means persistent instance, which can't be changed, that is to setup buildkitd, buildx, as a Docker extension, etc.. And, Docker is ephemeral, which is Docker-in-Docker, used in services of .gitlab-ci. – acd Apr 1 '20 at 16:50
  • @acd I'm not finding much in my search for gitlab immutable runner, perhaps you're referring to a shared runner? Immutable would mean no write access to /var/lib/docker which would mean all image builds would fail because of a read-only filesystem. It's also not clear from the comments that you attempted all three of the options suggested above. Output of the errors would help. – BMitch Apr 1 '20 at 18:02
0

Though it's not the direct answer to my question, but it's a goal, I was aimed at.

I tested several image build tools and stopped on 'Makisu', for building Docker images. It really works fine for me.

The details are here: https://github.com/uber/makisu

and here (starting from 'Using Makisu'): https://findwork.dev/blog/6-ways-speed-your-ci/

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.