2

We are in the middle of migrating from Self Hosted GitLab to Gitlab.com (Cloud). This also means that we are shifting building through shell, to building through Docker (using Kaniko).

A summary of our build can be found below.

Because GitLab is using Google Cloud for its runners, it is difficult for us to whitelist its IP. Therefor we would like to use the Gitlab.com owned runners for the build stage, and our own runner for the deploy stage.

The struggle is that cache is configured on a runner level rather on a global level. Therefor our runner in the deploy stage doesn't seem to be able to access the cache generated in the build stage.

Just a clarification. Right now the build stage doesn't use the image, as the image is only used to generate assets and composer and is then written to the cache. We think that this might be a faster method as it enables to share the image across multiple commits unless something in the docker configuration is changed.

What options are there to try?

Error in deploy_staging:

Checking cache for asset_build-feature/projectname-build... Runtime platform arch=amd64 os=linux pid=2787083 revision=f0a95a76 version=14.5.0 No URL provided, cache will not be downloaded from shared cache server. Instead a local version of cache will be extracted.

Build:

image_build:
  stage: image
  image:
    name: gcr.io/kaniko-project/executor:debug
    entrypoint: [""]
  script:
    - ls
    - cat ${CI_PROJECT_DIR}/DockerfileGitlab >> ${CI_PROJECT_DIR}/Dockerfile
    - mkdir -p /kaniko/.docker
    - echo "{\"auths\":{\"${CI_REGISTRY}\":{\"auth\":\"$(printf "%s:%s" "${CI_REGISTRY_USER}" "${CI_REGISTRY_PASSWORD}" | base64 | tr -d '\n')\"}}}" > /kaniko/.docker/config.json
    - >-
      /kaniko/executor
      --context "${CI_PROJECT_DIR}"
      --dockerfile "${CI_PROJECT_DIR}/Dockerfile"
      --destination "${CI_REGISTRY_IMAGE}:${CI_COMMIT_SHA}"

assets_build:
  image: ${CI_REGISTRY_IMAGE}:${CI_COMMIT_SHA}
  stage: build
  variables:
    GIT_STRATEGY: fetch
  needs: [ image_build ]
  before_script:
    - "pnpm install --frozen-lockfile"
  cache:
    - key: asset_build-$CI_COMMIT_REF_NAME-build
      paths:
        - $BUILD_FOLDER
      policy: push
    - key: asset_build-$CI_COMMIT_REF_NAME-pnpm
      paths:
        - /root/.pnpm-store
        - node_modules
      policy: pull-push
  script:
    - "NODE_ENV=prod ./node_modules/.bin/webpack --config webpack/webpack.config.js"

.deploy_global_template: &deploy_global
  stage: deploy
  tags: [ 'dev02' ]
  when: manual
  cache:
    key: 'asset_build-$CI_COMMIT_REF_NAME-build'
    paths:
      - $BUILD_FOLDER
    policy: pull
  needs:
    - job: assets_build
      artifacts: true
      optional: true
  environment: $DEPLOY_ENVIRONMENT
  script:
    - 'dep deploy $DEPLOY_ENVIRONMENT --branch=$CI_COMMIT_REF_NAME'

deploy_production:
  <<: *deploy_global
  variables:
    DEPLOY_ENVIRONMENT: production

deploy_staging:
  <<: *deploy_global
  variables:
    DEPLOY_ENVIRONMENT: staging

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.