Scenario
I'm using LFTP to deploy files via sftp. The transfer completes successfully and all files get to the destination but for some reason the job fails.
Question
How can I find out what is causing my deploy-stage
job to fail?
Attempts
- commented out
when
- no success - tried a couple variations of
lftp
usage - no success
Gitlab CI .yml
image: node:10.19.0 # https://hub.docker.com/_/node/
# image: node:latest
cache:
# untracked: true
key: my-project
# key: ${CI_COMMIT_REF_SLUG} # per branch
# key:
# files:
# - package-lock.json # only update cache when this file changes (not working) @jkr
paths:
- .npm/
- node_modules
- build
stages:
- prepare # prepares builds, makes build needed for testing
- test # uses test:build specifically @jkr
- build
- deploy
# before_install:
before_script:
- npm ci --cache .npm --prefer-offline
prepare:
stage: prepare
needs: []
script:
- npm install
- npm run build:test
test:
stage: test
needs: [prepare]
except:
- schedules
tags:
- linux
script:
- npm run test:ci-deps
- npm run test:e2e # runs puppeteer tests @jkr
artifacts:
reports:
junit: junit.xml
paths:
- coverage/
build:
stage: build
needs: [prepare]
only:
- schedules
before_script:
- apt-get update && apt-get install -y zip
script:
- npm run build:stage
- zip -r build.zip build
# cache:
# paths:
# - build
# <<: *global_cache
# policy: push
artifacts:
paths:
- build.zip
deploy-stage:
stage: deploy
needs: [build]
tags: [linux]
only:
- schedules
# # - branches@gitlab-org/gitlab
before_script:
- apt-get update && apt-get install -y lftp
script:
# temporarily using 'verify-certificate no'
# for more on verify-certificate @jkr: https://www.versatilewebsolutions.com/blog/2014/04/lftp-ftps-and-certificate-verification.html
# variables do not work with 'single quotes' unless they are "'surrounded by doubles'"
- lftp -e "set ssl:verify-certificate no; open my-project.com; user $LFTP_USERNAME $LFTP_PASSWORD; mirror --reverse --verbose build/ /var/www/domains/dev/projects/my-project/build/; bye"
environment:
name: staging
url: http://dev.my-project.com/projects/my-project/build
# url: https://stg2.my-project.org
when: manual
# allow_failure: true
deploy-prod:
stage: deploy
needs: [build]
tags: [linux]
only:
- schedules
# - master
before_script:
- apt-get update && apt-get install -y lftp
script:
- sh deploy-prod
environment:
name: production
url: http://www.my-project.com
when: manual
# allow_failure: true
Job Output
Running with gitlab-runner 13.2.2 (a998cacd)
on docker-auto-scale fa6cab46
Preparing the "docker+machine" executor
00:40
Using Docker executor with image node:10.19.0 ...
Pulling docker image node:10.19.0 ...
Using docker image sha256:01b816051d343a5aaa3b33e165f6b6d620b7e899961bf6e8955a7577a0129873 for node:10.19.0 ...
Preparing environment
00:04
Running on runner-fa6cab46-project-13332271-concurrent-0 via runner-fa6cab46-srm-1596764203-a2a74b3b...
Getting source from Git repository
00:06
$ eval "$CI_PRE_CLONE_SCRIPT"
Fetching changes with git depth set to 50...
Initialized empty Git repository in /builds/my-project/.git/
Created fresh repository.
Checking out d4675c89 as pipeline-deploy-fail...
Skipping Git submodules setup
Restoring cache
00:26
Checking cache for my-project...
Downloading cache.zip from https://storage.googleapis.com/gitlab-com-runners-cache/project/13332271/my-project
Successfully extracted cache
Executing "step_script" stage of the job script
00:25
$ apt-get update && apt-get install -y lftp
Ign:1 http://deb.debian.org/debian stretch InRelease
Get:2 http://security.debian.org/debian-security stretch/updates InRelease [53.0 kB]
Get:3 http://deb.debian.org/debian stretch-updates InRelease [93.6 kB]
Get:4 http://deb.debian.org/debian stretch Release [118 kB]
Get:5 http://deb.debian.org/debian stretch Release.gpg [2410 B]
Get:6 http://security.debian.org/debian-security stretch/updates/main amd64 Packages [546 kB]
Get:7 http://deb.debian.org/debian stretch-updates/main amd64 Packages [2596 B]
Get:8 http://deb.debian.org/debian stretch/main amd64 Packages [7080 kB]
Fetched 7895 kB in 2s (3798 kB/s)
Reading package lists...
Reading package lists...
Building dependency tree...
Reading state information...
The following NEW packages will be installed:
lftp
0 upgraded, 1 newly installed, 0 to remove and 65 not upgraded.
Need to get 711 kB of archives.
After this operation, 2235 kB of additional disk space will be used.
Get:1 http://deb.debian.org/debian stretch/main amd64 lftp amd64 4.7.4-1 [711 kB]
debconf: delaying package configuration, since apt-utils is not installed
Fetched 711 kB in 0s (5324 kB/s)
Selecting previously unselected package lftp.
(Reading database ... 29960 files and directories currently installed.)
Preparing to unpack .../lftp_4.7.4-1_amd64.deb ...
Unpacking lftp (4.7.4-1) ...
Setting up lftp (4.7.4-1) ...
$ lftp -e "set ssl:verify-certificate no; open my-project.com; user $LFTP_USERNAME $LFTP_PASSWORD; mirror --reverse --verbose build/ /var/www/domains/dev/projects/my-project/build/; bye"
Transferring file `asset-manifest.json'
Transferring file `favicon.ico'
Transferring file `index.html'
Transferring file `manifest.json'
Transferring file `precache-manifest.be0a0d937dde3502acb8ad9c21f53a42.js'
Transferring file `service-worker.js'
Making directory `images'
Transferring file `images/content-block-arrow.png'
Transferring file `images/logo-mobile.png'
Transferring file `images/primary-nav-bg.png'
Transferring file `images/secondary-nav-bg.png'
Transferring file `images/social-footer.png'
Transferring file `images/social-media-facebook.png'
Transferring file `images/social-media-instagram.png'
Transferring file `images/social-media-twitter.png'
Transferring file `images/social-media-youtube.png'
Making directory `static'
Making directory `static/css'
Transferring file `static/css/2.e01eb902.chunk.css'
Transferring file `static/css/2.e01eb902.chunk.css.map'
Transferring file `static/css/main.a760ea4c.chunk.css'
Transferring file `static/css/main.a760ea4c.chunk.css.map'
Making directory `static/js'
Transferring file `static/js/2.1b305e72.chunk.js'
Transferring file `static/js/2.1b305e72.chunk.js.LICENSE.txt'
Transferring file `static/js/2.1b305e72.chunk.js.map'
mirror: Access failed: 552 10099096: No space left on device (2.1b305e72.chunk.js.map)
Transferring file `static/js/main.9394eb63.chunk.js'
Transferring file `static/js/main.9394eb63.chunk.js.map'
Transferring file `static/js/runtime-main.76c0dd92.js'
Transferring file `static/js/runtime-main.76c0dd92.js.map'
Making directory `static/media'
Transferring file `static/media/GothamBook.b31b3549.woff2'
Transferring file `static/media/GothamMedium.39e89370.woff2'
Transferring file `static/media/Plantin.0e23fafd.woff2'
Transferring file `static/media/logo.81af0ace.png'
Transferring file `static/media/revicons.04eb8fc5.woff'
Transferring file `static/media/revicons.17629a5d.ttf'
Transferring file `static/media/revicons.2feb69cc.eot'
ERROR: Job failed: exit code 1
du
which is ~60% and inodes are at 80% so I have no idea why I'm getting that error. I'm assuming it's an issue with LFTP ? I'm thinking this whole program might be connected to LFTP as well. My resolution may be to try another ftps program. In the mean time I'm using this yaml as it works – Jacksonkr Aug 8 '20 at 15:38