I have a gitlab repository with a scheduled regular CI job on a gitlab runner that grabs a data file from the internet, does some analysis, and then writes the results to its artifacts so that it becomes a gitlab pages site. Hence when the file changes, the pages change to reflect the updated analysis.
What I'd like to do is not re-run the analysis if the data file is unchanged, but I'm not sure how to do this cleanly...
The CI job stores a copy of the data file to its artifacts and then that is available on the gitlab pages site. Hence the next CI job can compare the latest data file on the internet with the one used for the last analysis, and if its changed it can do the analysis, update the artifacts, and exit 0.
If the file hasn't changed since the last analysis the script can stop and exit cleanly, but if it doesn't put anything in the artifacts, the resulting gitlab pages will be empty. The only fix to this I can see is for the CI job to copy everything from the current pages to its own artifacts so it all appears again.
Or the CI script can fail with an error in which case the pages doesn't get updated, but I do get a "pipeline fail" error which might mask genuine pipeline fails.
Ideally I'd like to end the script so that Gitlab CI thinks that the job has completed successfully, but there's no need to update the gitlab pages. Is that possible?
I can think of other methods such as having a script run by cron on another machine, and doing the "has the data file changed?" test there, and not triggering the CI job if its not needed. But at the moment everything including the hourly scheduled run is done on gitlab and our runner. Which is nice.
.gitlab-ci.yml is pretty straightforward
pages file like this:
pages: stage: deploy script: - mkdir .public - cp -r static/* .public - mv .public public - ./build.sh artifacts: paths: - public only: - master