Testing Jenkins jobs and pipelines

Question

Currently, we have a fair number of Jenkins jobs and pipelines for builds, testing, deployments and other automated activities.

Every time we change or add a new job, we only test it manually - e. g. going over the "happy path" (when job is done with no errors), testing a couple of negative test cases when a job or pipeline fails - checking the error code and notifications.

This approach clearly is not reliable and does not scale well. How can we improve this process? Is there a place for test automation when it comes to checking how Jenkins jobs and pipelines work?

Answer 1

I'm posting this here not because I endorse these solutions (in fact, I've never tried them), but just because they are a potential answer to your question:

You can start with JenkinsPipelineUnit, a unit testing framework for Pipeline scripts.

There is also a project called jenkinsfile-runner which executes your Jenkinsfile in a transient, headless Jenkins instance. Supposedly, this can be used for integration testing Jenkinsfiles and Pipeline shared libraries. However, as of summer 2018 there is no documentation on how to use this tool for integration testing, and I have been unable to find any examples of anyone using this tool "in the wild".

See also the related bug report on the Jenkins JIRA: "Test framework for Jenkinsfile".