Currently, we have a fair number of Jenkins jobs and pipelines for builds, testing, deployments and other automated activities.

Every time we change or add a new job, we only test it manually - e. g. going over the "happy path" (when job is done with no errors), testing a couple of negative test cases when a job or pipeline fails - checking the error code and notifications.

This approach clearly is not reliable and does not scale well. How can we improve this process? Is there a place for test automation when it comes to checking how Jenkins jobs and pipelines work?

  • 5
    welcome to high-level devops - who tests the test? I think, you could start with a sandbox environment to replicate general system settings and evolve it to complete Configuration as Code appoach; this includes code-maintained job configurations.
    – Ta Mu
    Aug 28, 2017 at 17:04

1 Answer 1


I'm posting this here not because I endorse these solutions (in fact, I've never tried them), but just because they are a potential answer to your question:

You can start with JenkinsPipelineUnit, a unit testing framework for Pipeline scripts.

There is also a project called jenkinsfile-runner which executes your Jenkinsfile in a transient, headless Jenkins instance. Supposedly, this can be used for integration testing Jenkinsfiles and Pipeline shared libraries. However, as of summer 2018 there is no documentation on how to use this tool for integration testing, and I have been unable to find any examples of anyone using this tool "in the wild".

See also the related bug report on the Jenkins JIRA: "Test framework for Jenkinsfile".

  • I have been using jenkinsfile-runner to test my opensource jenkins shared library github.com/DontShaveTheYak/jenkins-std-lib The jobs I use to test the library are in jobs/ and the JFR image is in docker/jfr. I actually write my tests and execute them with pytest.
    – Levi
    Apr 22, 2022 at 1:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.