I'm in the process of converting multiple repositories into a single repository, our CI tool of choice is Jenkins due to the conversion of multiple repository structures into a single one 2 main issues have arisen.

  1. Build/test times have increased significantly as all the builds/tests have to be run for every single commit. This is partially alleviated by using a build tool, in our case we have gone with using Buck.

  2. After all the tests associated with the committed code are run, I have a deployment Jenkinsfile for each project. How will I be able to only trigger the Jenkinsfiles for projects that need to be re-deployed? And if I am able to do so, is this a correct practice?

  • How are your build hooks being implemented? Do you poll the repository with Jenkins? Do you have git hooks in place on each commit?
    – Preston Martin
    Commented Jun 22, 2018 at 13:49
  • We have githooks in place on each commit Commented Jun 25, 2018 at 1:38

2 Answers 2

  1. Get a list of changed files and use that to determine which tests to run.

  2. Load external Groovy scripts at run time in order to run your deploys.

  • 1
    Please copy the relevant bits from the link and post them inside a quote block in order to prevent that information would be lost if the link will be deprecated. Perhaps you could also add your personal view: what option would you choose and why?
    – 030
    Commented Jul 27, 2019 at 9:53
  • @030 I didn't provide two competing options. OP had two separate issues/questions labeled "1" and "2" so I gave one answer to each labeled "1" and "2". Furthermore each of those questions is duplicated by questions over at SO, which is what I linked to. I don't see the point in copy-pasting content from other SE sites since those questions and answers will be updated over time but my copy-paste will eventually become stale.
    – jayhendren
    Commented Feb 7, 2021 at 3:51

You can use the "when" block combined with the built in "changeset" condition to conditionally run only certain stages of your monorepo's pipeline. From the when.changeset documentation:

changeset- Executes the stage if the build’s SCM changeset contains one or more files matching the given string or glob. Example: when { changeset "**/*.js" }

Here is an example Jenkinsfile using this strategy:

pipeline {
    agent any
    stages {
        stage('build matchengine') {
            when {
                changeset "**/matchengine/*.*"
            steps {
                echo 'building match engine'
        stage('build posttrade') {
            when {
                changeset "**/posttrade/*.*"
            steps {
                echo 'building post trade'

, applicable to the monorepo project structure shown below:

   |-- Jenkinsfile
   |-- matchengine
   |-- posttrade
   |-- serverless
   |-- ui

This strategy won't scale past small codebases because it would be hard to keep track of which modules depend one each other. You would be better off using a build system like Bazel. Your CI job would simply issue a bazel build //... (build everything), and Bazel would calculate what actually needs to be built, and what needs to be tested. Further, there even exist bazel rules such as rules_docker and rules_k8s which can calculate which of your containers need to be rebuilt and pushed to a container registry, and which of your applications need to be redeployed to Kubernetes.

  • 2
    Unfortunatelly changeset doesn't contain all changes from change requests, only delta between last two commits. However custom check can be easily implemented.
    – Dima
    Commented Feb 16, 2020 at 9:25
  • the problem is if the build fails, the diff looks clean and a retry won't rebuild Commented Feb 23, 2021 at 20:55
  • yep I suggest Bazel in my answer, a build system built for this exact use case of only rebuilding / testing / deploying code which is affected by the commit
    – mancini0
    Commented Feb 24, 2021 at 14:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.