I'm in the process of converting multiple repositories into a single repository, our CI tool of choice is Jenkins due to the conversion of multiple repository structures into a single one 2 main issues have arisen.

  1. Build/test times have increased significantly as all the builds/tests have to be run for every single commit. This is partially alleviated by using a build tool, in our case we have gone with using Buck.

  2. After all the tests associated with the committed code are run, I have a deployment Jenkinsfile for each project. How will I be able to only trigger the Jenkinsfiles for projects that need to be re-deployed? And if I am able to do so, is this a correct practice?

  • How are your build hooks being implemented? Do you poll the repository with Jenkins? Do you have git hooks in place on each commit? – Preston Martin Jun 22 '18 at 13:49
  • We have githooks in place on each commit – YellowPillow Jun 25 '18 at 1:38
  1. Get a list of changed files and use that to determine which tests to run.

  2. Load external Groovy scripts at run time in order to run your deploys.

  • 1
    Please copy the relevant bits from the link and post them inside a quote block in order to prevent that information would be lost if the link will be deprecated. Perhaps you could also add your personal view: what option would you choose and why? – 030 Jul 27 '19 at 9:53

You can use the "when" block combined with the built in "changeset" condition to conditionally run only certain stages of your monorepo's pipeline. From the when.changeset documentation:

changeset- Executes the stage if the build’s SCM changeset contains one or more files matching the given string or glob. Example: when { changeset "**/*.js" }

Here is an example Jenkinsfile using this strategy:

pipeline {
    agent any
    stages {
        stage('build matchengine') {
            when {
                changeset "**/matchengine/*.*"
            steps {
                echo 'building match engine'
        stage('build posttrade') {
            when {
                changeset "**/posttrade/*.*"
            steps {
                echo 'building post trade'

, applicable to the monorepo project structure shown below:

   |-- Jenkinsfile
   |-- matchengine
   |-- posttrade
   |-- serverless
   |-- ui

This strategy won't scale past small codebases because it would be hard to keep track of which modules depend one each other. You would be better off using a build system like Bazel. Your CI job would simply issue a bazel build //... (build everything), and Bazel would calculate what actually needs to be built, and what needs to be tested. Further, there even exist bazel rules such as rules_docker and rules_k8s which can calculate which of your containers need to be rebuilt and pushed to a container registry, and which of your applications need to be redeployed to Kubernetes.

  • 1
    Unfortunatelly changeset doesn't contain all changes from change requests, only delta between last two commits. However custom check can be easily implemented. – mangolier Feb 16 '20 at 9:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.