Passing variables between scripts in a Jenkins pipeline

Question

I have a declarative Jenkins pipeline that looks like the following:

pipeline{
    agent {
        ...
    }
    stages {
        stage("Provision EC2 Instance and Set ENV Variable"){
            steps {
                dir("docker-image-builder"){
                    sh 'python ./provisonInstance.py'
                }
            }
        }
        stage("Configure EC2 Instance with Ansible"){
            steps {
                ansiblePlaybook('./ansible.playbook.yml') {
                    ansibleName('1.9.4')
                }
            }
        }
    }
}

In my first stage, I have a Python script that provisions an Amazon EC2 instance that will be used to create an AMI. In the second stage, I will then configure configure my instance before finally creating an AMI and terminating the instance. In order to do this, I need the instance-id of the EC2 instance created in the first stage. Is there an elegant way to pass values generated by a script between stages, and if so how?

The most obvious method that I can think of is to write the value to a file on the agent in the first stage, and read it in the second stage, but this does not seem like an elegant solution.

Answer 1

For Declarative Pipeline:

This question already has a highly-upvoted answer on StackOverflow:

If you want to use a file (since a script is the thing generating the value you need):

pipeline {
  agent { label 'docker' }
  stages {
    stage('one') {
      steps {
        sh 'echo hotness > myfile.txt'
        script {
          // trim removes leading and trailing whitespace from the string
          myVar = readFile('myfile.txt').trim()
        }
        echo "${myVar}" // prints 'hotness'
      }
    }
    stage('two') {
      steps {
        echo "${myVar}" // prints 'hotness'
      }
    }
    // this stage is skipped due to the when expression, so nothing is printed
    stage('three') {
      when {
        expression { myVar != 'hotness' }
      }
      steps {
        echo "three: ${myVar}"
      }
    }
  }
}

A second solution would be to write out output from your script directly or with writeFile and then later read it back in with readFile. However, this comes with the significant caveat that both the reading and writing must take place on the same node, so you can't use this to send data across nodes.

You can also can archive files centrally with archiveArtifact and retrieve them with copyArtifact (and then read in the file with readFile).

Finally, there are some answers to a near-identical question on StackOverflow that offer alternative solutions.

---

However, if you can, I would suggest using Scripted Pipeline instead, which allows a much simpler solution:

def String myVar

stage('my-first-stage') {
  myVar = sh(script: 'my-command', returnStdout: true)
}

stage('my-second-stage') {
  sh("my-other-command --var='${myVar}'")
}

(I also already answered this question for Scripted Pipeline over on ServerFault.)

---

It looks like some of these solutions have already been proposed in the comments, but you are concerned that they are not elegant enough. I'm not really sure what kind of elegance you're looking for, because I can't think of any way to simplify these answers in Jenkins-Pipeline-land.

Answer 2

There is a simple solution for declarative pipeline which doesn't involve having to write to files - define the variable-to-be-shared outside the pipeline block. This is much neater and means you should still be able to restart the job from the later stage if you need.

Example:

def variable = "initial value"
pipeline {
    agent {
        ...
    }
    stages {
        stage("Override variable") {
            steps {
                script {
                    variable = sh(script: 'some command', returnStdout: true).trim()
                }
            }
        }
        stage("Read new value of variable") {
            steps {
                script {
                    echo variable
                }
            }
        }
    }
}

Further explanation in video form