Transfer files and execute commands from jenkins scripted pipeline to AWS instance

Question

I want to transfer files and execute a few commands on remote AWS instance from Jenkins. I have the following methods in mind to do the same. Note that my Jenkins server is running in the local server, and I don't have SSH access to Jenkins

1) Using hardcoded pem file I have following scripted pipeline to transfer files from Jenkins pipeline to AWS EC2 instance.

stage('transfer docker image to ec2') {
  sh 'scp -oIdentityFile=/var/lib/jenkins/.key.pem docker_image.tar [email protected]:'

However it is not working, and waiting for a long time without showing logs.

2) Using Jenkins "credential"

I have added the pem file in Jenkins credential, and then invoked the credential by the following method. (Referred from - https://stackoverflow.com/a/47627460/9403545)

withCredentials([sshUserPrivateKey(credentialsId: "bindu-test", keyFileVariable: 'keyfile')]) {
 stage('transfer docker image to ec2') {
  sh 'scp -i ${keyfile} test [email protected]:'

Unfortunately, the above method also didn't work and got the following error.

/tmp/../Hello directory doesn't exist.

I have tried to use Publish over SSH, but it doesn't seem to be supported in scripted pipeline.

Is there any way I can do this? P.S: Please keep in mind that I'm AWS newbie, it would be nice if the answer is in detail :)

Answer 1

Below is sample groovy code for sending file or execute command over SSH in jenkins:

node(master)
{
  stage(Deploy)
  {
     sshPublisher(publishers: [sshPublisherDesc(configName: 'SERVER_NAME', transfers: [sshTransfer(cleanRemote: false, excludes: '', execCommand: 'apt-get update', execTimeout: 120000, flatten: false, makeEmptyDirs: false, noDefaultExcludes: false, patternSeparator: '[, ]+', remoteDirectory: '', remoteDirectorySDF: false, removePrefix: '', sourceFiles: '*.war')], usePromotionTimestamp: false, useWorkspaceInPromotion: false, verbose: false)])
  }
}

Answer 2

You can use 'Publish Over SSH plugin'. Using this plugin you can send files and execute the command on the remote server.

Click Here For install plugin.

Configure

1. Click “Manage Jenkins”
2. Click “Configure System”
3. Go to “Publish over SSH” section
4. Enter “/Users/Shared/Jenkins/.ssh/id_rsa” to “Path to Key”
5. Click “Add” at “SSH Servers”
6. Enter any logical name to “Name”
7. Enter IP Address or Hostname of the server to “Hostname”
8. Enter the user name to log in to “Username”
9. Enter any directory to “Remote Directory”
10. Click “Test Configuration”
11. Click “Save” at bottom of the page

Create a job

1. Create or copy a job that can build the WAR correctly.
2. As I wrote in How to specify a Git tag to be processed, make a job to can specify a tag to be processed.
3. Click “Add post-build action”
4. Click “Send build artifacts over SSH”
5. Enter “Source files”
6. Enter “Remove prefix”
7. Enter “Exec command”

For More reference:

1. https://nozaki.me/roller/kyle/entry/articles-jenkins-sshdeploy
2. https://medium.com/@weblab_tech/how-to-publish-artifacts-in-jenkins-f021b17fde71
3. https://wiki.jenkins.io/display/JENKINS/Publish+Over+SSH+Plugin

you can also do the same thing in pipeline script, see below images:

node(master)
{
  stage(Deploy)
  {
     sshPublisher(publishers: [sshPublisherDesc(configName: 'LAB-35', transfers: [sshTransfer(cleanRemote: false, excludes: '', execCommand: 'apt-get update', execTimeout: 120000, flatten: false, makeEmptyDirs: false, noDefaultExcludes: false, patternSeparator: '[, ]+', remoteDirectory: '', remoteDirectorySDF: false, removePrefix: '', sourceFiles: '*.war')], usePromotionTimestamp: false, useWorkspaceInPromotion: false, verbose: false)])
  }
}

Here configName: 'LAB-35' is your remote ssh server.