Jenkins pipeline job cannot access Docker Engine

Question

I am learning Jenkins. I have created Jenkins container with docker and trying to run jenkins from there.

services:
  jenkins:
    image: jenkins/jenkins:lts
    container_name: jenkins
    restart: always
    ports:
      - "8080:8080"
      - "50000:50000"
    volumes:
      - jenkins_home:/var/jenkins_home
    networks:
      - jenkins

networks:
  jenkins:

volumes:
  jenkins_home:

I have also installed docker-pipline plugin to my Jenkins Container. I checked the restart api button after the plugin is installed.

My script for pipeline job is:

pipeline {
    agent any

    stages {
        stage('w/o docker') {
            steps {
                sh 'echo "Without docker"'
            }
        }
        stage('w docker') {
            agent{
                docker{
                    image 'node:18-alpine'
                }
            }
            steps {
                sh 'echo "With docker"'
            }
        }
    }
}

But when I build it I am getting the following error:

Started by user Fahad Md kamal
[Pipeline] Start of Pipeline
[Pipeline] node
Running on Jenkins in /var/jenkins_home/workspace/Docker build Pipeline
[Pipeline] {
[Pipeline] stage
[Pipeline] { (w/o docker)
[Pipeline] sh
+ echo Without docker
Without docker
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (w docker)
[Pipeline] node
Running on Jenkins in /var/jenkins_home/workspace/Docker build Pipeline@2
[Pipeline] {
[Pipeline] isUnix
[Pipeline] withEnv
[Pipeline] {
[Pipeline] sh
+ docker inspect -f . node:18-alpine
/var/jenkins_home/workspace/Docker build Pipeline@2@tmp/durable-5dab51a9/script.sh.copy: 1: docker: not found
[Pipeline] isUnix
[Pipeline] withEnv
[Pipeline] {
[Pipeline] sh
+ docker pull node:18-alpine
/var/jenkins_home/workspace/Docker build Pipeline@2@tmp/durable-74f03549/script.sh.copy: 1: docker: not found
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // node
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
ERROR: script returned exit code 127
Finished: FAILURE

I have no idea what I am doing wrong here, and also kind of getting frustrated with it since I am facing the problem for over weeks now. I would be glad if you could help me.

Answer 1

Your pipeline is executing a shell stage, which exits with 127 -- this indicates that the command or program you tried to execute in the shell was not found. In your case, this was docker -- i.e. your Jenkins agent is not finding the docker command in its environment.

It looks like you're using the Jenkins controller as the agent as well, so the agent itself is running inside the same container as the controller. In order to make this pipeline work, you would need to provide access to the Docker socket.

In short, you need to mount the socket into the Jenkins container by binding it as a volume, as you did with Jenkins home.

Furthermore, since the binary was not found, you will need to add it to the Jenkins container (i.e. you need to install docker inside the Jenkins container).

See this thread

However, I should point out that if your objective is to learn how to use or deploy Jenkins, this is the completely wrong way to do it.

First of all, running an agent on the controller means that jobs can break the controller. Secondly running Docker from inside Docker can lead to unexpected consequences, unless it's done right. See this thread Finally, it obviously won't scale, but I suspect this might not be an issue for you until later, if ever.

So, long story short I would:

1. Add a Docker "cloud" to your controller
2. Create an agent compatible with your Jenkins and build environment
3. Define your jobs to use that agent

All the details are on the Jenkins agent setup page: https://www.jenkins.io/doc/book/using/using-agents/