how to decrease the docker image layers latency during the image build step?

Question

I am following the Microsoft document to create a custom image of vsts-agent on top of ubuntu and install the required capabilities. Since our environment is not having internet enabled, we used certain debian packages to install the capabilities and which resulted in multiple tasks in Docker file such as"COPY,"ENV setup for different apps,

Since we have these much steps, each is getting created in separate layer and which is causing vey slow image build.

FROM ubuntu:18.04
#2-Enable Ubuntu Packages
COPY ./sources.list /etc/apt/
#3- Install basic Softwares
ENV DEBIAN_FRONTEND=noninteractive
RUN echo "APT::Get::Assume-Yes \"true\";" > /etc/apt/apt.conf.d/90assumeyes
RUN apt-get update && apt-get install -y --no-install-recommends \
    ca-certificates \
    curl \
    wget \
    jq \
    git \
    iputils-ping \
    libcurl4 \
    libicu60 \
    libunwind8 \
    netcat \
    telnet \
    libssl1.0 \
    python \
    python3 \
    nodejs \
    python3-setuptools \
    python3-pip
    vim \
    openjdk-11-jdk-headless \
    gnupg \
    make \
    yarn\
    apt-transport-https \
    lsb-release \
  && rm -rf /var/lib/apt/lists/*
    #4. Copy Offline Packages
    COPY ./sw/* /tmp/
    #4-Install AzureCLI
    RUN  curl -sL https://packages.microsoft.com/keys/microsoft.asc | gpg --dearmor | tee /etc/apt/trusted.gpg.d/microsoft.gpg > /dev/null
    RUN AZ_REPO=$(lsb_release -cs) \
      && echo "deb [arch=amd64] https://packages.microsoft.com/repos/azure-cli/ $AZ_REPO main" | tee /etc/apt/sources.list.d/azure-cli.list \
      && apt-get update \
      &&  apt-get install azure-cli  
    #Installl helm,docker, googlechrome and kubectl
    RUN  tar zxvf /tmp/helm-v3.8.2-linux-amd64.tar.gz  && mv /linux-amd64/helm /usr/local/bin/ && mv /tmp/kubectl /usr/local/bin/ \
         &&  apt install /tmp/google-chrome-stable_current_amd64.deb \
         &&  apt install /tmp/containerd.io_1.6.9-1_amd64.deb \
         &&  apt install /tmp/docker-ce-rootless-extras_20.10.9_3-0_ubuntu-bionic_amd64.deb \
         &&  apt install /tmp/libslirp0_4.6.1-1build1_amd64.deb      \
         &&  apt install /tmp/docker-ce_20.10.9_3-0_ubuntu-bionic_amd64.deb \
         &&  apt install /tmp/docker-compose-plugin_2.6.0_ubuntu-bionic_amd64.deb \
         &&  apt install /tmp/pigz_2.6-1_amd64.deb \
         &&  apt install /tmp/docker-ce-cli_20.10.9_3-0_ubuntu-bionic_amd64.deb \
         &&  apt install /tmp/docker-scan-plugin_0.9.0_ubuntu-bionic_amd64.deb \
         &&  apt install /tmp/slirp4netns_1.0.1-2_amd64.deb
    #8-install maven 3.8.6
    RUN mkdir -p /usr/share/maven /usr/share/maven/ref \
      && tar -xzf /tmp/apache-maven-3.8.6-bin.tar.gz -C /usr/share/maven --strip-components=1 \
      && echo "Cleaning and setting links" \
      && rm -f /tmp/apache-maven.tar.gz \
      && ln -s /usr/share/maven/bin/mvn /usr/bin/mvn
        # 8.1- Define environmental variables required by Maven, like Maven_Home directory and where the maven repo is located
        ENV MAVEN_HOME /usr/share/maven \
            TARGETARCH=linux-x64 \
            MAVEN_CONFIG "$USER_HOME_DIR/.m2" \
            JAVA_HOME /usr/lib/jvm/java-11-openjdk-amd64
        #11- Agent Installation
        WORKDIR /azp
        COPY ./vstsagent/ .
        COPY ./start.sh .
        COPY ./docker.sh .
        RUN chmod +x start.sh docker.sh
        ENV TARGETARCH=linux-x64
        # Can be 'linux-x64', 'linux-arm64', 'linux-arm', 'rhel.6-x64'.
        ENV TARGETARCH=linux-x64
        ENV JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64
        ENV JAVA_HOME_11_X64=/usr/lib/jvm/java-11-openjdk-amd64
        ENV MAVEN_HOME=/usr/share/maven
        COPY policy-rc.d /usr/sbin/policy-rc.d
        RUN dpkg -i /tmp/docker-ce_20.10.9_3-0_ubuntu-bionic_amd64.deb
        COPY ./extensions/ /root/.azure/
        RUN dpkg -i /tmp/google-chrome-stable_current_amd64.deb
        RUN apt install -y --fix-broken
        COPY ./sw/zip/* /tmp/zip/
        WORKDIR /tmp/zip/
        RUN dpkg -i zip_3.0-11_amd64.deb unzip_6.0-21ubuntu1_amd64.deb
        WORKDIR /azp/_work
        COPY ./tools .
        RUN mkdir /root/.m2
        WORKDIR /root/.m2
        COPY ./settings.xml .
        WORKDIR /root
        COPY ./.npmrc .
        COPY ./jf .
        COPY ./config /root/.kube/
        COPY ./docker-compose /usr/local/bin
        COPY ./daemon.json /etc/docker/
        RUN sed -i 's/"$@"/"$@" --no-sandbox/' /opt/google/chrome/google-chrome
        ENV JFROG_CLI_HOME /root
        ENV TESTCONTAINERS_RYUK_DISABLED=true
        WORKDIR /azp
        CMD ["./docker.sh"]
        ENTRYPOINT ["./start.sh"]

So in the below sample, can we segregate repeating tasks to one layer so that the imagebuild step will be fast ?

Also is it allowed to start the vsts-agent without root user? if yes, how I can achieve the same using normal user?

Ralle Mc Black · Accepted Answer · 2023-01-01 15:41:37Z

First build an image with all the dependencie that want change and are not related to your code.

FROM ubuntu:18.04
#2-Enable Ubuntu Packages
COPY ./sources.list /etc/apt/
#3- Install basic Softwares
ENV DEBIAN_FRONTEND=noninteractive
RUN echo "APT::Get::Assume-Yes \"true\";" > /etc/apt/apt.conf.d/90assumeyes
RUN apt-get update && apt-get install -y --no-install-recommends \
    ca-certificates \
    curl \
    wget \
    jq \
    git \
    iputils-ping \
    libcurl4 \
    libicu60 \
    libunwind8 \
    netcat \
    telnet \
    libssl1.0 \
    python \
    python3 \
    nodejs \
    python3-setuptools \
    python3-pip
    vim \
    openjdk-11-jdk-headless \
    gnupg \
    make \
    yarn\
    apt-transport-https \
    lsb-release \
  && rm -rf /var/lib/apt/lists/*
    #4. Copy Offline Packages
    COPY ./sw/* /tmp/
    #4-Install AzureCLI
    RUN  curl -sL https://packages.microsoft.com/keys/microsoft.asc | gpg --dearmor | tee /etc/apt/trusted.gpg.d/microsoft.gpg > /dev/null
    RUN AZ_REPO=$(lsb_release -cs) \
      && echo "deb [arch=amd64] https://packages.microsoft.com/repos/azure-cli/ $AZ_REPO main" | tee /etc/apt/sources.list.d/azure-cli.list \
      && apt-get update \
      &&  apt-get install azure-cli  
    #Installl helm,docker, googlechrome and kubectl
    RUN  tar zxvf /tmp/helm-v3.8.2-linux-amd64.tar.gz  && mv /linux-amd64/helm /usr/local/bin/ && mv /tmp/kubectl /usr/local/bin/ \
         &&  apt install /tmp/google-chrome-stable_current_amd64.deb \
         &&  apt install /tmp/containerd.io_1.6.9-1_amd64.deb \
         &&  apt install /tmp/docker-ce-rootless-extras_20.10.9_3-0_ubuntu-bionic_amd64.deb \
         &&  apt install /tmp/libslirp0_4.6.1-1build1_amd64.deb      \
         &&  apt install /tmp/docker-ce_20.10.9_3-0_ubuntu-bionic_amd64.deb \
         &&  apt install /tmp/docker-compose-plugin_2.6.0_ubuntu-bionic_amd64.deb \
         &&  apt install /tmp/pigz_2.6-1_amd64.deb \
         &&  apt install /tmp/docker-ce-cli_20.10.9_3-0_ubuntu-bionic_amd64.deb \
         &&  apt install /tmp/docker-scan-plugin_0.9.0_ubuntu-bionic_amd64.deb \
         &&  apt install /tmp/slirp4netns_1.0.1-2_amd64.deb
    #8-install maven 3.8.6
    RUN mkdir -p /usr/share/maven /usr/share/maven/ref \
      && tar -xzf /tmp/apache-maven-3.8.6-bin.tar.gz -C /usr/share/maven --strip-components=1 \
      && echo "Cleaning and setting links" \
      && rm -f /tmp/apache-maven.tar.gz \
      && ln -s /usr/share/maven/bin/mvn /usr/bin/mvn
        # 8.1- Define environmental variables required by Maven, like Maven_Home directory and where the maven repo is located
        ENV MAVEN_HOME /usr/share/maven \
            TARGETARCH=linux-x64 \
            MAVEN_CONFIG "$USER_HOME_DIR/.m2" \
            JAVA_HOME /usr/lib/jvm/java-11-openjdk-amd64
        #11- Agent Installation

Build that image as your base image and push it in a dockerregistry.

docker build -t mybaseimage:latest .
docker image tag mybaseimage:latest registry-domain/mypath/mybaseimage:latest

docker image push registry-domain/mypath/mybaseimage:latest

Now your mybaseimage image is in the registry and you can use it and rebuild your image with your code from here.

FROM mybaseimage

       WORKDIR /azp
        COPY ./vstsagent/ .
        COPY ./start.sh .
        COPY ./docker.sh .
        RUN chmod +x start.sh docker.sh
        ENV TARGETARCH=linux-x64
        # Can be 'linux-x64', 'linux-arm64', 'linux-arm', 'rhel.6-x64'.
        ENV TARGETARCH=linux-x64
        ENV JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64
        ENV JAVA_HOME_11_X64=/usr/lib/jvm/java-11-openjdk-amd64
        ENV MAVEN_HOME=/usr/share/maven
        COPY policy-rc.d /usr/sbin/policy-rc.d
        RUN dpkg -i /tmp/docker-ce_20.10.9_3-0_ubuntu-bionic_amd64.deb
        COPY ./extensions/ /root/.azure/
        RUN dpkg -i /tmp/google-chrome-stable_current_amd64.deb
        RUN apt install -y --fix-broken
        COPY ./sw/zip/* /tmp/zip/
        WORKDIR /tmp/zip/
        RUN dpkg -i zip_3.0-11_amd64.deb unzip_6.0-21ubuntu1_amd64.deb
        WORKDIR /azp/_work
        COPY ./tools .
        RUN mkdir /root/.m2
        WORKDIR /root/.m2
        COPY ./settings.xml .
        WORKDIR /root
        COPY ./.npmrc .
        COPY ./jf .
        COPY ./config /root/.kube/
        COPY ./docker-compose /usr/local/bin
        COPY ./daemon.json /etc/docker/
        RUN sed -i 's/"$@"/"$@" --no-sandbox/' /opt/google/chrome/google-chrome
        ENV JFROG_CLI_HOME /root
        ENV TESTCONTAINERS_RYUK_DISABLED=true
        WORKDIR /azp
        CMD ["./docker.sh"]
        ENTRYPOINT ["./start.sh"]

Note: registry-domain/mypath/ could be docker.io/myaccountname or your private registry if you have one.

But I guess still the image build will happen layer per each task.. So better I am looking for some way, where I can Group all the ENV commands together with "&&" and similarly COP and RUN commands together wherever possible, so that in one layer all will be executed — Vowneee, Jan 1 at 18:05
Look also if there is a .dockerignore file and if here are files or folders specified that you wont to have rebuild. What ever doesnt change you can put it in the base image or before and between everything that can change. So it should not be rebuild. — Ralle Mc Black, Jan 1 at 18:10

Stack Exchange Network

how to decrease the docker image layers latency during the image build step?

1 Answer 1

Your Answer

Not the answer you're looking for? Browse other questions tagged
docker
docker-compose
dockerfile
dockerhub
or ask your own question.

Hot Network Questions

how to decrease the docker image layers latency during the image build step?

1 Answer 1

Your Answer

Sign up or log in

Post as a guest

Not the answer you're looking for? Browse other questions tagged dockerdocker-composedockerfiledockerhub or ask your own question.

Related

Hot Network Questions

Not the answer you're looking for? Browse other questions tagged
docker
docker-compose
dockerfile
dockerhub
or ask your own question.