Best tool for Docker Container test automation?

Question

In my project i am building about 200 containers I want to check in a smoke test if the containers are still running after a rebuild in pipeline.

I want to bring up the container with certain parameters and check if it is responding on ports (service being up), or run simple test inside the container. Best would be to use docker-compose for starting containers, so that paramateres can easily be configured.

Example for a nginx container

1. run "nginx -t" inside the container to check if the container still has a valid configuration file
2. check if nginx is exposing services on port 80 and 443

Similar for other containers, main topic is checking ports for rest services. After bringing up the container(s), the tests should be run and then the whole thing shoudl be shutdown again.

What is the best tool for doing such a thing automated - and not writing everything manually?

Answer 1

Although we're deploying to Kubernetes, we've found that docker-compose is a nice way of running tests in CI. Either set the IS_TEST flag in your container or use a separate Dockerfile where tests scripts are wired up. That way you can just run docker-compose build to test, and pull in any dependencies you might need as well.

Answer 2

One could use serverspec to test whether a service like nginx is exposing ports, .e.g:

require 'spec_helper'

describe port(80) do
  it { should be_listening }
end

describe port(443) do
  it { should be_listening }
end

Once the serverspec code has been written one could create a script that starts the nginx, subsequently runs the serverspec and finally destroys the nginx.