Is Docker right for my use case?

Question

My company has a system we sell that consists of basically a mini-computer "Smartbox" that is running Ubuntu 12.04. This box runs a Django application plus a number of different upstart processes related to it. Not much else. We have thousands of these boxes out on the field. We manage the package dependencies, process registration, etc. through a deb package with varying degrees of success.

We need a way to efficiently and robustly push updates out to our users out in the field. We also need something that as we upgrade the OS (we are way overdue for a Ubuntu upgrade as you can tell) we can feel relatively secure about our packages "just working".

I don't know much about Docker, but when I first heard about our problem (I'm a new hire), Docker was my first thought. But the more I thought about it I felt like maybe it wasn't, as these boxes are ours we control the OS on it which is a big part of the value proposition of Docker, or so I understand. So if we KNOW our boxes will always be Ubuntu and we basically just have a Django app plus some processes to run, is Docker any better than a deb package?

TL;DR: Docker vs deb packages for a distributed appliance that will always run Ubuntu so platform independence is not that important.

Answer 1

I'm not 100% sure I understand from the question, but it sounds like the Docker solution would be to go from having an (physical?) appliance with an OS and your app installed on it, to having an appliance with an OS and Docker on it, running a single container with your app in it. That doesn't obviate the need to update the OS in the host, and it adds a layer of complexity (and more updates to contend with, as you'll now have to keep Docker and the OS patched) with no readily-apparent benefit as far as the specific areas mentioned in the question are concerned.

However, if you're talking about going from a virtual appliance to a Docker container, that could potentially smooth things out for you, but it also adds Docker as a dependency for your product; you're shutting out anyone who isn't using Docker and doesn't want to add it to their stack just to use your product. You could continue to support those that don't/won't use Docker by continuing to ship the (now "legacy") virtual appliance as before, but now you've just doubled your workload because you have two distributions to support instead of one.

Answer 2

I've worked with Docker a long time. The platform independence is nice, but it's not what I consider most useful about Docker.

First and foremost, you get repeatability. You can create a Dockerfile, debug in a container on your developer machine, run tests on a continuous integration server, and then in your final product, and you know it will behave the same in all those environments. No forgetting a dependency that a developer had installed on their machine. Also, your developers don't have to use Ubuntu at their desk. Important to keep us Arch Linux users happy :-)

Second, for your upgrade scenario, you can have multiple versions pulled to a machine at once. If you do a docker pull myapp:2.0 while 1.0 is running, you can swap to 2.0 extremely quickly. Much faster than doing a full OS upgrade would normally take. If you use an orchestrator with multiple instances of microservices, you can even do rolling upgrades that don't interrupt service at all.

If you use a microservices model, Docker also provides sandboxes that can limit the amount of damage attackers can do in the event of an exploit. Instead of gaining control over an entire machine, they are only gaining control over one container.

The main downside is you need the host OS and some sort of orchestration. There are lots of choices for that, but don't underestimate the amount of work it takes to evaluate one, put it in place, and maintain it.

Answer 3

By and large docker provides many advantages for both developers and operations personnel. I use docker for some of my applications and find it to be a very reliable and robust approach.

My issue with you adopting docker is that I do not hear you asking the right questions and you could potentially make your life more complicated without addressing your most important requirements.

The first question you should ask (you said that you were new) is how are updates to the OS & application handled now? Does the current methodology work for you (your company)? What works well? What could be improved? Can you do a physical configuration audit on your target machines in the field to verify that they have the correct OS patches, application & were there any unauthorized changes.

I love docker, but I would not jump on the docker bandwagon without first assessing where you are right now including what works well and what needs to be improved.

Answer 4

While there is some tiny overlapping region Docker and the Debian packaging systems essentially solve two very different problems:

• The Debian packaging system is built to install software on a host and upgrade it as easily as possible. It is able to handle complex dependency and constraint patterns between software components, like “software X version A requires software Y with version B or newer installed” or “software X should never be installed with software Z version C”.

• The Docker system is conceived to easily describe and deploy services, especially micro-services, possibly on several hosts – e.g. a Docker swarm or a Kubernetes cluster.

These two problems are essentially orthogonal, which means that given the deployment problem to solve, one can use one of them, both of them, or maybe even none of them, as part of the solution. When using both of them, the Debian package is used in the production of the Docker image, and your Dockerfile (the recipes used to prepare the Docker image describing the “virtualised system” ran in a container) would essentially register your Debian repository in the sources of the Debian packaging system and install your package.

With this in mind, it seems to me that what you are really looking for is to implement the immutable server pattern. The recent development in cloud technologies made possible to upgrade software not by using the classical software upgrade system from a software package system (such as the Debian packaging system) but rather by simply replacing the full server at once. (Some persons did this before this development by having three OS-s on a server, two used in alternance to run the appliance and a mini-OS dedicated to performing the appliance replacement. While not overly complex, this seems to have always remained a niche.) This technique can be of interest for you because if you are used to upgrade software on your server using the package manager, the final state of the server depends of the “upgrade history” of the server – especially if errors occur in the upgrade process. This heterogeneity is bad, because it makes production problems hard to reproduce and diagnose, and your mixed experience

We have thousands of these boxes out on the field. We manage the package dependencies, process registration, etc. through a deb package with varying degrees of success.

could relate to this. The immutable server pattern wipes this source of errors by essentially destroying the notion of “upgrade history” from the problem.

Now there are various options to implement the immutable server pattern, two popular choices are to use Docker images, images or to use “master instance images” from your cloud provider (these are called AMIs in AWS and just Custom Images in Google Compute Engine). Your use-case forbids the use of cloud based techniques, I will therefore assume Docker images as the only eligible choice. (For the sake of completion, it is certainly possible to use other approaches, for instance using Virtual Box or similar virtualisation solution, as an alternative to Docker.)

When using the immutable server pattern technique, you introduce a new artefact (the Docker image) representing your server and this artefact can be tested as well, and it is very easy to obtain a setup replicating truthfully your production settings – aside from service load.

Now to consider the concrete problem you described, let's assume implementing the immutable server pattern with Docker is actually what you want. Since the Docker system and the Debian packaging system are complementary rather than mutually exclusive (cf. intro) we still have to address the question if you should you prepare a Debian package for your software.

The pertinence of using a Debian package to install your software (in the Docker image or on a host) lies in the complexity of the versioning problem you have to solve. If you run at the same time several versions of your software, occasionally need to downgrade, and have complex version requirements that you need to carefully document, having a Debian package is a must-be. Otherwise, this step can be skipped – but since you already put an effort to produce and deploy these packages, there is no real value into ditching your work. I would therefore suggest continue to produce your Debian packages.

Answer 5

I think it could be a good option (further tests are necessary)

You could provide a URL with all tags/version of the container you have made and the clients will read that URL to see if there is a new version of the container.

You can store personal files/settings on local and you will never lost that information in upgrades and you will ensure that what you have made and tested will work for everyone in the same way.

Even you could give the users the possibility to choose which version from the available they want to use (if you want to give that possibility).

It will be like ""only upgrade one package"", talking about only retrieve a new version of the container, much better that dealing with debian packages ;)

Answer 6

Docker sounds reasonable to me, as you could make and test changes to the container in house, and then depending on your release process, restart the containers always pulling :latest or something similar which would provide a tested upgrade.

Considerations you would need to deal with include data storage as containers don't retain changes on restart, so you would want a data volume. There are probably a lot more considerations you would have as well once you dig into it. The system that I am currently working with (all docker-based) has been in development for over a year now and we are still finding areas in which we need to make changes to the container, configurations, etc.