14

My company has a system we sell that consists of basically a mini-computer "Smartbox" that is running Ubuntu 12.04. This box runs a Django application plus a number of different upstart processes related to it. Not much else. We have thousands of these boxes out on the field. We manage the package dependencies, process registration, etc. through a deb package with varying degrees of success.

We need a way to efficiently and robustly push updates out to our users out in the field. We also need something that as we upgrade the OS (we are way overdue for a Ubuntu upgrade as you can tell) we can feel relatively secure about our packages "just working".

I don't know much about Docker, but when I first heard about our problem (I'm a new hire), Docker was my first thought. But the more I thought about it I felt like maybe it wasn't, as these boxes are ours we control the OS on it which is a big part of the value proposition of Docker, or so I understand. So if we KNOW our boxes will always be Ubuntu and we basically just have a Django app plus some processes to run, is Docker any better than a deb package?

TL;DR: Docker vs deb packages for a distributed appliance that will always run Ubuntu so platform independence is not that important.

  • 3
    Congrats for your first question, nicely written and with a practical goal, an example one :) – Tensibai May 16 '17 at 19:54
7

I'm not 100% sure I understand from the question, but it sounds like the Docker solution would be to go from having an (physical?) appliance with an OS and your app installed on it, to having an appliance with an OS and Docker on it, running a single container with your app in it. That doesn't obviate the need to update the OS in the host, and it adds a layer of complexity (and more updates to contend with, as you'll now have to keep Docker and the OS patched) with no readily-apparent benefit as far as the specific areas mentioned in the question are concerned.

However, if you're talking about going from a virtual appliance to a Docker container, that could potentially smooth things out for you, but it also adds Docker as a dependency for your product; you're shutting out anyone who isn't using Docker and doesn't want to add it to their stack just to use your product. You could continue to support those that don't/won't use Docker by continuing to ship the (now "legacy") virtual appliance as before, but now you've just doubled your workload because you have two distributions to support instead of one.

5

I've worked with Docker a long time. The platform independence is nice, but it's not what I consider most useful about Docker.

First and foremost, you get repeatability. You can create a Dockerfile, debug in a container on your developer machine, run tests on a continuous integration server, and then in your final product, and you know it will behave the same in all those environments. No forgetting a dependency that a developer had installed on their machine. Also, your developers don't have to use Ubuntu at their desk. Important to keep us Arch Linux users happy :-)

Second, for your upgrade scenario, you can have multiple versions pulled to a machine at once. If you do a docker pull myapp:2.0 while 1.0 is running, you can swap to 2.0 extremely quickly. Much faster than doing a full OS upgrade would normally take. If you use an orchestrator with multiple instances of microservices, you can even do rolling upgrades that don't interrupt service at all.

If you use a microservices model, Docker also provides sandboxes that can limit the amount of damage attackers can do in the event of an exploit. Instead of gaining control over an entire machine, they are only gaining control over one container.

The main downside is you need the host OS and some sort of orchestration. There are lots of choices for that, but don't underestimate the amount of work it takes to evaluate one, put it in place, and maintain it.

  • What does any of this have to do with what OP was asking? – Adrian Jun 5 '17 at 12:00
  • 1
    (Off-topic comment.) Hello Karl, I enjoyed a lot many of your contributions to Programmers/Software-Engineering, it's a pleasure to see you here as well! – Michael Le Barbier Grünewald Aug 28 '17 at 8:13
1

While there is some tiny overlapping region Docker and the Debian packaging systems essentially solve two very different problems:

  • The Debian packaging system is built to install software on a host and upgrade it as easily as possible. It is able to handle complex dependency and constraint patterns between software components, like “software X version A requires software Y with version B or newer installed” or “software X should never be installed with software Z version C”.

  • The Docker system is conceived to easily describe and deploy services, especially micro-services, possibly on several hosts – e.g. a Docker swarm or a Kubernetes cluster.

These two problems are essentially orthogonal, which means that given the deployment problem to solve, one can use one of them, both of them, or maybe even none of them, as part of the solution. When using both of them, the Debian package is used in the production of the Docker image, and your Dockerfile (the recipes used to prepare the Docker image describing the “virtualised system” ran in a container) would essentially register your Debian repository in the sources of the Debian packaging system and install your package.

With this in mind, it seems to me that what you are really looking for is to implement the immutable server pattern. The recent development in cloud technologies made possible to upgrade software not by using the classical software upgrade system from a software package system (such as the Debian packaging system) but rather by simply replacing the full server at once. (Some persons did this before this development by having three OS-s on a server, two used in alternance to run the appliance and a mini-OS dedicated to performing the appliance replacement. While not overly complex, this seems to have always remained a niche.) This technique can be of interest for you because if you are used to upgrade software on your server using the package manager, the final state of the server depends of the “upgrade history” of the server – especially if errors occur in the upgrade process. This heterogeneity is bad, because it makes production problems hard to reproduce and diagnose, and your mixed experience

We have thousands of these boxes out on the field. We manage the package dependencies, process registration, etc. through a deb package with varying degrees of success.

could relate to this. The immutable server pattern wipes this source of errors by essentially destroying the notion of “upgrade history” from the problem.

Now there are various options to implement the immutable server pattern, two popular choices are to use Docker images, images or to use “master instance images” from your cloud provider (these are called AMIs in AWS and just Custom Images in Google Compute Engine). Your use-case forbids the use of cloud based techniques, I will therefore assume Docker images as the only eligible choice. (For the sake of completion, it is certainly possible to use other approaches, for instance using Virtual Box or similar virtualisation solution, as an alternative to Docker.)

When using the immutable server pattern technique, you introduce a new artefact (the Docker image) representing your server and this artefact can be tested as well, and it is very easy to obtain a setup replicating truthfully your production settings – aside from service load.

Now to consider the concrete problem you described, let's assume implementing the immutable server pattern with Docker is actually what you want. Since the Docker system and the Debian packaging system are complementary rather than mutually exclusive (cf. intro) we still have to address the question if you should you prepare a Debian package for your software.

The pertinence of using a Debian package to install your software (in the Docker image or on a host) lies in the complexity of the versioning problem you have to solve. If you run at the same time several versions of your software, occasionally need to downgrade, and have complex version requirements that you need to carefully document, having a Debian package is a must-be. Otherwise, this step can be skipped – but since you already put an effort to produce and deploy these packages, there is no real value into ditching your work. I would therefore suggest continue to produce your Debian packages.

  • @Tensibai You are right, I reworked the answer according to this. – Michael Le Barbier Grünewald Aug 30 '17 at 16:16
  • 1
    Maybe I'm pedantic, but what about the various upstart processes mentioned in the question ? In my opinion introducing docker in the described stack is just introducing one more dependency, you still have to maintain the underlying host, and you now have to handle the complexity of sharing file systems between the containers and potentially the problem of inter process communication now they are in separate namespaces. Moreover there's probably a database somewhere behind the Django app (at least for Django itself) which is usually a bad candidate for immutable server pattern for newcomers. – Tensibai Aug 30 '17 at 16:25
  • 1
    @Tensibai Again, a very valid point :) – Michael Le Barbier Grünewald Aug 30 '17 at 16:31
0

Docker sounds reasonable to me, as you could make and test changes to the container in house, and then depending on your release process, restart the containers always pulling :latest or something similar which would provide a tested upgrade.

Considerations you would need to deal with include data storage as containers don't retain changes on restart, so you would want a data volume. There are probably a lot more considerations you would have as well once you dig into it. The system that I am currently working with (all docker-based) has been in development for over a year now and we are still finding areas in which we need to make changes to the container, configurations, etc.

  • 2
    It doesn't really answers how Docker is better than .deb packages. – AlexD May 17 '17 at 1:35
0

By and large docker provides many advantages for both developers and operations personnel. I use docker for some of my applications and find it to be a very reliable and robust approach.

My issue with you adopting docker is that I do not hear you asking the right questions and you could potentially make your life more complicated without addressing your most important requirements.

The first question you should ask (you said that you were new) is how are updates to the OS & application handled now? Does the current methodology work for you (your company)? What works well? What could be improved? Can you do a physical configuration audit on your target machines in the field to verify that they have the correct OS patches, application & were there any unauthorized changes.

I love docker, but I would not jump on the docker bandwagon without first assessing where you are right now including what works well and what needs to be improved.

0

I think it could be a good option (further tests are necessary)

You could provide a URL with all tags/version of the container you have made and the clients will read that URL to see if there is a new version of the container.

You can store personal files/settings on local and you will never lost that information in upgrades and you will ensure that what you have made and tested will work for everyone in the same way.

Even you could give the users the possibility to choose which version from the available they want to use (if you want to give that possibility).

It will be like ""only upgrade one package"", talking about only retrieve a new version of the container, much better that dealing with debian packages ;)

  • How can you ensure it will work the same for everyone ? an appliance which sat down for 3 years has good chances to have an old docker host and as such won't be able to run the latest docker image built. Read the question again, OP does provide the hosting system... – Tensibai Aug 28 '17 at 8:45
  • A tested docker image should work for all boxes that you know docker works fine. if you control de SO, you can meet all requirements for needed packages and configuration files that will support Docker. You should test if your image will work on the oldest boxes, maybe you should upgrade de SO or some packages. Sorry but I don't know what do you mean with "OP" – RuBiCK Aug 28 '17 at 8:55
  • OP = Original Poster (the question author if you prefer). So what you're saying is you have to test the docker package the same as you have to test a debian package, I can't see in your answer an added value over just testing a debian package and meet all the requirements also, I just see an added complexity by the addition of the docker layer. (and we're still talking about just one part of the question, not addressing the multiples upstart processes needed around the app itself) – Tensibai Aug 28 '17 at 9:08
  • You need to test whatever the solution you choose. IMHO it's easier to fail an upgrade process made by packages rather than running a new docker. – RuBiCK Aug 28 '17 at 9:25
  • We're more after verifiable facts and/or experience than opinion on Stack Exchange sites. Backed up opinions are ok, but for now I fail to see how your answer address the question exactly. Remember SE sites are not discussion forums, the format doesn't fit and is not made for this. – Tensibai Aug 28 '17 at 9:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.