4

I'm trying to get a better understanding of the reasons to use [and not use] Docker based on specific use cases.

From my current understanding, Docker helps to isolate applications and their dependencies within containers. This is useful to ensure consistent reproducible builds in varied environments.

However, I'm struggling to understand the rationale of using Docker where the environments are essentially the same, and the applications are relatively simple.

Say I have the following:

  • a cloud VM instance (DigitalOcean, Vultr, Linode, etc.) with 1Gb RAM running Ubuntu 20.
  • a Node.js Express app (nothing too complicated)

The following issues come to the fore:

  1. Dockerizing this application will produce an image that is ~100Mb after optimization (without optimization probably 500Mb or higher based on my research). The app could be 50Kb in size, but the Docker container dependencies to run it are significantly higher by a factor of up to 10,000 or above. This seems very unreasonable from an optimization standpoint.

  2. I have to push this container image to a hub before I can use Docker to consume it. So that's 500Mb to the hub, and then 500Mb down to my VM instance; total of about 1Gb of bandwidth per build. Multiply this by the number of times the build needs to be updated and you could be approaching terabytes in bandwidth usage.

  3. I read in a DigitalOcean tutorial that before I can run my container image, I have to do the following:

docker pull ubuntu

This pulls an image of Ubuntu. But, I'm already on Ubuntu, so does this mean I'm running a container that's running Ubuntu inside an existing VM that is running Ubuntu? This appears to be needless duplication, but I'd appreciate clarification.

  1. The Docker Docs specify that I should have 4Gb RAM. This means I have to use more expensive VM instances even when my application does not necessarily require it.

How exactly does containerization [using Docker or similar] optimize and enhance the DevOps experience, especially on an ongoing basis?

I'm not quite getting it but I'm open to clarification.

2
  • After you Dockerize things you no longer need VM. All you need is a service that runs the container for you. Of course you can still use a VM and handle the "orchestration" yourself. For this simple case you really don't need a container, unless you gonna run it in ECS (or such) and save on the DO droplet (or if you are lone developer). As for the "app could be 50Kb in size", yea but you need all dependencies (mind you with correct versions) on the host machine, with containers (Docker) you can leverage layers / caching or run the same app with different version of dependencies.
    – Kyslik
    Aug 30, 2022 at 14:43
  • @Kyslik Thanks for the answer. So with this "service that runs the container for you", they're still running virtualized hardware? Also, I now see the benefit of being able to run multiple versions of the same software on different containers for compatibility between different apps. Still, if you're doing that it might mean that you're spreading yourself too thin and should probably consolidate the applications you use i.e. be more boring.
    – ObiHill
    Aug 31, 2022 at 14:13

1 Answer 1

2

I'm struggling to understand the rationale of using Docker where the environments are essentially the same, and the applications are relatively simple.

In reality, it is highly unlikely that any development environment on any project would ever be anywhere near the same as staging/production.

  • Services running in staging/production will nearly always be physically hosted and managed somewhere which is not intended to be operated interactively by a human day-to-day, with an appropriate IT/security profile to match;
  • The nature of development work, and even internal build/testing typically requires a different IT profile to that of a production server.
  • Developers rarely have control over the underlying infrastructure or the organisation's IT/security policies.

There are many ways in which the IT profile of developer environments, including build agents and even test machines, can deviate from production:

  • Users/permissions or other security settings.
  • Installed tools, SDKs, runtimes, debug/test tools, OS features/packages, and other dependencies operating with debug/test configurations enabled.
  • Environment variables
  • Filesystem structure and the content of files in globally shared directories
  • Configurations of globally-installed dependencies such as web servers.

Furthermore, consider the nature of physical devices and VMs

  • They are stateful and mutable
  • Every change to any aspect of a device or VM, including installed software and configuration changes potentially affects its entire state for all processes running on it.
  • Physical devices and VMs typically run many processes/services concurrently, it would usually not be considered economical to have a whole server or VM just for a single running process.

What containers provide:

  • Isolation from the host device/VM and from an organisation's IT, Network and Infrastructure policies.
  • Isolation from each other - for example, consider the issue of requiring multiple versions of globally-installed runtime dependencies or modifications to shared host resources such as environment variables or local files.
  • Developers typically have full control over their choices of container images and the networking/orchestration inside the container runtime.
  • Images are based on immutable layers, meaning it is not possible for the state of any layer in an image to change, so a published image should always be a good, known, valid starting point.
  • The size of a parent image tends to be inconsequential because there's typically no reason to duplicate nor to re-download it unless a new version of that parent image is published.
  • A container is its own thin, mutable layer on top of an image, usually negligible in size, and uses the parent image for all dependencies.
  • if a container ends up in an invalid state, it may be quickly and cheaply disposed and replaced with a fresh, clean new container almost instantly by recreating another fresh new container layer.
  • The cheap, light-weight nature of containers makes it very efficient to run a single process per-container.
8
  • Ben, thanks a lot for this detailed answer. I appreciate the higher-level approach to the answer. I can appreciate the flexibility that containers provide for DevOps teams, especially in the enterprise environment you have described. Without dismissing Docker, I'm looking into building [in the open] an opinionated alternative to it (perhaps leveraging LXD) for a low-code platform [currently idea stage]; 5Mb would be the limit for container images (any larger would be from app dependencies). Could you share any resources, books, etc. that would help me boost knowledge on this?
    – ObiHill
    Aug 31, 2022 at 13:58
  • Also happy to have a quick chat if possible :)
    – ObiHill
    Aug 31, 2022 at 14:14
  • @ObiHill The underlying Linux Kernel feature (for Docker and I assume for LXD too) is Linux Namespaces, so you could consider starting there to get an understanding of those - the best place is likely to be in the official docs for some popular distro such as Red Hat. -- Docker itself is already really quite a thin layer of abstraction around namespaces (IMO), so while you could certainly roll your own using namespaces (and I obviously do not know your specific requirements), you may just end up reinventing docker without all of its useful and convenient features to help you manage it. Aug 31, 2022 at 17:25
  • Also obviously Docker's own docs are very well worth any time spent and its guides/tutorials for getting some simple containers working to learn its tools and capabilities (if nothing else, to understand the kind of stuff you'd be missing out and would very likely need to reinvent if you decided to create your own, at least if you wanted any chance of being able to support and maintain it in production -- e.g. the docker CLI, its logging, configuration model, various convenient ways to observe a running container and diagnose problems.) Aug 31, 2022 at 17:31
  • Also I think Docker in any system/project is usually just one in a much longer list of tools -- Docker is handy for the one very narrow and specific set of problems it solves (A tool which does one thing and does it well), but it still leaves you with plenty of other potentially tricky issues to consider -- 12factor.net (At least, I think these factors are all worth considering for anyone looking at a development platform because they tend to play a large role in how people support and maintain a production system over many years) Aug 31, 2022 at 17:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.