22

There are some questions about immutable servers, such as:

It seems obvious that it has to do with servers (that part I get). And just digesting the grammar of immutable, I would think it has something to do with "not possible to mute". If that guess is close, I'd have no clue what exactly cannot be muted (and I doubt it has to do with soundcards or something ...).

My questions:

  • What is actually a "immutable servers" (in the context of DevOps)?
  • Why are they used?
  • 4
    not possible to mutate – Evgeny Mar 7 '17 at 12:43
  • 3
    @Evgeny : ggggggggggggrrrrrrrrrrrrrrrr, I was close, but completely wrong with my mute guess (please don't laugh at me ...). – Pierre.Vriens Mar 7 '17 at 12:50
  • 3
    related question – Tensibai Mar 7 '17 at 12:57
  • I'm sorry to be so blunt, but was a simple internet search to define "immutable" that onerous? I get the question is larger than that, but looking up the meaning of the word rather than guessing might have given you a decent head start. – Adrian Mar 9 '17 at 21:12
  • @Adrian no problem abount being blunt (because I suffer ESL, I'd have to google that to really get the exact meening of blunt). However I wonder if your are aware of this meta.SA question? Apart from that, I would rather learn from DevOps experts, instead of Wikipedia-like alternatives. – Pierre.Vriens Mar 9 '17 at 21:57
17

Immutability is a term often used in computer science circles, which generally boils down to "not possible to change after creation". It is typically used in reference to parallelism, concurrency, and thread safety.

The discussion of that topic is fascinating, but can generally be found elsewhere on Stack Overflow. I'm resisting the urge to dive into it here. The key concept is 'not possible to change after creation'.

Imagine if, on Amazon, you deploy a web service by baking it into a Machine Image (AMI - a pre-built instance you can repeatedly re-provision). It connects to a backend database via credentials it gets out of a registry on startup. It dumps logs into a logging tool like Splunk. For normal day-to-day operation, you have no reason to ssh into this box. If you need to scale up that service, you just create more instances of that AMI and adjust a load balancer. Spinning it down is simply the destruction of instances and load balancers.

For the day-to-day operation, this box has no reason to change. We can just fire up more off of the AMI.

What happens when you need to provide an OS-level security patch? This is when you have a decision to make... do you bake a new AMI with the patch installed and redeploy all running instances, or do you ssh into existing images and update the patch? There are plenty of people who would just ssh in. The adherents of 'immutable architecture' just yelled at me for even suggesting such a thing is possible.

Immutabilists (if there is such a word) advocate baking the new ami. They advocate removing all reason to ssh into a machine ever. They advocate that any specific machine configuration should happen on the startup of that machine by pulling config details out of a repository. This is the ultimate expression of 'cattle, not pets'.

Immutable architecture is specifically about machine configurations that have no reason to change after the creation of the machine image. If something needs to change, bake a new instance image, shut down the old, bring up the new.

  • "shut down the old, bring up the new". Or if your architecture allows it, bring up the new, adjust load balancer, shut down the old. That way you can do it anytime you like, even in the middle of peak time. :) – Tim Malone Jun 16 '18 at 15:34
  • Immutability if from functional programming (1960s), it is now being realised that not only does it reduce bugs (something that is to often not cared about), but also aids with concurrency. – ctrl-alt-delor Mar 19 at 8:47
  • I'd really be interested in any addition to this fantastic answer on how monitoring agents or additional software that might be difficult to bake into the original might come into play here. For example, APM monitoring software that has to detect it's new machine environment after creation and change parameters. I'm exploring SSM and automation to deploy this in AWS, but have found very little to discuss the immutable with AMIs/images when dealing with additional agents that might be installed at a later time. – SheldonH Mar 21 at 7:25
10

Cloud technologies shifted the frontier between hardware and software so that many technical operations formerly exclusive citizen of the hardware world are also subjects of the software realm. Shared computing environments might be as old as computers themselves1 but cloud technologies could popularise them by offering convenient and familiar metaphors to interact with them: cloud users reserve an instance, a complete computer or a mimic, whereas older shared computing environments have all possible set of unwieldy limitations and “your program have to be uploaded on that FTP server, will run in environment X (usually with 10y old version of whatever software you want to use), for at most 60 minutes” could sound familiar to former or actual users of computing centres.

The practical consequence of this shift is that deployment procedures can now be represented by software artefacts. (Deployment procedures are the instructions telling how to setup an infrastructure, with databases, web-servers or whatever belongs to this infrastructure, together with the network where they run on.) Geared with these new lenses, manual maintenance of servers pretty much looks like manual patching of production code – which is only in very rare occasions a desirable thing. Manual maintenance is susceptible to introduce discrepancies between the systems actually running in production and the code describing these systems, which in turns mean irreproducible behaviour and impossible bug analysis, dual bug fixing, and other calamities.

The immutable server pattern is just the transposition for cloud operations of the above mantra, according to which we should avoid manual maintenance of running programs. Instead of manually configuring severs, the immutable server pattern recommends to automatise this configuration.

Implementation flavours

While the general idea of the immutable server pattern is quite clear, there is a lot of implementation nuances. For instance, some approaches suggest to not update servers at all but to systematically replace servers instead. This is because updating yields situation where a deployment consists of servers having been started at several distinct times and having gone through several, distinct, updating processes which implies an inhomogeneous set of servers and can lead to subtle differences in how servers handle their jobs. A second popular variation point is the discipline regarding remote access to the servers. Some like to disable completely remote administrative access to servers, in order to guarantee that manual maintenance never happens.

History note

To the best of my knowledge, the term “immutable server” has been popularised by Kief Morris but the idea itself is much older. In 1999 FreeBSD jails already popularised the idea of fully automatising the configuration of disposable computing environments, this is how I started to implement the “immutable server” pattern many years before I heard this name to describe this technique.

Immutability, in the guise of physical immutability based on CD-ROMS, has also been a popular measure to manufacture trusted computing systems. This is not to be mistaken with the immutable server pattern.


1If we don't count automatic loom tables or roller organs as computers.

  • 1
    Wow, yet another interesting explanation, merci! Now I'm really getting in trouble to decide about which answer to mark as accepted (patience on that please, and please know that I can only mark 1 at max, though at this time I am not at all decided on which one ...). – Pierre.Vriens Mar 7 '17 at 16:07
  • 1
    Avec plaisir! – I think it's a good idea to wait for several days to accept an answer, this increases the chances of users writing more answers. – Michael Le Barbier Grünewald Mar 7 '17 at 18:39
9

Immutable servers are servers on which no changes can be made (other than updates and security patches ideally). Instead of changing the software on the server, you spool up a new server with the desired software and then terminate the older one.

This concept helps to ensure that your test, development, and QA server are all identical, which is important for multiple reasons out of the scope of this question. Another benefit of immutable servers is the ability to rollback the application onto an older server. For example, I need to change K on production server 1, so I spool up server 2 and change K. Now after 10 minutes, I notice that K broke something with my application, rather than having to fix it right away which could take hours and potentially cause downtime for my customers, I redirect the traffic back to server 1, while I figure out what is wrong with 2.

  • Hm, interesting ... I need some time to further digest this ... Know the saying like "1 answer to a question triggers 10 new questions?" ... – Pierre.Vriens Mar 7 '17 at 12:48
  • 1
    That practice to quickly "rollback" is often called "Blue/Green Deployment" (also red/black, depends who does the calling). – Evgeny Mar 7 '17 at 12:54
  • @Evgeny and worse of all, could be named A/B deployments too, bluring line with A/B experiments. (and even worse, when this type of deployment, multiples version of the same app, are live to do A/B experiments instead of feature flags) – Tensibai Mar 7 '17 at 13:01
  • I was always told that Blue/Green Deployments were for the deployment of an update to an existing application , NOT the server itself. @Evgeny – Turtle Mar 7 '17 at 13:28
  • Yes. You can swap servers, containers, applications and whatnot and still call it Blue/Green. I think that deserves its own Q&A here. Just wanted to point out that the way you explained rollback in your answer is quite often called B/G. – Evgeny Mar 7 '17 at 13:30
6

The best explanation can be found (as always) on Martin Fowler's bliki article on Immutable Servers.

A server, be it hardware or a virtual server in the cloud, usually has an operating system and application running on it.

Often application, and components of the operating system, require configuration and require changes to be applied. For example security patches, deployment of new versions of the application and configuration changes.

When you consider that any change is a mutation on the state of the server, the term immutable starts to make more sense. It means that no mutations are allowed on such a server.

It is often the case, when people are involved in changing the state of the server - be it a deployment of a version, or configuration change, or a security path. The result is a server that is no longer working as expected. For example, the application might not run now because of misconfiguration, etc.

This is why a practice for creating immutable servers is established. With immutable servers, an image of a server is created with all the configuration, patches, application versions bundled in. Then that server image can be used to create servers in various environments.

The first environment where such an image is used, would be an environment where the image can be tested to work. Any abnormalities are detected, and only then such an image can be promoted to a production environment to replace the servers there with the new version (which is known to work well).

Once the process of creating the images and promoting the images is automated, you get a very failure-proof process that involves very little human effort and very low chance to introduce failure into your service.

Often immutable servers don't even include any way to "enter" them, such as for example the ssh server is missing. In this case it is also often the case that all the metrology of a server (metrics, logs) is shipped to systems outside such as a metrics database or log aggregation service.

With containers (see: Docker) there is also a process to create images, and then spawn these into running containers. These are quite often replaced by new containers based on updated images, and are never mutated. Meaning that no human enters into the container to "fix something" by introducing a change.

  • Interesting explanation, maybe you want to mutate it a very little bit, if you can, and if it makes sense, to elaborate on something which I believe is somehow related, i.e., "flushing" a system. Which I think is that you let (more or less) anybody "play" with something, and warn them upfront that (eg) every night there is some kind of reset to some initial state. It sounds that the input for such reset is ... euh ... what was I going to say ... right: such an immutable-thingie that can be used for it. – Pierre.Vriens Mar 7 '17 at 14:14
  • 2
    Running a service that returns the server to a known state (such a Chef/Puppet/Ansible/etc...), just means that you are not using immutable server. en.wikipedia.org/wiki/Promise_theory is great, but martinfowler.com/bliki/ImmutableServer.html is even better. – Evgeny Mar 7 '17 at 14:16
  • You're teasing me I believe, or is it rather "challenging me" (to try to discover the daily question limit). Isn't there also somewhere a SE-rule that says "you can only ask 50 question in 30 days"? – Pierre.Vriens Mar 7 '17 at 14:21
0

Let's start with the inverse, what is a mutable server?

Traditionally, a mutable server infrastructure is one that is continuously modified and updated in place. You can secure shell into it, upgrade packages, configure it, install services and deploy new code to it. This is what makes it mutable, you can mutate or modify it.

An immutable infrastructure is another infrastructure paradigm in which servers are never modified after they're deployed. If something needs to be updated, fixed, or modified in any way, new servers built from a common image with the appropriate changes are provisioned to replace the old ones. After they're validated, they're put into use and the old ones are decommissioned.

Why are they used? The benefits of an immutable infrastructure is more consistency and reliability in your infrastructure and a simpler, more predictable deployment process and it mitigates common server issues in mutable infrastructure, such as downtime from the server crashing or whatever.

But you have to know how to provision it efficiently via comprehensive deployment automations and fast server provisioning.

Imagine you are mining bitcoin, you would not want any downtime if your server crashed, you would need it back up as fast as possible, so an immutable infrastructure is supposed to be the solution.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.