39

A team of IT sysadmins that have exprience using shell scripting to solve their problems, are contemplating to start using Ansible instead.

Are there substantial differences and good reasons to start using Ansible vs. to continue writing shell scripts?

29

I never used Ansible but since a few weeks, I try to figure out what good Ansible could be in comparison with shell scrips–Which proves, at least in my case, that the haunting ad-campaigns they run are effective! After many unsuccessful attempts–which proves how their documentation fail at answering one of the most obvious question–I think I finally got it:

Now, let's watch the introduction video and go randomly as a potential new user through the introduction material to Ansible ans let's compare it to what a skilled shell programmer can produce right-off the shelf.

My conclusion is that over shell scripting, Ansible essentially offers 1. The possibility of checking that a system agrees with a desired state, 2. the ability to integrate with Ansible Tower, which is a paying system that seems to include monitoring abilities. In some important cases, like when implementing the immutable server pattern, the point 1 is probably not very useful, so the list, is rather thin.

My conclusion is that the benefits offered by Ansible over shell-scripting, as the documentation present them, could be sensible in a few handful of optimistic cases well covered by available modules but are small or even hypothetical in the general case. For a skilled shell-programmer probably, these benefits are most likely counter-balanced by other aspects of the trade-off.

But this maybe only proves how bad the introduction material is!

The quick start video:

There is a quick start video. It starts with a page claiming that… well these are not really claims, these are bullet lists, an artefact commonly used to suspend critical judgement in presentations (since the logic is not shown, it cannot be criticised!)

1. Ansible is simple:

1.1 Human readable automation – Specifications are technical documents, how could

  name: upgrade all packages
  yum:
    name: '*'
    state: latest

be easier to read than the corresponding yum invocation found in a shell-script? Furthermore, anybody who had contact to AppleScript dies laughing when they read “human readable automation”.

1.2 No special coding skills required – What is coding if not writing formal specifications? They have conditionals, variables, so, how is it not coding? And why would I need something I cannot program, that would henceforth be inflexible? The statement is happily inaccurate!

1.3 Tasks executed in order – Well, maybe some codegolf aficionados are aware of languages that execute tasks in disorder, but executing tasks in order hardly looks exceptional.

1.4 Get productive quickly – Skilled shell programmers are productive now. This counter-argument is just as serious as the initial argument.

2. Ansible is powerful

A popular salesman trick to sell artefacts is to fool people into believing they will acquire the “power” of these artefacts. The history of advertisement for cars or isotonic drinks should supply a convincing list of examples.

Here Ansible can do “app deployment” – but shell script surely do, “configuration management” but this is a mere statement of the purpose of the tool, not a feature, and “workflow orchestration” which looks a bit pretentious but no example goes beyond what GNU Parallel can do.

3. Ansible is agentless

To populate the column, they wrote in three different manners that this only needs ssh, which, as everybody knows is a daemon and has nothing to do with these agents pervading the world configuration management!

The rest of the video

The rest of the video introduces inventories, which are static lists of resources (like servers) and demonstrates how to deploy Apache on three servers simultaneously. This really does not match the way I work, where resources are highly dynamic and can be enumerated by command-line tooling provided by my cloud provider, and consumed by my shell functions using the pipe | operator. Also, I do not deploy Apache on three servers simultaneously, rather, I build a master instance image that I then use to start 3 instances which are exact replicas one of the other. So the “orchestrating” part of the argumentation does not look very pertinent.

Random documentation step 1: Integration with EC2

EC2 is the computing service from Amazon, interacting with it is supported by some Ansible module. (Other popular cloud computing providers are also provided):

# demo_setup.yml

- hosts: localhost
  connection: local
  gather_facts: False

  tasks:

    - name: Provision a set of instances
      ec2:
         key_name: my_key
         group: test
         instance_type: t2.micro
         image: "{{ ami_id }}"
         wait: true
         exact_count: 5
         count_tag:
            Name: Demo
         instance_tags:
            Name: Demo
      register: ec2

The corresponding shell-script would be essentially identical with YAML replaced by JSON:

provision_a_set_of_instances()
{
  aws --output=text ec2 run-instances --image-id …   
}

or the JSON version

provision_a_set_of_instances()
{
  aws --output=text ec2 run-instances --cli-input-json "$(provision_a_set_of_instances__json)"  
}

provision_a_set_of_instances__json()
{
  cat <<EOF
{
    "ImageId": … 
}
EOF
}

Both version are essentially identical, the bulk of the payload is the enumeration of the initialisation values in a YAML or JSON structures.

Random documentation step 2: Continuous Delivery and Rolling Upgrades

The largest part of this guide does not display any really interesting feature: it introduces variables (IIRC, shell scripts also have variables)!, and an Ansible module handles mysql, so that if instead of searching after “how do I create a mysql user with privileges on X Y” and end with something like

# Create Application DB User
mysql --host "${mysql_host}" --user "${mysql_user}" --password "${mysql_password}" "${mysql_table}" <<EOF
GRANT ALL PRIVILEGES ON *.* TO 'root'@'%';
EOF

you search after “how do I create a mysql user with privileges on X Y in ansible” and end up with

- name: Create Application DB User
  mysql_user: name={{ dbuser }} password={{ upassword }}
              priv=*.*:ALL host='%' state=present

The difference is still probably not very meaningful. On that page we also discover that Ansible has a template meta-Programming language

{% for host in groups['monitoring'] %}
-A INPUT -p tcp -s {{ hostvars[host].ansible_default_ipv4.address }} --dport 5666 -j ACCEPT
{% endfor %}

When I see this, I happen to really be in my comfort zone. This kind of simple meta-programming for declarative languages is exactly the same theoretical paradigm as BSD Makefiles! Which I happen to have programmed extensively This excerpt shows us that the promise of working with YAML file is broken (so I cannot run my playbooks through a YAML parser, e.g.). It also shows us that Ansible must discuss the subtle art of evaluation order: we have to decide if variables are expanded at the “declarative part” of the language or at the “imperative” meta-part of the language. Here shell programming is simpler, there is no meta-programming, aside from explicit eval or external-script sourcing. The hypothetical equivalent shell excerpt would be

enumerate_group 'monitoring' | {
  while read host; do
    …
  done
}

whose complexity in comparison to the Ansible variant is probably tolerable: it just uses the plain, regular, boring constructs from the language.

Random documentation step 3: Testing strategies

Last, we meet what turns out to be the first actually interesting feature of Ansible: “Ansible resources are models of desired-state. As such, it should not be necessary to test that services are started, packages are installed, or other such things. Ansible is the system that will ensure these things are declaratively true. Instead, assert these things in your playbooks.” Now it starts to be a bit interesting, but:

  1. Aside from a handful of standard situations readily implemented by available modules, I will have to feed the bits implementing the test myself, which will quite probably involve some shell commands.

  2. Checking for the conformity of installations might not be very relevant in the context where the immutable server pattern is implemented: where all systems running are typically spawned from a master image (instance image or docker image for instance) and never updated – they are replaced by new instead.

Unaddressed concern: the maintainability

The introductory material from Ansible ignores the question of the maintainability. With essentially no type system, shell-scripting has the maintainability ease of JavaScript, Lisp or Python: extensive refactorings can only be achieved successfully with the help of an extensive automated testsuite – or at least designs that allows easy interactive testing. That said, while shell scripting is the lingua franca from system configuration and maintenance, nearly each programming language has an interface to the shell. It is therefore totally feasible to leverage the maintainability advantage of advanced languages, by using them to glue together the various the bits of shell-configuration bits. For OCaml, I wrote Rashell that essentially provides a hand of common interaction patterns for subprocesses, which makes the translation of configuration scripts to OCaml essentially trivial.

On the side from Ansible, the very weak structure of playbooks and the presence of a meta-programming feature make the situation essentially as bad as it is for shell scripting, with the minus points that it is not obvious how to write unit tests for Ansible, and the argument of introducing ad-hoc a higher-level language cannot be mimiced.

Idempotency of configuration steps

The documentation of Ansible draws the attention on the necessity of writing idempotent configuration steps. More precisely, configuration steps should be written so that the step sequence a b a can be simplified to a b, i.e. we do not need to repeat configuration step. This is a stronger condition than idempotency. Since Ansible allows playbooks to use arbitrary shell commands, Ansible itself is unable to guarantee that this stronger condition is respected. This only relies on the programmer's discipline.

  • This seems like a manual ... impressive! – Pierre.Vriens Mar 7 '17 at 18:46
  • 4
    I couldn't agree more. We used Ansible for over 1 year now and are now using Docker containers, built with good, plain old bash scripts. Defining the state is also in my opinion by far the most interesting feature, but as you mentioned it already - there are so many services that don't have a corresponding Ansible module, so you always have to fallback to bash commands anyway. And yes, we also only deploy immutable containers to servers, so defining the state is not really any advantage in this case. – Andreas Apr 16 '17 at 10:34
  • 1
    After using ansible thoroughly I can confirm all the points I made a priori. Idempotency is possible but not enforced by ansible (see module vmware_guest for a bad citizen), working with their macro system is a real pain and it is incredenly hard to perform even the most basic treatments on structured data, some basic things are doing just wrong (the playbook format cannot eat a Unix file mode without a therapy) and the only real good thing the the load of useful functions written for ansible. So if it were not for Red Hat pushing that product, I cannot understand the wide adoption. – Michael Le Barbier Grünewald Mar 28 '18 at 9:28
  • 1
    @Andreas I agree you still have many cases where you need to fall back to the shell or command modules that doesn't mean your ansible play can't be idempotent. The core modules themselves maintain idempotency by just checking if the action should be done. You can do the same thing your self with shell or command module by first running a task that checks if something should be done and registering its output, then doing a conditional on the second task based on the output from the first task. – Levi Mar 17 at 2:31
  • 1
    @MichaelLeBarbierGrünewald, I have to agree overall, when I worked with Ansible, it was a real pain to get running and its work that takes weeks to get a playbook together to connect to the infrastructure at my former cloud-based company, provision the linux distro, install LAMP/LEMP or whatever. Once it was completed, it saved us time, but it took like a month just to get it up and running. None of us were master bash scripters so that was not an alternative. – Daniel Mar 21 at 12:14
22

When you put it this way, even if Ansible has some inherent advantages, the benefits of using familiar tools (in this case shell scripting) must be outweighed. I don't think there's a clear cut answer to that.

If the team can achieve the things Ansible offers with shell:

  1. Declarative, idempotent configuration management
  2. Access to re-usable snippets (i.e. playbooks) for industry popular services.
  3. Reliable management of remote execution, with retrying, rolling logic, etc.

then they could probably stick with what they know.

After all, you can implement "guards" in BASH. You can find lots of BASH existing work out there to solve a variety of server configuration tasks (essentially any Dockerfile out there is 90% bash installation code). You can get pretty close to what Ansible/Salt/Chef-Zero offer you, without actually having to port your entire existing solution to those tools.

It's a balancing act between NIH (not invented here) tendencies, and throwing out good, established scripts in favor a more robust solution.

One final consideration to keep in mind: how does your technology stack measure up when you try to recruit more people to the team. Finding people who have Ansible experience is a lot easier than finding people who have experience in your particular home-grown scripting CM tool. This isn't strictly a technical consideration, more a cultural one. Do you want to be the weird org that invents its own Ansible, or do you want to be the reasonable org that finds the right tool for the job? Those decision impact your ability to draw talent.

  • 4
    Liked your answer; I would also mention that, if the bash team is going for idempotency, execution management, and reuse, basically writing their own config management framework, there is a cost involved, and we all know that is can get really nasty for in-house projects. – ᴳᵁᴵᴰᴼ Mar 5 '17 at 20:58
  • I also subscribe to your answer, especially having put the available experience in the balance. I have two small critics: first is the idempotency This is of course an important aspect of configuration systems, but since it is possible to use any possible shell commands in ansible playbooks, the system can at best incitate to use idempotency. (We actually want something more powerful that idempotency aba = ab.) Second the reliable management of remote execution might be totally irrelevant in some important cases, e.g. when implementing the immutable server pattern using instance images. – Michael Le Barbier Grünewald Mar 7 '17 at 0:44
13

The above answer covers part of it but misses one of the important elements: convergent design. I wrote some words a while ago about this in the context of Chef at https://coderanger.net/thinking/ but the short version is that a bash script is a set of instructions, while an Ansible playbook (or Chef recipe, Salt state, etc) is a description of desired state. By documenting the state you want rather than the steps you want to take to get there, you can cope with a lot more starting states. This was the heart of Promise Theory as outlined in CFEngine long ago, and a design which we (the config management tools) have all just been copying since.

tl;dr Ansible code says what you want, bash code says how to do a thing.

  • 2
    Can you also add some reference to "promise theory", like books or articles, and any other valuable material if someone wants to learn about it? – Evgeny Mar 6 '17 at 3:26
  • 1
    That's actually what I alluded to when I said that you can write idempotent bash code (i.e. that can start at any point, be run multiple times, and converge onto the desired state). But your answer made it much clearer. – Assaf Lavie Mar 6 '17 at 5:08
  • Yeah, idempotence is an important property of convergent systems but the two are not always directly linked :) as for materials on Promise Theory, Mark Burgess (creator of CFEngine) has a few books, I can find links when I'm back at a laptop. – coderanger Mar 6 '17 at 6:17
  • 1
    @Evgeny, markburgess.org/TIpromises.html – Wildcard Sep 16 '17 at 0:59
3

One thing worth noting that you will have less issues in running your ansible playbooks on remote hosts too. As it's the main reason for running ansible. When you are using shell scripting you still need to have a way to script the scp'ing to the remote host.

  • Isn't Ansible just a wrapper around ssh in this sense? – Evgeny Mar 5 '17 at 23:54
  • Essentially, yes. It does ssh, copies python scripts remotely, and runs them. That means, btw, that if your ansible module depends on some python library, this library should be present on the remote machine, under some circumstances. – Assaf Lavie Mar 6 '17 at 5:10
  • 1
    And if you mess up the ssh config, you're locked out of your machine, usual drawback of push vs pull. – Tensibai Mar 6 '17 at 20:52
1

It's 2019 and I've just spent a few days on an ansible learning curve and here is the absolute truth: Ansible isn't worth the trouble.

it's not finished, it doesn't run on windows and the combination of YAML config and misleading error messages will make your eyes bleed. It seems almost deliberately terrible and I mean that seriously. It's clearly the product of a redhat sysadmins frustrated developer side project. Probably a hipster.

If you don't require any.of its features beyond provisioning, and you are only ever provisioning on one particular OS. For pities sake write a decent shell.script.

As of right now, the whole project reminds me of early linux forums where noobs were told to RTFM and derided for asking why someone couldn't write a GUI for configuring graphics settings. You just don't get it do you? You should stick to windows... perhaps I will mate.. happy VI-ing.

Use Docker. In preference to anything. Docker is outrageously simple and powerful.

But what if you absolutely must provision on pre-existing tin? What are the real alternatives?

Well... there aren't any, yet. But I will promise you this, unless ansible gets better, there will be soon. Because no matter how hard the fanboys push it, and forgive it's failings... it's a 5 out of 10 for effort.

SCP up a bash script, and save yourself the trouble.

  • First it does work on windows via Win_RM or SSH. Second the yaml syntax is very nice and can be programmatically generated and while some of the errors can be misleading its no different than Java or Python puking its guts during an exception. Third the notion of just SCPing a script to a server isn't applicable in highly dynamic cloud environments. Which server? The servers could change every day. Ansible allows easily configurable inventory plugins with easy ways to group servers and assigns variables to them. I dont think Ansible isn't worth it. I think Its overkill for your environment. – Levi Mar 17 at 2:44
  • @Levi yes. Its all my fault that ansible doesnt run on windows, has config that has no schema, and,has a longer learning curve and higher maintainance costs than any bespoke method of achieving the same task. – Richard Mar 17 at 10:40
  • For cloud environments there are other approaches for the kinds of large scale enterprises that might justify the learning curve. I understand what ansible does. I just don't see its niche. – Richard Mar 17 at 11:15
  • The niche is an easy to use automation framework for multiple machines. It's not as great for Windows as it is for Linux. Neither is it great for msdos and netware. It's 2019, windows servers are the small useless niche these days. – dyasny May 22 at 16:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.