A team of IT sysadmins that have exprience using shell scripting to solve their problems, are contemplating to start using Ansible instead.
Are there substantial differences and good reasons to start using Ansible vs. to continue writing shell scripts?
I never used Ansible but since a few weeks, I try to figure out what good Ansible could be in comparison with shell scrips–Which proves, at least in my case, that the haunting ad-campaigns they run are effective! After many unsuccessful attempts–which proves how their documentation fail at answering one of the most obvious question–I think I finally got it.
My conclusion is that over shell scripting, Ansible essentially offers 1. The possibility of checking that a system agrees with a desired state, 2. the ability to integrate with Ansible Tower, which is a paying system that seems to include monitoring abilities. In some important cases, like when implementing the immutable server pattern, the point 1 is probably not very useful, so the list of advantages is rather thin.
It seems to me that the benefits offered by Ansible over shell-scripting, as the documentation present them, could be sensible in a few handful of optimistic cases well covered by available modules but are small or even hypothetical in the general case. For a skilled shell-programmer, these benefits are most likely counter-balanced by other aspects of the trade-off.
But my conclusion maybe only proves how bad the introduction material is at displaying the actual advantages of this software!
Now, I propose to watch the introduction video and go randomly as a potential new user through the introduction material to Ansible an let's compare it to what a skilled shell programmer can produce in a reasonable time.
There is a quick start video. It starts with a page claiming that… well these are not really claims, these are bullet lists, an artefact commonly used to suspend critical judgement in presentations (since the logic is not shown, it cannot be criticised!)
1.1 Human readable automation – Specifications are technical documents, how could
name: upgrade all packages
yum:
name: '*'
state: latest
be easier to read than the corresponding yum invocation found in a shell-script? Furthermore, anybody who had contact to AppleScript dies laughing when they read “human readable automation.”
1.2 No special coding skills required – What is coding if not writing formal specifications? Ansible has conditionals, variables, so, how is it not coding? And why would I need something I cannot program, that would henceforth be inflexible? The statement is happily inaccurate!
1.3 Tasks executed in order – Well, maybe some codegolf aficionados are aware of languages that execute tasks in disorder, but executing tasks in order hardly looks exceptional.
1.4 Get productive quickly – Skilled shell programmers are productive now. This counter-argument is just as serious as the initial argument.
A popular salesman trick to sell artefacts is to fool people into believing they will acquire the “power” of these artefacts. The history of advertisement for cars or isotonic drinks should supply a convincing list of examples.
Here Ansible can do “app deployment” – but shell script surely do, “configuration management” but this is a mere statement of the purpose of the tool, not a feature, and “workflow orchestration” which looks a bit pretentious but no example in this document goes beyond what GNU Parallel can do.
To populate the column, they wrote in three different manners that this only needs ssh, which, as everybody knows is a daemon and has nothing to do with these agents pervading the world of configuration management!
The rest of the video introduces inventories, which are static lists of resources (like servers) and demonstrates how to deploy Apache on three servers simultaneously. This really does not match the way I work, where resources are highly dynamic and can be enumerated by command-line tooling provided by my cloud provider, and consumed by my shell functions using the pipe | operator. Also, I do not deploy Apache on three servers simultaneously, rather, I build a master instance image that I then use to start 3 instances which are exact replicas one of the other. So the “orchestrating” part of the argumentation does not look very pertinent.
EC2 is the computing service from Amazon, interacting with it is supported by some Ansible module. (Other popular cloud computing providers are also provided.):
# demo_setup.yml
- hosts: localhost
connection: local
gather_facts: False
tasks:
- name: Provision a set of instances
ec2:
key_name: my_key
group: test
instance_type: t2.micro
image: "{{ ami_id }}"
wait: true
exact_count: 5
count_tag:
Name: Demo
instance_tags:
Name: Demo
register: ec2
The corresponding shell-script would be essentially identical with YAML replaced by JSON:
provision_a_set_of_instances()
{
aws --output=text ec2 run-instances --image-id …
}
or the JSON version
provision_a_set_of_instances()
{
aws --output=text ec2 run-instances --cli-input-json "$(provision_a_set_of_instances__json)"
}
provision_a_set_of_instances__json()
{
cat <<EOF
{
"ImageId": …
}
EOF
}
Both version are essentially identical, the bulk of the payload is the enumeration of the initialisation values in a YAML or JSON structures.
The largest part of this guide does not display any really interesting feature: it introduces variables (IIRC, shell scripts also have variables)!, and an Ansible module that handles mysql, so that if instead of searching after “how do I create a mysql user with privileges on X Y” and end with something like
create_application_db_user()
{
mysql --host "${mysql_host}" --user "${mysql_user}" --password "${mysql_password}" "${mysql_table}" <<EOF
GRANT ALL PRIVILEGES ON *.* TO 'root'@'%';
EOF
}
you search after “how do I create a mysql user with privileges on X Y in ansible” and end up with
- name: Create Application DB User
mysql_user: name={{ dbuser }} password={{ upassword }}
priv=*.*:ALL host='%' state=present
The difference is still probably not very meaningful. On that page we also discover that Ansible has a template meta-Programming language
{% for host in groups['monitoring'] %}
-A INPUT -p tcp -s {{ hostvars[host].ansible_default_ipv4.address }} --dport 5666 -j ACCEPT
{% endfor %}
When I see this, I happen to really be in my comfort zone. This kind of simple meta-programming for declarative languages is exactly the same theoretical paradigm as BSD Makefiles! Which I happen to have programmed extensively This excerpt shows us that the promise of working with YAML file is broken (so I cannot run my playbooks through a YAML parser, e.g.). It also shows us that Ansible must discuss the subtle art of evaluation order: we have to decide if variables are expanded at the “declarative part” of the language or at the “imperative” meta-part of the language. Here shell programming is simpler, there is no meta-programming, aside from explicit eval or external-script sourcing. The hypothetical equivalent shell excerpt would be
enumerate_group 'monitoring' | {
while read host; do
…
done
}
whose complexity in comparison to the Ansible variant is probably tolerable: it just uses the plain, regular, boring constructs from the language.
Last, we meet what turns out to be the first actually interesting feature of Ansible: “Ansible resources are models of desired-state. As such, it should not be necessary to test that services are started, packages are installed, or other such things. Ansible is the system that will ensure these things are declaratively true. Instead, assert these things in your playbooks.” Now it starts to be a bit interesting, but:
Aside from a handful of standard situations readily implemented by available modules, I will have to feed the bits implementing the test myself, which will quite probably involve some shell commands.
Checking for the conformity of installations might not be very relevant in the context where the immutable server pattern is implemented: where all systems running are typically spawned from a master image (instance image or docker image for instance) and never updated – they are replaced by new instead.
The introductory material from Ansible ignores the question of the maintainability. With essentially no type system, shell-scripting has the maintainability ease of JavaScript, Lisp or Python: extensive refactorings can only be achieved successfully with the help of an extensive automated testsuite – or at least designs that allows easy interactive testing. That said, while shell scripting is the lingua franca from system configuration and maintenance, nearly each programming language has an interface to the shell. It is therefore totally feasible to leverage the maintainability advantage of advanced languages, by using them to glue together the various the bits of shell-configuration bits. For OCaml, I wrote Rashell that essentially provides a hand of common interaction patterns for subprocesses, which makes the translation of configuration scripts to OCaml essentially trivial.
On the side from Ansible, the very weak structure of playbooks and the presence of a meta-programming feature make the situation essentially as bad as it is for shell scripting, with the minus points that it is not obvious how to write unit tests for Ansible, and the argument of introducing ad-hoc a higher-level language cannot be mimiced.
The documentation of Ansible draws the attention on the necessity of writing idempotent configuration steps. More precisely, configuration steps should be written so that the step sequence a b a can be simplified to a b, i.e. we do not need to repeat configuration step. This is a stronger condition than idempotency. Since Ansible allows playbooks to use arbitrary shell commands, Ansible itself is unable to guarantee that this stronger condition is respected. This only relies on the programmer's discipline and the importance of this variation of idempotency when writing configuration scripts is certainly not a novelty.
Post-Scriptum. Since this answer seems to enjoy a relative popularity, I fixed a few embarrassing syntax errors and typos. By a twist of life I also had to use Ansible two years in my work. Overall my experience confirms what I foreseen here and I hardly can think about a situation where shell scripts would have been really outperformed by Ansible. On some aspects, Ansible is just worse than shell scripting. At least the shell has functions, these functions can be mocked, it is possible to test part of all of them, so overall the shell has much better software engineering features than Ansible has. In a shell script it is also possible to process data and awk can express all what SQL can, which is very important when programming configuration – the information we are working with here is not intrinsically hierarchical, so there is a need for extracting an rewriting. Ansible is so bad at extracting and rewriting data! Treatments must be expressed with a mixture of YAMl-templating at the playbook step level and a dialect of Jinja at the dictionary member level… this is cumbersome, ugly, hard to write, hard to test and poorly documented (I regularly looked up the Jinja filter implementations!).
When you put it this way, even if Ansible has some inherent advantages, the benefits of using familiar tools (in this case shell scripting) must be outweighed. I don't think there's a clear cut answer to that.
If the team can achieve the things Ansible offers with shell:
then they could probably stick with what they know.
After all, you can implement "guards" in BASH. You can find lots of BASH existing work out there to solve a variety of server configuration tasks (essentially any Dockerfile out there is 90% bash installation code). You can get pretty close to what Ansible/Salt/Chef-Zero offer you, without actually having to port your entire existing solution to those tools.
It's a balancing act between NIH (not invented here) tendencies, and throwing out good, established scripts in favor a more robust solution.
One final consideration to keep in mind: how does your technology stack measure up when you try to recruit more people to the team. Finding people who have Ansible experience is a lot easier than finding people who have experience in your particular home-grown scripting CM tool. This isn't strictly a technical consideration, more a cultural one. Do you want to be the weird org that invents its own Ansible, or do you want to be the reasonable org that finds the right tool for the job? Those decision impact your ability to draw talent.
The above answer covers part of it but misses one of the important elements: convergent design. I wrote some words a while ago about this in the context of Chef at https://coderanger.net/thinking/ but the short version is that a bash script is a set of instructions, while an Ansible playbook (or Chef recipe, Salt state, etc) is a description of desired state. By documenting the state you want rather than the steps you want to take to get there, you can cope with a lot more starting states. This was the heart of Promise Theory as outlined in CFEngine long ago, and a design which we (the config management tools) have all just been copying since.
tl;dr Ansible code says what you want, bash code says how to do a thing.
It's 2019 and I've just spent a few days on an ansible learning curve and here is the absolute truth: Ansible isn't worth the trouble.
it's not finished, it doesn't run on windows and the combination of YAML config and misleading error messages will make your eyes bleed. It seems almost deliberately terrible and I mean that seriously. It's clearly the product of a redhat sysadmins frustrated developer side project. Probably a hipster.
If you don't require any.of its features beyond provisioning, and you are only ever provisioning on one particular OS. For pities sake write a decent shell.script.
As of right now, the whole project reminds me of early linux forums where noobs were told to RTFM and derided for asking why someone couldn't write a GUI for configuring graphics settings. You just don't get it do you? You should stick to windows... perhaps I will mate.. happy VI-ing.
Use Docker. In preference to anything. Docker is outrageously simple and powerful.
But what if you absolutely must provision on pre-existing tin? What are the real alternatives?
Well... there aren't any, yet. But I will promise you this, unless ansible gets better, there will be soon. Because no matter how hard the fanboys push it, and forgive it's failings... it's a 5 out of 10 for effort.
SCP up a bash script, and save yourself the trouble.
One thing worth noting that you will have less issues in running your ansible playbooks on remote hosts too. As it's the main reason for running ansible. When you are using shell scripting you still need to have a way to script the scp'ing to the remote host.
