Running Ansible no-op's faster with a dynamic inventory and cached facts files

Question

I've got an ever expanding playbook that patches a few hundred machines to the latest version, I just add the same role with different parameters.

Obviously, the playbook is taking longer and longer to run on each machine, even though it's not really doing anything on them.

What I'm doing is in the pre-tasks, I'm getting a changelog file, checking that and running each of the roles when an identifier that a successful run puts in that changelog file does not exist. But, even so, ansible still takes a fairly onerous amount of time to do nothing.

When I strace the main ansible process it looks like its trying to hit a bunch of non-existent cached facts files and the inventory script.

I think I've created the dynamic inventory correctly, it accepts both the "--host" and "--list" parameters (although I'm not certain that --host is being called).

I've got about 100 devices that I haven't gathered facts for in the inventory file and I've got:

fact_caching_connection = /tmp/factscache
fact_caching = jsonfile

set up.

If I take that out, it doesn't try to hit up the missing facts files, but it there is a stat in the strace against the inventory generator file over and over again. I know it can't be generating the entire inventory because that take 5 seconds.

I've tried generating regular inventory files and using those, I've tried generating inventory files with both _meta and the regular host_vars (currently using _meta to house the host vars because I heard that was faster).

(running ansible 2.0)

Answer 1

Instead of storing the Ansible code centrally, one could extract the code and distribute it to every project. Another benefit of this approach is that config and code will be seen together and that developers will change config and Ops could fix e.g. bugs instead of working in silos. Running the code per project will prevent that the executions will take longer, longer and longer.

Answer 2

You could isolate the roles and run just them, you can generate a static inventory list before you run the particular role(and it's associated task).

Make sure that the task doesn't have a whole lot of inheriting/dependant roles. As 030 mentioned, it might be a good idea to implement some kind of way to semantically separate the roles.