Ansible - merging inventories to share variables across?

Question

Is it possible to merge inventories so I can share variables across them? Let's say I have two inventory files. Inventory A and Inventory B. They are both in a directory called inventories.

Variables are defined in B and I want to reference them in inventory A. Is this possible? I've heard of merging inventories so share variables across but I haven't gotten this to work.

Here's are the files in my inventories directory: /opt/ansible/inventories/aws_ec2.yaml /opt/ansible/inventories/secrets.yaml

contents of aws_ec2.yaml:

---
plugin: aws_ec2

aws_access_key: "{{ aws_access_key }}"
aws_secret_key: "{{aws_secret_key}} "
aws_security_token: "{{aws_security_token}}"

regions:
  - us-west-2

keyed_groups:
  - key: tags
    prefix: tag
  - key: placement.region
    prefix: aws_region

groups:
  TEST: "'prod' in (tags:list)"

compose:
  ansible_host: private_ip_address

Here is my secrets.yaml file:

---
  
aws_access_key: xxxxx
aws_secret_key: xxxxx
aws_security_token: xxxxx

When I run my playbook, referencing both inventories, inventory does not seem to occur and I get these errors. It looks like the variables aren't being referenced and expanded?:

me@workstation:/opt/ansible/inventories$ ansible-playbook -i /opt/ansible/inventories/aws_ec2.yaml -i /opt/ansible/inventories/secrets.yaml /opt/ansible/playbooks/test.yaml
[WARNING]:  * Failed to parse /opt/ansible/inventories/aws_ec2.yaml with aws_ec2 plugin: Failed to
describe instances: An error occurred (AuthFailure) when calling the DescribeInstances operation:
Credential must have exactly 5 slash-delimited elements, e.g. keyid/date/region/service/term, got '{{'
[WARNING]: Unable to parse /opt/ansible/inventories/aws_ec2.yaml as an inventory source
[WARNING]: Unable to parse /opt/ansible/inventories/secrets.yaml as an inventory source
[WARNING]: No inventory was parsed, only implicit localhost is available
[WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost
does not match 'all'

Or is my syntax for variable assigning and referencing wrong? Am I on the right path here? my /etc/ansible/ansible.cfg file, under [defaults] is as such:

[defaults]
inventory = /opt/ansible/inventory/aws_ec2.yaml

Answer 1

While I haven't worked with plugin: aws_ec2, here's some general Ansible information that might help you:

The order of inventory files matter; the files given first will be read first; the ones read last will take precedence. In your execution, you're referencing secrets.yaml last, so when aws_ec2.yaml is read, the secrets are still undefined.

You're trying to define a variable based on another variable with the same name. That's asking for trouble; rename the reference variables, as that will help you debugging.

When defining variables for the inventory, it's generally better to not do that directly on the inventory itself. See the link below on how to do that:

https://docs.ansible.com/ansible/latest/user_guide/intro_inventory.html#organizing-host-and-group-variables

Finally, keyed_groups, regions and all these other keywords are specific to aws_ec2, and they'd work because you have plugin: aws_ec2 at the top of the first file. You have no such definition on secrets.yaml, so Ansible would probably treat it as a normal inventory file, where that structure would not be accepted. Your approach here is probably wrong, then.

Again, I never used that inventory plugin. It may be the case that the inventory plugin does not allow for variables at that point.

Answer 2

Notice that what you're trying to do is not merging inventories. Merging inventories happens when you have two fully defined inventories and you try to, well, merge them. So, you have an inventory with a set of defined hosts, groups and variables and you combine that with another inventory with its own hosts, groups and variables.

In your example, what you're trying to do is to split the definition of a single inventory. The aws_ec2 plugin requires some information so it goes to Amazon and say, 'hey take this info and give me back my inventory'. You want part of that information in one file, part in another one.

Once you're using a plugin, the inventory's parsing will be controlled by the plugin, so I don't think there are any guarantees that stuff that works elsewhere (like variables) will work here.

You can test that; run ansible-playbook without secrets.yaml and set each of the variables as extra vars using -e:

ansible-playbook \
 -i /opt/ansible/inventories/aws_ec2.yaml  \
 -e aws_access_key=xxx \
 -e aws_secret_key=xxx \
 -e aws_security_token=xxx \
 /opt/ansible/playbooks/test.yaml

If that works, then you have validated that you can use Ansible variables in that place, and you just need to find a proper place to put them. I'm not sure where exactly to put that, though.

If you don't mind to put that data elsewhere, the plugin accepts those values you're trying to separate as environment variables:

ansible-doc -t inventory aws_ec2

- aws_access_key
    The AWS access key to use.
    (Aliases: aws_access_key_id)[Default: (null)]
    set_via:
      env:
      - name: EC2_ACCESS_KEY
      - name: AWS_ACCESS_KEY
      - name: AWS_ACCESS_KEY_ID
    
    type: str