0

I am trying to look up the IPv4 address (a single one, which I will assert on!) for a given name using Ansible and use that as a fact "down the line".

An MWE looks like this:

---
# vim: set autoindent smartindent tabstop=2 shiftwidth=2 softtabstop=2 expandtab filetype=ansible:

- name: Preparations
  hosts: localhost
  connection: local
  vars:
    host2lookup: "example.com"
  tasks:
    - getent:
        database: ahostsv4
        key: "{{host2lookup}}"
    - debug:
        var: getent_ahostsv4
    - name: Lookup for {{host2lookup}}
      debug:
        msg: "{{lookup('dig', '{{host2lookup}}./A')}}"

... and fails as follows (relevant excerpt):

  localhost failed | msg: An unhandled exception occurred while running the lookup plugin 'dig'. Error was a <class 'ansible.errors.AnsibleError'>, original message: The dig lookup requires the python 'dnspython' library and it is not installed

So I thought to myself I'd install the dnspython library for the appropriate Python version, making the playbook look as follows:

---
# vim: set autoindent smartindent tabstop=2 shiftwidth=2 softtabstop=2 expandtab filetype=ansible:

- name: Preparing Ansible host
  hosts: localhost
  connection: local
  tasks:
    - name: Install dnspython package on Ansible host
      block:
        - name: Installing python-dnspython package
          apt:
            pkg:
              - python-dnspython
            update_cache: yes
            install_recommends: no
            state: present
          when: ansible_python.version.major == 2
        - name: Installing python3-dnspython package
          apt:
            pkg:
              - python3-dnspython
            update_cache: yes
            install_recommends: no
            state: present
          when: ansible_python.version.major == 3
      become: true
- name: Preparations
  hosts: localhost
  connection: local
  vars:
    host2lookup: "example.com"
  tasks:
    - getent:
        database: ahostsv4
        key: "{{host2lookup}}"
    - debug:
        var: getent_ahostsv4
    - name: Lookup for {{host2lookup}}
      debug:
        msg: "{{lookup('dig', '{{host2lookup}}./A')}}"

But the ultimate failure remains the same. Either that error is misleading or something else is fishy ...

Executing playbook dbg.yml

- Preparing Ansible host on hosts: localhost -
Gathering Facts...
  localhost ok
Installing python-dnspython package...
Installing python3-dnspython package...
  localhost ok

- Preparations on hosts: localhost -
Gathering Facts...
  localhost ok
getent...
  localhost ok
debug...
  localhost ok: {
    "changed": false,
    "getent_ahostsv4": {
        "93.184.216.34": [
            "RAW"
        ]
    }
}
Lookup for example.com...
  localhost failed | msg: An unhandled exception occurred while running the lookup plugin 'dig'. Error was a <class 'ansible.errors.AnsibleError'>, original message: The dig lookup requires the python 'dnspython' library and it is not installed

- Play recap -
  localhost                  : ok=5    changed=0    unreachable=0    failed=1    rescued=0    ignored=0

What other than the appropriate dnspython package am I missing here?


The dnsutils package (containing dig) and its library dependencies is installed. Adding these as prerequisites into the apt: task therefore makes no difference.

Found another smoking gun, though:

$ python3 -m dnspython
/usr/bin/python3: No module named dnspython
$ sudo -H pip3 install dnspython
Requirement already satisfied: dnspython in /usr/lib/python3/dist-packages

I stand corrected. As per the examples page I should import something like dns.query (python3 -m dns.query). And that succeeds.

My ansible.cfg is this:

$ cat ansible.cfg
[defaults]
nocows = true
command_warnings = false
stdout_callback = unixy
display_skipped_hosts = false
interpreter_python=auto

The whole thing runs on Ubuntu 18.04.


As per request in a comment, I am providing more output. This is still on Ubuntu 18.04. With the (above shown) ansible.cfg in place, I get the following output:

$ ansible localhost -m setup -a 'filter=ansible_python*'
localhost | SUCCESS => {
    "ansible_facts": {
        "ansible_python": {
            "executable": "/usr/bin/python2",
            "has_sslcontext": true,
            "type": "CPython",
            "version": {
                "major": 2,
                "micro": 17,
                "minor": 7,
                "releaselevel": "final",
                "serial": 0
            },
            "version_info": [
                2,
                7,
                17,
                "final",
                0
            ]
        },
        "ansible_python_version": "2.7.17"
    },
    "changed": false
}

However, ansible-playbook behaves differently here. Consider the following playbook:

---

- hosts: all
  tasks:
    - debug:
        var: ansible_python
    - debug:
        var: ansible_python_version

The output becomes:

$ ansible-playbook -i localhost, -c local dbg.yml
Executing playbook dbg.yml

- all on hosts: all -
Gathering Facts...
  localhost ok
debug...
  localhost ok: {
    "ansible_python": {
        "executable": "/usr/bin/python3",
        "has_sslcontext": true,
        "type": "cpython",
        "version": {
            "major": 3,
            "micro": 9,
            "minor": 6,
            "releaselevel": "final",
            "serial": 0
        },
        "version_info": [
            3,
            6,
            9,
            "final",
            0
        ]
    },
    "changed": false
}
debug...
  localhost ok: {
    "ansible_python_version": "3.6.9",
    "changed": false
}

- Play recap -
  localhost                  : ok=3    changed=0    unreachable=0    failed=0    rescued=0    ignored=0

... comment out the following line from ansible.cfg:

interpreter_python=auto

... and the output (when rerunning the previous command) becomes:

$ ansible-playbook -i localhost, -c local dbg.yml
Executing playbook dbg.yml

- all on hosts: all -
Gathering Facts...
  localhost ok
debug...
  localhost ok: {
    "ansible_python": {
        "executable": "/usr/bin/python",
        "has_sslcontext": true,
        "type": "CPython",
        "version": {
            "major": 2,
            "micro": 17,
            "minor": 7,
            "releaselevel": "final",
            "serial": 0
        },
        "version_info": [
            2,
            7,
            17,
            "final",
            0
        ]
    },
    "changed": false
}
debug...
  localhost ok: {
    "ansible_python_version": "2.7.17",
    "changed": false
}

- Play recap -
  localhost                  : ok=3    changed=0    unreachable=0    failed=0    rescued=0    ignored=0

Nothing in the documentation suggests that this should happen.

  • Update the question and add the output of the command ansible localhost -m setup | grep ansible_python_version – Vladimir Botka Dec 18 '19 at 13:33
  • @VladimirBotka I honestly appreciate your effort, but what does the ansible command line tool have to do with behavior I see with ansible-playbook? I will amend the question shortly. You'll notice that the two tools behave differently in this exact case. No clue why, but it can be reproduced. – 0xC0000022L Dec 18 '19 at 13:42
  • @VladimirBotka amended the question with requested information ... and a little more, since the information you requested isn't representative of the issue I am encountering. – 0xC0000022L Dec 18 '19 at 13:58
  • You did not amend the question with the requested information. I asked you to use -hosts: localhost, but you used - hosts: all. This is really little more. – Vladimir Botka Dec 18 '19 at 14:15
  • @VladimirBotka Huh? Which part of ansible localhost -m setup -a 'filter=ansible_python*' (other than using a filter instead of grep) lacks the requested information? Your request was for output from ansible and I pointed out the discrepancy with that of ansible-playbook based on the interpreter_python=auto setting. – 0xC0000022L Dec 18 '19 at 14:23
1

Q: "None of that explains why - and that is/was my point - ansible_python wrongly claims to be running on Python 3.x on the local node. Because all I do by setting interpreter_python in ansible.cfg is to give Ansible free rein on how it interprets things."

A: Configuration option interpreter_python works as expected i.e. there is nothing wrong when Python3 is discovered (when asked by interpreter_python=auto). See Note 1) and examples below.

"The default value of auto_legacy provides all the same behavior, but for backwards-compatibility with older Ansible releases that always defaulted to /usr/bin/python, will use that interpreter if present (and issue a warning that the default behavior will change to that of auto in a future Ansible release."

interpreter_python=auto_legacy
$ ansible localhost -m setup -a 'filter=ansible_python_version'
[DEPRECATION WARNING]: Distribution Ubuntu 19.04 on host localhost should use /usr/bin/python3, but is using /usr/bin/python for backward compatibility with prior Ansible releases. A future
 Ansible release will default to using the discovered platform python for this host. See https://docs.ansible.com/ansible/2.9/reference_appendices/interpreter_discovery.html for more 
information. This feature will be removed in version 2.12. Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg.
localhost | SUCCESS => {
    "ansible_facts": {
        "ansible_python_version": "2.7.16", 
        "discovered_interpreter_python": "/usr/bin/python"
    }, 
    "changed": false
}
  • auto

See implementation details of the function discover_interpreter.

When the version of the distro is not in INTERPRETER_PYTHON_DISTRO_MAP

"return nearest previous version we're newer than"

def _version_fuzzy_match(version, version_map):
...
    # slot match; return nearest previous version we're newer than
    kpos = bisect.bisect(sorted_looseversions, find_looseversion)
interpreter_python=auto
$ ansible localhost -m setup -a 'filter=ansible_python_version'
localhost | SUCCESS => {
    "ansible_facts": {
        "ansible_python_version": "3.7.3", 
        "discovered_interpreter_python": "/usr/bin/python3"
    }, 
    "changed": false
}

Q: "What does the ansible command line tool have to do with the behavior I see with ansible-playbook?"

A: Both ansible command line tool and ansible-playbook will report the same value of ansible_python_version. For example

$ ansible localhost -m setup | grep ansible_python_version
        "ansible_python_version": "2.7.16",
$ cat playbook.yml
- hosts: localhost
  tasks:
    - debug:
        var: ansible_python_version

$ ansible-playbook playbook.yml

ok: [localhost] => {
    "ansible_python_version": "2.7.16"
}


Notes

1) interpreter_python configuration parameter says:

Path to the Python interpreter to be used for module execution on remote targets

See how to choose a python interpreter for Ansible playbook?

2) Ansible will tell what Python is used on master

$ ansible --version
ansible 2.9.2
  config file = /home/vlado/.ansible.cfg
  configured module search path = [u'/home/vlado/.ansible/my_modules']
  ansible python module location = /usr/lib/python2.7/dist-packages/ansible
  executable location = /usr/bin/ansible
  python version = 2.7.16 (default, Oct  7 2019, 17:36:04) [GCC 8.3.0]

3) dig says

The below requirements are needed on the local master node that executes this lookup: dnspython

4) It's possible to install both versions

$ dpkg -l | grep dnspython
ii  python-dnspython         1.16.0-1 all          DNS toolkit for Python
ii  python3-dnspython        1.16.0-1 all          DNS toolkit for Python 3

5) All Ansible packages in Ubuntu have been built with Python2 at the moment.

6) ansible_python_interpreter works as expected

With Ubuntu on master the playbook

$ cat playbook.yml
- hosts: test_01
  tasks:
    - getent:
        database: hosts
$ cat hosts
test_01 ansible_python_interpreter=/usr/local/bin/python3.6
$ ansible-playbook playbook.yml -i hosts -vvv | grep python

gives

ansible python module location = /usr/lib/python2.7/dist-packages/ansible python version = 2.7.16 (default, Oct 7 2019, 17:36:04) [GCC 8.3.0]

...

SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="admin"' -o ConnectTimeout=10 -o ControlPath=/export/home/vlado.config/.ansible/cp/5a3ab05cf7 -tt test_01 '/bin/sh -c '"'"'sudo -H -S -n -u root /bin/sh -c '"'"'"'"'"'"'"'"'echo BECOME-SUCCESS-xnvwkdksdqqoehvrgebdgcnrzcrmoszv ; /usr/local/bin/python3.6 /home/admin/.ansible/tmp/ansible-tmp-1576667662.62-67468482045094/AnsiballZ_getent.py'"'"'"'"'"'"'"'"' && sleep 0'"'"''

  • ansible-playbook on master runs with python version = 2.7.16
  • module on remote (test_01) runs with /usr/local/bin/python3.6

Change ansible_python_interpreter

When the ansible_python_interpreter is changed

$ cat hosts
test_01 ansible_python_interpreter=/usr/local/bin/python2.7

the same playbook gives

ansible python module location = /usr/lib/python2.7/dist-packages/ansible python version = 2.7.16 (default, Oct 7 2019, 17:36:04) [GCC 8.3.0]

...

SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="admin"' -o ConnectTimeout=10 -o ControlPath=/export/home/vlado.config/.ansible/cp/5a3ab05cf7 -tt test_01 '/bin/sh -c '"'"'sudo -H -S -n -u root /bin/sh -c '"'"'"'"'"'"'"'"'echo BECOME-SUCCESS-fidpgwptggdoizirumvjmzuorxqawypu ; /usr/local/bin/python2.7 /home/admin/.ansible/tmp/ansible-tmp-1576668475.22-210929624017605/AnsiballZ_getent.py'"'"'"'"'"'"'"'"' && sleep 0'"'"''

  • ansible-playbook on master runs with python version = 2.7.16
  • but module on remote (test_01) runs with /usr/local/bin/python2.7 as requested
| improve this answer | |
  • Yes, but none of that explains why - and that is/was my point - ansible_python wrongly claims to be running on Python 3.x on the local node. Because all I do by setting interpreter_python in ansible.cfg is to give Ansible free rein on how it interprets things. But since I am depending on its choices and its information being consistent (which it evidently isn't), things blow up in my face. Regarding point 4, it's completely beside the point, because I am attempting to install the appropriate version based on ansible_python, but Ansible simply lies to me ... – 0xC0000022L Dec 18 '19 at 9:01
  • There is no evidence that ansible_python wrongly claims to be running on Python 3.x on the local node. I've added more details that might help to clarify the situation. – Vladimir Botka Dec 18 '19 at 11:51
  • The evidence is the behavior one gets with when: ... the wrong package that gets installed (provided neither, i.e. python-dnspython/python3-dnspython, was installed before). This is actually reproducible. But you're right, this only happens with interpreter_python=auto set in ansible.cfg for me. But then I consistently see it (and I have actually used the debug module to verify the value of ansible_python is wrong). Only if python-dnspython is preinstalled (samba installs it, for example) don't I see the issue. Otherwise I can reproduce on all 18.04 so far. – 0xC0000022L Dec 18 '19 at 13:19
  • Just to make sure. I understand that the lookup plugins get called on the control/master. In my real-world case the Preparing Ansible host part is getting actually executed on the control/master, but the Preparations part gets executed on the remote machine(s). With the sole exception of the loopkup plugin, so I understand that the dnspython package needs to be installed on control/master and that the result of the lookup is passed (by value) to the remote machine(s). In my MWE I had to condense this for obvious reasons. – 0xC0000022L Dec 18 '19 at 13:33
  • A: Both ansible command line tool and ansible-playbook will report the same value of ansible_python_version. ... unfortunately not true, at least not unconditionally (see amendments to the question). – 0xC0000022L Dec 18 '19 at 14:12
0

Since this confused the hell out of me and cost me about an hour to figure out, I am going to post an answer.

As you can see I am using ansible_python to determine the Python version and install the appropriate version of dnspython. Since this is on Ubuntu 18.04 it defaults to python3. I am suppressing the respective warning with interpreter_python=auto in the ansible.cfg inside the same folder.

But here's the bummer. Apparently connection: local somehow violates the rule about using python3 whose dnspython package is installed. It made sense once I looked for dig.py (hoping for the Ansible lookup module of the same name). And indeed I found it:

$ locate dig.py
/usr/lib/python2.7/dist-packages/ansible/plugins/lookup/dig.py
/usr/lib/python2.7/dist-packages/ansible/plugins/lookup/dig.pyc

... alas, this means my version of Ansible uses Python 2.7 and so the dig module won't be able to make use of the python3-dnspython (Debian) package which I installed.

Still puzzling. But with the cause determined, I can work around it now.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.