known_hosts module reports changed when nothing has changed

Question

Why does my task report say that it has changed when nothing has changed?

- hosts: localhost
  become: false
  gather_facts: false
  tasks:
    - name: Remove non existing host key from known_hosts file
      known_hosts:
        name: 192.168.122.230
        key: 192.168.122.230 ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCbi2hyrvpTRKC37NOm46n4zCPBb9r6cKk8BPrN2eppFu/0PJlB4D+nRI5epHzs5+1LhPkppbOGLC2VIRl3phMDQci3RIszhEZx4lAyX/HAkM+2zdNJlH2WWs3cbw804B4iqfCvy/Gch5ZXl4qEfpVqMURCr/XjaMQETzbjbfgOoyYxw8M/5Kq8VQy+DzqxNNzPi4ElcFQztxxrKDFPwuDplFdxw3YK+iQ4JHxlLWSfgtwsFhg7Z7uM8/efP7ocB23i2GmmG67yM/T/8uAld9t73V8icfe9WnRk2WVY69p4TzC3tMl2KmUDVm5AwvH+FNm/67E9t2inWHgKZacdOaOrgJ7SimPz0ILYDKd4hXg4whz3vdp21M/acjX3jA+fiwx6/GDIofKhyWOP3SwaiprqHZb+rWxerIOZx1IeuIRDZBH5Hjz7UlE5yg1xnqPXXzrFMj9rsKp9S5VB3HGGDfuOU7VymhZiTHIAuGM+weV6r2cOjn5HgdqkU6ABuchMAJvzaj9a3E07Rzk6h/lgWfy5VT/yl7DA7sM0/YSqKPJKgxbstoaOAZl35SDxAx978T0xlomIxaJUehRefK+G1GgPeLMmk0QtpX1dMH8bD4qvKGoLQG1qeJ4W4HrnoTsGLCxsN5/ek3rnqCekYOSiJ/q9+sZyhcLN1hwrDrrFK5fRUw==
        state: absent
      register: reg_known_hosts
     
    - name: Show known_host register
      debug:
        var: reg_known_hosts

TASK [Remove non existing host key from known_hosts file] ***************************************
changed: [localhost]

TASK [Show known_hosts register] ****************************************************************
ok: [localhost] => {
    "reg_known_hosts": {
        "changed": true,
        "diff": {
            "after": "192.168.122.230 ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIIzlnSq5ESxLgW0avvPk3j7zLV59hcAPkxrMNdnZMKP2\n",
            "after_header": "/home/sxkx/.ssh/known_hosts",
            "before": "192.168.122.230 ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIIzlnSq5ESxLgW0avvPk3j7zLV59hcAPkxrMNdnZMKP2\n",
            "before_header": "/home/sxkx/.ssh/known_hosts"
        },
        "failed": false,
        "gid": 1000,
        "group": "sxkx",
        "hash_host": false,
        "key": "192.168.122.230 ssh-rsa ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCbi2hyrvpTRKC37NOm46n4zCPBb9r6cKk8BPrN2eppFu/0PJlB4D+nRI5epHzs5+1LhPkppbOGLC2VIRl3phMDQci3RIszhEZx4lAyX/HAkM+2zdNJlH2WWs3cbw804B4iqfCvy/Gch5ZXl4qEfpVqMURCr/XjaMQETzbjbfgOoyYxw8M/5Kq8VQy+DzqxNNzPi4ElcFQztxxrKDFPwuDplFdxw3YK+iQ4JHxlLWSfgtwsFhg7Z7uM8/efP7ocB23i2GmmG67yM/T/8uAld9t73V8icfe9WnRk2WVY69p4TzC3tMl2KmUDVm5AwvH+FNm/67E9t2inWHgKZacdOaOrgJ7SimPz0ILYDKd4hXg4whz3vdp21M/acjX3jA+fiwx6/GDIofKhyWOP3SwaiprqHZb+rWxerIOZx1IeuIRDZBH5Hjz7UlE5yg1xnqPXXzrFMj9rsKp9S5VB3HGGDfuOU7VymhZiTHIAuGM+weV6r2cOjn5HgdqkU6ABuchMAJvzaj9a3E07Rzk6h/lgWfy5VT/yl7DA7sM0/YSqKPJKgxbstoaOAZl35SDxAx978T0xlomIxaJUehRefK+G1GgPeLMmk0QtpX1dMH8bD4qvKGoLQG1qeJ4W4HrnoTsGLCxsN5/ek3rnqCekYOSiJ/q9+sZyhcLN1hwrDrrFK5fRUw==",
        "mode": "0600",
        "name": "192.168.122.230",
        "owner": "sxkx",
        "path": "/home/sxkx/.ssh/known_hosts",
        "size": 97,
        "state": "file",
        "uid": 1000
    }
}

When looking at the register it says in the diff property that before and after are the same, yet it reports that it made changes?

Something else I found out is that if I completely empty my .ssh/known_hosts file and run the playbook it will say ok: [localhost].

What I'm thinking is, is that when I specify a key to remove it will look for all the keys belonging to a particular host, making sure it does not contain the key in question but the before/after still holds a key (the ed25519 key) and then marks it as changed? It looks like when removing a key, the known_hosts module is expecting that all keys will be removed.

I can work around it by using the changed_when property but I rather understand why Ansible is saying it changed when it did not change at all.

This will work around the issue I'm having:

- name: Remove non existing host key from known_hosts file
  known_hosts:
    name: 192.168.122.230
    key: 192.168.122.230 ssh-rsa ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCbi2hyrvpTRKC37NOm46n4zCPBb9r6cKk8BPrN2eppFu/0PJlB4D+nRI5epHzs5+1LhPkppbOGLC2VIRl3phMDQci3RIszhEZx4lAyX/HAkM+2zdNJlH2WWs3cbw804B4iqfCvy/Gch5ZXl4qEfpVqMURCr/XjaMQETzbjbfgOoyYxw8M/5Kq8VQy+DzqxNNzPi4ElcFQztxxrKDFPwuDplFdxw3YK+iQ4JHxlLWSfgtwsFhg7Z7uM8/efP7ocB23i2GmmG67yM/T/8uAld9t73V8icfe9WnRk2WVY69p4TzC3tMl2KmUDVm5AwvH+FNm/67E9t2inWHgKZacdOaOrgJ7SimPz0ILYDKd4hXg4whz3vdp21M/acjX3jA+fiwx6/GDIofKhyWOP3SwaiprqHZb+rWxerIOZx1IeuIRDZBH5Hjz7UlE5yg1xnqPXXzrFMj9rsKp9S5VB3HGGDfuOU7VymhZiTHIAuGM+weV6r2cOjn5HgdqkU6ABuchMAJvzaj9a3E07Rzk6h/lgWfy5VT/yl7DA7sM0/YSqKPJKgxbstoaOAZl35SDxAx978T0xlomIxaJUehRefK+G1GgPeLMmk0QtpX1dMH8bD4qvKGoLQG1qeJ4W4HrnoTsGLCxsN5/ek3rnqCekYOSiJ/q9+sZyhcLN1hwrDrrFK5fRUw==
    state: absent
  register: reg_known_hosts
  changed_when: reg_known_hosts.diff.before != reg_known_hosts.diff.after

Ansible version: ansible [core 2.13.2]

What is going on here?

UPDATE - 18-Aug-2022

I will start by saying that I'm new to Ansible and I'm not a Python programmer.

When reading the source code for the known_hosts module I found the following.

Line: 228

def search_for_host_key(module, host, key, path, sshkeygen):
  '''search_for_host_key(module,host,key,path,sshkeygen) -> (found,replace_or_add,found_line)
  Looks up host and keytype in the known_hosts file path; if it's there, looks to see
  if one of those entries matches key. Returns:
  found (Boolean): is host found in path?
  replace_or_add (Boolean): is the key in path different to that supplied by user?
  found_line (int or None): the line where a key of the same type was found
  if found=False, then replace is always False.
  sshkeygen is the path to ssh-keygen, found earlier with get_bin_path
  '''
# ...

Looking at the comment at the top of the function I can conclude that:

• found will equal True because I have a key for that host (key of type ed25519).
• replace_or_add will equal True because the key is different from the one found (key of type ed25519).
• found_line will be None because no key of the same type was found.

With that in mind I think we can have a look at the enforce_state function.

Line: 117

def enforce_state(module, params):

  # ...

  # Next, add a new (or replacing) entry
  if replace_or_add or found != (state == "present"):
    try:
      inf = open(path, "r")
    except IOError as e:
      if e.errno == errno.ENOENT:
        inf = None
      else:
        module.fail_json(msg="Failed to read %s: %s" % (path, str(e)))
    try:
      with tempfile.NamedTemporaryFile(mode='w+', dir=os.path.dirname(path), delete=False) as outf:
        if inf is not None:
          for line_number, line in enumerate(inf):
            if found_line == (line_number + 1) and (replace_or_add or state == 'absent'):
              continue  # skip this line to replace its key
            outf.write(line)
          inf.close()
        if state == 'present':
          outf.write(key)
    except (IOError, OSError) as e:
      module.fail_json(msg="Failed to write to file %s: %s" % (path, to_native(e)))
    else:
      module.atomic_move(outf.name, path)

    params['changed'] = True

I think the culprit lies within this if block. For what I could make up from the comment in the previous function; replace_or_add is True, found is True and not equal to state == "present". Within this if block it will read the known_hosts file and loops over the lines and when a line (number) matches the found_line it will continue the loop, otherwise it will write that line to (a temporary) file. However no matter what it does within the loop, later on in the if block it will always set params['changed'] = True, meaning it will always report changed regardless of there actually being changes. A possible solution to this could be a counter to keep track of the number of times the loop was continued and then set the params variable property like so: params['changed'] = True if counter > 0 else False.

Something else I think is happening (regardless of changes) is that it will perform a write operation. If so then yes it would make some sense (would it?) that params['changed'] is set to True but I would much rather see the module only write out to .ssh/known_hosts if something actually changed.

UPDATE - 19-Aug-2022

This portion is not related to the state always being changed but rather something to do with the hash_host option in the known_hosts module, either your hostname or IP is lost and not set in the known_hosts file.

Let's assume the following entry is in my known_hosts file.

host.local,192.168.122.230 ssh-rsa <public_key>

When hashing the known_hosts file with ssh-keygen -H it represents the line above like so.

|1|<salt>|<hash> ssh-rsa <public_key>
|1|<salt>|<hash> ssh-rsa <public_key>

Where the first line is for host.local and the second line is for 192.168.122.230.

When attempting the same thing with the known_hosts module in Ansible something different will happen.

- name: add host key to known_hosts file
  known_hosts:
    name: host.local
    key: host.local,192.168.122.230 ssh-rsa <public_key>
    state: present

Will result in the following entry in the known_hosts file.

host.local,192.168.122.230 ssh-rsa <public_key>

However when you enable hashing things will change.

- name: add host key to known_hosts file
  known_hosts:
    hash_host: yes
    name: host.local
    key: host.local,192.168.122.230 ssh-rsa <public_key>
    state: present

|1|<salt>|<hash> ssh-rsa <public_key>

It will have only hashed the entry for host.local the IP version of that line is just gone. This is something to keep in mind when you want to use both the hostname and IP to access the target.

UPDATE: 24-Aug-2022

Opened an issue over on the Ansible github.

If you find any information to be incorrect please let me know or edit this post.

Answer 1

It seems that the issue you've filed, Ansible Issue #78598 was noticed. It was possible to reproduce, got a small fix and an easyfix label.

Since further verification and testing is outstanding, you might be able to add the three proposed lines of code into your own known_host.py and test it. You could provide after testing an update on the Ansible Issue.