How to have Ansible move a microservice?

Question

I am working with a microservices application that is using Ansible to deploy to a fleet of VMs. We have different numbers of replicas of each microservice type running across the VM fleet. Deployments are working fine using a single deploy playbook, passing it extra variables to say which service to install, and giving naming it a group of hosts in the inventory to run against.

Occasionally we need to move microservices between hosts to rebalance load. This isn't clean with how we have set up Ansible. Moving one microservice instance means adding one new host entry, and removing one old host entry, from the inventory group. The new instance is installed but no logic will run against the removed host. We have to run another playbook to uninstall the service from the removed host.

I am looking for a way to reorganize ansible so that it will quickly move a microservice. I am thinking that we should use a single inventory group of all hosts and use a list variable to say which hosts a microservice runs on. A custom fact can be set when a service is installed. The install task can check the fact and that the current host is in the list. The uninstall task can also check the fact and that the current host isn't in the list. Is this the right way to organise the logic or is there a better way?

Answer 1

I have created a demo with a role that can install many springboot microservices on a fleet of VMs where you name a subset of the VMs where you want to place each service. The playbook can then run on all hosts, compute whether each service should be on each host, and will either run the install.yaml or uninstall.yml tasks.

To install two of the same microservices with different runtime args you play something like:

roles:
    - {
      role: "ansible-role-springboot",
      sb_app_name: "microservices-registration",
      sb_app_group_id: "org.springframework.samples.service.service",
      sb_app_artifact_id: "microservices-demo",
      sb_app_version: "2.0.0.RELEASE",
      sb_app_run_args: '"registration 8082"',
      sb_app_healthcheck_urls: [
        "http://localhost:8082/actuator/health"
      ],
      sb_hosts: [
        "my-test-vm"
      ]
    }
    - {
      role: "ansible-role-springboot",
      sb_app_name: "microservices-web",
      sb_app_group_id: "org.springframework.samples.service.service",
      sb_app_artifact_id: "microservices-demo",
      sb_app_version: "2.0.1.RELEASE",
      sb_app_run_args: '"web 8083"',
      sb_app_healthcheck_urls: [
        "http://localhost:8083/actuator/health"
      ],
      sb_hosts: [
        "my-test-vm"
      ]
    }

The role is based upon orachide/ansible-role-springboot which seems very comprehensive. The new bit is in bold which is simply a list of host that should run the microservice.

Normally we would expect to use different artefact ids and versions each time we apply the role. In the example above, it is running an official springboot microservices mega jar that takes different RUN_AS args to run as different services. That seems a bit odd and not something I would recommend yet the role is agnostic to such details.

What thing that the original role requires is that you use an executable jar file:

        <plugin>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-maven-plugin</artifactId>
            <configuration>
                <executable>true</executable>
            </configuration>
        </plugin>

It role will create a symlink to that executable and create an environment files to run it and all that good stuff.

The modifications that I made were quite simple as the original role has logic to install or uninstall based on sb_app_state:

- import_tasks: install.yml
  when: >
        sb_app_state == 'present'

- import_tasks: uninstall.yml
  when: >
        sb_app_state == 'absent'

So it was simply a matter of adding some logic to check whether the current hostname is in the sb_hosts list then update the value of sb_app_state as appropriate. That is done with a single line change.

The demo codebase uses test kitchen ansible with testinfra to unit test that the playbook works. Getting that working was well worth the effort to do automated acceptance tests of the playbook logic.