5

Recently I came across a bottleneck in our ansible playbooks' code. We were deploying our clusters (e.g. a mongoDB Replica Set) sequentially - i.e. one VM after another, each waiting for the previous to be up and running.

This slowed down the whole cluster deploy time by a factor of the members on it.

To solve this, I started digging on ansible's async actions and pooling and found out a few examples on parallel loops and "fire-and-forget" strategies for scenarios like ours.

The particular thing is, we have defined our own "customize the VM and spawn it" ansible task (create_instance.yml) that gets included and receives the different customization variables from the playbook and abstracts the whole process by running different KVM/shell commands.

Using "Parallel task execution in Ansible" as reference, I ended up having something like:

- name: Generate VMs for DB
  hosts: hypervisor_fe
  tags: platform,mongodb
  tasks:
    - include: tasks/create_instance.yml
      vars:
        vm: "{{ item }}"
      with_items: "{{ mongodb.vms }}"
      register: mongo_instances
      async: 7200
      poll: 0

    - name: Wait for instance creation to complete
      async_status: jid={{ item.ansible_job_id }}
      register: mongo_jobs
      until: mongo_jobs.finished
      retries: 300
      with_items: "{{ mongo_instances.results }}"

However, this setup does seem to ignore all the new async code and keeps the old, sequential behavior. I'm guessing this has to do with the no. and granularity of plays inside the imported task. If I instead replace the include for a single, explicit long-running task - let's say, e.g.

- name: Test async operation
  shell: ping -c1 {{ item.hostname }} && sleep 20

This does seem to work just fine, running one ping to each item and then moving on to the next action.

Is this assumption right? Does someone has experience with include and async loops in ansible? Do I need to move the async declaration to a single play inside the imported code?

1

1 Answer 1

4

I advise you to rethink your playbook design in the following way:

- hosts: localhost
  tasks:
    - add_host:
        name: "{{ item.name }}"
        groups: new_vms
        vm: "{{ item }}"
      with_items: "{{ mongodb.vms }}"

- hosts: new_vms
  tasks:
    - include: create_instance.yml

And inside create_instance.yml use delegate_to: hypervisor_fe.

This gives you native Ansible host loop for every vm with concurrent execution of each task.

2
  • Won't this fail with "unreachable" when trying to gather the facts of new_vms (not yet created)? Is there a way to apply the delegate_to to all tasks on the include? (~30)
    – Alfageme
    Nov 13, 2018 at 10:33
  • 2
    You can disable it with gather_facts: no and collect them later with setup module. You can apply delegate_to: localhost to include statement or add block into your task file and apply delegate_to to the block. Nov 13, 2018 at 10:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.