Downloading Ansible galaxy roles in parallel

‹ letsencrypt / certbot incomplete certificate chain error | Decoding binary data in logstash ›

To always ensure that I have the latest version of a role from Ansible Galaxy, I've found I needed to use both the --force option (to re-download even if the directory already exists) and the --ignore-errors option (to keep going through all the roles). Overall, the command line ends up looking like this:

ansible-galaxy install --force --ignore-errors -r priv/ansible/requirements.txt -p priv/ansible/roles/

The downside is the downloads are serialized and take a long time to complete.  As of this writing, 10 seconds of searching the internet didn't uncover a better solution within Ansible, but slightly more than 10 seconds of scripting lets you run the downloads in parallel by creating many individual files within a requirements/ directory and running ansible-galaxy many times in the background using a shell script.  Replace *.txt with *.yml if you've exited the Ansible ice age and have switched to .yml for galaxy requirements files:

allpids=""; \
for i in priv/ansible/requirements/*.txt; do \
  ansible-galaxy install --force --ignore-errors \
    -r "$i" -p priv/ansible/roles/ & \
  pid=$!; \
  allpids="$allpids $pid"; \
done; \
for pid in $allpids; do \
  wait $pid; \

Subscribe to All Posts - Wesley Tanaka