Automating Your Ansible Deployment Using Shell Scripts

For the past 5 or so years I've been using Ansible to do all of my deployments.  It's a great tool to do everything from installing your operating system packages and installing postgres to deploying you Django code.  For the majority of projects, I use Jenkins to run my deployment process.  Jenkins is stable, and my co-founder open sourced this great project that makes setting up a Jenkins server easy.  For me, it doesn't make sense to use something like CircleCI or TravisCI as the costs are very high for what I can do with Jenkins.  I can use the above project to setup a Jenkins server on a $5.00 Digital Ocean instance.  There's very little maintenance involved and when I need to do an upgrade, it's very little work.

That being said, there are times when having a Jenkins server doesn't make sense either.  I have some very small personal projects that I don't deploy very often.  In these cases, it makes sense to run Ansible commands by hand.  Lately I've been doing a lot of maintenance and upgrades to this blog and I've been deploying a lot.  Everytime I deploy I have to grab my Ansible Vault password and I have to make sure I remember the specific command needed to run the Ansible job.  This was starting to get annoying so I decided to write a small script to automate the process.  I thought it would be pretty straight forward to set this up, however there were a few hurdles I ran into.  The main one being that I wanted to store my ansible-vault password in an environment variable.  Here's what I had to do to get everything setup.

What is Ansible Vault?

Ansible Vault allows you to encrypt your Ansible files.  When pushing your code to an online repo, it's a good idea to keep any sensitive information private.  Even if your repo is private, you may not want other people, who have access to that repo, to see any sensitive information.  You also don't know what the owners of the online repo you're using can see.  As a general practice, I like to encrypt my production Ansible file.  I typically keep my server passwords in this file, as well as other sensitive information needed to configure, login and deploy my production servers.  I typically have a single production YAML file that I keep all of my production details in.  I then encrypt this file and keep the password for the file in a safe place.

This is where things get a little annoying when deploying by hand.  Every time I want to deploy, I have to look up this password and provide it to Ansible.  I want this fully automated and I don't want the password stored in a place that could accidently get committed to my repo.  So what I've decided to do is keep the password in a environment variable for me to read when I run the Ansible script.  If you're using Ansible and are not using Ansible Vault, I highly recommend you get started with it.  For more details on how it works, check it out here. 

Deployment Script

Now, on to my deployment script.  I thought this would be a pretty easy script to setup.  I thought I could just create a bash script, enable my Ansible virtualenv and then pass the environment variable to Ansible.  Unfortunately, Ansible doesn't allow this and you can't easily pass the environment variable to the Ansible Valut password prompt.  They recommend that you use a file with your password in it to automate this process.  You can read more about that here.

There's three things that we need to do here.  Because of some of the limitations, I've decided to use a text file with my password in it to decrypt my ansible-vault encrypted file when deploying.  The one requirement I have is I don't want the password to be stored in the plan text file and stored in my repo.  In order to keep with these requirements, this is what I've done:

1. I created a shell script to read my environment variable and echo it.
2. I've configured my ansible.cfg to reference the shell script that's printing the password found in my environment variable.
3. I've created a deployment script that calls the Ansible playbook

That's it, 3 simple files to deploy my software.  It's a bit more complicated than I want, but once you see the contents of the files you'll understand.  I could have spent more time trying to figure out how to get Ansible to read the vault password from the command line but once I found this solution, it was straightforward enough that I didn't think it was worth spending more time on.

Ansible cfg file

This is my ansible.cfg file and you should already have this in place.  To make things simple, we're going to add the name of the script that'll echo the password to the screen.  You do this by setting the value_passowrd_file in the defaults section.  This is what my file looks like

[defaults]
host_key_checking=False
vault_password_file=../scripts/print_vault_pass.sh

[ssh_connection]
ssh_args = -o ForwardAgent=yes -o ControlMaster=auto -o ControlPersist=60s

As you can see I have the vault_password_file set to the location of the script that prints my vault password.  

Vault Password File

Since I do not want the password to be exposed in the file, I've set my password file to be a shell script.  I use virtualenv to manage my Ansible install.  So the shell script is going to activate my virtualenv and echo the password so Ansible can read it.

#!/bin/bash

# Enable the Virtual Environment that has Ansible installed
source ~/.venv/ansible2/bin/activate
source ~/.venv/ansible2/bin/postactivate

echo $ANSIBLE_PASSWORD

As you can see, this script is pretty straight forward.  It just activates Ansible and displays the password on the console (which you can't see when running the script) so Ansible can read it to decrypt your files.

Deployment Script

The final piece to our puzzle is the actual deployment script.  As you may have guessed, we're going to activate the virtualenv that has Ansible installed.  Then we simply run the ansible-playbook command to deploy our software.  Having the password file in the ansible.cfg file helps keep this script to a few simple lines.

#!/bin/bash

set -e

printf "\n\n>>> Deploying to the Production server...\n\n"

# Activate Ansible environment.
source ~/.venv/ansible2/bin/activate
source ~/.venv/ansible2/bin/postactivate

cd ansible
ansible-playbook -i production site.yml --tags="deploy,cron" 

All we're doing here is activating the virtualenv again, and calling the playbook.  My production file is encrypted and the playbook will use the password file to decrypt my Ansible files and deploy my code.   

I think there are some improvements that can be made but for me this is secure and easy enough that I no longer have to worry about remembering the exact command and finding my vault password.  The password is hidden away, and there's no chance it can accidently get committed with any new code push.  I think this is a secure and safe way to automatically deploy your code using Ansible Vault.

If your curious about our deployment process or Ansible in general you can check out the open source Ansible configuration we use for all of our apps.  My co-founder open sourced this project and it's gotten some traction.  There's a healthy community keeping it up to date and improving it so feel free to check it out!