Multi Environment Ansible Layout – Part 2: The Inventory

In yesterday’s post I did an overview of the Ansible repository layout.

Today, we’re gonna dive into the inv/ (inventory) folder.

.
├── group_vars/
│   ├── all/
│   │   ├── 10-global-secrets.yml
│   │   └── 10-global-variables.yml
│   ├── integration/
│   │   ├── 10-integration-secrets.yml
│   │   └── 10-integration-variables.yml
│   └── production/
│       ├── 10-production-secrets.yml
│       └── 10-production-variables.yml
├── host_vars/
│   └── backup.example.com/
│       └── 10-ansible.yml
├── int/
│   ├── ec2.ini
│   ├── ec2.py -> ../.shared/ec2.py
│   ├── group_vars -> ../group_vars/
│   ├── host_vars -> ../host_vars/
│   ├── localhost -> ../.shared/localhost
│   └── mapping.yml
├── prod/
│   ├── ec2.ini
│   ├── ec2.py -> ../.shared/ec2.py
│   ├── group_vars -> ../group_vars/
│   ├── host_vars -> ../host_vars/
│   ├── localhost -> ../.shared/localhost
│   └── mapping.yml
└── .shared/
    ├── ec2.py
    └── localhost

Reminder about terms

As I mentioned in yesterday’s post, there are two new variables/terms I’ve created for the environments:

  • environ — a "short" term for an environment (aka int or stage).

  • environment_long — the longer version of an environment (aka integration or production).

group_vars and host_vars

We create a single location for group_vars in the inv/ folder, this folder should contain subfolders one for all/ where the global variables end up, and one per environment, using the value of the environment_long variable.

This folder may also contain per-ansible-group folders (aka webservers/ or databases/)

Each of these per-environment folders can contain either Ansible variable files (YAML format), or an Ansible-Vault secret files.

I like to add numbering to them so that things sort properly on includes.

Also, spread your variables out. Don’t use a single file for the environment’s variables. I like to use a single file per "app" This makes maintaining much easier (esp with ansible-vault encrypted files).

Same thing goes for the host_vars folder. One folder per host. I often use this for a one-host exception to an Ansible connection setting (changing connection host or user for example).

Per environ inventory folders

Next, you’ll notice there’s a per environ (aka short name) folder for each environment. This is where we tell Ansible which environment we are using

ansible-playbook -i inv/int playbooks/site.yml

To make this layout work each per-environ folder contains a symlink to a single source for the ec2.py and localhost inventory files. You find these in the .shared folder. This reduces the places we have to make changes if we update either file.

Each per-environ folder has it’s own separate copy of the ec2.ini configuration file. This file is used to configure the dynamic inventory script specific to each environment.

Also we symlink group_vars and host_vars into this folder to make sure it’s in the place ansible expects it to be.

The ec2.ini configuration file

To have my dynamic inventory script return just the hosts related to the inventory for each environment I setup some filters in the ec2.ini file:

regions = all
regions_exclude = us-gov-west-1, cn-north-1

If you limit this to just the regions in which you have resources the execution time for ec2.py goes down considerably.

instance_filters = tag:environ=int

When I create instances in AWS I mark the instance with a tag related to its environment. Use instance_filters to get just the resources tagged with environ set to int.

In a later blog post I’ll show you how to use Boto profiles to automatically use the proper AWS credentials for each environment.

The mapping.yml file

This is where most of the heavy lifting occurs.

Here is an example of a mapping.yml file:

---
all:
  vars:
    environ: int
    environment_long: integration
  children:
    integration:

integration:
  children:
    web:
    app:
    db:

web:
  children:
    tag_ServerClass_web:

app:
  children:
    tag_ServerClass_app:

db:
  children:
    tag_ServerClass_app:

The vars section at the top properly sets the variables environ and environment_long. It also puts all of the groups into the integration group in the second section.

Each individual dynamic instance (which is tagged in AWS with the ServerClass tag) gets put into its proper group as well (see the web, app, and db groups above).

This is how the dynamic hosts from EC2 inventory end up with proper "clean" names.