Provisioning using Ansible in AWS

Discuss Docker, Chef, Puppet, Ansible, SaltStack

March 18, 2014

WHO AM I?

Aater Suleman

Geek, Architect, Developer, Ops, DevOps …

Co-founder & CEO Flux7 Labs

Part-time UT Austin Professor

Flux7 Labs: AWS and DevOps Solutions

■ Web

■ Big data

■ HPC

in/aatersuleman

@FutureChips

Flux7Labs

@Flux7Labs

www.flux7.com

What is Ansible?

IT automation tool

Advanced tasks such as continuous deployments or zero downtime rolling updates

Features:

● Simple

● Agentless: no agent on the client, uses SSH

● Scalable

No Databases or daemons are added after installation

Root permissions are not required

OS package manager for Red Hat Enterprise Linux (TM), CentOS, Fedora, Debian, or Ubuntu

pip install for MacOS

Installation

$ git clone git://github.com/ansible/ansible.git

$ cd ./ansible

$ source ./hacking/env-setup

Installation from source

To install from source

$ sudo pip install paramiko PyYAML jinja2 httplib2

Install Python Modules

● Ansible’s inventory file has the list of all managed host names one

line per host

● Files are organized as hosts and groups.

● A set of hosts can be under a group name.

● A host can be in more than one group

● Dynamic Inventory to pull files from dynamic sources or clouds

Ansible’s Inventory

$ echo "127.0.0.1" > ~/ansible_hosts

$ export ANSIBLE_HOSTS=~/ansible_hosts

Inventory Example

The format for /etc/ansible/hosts is in INI format and is as follows:

webserver1

webserver2

dbserver1

[webserver]

webserver1

webserver2

[dbserver]

dbserver1

Patterns: Pattern refers to the way we decide to manage hosts

Example:● How to communicate with hosts

● Which hosts need a particular config management

Patterns

ansible <pattern_goes_here> -m <module_name> -a <arguments>

Basic Structure:

one.example.comone.example.com:two.example.com192.168.1.50192.168.1.*

Example: Address Specific host or set of hosts by name

webserverswebservers:dbservers

Example: Address one or more groups

webserverswebservers:dbservers

Example: Address one or more groups

Pattern Examples

Basic Commands

ansible -m ping -i hosts all

Ping all hosts:Example Inventory:/etc/ansible/hosts

webserver1

webserver2

dbserver1

[webserver]

webserver1

webserver2

[dbserver]

dbserver1

ansible webserver1 -m

command -a whoami

Who am I:

Policy for a remote system or a set of steps for a general IT process

Language for Ansible’s configuration, deployment and orchestration

Human readable

Playbooks

Playbook Example

---# possibly saved as tasks/foo.yml

- name: placeholder foo command: /bin/foo

- name: placeholder bar command: /bin/bar

Simple playbook template:

ansible-playbook playbook.yml -f 10

Execute a playbook: Using a parallelism level of 10

Playbook Example

---- hosts: webservers remote_user: root tasks: - name: test connection ping: remote_user: yourname

Defining remote users per task

tasks: - name: make sure apache is running service: name=httpd state=running

Simple task definition

Modules control system resources - services, packages, files, system commands, etc.

In module directories that can be executed directly or through playbooks.

Language independent -- Return JSON format data

Idempotent - avoids change to system unless needed

Modules

Module Examples

# Example action to start service httpd, if not running

- service: name=httpd state=started

# Example action to start service foo, based on running process

/usr/bin/foo

- service: name=foo pattern=/usr/bin/foo state=started

Service Module: Controls service on remote hosts

# Update repositories cache and install "foo" package

- apt: pkg=foo update_cache=yes

# Update all packages to the latest version

- apt: upgrade=dist

apt Module : Manages apt packages

Module Examples

$ ansible all -m user -a "name=foo password=<crypted password here>"$ ansible all -m user -a "name=foo state=absent"

User Module: Manages user accounts

# Basic provisioning example

- local_action:

module: ec2

key_name: mykey

instance_type: c1.medium

image: ami-40603AD1

wait: yes

group: webserver

count: 3

ec2 Module: create, terminate, start or stop an instance in ec2, return instanceid

Hub of all automation tasks

Web-based solution

Controls access

Manages Inventory

Supports autoscaling topologies through provisioning callbacks

Ansible Tower

ProvisioningUsing Ansible in AWS

Python Module: Boto 2.5 or higher

Basic Requirements

$ yum install python-boto

Add localhost to inventory[local]

localhost

- hosts: localhost

connection: local

gather_facts: False

Pattern used in playbooks for

provisioning

Install this python module on the execution host:

● ec2 module allows provisioning of EC2 instances

● Provisioning will be against Ansible master server in a play that operates on localhost

● Specify access and secret key using ENV variables to provide authentication to AWS related modules

Provisioning in AWS

# ansible localhost -m ec2 -a "image=ami-6e649707

instance_type=m1.large keypair=mykey group=webservers

wait=yes" -c local

Example of provisioning a number of instances in ad-hoc mode

tasks:

- name: Provision a set of instances

ec2: >

keypair={{mykeypair}}

group={{security_group}}

instance_type={{instance_type}}

image={{image}}

wait=true

count={{number}}

register: ec2

Translates in play as follows:

Registering allows dynamic creation of a host group for the new instances

Provisioning in AWS

- name: Add all instance public IPs to host groupadd_host: hostname={{ item.public_ip }} groupname=ec2hostswith_items: ec2.instances

Registering allows configuration actions on the hosts in a subsequent task:

Include the configuration as a task include or a role rather than inline inclusion

Provisioning in AWS

# ./ec2.py --refresh-cache

Advanced Usage of Ansible in AWS

Host Inventory: Use of ec2 inventory plugin when you need to talk to a

node again. Schedule a regular refresh of the inventory cache using:

Tags: Helps manage groups dynamically without maintaining a separate inventory

Pull Configuration: Using ansible-pull - which checks out a repo of configuration instructions from git

Autoscaling using Ansible Tower: By using a simple curl script

Questions?