ansible - a 'crowd' introduction

SOFTWARE CRAFTSMANSHIP TOLEDO

A “crowd” introduction


Software Craftsmanship Toledo• Pasión por la ingeniería software y el desarrollo de

software en la comunidad de Castilla-La Mancha. • Clara vocación por aprender, fomentar y compartir las

metodologías ágiles en la región.• Grupo totalmente abierto: http://www.meetup.com/es/

Software-Craftsmanship-Toledo-ES• Miembros profesionales y/o apasionados del desarrollo

software.• Tenemos un canal de Slack… Apúntante!

http://www.meetup.com/es/Software-Craftsmanship-Toledo-ES


Meet me• Manuel de la Peña

• @mdelapenya

• Support > Engineering > Testing > IT … at Liferay, Inc.

• DEVOPS?

https://twitter.com/mdelapenya

Meet the team

Meet our server room

OK, THIS server room


This was my first day at work…

moss@itcrowd1:~$ ssh root@server1password:root@server1:~$ apt-get updateroot@server1:~$ apt-get install apache2root@server1:~$ apt-get install htoproot@server1:~$ vi /etc/fstabroot@server1:~$ mount /mnt/devopsroot@server1:~$ exitmoss@itcrowd1:~$

server2, server3, server4, server5 … server30buzz, scar, ariel, mickey, mulan … goofieares, odin, hera, atenea, loki … melkart

logan, natasha, clark, peter, bruceW … bruceB

And all these following servers too:

Every task,every day.


Why don’t you automate THIS?

Why don’t you automate THIS?

Automateserver

management???


AUTOMATION FOR EVERYONEDeploy apps. Manage systems. Crush complexity.

Ansible helps you build a strong foundation for DevOps.


How it works• It uses a very simple language (YAML, in the form

of Ansible Playbooks)

• Plain English!

• By default manages machines over the SSH protocol.

• It uses no agents and no additional custom security infrastructure.


SSH Keys

• SSH keys with ssh-agent are recommended.

• Root logins are not required, you can login as any user, and then su or sudo to any user.


Installation

• No database, and no running daemons!

• Install it on only one machine (a laptop?) as central point.

• It does not leave software installed or running on remote machines —> upgrades super easy :D


Control Machine

• “Any” machine with Python 2.6 or 2.7 installed.

• This includes Red Hat, Debian, CentOS, OS X, any of the BSDs, and so on.

• Windows isn’t supported for the control machine.


Managed Nodes

• You need a way to communicate, normally ssh.

• SSH uses sftp. If not available, switch to scp.

• Python 2.4 or later. (If Python < 2.5 also need python-simplejson)


Remote connection• Native OpenSSH for remote communication when

possible.

• If OpenSSH is too old in control machine (Enterprise Linux 6 OS)—> Fallback to Paramiko: a Python implementation.

• SSH keys are encouraged but password authentication can also be used (--ask-pass).


sudo

Ansible Inventory


Inventory

• What machines/hosts you are managing using a very simple INI file.

• Managed machines/hosts in groups of your own choosing.

• Define variables by scope.

Groups


Inventory variables

• Host variables

• Group variables

• Advanced variables


Inventory: Host vars


Inventory: Group vars


Inventory: Advanced vars

Affects to groups

Specific group of servers

Arbitrary criterium


Host matching: Patterns

• all

• *

• 192.168.1.*, *.example.com

• OR —> webservers:dbservers

• AND —> webservers:&staging


Host matching: Patterns

• Exclusions —> webservers:!phoenix

• Combinations —> webservers:dbservers:&staging:!phoenix

• Host selection —> webservers[0], webservers[0:1], webservers[1:]

• Regexp —> ~(web|db).*\example\.com


Host matching: Limit


Ansible Modules


Ansible modules

• copy

• file

• yum

• user

• service

• git

• apache

• apt

• azure

• bower

• cron

• mysql_db

• npm

• puppet

• django

• ec2And many more!


Modules


git module


Ad-hoc commands

Fork 10 parallel threads


Ansible Variables


Variables• Should be letters, numbers, and

underscores.

• Variables should always start with a letter.

• YAML also supports dictionaries which map keys to values.

• There are reserved Python-related keywords.


Variables

• Defined in Inventory.

• Defined in Playbooks.

• Defined in includes.

• Defined in Roles.


Variables: Jinja2

• Jinja2 —> Template system

• How to use a variable? —> {{ variable_name }}

• Safer —> “{{ variable_name }}”


Variables: register• Run a command and save its result into a

variable.


Variables: external

• Private information?? Hide them in variables!

• In a task list —>

vars_files: /opt/private/my_vars.yml


Variables: command• Directly pass variables to command line:

• --extra-vars “version=1.3 my_var=foo”

• --extra-vars ‘{"pacman":"mrs","ghosts":["inky","pinky","clyde","sue"]}'

• --extra-vars "@some_file.json"


Variables: precedence• role defaults < inventory vars < inventory

group_vars < inventory host_vars < playbook group_vars < playbook host_vars < host facts < registered vars < set_facts < play vars < play vars_prompt < play vars_files < role and include vars < block vars (only for tasks in block) < task vars (only for the task) < extra vars


Ansible Playbooks


Playbooks• Orchestrate steps/tasks of any

manual ordered process.

• Executed a/synchronously.

• YAML format.

• Minimum syntax —> not a language but a model.


Playbooks: Tasks

• Should have a name: included in output and readable by humans.

• Accept variables:

template: src=somefile.j2 dest=/etc/httpd/conf.d/{{ vhost }}


Playbooks: Tasks lists• Each play contains a list of tasks.

• Tasks are executed in order, one at a time, against all machines matched by the host pattern.

• Important! Same task directives to all hosts.

• Tasks goal? execute a module, with arguments.

• Modules are idempotent.


Playbooks:Hosts&Users• For each play, choose machines to

target and remote user to complete the steps/tasks as.

• Define remote users per task.

• Use sudo on a particular task instead of the whole play.


Playbooks: Handlers• Playbooks respond to change.

• Can notify at the end of each block of tasks.

• Triggered only once, even if notified by multiple tasks.

• Best used to restart services and trigger reboots.


ansible-playbook playbook.yml


Ansible Roles


Roles• Reuse playbooks.

• Combine files to form clean, reusable abstractions.

• Grouping allows easy sharing of roles with other users.

• include directive —> Include files.


Roles: structure

• Automatically loads certain vars_files, tasks, and handlers based on a known file structure.

• Where is the magic? Improvements to search paths for referenced files.


‘common’ role structure

‘webservers’ role structure


Roles: main.yml• roles/x/tasks/main.yml —> Add tasks to the

play.

• roles/x/handlers/main.yml —> Add handlers to the play.

• roles/x/vars/main.yml —> Add variables to the play.

• roles/x/meta/main.yml —>, Add roles as dependency (1.3 and later).


Roles: paths

• Any copy, script, template or include tasks (in the role) can reference files in roles/x/{files,templates,tasks}/ without having to path them relatively or absolutely


Roles:parameters


Roles:defaults• Add a defaults/main.yml file in your role

directory.

• Set default variables for included or dependent roles.

• Lowest priority of any variables available, so they are easily overridden, including inventory variables!


Roles:dependencies• A list of roles and parameters to

insert and execute before the specified role.

• Automatically pull in other roles.

• meta/main.yml within the role.

• Executed recursively.


Roles:dependencies


https://galaxy.ansible.com

Where can I findready to use Roles?

Ansible Galaxy

https://galaxy.ansible.com/


https://github.com/ansible/ansible-examples/tree/master/lamp_haproxy

Ansible Real Example (1)

https://github.com/ansible/ansible-examples/tree/master/lamp_haproxy


Ansible Real Example (2):LELK


LELK

• Liferay Portal

• Elasticsearch

• Logstash

• Kibana

• generate

• index

• process

• display


Liferay + Tomcat Liferay + Tomcat

Liferay + Tomcat Liferay + Tomcat

LELK


LELK: Roles

• Server —> ElasticSearch + Logstash + Kibana

• Clients -> Liferay Deployment


Role: liferay-deployment• Ansible Role: liferay-users (add liferay

user with UID=1000)

• Ansible Role: liferay-os-tools (vim, htop, curl, rsync, tree, zip)

• Ansible Role: liferay-dev-tools (java)

• Ansible Role: liferay-logstash-forwarder


Role: logstash-forwarder

• Copy SSL certificate

• Copy init.d startup

• Copy logstash-forwarder configuration

logstash-forwarder.conf.j2


Role: logstash-forwarder


Role: elastic-search-server

• Install Logstash

• Install ElasticSearch

• Install Kibana

• Install Nginx


Role: elastic-search-serverlogstash-input.conf.j2

logstash-output.conf.j2


LELK Next??

• Applied logstash-forwarder role to more machines, setting log_paths and tags variable to the desired file.

• Add more client types —> Apache Server, Firewall


With tests!!!


Recap• Server are hard to maintain up-to-date

• Automation over infrastructure

• Infrastructure as code

• Ansible Inventory & Modules

• Variables & template system

• Playbooks & Roles & Galaxy


What about using GITto version

Inventory and Roles??

Meet OUR server room

Thanks!

ansible - a 'crowd' introduction

Technology