harmonious development: standardizing the deployment process via vagrant and puppet

54
1 Title: Harmonious Development: Standardizing The Deployment Process via Vagrant and Puppet Achieve – Quality on Time Prepared For: Acquia October 4, 2012

Upload: acquia

Post on 08-May-2015

2.257 views

Category:

Documents


2 download

TRANSCRIPT

Page 1: Harmonious Development: Standardizing The Deployment Process via Vagrant and Puppet

1

Title: Harmonious Development: Standardizing The Deployment Process via Vagrant

and Puppet

Achieve – Quality on Time

Prepared For: Acquia October 4, 2012

Page 2: Harmonious Development: Standardizing The Deployment Process via Vagrant and Puppet

Achieve Internet Overview

7+ years developing media and entertainment web

properties

Over 60,000 development hours developing Drupal

web & mobile-optimized solutions

Deep expertise in UX (theming), Globalization,

Integration, and Performance

Company commitment to quality, transparency,

continuity, reliability and client experience

Page 3: Harmonious Development: Standardizing The Deployment Process via Vagrant and Puppet

Key Clients: Media & Entertainment

Media

Page 4: Harmonious Development: Standardizing The Deployment Process via Vagrant and Puppet

Key Clients: Other

Other

Page 5: Harmonious Development: Standardizing The Deployment Process via Vagrant and Puppet

Pop Quiz

<?php/** * Transform upper camel case to lower camel case. * * @param string $word * Upper camel case word. * * @return string * $word with first letter in lower case. * * @see http://www.php.net/manual/en/function.lcfirst.php */function my_module_camel_case($word) { return lcfirst($word);}

echo my_module_camel_case('HelloWorld');

Page 6: Harmonious Development: Standardizing The Deployment Process via Vagrant and Puppet

Agenda

+ =

Page 7: Harmonious Development: Standardizing The Deployment Process via Vagrant and Puppet

Agenda (Cont.)

What’s all the fuss?

What is Virtualization?

Vagrant – Creating (and destroying) environments on the fly

Provisioning (via Puppet)

Examples

Future Directions

Q & A

Page 8: Harmonious Development: Standardizing The Deployment Process via Vagrant and Puppet

What We Are Not Covering

Performance Tuning

These are examples only

Version Control

All provisioning can be placed into VCS

Puppet vs. Chef

Page 9: Harmonious Development: Standardizing The Deployment Process via Vagrant and Puppet

What’s all the fuss?

“Write once, run anywhere” -- Sun Microsystems

“Write Once, Debug Everywhere” -- Anonymous /

Reality

“In many distributed computing environments,

failures are reported in a way that violates even

the simplest notions of consistency.” -- IEEE

Software, Bradford Glade. K. Birman

Page 10: Harmonious Development: Standardizing The Deployment Process via Vagrant and Puppet

The Good Old Days

Page 11: Harmonious Development: Standardizing The Deployment Process via Vagrant and Puppet

Today….

Page 12: Harmonious Development: Standardizing The Deployment Process via Vagrant and Puppet

How It Compares

Old Approach

Old Approach… Today

Page 13: Harmonious Development: Standardizing The Deployment Process via Vagrant and Puppet

The Drupal Ecosystem

Page 14: Harmonious Development: Standardizing The Deployment Process via Vagrant and Puppet

Site Building Roles

Page 15: Harmonious Development: Standardizing The Deployment Process via Vagrant and Puppet

How It All Fits

Virtual Box(Virtual Environment)

Vagrant(Virtual Machine Builder)

Puppet(Provisioner)

VeeWee(Vagrant Box

Builder)

Page 16: Harmonious Development: Standardizing The Deployment Process via Vagrant and Puppet

Versions

VirtualBox v4.1.18

(https://www.virtualbox.org/wiki/Downloads)

Vagrant v1.0.3 (http://downloads.vagrantup.com/)

Puppet v2.7.19

Veewee v0.3.0.beta1

(https://github.com/jedi4ever/veewee)

Ubuntu v8.04 Hardy Heron

Page 17: Harmonious Development: Standardizing The Deployment Process via Vagrant and Puppet

Virtual Box

Virtual Box(Virtual Environment)

Vagrant(Virtual Machine Builder)

Puppet(Provisioner)

VeeWee(Vagrant Box

Builder)

Page 18: Harmonious Development: Standardizing The Deployment Process via Vagrant and Puppet

What is Virtualization?

Technical Definition….

A logical representation of a computer in software. By decoupling the physical hardware from the operating system, virtualization provides more operational flexibility and increases the utilization rate of the underlying physical hardware.1

In a Nutshell….

Putting a small computer [guest/virtual machine] (operating system, memory, video display, peripherals, etc.) into the confines of another bigger physical system [host].

1. "Virtualization in education". IBM. October 2007

Page 19: Harmonious Development: Standardizing The Deployment Process via Vagrant and Puppet

VirtualBox

Open Source virtualization software for Windows, Mac, and Linux

Can host multiple operating systems at once (within memory and CPU resources)

Free!

Page 20: Harmonious Development: Standardizing The Deployment Process via Vagrant and Puppet

Vagrant

Virtual Box(Virtual Environment)

Vagrant(Virtual Machine Builder)

Puppet(Provisioner)

VeeWee(Vagrant Box

Builder)

Page 21: Harmonious Development: Standardizing The Deployment Process via Vagrant and Puppet

Vagrant

Manages your virtual machines

Works with VirtualBox

Extremely customizable

Over 50+ pre-built base boxes

Free!

Page 22: Harmonious Development: Standardizing The Deployment Process via Vagrant and Puppet

Why Vagrant?

Creates consistent, reproducible environments

Rapid setup – Start development in a few quick steps

Reduce duplicated effort

Increased reliability

Allows you to tinker around, more on this later…

Cost savings!

Page 23: Harmonious Development: Standardizing The Deployment Process via Vagrant and Puppet

Agenda - Revisited

What’s all the fuss?

What is Virtualization?

Vagrant – Creating (and

destroying) environments on the

fly.

Provisioning (via Puppet)

Examples

Future Directions

Q & A

Page 24: Harmonious Development: Standardizing The Deployment Process via Vagrant and Puppet

Using VagrantObjective:Part 1: Create a Ubuntu 8.04 LTS Virtual Machine (VM)

One Time Steps:List available boxes:$ vagrant box list

Add a new Vagrant base box to inventory if not available:(@see http://www.vagrantbox.es/)Usage: vagrant box add <alias> <location>$ vagrant box add hardy /path/to/hardy.box

Create a directory for your Vagrant VMs$ mkdir <vm_dir>/myproject

Create a Vagrant file (do this in <vm_dir>/myproject)Usage: vagrant init <alias>$ vagrant init hardy

Page 25: Harmonious Development: Standardizing The Deployment Process via Vagrant and Puppet

Using Vagrant

Let’s Rock!!

Start up a virtual machine:$ vagrant up

Delete a virtual machine:$ vagrant remove

Connect to virtual machine:$ vagrant ssh

Success! Instant VM. Do the happy dance!

Page 26: Harmonious Development: Standardizing The Deployment Process via Vagrant and Puppet

Using Vagrant

Other Useful Commands

Go to lunch:$ vagrant suspend # Save state of virtual machine$ vagrant resume # Start virtual machine up again

Reboot:$ vagrant halt # Shutdown the VM$ vagrant up –no-provision # Restart VM, skip provisioning*

* = Online documentation says no reprovisioning will occur…

Reload configuration (automatically does a provisioning)$ vagrant reload

Reprovision a system:$ vagrant reprovision

Repackage running system:$ vagrant package

Page 27: Harmonious Development: Standardizing The Deployment Process via Vagrant and Puppet

Example 1

$ mkdir <vm_dir>/myproject$ vagrant init hardy

Customize our VM Instance (Excerpts of Vagrantfile)config.vm.network :hostonly, "192.168.33.10”

# Host: <vm_dir>/<project_name> == Guest: /vagrantconfig.vm.share_folder "v-root", "/vagrant", ".", :owner => 'www-data', :group => 'www-data’

# Configure the VM with puppet.config.vm.provision :puppet, :module_path => "modules", :options => ["--environment", "local"] do |puppet| puppet.manifests_path = "manifests” puppet.manifest_file = ”myproject.pp”End

$ vagrant up

Page 28: Harmonious Development: Standardizing The Deployment Process via Vagrant and Puppet

Example 1

Scenario: Admin for a Day

Wonder what this does… (WARNING)*

$ sudo chmod 444 /etc/sudoers

* Warning, doing this on a Production system may have an adverse effect on your career. This will remove any ability to sudo.

Solution (Assuming all configuration is provisioned):$ vagrant destroy$ vagrant up

Page 29: Harmonious Development: Standardizing The Deployment Process via Vagrant and Puppet

Vagrant++

Instant development environments!

No physical hardware required.

Creates a safe environment for configuration

optimization.

Page 30: Harmonious Development: Standardizing The Deployment Process via Vagrant and Puppet

Puppet

Virtual Box(Virtual Environment)

Vagrant(Virtual Machine Builder)

Puppet(Provisioner)

VeeWee(Vagrant Box

Builder)

Page 31: Harmonious Development: Standardizing The Deployment Process via Vagrant and Puppet

Puppet

Provisioning = preparing a device/server for usage, aka configuration

Allows enforceable system configuration

Similar to Microsoft’s System Center Configuration Manager/SMS or Apple’s Profile Manager

Uses a Resource Abstraction Layer (RAL) to interact with the host.Core Resource Types include: notify, file, package, service, exec, cron, user, group

RAL can be used to read and modify resources

Configurations can be store in manifests. See resources for prebuilt modules.

Page 32: Harmonious Development: Standardizing The Deployment Process via Vagrant and Puppet

Puppet: Miscellaneous Info

Facter: Inspects operating system “facts” that can be used in modules.$ facter

… id => vagrantinterfaces => eth0,eth1,loipaddress => 10.0.2.15ipaddress_eth0 => 10.0.2.15ipaddress_eth1 => 192.168.33.10ipaddress_lo => 127.0.0.1is_virtual => truekernel => Linuxkernelmajversion => 2.6kernelrelease => 2.6.24-26-serverkernelversion => 2.6.24lsbdistcodename => hardylsbdistdescription => Ubuntu 8.04.4 LTSlsbdistid => Ubuntulsbdistrelease => 8.04lsbmajdistrelease => 8memoryfree => 1.85 GBmemorysize => 1.98 GBmemorytotal => 1.98 GBnetmask => 255.255.255.0netmask_lo => 255.0.0.0network_eth0 => 10.0.2.0network_eth1 => 192.168.33.0network_lo => 127.0.0.0operatingsystem => Ubuntuoperatingsystemrelease => 8.04…

Page 33: Harmonious Development: Standardizing The Deployment Process via Vagrant and Puppet

Puppet: Miscellaneous Info (Cont.)

Describes a resource, use -s to summarize$ puppet describe <resource>

Inspect individual resource in “Puppet-speak”$ puppet resource service apache2service { 'apache2': ensure => 'stopped', enable => 'true',}

Test a manifest on the local system$ puppet apply --modulepath=<module_path> <manifest>.pp

Page 34: Harmonious Development: Standardizing The Deployment Process via Vagrant and Puppet

Agenda - Revisited

What’s all the fuss?

What is Virtualization?

Vagrant – Creating (and destroying)

environments on the fly.

Provisioning (via Puppet)

Examples

Future Directions

Q & A

Page 35: Harmonious Development: Standardizing The Deployment Process via Vagrant and Puppet

Puppet: Important Notes

Common Pattern:Package, File, Service

Resources are processed asynchronously! Use dependency relationships (require/before) if any ordering is necessary.

@see http://forge.puppetlabs.com/ for prebuilt Puppet modules.

Page 36: Harmonious Development: Standardizing The Deployment Process via Vagrant and Puppet

Puppet Main Manifest

class myproject { $site = ’myproject'

# Vagrant specific setup, set up hostname, hosts file class { 'vagrant': site => $site, }

# Cool stuff class { 'vim': }

# LAMP stack class { 'linux': } class { 'apache': } class { 'mysql': } class { 'php': error_reporting => 'E_ALL & ~(E_WARNING|E_NOTICE)', } class { 'phpmyadmin': }

# Drupal class { 'drupal': server_type => 'dev', }

# Solr class { 'tomcat': } class { 'solr': }

# Site specific stuff apache::site { 'default': vhost => 'default', ensure => 'absent'; $site:

ensure => 'present'; } apache::mod { 'rewrite': ensure => 'present' }

mysql::schema { $site: user => $site, password => $site, }

class { 'drupal::prepare': site => $site, admin_pw => 'admin', }}

class { 'myproject': }

myproject.pp

Search

Base LAMP

Site specific Site/DB

Page 37: Harmonious Development: Standardizing The Deployment Process via Vagrant and Puppet

Example 2: Puppet Module

# Class: apache## This module installs apache and sets up the virtual.## Mike Lee <[email protected]># 2012-09-07## Tested platforms:# - Ubuntu 8.04 Hardy Heron## Parameters:# N/A## Actions:# Installs and configures apache. Will automatically create the site file.## Requires:# N/A## Sample Usage:## Make sure apache is present.# class { 'apache': }## Enable/disable sites.# <site_name> = Simple site name. The vagrant domain will be added automatically.# i.e. <site_name> = "example" creates a site named "example.vagrant".# <docroot> = Docroot. Defaults to /vagrant/<site_name>/docroot.# If this is overridden, this must be the ABSOLUTE path to the docroot.#

# apache::site {# 'default':# vhost => 'default',# ensure => 'absent';# '<site_name_1>':# ensure => 'present';# '<site_name_2>':# docroot => '/custom/path',# ensure => 'present';# }## Enable/disable modules.# apache::mod {# 'rewrite': ensure => 'present’# }

/modules/apache/init.pp (Header)

Base usage

Advanced usageEnable host/mod

Page 38: Harmonious Development: Standardizing The Deployment Process via Vagrant and Puppet

Puppet Sample Module

class apache { $apache2_sites = '/etc/apache2/sites' $apache2_mods = '/etc/apache2/mods'

# OS Specific settings case $::operatingsystem { ubuntu: { $package_name = 'apache2' $conf_template = 'site.conf.debian.erb' } default: { notify { "${module_name}_unsupported": message => "The ${module_name} module is not supported on ${::operatingsystem}.", } error("OS support for ${::operatingsystem} needs to be configured") } }

# Define new type: site define site ($ensure = 'present', $vhost = undef, $docroot = undef) { if $vhost { $site_name = $vhost } else { $site_name = "${title}.vagrant" }

if $docroot { $path = $docroot } else { $path = "/vagrant/${title}/docroot" }

case $ensure { present: { file { "/etc/apache2/sites-available/$site_name": ensure => file, require => Package[$apache::package_name], content => template("apache/${apache::conf_template}"), }

exec { "/usr/sbin/a2ensite $site_name": unless => "/bin/readlink -e ${apache2_sites}-enabled/$site_name", notify => Exec['force-reload-apache'], require => Package[$apache::package_name], } } absent: { exec { "/usr/sbin/a2dissite $site_name": #onlyif => "/bin/readlink -e ${apache2_sites}-enabled/$site_name", # readlink not returning 0 notify => Exec['force-reload-apache'], require => Package[$apache::package_name], } } default: { err("Unknown ensure value: '$ensure'") } } }

/modules/apache/init.pp (Code)

Define type: site

OS Specific

Page 39: Harmonious Development: Standardizing The Deployment Process via Vagrant and Puppet

Puppet Sample Module

# Define new type: mod define mod ($ensure = 'present') { case $ensure { present: { exec { "/usr/sbin/a2enmod $title": unless => "/bin/readlink -e ${apache2_mods}-enabled/${title}.load", notify => Exec['force-reload-apache'], require => Package[$apache::package_name], } } absent: { exec { "/usr/sbin/a2dismod $title": #onlyif => "/bin/readlink -e ${apache2_mods}-enabled/${title}.load", # readlink not returning 0 notify => Exec['force-reload-apache'], require => Package[$apache::package_name], } } default: { err("Unknown ensure value: '$ensure'") } } }

# Force reload all the time, doesn't take that much more in resources. exec { 'force-reload-apache': command => '/etc/init.d/apache2 force-reload', refreshonly => true, }

package { $package_name: ensure => installed, }

service { $package_name: ensure => running, hasstatus => true, hasrestart => true, require => Package[$package_name], }}

/modules/apache/init.pp (Code)

Define type: mod

Package/Service

Page 40: Harmonious Development: Standardizing The Deployment Process via Vagrant and Puppet

Puppet Sample Module

# Puppet generated file. DO NOT EDIT!# Managed by Class['apache']

<VirtualHost *:80> ServerAdmin [email protected]

DocumentRoot <%= path %> ServerName <%= site_name %>

<Directory <%= path %>> Options Indexes FollowSymLinks MultiViews AllowOverride All Order allow,deny allow from all </Directory></VirtualHost>

ERB Templates: Create files on the fly./modules/apache/templates/site.conf.debian.erb

Page 41: Harmonious Development: Standardizing The Deployment Process via Vagrant and Puppet

Puppet++

Construct fully functioning development

environments on the fly!

Bring new team members up to speed in a few

simple commands.

CONSISTENCY! CONSISTENCY! CONSISTENCY!

Everyone is working with the same virtual machine.

Page 42: Harmonious Development: Standardizing The Deployment Process via Vagrant and Puppet

Agenda - Revisited

What’s all the fuss?

What is Virtualization?

Vagrant – Creating (and destroying)

environments on the fly.

Provisioning (via Puppet)

Examples

Future Directions

Q & A

Page 43: Harmonious Development: Standardizing The Deployment Process via Vagrant and Puppet

Veewee

Virtual Box(Virtual Environment)

Vagrant(Virtual Machine Builder)

Puppet(Provisioner)

VeeWee(Vagrant Box

Builder)

Page 44: Harmonious Development: Standardizing The Deployment Process via Vagrant and Puppet

Next Steps: Veewee

Want your own custom Vagrant Box or one doesn’t exist?

Veewee has predefined definitions for creating a variety of Vagrant base boxes.

Installation Requirements (Mac OS X)- Install RVM (Ruby Version Manager)

$ curl -L https://get.rvm.io | bash -s stable

- Install Xcode, Command Line Tools for Xcode (required by Homebrew)

- Install Homebrew$ ruby <(curl -fsSkL raw.github.com/mxcl/homebrew/go)

- Install apple-gcc42 $ brew update $ brew tap homebrew/dupes $ brew install autoconf automake apple-gcc42 $ rvm pkg install openssl

GCC Required!

Page 45: Harmonious Development: Standardizing The Deployment Process via Vagrant and Puppet

Next Steps: Veewee

Veewee Installation (Mac OS X)

$ rvm install 1.9.2$ git clone https://github.com/jedi4ever/veewee.git$ cd veewee

(official – Never worked for me…)$ gem install bundler$ bundle install

(unoffical)$ gem build veewee.gemspec$ gem install veewee-*.gem (run as sudo)$ unalias veewee

Page 46: Harmonious Development: Standardizing The Deployment Process via Vagrant and Puppet

Next Steps: Veewee

Veewee Base Box Creation (Mac OS X)

In veewee directory:- Copy new iso into [currentDir]/iso (optional, saves veewee from finding)

Create VM definition off of template:Usage: veewee vbox define <box_name> <template>$ veewee vbox define hardy ubuntu-8.04.4-server-i386

Customize VM Definition:- Edit <veewee_dir>/definitions/<box_name>/definition.rb

Build virtual box (This should start VirtualBox)Usage: veewee build <box_name>$ veewee vbox build hardy # go get a coffee

Usage: veewee vbox validate <box_name>$ veewee vbox validate hardy

Build Vagrant base box:Usage: vagrant basebox export <box_name>$ vagrant basebox export hardy

Page 47: Harmonious Development: Standardizing The Deployment Process via Vagrant and Puppet

Next Steps: Puppet Enterprise

Allow for remote management of puppet nodes.

“One server to rule them all.”

Clone your configurations to EC2.

Audit your configurations.

Page 48: Harmonious Development: Standardizing The Deployment Process via Vagrant and Puppet

Gotchas

Be mindful of Puppets asyncronous execution order:

Make sure dependecies are properly set.

Starbucks syndrome:

Switching networks can sometimes cause your VM’s networking to get confused. Halt and restart to correct.

Provision everything… or as much as you can:

This will make resetting your environment go much more smoothly.

Page 49: Harmonious Development: Standardizing The Deployment Process via Vagrant and Puppet

Versions

VirtualBox v4.1.18

(https://www.virtualbox.org/wiki/Downloads)

Vagrant v1.0.3 (http://downloads.vagrantup.com/)

Puppet v2.7.19

Veewee v0.3.0.beta1

(https://github.com/jedi4ever/veewee)

Ubuntu v8.04 Hardy Heron

Page 50: Harmonious Development: Standardizing The Deployment Process via Vagrant and Puppet

Further Resources

Prebulit Vagrant Base Boxes (

http://www.vagrantbox.es/)

Puppet Forge (http://forge.puppetlabs.com/)

Learn Puppet (http

://docs.puppetlabs.com/learning/ral.html)

Page 51: Harmonious Development: Standardizing The Deployment Process via Vagrant and Puppet

Conclusion

…makes the "works on my machine" excuse a relic of the past. -- Vagrant

Created a disposable test bed to play with configuration changes

Created reusable modules to easily build customized setups

Provided canned environment for those who simply want to jump into a project. Allows team members to focus on core competencies.

Allows you to quickly and efficiently archive projects and come back to them later

CONSISTENCY - REDUCED ERRORS

EASY & INEXPENSIVE

INSTANT DEVELOPMENT - SAVE $$$

Page 52: Harmonious Development: Standardizing The Deployment Process via Vagrant and Puppet

Achieve Internet Pillars of Expertise

Technical Foundation Built On the Following Core Areas of Expertise and Knowledge:

1. User Experience (UX) / Theming

2. Globalization/Localization

3. Performance & Deployment

4. Integration

Page 53: Harmonious Development: Standardizing The Deployment Process via Vagrant and Puppet

QA

Page 54: Harmonious Development: Standardizing The Deployment Process via Vagrant and Puppet

Michael LeeSenior Architect

[email protected]

mlee11111@ach_mikelee

/pub/michael-lee/4/297/a6bDirect: 858-453-5760

Thank You