nlit 2011: chef & capistrano

NREL is a national laboratory of the U.S. Department of Energy, Office of Energy Efficiency and Renewable Energy, operated by the Alliance for Sustainable Energy, LLC.

Automated Server Configuration and Web Site Deployments with Chef and Capistrano

NLIT Summit 2011

Nick Muerdter

06/15/2011

NATIONAL RENEWABLE ENERGY LABORATORY 2

Photo from National Library NZhttp://www.flickr.com/photos/nationallibrarynz_commons/5015573731

NATIONAL RENEWABLE ENERGY LABORATORY 3

Bringing Order

Photo from U.S. National Archiveshttp://www.flickr.com/photos/usnationalarchives/4011449131

NATIONAL RENEWABLE ENERGY LABORATORY 4

Party like it’s 2011!

Image © Hanna-Barbera and Warner Bros. Entertainment Inc.

NATIONAL RENEWABLE ENERGY LABORATORY 5

Photo by hey skinnyhttp://www.flickr.com/photos/heyskinny/1464641723

NATIONAL RENEWABLE ENERGY LABORATORY 6

• Web site deployment tool• Automates commands over SSH to

multiple servers• Encapsulates deployment best practices• Ruby on Rails based, but easily does more

What Is Capistrano?

NATIONAL RENEWABLE ENERGY LABORATORY

Getting Started With Capistrano

7

$ sudo gem install capistrano capistrano-ext

$ cd my_project$ capify .[add] writing './Capfile'[add] making directory './config'[add] writing './config/deploy.rb'[done] capified!

NATIONAL RENEWABLE ENERGY LABORATORY

Sample config/deploy.rb

8

require "capistrano/ext/multistage"

set :application, "my_project"

# Supported: `accurev`, `bzr`, `cvs`, `darcs`, `git`,# `mercurial`, `perforce`, `subversion` or `none`set :scm, :subversion

set :repository, "https://svn.nrel.gov/my_project/trunk"

set :deploy_to, "/var/www/my_project"

NATIONAL RENEWABLE ENERGY LABORATORY

Sample config/deploy/development.rb

9

# Set the servers for this stage.role :app, "dev.nrel.gov"role :web, "dev.nrel.gov"role :db, "dev-db.nrel.gov"

# Reduce the number of copies kept for the# development environment.set :keep_releases, 2

NATIONAL RENEWABLE ENERGY LABORATORY

Sample config/deploy/staging.rb

10

# Set the servers for this stage.role :app, "staging.nrel.gov"role :web, "staging.nrel.gov"role :db, "staging-db.nrel.gov"

NATIONAL RENEWABLE ENERGY LABORATORY

Sample config/deploy/production.rb

11

# Set the servers for this stage.role :app, "nrel.gov"role :web, "nrel.gov"role :db, "db.nrel.gov"

NATIONAL RENEWABLE ENERGY LABORATORY

Running Capistrano

12

$ cap staging deploy * executing `development' triggering start callbacks for `deploy' * executing `multistage:ensure' * executing `deploy' triggering before callbacks for `deploy' * executing `deploy:try_setup' * executing `deploy:setup' triggering before callbacks for `deploy:setup' * executing `deploy:gem_bundler:setup' * executing "mkdir -p /srv/afdc/staging/common/my_project /srv/afdc/staging/common/my_project/releases /srv/afdc/staging/common/my_project/shared /srv/afdc/staging/common/my_project/shared/log" servers: ["staging.nrel.gov"]Password: [staging.nrel.gov] executing command command finished in 795ms triggering after callbacks for `deploy:setup' * executing `deploy:shared_children_file_tasks:setup' * executing "mkdir -p /srv/afdc/staging/common/my_project/shared/public/linkcheck && chmod g+w /srv/afdc/staging/common/my_project/shared/public/linkcheck" servers: ["staging.nrel.gov"] [staging.nrel.gov] executing command command finished in 62ms

...

NATIONAL RENEWABLE ENERGY LABORATORY

• Customize with before/after callbacks

• What we’ve done:• Per-developer

sandbox creation on development

• Per-branch deployment on staging

13

Where To Next?

Photo from Nationaal Archiefhttp://www.flickr.com/photos/nationaalarchief/402623048

NATIONAL RENEWABLE ENERGY LABORATORY 14

What Is Chef?• Automated server

configuration tool• Programmatically

define:• System

requirements• System

configuration• Replicate changes

between environments.• Easily setup similar

servers.Image © Comedy Central

NATIONAL RENEWABLE ENERGY LABORATORY 15

• Cookbooks• Recipes• Attributes• Files• Templates

• Roles• Nodes

• Chef Server• Chef Clients

Chef Terminology

Image © Iannucci's Pizzeria & Italian Restaurant

NATIONAL RENEWABLE ENERGY LABORATORY 16

package "apache2" do case node[:platform] when "centos", "redhat", "fedora", "suse" package_name "httpd" when "debian", "ubuntu" package_name "apache2" when "arch" package_name "apache" end

action :installend

service "apache2"

template "#{node[:apache][:dir]}/envvars" do source "envvars.erb" group "root" owner "root" mode 0644 notifies :reload, "service[apache2]"end

# ...

Sample Recipe

NATIONAL RENEWABLE ENERGY LABORATORY 17

name "ctts_base"description "A base role for ctts.nrel.gov servers."

run_list([ "role[base]”, "role[apache]", "role[capistrano]", "role[passenger_apache]", "role[ruby]", "role[tomcat]",

"recipe[apache2::mod_cgi]", "recipe[apache2::mod_include]", "recipe[apache2::mod_php5]", "recipe[apache2::mod_ssl]", "recipe[awstats]",])

default_attributes({ :awstats => { :log_file => "/srv/afdc/ctts/eere/current/log/access.log-%YYYY-0%MM-0%DD-0", :domain => "www.afdc.energy.gov", }, :php => { :module_oci8 => { :version => "1.4.4" }, },})

Sample Role

NATIONAL RENEWABLE ENERGY LABORATORY 18

NATIONAL RENEWABLE ENERGY LABORATORY 19

$ sudo chef-clientRun List is [role[developer_development]]Run List expands to [sudo, chef-client::config, etc::environment, opsview::client, iptables, logrotate, screen, ack, vim, nano, zsh, capistrano, subversion, nginx::source, passenger::nginx, rvm::install, rubygems::client, bundler, java, tomcat, db_config, haproxy, jammit, postgresql::client, supervisor, opsview::check_haproxy, opsview::check_nginx, opsview::check_supervisorctl, opsview::server, opsview::apache, chef-server, docs_site::nginx]Starting Chef Run for devdev.nrel.govLoading cookbooks [ack, apache2, apt, bluepill, build-essential, bundler, capistrano, chef-client, chef-server, couchdb, daemontools, db_config, docs_site, erlang, etc, gecode, git, haproxy, iptables, jammit, java, logrotate, mysql, nano, nginx, openssl, opsview, passenger, perl, php, postgresql, python, rsync, rsyslog, rubygems, runit, rvm, screen, subversion, sudo, supervisor, tomcat, ucspi-tcp, vim, xml, yum, zlib, zsh]Skipping opsview::client recipe because conflicting opsview::server recipe is enabledCould not find previously defined grants.sql resourceProcessing package[sudo] action upgrade (sudo::default line 20)Processing template[/var/cache/chef/sudoers] action create (sudo::default line 40)Processing template[/etc/sudoers] action create (sudo::default line 48)Processing directory[/var/run/chef] action create (chef-client::config line 30)Processing directory[/var/cache/chef] action create (chef-client::config line 30)Processing directory[/var/lib/chef] action create (chef-client::config line 30)Processing directory[/var/log/chef] action create (chef-client::config line 30)Processing template[/etc/chef/client.rb] action create (chef-client::config line 38)Processing ruby_block[reload_client_config] action nothing (chef-client::config line 47)Processing template[/etc/environment] action create (etc::environment line 10)Processing package[iptables] action install (iptables::default line 20)Processing execute[rebuild-iptables] action nothing (iptables::default line 22)Processing directory[/etc/iptables.d] action create (iptables::default line 27)Processing cookbook_file[/usr/sbin/rebuild-iptables] action create (iptables::default line 31)...

Running Chef

NATIONAL RENEWABLE ENERGY LABORATORY 20

S l o w G o i n g ?

Photo from State Library and Archives of Floridahttp://www.flickr.com/photos/floridamemory/3266993225

NATIONAL RENEWABLE ENERGY LABORATORY 21

Payoff?Totally,

dude.

Photo by Calsidyrosehttp://www.flickr.com/photos/calsidyrose/3198309214

NATIONAL RENEWABLE ENERGY LABORATORY 22

Capistrano:https://github.com/capistrano/capistrano/wiki

Chef:http://www.opscode.com/chef/http://wiki.opscode.com/display/chef/Home

Chef Cookbooks:http://community.opscode.com/cookbookshttps://github.com/opscode/cookbooks

Resources & Support

Photo from Musée McCord Museumhttp://www.flickr.com/photos/museemccordmuseum/5348751435

NATIONAL RENEWABLE ENERGY LABORATORY 23

Photo by Eleafhttp://www.flickr.com/photos/eleaf/2536358399

http://slideshare.net/NickBlah

nick.muerdter@nrel.gov