chefconf 2012 spiceweasel

Spiceweasel: Your Infrastructure as Code Table of Contents

Matt Raymatt@opscode.com

Twitter/IRC/GitHub: mattray

Infrastructure as Code

Enable the reconstruction of the business from nothing but

a source code repository, application data backup and

bare metal resources.

That's great but...how did I get here?

http://www.flickr.com/photos/mrchippy/443960682/

Spiceweasel

• Infrastructure deployment manifest

• Converts YAML/JSON to knife commands

• http://github.com/mattray/spiceweasel

http://www.flickr.com/photos/mrchippy/443960682/

YAML

• Simple

• Easy to read

• Easy to reason

• spiceweasel infrastructure.yml

http://www.flickr.com/photos/mrchippy/443960682/

JSON

• Patches welcomed.

• spiceweasel infrastructure.json

cookbooks:- annoyances:- apache2:- chef-client: - 1.1.5- java:- mysql:- tomcat:

knife cookbook upload annoyancesknife cookbook site download apache2 --file cookbooks/apache2.tgz tar -C cookbooks/ -xf cookbooks/apache2.tgzrm -f cookbooks/apache2.tgzknife cookbook upload apache2knife cookbook site download chef-client 1.1.5 --file cookbooks/chef-client.tgz tar -C cookbooks/ -xf cookbooks/chef-client.tgzrm -f cookbooks/chef-client.tgzknife cookbook upload chef-clientknife cookbook upload javaknife cookbook upload mysqlknife cookbook upload tomcat

Cookbooks

http://www.flickr.com/photos/mrchippy/443960682/

environments:- development:- qa:- preprod:- production:

knife environment from file development.rbknife environment from file qa.rbknife environment from file preprod.rbknife environment from file production.rb

Environments

http://www.flickr.com/photos/mrchippy/443960682/

Roles

roles:- base:- glance-single-machine:- nova-db:- nova-multi-compute:- nova-multi-controller:- nova-rabbitmq-server:- nova-single-machine:

http://www.flickr.com/photos/laenulfean/374398044/

knife role from file base.rbknife role from file glance-single-machine.rbknife role from file nova-db.rbknife role from file nova-multi-compute.rbknife role from file nova-multi-controller.rbknife role from file nova-rabbitmq-server.rbknife role from file nova-single-machine.rb

http://www.flickr.com/photos/mrchippy/443960682/

Data Bags

data bags:- users: - alice - bob - chuck- data: - *- passwords: - secret secret_key - mysql - rabbitmq

knife data bag create usersknife data bag from file users alice.jsonknife data bag from file users bob.jsonknife data bag from file users chuck.jsonknife data bag create dataknife data bag from file data dataA.jsonknife data bag from file data dataB.jsonknife data bag create passwordsknife data bag from file passwords mysql.json --secret-file secret_keyknife data bag from file passwords rabbitmq.json --secret-file secret_key

http://www.flickr.com/photos/75659300@N00/2615848530/

nodes:- ubuntu1-1004.vm ubuntu2-1004.vm ubuntu3-1004.vm: - role[base],role[tc],role[sample] - -i ~/.ssh/mray.pem -x ubuntu --sudo

knife bootstrap ubuntu1-1004.vm -i ~/.ssh/mray.pem -x ubuntu --sudo -r 'role[base],role[tc],role[sample]'knife bootstrap ubuntu2-1004.vm -i ~/.ssh/mray.pem -x ubuntu --sudo -r 'role[base],role[tc],role[sample]'knife bootstrap ubuntu3-1004.vm -i ~/.ssh/mray.pem -x ubuntu --sudo -r 'role[base],role[tc],role[sample]'

Nodes

http://www.flickr.com/photos/75659300@N00/2615848530/

nodes:- windows_winrm winboxA: - role[base],role[sqlserver] - -x Administrator -P 'super_secret_password'- windows_ssh winboxB winboxC: - role[base],role[iis] - -x Administrator -P 'super_secret_password'

knife bootstrap windows winrm winboxA -x Administrator -P 'super_secret_password' -r 'role[base],role[sqlserver]'knife bootstrap windows ssh winboxB -x Administrator -P 'super_secret_password' -r 'role[base],role[iis]'knife bootstrap windows ssh winboxC -x Administrator -P 'super_secret_password' -r 'role[base],role[iis]'

Nodes: Windows

http://www.flickr.com/photos/75659300@N00/2615848530/

nodes:- ec2 5: - role[base],role[tc],role[sample] - -S mray -i ~/.ssh/mray.pem -x ubuntu -I ami-0c6ebd65 -f m1.small- rackspace 5: - role[base],role[tc],role[sample] - --image 114 --flavor 2 -i ~/.ssh/mray.pem- openstack 5: - role[base],role[tc],role[sample] - --image 7 --flavor 2 -x ubuntu -i ~/.ssh/mray.pem

seq 5 | parallel -j 0 -v "knife ec2 server create -S mray -i ~/.ssh/mray.pem -x ubuntu -I ami-0c6ebd65 -f m1.small -r 'role[base],role[tc],role[sample]'"seq 5 | parallel -j 0 -v "knife rackspace server create --image 114 --flavor 2 -i ~/.ssh/mray.pem -r 'role[base],role[tc],role[sample]'"seq 5 | parallel -j 0 -v "knife openstack server create --image 7 --flavor 2 -i ~/.ssh/mray.pem -r 'role[base],role[tc],role[sample]'"

Nodes: Cloud Providers

Validation

• Cookbook versions and their dependencies exist

• Environment names and referenced cookbooks are correct

• Role names and referenced roles and cookbooks are correct

• Data bags parse, secret keys are present

• Node run lists items are correct and their Environment exists

http://www.flickr.com/photos/mrchippy/443960682/

What's the Point?

• Nodes are ephemeral

• Everything we do is in code.

• Everything is checked into version control.

• Basic roadmap for deployment.

http://www.flickr.com/photos/mrchippy/443960682/

Break it Down

How do we recreate our infrastructure?

http://www.flickr.com/photos/mrchippy/443960682/

Real World Example

• Whiteboard it out

• Break it down into components

• Comment it all out and start working

http://www.flickr.com/photos/mrchippy/443960682/

Here's the YAML

cookbooks:- annoyances:- chef-client:- java:- jpackage:- ntp:- sudo:- tomcat:- users:- sample:roles:- base:- tc:- sample:data bags: - users: - mraynodes:- ec2 1: - role[base],role[tc],role[sample] - -S mray -i ~/.ssh/mray.pem -x ubuntu -G default -I ami-0c6ebd65 -f m1.small- rackspace 1: - role[base],role[tc],role[sample] - --image 114 --flavor 2 -i ~/.ssh/mray.pem

http://www.flickr.com/photos/mrchippy/443960682/

Here it is on GitHub

http://www.flickr.com/photos/mrchippy/443960682/

Versioned Infrastructure

• Documentation for your deployment

• Iterate over it

• Stored in version control

• Share with others

http://www.flickr.com/photos/mrchippy/443960682/

What's Next?

•Extract from chef-repo

•Extract from Chef Server

•Execute as knife plugin

http://www.flickr.com/photos/mrchippy/443960682/

Thanks!

• gem install spiceweasel

• http://github.com/mattray/spiceweasel

• Twitter/IRC/GitHub: mattray

• matt@opscode.com