chefconf 2012 spiceweasel

Spiceweasel: Your Infrastructure as Code Table of Contents

Matt [email protected]

Twitter/IRC/GitHub: mattray

mailto:[email protected]




Infrastructure as Code

Enable the reconstruction of the business from nothing but

a source code repository, application data backup and

bare metal resources.

That's great but...how did I get here?

http://www.flickr.com/photos/mrchippy/443960682/

Spiceweasel

• Infrastructure deployment manifest

• Converts YAML/JSON to knife commands

• http://github.com/mattray/spiceweasel



https://github.com/mattray/spiceweasel





YAML

• Simple

• Easy to read

• Easy to reason

• spiceweasel infrastructure.yml




JSON

• Patches welcomed.

• spiceweasel infrastructure.json



cookbooks:- annoyances:- apache2:- chef-client: - 1.1.5- java:- mysql:- tomcat:

knife cookbook upload annoyancesknife cookbook site download apache2 --file cookbooks/apache2.tgz tar -C cookbooks/ -xf cookbooks/apache2.tgzrm -f cookbooks/apache2.tgzknife cookbook upload apache2knife cookbook site download chef-client 1.1.5 --file cookbooks/chef-client.tgz tar -C cookbooks/ -xf cookbooks/chef-client.tgzrm -f cookbooks/chef-client.tgzknife cookbook upload chef-clientknife cookbook upload javaknife cookbook upload mysqlknife cookbook upload tomcat

Cookbooks


environments:- development:- qa:- preprod:- production:

knife environment from file development.rbknife environment from file qa.rbknife environment from file preprod.rbknife environment from file production.rb

Environments




Roles

roles:- base:- glance-single-machine:- nova-db:- nova-multi-compute:- nova-multi-controller:- nova-rabbitmq-server:- nova-single-machine:

http://www.flickr.com/photos/laenulfean/374398044/

knife role from file base.rbknife role from file glance-single-machine.rbknife role from file nova-db.rbknife role from file nova-multi-compute.rbknife role from file nova-multi-controller.rbknife role from file nova-rabbitmq-server.rbknife role from file nova-single-machine.rb




Data Bags

data bags:- users: - alice - bob - chuck- data: - *- passwords: - secret secret_key - mysql - rabbitmq

knife data bag create usersknife data bag from file users alice.jsonknife data bag from file users bob.jsonknife data bag from file users chuck.jsonknife data bag create dataknife data bag from file data dataA.jsonknife data bag from file data dataB.jsonknife data bag create passwordsknife data bag from file passwords mysql.json --secret-file secret_keyknife data bag from file passwords rabbitmq.json --secret-file secret_key



http://www.flickr.com/photos/75659300@N00/2615848530/

nodes:- ubuntu1-1004.vm ubuntu2-1004.vm ubuntu3-1004.vm: - role[base],role[tc],role[sample] - -i ~/.ssh/mray.pem -x ubuntu --sudo

knife bootstrap ubuntu1-1004.vm -i ~/.ssh/mray.pem -x ubuntu --sudo -r 'role[base],role[tc],role[sample]'knife bootstrap ubuntu2-1004.vm -i ~/.ssh/mray.pem -x ubuntu --sudo -r 'role[base],role[tc],role[sample]'knife bootstrap ubuntu3-1004.vm -i ~/.ssh/mray.pem -x ubuntu --sudo -r 'role[base],role[tc],role[sample]'

Nodes




nodes:- windows_winrm winboxA: - role[base],role[sqlserver] - -x Administrator -P 'super_secret_password'- windows_ssh winboxB winboxC: - role[base],role[iis] - -x Administrator -P 'super_secret_password'

knife bootstrap windows winrm winboxA -x Administrator -P 'super_secret_password' -r 'role[base],role[sqlserver]'knife bootstrap windows ssh winboxB -x Administrator -P 'super_secret_password' -r 'role[base],role[iis]'knife bootstrap windows ssh winboxC -x Administrator -P 'super_secret_password' -r 'role[base],role[iis]'

Nodes: Windows




nodes:- ec2 5: - role[base],role[tc],role[sample] - -S mray -i ~/.ssh/mray.pem -x ubuntu -I ami-0c6ebd65 -f m1.small- rackspace 5: - role[base],role[tc],role[sample] - --image 114 --flavor 2 -i ~/.ssh/mray.pem- openstack 5: - role[base],role[tc],role[sample] - --image 7 --flavor 2 -x ubuntu -i ~/.ssh/mray.pem

seq 5 | parallel -j 0 -v "knife ec2 server create -S mray -i ~/.ssh/mray.pem -x ubuntu -I ami-0c6ebd65 -f m1.small -r 'role[base],role[tc],role[sample]'"seq 5 | parallel -j 0 -v "knife rackspace server create --image 114 --flavor 2 -i ~/.ssh/mray.pem -r 'role[base],role[tc],role[sample]'"seq 5 | parallel -j 0 -v "knife openstack server create --image 7 --flavor 2 -i ~/.ssh/mray.pem -r 'role[base],role[tc],role[sample]'"

Nodes: Cloud Providers



Validation

• Cookbook versions and their dependencies exist

• Environment names and referenced cookbooks are correct

• Role names and referenced roles and cookbooks are correct

• Data bags parse, secret keys are present

• Node run lists items are correct and their Environment exists


What's the Point?

• Nodes are ephemeral

• Everything we do is in code.

• Everything is checked into version control.

• Basic roadmap for deployment.




Break it Down

How do we recreate our infrastructure?


Real World Example

• Whiteboard it out

• Break it down into components

• Comment it all out and start working




Here's the YAML

cookbooks:- annoyances:- chef-client:- java:- jpackage:- ntp:- sudo:- tomcat:- users:- sample:roles:- base:- tc:- sample:data bags: - users: - mraynodes:- ec2 1: - role[base],role[tc],role[sample] - -S mray -i ~/.ssh/mray.pem -x ubuntu -G default -I ami-0c6ebd65 -f m1.small- rackspace 1: - role[base],role[tc],role[sample] - --image 114 --flavor 2 -i ~/.ssh/mray.pem




Here it is on GitHub


Versioned Infrastructure

• Documentation for your deployment

• Iterate over it

• Stored in version control

• Share with others




What's Next?

•Extract from chef-repo

•Extract from Chef Server

•Execute as knife plugin












Thanks!

• gem install spiceweasel

• http://github.com/mattray/spiceweasel

• Twitter/IRC/GitHub: mattray

• [email protected]









chefconf 2012 spiceweasel

Technology

knife rolefrom file

knife environmentfrom

data data

json knife data bag

key knife data bag

secretfile secret

file data datab

file data dataa