chef or how to make computers do the work for us
DESCRIPTION
My KRUG (Kraków Ruby Users Group) presentation about automating boring tasks with Opscode's Chef.TRANSCRIPT
![Page 1: Chef or how to make computers do the work for us](https://reader033.vdocuments.mx/reader033/viewer/2022060107/554a335cb4c90520578b5352/html5/thumbnails/1.jpg)
C H E For how to make computers do the work for us
Marcin Kulik, Lunar Logic Polska
KRUG 2011/11/08
![Page 2: Chef or how to make computers do the work for us](https://reader033.vdocuments.mx/reader033/viewer/2022060107/554a335cb4c90520578b5352/html5/thumbnails/2.jpg)
Everyday we're dealing with mechanical, repetitive tasks... wecan automate.
![Page 3: Chef or how to make computers do the work for us](https://reader033.vdocuments.mx/reader033/viewer/2022060107/554a335cb4c90520578b5352/html5/thumbnails/3.jpg)
What is Chef?
![Page 4: Chef or how to make computers do the work for us](https://reader033.vdocuments.mx/reader033/viewer/2022060107/554a335cb4c90520578b5352/html5/thumbnails/4.jpg)
Automation tool
written in Ruby
![Page 5: Chef or how to make computers do the work for us](https://reader033.vdocuments.mx/reader033/viewer/2022060107/554a335cb4c90520578b5352/html5/thumbnails/5.jpg)
DSL
![Page 6: Chef or how to make computers do the work for us](https://reader033.vdocuments.mx/reader033/viewer/2022060107/554a335cb4c90520578b5352/html5/thumbnails/6.jpg)
Created by Opscode
![Page 7: Chef or how to make computers do the work for us](https://reader033.vdocuments.mx/reader033/viewer/2022060107/554a335cb4c90520578b5352/html5/thumbnails/7.jpg)
"Chef is an open source systems integration framework builtto bring the benefits of configuration management to your
entire infrastructure."
"You write source code to describe how you want each part ofyour infrastructure to be built, then apply those descriptions
to your servers."
"The result is a fully automated infrastructure: when a newserver comes on line, the only thing you have to do is tell Chef
what role it should play in your architecture."
![Page 8: Chef or how to make computers do the work for us](https://reader033.vdocuments.mx/reader033/viewer/2022060107/554a335cb4c90520578b5352/html5/thumbnails/8.jpg)
Why do you need it?
![Page 9: Chef or how to make computers do the work for us](https://reader033.vdocuments.mx/reader033/viewer/2022060107/554a335cb4c90520578b5352/html5/thumbnails/9.jpg)
Economics + Efficiency + Scalability
![Page 10: Chef or how to make computers do the work for us](https://reader033.vdocuments.mx/reader033/viewer/2022060107/554a335cb4c90520578b5352/html5/thumbnails/10.jpg)
Terms
![Page 11: Chef or how to make computers do the work for us](https://reader033.vdocuments.mx/reader033/viewer/2022060107/554a335cb4c90520578b5352/html5/thumbnails/11.jpg)
Noderemote server, local machine...
![Page 12: Chef or how to make computers do the work for us](https://reader033.vdocuments.mx/reader033/viewer/2022060107/554a335cb4c90520578b5352/html5/thumbnails/12.jpg)
Roleweb server, database server, ruby dev workstation...
![Page 13: Chef or how to make computers do the work for us](https://reader033.vdocuments.mx/reader033/viewer/2022060107/554a335cb4c90520578b5352/html5/thumbnails/13.jpg)
Cookbookmysql, ssh-access, dotfiles...
![Page 14: Chef or how to make computers do the work for us](https://reader033.vdocuments.mx/reader033/viewer/2022060107/554a335cb4c90520578b5352/html5/thumbnails/14.jpg)
Recipeinstall mysql-server, create database, add user...
![Page 15: Chef or how to make computers do the work for us](https://reader033.vdocuments.mx/reader033/viewer/2022060107/554a335cb4c90520578b5352/html5/thumbnails/15.jpg)
Resourcefile, dir, user, package, service, gem, virtual host...
![Page 16: Chef or how to make computers do the work for us](https://reader033.vdocuments.mx/reader033/viewer/2022060107/554a335cb4c90520578b5352/html5/thumbnails/16.jpg)
Run listlist of recipes to run in order
![Page 17: Chef or how to make computers do the work for us](https://reader033.vdocuments.mx/reader033/viewer/2022060107/554a335cb4c90520578b5352/html5/thumbnails/17.jpg)
{ "run_list": [ "recipe[mysql]", "recipe[git]", "recipe[ruby19]" ]}
![Page 18: Chef or how to make computers do the work for us](https://reader033.vdocuments.mx/reader033/viewer/2022060107/554a335cb4c90520578b5352/html5/thumbnails/18.jpg)
Cookbook structure |-- config | |-- node.json | `-- solo.rb |-- cookbooks | |-- book1 | | |-- attributes | | |-- files | | |-- metadata.rb | | |-- recipes | | | |-- default.rb | | | `-- source.rb | | `-- templates | |-- book2 | | |-- attributes | | | `-- default.rb | | |-- files | | |-- recipes | | | `-- default.rb | | `-- templates
![Page 19: Chef or how to make computers do the work for us](https://reader033.vdocuments.mx/reader033/viewer/2022060107/554a335cb4c90520578b5352/html5/thumbnails/19.jpg)
| | `-- templates | | `-- default | | `-- authorized_keys.erb | |-- book3 | | |-- attributes | | |-- files | | | `-- default | | | `-- secret-key | | |-- recipes | | | `-- default.rb | | `-- templates |-- config | |-- node.json | `-- solo.rb |-- cookbooks | |-- book1 | | |-- attributes | | |-- files | | |-- metadata.rb | | |-- recipes | | | |-- default.rb | | | `-- libs.rb | | `-- templates
Installation
![Page 20: Chef or how to make computers do the work for us](https://reader033.vdocuments.mx/reader033/viewer/2022060107/554a335cb4c90520578b5352/html5/thumbnails/20.jpg)
$ gem install chef
![Page 21: Chef or how to make computers do the work for us](https://reader033.vdocuments.mx/reader033/viewer/2022060107/554a335cb4c90520578b5352/html5/thumbnails/21.jpg)
Modes of operation
![Page 22: Chef or how to make computers do the work for us](https://reader033.vdocuments.mx/reader033/viewer/2022060107/554a335cb4c90520578b5352/html5/thumbnails/22.jpg)
Cookbooks stored
in central repository(free cookbooks hosting by Opscode:
https://manage.opscode.com/)
![Page 23: Chef or how to make computers do the work for us](https://reader033.vdocuments.mx/reader033/viewer/2022060107/554a335cb4c90520578b5352/html5/thumbnails/23.jpg)
$ sudo chef-client
![Page 24: Chef or how to make computers do the work for us](https://reader033.vdocuments.mx/reader033/viewer/2022060107/554a335cb4c90520578b5352/html5/thumbnails/24.jpg)
Cookbooks stored
on the node
![Page 25: Chef or how to make computers do the work for us](https://reader033.vdocuments.mx/reader033/viewer/2022060107/554a335cb4c90520578b5352/html5/thumbnails/25.jpg)
$ sudo chef-solo -c /path/to/cfg.rb -j /path/to/node-data.json
![Page 26: Chef or how to make computers do the work for us](https://reader033.vdocuments.mx/reader033/viewer/2022060107/554a335cb4c90520578b5352/html5/thumbnails/26.jpg)
Use cases
![Page 27: Chef or how to make computers do the work for us](https://reader033.vdocuments.mx/reader033/viewer/2022060107/554a335cb4c90520578b5352/html5/thumbnails/27.jpg)
Configure new machine
(in the cloud with Knife)Amazon EC2, Engine Yard, Linode, BrightBox...
![Page 28: Chef or how to make computers do the work for us](https://reader033.vdocuments.mx/reader033/viewer/2022060107/554a335cb4c90520578b5352/html5/thumbnails/28.jpg)
Manage config of existing
company serversClient demo apps (directory, vhost, god config), developers' ssh
keys...
![Page 29: Chef or how to make computers do the work for us](https://reader033.vdocuments.mx/reader033/viewer/2022060107/554a335cb4c90520578b5352/html5/thumbnails/29.jpg)
Bootstrap workstation!rvm + ruby 1.9, git, mysql, vim/emacs...
![Page 30: Chef or how to make computers do the work for us](https://reader033.vdocuments.mx/reader033/viewer/2022060107/554a335cb4c90520578b5352/html5/thumbnails/30.jpg)
Enough with theory!
![Page 31: Chef or how to make computers do the work for us](https://reader033.vdocuments.mx/reader033/viewer/2022060107/554a335cb4c90520578b5352/html5/thumbnails/31.jpg)
Lunar Stationhttps://github.com/LunarLogicPolska/lunar-station
![Page 32: Chef or how to make computers do the work for us](https://reader033.vdocuments.mx/reader033/viewer/2022060107/554a335cb4c90520578b5352/html5/thumbnails/32.jpg)
Lunar Station is a set of Chef cookbooks and a bash script (???)for bootstrapping developers machines at Lunar Logic Polska.
![Page 33: Chef or how to make computers do the work for us](https://reader033.vdocuments.mx/reader033/viewer/2022060107/554a335cb4c90520578b5352/html5/thumbnails/33.jpg)
You need ruby to run Chef
![Page 34: Chef or how to make computers do the work for us](https://reader033.vdocuments.mx/reader033/viewer/2022060107/554a335cb4c90520578b5352/html5/thumbnails/34.jpg)
(We assume) you use RVMNo need for system ruby for ruby devs nowadays
![Page 35: Chef or how to make computers do the work for us](https://reader033.vdocuments.mx/reader033/viewer/2022060107/554a335cb4c90520578b5352/html5/thumbnails/35.jpg)
bootstrap.sh
![Page 36: Chef or how to make computers do the work for us](https://reader033.vdocuments.mx/reader033/viewer/2022060107/554a335cb4c90520578b5352/html5/thumbnails/36.jpg)
detects platform (Ubuntu, Fedora, OSX)
installs compilers and other RVMdependencies
installs RVM & ruby 1.9 & chef gem
downloads latest Lunar Stationcookbooks
runs chef-solo
![Page 37: Chef or how to make computers do the work for us](https://reader033.vdocuments.mx/reader033/viewer/2022060107/554a335cb4c90520578b5352/html5/thumbnails/37.jpg)
$ curl -skL http://bit.ly/lunar-station | bashInitializing Lunar Workstation...>> Fedora Linux detected.>> Checking for RVM...>> Fetching latest version of Lunar Station cookbooks...>> Starting chef-solo run...[Mon, 07 Nov 2011 22:19:54 +0100] INFO: *** Chef 0.10.4 ***[Mon, 07 Nov 2011 22:19:54 +0100] INFO: Setting the run_list to ...
![Page 38: Chef or how to make computers do the work for us](https://reader033.vdocuments.mx/reader033/viewer/2022060107/554a335cb4c90520578b5352/html5/thumbnails/38.jpg)
Nodes
![Page 39: Chef or how to make computers do the work for us](https://reader033.vdocuments.mx/reader033/viewer/2022060107/554a335cb4c90520578b5352/html5/thumbnails/39.jpg)
# linux-rubydev.json
{ "run_list": [ "role[rubydev]" ]}
![Page 40: Chef or how to make computers do the work for us](https://reader033.vdocuments.mx/reader033/viewer/2022060107/554a335cb4c90520578b5352/html5/thumbnails/40.jpg)
# osx-rubydev.json
{ "run_list": [ "role[osx]", "role[rubydev]" ]}
![Page 41: Chef or how to make computers do the work for us](https://reader033.vdocuments.mx/reader033/viewer/2022060107/554a335cb4c90520578b5352/html5/thumbnails/41.jpg)
Roles
![Page 42: Chef or how to make computers do the work for us](https://reader033.vdocuments.mx/reader033/viewer/2022060107/554a335cb4c90520578b5352/html5/thumbnails/42.jpg)
# base.rb
run_list 'recipe[repos]', 'recipe[curl]', 'recipe[wget]', 'recipe[git]', 'recipe[libxml2]', 'recipe[ack]', 'recipe[vim]', 'recipe[ctags]', 'recipe[skype]', 'recipe[firefox]' , 'recipe[google-chrome]'
![Page 43: Chef or how to make computers do the work for us](https://reader033.vdocuments.mx/reader033/viewer/2022060107/554a335cb4c90520578b5352/html5/thumbnails/43.jpg)
# rubydev.rb
run_list 'role[base]', 'recipe[mysql]'
![Page 44: Chef or how to make computers do the work for us](https://reader033.vdocuments.mx/reader033/viewer/2022060107/554a335cb4c90520578b5352/html5/thumbnails/44.jpg)
# osx.rb
run_list "recipe[homebrew]"
![Page 45: Chef or how to make computers do the work for us](https://reader033.vdocuments.mx/reader033/viewer/2022060107/554a335cb4c90520578b5352/html5/thumbnails/45.jpg)
Cookbooks
![Page 46: Chef or how to make computers do the work for us](https://reader033.vdocuments.mx/reader033/viewer/2022060107/554a335cb4c90520578b5352/html5/thumbnails/46.jpg)
repos cookbook
![Page 47: Chef or how to make computers do the work for us](https://reader033.vdocuments.mx/reader033/viewer/2022060107/554a335cb4c90520578b5352/html5/thumbnails/47.jpg)
# cookbooks/repos/recipes/default.rb
case node[:platform]when 'fedora' path = "/tmp/rpmfusion-free-release-stable.noarch.rpm"
bash "download rpmfusion free package" do code "wget http://download1.rpmfusion.org/.../" + "rpmfusion-free-release-stable.noarch.rpm -O #{path}"
not_if { File.exist?(path) } end
package "rpmfusion-free-release-stable" do source path options "--nogpgcheck" end
when 'ubuntu' ...end
![Page 48: Chef or how to make computers do the work for us](https://reader033.vdocuments.mx/reader033/viewer/2022060107/554a335cb4c90520578b5352/html5/thumbnails/48.jpg)
end
# cookbooks/repos/recipes/default.rb
case node[:platform]when 'fedora' ...
when 'ubuntu' bash "enable multiverse repo" do code "head -n 1 /etc/apt/sources.list | " + "sed 's/main universe/multiverse/' " + ">> /etc/apt/sources.list"
not_if "egrep '^deb.+multiverse' /etc/apt/sources.list" endend
![Page 49: Chef or how to make computers do the work for us](https://reader033.vdocuments.mx/reader033/viewer/2022060107/554a335cb4c90520578b5352/html5/thumbnails/49.jpg)
vim cookbook
![Page 50: Chef or how to make computers do the work for us](https://reader033.vdocuments.mx/reader033/viewer/2022060107/554a335cb4c90520578b5352/html5/thumbnails/50.jpg)
# cookbooks/vim/recipes/default.rb
case node[:platform]when "ubuntu" package "vim" package "vim-gnome"
when "fedora" package "vim-enhanced" package "vim-X11"
when 'mac_os_x' package "macvim"end
![Page 51: Chef or how to make computers do the work for us](https://reader033.vdocuments.mx/reader033/viewer/2022060107/554a335cb4c90520578b5352/html5/thumbnails/51.jpg)
skype cookbook
![Page 52: Chef or how to make computers do the work for us](https://reader033.vdocuments.mx/reader033/viewer/2022060107/554a335cb4c90520578b5352/html5/thumbnails/52.jpg)
# cookbooks/skype/recipes/default.rb
case node[:platform]when 'ubuntu' include_recipe 'init::ubuntu' # for partner repo
package 'skype'
when 'mac_os_x' dmg_package "Skype" do source "http://www.skype.com/go/getskype-macosx.dmg" action :install end
when 'fedora' ...end
![Page 53: Chef or how to make computers do the work for us](https://reader033.vdocuments.mx/reader033/viewer/2022060107/554a335cb4c90520578b5352/html5/thumbnails/53.jpg)
Lunar Kitchen
![Page 54: Chef or how to make computers do the work for us](https://reader033.vdocuments.mx/reader033/viewer/2022060107/554a335cb4c90520578b5352/html5/thumbnails/54.jpg)
Source of LLP servers configuration data and a set of Chefcookbooks
![Page 55: Chef or how to make computers do the work for us](https://reader033.vdocuments.mx/reader033/viewer/2022060107/554a335cb4c90520578b5352/html5/thumbnails/55.jpg)
chef-solo invoked onremote machines
no chef server
![Page 56: Chef or how to make computers do the work for us](https://reader033.vdocuments.mx/reader033/viewer/2022060107/554a335cb4c90520578b5352/html5/thumbnails/56.jpg)
Each server we configure has its corresponding nodeconfiguration file in nodes/ directory of kitchen project that
specifies run_list and few other settings
![Page 57: Chef or how to make computers do the work for us](https://reader033.vdocuments.mx/reader033/viewer/2022060107/554a335cb4c90520578b5352/html5/thumbnails/57.jpg)
# nodes/deneb.json
{ "run_list": [ "recipe[ssh_access]" ],
"ssh_access": [ "marcin.kulik", "anna.lesniak", ...],
"opened_ports": { "tcp": [80, 443, 22, 8080], "udp": [] }, ...
![Page 58: Chef or how to make computers do the work for us](https://reader033.vdocuments.mx/reader033/viewer/2022060107/554a335cb4c90520578b5352/html5/thumbnails/58.jpg)
How do we run chef-soloon remote machine?
![Page 59: Chef or how to make computers do the work for us](https://reader033.vdocuments.mx/reader033/viewer/2022060107/554a335cb4c90520578b5352/html5/thumbnails/59.jpg)
Capistrano!
![Page 60: Chef or how to make computers do the work for us](https://reader033.vdocuments.mx/reader033/viewer/2022060107/554a335cb4c90520578b5352/html5/thumbnails/60.jpg)
# See the list of configured servers:
$ cap -T
# Make the changes happen on the server:
$ cap configure:deneb
![Page 61: Chef or how to make computers do the work for us](https://reader033.vdocuments.mx/reader033/viewer/2022060107/554a335cb4c90520578b5352/html5/thumbnails/61.jpg)
How does Capfile look like?
![Page 62: Chef or how to make computers do the work for us](https://reader033.vdocuments.mx/reader033/viewer/2022060107/554a335cb4c90520578b5352/html5/thumbnails/62.jpg)
set :user, 'chef'
NODE_LIST = Dir["nodes/*.json"].map do |nodefile| File.basename(nodefile, '.json')end
NODE_LIST.each do |node| role node.to_sym, nodeend
NODE_CONFIG = <<-EOS file_cache_path '/tmp/chef-solo' cookbook_path '/tmp/chef-solo/cookbooks' role_path '/tmp/chef-solo/roles'EOS...
![Page 63: Chef or how to make computers do the work for us](https://reader033.vdocuments.mx/reader033/viewer/2022060107/554a335cb4c90520578b5352/html5/thumbnails/63.jpg)
...namespace :configure do NODE_LIST.each do |node| desc "Configure #{node}" task node.to_sym, :roles => node.to_sym do run "if [ ! -e /tmp/chef-solo ]; then mkdir /tmp/chef-solo; fi" upload("cookbooks", "/tmp/chef-solo/", :via => :scp, :recursive => true) upload("roles", "/tmp/chef-solo/", :via => :scp, :recursive => true) upload("nodes/#{node}.json", "/tmp/chef-solo/node.json", :via => :scp) put(NODE_CONFIG, "/tmp/chef-solo/solo.rb") run "rvmsudo chef-solo " + "-c /tmp/chef-solo/solo.rb " + "-j /tmp/chef-solo/node.json" end endend
![Page 64: Chef or how to make computers do the work for us](https://reader033.vdocuments.mx/reader033/viewer/2022060107/554a335cb4c90520578b5352/html5/thumbnails/64.jpg)
SSH access
![Page 65: Chef or how to make computers do the work for us](https://reader033.vdocuments.mx/reader033/viewer/2022060107/554a335cb4c90520578b5352/html5/thumbnails/65.jpg)
├── Capfile├── config├── cookbooks├── nodes├── README.md├── roles└── ssh_keys ├── anna.lesniak ├── artur.bilski ├── ... └── marcin.kulik
![Page 66: Chef or how to make computers do the work for us](https://reader033.vdocuments.mx/reader033/viewer/2022060107/554a335cb4c90520578b5352/html5/thumbnails/66.jpg)
# cookbooks/access/recipes/default.rb
username = 'dev'
ssh_keys = node[:ssh_access].map do |f| File.read("/tmp/chef-solo/ssh_keys/#{f}")end
template "/home/#{username}/.ssh/authorized_keys" do source "authorized_keys.erb" owner username group 'users' mode "0600" variables :ssh_keys => ssh_keysend
![Page 67: Chef or how to make computers do the work for us](https://reader033.vdocuments.mx/reader033/viewer/2022060107/554a335cb4c90520578b5352/html5/thumbnails/67.jpg)
# cookbooks/access/templates/authorized_keys.erb
# Generated by Chef, do not edit!
<%= @ssh_keys.join("\n") %>
![Page 68: Chef or how to make computers do the work for us](https://reader033.vdocuments.mx/reader033/viewer/2022060107/554a335cb4c90520578b5352/html5/thumbnails/68.jpg)
Tips
![Page 69: Chef or how to make computers do the work for us](https://reader033.vdocuments.mx/reader033/viewer/2022060107/554a335cb4c90520578b5352/html5/thumbnails/69.jpg)
Learn step by stepEC2 + Chef + Knife + Opscode... = Fuuuuuuuuuuuuuuuuuuuuu
![Page 70: Chef or how to make computers do the work for us](https://reader033.vdocuments.mx/reader033/viewer/2022060107/554a335cb4c90520578b5352/html5/thumbnails/70.jpg)
Start with chef-solo
![Page 71: Chef or how to make computers do the work for us](https://reader033.vdocuments.mx/reader033/viewer/2022060107/554a335cb4c90520578b5352/html5/thumbnails/71.jpg)
Run on local machineEasy to troubleshoot problems
![Page 72: Chef or how to make computers do the work for us](https://reader033.vdocuments.mx/reader033/viewer/2022060107/554a335cb4c90520578b5352/html5/thumbnails/72.jpg)
Use Vagranthttp://vagrantup.com/
Great for testing cookbooks - doesn't pollute your system
![Page 73: Chef or how to make computers do the work for us](https://reader033.vdocuments.mx/reader033/viewer/2022060107/554a335cb4c90520578b5352/html5/thumbnails/73.jpg)
Q?
![Page 74: Chef or how to make computers do the work for us](https://reader033.vdocuments.mx/reader033/viewer/2022060107/554a335cb4c90520578b5352/html5/thumbnails/74.jpg)
[email protected] | @sickill | https://github.com/sickill