cooking perl with chef
DESCRIPTION
Reliable and scalable applications need repeatable, automated application deployment. Configuration management tools like Chef, Puppet and others make it easy to deploy an entire application stack, but support for Perl applications has lagged behind other popular, dynamic languages. The Perl community has responded to these challenges with tools like perlbrew, local::lib, carton and others to make it easier to manage an application and its dependencies in isolation. This presentation will show you how to make those tools work with Chef for complete automation of Perl application deployment.TRANSCRIPT
ConfigurationManagement
(e.g. chef, puppet, cfengine, ...)
Unknown state
↓Target state
New machine
↓Deployed app
Infrastructure as code
automated!repeatable!
testable!
(no manual steps, checklist, etc.)
One tool to deploythe whole stack
(DB, caching, messaging, ...)
But wait!Isn't that hard to do for Perl apps?
Perl applications are complex
Dependency hell
App = perl + CPAN + your code
App = perl + CPAN + your code
CHIDateTimeDBIJSONMoosePlackPOETry::Tiny...
App = perl + CPAN + your code
CHIDateTimeDBIJSONMoosePlackPOETry::Tiny...
your application is the versioned set of all its compontents
App = perl + CPAN + your code
CHIDateTimeDBIJSONMoosePlackPOETry::Tiny...
v1.0.0
your application is the versioned set of all its compontents
App = Perl + CPAN + your code
CHIDateTimeDBIJSONMoosePlackPOETry::Tiny...
v1.0.0 v5.14.2
your application is the versioned set of all its compontents
App = Perl + CPAN + your code
0.550.761.6222.532.06030.99891.3540.11...
v1.0.0 v5.14.2
your application is the versioned set of all its compontents
App = Perl + CPAN + your code
0.550.761.6222.532.06030.99891.3540.11...
v1.0.0 v5.14.2 v1.0
your application is the versioned set of all its compontents
App = Perl + CPAN + your code
0.550.761.6222.532.06030.99891.3540.11...
v1.0.0 v5.14.2 v1.0
change one piece...
App = Perl + CPAN + your code
0.550.761.6222.532.06030.99891.3540.11...
v1.0.0 v5.16.0 v1.0
change one piece...
App = Perl + CPAN + your code
0.550.761.6222.532.06030.99891.3540.11...
v1.0.1 v5.16.0 v1.0
… and you have a new version of your application
Repeatable deployment means...
Repeatable deployment means...
... the same Perl
Repeatable deployment means...
... the same Perl
... the same modules
Repeatable deployment means...
... the same Perl
... the same modules
... the same code
Repeatable deployment means...
... the same Perl
... the same modules
... the same code
... on demand
Easy...
If we have the right tools
Use one-size fits all distribution packagers like apt, yum, etc...?
(How much do you like your system perl?!)
This problem is notunique to Perl
Let's be inspired by Larry
[larry hat pic]
Or better yet, Paul
YARRR!
Great hackers steal!
Great hackers steal ideas
Consider Chef...
(written in ruby; various ruby app stacks)
Chef ❤ Ruby
(virtualenv; pip; django apps)
Chef ❤ Python
Common patterns emerge
python → virtualenv + pipruby → rvm + Bundler
We've built tools like these, too
Kang-min Liu (gugod)
Matt S Trout (mst)
Tatsuhiko Miyagawa
(and a big community helping them)
perlbrew – multiple perl manager
Matt S Trout (mst)
Tatsuhiko Miyagawa
(and a big community helping them)
perlbrew – multiple perl manager
local::lib – custom @INC manager
Tatsuhiko Miyagawa
(and a big community helping them)
perlbrew – multiple perl manager
local::lib – custom @INC manager
carton – versioned dependency installer
(and a big community helping them)
So we have the pieces
Repeatable deployment in five parts
Repeatable deployment in five parts
application-specific Perl
application-specific @INC path
versioned application code
versioned module dependencies
automate the previous four
Repeatable deployment in five parts
perlbrew
application-specific @INC path
versioned application code
versioned module dependencies
automate the previous four
Repeatable deployment in five parts
perlbrew
local::lib
versioned application code
versioned module dependencies
automate the previous four
Repeatable deployment in five parts
perlbrew
local::lib
git
versioned module dependencies
automate the previous four
Repeatable deployment in five parts
perlbrew
local::lib
git
carton
automate the previous four
Repeatable deployment in five parts
perlbrew
local::lib
git
carton
@&$%!
[swedish chef FAIL pic]
No support in Chef...
So I implemented it
(In Ruby)
(After I learned some Ruby)
(perlbrew; local::lib; carton)
Chef ❤ Perl
Now...
“Cookbook”
A collection of components to configurea particular application
Typically includes recipes, providers, templates, etc.
(CPAN analogy → “distribution”)
“Recipe”
Component applied that deploys an application or service
Typically declarative, specifying desired resources and associated configuration
“Resource”
An abstraction of something to be deployed
“Provider”
Platform-specific implementation to deliver a resource
“Node”
A host computer managed with Chef
Often means the configuration file that defines recipes, attributes and roles that define the target state of a host
“Attribute”
A variable used in a recipe and/or provider that customizes the configuration of a resource
Attributes have defaults, but can be customized for nodes or roles
“Role”
A collection of recipes and attributes used to apply common configuration across multiple nodes
Summary...
cookbooks include recipes and providers
roles, recipes and attributes get applied to nodes
recipes specify desired resources and customize them with attributes
providers do the work of deploying resources
I wrote two Perl Chef cookbooksfor the Chef community repository
(which is like CPAN circa 1996 or so)
http://community.opscode.com/
1. perlbrew – for managing perls
2. carton – for deploying apps
Also available here: https://github.com/dagolden/perl-chef
perlbrew cookbook resources:
perlbrew_perl – install a perl
perlbrew_lib – create a local::lib
perlbrew_cpanm – install modules to perl or lib
perlbrew_run – run shell commands under aparticular perlbrew and/or lib
carton cookbook resource:
carton_app – deploy an app with carton
– start in directory with the app source– configure for a specific perlbrew perl– install versioned dependencies with carton– create a runit service for the app– start the app
Time for an example:
Deploying a “Hello World” Plack app
https://github.com/dagolden/zzz-hello-world
1. Write the application
2. Use carton to create a carton.lock file with versioned dependency info
3. Write a simple cookbook for the application
4. Check it all into git
5. Deploy the application with Chef
Steps for creating Hello World
$ tree.├── Changes├── Makefile.PL├── app.psgi├── carton.lock├── cookbook│ └── hello-world│ ├── README.md│ ├── attributes│ │ └── default.rb│ ├── metadata.rb│ └── recipes│ └── default.rb└── lib └── ZZZ └── Hello └── World.pm
$ tree.├── Changes├── Makefile.PL├── app.psgi├── carton.lock├── cookbook│ └── hello-world│ ├── README.md│ ├── attributes│ │ └── default.rb│ ├── metadata.rb│ └── recipes│ └── default.rb└── lib └── ZZZ └── Hello └── World.pm
use strict;use warnings;use ZZZ::Hello::World;my $app = sub { ZZZ::Hello::World->run_psgi(@_) };
(this Plack app just invokes a simple module)
$ tree.├── Changes├── Makefile.PL├── app.psgi├── carton.lock├── cookbook│ └── hello-world│ ├── README.md│ ├── attributes│ │ └── default.rb│ ├── metadata.rb│ └── recipes│ └── default.rb└── lib └── ZZZ └── Hello └── World.pm
use 5.008001;use strict;use warnings;
package ZZZ::Hello::World;our $VERSION = "1.0";
use Plack::Request;
sub run_psgi { my $self = shift; my $req = Plack::Request->new(shift); my $res = $req->new_response(200); $res->content_type('text/html'); $res->body(<<"HERE");<html><head><title>Hello World</title></head><body><p>Hello World. It is @{[scalar localtime]}</p>...</body></html>HERE return $res->finalize;}
1;(the module just returns some dynamic HTML)
$ tree.├── Changes├── Makefile.PL├── app.psgi├── carton.lock├── cookbook│ └── hello-world│ ├── README.md│ ├── attributes│ │ └── default.rb│ ├── metadata.rb│ └── recipes│ └── default.rb└── lib └── ZZZ └── Hello └── World.pm
use inc::Module::Install;name 'ZZZ-Hello-World';version '1.0'; requires 'Plack';requires 'Starman'; WriteAll;
(the Makefile.PL also includes deployment dependencies like Starman)
During development, carton installs dependencies locally and creates a versioned dependency file called carton.lock
$ carton install# installs dependencies into a local directory# creates carton.lock if it doesn't exist# carton.lock is a JSON file of dependency info
During deployment, carton installs dependencies from carton.lock and runs the app with them
$ carton install# installs dependencies into a local directory
$ carton exec Ilib starman p 8080 app.psgi# runs the app using carton installed deps
$ tree.├── Changes├── Makefile.PL├── app.psgi├── carton.lock├── cookbook│ └── hello-world│ ├── README.md│ ├── attributes│ │ └── default.rb│ ├── metadata.rb│ └── recipes│ └── default.rb└── lib └── ZZZ └── Hello └── World.pm
# carton.lock JSON{ "modules" : { "Class::Inspector" : { "dist" : "Class-Inspector-1.27", "module" : "Class::Inspector", "pathname" : "A/AD/ADAMK/Class-Inspector-1.27.tar.gz", ... }. "Data::Dump" : { ... }, "Devel::StackTrace" : { ... }, "Encode::Locale" : { ... }, ...}
(carton.lock associates module names to specific versions of those module)
$ tree.├── Changes├── Makefile.PL├── app.psgi├── carton.lock├── cookbook│ └── hello-world│ ├── README.md│ ├── attributes│ │ └── default.rb│ ├── metadata.rb│ └── recipes│ └── default.rb└── lib └── ZZZ └── Hello └── World.pm
# perlbrew to execute withdefault['hello-world']['perl_version'] = 'perl-5.16.0'
# Install directory, repo and tagdefault['hello-world']['deploy_dir'] = '/opt/hello-world'default['hello-world']['deploy_repo'] =
'https://github.com/dagolden/zzz-hello-world.git'default['hello-world']['deploy_tag'] = 'master'
# Service user/group/portdefault['hello-world']['user'] = "nobody"default['hello-world']['group'] = "nogroup"default['hello-world']['port'] = 8080
(attributes are variables used in the recipe; can be customized per-node during deployment)
$ tree.├── Changes├── Makefile.PL├── app.psgi├── carton.lock├── cookbook│ └── hello-world│ ├── README.md│ ├── attributes│ │ └── default.rb│ ├── metadata.rb│ └── recipes│ └── default.rb└── lib └── ZZZ └── Hello └── World.pm
include_recipe 'carton'
package 'git-core'
git node['hello-world']['deploy_dir'] do repository node['hello-world']['deploy_repo'] reference node['hello-world']['deploy_tag'] notifies :restart, "carton_app[hello-world]"end
carton_app "hello-world" do perlbrew node['hello-world']['perl_version'] command "starman -p #{node['hello-world']['port']} app.psgi" cwd node['hello-world']['deploy_dir'] user node['hello-world']['user'] group node['hello-world']['group']end
carton_app "hello-world" do action :startend
(recipe ensures carton and git are available...)
include_recipe 'carton'
package 'git-core'
git node['hello-world']['deploy_dir'] do repository node['hello-world']['deploy_repo'] reference node['hello-world']['deploy_tag'] notifies :restart, "carton_app[hello-world]"end
carton_app "hello-world" do perlbrew node['hello-world']['perl_version'] command "starman -p #{node['hello-world']['port']} app.psgi" cwd node['hello-world']['deploy_dir'] user node['hello-world']['user'] group node['hello-world']['group']end
carton_app "hello-world" do action :startend
(git resource specifies where application code goes...)
include_recipe 'carton'
package 'git-core'
git node['hello-world']['deploy_dir'] do repository node['hello-world']['deploy_repo'] reference node['hello-world']['deploy_tag'] notifies :restart, "carton_app[hello-world]"end
carton_app "hello-world" do perlbrew node['hello-world']['perl_version'] command "starman -p #{node['hello-world']['port']} app.psgi" cwd node['hello-world']['deploy_dir'] user node['hello-world']['user'] group node['hello-world']['group']end
carton_app "hello-world" do action :startend
(attributes parameterize the resource statement...)
include_recipe 'carton'
package 'git-core'
git node['hello-world']['deploy_dir'] do repository node['hello-world']['deploy_repo'] reference node['hello-world']['deploy_tag'] notifies :restart, "carton_app[hello-world]"end
carton_app "hello-world" do perlbrew node['hello-world']['perl_version'] command "starman -p #{node['hello-world']['port']} app.psgi" cwd node['hello-world']['deploy_dir'] user node['hello-world']['user'] group node['hello-world']['group']end
carton_app "hello-world" do action :startend
(carton_app resources installs deps and sets up runit service...)
include_recipe 'carton'
package 'git-core'
git node['hello-world']['deploy_dir'] do repository node['hello-world']['deploy_repo'] reference node['hello-world']['deploy_tag'] notifies :restart, "carton_app[hello-world]"end
carton_app "hello-world" do perlbrew node['hello-world']['perl_version'] command "starman -p #{node['hello-world']['port']} app.psgi" cwd node['hello-world']['deploy_dir'] user node['hello-world']['user'] group node['hello-world']['group']end
carton_app "hello-world" do action :startend
(again, attributes parameterize the resource...)
include_recipe 'carton'
package 'git-core'
git node['hello-world']['deploy_dir'] do repository node['hello-world']['deploy_repo'] reference node['hello-world']['deploy_tag'] notifies :restart, "carton_app[hello-world]"end
carton_app "hello-world" do perlbrew node['hello-world']['perl_version'] command "starman -p #{node['hello-world']['port']} app.psgi" cwd node['hello-world']['deploy_dir'] user node['hello-world']['user'] group node['hello-world']['group']end
carton_app "hello-world" do action :startend
(finally, the resource is idempotently started...)
These files – and the Perl Chefcookbooks – are all you need
Enough code... let's see how to deploy it
1. Set up a Vagrant virtual machine
2. Prepare Pantry to manage Chef Solo
3. Get Hello World cookbook and dependencies
4. Configure virtual machine for Hello World
5. Deploy
Steps for deployment of Hello World
1. Set up a Vagrant virtual machine
2. Prepare Pantry to manage Chef Solo
3. Get Hello World cookbook and dependencies
4. Configure virtual machine for Hello World
5. Deploy
Steps for deployment of Hello World
Vagrant is a tool for managing virtual machines
“Can I have a VirtualBox now, please?”
Vagrant is a tool for managing virtual machines
$ vagrant box add base \ http://files.vagrantup.com/lucid32.box
$ vagrant init
$ vagrant up
Vagrant is great for testing Chef deployment
(and other things, besides)
1. Set up a Vagrant virtual machine
2. Prepare Pantry to manage Chef Solo
3. Get Hello World cookbook and dependencies
4. Configure virtual machine for Hello World
5. Deploy
Steps for deployment of Hello World
Chef Solo is Chef without a centralconfiguration server
(good for demos and smaller deployments)
Chef – you push config data to Chef Server – nodes run Chef Client to pull config
from Chef Server and execute it
Chef Solo – you push config data to nodes– you run Chef Solo remotely
One advantage of Chef Solo...
Your config repo is canonical
(i.e. you don't have to track what you've pushed to the central server)
One dis-advantage of Chef Solo...
Manual rsync/ssh required (yuck!)
1. Set up a Vagrant virtual machine
2. Prepare Pantry to manage Chef Solo
3. Get Hello World cookbook and dependencies
4. Configure virtual machine for Hello World
5. Deploy
Steps for deployment of Hello World
Pantry is a tool for automating Chef Solo
Pantry is a tool for automating Chef Solo
$ pantry create node server.example.com
$ pantry apply node server.example.com \ --role web --recipe myapp
$ pantry sync node server.example.com
Pantry is written in Perl and available on CPAN
(Similar to pocketknife [Ruby] and littlechef [Python])
Finally, a demonstration...
Screencast available athttp://youtu.be/H93rt-KtwBE
1. Set up a Vagrant virtual machine
2. Prepare Pantry to manage Chef Solo
3. Get Hello World cookbook and dependencies
4. Configure virtual machine for Hello World
5. Deploy
Steps for deployment of Hello World
$ vagrant init# create config file
$ vim Vagrantfile# edit to forward local port 8080 to# virtual machine port 8080
$ vagrant up# launch it
$ vagrant ssh# check that it's up, then exit
1. Set up a Vagrant virtual machine
2. Prepare Pantry to manage Chef Solo
3. Get Hello World cookbook and dependencies
4. Configure virtual machine for Hello World
5. Deploy
Steps for deployment of Hello World
$ pantry init# create directories to hold Chef Solo config
$ pantry create node vagrant \ host localhost \ port 8080 \ user vagrant# create a node config file
$ sshadd ~/.vagrant.d/insecure_private_key# allow pantry to ssh to vagrant machine
(Important tip: remove the insecure_private_key after you no longer need it because Github chokes on it.)
1. Set up a Vagrant virtual machine
2. Prepare Pantry to manage Chef Solo
3. Get Hello World cookbook and dependencies
4. Configure virtual machine for Hello World
5. Deploy
Steps for deployment of Hello World
Four cookbooks must be downloadedand copied to the 'cookbooks' directory
– hello-world– carton– perlbrew– runit
1. Set up a Vagrant virtual machine
2. Prepare Pantry to manage Chef Solo
3. Get Hello World cookbook and dependencies
4. Configure virtual machine for Hello World
5. Deploy
Steps for deployment of Hello World
$ pantry apply node vagrant recipe helloworld# apply recipe to node configuration
$ pantry apply node vagrant default \ helloworld.perl_version=perl5.14.2# override a default attribute
$ pantry show node vagrant# see the resulting JSON config file{ "helloworld" : { "perl_version" : "perl5.14.2" }, "run_list" : [ "recipe[helloworld]" ], "name" : "vagrant", "pantry_user" : "vagrant", "pantry_port" : "2222", "pantry_host" : "localhost"}
1. Set up a Vagrant virtual machine
2. Prepare Pantry to manage Chef Solo
3. Get Hello World cookbook and dependencies
4. Configure virtual machine for Hello World
5. Deploy
Steps for deployment of Hello World
$ pantry sync node vagrant# ... wait for everything to deploy ...# then load browser and test it!
It works
You can do this, too
Don't be afraid. Try it out. Get involved.
tutorial and screencast → http://perlchef.com
mailing list → [email protected]