Manageable Puppetinfrastructure
~April 2014 edition~
PuppetCamp Berlin
Ger Apeldoorn - http://puppetspecialist.nl
1 / 44
Freelance Puppet Consultant
Trainer for PuppetLabs Benelux
Who's this?
2 / 44
ScopeAlso... why this talk?
3 / 44
Commonpitfalls
4 / 44
Pitfalls
Cause & effectPitfalls
Lots of WorkaroundsUnmaintainable codebaseCollaboration difficulties
5 / 44
Pitfalls
Cause & effect
Quick Wins
Fix your codebase!Quick wins:
Move data to Hiera
Implement Code Review
Use Puppet-lint in a git-hook
REFACTOR CONSTANTLY
6 / 44
A Manageable DesignApril 2014 edition
7 / 44
RequirementsWhadda we need
8 / 44
Our environment should be:Easy to Use
Easy to Comprehend
Easy to Update
and... Safe
9 / 44
This stuffisn't exactly
easy
10 / 44
But we cán make it safe andmanageable
11 / 44
Requirements
Easy to:UseComprehendUpdate
Safe
SafeUse environments to test everything
Create a huge testing environment
Use Git to promote your code
12 / 44
Requirements
Easy to:UseComprehendUpdate
Safe
Manageable
ManageableKeep a consistent module structure
Using roles for abstraction
Facilitate collaboration
13 / 44
DomainsServer Roles
All things data
Deployment & Workflow
14 / 44
OverviewSoftware Components
15 / 44
Software ComponentsPuppet Enterprise or The Foreman
Hiera and hiera-eyaml (Hierarchical Data lookup)
Gerrit (Code review system)
Git (what else?)
Git Flow, adapted version for Gerrit
R10K (Environment deployment tool)16 / 44
Domain #1:
Server Roles
17 / 44
A layer of abstraction
18 / 44
How to do it?Create roles moduleroot@puppet# puppet module generate gerapeldoorn-role
Create a base-role to cover generic settings# modules/role/manifests/base.pp:class role::base { include users include ssh include motd ...
19 / 44
How to do it? -Cont'd-Put all required resources in the classes# modules/role/manifests/app.pp:class role::app { include apache include tomcat apache::virtualhost { 'default': ...
Include role in node definition# site.pp:node 'app01.autiplan.com' { include role::base include role::app}
20 / 44
Domain #2:
All things Data
21 / 44
HieraHierarchical data lookup tool
22 / 44
Configured Hierarchy:#/etc/puppet/hiera.yaml::hierarchy: - "%{::clientcert}" - "%{::environment}" - common
Node app01.autiplan.com:
environment: testing
Hieradata# hiera/app01.autiplan.com.yaml---examplekey: value for \ app01.autiplan.com
# hiera/testing.yaml---examplekey: value for nodes in \ testing environment
# hiera/common.yaml---examplekey: value for all nodes
It's all about Hierarchy
What will be in $test?$test = hiera('examplekey')
23 / 44
Types of HieradataRegular values# hiera/app01.autiplan.com.yaml---examplekey: value
24 / 44
Types of HieradataArrays# hiera/app01.autiplan.com.yaml---array: [ item1, item2, item3 ]
otherarray: - item1 - item2 - item3
Note: Never use tabs in Hiera files!
25 / 44
Types of HieradataHashes# hiera/app01.autiplan.com.yaml---hash: key1: value key2: value
26 / 44
Types of HieradataCombinations# hiera/app01.autiplan.com.yaml---hash: key1: value key2: value key3: - arrayvalue1 - arrayvalue2 key4: subhashkey1: value subhashkey2: value
27 / 44
Hiera-related functions...and what to use them for
28 / 44
What does it do?Retrieves the first-found value in thehierarchy. (top-down)
What to use it for?Basic variable-lookup.Very easy to create exceptions!
How to use it?
$smarthost = hiera('smarthost')
Example Hieradata# hiera/mail.autiplan.com.yaml---smarthost: smtp.myprovider.nl
# hiera/testing.yaml---smarthost: testsmtp.autiplan.com
# hiera/common.yaml---smarthost: mail.autiplan.com
hiera('key' [, default_value])
29 / 44
What does it do?Retrieves an array or hash valuein the hierarchy, concatinates allfound results
What to use it for?Combining data from allhierarchy levels.
How to use it?
$users = hiera_array('users')
Example Hieradata# hiera/app01.autiplan.com.yaml---users: [ 'user1', 'user2' ]
# hiera/testing.yaml---users: [ 'testuser' ]
# hiera/common.yaml---users: [ 'user3', 'user4' ]
hiera_array('key' [, default_value]) (and hiera_hash)
30 / 44
What does it do?Includes all classes listed in thearray that is loaded from Hiera.Takes elements from ALLhierarchy levels.
What to use it for?Lightweight ENC.Put all classes / roles in Hiera.
How to use it?
node default { hiera_include('roles')}
Example Hieradata# hiera/web01.autiplan.com.yaml---roles: - role::web
# hiera/common.yaml---roles: - role::base
hiera_include('classes')
31 / 44
What does it do?Generates resources from aHASH.
What to use it for?Generate any resource based ondata from Hiera.Can also be used withhiera_hash to create resourcesfrom all levels!
How to use it?
create_resources ('apache::vhost', hiera('vhosts', {}))
Example Hieradata# hiera/web01.autiplan.com.yaml---vhosts: autiplan.com: alias: www.autiplan.com autiplan.dk: alias: www.autiplan.dk docroot: /var/www/html/autiplan.dk autiplan.nl: alias: www.autiplan.nl cdn.autiplan.com: port: 81 docroot: /var/www/html/cdn
create_resources('type', HASH [, default_values])
32 / 44
Data bindingsAuto-loading of Hiera data for parameterized classes.
33 / 44
What does it do?Automatically loads classparameters from Hiera.
What to use it for?Specify all class parameters inHiera.Use all hierarchical benefits forclass parameters.Simplify the use ofparameterized classes.
How to use it?
include mysql::server
Example Hieradata# hiera/web01.autiplan.com.yaml---mysql::server::root_password: m0ars3cr3t
# hiera/common.yaml---mysql::server::root_password: t0ps3cr3tmysql::server::package_name: mysql-servermysql::server::restart: true
Data bindings
34 / 44
Putting it all togetherAnything node-specific should be in Hiera!
35 / 44
A Puppet Run: What calls what?
36 / 44
Domain #3:
Deployment & Workflow
37 / 44
EnvironmentsKeeping the environmentalists happy
38 / 44
EnvironmentsWhat is an environment?
Seperate modulepaths/site.pp.Common environments: development, testing, production.Nodes request a specific environment.
Why?Essential to prevent mistakes.NEVER edit code in production!The workflow helps us to 'promote' our code to production.
39 / 44
Demo!
40 / 44
R10k overview
41 / 44
Final remarksKeep public modules as-is, wherever possible
Create wrapper classes in company-module.Create fork if needed, submit pull request for fixes.
Add forked module (gitrepo) to Puppetfile.
Think aheadAlways try to anticipate future applications.If it feels overly complicated, yer doin it wrong.Refactor!
42 / 44
Questions?
43 / 44
Freelance Puppet Consultant
Trainer for PuppetLabs Benelux
Thank you!A howto of setting up this environment (and the workflow!) is available on my
blog: http://puppetspecialist.nl/mpi
44 / 44
