puppet for everybody! - federated and hierarchical puppet enterprise - puppetconf 2014

Puppet for Everybody!Federated and Hierarchical

Puppet Enterprise 

Chris Bowles, Senior Systems Administrator

University of Texas at Austin

Puppet for Everybody? Absolutely!• Development• Operations• Management

source: http://goo.gl/Mjr0dy

Continuum of Expertise

Novice• Puppet Console• Variables

Medium• Hiera

Expert• Code• Custom

Facts• Custom

Functions

UT Puppet Canon• Inclusive• Secure by Default• Federation

UT Puppet Toolset

UT Puppet Community

Nested Configs

Puppet Enterprise

Code/DataFederation

UT Puppet Community

UT Puppet Community

Nested Configs

Puppet Enterprise

Code/DataFederation

UT Puppet Culture• Module Coding Standards• Module Documentation Standards• Power to the People

Puppet Console• Classes• Console Groups

(role/profile)• Console Variables

Configured Server!

Module CodeHieraExpert

UT Puppet Diagram

Novice

Nested Configs

UT Puppet (standards,

culture)

Nested Configs

Puppet Enterprise

Code/DataFederation

Nesting: Roles/Profiles

• Wiki server configurationsRoles

• Apache configurationsProfiles• Secure by default• standardized• configurableBASE

Minifigure Metaphor

• Default “torso” provided• Configurable: can change the

color of the cowl (black or very, very dark grey)

• Role/Profile: Can choose the head and arms, cape, etc…

From: https://www.flickr.com/photos/spielbrick/8201894577

Nest all the things!• Groups• Variables• hiera? (yup, more on that later)

Puppet Console• Nested groups• role/profile• assign classes &

variables to nodes

Configured Server!

Module CodeHieraExpert

Roadmap: Console Nesting

Novice

Nested Console Groups

source: http://goo.gl/tUdl5U

Nested Console Groups

BASE

profile_apache

role_wiki

wiki-01

secure defaults

Apache configs

Wiki configs

Node-specific configs

Nesting (from the node POV)

Contains Classes/Variables

from:

Node wiki-01

BASE profile_apache role_wiki

Don’t forget the Blog!

Node-level

Roles

Profiles

Secure Defaults BASE

profile_apache

role_blog

blog-01 blog-02

role_wiki

wiki-01 wiki-02

• Configurations come from nested groups

• No repetition!

What’s in a name (prefix)?

Role

Profile

Top BASE

profile_apache

role_blog role_wiki

Puppet Console will display:(alphabetical)• BASE• profile_apache• role_blog• role_wiki

Console Building Blocks!

source: http://goo.gl/CHwab0

BASE: BASE group

Profile: profile_apache group

Role: role_wiki group

Node: wiki-01.puppetconf.com

Puppet Console components• Classes

• Variables

• Group(s)

• Nodes

ssh

$::ssh_port

BASE, Profile_apache, role_wiki

wiki-01

Class Inheritance (immutable)BASE

assigns: ssh

profile_apacheinherits: ssh assigns: apache

role_wiki inherits: ssh, apache

nodeInherits: ssh, apache

Variable Inheritance (child wins)BASEN/A

profile_apachehttp_port = 80

role_blogN/A

blog-01http_port= 80

role_wikihttp_port = 8080

wiki-01http_port= 8080

All together now!

source: http://goo.gl/K91CJA

wiki-01 (annotated)Variable overrides from role_wiki group

Group membership and source(s)

Classes: combined from nested groups

Puppet Console• Console Groups

(role/profile)• Console Variables

Configured Server!

Module CodeHieraExpert

Roadmap: Hiera Nesting

Novice

Hiera: for complex variables

key: value

key2: value2

• Arrays • Hashes

source: http://goo.gl/ge45I1

Think backend data mapping

Nested Groups => Hiera pathsBASE

N/A

profile_apacheprofile = apache

role_wikirole = wiki

wiki-01Inherits:

profile,role

./

./apache/

./apache/wiki/

Nesting Hiera w/ group variables

Role(s)• ./$profile/$role

Profile(s)• ./$profile/

BASE• ./

(no variable)

$profile

$role

Broadto

Specific

Hiera.yaml – specific to broad

Specificto

Broad

---:backends: - yaml:hierarchy: - '%{profile}/%{role}/common' - '%{profile}/common' - 'common':logger: console:yaml: :datadir: /etc/puppetlabs/puppet/hieradata

Hiera.yaml – specific to broad

Specificto

Broad

---:backends: - yaml:hierarchy: - '%{profile}/%{role}/common' - '%{profile}/common' - 'common':logger: console:yaml: :datadir: /etc/puppetlabs/puppet/hieradata

Putting it together

"Denslow's Humpty Dumpty 1904" by William Wallace Denslow – Library of Congress [1]. Licensed under Public domain via Wikimedia Commons – http://commons.wikimedia.org/wiki/File:Denslow%27s_Humpty_Dumpty_1904.jpg

Console => HieraNested Console Groups Hiera

profile_apacheprofile = apache

Role_wikirole = wiki

./apache/

./apache/wiki/

wiki-01 nodeprofile= apache, role= wiki

BASEnone

./

Hiera search order1. ./apache/wiki/common.yaml2. ./apache/common.yaml3. ./common.yaml

Advanced Hiera Usage• “This data is exactly what I need… almost”• firewall, sudoers• +1

• Check out: – hiera_hash – hiera_array

Code/Data Separation

UT Puppet (standards,

culture)

Nested Configs

Puppet Enterprise

Code/DataFederation

Code/Data Federation

Wiki source

Apachesource

BASEsource

Puppet Server

Code Federation

puppet.conf

modulepath=

/opt/puppet/modules/base:

/opt/puppet/modules/apache:

/opt/puppet/modules/wiki:

BASE Repo

Apache Repo

WikiRepo

VCSREPO

Separate sources enable role separation via ACLs

Data Federation

• ./hieradata/ =

./common.yaml

./apache/common.yaml

./apache/wiki/common.yaml

BASERepo

ApacheRepo

WikiRepo

VCSREPO

Separate ACLs for Hiera data as well

A peek into the future…

source: http://goo.gl/9GwKyQ

Git Workflow• Instead of this… • 1 git repo / module

Core SVN repo(modules)

Apache SVN repo(modules)

Head (production) branch

Non-production branches (created as needed)

CI/CD• r10k push deployments (faster!)• Puppet Environments defined by code (Puppetfile)• Automated Testing/Deployment

Git repos

r10k

Puppet

Takeaways• Puppet Enterprise can be:

– Inclusive– Secure by Default– Highly Federated

• Nurture your Puppet community• Nest your configs!

Thanks! Any Questions?• Slide deck available from PuppetLabs• UT Puppet Architecturehttps://wikis.utexas.edu/x/

OreZAw• Contact information:

– Chris Bowles• Email: cbowles@austin.utexas.edu• Twitter: @cbowlesUT

Puppet Man, Sulayman Bowles 2014