puppet fundamentals

PUPPET FUNDAMENTALS

About Puppet

• Puppet is an infrastructure automation and configuration management tool

• Created by Luke Kanies in 2003 using Ruby.

• First commercial product released by PuppetLabs in 2011

• 4000+ community members

• 50000+ nodes managed by largest deployments.

• Support for RedHat, ubuntu, Debian, Suse, Solaris 10, Windows, MacOSX

WHY PUPPET ?

Common issues in traditional IT Ops

• Manually configure new nodes (servers)

• Custom scripts are written for specific OS, environments

• Managing packages, patches across large infrastructure.

• Configuration consistency across all nodes.

• Managing large infrastructure becomes expensive.

Using puppet

• Enforces a defined state of the infrastructure

• Manages, automate tasks on 1000s of nodes

• Enable Infrastructure as a code

• Configuration consistency

• Increased productivity.

• Visibility of the infrastructure changes.

• Operational efficiency.

• Scalability

How Puppet works

Example: define

Pa kage { sshd : Ensure => installed.

}

file { /et /sshd/sshd_config : ensure => file,

owner => root,

group => root,

}

“e i e { sshd : ensure => running.

enable => true.

}

Use Case

• You need to manage an user max

> verify user exist ?

> what is the primary group

> what is the secondary group

> what is the home directory

Existing Tools in *nix

• useradd

• usermod

• groupadd

• groupmod

• mkdir

• chmod

• chgrp

• chown

Command line concerns

• Do I have to use useradd or adduser

• What are the options to use ( -l or –L )

• If I run the same command again, does it

work?

Use a script

#! /bin/sh

USER=$1; GROUP=$2; HOME=$3

If [ 0 –ne $(getent passwd $USER > /dev/null)$? ]

then useradd $USER –home $HOME –gid $GROUP –n; fi

OLDGID=`getent passwd $USER | awk –F: {p i t $ } ` OLDGROUP=`getent group $OLDGID | awk –F: {p i t $ } ` OLDHOME=`getent passwd $USER | awk –F: {p i t $ } ` If [ $G‘OUP != $OLDGID ] && [ $G‘OUP != $OLDG‘OUP ] then usermod –gid $GROUP $USER; fi

if [ $HOME != $OLDHOME ] then usermod –home $HOME $USER; fi

But What about

• Readability

• What if Windows & other OS support

• Robust error checking and logging

• How about other complex tasks

Puppet way

user { max :

ensure => present,

gid => admin ,

home => /data/max ,

managehome => true ,

}

Desired State

Desired State

Use { elmo : Ensure => present,

Gid => staff , Home => /mnt/home/elmo , ...

}

Convergence

gid => sysadmin , home => /mnt/home/elmo ,

Drift

Node State

Use { elmo : Ensure => present,

Gid => staff , Home => /ho e/elmo , }

Desired State

• Any convergence are reported back to the

server

• Provision a node, pupet configure it and

maintain the state.

• Puppet enforce the desired state in an

idempotent way.

Resource Abstraction Layer

File Package Service User

Ruby Apt

Yum

Gems

Dev

RPM

Redhat

Launched

SMF

Debian

Useradd

Ldap

Netinfo

Resource abstraction Layer

Facter

• Puppet uses facter to gather information

about node.

• Run $facter to know what are the facters on

your system

• The returned key-value pairs are called facts

• You can use these facts in your puppet

manifest.

Catalog compilation

Module structure

[root@master puppet]# tree modules/

modules/

`-- hello_world

|-- files

| `-- hello_world

|-- manifests

| `-- init.pp

|-- templates

`-- tests

`-- init.pp

Puppet Classes Classes define a collection of resources that are managed together as a single Unit.

# /etc/puppetlabs/puppet/modules/ssh/manifests/init.pp

Class ssh {

Pa kage { openssh- lie ts : Ensure => present,

}

File { /et /ssh/ssh_config : O e => oot , G oup => oot , Mode => , ‘e ui e => pa kage [ openssh- lie ts ], “ou e => puppet:/// odules/ssh/ssh_ o fig , }

“e i e { sshd : Ensure => stopped,

Enable => false,

}

}

Node Definition

Multiple classes are declared together to represent a Role

node agent.example.com {

include ssh

include mysql

include apache

}

Note: You can also use like below

class { ssh : }

include ssh

include ha_proxy_mbr

include apache

include web_app

include ssh

include ha_proxy

include ssh

include mysql

Node

web1.example.com

Node

lb.example.com

Node

db.example.com

Classes can be re-used

Class Naming Convention

• Class name must be unique

• Can be used only once on a given node

• Classes cannot be re-declared

• Ex – class hello_world

class apache::virtual_host

class myusers::group::finance

• Validate a class

puppet parser validate init.pp

Class Auto loading

• Classes load as per the modulepath

Read #man 5 puppet.conf for more details

Your First Module: Exercise

• Create a module for managing your user

account

• Steps:

> Create the module structure & support files

> Validate the syntax of your class

> Test the class on one of the agent node

Resources

• File

• Package

• User

• Service

#puppet resource file /etc/passwd

#puppet resource service sshd

#puppet describe user

Resource Abstraction Layer

File Package Service User

Ruby Apt

Yum

Gems

Dev

RPM

Redhat

Launched

SMF

Debian

Useradd

Ldap

Netinfo

Providers

•Providers are the interface between underlying OS and

resources.

Resource Relationship

• require

• subscribe

• notify

• before

(2) Containing Resource

(1) Reference

^

|

|

|

|

|

|

|

|

|

require

|

|

|

|

|

|

|

|

|

A containing resource can require a referenced resource to be applied first

require

Example

Pa kage { openssh : Ensure => present,

}

“e i e { sshs : Ensure => running,

Enable => true,

‘e ui e => Pa kage[ openssh ], }

(1) Containing Resource

(2) Reference

A containing resource can request to be applied before a referenced resource

|

|

|

|

|

|

|

|

|

|

before

^

|

|

|

|

|

|

|

|

before

Example

Pa kage { openssh : Ensure => present,

Befo e => se i e[ sshd ], }

“e i es { sshd : Ensure => running,

Enable => true,

}

Refresh Events

• Resource changes can refresh other events

• A change in configuration file must refresh

the service associated to it

• The subscribe and notify meta-parematers

can do it.

subscribe

(1) Reference

^

|

|

|

|

|

|

|

subscribe

|

|

|

|

|

|

|

|

|

(2) Containing Resource

refresh

Example: Pa kage { ntp : Ensure => present,

}

File { /et /ntp.conf : O e => oot , G oup => oot , Mode => , “ou e => puppet:/// odules/ tp/ tp. o f , ‘e ui e => Pa kage[ ntp ], }

“e i es { ntpd : Ensure => running,

Enable => true,

“u s i e => File[ /et /ntp.conf ], }

notify

(1) Reference

^

|

|

|

|

|

|

|

notify

|

|

|

|

|

|

|

|

|

(2) Containing Resource

refresh

Common Use Case

• Package | File | Service

• One of the common design pattern used in

production

• Reasonable workflow

> Install a package

> configure one or more config files

> Enable the service

Language Constructs (DSL)

• How variables are used in Puppet

• How to use Arrays

• Conditional logic options in DSL

• Create a puppet manifest that works in

multiple OS

Variables

• Variables are prefixed with $

Ex: $application = /var/tmp/prodevans.war

$confdir = /etc/httpd/conf.d

You can use double quote while using a string.

$string = My httpd config directory is $confdir

Variable Example

$httpd_dir = /et /httpd/conf.d

File {

O e => oot , G oup => oot , Mode => , }

Scope

• Scope is a specific area of the code which is

isolated from other area

Ex:

Class example

Class example::other

Local scope locally override the variables

defined in the parent.

Global variables

• All facts are global variables

• :: is the scope operator for top scope facts

Ex:

$string = This is the home page for ${::hostname}

• Variables are constants, cannot be

reassigned.

Namevar

• Each resource has a special attribute called namevar.

• It is the unique identifier for the resource

• When it is omitted, by default it is same as that of title.

Ex: user { Max Anderson :

ensure => present,

name => max ,

gid => wheel .

}

• For packages package name is the namevar

• For files the path is the namevar

• The title of the resource can be different from the namevar

Arrays

• Puppet support Simple array.

Ex: $sample_array = [ one , two , three ]

File { [ /tmp/o e , /tmp/o e/t o , /tmp/o e/t o/th ee ] : Ensure => directory

O e => oot , G oup => oot , Mode => , }

Conditional statements

• Puppet supports three conditional expression.

> The selector

> case statement

> if-else / elsif statements

• Selectors return a value

• If-else & case alter the logic flow of puppet code

Selector Values

• The value returned by selector can be used

Ex: package { ssh :

ensure => present ,

name => $::operatingsystem ? {

Ubuntu => ssh ,

default => openssh ,

},

}

• Selectors return a value, but do not evaluate a block of code

• Ideal for setting a variable or an attribute.

Case statement

• Case statements can be used around resources or a

collection of resources or other logical constructs

case $::operatingsystem {

debian : { $ssh_name = ssh }

RedHat : { $ssh_name = sshd }

default: { warning ( OS not supported ) }

}

• Always use the default match to avoid compilation issues.

Example: Case $::operatingsystem {

ubuntu : { $ssh_pkg = ssh

}

solaris : { $ssh_pkg = [ SUNWsshcu , SUNWsshdr , SSNWsshu ] }

# default assumes CentOs, RedHat

Default: {

$ssh_pkg = [ openssh , openssh- lie t , openssh-se e ] }

}

Package { $ssh_pkg:

Ensure => present,

}

If-else / elsif

• These conditions act on boolean expressions

• Following values return false

> Undefined or nil value

>

> false

• Ex: if $mailserver {

file { /etc/mail : ensure => present }

}

else {

file { /etc/mail : ensure => absent }

}

Conditional expressions

• Boolean expressions

and , or , not

• Comparison expressions

== , != , =~ , < , >, <= , >=

• Arithmetic expressions

+ , - , / , *, << , >>

Example:

$server = ProdDBlapp01

if $server =~ /ProdDB/ {

notify { This is a database instance : }

}

else {

notify { This is not a database instance : }

}

ERB Templates

• Manage configuration files with dynamic

contents

• Use this technique to manage configuration

file for apache or tomat or JBoss

Templates

• Templates are usually text files

• Inserting ERB tags allows you to display or act

on content of the variable

• Ex: The system IP address is <%= @ipaddress %>

• Can be assigned to an attribute

File { /et / a i g : ensure => present,

content =>

te plate apa he / a i g.e , }

Example:

• ssh_config template, enable X11 forwarding

only for CentOS hosts

#Puppet managed ssh_config file

Host *

GSSAPIAuthentication yes

<% if @operatingsystem == Ce tO“ then -%>

ForwardX11 yes

forwardX11Trusted yes

# virtually no client support untrusted mode

<% else -%>

Forward X11 no

<%end -%>

SendEnv LANG LC_*

Puppet Forge

• Puppet modules shared by community

• Search modules in the forge

• Share modules with others using the forge

• forge.puppetlabs.com

• From command line search

#puppet module search <modulename>

• Install a module

#puppet module install <modulename>

• Verify installed modules

#puppet module list --tree

Custom facts

• Facts written in ruby programming language

• Usually shell commands are issued as part of

the fact to return value

• Environment variable FACTERLIB

• use pluginsync = true in the main section of

/etc/puppet/puppet.conf

Hiera

• Installed by default after pupet 3.0 or later

• Hiera is a key-value lookup tool to provide node specific data

• Easy to configure data on per node basis

• Keep node configuration in one place and managing the node specific variables/data will be easy

• Hiera implies hierarchical data

Hiera configuration

• /etc/hiera.yaml

(Config file for puppet opensource )

• /etc/puppetlabs/puppet/hiera.yaml

( Config file for enterprise puppet )

• Use hiera command to find out specific data from hiera

• Referred inside puppet module as

$package_name = hiera( package_name )

Troubleshooting & Best practice

Managing puppet certificates

• puppet cert list

• puppet cert list –a

• puppet cert sign <hostname>

• puppet cert sign all

• puppet cert clean <hostname>

debug

• Agent run with –debug –verbose

• Verify the classname

• Verify if you have proper node classification

in site.pp or nodes.pp

• Verify agent configuration

Best practice

• Follow Package | config | service model

• Each small components must be created as

module and include them in other classes.

• Use editor to avoid syntax issue