learning puppet chapter 1

LEARNINGPUPPET - 01

SlideBook (Book on slides)

Inspired by Slidedoc -

http://www.duarte.com

- Vishal Biyani

www.vishalbiyani.com

Puppet Puppet –spreading wings

www.vishalbiyani.com Learning Puppet

Why I wrote Puppet SlideBook?

Going through tons of documentation and then doing some

hands on just seemed counter intuitive

Puppet documentation is very good – just that I wanted to

learn it gradually and relate to it while building stuff

3|•You should definitely give a shot to Puppet learning VM: https://puppetlabs.com/download-learning-vm

•Some basic awareness of “what is Puppet used for” is assumed in tutorial. Even if you don’t have hang on!

Puppet learning VM* sounded great for this, but I wanted to

build it from scratch and then learn components one by one.

I wanted it to be fun & concise – maximum returns on easy to

digest format with minimum text!

Lastly since I thought of sharing my Puppet learning

experience from which others might benefit.


Get Set Get source code at https://github.com/vishal-biyani/puppet-lab

clone on your machine in a convenient directory.

Configure the number of agents you want to spin up and RAM you

want to allocate to master & agents in Vagrantfile with parameters

MASTER_MEMORY & AGENT_MEMORY. Ideally keep at least 1GB

RAM for server, although in first few chapters 512M is fine too.

Now all you need to bring up the whole setup is fire a simple

command (Provided you have done the installation suggested in box

on left side)

Install VirtualBox and Vagrant

4|

vagrant up

The setup will take some time and will do following:

� Download a lightweight Linux machine image and create required

number of master & agent instances. (~200MB download)

�Master instance will be installed and configured with Puppet

Master and agent instances with Puppet Agent – they will also be

connected to each other. (~ 100 MB download)

Install VirtualBox and Vagrant

on your machine before you

start.

You will need to know very

basics of Git – and I will

introduce Vagrant, but

otherwise much of tutorial is

self contained and Puppet

oriented.

A very basic and sufficient introduction of Vagrant can be found at https://docs.vagrantup.com/v2/getting-started/index.html


MASTER_MEMORY=2048

AGENT_MEMORY=256

We start by setting some parameters in

beginning of script

Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|

config.vm.define "puppet_master" do |pmaster|

Then we start configuration for the

master box. We name it pmaster and rest

of configs will be pmaster.something

The Vagrant (Black) magic!

What’s going on in Vagrant?

5|

Do not worry about learning Vagrant much – our aim is to focus on Puppet. This is only for information

pmaster.vm.box = "centos_6_3_x86_64"

pmaster.vm.network "private_network", ip: "#{PUPPET_MASTER_ADDRESS}"

pmaster.vm.hostname = "puppet.learn.com“

Then we define a CentOS box and we

provide an IP within a private network

along with a domain name

pmaster.vm.provider :virtualbox do |vb|

vb.customize ["modifyvm", :id, "--memory", MASTER_MEMORY]

end

pmaster.vm.provision "shell", path: "scripts/installPuppetMaster.sh"

end

We modify the RAM as per our need and

finally we call an script on newly created

box. We will look at this script shortly but

it basically setups the whole box for us. A

simple shell script –

installPuppetMaster.sh

Puppet setup+Basic puppet configurationconfiguration+Playing with Puppet


Puppet Master Installation in 11 lines!1 sudo rpm -ivh http://yum.puppetlabs.com/puppetlabs-release-el-6.noarch.rpm

2 sudo yum -y install puppetserver

3 # We are adding Puppet labs repo to RPM and then installing it.

4

5 sudo cp /vagrant/conf/puppet.conf /etc/puppet/puppet.conf

6 # Copying a config file, which we will look in details later

7

8 sudo echo "192.168.17.99 puppet.learn.com puppet puppetmaster" >> /etc/hosts

9 # Add IP of server with domain name across all machines

10

7|

10

11 sudo iptables -A INPUT -p tcp --dport 8140 -m state --state NEW -j ACCEPT

12 sudo service iptables save

13 sudo iptables -F


15 # We are opening server's port 8140 to world & flushing iptables so that they

behave well!

16

17 sudo puppet master start

18 # Started Puppet master

19

20 sudo cp /vagrant/puppet_data/site.pp /etc/puppet/manifests

21 sudo echo "*" > /etc/puppet/autosign.conf

22 # Copying some more conf file - more on it later


Installing & Connecting Puppet Agent

1 sudo rpm -ivh http://yum.puppetlabs.com/puppetlabs-release-el-6.noarch.rpm

2 sudo yum -y install puppet

3 # Add puppet repo to list & install Puppet (Client)

4

5 sudo cp /vagrant/conf/puppet.conf /etc/puppet/puppet.conf

6 sudo echo "192.168.17.99 puppet.learn.com puppet puppetmaster" >> /etc/hosts

7 # Copying some configuration files which we will see shortly.

8

9 sudo iptables -F

8|

-: A NOTE OF CAUTION :-If you are going to use sudo before every puppet command – then use it uniformly for all

commands. If you are not going to – then don’t do it for any command.Due to access permissions if you don’t use sudo then all directories will be created under

$HOME/.puppet.So use one and leave other to avoid confusion! But decide right now.

I am going to use sudo everywhere to avoid any issues at all

9 sudo iptables -F


11 # Some iptables magic - nothing to worry here

12

13 sudo puppet agent -t

14 # Test run the puppet agent


1 Puppet.conf is a configuration file which exists on every

node – be it master or agent. Typical location is

/etc/puppet/puppet.conf (Or /etc/puppetlabs/puppet/puppet.conf)

Puppet.conf

2 There are three sections – [main] settings applicable to

all nodes, [master] has settings only for master nodes

and [agent] has settings meant for agent nodes

3 The only setting we are adding to default puppet.conf

right now is “server = puppet.learn.com” – so that all nodes

point to the server.

9|

* - the other configuration of puppet called serverless Puppet – is in which you run puppet stand alone without need for a master. We will get a basic introduction of serverless puppet towards end of this chapter

4 To ensure that “puppet.learn.com” is resolved to a valid IP

– we made an entry in /etc/hosts (Recall from previous

page?)

5In a typical server-agent setup* of Puppet there will be

one or more master and n number of nodes. The node

has to connect to master – and authenticate itself.

6 The autosign.conf that we configured in master script has

“*” which means all nodes will be auto approved as

soon as they connect to master – removing need for a

manual approval. Just a convenience for our test.


Let’s play with what we setup!

1 Assuming you are in same directory where

Vagrantfile is present fire command:

vagrant status

In my case I have one master and two agents

configured so I get result like following, you will get

a similar result:

“puppet” is the command you will use irrespective of you are running on

“master” or “agent” with those names as argument for example. We

used following command to start puppet server:

sudo puppet master start

And to test agent (In shell scripts after boxes were provisioned):

sudo puppet agent –t

For any help simply type command “puppet help”, for a specific

command help type “puppet help command_name”

3

To know various configurations of puppet there is a handy command:

sudo puppet config print

But that is going to print a whole lot of configurations, so we can choose

to see only specific configurations:

4

10|

#: To get “vagrant ssh” working on windows seamlessly there are some hacks listed at http://stackoverflow.com/questions/9885108/ssh-to-vagrant-box-in-windows

To get into any of boxes fire a ssh command with

box name. If you are on windows OS check the

bottom note #. I fired following command to get

into master for example:

vagrant ssh puppet_master

Now you are in the Linux box that we just created

and can fire any command.

2 So what exactly happens when we run “puppet agent -t”

Puppet is getting info from server and applying configurations to node.

5

Remember the “autosign.conf” in which we added “*” – that ensured that all agents are automatically authenticated? Want to see

them? Then fire the command: sudo puppet cert list –all

If we had not configured that file then you would have to manually approve the certificate request by firing command like “sudo

puppet cert sign node_name”. Get more familiarity with command by firing “sudo puppet help cert”

6


Puppet Terminology in short Everything in puppet is a resource – a file, a service, a package to be installed

etc. Each resource has a “type” and other attributes. For example file is a type

of resource or exec is a type which can execute external commands. Puppet

provides lot of types in built plus we can write our own.*

Manifest is where we write our Puppet code, typically extension is “.pp”. There

might be classes etc. to provide structure to our code within manifest files.

Manifests are compiled to catalog and then sent to nodes for actual execution.

ERB – stands for Embedded RuBy. Used in templates with embedded code.

Templates can be for a configuration file and code is resolved at runtime to

resource

manifest

ERB template

11|* - Check all types that Puppet has built in at:http://docs.puppetlabs.com/references/latest/type.html

Templates can be for a configuration file and code is resolved at runtime to

populate appropriate values.

Every system has certain facts – like IP address, OS type etc. which are reported

back to server and can be used in code to reduce hard coding. We can also build

custom facts of our own.

Module is a logical unit of puppet code & configuration – which is self contained.

Typically contains classes/manifests, configuration files and templates, files

needed & any other libraries/plugins etc. Think of module as a logical

packaging in other languages like JAR in Java or gem in Ruby (That is

oversimplified but to get the point). For example you might write a module

which can install and configure Tomcat – so the module will have configuration

files & manifests for doing that. You can also find modules built by community

on Puppet Forge.

ERB template

facts

Much more to come!!

module


Getting hands dirty with Puppet

1 We talked about resources very briefly in previous

slides. Puppet has certain in built resource types

which it can manage, for example a file, a service or

a group and so on. To know which all types puppet

has in built fire following command:

sudo puppet describe –list

The manifest config gives you location of site.pp – think of this as “the

king manifest” - a manifest which eventually encompasses all other

manifests. We will play with site.pp in coming chapters.

2

Puppet module is another useful command and for now we will look at

four usages of it which are helpful to us.

sudo puppet module list

Will list modules already installed on your server. Fire this command to

see which modules are present on your setup

sudo puppet module search puppetlabs

Will search for modules on puppet Forge whose name contains

3

12|

sudo puppet describe –list

You will get a big list of things which puppet

supports natively. If you want to know more about

specific type for example to know more about

“host”:

sudo puppet describe host

Each type has three main sections – description,

parameters that it can take and provider (We will

see provider a bit later)

At this point in time don’t worry much about

knowing everything about types – but this is a good

command to recall when you want to inspect a

type.

Will search for modules on puppet Forge whose name contains

“puppelabs”

sudo puppet module generate

Will generate a directory structure & bare bones files so you can wrote

your own module. We will do this in coming chapters

sudo puppet module install <ARGUMENTS>

Install a module from Puppet forge or from a archive file.

If you don’t want to execute/apply any code and just want to test your

code you can pass the flag “--noop” which is dry run mode. The flag goes

with almost all puppet commands and gives you a kind of simulation of

what is going to happen without actually changing anything on system!

4


Curious case of Puppet Apply

Most of what we have seen and will see through the book is puppet

master-agent way of working but puppet can work on a standalone

machine without needing a puppet master – called serverless puppet and

this is achieved with puppet apply command. So how does it differ?

�The manifests/code is typically downloaded directly from a source code

repository based on role etc. of node

�The catalog can be applied periodically, often through a cron job.

�You can pass a single manifest, include modules or pass a JSON catalog

generated by compiling catalogs. (Catalog can be generated on puppet

master by firing command “puppet master –compile” Why would someone use serverless

13|

master by firing command “puppet master –compile”

To apply from a manifest and to apply from a module by including a class

respectively, code would look like below:

There are lot more options and I suggest you take a quick look at

documentation of puppet apply. (Of course by firing command “sudo

puppet help apply” ☺)

3 $ puppet apply -l /tmp/action_log.log tomcat_manifest.pp

4 $ puppet apply --modulepath=/home/dev/modules -e "include tomcat"

There is a good case study on using Puppet apply or masterless puppet at https://puppetlabs.com/presentations/de-centralise-and-conquer-masterless-puppet-dynamic-environment

Why would someone use serverless

puppet instead of a master-agent

puppet? The reasons can be many and

some of them may not be relevant as

puppet evolves more. Some of points

mentioned in presentation in the

footnote for example are scalability to

single point of failure if Master fails

etc. As always there are multiple

solutions to any problem and

serverless puppet can be sometimes

an easy and simple solution


But this is not only it..

Facter is a system profiling

library which provides facts

about the node. Imagine

There is much more to Puppet than meets the eye

Hiera is a key/value

storage tool so you can

store configurable data

Mcollective is a

orchestration framework

which allows you to run

PupetDB is the storage

engine used by Puppet

which also provides an

What we have setup so far is bare minimum basic Puppet server and agent(s) – which is great for learning

Puppet as beginner. But to harness the real power there are lot more things we will learn by end of this book.

14|

about the node. Imagine

having to hard code IP

address of system? With

facter you won’t ever need

to do that

store configurable data

and retrieve when needed

so that you can avoid hard

coding and make code

more configurable

which allows you to run

commands on set of

servers in real time

which also provides an

API.

Puppet – of course is the

core declarative language

framework which allows

you to write code for

controlling platform

components

Puppet Enterprise

combines all previous

components with a

powerful UI – the Puppet

Console, is free for upto

10 nodes

Geppetto is a IDE for

puppet so that you can

write Puppet code with

ease

Puppet Forge is

repository of modules

(Reusable components)

written by Puppetlabs

team and community at

large


What did we learn?

Apart from core puppet there is

an ecosystem of libraries and

frameworks which allow you to

do a vast number of things

around infrastructure

provisioning, handling and

maintaining. We will learn most

of these one at a time in coming

chapters.

Typically puppet runs on master-

agent model. The agent

connects to master using

“server” setting in puppet.conf.

Puppet can also be run in

serverless mode – without a

server. Which means puppet

library for agent/server is same.

Puppet.conf is the key

15|

chapters.Puppet.conf is the key

configuration file for controlling

various parameters.

-: SOMETIMES YOU WILL HALT THOSE VAGRANT BOXES:-When you halt the vagrant boxes – and then bring back up and it might seem like nothing is working. Don’t worry follow following steps:1) Ensure puppet master is alive, else fire command “sudo puppet master start” on master box2) For a given agent the certificates need to be generated fresh & needs cleaning up on master. So first on master

machine “sudo puppet cert clean _AGENT_NAME_”3) Then on agent “find /home/vagrant/.puppet/ssl -name _AGENT_NAME_.pem -delete”4) And then fire on agent “sudo puppet agent -t” – and this should fix it.Option 21) If above steps don’t work for a given agent then destroy only that agent with “vagrant destroy _AGENT_NAME_”2) And then bring up the agent with command “vagrant up _AGENT_NAME_”

learning puppet chapter 1

Software