puppet at github - puppetconf 2013

Puppet at GitHub

@wfa r rGitHub

Operations

Known Aliases:

King of Kebabs

The Chairman

Mr. Caremad !

The State of Puppet at GitHub

" The State of Puppet at GitHub

github/boxen

~1.5 years old

open-sourced ~7 months ago

~240 open-source puppet modules

puppet 3.latest

supports ruby 1.8.7, 1.9.3, 2.0.0

Linux support in-progress

#tomorrow @ 2:20pm

github/puppet

~5 years old

0.24.x — 2.7.x

121 modules

~280k lines of code

every single employee has commit access

15.5k commits to master past 12 months

by 86 contributors

% % % % % % % % % % % % % % % % % % % %% % % % % % % % % % % % % % % % % % % %% % % % % % % % % % % % % % % % % % % %% % % % % % % % % % % % % % % % % % % %% % % % % % % % % % % % % % % % % % % %% % % % % % % % % % % % % % % % % % % %% % % % % % % % % % % % % % % % % % % %% % % % % % % % % % % % % % % % % % % %% % % % % % % % % % % % % % % % % % % %% % % % % % % % % % % % % % % % % % % %

now with ops taken out

-12-15

commits on master per week, last 12 months

4000total commits by author past year

700total commits by author past year, except ops

700total commits by authors with >10 commits, past year, except ops

single puppetmaster

rubygems

ruby 1.8.7

unicorn

puppet 2.7.latest

~600 nodes

run hourly via crond

puppetdb

nagiosdb

"yo puppetdb, gimme all your nagios::object::* resources so I can

update this nagios config"

filtergendb

"yo puppetdb, gimme all the filtergen::rule resources I would realize

so I can update this filtergen config"

⚡ puppetdb ⚡

as it turns out, an api call is faster than running puppet on a host

gpanel

"Imagine Puppet Dashboard meets Razor and went on a weekend trip to the

beach with the Heroku API and drank epic amounts of blue drink"

aka we reinvented our own, smaller version of Foreman

inventory

app configuration

versioning of configuration values

Create nil => 1Update 1 => 2Delete 2 => nil

let's make credential rolling less awful

provisioning

provisioning is typically awful

we sprinkled in some ChatOps

a little bit later...

we have the dumbest ENC out there

# /usr/local/sbin/fetch_gpanel_enc \ fe1.rs.github.com

---parameters: gpanel_cabinet: D20-13 gpanel_enabled: true gpanel_monitored: false

we never specify classes via the ENC

any variables we pass through are prefixed with gpanel_

How GitHub writes Puppet

( How GitHub Writes Puppet

rodjek/puppet-lint

if you aren't using puppet-lint to audit your puppet codebase,

you are doing it wrong

puppet-lint enforces the Puppet Labs style guide

puppet-lint can even fix a ton of linter errors for you

put it in a pre-commit hook

$ git commit -am "can't lint this"

modules/github/manifests/role/redis.pp: syntax okmodules/github/manifests/role/redis.pp - WARNING: => is not properly aligned on line 118

1 errors found, aborting commit.

and then buy rodjek a beer

rodjek/rspec-puppet

if you aren't writing tests for your puppet code before

running it on a server,you are doing it wrong

use whatever framework/library

rspec-puppet just happens to be a pretty good one

put it in a pre-commit hook

$ git commit -am "tests dont pass but whatever lol"

1) Expected redis::server would include class "more_than_a_single_c_thread"

1 failures encountered, aborting commit.

and then buy rodjek another beer

node definitions

we don't use an ENC to describe node classes

node /^github-redis\d+/ { class { 'github::role::polling_redis': enabled => $::gpanel_enabled, environment => $::gpanel_environment, private_ipv4 => $::ipaddress, }}

abstractions all the way down

treat your site classes as cascades down to your dist classes

class redis::server( # params) {

class { 'redis::config': ... } -> class { 'redis::package': ... } ~> class { 'redis::service': ... }

class github::redis( # params) inherits github::defaults {

$memory = $environment ? { 'stg' => '2G', default => $half_memory_gb }

class { 'redis::server': ... }

class github::role::polling_redis( # params) {

class { 'github::redis': # overrides based on specific node }

augeas

you know what's not awesome?

an erb template that requires your class to take 52876423 parameters so you can configure every possible

value in my.cnf

augeas { 'my.cnf/performance': context => '/files/etc/mysql/my.cnf/mysqld', changes => [ # automatic dump/restore 'set innodb_auto_lru_dump 18000',

# innodb "set innodb_buffer_pool_size ${innodb_buffer_pool_size}", 'set innodb_log_file_size 256M', 'set innodb_log_buffer_size 8M', 'set innodb_lazy_drop_table 1', ],require => Percona::Server[$::fqdn]

it can seem complex and scary

it is an amazing tool

How GitHub ships Puppet

) How GitHub Ships Puppet

continuous integration

commit gets pushed

jenkins runs the test suite

status gets posted back to GitHub.com

tmm1/test-queue

remembers how to better parallelize tests

continuous deployment

tests passed for commit, Hubot auto-deploys

branch deploy everything

a lot of shops have a few environments

testing

staging

production

currently we have 181 environments

that's not the same as 181 nodesnot running production

we only have 5 of those

Hubot automatically merges the master branch before allowing any

branch to deploy

ChatOps

202 employees100% access to Puppet

100% trust

the list of roles you'd never think would touch production puppet

webkit developercore git developers

core ruby developerssvn developers

search developerfrontend designers

windows developersmac developers

supportenterprise sales

every puppet run happens in chat

all puppet output goes to chat

everyone can see everything

everyone can do anything

hands-on learning, by accident

eliminate disruptive questions

The Future of Puppet at GitHub

* The Future of Puppet at GitHub

puppet 3.x

upgrading a large, old codebase from 2.7 to 3 is really painful

we've been talking about it for a year

we're still trying to push forward, but...

helping maintain security fixes for 2.7

goal is to be on 3.2 by end of year

ruby 2.0

moar faster rubby

we get this for free when we move to 3

mcollective

aka the story of github/shell and the wonders of ssh in a for-loop

there are a lot of cool things about mcollective

I think it will be a part of our stack in the future

but you can still go pretty damn far with bash and ssh

even more puppetdb tooling

waiting for multiple runs to converge exported resources is painful

use the puppetdb API to skip all that

even more gpanel integration

"databags"

"node search"

steal good ideas from other tools

closing thoughts

"all software is terrible"

— anyone who's worked with software long enough

"no software is better than no software"— rtomayko

the plumbing doesn't matter when all you care about is the porcelain

write some damned good porcelain

THIS IS RODJEK

HE'S NORMALLY NOT THIS BLURRY

HE IS ALSO NOT MY BROTHER, BUT YOU REALLY SHOULD BUY

HIM A BEER

YOU SHOULD BUY THIS MAN A LOT OF BEER

THANKS

https://speakerdeck.com/wfarr/

puppet-at-github-puppetconf-2013

puppet at github - puppetconf 2013

state of puppet

puppet dashboard

github puppetdb

github gpanel

github provisioning

github enc

github githubboxen

github hiera

Technology

puppetconf 2016: puppet on windows – nicolas corrarello,...

getting started with puppet - puppetconf 2014

puppetconf 2016: testing and delivering puppet – michael...

puppet for windows users - puppetconf 2014

puppetconf 2016: running puppet software in docker...

puppetconf 2016: the future of testing puppet code –...

puppetconf 2016: successful puppet implementation in large...

puppetconf 2013 vcloud hybrid service and puppet

puppetconf 2016: puppet as security tooling – bill weiss,...

puppetconf. 2016: puppet best practices: roles & profiles...

writing and publishing puppet modules - puppetconf 2014

puppetconf track overview: puppet 4

getting started with puppet - puppetconf 2013

puppet for everybody! - federated and hierarchical puppet...

state of the puppet community - puppetconf 2012

puppet design patterns - puppetconf

using docker with puppet - puppetconf 2014

packaging software, puppet labs style - puppetconf 2014

test-driven puppet development - puppetconf 2014

puppetconf 2016: keynote - luke kanies, puppet founder