CHEF VS. PUPPET IN THE CLOUD: HOW TELEPICTURES AND

MONEYSUPERMARKET DO IT

POLLING QUESTIONS

•  Doug Babst •  Systems Administrator •  Telepictures

•  Jim Davies •  Head of DevOps •  MoneySuperMarket

Panelists

DevOps Adoption Rises in 2015

2015 DevOps Tools – All Respondents

Telepictures Chef UsageDoug Babst

doug.babst@wbdevops.com

Who we are• Telepictures Productions is an industry-leading and Emmy

Award-winning producer of innovative, advertiser-friendly, multi-platform television and digital content for the syndication, cable and digital marketplaces !

• Our group manages the CMS for several popular shows

Current Setup• All servers / services are hosted in AWS • Manage ~150 servers

• We’ve been using RightScale for 3-4 years • Inherited a legacy setup built using RightScript

Design Considerations• Typically run ~40 web servers to handle normal load • Need to be able to handle spikes for popular stories • Boot times need to be kept to a minimum • Almost everything is baked into the base image • AMI (Amazon Machine Image) is entirely automated

Old Dev Process - RightScript

Chef Migration• Rewriting everything from the ground up in Chef • Code written and tested first on local system in vagrant • We do a lot of chef wrappers. No Forking!

Why Chef?• Experience with both Chef and Puppet • Last used Puppet ~5 years ago • Both are very similar

• Key differences for us: • Chef uses Ruby natively, Puppet uses a Ruby-like DSL • Chef has better control for order of operations • Chef has better industry support for our purposes

New Dev Process - Chef Based

Learning Curve• Challenge learning chef and ruby at the same time • Take the time to learn ruby fundamentals first

• Get Chef Training • Chef Fundamentals Webinar Series on YouTube

• RightLink implements some standard concepts differently • Start simple, make mistakes, use vagrant

Benefits• Forces admins to use version control • Allows tighter integration between system and app code • Consistency • Inputs now have descriptions and default values • Local vagrant image and AWS image now match

Challenges• RightScripts can be updated immediately • Chef recipes take at least 10 minutes

• Chef development inside RightScale alone would be difficult • Pairing with vagrant simplifies everything

• 95% of what works in vagrant works in RightScale • That last 5% leads to a loss of hair

The Future• Our next phase is moving to Chef 12 and RightLink v10 • RightLink v10 will manage vagrant in RightScale

www.moneysupermarket.com www.travelsupermarket.com

www.moneysavingexpert.com

Jim Davies Head Of DevOps Moneysupermarket Group (LSE:MONY)

Had Puppet skills and code already from previous project Based around a declarative, readable DSL Enforces configuration and raises confidence Cloud migration of business services demanded total infrastructure-as-code …but a traditional Puppetmaster setup just won’t work in a dynamic cloud

“Dynamic infrastructure” -ne “Static list of FQDNs”

Tell an instance what it is and let it PULL its own configuration

Instance only enters service when it is ready

For this, we need master-less Puppet…

We use Github – it’s there when you need it (well, usually) Everything in one place Self-documenting infrastructure Linting and CI/CD for deployment code Change management via pull requests and peer review Stores role configuration including Puppet code and parameters

Parameterized shell script v

Facter v

Puppet-librarian v

Puppet apply v

Hiera

#!/bin/bash ... TAG_LIST="msmid:account=$MSMID_ACCOUNT," TAG_LIST+="msmid:userbase=$MSMID_USERBASE," TAG_LIST+="msmid:env=$MSMID_ENV," TAG_LIST+="msmid:servicegroup=$MSMID_SERVICEGROUP," TAG_LIST+="msmid:role=$MSMID_ROLE," FQN="internal" ... # Write tags to Rightscale tags, populate Facter facts.d files and # concatenate into an 'FQN' # Using... # rs_tag --add "$TAGVAL" # echo $KEY=$VAL > /etc/facter/facts.d/$KEY.txt ... git clone https://github.com/$TRUSTRAP_REPOUSER/tru-strap.git $TRUSTRAP_BASEDIR cd $TRUSTRAP_BASEDIR ./init.sh --role $MSMID_ROLE --environment $MSMID_ENV

#!/bin/bash ... # Install Puppet, Hiera and Facter (note we use official gems for this) gem install puppet:3.7.4 hiera facter --no-ri --no-rdoc ... # The instance downloads the role configurations locally git clone -b $CONFIG_REPO $TARGET_DIR ... # The instance finds the list of Puppet modules for its role # and runs librarian-puppet librarian-puppet install --verbose ... # The Puppet run begins puppet apply /etc/puppet/manifests/site.pp

# Puppetfiles/Puppetfile.lb-agg1 # Role specific modules mod “puppetlabs/apache”, “1.1.1” mod “MSMFG/haproxy”, :git => “git@github.com:MSMFG/puppet-haproxy.git", :ref => “0.0.16” mod "MSMFG/skydns_client", :git => “git@github.com:MSMFG/puppet-skydns-client.git”, :ref => “0.0.5”

https://github.com/rodjek/librarian-puppet

:yaml: :datadir: /etc/puppet/hieradata :hierarchy: - "%{::init_env}/%{::init_role}" - "%{::init_role}" - "%{::init_env}" - common

node default { hiera_include('classes') }

--- classes: - role::lb_agg1 ...

http://docs.puppetlabs.com/hiera/1/

Code mixing

with

configuration

Over-reliance on

third-party services and repositories

Slow boot

times

Hundreds of instances, scores of roles, many environments

One ServerTemplate and one machine image

‘Phoenix’ servers with hot-swap deployments

Platform-agnostic principles (from development in Vagrant to production in Rightscale)

Infrastructure-as-code and ‘copy/paste datacenters’

•  E-book: Continuous Integration and Delivery in the Cloud: How We Do It at RightScale: How RightScale Does It

•  www.rightscale.com/ci-cd-ebook!

Q&A

6