devops for drupal: why we cook with chef

DevOps for Drupal: Why We Cook with Chef

Wednesday May 22nd DevOps Track

Beginner

Overview: Purpose

Why are we all here?

•  Interested in DevOps practices

•  Interested in automation tools like Chef

•  Interested in using them with Drupal

+

Overview: Sharing

How are we going to do this?

•  Team style presentation and discussion with views from both sides - Development and Operations

•  Share our experiences and examples... we will be keeping it real

•  We are NOT doing a tool comparison

+

Overview: Learn

What do we hope you will learn?

•  Why DevOps matters

•  Why automation is a must

•  Why you can use Chef to help yourselves, make life easier, and save time

+

For more info on Chef, check out fellow DevOps track session:

The Joy of Cooking - Whip Up a Drupal Environment with Chef

•  Thursday May 23rd at 2:15pm •  OR 203 •  Presented by Opscode's Nathen Harvey

Overview: More on Chef +

Who We Are

A software development shop... •  specializing in custom Drupal development,

systems integration, mobile, DevOps, and 24x7 support

•  based in Chicago with 30+ team members worldwide

...and yes we are hiring! Come talk to us!

+

Who We Are Playing the part of:

Moderator and Product Owner

@ Promet Source: Director of Products jay@promethost.com

+

Jay Uhlinger

Who We Are Playing the part of:

Development

@ Promet Source: Solutions Architect will@promethost.com

drupal.org: wamilton twitter/hub: @winmillwill

+

Will Milton

Who We Are Playing the part of:

Operations

@ Promet Source: Systems Administrator greg@promethost.com

+

Greg Palmier

Audience Quiz Time

How many are Developers? (a Will)

+

CC Image courtesy of Pedro Lozano on Flickr

Audience Quiz Time

How many are Sysadmins? (a Greg)

+

CC Image courtesy of Sharyn Morrow on Flickr

Audience Quiz Time

How many are both? (changing Prod servers)

+

CC Image courtesy of Arthur Caranta on Flickr

Audience Quiz Time

How many are other? (the Product Manager)

+

Audience Quiz Time

How many are using automated configuration management tools and processes now in your everyday work with Drupal?

How many are using Chef?

+

DevOps: What It Is

Key Culture •  Foster a collaborative working relationship between

Development and Operations

Key Concept •  Infrastructure as code

Key Results •  Higher deployment rates •  Better reliability, stability, and resilience of Production

+

DevOps: 3 Key Patterns

1. Make environments available early in the Development process

2. Shorten and amplify feedback loops 3. Create reusable deployment procedures

+

DevOps: What It Isn't

It is not a collection of bash, Drush, insert scripts here, etc.

+

DevOps is not about the tools... they don't fix everything magically But DevOps relies on some tools to implement its principles

DevOps: Why Is It Important

One-off environments and complicated manual builds and deployments = FAILURE

Must automate to meet business demands -- support agile, lean, faster, better...

Every environment is Production to someone

Drupal is not different -- all of these apply

+

A Classic Scenario

Jay: Will, you said this was working.

Will: It works in Dev.

Jay: Greg, can you look into why it isn't working in Prod?

Greg: Ok. (later on) @#$%! Dev and Prod are not the same!

+

Another Classic One

The Blog Post: Installing Drupal on AWS

1. Start wizard

2. 50+ steps later

3. A Drupal install

+

Why We Chose Chef

The orange color matches ours of course!

We already had experience (Marius) doing DevOps work with Chef in Bay Area with leading tech companies

We build Drupal applications -- we found Chef to be a good fit for application configuration management

+

The Operations part... ...out with the old and in with the new

+

Our Operations Story

How we started with Chef at Promet

Inconsistency was killing our time

How realistic was automation and configuration management of all of our stuff

+

Wasting Time

You are not in the sudoers file. This incident will be reported.

$ which git $

Fatal error: Class 'PDO' not found in blahblahblah

All of these messages diverted your work

+

Manual Config Run Amok

•  Dev environments != Prod environments •  Shared Dev/Prod/?? environments were

continually bloated •  Over time modifications amounted to more

inconsistency between environments •  Documentation was lacking

o  No records explaining changes o  Constant paper trail "paving" o  No great place for it anyways (what wiki?!?)

•  More customizations -> High Fragility

+

Typing Faster != Solution

•  Runbooks, Bash and SSH o  Process to stand up new infrastructure was done

manually via a "runbook”

•  Any configuration time was repeated extensively

•  Environment “discovery” bled time o  Snooping around to analyze how things were initially

stood up and potentially modified (caching, etc.)

•  Endless SSH-ing into servers

+

Chef Migration •  In with the New

o  2 dedicated servers §  VM hosts

o  Similar model, consolidated o  Entirely Chef managed o  Individual client Chef-spun Dev instances

•  Stand Up, QA, and Cutover o  Virtualize legacy o  Considerations mainly PHP

•  Monitoring customizations •  Special clients

o  Non-Drupal o  Contractually unique

+

Chef Migration

And migrating wasn't a complete nightmare....

+

Righting the Ship •  Server configuration in Git

o  Mitigated "paper trail" o  Team awareness of configs by notifications o  PHP, Apache, MySQL

•  System config changes pushed to Chef for Server propagation o  No more endless SSH-ing o  No more runbook config for every server o  Undoes "helpful" customizations by others

•  Direct between me and Config Mgmt o  Configuration entirely “local” with Vagrant o  Git driven accountability and awareness (--stat) o  `knife` is to Chef as drush is to Drupal

+

Client Ownership •  Clients maintain ownership of their assets

o  Any platform with a knife plugin o  Legacy apps migrated o  Non-Drupal or Promet "old" clients

•  Give us their Keys o  Let us be the Designated Driver o  Developers drop in, get pitted

+

Client Ownership

•  Excessive customization (not perfect) o  Non-Drupal clients or other work o  More so, use client's Hosted Chef

•  Contract flexibility o  No Dysfunctional Marriage

•  Honor prior client / hosting relationships o  Help clients where they are

+

Things That Are Solved •  User Configs

o  .gitconfig o  .tmux.conf o  .ssh/config o  shell customizations o  Not necessary but....doable

•  Server Configs o  logwatch o  mail o  mysql o  apache o  ssh o  automysqlbackup

+

Sysadmin Transformation

Have time to explore/make new tools

Do support in a consistent fashion

Automate all the Things!

"Grow" your team and infrastructure not with numbers but with talent

We do more work with less resources!

+

Sysadmin Team Evolution

We even spun-off a Startup company for infrastructure and applications automation and scaling using Chef in the Bay Area (non-Drupal)

I don't care what you automate with... pick one and do it!

+

The Dev part... ...it's not what you think

+

What I Thought I Would Say

•  What it was like... •  How it sucked for Devs •  How we changed it •  How it sucked in new ways •  How that lead to the new thing •  Repeat.... •  Where we are now •  What's really fun about this for Devs

+

The Problem

•  We went through too much shit •  It's not even fun to talk about •  It's probably boring •  Someone else already said what I was trying

to justify •  You're probably the choir if you're here

+

Drupal: OH @crell

"If you're not using Features, you are not doing Drupal professionally"

Also: •  If you don't have a one step build, you aren't

using Features correctly => Sucking at doing Drupal professionally

•  Configuration module is also probably fine •  It's going into core

+

So Now My Presentation Is

•  What we're doing now •  Why you shouldn't NOT do what we're doing •  Actual examples of how it has helped •  Why Chef is really awesome •  A surprise

+

What We're Doing Now

•  Everything you need in the repo o  Any scripts or scripting examples o  README files o  (small(-ish)) csv's/xml/etc o  ...pretty much any text file that is not too large and is

useful

•  ...Still need a way to automate sourcing archives of files and sql...

•  Vagrantfile + Berksfile

+

Berks What?

+

+

Kirkberks? +

What a Dev Has to Do on Our Team

•  Install git somehow •  Install the vagrant package •  Install the vagrant-berkshelf plugin •  git clone/pull && cd-to-the-root •  vagrant up (or reload) •  edit the hosts file or use dnsmasq or similar

+

Some Concrete Examples of This Helping Us Do Better

•  Audits o  http://reload.github.io/phing-drupal-template o  Not perfect... o  But now no one has to repeat what is automated

•  Every single deployment ever since o  Your Production push should not be the first time

you attempt sync'ing to a Production environment o  ...unless you crave excitement

•  Last minute accommodations o  Ever test RewriteBase?

+

Add the .htaccess directive... +

...change the docroot in code!

Potential Examples: You Can't Prove the Negative

•  PECL/Pear packages o  Shame on professionals who can't figure it out... o  ...but shame on me for betting the project

•  Anything to do with email ever since ever •  Anything to do with external services

o  ...memcached o  ...SOLR o  ...even MySQL

+

Why You Shouldn't NOT Do As We Do

Challenges you can deal with, excuses that are bullshit

On stupid things people say re: vagrant and chef

•  We make great docs! •  My native environment is fine •  Virtualization consumes resources •  I don't have enough time

+

+

My Native Environment is Fine

1. No, it isn't •  Your job is to ship to your downstream

o  ...not manicure and fetishize your tools

•  You are getting strictly less help than you could get o  ...even if you're getting a lot of help o  ...even if you're paying for help

•  You are keeping yourself from helping

+

My Native Environment is Fine (cont'd)

2. Your native environment is personal •  You may even obsess over it •  You may be completely wrong •  You will be happier without each other

+

Virtualization Consumes Resources +

I Don't Have Time...

...to do everything perfectly

•  We didn't either, and we're doing far better than we were

•  We do not have a massive continuous delivery, cloud orchestration and scaling infrastructure

•  ... But now what we have is in code

+

Chef is Really Awesome

...Puppet is probably cool too

•  Stewardship and leadership •  Constantly evolving ecosystem •  A community hell-bent on sharing •  Kicks ass even if you don't

+

So Easy I Can Share It with You!

Don't try it on this connection

bash <(curl -L -s http://bit.ly/10McZSZ)

github.com/promet/drupal_cookbook

Get to work already!

+

Why You Should Use this Project (or something like it)

We must advance the state of the art for Drupal deployment and application management

The CMS that is easiest to flexibly deploy will be the CMS of the future

All the features and D8 config exports in the world can't help you if you can't automate it

+

Other Projects/Strategies

•  If you have a dope Jenkins workflow... o  help me automate it with Chef so we can all use it!

•  Please use Puppet if it suits you o  and be vocal about hardcoded conventions that

make your work hard

•  Kraftwagen - kraftwagen.org •  Ariadne by patcon - github.com/

myplanetdigital/vagrant-ariadne

+

The Wrap-Up part... ...what should you remember

+

Key Summary Points

Process and team is the focus •  Get your Devs and Sysadmins in the same

room and start communicating

Tools are there to help •  Start automating now

Chef is an awesome tool •  It works and it helps

+

Q&A

Ask us questions now

Talk to us afterward

Stop by and see us at Booth 201

Thanks!

+

Feedback? Evaluate session at: portland2013.drupal.org/schedule