drupal integrated devops workflow

Drupal IntegratedDevOps Workflow

make yr shit right, son

Steve Rifkin@crifkincrifkin.com SandCAMP 2013 1/25/12

Craig McEldowney

About The PresentersSteve Rifkin

Site Architect, Consultant, leans toward front end UI / UXBut not afraid of the backend?...ahem...can whip up some mean php when called upon)7+ years building sites and data modeling (RoR)LA Drupal User Group OrganizerNicer than Craig

Steve Rifkin@crifkincrifkin.com SandCAMP 2013 1/25/12

Craig McEldowney

Craig McEldowneyBackend ninja, SOLR maven, Level 36 lawful evil Drupal mage14 years building content management systems, 7+ years DrupalBaby's got back-end development. not so keen on UI / UXCrankier than Steve

About CrifkinCraig McEldowney + Steve Rifkin = crifkin

Craig gets less namespace, but more clout

Steve Rifkin@crifkincrifkin.com SandCAMP 2013 1/25/12

Craig McEldowney

● Virtual CTO/CIO Services● Cloud architecture / turnkey HA/HP solutions● Developers looking for interesting problems to solve

Agenda● Framing the Problem● Playing the DevOps workflow game

○ Vagrant○ Chef Server

● "The Cloud Mentality" -- HA/HP from local dev to production

Steve Rifkin@crifkincrifkin.com SandCAMP 2013 1/25/12

Craig McEldowney

In the beginningYOU build sites.

YOU do quality work. More people hire YOU.

...YOU become the limiting factor in this equation.

YOU need to scale yourself, your process, your team, your clients.

Steve Rifkin@crifkincrifkin.com SandCAMP 2013 1/25/12

Craig McEldowney

Framing the ProblemDeveloper/Process side

How do I support Client K who is running in infrastructure Y while still supporting Client 4 who is

running in infrastructure 11.

How do I ingest new clients, yet still be able to support older ongoing clients without getting cranky?

Steve Rifkin@crifkincrifkin.com SandCAMP 2013 1/25/12

Craig McEldowney

Framing the ProblemTeam side

I need to hand off some coding/theming/support responsibilities to another resource...

What's the fastest/cheapest/most effective way to get them up and running so I can scale development

efforts?

Steve Rifkin@crifkincrifkin.com SandCAMP 2013 1/25/12

Craig McEldowney

Framing the ProblemClient side

I need to be able to handle configuration changes, scale for traffic, address and balance High

Performance/High Availability requirements, and provide a platform to grow with the client.

Steve Rifkin@crifkincrifkin.com SandCAMP 2013 1/25/12

Craig McEldowney

Playing the DevOps workflow game

Let's flip the problem on its head. If we know where we're headed, we can make some smart choices on the way to solving

them.

Steve Rifkin@crifkincrifkin.com SandCAMP 2013 1/25/12

Craig McEldowney

In a perfect world...

How do we meet the client's needs for HA/HP/scaling?

Configuration management and a supporting cloud platform.

Crifkin spells this Chef + AWS = ChAWS.

Steve Rifkin@crifkincrifkin.com SandCAMP 2013 1/25/12

Craig McEldowney

Huh?

Steve Rifkin@crifkincrifkin.com SandCAMP 2013 1/25/12

Craig McEldowney

Huh? (Chef Server)

● Chef = Configuration Management System● Written in Ruby● Idempotent execution -- e.g. running recipe(s)

multiple times will result in the same outcome● Track stack architecture and allow servers to

register with the stack and query for specific resources

● Other options: Puppet, Ansible, bash scripts and/or hating your life

Steve Rifkin@crifkincrifkin.com SandCAMP 2013 1/25/12

Craig McEldowney

How does this help me?I wrote all this cool chef stuff (recipes, roles). How

does this help me with my process/team and my own efforts working for multiple clients?

The answer rhymes with Ragrant

Steve Rifkin@crifkincrifkin.com SandCAMP 2013 1/25/12

Craig McEldowney

Huh? (Vagrant)

● Ruby command-line toolset for managing VirtualBox VMs

● Already built to pull configuration from Chef, Puppet, Bash, and other scripting methodologies

● Package Virtual Environments● Multi-VM Host Environments-- e.g. run a scaled out

production-like stack on your local environment

Steve Rifkin@crifkincrifkin.com SandCAMP 2013 1/25/12

Craig McEldowney

How does it fit together?Why spend a bunch of time putting together a stack and methodology for your production environment, without giving yourself options and tools to manage your local development efforts?

Let's talk about environments...

Steve Rifkin@crifkincrifkin.com SandCAMP 2013 1/25/12

Craig McEldowney

Chef: EnvironmentsChef allows you to create and assign nodes/servers to environments. Environments allow you to override server attributes/metadata, configuration parameters, recipe run lists, etc.

So what if...

We break up our deployment into environments, with resources to match each environment's needs.

Steve Rifkin@crifkincrifkin.com SandCAMP 2013 1/25/12

Craig McEldowney

Chef: EnvironmentsBut instead of doing it in a stupid and boring way

● Lovingly handcraft servers and back them up hourly, daily, weekly, never?

● Rats nest of client environment on local machine● Steve doesn't understand what I do. I don't

understand what he does...custom scripting.

Let's use the "Cloud Mentality"

Steve Rifkin@crifkincrifkin.com SandCAMP 2013 1/25/12

Craig McEldowney

The Cloud MentalitySome hard realities of cloud computing...

● Your servers are not beautiful and unique snowflakes

● Things will break and you will cry● Steve is nicer about these things than I am

Steve Rifkin@crifkincrifkin.com SandCAMP 2013 1/25/12

Craig McEldowney

The Cloud MentalityThe most important thing about your server is its configuration. Chef takes care of this.

Chef has recipes and roles to define what a Webhead is, what a Reverse Proxy is, what a BananaFarm is.

So crunch all you want, we'll make more.

Steve Rifkin@crifkincrifkin.com SandCAMP 2013 1/25/12

Craig McEldowney

The Cloud MentalityThe difference between production and staging is the number of servers. The difference between staging and dev is the number of servers. The difference between dev and your local machine is....

The number of Virtual Machines!

Steve Rifkin@crifkincrifkin.com SandCAMP 2013 1/25/12

Craig McEldowney

Prod/Live

Steve Rifkin@crifkincrifkin.com SandCAMP 2013 1/25/12

Craig McEldowney

Staging

Steve Rifkin@crifkincrifkin.com SandCAMP 2013 1/25/12

Craig McEldowney

Local Dev

Steve Rifkin@crifkincrifkin.com SandCAMP 2013 1/25/12

Craig McEldowney

Chef Server is the glue

Steve Rifkin@crifkincrifkin.com SandCAMP 2013 1/25/12

Craig McEldowney

Huh? (Howzit fit together?)Local Dev Setup:Install VirtualBoxInstall VagrantEdit Vagrantfile and point it at your chef serverSpin up virtual machine(s)Download DB/file assets from S3 or local-- use drush aliases to take DB dumps from DB slaves where possible

Edit code on local machine IDE by either mounting WebDAV drive, or setting up nfs share into Vagrant

Extra credit for running this all from a bash script and/or through a recipe in Chef!Double credit for writing a homebrew...you Mac nerds

Steve Rifkin@crifkincrifkin.com SandCAMP 2013 1/25/12

Craig McEldowney

Um...right, this is DrupalGotchas and considerations when using templated configuration for Drupal:1. DB credentials/discovery2. Other resources:

i. memcachedii. SOLRiii. Reverse Proxyiv. etc...

3. Providing options for environment specific overrides -- e.g. no preprocess css in Dev, etc.

Steve Rifkin@crifkincrifkin.com SandCAMP 2013 1/25/12

Craig McEldowney

More advanced use-casesSet up Continuous Integration

Use Jenkins or other post commit hooks to automagically spin up a stack in a dedicated testing environment, and run load, smoke, other tests

Dashboard controls for controlling environments-- a la Aegir, but with Chef Server controlling the system configuration, and manage scaling parameters (autoscale), scheduling scaling, etc.

Steve Rifkin@crifkincrifkin.com SandCAMP 2013 1/25/12

Craig McEldowney

In a less perfect world...Client is committed to existing

resources/deployment strategies?

● If this doesn't pose a business risk over an acceptable time frame, then save that fight for another day...

● Migrate some portion of their stack to new environment-- e.g. dev server/staging server?OR

● Use smart DevOps magic for you and your developer(s) to your side of this part of the process tight and efficient. Build up recipes/configurations so that they can be used in prod later on

Steve Rifkin@crifkincrifkin.com SandCAMP 2013 1/25/12

Craig McEldowney

Questions/Demo