Zero to Test Driven Infrastructure

Sean Carolan – Solutions Architect, Chef

History of Software Testing

< 1956 Debugging: You found a bug in the code and fixed it.

1957–1978 Demonstration: Does the code run? Does it solve the problem?

1979–1982 Destruction: Kill all the bugs.

1983–1987 Evaluation: SDLC – Each stage has testing methods.

1988–2000 Prevention: Catch bugs and issues *before* they cost $$$(1)

2001-2009 Automation: Automate as much of your testing as possible.

2010–Today Velocity: Test fast and deploy often.

1) Gelperin, D. & Hetzel, B., (1988). The Growth of Software Testing

Why test?

• The closer a defect is discovered to its

introduction, the cheaper and easier it is to

fix.

• It is better to find a security violation before

the configuration code touches a production

system, instead of three months later during

a security audit.

• Rigorous testing allows you to deploy

infrastructure changes with confidence. The

more often you deploy code, the more

routine it becomes.

Automation is Essential

• Automate the setup, provisioning and deployment of an Apache webserver

• Infrastructure setup steps are defined as code, using the Chef DSL

• Tests are conducted during each stage of the continuous delivery pipeline

Three Types of Testing

Signal in: Did I send a properly

formed piece of code to the

interpreter?

Signal processing: Was my well-

formed signal properly interpreted?

Signal out: Did my well-formed

code, correctly interpreted, actually

achieve the thing I set out to do?

The Breakfast Taco API

What are the ingredients for success?

Chef workstation.

Can be your laptop

or a VM.

Somewhere to spin

up VMs/containers.

Source Code

Repository

CI system for

automated testing

Tools of the TradeWhat’s in the Kitchen?

RubocopBecause Ruby

• Identify potential Ruby errors

• Unclosed strings, etc.

• Identify style/convention that helps write better code

• Single quotes vs. double quotes

• This is useful for new Chefs, and helps make the code more readable

• Exclude rules that do not apply

• Not everyone is the same – some tests won’t work for your organization or for Chef code

• Run against static code, so tests are very fast (<1 second to run)

Food CriticTest Your “Chef Style”

• Flag problems that might cause your Chef cookbooks to fail

on converge

FC010: Invalid search syntax

• Identify style/convention that has been adopted by the Chef

community

FC004: Use a service resource to start and stop

services

• Create custom rules for your own organization’s

compliance/standards

COMP001: Do not allow recipes to mount disk volumes

• Run against static code, so tests are very fast (<5 seconds to

run)

ChefSpecSimulate Chef

• Did I send a properly formed piece of code to Chef?

• Especially important if there is mutability in your code

• This can cover things like feature flags, or behavior that changes on a

per-OS basis

• "If on Debian-based Linux I need to install package apache2, but on EL

variants I need to install package httpd.”

• Tests are very fast to run

• Even with simple code, it is useful for new Chefs to find simple errors

quickly

• Useful for regression testing – to make sure new changes

don’t break stuff that worked before.

Test KitchenLet’s do this (almost) for real

• “Signal Out” testing for Chef code

• Just because I said something and it was interpreted correctly, doesn't mean that

what I said is what I meant.

• Executes your Chef code on a real node. These nodes should be

disposable (local virtualization, cloud instances, etc.)

• Use any number of “signal out” testing products to ensure expected

results

• BATS

• ServerSpec

• Pester

• Can pull in other cookbook dependencies as well, and execute against

a machine that looks the same as your standard image

Six hours? Really?

• Hour 1: Visit www.learnchef.com and learn the Chef DSL basics

• Hour 2: Install ChefDK, Vagrant, Virtualbox and the kitchen-vagrant driver

• Hour 3: Write a few ChefSpec and ServerSpec tests for Apache

• Hour 4: Write Chef code so your VM passes the tests (Install, config apache)

• Hour 5: Debug, troubleshoot, fix any errors.

• Hour 6: Sit back, enjoy a Texian Martini and reflect on how awesome you are

Pre-built Apache cookbook

• Download a pre-built sample Apache

cookbook to use for testing.

• This is the same cookbook we write in

Chef Fundamentals training.

• Some sample unit and integration tests

are included to get you started.

http://bit.ly/1Iauoft

Questions?