behaviour driven infrastructure
DESCRIPTION
Does Behaviour Driven Development have a role in the infrastructure world? Enter Behaviour Driven Infrastructure where systems administrators can apply some simple open source tools and BDD principles to make infrastructure management more powerful, more insightful and deliver more value to their customers.The typical enterprise monitoring and configuration management set-up for a website is: - Is the Apache package installed and the appropriate version? - Is the Apache service running? - Can I connect to the HTTP port and is HTML returned? - Multiply this by a few hundred iterations of hosts and types of services and you’re probably looking at your typical Nagios, Puppet, Cfengine, Hyperic set-up.All this monitoring misses something critical though – we’re not actually monitoring that the service does what it should. Yes, it matters whether Apache is installed, the Apache service is running, and you can connect to HTTP but does this actually prove anything about the availability of the service we’re managing and providing for our customers? Nope…We need to demonstrate more than just a check that says the Apache server is up. We need to demonstrate that the service delivered by that infrastructure was available to our customers AND functioning as intended.Enter Behaviour Driven Infrastructure or BDI which applies the principles of Behavioural Driven Development to the management of infrastructure. In this presentation you’ll learn: - How Behaviour Driven Development works - What makes a behavioural test - How to install and use Cucumber to perform BDI - Practical examples of behavioural tests in Cucumber, and - How to integrate BDI into your work flow and your enterprise monitoring and configuration management frameworks.TRANSCRIPT
BehaviourDriven
InfrastructureLindsay Holmwood
DevOps
Behaviour Driven
Development
Behaviour Driven
Infrastructure?
What is a behavioural test?
origins
Test Driven Development
write test
write testrun test
write testrun test
test fails
write testrun test
test fails
make test pass
write testrun test
test fails
make test pass
refactor
unit tests
input/outputof
functions
result = bar(foo)assert(result, true)
xUnit
a reaction to test focus
Behavior Driven Development
testing the
flow of data in a system
testing the
flow of data in a system
test auser canperform atask
make a donationsearch for productcheckout cart
verifying a business’sfunctional
requirementsare met
underlying
implementation isirrelevant
to the business
underlying
implementation isirrelevant
to the business
Business: “assert(bar(foo),true)?”Tech: “Yes”Business: “Great”
Business: “assert(bar(foo),true)?”Tech: “Yes”Business: “Great”
Business: “Can I search for things?”Tech: “Yes”Business: “Great”
functionis importantfunctionis important
implementationis not
implementationis not
verifying a business’sfunctional
requirementsare met
integration testsacceptance testsintegration testsacceptance tests
“outside-in tests”“outside-in tests”
executable specificationexecutable specification
written in
spoken languagewritten in
spoken language
so the business understands what it is
paying for
so the business understands what it is
paying for
Enough business touchy-feely mumbo
jumbo!
I thought this was a
Behaviour Driven
Infrastructure talk?
I thought this was a
Behaviour Driven
Infrastructure talk?
Infrastructure as code
“The ‘Infrastructure’ is an application, a long running process with inputs, outputs and state.”
- Andrew Schafer, 2009http://bit.ly/he3HBt
an abstractionan abstraction
infrastructureis the
application
daemonsare the
libraries
configuration managementis the
programming language
infrastructureis
built with code
code without testsis
bad
code without testsis
evilevil
Why?
You can’t verify that your system works.
Behaviour Driven
Infrastructure
Taking BDD
principals & tools
Taking BDD
principals & tools
Adapting them to
infrastructure development
Show me the
tools!
Cucumber
Executable format for software specifications
Tool to execute that specification
Terminology
featureis a
module of common functionality
Feature: Google search To increase her knowledge A user Needs a search interface To discover information
featurehas many
scenarios
Feature: Google search To increase her knowledge A user Needs a search interface To discover information
Scenario: Home page When I visit "http://google.com.au/" And I fill in "q" with "great balls of fire" Then I should see "great balls of fire"
scenariohas many
steps
Given
When
Then
And*
stepsare like
unit tests
they mapto
blocks of code
Feature: Google search To increase her knowledge A user Needs a search interface To discover information
Scenario: Home page When I visit "http://google.com.au/" And I fill in "q" with "great balls of fire" Then I should see "great balls of fire"
When /^I visit "([^"]*)"$/ do |location| # codeend
When /^I fill in "([^"]*)" with "([^"]*)"$/ do |input, value| # codeend
Then /^I should see "([^"]*)"$/ do |string| # codeend
When /^I visit "([^"]*)"$/ do |location| visit(location)end
When /^I fill in "([^"]*)" with "([^"]*)"$/ do |input, value| fill_in(input, :with => value)end
Then /^I should see "([^"]*)"$/ do |string| response.body.should match(string)end
blocks of codeare like
unit tests
∴ scenariois a
serial execution of unit tests
Installing
$ apt-get install rubygems $ gem install cucumber
Using
$ apt-get install rubygems $ gem install cucumber
$ apt-get install rubygems $ gem install cucumber$ mkdir -p project/features/steps$ cd project
Testing Workflow
write test
write testrun test
write testrun test
test fails
write testrun test
test fails
make test pass
write testrun test
test fails
make test pass
refactor
$ apt-get install rubygems $ gem install cucumber$ mkdir -p project/features/steps$ cd project$ vim features/site.feature
Feature: Google search To increase her knowledge A user Needs a search interface To discover information
Scenario: Home page When I visit "http://google.com.au/" And I fill in "q" with "great balls of fire" Then I should see "great balls of fire"
$ apt-get install rubygems $ gem install cucumber$ mkdir -p project/features/steps$ cd project$ vim features/site.feature$ cucumber features/site.feature$ vim features/steps/site_steps.rb
$ apt-get install rubygems $ gem install cucumber$ mkdir -p project/features/steps$ cd project$ vim features/site.feature$ cucumber features/site.feature$ vim features/steps/site_steps.rb$ cucumber features/site.feature
Implications
Continuous Integration
Test server builds
Execute tests
on commit to config management
What environments are the tests run against?
UAT?
Staging?Production?
Destructive tests?
Fixtures
Setup / TeardownPattern
A/B testing
Migration to config management
Continuous Integration&
Monitoring Systems
build
testnotify
deploy
build
testnotify
deploy
testnotify
We’ve been asking the wrong questions
pingconnect
is the host up?is the service available?
is the host up?is the service available?
misconfiguration, bugs triggered by user data, hacked
network stack up
but otherwise broken
Why should I care?
“Nagios checks already do this!”
Cucumber
provides a framework
to phrase questions
Given
When
Then
Lowers the barrier of entry to
writing good checks
Caveat:
You need a firm grasp of language
Feature: Candy Store Wooooooooo!
Scenario: Lollipops When I fly to the moon Then I should see monkeys
Feature: Candy Store Wooooooooo!
Scenario: Lollipops When I fly to the moon Then I should see monkeys
completely valid,
completely useless
Cucumber
provides a common
specification format
dev & ops can share
Cucumber
provides a common
specification format
IT & business can share
Duplication of tests
Development
Production
Cucumber featuresare librarised tests
Cucumber featuresare librarised tests
Same specificationSame specification
Different implementationDifferent implementation
Where to from here?
Write more code.
Patterns.
Expand the library of tests.
Explain to your friends, colleagues, family, pets
these ideas.
Discuss!Thank you.
http://www.flickr.com/photos/51746218@N03/5077975818/http://www.flickr.com/photos/abrinsky/4981594203/http://www.flickr.com/photos/andresrueda/3020304543/http://www.flickr.com/photos/communityfriend/2342578485/http://www.flickr.com/photos/damienroue/2775342201/http://www.flickr.com/photos/elsie/3917813380/http://www.flickr.com/photos/hugo90/5343453489/http://www.flickr.com/photos/iamphejom/4930805430/http://www.flickr.com/photos/jiuck/5060618037/http://www.flickr.com/photos/kusamakura/2435343128/http://www.flickr.com/photos/laurelfan/100700370/http://www.flickr.com/photos/littledebbie11/3208631777/http://www.flickr.com/photos/master-blitzy/1192778834/http://www.flickr.com/photos/nasacommons/4858567480/http://www.flickr.com/photos/nhankamer/4703022414/http://www.flickr.com/photos/normanlowery/4674154988/http://www.flickr.com/photos/paul_lowry/2266388742/http://www.flickr.com/photos/pinprick/467320431/http://www.flickr.com/photos/superciliousness/91346618/http://www.flickr.com/photos/xaviergp/4003730145/
Credits