Treating Your InfrastructureLike Garbage

Saturday, April 20, 13

Mike Fiedler, DatadogTwitter: @mikefiedler

Saturday, April 20, 13

What dowe care about?

• servers

• uptime

• load

• alerts

Saturday, April 20, 13

What shouldwe care about?

• services

• uptime

• performance

• alerts

Saturday, April 20, 13

What’s the difference?

• approach

• perspective

• business goals

Saturday, April 20, 13

Stop server hugging!

(C) GigaOM

Saturday, April 20, 13

Everything fails.Be ready for it.

http://youtu.be/drQlSptFXXIWatch this:

Saturday, April 20, 13

So, how?

Saturday, April 20, 13

Popular stack

LB

Data

User

Web

Saturday, April 20, 13

Choose elasticity• Use distributed, self-healing storage

Data

Saturday, April 20, 13

Web Choose elasticity

• Design web tier to tolerate failure

• session state?

• application behavior?

service = n + 1

Saturday, April 20, 13

Choose elasticity

• Monitor web tier health

• Not always as easy at it seems!

LB

Saturday, April 20, 13

Choose elasticityUser

Saturday, April 20, 13

Missing piece?

LB

Data

User

Web

Saturday, April 20, 13

Monitoring

LB

Data

Web

UserMonitor

Saturday, April 20, 13

[demo]https://github.com/miketheman/fullstack

Saturday, April 20, 13

Tools used• AWS EC2

• MongoDB

• Bottle.py

• Python

• Apache HTTP

• HAProxy

• Siege

• Chef

• Ruby

• Spiceweasel

• Datadog

• Money

Saturday, April 20, 13

Skills employed

• System architecture design

• Performance monitoring

• Capacity & disaster planning

Saturday, April 20, 13

What did you just see?

• A design pattern

• Experiencing failure and recovery

• Disposable servers, valuable service

Saturday, April 20, 13

#HugOpsSaturday, April 20, 13