Mark R. Hinkle VP of Community Cloud.com Twitter: @clouddotcom @mrhinkle Email: mrhinkle@cloud.com mrhinkle@gmail.com

Open Source Toolchains to Manage Cloud Infrastructure

2

Introduction •  Responsible for Driving Adoption of

CloudStack Open Source Cloud Computing Software

•  Former manager of Zenoss Open Source project 100,000 users, 1.5 million downloads

•  Former Linux Desktop Advocate (Zealot?) •  Former LinuxWorld Magazine Editor-in-Chief •  Open Management Consortium Conspirator •  Open Desktop Consortium Instigator •  Author - “Windows to Linux Business Desktop

Migration” - Thomson •  NetDirector Project - Open Source

Con!guration Management Project •  Sometimes Author and Blogger at

SocializedSoftware.com/NetworkWorld •  Start-up junkie, Glutton for punishment

3

Agenda

•  Introductions •  Cloud & Systems Management Overview •  Open Source Tools for Cloud

Management •  Creating Open Source Toolchains •  Questions (and Hopefully Answers)

Cloud Computing & Systems Management Overview

5

Jevon’s Paradox and the Cloud

1865, the English economist William Stanley Jevons

…the efficiency with which a resource is used tends to increase (rather than decrease) the rate of consumption of that resource.

…in cloud computing increased consumption can introduces signi!cant management overhead despite higher hardware utilization and other bene!ts

6

Balancing the Equation: Creating the Cloud

+ =

7

Balancing the Equation: Managing the Cloud

+ = ?

8 Automation Balances the Equation •  MeatCloud, Can’t Keep up with

Cloud Computing •  Devops & Agile IT Philosophy •  Script Repetitive Tasks •  Integrate Tools •  Automate, Automate, Automate

9 DevOps – Agile Management for the Cloud

•  Agile, fast moving •  Smaller incremental change •  Increased coordination across

groups (development/operations) •  Eye towards Automation

10

Management Toolchains

Con!guration

Patching and Provisioning

Monitoring Toolchain (n): A set of tools where the output of one tool becomes the input of another tool

11

The Myth of the Nines Availability % Downtime per

Year Downtime per Month

Downtime per Week

99.9% (three nines) 8.76 hours 43.2 minutes 10.1 minutes

99.95% 4.38 hours 21.56 minutes 5.04 minutes

99.99% (four nines) 52.6 minutes 4.32 minutes 1.01 minutes

99.999% (!ve nines) 5.26 minutes 25.9 seconds 6.05 seconds

99.9999% (six nines) 31.5 seconds 2.59 seconds .0605 seconds

Average polling interval for monitoring? 5 minutes? Even superhuman operations people can’t be alerted and take action in under 5 minutes.

One outage per year could drop service level to three nines or worse.

Open Source Tools to Manage the Cloud

13

2011 CLOUD COMPUTING OUTLOOK SURVEY http://www.cloud.com/cloud-computing-outlook

14

4 Types of Management Tools

Provisioning Installation of operating systems and other software

Configuration Management Sets the parameters for servers, can specify installation parameters

Orchestration/Automation Automate tasks across systems

Monitoring Records errors and health of IT infrastructure

15

Cloudy Tools

•  Network Capable •  Cloud “Aware” •  Easy-to-Integrate •  Adhere to Open Standards •  Lend Themselves to Automation

16 Open Source Provisioning Tools

Year Started Language License Installation Targets

Cobbler 2007 Python GPL Red Hat, OpenSUSE Fedora, Debian, Ubuntu

Fully Automatic Installation (FAI)

2000 Perl GPL Debian

Kickstart ? Python GPL Most .dep and RPM based Linux distros

Spacewalk 2008 Perl, Python, Java GPL Fedora, Centos

Crowbar/Chef 2011 Ruby Apache ?

17 Open Source Configuration Management Tools

Year Started Language License Client/Server

Bcfg2 2003 Python BSD Yes

Cfengine 1993 C Apache Yes

Chef 2009 Ruby Apache Chef Solo – No Chef Server - Yes

Puppet 2004 Ruby GPL yes

18 Open Source Monitoring Tools

Year Started License Language Type of Monitoring

Collection Methods

Cacti (RRDTool) 2001 GPL PHP Performance SNMP, syslog

Nagios 1999 GPL C/PHP Availability SNMP,TCP, ICMP, IPMI, syslog

OpenNMS 2000 GPL Java Availability/Performance

SNMP,

Zabbix 2001 GPL C/PHP Availability/ Performance and more

SNMP, TCP/ICMP, IPMI, Synthetic Transactions

Zenoss 2005 GPL Python Availability, Performance, Event Management

SNMP, ICMP, SSH, syslog, WMI

19 Open Source Automation/Orchestration Tools

Year Started

Language License Client/Server

Support Organization

Capistrano 2006 Ruby MIT Yes None

RunDeck 2010 Java Apache Yes DTO Solutions

Func 2007 Python GPL Yes Fedora Project

MCollective 2009 Ruby Apache Yes PuppetLabs

20 Cloud APIs Aren’t Created Equal Open Source Abstractions

API Language

jclouds java

deltacloud python

Lib cloud python

Fog Ruby

Open Source Management Toolchains

22

Toolchain Concept

Source:http://code.google.com/p/devops-toolchain/wiki/ArchitectureComponents

23

Automated Toolchain

Cloud Image Launch

OS Install

System Con!guration

Application Service Orchestration

Capistrano RunDeck Fabric Func

BCFG2 Cfengine Chef Puppet

Cloud: Eucalyptus OpenStack CloudStack Abiquo

OS Install: Kickstart Cobbler Spacewalk

Command and Control

Con!guration

Bootstrapping Prov

isio

ning

Act

ivity

24

Example: GeekNet

Hundreds of servers, serving web, databases, and other infrastructure for some of the world’s most highly trafficked websites – over 40 million visitors per month.

•  Servers are automatically built using con!guration management software (Puppet)

•  Discovery tool (Zenoss) !nds infrastructure and populates a CMDB then spits out information to scripts that translate information to BIND con!gurations for DNS

•  Monitoring tool adds hosts to polling tool (Nagios) to check servers for availability

•  As infrastructure changes systems are updated automatically

•  Servers can be spun up and managed in minutes, not hours automatically with little or no human interaction

25 Example Toolchain: Online Gaming Studio

Questions?

27

Contact Me Professional: mrhinkle@cloud.com Personal: mrhinkle@gmail.com

Professional: mrhinkle@cloud.com Personal: mrhinkle@gmail.com

Professional: http://open.cloud.com Personal: http://socializedsoftware.com

Twitter: @clouddotcom @mrhinkle

Mark R. Hinkle

VP of Community Open Source Enthusiast

Open Source Toolchains to Manage Cloud Computing

http://www.slideshare.net/socializedsoftware

by Mark R. Hinkle is licensed under a Creative Commons Attribution-ShareAlike 3.0 United

States License.