openstack in action2! automate and accelerate cloud deployments with dell crowbar

Automate and accelerate cloud deployments with Dell CrowbarOpenStack in Action 2!Paris, May 31st 2012

Taco Scargo, EMEA Enterprise Technologist

Why does Dell have the right to be here?

• Dell powers 20 of the 25 largest clouds• Processes 13 billion security events each

day• Managing over 6,000,000 SaaS based

users• Managing more than 4 billion diagnostic

imaging objects in cloud storage• Over 1 billion connections to the Dell’s

own Cloud infrastructure each day• And last but not least: founding member of

OpenStack

Proven Cloud Experience

3 Dell Confidential

Value creation

Just as IT is not a commodity, cloud is not a destination. Cloud is a journey, placing IT squarely in the center of the enterprise as both facilitator and enabler of value-creation. Michael Dell, Chairman & CEO

The cloud is never at rest.

The cloud is always ready,

never finished.

How do you manage a cloud that`won’t stay still?

ProblemCloud

Operator Challenge

Cloud User Challenge Solution

Code is constantly changing

Deployment time > code delivery rate

Pace of innovation is

fast

Continuous deployment

Resources are constantly changing

At scale, no system is

steady state

Match output with

consumption demand

Automated deployment

Too many components

to easily track

Complexity is expensive

Require dynamic

integration

Centralized configuration management

Clouds demand significant operational and process controls

Operational decisions drive hardware and software decisions

We are finding ways to productize operations into best practices

Clouds require an Operational Focus

HW

SW OPS

OPS

SW

Dell OpenStack Cloud Solution

HW

SW OPS

“Crowbar” CloudOps Software

Services & Consulting

Reference Architecture

How DevOps is DifferentImages vs. Layers

What is DevOps?

StaffCommodityHardware

PackagedSoftware

OperationsScripts & Tools

CrowbarIT’S ALL

SOFTWARE!

Is an emerging set of principles, methods and practices for communication, collaboration and integration between application/software engineering and IT Operations that DRIVE AUTOMATION

Images vs. Layers: Overview

Images: Single UnitLayers: Stacked

Pieces

Integrations + Applications +

Utilities + Operating

System

Configuration

Operating System

Utilities

Application Bar

Application Foo

Integrations

Con

fig

ura

tion

Images vs. Layers: Lifecycle

Images: Replacement Layers: Upgrade

I+A+U+O/S

Config

OSU

Bar v1Foo

I

Config

I+A+U+O/S

Config

I+A+U+O/S

Config

I+A+U+O/S

Config

OSU

Bar v2 Foo

I

Config

Bar v2

There are lot of pieces, but they share many elements!

CloudOps for OpenStack

OSU

NovaNode

I

Config

OSU

NovaAPI

I

Config

OSU

NovaNode

I

Config

OSU

NovaNode

IC

onfig

OSU

NovaNode

I

Config

OSU

NovaNode

I

Config

OSU

NovaNode

I

Config

OSU

NovaDB

I

Config

OSU

NovaCntrl

I

Config

OSU

Dash Board

I

Config

OSU

Nagios

I

Config

OSU

CrowbarChef

I

Config

Plus interconnections are highly complex and dynamic

CloudOps for OpenStack

OSU

NovaNode

I

Config

OSU

NovaAPI

I

Config

OSU

NovaNode

I

Config

OSU

NovaNode

IC

onfig

OSU

NovaNode

I

Config

OSU

NovaNode

I

Config

OSU

NovaNode

I

Config

OSU

NovaDB

I

Config

OSU

NovaCntrl

I

Config

OSU

Dash Board

I

Config

OSU

Nagios

I

Config

OSU

CrowbarChef

I

Config

Mission: “A Zero Touch Cloud Installer”Servers in boxes to full function cloud in under 2 hours

Fast & Flexible Bare metal install including BIOS & RAID config Users can choose how their system is configured (“barclamps”)

DevOps Embracing Ongoing Operations Model (DevOps for Clouds) Leverages & Wraps Opscode Chef

Open Not specific to OpenStack – Dell using for other Apps Not restricted to Dell hardware Apache 2 licensed

What is Crowbar?

Crowbar Builds Up Clouds

Dell “

Cro

wb

ar”

Op

s M

an

ag

em

en

t

Core Components & Operating Systems

Cloud Infrastructure

Physical Resources

APIs, User Access, & Ecosystem Partners

1) Hardware Configuration

Dell “

Cro

wb

ar”

Op

s M

an

ag

em

en

t

Core Components & Operating Systems

Cloud Infrastructure

Physical Resources

APIs, User Access, & Ecosystem Partners

2) OS and Ops Infrastructure

Dell “

Cro

wb

ar”

Op

s M

an

ag

em

en

t

Core Components & Operating Systems

Cloud Infrastructure

Physical Resources

APIs, User Access, & Ecosystem Partners

* Suse pending, based on open source activity

*

3) Cloud Platform Integration

Dell “

Cro

wb

ar”

Op

s M

an

ag

em

en

t

Core Components & Operating Systems

Cloud Infrastructure

Physical Resources

APIs, User Access, & Ecosystem Partners

*

4) Cloud App ManagementD

ell “

Cro

wb

ar”

Op

s M

an

ag

em

en

t

Core Components & Operating Systems

Cloud Infrastructure

Physical Resources

APIs, User Access, & Ecosystem Partners

Server

How does Crowbar work?D

ell “

Cro

wb

ar”

Op

s M

an

ag

em

en

t

Operating System

Network Config

Chef-Client

Chef-Client

Chef-Client

Chef-Client

Chef Server

Network Config

Status (post)

Apps

DHCP

Cro

wb

ar

Mag

ic H

ap

pen

s!

Orchestration

State Machine

Run List

Config.

PXE Boot

Hardware Config(uses

“Sledgehammer”)

Apps

Crowbar = State Machine

Modular Design: BarclampsWhat Configuration You want to Deploy

Dell “

Cro

wb

ar”

Op

s M

an

ag

em

en

t

Core Components & Operating Systems

Cloud Infrastructure & Dell IP Extensions

Physical Resources

APIs, User Access, & Ecosystem Partners

* Crowbar has potential to service other programs beyond OpenStack

Crowbar

Deployer

Provisioner

Network RAID

BIOS IPMI

NTP

DNS Logging

Nova Swift Glance

Nagios Ganglia

enStratus

Dashboard

Ceph

Horizon

KeystoneForce10

Quantum

Dell “

Cro

wb

ar”

Op

s M

an

ag

em

en

tWhat is a barclamp?

Barclamp

Chef Recipes

O/S App Packages

Crowbar API &

Partial UI

Components & Scripts

• Crowbar API & Partial UI• Adds states and transitions to Orchestration• Customizes UI specific to barclamp• Visual extensions to nodes/networks/utils

• Chef Recipes• DevOps description of application• Enforced repeatable configuration

• O/S App Packages• Operating system dependencies (Debs & RPMs)

• Components & Scripts• Ad hoc configuration not covered by Chef

Dell “

Cro

wb

ar”

Op

s M

an

ag

em

en

tWhat is a barclamp?

Barclamp

Chef Recipes

O/S App Packages

Crowbar API &

Partial UI

Components & Scripts

Barclamp

Barclamp

• Interact as service layers• Have dependencies and pre-requisites• Can operate on multiple operating systems• Have independent version streams• Leverage/Reuse Chef Cookbooks

Networking in Crowbar

Server

IPMI

eth0

eth1

eth#

VLAN

VLAN

VLAN

VLAN

TeamedTeamed

IP Allocations

DHCP

Network Barclamp

Chef Client

not mgm’d

1g1

1g2

1g3Conduit

Conduit

Conduit

Other BarclampsRequest“Conduits”

Interface &ConduitsNormalizeNetworksBetweenNodes.

Networking in Crowbar

Server

IPMI

eth0

eth1

eth#

VLAN

VLAN

VLAN

VLAN

TeamedTeamed

IP Allocations

DHCP

Network Barclamp

Chef Client

Crowbar Does NOTConfigureThe Switch!O/S Onlynot

mgm’d

Crowbar DiscoversSwitches and Ports

DashboardView system information and status

Barclamps and ProposalsChoose what to deploy

Edit Proposals Configure and Select Servers

MonitoringView performance metrics and health

Nagios® and Ganglia are installed by default and are configured to correctly monitor the services installed by crowbar

A self contained cloud requires some supporting services. Crowbar also includes: DNS: Can be forwarded to an upstream

server NTP: Can be synced to an upstream time

server Logging: Can be sent to a centralized

logging server

Supporting Services

Questions?More info:• http://Dell.com/OpenStack - White Papers & More!• http://github.com/dellcloudedge/crowbar - Source code

and instructions• http://lists.us.dell.com/mailman/listinfo/crowbar -

Discussions and questions• Contact me at Taco_Scargo@dell.com

How many watt does it take torun a 48-node OpenStack cloud?

just 750 watt!

48 ARM-based quad-core servers in the3U PowerEdge C5000 chassis,aimed at the microserver market

Each ARM server node includes:1 SATA hard drive or flash drive1 DIMM slot for up to 8GB RAM1 GbE input per server into a non-blocking L2 switch per sled

Introducing Dell “Copper”

ARM Server Market Status

ARM Server Market Status

Product ARM Support Status

Ubuntu Available Now - http://www.ubuntu.com/download/arm

Fedora In Dev - Fedora-ARM Project

LAMP Stack Open Source Available Now – continued optimization

Hadoop Open Source Available Now – continued optimization

OpenStack Public demos May 2012

Oracle Java Available Now – continued optimization, http://www.oracle.com/technetwork/java/embedded/overview/embedded-faq-159987.html#faq4 

KVM Open Source Expected w/Cortex A15

Availability

• Seed Program• Continued delivery of Dell "Copper" ARM servers to select customers WW

• Includes DCS Classic and non-Classic customers

• Expected use: Test and development with web front-end and Hadoop workloads. No general availability at this time.

• Timeframe: Systems have already begun shipping, and customers have started testing

• Ecosystem Enablement:• Seed units will in-progress for key ecosystem partners, including Canonical,

Cloudera and others

• Remote accessible clusters for developer use include:• Dell Solution Centers: Centers planned worldwide, we will deploy multiple

clusters over time• Texas Advanced Computing Center (UT): Austin-based cluster as part of their

support of the development community

Questions?More info:• http://content.dell.com/us/en/enterprise/d/campaigns/

project-copper.aspx• Contact me at Taco_Scargo@dell.com