Radical innovations in storage for multi-tenant infrastructure John Griffith, Lead Open Source Developer & PTL, OpenStack Block Storage

How did we get to this point? •  First there was virtualization…and it was

good •  For smaller scale use cases it still is good •  But, when scaling virtual environments…

•  Adding and deploying hypervisors •  Networking headaches •  Management complexity •  Storage performance degradation

•  Something had to change

Graphic Source: http://crystaltec.com.au/services/virtualization

Enter Cloud Computing •  It’s no longer just about a collection of

virtual machines •  It’s all about:

•  Abstraction •  Automation •  Resource utilization •  Maximum efficiency •  Utility consumption model •  Dynamic and massive scale •  Common interfaces / APIs

Graphic Source: http://www.intelligentitnyc.com

Enter OpenStack •  Project launched in 2010 •  Open source software for building

and managing clouds •  A collection of software projects that

tie into common management layer •  Created to

•  Drive industry standards •  Accelerate cloud adoption •  Put an end to cloud “lock-in” •  Deliver AWS-like functionality and

economics outside of AWS

Storage Infrastructure Implications •  One of many abstracted resources tied into common management

layer •  Virtual machines, Bare Metal, Storage, Networking, etc.

•  Must be… •  Capable of granular & dynamic resource provisioning •  Designed for multi-tenancy & scale •  Easy to use REST-API •  Enabling self-service virtual environments for end users

Storage in the OpenStack Context Cinder / Block Storage Swift / Object Storage

Objectives •  Storage for running VM disk

volumes on a host •  Ideal for performance sensitive apps •  Enables Amazon EBS-like service

•  Ideal for cost effective, scale-out storage •  Fully distributed, API-accessible •  Well suited for backup, archiving, data retention •  Enables Dropbox-like service

Use Cases

•  Production Applications •  Traditional IT Systems •  Database Driven Apps •  Messaging / Collaboration •  Dev / Test Systems

•  VM Templates •  ISO Images •  Disk Volume Snapshots •  Backup / Archive •  Image / Video Repository

Workloads •  High Change Content •  Smaller, Random R/W •  Higher / “Bursty” IO

•  Typically More Static Content •  Larger, Sequential R/W •  Lower IOPS

Cinder 101 •  Architected to provide traditional block level storage resources to other OpenStack services

•  Presents persistent block level storage volumes for use with OpenStack Nova compute instances

•  Manages the creation, attaching and detaching of these volumes between a storage system like SolidFire and different host servers

How Cinder Works

•  Plug-in architecture, use your own vendors backend(s) or use the default

•  Default implementation built on LVM and iSCSI •  Mix and match, add and remove •  Back-end devices can be invisible to end-user/

tenant •  Consistent API regardless of backend selection •  Filter scheduling to auto place newly create volumes •  More specific placement based on volume-type

selection •  Expose differentiating features via custom volume-

types and extra-specs

Cinder Base Features •  Create/delete volumes •  Specify custom "types/extra-specs” •  Clone •  Copy image to volume and volume to image •  Point in time copy (snapshots of volumes) •  Create volume from snapshot •  Backup volume (to object store, SWIFT and CEPH) •  Transfer volume ownership •  Customized scheduling filters •  Per tenant usage quotas

Cinder Design View

Cinder Benefits

•  Dynamic pool of block storage resources •  Horizontally scalable •  Well defined, easy to use API

•  Strict adherence to API compatibility regardless of backend

•  Back-End Storage Differentiators •  To the Admin

•  Cost

•  Management

•  Reliability

•  To the End-user •  Performance

•  Reliability

Vendor Unique Features In Cinder •  Exposed through custom types or extensions •  Different back-ends for different use cases •  Back-End selected by filter scheduler •  Back-End is setup based on desired capabilities and

characteristics

Striving for Self-Service

•  Idea is for the end user to be able to request resources on demand

•  “Give me 100 GB of block storage with XYZ characteristics please”

•  “It’s mine, now what can I do with it” •  Attach it, boot it, clone it, back it up

SolidFire’s Scale-Out Block Storage System Designed for OpenStack and other cloud platforms

•  Scale-Out Design, 100 nodes, 3.4PB, 7.5M IOPS •  Industry standard hardware, 10gb iSCSI •  Linear non-disruptive growth •  Automation top priority in API design •  Built to deploy in an OpenStack environment •  Extreme fault tolerance •  Best-in-Class QoS Architecture

Eliminating Noisy Neighbors

The Noisy Neighbor Effect •  Individual tenant impacts other applications

•  Unsuitable for performance sensitive apps

SolidFire QoS in Practice •  Create fine-grained tiers of performance

•  Application performance is isolated

•  Performance SLAs enforced

Traditional Multi-Tenant Performance

Noisy Neighbor

Traditional Multi-Tenant Performance

Application of SolidFire QoS controls

SolidFire & Cinder

•  Full SolidFire driver integration with latest OpenStack software release

•  Set and maintain true QoS levels on a per-volume basis

•  Create/snapshot/clone/manage SolidFire volumes using OpenStack clients and APIs

•  Run instances on a SolidFire volume •  Enhanced boot from volume capabilities •  Web-based API exposing all cluster

functionality •  SolidFire/Cinder integration can be

configured in less than a minute

SolidFire & Nebula

•  Brings together the founding engineers and leaders of the Cinder project

•  Validated interoperability •  Integrated user experience for customers

adding SolidFire storage appliances to their Nebula One system

"In a cloud infrastructure, some applications require very high levels of consistent storage performance. Adding a SolidFire appliance to the Nebula One system brings a level of performance predictability not possible from spinning disks in our certified industry standard servers.”

- Chris C. Kemp, Co-Founder & CEO, Nebula

Why SolidFire & OpenStack?

•  No better block storage option for customers seeking guaranteed performance, high availability and scale,

•  Designed from the ground up to integrate with orchestration layers like OpenStack

•  Unrivaled integration & support •  Breadth/depth of API creates significant flexibility •  All core Cinder API features implemented in SolidFire

driver •  No additional features/licenses required;

•  Driver included in OpenStack distribution •  Don't have to buy license to enable snapshots, clones, etc. •  Don't have to buy an SDK, or management server, etc.

Build your cloud… Out of technology built for the cloud

Key Cloud Design Principles Relevant SolidFire Features

Abstraction Performance Virtualization

Automation All cluster operations emanate from API

Resource Utilization •  Fine grained QoS •  Provision capacity independent of performance

Maximum Efficiency •  Built-In Data Reduction •  Best-in-Class hardware density

Utility Consumption Model Dynamic Provisioning & Modifications

Dynamic & Massive Scale Linear, Non-Disruptive, Single Management Domain

Common Interface & APIs Enabled via REST-based APIs

1620 Pearl Street, Boulder, Colorado 80302 Phone: 720.523.3278 Email: info@solidfire.com www.solidfire.com

Questions?