openstack day italy: openattc as an open storage platform for openstack

www.it-novum.com© it-novum GmbH

Seite 1

Building a flexible and large-scale software-defined storage platform for OpenStack

OPENSTACK DAY ItalyMay 30th 2014, Milan

Michael Kienle, CEOit-novum GmbH

[email protected]


Seite 2

Source: RED HAT

Open Clouds are picking up right now!


Seite 3

Open Stack: Open cloud operating system supported by theOpenStack Foundation backed up by ### well-known companies

Want to learn more: listen to the other tracks

Major Design Guidelines are OPEN:Open source (apache), Open design (blueprints), Open development(branches), Open community (tech leads)

Although OpenStack is purposely open it partly relies on underlying proprietary storage products.

To build your cloud upon a truly OPEN software-defined storage platform (SDS) you might achieve greater flexibility and $$$$-savings.

openATTIC is a SDS cloud storage platform optimized for OpenStack'sCinder component.

What is this all about?


Seite 4

Introduction & AgendaOpenStack

Focus on Storage within OpenStack, esp. CinderRequirements & Optimization of current approaches

SDS – software-defined storageHow it worksSDS as part of the software-defined datacenter (SDDC)Why now?Advantages

openATTICHow to bring SDS and OpenStack togetherVideo/Examples

Agenda


Seite 5

Leading provider of open source business solutions75 employees currently; approx. 20% growth p.a.Headquarters in Fulda/DE; Branch office in Vienna/ATHundreds of mid-sized and large corporate clientsOperating public listed parent company‘s data center for 20+ yearsand migrated most of this to open source

Mission: Help customers to build and run a 100% open-source software-defined datacenter

Proven Expertise in Business Open Source Infrastructure and ApplicationsSAP Extended Business Partner (for integrating SAP into open sourcesolutions)

Open Business Solutions. Delivered!


Seite 6

Generic Cloud Drivers: Abstraction, Automation, Self-Service, efficient use of resource pools, dynamic & massive scale, commonAPIs

Manage Virtualization across different Hypervisors (KVM, VMware, Xen, Hyper-V etc.)

Support the whole IaaS stack (Storage, Security, Networking etc.)

Scale far beyond the limitations of an „old virtualized landscape“

Enable the software-defined datacenter (SDDC)

OpenStack: Value Proposition


Seite 7

Generic Cloud Drivers: Abstraction, Automation, Self-Service, efficient use of resource pools, dynamic & massive scale, commonAPIs

Manage Virtualization across different Hypervisors (KVM, VMware, Xen, Hyper-V etc.)

Support the whole IaaS stack (Storage, Security, Networking etc.)

Scale far beyond the limitations of an „old virtualized landscape“

Enable the software-defined datacenter (SDDC)

OpenStack: Value Proposition


Seite 8

Storage costs optimization ($$$$)Data growth of >>40% p.a. needs to be managed automatically and waycheaper

Flexibility and reduced vendor lock-in (open)Incumbent players are perfect in securing their huge marginsUsing innovative technology without renewing existing assets

Cloud Computing has characterized the next generation IT delivery model, easy orchestration/automation of (storage) services is essential

need scale-up/scale-out of underlying storage at lesser costsSoftware-Defined DataCenter (SDDC) becomes a major disruptive IT trend(SDS beeing part of this)

Standardisation enables automation & abstraction of software from hardware-> Commodity hardware and SDS gained strength

Customers‘ Perspective on Storage


Seite 9

Cinder / Block Storage Swift / Object StorageObjectives • Storage for running VM disk

volumes on a host• Ideal for performance

sensititve apps

• Ideal for cost effective,scale-out storage

• Backup & Archive• Dropbox-like services

Use Cases • Production Applications• Traditional IT Systems• Database-driven Apps• Dev-/Test Landscape

• ISO Image- and VM-Templates

• Snapshots• Backup & Archive• Repository

Workloads • High chance content• Smaller, Random R/W• Higher / Bursty IO

• More static content• Larger, sequential r/w• Lower IOPS

Storage in OpenStack


Seite 10

Architecture to provide block-level resources to OpenStack („block storage asa service“)Persistent block-level storage volumes for use with OpenStack Nova compute instances (e.g. iSCSI target)Manages creation, attach/detach, clone of these storage volumensIntention of Cinder: keep you from beeing locked-in:

Ability to change the backend (the storage system) easilyMigrate data & expect the same base level of functionality

Many vendor/storage plug-ins for Cinder available;the choice between functionality and $$$$

Cinder - Details

LegendPresistent volume controlPresistent volume data

VM instance

Note that iSCSI isjust an example –several additional

protocols aresupported (e.g.,

FC, NFS)

Nova

/dev/vda

iSCSIinitiator

Cinder

KVM

iSCSItarget


Seite 11

Limitations of Commodity Storage (like a simple Linux server orcomparable) because of lacking features like:

node redundancy & clustering & HA, scalingQoS, Deduplication/Compression, ReplicationExample: Image Cloning (e.g. image from Glance) could be fasterthan block-based copy&paste…

Limitations of Enterprise Storage (the big names):Vendor lock-in$$$$$

Consider Open Source & SDS !

Choose your underlying storage system!


Seite 12

SDS is the abstraction of the software from the hardware.

Benefits:Independence from big-name commercial developer of proprietarysystems (bundle of hardware and storage os) for better pricing andfreedom of choiceHardware is treated as a pool, scalable across multiple storagesystemsInnovative functionality (e.g. top-down policy driven), easier toadministrate, faster to implement (software can be extended andmaintained more easily than hardware)Part of the Software-defined DataCenter (SDDC)

By 2016 the avail market for OpenSource-Storage is 2,5$B (Gartner, “Tech Go-to-Market: Storage Providers Should Focus on Three Growth Opportunities Through 2016”, Nov 2013), by 2018 OpenSource Storage will gain 20% of the market , approx. 10$B (Gartner, “2014 Strategic Roadmap for Storage”, March 2014).

Software-defined Storage (SDS)


Seite 13

ConfigurationManagement

System (CMS)

Service Transition Service OperationService Design

Monitoring

Event Correlation

User

Oth

erM

onito

rs

CapacityManagement

RiskManagement

Cloud Self Service

Sys

tem

App

licat

ion

End

-2-E

nd

Proc

ess

ChangeManagement

ProblemManagement

SAP

ApplicationsService DeskNetwork Operation Center

ReleaseManagement

Cloud Management

IncidentManagement

Infrastructure

Software-defined Storage Virtualization

CMDB

Discovery

Rep

ortin

gSe

rvic

e Le

vel M

anag

emen

t

The Big Picture: SDDC with 100% open sourceThe Big Picture: SDDC with 100% open source


Seite 14

ITSMi-doit & OTRS

Umbrella MonitoringopenITCOCKPIT

Cloud ManagementOpenStack

ReportingPentaho

Billing, Backup,...SAP, SEP sesam

ConfigurationDeployment

Services & Tools

API

Web

CLI

Apps

StatusChecking

ApplicationManagement

AccessSamba, NFS, AFP, iSCSI, FC

AvailabilityDRBD, Pacemaker,

Corosync, ...

EfficiencyZFS, LVM, BTFRS, XFS

BackupSnapshotting, Tape-

Backup, ...

AnalysisopenITCOCKPIT,

Nagios, ...

Back end

Core

Front end

openATTIC Framework - Architecture

Interfaces


Seite 15

MirroringBlock- & File-basedSynchronous & asynchronousObject-based

ClusteringAutomated Failover Multi Node Support

Support & ServicesTraining & OptimizationIntegration & Development Guaranteed Response Time Guaranteed Restore Time(Appliance only)

ProtectionIntegration Tape-SystemsSnapshottingConsistent Snapshotting

EfficiencyDeduplicationCompressionPerformance Config-Set Config-Set for Virtualization

Unified and versatileSAN / Block Storage (FC, iSCSI, FCoE)NAS / File-based (CIFS, NFS)Object Storage

Central ConfigurationWeb FrontendCommand Line Interface

Dynamic MonitoringAvailabilityPerformanceCapacity

Cloud SupportOpenStack/CinderopenQRM

Umbrella ManagementMaster/Slave Architecture

Current FeaturesOpen API, Scalable, Cinder-ready


Seite 16

Current FeaturesBenchmarkExample: Read/write performance using ZFS Pool


Seite 17

All access protocols, block (FC,iSCSI, FCoE) and files storage systems (CIFS, NFS) in every single openATTIC systemIn its latest definition Unified Storage contains object storageas well (Ceph, Swift). Object storage architecture manages data as objects and allows inexpensive, scalable and self-healing retention of massive amounts of unstructured DataopenATTIC can use Ceph for Object Storage and enables the following

Extending storage capacity and performance seamlessly on demand (multi-node scalibility)No upfront investment costs for eventual higher requirements in the futureScale out capacity and performance without migrate the existing infrastructure

Unified StorageExpample: Scale out with Object Storage


Seite 18

More Cloud Support/Integration: openstack Swift, CloudStack, Eucalyptus…QoS for top-down orchestration of services and policy enforcementBusiness-aware/application-aware SnapAppsSnapLock: Permanent and compliant long-term storage for meetingregulatory requirementsFlash/SSD Support for AppliancesBetter Security Hardening and Performance Optimization forAppliancesBetter support for easy scale-up/scale-out Deeper Integration of Big Data solutions: Hadoop, SparkEye-Candy GUI front-end Better documentation and more video tutorials….

Product Roadmap


Seite 19

Volume create/deleteSnapshot create/deleteVolume attach/detach (called by Nova)Create Volume from SnapshotCopy Image to Volume / Volume to ImageGet Volume Stats, Clone VolumeExtend Volume (mandatory since Icehouse)

openATTIC Cinder plug-in: VideoWe show you some examples how this works…

How it works: Cinder API (Icehouse)


Seite 20

Unit testsCinder driver certification fileRelease hopefully next one…

More Info: www.openattic.org/enIntegration in other cloud platforms has been done already

OpenStack Cinder / openAttic: Status


Seite 21

Open and comprehensive API achivies great flexibility forintegration into different cloud platformsOptimization of huge data through intelligent openATTICfunctionalities

Consistent snapshots relieve backup, accelerate desaster recoveryand reduce downtimesDeduplication, Compression, efficient mirroring and redundancy

Huge cost savings:openATTIC runs on commodity hardwarereducing vendor lock-in Open Source approach saves license costs

Are there limitations of software-defined storage / software-defineddatacenter?

Summary: openATTIC within OpenStack


Seite 22

Open and comprehensive API achivies great flexibility forintegration into different cloud platformsOptimization of huge data through intelligent openATTICfunctionalities

Consistent snapshots relieve backup, accelerate desaster recoveryand reduce downtimesDeduplication, Compression, efficient mirroring and redundancy

Huge cost savings:openATTIC runs on commodity hardwarereducing vendor lock-in Open Source approach saves license costs

However, the bad news is: even software-defined datacenters still need some underlying hardware…

Summary: openATTIC within OpenStack


Seite 23

Thank you!Questions? - More than welcome!

Cloud Storage

Intelligent SDS

Unified Storage

Power your cloudPower your cloud Rock-solid storageRock-solid storage Scale out data center Scale out data center

Power your cloud & scale out your data center with rock-solid Software-Defined Storage (SDS) truly on open source, join the community & more

about the openATTIC project at http://openattic.org/en

Power your cloud & scale out your data center with rock-solid Software-Defined Storage (SDS) truly on open source, join the community & more

about the openATTIC project at http://openattic.org/en

openstack day italy: openattc as an open storage platform for openstack

Software

openstack block storage

underlying storage

novum gmbh seite

storage system

open stack

sds cloud storage platform

iaas stack storage

storage costs optimization