webinar- understanding ibm cloud storage and sds solutions - may 16 2014 v2.ppt

© 2014 IBM Corporation

IBM STG - Understanding IBM Cloud Storage and Software-Defined Storage (SDS) Solutions

May 16th 2014 - STG Enablement Webinar May 16th 2014 - STG Enablement Webinar


Purpose of Today’s Webinar

Understand IBM STG’s Cloud Storage

and SDS Strategy

Deliver Vision and Strategy to your

Clients

Technical and Sales Enablement &

Roadmap

Introduce Experts available on Technical Leadership Council (TLC), and create basis for ongoing deeper dive webinar series

Major shifts occurring at our clients in place of traditional IT Infrastructure models. Understand these changes and IBM’s vision

Content designed such that this presentation (or subset) can be used by you to deliver our vision, strategy, and solutions to your clients

2


Cloud is a growth engine for business

1 IBM, The power of cloud: Driving business model innovation, February 2012.2 IBM Center for Applied Insights, Global Cloud Enterprise Study, 2013.

more companies will use the cloud for significant innovations by 2015.12x

higher gross profit for enterprises aggressively adopting cloud for competitive advantage.22.4x

Rapidly optimize data architectures yielding creative and unrealized business insights

InfrastructureMatters!

Cloud helps you deliver solutions faster and realize benefits sooner

3


Cloud Delivery Models

A private cloud is one in which the cloud infrastructure is provisioned for exclusive use by a single organization, comprising multiple consumers. It may be owned, managed and operated by the organization itself or a third party, and may exist on or off premises.

A hybrid cloud combines multiple elements of public and private cloud, including any combination of providers and consumers, and may also contain multiple service layers.

A public cloud is one in which the consumer of cloud services and the provider of cloud services exist in separate enterprises. The ownership of the assets used to deliver cloud services remains with the provider.

The IT department is in direct competition with

Public Cloud alternatives for Lines of Business

funding

4

© 2014 IBM Corporation5

Open Cloud Platforms OpenStack, VMWare, Hyper-V


Brian Sherman – Distinguished Engineer Brian Sherman – Distinguished Engineer


Cinder and Horizon Automation

Deploy XIV and Storwize storage in Horizon Automated volume, host and initiator creation Cinder development contributor

Complete Storage Integration All Storage APIs consumed Co-development on vVol and VASA Automation SRM enablement Disk Management from vCenter and vCenter Operations

Suite (vCOPs)

Storage Enablement

Manage XIV in Microsoft System Center Virtual Machine Manager (SCVMM)

Space Reclamation

Smart Cloud User portal for self service Proven IBM services platform and SLAs

IBM Comprehensive Storage Management in Cloud

6


Enable and Build Better Clouds: RESTful API

What the XIV RESTful API offers: Agnostic/Common to OS and cloud

platform programming language Uses http in browser

– Doesn't need a client (like other APIs)

– Every object has a unique identifier (URL)

Uses the following http actions:– Get (Read)

– Put (Create)

– Delete (Delete)

– Post (Update)

– SnapShot and Copy Services support plus more

EX : GET XIV/ (lists XIVs)

PUT XIV/:xiv1/volumes/:vol1 size:,pool

Maximum flexibility in automating your custom cloud

Benefits:• Management API for automating your cloud• Integrates XIV natively with your cloud

management software• Delivered through IBM Hyper-Scale

Manager (this is your web server)• Offers flexibility to meet needs beyond

standard cloud tools

XIV RESTful API is one of the first on the market!

7


Overview of OpenStack: Key components

HorizonHorizon

NovaNova

CinderCinder

SwiftSwift

NeutronNeutron

KeystoneKeystone

GlanceGlance

OsloShared ServicesOsloShared Services

8

Storage Tiers New in Icehouse :– Use for performance or Quality of

Service– Achieved through volume "types“– Ability to retype– Host or Volume will handle the

conversion depending on driver


Gold SponsorsGold Sponsors

IBM is #2 in contributions to integrated projects

IBM is #2 in contributions to integrated projects

IBM is working to accelerate OpenStack Foundation success

Because an open interoperable Cloud is critical for flexible cloud deployment and customer success…

IBM has 15 core contributors IBM has 15 core contributors15

IBMers working on OpenStack – from formation of the Foundation to Code Quality and New Function

IBMers working on OpenStack – from formation of the Foundation to Code Quality and New Function380

IBMers active in the projects IBMers active in the projects

2

107

Mar2013

April 2014859

Contributors8,500 Individuals

2130Contributors15,800+ Individuals

Exponential growth in ~1 YR

Platinum SponsorsPlatinum Sponsors

OpenStack Participant GrowthOpenStack Participant Growth

9


IBM Storage and OpenStack

IBM is a leading contributor to OpenStack Cinder– IBM SVC/Storwize family, IBM XIV,

TPC/VSC, Elastic Storage and IBM DS8870– Supports Essex to Icehouse releases– iSCSI or Fibre Channel– Full ability to leverage Easy Tier, Real Time

Compression, volume movement, etc• Software defined placement via OpenStack

filter scheduler

IBM is driving forward OpenStack storage:– Extending with volume migration – Adding enterprise attributes to OpenStack

Object Store (Swift)

IBM Storwize family IBM XIV

Cinder

“Managing IBM Storwize V7000 Storage with OpenStack”

http://www.youtube.com/watch?v=JWMm6-zwrMw

IBM DS8000

Storage for the Cloud

Austin10/21/2010

Bexar3/3/2011

Cactus4/15/2011

Diablo9/22/2011

Essex4/2/2012

Folsom9/27/2012

Grizzly4/4/2013

Havana10/17/2013

Icehouse4/17/2014

Juno10/17/2014

10


OpenStack Deployment at eBay

ProductionMagento GOMagento CommerceVarious Public Products

DevelopmentX.Commerce QALnPDevelopment

Test

OpenStack TestUpgradesPerformance

Host all environments

Mirror sites for HA between Colo’s

“XIV is able to ingest any workload without having to configure LUNs…other vendors do not give the same level of ease of use, performance, or predictability of I/O as XIV.”

Roark Hilomen, eBay, at Edge2013

Watch the video

eBay

11


MySQL PerformancePerformance improvement on XIV and OpenStack private cloudBare Metal (BM) = Standard server with local storageBased on Production workloads

OpenStack Production Example


Cloud Economics… Infrastructure Matters

Simplified Management– Improved admin productivity– Lower skills required

Fewer Servers and Network Connections – Fewer cores and connections– No need for Fibre Channel SAN

Higher Storage Utilization– Deliver SLAs with increased utilization

Lower Software License Costs– Fewer cores– Database, infrastructure, SW

Lower Environmental Costs– Power / cooling, floor space

Cloud Optimized Storage

Lower Entry Costs; Inefficient at Scale

Increasing Complex and Inefficient for Cloud and Big Data Needs

Commodity Storage

TIME/CAPACITY

TIME/CAPACITY

OPEX

CAPEX

$

$


Commodity Storage

Commodity Storage


OPEX: Up to 75% of Cloud Storage Costs

13


IBM Hyper-Scale Vision – support large growth without overhead

14

IBM Hyper-Scale Mobility

Cross-system data mobility that is transparent to host applications and governed by QoS and management

policies

IBM Hyper-Scale Mobility

Cross-system data mobility that is transparent to host applications and governed by QoS and management

policies

IBM Hyper-Scale Consistency

Smart, store-aware software ensures application consistency upon cross-system data mobility

IBM Hyper-Scale Consistency

Smart, store-aware software ensures application consistency upon cross-system data mobility

IBM Hyper-ScaleElastic, scale-out data store of

multiple XIV frames

Managed as a single system; easily add or remove frames;

XIV SW license – all inclusive

IBM Hyper-ScaleElastic, scale-out data store of

multiple XIV frames

Managed as a single system; easily add or remove frames;

XIV SW license – all inclusive

IBM Hyper-Scale Manager

An application that provides a single point of management – planning,

monitoring, configuring, setting QoS and management policies

IBM Hyper-Scale Manager

An application that provides a single point of management – planning,

monitoring, configuring, setting QoS and management policies

4Q 2012

2Q 2013

4Q 2013


Machine 2

Delivered when Machine 1 hits expansion threshold (70% utilization)

$1 Initial invoice for Machine 2

Remaining 40% payment for Machine 1

Delivered when Machine 1 hits expansion threshold (70% utilization)

$1 Initial invoice for Machine 2

Remaining 40% payment for Machine 1

Machine 3 to N

Delivered when prior Machine hits expansion threshold

$1 Initial invoice for current Machine

Remaining payment (100% less $1) for prior machine

(at end of 1 yr term, client pays for any delivered system)

Delivered when prior Machine hits expansion threshold

$1 Initial invoice for current Machine

Remaining payment (100% less $1) for prior machine

(at end of 1 yr term, client pays for any delivered system)

Machine 1

100% Authorized for performance and capacity

Initial payment of 60% of total price

100% Authorized for performance and capacity

Initial payment of 60% of total price

Key points:• Client enjoys full benefit of system well in advance of payment• Fully populated system – not detuned!• Fully Authorized Capacity and Performance! No need for key activation• Ability to run several parallel ASP tracks for multi-site data centers and

client needs• Ability to construct offering as a Capital Acquisition Model or, in many

cases, as an Opex model with payments spread over 36 months (60% stepping up to 100%, or no payments stepping up to 100%)

Key points:• Client enjoys full benefit of system well in advance of payment• Fully populated system – not detuned!• Fully Authorized Capacity and Performance! No need for key activation• Ability to run several parallel ASP tracks for multi-site data centers and

client needs• Ability to construct offering as a Capital Acquisition Model or, in many

cases, as an Opex model with payments spread over 36 months (60% stepping up to 100%, or no payments stepping up to 100%)

Overview: New program launching in 2Q 2014 – separate from

existing CUoD / Utility programs

Ideally suited for Cloud environments, often providing the ability to charge out to clients well in advance of IBM acquisition costs

Typically provides optimal acquisition structure for variable workloads

IBM installs a full system with partial payment based on initial usage

Depending on client growth rates and SLA, IBM pre positions subsequent systems in advance of client needs – typically at 70% Utilization

Simple acquisition model:–Minimal CE involvement in capacity upgrades–Detailed capacity monitoring not required

Best for clients with rapidly growing storage requirements needing highly flexible acquisition models

Can be structured as CAPX and OPEX offering via IGF

May be sold direct today with Partner Channel in 2Q

Available for 15 module XIVs and fully configured FlashSystem 840s.

Advanced System Placement (Frame Ahead) for XIV

15


A Path to Cloud with Software Defined Storage (SDS)


Chris Vollmar - Client Technical SpecialistChris Vollmar - Client Technical Specialist


ApplicationAware

ServiceDriven

ResourceSmart

On or Off Premise

Dynamic Cloud

Software Defined Environments are the foundation for building a Dynamic Cloud

17


Integrate front-office Systems of Engagement with back-office Systems of Record

Rapidly develop services for new interactions beyond the four walls of the enterprise

Quickly and easily scale to capture all of the different endpoints from front-office engagements

Act immediately on data from real-time integration to enable continuous insights

Systems of Interaction

The new era of Systems of Interaction transforms how companies engage, interact and transact with their customers

Exploding Data Volumes

Diverse Data Types

Increasing Valueof Information

Mobile and Social Engagement

18


Application Aware infrastructure leverages best practices with patterns of expertise

What the business wants:

• Define business needs

• Identify service opportunities and requirements

• Quickly experiment and test new services

What Software Defined Environments provides:

• Patterns of Expertise to link solution to infrastructure based on business rules

• Automated orchestration of workloads

• Analytics-based optimization of workload to maximize outcomes

Co

ntin

uo

us

Op

timiz

atio

n

PriorityPolicy

Service

PriorityPolicy

Service

Solution Definition

19


What are Software Defined Environments (SDE)?

With IBM’s Software Defined Environment, infrastructure is fully programmable to rapidly deploy workloads on optimal resources and to instantly respond to changing business demands

Application AwareApplication Aware

ComputeCompute StorageStorage NetworkNetwork

AnalyticsPatternsDefinition

Resource SmartResource Smart

Software

Defined

Environments

Abstracted and virtualized IT infrastructure resources managed by software

Applications that define infrastructure requirements and configuration

IT infrastructure that extends multiple environments to go beyond the data center

20


Software Defined Storage (SDS) as a component of SDE

SDE uses a Resource Abstraction layer to make Compute, Network, and Storage facilities appear as needed by Workloads

SDE relies on virtualized physical resources, which can be configured to match the SLAs of services defined in the Service Catalogue

2

Workloads

Traditional Middleware Based

Cloud Based Services

Software Defined Compute

Software Defined Network

Software Defined Storage

Heterogeneous Compute Resources

Virtualized Network

Virtual Storage Layer

Resource Abstraction

Software Defined Infrastructure

Workload Definition

Workload Orchestration

Workload Definition and Orchestration

Software Defined Environment

21

SDS creates smarter interactions between workloads and storage by:

– Exposing storage capabilities for the workloads to dynamically provision storage with the most suitable characteristics (i.e. OpenStack, VMWare)

– Introducing new operations and concepts between workloads and storage to let storage better adapt to the needs of workloads (i.e. Applied Analytics)

– Moving some of the storage functions closer to the workloads to leverage higher-level knowledge and/or lower cost (i.e. Ability to lift storage services off of disk systems)

– increasing reliance on commodity hardware


Software Define Storage – Control and Data Planes

IBM SDS Control Plane Policy

AutomationAnalytics and Optimization

Integration and API Services

Backup and Copy

Management

Storage Virtualization

Storage Cloud

IBM SDS Data Plane Scalable storage Capacity

Virtual StorageVirtual Storage

Open Common Control Plane to Manage Storage Services for the Workloads

22

Storage SystemsStorage Systems

ISISVMWare integration

SCSA Cloud service portal

OpenStack APIs Cinder support

TPC, FCM, TSM Enterprise Storage management

VSCVirtual StorageCenter

Workload Optimized and Transaction Integrity

Agility and Rapid Scale


Current SDS Building Blocks in the Control and Data Planes

Backup Integration and Automation

OpenStack VMware Self-Service PortalIBM SmartCloud Orchestrator

File and Hadoop Storage

IBM and 3rd Party Block Storage

Object Storage

IBM Storage Integration Server Storage integration for virtualized

environments (VMWare)

VSCSDS Control Plane

SDS Data Plane

23


Why SDS? SDS addresses top storage pain points…

Controlling out-of-control storage growth

Reducing Costs

Minimizing performance bottlenecks

Integration of Provisioning

Supporting Server Virtualization

Source: The Info Pro 2012

24


IBM SmartCloud Virtual Storage Center (VSC)




Storage services in a block Software Defined Storage Solution

PCS / IBM Confidential

Thin ProvisioningNon-disruptive data

migration

Integration with Server Hypervisor's

Storage AnalyticsEngine

Real-time Compression

Application Integrationwith FlashCopy

Manager

EasyTierAdvanced Mirroring

andCopy Services

Common Management

CloudReady

26


IBM SmartCloud Virtual Storage Center• Enables storage for cloud computing• Easy to buy and deploy• Pay-as-you-go pricing

SAN

Snapshot Data Protection

Snapshot Data ProtectionStorage VirtualizationStorage Virtualization

Infrastructure Resource Management

Infrastructure Resource Management

IBM SmartCloud Virtual Storage Center (VSC)

Storage Optimization, Provisioning andTransformation

Storage Optimization, Provisioning andTransformation

27


IBM SmartCloud Virtual Storage Center (VSC) - Portfolio

IBM SmartCloud

VSC for Storwize

• Restricted to the Storwize V7000 and V5000 hardware

• ‘Per Enclosure’ price metric for all internal and externally virtualized Storage enclosures

• Includes all features of VSC (external virtualization, Mirroring, Advanced Analytics);

• License does not include base SW license for Storwize HW

IBM SmartCloud

VSC -Entry

• Restricted to 250 TB’s and 2 SVC I/O groups

• ‘Per TB’ price metric

• Includes all features of standard VSC license (virtualization, FlashCopy, Mirroring, Advanced Analytics)

• Leverage Real-Time Compression to extend capacity management beyond 250 TB

IBM SmartCloud

Virtual Storage Center

• Global Enterprise License Structure

• Storage profile of 200TB to 1PB +

• Leverages IBM SVC Engines or I/O groups

• Single per TB license to include:

• SVC Virtualization• SVC FlashCopy• SVC Remote Mirroring• TPC • FCM• Storage Analytics Engine

Storage as the Cloud or Storage for the Cloud

28


Advanced Analytics and Cloud Functionality in VSC

Advanced Analytics• Automated Storage Tiering: – Automatically migrate volumes between service

classes / storage pools based on user defined policy– schedule recurring analysis

• Dynamic Storage Pool Utilization– Real-time view of storage pool utilization with additional

tags for attributes like name and location

• Storage Volume Transformation– Ability to transform an existing storage volume into a

different type of volume to take advantage of built-in efficiency features of the storage virtualization layer within VSC

• Service Classes / Storage Pools– Manage service classes and storage pools within the

graphical interface with ability to create, modify, anddelete resources

• Self-Service Provisioning– Improved interfaces for submitting storage requests

either via the user interface (with role-based authorization) or via external tools like SmartCloud Storage Access via integration through available APIs

29


SmartCloud Storage Access (SCSA) – “Self Service Storage”




IBM SmartCloud Storage Access R1.2 at a glance

Self-service – Capacity provisioning, monitoring, reporting

Web-based access– Simple GUI via Internet

Monitoring and reporting capability– End-user and administrator– Usage metrics export function

Service Class customization Control

– Complete approval control process– Automatic expiration policy control

Open– Storage cloud integration API (REST) for 3rd party application

integration Support

– File (CIFS/NFS) storage– Block (FC) storage– SCSA Instrumentation Layer (powered by TPC)– Federated authentication

31


End to end private cloud storage solutionEnd to end private cloud storage solution

Self service portalSelf service portal

Required elements that may already exist at customerRequired elements that may already exist at customer

Designed to offer a simple cloud storage architecture

Cloud, Dept admin

Cloud service users

Developers

Testers

Storwize family

XIV SONAS

Instrumentation Layer (SCSAIL) powered by TPC or

Tivoli Storage Productivity Centeror

Virtual Storage Center

Authentication Server

32


Software Defined Storage for Unstructured data – Code name: Elastic Storage


Leonard Degollado – Client Technical SpecialistLeonard Degollado – Client Technical Specialist


Storage services in an Unstructured Software Defined Storage Solution

PCS / IBM Confidential

Commodity HardwareNon-disruptive data

migration

FlashAcceleration

Network performance

monitoring

File PlacementOptimization

Native Encryption and

Secure Erase

Global Active File

Management

Advanced Mirroring and

Caching Services

Common Management

CloudReady

34

Storage ResourcePools

Global Namespace


Elastic Storage

Proven, scalable, high-performance software defined storage solution – Simplified data management and automated information lifecycle tools – Capable of managing petabytes of data and billions of files

15 years in the marketplace Broad adoption across many industries

– Technical computing • Electronic Design Automation, Life Sciences, Oil and Gas Exploration, Government,

Media, Financial Services, HPC– Enterprise computing

• Business Analytics, OLTP

Over 3,000 customers with 100K+ systems world-wide Industry proven, minimizing risk

35


Elastic Storage deployment models

36


Cold Storage In the Industry – Facebook, Google and IBM

‘Tape will never be the whole answer to storing data, according to Dr Eleftheriou. But it forms a crucial part of a “storage hierarchy”.’1

1 “Tape Rescues Big Data” The Economist, Blogs, Science and Technology September 2013http://www.economist.com/blogs/babbage/2013/09/information-storage

2 “Facebook Crafts Cold Storage Library From Blu-ray Discs” EnterpriseTech Storage Editionhttp://www.enterprisetech.com/2014/01/28/facebook-crafts-cold-storage-library-blu-ray-discs/

“The setup can cram an amazing 1 PB of data into a single 48U rack (seven feet high)….”2

Facebook Crafts Cold Storage Library From Blu-ray Discs

“….the as-yet-unnamed Blu-ray library would burn 80 percent less energy than the Open Vault cold storage, and would be 50 percent less expensive than that disk-based setup as well.”2

Google reveals TAPE-TASTIC data centre…"We store another copy of the most important data on digital tape.“3

3. Google reveals TAPE-TASTIC data centre in saucy vidSay it loud: We've tape and we're proudThe Register – 28, 01, 2013http://www.theregister.co.uk/2013/01/28/google_oracle/

Tape Rescues Big Data

37


Linear Tape File System Enterprise Edition (LTFS EE)

LTFS EE integrates LTFS LE with Elastic Storage

– Policy based migration

Elastic Storage provides global name space

LTFS EE is installed on one or more Elastic Storage nodes

Workload is distributed over all LTFS EE nodes and tape drives

Elastic Storage Node 1

TSxxxx Tape Library

LTFS EE

Elastic Storage Node 2

LTFS EE

Global name space

Elastic Storage file systems (user data and metadata)Elastic Storage file systems (user data and metadata)

Users and applications

User data


Self Service and Elastic Storage

NFS

•Storage Infrastructure Admin – work with physical devices•Mission: Monitor/manage physical devices; Create virtual resource pool•Tools: LTFS EE CLI

•Cloud Admin - work with virtual resources•Mission - Connect virtual resource pool to Cloud Management;

-Manage virtual resources; -Manage business processes;

•Tools: SCSA (Admin)

•End User – work with virtual resources•Mission: Use virtual resources•Tools: SCSA (User)

SCSA

•Dept Admin – work with virtual resources•Mission: - Manage virtual resources; -Manage business processes;•Tools: SCSA (User)

Storage Cloud

Authentication Server

Mgmt flowData flow

Node 1

ES

LTFS

Node 2

ES

LTFS

Node N

ES

Elastic Storage Architecture with LTFS EENode k

ES

LTFS

http://www.redbooks.ibm.com/Redbooks.nsf/RedbookAbstracts/tips1129.html?Open

39


Storage Cloud Focus Plays




Software Defined Anything (SDx) is transforming the market

41


Five Cloud Storage and Software Defined Storage Conversations ….you can have today

XIV for Cloud Providers and MSPs – – Featuring the Frame ahead offering for

GridScale

IBM Virtual Storage Center (VSC) for Private Storage Clouds -

– Featuring 3 VSC license offerings

Self Service Modular Cloud Storage – – Storwize with Active Cloud Engine (ACE)

and SmartCloud Storage Access (SCSA)

Cold Storage for Private Clouds – – With LTFS EE Rapid Application

Configurations

Software Defined Storage for Unstructured Data

– Code name: Elastic Storage

42


-- Brian Hopkins et al., Forrester Research, “Forrester’s Top 15 Emerging Technologies To Watch: Now To 2018”, February 2013.

Infrastructure matters because business outcomes matter and new outcomes entail new infrastructure demands.

“A wide range of innovations is happening at the infrastructure layer, as continued hardware improvements drive new capabilities for compute, storage, and networking and combine with new innovations in software. These will recombine in numerous ways to create much more capable infrastructure and application platforms. “

43


Resources – Cloud Storage and SDS – IBM Technical Leadership Council

Matt Bedernjak – Storage CTM

Brian Sherman – Distinguished Engineer

Leonard Degollado – Client Technical Specialist

Pete Kisich - Client Technical Specialist

Mike Lovering - Client Technical Specialist

Todd Nordquist - Client Technical Specialist

Jim Rogers - Storage Architect

Chris Vollmar - Client Technical Specialist

Mike Grove - Client Technical Specialist

44

webinar- understanding ibm cloud storage and sds solutions - may 16 2014 v2.ppt

Documents