Copy Data Management for Analytics

Catalogic ECXManage, Orchestrate and Analyze

Copy Data for Analytics

Agenda

2

Copy Data Analytics Challenges

Copy Data Management Value Proposition

Catalogic Software’s Solution Overview

Product Demonstration – http://data.catalogicsoftware.com/ibm

IBMinfo@catalogicsoftware.com

3

• Four use cases• Each requiring its own solution• All with common, overlapping

storage services• Separate and complex to

deploy• Costly to implement & manage• Results in data silos• Creates software, hardware and

data sprawl

Multiple Solutions – Multiple ‘Common’ Services

BACKUP/DR

Dev/Test - DevOps

TEST/DEV, DEVOPS

ANALYTICS

DATA SERVICES

COPY OPTIMIZE

MOVE STORE

DATA SERVICES

COPY OPTIMIZE

MOVE STORE

DATA SERVICES

COPY OPTIMIZE

MOVE STORE

DATA SERVICES

COPY OPTIMIZE

MOVE STORE

4

Big Data workload(Analytics)

Resilient workload(Mirror)

Non productionworkload(Test/Dev or DevOps)

Compliance workload(Archive)

Resilient workload(Disk Backup)

Recovery workload(Tape Backup)

PrimaryData

Today’s IT Challenge: Drowning in a Deluge of Copy DataResult: Increased cost and complexity and no additional business value

Source: IDC, The Copy Data Problem: An Order of Magnitude Analysis, doc #239875

COPY DATA GROWTH

Stor

age

Gro

wth

Time

Primary Data

~35%YoY

23

6

1

1

?

Copy Data will be a $51B problem by 2018• Consumes as much as 60% of disk capacity• Drives 65% of Storage Software and 85% of the

Storage Hardware spending• Almost all copies sit idle

Copy Data Mgmt Gap

Geometric CopyData Growth

Appropriate Data Growth

Linear Data Growth

50+Copies

231

1

Analytics Process – Before

• Challenge with this process

– ETL data is ‘stale’

– Difficult, slow, long process – each group needs

separate ETL process for data – lots of scripting

– Resultant analysis is not useful, stale, not real-

time

– No competitive advantage

5

Data set made available via data recovery process

(never refreshed)

Marketing team performs analytics on

some ETL data set

Sales team performs analytics on some ETL

data set

M

S

Business Decision Made

F

Finance team performs analytics on some ETL

data set

LEV

ERA

GE

Catalogic’s Copy Data Management Platform

6

Your Infrastructure

VM VM VMVM VM VM

Virtual snapshot and replication capabilities

Catalogic ECX

Copy DataManagement

Platform

Catalog, Manage, Orchestrate and Analyze

Copy Data across your Enterprise and Cloud

Downloadable VM

Agentless

Use Cases

• Automated DR, Enhanced Test/Dev, DevOps Automation, Analytics and Enhanced Recovery

• Orchestration and Automation of application consistent Snapshots and VMware VADP Copy Images

Copy Data Orchestration and Automation

• Full actionable catalog for Snapshot / File analytics /Protection Compliance

• Search, Report and Analyze

Copy Data Catalog and Analytics

IBM(GA ECX 2.2): Storwize, SVC, v9000, VersaStack,

Flash Copy Manager & SoftLayer

AND/OR • Efficient management of Copy Data through full lifecycle (Copy Creation, Copy Use, Compliance and EOL), across heterogeneous storage infrastructure

• Eliminate the need for scripting based upon unique APIs per storage platform

Copy Data Lifecycle Management

• Self-service provisioning as well as Self-Service automated DR, Test/Dev, DevOps and analytics

• App teams gain speed and IT keeps control and gains SLA management

Self-Service for App teams or Admins

7

How it Works

VM VM VM

VM VM VM

VM VM VM

VM VM VM

The Copy Data Reality

8

Current Environment• Highly efficient and resilient• Replicated• Primary and secondary

Snapshots

Separate Environments for:• Test/Dev• Analytics or Data warehouse• Operational Recovery• DR• Archive

…Over budget and …“data starved”

PRIMARY SITE

Storage Array

VMware

VM VM VM

VM VM VM

SECONDARY SITE

Storage Array

VMware

VM VM VM

VM VM VM

VM VM VM

VM VM VM

VM VM VM

VM VM VM

Array Replication

Copy Data Management

9

PRIMARY SITE

Storage APIs

SECONDARY SITE

Storage APIs

VMware API

VM VM VM

VM VM VM

VMware API

VM VM VM

VM VM VM

Array Replication

CatalogicECX

Copy Data Management“Catalog” provides:

• Catalog all snaps, replicas, and VMware objects

• Detailed visibility and context of your current environment

• Copy Lineage and location

• Compliance reporting

Copy Data Management

10

PRIMARY SITE

Storage Array

SECONDARY SITE

Storage Array

VM VM VM

VM1 VM2 VM3

VM VM VM

VM VM VM

Copy Data Managementactionable catalog provides:

• Catalog all snaps, replicas and VMware objects

• Detailed visibility and context of your current environment

• Copy Lineage and location

• VMware / Storage correlation

• Compliance / Audit reporting

VMware VMware

Array Replication

Production Data

Full Copy

Snapshot

CatalogicECX

Copy Data Management

11

PRIMARY SITE

Storage Array

SECONDARY SITE

Storage Array

VM VM VM

VM1 VM2 VM3

VM VM VM

VM VM VM

Automation & orchestration enables powerful use cases:• Enhanced Recovery, Automated

Daily DR, DevOps, Analytics

• Workflows run scheduled, ad hoc or via REST API

• Refresh Data as often as you snapshot

VMware VMware

Array Replication

Production Data

Full Copy

Snapshot

CatalogicECX

Copy Data Lifecycle Management• Simple process to consistently

configure and deploy snapshots/Replication

• Replaces complex custom scripting

• Full visibility, analytics and SLA reporting

VMware VMware API

Copy Data Management

12

PRIMARY SITE

Storage Array

SECONDARY SITE

Storage Array

VM VM VM

VM1 VM2 VM2

Automated DR / DevOps / Analytics:

• Workflows leverage actionable catalog

• Bring up applications in a fenced network

• Bring up applications in order leveraging existing application consistent snapshots

• Test DR every day

• Enable DevOps agility; faster time to market

• Fresh data for analytics; greater accuracy

VM VM VM

VM VM VM VM1’ VM2’ VM3’

Fenced VM Environment

VM VM VM

VM VM VM

VM1’ VM2’ VM3’

FencedVM Environment

Array Replication

Production Data

Full Copy

Snapshot

CatalogicECX

Copy Data Management

13

PRIMARY SITE

Storage Array

SECONDARY SITE

Storage Array

VM VM VM

VM VM VM

Environments can be:

• Cleaned up

• Promoted to production

• Moved to permanent storagefor longer use

VM VM VM

VM VM VMVM1 VM2 VM3 VM1’ VM2’ VM3’

Fenced VM Environment

VM VM VM

VM VM VM

• Cleaned up

VMware VMware

Array Replication

Production Data

Full Copy

Snapshot

CatalogicECX

Copy Data Management

14

PRIMARY SITE

Storage Array

SECONDARY SITE

Storage Array

VM VM VM

VM VM VM

Environments can be:

• Cleaned up

• Promoted to production

• Moved to permanent storagefor longer use

VM VM VM

VM VM VMVM1 VM2 VM3 VM1’ VM2’ VM3’

Fenced VM Environment

VM VM VM

VM VM VM

• Promoted to production

VMware VMware

Array Replication

Production Data

Full Copy

Snapshot

CatalogicECX

VMware VMware

Copy Data Management

15

PRIMARY SITE

Storage array

SECONDARY SITE

Storage Array

VM VM VM

VM VM VM

Environments can be:

• Cleaned up

• Promoted to production

• Moved to permanent storagefor longer use

VM VM VM

VM VM VMVM1 VM2 VM3 VM1’ VM2’ VM3’

Fenced VM Environment

VM VM VM VM

VM VM VM2’VM1’ VM3’

• Moved to permanent storagefor longer use

Array Replication

Production Data

Full Copy

Snapshot

CatalogicECX

16

The Power of Storage Workflow Templates

Current situation

Measured on:Up-time

Containing costAgility for the Business

Measured on:Speed of software

deliveryQuality of code

not up-time

STORAGE TEAM

• Understand the environment and has the storage skills to set the environment up and provide the required up-time

• Setting up environments is a manual and error prone task that takes time

• Doesn’t have the tools to provide self service to App teams

APP TEAM

• Would like more control over how storage for an application is designed.

• Would like to self-serve storage – eliminate the delay created by Storage team

• At best, the App Team is frustrated with the process

• At worst, the App team goes around the processes and does their own exports or dumps to take control and speed up their work

INTERNAL COMMUNICATION GAP

CIO

Business Unit Leader

17

The Power of Storage Workflow Templates

Templates provide “Common Ground”

CIO

Business Unit Leader

• Understand the environment and has the storage skills to set the environment up and provide the required up-time

• Setting up environments is a manual and error prone task that takes time

• Doesn’t have the tools to provide self service to App teams

• Would like more control over how storage for an application is designed.

• Would like to self-serve storage – eliminate the delay created by Storage team

• At best, the App Team is frustrated with the process

• At worst, the App team goes around the processes and does their own exports or dumps to take control and speed up their work

INTERNAL COMMUNICATION GAP

STORAGE TEAM

APP TEAM

Self-services Provisioning, Automated DR, Test/Dev Refresh, DevOps Automation and Analytics

Storage team: Keeps control and gain SLA management

App Team: Gets the speed

they need through

Self-Service

Measured on:Up-time

Containing costAgility for the Business

Measured on:Speed of software

deliveryQuality of code

not up-time

Big Data / Analytics – After

18

Sales team performs analytics on full, fresh

data set

Finance tam performs analytics on full, fresh

data set

S

F

Marketing team performs analytics on full, fresh data set (any location)

M

Data set made available via automation & orchestration with

application consistent snaps / mirrors

BetterBusiness Decision Made

• Volume, Velocity, Veracity– Data sets growing exponentially– Data sets changing quickly – Data sets take on more and different profiles

• All of this requires data is updated as frequently as (real-time) possible (as often as you take a snap) to yield better results

Data Analytics Automate and Orchestrate Data Copies for Business Agility

19

Production Environment

If you’ve got it, Mount it!Leverage your Copy Data!

• Big Data / Analytics– Same ‘copy’ / ‘use’ data workflows as data

recovery

– Full ECX automation / orchestration

– No manual effort / scripting

– Exception reporting

– Real-time data analytics

– Analyze latest production data

– More accurate analytic results

– Analyze in any location, including the

cloud

– Ensure analytic teams have direct access

to the right data, instantly regardless of

location

– Op/Ex and Cap/Ex savings

Leverage Copy Data

Save CapEx and OpEx – Improve ROA• General operational savings

– Eliminate complex and hard to maintain scripts

– Self Service storage provisioning, DR, Test/Dev,

DevOps and Analytics

– Decrease unplanned downtime through compliance reporting and availability audits

– Replaces multiple tools

• Automatic Daily DR– Leverage existing replicated snapshots – better ROA

– No need for dedicated storage

– Eliminate time consuming and rarely successful DR weekends

• Enhanced Test/Dev or DevOps– Leverage existing replicated snapshots – better ROA

– No need for dedicated storage

– Reduce manual tasks for data population

– Dramatically increase developer productivity

– Increase time to market

• Near Real-time data access for analytics– Eliminate data extraction process via scripting

– Increase analytic benefits through Near-Real-time data access

20

Catalogic Software & IBM Partnership with ECX

21

Catalogic ECX

Manage, Orchestrate and Analyze Copy Data across your Enterprise

and Cloud

Downloadable VM

Agentless

Flash Copy Manager• New & exciting uses cases for Flash Copy

Manager volumes• Automated DR, Enhanced Test/Dev,

DevOps Automation and Analytics

Flash System v9000• Differentiate against other Flash Arrays

with Copy Data use cases– Automated DR, Test/Dev, Analytics and

hybrid cloud

• Leverage application consistent snapshots for dramatically more powerful Test/Dev, DevOps automation and analytics use cases

• Leverage all of v9000’s advanced Copy Services (Snapshots, replication, compression and Easy Tier)

Storwize and SVC• Copy Data use cases for all Storwize and

SVC environments• Automated DR, Enhanced Test/Dev,

DevOps Automation, Analytics and hybrid cloud

• Additionally provides entry point into any VMware environment

• Leverage all of Storwize’s advanced Copy Services (Snapshots, replication, compression and Easy Tier)

VersaStack• Copy Data use cases for VersaStack

environments• Automated DR, Enhanced Test/Dev,

DevOps Automation, Analytics and hybrid cloud

• Additionally provides entry point into any VMware environment

• Leverage all of Storwize’s advanced Copy Services (Snapshots, replication, compression and Easy Tier)

Product Demonstration

Catalogic Software

http://catalogicsoftware.com/ibm/landingpage_demo

23

Software-Defined Copy Data Management

Catalogic ECX

Reduced cost and complexityand increased business agility

Result

Available for the IBM Storwize family, SVC, V9000 and Flash Copy Manager

in Catalogic ECX 2.2

For more information: Data.CatalogicSoftware.com/IBM or email us at IBMinfo@catalogicsoftware.com

24

Thank You