how slow data kills agile! - agileconference.org slow data kills agile! jes breslaw head of...

© 2014 Delphix. All Rights Reserved. Private & Confidential. P1.© 2014 Delphix. All Rights Reserved. Private & Confidential.

How slow data kills Agile!

Jes Breslaw

Head of Strategy, EMEA

October 2014

© 2014 Delphix. All Rights Reserved. Private & Confidential. P2.© 2014 Delphix. All Rights Reserved. Private & Confidential. P2.

Governance,

Risk & Compliance

Modernization

& Migration

App Quality,

Time-to-Market

Analytics

& Reporting

Data

Protection

⌃ ⌃ ⌃ ⌃⌃

Platform Modernization

Infrastructure Savings

Datacenter Consolid’n, Migration

Data Loss Prevention, Recovery

Private Cloud Backup, DR

Very Large DB, DW Backup

Improve RPO, RTO, Granularity

Severity 1 Fire Fighting, RecoverySingle Custm’r View: Cross/Upsell

Fresher, 24x7 Data for BI

Refresh, Dev for Very Large DW

Offload Batch ETL, Reports

Historical, Time-Series ReportingCloud, Private Cloud Migration

Enterprise Data Hub, Governance

SOX: Fin’l Close, Reconciliation

Cloud Compliance, Security

Data Privacy, Security, Masking

Regulatory Reporting (DF, CCAR) App Consolidation, Retirement

Enterprise Release Mgmt for ERP

Mobile Application Development

Streamline Integration Testing

Test Quality, Test Data Mgmt

Agile Dev, Continuous Delivery

© 2014 Delphix. All Rights Reserved. Private & Confidential.


Agenda

Introduction

What is slow data & why does it hurt Agile?

How Delphix speeds up data

City Index case study

The DevOps question

Summary


Agile Development Trends

• Improved project timelines

• Greater productivity

• Better alignment of IT and business objectives

88%of practicing organisations

are realisingOver

72% Companies practicing agile for more than 2 yearsOnly

48% are using agile in less than half of all projectsYet


Agile Development Characteristics

Shorter cycles

More projects in parallel

Frequent iteration

Continuous testing

Small, integrated teams


Barriers to Agile Adoption

Setup and teardown effort

• Dozens of copies (per project team – even per developer)

• Days to weeks to setup configure, de-commission

Environment costs

• Greater parallelization

• 10 – 20X feature projects 10 – 20X infrastructure costs

Testing and recovery

• Integrating projects for final QA

• Reproducing test environments to a specific point in time


Server / App Tier Virtualisation Helps….but


How Delphix speeds up data?

Setup and teardown effort

• Full function virtual DBs provisioned in minutes

• 95% reduction in operational overhead

Environment costs

• 90% of storage for copies eliminated

• 95% of backup costs eliminated

Testing and recovery

• On-demand access to latest production data across sources

• Superior RPO and RTO

© 2014 Delphix. All Rights Reserved. Private & Confidential. P9.

Dev QA Dev Integration

1 TB

App Tier – Prod

1 TB 1 TB 1 TB 1 TB

Unique block mapping

Block aware filtering

Efficient compression

90% reduction in EBS storage costs

.5 TB

App Data Files

90%+ Common, Repeated Data


Global “hot” read cache

Global write log

SSD flash cache

On-the-fly compression,

de-compression

5 sources and 45 virtual environments consolidated per Delphix Engine

Built for VDB Performance: 20 VDBs on Average


Data Orchestration: Right Data, Time, Place

FEATURE BENEFIT

SYNC• Efficiently sync heterogeneous sources in near real time

• Deliver right data to right team at right time

RECORD• Synthesize, record all changes into a continuous TimeFlow

• Database continuity, superior Recover Point Objective (RPO)

PLAY• Fast database provisioning, refresh, rollback, data integration

• Reduce time from 10 days to 10 minutes, from 4 teams to 1 team

MOVE• Promote, demote, consolidate, and recover databases

• Quickly move data through application, development lifecycle stages

REPLICATE• Efficient replication to secondary Delphix virtual appliance

• High availability, disaster recovery, backup


on HP-UX

EMC storage

on AIX

NetApp storage

on RHEL

NetApp storage

EXADATA

Appliance

on AIX

EMC storage

SAP SYBASE ASEon RHEL

NetApp storage

Remote

Any

HW

Any

Storage

Cloud Ready

Remote

on Windows

Hitachi storageon RHEL

IBM storage

PostgreSQL Unix/Linux SAP ASE Windows

Auto Sync for 10+ Heterogeneous Sources


Local

Remote

Intra-daily refreshes of virtual copies tied to trading systems

12:36:38

12:36:38

12:36:38

12:36:41

Maintain sync within seconds of

last transactions

Ensure access to fresh data

Enable use of virtual copies for HA

Near Real Time Log Shipping


90% load reduction with automated DB aware, change data only sync

5% Read I/O

5% Process I/O 5% Network Transfer Load

5% Write I/O

1x Read I/O

1x Process I/O

1x Dump I/O

1x Read I/O 1x Transfer I/O1x Write I/O

Sync (Reduced Load)


on HP-UX

EMC storage

Weekly scheduled refreshes, daily rollback for patch and upgrade test environments

Aug 1, 3:36:51

Provision

On-demand access to source data

Superior data management SLA’s

Higher application quality

Current Time

Refresh

Jul 25, 6:42:18

Rollback / Recover

Refresh, Restore, Reset, Bookmark, Branch


6 sources integrated for MDM initiative (50% acceleration)

Aug 5, 7:28:29

Integrated

Provisioning

Decouple reliance on multiple production teams

Synchronize data delivery across sources

Maximize test time and improve data quality

Integrated

Refresh

Aug 6, 3:17:51

Integrated

Rollback

Aug 1, 11:50:33

Data Integration

© 2014 Delphix. All Rights Reserved. Private & Confidential. P17.© 2014 Delphix. All Rights Reserved Private and confidential 17

REFRESH

Peter the developer wants the latest production data

Without Jet Stream:

• Peter files a ticket with IT to refresh his VDB

With Jet Stream:

• Peter logs into the Jet Stream UI and clicks refresh


BRANCH

Peter needs to fix a bug while developing a new feature

Without Jet Stream:

• Request a new VDB to host the data required to fix the bug

• File a ticket to rewind or refresh the existing VDB with the required data

With Jet Stream:

• Peter creates a new branch in his data container

Feature Development

Bug Fix


SHARE

Peter the developer wants Jane in QA to test his changes

Without Jet Stream:

• Peter gives his environment to Jane

• Peter asks IT to take a snapshot of his VDB and provision a new VDB

With Jet Stream:

• Peter creates a bookmark and shares it

• Jane restores her data container from the bookmark


SHARE (continued)

Jane finds a problem with the code and wants Peter to diagnose it

Without Jet Stream:

• Jane gives her environment to Peter

• Jane asks IT to take a snapshot of her VDB and provision a new VDB

With Jet Stream:

• Susie creates a bookmark and shares it

• Peter restores his data container from the bookmark


SHARE (continued)

Peter needs to test a fix against a previous release

Without Jet Stream:

• IT has to map an existing snapshot on the timeflow to the required release

With Jet Stream:

• Peter browses the available bookmarks, searches for the release version

in the bookmark tags

• Peter restores his data container from the bookmark

Data Template

1.0 1.1 1.2 2.0


City Index one of worlds leaders in Spread Betting

Highly competitive market

Already focused on

- Release management automation

- Continuous integration

- Automated Testing

- Fast spin up and tear down

Development group adopted using scrum, TDD and BDD

Scaled Agile using SAFe and AgilePM

“Best Mobile Trading Platform” atMoneyAM Online Finance Awards

Background

“We turned to Agile to increase

ITs value to the business and

make it more effective.“


Bottleneck had moved downstream in testing and the

release management

Release time and other advantages of Scrum at are

nullified

8 projects at one time

Restoring single prod DB taking 4 hours

Development cycles were long

Harder to design out dependencies

Sacrificing innovation

Challenges

“We found ourselves having to

sacrifice new innovative projects

in order to prevent delivery rates

from slipping.”


Delphix Virtual Data Platform for MS SQL Server 2008

3 months POC

Immediate results

Fill implementation in 1 month

Solution


Time to create Db reduced from 4 hours to 3 mins

Removal of shared QA environment

30 Developers now with self-service

Infrastructure freed for more complex data work

Development cycles reduced by 75%

20% more applications and features delivered to the business

Results

“Delphix has unlocked the potential of Agile helping us

increase our output to the business by 20%.”


The DevOps question

?


Summary

Slow Data is the antitheist of Agile development

By removing this constraint, it releases creativity

Dev/test become closer

DevOps becomes reality

how slow data kills agile! - agileconference.org slow data kills agile! jes breslaw head of...

Documents