dueling duplications rman vs delphix

Dueling Duplications

Steve Karam – oraclealchemist.comKyle Hailey – kylehailey.com

1

Physical vs. Virtual Data Cloning

Your Presenters• Kyle Hailey

– Oracle Ace, Oaktable Member

– A principle designer of Oracle EM performance pages

– Twitter: @kylehhailey

– Blog: kylehailey.com

• Steve Karam– Joined Delphix in December

– Oracle Certified Master, ACE, and other acronyms

– Just a little social

• Blog: http://www.oraclealchemist.com/

• Twitter: @OracleAlchemist

• Facebook: OracleAlchemist

Introduction

In this corner…we have RMAN!

• Undisputed(ish) Oracle Database Heavyweight Backup Champion• Raised on the mean streets of Oracle 8• Became really useful in Oracle 9i• Became really really useful in Oracle 10g• Just keeps getting better and better• Used as a tool for:

– Backup and Recovery– Cloning Oracle Databases– Cross Platform Migrations– Taking up lots of disk space

And in this corner, the contender: Delphix!• The Agile powerhouse out of Menlo Park, CA• Light on its feet, and built for speed• Compatible with 9.2.0.8 to 12c • Provisions virtual databases (VDBs) off any Oracle or SQL Server• Ready to disrupt your business practices—but in a good way• Used as a tool for:

– Cloning Oracle and SQL Server fast– Reducing the storage – Removing data movement bottlenecks– Making sure everyone gets a copy of the database– Making people jealous on Twitter

Why We Clone

There are plenty of good reasons to clone a database.

•Creating Dev, QA, Reporting, UAT, and other environments•Migrating data from one server to another•ETL or other warehouse workflow requirements•Data versioning•Data archiving

The problem is…

Developers

QA and UAT

Reports

First copyProduction

• CERN -  European Organization for Nuclear Research

• 145 TB database• 75 TB growth each year• Dozens of developers want copies.

What We’ve Seen

1. Inefficient QA: Higher costs of QA2. QA Delays : Greater re-work of code3. Sharing DB Environments : Bottlenecks4. Using DB Subsets: More bugs in Prod5. Slow Environment Builds: Delays

“if you can't measure it you can’t manage it”

1. Inefficient QA: Long Build times

Build TimeBuild TimeQA Test

96% of QA time was building environment$.04/$1.00 actual testing vs. setup

Build

2. QA Delays: bugs found late require more code re-work

Build QA EnvBuild QA Env QA Build QA EnvBuild QA Env Q

A

Sprint 1 Sprint 2 Sprint 3

Bug Code

X

Delay in Fixing the bug

Cost ToCorrect

Software Engineering Economics – Barry Boehm (1981)

3. Full Copy Shared : Bottlenecks

Frustration Waiting

Old Unrepresentative Data

4. Subsets : cause bugs

Productio

n

4. Subsets : cause bugs

Classic problem is that queries that run fast on subsets hit the wall in production.

Developers are unable to test against all data

The Production ‘Wall’

5. Slow Environment Builds:

Developer Asks for DB

Get Access

Manager approves

DBA Request system

Setup DB

System Admin

Requeststorage

Setup machine

Storage Admin

Allocate storage (take snapshot)

Slow Environment Builds: 3-6 Months to Deliver Data

Why We Can’t Clone

And there are many reasons that we can’t make the clones we need.

•We don’t have the disk space for another copy (or three)•We don’t have enough environments

•The source database is too huge•There’s just not enough time

5. Slow Environment Builds: culture of no

DBA Developer

Never enough environments

bottlenecks

What We’ve Seen

1. Inefficient QA: Higher costs2. QA Delays : Increased re-work3. Sharing DB : Bottlenecks4. Subset DB : Bugs5. Slow Environment Builds: Delays

The problem is…

The problem is that RMAN is still a backup and recovery tool. It’s capable of duplicating databases, it has great features for duplicating databases, but it’s not made to duplicate databases. It’s not made for agility.

In short…

Round 1 - RMAN

RMAN is…

The official description: Oracle Recovery Manager (RMAN) provides a comprehensive foundation for efficiently backing up and recovering the Oracle database.

So how does it do that?

•Backups to disk or tape (or 3rd party media management layers)•Integrated catalog , retention policies and cleanup•Backup-time corruption detection•Full and incremental backup capabilities•Backup set and Copy backup types

Backup and Recovery in RMAN

Backup and Recovery in RMAN is dead simple, particularly if the environment is configured ahead of time with proper paths and requirements.

Backup the whole database:

backup database;

Restore and recover the whole database:

restore database;

recover database;

So what does a backup look like?

RMAN backups are, of course, collections of files. What those collections look like is determined by the type of backup you’ve performed.

• Backup as Backupset• Backs up non-empty blocks into one or

more backupset files• Multiple datafiles per backupset file• Datafiles are reconstructed at restore

time

• Backup as Copy• Backs up complete files to complete

copies, byte-for-byte, old school• Fast recovery• Takes up more space• Useful for incrementally updated

backups, recover into full backup

Surely it does more than that!No doubt! RMAN is capable of many things, all centered around its ability as a backup and recovery tool.

•Cross platform conversions – Convert a database from Windows to Linux, Linux to Windows, or even to

platforms with a different endian format like SPARC Solaris

•Incrementally updated backups – A single full backup copy that is recovered into over time by way of

incremental backups

•Proactive health checks– Database validation to spot signs of corruption

•Compression– Compression during backup using BZIP2 or ZLIB

•Cleanup policies – Using backup retention policies, it is easy to report on and delete unnecessary

backup files

And 12c action

• Table and Pluggable Database Recovery – Recover at the table level instead of the tablespace or datafile level, or recover

pluggable databases with the Multitenant Database option

• Simplified Cross Platform backup and recovery – Now you can use BACKUP TO PLATFORM and RESTORE FROM PLATFORM

instead of doing separate conversions. Solaris: “Backup to Linux”

• Cross Platform incremental backups – Make it possible to do cross platform conversions with minimal downtime

• Multi-Section Image Copy and Incremental Backups – Divide individual data file backups into smaller sections to backup in parallel

But can it clone?

Yes Virginia, RMAN can clone. Since RMAN’s primary functionality is to backup databases and then restore them, it makes sense that you could backup from one server and restore on another.

Even better, RMAN has allowed duplicating with a relatively simple DUPLICATE command since Oracle 8i. This feature has of course grown better over time. As of Oracle 12c you can:•Duplicate from Backup ( only option in pre-12c)•Duplicate from an active database, no backup required•Use compression and multi-section backups for DUPLICATE FROM ACTIVE DATABASE commands

64% of DBAs use RMAN to clone

So what’s the problem?

• Backups take space. – They can be compressed but slow, BZIP2, built in– ZLIB fast but have to pay advanced compression package

• Clones take space– The cloned databases take up lots of space as well. They will always at least

double your space usage.

• Takes time– Even if space isn’t a concern, the process takes itme– While several features have sped up cloning with RMAN, you still need to

physically copy a database from one system to another.

• Start RMAN backup

Round 2 - Delphix

Clone 1 Clone 3Clone 2

99% of blocks are identical

Clone 1 Clone 2 Clone 3

Thin Clone

Where we Are

Production Development QA UAT

Instance Instance Instance Instance

Database

File systemFile system

Database

File systemFile system

Database

File system

Database

File systemFile systemFile systemFile system

Database

Production

Database

Development

Database

QA

Database

UAT

Snapshots

Instance Instance Instance Instance

Want be here

Database Virtualization

Three Physical Copies Three Virtual Copies

Data Virtualization Appliance

Install Delphix on x86 hardware

Intel hardware

Allocate Any Storage to Delphix

Allocate storage of any type

One time backup of source database

Database

Production

Instance

File system

File system

Beta or in Development

Production

DxFS (Delphix) Compress Data

Database

Production

Data is compressed typically 1/3 size

File system

InstanceInstance

• Start Delphix linking

Incremental forever change collection

Database

File system

Changes

• Collected incrementally forever• Old data purged

File system Time Window

Production

InstanceInstance

Source Full Copy Source backup from SCN 1

Snapshot 1Snapshot 2

Snapshot 1Snapshot 2

Backup from SCN

Snapshot 1Snapshot 2

Snapshot 3

DropSnapshot 1

Snapshot 2Snapshot 3

Cloning

Database

Production

Instance

File systemFile system Time

Window

Database

Instance

InstInstance

Typical Architecture

Production Development QA UAT

Instance Instance Instance Instance

Database

File systemFile system

Database

File systemFile system

Database

File system

Database

File systemFile systemFile systemFile system

With Delphix

Database

Production

Database

Development

Database

QA

Database

UAT

Instance Instance Instance Instance

File system

Three Core Parts

Production

File System Instance

DevelopmentStorage

21 3

Source Sync’ingOriginal copyPull in changesPurging > time window

StorageSnapshotCompressShare CacheStorage Agnostic

Self ServiceMount, recover, renameRoles & Security Rollback & Refresh Branch & Tag

Instance

Overview of Database Virtualization

1. Fast, Fresh, Full2. Free3. Branching4. Federated5. Self Serve

Fast, Fresh, Full

Instance

Time Window

Instance

Development VDB

Source

Free

Instance

Instance

Instance

Instance

Source

Branching

Instance Instance

Instance

Source Dev

QA branched from Dev

Federated

Instance

Instance

Instance

Instance

Source1

Source2

Source1

Source2

VDB Source 2

VDB Source 1

Battle Royale

Prepararation

• Source and Target– ORACLE_HOME /home/oracle/oracle1123/product– Backup directory /home/oracle/rman

• Target Preparation– directories

• mkdir -p /home/oracle/oracle1123/fast_recovery_area/dup• mkdir -p /home/oracle/oracle1123/admin/dup • mkdir /home/oracle/oradata/dup• mkdir -p /home/oracle/oradata/O1123/changetracking/

– init.ora /home/oracle/oracle1123/product/dbs/initdup.ora• added

– *.db_file_name_convert='o1123','dup’– *.log_file_name_convert='o1123','dup’

– Create a TNSNAMES entry for 'source’

RMAN

SourceSource

1. RMAN backup set

TargetTarget

/home/oracle/rman /home/oracle/rman

2. scp 3. RMAN duplicate

/home/oracle/oradata /home/oracle/oradata

Delphix

SourceSource

/home/oracle/oradata

DelphixDelphix TargetTarget

/home/oracle/oradata

Initial Link Create VBD

Delphix

SourceSource

/home/oracle/oradata

DelphixDelphix TargetTarget

/home/oracle/oradata

Initial Link

Create VBD

Target 2Target 2

/home/oracle/oradata

Delphix

SourceSource

/home/oracle/oradata

DelphixDelphix TargetTarget

/home/oracle/oradata

Initial Link

Create VBD

Target 2Target 2

/home/oracle/oradata

/home/oracle/oradata