recovery management solution 50 ways to move your data (a.k.a., make a new plan stan) phil grainger...

Recovery Management Solution50 Ways to Move Your Data(a.k.a., Make a New Plan Stan)Phil GraingerDonna Di CarloBMC SoftwareSession Code: 414Wed, May 01, 2013 01:00 PM - 02:00 PM| Platform: DB2 for z/OS

Click to edit Master title style

Agenda

• Data migration overview• Capturing a consistent image• Migrating with RMD Solution V10.1.00• New V11.1.00 Copy Migration feature • Migration scenarios• Summary• Questions

2

Click to edit Master title style3

Why do we move data around?

• Production data is our business• And it is often handy to have that data elsewhere

• From production DB2 to “somewhere else”• Data analytics and decision support• Application testing• System testing• Problem diagnostics


What are our choices?• As with most things in DB2, we have choices to make• We could use Unload and Load

• Costly process and time consuming• We could use DSN1COPY

• Much faster• But no knowledge of DB2 internals• DSN1COPY can complete without errors, but the data may still be inaccessible• There are more and more reasons NOT to use DSN1COPY:

• Schema changes that are not yet instantiated• RRF vs BRF• Object definition differences

• We could use hardware snapshots• IF the object structures are IDENTICAL between subsystems

• For safety we would need to revert to Unload and Load• Costly process and time consuming


There has to be a better way

• And there is• Restore the data using a consistent image

• This process is very efficient

• And if the data movement needs to be repeated, we could• Refresh the migrated data• Or even propagate changes via log


Consistent image copy

• Image copy can’t have in-flight transactions • All updates must be committed

UR 1

UR 2UR 3

UR 4

ConsistentPoint

InconsistentPoint


Obtaining a consistent image• SHRLEVEL REFERENCE image copy

• Requires a large outage• Instant Snapshot Copy

• Uses intelligent storage to snap a SHRLEVEL REFERENCE copy • Requires a small outage

• OUTCOPY ONLY• Merges existing image copy with log to produce a copy• Must specify a quiet point or use Timestamp Recovery feature• Can use Log Master to find a quiet point

• QUIESCE not necessary

• Online Consistent Copy (OCC)• Creates a SHRLEVEL CHANGE Snapshot• Uses log to back out in-flight transactions• No outage required


Recover Plus INCOPY feature

• Use when the target subsystem is remote• Specify an unregistered consistent copy in the INCOPY clause• Supply OBID translation values

• Values can be defaulted for single-table table spaces and indexes

DB2A DB2B


Recover Plus INCOPY feature (cont.)

• Recover Plus INCOPY feature (cont.)• Difficult to use with OBJECTSET

• An OBJECTSET allows you to process a group of objects in a single statement

• All OBID translation values must be defaulted• All image copy data set names must use the same data set model

• Can’t use cabinet copies• A cabinet copy contains several small images in one data set

• Eliminates several data set allocate/open/close commands• Needs BMCXCOPY information to read the cabinet


Recover Plus INDEP OUTSPACE• Use when shared DASD is available• Specify source object name and quiet point

• The appropriate image copy will be used based on the quiet point• Will merge log with image copy to create a consistent image• Can use Timestamp Recovery and specify a time instead of a quiet point

• Specify VSAM data set name of target• Supply OBID translation values

• Defaults source values for single-table table spaces and indexes

DB2ADB2B

VSAM DSDB2A

Log


Recover Plus INDEP OUTSPACE (cont.)

• Can’t be used with cabinet copies or an OBJECTSET that contains multi-table table spaces


V11.1.00 Copy Migration feature

• Copy Migration feature helps automate migration• The migration file is key

• Supplies image copy names and translation ID values• Used to verify compatibility between source and target attributes


V11.1.00 Copy Migration feature (cont.)• Uses the following products

• Copy Plus • EXPORT creates the migration file

• Recover Plus processes the migration file• IMPORT automates INCOPY• MIGRATE automates INDEP OUTSPACE

• Change Manager • EXPORT• IMPORT• Allows for multi-tasking• Allows BMCXCOPY access

• Instant Snapshot• Cabinet copies• Encrypted copies

• Allows LOB migration


Migration file• Brains of Copy Migration• Contains information from SYSCOPY,

BMCXCOPY and the DB2 catalog• Image copy names• Object ID values• Schema definitions• Versioning information

• Allows for a high level of error checking• Created by the Copy Plus EXPORT

command• Input to the Recover Plus IMPORT and

MIGRATE commands Migration

File

DB2 Catalog

SYSCOPY

BMCXCOPY


Copy Plus EXPORT command• The output descriptor defines the migration file allocation• The object list names the table spaces to be exported

• TABLESPACE• Can be wildcarded

• OBJECTSET• APPLICATION

• Table spaces will be included if a full image copy is found• Can request to include indexes with INDEXES YES

• Must have an available index image copy• Image copy selection

• Most recent copy is default• If INDEXES YES, index copy log point doesn’t have to match table space copy

• Can request a copy at a specific point• If INDEXES YES, index copy log point must match table space copy


Copy Plus EXPORT example

OUTPUT EXPFILE UNIT WORK EXPOUT YES DSNAME RDADMB.PBGMIG.EXPORTEXPORT TABLESPACE DMDSRC*.* EXPORTDDN(EXPFILE) ATLOGPOINT LASTFULLCOPY

INDEXES YES


Recover Plus IMPORT command• Automates the INCOPY technology• The EXPORT runs on the source while the IMPORT runs on the target• The object list names the table spaces to be imported

• TABLESPACE• OBJECTSET• No specification means import everything

• You have a choice on how to handle indexes• REBUILD• RECOVER• AUTO - let Recover Plus decide• NO - don’t process any indexes

• Command available to map object names from source to target• Ability to migrate only data that has changed since last

EXPORT/IMPORT

Click to edit Master title style

IMPORT

18

IMPORT

TS

IX

TS

IX

IC

MIGFILE

DB2CATALOG

BMCXCOPY

SYSCOPY

DB2CATALOG

BMCXCOPY

EXPORT

COPY

SOURCE SUBSYSTEM TARGET SUBSYSTEM


Recover Plus IMPORT example

IMPORT USING RDADMB.PBGMIG.EXPORT NOCOPYPEND INDEXES AUTO SYNC AUTO CHANGE TABLE FROM RDADMB.DMDSRC* TO RDADMB.DMDTGT* CHANGE INDEX FROM RDADMB.DMDSRC* TO RDADMB.DMDTGT*


Recover Plus MIGRATE command• Automates the INDEP OUTSPACE technology• The EXPORT runs on the target while the MIGRATE runs on the

source• The object list names the table spaces to be exported

• TABLESPACE• OBJECTSET• No specification means migrate everything

• Choices to handle indexes are limited• Recovering the indexes is advisable• Cannot rebuild the indexes with MIGRATE

• Must supply a model of the VSAM data set name• Command available to map object names from source to target


MIGRATE

EXPORT

TS

IX

IC

MIGFILE

DB2Catalog

BMCXCOPY

SOURCE SUBSYSTEM TARGET SUBSYSTEM

DB2CATALOG

MIGRATE

Log

SYSCOPY


Recover Plus MIGRATE example

MIGRATE USING RDADMB.PBGMIG.EXPORT1

CHANGE TABLE FROM RDADMB.DMDTGT* TO RDADMB.DMDSRC* CHANGE INDEX FROM RDADMB.DMDTGT* TO RDADMB.DMDSRC* INDEP OUTSPACE

MODEL &VCAT.DSNDBC.&DB.&TS.I0001.&PART


Migration scenarios

• BACKOUT updates• Log Master ongoing process• Copy Migration refresh


BACKOUT updates• IMPORT production to test

• On the source subsystem• Create an OCC for the table spaces• Execute Copy Plus EXPORT to create a migration file

• On the target subsystem• Use Change Manager to

• Create the target objects• Specify IMPORT to migrate the data• Rebuild the indexes

• Execute Copy Plus to create copies of the new table spaces

• Perform testing• BACKOUT AUTO to refresh data

• Forward recovery will be performed if one of the following occurs• REORG• LOAD• REBUILD


Log Master ongoing process

• MIGRATE production to RO subsystem• On the target subsystem

• Use Change Manager to create the target objects• Create an image copy for the table spaces• Execute Copy Plus EXPORT to create a migration file• Stop the table spaces

• On the source subsystem• Find and generate a quiet point with Log Master • Run Recover Plus MIGRATE to move the data to the target VSAM

• On the target subsystem• Start the table spaces• Rebuild the indexes


Log Master ongoing process (cont.)

• Refresh from production• On the source subsystem

• Collect Logical Log (LLOG) with Log Master ongoing process• Use the quiet point from the MIGRATE as a starting point• Specify end point with a date/time, LRSN or LOGMARK

• On the target• Use High Speed Apply Engine to apply LLOG

• Repeat this process on a regular basis• Some customers repeat as regularly as every 15 minutes


Copy Migration refresh

• IMPORT production to test• On the source subsystem

• Create an OCC for the table spaces and indexes• Execute Copy Plus EXPORT to create a migration file

• On the target subsystem• Use Change Manager to

• Create the target objects• Use Recover Plus to

• IMPORT the table space and indexes


Copy Migration refresh (cont.)

• Refresh test data• On the source subsystem

• Create an OCC for the table spaces and indexes• Execute Copy Plus EXPORT to create a migration file

• On the target subsystem• Use Recover Plus to IMPORT with SYNC AUTO

• Only refreshes data that was changed on the source and/or the target• No need to rebuild indexes


Summary

• BMC Software provides several different ways to• Create consistent image copies• Migrate data• Refresh data

• Mix and match to suit your environment• All options provide

• Speed• Efficiency• Ease of use

• If you would like a copy of this presentation• We will be at the booth or email us

Phil GraingerDonna Di CarloBMC [email protected] [email protected] Code: 414Recovery Management Solution50 Ways to Move Your Data(a.k.a., Make a New Plan Stan)

Evaluate my session online: www.idug.org/na2013/eval

http://www.idug.org/na2013/eval

recovery management solution 50 ways to move your data (a.k.a., make a new plan stan) phil grainger...

Documents

production data

image copy data set

master title style recover

cabinet copy

data analytics

data movement

migrated data

log master