defy murphy's law with db2 backup and restore - act … murphy's law with db2 backup...
TRANSCRIPT
© Copyright 3/4/2011 BMC Software, Inc 2
Murphy's Law ("If anything can go wrong, it will") was born at Edwards Air Force Base in 1949 at North Base.
It was named after Capt. Edward A. Murphy, an engineer working on Air Force Project MX981, (a project) designed to see how much sudden deceleration a person can stand in a crash.
One day, after finding that a transducer was wired wrong, he cursed the technician responsible and said, "If there is any way to do it wrong, he'll find it.“
"If there's more than one way to do a job, and one of those ways will result in disaster, then somebody will do it that way.“
© Copyright 3/4/2011 BMC Software, Inc 3
Agenda
Making sure everything is copied and well taken care of - Listdef, template switching, scope(pending)
Making sure the copied data is usable - Checkpage; Copytocopy; Objects in error; avoiding specific Image Copy
Keep the DB2 Catalog Manageable- MODIFY RECOVERY
Is everything OK for and during recovery - Report Recovery Display progress of Recover in LOGAPPLY phase
Is point in time recovery again a feasible option - RECOVER PIT with Consistency
Don´t challenge Murphy - Some things to be prepared (watch out) for - Tables that contain identity columns- Not logged objects- Recover Clone(d) objects
© Copyright 3/4/2011 BMC Software, Inc 4
COPY LISTDEF
LISTDEF will automatically pull in new objects if wildcard chars are used- Related sets of objects
Well defined data set names for templates will automatically handle the creation of output image copy data sets- Ensure unique data set names with substitution variables
Templates will automatically define the GDG base for new objects
Let Utility calculate the dasd space requirements for the template datasets
TEMPLATE LP
UNIT SYSALLDA
MODELDCB model GDGLIMIT(60)
DSN &DB..&SN..&IC..&LR..&PB..P&PA..(+1)
© Copyright 3/4/2011 BMC Software, Inc 5
LISTDEF – Considerations
Exclude a partition - the only way to exclude a partition is if it was first included explicitly as a partition
– YES :LISTDEF LST1 INCLUDE TABLESPACE DB.TS1 PARTLEVEL
EXCLUDE TABLESPACE DB.TS1 PARTLEVEL(7)
– NO : LISTDEF LST2 INCLUDE TABLESPACE DB.TS1
EXCLUDE TABLESPACE DB.TS1 PARTLEVEL(7)
Limitation : not permissible to create all-inclusive lists- DATABASE * - TABLESPACE *.*.
The RI parameter can be included on the INCLUDE - RI parameter cannot be used in conjunction with PARTLEVEL
© Copyright 3/4/2011 BMC Software, Inc 6
LISTDEF – more considerations
If CLONE Keyword (or Option) exists - Utility will operate against BASE by default
LISTDEF - INCLUDE and EXCLUDE for both BASE and CLONE
Utility will only process BASE or CLONE objects - Example
LISTDEF final list 100 Objects 5 of them have clones Final list used with COPY without CLONE keyword
– 100 Base objects will be copied
Final list used with COPY with CLONE keyword specified– 5 Clone objects will be copied
© Copyright 3/4/2011 BMC Software, Inc 7
LISTDEF Warnings….
RI and PARTLEVEL are mutually exclusive
Object restriction still apply!! No selection on status ( see later)
LIST is NOT re-evaluated at restart time- objects could have been dropped or created
© Copyright 3/4/2011 BMC Software, Inc 8
COPY – Template Switching
Template switching - copy to tape if large; to disk if small
Only supported for image and inline copies
- (COPYDDN, RECOVERYDDN);
- LIMIT is ignored if the TEMPLATE on which it is specified is used for other purposes (e.g.WORKDDN, PUNCHDDN)
TEMPLATE BIG DSN &DB..&TS..D&DA..T&TI. UNIT=TAPE
TEMPLATE SMALL DSN &DB..&TS..D&DA..T&TI. UNIT=SYSALLDA LIMIT(20 CYL, BIG)
COPY TABLESPACE SMALL.TS COPYDDN(SMALL)
COPY TABLESPACE LARGE.TS COPYDDN(SMALL)
© Copyright 3/4/2011 BMC Software, Inc 9
COPY – Template Switching
The decision to switch is made based on the ESTIMATED output data set size- which may differ from the actual final size of the output data set.
- for incremental image copies estimated at 10% of the space required for a full image copy.
Remember - constrain and quantify the allocation extents
- PCTPRIME – 100% or required space is alllocated as PRIMARY- MAXPRIME – upperlimit value (not percentage)- NBRSECND – number of secondary extents
© Copyright 3/4/2011 BMC Software, Inc 10
COPY – SCOPE(PENDING)
Objects listed will only be copied if in:- COPY pending status- ICOPY pending status
If DSNUM(ALL) is specified - if any of the partitions are in COPY or ICOPY, all of the partitions will be copied
Not Logged objects- Only SHRLEVEL(REFERENCE) copies are allowed
PBG (partition by growth) table spaces- SHRLEVEL(CHANGE) copy at the table space level will copy partitions that are grown while
the copy is running
© Copyright 3/4/2011 BMC Software, Inc 11
COPY – CHECKPAGE
Since DB2 9 always CHECKPAGE
no longer sets copypending, so…. Check those RCs!
new SYSCOPY record type is written if a broken page is detected to force a full image next since dirty bits may have already been flipped off in the space map pages
- TTYPE column, with a value of “B” (ICTYPE = T)
© Copyright 3/4/2011 BMC Software, Inc 12
COPYTOCOPY Utility
Use a LISTDEF list with templates for the output datasets
Does not support parallelism, but sorts, then processes the objects based on: Input image copies‟ tape stack(s) Output image copies‟ tape stack(s) Size – ascending order
- For tapes, minimizes mounts/demounts/rewinds automatically
Not possibel for Concurrent Copies
Does not support - DSNDB01.SYSUTILX and its indexes - DSNDB01.DBD01 and its indexes - DSNDB06.SYSCOPY and its indexes
© Copyright 3/4/2011 BMC Software, Inc 13
COPYTOCOPY Utility:
Output image
copy stack for RP:
fsn=1 WILDLIFE.MOUSE
fsn=2 WILDLIFE.BEAVER
fsn=3 WILDLIFE.BEAR
fsn=4 WILDLIFE.HIPPO
fsn=5 WILDLIFE.SQUIRREL
fsn=6 WILDLIFE.LYNX
fsn=7 WILDLIFE.TIGER
fsn=8 WILDLIFE.ELEPHANT
Sorted by input image
copy stack 2:
WILDLIFE.SQUIRREL
WILDLIFE.LYNX
WILDLIFE.TIGER
WILDLIFE.ELEPHANT
Sorted by input image
copy stack 1:
WILDLIFE.MOUSE
WILDLIFE.BEAVER
WILDLIFE.BEAR
WILDLIFE.HIPPO
WILDLIFE.BEAR
WILDLIFE.BEAVER
WILDLIFE.ELEPHANT
WILDLIFE.HIPPO
WILDLIFE.LYNX
WILDLIFE. MOUSE
WILDLIFE.SQUIRREL
WILDLIFE.TIGER
LISTDEF ANIMALS
INCLUDE TABLESPACES DATABASE WILDLIFE
COPYTOCOPY LIST ANIMALS
FROMLASTFULLCOPY RECOVERYDDN(RP)
© Copyright 3/4/2011 BMC Software, Inc 14
Avoiding Specific Copy
Recover RESTOREBEFORE x‟rba/lrsn‟
Directs the Recover Utility to use a recovery base prior to the rba or lrsn specified
Mutually exclusive with TOCOPY, TOLASTCOPY, or TOLASTFULLCOPY.
© Copyright 3/4/2011 BMC Software, Inc 15
COPY INDEXES
Options event(itemerror,skip) and copy shrlevel(change)- Keeps objects in the list in UTRW state *only* as each object is being copied instead of for the
duration of the Copy Utility
Specifying OPTIONS EVENT(ITEMERROR,SKIP)
UTRW – utility allows read/write access by applications, but no access for exclusive utilities
© Copyright 3/4/2011 BMC Software, Inc 16
COPY INDEX – Informational COPY-pending status
Setting and clearing the informational COPY-pending status
Index with COPY YES attribute the following utilities can place the index in the informational COPY-pending (ICOPY) status set by:
- REORG INDEX - REORG TABLESPACE LOG YES or NO - LOAD TABLE LOG YES or NO - REBUILD INDEX
© Copyright 3/4/2011 BMC Software, Inc 17
MODIFY RECOVERY
Specify what is to be retained instead of what is to be deleted
Instead of DELETE parameter - Date or AGE
Code a RETAIN parameter
© Copyright 3/4/2011 BMC Software, Inc 18
MODIFY RECOVERY – RETAIN LAST
Deletion works on dates and not on timestamps.- As a result, more entries than requested might be kept.
Example, - Five most recent copies were taken on the same day - RETAIN LAST(2) is specified- Records for all five copies that were taken on that day are retained in SYSCOPY
© Copyright 3/4/2011 BMC Software, Inc 19
MODIFY RECOVERY
Keep SYSCOPY and SYSLGRNX in a manageable state
REORG SYSIBM.SYSCOPY and SYSIBM.SYSLGRNX regularly- If timeout or deadlock is received while deleting syslgrnx records, we commit what has been
done so far - Next run of Modify Recovery will pick up where the previous run left off
Indexes with the COPY YES attribute processed automatically- Index recovery is dependent on recovery information for the underlying table space
Will remove records from SYSIBM.SYSLGRNGX without removing SYSIBM.SYSCOPY
© Copyright 3/4/2011 BMC Software, Inc 20
MODIFY RECOVERY – GDGLIMIT
Full image copies using one GDG and incremental image copies using a separate GDG
The date and time of creating the image copy are (not) visible in data set- only in SYSIBM.SYSCOPY
© Copyright 3/4/2011 BMC Software, Inc 21
Report Recovery Information
Information about the image copy data sets and archive log data set that might be required during the recovery.
Three sections
- Recovery history from the SYSIBM.SYSCOPY catalog table Including information about any system level backup copies that can be used for an object
level recovery. - Log ranges from SYSIBM.SYSLGRNX.
- Volume serial numbers where archive log data sets from the BSDS reside.
-
© Copyright 3/4/2011 BMC Software, Inc 23
Display Progress of Recover LOGAPPLY phase
To help see progress and estimate elapsed time
Progression of Recover logapply shown via –DIS UTIL command- only issued during RECOVER LOGAPPLY phase
DSNU116I csect-name RECOVER LOGAPPLY PHASE DETAILS:
STARTING TIME = timestamp
START RBA = ss START LRSN = rr
END RBA = ee END LRSN = nn
LAST COMMITTED RBA = cc LAST COMMITTED LRSN = ll
ELAPSED TIME = hh:mm:ss
© Copyright 3/4/2011 BMC Software, Inc 24
Recover on a LISTDEF list with PARALLEL
Build LISTDEF lists for sets of related objects
Parallelism in the RESTORE phase…objects are sorted by:- Device type of image copy(s)
DASD Tape – objects sorted to avoid mounts/demounts/rewinds
– File sequence number– Volser(s)
- Size – ascending order- Type of image copy
DB2 image copy Concurrent copy
Do not mix objects to be restored from image copies and objects to be restored
from concurrent copies in the same RECOVER Utility statement
© Copyright 3/4/2011 BMC Software, Inc 25
RECOVER – Parallelism
RESTORE phase : PARALLEL(n) keyword TAPEUNITS(n) keyword
- For image copies on tape - consider full + incremental copies- System limit for threads- Storage in the batch Utility address space- Restore of some objects will be deferred if there is a problem with the image copies
LOGAPPLY phase (fast logapply) :- 98 subtasks to apply the log records
Each subtask handles a table space or partition Depends on the mix of log records in the current log read buffer Log records sorted by dbid, psid, partition number, page number
LISTDEF ANIMALS
INCLUDE TABLESPACES DATABASE WILDLIFE
RECOVER LIST ANIMALS
PARALLEL(4)
© Copyright 3/4/2011 BMC Software, Inc 26
Recover to any point in time (PIT)
With consistency
- Uncommitted changes are backed out
- Significantly reduces (eliminates?) the need to run QUIESCE which can be disruptive to applications
- Does not apply to RECOVER TOCOPY, TOLASTCOPY and TOLASTFULLCOPY using SHRLEVEL CHANGE copy (consistency is not ensured – use RBA/LRSN after COPY point)
- Include all relevant objects in same RECOVER to ensure data consistency from the application point of view
© Copyright 3/4/2011 BMC Software, Inc 27
Recover to any point in time (PIT)
With consistency – TOLOGPOINT or TORBA
LOGAPPLY phase – applies log records to the recovery point specified
Introduced new log phases
– LOGCSR – analyzes URs active at the recovery point– LOGUNDO – rolls back uncommitted changes
© Copyright 3/4/2011 BMC Software, Inc 28
Recovering to a point in time with consistency
RECOVER TOLOGPOINT and TORBA have recover with consistency as its default behavior.
No consistency is ensured if - RECOVER TOCOPY, TOLASTCOPY, and TOLASTFULLCOPY using SHRLEVEL CHANGE
To achieve consistency with a recovery to a SHRLEVEL CHANGE copy- Use a desired SHRLEVEL CHANGE copy as the recovery base - Specify a recovery point that was right after the copy completed via the TORBA or
TOLOGPOINT syntax.- Locate the entry in the SYSCOPY table for the SHRLEVEL CHANGE copy that is being used
and use the RBA or LRSN value in the PIT_RBA column as the recovery point
Use the REPORT RECOVERY utility to see the report of the SYSCOPY records
© Copyright 3/4/2011 BMC Software, Inc 29
Recovering tables that contain identify columns
Column attribute AS IDENTITY
When recovering a table that has an identity column to a point in time, you can create a gap in the identify column values. When you insert a row after this recovery, DB2 produces an identity value for the row as if all previously added rows still existed.
CREATE TABLE EMPL
(EMPNO INTEGER GENERATED ALWAYS AS IDENTITY,
INSERT
INSERT
INSERT
EMPL 1 1 1 1
2 2 2 2
3 3 3 3
4 7
5 8
6 9
IMAGE COPY INSERT
INSERT
INSERT
RESTORE IC INSERT
INSERT
INSERT
© Copyright 3/4/2011 BMC Software, Inc 30
30
Recovery of Objects in Error
-DISPLAY DB(*) SPACE(*) LIMIT(*) RESTRICT- COPY, LPL, WEPR, GRECP.
-DISPLAY DB(*) SPACE(*) LIMIT(*) ADVISORY- ICOPY, AUXW, and AREO*.
-DISPLAY DB(*) SPACE(*) LPL ONLY
© Copyright 3/4/2011 BMC Software, Inc 31
Recoverability of Not Logged objects
Say what ?
Not Logged objects can only be recovered to a point in time that is a “recoverable point”
- Object will be recovered to the most recent “recoverable point”:
Alter to Not Logged Image copy taken with SHRLEVEL(REFERENCE) made while Not Logged
© Copyright 3/4/2011 BMC Software, Inc 32
NOT LOGGED Table Spaces – Catalog Changes
LOG column of SYSIBM.SYSTABLESPACE reflects - non-LOB table spaces as well as LOB table spaces.
• Possible values are:– „Y‟, LOGGED.– „N‟, NOT LOGGED.– „X‟, NOT LOGGED, linked to the base.
SYSLGRNX records are not maintained for NOT LOGGED table spaces.
SYSCOPY records are written for the following events. When a table space is created as LOGGED; ICTYPE='C„ STYPE='L' When a table space is created as NOT LOGGED; ICTYPE='C„ STYPE=„O‟ When a table space is altered to LOGGED; ICTYPE='A„ STYPE='L' When a table space is altered to NOT LOGGED; ICTYPE='A' STYPE='O„
- The LRSN in these SYSCOPY records reflects the point in the log at which the logging attribute was altered.
- A new column, LOGGED, is added to the SYSCOPY Catalog table. Values can be: „Y‟ - indicates the LOGGED attribute. „N‟ - indicates the NOT LOGGED attribute. blank - indicates that the row was inserted prior to V9.
- 32 -
© Copyright 3/4/2011 BMC Software, Inc 33
Recover of CLONED Objects
Base and clones objects must be recovered in separate Recover Utility statements
May recover fully (to current) even when the recovery base is before EXCHANGE DATA
Recovery to point in time prior to EXCHANGE DATA is not allowed
© Copyright 3/4/2011 BMC Software, Inc 34
COPY/RECOVER – To Remember
MODIFY Recovery enhancements- “Retain” keyword added to improve management of copies
LAST(n), LOGLIMIT, GDGLIMIT
RECOVER to any point-in-time with consistency
RECOVER RESTOREBEFORE to use an earlier image copy
Display progress of RECOVER during log apply
COPY CHECKPAGE option always active- “Copy Pending” avoided if broken page encountered
COPY SCOPE PENDING - to copy only objects in “Copy Pending”