ims tools recovery solution pack 2.1 phoenix rug 11 10-16

© 2016 IBM Corporation

IBM IMS Recovery Solution Pack V2.1

Phoenix, AZ IMS User Group Meeting

November 10, 2016

© 2016 IBM Corporation2

Today’s Agenda

� IMS Tools Solutions

� IBM IMS Recovery Solution Pack V2.1

– Definitions

– IMS Backup Resources Supported

– Local IMS Application Recovery

– IMS Disaster Recovery

– IMS Recovery Smarter Tooling using Operational Analytics


Total Recovery Time… “70% is Thinking Time”

� IBM IMS Recovery Solution Pack V2.1 goals1. Create recovery assets with minimal impact to databases2. Recover databases quickly and efficiently

• Process multiple databases in parallel3. Recover databases to “any point in time”

• Whether allocated or unallocated at recovery time4. Validate database recoveries

• Ensure databases and indexes are recovered correctly5. Remove the “70% Think Time” in database recovery

Source : McGladrey and Pullen


IMS Tools Solutions


NEW IMS Tools Solutions: GA: September 2, 2016


NEW IMS Tools Solution: GA: October 21, 2016

IBM IMS Recovery Solution Pack for z/OS V2.1

A backup and recovery solution built on high-performance tools


Today’s Agenda


� IMS Recovery Solution Pack V2.1

– Definitions






IMS Recovery Solution Pack V2.1

� Local Recovery

– Failure is localized at the Primary Production site

• Recovery is for one or more groups of IMS database and/or DB2

tablespaces

• Typically caused by application errors, operational errors, or local

hardware problems

– Resources that can be used in recovery

• Image Copies

• Change Accumulation data sets

• Archived logs

• System Level Backups



� Disaster Recovery– System-wide failure affecting the Primary Production site

• Switchover to the remote site required

– Two disaster recovery strategies• Recovery with IMS and DB2 backup resources

• Image copies

• Change Accumulation or Point-In-Time Change Accumulation

data sets

• Archived logs

• System Level Backups

• Resources often transmitted to remote site using virtual tape

• Storage Mirroring (IBM, EMC, Hitachi)

• Data is replicated from Primary site to Remote site


� Recovery Time Objective (RTO)– Time allowed to recover an application– Critical operations are up and running again– Considerations include:

• For DR, recovery of the subsystem, databases, and the network

� Recovery Point Objective (RPO)– Amount of data lost for local applications or for disaster event– Considerations include:

• Frequency of creating recovery points

• Ability to recover to “any” timestamp with consistency• Ability to coordinate the recovery between IMS and DB2

• Frequency of transferring data to the remote site for DR



� IMS Recovery

–Only committed updates are applied to the database

–Full or timestamp recovery is to a time when the database is not allocated

• Not typically the case for recovering from errors

–Point-in-time recovery (PITR) can be to any time, whether the database is allocated or not

–DBRC RECON shows when the database is allocated

• Useful for full or timestamp recoveries

–“Quiet time log analysis” can find times when the database is allocated but no uncommitted work is in flight

• For recovering multiple related databases, need to find the time(s) when none of them have work in flight



�Creating Recovery Points in IMS• Take Database offline or make read-only

• UPDATE DB STOP(ACCESS) or /DBR

• UPDATE DB STOP(UPDATES) or /DBD

• Very costly

• IMS command to quiesce one or more databases• UPDATE DB START(QUIESCE) OPTION(HOLD | NOHOLD)

• Less costly

� IMS Recovery Solution Pack V2.1 Batch Utility

–Creates Recovery Points for IMS and/or DB2

1. Issues IMS DB QUIESCE START with HOLD for IMS databases

2. Drives Quiesce Utility (DSNUTILB) for DB2 databases

3. Issues IMS DB QUIESCE STOP for IMS databases



� “Quiet Time Log Analysis” – Finds Recovery Points in IMS or DB2 log data sets

• Focuses on Transaction Unit of Recovery (UOR)

• Identifies commit points within each IMS and DB2 transaction• Database may show it is allocated in the DBRC RECON

– Coordinated IMS and DB2 “Quiet Time Log Analysis”

• Uses IMS and DB2 log data sets

• Recovery Point is intersection of IMS and DB2 Quiet Times

14


IMS UOR 2

Coordinated IMS and DB2 Quiet Time

IMS UOR1

DB2 UOR 1

IMS UOR1

DB2 UOR 1

IMS UOR 1

DB2 UOR 1

IMS UOR1

DB2 UOR 1

DB2 UOR 2 DB2 UOR 3

IMS UOR 3

IMS Quiet TimeDB2 Quiet Time


Today’s Agenda



– Definitions






IMS Recovery Solution Pack V2.1 – Image Copies

� Created when database is Deallocated in DBRC

� Registered in DBRC as a BATCH Image Copy

Batch Image Copy

ARCHIVED IMS LOG 1

UPD UPD CMTUPD UPD

Tran 1 on Database 1

UPD UPD UPD UPD


CMT

CMT

UPD

Database 1

(Image Copy)

ALLOC

ALLOC

DEALLOC

DEALLOC


IMS Recovery Solution Pack V2.1 – Image Copies

Concurrent Image Copy

� Created when database is Allocated in DBRC

� Registered in DBRC as a CONCURRENT Image Copy

� Requires the archived logs to resolve uncommitted updates

ARCHIVED IMS LOG 1

UPD UPD CMTUPD UPD


UPD UPD UPD UPD


CMT

CMT

UPD

ALLOC

ALLOC

DEALLOC

DEALLOC

Database 1

(Image Copy)


� Reduces the # of log records needed for recovery

� Supports standard IMS Change Accumulation data sets

� Records from unpaired logs are “spilled” to the Change Accumulation

� Archived logs used with recovery to resolve uncommitted work

IMS Change Accumulation Data Set

Archvied IMS LOG 1

UPD UPD UPD


UPD UPD UPD UPD


CMT

Database 1

(Image Copy)

Archived IMS LOG 2

UPD UPD


UPD UPD UPD


CMT

CMT

UPD

All Databases

(IMS Change Acumulation 1)

ALLOC

ALLOC

IMS Recovery Solution Pack V2.1 – Change Accum


IMS Backup Resources – Change Accumulation

IMS Point-In-Time Change Accumulation (PITCA)

Database 1

(Image Copy)

Archvied IMS LOG 1 Archived IMS LOG 2

UPD1 UPD2 UPD3


UPD1 UPD2 UPD3


CMT UPD4 UPD5


UPD4 UPD5 UPD6


CMT

CMT

UPD6

CMT

UPD1 UPD2 UPD3


UPD1 UPD2 UPD3


CMT UPD4 UPD5


CMT

CMT

IMS PITCA

Point In Time

� Available only with IMS Recovery Solution Pack

� Reduces the # of log records needed for recovery

� No uncommitted updates on PITCA

� Recovery does not use any archived log data sets

ALLOC

ALLOC

MD7

MD7 The creation of the PITCA will perform UOR processing to ensure that only comitted updates are placed on the PITCAMitchell Dooley, 10/31/2016


IMS Recovery Solution Pack V2.1 – Incremental ICs


� Incremental Image Copy is a standard Image Copy

� Creates new IC from prior IC and archived logs

� Database is not accessed by this process

� If database is Deallocated– Registered to DBRC as a BATCH Image Copy

Batch Incremental Image Copy

Database 1

(Image Copy)

Archived IMS LOG 2

UPD UPD


UPD UPD


CMT

Database 1

(Incremental Image Copy)

Archvied IMS LOG 1

UPD UPD UPD


UPD UPD UPD UPD


CMTALLOC

ALLOC

DEALLOC

CMT DEALLOC


IMS Recovery Solution Pack V2.1 – Incremental ICs


� Incremental Image Copy is a standalone Image Copy

� Creates new IC from prior IC and logs

� Database is not accessed by this process

� If database is Allocated– Registered to DBRC as a CONCURRENT Image Copy

Concurrent Incremental Image Copy

Archived IMS LOG 2

UPD UPD


UPD UPD UPD


CMT

CMT

Database 1

(Incremental Image Copy)

UPD

Archvied IMS LOG 1

UPD UPD UPD


UPD UPD UPD UPD


CMTALLOC

ALLOC

Database 1

(Image Copy)

UPD CMT

UPD CMT

DEALLOC

DEALLOC


IMS Backup Resources – System Level Backup (SLB)

IMS System Level Backup (SLB)

IMS System Data Sets

(ACBLIB, DBDLIB, IMS Catalog, etc.)

DBRC RECON, IMS OLDS, WADS, etc.

Associated ICF User Catalogs

IMS Database Data Sets IMS System Level Backup (SLB)


� Contains all IMS data sets at consistent point in time

� Image Copies can be created “upon request”– Pseudo ICs can be registered in DBRC

� When used for Local IMS Application Recovery only– Can include only the IMS Database Data Sets

� Fast Replication (IBM, EMC, Hitachi) is used– Storage processor used instead of Host CPU– Background copy used for physical copy

IMS LOG 1

UPD UPD UPD


UPD UPD UPD UPD


CMTALLOC

ALLOC


IMS Recovery Solution Pack v2.1

� Benefits of the System Level Backup (SLB)– Multiple volumes can be backed up simultaneously and

almost instantaneously

– Can take the place of 1000’s of image copies

– User chooses whether to backup the entire IMS system or

only the database data sets

• Tailored to your needs

– SLB can be offloaded (up to 4 copies) for local and remote

recoveries

– For DR, pre-built recovery jobs can be created with each SLB

and sent to the DR site


Today’s Agenda



– Definitions







� Supports multiple recovery strategies:

• IMS Timestamp Recovery

Recover to Current Time (as long as database is Deallocated)

Recover to any time when the database is Deallocated

• IMS Point-In-Time-Recovery (PITR)

Recover to any timestamp (only committed updates applied)

• Coordinated IMS and DB2 Recovery

Uses Coordinated IMS and DB2 Application Profiles

Recover to a consistent timestamp for IMS and DB2

Can use “Quiet Time Log Analysis” to find timestamp



� Methods to drive recovery– DRF JCL to recover one or more databases

• API will interface with IMS Recovery Expert to allow the use of an SLB when it is the best fit for IC

– IMS Recovery Expert Application Profiles• Profile that can include DBs, IXs or RECON groups to specify the

DBs to be recovered• Can specify to include related indexes or logically related DBs• List of databases expanded dynamically at execution time• IMS Recovery Expert determines the appropriate resources and

best method for performing recovery and dynamically builds and executes the necessary jobs and processes

• Can drive recovery, change accumulation, index rebuild (FF, HALDB and FP), and post-recovery image copy


� IMS Timestamp Recovery to a consistent Recovery Point

►Databases must be Deallocated at the recovery timestamp

● If timestamp not specified, recovered to current time

►Recovery Points are found:

● DBRC RECON using ALLOC records


Database 1

(IC)

All Databases

(IMS SLB)

Database 1

(Incremental IC)

ARCHIVED IMS LOG 1

ALLOC UPD CMT UPD UPD UPD UPD


ALLOC UPD UPD UPD UPD UPD


CMT

IMS Recovery Point

Lost IMS Data

ARCHIVED IMS LOG 2

UPD


UPD UPD


CMT

CMT

UPD UPD


UPD UPD UPD


CMT

CMT

DEALLOC

DEALLOC



� IMS Point-In-Time-Recovery (PITR) to any timestamp

• Available only with IMS Recovery Solution Pack

• Only committed updates are applied

IMS “Any” Timestamp

Lost IMS Data

ARCHIVED IMS LOG 1

UPD CMT UPD UPD UPD UPD


UPD UPD UPD UPD UPD


CMT

ARCHIVED IMS LOG 2

UPD UPD


UPD UPD UPD


CMT

CMT

UPD UPD

UPD UPD UPD

CMT

CMT

UPD

Database 1

(IC)

All Databases

(IMS SLB)

Database 1

(Incremental IC)DEALLOC

DEALLOC

ALLOC

ALLOC



� Coordinated Local IMS and DB2 Application Recovery

►Requires DB2 Recovery Expert

IMS Database 1

(IC)

All Databases

(IMS SLB)

DB2 Database 1

(IC)

All Databases

(DB2 SLB)

“Quiet Time Log Analysis” TimestampLost Data

ARCHIVED IMS LOG 1

ALLOC UPD


ALLOC UPD UPD


CMT

CMT

UPD UPD

UPD UPD

CMT

CMT

UPD

UPD

ARCHIVED DB2 LOG 1

ALLOC UPD CMT UPD UPD

DB2 Tran 4 on Database 2


DB2 Tran 5 on Database 2

CMT

CMT UPD

UPD DEALLOC

DEALLOC

DEALLOC

DEALLOC



DBRC

RECON

Recovery

Repository


Recover

IMS

Databases

Rebuild

IMS

HALDB

ILDS/Index

Create New

IMS

Image Copies

and Pointer

Check

Data Set

Restore

(DBRC

Notifications)

IBM

Fla

shcopy

EM

C T

imeF

inder

IBM

DfS

MS

dss

Recoveries Performed in Parallel for One or More

IMS Databases

Apply

IMS

Change

Accums

Local IMS Application Recovery

IMS HP

Change

Accumulation

Utility

IMS

Database

Recovery

Facility

Rebuild IMS

Indexes,

Fast Path

Secondary

Indexes

IMS Index

Builder

IMS Fast

Path Solution

Pack

IMS

Index

Builder

IMS HP

Image Copy,

IMS HP

Pointer

Checker

Hitachi F

lashcopy

Supports non-IBM products for

migration to IMS Recovery Solution

Pack V2.1

Fast Replication (Data Set Restore)

IMS System

Level Backup

(SLB)

Newer Image

Copies / CA Newer IMS

Archived

Logs IMS

Databases

New Image

Copies

IC or SLB automatically determined


Today’s Agenda



– Definitions







Virtual Tape Replication

IP Link

Remote SitePrimary Site

Backup

Resources

Backup

Resources

� Provides digital replication of IMS and DB2 backup resources

� Virtual Tape gives significant backup replication performance

� Primary “Tape” becomes available at the Remote site

� Secure data transferred achieved through IP Link Encryption




Recover

IMS

Databases

Rebuild

IMS

HALDB

ILDS/Index

Create New

IMS

Image Copies

and Pointer

Check

Data Set

Restore

(DBRC

Notifications)

IBM

Fla

shcopy

EM

C T

imeF

inder

IBM

DfS

MS

dss

Recoveries Performed in Parallel for One or More

IMS Databases

Apply

IMS

Change

Accums

IMS Disaster Recovery

IMS HP

Change

Accumulation

Utility

IMS

Database

Recovery

Facility

IMS

Index

Builder

IMS HP

Image Copy,

IMS HP

Pointer

Checker

Hitachi F

lashcopy

Supports non-IBM products for

migration to IMS Recovery Solution

Pack V2.1

Fast Replication (Data Set Restore)

Newer Image

Copies / CA Newer IMS

Archived

LogsIMS

Databases

New Image

Copies

IC or SLB automatically determined

IMS System

Level Backup

(SLB)

Including

Conditioned

DBRC

RECON

Pre-Built

Recovery

Jobs

Recovery

Repository

Rebuild IMS

Indexes,

Fast Path

Secondary

Indexes

IMS Index

Builder

IMS Fast

Path Solution

Pack



IMS Volume 1

IMS Volume 2

IMS Volume n

DB2 Volume 1

DB2 Volume 2

DB2 Volume n

• IMS and DB2 Disaster Recovery solutions• Coordinated IMS and DB2 Restart Solutions

• One combined IMS and DB2 System Level Backup (SLB) is restored

Combined IMS and DB2 SLB


� Restore the Combined IMS and DB2 SLB (SLB)

►Restart IMS with Emergency Restart (/ERE), DB2 with REDO processing


IMS Active LOG 1

ALLOC UPD

IMS Tran 1

ALLOC UPD UPD

IMS Tran 2

CMT

CMT UPD UPD UPD

UPD UPD CMT

CMT

Combined IMS and DB2 SLB

DB2 Active LOG 1

ALLOC UPD

DB2 Tran 1

ALLOC UPD UPD

DB2 Tran 2

CMT

CMT UPD UPD UPD

UPD UPD CMT

CMT

UPD UPD

UPD UPD



• IMS and DB2 Disaster Recovery solutions• Coordinated IMS and DB2 Recovery Solutions

• Separate IMS and DB2 System Level Backups (SLB) are restored

• Point-In-Time-Recovery (PITR) recovers to most recent consistent time

DB2 SLB

Newer Image

Copies / CA

Newer DB2

Archived

Logs

IMS SLB

Newer Image

Copies / CA

Newer IMS

Archived

LogsIMS Databases

DB2 Databases


� Restore the separate IMS and DB2 System Level Backup (SLBs)

► Uses latest timestamp of earlier of the IMS and DB2 logs for Recovery

► Point-In-Time-Recovery for IMS and DB2

IMS ARCHIVED LOG 1


IMS Tran 1


IMS Tran 2

CMT

IMS ARCHIVED LOG 2

UPD UPD

IMS Tran 1

UPD UPD UPD

IMS Tran 2

CMT

CMT UPD UPD UPD

UPD UPD CMT

CMT

DB2 SLB

DB2 ARCHIVED LOG 1


DB2 Tran 1


DB2 Tran 2

CMT

DB2 ARCHIVED LOG 2

UPD UPD

DB2 Tran 1

UPD UPD UPD

DB2 Tran 2

CMT

CMT UPD

UPD UPD

IMS SLB

Lost DB2 Data

Lost IMS Data



Today’s Agenda



– Definitions






Administration Optimization

Administration Optimization Higher valuetasks

Higher valuetasks

Database Administrator

System Programmer

Database Administrator

System Programmer

Target: Smarter Tools enable productivity

Today

IT Operations Analytics (ITOA) allows data to be retrieved, analyzed

and intelligently viewed. ITOA applies big data analytics to large-scale

operations to extract unique insights and automate actions.


The Value of IMS Smarter Tooling


History Data Store

Collection of Current

System Conditions

Analysis Engine for Intelligent Decisions

Automated Execution of Actions

IBM IMS and DB2 Management Console User Interface (Browser-based)


Leveraging IMS Smarter Tooling


History Data Store

Collection of Current

System Conditions


� IMS Recovery Sensor examines DBRC RECON Data Set

� Image Copy• Image Copy needed flag• Image Copy Recommended flag• # of hours since last Image Copy• Database is not in CA group flag

� Recovery• EEQE counts• Recovery needed flag• # of logs required for a recovery

� Database Backout• Backout needed flag• Backout needed IMS id

� Change Accumulation• # of hours since last CA created• # of logs produced since last CA• Name of CA Group

What “Big Data Analytics” are collected for IMS Recovery?



What “Insights” are gleaned from the IMS Recovery data?

� “Insights” are identified as IMS Exceptions• Conditions exceed user-defined Policies

• Image Copy (DBDS)

• An Image Copy is overdue or missing

• A data set is missing in CA Group

• Recovery (DB/Partition/Area)

• A database needs recovery

• Backout (DBDS)

• A database needs to be backed out

• CA (CAGRP)

• A Change Accum is overdue

Analysis Engine for Intelligent Decisions



What “Actions” are taken for the IMS Recovery data?

� “Actions” can be Passive or Active

• Today we use “Passive Actions” for IMS Recovery• Exceptions are noted

• Viewed on IBM Management Console or ISPF

• Notifications are sent

• In the future, we may use “Active Actions”

• Jobs are created automatically

• Image copies and Change Accums • Recovery jobs will not be run automatically

• Jobs are scheduled to correct problems

• Jobs are not executed if not needed

• Image copies will not run if not needed• Similar to Conditional Reorg jobs that are not run

Automated Execution of Actions


�Recovery Preparedness (HCHECK Validation feature)–Identifies problems that can affect recovery

–Useful for Local IMS Application Recovery and IMS Disaster Recovery

–Driven from ISPF user interface or when DR Recovery Jobs are built

–IMS Backup and Recovery Asset Verification

• Image Copies• No image copy has been taken• An image copy is needed• An image copy is recommended

• HALDB Partitions or Fast Path Areas• Partitions are disabled• Partitions are not initialized• Areas are not available

• Database Backout conditions• Databases with potential backout or authorization problems



System/DBD Properties Exceptions and Actions Utility Reports

Sensor Data Elements


IMS Recovery “Insights”


�IBM IMS Recovery Solution for z/OS, V2.1–PID: 5655-ISR

�IMS Tools Base for z/OS, V1.6–PID: 5655-V93

New IMS Solution Version: Available Oct 21, 2016

ims tools recovery solution pack 2.1 phoenix rug 11 10-16

Technology