ims database solution pack overview with autonomics - ims ug nyc-2015

© 2015 IBM Corporation

IMS Database Solution Pack

Overview with Autonomics

Richard Phipps IBM IMS and IMS Tools

[email protected]

NY IMS Regional User Group Meeting

19 May 2015

mailto:[email protected]

© 2015 IBM Corporation 2

IMS Tools – Why ?

The IMS base product includes a number of basic utilities for the

administration and management of IMS and it‟s underlying data bases

• Backup, Recovery, Reorganization, Performance reporting, etc.

• Sufficient for some customers with lesser demands from IMS data

For many IBM customers, the „business critical‟ nature of IMS data

demands more from what IMS utilities provide.

IMS Tools are needed for:

• Better Performance

• Enriched Features and Functions

• Higher Data Availability

• Integrated Capabilities

• Ease of Use

“Doing more with less” … this is the role of IMS Tools


Today - Typical Concerns and Challenges

Increased use of data assets - Bigger database size - Higher demands for continuous data availability

New business requirements - Enabling host data for access outside of the zSeries (host) platform - New governmental regulations

Cost containment - Reduce CPU, I/O, and elapsed execution times - Budgetary Concerns

Support staff concerns - Head count reductions - Retirements - Experience and knowledge level

The “shrinking skill set” - A top concern for all


The “Shrinking Skill Set” – Some observations

Issue: The “z” support community is getting older which creates an impending skills

problem that IT organizations are already beginning to deal with Facts:

80% of IMS DBAs are over the age of 55 Within the next 10 years, this entire group will be moving on to retirement

Most companies have IMS DBAs with 25+ years of experience

Estimates are it takes 5-7 years for someone to become a truly effective DBA

Industry statistics show that on average, new employees will remain on the same job for between 3-5 years

Response:

IT organizations have some “big shoes to fill”

Provide tools that can capture the wealth of experience

Put new tools into place to accelerate the learning curve for next generation IT support people


IMS Tools Vision - Autonomics

Sensors collect resource statistics

Policies evaluate sensor data and identify potential problems

Automation orchestrates the collection and evaluation of sensor data

Modernization presents an interactive modern interface for managing the system

Putting information to work

Sensors Policies

Automation Modernization

Not a vision, but a reality


IMS Automated Database Solutions

Checks data base status on a regular basis

User specified thresholds for key indicators

Performs reorganization only when necessary

Performs auxiliary functions: IC, PC, IB, etc.

Keeps data bases performing optimally

Saves human resources

Saves computer resources

Cost efficient solution for both predictable and unpredictable data base growth and

activity


Sensors: Collecting the Basic Information You Need

Statistical point-in-time sensor data on your FF/FP Databases

Stored in IMS Tools Knowledge Base repository

Historically maintained per user specifications

Over 60 separate data elements related to space usage, optimization, and

fragmentation

• data set extents, DASD volume usage, data set free space, roots distribution, RAP usage, CI/CA

splits, and IMS free space, etc

Two methods of collection:

Standalone database sensor utilities for full-function and Fast Path databases

Integrated with existing IMS Tools

Integrated Tools support

High Performance Image Copy, High Performance Pointer Checker

Fast Path Analyzer, Fast Path Online Pointer Checker

Sensors


List of Full Function Sensor Data collected

Database Record Statistics (per database or HALDB partition) Nbr. of DB records Avg. DB record length

Randomizer Statistics (per HDAM or PHDAM partition) Nbr. of total RAPs Nbr. of unused RAPs % of number of unused RAPs Nbr. of synonyms

% of number of synonyms Nbr. of root not on home block % of root not on home block % of segment data in overflow

Nbr. of roots in overflow % of number of roots in overflow Bytes of segments in RAA

Volume/Extents Statistics (per data set) Allocation type (CYL, TRK, …) Primary allocation amount Secondary allocation amount SMS-managed or not

Max. nbr of extents for the d.s. Max. nbr. of extents for the volume Nbr. of extents allocated Nbr. of volumes used

Nbr. of unused volumes Nbr. of unused assigned volumes Nbr. of unused candidate volumes

Nbr. of available remaining extents determined by the max. nbr. of data set extents and the max. nbr. of extents available on volumes assigned to the data set

Data Set Space Usage Statistics (per data set) Block/CI size Nbr. of blocks/CIs used Max. size of the data set % of data set size against the max.

High-Allocated-RBA High-Used-RBA

IMS Space Utilization Statistics (per data set) Total bytes of segment data Total bytes of free spaces Total bytes of slack bytes % of free spaces

% of segment data % of unused bytes in the data set Total nbr. of segments Total nbr. of VL segments

Total nbr. of VL-split segments % of nbr. of VL-split segments Total nbr. of slack bytes Avg. nbr. of slack bytes per block

Total nbr. of FSEs Avg. nbr. of FSEs per block Nbr. of FSEs valid for shortest segments Nbr. of FSEs valid for longest segments

Avg. nbr. of non-reusable FSEs Total nbr. of pointers Total nbr. of ptrs pointing external block % of nbr. Of ptrs pointing ext. block

HISAM/SHISAM Statistics (for HISAM) Logical record length Total nbr. of CI splits % of nbr. of CI splits Total nbr. of CA splits

% of nbr. of CA splits Total nbr. of HISAM delete bytes % of nbr. of HISAM delete bytes

Sensors


Policies: Using Sensor Data to Make Decisions

Policy definitions are used to evaluate specific database states

Threshold values are compared against sensor data for a given database or group of databases

When thresholds are met or exceeded, exceptions occur

Works “out of the box”

Ships with predefined policies and threshold values

Full ISPF interface provided for policy management

Customizable to fit your shop

You can define your own sets of threshold values

Customize the messages sent when exceptions do occur

Specify who receives which messages and how • WTO, e-mail, or text

Policies


Automation

IBM Tools Autonomics Director (Passive)

Automates collection and analysis of Sensor Data

Recommends when databases should be reorganized

• With email or text notifications

Provides a scheduling feature that allows you to control how frequently sensor data is

collected and how frequently policies are evaluated

Flexible scheduling around pre-defined PEAK times

IBM Tools Autonomic Director (Active)

Actively initiate recommended actions on user-defined database groups • Discovery feature for identifying related database groups

• Ability to manage and coordinate reorganization of multiple IMS database groups as if

reorganizing a single database

• Flexible scheduling only in pre-defined Maintenance windows

Automation


Provide a single holistic, easy-to-use interface

to manage IMS/DB2 systems and databases

Consolidate information from various tools to paint a more complete picture of IMS/DB2

systems and databases

Leverage the latest web technologies for a richer user experience

Access from anywhere via the Internet using standard web browsers

Prepare the next generation of DBMS DBAs and System Programmers

through an integrated, context sensitive help system

Modernization

IBM Management Console for IMS & DB2 Databases for z/OS


Object Health

and Autonomics Enterprise-wide

Navigation

Integrated Help

Graphical Visualization

of data not possible in ISPF

Provides a single, holistic easy-to-use interface to manage IMS and DB2

Zero-install web-based interface

Consolidate information from IMS, DB2 and tools from across the entire enterprise

Reduced time for problem identification and resolution through tight integration with IMS and DB2 Autonomics

Dramatically reduced learning curve for new users of IMS and DB2

Now a separate product available no-charge (5655-TAC)

Extensible by growing number of products and solution packs adding additional value

IMS HP

Tools

Autonomics

Director Discovery

Sensors



Drill down from the IMSPLEX level through:

IMSPLEX members

IMS Connect

IMS Subsystems

Transactions

Programs

Routing Code

Databases

Identify, stop, start IMS resources

Explore resource relationships

OM Command Builder



Graphical control of IMS

Autonomics Director

Define monitor lists, setup

peak times and

maintenance windows

Web access to IMS reports

from a variety of Tools

Search and filter by type,

date, completion code, etc

ITKB Reports

Autonomics Control



Extended IMS Autonomics and

Database information

Easily identify and diagnose

symptoms and recommended

actions through IMS Database

Autonomics

Identify trends and make projections

through database sensor charts

Visualize IMS FF/FP Databases

through Library Integrity Utilities

LIU Reports

Extended with the IMS FF/FP Solution Pack



Progressive drill down through

variety of DB2 object dashboards

Autonomics Director for DB2 for

z/OS (in the no-charge Tools

Base) enables:

Charting of DB2 object statistics

through RTS snapshots with

Autonomics control to define

profiles and maintenance

windows

Integrated support for the DB2

Admin Task Scheduler

DB2 Subsystem

Maintenance Windows



Identification and Diagnosis of

symptoms and recommended

actions for REORGs, ICs and

Runstats

Reporting on historical utility

execution including timestamp,

elapse time, system output, etc.

Graphical interfaces to define

Automation Tool Object, Utility,

Exception, and Job Profiles

Extended with the DB2 Utility Solution Pack



Quickly identify and drill down to databases and objects that need your attention from a single starting point

Easily manage by exception and recommendation, taking action before problems occur

Rapidly interpret statistical trends to verify and project

Shorten the learning curve for new administrators

…all from a unified IMS and DB2 interface



Simplified use of IMS data management processes

through tool integration

Designed to reduce technical expertise by providing

an integrated, high performance solution

Provides a complete set of high performance utilities to unload,

load, reorganize, verify, and report on Full Function and HALDB

databases

Functions to assist DBAs in identifying and performing the tasks

of maintaining the health, performance, and recoverability of IMS

databases

IMS Database Solution Pack for z/OS

IMS Tools provided: • IMS Online Reorg Facility

• IMS DB Reorg Expert

• HP Unload

• HP Load

• IMS Index Builder

• HP Prefix Resolution

• HP Image Copy

• HP Pointer Checker

• IMS Library Integrity Utilities

• HALDB Toolkit

Features for the 'smart' reorganization of IMS databases that can help improve system availability

Utilities to validate, compare, map, recover, and report on IMS libraries

Provides tool for fast backup of IMS databases


IMS Database Reorganization Today – How are you doing it now?

Workload for data management increases as your business grows

Increasing volume of repetitive tasks such as free space defrag

Processing efficiency is becoming important

More frequent needs for database logical/physical structure changes

Deep IMS DBA skills are needed for these complex tasks

Plan/Do Actions Diagnose Databases

Pointer

Checker

Reports % of segment splits

Avg nbr of FSEs

% of max. dataset size

Number of extents

Statistics Values

. . .

Diagnosis Result

Exception 1

Exception 4

. . .

Each may be simple,

but the total work volume

is getting larger

. . .

Processing efficiency

is becoming important

How to do it

this time?

When can

it be done?

Deep IMS DBA

skills are needed

in planning actions

Exception 2

Exception 3

Check Key

Stats Items

IMS DBA

Simple Repetitive Actions

(e.g. free space defrag)

Complex Actions

(e.g. Structure Change)


IMS Database Reorganization Expert

Provides a capability to perform a database reorganization while the database

remains in read only status

Provides automated one step batch job process to:

Reorganize IMS databases to shadow datasets

Rebuild the indexes

Perform image copy after the reorg with hash pointer checking

Rename shadow datasets to original production name

Perform automated /DBR and /STA


IMS Online Reorganization Facility

Provides a capability to perform a database reorganization while the database

remains online and available for reads and updates

Designed to address customer needs for 24x7 continuous data availability

Provides automated one step batch job process to:

• Reorganize IMS databases to shadow datasets

• Rebuild the indexes

• Perform image copy after the reorg

• Capture changes to the original datasets

• Apply captured changes to the shadow datasets

• Rename shadow datasets to original production name

• Perform automated /DBR and /STA


Conditional Reorganization – Only reorg what needs it!

Conditional Reorganization Process Steps

DB Administrator submits conditional reorganization job

Reorganization utility collects database statistics and stores them in repository

• Tools Base (Policy Services) evaluates the statistics by predefined policies

Reorganization utility reorganizes the database only when it is necessary

Reorganization utility generates diagnosis report and stores it in repository

DB Solution Pack

Reorganization Utility

• DB Reorg Expert

• Online Reorg Facility

TSO

Submit REORG job

IBM Tools Base

IMS Tools Knowledge Base

Policy

Repository

Store reports

Statistics

Evaluation

Results Diagnosis Report


Specifying a Conditional Reorg Mode of Operation

The following modes are supported in both DB Reorg Expert and ORF:

Conditional Reorganization mode

• Database is reorganized only when necessary.

• DB Reorg Expert: CONDREORG=YES

• ORF: CONDREORG(Y)

Reorganization and Diagnosis mode

• Database is always reorganized and Diagnosis report is generated.

• DB Reorg Expert: CONDREORG=YES,REORGDIAG

• ORF: CONDREORG(Y,REORGDIAG)

Diagnosis-only mode

• Database is not reorganized. DB statistics are collected and evaluated.

• DB Reorg Expert: CONDREORG=YES,DIAGONLY

• ORF: CONDREORG(Y,DIAGONLY)


Conditional Online Reorganization Work Flow

Benefits of Conditional Online Reorganization

A database can be reorganized only when it is really needed

The Diagnosis Report provides an evidence of the necessity of reorganization and

effectiveness of the reorganization


Dia

gn

os

is o

f O

nli

ne

DB

Renamed to

original name Renamed to

original name

Renamed to

original name Renamed to

original name

Log R

ecovery

Pro

cessin

g

Shadow DBD Shadow DBDS

Index Index Shadow Index

IC

IC IC

DB is Offline

NO

TIF

Y.R

EO

RG

DB

DS

Nam

e S

wappin

g

CH

AN

GE

.DB

AU

TH

RE

AD

OF

F

Co

py T

as

k

NO

TIF

Y.I

C/U

IC

Resta

rtin

g D

B

/STO DB

&

/STA DB

/STA DB

Online Online

Original Index Original Index

Original Index

Original DBDS Original DBDS Renamed to

original name

Online Reorganization Workflow Control (used only when reorg needed)

Sta

rt L

og C

aptu

re

Reorganization need is checked

z/OS consoles (and/or TSO users)

Detected or

remaining exceptions

are notified

(based on the policy)

Un

loa

d T

as

k

/STO DB

&

/STA DB

Sw

itch t

o C

hange C

aptu

re

Unload

Data Set

Relo

ad

Ta

sk

Shadow DBDS Shadow DBDS

Ind

ex

Bu

ild

Ta

sk

Ima

ge

Co

py T

as

k

(can

in

clu

de P

oin

ter

Ch

eck)

/DBR DB

Makin

g D

B O

fflin

e

Ap

ply

Ta

sk

Dia

gn

os

is o

f O

nli

ne

DB

Co

nd

itio

na

l R

eo

rg

Po

st

Pro

ce

ss

ing

Online Online


Conditional Online Reorganization – Sample JCL

Three types of policy selection

By database type

Default

By database name

Used to specify a database

specific policy

By policy name

Used to specify a policy for a

group of databases

//ORFJOB EXEC PGM=HRFREORG,REGION=80M,DYNAMNBR=999 //STEPLIB DD DISP=SHR,DSN=ITB.SFOILOAD // DD DISP=SHR,DSN=ITB.SHKTLOAD // DD DISP=SHR,DSN=DBSP.SHRFLOAD // DD DISP=SHR,DSN=DBSP.SHPSLMD0 // DD DISP=SHR,DSN=IMS.SDFSRESL //IMSDALIB DD DISP=SHR,DSN=IMS.MDALIB //IMS DD DISP=SHR,DSN=IMS.DBDLIB //SYSPRINT DD SYSOUT=* //HRFSYSIN DD * REORG DBD(SAMPLEDB) - LOG1DSN(DATADS.LOG.&DBD.&DATA1.&TIME2) - IC1DSN(DATADS.IC1.&DBD.&DATA1.&TIME2) - CONDREORG(Y) - ITKBSERVER(FPQSVR00) - CR.POLICYBY(DBTYPE) /*

CR.POLICYBY(DBTYPE)

CR.POLICYBY(DBDNAME)

CR.POLICYBY(NAME) CR.POLICYNM(MY.POLICY.GROUP1)

By database type

By database name

By policy name

Policy to be selected

SYS.DBDTYPE.type

SYS.DBDNAME.dbdname

MY.POLICY.GROUP1

(type = HDAM, HIDAM, PHDAM,

PHIDAM, HISAM, or SHISAM)

Specifies Conditional Reorg Mode

Specifies XCF Group Name for

ITKB Server to connect

Specifies how a policy is to be selected

,DIAGONL(Y) -

Specifies Diagnosis Mode


Database Diagnosis Report

Processing result of Smart Reorg job is summarized in the Database

Diagnosis Report

The report provides the following information:

Summary of Policy Evaluation

•Policy that was applied to the database

•Reorganization need

•Result of policy evaluation

•Result of policy evaluation after reorganization (if performed)

•Evaluation summary message (in the previous chart)

Sensor data values before and after reorganization

•Differences between the two


Database Diagnosis Report - Example

Basic attributes of the subject database is printed in the title page.

IMS DB Reorg Expert - V4R1 Database Diagnosis Report Page: 1 5655-S35 Date: 12/16/2009 Time: 00.05.14 Summary of Database Definition ------------------------------- Database.......................... BKDB Database Type..................... HIDAM Data Set Organization............. VSAM Number of Data Set Groups......... 1

The following sections follow this title page:

Summary of Policy Evaluation

Sensor data values before and after reorganization

1. Database Record Statistics section

2. Randomizing Statistics section (only for HDAM and PHDAM)

3. Volume and Extent Statistics section

4. Data Set Space Usage Statistics section

5. IMS Space Utilization Statistics section

6. HISAM Statistics section (only for HISAM and SHISAM)

※ Sections 3 through 5 are printed for each data set group


Database Diagnosis Report – Summary of Policy Evaluation IMS DB Reorg Expert - V4R1 Database Diagnosis Report Page: 2 5655-S35 Date: 12/16/2009 Time: 00.05.14 Summary of Policy Evaluation (DBD: BKDB ) ----------------------------------------------------------------- Name of Policy Applied............ SYS.DBDTYPE.HIDAM Policy Locale..................... Global Reorganization Need............... Yes Exceptions before Reorganization -------------------------------------------------------------------------------- The number of available extents for a data set of BKDB is small Class: DATA_SET_EXTENTS_AVAILABILITY Level: CRITICAL Rule: G:IBM.DBDS_EXTENTS.10 Threshold Set: HIGH The size of a data set in BKDB, which still has a certain amount of free space, has increased Class: GROWING_DBDS_WITH_FREE_SPACES Level: CRITICAL -> REORG Rule: G:IBM.DBDS_GROWTH.20 Threshold Set: HIGH The fragmentation of free space in BKDB has increased Class: FRAGMENTED_FREE_SPACES Level: CRITICAL -> REORG Rule: G:IBM.FRAGMENTATION.10 Threshold Set: HIGH A data set of BKDB has many pointers that point to other bocks or CIs Class: EXCESSIVE_SEGMENT_SCATTERING Level: SEVERE Rule: G:IBM.SEGM_SPREAD.10 Threshold Set: MED -------------------------------------------------------------------------------- Exceptions after Reorganization -------------------------------------------------------------------------------- BSN2800I GENERAL STATUS: RESOURCE=BKDB ACTION_NAME=REORG EXECUTION_STATUS=SUCCESSFUL The number of available extents for a data set of BKDB is small Class: DATA_SET_EXTENTS_AVAILABILITY Level: CRITICAL Rule: G:IBM.DBDS_EXTENTS.10 Threshold Set: HIGH -------------------------------------------------------------------------------- Summary of Policy Evaluation and Action: -------------------------------------------------------------------------------- BSN2904I BKDB HAS BEEN REORGANIZED, BUT IT IS STILL IN A CRITICAL STATE --------------------------------------------------------------------------------

Policy name

The locale where the

policy is defined

Result of decision on

reorganization need

Summary message, which shows that a

CRICTICAL exception still remains after

the reorganization.

Exceptions detected before the

reorganization. In this example,

Three CRITICAL exceptions were

detected

Two of them can be removed by

REORG (unload and reload)

One sever exception was detected

Exceptions that still remain after the

reorganization. In this case, one CRITICAL

exception remains.

Result of the reorganization

processing (success/failure)

1

2

3

4

1’


Database Diagnosis Report – Database Record Statistics

IMS DB Reorg Expert - V4R1 Database Diagnosis Report Page: 3 5655-S35 Date: 12/16/2009 Time: 00.05.14 Database Statistics (DBD: BKDB ) -------------------------------------------------------------------------------- ============================== Database Record Statistics ============================== Note: The mark * in column P means that the data element is used in the policy. Data Element Name P Before Reorg After Reorg Difference ------------------------ - ---------------- ---------------- ------------------ DB_NUM_ROOT * 8,560,000 8,560,000 0 DB_AVG_DBREC_LENGTH * 288.06 288.06 0

The number of

database records

Average database

record length


Database Diagnosis Report – Volume/Extent Statistics ... Data Set Statistics (DBD: BKDB , DSG: 01) -------------------------------------------------------------------------------- ============================ Volume/Extent Statistics ============================ Note: The mark * in column P means that the data element is used in the policy. Data set definition and environment information ================================================= Data Element Name P Before Reorg After Reorg Difference ------------------------ - ---------------- ---------------- ------------------ DB_FLAG_SPACE_TYPE C C No change DB_NUM_PRI_SPACE 1,000 1,000 0 DB_NUM_SEC_SPACE 100 100 0 DB_FLAG_SMS * Y Y No change DB_MAX_EXT_DS 251 251 0 DB_MAX_EXT_VOL 123 123 0 Data set usage information ============================ Data Element Name P Before Reorg After Reorg Difference ------------------------ - ---------------- ---------------- ------------------ DB_NUM_EXT 37 32 -5 DB_NUM_VOL 3 3 0 DB_NUM_UNUSED_VOL 0 0 0 DB_NUM_UNUSED_VOL_SER 0 0 0 DB_NUM_UNUSED_VOL_CAND * 0 0 0 Remaining available data set extent estimation ================================================ Data Element Name P Before Reorg After Reorg Difference ------------------------ - ---------------- ---------------- ------------------ DB_AVAIL_EXT_LESS_100 * Y Y No change DB_NUM_AVAIL_EXT * 0 11 +11 DB_AVAIL_EXT_LIMIT VOL_FREE_EXTENTS VOL_FREE_EXTENTS No change Remark: If DB_NUM_UNUSED_VOL_CAND is not zero, more extents than those indicated by DB_NUM_AVAIL_EXT might be available.

Information on extents and used/unused

volumes for the subject data set

Estimation for extent availability

Information on data set allocation

parameters and DFSMS environment


Database Diagnosis Report – Data Set Space Usage Statistics

IMS DB Reorg Expert - V4R1 Database Diagnosis Report Page: 5 5655-S35 Date: 12/16/2009 Time: 00.05.14 Data Set Statistics (DBD: BKDB , DSG: 01) -------------------------------------------------------------------------------- =================================== Data Set Space Usage Statistics =================================== Note: The mark * in column P means that the data element is used in the policy. Data Element Name P Before Reorg After Reorg Difference ------------------------ - ---------------- ---------------- ------------------ DB_BLOCK_SIZE 4,096 4,096 0 DB_NUM_DBDS_BLOCKS 999,179 673,780 -325,399 DB_MAX_DS_SIZE 4G 4G 0G DB_PCT_OF_MAX_DS_SIZE * 96% 65% -31 DB_RBA_HIGH_ALLOC 4,128,768,000 2,801,664,000 -1,327,104,000 DB_RBA_HIGH_USED 4,092,641,280 2,759,806,976 -1,332,834,304

This shows that, within data elements in

this category, only the value for the data

element DB_PCT_OF_MAX_DS_SIZE is

referrred to in one or more rules defined in

the selected policy.

You can see that the size of data set is

reduced by the reorganization.


Database Diagnosis Report – IMS Space Utilization Statistics

IMS DB Reorg Expert - V4R1 Database Diagnosis Report Page: 6 5655-S35 Date: 12/16/2009 Time: 00.05.14 Data Set Statistics (DBD: BKDB , DSG: 01) -------------------------------------------------------------------------------- ==================================== IMS Space Utilization Statistics ==================================== Note: The mark * in column P means that the data element is used in the policy. Data Element Name P Before Reorg After Reorg Difference ------------------------ - ---------------- ---------------- ------------------ DB_BYTES_SEG 2,465,800,000 2,465,800,000 0 DB_BYTES_FREE_SPACE 1,610,723,680 283,136,678 -1,327,587,002 DB_BYTES_UNIDENTIFIED * 110 0 -110 DB_PCT_BYTES_FREE_SPACE * 39% 10% -29 DB_PCT_BYTES_SEG * 60% 89% +29 DB_PCT_UNUSED_BYTES * 1% 1% 0 DB_NUM_SEG 85,620,000 85,620,000 0 DB_NUM_VLSEG 8,560,000 8,560,000 0 DB_NUM_VLSEG_SPLIT * 0 0 0 DB_PCT_NUM_VLSEG_SPLIT * 0% 0% 0 DB_NUM_UNIDENTIFIED * 51 0 -51 DB_AVG_NUM_UNIDENTIFIED * 0 0 0 DB_NUM_FSE * 23,728,638 673,759 -23,054,879 DB_AVG_NUM_FSE * 23.75 1.00 -22.75 DB_NUM_FSE_MIN * 23,558,660 673,759 -22,884,901 DB_NUM_FSE_MAX * 718,662 673,759 -44,903 DB_AVG_NUM_NOREUSE_FSE * .17 0 -.17 DB_NUM_PTR 85,070,408 84,946,241 -124,167 DB_NUM_PTR_DIFF_BLK * 32,424,879 663,666 -31,761,213 DB_PCT_NUM_PTR_DIFF_BLK * 38% 1% -37

Free spaces fragmentation on each

database block has been resolved by

unload and reload of the database.

Segments of a database record fit in a

single database block in almost all

database records.

Allocated space is more efficiently used by

segment data.


IMS High Performance Unload

IMS High Performance Unload (HP Unload) provides high performance database

retrieval service called High Speed Sequential Retrieval (HSSR) Engine

Provides DL/I call analyzer, segment retriever, and buffer handler that are optimized for

GN calls

• Similar to OSAM Sequential Buffering, but HSSR Engine supports VSAM ESDS also

Supports various unload record formats (HD, SH, CS, CP, F1, F2, F3, F6, A1, A2, A3,

or A4)

Supports user-defined unload record formats

Provides an Unload API

• DL/I call and EXEC DLI commands: GU, GHU, GN and GHN call


IMS High Performance Load

Provides a high performance database load service

Processes compressed segment data as input

If unload dataset created by HP Unload utility

Supports various input record formats, including

Formats supported by HP Unload

User-defined unload record formats produced by HP Unload utility

Provides a user exit interface for additional segment processing

Generates statistics reports to aid in tuning the database

Checks for database updates in between the unload and reload to prevent data

corruption


IMS High Performance Load (con‟t)

Can initialize an empty database or partition

HDAM/PHDAM partition: the entire root addressable area is preformatted

HIDAM/PHIDAM partition: a root segment with a key of all X'FF' is inserted

Provides options for whether/how to build ILDSs for HALDB partitions

IMS HP Load provides a Load API

Load API is optimized for ISRT calls

User initial load program can run on HP Load much faster than that on IMS DL/I

• The load program must specify a PCB with PROCOPT=L or LS

Calls that use AIB is supported

EXEC DLI is supported


IMS High Performance Prefix Resolution

Enables you to resolve and update prefixes of IMS databases that are involved in

logical relationships as a single job step

Supports the following three modes of operation:

Stand-Alone Prefix Resolution

• Replacement of IMS Database Prefix Resolution utility (DFSURG10)

Stand-Alone Prefix Update

• Replacement of the IMS Database Prefix Update utility (DFSURGP0),

Concurrent Prefix Update

• Running prefix resolution and prefix update in a single job step

• This function is not available in IMS standard utilities


IMS Index Builder

Build or rebuild index databases

Secondary indexes: multiple indexes in one job step

Primary index of HIDAM or PHIDAM partitions

HALDB Indirect List Data Sets (ILDS)

Streamlines index creation, reorganization, maintenance, and recovery.

Eliminates the need for the traditional unload and reload processes at the time of reorganization of the primary database

Eliminates the need to image copy indexes


IMS High Performance Image Copy

High performance utility to create database back ups easily and quickly

Integration with reorganization utilities included in this solution

IMS Reorganization Expert


Integration with High Performance Pointer Checker to ensure integrity of the

backup taken

Can be executed as a stand alone utility to create backups necessary to support a

recovery strategy

Provides a quick copy method (FlashCopy) with simple interface

Create stacked tape easily without special JCL control

Autonomics capability: switch copy technique based on database state


HPIC Quick Copy Method for Stacked Image Copies

Requirement:

Reduce DB outage time when

creating stacked image copy

Solution: HPIC

Create shadow DB using FlashCopy

DB is brought back online

automatically

Read DB records from shadow

database

Create stacked output

DBDS1

DBDS2

DBDS3

Stop HDAM1 database

Read record in DBDS1

Put record into ICDS


Put record into ICDS


Put record into ICDS STACK

HDAM1

Delete shadow

Create Shadow by FlashCopy

Start HDAM1 database

DBDS1

DBDS2

DBDS3

Shadow

IMS command

IMS command


Autonomics: Internally switches IC process to CIC

Requirement:

Daily backup needed

Batch Image Copy is ideal but

active database may be busy when

job is submitted

Solution: HPIC

Checks DB authorization status,

and switch Image Copy process

Batch Image Copy (IC) to

Concurrent Image Copy (CIC)

automatically

41

DB1 DB2

Program Program Program

Suspend DB by DB QUIESCE

DB1 Busy (Command timeout )

Stop DB2

Check DB authorization status

RECONs

CIC DB1

Request Clean IC

IC DB2


Integration with HP Pointer Checker

Pseudo Online Pointer Check

DB suspended by DB QUIESCE

Shadow DB created by FlashCopy

after DB is quiesced

Checking data in parallel

DBDS1

DBDS2

DBDS3

Suspend DB by DB QUIESCE


Run pointer check


Run pointer check


Run pointer check

HDAM1

Delete shadow

Create Shadow by FlashCopy

Stop DB QUIESCE

DBDS1

DBDS2

DBDS3

Shadow Pointer Evaluation


IMS High Performance Pointer Checker

Ensures IMS Full Function and HALDB databases are are operational, tuned,

repaired, and ready for use.

Consists of six utilities:

HD Pointer Checker (HPPC) – Main Component

• Two modes of execution

• Standard (Full) checking

• Hash checking

• Stand alone execution or integration with High Performance Image Copy

Space Monitor

HD Tuning Aid

DB Historical Data Analyzer

DB Segment Restructure

IMS Database Repair Facility


HALDB Toolkit

Provides functions that can be used for:

Simulating conversion to HALDB

Converting to HALDB

Re-partitioning of a HALDB

Maintaining a HALDB

Analyzer function

DBRC handling

Miscellaneous utilities which assist in HALDB handling tasks

• Indirect list key (ILK) rebuild

• This utility enables you to rebuild all ILKs and their references, and allows you to resolve

conflicting ILKs.


CONVERT command

Backup

HALDB Toolkit

CONVERT

DBDLIB/ACBLIB

RECON FF DB

MDALIB

DBDLIB

RECON

IC datasets

HALDB

backup

//IHCSYSIN DD *

CONVERT –

DBD(HDAM1) –

DSNPREF(HALDB.HDAM1) –

PARTNUM(3) -

:

HSSR Engine

HP Load

HPIC+HPPC Hash


Convert / Migrate to HALDB

CONVERT command can be used to:

•convert to HALDB in a single step batch job

•Integrate with ORF to allow „online‟ conversion

Partitioning can be determined by:

•PARTNUM: specify the number of partitions

•PARTSIZE: specify the size of partitions

•KEYS: specify the high keys

ISPF Interface available to help orderly tracking of large conversion projects

Database Types that can be converted:

•HDAM to PHDAM

•HIDAM to PHIDAM

•HISAM to PHIDAM

•SHISAM to PHIDAM

•PDB/PDF to PHDAM/PHIDAM


Operations of DL/I Batch program for HALDB with HALDB Toolkit

• HALDB Toolkit provides a function to treat a HALDB as if it is not registered.

(DBRC=N) • No SUBSYS record created

• No batch log created

• Normal procedure:

(1) /DBR

(2) Create an image copy#1

(3) Run DL/I Batch program

(4) Create an image copy#2

(5) /STA

DL/I Batch program

(DBRC=Y)

Create IC

IC#1

/DBR DB Create IC

IC#2

/STA DB

• Recovery procedure:

(1) Re-allocate DBDS

(2) Recovery from image copy#1

(3) Re-build ILDS/Primary Index

(4) Re-run DL/I Batch program

from the beginning

RECON info. SUBSYS record

Batch log record IC record IC record

RECON

X


Library Integrity Utilities

A set of utilities designed to validate, compare, map, report, and regenerate a number of

IMS libraries.

IMS libraries supported

• ACB,DBD,PSB, and MFS libraries

• RECON data sets

• IMS Catalog

LIU utilities

• DBD/PSB/ACB Reversal

• DBD/PSB/ACB Mapper

• DBD/PSB/ACB Compare

• Consistency Checker

• Multiple Resource Checker

• Advanced ACBGEN, ACBLIB Analyzer

• MFS Reversal, MFS Compare

• Integrity Checker

• Catalog Manager =


Advanced ACBGEN Utility

Advanced ACBGEN is faster than IMS standard ACBGEN.

Reduces I/O, elapsed time and CPU time significantly.

In our test system, IMS ACBGEN takes 97 seconds, Advanced ACBGEN takes 14

seconds to build 2600 members.

Preview function

Prior to an ACBGEN, a user can know how many members and which members will be

built.

Getting ACBGEN result in more concise format and unique information

A user can know which members are built.

Information of processed members Before and after ACBGEN.


Multiple Resource Checker

DBD Library PSB Library

ACB Library RECON

DBD Library DBD Library

PSB Library PSB Library

ACB Library ACB Library RECON

RECON

Resource Check

Summary Report

Multiple

Resource

Checker

Input: Multiple Resources (DBD, PSB, ACB and RECON)

Up to 10 DBD/PSB/ACB libraries, 10 RECONs

If a user has many resources…..

• Other compare utilities • Process 1-1 for two resources in one job step.

• Need to run many jobs and analyze many reports for many members.

• Multiple Resource Checker compares multiple IMS resource in one job step. • Prints a consolidated summary report.


IMS Catalog – An issue

DBDLIB

PSBLIB

ACBLIB ACBGEN

IMS ACBLIB and IMS Catalog must be synchronized.

However, there is no method to confirm whether ACBLIB, IMS Catalog and

directory are consistent.

Are IMS Catalog and

ACBLIB

synchronized?

? Populate

Are ACBs in the catalog consistent

to ACBLIB and consistent within IMS Catalog?

DBD

PSB

IMS Catalog

Directory

ACB (DMB) ACB (PSB)

Solution: LIU Catalog Manager Utility


Catalog Manager Utility

Ensures DBDs and PSBs in the IMS catalog are maintained correctly.

Time-stamp validation

• Checks time-stamp of each DBD and PSB between ACBLIB and IMS Catalog

• Checks time-stamp of each ACB (DMB and PSB) member in the directory, If IMS

loading ACB from Catalog is activated

Database Version validation

• If database versioning is enabled, the utility also checks whether the database

version numbers are the same for each DBD.

WTO message & IMS Catalog Validation report


Catalog Manager Utility - WTO messages

RC=00 : All of DBDs and PSBs are consistent between ACBLIB and IMS Catalog

RC=04 : Some of DBDs and PSBs are inconsistent between ACBLIB and IMS

Catalog.

If inconsistency is found, run the ACB Generation and Catalog Populate utility

(DFS3UACB) or the IMS Catalog Population utility (DFS3PU00) to correct the

IMS catalog.


IMS Catalog Validation Report

Valid : same time-stamp and DB Version between ACBLIB and IMS Catalog

Invalid : different time stamp or member does not exist in IMS catalog

This report is available

for IMS V12 or V13


Summary

IMS Database Solution Pack is a single, simplified, integrated solution which includes all

the necessary components to effectively and efficiently manage your IMS Full Function

Database Environment


IMS High Availability Large Database Toolkit

IMS Database Reorganization Expert

IMS High Performance Unload

IMS High Performance Load

IMS High Performance Prefix Resolution

IMS Index Builder

IMS High Performance Image Copy

IMS High Performance Pointer Checker

IMS Database Repair Facility (delivered as part of HP Pointer Checker)

IMS Library Integrity Utilities


IMS Tools from IBM

Providing Modernization and Autonomics

IMS Fast Path Solution Pack

Autonomics IMS HP Fast Path Utilities IMS DIB Repair Facility IMS HP Image Copy IMS Library Integrity Utilities

IMS Recovery Solution Pack

IMS HP Image Copy IMS Database Recovery Facility IMS HP Change Accumulation IMS Index Builder IMS DRF Extended Functions

IMS Performance Solution Pack IMS Connect Extensions IMS Performance Analyzer IMS Problem Investigator

IMS Database Solution Pack Autonomics IMS DB Reorganization Expert - Unload, Load, Index Build, Prefix Resolution/Update IMS HP Image Copy IMS HP Pointer Checker IMS Library Integrity Utilities IMS HALDB Toolkit IMS Online Reorg Facility

IMS Sequential Randomizer Generator Data Base Administration

IMS Cloning Tool IMS Database Control Suite

Utility Management

IMS HP Image Copy IMS DEDB Fast Recovery IMS Recovery Expert

Backup and Recovery Batch Terminal Simulator

Program Restart Facility Application Management

Performance Management

IBM Transaction Analysis Workbench IMS Buffer Pool Analyzer IMS Network Compression Facility

System / TM Administration

System IMS Command Control Facility IMS ETO Support IMS HP Sysgen Tools IMS Queue Control Facility IMS Workload Router TM IMS Configuration Manager IMS Sysplex Manager

Regulatory Compliance

Guardium ITAP for IMS IBM Infosphere Guardium Data Encryption for DB2 and IMS Databases

IBM Tools Base for z/OS


Thank You

Richard Phipps IBM IMS and IMS Tools

[email protected]

mailto:[email protected]


Legal Disclaimer

• © IBM Corporation 2014. All Rights Reserved.

• The information contained in this publication is provided for informational purposes only. While efforts were made to verify the completeness and accuracy of the information contained

in this publication, it is provided AS IS without warranty of any kind, express or implied. In addition, this information is based on IBM‟s current product plans and strategy, which are

subject to change by IBM without notice. IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, this publication or any other materials. Nothing

contained in this publication is intended to, nor shall have the effect of, creating any warranties or representations from IBM or its suppliers or licensors, or altering the terms and

conditions of the applicable license agreement governing the use of IBM software.

• References in this presentation to IBM products, programs, or services do not imply that they will be available in all countries in which IBM operates. Product release dates and/or

capabilities referenced in this presentation may change at any time at IBM‟s sole discretion based on market opportunities or other factors, and are not intended to be a commitment to

future product or feature availability in any way. Nothing contained in these materials is intended to, nor shall have the effect of, stating or implying that any activities undertaken by

you will result in any specific sales, revenue growth or other results.

• If the text contains performance statistics or references to benchmarks, insert the following language; otherwise delete:

Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will

experience will vary depending upon many factors, including considerations such as the amount of multiprogramming in the user's job stream, the I/O configuration, the storage

configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve results similar to those stated here.

• If the text includes any customer examples, please confirm we have prior written approval from such customer and insert the following language; otherwise delete:

All customer examples described are presented as illustrations of how those customers have used IBM products and the results they may have achieved. Actual environmental costs

and performance characteristics may vary by customer.

• Please review text for proper trademark attribution of IBM products. At first use, each product name must be the full name and include appropriate trademark symbols (e.g., IBM

Lotus® Sametime® Unyte™). Subsequent references can drop “IBM” but should include the proper branding (e.g., Lotus Sametime Gateway, or WebSphere Application Server).

Please refer to http://www.ibm.com/legal/copytrade.shtml for guidance on which trademarks require the ® or ™ symbol. Do not use abbreviations for IBM product names in your

presentation. All product names must be used as adjectives rather than nouns. Please list all of the trademarks that you use in your presentation as follows; delete any not included in

your presentation. IBM, the IBM logo, Lotus, Lotus Notes, Notes, Domino, Quickr, Sametime, WebSphere, UC2, PartnerWorld and Lotusphere are trademarks of International

Business Machines Corporation in the United States, other countries, or both. Unyte is a trademark of WebDialogs, Inc., in the United States, other countries, or both.

• If you reference Adobe® in the text, please mark the first use and include the following; otherwise delete:

Adobe, the Adobe logo, PostScript, and the PostScript logo are either registered trademarks or trademarks of Adobe Systems Incorporated in the United States, and/or other

countries.

• If you reference Java™ in the text, please mark the first use and include the following; otherwise delete:

Java and all Java-based trademarks are trademarks of Sun Microsystems, Inc. in the United States, other countries, or both.

• If you reference Microsoft® and/or Windows® in the text, please mark the first use and include the following, as applicable; otherwise delete:

Microsoft and Windows are trademarks of Microsoft Corporation in the United States, other countries, or both.

• If you reference Intel® and/or any of the following Intel products in the text, please mark the first use and include those that you use as follows; otherwise delete:

Intel, Intel Centrino, Celeron, Intel Xeon, Intel SpeedStep, Itanium, and Pentium are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States

and other countries.

• If you reference UNIX® in the text, please mark the first use and include the following; otherwise delete:

UNIX is a registered trademark of The Open Group in the United States and other countries.

• If you reference Linux® in your presentation, please mark the first use and include the following; otherwise delete:

Linux is a registered trademark of Linus Torvalds in the United States, other countries, or both. Other company, product, or service names may be trademarks or service marks of

others.

• If the text/graphics include screenshots, no actual IBM employee names may be used (even your own), if your screenshots include fictitious company names (e.g., Renovations, Zeta

Bank, Acme) please update and insert the following; otherwise delete: All references to [insert fictitious company name] refer to a fictitious company and are used for illustration

purposes only.

http://www.ibm.com/legal/copytrade.shtml

ims database solution pack overview with autonomics - ims ug nyc-2015

Technology