ims batch message processing - ims ug singapore 8-2013

© 2013 IBM Corporation

IMS Batch Message Processing


Agenda

BMP Overview

Typical daily processing workload

Recent IMS function to help BMPs

Converting a batch program to a BMP

BMP Programming facilities


Batch Message Program (BMP)

BMP is not started by the IMS control region, but by submitting a batch job.

The batch job then connects to an IMS control region

Many BMPs can run concurrently – same or different programs

BMPs do not normally process online transactions, but are designed for larger bulk processing of data.

ONLINE LOG DATA

SET RECON CORPORATE

DATABASE

z/OS

APPL PGM

CICS Transaction Processing

IMSCTL Control Region

TELECOMMU- NICATIONS

MESSAGE SCHEDULING

LOGGING/ RESTART

DBRC Recovery Control

LOG CTL

DLISAS DL/I

Subordinate Address-space

DL/I ACTION MODULES

DB/DC SUBSYSTEM

APPL PGM

BMP Batch

Message Processing


Typical Banking Batch (BMP) Processing over 24 hours

Overnight BMPs

Online Transactions in CICS or IMS

Start overnight BMP Processing

BMPs during the day

Large Australian Bank BMP workload 4176 steps run over 24 hour period Of these, 2227 (53%) ran between 08:00 and 18:00


Comments from Large Australian Bank

“all the BMPs must checkpoint if they are locking data, and checkpoint frequently enough to not cause any undue impact on the online systems.”

“There has to be enough capacity to ensure the BMP runs and doesn’t get delayed because of CPU/paging..etc while they hold locks.”

“The database pools have to sized correctly”

...etc


Overnight Batch Processing Benchmark

IMS Version 2.2 Overnight Processing Redbook – Strict timeframe for processing Batch (BMP) workloads – For a Major Bank, with their Retail Banking application – Goal: Prove the overnight batch processing could be finished within the timeframe

Conclusions – IMS is able to process the Batch Applications in the Overnight Timeframe.

– Store data into multiple key ranges, across separate IMS databases

• Similar to HALDB function which is available today in IMS – Parallel application processing,

• One stream for each key range • Merge the data results at the end of the parallel processing


IMS V12 Dynamic Full Function Database Buffer Pools

IMS 12 adds dynamic buffer pool support for full function databases – Users can dynamically manage full function buffer pools – Buffer pool definitions can change without taking IMS down – Application activity is internally quiesced

• Allows new buffer pools to be created • Allows existing buffer pools to be changed or deleted

Benefits – Improved buffer pool management

• Provides type-2 commands for better usability • Eliminate system down time for modifications to buffer pool definitions • Improve application performance with improved buffer pool specifications

8.7

© 2013 IBM Corporation 8.8

Overview of Dynamic Full Function Dynamic Buffer Pools

IMS Proclib

DFSVSMxx

POOLID=VCCC 2048,10000 4096,10000

System Initialization

2048,10000 4096,10000

IMS CONTROL REGION

UPDATE POOL TYPE(DBAS) SECTION(VSAM111)

IMS Proclib

DFSDFxxx

<SECTION=VSAM111> POOLID=VCCC

2048,30000 4096,30000

Define OSAM and VSAM Changes in DFSDFxxx

2048,30000 4096,30000

IMS CONTROL REGION

1

2

3

© 2013 IBM Corporation 9 IMS 11 | May 2010 © Copyright IBM Corporation 2010

Fast Path 64 Bit Buffer Manager

Fast Path buffers above the 2GB bar in control region address space – Optional – Multiple subpools with different buffer sizes

• Subpools expand and compress dynamically

Implementation – DFSDFxxx PROCLIB member

• FPBP64=N is the default • Changing FPBP64 value requires a cold start of IMS • User does not specify the number of buffers. FP handles this

DBBF, BSIZ, and DBFX are ignored when FPBP64=Y is specified

<SECTION=FASTPATH> FPBP64=Y


Fast Path 64 Bit Buffer Manager

64 bit buffer pool – Initial allocation of buffers in each subpool is determined by the number of areas using

the CI size – Subpools expand and compress depending on buffer use by applications

• Maximum storage used above the bar is 2GB

ECSA pool – Every subpool has a small number of buffers in ECSA

• Used for: MSDBs Buffer headers (DMHRs) and some control blocks System buffers including those used for SDEP inserts

– Initial allocation of ECSA storage is 64K (for tracking of 2GB above the bar)

• ECSA storage may grow as needed.

10 IMS 11 | May 2010 © Copyright IBM Corporation 2010

© 2013 IBM Corporation © Copyright IBM Corporation 2008

Converting a Batch Program to BMP

Requirements for converting a batch program to a BMP are:

– A BMP must have an I/O PCB

• You can obtain an I/O PCB in batch by specifying the compatibility option in the

Program Specification Block (PSB) for the program.

– BMPs should issue Checkpoint calls more frequently than batch programs

• Batch programs in a data-sharing environment must also issue Checkpoint calls frequently.


Additional Programming Features

"DUMMY" I/O-PCB

System Service Calls – Checkpoint – Restart

GSAM: Generalized Sequential Access Method – Simple, non-hierarchic data set – No update capability – Compatible with non-IMS data sets


I/O PCB

If an application does not have an I/O-PCB, it prevents: – INQY,SETS – CHKP, XRST – System Service Calls

'AD' status code if non I/O-PCB is used for these calls


PROGRAM LOGIC PCB-AREA

ENTRY .. CALL 'XRST' .. CALL 'LOG' .. CALL 'GU' .. CALL 'ISRT' .. CALL 'GN' .. CALL 'CHKP'

I/O-PCB DB-PCB-1 DB-PCB-2 GSAM-PCB

"Dummy" I/O-PCB Input/Output PCB is normally used

only for "terminal" I/O

Some DB calls require the I/O-PCB – CHECKPOINT – RESTART – LOG

Built by CMPAT=Y parameter during PSBGEN

Also used for: – Testing "online" programs – Conversion to "online"


DATA AREAS FUNCTION CODES I/O-AREAS WORK AREA LIST OF USER-DEFINED AREAS PROGRAM LOGIC

ENTRY CALL 'XRST' .. PROCESS .. CALL 'CHKP'

TERMINATION

RESTART LOGIC

TREAT XRST AS NOP

--------------------- SET FLAG FOR

CHKP

CONTINUE NORMAL PROCESSING

RESTART PROCESSING

CHKPT-ID IN WORK AREA

CHKPT-ID IN PARM FIELD

// EXEC .... PARM='... CHKPT-ID DATE/TIME

NO NO

YES YES

Checkpoint/Restart Program Flow


MASTER 1

3

4

5

TRAN A

PGM X

TRAN PGM LTERM ABEND CODE SEG1 OF INPUT

2

STOP!

STOP!

OUTPUT MSG

INPUT MSG

QPOOL MSG/BMP GU IOPCB ISRT DBPCB ISRT IOPCB

ABEND!!

DATA BASE

Dynamic backout! (of all updates since last sync point)

Delete message In process

Cancel all msgs ISRT'd (except express) since last sync point

Dynamic Backout


PARAMETER COUNT FUNCTION CODE ('CHKP') FIRST PCB ADDR (I/O-PCB) I/O AREA LENGTH (NOT USED) 30 BYTE I/O AREA (CONTAINS CHECKPOINT ID) CHECKPOINT AREA LENGTH CHECKPOINT AREA

CALL langTDLI count, function, pcb-name, ioalen, ioarea, arealen, area,... 'CBLTDLI' PLITDLI

Checkpoint Programming Call

The "checkpoint area length, checkpoint area" pair may be specified up to 7 times.


PARAMETER COUNT FUNCTION CODE ('XRST') FIRST PCB (I/O-PCB) I/O AREA LENGTH (NOT USED) 30 BYTE I/O AREA CHECKPOINT AREA LENGTH CHECKPOINT AREA

CALL langTDLI count, function, pcb-name, ioalen, ioarea, arealen, area,... 'CBLTDLI' PLITDLI

Restart Programming Call

The "checkpoint area length, checkpoint area" pair may be specified up to 7 times, and must (of course) be the same as specified on the CHKP call.


Exception Handling

ROLL Calls – The Roll Back (ROLB) call will dynamically back out database changes and return

control to your program – The Roll (ROLL) call will abnormally terminate your program and dynamically back out

database changes. – The Roll Back to SETS (ROLS) call will back out to a processing point set by a prior

SETS or SETU call.

SET Calls – The Set a Backout Point (SETS) call will set an intermediate backout point or cancel all

existing backout points. – The SET Unconditional (SETU) call operates like the SETS call, except that the SETU

call is accepted even if unsupported PCBs exist or an external subsystem is used.


PARAMETER COUNT FUNCTION CODE ('ROLL') I/O PCB

CALL langTDLI count, function, pcb-name 'CBLTDLI' PLITDLI

'ROLL' Call ROLL (Rollback and ABEND) Call

Requests that the current application be ABENDed: – Current application's database updates are backed-out – U0778 ABEND issued

The ROLL call is not a substitute for good programming.


PARAMETER COUNT FUNCTION CODE ('ROLB') I/O PCB

CALL langTDLI count, function, pcb-name 'CBLTDLI' PLITDLI

'ROLB' Call ROLB (Rollback, no ABEND) Call

'ROLB' like the 'ROLL' call, except the application retains control

Current application's database updates are backed-out – Log dataset must be on DASD – BKO=Y parameter must be coded in JCL PARM

Control returns to the application program


CALL langTDLI count, function, pcb-name, ioarea, token 'CBLTDLI' PLITDLI

'SETS' Processing:

LL ZZ DATA TO BE RETURNED

XXXX

SETS Call

Establishes (sets) up to 9 intermediate backout points

I/O-AREA (before call):

Data to be returned on corresponding 'ROLS‘

TOKEN (after call): 4-byte identifier to be used on later 'ROLS‘ – Used by later ROLS call to control "partial" rollback


ROLS Call

Backs out database changes to 'SETS' "token" – Program retains control

I/O-AREA (after call):

– Same format as area supplied on the SETS call

TOKEN (before call): – 4-byte identifier set on prior 'SETS' call

Requires prior SETS call to establish token If DB-PCB is supplied, it causes U3303 ABEND

– Is an appropriate response to 'BA' status code after INIT

'ROLS' Call

LL ZZ RETURNED DATA

XXXX

CALL langTDLI count, function, pcb-n , ioarea, token 'CBLTDLI' PLITDLI

IO-PCB or DBPCB


USING

OR VSAM(ESDS) BSAM

RECFM

• Fixed • Variable • Undefined

• Fixed • Variable

Generalized Sequential Access Method

Non-Hierarchic

Sequential Database


CALL langTDLI parm-count, call-func, pcb-name, ioarea, rsa-addr

PARAMETER COUNT ADDRESS FUNCTION CODE ADDRESS (GU,GN,ISRT,OPEN,CLSE) GSAM PCB ADDRESS

I/O AREA ADDRESS RECORD SEARCH ARGUMENT ADDRESS

GSAM

RSA FORMAT: APPLICATION PGM MOVE RSA CALL 'GU' CALL 'GN' or 'ISRT'

2 FULL WORDS: • BSAM TAPE - RELATIVE

BLOCK ADDRESS • BSAM DASD - ACTUAL

TRACK ADDRESS • VSAM - RELATIVE BYTE

ADDRESS

RSA ADDR REQUIRED

OPTIONAL

RSA ADDR

'CBLTDLI' PLITDLI

GSAM Programming Calls


Summary

BMP Overview

Typical daily processing workload

Recent IMS function to help BMPs

Converting a batch program to a BMP

BMP Programming facilities

ims batch message processing - ims ug singapore 8-2013

Technology