b05 rexx & db2

42
1 Jim Addison Standard Life B05 3 October 2006 • 10:15 a.m. – 11:15 a.m. Platform: z/OS Rexx & DB2 The challenging world of DBA attracts frequent interruptions for often repetitive tasks. Rexx is a powerful, easy to use language which can be used to reduce or eliminate manual effort. The Rexx/DB2 interface can be used for further exploitation by DBAs - Stored Procedures written in Rexx provide additional advantages. This presentation demonstrates the fundamentals of Rexx, using the DB2 interface and the use of Rexx stored procedures. Examples of the use of Rexx DBA applications at Standard Life are included.

Upload: others

Post on 27-Apr-2022

26 views

Category:

Documents


2 download

TRANSCRIPT

Page 1: B05 Rexx & DB2

1

Jim AddisonStandard Life

B05

3 October 2006 • 10:15 a.m. – 11:15 a.m.

Platform: z/OS

Rexx & DB2

The challenging world of DBA attracts frequent interruptions for often repetitive tasks. Rexx is a powerful, easy to use language which can be used to reduce or eliminate manual effort. The Rexx/DB2 interface can be used for further exploitation by DBAs - Stored Procedures written in Rexx provide additional advantages.

This presentation demonstrates the fundamentals of Rexx, using the DB2 interface and the use of Rexx stored procedures. Examples of the use of Rexx DBA applications at Standard Life are included.

Page 2: B05 Rexx & DB2

2

2

Rexx & DB2• Demonstrate that Rexx is a readily exploitable

language for DB2 applications.• Provide the building blocks for Rexx/DB2 applications.• Provide confidence for exploring Rexx functionality.• Demonstrate the advantages of Rexx Stored

Procedures.• Give an insight into how Rexx/DB2 is proving

invaluable at Standard Life.

Rexx is attractive to novice and expert programmers whether DBAs or applications staff. We will introduce the characteristics which provide Rexx with its popularity. We will focus on the use of Rexx with DB2 which covers utilities, commands, SQL and binding.

We will build up a some example programs, however some of the most important parts of the presentation will be contained in the notes, where more complete examples of Rexx will be provided.

We will see several examples which you can change and build upon.

Rexx stored procedures introduce some constraints to our Rexx programs. We will see why it is worth working with these constraints.

We will see examples of how Standard Life has taken advantage of the Rexx functionality for DB2 processes to maximise return on investment of ISV products and minimise disruption to DBAs.

Page 3: B05 Rexx & DB2

3

3

Standard Life Environment • 7M IMS transactions per day • <1M CICS transactions per day • 350K Stored Procedures per day• 6 DB2 V7 subsystems, no Data Sharing• >1 TB of DB2 data • Peoplesoft

This is a summary of Standard Life’s production environments.

IMS is our primary transaction manager.

Stored procedure use is growing rapidly (SQL/PL and Cobol)

3 production IMS subsystems (8 development subsystems)15 further development DB2 subsystems DB2 data is compressed (saving many hundred GB)

Page 4: B05 Rexx & DB2

4

4

Why Use Rexx ?• Free• Easy to code • Easy to debug• Provides powerful functionality from:

• Built-in Functions• Interfaces to various environments

• (e.g. TSO, ISPF, DB2)

Rexx is easy to code – it is an interpreted language which means it does not require to be compiled (although a compiler is available). There is no need to set up working storage areas or to be very concerned about data types (everything is character). Rexx is very forgiving in terms of the format, column spacing etc.

Generally speaking Rexx is not ideal for high volume applications as it is an interpreted language.

Debugging is straightforward using the TRACE function.

There are many functions available to Rexx some of which are similar to DB2 functions.

Arguably the most powerful feature of Rexx it the ease with which it can communicate with a number of external (host) environments e.g. IMS, DB2, TSO, ISPF, ISV products. When Rexx encounters a statement it does not recognise it passes the statement to the current host environment – we will introduce the host environments which are most useful to Standard Life…

Page 5: B05 Rexx & DB2

5

5

Host Environments - TSOUsed for

• Job Submission• Interface with Workload Scheduler• DSN Command Processor• Dataset Allocation

There are a large number of TSO services which are available, these are the services which Standard Life most frequently use.

Job submission – Using the TSO SUBMIT command.

At Standard Life rather than use TSO SUBMIT we more frequently use a TSO interface to our batch scheduler to control running of jobs which ensures appropriate control of resources.

DSN Command Processor – We can start up a TSO DSN session from a Rexx which is useful for issuing DB2 commands and BIND statements.

Dataset Allocation – Used for allocating datasets for input or output e.g. when using the DSN processor to perform a Bind we may require to allocate DBRM libraries.

Page 6: B05 Rexx & DB2

6

6

Host Environments - ISPFUsed for

• Browse/Edit Datasets• Edit Macros• Panel Processing• ISPF Tables • File Tailoring

There are a number of ISPF services that are very useful to Rexx programming.

The last three on this slide tend to be the most useful.

Panel processing is used for providing a good interface for the users.

ISPF tables – not related to DB2 tables. ISPF allows you to define a set of typeless variables which can be considered columns and a number of observations i.e. rows. This is particularly useful when combined with display services – ISPF tables can be created from the results of DB2 queries (for example) and displayed on a panel without the need for application code to deal with scrolling etc – the ISPF display service will do this for you.

File Tailoring services allow you to define skeleton JCL which can be brought into a Rexx program and written out with substitution of the skeleton variables. File tailoring can be used on any type of dataset – it is not limited to JCL.

Page 7: B05 Rexx & DB2

7

7

Host Environments – DSNREXXUsed for SQL requests

• DML• DDL

A SQLCA is set up by this environment so that the SQLCA variables can be tested when required, in particular SQLCODE.

As each SQL statement is passed to the external environment, in addition to the SQLCA, the Rexx special variablerc is also set each time. This is zero for successful requests, +1 for SQL warnings, -1 for SQL errors and –3 for other errors.

Page 8: B05 Rexx & DB2

8

8

Example – Single RunstatsInput: Subsystem Id

Database NameTablespace Name

//DBASTATS JOB PROD,'RUNSTATS',CLASS=4,MSGCLASS=X //* //UTIL0001 EXEC PGM=DSNUTILB,PARM='&SSID' //SYSPRINT DD SYSOUT=* //UTPRINT DD SYSOUT=* //SYSIN DD * RUNSTATS TABLESPACE &DBNAME..&TSNAME SHRLEVEL CHANGE TABLE INDEX ALL HISTORY ALL

Here we build a Rexx which creates and submits RUNSTATS for a given tablespace.

The slide shows skeleton JCL for RUNSTATS of a single tablespace. There exists skeleton variables for subsystem id, database and tablespace which need to be substituted before the job can be submitted.

This skeleton JCL is held as a member of a PDS library, say DBA.PROD.SKELETON(RUNSTATS).

Page 9: B05 Rexx & DB2

9

9

Example – Single RunstatsTSO EXEC ‘DBA.PROD.EXEC(RUNSTATS)’ ‘DSA DB01 TS01’

/* Runstats Rexx for a given tablespace */arg ssid dbname tsnameaddress ISPEXEC “LIBDEF ISPSLIB DATASET ID('DBA.PROD.SKELETON')”“LIBDEF ISPFILE DATASET ID('DBA.PROD.JOB')”“FTINCL RUNSTATS”“FTCLOSE NAME(RUNSTATS)”address TSO“SUBMIT JOB(‘DBA.PROD.JOB(RUNSTATS)’)”If rc=0 then say “Runstats job submitted”Else say “Error in RUNSTATS submit rc=” rc

Here is the Rexx code which accepts the input and creates and submits the runstats job. The arg statement is used to assign the values used when the Rexx was invoked into the variables ssid, dbname and tsname.

address ISPEXEC is used to direct any subsequent non-Rexx code to the ISPF environment.LIBDEF ISPSLIB is an ISPF statement which defines where ISPF skeletons are foundLIBDEF ISPFILE is an ISPF statement which defines where output for file tailoring is directed

FTINCL is used to include the RUNSTATS skeleton member for file tailoring. At this point ISPF locates our skeleton member, searches for skeleton variables and replaces these with the values currently held in the corresponding Rexx variables. i.e. the skeleton ssid variable will be replaced with DSA, dbname with DBPROD and tsname with TSPROD.

No special action has been require to put the desired values in to the skeleton variables other than use corresponding names.

FTCLOSE is used to create the output PDS member on DBA.PROD.JOB called RUNSTATS i.e. following the variable substitution.

address TSO is used to direct any subsequent non-Rexx code to the TSO environment.

SUBMIT is a TSO service which submits the named job i.e. RUNSTATS.

Notice that there are very few lines of Rexx code in this program. Rexx is a very good glue for stick together different services in different environments.

Error checking has been minimised in this example. Every time a line is passed to an external environment a special variable, rc, is set according to whether the request has been successful or not.

The trouble with this example is that you do not want to have to remember the name of the Rexx or details of the parameter list. A much better approach is to have an ISPF option which displays a panel with the input variables and invokes the Rexx for you...

Here is an ISPF panel definition which could be used for this purpose (next slide)…..

Page 10: B05 Rexx & DB2

10

10

Example – Single Runstats)ATTR £ TYPE(OUTPUT) INTENS(HIGH) COLOR(RED) CAPS(OFF) ! TYPE(TEXT) COLOR(BLUE) $ TYPE(TEXT) COLOR(YELLOW) _ TYPE(INPUT) COLOR(GREEN) PADC(_)

)BODY WINDOW(64,9)

!Please enter details for your RUNSTATS Request:

$Subsystem : _ssid$ $Database : _dbname $ $Tablespace: _tsname $

£statmsg

)END

The ATTRibute section defines some attributes for painting the screen which is done in the BODY section. Attributes provide information on what are input or output variables, what colour they are and also general text properties.

This was a slightly artificial example to introduce Rexx, ISPF Panels and File Tailoring. We can now build up a more realistic example....

Additional notes

Below is the Rexx program updated to use the ISPF panel on this slide. Note that the variable names do not necessarily have to be used anywhere in the Rexx code.

Page 11: B05 Rexx & DB2

11

11

Example – Single Runstats)ATTR £ TYPE(OUTPUT) INTENS(HIGH) COLOR(RED) CAPS(OFF) ! TYPE(TEXT) COLOR(BLUE) $ TYPE(TEXT) COLOR(YELLOW) _ TYPE(INPUT) COLOR(GREEN) PADC(_)

)BODY WINDOW(64,9)

!Please enter details for your RUNSTATS Request:

$Subsystem : _ssid$ $Database : _dbname $ $Tablespace: _tsname $

£statmsg

)END

/* Runstats Rexx */ /* */ /* This Rexx creates and submits a RUNSTATS job. It displays a panel for input of subsystem id, */ /* database name and tablespace name. It performs validation against these. */ /* A message is sent to the panel indicating if the job was submitted or an error encountered */

call initialise call display_panel do while end_key_not_used

call validate_panel if statmsg = "" then call create_and_submit_runstatscall display_panel

end address ISPEXEC "REMPOP" /* Remove Popup */

return

initialise:statmsg = "" valid_ssids = "DSN DSA DSB DSH DSBK"||,

" DSS DSX DS1" /* List of subsystems*/zwinttl = "DB2 Runstats Requests" /* Set Panel Title */address ISPEXEC "LIBDEF ISPPLIB DATASET ID('DBA.PROD.PANEL')" /* Define Panel lib */"ADDPOP COLUMN(5) ROW(9)" /* Use Popup Panels */return

display_panel: address ISPEXEC /* display panel for input of */"DISPLAY PANEL(RUNREQ)" /* variables - if user exits */ if rc=0 then end_key_not_used = 1 /* by "END" or PFKey then rc>0 */ else end_key_not_used = 0

return

Page 12: B05 Rexx & DB2

12

12

Example – Single Runstats)ATTR £ TYPE(OUTPUT) INTENS(HIGH) COLOR(RED) CAPS(OFF) ! TYPE(TEXT) COLOR(BLUE) $ TYPE(TEXT) COLOR(YELLOW) _ TYPE(INPUT) COLOR(GREEN) PADC(_)

)BODY WINDOW(64,9)

!Please enter details for your RUNSTATS Request:

$Subsystem : _ssid$ $Database : _dbname $ $Tablespace: _tsname $

£statmsg

)END

validate_panel:statmsg = "" select when (wordpos(ssid,valid_ssids) = 0), then do

statmsg = "Please enter a valid subsystem name"cursvar = "ssid"

end when dbname = "" then do

statmsg= "Please enter a database name"cursvar = "dbname"

end when tsname = "" then do

statmsg= "Please enter a tablespace name"cursvar = "tsname"

end otherwise end

return

create_and_submit_runstats: address ISPEXEC "LIBDEF ISPSLIB DATASET ID('DBA.PROD.SKELETON')" "LIBDEF ISPFILE DATASET ID('DB.SL015D.JOB')" "FTINCL STATS" "FTCLOSE NAME(RDB2STAT)" address TSO "SUBMIT 'DB.SL015D.JOB(RDB2STAT)'" if rc=0 then statmsg = "Runstats submitted" else statmsg = "Submit failed rc="rc

return

Page 13: B05 Rexx & DB2

13

13

Example – Multiple RunstatsInput: Subsystem IdISPF SKELETON JCL HEADER (STATHDR)//DBAPROD JOB PROD,'DB2 AUTO STATS',CLASS=A,MSGCLASS=X //UTIL0001 EXEC PGM=DSNUTILB,PARM='&SSID' //SYSPRINT DD SYSOUT=* //UTPRINT DD SYSOUT=* //SYSIN DD * LISTDEF STATS

ISPF SKELETON FOR EACH TABLESPACE (STATTS)INCLUDE TABLESPACES TABLESPACE &DB..&TS

ISPF SKELETON TO INVOKE RUNSTATS (STATEND) RUNSTATS TABLESPACE LIST STATS SHRLEVEL CHANGE TABLE INDEX ALL SAMPLE HISTORY ALL

What we want to do in this example is build a runstats job for all our objects which require to be runstatted. Inotherwords we have a runstats policy which we want to execute. We will do this by building a LISTDEF called STATS then invoke RUNSTATS on the list.

Our first step is to break up our previous runstats skeleton into three pieces – a static JCL header, an include statement to build the LISTDEF components which will appear a variable number of times and finally the RUNSTATS statement itself.

Page 14: B05 Rexx & DB2

14

14

Example – Multiple Runstats/* Rexx to execute Runstats policy */arg ssidcall environment_setup call set_stat_policy call db2_initialisation call get_job_header call generate_runstatscall disconnect_and_submit

return

As this is a more complex program, we will format the program more neatly by breaking the functionality into chunks called internal subroutines. These are contained within the same Rexx program and are executed using therexx statement call subroutine_name. The subroutine is identified by the subroutine label which is the subroutine_name followed by a colon. A return statement is required at the end of the subroutine. Subroutines are particularly useful for chunks of code which need to be executed at different points in a program, for example checking sqlcodes.

So we want to set up our environments, determine the runstats criteria (or policy), initialise DB2, start file tailoring by preparing the job header, generate the appropriate statements for runstats, then some cleanup activities.

We require a return at the end of this section of code to let the Rexx know to stop at this point otherwise it will attempt to continue through to the first internal subroutine.

We will now go through each internal subroutine in turn….

Page 15: B05 Rexx & DB2

15

15

Example – Multiple Runstatscall environment_setup

environment_setup:rc = RXSUBCOM('ADD','DSNREXX','DSNREXX') address ISPEXEC "LIBDEF ISPSLIB DATASET ID('DBA.PROD.SKELETON')" "LIBDEF ISPFILE DATASET ID('DBA.PROD.JOB')"

return

RXSUBCOM is used to set up the DSNREXX environment. We did not require to do this for TSO or ISPF as these are default environments. There are a number of default environments however DSNREXX is not one of these. RXSUBCOM in this case will add the DSNREXX environment.

The success of the RXSUBCOM will be passed back into variable rc which will be zero if there have been no problems, +4 if the environment already existed or >+4 if an error was encountered. The return code should ideally be tested.

The LIBDEFs we used in the previous example are repeated.

Page 16: B05 Rexx & DB2

16

16

Example – Multiple Runstatscall set_stat_policy

set_stat_policy:

stat_policy = "SELECT DBNAME, NAME ",

" FROM SYSIBM.TABLESPACESTATS ",

" WHERE DBNAME NOT IN ('DSNDB01','DSNDB07') ",

" AND (STATSLASTTIME IS NULL ",

" OR STATSMASSDELETE > 0 ",

" OR LOADRLASTTIME > STATSLASTTIME ",

" OR DECIMAL(STATSINSERTS+STATSDELETES+STATSUPDATES)",

" *100/MAX(TOTALROWS,1)>1) "

return

In this section we define the RUNSTATS policy.

The runstats policy is an SQL statement. Commas at the end of the line are continuation characters.

Our runstats policy is based on the real time statistics table TABLESPACESTATS (which is a very useful table). We identify rows where there has been activity since the last runstats which we feel should trigger another runstats, in particular where there as been >1% change activity.

As an aside, this is very similar to our actual runstats policy except that TOTALROWS is not used as it is only populated at LOAD REPLACE or REORG. We join with some catalog tables instead.

Page 17: B05 Rexx & DB2

17

17

Example – Multiple Runstatscall db2_initialisation

db2_initialisation: address DSNREXX "CONNECT" ssid"EXECSQL DECLARE C1 CURSOR FOR S1" call check_sqlcode"EXECSQL PREPARE S1 INTO :OUTSQLDA FROM :stat_policy" call check_sqlcode"EXECSQL OPEN C1" call check_sqlcode

return

We defined a SQL statement which we want to use, now we prepare the statement and open the cursor.

We use address DSNREXX to direct any non-Rexx statements to DSNREXX.

The CONNECT statement will connect to the input subsystem id (ssid). We will receive a return code in special variable rc to show the success of this (all statements which are passed to another environment set the special variable rc).

We then declare the cursor. DSNREXX requires use of a predefined set of cursor names and statement names which must correspond to each other. Cursor names are C1 to C100 and prepared statements names are S1 to S100. When using C5 (for example) the statement must be S5.

Note that if you check the value of rc to ensure that the connect has been successful, then it should be done immediately after the CONNECT statement has been issued as the EXECSQLs (for example) will cause rc to be reset.

We will be retrieving a number of rows here, however if you have an SQL statement which can only ever return a single row this approach will still be required as Rexx cannot use SELECT INTO.

There are a number of host variable names which should be avoided: C1-C100, S1-S100, names beginning with 'SQL', 'RDI', 'DSN', 'RXSQL', or 'QRW' and names ending with a period.

Page 18: B05 Rexx & DB2

18

18

Example – Multiple Runstatscall get_job_header

get_job_header: address ISPEXEC "FTINCL STATHDR"

return

This pulls in the header for the runstats job i.e. no RUNSTATS control statements yet.

Page 19: B05 Rexx & DB2

19

19

Example – Multiple Runstatscall generate_runstats

generate_runstats: do until sqlcode ¬= 0

address DSNREXX "EXECSQL FETCH C1 INTO :db, :ts" call check_sqlcodeif sqlcode = 0 then do

db = strip(db) address ISPEXEC "FTINCL STATTS"

end end address ISPEXEC "FTINCL STATEND"

return

This section generates all the RUNSTATS statements.

We have a program loop which we use until we have a non-zero sqlcode.

In that loop we fetch the next row from the cursor (from TABLESPACESTATS). This uses host variables db and ts.

We check the sqlcode and if it is zero we remove trailing spaces from the database name (db). We require to do this because DB2 has told Rexx that the length of the variable db is 8 (DB2 V7). We then use the File Tailoring service to add a control line to our job.

We continue to loop round reading from the cursor and adding further INCLUDE statements until we have a non-zero sqlcode.

We can then add the RUNSTATS statement itself using the skeleton STATEND.

Page 20: B05 Rexx & DB2

20

20

Example – Multiple Runstatscall disconnect_and_submit

disconnect_and_submit: address DSNREXX "DISCONNECT" address ISPEXEC "FTCLOSE NAME(RUNSTATS)" address TSO "SUBMIT 'DBA.PROD.JOB(RUNSTATS)'"

return

Here we disconnect from DSNREXX. We could also remove the environment using RXSUBCOM(‘DELETE’,’DSNREXX’,’DSNREXX’)

We then close the output JCL member and submit it.

What we have in place at Standard Life is slightly more complex in that we have a variable number of runstats jobs which can run in parallel. The rexx program reads the maximum number of streams from a DB2 table then distributes the runstats requests across these streams in order to balance them.

Page 21: B05 Rexx & DB2

21

21

Example – Multiple Runstatscheck_sqlcode:

if sqlcode<0 then do say "Sqlcode = " sqlcodesay "Sqlerror = " sqlerrmc rc=16 call error

end return

error: address ISPEXECzispfrc=rc "VPUT (ZISPFRC) SHARED" exit

return

Here is a typical piece of error handling code. The code in the error section is used to set a special ISPF variable called ZISPFRC. This is used in batch to ensure the condition code is set correctly when an error has been encountered. This is required when running a Rexx with ISPF services in batch.

If the Rexx is not running under ISPF then the return code can be passed back to the job using:

return rcOr

exit rc

The JCL for ISPF and non-ISPF is quite different…

Page 22: B05 Rexx & DB2

22

22

Rexx in Batch (non-ISPF)//DBAREXX JOB DBA,REXX,CLASS=A,MSGCLASS=J

//*

//STEPREXX EXEC PGM=IKJEFT01

//SYSEXEC DD DSN=DBA.PROD.REXX,DISP=SHR

//SYSTSPRT DD SYSOUT=W

//SYSTSIN DD *

REXXNAME REXXPARMS

//*

Here is a job which can execute a rexx which does not contain ISPF services.

Page 23: B05 Rexx & DB2

23

23

Rexx in Batch (ISPF)//DBAREXX PROC REXX=,PARMS= //DBAREXX EXEC PGM=IKJEFT01,DYNAMNBR=99,ACCT=SF,REGION=0M, // PARM='ISPSTART CMD(&REXX &PARMS)' //SYSEXEC DD DISP=SHR,DSN=DBA.PROD.REXX // DD DISP=SHR,DSN=SYS1.SISPEXEC //ISPPLIB DD DISP=SHR,DSN=DBA.PROD.PANEL //ISPSLIB DD DISP=SHR,DSN=DAB1.PROD.SKELETON //ISPMLIB DD DISP=SHR,DSN=SYS1.SISPMENU //ISPTABL DD DISP=SHR,DSN=DBA.PROD.TABLE //ISPTLIB DD DISP=SHR,DSN=SYS1.SISPTENU // DD DISP=SHR,DSN=SYS1.SISFTLIB //ISPPROF DD UNIT=VIO,SPACE=(TRK,(1,,1)), // DCB=(LRECL=80,RECFM=FB,BLKSIZE=800) //ISPLOG DD SYSOUT=*, // DCB=(LRECL=125,RECFM=VA,BLKSIZE=129) //SYSTSIN DD DUMMY //SYSTSPRT DD SYSOUT=*

Here is a general purpose JCL procedure which we use to run a Rexx in batch which in turn uses ISPF services. The Rexx to be executed is passed as a variable as well as the parameter list.

Some datasets are validated by ISPF even if they are not used by the Rexx e.g. ISPPLIB.

Page 24: B05 Rexx & DB2

24

24

Housekeeping at Standard LifeRTS

IMAGE COPY POLICY

MODIFY POLICY

REORG POLICY

QUIESCE POLICY

RUNSTATS POLICY

Catalog

Reorg

UTILITIES

UTILITIES

SELECTION POLICY

Adhoc ReorgRequests

Image Copy, Quiesce, Modify and Reorgs work in a similar way to the RUNSTATS example seen on the previous slides. Each has a Rexx program controlling the policy which requests scheduling of utilities via our Batch Scheduling software.

The daily Image Copy policy checks the RTS table to determine whether any changes have been made eachtablespace since the last full copy. If not then another IC is not required. Otherwise an incremental or full IC is taken depending on the proportion of changes. The policy has an overriding rule that no tablespace can go more than 3 days without a full image copy being taken. A single IC job is scheduled on each subsystem usingLISTDEFs and taking advantage of the PARALLEL keyword.

The reorg policy runs each week following completion of the runstats jobs. The reorg criteria is applied to thecatalog using essentially the criteria from the admin guide and populates a table which contains candidate objects for reorg (tablespaces and indexspaces). Each candidate has a current size, reorg priority based on how bad a shape the object is in, the reason(s) for the priority, reorg elapsed time estimate which takes account of the current size and any previous reorgs for that object.

Adhoc requests can be added to the table through a DBA panel (e.g. to implement changes).

Each day a reorg selection policy is run on the reorg table to identify objects to reorg. This takes account of a number of factors including number of parallel reorg streams available to accept reorgs on the subsystem, priority of reorgs, estimated time of reorg and preferred maximum batch window size for reorgs. This then creates the jobs and passes information to the batch scheduler to process. We run around 4000 reorgs per month all going through this process, most of which are index reorgs. The File Tailoring services we use for reorgs have steps before and after each reorg step which update the appropriate row on the reorg table with the start and end times of the reorg and whether the reorg was successful.

Quiesce policy runs daily and Modify policy runs weekly on each subsystem and are more straightforward.

No intervention is required – if there is a problem it is flagged through our incident management software.

Page 25: B05 Rexx & DB2

25

25

Standard Life Examples• Maximising RoI:

• Utilities• Referential Extracts• Performance Database• Remove toolsets

• Minimising Disruption• Repetitive Tasks

Using features explored on the previous slides Standard Life has levered increased return on investment from various vendors toolsets:

Utilities (previous slide) – we optimise the use of our utilities by creating Rexx processes to drive them.

Referential Extracts – Standard Life has a product which extracts and loads referentially intact sets of data between environments. This works well if you are familiar with the database structure and the product itself. Familiarity with these components is often difficult for business testers and developers (especially off-mainframe). We created a set of panels which guides a user through the options to decide what sets of data they require, where to get it from and where to put it. The application then creates a job which executes the product with all the appropriate information. Optional steps can also included to, for example, age data, keep the data for reload, selective clear out the target environment before reload (functions which are not available in the product itself).

As part of this process we realised we could use the Referential Extract product to clone data by using skeleton sets of data.

We have a performance monitor product which we use to keep extensive performance information on a DB2 database. We created a set of panels which bring back information from the tables using the DSNREXX interface.

We have also been able to remove some products by replacing their functionality with Rexx applications.

Page 26: B05 Rexx & DB2

26

26

Standard Life Examples• Maximising RoI:

• Utilities• Referential Extracts• Performance Database• Remove toolsets

• Minimising Disruption• Repetitive Tasks

Minimising Disruption

We (Standard Life DBAs) have developed a large number of options on a DBA panel for adhoc tasks which are regularly required. We also monitor requests made to us from developers and other areas – any which occur frequently we will strive to create a process, usually Rexx, which services these requests without our involvement, for example:

We regularly are requested for extracts of production data from various business areas. We created a Rexxapplication which allowed requests to be put on a DB2 table (with a frequency if required). A daily Rexx process reads rows from this table to decide which unloads are required and schedules the appropriate unload jobs.

Another example is our recovery process. We have a set of panels which will query SYSCOPY for valid recovery points for input database and/or tablespace criteria. The user can select one of these and a recovery job is generated.

There are dozens of other tasks which we have tackled in this manner.

Page 27: B05 Rexx & DB2

27

27

DB2 CommandsTSO DSN can be used:

issue_db2_commands: address TSO “NEWSTACK”do i=1 to max_command

queue db2_command.i end queue "END" x=outtrap('cmdoutput.',,noconcat) "DSN SYSTEM("ssid")" db2_rc = rcx=outtrap("OFF") “DELSTACK”

return

The DSN command processor can be invoked from within a Rexx to issue DB2 commands.

The slide shows a general purpose section of code to start a DSN session and issue the commands which are contained in an array of variables called db2_command. For example:

db2_command.1 = “-START DATABASE(DBAPROD) SPACENAM(SPAC1) ACCESS(FORCE)”db2_command.2 = “-START DATABASE(DBAPROD) SPACENAM(SPAC2) ACCESS(FORCE)”db2_command.3 = “-START DATABASE(DBAPROD) SPACENAM(SPAC3) ACCESS(FORCE)”db2_command.4 = “-DISPLAY DATABASE(DBAPROD) SPACENAM(*) RESTRICT LIMIT(1000)”Max_command = 4

The Rexx function “queue” will place the commands on a data stack (buffer). When the DSN session starts it will pull commands from this stack and process them. Once the commands on the stack have been exhausted, DSN will expect further input from the user. To avoid this the DSN command “END” is put on the stack after all the other commands to allow the DSN session to close and control to pass back to the next line of the program.

The Rexx function “outtrap” is used to trap the output from commands. DSN output is placed in an array of variables called cmdoutput I.e.

Cmdoutput.1Cmdoutput.2Cmdoutput.3Etc

The number of lines of output is held in a special array variable called cmdoutput.0

The DB2 commands can be entered or DSN commands such as BIND PACKAGE, RUN PROGRAM, FREE PLAN and SPUFI.

NEWSTACK and DELSTACK will not be required in most cases, however their use is recommended as there are cases when DSN may not be able to complete processing of the queued commands resulting in later problems if there are unprocessed entries on the TSO stack.

Page 28: B05 Rexx & DB2

28

28

DB2 CommandsTSO DSN can be used:

issue_db2_commands: address TSO “NEWSTACK”do i=1 to max_command

queue db2_command.i end queue "END" x=outtrap('cmdoutput.',,noconcat) "DSN SYSTEM("ssid")" db2_rc = rcx=outtrap("OFF") “DELSTACK”

return

An example of a Rexx program which uses this section of code (with ISPF Tables) follows:

/* This Rexx will issue a DISPLAY DATABASE (db) SPACENAM(*) RESTRICT */ /* on subsystem ssid and put the results on an ISPF table. The */ /* table will then be displayed using ISPF services. The ssid and */ /* database name can be repeatedly changed. */ call initialisation call display_panel do while end_key_not_used

call build_screen call display_panel

end call cleanup

return

initialisation: /* Define the datasets for ISPF panels and ISPF tables */ /* The current userid is chosen as a way of allowing different users */ /* to use the application concurrently */ address ISPEXEC "LIBDEF ISPPLIB DATASET ID('DAB1.PROD.PANEL')" "LIBDEF ISPTLIB DATASET ID('DAB1.PROD.TABLE')" "LIBDEF ISPTABL DATASET ID('DAB1.PROD.TABLE')" "ADDPOP COLUMN(3) ROW(-3)" /* use Popup Panels */ "TBCREATE " userid(), /* create an ISPF table */ "names (nam typ prt stat) REPLACE" /* named using the current */

/* userid with 4 variables */ /* (i.e. columns) */

zwinttl = "Display Spaces" /* set panel title */ return

Page 29: B05 Rexx & DB2

29

29

DB2 CommandsTSO DSN can be used:

issue_db2_commands: address TSO “NEWSTACK”do i=1 to max_command

queue db2_command.i end queue "END" x=outtrap('cmdoutput.',,noconcat) "DSN SYSTEM("ssid")" db2_rc = rcx=outtrap("OFF") “DELSTACK”

return

display_panel: /* the section displays the ISPF table whose value is USERID() */ /* using the panel definition DISREST */

address ispexec"TBDISPL " userid() "PANEL(DISREST)" if rc=8 then end_key_not_used = 0 else end_key_not_used = 1dbmsg = ""

return

build_screen: address ISPEXEC db2_command.1 = "-DISPLAY DB("db") spacenam(*) restrict limit(1000)" max_command = 1 /* only 1 command */ call issue_db2_commands /* execute command */ if db2_rc > 4 then dbmsg = cmdoutput.1 /* error occurred DSN msg */

/* will be in first line of output */ do m= 1 to cmdoutput.0 until (pos("DSNT397I",cmdoutput.m)>0)

/* go through output looking for DSNT397I message */ end if pos("DSNT397I",cmdoutput.m)=0 then dodbmsg = "Unexpected error while formatting output" return /* Unknown format */

end m=m+1 /* move to next line */

nampos = pos("NAME",cmdoutput.m) /* which contains the */typpos = pos("TYPE",cmdoutput.m) /* field headers - use */prtpos = pos("PART",cmdoutput.m) /* these to calculate */statpos = pos("STAT",cmdoutput.m) /* position of field values */

Page 30: B05 Rexx & DB2

30

30

DB2 CommandsTSO DSN can be used:

issue_db2_commands: address TSO “NEWSTACK”do i=1 to max_command

queue db2_command.i end queue "END" x=outtrap('cmdoutput.',,noconcat) "DSN SYSTEM("ssid")" db2_rc = rcx=outtrap("OFF") “DELSTACK”

return

m=m+2 /* move to next line then skip again as it */ /* only contains underlines for field headers */

"TBCREATE " userid() "names (nam typ prt stat) REPLACE" /* Recreate ISPF Table */ if pos("NO SPACES FOUND",cmdoutput.m) > 0 then,dbmsg = "No Spaces Found" /* Check for no spaces msg*/ else do i=m to cmdoutput.0-2 /* The DISPLAY command ends with a display ended msg and */

/* a DSN9022I msg hence the final set of field values will be *//* located at line cmdoutput.0 less two */

nam = substr(cmdoutput.i,nampos,8) /* Set table values to */typ = substr(cmdoutput.i,typpos,4) /* the DISPLAY output values */prt = substr(cmdoutput.i,prtpos+1,3) stat = substr(cmdoutput.i,statpos,12) if prt = "" then part=prt /* Format partition no */ else part = "PART("prt")" "TBADD " userid() /* Add current line to ISPF table */

end return

issue_db2_commands: address TSO “NEWSTACK" /* Create a new stack */ do i=1 to max_command

queue db2_command.i /* Place DB2 commands on stack */ end queue "END" /* Ensure DSN session ENDs */ x=outtrap('cmdoutput.',,noconcat) /* Prepare to collect output */ "DSN SYSTEM("ssid")" /* Start DSN session which will then */

/* pull commands from the stack */ db2_rc = rcx=outtrap("OFF") /* Stop output collection */ "DROPBUF" /* Remove stack and any outstanding contents */

return

Page 31: B05 Rexx & DB2

31

31

DB2 CommandsTSO DSN can be used:

issue_db2_commands: address TSO “NEWSTACK”do i=1 to max_command

queue db2_command.i end queue "END" x=outtrap('cmdoutput.',,noconcat) "DSN SYSTEM("ssid")" db2_rc = rcx=outtrap("OFF") “DELSTACK”

return

cleanup: /* Remove ISPF table */ address ISPEXEC "TBCLOSE " userid() address ISPEXEC "TBERASE " userid()

return

This is the ISPF panel used in the above example:

)PANEL )ATTR £ TYPE(TEXT) COLOR(GREEN) + TYPE(TEXT) COLOR(BLUE) @ TYPE(INPUT) INTENS(HIGH) COLOR(YELLOW) # TYPE(OUTPUT) INTENS(HIGH) COLOR(YELLOW) ¬ TYPE(INPUT) CAPS(ON) INTENS(HIGH) COLOR(RED) HILITE(USCORE) )BODY £Command ===>¬ZCMD £ £Subsystem ===>@ssid£ £Database ===>@db ££ #dbmsg£+ Spacename Type Partn Status + --------- ---- ----- ------)MODEL

#Z + #Z #Z #Z + )INIT .ZVARS = '(nam typ prt stat)'

)END

Page 32: B05 Rexx & DB2

32

32

Rexx Stored ProceduresConsiderations

• Stored Procedure definitions• Maximum of 1 variable for output• WLM address space cannot be used for full range

of functions e.g. DSN processor• Overhead of CALL

Using Rexx stored procedure does introduce some complications:

Stored procedure definitions are required e.g. a WLM address space needs set up coded with a SYSEXEC which contains your stored procedure Rexx code. We recommend a separate library for Rexx code which is used in a stored procedure and that at least one separate address space is used for Rexx stored procedures (i.e. do not mixRexx and non-Rexx stored procedures in the same WLM). NUMTCB must be 1 for rexx stored procedures. DDL for the stored procedure needs to be created/maintained and authorities granted.

There is a constraint that only 1 variable can be used for output. Many can be used for input. This can be worked around by defining a large output parameter and passing back several variables as a single parameter list.

All stored procedure variables must be initialised before the stored procedure is called.

Rexx stored procedures will incur a performance overhead for the call. There is also an overhead because at least two programs are involved i.e. the Rexx stored procedure program and the program which invokes it, which is likely to be Rexx also.

Page 33: B05 Rexx & DB2

33

33

Why Use Rexx Stored Procedures• Security and control – SECURITY

DEFINER/SECURITY DB2• Note: Does not apply to SQL

• Remote access through DDF• Executing remote DB2 commands• Non-DB2 functions

• Potential for “unusual” processing

So why use Rexx stored procedures if there are these complications ?

Rexx applications may want to perform operations such as issue DB2 commands or update datasets. A Rexxapplication runs with the authority of the TSO userid invoking the application therefore there is scope for the user to bypass the Rexx application and update a dataset directly avoiding the validation contained in the Rexx code.

By using the SECURITY option you can avoid the need to provide the TSO userid the ability to manipulate the underlying resources, thereby ensuring that application validation is performed.

An example of how this could be used is to provide a panel which allows developers to start a specific tablespacewith access force without allowing them to start any other tablespaces in the database. A stored procedure could be written which issues the –START command which runs with SECURITY DB2 (the WLM address space will require STARTDB authority). The user does not require STARTDB and can only execute the Rexx stored procedure in order to start the tablespace. It is therefore important to have solid controls on the datasets referenced in the SYSEXEC DD name of the WLM address spaces.

Page 34: B05 Rexx & DB2

34

34

Why Use Rexx Stored Procedures• Security and control – SECURITY

DEFINER/SECURITY DB2• Note: Does not apply to SQL

• Remote access through DDF• Executing remote DB2 commands• Non-DB2 functions

• Potential for “unusual” processing

Standard Life has a change management procedure for application BIND statements. In some of our environments, e.g. pre-production, we do not want developers to be able to BINDADD as it represents a risk to the environment. Therefore we use SECURITY DB2 on this stored procedure in order that the binds are performed with the userid of the WLM address space running the stored procedure (could also have used SECURITY DEFINER). The Rexx stored procedure interfaces with our Change Management tool in order to perform validation.

Note that the usefulness of this does not apply to SQL resources. If you want to protect DB2 data in this way then you need to use static SQL (for example in a Cobol stored procedure) or invoke a batch job with a trusted userid.

The ability for Rexx programs to be able to communicate across LPARS is very useful perhaps due to installation, licensing or other constraints. This may be unrelated to DB2 – for example a Rexx running on one LPAR may wants to access non-shared DASD on another LPAR.

Standard Life has a change management process for application stored procedures where the stored procedure statement source is controlled by an ISV product. The product is only installed on one LPAR however the destination of the stored procedure could be one of several LPARS. The ISV invokes Rexx code on its home LPAR which then executes a stored procedure on the appropriate target LPAR to do the work. The ISV process receives information on the success or otherwise of the request and can proceed accordingly.

Unusual processing: It would be possible, but generally discouraged, to have say a Rexx SP which is called from a CICS/IMS transaction to perform some activities e.g. submit jobs, update PDS members.

Page 35: B05 Rexx & DB2

35

35

Rexx Stored Procedure DDLCREATE PROCEDURE DA01PROD.ADD_PLAN

( IN SSID CHARACTER(4) ,IN PLANNAME CHARACTER(8) ,IN PLANOWNER CHARACTER(8) ,IN EXECUTETO CHARACTER(8) ,OUT RETCODE CHARACTER(6))

EXTERNAL NAME 'BINDADD' RESULT SET 0 LANGUAGE REXX PARAMETER STYLE GENERAL COLLID DSNREXCS WLM ENVIRONMENT DSADBA PROGRAM TYPE MAIN SECURITY DB2

The DA01PROD.ADD_PLAN procedure has 4 input parameters and a single output parameter. The member name of the Rexx code in the WLM SYSEXEC DD which is picked up when the stored procedure is called is BINDADD.

Language is Rexx and the collection id is DSNREXCS (could also be RR, UR or RS).

Notice that we use SECURITY DB2 in this case.

Rexx stored procedures can process results sets (however my experience has been this can be slow when processing larger result sets using Rexx in a TSO environment).

Page 36: B05 Rexx & DB2

36

36

Rexx Stored Procedures • Rexx Address Space JCL

//DSADBA PROC RGN=0K,APPLENV=DSADBA,DB2SSN=DSA,NUMTCB=1

//IEFPROC EXEC PGM=DSNX9WLM,REGION=&RGN,TIME=NOLIMIT,

// PARM='&DB2SSN,&NUMTCB,&APPLENV'

//STEPLIB DD DISP=SHR,DSN=SYS1.SCEERUN

//SYSUDUMP DD SYSOUT=X

//SYSTSPRT DD SYSOUT=X

//SYSOUT DD SYSOUT=X

//SYSEXEC DD DISP=SHR,DSN=DBA.PROD.SPREXX

Note that NUMTCB must be 1.

SYSEXEC contains the stored procedure Rexx code.

Page 37: B05 Rexx & DB2

37

37

Rexx Stored Procedures• DB2 Commands (using DSNWLIR)

/* REXX */ parse upper arg cmdcommand = substr("COMMAND",1,18," ")ifca = substr('00'x,1,180,'00'x)ifca = overlay(d2c(length(ifca),2),ifca,1+0)ifca = overlay("IFCA",ifca,4+1)rtrnareasize = 262144rtrnarea = d2c(rtrnareasize+4,4)left(' ',rtrnareasize,' ') output = d2c(length(cmd)+4,2)||'0000'x||cmdbuffer = substr(" ",1,16," ") ADDRESS LINKPGM "DSNWLIR COMMAND IFCA RTRNAREA OUTPUT"dsnwli_rc = rc rtrn = c2d(substr(ifca,13,4))reas = c2x(substr(ifca,17,4))totlen = c2d(substr(ifca,20+1,4)) return dsnwli_rc rtrn reas totlen strip(rtrnarea)

This slide is included for completeness as it is a more complex example. However it can be used without changes to execute a DB2 command from within a stored procedure.

Here the stored procedure is using only one variable in which to accept input and return a number of output components.

More information on the IFI data areas can be found in Appendix E of the DB2 Admin Guide.

An example program which calls this stored procedure follows:

/* The rexx will issue a display thread command on the input ssid. *//* The stored procedure must be defined on that subsystem. */arg ssidcall initialise call invoke_stored_proc call format_and_display_output

return

initialise: /* set up DSNREXX environment, DB2 command and variables for formatting output */ s_rc = RXSUBCOM('ADD','DSNREXX','DSNREXX') /* Prepare to call SP */rtrnarea = left("-DIS THD(*)",2500," ") /* Format command */pos15 = 9prevpos=9 finished = 0

return

Page 38: B05 Rexx & DB2

38

38

Rexx Stored Procedures• DB2 Commands (using DSNWLIR)

/* REXX */ parse upper arg cmdcommand = substr("COMMAND",1,18," ")ifca = substr('00'x,1,180,'00'x)ifca = overlay(d2c(length(ifca),2),ifca,1+0)ifca = overlay("IFCA",ifca,4+1)rtrnareasize = 262144rtrnarea = d2c(rtrnareasize+4,4)left(' ',rtrnareasize,' ') output = d2c(length(cmd)+4,2)||'0000'x||cmdbuffer = substr(" ",1,16," ") ADDRESS LINKPGM "DSNWLIR COMMAND IFCA RTRNAREA OUTPUT"dsnwli_rc = rc rtrn = c2d(substr(ifca,13,4))reas = c2x(substr(ifca,17,4))totlen = c2d(substr(ifca,20+1,4)) return dsnwli_rc rtrn reas totlen strip(rtrnarea)

invoke_stored_proc: address DSNREXX "CONNECT" ssid"EXECSQL CALL DA01PROD.DBACMD (:rtrnarea)" /* Call Stored Proc */parse var rtrnarea dsnwli_rc rtrn reas totlen rtrnarea

/* Only 1 output variable which contains 6 “real” outputs *//* We split these into their component parts */

say "SQLCODE: " sqlcode /* Display outputs */say "DSNWLI rc:" dsnwli_rcsay "RTRN: " rtrnsay "REAS: " reassay "TOTLEN: " totlen

return

format_and_display_output: /* break up the return area line by line - each line is punctuated */ /* by a hex '15' character and the first 4 bytes of each line can */ /* be discarded */ do until finished

pos15 = pos(x2c('15'),rtrnarea,prevpos) if pos15 = 0 then finished = 1 else do

output_line = substr(rtrnarea,prevpos,pos15-prevpos) if output_line ¬= " " then say output_lineprevpos = pos15+5

end end

return

Page 39: B05 Rexx & DB2

39

39

Rexx Stored Procedures• BIND in a Stored Procedure

• Use DSNESM71 – with caution

As the DSN command processor cannot be readily used from a stored procedure address space another method is required.

We use DSNESM71 for this.

Warning – this is not a documented interface, therefore support from IBM will not be available in the event of issues.

DSNESM71 needs to be called from the Rexx code as an external program with the BIND statement set up as input. If DBRM libraries are required the need to be allocated before DSNESM71 is called.

Sample code for the call is:

bind: x = outtrap('bindmsg.',,concat) do l=1 to j

"NEWSTACK" queue bindstmt.l queue "END" ADDRESS ATTCHMVS "DSNESM71"bindrc = rc"DELSTACK"

end return

Page 40: B05 Rexx & DB2

40

40

Final Observations• Give thought to program design• Create independent checking processes• Rexx data type conversion is good• Rexx is not suitable for everything – consider

alternatives

Programs often end up being more complicated than expected. Writing a brief flow or design document is always valuable.

Ensure you are protected against failures in the processing. E.g. we do not want our Copy policy to stop due to scheduling errors or an incorrect update to the Copy policy. We have an independent process which reads SYSCOPY and raises an incident if the youngest row for any object is more than 3 days old.

Rexx treats all variables as character. When passing information between Rexx and DB2 the conversion for data types generally works well. You can be more specific by making use of the SQLDA although Standard Life has not yet had a requirement to do this. One area where we have had to take more care is if we want to pass a numeric variable to a character type in DB2 – “’”100”’” is required.

Sometimes static SQL is best or a process written in another language (e.g. Cobol).

Page 41: B05 Rexx & DB2

41

41

Summary• Rexx is great for DB2

We have seen how easy it is to quickly build powerful DB2 applications, looking at utilities, DB2 commands, binding and SQL processing.

The more practice you have the quicker these applications can be turned around.

Useful Documentation:

TSO/E Rexx User GuideDB2 for z/OS Stored Procedures: Through the CALL and BeyondDB2 UDB for z/OS Version 8 Application Programming and SQL Guide

Page 42: B05 Rexx & DB2

42

42

Jim AddisonStandard Life

[email protected]

Session B05Rexx and DB2