integrating ibm tivoli workload scheduler and content manager ondemand to provide centralized job...

7/31/2019 Integrating IBM Tivoli Workload Scheduler and Content Manager OnDemand to Provide Centralized Job Log Processing Sg246629

1/278

ibm.com/redbooks

Integrating IBM Tivoli WorkloadScheduler and Content ManagerOnDemand to Provide

Centralized Job Log Processingssing

Vasfi Gucer

Cheryl Brown

Budi Darmawan

Doroti Almeida Dias Garcia

Tina Lamacchia

Martin PepperPete Soto

Michael R Tucci

Bob Watling

Solution to integrate ITWS Suite and

Content Manager OnDemand

Quick and centralized access to

ITWS and ITWS for z/OS logs

Scripts provided for

additional customization

Front cover
http://www.redbooks.ibm.com/http://www.redbooks.ibm.com/http://www.redbooks.ibm.com/http://www.redbooks.ibm.com/


2/278


3/278

Integrating IBM Tivoli Workload Scheduler andContent Manager OnDemand to Provide CentralizedJob Log Processing

October 2003

International Technical Support Organization

SG24-6629-00


4/278

Copyright International Business Machines Corporation 2003. All rights reserved.

Note to U.S. Government Users Restricted Rights -- Use, duplication or disclosure restricted by GSA ADPSchedule Contract with IBM Corp.

First Edition (October 2003)

This edition applies to Tivoli Workload Scheduler Version 8.2 and IBM Content ManagerOnDemand 7.1.

Note: Before using this information and the product it supports, read the information inNotices on page xiii.


5/278

Copyright IBM Corp. 2003. All rights reserved. iii

Contents

Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii

Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .xi

Notices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiiiTrademarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiv

Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xvThe team that wrote this redbook. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xv

Become a published author . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xviiComments welcome. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xvii

Chapter 1. Overview and benefits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.1 IBM DB2 Content Manager OnDemand overview. . . . . . . . . . . . . . . . . . . . 31.2 Servers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51.3 OnDemand concepts. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

1.3.1 Definition files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61.3.2 Indexing methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81.3.3 OnDemand Web Enablement Kit . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

1.4 IBM Tivoli Workload Scheduler overview . . . . . . . . . . . . . . . . . . . . . . . . . 131.5 IBM Tivoli Workload Scheduler concepts . . . . . . . . . . . . . . . . . . . . . . . . . 16

1.5.1 IBM Tivoli Workload Scheduler for z/OS. . . . . . . . . . . . . . . . . . . . . . 171.5.2 IBM Tivoli Workload Scheduler for Distributed . . . . . . . . . . . . . . . . . 181.5.3 IBM Tivoli Workload Scheduler in an end-to-end configuration . . . . 20

Chapter 2. Integrating OnDemand and IBM Tivoli Workload Scheduler . 23

2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 242.2 The business case of integration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 252.2.1 Benefits of ITWS/OnDemand integration in a nutshell . . . . . . . . . . . 26

2.3 Distributed environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 272.3.1 Data to be collected . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

2.3.2 Data collection process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 282.4 z/OS and end-to-end environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

2.4.1 Data to be collected . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 292.4.2 Data collection process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

Chapter 3. IBM Tivoli Workload Scheduler for z/OS implementation . . . . 313.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 323.2 Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34


6/278

iv Integrating IBM Tivoli Workload Scheduler and Content Manager OnDemand

3.2.1 Single z/OS domain . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 343.2.2 Multiple z/OS domains . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

3.3 Prerequisites . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 363.3.1 Software versions and releases tested . . . . . . . . . . . . . . . . . . . . . . . 36

3.3.2 JES2 sysout classes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 363.3.3 Additional documentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 373.4 Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

3.4.1 Overview of the implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 383.4.2 Install the ARSSOCKD procedure . . . . . . . . . . . . . . . . . . . . . . . . . . 393.4.3 Download, unzip, and import the group and folder definitions . . . . . 403.4.4 Create user IDs and passwords for the OnDemand database . . . . . 453.4.5 Overview of the JES Spool data capture facility (ARSYSPIN) . . . . . 46

3.4.6 Install the ARSYSPIN started task . . . . . . . . . . . . . . . . . . . . . . . . . . 47

3.4.7 Install the ARSLOAD procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . 643.4.8 Install the ARSBATCH procedure. . . . . . . . . . . . . . . . . . . . . . . . . . . 663.4.9 Summary of started tasks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 683.4.10 Collecting ITWS for z/OS reports from the JES Spool . . . . . . . . . . 693.4.11 Collecting ITWS for z/OS reports from datasets . . . . . . . . . . . . . . . 70

3.4.12 ITWS for z/OS database and planning reports . . . . . . . . . . . . . . . . 713.4.13 Collecting IBM Tivoli Workload Scheduler for z/OS log files

(EQQMLOG). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73

Chapter 4. IBM Tivoli Workload Scheduler Distributed implementation . 774.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78

4.2 Prerequisites . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 794.2.1 Hardware requirements. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 804.2.2 Software requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 814.2.3 IBM Tivoli Workload Scheduler specific requirements . . . . . . . . . . . 814.2.4 Environmental variables requirements . . . . . . . . . . . . . . . . . . . . . . . 814.2.5 OnDemand specific requirements. . . . . . . . . . . . . . . . . . . . . . . . . . . 83

4.2.6 Download, unzip, and import the OnDemand definitions . . . . . . . . . 844.3 Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 884.3.1 Method type to use . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 894.3.2 Push method. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 904.3.3 Pull method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96

4.4 Scenarios . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109

4.5 Troubleshooting. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111

Chapter 5. IBM Tivoli Workload Scheduler end-to-end environment . . . 113

5.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1145.2 Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1145.3 Prerequisites . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115

5.4 Data collected . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115


7/278

Contents v

5.5 Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116

Chapter 6. Reporting: IBM Content Manager user interface . . . . . . . . . . 1176.1 IBM Content Manager OnDemand user interfaces . . . . . . . . . . . . . . . . . 1186.2 Using the OnDemand client program . . . . . . . . . . . . . . . . . . . . . . . . . . . 119

6.2.1 Starting the IBM OnDemand Interface (GUI) . . . . . . . . . . . . . . . . . 1196.2.2 Updating server information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121

6.2.3 Opening folders. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1216.2.4 Searching for documents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1256.2.5 Search with wildcards . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1286.2.6 Searching with dates and date ranges . . . . . . . . . . . . . . . . . . . . . . 1296.2.7 Search logical operators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134

6.2.8 Printing selected documents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135

6.2.9 Selecting and viewing documents. . . . . . . . . . . . . . . . . . . . . . . . . . 1366.2.10 How to select documents for viewing . . . . . . . . . . . . . . . . . . . . . . 1376.2.11 Saving and recalling successful queries . . . . . . . . . . . . . . . . . . . . 1406.2.12 Finding information in documents or notes . . . . . . . . . . . . . . . . . . 1426.2.13 Adding and viewing notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145

Chapter 7. Administration of the solution. . . . . . . . . . . . . . . . . . . . . . . . . 1517.1 Report administration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152

7.1.1 Storage sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153

7.1.2 Application groups. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1537.1.3 Applications. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161

7.1.4 Folders . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1657.2 User and group administration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171

7.2.1 User types, authorities, and functions . . . . . . . . . . . . . . . . . . . . . . . 1717.2.2 OnDemand system parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . 1737.2.3 Add a user. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1737.2.4 Organizing users into groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174

Appendix A. Scripts used in the solution . . . . . . . . . . . . . . . . . . . . . . . . . 177UNIX scripts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179Setup variables: tws_ondemand_env.sh . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179

jobmanrc. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180Main script: tws_ondemand_main.sh. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189Real script: tws_ondemand_real.sh . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189Logpush script: tws_ondemand_logpush.sh . . . . . . . . . . . . . . . . . . . . . . . . . 191Batch script: tws_ondemand_batch.sh. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194Monitor script: tws_ondemand_monitor.sh. . . . . . . . . . . . . . . . . . . . . . . . . . . 196Event Monitor script: tws_ondemand_event.sh . . . . . . . . . . . . . . . . . . . . . . . 197BmEvents.conf . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 200

Convert script: tws_ondemand_convert.sh (pull method) . . . . . . . . . . . . . . . 201Netman script: tws_ondemand_netman.sh . . . . . . . . . . . . . . . . . . . . . . . . . . 207


8/278

vi Integrating IBM Tivoli Workload Scheduler and Content Manager OnDemand

TWSmerge script: tws_ondemand_twsmerge.sh. . . . . . . . . . . . . . . . . . . . . . 207Database script: tws_ondemand_database.sh (pull method) . . . . . . . . . . . . 208Plan script: tws_ondemand_plan.sh. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 208Windows scripts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 209

tws_ondemand_netman.cmd . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 209tws_ondemand_twsmerge.cmd . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 209tws_ondemand_database.cmd . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 210

tws_ondemand_plan.cmd . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 210

Appendix B. Flowcharts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213Environmental variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 215Push method job stdlist real-time or batch mode . . . . . . . . . . . . . . . . . . . . . . 216

Push method batch mode netman, TWSmerge, database, and plan. . . . . . . 218

Push method batch mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221Pull method monitor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222Pull method event.log and real time . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223Pull method convert logs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 226Pull method UNIX netman . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234Pull method UNIX TWSmerge . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235Pull method UNIX database audit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 236Pull method UNIX plan audit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 237Pull method Windows Netman . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 238

Pull method Windows TWSmerge . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 239Pull method Windows database audit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 240

Pull method Windows plan audit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 241

Appendix C. Additional material . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 243Locating the Web material . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 243Using the Web material . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 243

System requirements for downloading the Web material . . . . . . . . . . . . . 244

How to use the Web material . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 244

Abbreviations and acronyms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 245

Related publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 247IBM Redbooks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 247Other publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 247Online resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 248How to get IBM Redbooks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 248

Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 249


9/278

Copyright IBM Corp. 2003. All rights reserved. vii

Figures

1-1 Typical OnDemand system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41-2 Relationship between folders, application groups, and applications . . . . 81-3 Applications and documents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101-4 IBM Tivoli Workload Scheduler for z/OS structure. . . . . . . . . . . . . . . . . 181-5 IBM Tivoli Workload Scheduler structure (distributed). . . . . . . . . . . . . . 20

1-6 IBM Tivoli Workload Scheduler end-to-end structure . . . . . . . . . . . . . . 223-1 OnDemand for z/OS architecture. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 333-2 OnDemand and TWS for z/OS (one JES domain) schematic . . . . . . . . 34

3-3 OnDemand and TWS for z/OS (two or more JES domains) schematic. 353-4 TWS4ZOS.ZIP file structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

3-5 OnDemand administrator panel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 453-6 ARSYSPIN (APKACIF) exit creation job flow . . . . . . . . . . . . . . . . . . . . 483-7 MSGCLASS files that have been collected for a given day. . . . . . . . . . 643-8 Supported DB and Planning reports for TWS for z/OS . . . . . . . . . . . . . 723-9 Example OnDemand screen showing the TWS for z/OS reports . . . . . 733-10 OnDemand screen showing the TWS for z/OS EQMLOGS collected. . 76

4-1 Our lab environment for the ITWS/OnDemand integration solution. . . . 80

4-2 TWS4DIST.ZIP. file structure. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 854-3 Scenario for each CPU pushing data to the OnDemand Server . . . . . 1104-4 Scenario showing a combination of Push and Pull methods . . . . . . . . 1115-1 ITWS and OnDemand integration (ITWS end-to-end environment) . . 1156-1 Starting the OnDemand client program . . . . . . . . . . . . . . . . . . . . . . . . 1196-2 Logging on to the OnDemand Server . . . . . . . . . . . . . . . . . . . . . . . . . 1206-3 Open a Folder window . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1226-4 Search Criteria and Document List dialog box. . . . . . . . . . . . . . . . . . . 123

6-5 Wildcard usage example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1296-6 Searching for documents operation: date and time range . . . . . . . . . . 1306-7 Searching for documents operation: a certain date . . . . . . . . . . . . . . . 131

6-8 Sort operation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1326-9 Print dialog . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1366-10 TWSMERGE logs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1386-11 Viewing the TWSMERGE log . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1396-12 Selecting the next page . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1406-13 Save a Named Query . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141

6-14 Select a Named Query. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1426-15 Find operation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1436-16 Adding a note to a document. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1466-17 Viewing a note . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148


10/278

viii Integrating IBM Tivoli Workload Scheduler Suite and Content Manager OnDemand

7-1 OnDemand end-user system flow . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1527-2 Application Group Database Information category . . . . . . . . . . . . . . . 1547-3 Annotation flags in document database table: Yes . . . . . . . . . . . . . . . 1557-4 Application group storage management . . . . . . . . . . . . . . . . . . . . . . . 1557-5 Application Group permissions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1577-6 Application group field information. . . . . . . . . . . . . . . . . . . . . . . . . . . . 1587-7 Type attribute. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159

7-8 OnDemand results using index field search . . . . . . . . . . . . . . . . . . . . 1597-9 OnDemand results using filter field search . . . . . . . . . . . . . . . . . . . . . 1607-10 Segment attribute. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1607-11 Application ID attribute. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1617-12 Application Indexer Information category. . . . . . . . . . . . . . . . . . . . . . . 162

7-13 Application Load Information: File Format . . . . . . . . . . . . . . . . . . . . . . 163

7-14 Application Load Information, Preprocessor Parameters . . . . . . . . . . 1637-15 Folder General information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1657-16 Display Document Location icons . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1667-17 Folder permissions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1677-18 Field Definition folder . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169

7-19 User types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1717-20 OnDemand default system parameters . . . . . . . . . . . . . . . . . . . . . . . . 1737-21 Add a user . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1747-22 Add a group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175

7-23 Groups and users list . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1767-24 tws_ondemand_env.sh flowchart. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 215

7-25 tws_ondemand_env.sh flowchart. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2167-26 Push method job stdlist real-time or batch mode flowchart . . . . . . . . . 2177-27 Push method batch mode netman, TWSmerge, database, and plan . 2187-28 Push method batch mode netman, TWSmerge, database, and plan . 2197-29 Push method batch mode netman, TWSmerge, database, and plan . 2207-30 Push method batch mode netman, TWSmerge, database, and plan . 221

7-31 Push method batch mode flowchart. . . . . . . . . . . . . . . . . . . . . . . . . . . 2227-32 Pull method monitor flowchart . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2237-33 Pull method event.log and real-time flowchart . . . . . . . . . . . . . . . . . . . 2247-34 Pull method event.log and real-time flowchart . . . . . . . . . . . . . . . . . . . 2257-35 Pull method event.log and real-time flowchart . . . . . . . . . . . . . . . . . . . 2267-36 Pull method convert logs flowchart . . . . . . . . . . . . . . . . . . . . . . . . . . . 227

7-37 Pull method convert logs flowchart . . . . . . . . . . . . . . . . . . . . . . . . . . . 2287-38 Pull method convert logs flowchart . . . . . . . . . . . . . . . . . . . . . . . . . . . 2297-39 Pull method convert logs flowchart . . . . . . . . . . . . . . . . . . . . . . . . . . . 230

7-40 Pull method convert logs flowchart . . . . . . . . . . . . . . . . . . . . . . . . . . . 2317-41 Pull method convert logs flowchart . . . . . . . . . . . . . . . . . . . . . . . . . . . 2327-42 Pull method convert logs flowchart . . . . . . . . . . . . . . . . . . . . . . . . . . . 2337-43 Pull method convert logs flowchart . . . . . . . . . . . . . . . . . . . . . . . . . . . 234


11/278

Figures ix

7-44 Pull method UNIX netman flowchart . . . . . . . . . . . . . . . . . . . . . . . . . . 2357-45 Pull method UNIX twsmerge flowchart . . . . . . . . . . . . . . . . . . . . . . . . 2367-46 Pull method UNIX audit database flowchart . . . . . . . . . . . . . . . . . . . . 2377-47 Pull method UNIX plan audit flowchart . . . . . . . . . . . . . . . . . . . . . . . . 2387-48 Pull method Windows Netman flowchart . . . . . . . . . . . . . . . . . . . . . . . 2397-49 Pull method Windows TWSmerge flowchart . . . . . . . . . . . . . . . . . . . . 2407-50 Pull method Windows database audit . . . . . . . . . . . . . . . . . . . . . . . . . 241

7-51 Pull method Windows plan audit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 242


12/278

x Integrating IBM Tivoli Workload Scheduler Suite and Content Manager OnDemand


13/278

Copyright IBM Corp. 2003. All rights reserved. xi

Tables

3-1 Additional documentation references. . . . . . . . . . . . . . . . . . . . . . . . . . . 373-2 APKACIF exit condition code handling . . . . . . . . . . . . . . . . . . . . . . . . . 513-3 ARSYSPIN keywords and descriptions . . . . . . . . . . . . . . . . . . . . . . . . . 523-4 ARSYSPIN JCL explained . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 616-1 Search operators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126

6-2 Search wildcards . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1286-3 T format string . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1336-4 Examples of using the T format string . . . . . . . . . . . . . . . . . . . . . . . . . 133

6-5 Options to view documents when AutoView is selected . . . . . . . . . . . 1377-1 Date format symbols . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164


14/278

xii Integrating IBM Tivoli Workload Scheduler Suite and Content Manager OnDemand


15/278

Copyright IBM Corp. 2003. All rights reserved. xiii

Notices

This information was developed for products and services offered in the U.S.A.

IBM may not offer the products, services, or features discussed in this document in other countries. Consultyour local IBM representative for information on the products and services currently available in your area.Any reference to an IBM product, program, or service is not intended to state or imply that only that IBMproduct, program, or service may be used. Any functionally equivalent product, program, or service thatdoes not infringe any IBM intellectual property right may be used instead. However, it is the user'sresponsibility to evaluate and verify the operation of any non-IBM product, program, or service.

IBM may have patents or pending patent applications covering subject matter described in this document.The furnishing of this document does not give you any license to these patents. You can send licenseinquiries, in writing, to:IBM Director of Licensing, IBM Corporation, North Castle Drive Armonk, NY 10504-1785 U.S.A.

The following paragraph does not apply to the United Kingdom or any other country where such provisionsare inconsistent with local law: INTERNATIONAL BUSINESS MACHINES CORPORATION PROVIDESTHIS PUBLICATION "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED,INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF NON-INFRINGEMENT,MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Some states do not allow disclaimerof express or implied warranties in certain transactions, therefore, this statement may not apply to you.

This information could include technical inaccuracies or typographical errors. Changes are periodically madeto the information herein; these changes will be incorporated in new editions of the publication. IBM may

make improvements and/or changes in the product(s) and/or the program(s) described in this publication atany time without notice.

Any references in this information to non-IBM Web sites are provided for convenience only and do not in anymanner serve as an endorsement of those Web sites. The materials at those Web sites are not part of thematerials for this IBM product and use of those Web sites is at your own risk.

IBM may use or distribute any of the information you supply in any way it believes appropriate withoutincurring any obligation to you.

Information concerning non-IBM products was obtained from the suppliers of those products, their publishedannouncements or other publicly available sources. IBM has not tested those products and cannot confirm

the accuracy of performance, compatibility or any other claims related to non-IBM products. Questions onthe capabilities of non-IBM products should be addressed to the suppliers of those products.

This information contains examples of data and reports used in daily business operations. To illustrate themas completely as possible, the examples include the names of individuals, companies, brands, and products.All of these names are fictitious and any similarity to the names and addresses used by an actual businessenterprise is entirely coincidental.

COPYRIGHT LICENSE:This information contains sample application programs in source language, which illustrates programmingtechniques on various operating platforms. You may copy, modify, and distribute these sample programs inany form without payment to IBM, for the purposes of developing, using, marketing or distributing application

programs conforming to the application programming interface for the operating platform for which thesample programs are written. These examples have not been thoroughly tested under all conditions. IBM,therefore, cannot guarantee or imply reliability, serviceability, or function of these programs. You may copy,modify, and distribute these sample programs in any form without payment to IBM for the purposes ofdeveloping, using, marketing, or distributing application programs conforming to IBM's applicationprogramming interfaces.


16/278

xiv Integrating IBM Tivoli Workload Scheduler Suite 8.2 and Content Manager OnDemand to provide

Trademarks

The following terms are trademarks of the International Business Machines Corporation in the United States,other countries, or both:

Advanced FunctionPresentationAFPAIXAS/400CICSDB2 Universal DatabaseDB2^Infoprint

IBMiSeriesLanguage EnvironmentLotusMaestroMVSNetViewNotesOS/390OS/400

Print Services FacilityRedbooksRedbooks (logo) S/390Tivoli EnterpriseTivoli Enterprise ConsoleTivoliVTAMWebSpherez/OS

The following terms are trademarks of other companies:

Java and all Java-based trademarks and logos are trademarks or registered trademarks of SunMicrosystems, Inc. in the United States, other countries, or both.

UNIX is a registered trademark of The Open Group in the United States and other countries.

Other company, product, and service names may be trademarks or service marks of others.


17/278

Copyright IBM Corp. 2003. All rights reserved. xv

Preface

This IBM Redbook implements a solution that integrates IBM Tivoli WorkloadScheduler and IBM DB2 Content Manager OnDemand products to providecentralized output processing for job logs (job outputs, message files, and auditfiles) from IBM Tivoli Workload Scheduler and IBM Tivoli Workload Scheduler forz/OS. The solution provides immediate benefit by integrating the job logs into anonline, electronic information archive and retrieval system, which is used forquick search and problem resolution purposes.

As part of the solution, we cover defining and implementing the required

infrastructure, to access these reports from a simple user interface. We alsoinclude examples and scenarios for using this solution.

We include all scripts that make up this solution so that you will be ablecustomize the solution according to your needs.

We anticipate that the solution covered in this book will provide great value forIBM Tivoli Workload Scheduler and IBM Tivoli Workload Scheduler for z/OScustomers who are planning to deploy a centralized job logging and browsing

system.

The team that wrote this redbook

This redbook was produced by a team of specialists from around the worldworking at the International Technical Support Organization, Austin Center.

Vasfi Gucer is an IBM Certified Consultant IT Specialist working at the

International Technical Support Organization, Austin Center. He worked withIBM Turkey for 10 years, and has been with the ITSO since January 1999. Hehas more than 10 years of experience in systems management, networkinghardware, and distributed platform software. He has worked on various Tivolicustomer projects as a Systems Architect in Turkey and the U.S. Vasfi is also aCertified Tivoli Consultant.

Cheryl Brown is an independent consultant for the IBM DB2 Content ManagerOnDemand for Multiplatforms product. She is an IBM retiree who became a

contractor working on the development of OnDemand education for seven years,beginning in 1996. In addition to education, she has also provided consulting toIBM customers both in the US and abroad.


18/278

xvi Integrating IBM Tivoli Workload Scheduler Suite and Content Manager OnDemand

Budi Darmawan is a Tivoli Specialist at the International Technical SupportOrganization, Austin Center. He writes extensively and teaches IBM classesworldwide on Tivoli, DB2 database, and OS/390. Before joining the ITSO inFebruary 1999, he worked in IBM Global Services, Indonesia as the lead solutionarchitect for Tivoli system management and business intelligence services. Budi

is also a Tivoli Certified Instructor and a Tivoli Certified Enterprise Consultant.

Doroti Almeida Dias Garcia is IT Specialist working in Tivoli Customer Supportin Brazil. She has three years of experience with Tivoli products, focusing onTivoli Workload Scheduler and User Administration. She holds a degree inMathematical Science and she has an MBA in E-Management InformationTechnology from Fundao Getulio Vargas. Her areas of expertise includes theAIX operational system.

Tina Lamacchia has been in the IT realm for 18 years. Her career started withprogramming, and then administrating hundreds of UNIX systems. She hasbeen with Tivoli Systems for six years. She has worked with IBM Tivoli WorkloadScheduler for nine years as a customer and as an IBM Tivoli WorkloadScheduler advocate. Currently, she is working in the SWAT team in Austin toassist with sales of various deployments in IBM Tivoli Workload Scheduler andall PACO products.

Martin Pepper does EMEA pre-sales technical support for OnDemand for

Multiplatforms and is based out of Warwick in the United Kingdom. He has fouryears of experience in the Content Management field. He holds a Mastersdegree in Computer Science from the University of Manchester. Martin is an IBMcertified AIX system administrator and a certified solution expert for OnDemandfor Multiplatforms.

Pete Soto is currently the L2 Senior Software Engineer for IBM Tivoli WorkloadScheduler. He has worked for IBM for nine years in training, implementing, andsupporting IBM Tivoli Workload Scheduler. He also supported the Tivoli Software

Distribution and Tivoli Inventory products. He is Tivoli Certified for TivoliFramework, Tivoli Software Distribution, Tivoli Inventory, and Tivoli WorkloadScheduler. He has a BBA with a Major in Accounting and a Minor in InformationSystems from the University of Texas at San Antonio.

Michael R. Tucci started with IBM Tivoli Software in Austin in 1999 as an L2Software Engineer for IBM Tivoli Workload Scheduler. He has done extensivetravel to customer sites as well as spent time in Rome, Italy testing IBM TivoliWorkload Scheduler 8.2 in a Windows environment. He also assisted with thecompletion of the IBM Tivoli Workload Scheduler Installation Guide. Currently, heis responsible for Austins L2s test environment with Windows, UNIX and Linux.

Bob Watling is a Systems Management Technical Consultant based in theUnited Kingdom with more than 30 years of Systems Management experience.


19/278

Preface xvii

He has been the IBM representative at the UK-based OPC user group for morethan 15 years and has accumulated a vast amount of knowledge and experienceover this period of time in the area of batch management.

Thanks to the following people for their contributions to this project:

Wade WallaceInternational Technical Support Organization, Austin Center

Wei-Dong Jackie ZhuInternational Technical Support Organization, San Jose Center

Robert HaimowitzInternational Technical Support Organization, Poughkeepsie Center

Jackie Biggs, Gregory Felderman, Warren Gill, Louise E. HawleyIBM USA

Fabio Barillari, Maria Pia Cagnetta, Antonio Di Cocco, Riccardo Colella, PietroIannucci, Valeria PerticaraIBM Italy

Become a published authorJoin us for a two- to six-week residency program! Help write an IBM Redbookdealing with specific products or solutions, while getting hands-on experiencewith leading-edge technologies. You'll team with IBM technical professionals,Business Partners and/or customers.

Your efforts will help increase product acceptance and customer satisfaction. Asa bonus, you'll develop a network of contacts in IBM development labs, andincrease your productivity and marketability.

Find out more about the residency program, browse the residency index, andapply online at:

ibm.com/redbooks/residencies.html

Comments welcome

Your comments are important to us!

We want our Redbooks to be as helpful as possible. Send us your commentsabout this or other Redbooks in one of the following ways:
http://www.redbooks.ibm.com/residencies.htmlhttp://www.redbooks.ibm.com/residencies.htmlhttp://www.redbooks.ibm.com/residencies.htmlhttp://www.redbooks.ibm.com/residencies.html


20/278

xviii Integrating IBM Tivoli Workload Scheduler Suite and Content Manager OnDemand

Use the online Contact us review redbook form found at:

ibm.com/redbooks

Send your comments in an Internet note to:

[email protected]

Mail your comments to:

IBM Corporation, International Technical Support OrganizationDept. JN9B Building 003 Internal Zip 283411400 Burnet RoadAustin, Texas 78758-3493
http://www.redbooks.ibm.com/http://www.ibm.com/redbooks/http://www.redbooks.ibm.com/contacts.htmlhttp://www.redbooks.ibm.com/contacts.htmlhttp://www.ibm.com/redbooks/http://www.ibm.com/redbooks/http://www.redbooks.ibm.com/


21/278

Copyright IBM Corp. 2003. All rights reserved. 1

Chapter 1. Overview and benefits

In this chapter, we provide an overview of the IBM DB2 Content ManagerOnDemand (OnDemand) system and the IBM Tivoli Workload Scheduler

product. We also provide information that can help you better understand howthese products work. This information is important, because you have to have agood understanding of both products to be able to successfully implement theIBM DB2 Content Manager OnDemand and IBM Tivoli Workload Schedulerintegration solution (hereafter referred to as the ITWS/OnDemand integrationsolution). This integration is the main topic of this book. If you already haveexperience with both products, you can skip this chapter and go to Chapter 2,Integrating OnDemand and IBM Tivoli Workload Scheduler on page 23.

First, we describe how OnDemand manages reports and indexes data. We haveincluded important information about how OnDemand, its database manager,and the storage manager work together to index, load, and retrieve reports andlog files. Also, we provide a list of tasks that OnDemand administrators typicallyperform to manage an OnDemand system.

Second, we describe how IBM Tivoli Workload Scheduler is structured and how itcan schedule your entire enterprise.

The following sections will be covered in this chapter:

Section 1.1, IBM DB2 Content Manager OnDemand overview on page 3 Section 1.2, Servers on page 5 Section 1.3, OnDemand concepts on page 6

1


22/278

2 Integrating IBM Tivoli Workload Scheduler Suite and Content Manager OnDemand

Section 1.4, IBM Tivoli Workload Scheduler overview on page 13 Section 1.5, IBM Tivoli Workload Scheduler concepts on page 16


23/278

Chapter 1. Overview and benefits 3

1.1 IBM DB2 Content Manager OnDemandoverview

IBM DB2 Content Manager OnDemand is an archival and retrieval system fordocuments, log files, and reports. It provides online access to hard copy and isan excellent tool for microfiche replacement, providing instant access toinformation. An OnDemand system can support small office environments andlarge enterprise installations with hundreds of system users. It can dramaticallyimprove productivity and customer service in many businesses by providing fastaccess to information stored in the system.

OnDemand processes the print output of application programs, extracts indexfields from the data, stores the index information in a relational database, andstores one or more copies of the data in the system. With OnDemand, you canarchive newly created and frequently accessed reports on high speed, diskstorage volumes and automatically migrate them to other types of storagevolumes as they age.

OnDemand fully integrates the capabilities of the Advanced Function

Presentation (AFP) data stream, including management of resources,indexes, and annotations, and supports full fidelity reprinting and faxing of files todevices attached to a PC, OnDemand Server, or other server on the network.

OnDemand provides administrators with tools to manage OnDemand Servers,authorize users to access OnDemand Servers and data stored in the system,and back up the database and data storage.

OnDemand provides users the ability to view, print, send, attach electronic notes,

and fax copies of log files and reports.

OnDemand offers several advantages to IBM Tivoli Workload Scheduler users; itcan:

Easily locate data without specifying the exact report.

Retrieve the pages of the report that you need without processing the entirereport.

View selected data from within a report.

Note: Part of this chapter has been taken from the redbook Content ManagerOnDemand Guide, SG24-6915. For more information on IBM DB2 ContentManager OnDemand, you can refer to this redbook.


24/278


OnDemand provides an information management tool that will increase youreffectiveness when working with customers. It achieves this by doing thefollowing:

Integrates data created by application programs into an online, electronicinformation archive and retrieval system.

Provides controlled and reliable access to all reports of an organization.

Retrieves data when needed.

Provides a standard, intuitive client with features such as thumbnails,bookmarks, notes, and shortcuts.

These features mean that OnDemand can help you quickly retrieve the specificpage of a report that you need to provide fast customer service.

An OnDemand system consists of client programs and server programs thatcommunicate over a network running the TCP/IP communications protocol. Anexample of a typical OnDemand system is shown in Figure 1-1.

Figure 1-1 Typical OnDemand system

OnDemand client programsOnDemand client programs run on CICS terminals and Windows operatingsystem PCs attached to a network communicating with the OnDemand Servers.

Using the client program, users construct queries to initiate the search and thenare presented with a list of documents that match the query. Once a documentlist has been displayed, users have the following options:


25/278


View

Print (all or some pages)

Fax (all or some pages)

Attach electronic notes to pages of a document.

1.2 Servers

The OnDemand Server environment includes a library server and one or moreobject servers residing on one or more nodes connected to a TCP/IP network.An OnDemand Server is supported in one of the following operating systems:

AIX

Sun OS HPUX OS/400 z/OS

OnDemand library serverThe OnDemand library server manages control information in the databaseabout the OnDemand users and the reports stored on the system. The databasemanager provides the database engine and utilities to administer the database.The library server processes client logons, queries, and print requests andupdates to the database. The major functions that run on the library server arethe request manager, the database manager, and the server print manager.

OnDemand object server

The OnDemand object server maintains documents on cache storage volumesand, optionally, works with an archive storage manager to maintain documentson archive media, such as optical and tape storage libraries. An object serverloads data, retrieves documents, and expires documents. The major functionsthat run on an object server are the cache storage manager, OnDemand dataloading and maintenance programs, and optionally, the archive storage manager.

An OnDemand system consists of one or more object servers. An object servercan operate on the same server as the library server, or on a separate serverthan the library server.

When a user submits a query, the client program sends a search request to theOnDemand library server. The library server returns a list of the documents thatmatch the query to the user. When the user selects a document for viewing, the

Note: An OnDemand system consists of only one library server.


26/278


client program retrieves a copy of the document from the object server where thedocument is stored, opens a viewing window, and displays the document.

1.3 OnDemand conceptsIn order to load and view report files or log files, OnDemand definition files mustbe created. The terms application, application group, andfolderrepresent thesedefinition files. It is in these files that the determination is made as to howOnDemand stores, manages, retrieves, views, and prints reports and index data.When defining a new report or type of data to OnDemand, an administrator mustcreate an application and assign the application to an application group. Youmay define one or more applications per application group.

Before users can search for and retrieve documents, an administrator mustcreate or update a folder to use the application group and application.

1.3.1 Definition files

The following gives an introduction to the OnDemand definition files.

ApplicationEach application represents a unique report type. An application defines thefollowing areas to OnDemand:

File characteristics Describes the input file characteristic, that is, format of thedata, the orientation of data on the page, the paper size,the record length, and the code page.

Index information Includes parameters used by the indexing program tolocate and extract index data.

Load instructions Includes both pre- and post-processing parameters whenloading index data in the database, date format settings,and data compression selection.

Note: If an application group does not exist, the administrator must first createone.


27/278


Application groupWhen searching for a report or log file, a SQL query is run against an applicationgroup database table. An application group defines three key areas toOnDemand:

Database tables This includes retention of the indexes, database fieldattributes as well as database organization, table length,and size.

Data storage This includes retention period of the data for short-termdata storage and interfacing with storage manager forlong-term data storage.

Access Assigns access permissions to users or groups of usersto query the database and access the report files.

When a report is loaded into OnDemand, you must identify the application groupwhere OnDemand will load the index data and store the documents. Anapplication group is a collection of one or more OnDemand applications withcommon indexing and storage management attributes. You typically groupseveral different reports in an application group so that users can access theinformation contained in the reports with a single query.

Folder

A folder provides users with a convenient way to find related information stored inOnDemand, regardless of the source of the information or how the data wasprepared. A folder is similar to a directory structure in a server where the fileswithin that directory are virtual fi les, that is, a link to the actual physical location ofthe files are accessible. The following areas are defined through the folder:

Query fields Defines which database fields within the application groupwill be searched and all attributes about those fields, forexample, default value and search operators (Equal To orLess Than, and so on)

Access Assign access permissions to users or groups of users toquery the database and access the report files.

A folder allows an administrator to set up a common query screen for severalapplication groups that may use different indexing schemes so that a user canretrieve the data with a single query. For example, a folder called Log Files mightcontain NETMAN, JOBMAN, and MAILMAN entries in addition to Standard ListLog Files, which represents information stored in different application groups,

defined in different applications, and created by different programs.Figure 1-2 on page 8 illustrates the concepts described in this section.


28/278


Figure 1-2 Relationship between folders, application groups, and applications

1.3.2 Indexing methods

OnDemand provides two methods of indexing data

Document indexingis used for reports that contain logical items, such as logfile information. Each of the items in a report can be individually indexed onvalues, such as server job number, job name, start date and time, and enddate and time. With document indexing, the user does not necessarily need toknow about reports or report cycles to retrieve a document from OnDemand.

Report indexingis used for reports that contain many pages of the same kind

of data, such as a merge transaction log. Each line in the report usuallyidentifies a specific transaction, and it would not be cost effective to indexeach line. OnDemand stores the report as groups of pages and indexes eachgroup. When reports include a sorted transaction value (for example, startdate), OnDemand can index the data on the transaction value. This is doneby extracting the beginning and ending transaction values for each group ofpages and storing the values in the database. This type of indexing lets usersretrieve a specific transaction value directly.

The reports that are stored in OnDemand must be indexed. OnDemand supportsseveral types of index data and indexing programs. The AFP Conversion andIndexing Facility (ACIF) can be used to index S/390 line data, ASCII data, andAFP files, collect resources required to view the reports, and convert line data

Log File

App Group

Log File

Application

NETMAN

Application

MAILMAN

Application

Log Files Folder

Merge File

App Group

JOBMAN

Application


29/278


files to AFP data. The OS/400 Indexer can be used to index a variety of datatypes and is the most common OnDemand indexer for OS/400 spooled files. TheOnDemand PDF Indexer can be used to create index data for Adobe AcrobatPDF files. The OnDemand Generic Indexer can be used to create index data foralmost any other type of data, such as HTML documents, Lotus WordPro

documents, and TIFF files.

For OS/400, there is the OS/400 Indexer that can index a variety of data types forOS/400 spooled files. Refer to IBM Content Manager OnDemand forMultiplatforms Indexing Reference Version 7.1, SC27-0842, IBM Content

Manager OnDemand for iSeries Common Server: Indexing Reference,SC27-1160, and IBM Content Manager OnDemand for z/OS and OS/390:Indexing Reference, SC27-1375for details about the indexing programsprovided with OnDemand for various platforms.

DocumentsOnDemand documents represent indexed groups of pages. Typically, anOnDemand document is a logical section of a larger report, such as a log filewithin a report of hundreds of log files. An OnDemand document can also

represent a portion of a larger report. For reports that do not contain logicalgroups of pages, such as merge transaction logs, OnDemand can divide thereport into groups of pages. The groups of pages are individually indexed andcan be retrieved much more efficiently than the entire report. Documents areusually identified by date, with one or more other fields, such as CPU name.

Figure 1-3 on page 10 illustrates OnDemand applications and documents. Anadministrator could define the Log File application for a report that containslogical items. The Log File application uses the document indexing method to

divide the report into documents. Each log file in the report becomes a documentin OnDemand. Users can retrieve a statement by specifying the date and anycombination of name and number.

Note: The ITWS/OnDemand solution uses ACIF. See Indexer information onpage 161 for a more detailed explanation about indexing.


30/278


Figure 1-3 Applications and documents

An administrator could define the MERGE FILE application for a report thatcontains lines of sorted transaction data. The MERGE FILE application uses the

report indexing method to divide the report into documents. Each group of 100pages in the report becomes a document in OnDemand. Each group is indexedusing the first and last sorted transaction values that occur in the group (in themerge file the start date is used as a transaction value). Users can retrieve thegroup of pages that contains a specific start date and specifying the CPU name.OnDemand retrieves the group that contains the value entered by the user.

The basic OnDemand configuration is a library server and an object server onthe same physical system or node. This single library/object server configuration

supports the database functions and cache storage on one system. You can addan archive storage manager to the single library/object server configuration tomaintain documents on archive media.

You can also configure your OnDemand system with a library server on one nodeand one or more object servers on different nodes. This configuration is knownas a distributed library/object server system. The distributed library/object serverconfiguration supports caching of documents on different servers. You can addan archive storage manager to one or more of the object servers to maintain

documents on archive media attached to different servers.

Std List Log File

20030716

Log FileApplication

Job NumberCPU Name

Job Name

Start Date

End Date

Merge File

20030716

Merge FileApplication

CPU Name

Start Date

End Date

Job #28384 Job #35674

Page 1 to 100


31/278


An OnDemand Server environment contains several components:

A request managerthat provides client, network, and operating systemservices, security, and accounting. The request manager resides on thelibrary server.

A database managerthat maintains the index data for the reports that youstore on the system. The database manager is a relational databasemanagement product, such as DB2. The database manager resides on thelibrary server.

Database control information about the users, groups, application groups,applications, folders, storage sets, and printers that you define on the system.The control information determines who can access the system, the foldersthat a user can open, and the application group data that a user can queryand retrieve. The database resides on the library server.

A cache storage managerthat maintains documents in cache storage. Cachestorage is for high-speed access to the most frequently used documents.

An archive storage manager, which is an optional part of the system. Thearchive storage manager is for the long-term storage of one or more copies ofdocuments on archive media, such as optical and tape storage libraries.

A download facility that automatically transfers spool files to a server at highspeed. We recommend that you use Download for OS/390, a licensed featureof Print Services Facility (PSF) for OS/390. Download provides theautomatic, high-speed download of JES Spool files from an OS/390 system toOnDemand Servers. The download facility is not applicable to iSeries.

Data loading programs that can be set up to automatically store report datainto application groups and update the database. The data loading programscan run on any OnDemand Server.

Archivedreports and resources.

Aserver printfacility that allows users to reprint a large volume of documents

at high speed. OnDemand uses Infoprint, which must be purchasedseparately, to manage the server print devices.

OnDemand management programs to maintain the OnDemand database anddocuments in cache storage.

Asystem loggingfacility that provides administrators with tools to monitorserver activity and respond to specific events as they occur. The interface tothe system logging facility is through the system log folder and the system loguser exit.

Management programsOnDemand provides programs to maintain and optimize the database andmaintain documents on cache storage. An administrator usually determines the


32/278


processing parameters for these programs, including the frequency with whichthe programs should run. When you create an application group, you specifyother parameters that these programs use to maintain the report data stored inthe application group.

For example, when creating an application group, the administrator specifies howlong documents should be maintained on the system and whether index datashould be migrated from the database to archive media. The programs use theinformation to migrate documents from cache storage to archive media, deletedocuments from cache storage, migrate index data from the database to archivemedia, and delete index data from the database. These functions are usefulbecause OnDemand can reclaim the database and cache storage spacereleased by expired and migrated data. We recommend that you configure yourOnDemand system to automatically start these management programs on a

regular schedule, usually once every night or week.

The archive storage manager deletes data from archive media when it reachesits storage expiration date. An administrator defines management information tothe archive storage manager to support the OnDemand data it manages. Themanagement information includes the storage libraries and storage volumes thatcan contain OnDemand data, the number of copies of a report to maintain, andhow long to keep data in the archive management system.

OnDemand and the archive storage manager delete data independently of eachother. Each uses its own criteria to determine when to remove documents. Eachuses its own utilities and schedules to remove documents. However, for finalremoval of documents from the system, you should specify the same criteria toOnDemand and the archive storage manager.

1.3.3 OnDemand Web Enablement Kit

The OnDemand Web Enablement Kit (ODWEK) is an optional feature of

OnDemand that allows people to use a Web browser to access data stored in anOnDemand system. For example, you can provide some people with the UniformResource Locator (URL) of a Web page that permits them to log on to anOnDemand Server; you can provide other people with the URL of a Web pagethat permits them to search a specific folder. ODWEK verifies that the userinformation is valid on the OnDemand Server, such as permission to access theserver and data stored in an application group. After the user submits a search,ODWEK displays a Web page that contains a list of the documents that matchthe query. The user selects a document to view and ODWEK sends the

document to the browser.

ODWEK t i l t


33/278


ODWEK contains several components:

The Web server program. The server program uses standard OnDemandinterfaces and protocols to access data stored in an OnDemand Server. Noadditional code is needed on the OnDemand Server to support ODWEK. Youcan use one of the following Web server programs to control ODWEK:

CGI program. The CGI program runs against an HTTP server, such as theIBM HTTP Server.

Java servlet. The servlet runs on a Java-enabled HTTP server with aJava application server, such as the IBM WebSphere Application Server.

TheAFP Web Viewer. The AFP Web Viewer lets users search, retrieve, view,navigate, and print AFP documents from a Web browser. The AFP WebViewer must be installed on the viewing workstations.

The Image Web Viewer. The Image Web Viewer lets users search, retrieve,view, navigate, and print BMP, GIF, JPEG, PCX, and TIFF documents from aWeb browser. The Image Web Viewer must be installed on the viewingworkstations.

TheLine Data Java applet. The Line Data applet lets users view line datadocuments from a Web browser.

TheAFP2HTML Java applet. The AFP2HTML applet lets users view theoutput generated by the IBM AFP2WEB Transform service offering. The

AFP2WEB Transform converts AFP documents and resources into HTMLfiles that can be displayed with the AFP2HTML applet. If you plan to use theAFP2HTML applet, then you must obtain the AFP2WEB Transform from IBMand install and configure it on the Web server.

1.4 IBM Tivoli Workload Scheduler overview

The importance of end-to-end business systems management is recognized

across the industry; batch workload is the backbone of business applications andtherefore drives the business. As a crucial part of business systems, theworkload must be managed seamlessly from end to end.

IBM Tivoli Workload Scheduler is the state-of-the-art production workloadmanager that is designed to help you meet your present and future dataprocessing challenges. Its scope encompasses your entire enterpriseinformation system, including heterogeneous environments.

The workload scheduler suite simplifies systems management across


34/278


The workload scheduler suite simplifies systems management acrossheterogeneous environments by integrating systems management functions.There are three main components to the suite:

Tivoli Workload Scheduler for z/OS (The scheduler in OS/390 and z/OSenvironments)

Tivoli Workload Scheduler (The scheduler in distributed environments)

Tivoli Job Scheduling Console (The common user interface for both TivoliWorkload Scheduler for z/OS and Tivoli Workload Scheduler)

Automatic driverIBM Tivoli Workload Scheduler provides leading-edge solutions to problems inproduction workload management. It can automate, plan, and control theprocessing of your enterprises entire production workload, not just the batchsubset. IBM Tivoli Workload Scheduler functions as an automatic driver foryour production workload to maximize the throughput of work and optimize yourresources, but also allows you to intervene manually as required. When IBMTivoli Workload Scheduler interfaces with other system management products, itforms part of an integrated automation and systems management platform foryour DP operation.

IBM Tivoli Workload Scheduler can manage the complete flow of work throughyour enterprises entire DP operation, on both local and remote systems.

Single point of controlFrom a single point of control, IBM Tivoli Workload Scheduler analyzes thestatus of the production work and drives the processing of the workloadaccording to installation business policies. It supports a multiple-end-userenvironment, enabling distributed processing and control across sites anddepartments within your enterprise.

Enterprise application integrationITWS provides advanced integration with enterprise applications like SAP R/3,Oracle Applications, and PeopleSoft. ITWSs underlying architecture ensuresthat business and mission-critical applications run within an infrastructure that issecure, fault-tolerant, and scalable.

Systems management integrationSolutions to todays systems management problems require an integration of

application programs and processes. IBM Tivoli Workload Scheduler interfacesdirectly with some of the z/OS products as well as with a number of other IBMproducts to provide a comprehensive, automated processing facility and anintegrated approach for the control of complex production workloads.

For example:


35/278


For example:

IBM DB2 Content Manager OnDemandhas the ability to archive not only theoutput from jobs being processed under the control of IBM Tivoli WorkloadScheduler but also the logs and reports produced by the scheduler itself.Using the Windows client or WEB access feature, users can historically

search the DB2 database by specifying the same user defined keywords asthose used when the data was archived. Searching for archived output byJobname, date and time of run, completion code, or even freeform strings areall possible.

Workload Scheduler sends workload plan information andIBM TivoliBusiness Systems Manageris the solution to unifying the management ofbusiness systems. IBM Tivoli Workload Scheduler for z/OS has beenenhanced to support monitoring from Tivoli Business Systems Manager.

From Tivoli Business Systems Manager, you can monitor status changes tojobs, addition of jobs to the plan, and alert conditions relating to jobscheduling.

Workload Scheduler can initiate software distributions done by IBM TivoliConfiguration Manager; Configuration Manager can install and deployWorkload Scheduler Agents.

Workload Scheduler for z/OS can send job related information toInformationManagementfor service related interactions. Information Managementsupports the administration of the systems management process of anenterprises hardware, software, and related resources. An interface withInformation Management is provided for reporting problems detected whileprocessing the production workload.

Workload Scheduler schedules IBM Tivoli Data Exchange to manage thecustomers data movement among systems.

Integrating Workload Scheduler with IBM Tivoli NetView gives networkmanagers the ability to monitor and diagnose Workload Scheduler networksfrom a NetView management node. It includes a set of submaps and symbolsto view scheduler networks topographically, and determine the status of jobscheduling activity and critical scheduler processes on each workstation.Menu actions are provided to start and stop scheduler processing, and to runconman on any workstation in the network.

System Automation for 390 (SA/390)initiates automation procedures thatperform operator functions to manage OS/390 components, data sets, andsubsystems. SA/390 includes an automation feature for IBM Tivoli WorkloadScheduler for z/OS.

TheIBM Tivoli Enterprise Console (TEC) is a powerful, rules-based eventmanagement application that integrates network, systems, database, andapplication management. It offers a centralized, global view of your computing

enterprise while ensuring the high availability of your application and


36/278


enterprise while ensuring the high availability of your application andcomputing resources. Tivoli Enterprise Console acts as a central collectionpoint for alarms and events from a variety of sources, including those fromTivoli applications. IBM Tivoli Workload Scheduler runs a Tivoli EnterpriseConsole adapter that reads events from the IBM Tivoli Workload Scheduler

log file. Enterprise Console in turn can initiate actions in Workload Schedulerbased upon the collective correlation of other events.

Workload Scheduler can initiate and monitor workstation backup andrecovery activities by integrating withIBM Tivoli Storage Manager.

IBM Tivoli Monitoringcan report on the status and health of WorkloadSchedulers management processes and queue files.

IBM Tivoli Decision Support for z/OShelps you effectively manage theperformance of your system by collecting performance data in a database

and presenting the data in a variety of formats for use in systemsmanagement. Decision Support uses data from IBM Tivoli WorkloadScheduler for z/OS to produce summary and management reports about theproduction workload, both planned and actual results.

Removable Media Manager (RMM) works with restart and cleanup to assistwith the management of data sets in a z/OS environment. Removable MediaManager verifies that a data set exists on a volume and then takesuser-requested actions on the data set. Consequently, IBM Tivoli WorkloadScheduler for z/OS works with Removable Media Manager to properly markand expire data sets as needed.

Workload Scheduler will store batch job run statistics such as job duration,CPU usage, and status information in theIBM Tivoli Data Warehouse. Thereare many predefined reports provided with the Data Warehouse for mostcommon reporting needs

This document is specifically targeted at the integration between OnDemand andIBM Tivoli Workload Scheduler and/or IBM Tivoli Workload Scheduler for z/OS.

1.5 IBM Tivoli Workload Scheduler concepts

This topic summarizes the differences between IBM Tivoli Workload Schedulerfor z/OS, IBM Tivoli Workload Scheduler for Distributed, and IBM Tivoli WorkloadScheduler for an end-to-end configuration. For a more detailed explanation, youcan refer to the End-to-End Scheduling with Tivoli Workload Scheduler 8.1,SG24-6022redbook or refer to the IBM Tivoli Workload Scheduler and IBM Tivoli

Workload Scheduler for z/OS product manuals directly.

1.5.1 IBM Tivoli Workload Scheduler for z/OS


37/278


1.5.1 IBM Tivoli Workload Scheduler for z/OS

IBM Tivoli Workload Scheduler for z/OS has been scheduling and controllingbatch workloads in data centers since 1977. Originally called OperationsPlanning and Control (OPC), the product has been extensively developed andextended to meet the increasing demands of customers worldwide. An overnightworkload consisting of 100,000 production jobs is not unusual, and IBM TivoliWorkload Scheduler for z/OS can easily manage this kind of workload.

IBM Tivoli Workload Scheduler for z/OS databases contains information aboutthe work that is to be run, when it should run, and the resources that are neededand available. This information is used to calculate a forward forecast called thelong term plan. The data center staff can check this forecast to confirm that thedesired work is being scheduled when required. The long term plan usuallycovers a period of time from four to twelve weeks. A second plan is producedthat uses the long term plan and databases as input. The current plan usuallycovers 24 hours and is a detailed production schedule. IBM Tivoli WorkloadScheduler for z/OS uses this to submit jobs to the appropriate processor at theappropriate time. All jobs in the current plan have IBM Tivoli Workload Schedulerfor z/OS status codes that indicate the progress of work. When a jobspredecessors are complete, IBM Tivoli Workload Scheduler for z/OS considers itready for submission. It checks that all requested resources are available, andwhen these conditions are met, it causes the job to be submitted.

StructureIBM Tivoli Workload Scheduler for z/OS consists of a controller and one or moretrackers (Figure 1-4 on page 18). The controller runs on an z/OS system. Thecontroller manages the IBM Tivoli Workload Scheduler for z/OS databases andthe long term and current plans. The controller schedules work and cause jobs tobe submitted to the appropriate system at the appropriate time. Trackers areinstalled on every system managed by the controller. The tracker is the linkbetween the controller and the managed system. The tracker submits jobs when

the controller instructs it to do so, and it passes job start and job end informationback to the controller. The controller can schedule jobs on z/OS systems as wellas distributed systems. The IBM Tivoli Workload Scheduler for z/OS controlleruse fault tolerant workstations for job scheduling on distributed systems. A faulttolerant workstation is actually a IBM Tivoli Workload Scheduler Fault TolerantAgent. For more details on the use of fault tolerant workstations, see 1.5.3, IBMTivoli Workload Scheduler in an end-to-end configuration on page 20.


38/278


Figure 1-4 IBM Tivoli Workload Scheduler for z/OS structure

1.5.2 IBM Tivoli Workload Scheduler for Distributed

IBM Tivoli Workload Scheduler is descended from the Unison Maestroprogram. Unison Maestro was developed by Unison Software onHewlett-Packards MPE operating system. It was then ported to UNIX and

Windows. In its various manifestations, IBM Tivoli Workload Scheduler has a 15year track record. During the processing day, IBM Tivoli Workload Schedulersproduction control programs manage the production environment and automatemost operator activities. It prepares jobs for execution, resolvesinterdependencies, and launches and tracks each job. Because jobs begin assoon as their dependencies are satisfied, idle time is minimized and throughputimproves significantly. Jobs never run out of sequence, and, if a job fails, IBMTivoli Workload Scheduler handles the recovery process with little or no operatorintervention.

There are two basic aspects to job scheduling in IBM Tivoli Workload Scheduler:the database and the plan. The database contains all the definitions forscheduling objects, such as jobs, job streams, resources, and workstations. It

z/OS

AIX Windows 2000 Solaris

Tracker

OS/400

SchedulingController

Tracker Tracker Tracker

also holds statistics of job and job stream execution, as well as information on the


39/278


user ID that created an object and when an object was last modified. The plancontains all job scheduling activity planned for a period of one day. In IBM TivoliWorkload Scheduler, the plan is created every 24 hours and consists of all the

jobs, job streams, and dependency objects that are scheduled to execute for that

day. All job streams for which you have created a run cycle are automaticallyscheduled and included in the plan. As the day goes by, the jobs and job streamsthat do not execute successfully can be rolled over into the next days plan.

StructureIBM Tivoli Workload Scheduler consists of a Master Domain Manager (MDM)that contains the centralized database files used to document schedulingobjects. It creates the production plan at the start of each day and performs alllogging and reporting for the network. All communications to agents are routed

through the Domain Manager (DM), which is the management hub in a domain.The network can be managed by a mix of agents. Fault tolerant agents arecapable of resolving local dependencies and launching their jobs should anetwork interruption cause a loss of communication with their Domain Managers,because each one is given a set of scheduling instructions at the beginning ofevery processing day. IBM Tivoli Workload Scheduler Version 7.0 introduced anew Java GUI, the Job Scheduling Console (JSC). The current version of JSChas been updated with several new specific IBM Tivoli Workload Schedulerfunctions. The JSC provides a common interface to both IBM Tivoli Workload

Scheduler and IBM Tivoli Workload Scheduler for z/OS.

Figure 1-5 on page 20 shows the IBM Tivoli Workload Scheduler Distributedarchitecture.


40/278


Figure 1-5 IBM Tivoli Workload Scheduler structure (distributed)

1.5.3 IBM Tivoli Workload Scheduler in an end-to-end configuration

In an end-to-end environment, E2E scheduling directly connects IBM TivoliWorkload Scheduler Domain Managers and their underlying agents and domainsto IBM Tivoli Workload Scheduler for z/OS. IBM Tivoli Workload Scheduler forz/OS is seen by the distributed network as the Master Domain Manager.

IBM Tivoli Workload Scheduler for z/OS also creates the production plan for the

distributed network and sends it to the Domain Managers. The DomainManagers send a copy of the plan to each of their agents and subordinateDomain Managers for execution.

The IBM Tivoli Workload Scheduler Domain Managers function as the brokersystems for the distributed network by resolving all dependencies for theirsubordinate managers and agents. They send their updates (in the form ofevents) to IBM Tivoli Workload Scheduler for z/OS so that it can update the planaccordingly. IBM Tivoli Workload Scheduler for z/OS handles its own jobs andnotifies the Domain Managers of all the status changes of the IBM TivoliWorkload Scheduler for z/OS jobs that involve the IBM Tivoli Workload Schedulerplan. In this configuration, the Domain Managers and all the distributed agentsrecognize IBM Tivoli Workload Scheduler for z/OS as the Master Domain

Job

Scheduling

Console

FTA7

DomainEDomainDDomainC

FTA2

DomainBDomainA

AIX

AIX

Domain

Manager

DMA

HPUX

Domain

Manager

DMB

AIX

Solaris

Solaris

MASTERDM

DMC DMD DME

AIX

FTA3

HPUX

FTA1

FTA6

Win NT

FTA4 FTA5

AIX HPUX

FTA9FTA8

AIXWin 2K

AIX

OS/400

Master

Domain

Manager

FTA7

DomainEDomainDDomainC

FTA2

DomainBDomainA

AIX

AIX

Domain

Manager

DMA

HPUX

Domain

Manager

DMB

AIX

Solaris

Solaris

MASTERDM

DMC DMD DME

AIX

FTA3

HPUX

FTA1

FTA6

Win NT

FTA4 FTA5

AIX HPUX

FTA9FTA8

AIXWin 2K

AIX

OS/400

Master

Domain

Manager

Manager and notify it of all the changes occurring in their own plans. At the sametime the agents are not permitted to interfere with the IBM Tivoli Workload


41/278


time, the agents are not permitted to interfere with the IBM Tivoli WorkloadScheduler for z/OS jobs, since they are viewed as running on the master that isthe only node that is in charge of them.

IBM Tivoli Workload Scheduler for z/OS also allows you to access job streamsand add them to the current plan in IBM Tivoli Workload Scheduler for z/OS. Inaddition, you can build dependencies among IBM Tivoli Workload Scheduler forz/OS job streams and IBM Tivoli Workload Scheduler jobs. From IBM TivoliWorkload Scheduler for z/OS, you can monitor and control the distributed agents.

In the IBM Tivoli Workload Scheduler for z/OS current plan, you can specify jobsto run on workstations in the IBM Tivoli Workload Scheduler network. IBM TivoliWorkload Scheduler for z/OS passes the job information to the Symphony file inthe IBM Tivoli Workload Scheduler for z/OS server, which in turn passes theSymphony file to the IBM Tivoli Workload Scheduler Domain Managers (DMZ) todistribute and process. In turn, IBM Tivoli Workload Scheduler reports the statusof running and completed jobs back to the current plan for monitoring in the IBMTivoli Workload Scheduler for z/OS engine.

StructureFigure 1-6 on page 22 shows an end-to-end environment where a IBM TivoliWorkload Scheduler network is managed by IBM Tivoli Workload Scheduler for

z/OS.


42/278


Figure 1-6 IBM Tivoli Workload Scheduler end-to-end structure

This concludes our discussion on IBM DB2 Content Manager OnDemand andIBM Tivoli Workload Scheduler. In the next chapter, we will start discussing the

ITWS/OnDemand integration solution.For more information on IBM DB2 Content Manager OnDemand, please refer tothe Content Manager OnDemand Guide, SG24-6915.

For more information on IBM Tivoli Workload Scheduler, please refer toEnd-to-End Scheduling with Tivoli Workload Scheduler 8.1, SG24-6022.

DomainZ

MASTERDM

Domain

Manager

AIX

AIX

Domain

Manager

DMA

HPUX

Domain

Manager

DMB

AIX Windows 2000 Solaris

DomainA DomainB

FTA1 FTA2 FTA3 FTA4

OS/400

Master

Domain

Manager

z/OS


43/278

Copyright IBM Corp. 2003. All rights reserved. 23

Chapter 2. Integrating OnDemand and

IBM Tivoli WorkloadScheduler

In this chapter, we introduce the solution of conceptional integration ofcentralized log storage on DB2, and discuss this applications role in theconstruction of the OnDemand even