ms open database (odbm) and the ims catalog · pdf file3 ims catalog metadata • database...
TRANSCRIPT
Exploring the IMS Catalog Using the Metadata and IMS Open Database (ODBM)
#12548 - Golden Gate 8February 4,2013
2
IMS Java Metadata • IMS 7 – 11 Generated into local file
• IMS DBD + PSB source• COBOL COPYBOOKs and PL/I INCLUDEs map
PSBLIB DBDLIBACBLIBCOBOL/PLI
source
DBD sourcePSB source
–IMS –User maintained
IMS ES Explorer
Java classes
DLIDatabaseView IMS Metadata
2
3
IMS Catalog Metadata • Database and Application Program resources are managed by IMS
–Database definitions
–Segments, Mappings, Fields and data types
–Application program specifications
IMS Metadata
3
ACBLIB
ACBGEN
DBDGEN/PSBGEN
Extended
PSB source
data
Extended
DBD source
data
4
Dynamic Metadata management • Database and Application Program resources are managed by IMS
IMS Metadata
4
ACBLIB
ACBGEN
DBDGEN/PSBGEN
Extended
PSB source
data
Extended
DBD source
data
IMS Explorer
–Database Definitions
– Segments
– Fields
–Program Definitions
– Data Bases PCB Schema
– PCBNAME
–Application program specifications
� Arrays and Structures
� Field data types and data type conversion
� Field redefines
� Alternative Field maps for a segment
5
DBD + PSB = IMS DB Schema
DA BB DC
a b c d
e f g j k l
A
B D
h i
C
DBD, PCB,Copybooks
DBD/PCB Schema
On Disk I/O
IMS HierarchicalModel
Program I/O Area
A B C
6
It’s the Metadata
Defined inDBD
Defined inCopylibs
(IMS Java)
Defined inXML SchemaLocal file orIMS Catalog
0101010010111101010100001001110100101111
10110110010001010011010111011100011001001001
<PurchaseOrder number=“ ”><lastName> </lastName><firstName> </firstName><date> </date><payment type=“ ”> </payment>
10100010111001
01011010101101100101010010101001010101010000111011101100101011101010000010101011
1011000100110101110101001001
101100000101001001101011101011001001001001
number lastName firstName payment type dateINT CHAR CHAR CHAR CHAR DATE
113246IBMIMS
04/26/1968MC 5414 2263 4895 1145
7
Data typesData types have multiple metadata elements
• Application data type• Universal drivers use application data type to present data to clients
• Physical data type• Universal drivers use physical data type to marshal/unmarshal data to and from the
database• Example
• Application data type is DECIMAL(10,2) [decimal with precision 10 and scale 2]• Physical data type is a signed packed decimal (AD community doesn’t need to know
this)
New data type support• Structs (nested n levels with no constraint on element data types)
• Accessed via SQL and DLI• Arrays (nested n levels with no constraint on element data types)
• Accessed via SQL and DLI• User-defined
• Name of UDT can be defined to the catalog and intended to be used at runtime by Universal drivers to marshal/unmarshal data
• Can be part of a Struct or Array element
8
Maps
Mapping support• A Map is metadata that describes how a field (or set of fields) are mapped for
a particular segment instance• Metadata captures the various cases and for each case defines the set of
fields to be used for that case• Maps can be defined to the catalog• Example
• Insurance segment mapped multiple ways depending on value of a ‘Policy Type’ field
-----555 Disk
Drive Way, 95141
500K5Single Family
H
Red2K1989EscortFord----M
ColorValueYearModelMakeAddressValueRoomsProperty Type
Policy Type
9
PSB PCB statements changes
• PCB • EXTERNALNAME=
• An optional alias for the PCB label or the PCBNAME= parameter
• Java application programs use the external name to refer to the PCB
• REMARKS= • Optional user comments. A 1- to 256-character field. Added to
PSBGEN statements
10
DBD statement changes
• DBD • ENCODING=
• Specifies the default encoding of all character data in the database defined by this DBD.• This value can be overridden in individual segments or fields.
• SEGM• EXTERNALNAME=
• An optional alias for the NAME= parameter used by Java application programs to refer to the segment.
• FIELD• CASENAME=
• The name of the map case when alternative mappings are defined for the fields in a segment• DATATYPE=
• Specifies the external data type of the field.• EXTERNALNAME=
• An optional alias for the NAME= parameter used by Java application programs to refer to the field.
• DFSMARSH • Specifies the DATATYPE converter routine to transform binary data to external data format
• DFSMAP • Defines alternative field mappings in a segment.
• DFSCASE • Defines a map case for a segment type that uses DFSMAP alternative field mapping.
• REMARKS= • Optional user comments. A 1- to 256-character field. Added to DBDGEN statements
11
DBDGEN Support for Unsigned Data Types
• APAR - PM64418 PTF - UK79443• Problem
• IMS Explorer generates DBD source which includes new unsigned data types and internal type converters which are not supported by DBDGEN
• Large positive signed 2's complement data element values are made negative.
• Solution• FIELD macro add support UBYTE, UINT, ULONG, and
USHORT data type values on the DATATYPE= parameter• DFSMARSH macro add support UBYTE, UINT, ULONG, and
USHORT internal type converter values on the INTERNALTYPECONVERTER= parameter
12
DBDGEN FIELD Statement>>-FIELD >-DATATYPE=+-ARRAY+-BINARY+-BIT
+-BYTE +-UBYTE+-CHAR+-DATE+-DECIMAL(pp,ss)+-DOUBLE+-FLOAT
+-INT +-UINT+-LONG +-ULONG+-OTHER
+-SHORT +-USHORT+-STRUCT+-TIME+-TIMESTAMP+-XML
13
DBDGEN DFSMARSH Statement
>>-DFSMARSH
>--+-,INTERNALTYPECONVERTER=
+-CHAR
+-BIT
+-BINARY
+-BYTE +-UBYTE
+-SHORT +-USHORT
+-INT +-UINT
+-LONG +-ULONG
+-FLOAT +-DOUBLE
+-PACKEDDECIMAL +-ZONEDDECIMAL
+-CLOB +-BLOB +-XML_CLOB +-ARRAY +-STRUCT
14
IMS Universal Drivers Unsigned Type Support • APAR - PM62076 PTF - UK80831• Problem
• NullPointerException in the IMS Universal Drivers when accessing a Secondary Index with a single source field that is defined as packed decimal.
• Solution• The IMS Universal Drivers have added support for the following data types:
• UTINYINT • USMALLINT • UINTEGER • UBIGINT
• Benefits• Users are now able to access a Secondary Index with a single source field
of any type • Users will no longer receive an UnsupportedTypeConversion message
when accessing a variable length segment where the length field (LL) is defined as BINARY.
15
COBOL data formats mapped to IMS Java data constants
• PIC X(25) CHAR• PIC 9(02) COMP UTINYINT• PIC S9(04) COMP SMALLINT• PIC 9(04) COMP USMALLINT• PIC S9(06) COMP-4 INTEGER• PIC 9(06) COMP-4 UINTEGER• PIC 9(06)V99 COMP or COMP-4 BINARY• PIC S9(12) BINARY BIGINT• PIC 9(12) BINARY UBIGINT• COMP-1 FLOAT• COMP-2 DOUBLE• PIC S9(06)V99 ZONEDDECIMAL• PIC 9(06).99 ZONEDDECIMAL• PIC S9(06)V99 COMP-3 PACKEDDECIMAL
16
Catalog Metadata definition in DBD and PSB source
• IMS 12 HALDB Catalog • Contains information about IMS program resources, database
resources, and relevant application metadata• Database Metadata
• describes a physical database • Application Program Metadata
• describes application program view of the data in the database• Arrays and Structures • Field data types and data type conversion • Field redefines • Alternative Field maps for a segment
• ACBGEN• Used to auto-populate the catalog
• Benefits• consolidates the application program data requirements into
trusted location
17
Extended
DBD source
IMS Catalog Lifecycle
Catalog
Extended
PSB source PSBLIB++
DBDLIB++
PSBGEN
DBDGEN
ACBGENIMS Explorer DFS3PU00
DB Client Route
ACBLIB++PSBLIB
DBDLIB
ACBLIB
� ACBGEN will populate ACBLIB and catalog in same UOW• Populates ACBLIB with standard ACB info and extended info• Populates the catalog with extended info
� Key points• Only way to update catalog is via the Populate Utility or ACBGEN process• Extended info stored in ACBLIB members for recoverability• Extended info is acquired via the IMS Explorer
Populate Utility will populate catalog from ACBLIB
17
18
IMS Universal Database Drivers
• A “Driver” is the java code that provides an API to a java application, and interfaces to the target DB system
• A Remote Client uses industry standard Distributed Relational Database Architecture (DRDA) for communication with IMS Connect
• IMS 11 introduced three new Java Universal Drivers• IMS Universal DB Resource Adapter
• Exploits JCA 1.5 in a J2EE Java environment (e.g. in WAS), and supports JDBC SQL calls
• IMS Universal JDBC Driver • In any Stand-alone Java SE environment, supports JDBC SQL calls
• IMS Universal DL/I Driver • Used in a Stand-alone Java SE environment to support DL/1 calls
• Each driver can implement a Type-2 interface for local z/OS access or a Type-4 interface for access via TCP/IP and IMS Connect
19
Immediate Benefits
• All Universal drivers leverage the IMS catalog
• Trusted online (IMS) source for metadata used by the drivers
• Direct access to IMS metadata in the catalog
• No longer require the separate Java metadata class• Virtual deployment support
• No longer file-system dependent for metadata• Virtual deployment support
• Metadata is trusted and up-to-date
20
SQL and IMS Concepts ASegment(Table)
Fields(Columns)A11|A2|A3
BA11A11B1
CA11A11B1B1
C1 Segment Instances(Table Row)
Segment Key(Table Primary Key)
IMS 11IMS Foreign Key
(Table Foreign Key)
21
IMS JavaHierarchical to Relational Mapping
• Each Segment Type maps to a Table• Each segment is a row, each field is a column
Relational Design
DealerNo DealerName DealerAddressDealer Table
53SJ7 George 555 Bailey Ave.
53SJ8 Bob 240 Elm St.
53SJ9 Mary 111 Penny Ln.
... ... ...
UU45 | Dodge | Viper
PR27 | Dodge | Durango
53SJ9 | Mary | 111 Penny Lane 53SJ8 | Bob | 240 Elm St.
DEALER Segments
MODEL Segments
FF13 | Toyota | Camry
53SJ7 | George | 555 Bailey Ave.
Hierarchical Design
JPR27 | Dodge | Durango
WJ45 | Mercury | Cougar
DLRNO NAME ADDRESS
MODID MAKE MODTYPE
DEALER
SALESINF
SALESPER
STOCKSALES
MODEL
ORDER
Model TableID Make ModType DealerNo
UU45 Dodge Viper 53SJ7
PR27 Dodge Durango 53SJ7
FF13 Toyota Camry 53SJ7
JR27 Dodge Durango 53SJ8
WJ45 Mercury Cougar 53SJ8
... ... ... ...
Foreign Key
Reminder
Relational JOIN
DB
DJO
IN
IMS Foreign Key
22
Query governance – join operations
A
EDB
C F G
Logical Child
Logical Parent
23
Application programming with the IMS 12 Catalog
• IMS catalog resident PSBs for application programs
• DFSCP000 - High-level assembler and COBOL applications• DFSCP002 PL/I - applications
• DFSCP003 - PASCAL applications
• The following PCBs are included to support different catalog processing models:
• DFSCAT00 - The primary PCB to access all data in the catalog database..• DFSCATSX - Use this PCB to access the catalog database via the catalog
secondary index.
• DFSCATX0 - Use this PCB to process the catalog secondary index database.
• All catalog processing is performed with PROCOPT=G.
• GUR call• application programs can use the Get Unique Record (GUR) DL/I call to
retrieve catalog database record
24
GUR Call
• New “GUR” DL/I call
• Get Unique Record
• Restricted to use with IMS Catalog database• Functions like a GU followed by a series of GNP calls
• Returns the entire database record in one call
• Saves overhead of issuing GU & GNP to retrieve all the metadata for a catalog member
• Using an AIB token, the call can be continued if the I/O area is too small for entire catalog database record
• Data returned will be in XML format • The XML schemas are included in the IMS.ADFSSMPL data set:
• DFS3XDBD.xsd (for DBD records) • DFS3XPSB.xsd (for PSB records)
• Support added for DFSDDLT0 and IMS REXX
25
GUR Call…
• IMS catalog has a structure that uses a header segment as the root for each record. • Each header segment instance has either a PSB or DBD child segment
instance. • GUR AIB IOAREA HEADER PSB
• This call locates the first record, which is always a DBD record because DBD precedes PSB in alphanumeric order.
• GUR AIB IOAREA HEADER (TYPE == PSB ) PSB• A GUR call that is issued without a qualification at the PSB or DBD level
retrieves the record for the member that is currently active in the ACB library.
• GUR AIB IOAREA HEADER (RHDRSEQ == PSB BMP255 )• GUR call fails if there is no active ACB library member for BMP255• AIB return code X’108’ and reason code X’338’
• GUR AIB IOAREA HEADER (RHDRSEQ = PSB BMP255 ) PSB (TSVERS ==xxxxxxxxxxxxx)
• To retrieve the record for an inactive or removed ACB library member, add an SSA qualification for the correct ACB generation timestamp
26
Runtime Catalog Access
DBDB
Catalog
DL/I or Type 2
Universal Drivers
(SQL/DLI/XML)
WAS z
DRDA/TCPIP
SQL/DLI
SQL/DLISCI
DLI
UsersIMS Explorer
Tools
Type 4UniversalDrivers
(SQL/DLI/XML)
Applications
SQL/DLIDependent
Regions
(MPP,JMP,IFP
,BMP,JBP)
Type 2
Universal Drivers
(SQL/DLI/XML)
CICSDB2 zSP
Type 4
Universal Drivers
(SQL/DLI/XML)
DRDA/TCPIP
27
IMS Catalog Supported Interfaces
• Open systems• Universal drivers - Type 4
• SQL and DLI interfaces directly to the catalog• XML - render catalog information as XML instance document
• z/OS• Universal drivers - Type 2 and Type 4
• SQL and DLI interfaces directly to the catalog• XML - render catalog information as XML instance document
• Traditional IMS languages• DLI access directly to the catalog• Batch access supported
27
28
Open Database Components
IMS
SCI
OD
BA
/CC
TL
DDM
TCP/IP
DRDA/DDM
DLI
IMSOpen DBUniversal
Type-4Drivers
z/OS LPAR 2 z/OS
LPAR 1
Operations
Manager(OM)
SCI
API
Structured
CallInterface
(SCI)SCI
API
IMS
Connect
ODBM
Client
S
C
I
DR
DA
PO
RT
Open DB
Manager(ODM)
SCI
DR
A
IMS DBIMSCatalog
GUR
DLI
2929
IMS Enterprise Suite V2.2 Explorer for DevelopmentQuery IMS database with Universal JDBC driver
Create SQL scripts with Select, Update, Delete, and Insert
statements
Create SQL scripts with Select, Update, Delete, and Insert
statements
Build a SQL statementusing the SQL Builder
with content assistance
Build a SQL statementusing the SQL Builder
with content assistance
Viewthe SQL results
Viewthe SQL results
PSB =schemaPSB =
schema
DB PCB = database
DB PCB = database
Database segments =
tables
Database segments =
tables
30
IMS Enterprise Suite V2.2 Explorer for Development
• Visualization and editing of IMS Database and Program Definitions • Provide graphical editors to:
• Display IMS database hierarchical structures• Display/create/edit PSBs• Change/add fields on a DBDs
• Import Cobol CopyBooks and PL/I Structures to a database segment*• Generate of DBD and PSB source
• Ability to easily access IMS data using SQL statements• Leveraging IMS Universal JDBC driver
• Ability to access the IMS Catalog • Connectivity to the z/OS system
• Browse a Data Set and submit JCLs• Import and export DBD and PSB source files from a Data Set to the IMS
Explorer, and vice-versa• Supports cross-product integration:
• IBM® Rational® Developer for System z®• IBM Optim™ Development Studio • IBM Problem Determination Tools Plug-ins for Eclipse
• Benefit• Simplify IMS application development tasks *Requires RDz 8
3131
IMS Enterprise Suite V2.2 Explorer for Development
• IMS 12 HALDB Catalog Metadata• Universal Driver Type 4 connection• Can add COBOL or PL/I metadata to the catalog
• Direct update of the catalog metadata is not available• Requires intermediate ACBGEN to populate the catalog with the
extended information
• PSBs and DBDs editing• Send updated DBDGEN and PSBGEN source files to the
host • run DBDGENs, PSBGENs, ACBGENs, and catalog
population
32
IMS Enterprise Suite V2.2 Explorer for DevelopmentView physical IMS database structure
Logical relationship
between databases
Logical relationship
between databases
Additional properties of a
segment or field
Additional properties of a
segment or field
33
IMS Enterprise Suite V2.2 Explorer for DevelopmentView logical IMS database structure
A concatenated segment and its
underlying source segments
A concatenated segment and its
underlying source segments
34
IMS Enterprise Suite V2.2 Explorer for Development Build PCB definition
34
Point and clickto select
SENSEGs
Point and clickto select
SENSEGs
GeneratedPSB sourceGenerated
PSB source
3535
IMS Enterprise Suite V2.2 Explorer for DevelopmentBrowsing Data Sets and Submitting JCL’s
Browsedata setsBrowse
data sets
Edita data set member
Edita data set member
ViewJES output
ViewJES output
36
IMS Enterprise Suite V2.2 Explorer for Development
• New for V2.2• COBOL and PL/I importer plug-ins have now been packaged with
IMS Explorer for Development.• An Eclipse Task Launcher has been added to provide a single UI
for launching wizards and online help topics.• Internal changes have been made to improve the performance of
IMS sources import.• Data Source Explorer view displays additional IMS related
information.• Enable Data Studio supports for pureQuery with IMS.
37
IMS Explorer V2.2 Joint Project with IBM Data Studio V3.2
• IBM® Data Studio support for pureQuery with IMS
• Can generate pureQuery code for IMS databases • All generated SQL supported by IMS
Annotated pureQuery interface
38
IMS Explorer for Development – Runtime Access
• Distributed Relational Database Architecture (DRDA)
• Set of protocols and functions for client and database servers connectivity
• Communication protocol
• Two-Phase commit protocol• Security
Windows z/OS
Cognos
IMS
Universal
Database
Driver
T
C
P
I
P
T
C
P
I
P
IMS Connect
S
C
I
ODBM
S
C
I
O
D
B
A
CTL IMS
DB
PC
IMS
TCPIP
IMS Explorer
SQL
DLIDDM
IP
DLIDDM
SQL code orresult set DLI
Distributed Data Management : DRDA protocol
IMS Connect is the server for DRDA messages sent via the TCP/IP protocol
IMSCatalog
39
DLIModel
• IMS Enterprise suite for z/OS , V2.1 is the last release to provide the DLIModel Utility plug-in.
• Explorer provides ability to import DLIModel projects• Explorer does not support XML DB or DB Web Services
• Customers using IMS Database Web Services should transition to using the IBM® Data Studio which leverages the IMS Universal Drivers
40
Summary
• IMS 12 HALDB Catalog • Contains information about IMS program resources, database
resources, and relevant application metadata• ACBGEN
• Used to auto-populate the catalog• IMS Enterprise Suite Explorer for Development
• Generates DBD/PSB source to populate the catalog
4:30pm today - IMS Explorer for DevelopmentNate Church#12651Golden Gate 7