copyright 2007, information builders. slide 1 internals of retrieval for table/tablef/match file...
Post on 17-Dec-2015
213 Views
Preview:
TRANSCRIPT
Copyright 2007, Information Builders. Slide 1
Internals of Retrieval for TABLE/TABLEF/MATCH FILE
Cesare PetrizioInformation Builders
Copyright 2007, Information Builders. Slide 2
Author: Cesare Petrizio Company: Information Builders Presentation Title: Internals of Retrieval for TABLE/TABLEF/MATCH
FILE Abstract: This presentation looks at what FOCUS is doing under the
covers when a TABLE is executed in FOCUS on any platform, or through WEBFOCUS. Application developers can create more efficient requests when they understand how FOCUS sets up to retrieve data from FOCUS files, Relational TABLEs, and other non-FOCUS data sources. Learn the answers to such questions as: When would an index be more efficient? One pass or two? DEFINE vs. COMPUTE? What is AUTOPATH and AUTOINDEX, and what do they mean in terms of efficiency?
Presentation Information
Copyright 2007, Information Builders. Slide 3
Agenda
Reporting Internals Retrieval Internals Analysis Phase Retrieval Phase Internal Matrix Generation
Copyright 2007, Information Builders. Slide 4
Retrieval Internals
Retrieve
DEFINE
Screening
FOCUS or External Sort
Aggregation
COUNT/MIN/MAX
TOTAL Screens
COMPUTE
Format
Extracts
Analysis Report
Generator
Sort File
Output
Generator
RETYPE
REPLOT
HOLD
Copyright 2007, Information Builders. Slide 5
FILE=EMPLOYEE,SUFFIX=FOC
SEGNAME=SEG1,SEGTYPE=S1
FIELD=EMP_ID, EID,A9,$
FIELD=FIRST_NAME, FN,A15,$
FIELD=LAST_NAME, LN,A15,$
SEGNAME=INSSEG,SEGTYPE=SH2
FIELD=COVER_DT, ,YMD,$
FIELD=TYPE_COVER, ,A3,$
FIELD=FAMILY, ,A1,$
Retrieval InternalsThe Files
FILE=KIDS,SUFFIX=DB2
SEGNAME=SEGK,SEGTYPE=S2
FIELD=EMP_ID, EID ,A9,$
FIELD=CHILD_DOB, ,YYMD,$
FIELD=CHILD_NAME, ,A20,$
FILE=SPICE,SUFFIX=FOC
SEGNAME=SEGS,SEGTYPE=S1
FIELD=EMP_ID, EID, A9, INDEX=I,$
FIELD=SPOUSE,SPOUSE_NME,A20,$
Copyright 2007, Information Builders. Slide 6
Retrieval InternalsExample
SH2KU
KEMP_ID
SPOUSE
JOIN EMP_ID IN EMPLOYEE TO EMP_ID IN SPICE AS AJ
EMP_ID
FIRST_NAME
LAST_NAME
COVER_DT
TYPE_COVER
FAMILY
JOIN EMP_ID IN EMPLOYEE TO ALL EMP_ID IN KIDS AS BJ
EMP_ID
CHILD_DOB
CHILD_NAME
K
KM
Copyright 2007, Information Builders. Slide 7
DEFINE
Segment is determined by: WITH field Fields used in expression
Constant expressions are evaluated when file is opened (segment 0)
DEFINEs are evaluated ONLY if required by requestDEFINEs are evaluated at detail level only
Each DEFINE field is associated with a specific segment
Copyright 2007, Information Builders. Slide 8
Analysis
Verify DBA access to file(s)
Determine I/O access modules required
Retrieve DEFINE field expressions for pertinent files
Read and parse the request
Check DBA and set up access restrictions at SEGMENT and FIELD levels
Open and CHECK Master File Descriptions
Copyright 2007, Information Builders. Slide 9
Analysis
Activate fields (real and DEFINEd) Read USE list and/or check Access Files and
FILEDEF/ALLOCATES Open and Verify files
Activate segments: referenced subtree(s) – smallest subtree which contains “root” and every segment referenced directly or indirectly (HEADING PARENT AND GRAND CHILD)
Copyright 2007, Information Builders. Slide 10
Retrieval InternalsThe Files
SEG1 01 S1 ************** *EMP_ID ** *FIRST_NAME ** *LAST_NAME ** * ** * ** *************** ************** I I I I INSSEG 02 I SH2 ************** *COVER_DT ** *TYPE_COVER ** *FAMILY ** * ** * ** *************** **************
SEGK 01 S2 ************** *EMP_ID ** *CHILD_DOB ** *CHILD_NAME ** * ** * ** *************** **************
SEGS 01 S1 ************** *EMP_ID **I*SPOUSE ** * ** * ** * ** *************** **************
KIDS EMPLOYEE SPICE
Copyright 2007, Information Builders. Slide 11
SEG1 01 S1 ************** *EMP_ID ** *FIRST_NAME ** *LAST_NAME ** * ** * ** *************** ************** I +-----------------+-----------------+ I I I I SEGS I SEGK I INSSEG 02 I KU 03 I KM 04 I SH2 .............. .............. ************** :EMP_ID :K :EMP_ID ::K *COVER_DT **:SPOUSE : :CHILD_DOB :: *TYPE_COVER **: : :CHILD_NAME :: *FAMILY **: : : :: * **: : : :: * **:............: :............:: *************** JOINED SPICE .............: ************** JOINED KIDS
Retrieval Internals
JOIN EMP_ID IN EMPLOYEE TO EMP_ID IN SPICE AS J1
SEG1 01 S1 ************** *EMP_ID ** *FIRST_NAME ** *LAST_NAME ** * ** * ** *************** ************** I +-----------------+ I I I SEGS I INSSEG 02 I KU 03 I SH2 .............. ************** :EMP_ID :K *COVER_DT **:SPOUSE : *TYPE_COVER **: : *FAMILY **: : * **: : * **:............: *************** JOINED SPICE **************
JOIN EMP_ID IN EMPLOYEE TO ALL EMP_ID IN KIDS AS J2
Copyright 2007, Information Builders. Slide 12
Retrieval InternalsDEFINE FILE EMPLOYEE INS_TYPE/A6 = DECODE TYPE_COVER(
DDS 'DENTAL' EYE 'VISION'
MDS 'HEALTH' ELSE 'OTHER');
TODAY/YYMD = '&YYMD'; AGE/I3 = (TODAY - CHILD_DOB) / 365.25;
END TABLE FILE EMPLOYEE PRINT INS_TYPE
CHILD_NAME AGE
BY LAST_NAME BY FIRST_NAME
BY SPOUSE IF AGE LE 20
END
Seg 4 Define Based on TYPE_COVER
Seg 0 Define (No Reference) Seg 3 Define Based on
CHILD_DOB
Copyright 2007, Information Builders. Slide 13
SEG1 01 S1 ************** *EMP_ID ** *FIRST_NAME ** *LAST_NAME ** * ** * ** *************** ************** I +-----------------+-----------------+ I I I I SEGS I SEGK I INSSEG 02 I KU 03 I KM 04 I SH2 .............. .............. ************** :EMP_ID :K :EMP_ID ::K *COVER_DT **:SPOUSE : :CHILD_DOB :: *TYPE_COVER **: : :CHILD_NAME :: *FAMILY **: : : :: * **: : : :: * **:............: :............:: *************** JOINED SPICE .............: ************** JOINED KIDS
Retrieval InternalsSegment Activation
TABLE FILE EMPLOYEE PRINT INS_TYPE
CHILD_NAME AGE
BY LAST_NAME BY FIRST_NAME
BY SPOUSE
IF AGE LE 20 END
Seg 1 Activated by FIRST_NAME and LASTNAME
Seg 2 Activated by SPOUSE
Seg 3 Activated by CHILD_DOB and CHILD_NAME
Seg 4 Activated by TYPE_COVER Via INS_TYPE DEFINE
Copyright 2007, Information Builders. Slide 14
Retrieval InternalsField Activation
Fields Mentioned in the Body of TABLE are ACTIVATED
FIRST_NAME LASTNAME SPOUSE CHILD_NAME
CHILD_DOB (Via AGE DEFINE) TYPE_COVER (Via INS_TYPE DEFINE)
DEFINE FILE EMPLOYEE INS_TYPE/A6 = DECODE TYPE_COVER(
DDS 'DENTAL' EYE 'VISION'
MDS 'HEALTH' ELSE 'OTHER');
TODAY/YYMD = '&YYMD'; AGE/I3 = (TODAY - CHILD_DOB) / 365.25;
END TABLE FILE EMPLOYEE PRINT INS_TYPE
CHILD_NAME AGE
BY LAST_NAME BY FIRST_NAME
BY SPOUSE IF AGE LE 20
END
Copyright 2007, Information Builders. Slide 15
Retrieval
Unique segments are promoted into path of parent and are never missing
IF/WHERE tests on database fields are evaluated upon retrieval
DEFINE fields are evaluated if needed IF/WHERE tests are evaluated on DEFINE fields on
segment by segment basis, AFTER all other selection tests are passed
Each Path is processed independently, and fields are merged on common sort fields
Retrieval is “Top Down, Left Right” within the referencedsub-tree
Copyright 2007, Information Builders. Slide 16
Retrieval Order of Retrieval – FOCUS Files
Root segment instances are obtained in physical order
Lower level instances obtained in SEGTYPE order within parent segment
TABLEF Root and child segment instances are obtained in
SEGTYPE order No sorting
TABLE/GRAPH/MATCH
Copyright 2007, Information Builders. Slide 17
Retrieval Order of Retrieval – FOCUS Files
Fieldname is not indexedSegment containing “fieldname” becomes
ROOT of the view, and is retrieved physically Fieldname is indexed
Segment containing “fieldname” is accessed via the equality test on INDEX
Other segments become children of the new root segment
TABLE FILE filename.fieldname
Copyright 2007, Information Builders. Slide 18
Retrieval Order of Retrieval – FOCUS Files
Alternate Physical View is created through the referenced segment highest in the hierarchy
AUTOINDEX=ON Alternate Indexed View is created if there is an
Equality test specified for an indexed field on the referenced segment highest in the hierarchy
AUTOPATH = ON
Copyright 2007, Information Builders. Slide 19
Retrieval Autostrategy – FOCUS Files
Equality IF/WHERE test on primary key field in logically retrieved segment will terminate “chain chasing” as soon as possible
STATE
DATE
Copyright 2007, Information Builders. Slide 20
Retrieval Autostrategy – FOCUS Files
IL MI
00010002 9912 9911 9910 9911 9910 9909
WHERE DATE EQ ‘9912’
1 STATE=IL, DATE=0002 Continue Search2 STATE=IL, DATE=0001 Continue Search3 STATE=IL, DATE=9912 Continue Search4 STATE=IL, DATE=9911 Next Parent5 STATE=MI, DATE=9911 Next Parent
Copyright 2007, Information Builders. Slide 21
Retrieval FIXRETRIEVE – SUFFIX=FIX
Equality IF/WHERE test on primary specified sort field in single segment FIX file will terminate retrieval as soon as possible
FILE=TESTFI,SUFFIX=FIX
SEGNAME=TESTSEG,SEGTYPE=S1
FIELD=COUNTRY,E01,A10,A10,$
FIELD=CAR ,E02,A16,A16,$
Copyright 2007, Information Builders. Slide 22
FIXRETRIEVAL
ENGLAND JAGUAR
ENGLAND JENSEN
ENGLAND TRIUMPH
FRANCE PEUGEOT
ITALY AUDI
WHERE COUNTRY EQ ‘FRANCE’
Retrieval stops if:
Set FIXRETRIEVAL = ON
Copyright 2007, Information Builders. Slide 23
Retrieval Multiple Paths
Unique segments are always in the path of their parent
Unique segments are SEGTYPE = U, KU, DKU, KLU JOIN TO creates DKU segments JOIN TO ALL creates DKM segments
TABLE/MATCH/GRAPH Instances from each path will be merged in the sort
process by the common parent(s)TABLEF does not merge data from multiple pathsAlternate file views can be used to create single pathsSET MULTIPATH controls multiple path retrieval
Retrieval is performed for each path separately
Copyright 2007, Information Builders. Slide 24
Multiple Paths
ARRIVE_TIME
ARR_FLIGHT
ARR_FROM
DEPART_TIME
DEP_FLIGHT
DEPART_TO
Airport
Copyright 2007, Information Builders. Slide 25
Multiple PathsDEFINE FILE AIRPORTS.ARRIVE_TIME
DELAY=DEPART_TIME – ARRIVE_TIME;
END
TABLE FILE AIRPORTS.ARRIVE_TIME
PRINT DEP_FLIGHT DEPART_TIME
WHERE AIRPORT EQ ‘O’’HARE’
WHERE DELAY GT 60
WHERE DEPART_TO EQ ‘LAX’
WHERE ARR_FROM EQ ‘TOR’
END
ARRIVE_TIME
ARR_FLIGHT
ARR_FROM
DEPART_TIME
DEP_FLIGHT
DEPART_TO
Airport
Copyright 2007, Information Builders. Slide 26
Multiple Paths
FAMILY
INVESTMENT
FAMILY
COLLEGE
FAMILY
Copyright 2007, Information Builders. Slide 27
Retrieval Multiple Paths Instances
1
A I
B M
2
B M
C
3
C I
D
FAMILY
FAMILYCOLLEGE
FAMILYINVESTMENT
Copyright 2007, Information Builders. Slide 28
Multiple PathsTABLE FILE SAMPLEPRINT COLLEGE INVESTBY FAMILYWHERE COLLEGE EQ ‘B’
WHERE INVEST EQ ‘I’END
FAMILY COLLEGE INVEST
------ ------- ------
1 B I
2 B .
3 . I
FAMILY COLLEGE INVEST
------ ------- ------
1 B I
SET MULTIPATH=SIMPLE SET MULTIPATH=COMPOUND
Copyright 2007, Information Builders. Slide 29
Retrieval Short Paths
High level segments with missing referenced descendents rejected (short path)
SET ALL=ON High level segments with missing referenced
descendents accepted (missing fails IF/WHERE tests)
SET ALL=PASS High level segments with qualified or missing
referenced descendents accepted (missing passes IF/WHERE tests)
Valid only for FOCUS files
SET ALL=OFF
Copyright 2007, Information Builders. Slide 30
A B
2
C
21
&ALL &IF RESULT ON A 1, A 2, B .,C 2&ALL &IF RESULT OFF A 1, A 2, C 2
A B
2
C
21
&ALL &IF RESULT ON SSN EQ 1 A 1
A B
2
C
21
Retrieval Short Paths
SET ALL = &ALLTABLE FILE DEPTSPRINT *&IFEND
DEPT
SSN
SEX
LNAME
I
I
&ALL &IF RESULT PASS SSN EQ 1 A 1, B .
A B
2
C
21
Copyright 2007, Information Builders. Slide 31
A B
2
C
21
DEPT
SSN
SEX
LNAME
I
I
A B
2
C
21
Retrieval Short Paths
SET ALL=PASS
TABLE FILE DEPTS
PRINT DEPT SSN
WHERE SSN EQ 1 AND SSN EQ 2
END
Copyright 2007, Information Builders. Slide 32
Retrieval Short Paths
Referenced, not present, Unique Segments are defaulted to blanks for alpha fields, zeros for numerics
Unique segments do not create short paths. Therefore, ALL settings are irrelevant for Unique segments.
Note: Missing referenced descendents of missing unique segments revert to ALL=OFF logic
Unique segments (U, KU, DKU, and KLU) are never considered missing
Copyright 2007, Information Builders. Slide 33
Retrieval – Short Paths
DEPT
SSN
SEX
LNAME
I
I
A B C
C1
A2
A1
JOIN DEPT IN DEPT1 TO DEPT IN SSNS AS AJ
TABLE FILE DEPT1
PRINT DEPT
WHERE DEPT1.DEPT NE SSNS.DEPT
END
A B C
C1
A2
A1
Copyright 2007, Information Builders. Slide 34
Internal Matrix Generation
One row for each distinct combination of sort keys
One column for each verb object Computed columns, row and column totals,
subtotals, and summaries are not yet done All data is in internal form
The output of the SORT/MERGE phase is conceptually a matrix
Copyright 2007, Information Builders. Slide 35
Internal Matrix Generation
Fields following a verbFields used in COMPUTEs not previously
mentionedFields used in HEADINGs or FOOTINGsFields used in SUBHEADs or SUBFOOTs, not
previously mentionedMISSING fields are not counted
Sort FieldsBY fieldsACROSS fields (used as BY at sort phase)FOCLIST (verb is PRINT or LIST)
Verb Objects (both SUM and COUNT are calculated)
Copyright 2007, Information Builders. Slide 36
Internal Matrix GenerationMultiple Verb Sets
SUM SALES BY DIVISION
SUM SALES BY DIVISION BY YEAR
LIST SALES BY DIVISION BY YEAR
Up to 16 verbs with associated BY fields can be specified
Up to 256 verb objects may be specified
Copyright 2007, Information Builders. Slide 37
Internal Matrix GenerationMultiple Verb Sets
EAST 210
NORTH 150
SOUTH 30
WEST 10
SUM SALES BY DIVISION
1 40
2 60
3 110
1 60
2 90
1 10
2 20
1 10
99 100
00 110
99 60
00 90
99 30
99 10
DIV SALES
SUM SALES BY DIVISION BY YEAR
YEAR SALES
LIST SALES BY DIVISION BY YEAR
FOCLIST SALES
Copyright 2007, Information Builders. Slide 38
Reporting Internals Output Processing
Attend the Follow-On Session
Output Processing Phase
See You There!
top related