BUILD, OVERLAY and IFTHENUsing Build, we can reformat each record by

specifying all of its item one by one.We can delete, rearrange and insert fields and

constants.An example of reformatting with OUTFIL and

BUILD: OUTFIL FNAMES=OUT1,BUILD=(21,20,5,3,PD,ADD,+20,TO=PD,LENGTH=3,

6C‘*',40:X)

COND….Using OVERLAY, we can reformat each

record by specifying just the items that overlay specific columns.

With this we can change specific existing columns without affecting the entire record.

An example of reformatting with OUTREC and OVERLAY:OUTREC OVERLAY=(45:45,8,TRAN=LTOU)

COND…Using IFTHEN clauses, we can reformat

different records in different ways by specifying how BUILD or OVERLAY items are applied to records that meet given criteria.

In IFTHEN clauses we can use conditional logic to choose different record types.

INREC IFTHEN=(WHEN=INIT,OVERLAY=(73:SEQNUM,8,ZD)),

IFTHEN=(WHEN=(4,5,CH,EQ,C'TYPE1'),

BUILD=(1,40,61:C‘001',73:73,8)),

IFTHEN=(WHEN=(4,5,CH,EQ,C'TYPE2'),

BUILD=(1,30,61:C‘002',73:73,8)),

IFTHEN=(WHEN=NONE,BUILD=(1,50,61:C‘000',73:73,8))

Fields and ConstantsAn example of rearranging fields and adding constants:OUTREC BUILD=(5,2,5C‘*',18,6,C'ABC',+30,TO=BI,

LENGTH=4,X‘01‘, 80:X)

The output records for the SORTOUT data set will contain the following:

In output positions 1-2, the characters from input positions 5-6. In output positions 3-7, five asterisks. In output positions 8-13, the characters from input positions 18-

23. In output positions 14-16, the characters ABC. In output positions 17-20, the decimal number +30 converted to a

4-byte binary value (X'0000001E'). In output position 21, the hexadecimal value 01. Blanks in output positions 22-80.

Timestamps An example of timestamps: INREC IFTHEN=(WHEN=(5,1,CH,EQ,C'A'),

BUILD=(2X,DATE=(MD4/),X,TIME,X,1,20)),

IFTHEN=(WHEN=(5,1,CH,EQ,C'B'),

BUILD=(2X,DATE=(DM4-),X,TIME=(24.),X,1,20))

The output records for the SORTOUT data set will contain the following:

Blanks in positions 1-2. In output positions 3-12, the current date in the form 'mm/dd/yyyy'

(e.g. '03/28/2005') if position 5 has 'A', or in the form 'dd-mm-yyyy' (e.g. '28-03-2005') if position 5 has 'B'.

A blank in position 13. In output positions 14-21, the current time in the form 'hh:mm:ss'

(e.g. '11:05:31') if position 5 has 'A', or in the form 'hh.mm.ss' (e.g. '11.05.31') if position 5 has 'B'.

A blank in position 22. In output positions 23-42, the characters from input positions 1-20.

Hexadecimal DisplayAn example of Hexadecimal display: OUTFIL BUILD=(1,16,HEX,C‘**',17,16,HEX)

The output records for the SORTOUT data set will contain the following: In output positions 1-16, the hexadecimal representation of input positions 1-

16. Two asterisks in output positions 17-18. In output positions 19-34, the hexadecimal representation of input positions 17-

32.

You can even display an entire variable-length record, including the variable part, in hexadecimal. OUTREC BUILD=(1,4,1,4,HEX,5,HEX)

The output records for the SORTOUT data set will contain the following: In output positions 1-4, the record descriptor word (RDW). In output positions 5-8, the hexadecimal representation of the RDW. Starting in output position 9, the hexadecimal representation of the variable

bytes starting in input position 5.

TranslationLowercase to uppercase translation:INREC OVERLAY=(21:21,11,TRAN=LTOU,48:48,8, TRAN=LTOU)

• Uppercase to lowercase translation:OUTREC BUILD=(1,4,5,TRAN=UTOL)

Translation using an ALTSEQ table:ALTSEQ CODE=(0040,5C40)

OUTFIL BUILD=(1,20,21,30,TRAN=ALTSEQ,C'END')

Writing Constants Character Strings

The format for writing a character string is:C’x...x’ where x is an EBCDIC character. For example, C'FERN'. Hexadecimal Strings

The format for writing a hexadecimal string is: X'yy...yy' where yy is a pair of hexadecimal digits. For ex: X'7FB0'.

Decimal Numbers The format for writing a decimal number is:

n...n or ±n...n where n is a decimal digit. Examples are 24, +24, and -24.

Substring SearchSubstring Search for INCLUDE and OMIT:

INCLUDE COND=(106,5,CH,EQ,C’BIOL’,OR, 106,5,CH,EQ,C’HIST’,OR,

106,5,CH,EQ,C’BUSIN’,OR,

106,5,CH,EQ,C’PSYCH’)

Substring Search (SS format):

INCLUDE COND=(106,5,SS,EQ,C’BIOL ,HIST ,BUSIN,PSYCH’)

With substring search (SS format), you only write the field once and write the character constant so it includes all of the strings you want to search for. If the value in the field matches any of the strings (for example, ″BUSIN″), the record is included. If the value in the field does not match any of the strings, the record is omitted.

VB Data Set Considerations Short control fields

If you know you have VB records with short compare fields, you can specify the VLSCMP option, if appropriate, to prevent DFSORT from terminating. For example: OPTION COPY,VLSCMP INCLUDE COND=(21,8,CH,EQ,C’Type 200’)

VLSCMP tells DFSORT that you want to temporarily replace any missing compare field bytes with binary zeros, thus allowing the short fields to be validly compared (the zeros are not kept for the output records). In the example above, records less than 28 bytes are not included because the binary zeros added for the missing bytes in the field prevent it from being equal to 'Type 200'.

Another way you can prevent DFSORT from terminating for VB records with short compare fields, if appropriate, is by specifying the VLSHRT option. For example:

OPTION COPY,VLSHRT

INCLUDE COND=(21,8,CH,EQ,C’Type 200’)

VLSHRT tells DFSORT to leave records with short summary fields unsummed. That is, when one of the two records involved in a summary operation has a short summary field, the records are kept unchanged; neither record is deleted. In the example above, any records less than 28 bytes are not included.

Numeric EditingYou can use the following OUTREC statement

to make the PD values meaningful: OUTREC FIELDS=(1,8, 5X, 11,6,PD,M4, 5X, 31,7,PD,EDIT=(SI,III,III,IIT.TTT),SIGNS=(,-), 5X, 41,5,PD,EDIT=(TT-TT-TTTT))

COND…M4 is one of DFSORT’s 27 pre-defined edit

masks. It edits a numeric field according to the pattern SI,III,III,III,IIT.TT. EDIT=(SI,III,III,IIT.TTT) is a user-defined edit mask. It edits a numeric field according to the pattern SI,III,III,IIT.TTT. EDIT=(TT-TT-TTTT) is another user-defined edit mask. It edits a numeric field according to the pattern TT-TT-TTTT.

COND…In the patterns: I indicates a leading insignificant digit to be

displayed as 1-9, or as blank for a leading 0. T indicates a significant digit to be displayed as 0-9. S before the digits indicates a leading sign. S after

the digits indicates a trailing sign. For M4, the leading sign is to be displayed as + for a positive value or as − for a negative value. For EDIT=(SI,III,III,IIT.TTT),SIGNS=(,−), the leading sign is to be displayed as blank for a positive value or as − for a negative value.

Any other character (for example, comma, decimal point or hyphen) is just displayed as appropriate. The results produced for the OUTREC statement are:

Numeric ConversionAn example of numeric conversion:OUTREC BUILD=(21,5,ZD,TO=PDF,8,4,ZD,TO=FI,LENGTH=2)

The zoned decimal value in positions 21-25 of the input record is converted to a packed decimal value in positions 1-3 of the output record (with an F for a positive sign or a D for a negative sign). The zoned decimal value in positions 8-11 of the input record is converted to a fixed-point value in positions 4-5 of the output record. .

Sequence Numbers INREC, OUTREC and OUTFIL let us generate BI, PD, ZD or FS

sequence numbers in the output records.Starting values and increment values can be specified or defaulted.

An example of sequence numbers assigned before sorting with INREC:INREC OVERLAY=(73:SEQNUM,8,ZD,START=1000,INCR=1000)

SORT FIELDS=(25,5,CH,A) An example of sequence numbers assigned after sorting with

OUTREC:SORT FIELDS=(25,5,CH,A)

OUTREC OVERLAY=(73:SEQNUM,8,ZD,START=1000,INCR=100)We can also restart the sequence numbers at the START value

each time the value in a specified field changes. An example with OUTFIL:OUTFIL OVERLAY=(81:SEQNUM,5,ZD,RESTART=(5,3))

Justifying(JFY)Left-Justify:

OPTION COPYOUTREC FIELDS=(1,30,JFY=(SHIFT=LEFT))

Right-Justify:OPTION COPYOUTREC FIELDS=(1,30,JFY=(SHIFT=RIGHT))

Leading, Trailing String:OPTION COPYOUTREC FIELDS=(1,30,JFY=(SHIFT=LEFT,LEAD=C'<',TRAIL=C'>'))

OPTION COPYOUTREC FIELDS=(1,30,JFY=(SHIFT=RIGHT,LEAD=C'<',TRAIL=C'>'))

Squeezing(SQZ)Left Squeeze:For a left-squeezed field, all blanks are removed and

the characters from the first nonblank to the last nonblank are shifted left, with blanks inserted on the right if needed.

OPTION COPY

OUTREC FIELDS=(1,40,SQZ=(SHIFT=LEFT))

OPTION COPY

OUTREC FIELDS=(1,40,SQZ=(SHIFT=LEFT,

LEAD=C'<tag1>‘,TRAIL=C'</tag1>'))

Squeezing(SQZ)Right Squeeze:For a right-squeezed field, all blanks are removed

and the characters from the last nonblank to the first nonblank are shifted right, with blanks inserted on the left if needed.

OPTION COPY

OUTREC FIELDS=(1,40,SQZ=(SHIFT=RIGHT))

OPTION COPY

OUTREC FIELDS=(1,40,SQZ=(SHIFT=RIGHT,

LEAD=C'<tag1>‘,TRAIL=C'</tag1>'))

Parsing Delimited FieldsThis very powerful DFSORT function can be used to

extract delimited variable position/length data in many different forms into %nn fields, and then edit, convert, justify, squeeze, translate, lookup and change, or do arithmetic with the %nn fields.

Here's an example of parsing and reformatting comma separated values (CSV):

OUTREC PARSE=(%01=(ENDBEFR=C',',FIXLEN=8),

%=(ENDBEFR=C','),

%03=(ENDBEFR=C',',FIXLEN=5),

%04=(FIXLEN=12)),

BUILD=(%01,14:%03,TRAN=LTOU,25:%04,UFF,TO=FS,LENGTH=15)

Numeric and Non-Numeric Tests (NUM)We can test a field for numerics

(field,EQ,NUM) or non-numerics (field,NE,NUM) in character (FS), zoned decimal (ZD) or packed decimal (PD) format.

Examples:

INCLUDE COND=(18,4,FS,NE,NUM)

INCLUDE COND=(22,6,PD,EQ,NUM,

AND,28,6,PD,EQ,NUM)

Splitting a File Contiguously (SPLIT1R)SPLIT and SPLITBY=n rotate many times among

the output data sets, SPLIT1R=n only rotates once among the output data sets, resulting in contiguous records in each output data set.

Examples:OUTFIL FNAMES=(OUT1,OUT2,OUT3),SPLITOUTFIL FNAMES=(OUT1,OUT2,OUT3),SPLITBY=5OUTFIL FNAMES=(OUT1,OUT2,OUT3),SPLIT1R=5

Symbols for Columns (sym:) ... //SYMNAMES DD

Col1,5

Col2,25

Col3,45

End_rec,80

First,28,10

Last,7,15

Amount,45,4,FS /* //SYSIN DD

OPTION COPY

OUTFIL REMOVECC,

HEADER2=(Col1:'First',Col2:'Last',Col3:'Amount',/,

Col1:10'-',Col2:15'-',Col3:6'-'),

BUILD=(Col1:First,Col2:Last,

Col3:Amount,EDIT=(IIT.TT),End_rec:X),

TRAILER2=(Col1:5'=',Col3:6'=',/,

Col1:'Total',Col3:TOT=(Amount,EDIT=(IIT.TT))) /*

ICEGENER: Make IEBGENER Jobs FlyDFSORT‘s ICEGENER facility is a replacement

for IEBGENER.ICEGENER as a direct replacement for

IEBGENER, so ICEGENER is then called automatically whenever IEBGENER is requested either directly or from a program.

ICEGENER uses DFSORT to process IEBGENER jobs when possible and transfers control to IEBGENER when DFSORT can't be used.

DFSORT will issue information such as the number of records copied and the RECFM, LRECL, and BLKSIZE.

Multiple Output (OUTFIL): Slash Elapsed Time, EXCPs and CPU Time

With OUTFIL, sort, merge and copy applications can create multiple output data sets containing unedited or edited records, different ranges, subsets or samples of records, reports, and so on, from a single pass over one or more input data sets.

We can see range of Performance improvements with DFSORT using a single sort step with OUTFIL vs DFSORT using a sort step followed by multiple copy steps

INTRODUCTIONICETOOL, a versatile data set processing and

reporting utility, provides an easy-to-use batch front-end for DFSORT. ICETOOL combines new features with previously available DFSORT features to perform complex sorting, copying, reporting and analytical tasks using multiple data sets in a single job step.

The thirteen ICETOOL operators are: COPY, COUNT, DEFAULTS, DISPLAY, MODE, OCCUR, RANGE, SELECT, SORT, SPLICE, STATS, UNIQUE, and VERIFY. By using combinations of these thirteen operators, we can easily create applications that perform many complex tasks. By supplying these operators in a parameter list, we can use the information returned by ICETOOL in your program.

ICETOOL Return Codes ICETOOL sets a return code for each operation it performs.

The return codes are:

0 - Successful completion. No errors were detected. 4 - Successful completion. DFSORT detected one or more

warning conditions. 12 - Unsuccessful completion. ICETOOL detected one

or more errors. Can also be set if the record count meets a specified criteria (for example, a data set is empty, or a data set contains more than 50000 records).

16 - Unsuccessful completion. DFSORT detected one or more errors.

20 - Message data set error. The TOOLMSG DD statement was not present or the TOOLMSG data set was not opened.

24 - Unsupported operating system. The operating system was not z/OS or OS/390.