sas data integration studio take control with conditional
TRANSCRIPT
![Page 1: SAS Data Integration Studio Take Control with Conditional](https://reader033.vdocuments.mx/reader033/viewer/2022043020/626bc01c79feaf484b13817e/html5/thumbnails/1.jpg)
SAS Data Integration Studio
Take Control with Conditional &
Looping Transformations
Harry Droogendyk – Stratia Consulting Inc.
![Page 2: SAS Data Integration Studio Take Control with Conditional](https://reader033.vdocuments.mx/reader033/viewer/2022043020/626bc01c79feaf484b13817e/html5/thumbnails/2.jpg)
Introduction
� DIS is real good at linear
� not linear ?� we love macro, roll yer own non-linear
� why use DIS properly ?� metadata trail
� maintenance
� non-linear Transformations� Loop
� Conditional
![Page 3: SAS Data Integration Studio Take Control with Conditional](https://reader033.vdocuments.mx/reader033/viewer/2022043020/626bc01c79feaf484b13817e/html5/thumbnails/3.jpg)
Business Need
� 3rd party supplied text files
� imported into SAS �Teradata
� daily – kinda
� must be processed in date order
� control table
� if a day is missing...
� send email
� continue to look for files
� don’t import any more data
![Page 4: SAS Data Integration Studio Take Control with Conditional](https://reader033.vdocuments.mx/reader033/viewer/2022043020/626bc01c79feaf484b13817e/html5/thumbnails/4.jpg)
Technical Solution
� outer job - looping� read control table to get last imported date
� create driver table of dates to process - last+1 to today-1
� loop over driver table� pass next date to inner, import job
� execute inner job
� inner job – looped over� accept next date to process from outer job
� check if file for this date exists
� send email or import
� stop importing after missing file is detected
![Page 5: SAS Data Integration Studio Take Control with Conditional](https://reader033.vdocuments.mx/reader033/viewer/2022043020/626bc01c79feaf484b13817e/html5/thumbnails/5.jpg)
DIS Jobs
![Page 6: SAS Data Integration Studio Take Control with Conditional](https://reader033.vdocuments.mx/reader033/viewer/2022043020/626bc01c79feaf484b13817e/html5/thumbnails/6.jpg)
DIS Jobs
![Page 7: SAS Data Integration Studio Take Control with Conditional](https://reader033.vdocuments.mx/reader033/viewer/2022043020/626bc01c79feaf484b13817e/html5/thumbnails/7.jpg)
DIS Jobs
� note the icons
� & is a job with a parameter
� blue arrow is a deployed job
![Page 8: SAS Data Integration Studio Take Control with Conditional](https://reader033.vdocuments.mx/reader033/viewer/2022043020/626bc01c79feaf484b13817e/html5/thumbnails/8.jpg)
Creating Inner Job
� Job Properties
� prompts aren’t magic, simply macro vars
![Page 9: SAS Data Integration Studio Take Control with Conditional](https://reader033.vdocuments.mx/reader033/viewer/2022043020/626bc01c79feaf484b13817e/html5/thumbnails/9.jpg)
Creating Inner Job
� name must be macro variable name in code
![Page 10: SAS Data Integration Studio Take Control with Conditional](https://reader033.vdocuments.mx/reader033/viewer/2022043020/626bc01c79feaf484b13817e/html5/thumbnails/10.jpg)
Creating Inner Job
� completed parameter
![Page 11: SAS Data Integration Studio Take Control with Conditional](https://reader033.vdocuments.mx/reader033/viewer/2022043020/626bc01c79feaf484b13817e/html5/thumbnails/11.jpg)
Inner Job – Set Condition
� Check File Presence – code node
![Page 12: SAS Data Integration Studio Take Control with Conditional](https://reader033.vdocuments.mx/reader033/viewer/2022043020/626bc01c79feaf484b13817e/html5/thumbnails/12.jpg)
Inner Job – Set Condition
� using the job parameter
![Page 13: SAS Data Integration Studio Take Control with Conditional](https://reader033.vdocuments.mx/reader033/viewer/2022043020/626bc01c79feaf484b13817e/html5/thumbnails/13.jpg)
Inner Job – Set Condition
� checking if file exists
![Page 14: SAS Data Integration Studio Take Control with Conditional](https://reader033.vdocuments.mx/reader033/viewer/2022043020/626bc01c79feaf484b13817e/html5/thumbnails/14.jpg)
Inner Job – Check Condition
� No Missing Files? Conditional Start Transformation
![Page 15: SAS Data Integration Studio Take Control with Conditional](https://reader033.vdocuments.mx/reader033/viewer/2022043020/626bc01c79feaf484b13817e/html5/thumbnails/15.jpg)
Inner Job – Check Condition
� continue if TRUE
![Page 16: SAS Data Integration Studio Take Control with Conditional](https://reader033.vdocuments.mx/reader033/viewer/2022043020/626bc01c79feaf484b13817e/html5/thumbnails/16.jpg)
Inner Job – Check Condition
� Skip to Conditional End if FALSE
![Page 17: SAS Data Integration Studio Take Control with Conditional](https://reader033.vdocuments.mx/reader033/viewer/2022043020/626bc01c79feaf484b13817e/html5/thumbnails/17.jpg)
Inner Job – Check Condition
� Conditional Start generated code
![Page 18: SAS Data Integration Studio Take Control with Conditional](https://reader033.vdocuments.mx/reader033/viewer/2022043020/626bc01c79feaf484b13817e/html5/thumbnails/18.jpg)
Inner Job – real work
� if TRUE, the real work occurs
� everything between Conditional Start and Conditional End
![Page 19: SAS Data Integration Studio Take Control with Conditional](https://reader033.vdocuments.mx/reader033/viewer/2022043020/626bc01c79feaf484b13817e/html5/thumbnails/19.jpg)
Outer Job
� Create Dates Parameter – code node
� Loop
� inner job
� Loop End
![Page 20: SAS Data Integration Studio Take Control with Conditional](https://reader033.vdocuments.mx/reader033/viewer/2022043020/626bc01c79feaf484b13817e/html5/thumbnails/20.jpg)
Outer Job – Create Dates Parms
� read Control table to get last date
� data set with dates to process
![Page 21: SAS Data Integration Studio Take Control with Conditional](https://reader033.vdocuments.mx/reader033/viewer/2022043020/626bc01c79feaf484b13817e/html5/thumbnails/21.jpg)
Outer Job – Create Dates Parms
� right click on green grid, output data set of xform
� properties
� when the job runs, output data set will have n rows
![Page 22: SAS Data Integration Studio Take Control with Conditional](https://reader033.vdocuments.mx/reader033/viewer/2022043020/626bc01c79feaf484b13817e/html5/thumbnails/22.jpg)
Outer Job – Create Dates Parms
� name the output data set
![Page 23: SAS Data Integration Studio Take Control with Conditional](https://reader033.vdocuments.mx/reader033/viewer/2022043020/626bc01c79feaf484b13817e/html5/thumbnails/23.jpg)
Outer Job – Create Dates Parms
� define the file_dt column
![Page 24: SAS Data Integration Studio Take Control with Conditional](https://reader033.vdocuments.mx/reader033/viewer/2022043020/626bc01c79feaf484b13817e/html5/thumbnails/24.jpg)
Outer Job – Loop, Inner Job
� order is important – DIS will help you
� parameter data set
� Loop transformation
� Inner Job
![Page 25: SAS Data Integration Studio Take Control with Conditional](https://reader033.vdocuments.mx/reader033/viewer/2022043020/626bc01c79feaf484b13817e/html5/thumbnails/25.jpg)
Outer Job – Loop, Inner Job
� DIS “sees” data set columns and Inner Job parameter
� Loop Properties
� map the parameter
� select source column
� add Loop End, save
![Page 26: SAS Data Integration Studio Take Control with Conditional](https://reader033.vdocuments.mx/reader033/viewer/2022043020/626bc01c79feaf484b13817e/html5/thumbnails/26.jpg)
Outer Job – Loop code
� 316 lines of goobly-gook
� Inner Job code is in Loop xform
![Page 27: SAS Data Integration Studio Take Control with Conditional](https://reader033.vdocuments.mx/reader033/viewer/2022043020/626bc01c79feaf484b13817e/html5/thumbnails/27.jpg)
Finished Result
� generated Loop transformation code included Inner Job code
� recall the Job folder display
� only the Outer Job was deployed
� if changes are made to either job...
� must re-deploy Outer Job
� multiple jobs within a loop
� multiple parameters
� execute loops in parallel in grid environment
![Page 28: SAS Data Integration Studio Take Control with Conditional](https://reader033.vdocuments.mx/reader033/viewer/2022043020/626bc01c79feaf484b13817e/html5/thumbnails/28.jpg)
Unexpected Behavior - kinda
![Page 29: SAS Data Integration Studio Take Control with Conditional](https://reader033.vdocuments.mx/reader033/viewer/2022043020/626bc01c79feaf484b13817e/html5/thumbnails/29.jpg)
Unexpected Behavior - kinda
![Page 30: SAS Data Integration Studio Take Control with Conditional](https://reader033.vdocuments.mx/reader033/viewer/2022043020/626bc01c79feaf484b13817e/html5/thumbnails/30.jpg)
Wrap
� DIS Loop & Conditional Transformations
� stay in the DIS envelope
� metadata trail
� maintenance
� non-linear processing
Harry Droogendyk [email protected]
www.stratia.ca