simple data automation using base sas in a ... group...automation best practices: • always test...
TRANSCRIPT
SIMPLE DATA AUTOMATION USING BASE SAS IN A WINDOWS ENVIRONMENT
Jennet Gambino
Bond Brand Loyalty
“You’re either the one that creates
the automation or you’re getting
automated.”
— Tom Preston-Werner
Why do we automate data tasks?
• Increased productivity
• Reduced direct human
labor cost and expenses
• Improved quality and
consistency of output
Client customer transactional files and survey
response data files originating from multiple
application systems will need to be
extracted, transformed and load (ETL) at
specific time daily.
CHALLENGES:
• Multiple files are coming from different systems at
different time (daily or hourly)
• Data processing business rules are massive
• Need email alerts to notify someone if any of files
are missing or if there are any issues with the content
SAS Process
ReportsMFT Server
Reports/ Error Log
What a manual work flow looks like
Check that each file is
posted on the ftp
hourly
Other
Systems/
Applications
At specific time of the
day submit each SAS
code needed to
transform and generate
data files
Manual
Data Upload
SAS Process
Reports / Emails
(Automated)MFT
Server
Reports / Error Log
Simple Workload Automation with SAS Processing
Scheduled Data Feed
(Automated)Other
Systems
Applications
Scheduled SAS Batch
Processing (Automated)
Email File
Issues
Scheduled
Data Feed
(Automated)
SETUP REQUIREMENTS:
• Managed File Transfer (MFT) tasks
• SAS code (performing all the business rule/ data
processing)
• Windows Task Scheduler (server)
• Windows batch file script calling the necessary SAS
code and program (ex. Content of Step1_runSAS.bat)
"C:\PROGRAM FILES\SASHOME\SASFOUNDATION\9.X\SAS.EXE“ -SYSIN
T:\DATA_CLEANER.SAS -CONFIG
"C:\PROGRAM FILES\SASHOME\SASFOUNDATION\9.X\SASV9.CFG"
THE WINDOWS TASK SCHEDULER
1. At the windows explorer, search “Task Scheduler”,
2. Choose Action in the menu, create basic task, fill all the necessary information,
name and description
Create a Basic Task
THE WINDOWS TASK SCHEDULER
3. Next 2 screens, choose the frequency (and the time if applicable)
Create a Basic Task
THE WINDOWS TASK SCHEDULER
3. Then, next 2 screens, will be the Action section settings. Enter the location of the
windows batch script.
Create a Basic Task
THE WINDOWS TASK SCHEDULER
3. Final screen, review the summary of the tasks , then complete the settings by
clicking on the Finish button.
Create a Basic Task
OUTPUT OF THE SCHEDULED TASK
• Formatted data files needed by other systems
• .LST file and output file containing all output information
• Emailed report(s) only if applicable
Create a Basic Task
AUTOMATION BEST PRACTICES:
• Always test all your SAS codes manually before
scheduling the automated tasks.
• Create the overall data flow chart, data processing and
schedule information as part of your documentation.
• Obtain all the necessary approvals from your clients.
Think about data security and privacy.
• Before going “LIVE” in the server, schedule a performance
test with your IT department.
Me after a successful automation deployment.