sas 9 3 and subversion integration with sas data integration studio

make connections • share ideas • be inspired

Copyright © 2012, SAS Institute Inc. All rights reserved.www.sasprofessionals.net

SAS 9.3 and Subversion Integration with SAS Data Integration Studio

Adam Upton – Lead SAS Developer, Aviva Health

http://www.sasprofessionals.net/



Why Use Integrated Versioning?

o Backup/Restore. o Files are saved as they are edited, and you can jump to any moment

in time. o Revert back. o Sometimes changes made over time have consequences that only

become apparent many versions later. So this allows these changes to be “Thrown Away”, allowing the revival of a working version.

o Development. o Can work rapidly changing and reverting back to older versions, in a

“throw away fashion”.



Why Use Integrated Versioning? (Cont.)

o Synchronization. o Lets people share files and stay up-to-date with the latest version

o Track Changes. o Notes explaining the changes over time, rationale etc. Problem

solving.o Ownership. o Creating owners and accountability.



Where We Were (Aviva UK Health)?

o Background where we were: o SAS 9.1.3, EDIEBI installation, where DI Studio 3.4 was used widely

for the creation of a data warehouse.o Multiple projects in flight with up to 6 developers working on DI jobs

concurrently, utilizing change management functionality.o Rapid changing and development.o Daily metadata backups.o Legacy batch code.



Where We Wanted To Be?

o Despite using change management, DI notes and taking regular metadata backups, as were our processes using SAS 9.1.3. This had the following issues we wanted to address:o Ability to roll back specific objects. Restoring from a metadata backup

was good for backing out a change, however this also meant that any other objects developed were also rolled back. Thus causing repeated development effort.

o Comparing job versions etc. was practically impossible, without creating a bespoke process for packaging DI code and then reimporting with differing names, object IDs etc.

o Code comparison issues, comparing code across development levels.o No single point/repository for all related changes.



What Were Our Options?

o Do Nothing (unlikely)!o We were upgrading our existing SAS 9.1.3 EBIEDI installations to

SAS 9.3 anyway, so exploring/exploiting new functionality seemed sensible.

o So time to take action.o SAS released integration capabilities built straight in to DI Studio 4.3.o CVS and SVN work with DI Studio 4.3 out of the box.o They are both free.o SVN has out of the box install and configuration and is rapidly

becoming the industry standard.o Open source risks, versus no version control.



Time For Action - Implementing SVNo I found the following paper that describes a simple step by step

guide ‘How To Setup Version Control for SAS 9.3 Data Integration Studio Using CollabNet Subversion Edge (SVN)’.

o http://www.scorpiosoftware.com.au/2012/01/configuring-version-control-in-sas-9-3-enterprise-data-integration-server/

o After following the step-by-step guide, as above, which took no more than 20mins. Here is a list of the prerequisites.o A machine to host Apache Subversion Server on either a Windows,

Linux or Solaris platform, with enough rights to install software.o Local clients to include:o SAS DI Studio 4.3 (at least) on a SAS 9.3 installation.o Third Party SVN product such as TortoiseSVN.o Write access to a local drive that is required for local repository.

http://www.scorpiosoftware.com.au/2012/01/configuring-version-control-in-sas-9-3-enterprise-data-integration-server/

http://www.scorpiosoftware.com.au/2012/01/configuring-version-control-in-sas-9-3-enterprise-data-integration-server/



Take Note And Be Prepared

o With your new environment ensure that you note where your local repository will reside.

o Ensure that you document where the SVN server/repository is.o Beware that when packaging, that memory limitations may mean that

the packaging will fail to archive, especially large ones, so the following recommendations should be used:o Changing parameters in the ‘distudio.ini’ file for memory options, such as:

o JavaArgs_1=Xmx1024m JavaArgs_13=-XX:MaxPermSize=128mo Create packages on your server side, this will typically have more RAM.o Breakdown the metadata into smaller packages and check that you are

packaging only the required objects.



Getting Started With Version Control And DI Studio

o Baseline your environment.o Applying to an existing environment.

o Archiving the packages.



Archiving A Packageo Select ‘Custom select dependent objects’.o Ensures consistent approach to packaging.o Allows multiple objects traversing folder structure to be included in each

package.o Note: ‘Exporting SAS code for jobs’ appears to not export the actual code, rather

creating a note in the metadata folder of the job you want to export the code for. The notes are then used for comparison (this may create unwanted objects in your environment). These notes are also not dynamically updated, if a job is amended, so may cause confusion.



Archiving A Package - cont.

o Selecting the objects to be archived.



Archiving A Package - cont.

o Package Summary.



Archiving A Package - Behind The Scenes

Local Repository

Remote SVN Repsitory

Package committed to

local repository.

Package committed to

Remote repository and

archive updated.



Comparing Versions

o Go compare ! o Ability to compare in a very flexible manor:o Object to object/archived object/packaged object comparison.o Archive to archive/object/packaged object.

o Automatic filtering:o Comparing a selected object/archive against an archived package will

automatically filter out for only the packages that contain the same object/objects.

o Highlighting differences :o Clear highlighting of differences between packages.o Owner/author listed, greater accountability.



Comparing Versions -cont.

• Select a package to compare.

• Compare to archive.

• A Filtered list containing only those packages that contain the object/objects that are to be compared.

• Archives package selected for comparison.• Comparison screen highlighting areas of the package that differ.



Comparing A Package - Behind The Scenes

Local Repository

Remote SVN Repsitory

If package being

compared in archive, is not

in local repository,

then copied from server.Archive

information read by client

plugin



Archive Administration

o Editing in situ.o Define a package of objects and once built this can readily be re-archived,

automatically include the objects previously selected in the archive.o House keeping:o Delete obsolete packages, or those created by mistake.o Changed names, notes.

o Important, notes using SVN:o When performing housekeeping, the actual metadata (or archive file is

updated, defining the information available on the plugin). Not the packages themselves.



Archive Administration

o Metadata available v’s repository information.



Potential Outside of DI Studio

o Roll out approach beyond DI to base SAS legacy codes, offering the following benefits:o Ability to lock source table whilst already being change, much like the DI

Studio change management functionality.o Single repository for all source data, lowering environment complexity.o Create SAS Batch jobs comparing deployed packages automatically using

SVN commands.


sas 9 3 and subversion integration with sas data integration studio

Documents