SAP Data Services

The Future Of Enterprise ETL ?

Housekeeping

• Venue

• Tea / Coffee

• Comfort Break (maybe)

• Presentations

• Duration

• Questions welcome

Speaker Bio

• Richard Munn

• DBA @ Nationwide Building Society

• Experience: 15yrs IT, 11yrs SQLServer

• Blog: http://sqlmunkee.blogspot.com

• Twitter: @sqlmunkee

• Mail: SQLMunkee@gmail.com

• Social status: Northener

Presentations

• 1 - Quick comparisons between SSIS and SDS

• 2 - Overview / Explanation

• 3 – Demonstration (I hope)

• Knowledge of SSIS / ETL terminology is assumed• Disclaimer

<presentation1>

SSIS vs. SDS

Paradigm Shift ?

The Future Is Yellow

• SAP Data Services (SDS) is an Enterprise Solution for ETL.

• Platform agnostic

• Repository based ETL development.

• Any object can be shared (I think)

• Supports CheckOut /CheckIn usage

Comparisons

Feature SSIS SDS

ETL Operations Yes Yes

Sharing No Yes

Wide range of standard objects Yes No

Custom objects Yes Yes

Standalone development Yes Yes (but…)

Try…Catch Custom Built-in

Metadata and Lineage No Yes

“Push down” WTF ? Yes

Script support Yes Yes

Script Language VB.net / C# (2008) Python

Job Execution

• SSIS just requires a command line (DTExec)

• SDS (even in development) needs:– A job to contain Workflow / Dataflow– A job server to execute the job

• More infrastructure is required to provide service

Restartable Jobs

• Must be hand-crafted in SDS

• Complex logic required with multiple Workflows, Try…Catch, Parameters, Variables, etc…

• Built-in with SSIS

Challenges

• Requires dedicated, experienced developers and support staff (IMHO)

• Does the job in a controlled and ordered manner.

• Not as “user-friendly” as SSIS.

</presentation1>

<presentation2>

All aboard the big yellow fun bus !

Key Elements

• Repository / Repositories

• Data Services Designer

• Job Server / Server Group

• Management Console

Repositories (1)

• Somewhere to store your objects:– Projects– Jobs– DataSources– DataFlows– WorkFlows– Transforms

More on these later…

Repositories (2)

• 4 types:

– Local - Held on a local DBMS

– ‘Personal’ * - Work in progress

– Central - Items to be used / shared

– Secure - Only Admin access (this can be a problem)

* - not an SAP option, more ‘best practice’

Data Services Designer

• Local installation of Development Environment for creating the objects to store in the Repository

• …but can be delivered over Citrix or Softgrid/AppV

• Fairly simple in look and feel, but actually quite complex

• Would anyone like to take a look ?

Starting The Designer

Jobs (1)

• Anything you want to do is treated as a ‘job’ in SDS

• Jobs usually contain WorkFlows and DataFlows (but you can have one without the other, unlike SSIS)

• 2 types:– Batch Job– Real-Time Job

Jobs (2)

• Batch Job– Commonly single unit-of-work– Overnight / Off-peak ETL processing

• Real-Time Job– Creates a Web service– Typically 3Flow/2Transform data movement– Simple operations ONLY please

Job Servers

• Actually runs your job (even if you’re moving data about locally)

• Multiple Job Servers can function as a “Server Group”

• Server Groups share the submitted jobs between them, but once a job starts it can’t move servers (ask me about this)

Management Console

• Single point of administration for…– Users– Job Servers (reporting)– Repositories

• Browser-based, so requires Apache Tomcat (later versions support other web servers like IIS)

• Typically you’ll have one DSMC per landscape unit

I hope there’s a WhiteBoard…

• Shall we draw some pictures of what it might look like ?

</presentation2>

Comfort Break

If we haven’t already had one..

<presentation3>

Let’s go look see…

</presentation3>

Questions ?

GOTO: Pub()