artofbi odi10g salesforce integration deep dive (1)

ORACLE DATA INTEGRATOR (ODI) 10G

AND SALESFORCE.COM INTEGRATION

PART 1

Written by: Christian Screen ArtOfBI.com (The Art of Business Intelligence)

Version 1 (2010/03/22)

The Art of BI (http://www.artofbi.com) Oracle Data Integrator (ODI) and SalesForce.com Integration

Last Saved: 10/8/2010 11:34 PM 2

http://www.artofbi.com

Table of Contents ABOUT THIS TUTORIAL .......................................................................................................................................... 4

PREFACE .............................................................................................................................................................. 4

OBJECTIVE ............................................................................................................................................................ 4

SOFTWARE DEPENDENCIES ....................................................................................................................................... 4

PROJECTED COMPLETION TIME .................................................................................................................................. 4

PRE-REQUISITES ..................................................................................................................................................... 4

INSTALL THE ODI 10.3.5.5 PATCH ........................................................................................................................... 6

DIRECTION ............................................................................................................................................................ 6

INSTALL THE JAVA JDK ........................................................................................................................................... 7

DIRECTION ............................................................................................................................................................ 7

FILE DOWNLOAD & INSTALLATION .............................................................................................................................. 7

SETTING-UP THE SAMPLE DATA SOURCE ............................................................................................................... 8

DIRECTION ............................................................................................................................................................ 8

TUTORIAL FILE DOWNLOAD & INSTALLATION................................................................................................................. 8

SETTING-UP SALESFORCE.COM 3RD

PARTY SOFTWARE........................................................................................... 9

DIRECTION ............................................................................................................................................................ 9

SOFTWARE DOWNLOADS.......................................................................................................................................... 9

INSTALL TUTORIAL FILES / DATA PROCEDURE .............................................................................................................. 10

TEST SALESFORCE.COM API LOGIN CREDENTIALS ......................................................................................................... 10

CREATE WORK REPOSITORY DATABASE SCHEMA ................................................................................................ 13

DIRECTION .......................................................................................................................................................... 13

CREATE SCHEMA .................................................................................................................................................. 13

SETUP TOPOLOGY MANAGER .............................................................................................................................. 14

DIRECTION .......................................................................................................................................................... 14

CREATE REPOSITORY CONNECTION LOGIN .................................................................................................................. 14

CREATE WORK REPOSITORY .................................................................................................................................... 15

ADD SALESFORCE.COM TECHNOLOGY ........................................................................................................................ 17

CREATE AN AGENT ............................................................................................................................................... 18

CREATE A CONTEXT ............................................................................................................................................... 19

CREATE A LOGICAL AGENT ...................................................................................................................................... 20

CONFIGURE THE AGENT ....................................................................................................................................... 22

DIRECTION .......................................................................................................................................................... 22

CONFIGURE AGENT BATCH FILE ............................................................................................................................... 22

CREATE AGENT START/STOP BATCH SCRIPTS ............................................................................................................... 23

SETUP DATA SERVERS & DATA STORES ................................................................................................................ 25

ADD FILE DATA SERVER ......................................................................................................................................... 25

ADD SALESFORCE.COM TECHNOLOGY DATA SERVER ..................................................................................................... 26

PREPARE THE INTEGRATION IN DESIGNER ........................................................................................................... 28

DIRECTION .......................................................................................................................................................... 28

INITIAL DESIGNER CONNECTION ............................................................................................................................... 28


Last Saved: 10/8/2010 11:34 PM 3


CREATE A PROJECT ................................................................................................................................................ 29

IMPORT THE KNOWLEDGE MODULES ......................................................................................................................... 29

CREATE A FILE SYSTEM LOG FOLDER .......................................................................................................................... 30

CONFIGURE RKM OPTIONS .................................................................................................................................... 30

CONFIGURE IKM OPTIONS ..................................................................................................................................... 30

CREATE THE MODELS ............................................................................................................................................ 31

GO, GO INTEGRATION .......................................................................................................................................... 35

DIRECTION .......................................................................................................................................................... 35

ADD FILE DATASTORE ............................................................................................................................................ 35

CREATE THE INTERFACE .......................................................................................................................................... 36

VERIFY SALESFORCE.COM CONTACTS LOAD......................................................................................................... 40

DIRECTION .......................................................................................................................................................... 40

VERIFY ............................................................................................................................................................... 40

CONCLUSION ....................................................................................................................................................... 42

REFERENCES ......................................................................................................................................................... 42


Last Saved: 10/8/2010 11:34 PM 4


About this Tutorial PREFACE In the past several years SalesForce.com (SFDC) has become a mecca for small to large organizations to manage their customer relationship management (CRM) data. As such, the core data often resides on an island, which is unavailable for integration into a larger corporate enterprise reporting strategy to analyze data seamlessly as a whole. One approach to integrating this mass collection of sales opportunity and CRM data from SalesForce.com into an enterprise data warehouse environment is the utilization of Oracle Data Integrator (ODI). ODI is an enterprise Extract Load and Transformation (ELT) tool. ODI offers modularized integrations of heterogeneous data sources via the use of Knowledge Modules (KMs). One such grouping of KMs developed circa 2006 near the time of Oracle purchase of the tool from Sunopsis is one designed for Salesforce.com integration. However after the purchase, no further documentation of the integration between ODI and Salesforce.com was released. This functionality is highly sought after.

OBJECTIVE This tutorial was developed for users with an intermediate understanding of ODI, data warehousing, ETL (or ELT as ODI labels itself), and Salesforce.com. During this exercise a single comma-separated values (CSV) file containing sales contacts will be interfaced with ODI and pushed into the users SFDC contacts list. The most important part of this tutorial is getting the set-up steps configured correctly. There may seem to be a lot of initial steps but without them the integration is not possible. Please follow the first several steps very closely.

SOFTWARE DEPENDENCIES In order to use this tutorial the following software must be installed and functioning properly:

Windows Server 2003 Oracle database (Oracle XE recommended) Oracle Data Integrator (ODI) Client/Server SalesForce.com Software

PROJECTED COMPLETION TIME Assuming that all prerequisites have been established, the exercise completion time is 90 minutes.

PRE-REQUISITES The SalesForce.com KMs provided by ODI are rough, initial releases and have not had much in the way of updates since 2006. Due to this lapse, the following items must be secured prior to continuing in this exercise. The instruction outlined in this tutorial was created for a user on a


Last Saved: 10/8/2010 11:34 PM 5


Windows Server 2003 environment. By design the SalesForce.com KMs utilize a cmd.exe execution in the KM XML files to run JAR files. Clearly this can be updated by the developer and saved for another environment such as Linux outside of this tutorial but this tutorial lives in a Windows environment. The following pre-requites must be fulfilled before continuing:

Installed Oracle Data Integrator (ODI Quality or Profiler not necessary) o Assumes a Master Repository is already configured

Java JDK 1.5 or Greater Download and install the ODI 10.3.5.5 patch

o Patch available on Oracle Support or Mirrored here o The most important part of this patch is the SalesForce.com RKM

A SalesForce.com Developer Edition Account (Free) o Registration Page http://www.developerforce.com/events/regular/registration.php o Obtain (reset) security token for API login access

Setup > Reset Security Token

Download latest ApexDataLoader.exe application o Login to your SaleForce.com Developer Edition account o Click Setup > Data Management > Data Loader

Download the open source SalesForce.com SForceDataLoader application Download the SalesForce.com QuickStart.jar file Download the Sample Code provided with this tutorial

As part of the prerequisites for this tutorial your ODI environment should be setup with a confirmed connection to a Master Repository. It is assumed that you as the conductor of this exercise have Administrator-like rights and privileges to all environments surrounding the interaction. Although full Administrator rights are not necessary for all steps, many items such as setting folder security, etc. will require a high-level of privileges.


Last Saved: 10/8/2010 11:34 PM 6


Install the ODI 10.3.5.5 Patch DIRECTION The ODI 10.3.x version had an inherent problem connecting to SalesForce.com. In the 10.3.5.5 patch release this problem was fixed with new Java coding in the SalesForce.com RKM. The patch fixes several issues outside of the SalesForce.com connectivity issue. Regardless, installation of the patch should be considered for long-term compatibility though it is not absolutely necessary in order to complete this tutorial. The installation of the patch is outside the scope of this article


Last Saved: 10/8/2010 11:34 PM 7


Install the Java JDK DIRECTION The ODI 10.3.x version comes with only the Java Runtime Environment (version 1.4.2) which does not support web service calls. In order to communicate with the SalesForce.com this is clearly necessary.

FILE DOWNLOAD & INSTALLATION The Java JDK can be downloaded from the Oracle/Sun Microsystems Java Developer website, http://java.sun.com/javase/downloads/widget/jdk6.jsp (JDK 5 is synonymous with 1.5 and JDK 6 is synonymous 1.6).

1. Download and install the JDK 1.5 or greater on the same machine as the ODI installation from, http://java.sun.com/javase/downloads/widget/jdk6.jsp

2. Open your computers System Properties prompt by right-clicking on My Computer from Windows Explorer.

3. Select the Advanced tab 4. Click the Environment Variables button 5. In the System Variables section, locate ODI_JAVA_HOME 6. Change its value to the path of the JDK that you just installed but do not include the bin

path. For example if the JDK installation path is C:\Program Files\Java\jdk1.6.0_16 the enter C:\Program Files\Java\jdk1.6.0_16. Do not include the bin folder in this path as the JAVA_HOME system variable does. ODI adds the bin path when it references the Java executables.

7. Restart the server to ensure that this environment variable change is set.


Last Saved: 10/8/2010 11:34 PM 8


Setting-Up the Sample Data Source DIRECTION Based on the ODI documentation and the KMs provided by ODI, all integration between ODI and SaleForce.com is conducted via file interfaces (see ODI KMs Reference Guide). There are no immediate relational database to SalesForce.com KMs available from ODI. As such, this tutorial uses a simple CSV file containing contact records as the data source from which we will update the SalesForce.com contacts (object) list.

TUTORIAL FILE DOWNLOAD & INSTALLATION Please download the tutorial sample code ZIP file from, http://www.artofbi.com/downloads/ODI_SFDC_Code.zip to a temp folder on your local machine.

1. Unzip the contents of the ODI_SFDC_Code.zip file into a temporary directory. 2. Create a new folder called FileStorage on the main drive that Oracle Data Integrator is

installed (ex: C:\FileStorage). 3. Copy the contents of the extracted sample code FileStorage folder into your newly

created directory. This should contain the data file(s) for this tutorial.

4. Open the new folders permission and security tab and set Modify permissions on the folder to Everyone.

5. Regardless of whether or not you were able to download the ODI 10.3.5.5 patch from Oracle support you will need to use the adjusted SalesForce.com RKM contained in the sample code for this article as it fixes a naming issue the patch did not. This file, KM_RKM Salesforce.com.xml, is located in the ODI\oracledi\impexp\ folder when the sample code zip file is extracted.

a. Make a back-up copy of the original file in the ODI directory before you overwrite it.

b. Copy and paste the KM_RKM Salesforce.com.xml file from the sample code extract directory into the \oracledi\impexp\ folder of your ODI installation. This is the default location for all ODI KMs.


Last Saved: 10/8/2010 11:34 PM 9


Setting-Up SalesForce.com 3rd Party Software DIRECTION Two Java executables from SalesForce.com are required for the ODI SalesForce.com integration, the SForceDataLoader.jar and Quickstart.jar. The SalesForce.com SForceDataLoader is an out-dated application developed by SalesForce.com in 2006. It has recently been superseded by its predecessor application, ApexDataLoader. However, the ODI integration coding and modules are still based on the legacy open-source application and does not integrate with ApexDataLoader, only SForceDataLoader, so the latter must be obtained and installed.

The Quickstart.jar file provides driver access to ODI and will be downloaded and placed in the driver path of the ODI install directory.

As an ancillary tool to test that your SalesForce.com API login credentials are connecting properly you will download the ApexDataLoader.exe from SalesForce.com and install it on your local machine. Unfortunately, there is no way to test your SalesForce.com login credentials from within ODI. The credentials are only passed during runtime execution. This SFDC tool will allow you to test that your login is working. It also has other extraction benefits outside of this tutorials scope.

SOFTWARE DOWNLOADS SForceDataLoader is an open-source Java based application from SFDC and can be downloaded freely at http://sourceforge.net/projects/sforce-app-dl/files/SforceDataLoader/7.0%20zip%20with%20upsert/sforcedataloader70.zip/download

Quickstart.jar is a Java based executable the may be downloaded from SalesForce.com at http://adnsandbox.com/appexchange/quickstartJava.zip the information page on the Quickstart.jar file usage can be found at http://wiki.developerforce.com/index.php/Java_Quick_Start

The SalesForce.com ApexDataLoader.exe is an installation executable that can be downloaded from your profile dashboard on the SalesForce.com website by clicking Setup > Data Management > Data Loader.


Last Saved: 10/8/2010 11:34 PM 10


INSTALL TUTORIAL FILES / DATA PROCEDURE The downloaded sforcedataloader70.zip file contains a folder structure with several files relating to the operation of the software. The zip file is platform independent however for the purpose of this tutorial the contents should be extracted to the root of the C:\ drive. Please note that the ODI SFDC KMs have several bugs that conflict with spaces in folder and file names. Extracting to the base of the C:\ drive prevents previously detected issues from arising during this tutorial. Next,

1. Extract the quickstartjava.zip into a temporary directory. 2. Copy and paste the quickstart.jar file into the \drivers\ folder of your ODI installation

path. On the C:\ drive this should be C:\ODI\oracledi\drivers\. 3. Locate the downloaded ApexDataLoader.exe file and run the executable to install the

application. Accept the installation defaults and default installation directory which is usually C:\Program Files\.... and complete the install.

TEST SALESFORCE.COM API LOGIN CREDENTIALS After the installation of the ApexDataLoader file, a new program should be installed under Programs > salesforce.com > Apex Data Loader 18.

1. Launch this application. Click Cancel on the default prompt. 2. On the menu bar, select the Settings menu and click Settings. 3. Ensure the following settings are configured:

o Server host: https://www.salesforce.com o Reset URL on Login: Checked


Last Saved: 10/8/2010 11:34 PM 11


2. Click OK. 3. From the file menu, choose Insert. 4. On the login prompt enter your SalesForce.com username which is usually your email

address. For your password you will need your concatenated SalesForce.com password and security token. For example if your password is password1 and your security token is ABCDEFGH then your API combination password would be password1ABCDEFGH. Please reset your security token via the SalesForce.com dashboard if you do not have it ready. It will then be sent to you via email.

Once your API password has been entered, click Log in to confirm your connection. If the login is successful you will know that your API credentials are working. These will be the credentials used for the SalesForce.com Technology Data Server in ODI.


Last Saved: 10/8/2010 11:34 PM 12



Last Saved: 10/8/2010 11:34 PM 13


Create Work Repository Database Schema DIRECTION In order to maintain a clean work environment for this tutorial a new ODI work repository will be created. An Oracle XE database schema called ODI_WORK will be created for this purpose. Since ODI_WORK is the schema owner its credentials will be used. If you are using any alternate users for an existing schema due to any permission restrictions you may have, please ensure that the user has all of the necessary create, read, and write privileges on the schema if not using the schema owner credentials. It is assumed a Master Repository has already been created and the credentials to it known.

CREATE SCHEMA Connect to your Oracle XE database instance using your favorite IDE or SQL*PLUS using a system user and create a new user, ODI_WORK with a password of odiwork. Be sure that CONNECT and RESOURCE roles are granted to this user.


Last Saved: 10/8/2010 11:34 PM 14


Setup Topology Manager DIRECTION The topology manager contains all information about technologies ODI uses and the data server connections for those technologies. We must establish a new login, configure the work repository, and set-up data stores for the two technologies that this tutorial will use.

CREATE REPOSITORY CONNECTION LOGIN Launch ODI Topology Manager from the Start > All Programs > Oracle > ODI menu. Upon starting the Topology Manager a prompt will appear for you to login. Create a new repository connection by clicking the new icon.

For the Oracle Data Integrator Connection section please enter the following: Login Name: ODI_SFDC_DEMO. User: SUPERVISOR Password: SUNOPSIS

For the Database Connection (Master Repository) section please enter the database connectivity credentials for the relational db/schema that holds your ODI Master Repository data.


Last Saved: 10/8/2010 11:34 PM 15


Click the Test button to confirm your credentials are accurate. Click OK if a successful connection is made.

Now on the login prompt, enter the username and password you just created for the repository credentials. Click OK to login.

CREATE WORK REPOSITORY Once logged into Topology Manager, locate the tabs running along the bottom of the left-pane of this IDE. Click the Repositories tab that looks like a book with AB on it.

Find the tab for Repositories AB at the bottom and click on it.

1. In the left-pane right-click on Work Repositories and click Insert Work Repository. 2. You are prompted to enter information for a new Data Server connection. Enter the

following information: Name: ODI_SFDC_DEMO_DS Technology: [select the DB type of your Work Repository (ex: Oracle)] Instance / dblink (Data Server): [the name of your database server, ex: localhost] User: ODI_WORK Password: odiwork


Last Saved: 10/8/2010 11:34 PM 16


3. Click the JDBC tab and enter the following based on the database and SID of your instance: JDBC Driver: oracle.jdbc.driver.OracleDriver JDBC Url: jdbc:oracle:thin:@demomachine:1521:xe

4. Click the Test button to test the connection. When prompted with selection for an Agent just click Test. If successful then the connection to the database is good.

5. Click OK. A prompt appears and requires you to enter the Work Repository information which creates the ODI reference.

6. Enter the following values in this prompt: In the ID field enter 222 or another arbitrary number as this is just a reference. In the Type dropdown list select Development. In the Name field enter ODI_SFDC_DEMO_WORK_REP.

7. Click OK.

The Work Repository should now be successfully created.


Last Saved: 10/8/2010 11:34 PM 17


ADD SALESFORCE.COM TECHNOLOGY In Topology Manager select the Physical Architecture tab from the bottom of the left pane.

1. Expand Technologies. a. If the SalesForce.com Technology is already available in your list of

Technologies you may skip this section and proceed to the next section. 2. Right-click Technologies, click Open in New Window. The new window opens as a

window pane on the right-hand side of the IDE. 3. Click the second icon Import Technology.

4. From the list select Salesforce.com and click OK.

5. Verify that Salesforce.com is now in your Technology list.


Last Saved: 10/8/2010 11:34 PM 18


CREATE AN AGENT An agent is required by ODI to execute most KMs and needs a server process to run under. If no agent currently exists then one must be created. There are two types of agents. We will be created a non-scheduler agent.

In the Physical Architecture tab, right-click Agents and click Insert Agent.

In the new Agent prompt enter the following: Name: ODI_SFDC_AGENT Host: [Server ODI is installed on, ex: localhost] Port: [Leave with default value]

o You may choose any port that is not being used. However, if you change from the default port you will need to modify the port for the agent in either the config file or a batch file that launches the agent by name as a command-line attribute.


Last Saved: 10/8/2010 11:34 PM 19


You cannot test the agent yet as we must first set up a context and a logical agent.

Click OK.

CREATE A CONTEXT As something akin to grouping like objects, ODI has a context object. We will create one specific to the SFDC integration. Towards the bottom of the left-pane click the Contexts tab with the icon that looks like a mountain with a flag atop it.

1. Click the icon for Insert Context.

2. In the new Context prompt enter the following: Name: ODI_SFDC_CONTEXT


Last Saved: 10/8/2010 11:34 PM 20


Code: ODI_SFDC_CONTEXT

3. Check the checkbox for Default. Accept the prompt. 4. Click OK.

CREATE A LOGICAL AGENT Navigate to the third tab, Logical Architecture, at the bottom of the left-pane. This will be used to logically link the context to the physical Agent.

1. Right-click on Agents and click Insert Logical Agent.

2. In the Name field enter ODI_SFDC_LOGICAL_AGENT 3. On the Definition tab, select the ODI_SFDC_AGENT Physical Agent for the

ODI_SFDC_CONTEXT.


Last Saved: 10/8/2010 11:34 PM 21


4. Click OK.


Last Saved: 10/8/2010 11:34 PM 22


Configure the Agent DIRECTION Since our ODI installation was fresh and no agent had yet been created our Agent batch file has not yet been updated. We must update this batch file in order for the Agent we have created to access both the Master Repository and Work Repository.

CONFIGURE AGENT BATCH FILE Open the batch file within the ODI installation path \oracledi\bin\odiparams.bat, in Edit mode. Create a backup of the file before modifying it.

The section under rem Repository Connection Information must be edited in order to link the newly created Master and Work Repositories to the Agent. Change the following settings as outlined below based on your installation and server:

set ODI_SECU_DRIVER=oracle.jdbc.driver.OracleDriver set ODI_SECU_URL=jdbc:oracle thin:@demomachine:1521:xe set ODI_SECU_USER=ODI_MASTER set ODI_SECU_ENCODED_PASS=fDyXMwUP4Dv5W89gwI,p set ODI_SECU_WORK_REP=ODI_SFDC_DEMO_WORK_REP set ODI_USER=SUPERVISOR set ODI_ENCODED_PASS=LELKIELGLJMDLKMGHEHJDBGBGFDGGH

The variable ODI_SECU_USER should contain the value for the Master Repository schema user. The ODI_SECU_ENCODED_PASS variable should contain the value for the Master Repository schema user password in an encrypted format To encrypt the password you must run the ODI Agent encode function from the command-line. To do so, launch the MS-DOS command prompt from Start > Run, change the directory to \oracledi\bin\ and execute the following:

Agent encode [Your Password] o The screenshot below shows several passwords being encrypted on the machine

this tutorial was written on.


Last Saved: 10/8/2010 11:34 PM 23


Update the password in the batch script.

The ODI_SECU_WORK_REP variable should contain the name of the Work Repository. And, the ODI_USER and ODI_ENCODED_PASS should contain the respective credentials to log in to the ODI Repository. If you did not stray from this tutorial using (SUPERVISOR/SUNOPSIS) you can leave these as they are.

CREATE AGENT START/STOP BATCH SCRIPTS The Agent can be stopped and started by creating simple batch files. If you changed the default port of the Agent you cannot use the default agent batch files to start the agent and can only start the agent when including the port number switch from the command-line. To prep for either scenario, create two batch files with the following lines of code respective to your ODI installation paths.

Create a folder in the root of your ODI_HOME path called _Agent_Scripts and save the scripts here.

Create the first batch file script with the following code and save as start_sfdc.bat: H: cd\ CD ODI\oracledi\bin agent "-PORT=20910" "-NAME=ODI_SFDC_AGENT"

Create the second batch file script with the following code and save as stop_sfdc.bat: H: cd\ CD ODI\oracledi\bin\ agentstop "-port=20910"


Last Saved: 10/8/2010 11:34 PM 24


Run the startAgent.bat file you just created. You may now return to the Topology Manager Physical Agent and test that the Agents connection is working.


Last Saved: 10/8/2010 11:34 PM 25


Setup Data Servers & Data Stores ADD FILE DATA SERVER

1. Open Topology Manager and navigate to the Physical Architecture tab. 2. Expand Technology, right-click on Files and select Insert Data Server. 3. On the Definition tab, for the Name field value enter File_Storage. 4. On the JDBC tab select or enter the following content:

o JDBC Driver: com.sunopsis.jdbc.driver.file.FileDriver o JDBC Url: jdbc:snps:dbfile

5. Click Apply.

The Physical Schema prompt will appear. On the Definition tab the root file Directory where the files reside must be defined. Enter the folder path that you created earlier and copied the tutorial sample data files into. Please note that there is no browse tool so you will need to click inside the dropdown and type in the path. Pay close attention and ensure that you enter a forward slash (not backslash) for the path name, ex: H:/FilesStorage. Be sure to spell it correctly.

6. Click the Context tab. 7. Click the add new row icon which sits to the left of the garbage can icon. The context we

created earlier appears. Now, in the logical schema column a default schema will appear. 8. Click inside of this drop-down, delete its content and enter, SFDC_FILE_SCHEMA.


Last Saved: 10/8/2010 11:34 PM 26


9. Click OK twice.

ADD SALESFORCE.COM TECHNOLOGY DATA SERVER 1. In Topology Manager, expand Technology. 2. Right-click on the Salesforce.com technology and select Insert Data Server. 3. On the Definition tab enter the following information:

Name: SFDC_DS User: [Your SalesForce.com Developer Account Email Address] Password: [Your SalesForce.com Developer API Password (password + security

token)] o Please see the SalesForce.com API document for more information on the use

of this password. o A test was conducted earlier in this article to ensure your credentials were

sound. You may copy and paste them.


Last Saved: 10/8/2010 11:34 PM 27


Please note an undocumented caveat of ODI is that the password field only accepts 35 characters in the password field. For the API password (password + security token) this means that you may need to change your SalesForce.com password and shorten it so that it is less than 11 characters in length as the security token is 24 characters long.

The JDBC does need to be modified as it is not necessary for this type of technology.

4. Click OK and the Physical Schema prompt will appear. 5. On the Context tab, click the new row to the left of the trash can icon to add a new

context row. Select ODI_SFDC_CONTEXT from the Context dropdown list. Then manually enter into the Logical Schema column the value SFDC_SCHEMA.

6. Click OK.


Last Saved: 10/8/2010 11:34 PM 28


Prepare the Integration in Designer DIRECTION The ODI Designer is where most of the magic of this integration will take place. We will set up a project, create and define the file and SalesForce.com models, create a simple interface, and use the interface to load data to the SalesForce.com account.

INITIAL DESIGNER CONNECTION 1. Open ODI Designer from the Start > All Programs > Oracle > ODI menu. 2. When you open the Designer you will need to click the new icon to create a new login.

For the Oracle Data Integrator Connection enter the following: Login Name: SFDC_DEMO User: SUPERVISOR Password: SUNOPSIS

3. Enter the credentials the Master Repository and then select the Work Repository, ODI_SFDC_DEMO_WORK_REP that was created earlier.

4. Check Default Connection 5. Test. Click OK to close then login with SFDC_DEMO.


Last Saved: 10/8/2010 11:34 PM 29


CREATE A PROJECT In Designer, on the Projects tab of the left pane, click the first icon at the top for Insert Project. For the new project prompt enter ODI_SFDC_PROJECT for the Name and Code fields.

Click OK.

IMPORT THE KNOWLEDGE MODULES Right-click on the main project you have just created and choose the Import Knowledge Modules option from Import sub-item.

On the Import Knowledge Modules prompt, make sure that the File Import Directory is set to the ODI_HOME\Oracledi\impexp folder. At this point select the SalesForce.com KMs from the list below. Hold down the ctrl key to select multiple KMs in one sitting. Select the following files from the list:

IKM File to Salesforce (Upsert) RKM Salesforce.com


Last Saved: 10/8/2010 11:34 PM 30


Click OK.

CREATE A FILE SYSTEM LOG FOLDER The SalesForce.com Knowledge Modules allow for logging which aids in troubleshooting efforts. Using Windows Explorer create a folder in the ODI_HOME path called _Logs. In the properties menu for the newly created folder set permissions for Everyone to Modify.

CONFIGURE RKM OPTIONS In the ODI Designer, under the projects tab, expand the new project. Expand Knowledge Modules > Reverse-engineering (RKM) > RKM Salesforce.com to expose the KMs options. Double-click on LOG_FILE_NAME to open up the edit window. Change the Default Value field value to ODI_HOME\_Logs\reverse.log (ex: C:\ODI\_Logs\reverse.log). Click OK. This path should now reflect a non-existing file in the path of the logs folder that you created above.

CONFIGURE IKM OPTIONS In the ODI Designer, under the projects tab, Expand Knowledge Modules > Integration (IKM) > IKM File to Salesforce (Upsert) to expose the KMs options. Double-click on each option listed


Last Saved: 10/8/2010 11:34 PM 31


below and modify its default values replacing ODI_HOME with your ODI installation directory which if installed on the C:\ drive will be C:\ODI:

ERROR_FILE: ODI_HOME\_Logs\sforcedataloader.error JVM_PATH: ODI_HOME\jre\1.4.2\bin LOG_FILE: ODI_HOME\_Logs\sforcedataloader.log RESULT_FILE: ODI_HOME\_Logs\extract.csv SF_PATH: C:\\sforcedataloader SUCCESS_FILE: ODI_HOME\_Logs\sforcedataloader.success

Please note that in the SF_PATH value that the backslash is escaped. This is necessary due to the programming of the KM.

CREATE THE MODELS Models are necessary to contain the Data Stores for each individual object referenced in our soon to be mapped interface of flat file to SalesForce.com contact object. In Designer, navigate to the Models tab in the left-pane by selecting Windows > Show View > Models from the menu bar.

In the Models view click the Insert Model Folder icon and in the new model folder prompt enter SFDC_MODELS in the Name field. Click OK.

Now, we will create two models to be stored inside the SFDC_MODELS folder. One to hold the flat file objects and one to hold the SalesForce.com objects.

Click the icon to Insert Model (4th yellowish icon with a plus). In the new model prompt Definition tab enter the following:

Name: SFDC_MODEL Code: SFDC_TARGET Technology: Salesforce.com Logical Schema: SFDC_SCHEMA


Last Saved: 10/8/2010 11:34 PM 32


On the Reverse tab, click the Customized radio button. Choose the following values from the respective dropdown lists:

Context: ODI_SFDC_CONTEXT Local Agent: ODI_SFDC_LOCIGAL_AGENT Select your KM: RKM Salesforce.com.ODI_SFDC_PROJECT

Now at the very bottom of the Reverse tab, click the Reverse button. This will begin your initial connection via the SalesForce.com API and pull down all existing SalesForce.com objects available under your user account.


Last Saved: 10/8/2010 11:34 PM 33


Click OK or Yes for all prompts that follow after clicking the Reverse button. This can take up to 5 minutes. Click OK to close the new model window.

To create the flat files model, navigate to the Models tab of the left-pane. Click the icon to Insert Model. In the fields and dropdowns select or enter the following values:

Name: SFDC_FLAT_FILE Code: SFDC_FLAT_FILE Technology: File Logical Schema: SFDC_FILE_SCHEMA

On the Reverse tab of the SFDC_FLAT_FILE model change the Context dropdown value to ODI_SFDC_CONTEXT.

Click OK to finish and verify that your two models have been created. If the models dont immediately line-up under the SFDC_MODELS folder simply click each model and drag it into the parent folder.


Last Saved: 10/8/2010 11:34 PM 34



Last Saved: 10/8/2010 11:34 PM 35


Go, Go Integration DIRECTION Establishing a project, setting up the models and reverse engineering SalesForce.com objects has the ODI Designer prepped and ready to. Now, we will add our two data stores and create an interface to load data into SalesForce.com.

ADD FILE DATASTORE To create a Datastore for the contacts flat file, right-click the SFDC_FLAT_FILE model and choose Insert Datastore (at the bottom of the menu). In the prompt we will locate the specific flat file with the sample data that we wish to insert into our SFDC contact object. In the prompt enter Contacts in the Name field. Then, for the Resource Name use the elipse button and select the base_contacts.csv that you placed in the FileStorage directory location in a previous step.

On the Files tab, configure the settings with the following values: File Format: Delimited Heading: 1 Field Separator: , (comma) Text Delimiter: (double quote)


Last Saved: 10/8/2010 11:34 PM 36


On the Columns tab, click the Reverse button. Click Yes to save changes.

You should now see the columns names from the file and the associated metadata. Click OK to finish.

CREATE THE INTERFACE The interface will map the specific object columns from source to target and define the KMs to be used during the inserting of records to SalesForce.com. On the left-side panel select the Project tab. Expand the ODI_SFDC_PROJECT > First Folder to expose the Interfaces item. Right-click on Interfaces and select Insert Interface


Last Saved: 10/8/2010 11:34 PM 37


In the new Interface prompt enter i_push_contacts_to_SFDC for the Name field value and ensure that the ODI_SFDC_CONTEXT is selected. Check the Staging Area Different From Target checkbox and choose SFDC_FILE_SCHEMA from the dropdown.

Click the Diagram tab. Click OK on the default help prompt if it appears. In the Diagram we need to access the source and target datastores that were just created. Leave the Interface prompt alone for a second and change the tab on the left-pane to the Models tab. You will see that the new Interface prompt does not disappear.

Expand the SFDC_MODELS parent folder to expose the two SFDC models. Expand the SFDC_FLAT_FILE model to expose the Contacts datastore. Click on the Contacts object and drag it into the sources section of the new Interface window.


Last Saved: 10/8/2010 11:34 PM 38


Expand the SFDC_Model to expose the underlying SalesForce.com objects. Click on the Contact object and drag it to the Target Datastore section of the new Interface window.

When the warning prompt appears and asks Do you want to perform an Automatic Mapping? click Yes. The mapping will be performed and should line-up the columns nicely.

Click the Flow tab and notice the lineage from source to target has been created. Click on the header of the Target box to reveal its properties. Ensure that the IKM dropdown box has IKM File to Salesforce (Upsert) selected as its value.

Click Apply at the bottom of the window. If prompted to lock the object click No.

To execute the interface and see data loaded into your SalesForce.com account, click the Execute button at the bottom right of the Interface window. A prompt will appear to choose the Agent and context under which the interface should execute. Choose ODI_SFDC_CONTEXT as the Context and ODI_SFDC_LOGICAL_AGENT as the Agent.


Last Saved: 10/8/2010 11:34 PM 39


Click OK to begin the execution session.


Last Saved: 10/8/2010 11:34 PM 40


Verify SalesForce.com Contacts Load DIRECTION In order to verify that data has been loaded successfully to SFDC, three locations may be referenced: the ODI Operator, the log success/error files or the SalesForce.com Contacts tab.

VERIFY Open the ODI Operator from the Start menu by All Programs > Oracle > Oracle Data Integrator > Operator.

On the left side of the window under Session List expand the Physical Agent node > ODI_SFDC_AGENT. Expand the session based execution of the interface you just launched. If all icons glow green then the execution was successful.

To view the log files, navigate to the folder ODI_HOME\_Logs and open the file prefixed with extract.csv.i_push_contacts_to_SFDC followed by a session number using your favorite text editor. Any errors or successes will be articulated in this file.

Finally, log into your SalesForce.com account, click on the contacts tab or navigate to your contacts object to view the updated list.


Last Saved: 10/8/2010 11:34 PM 41



Last Saved: 10/8/2010 11:34 PM 42


Conclusion Oracle Data Integrator is a great ETL (ELT) to for moving data between heterogeneous data sources. This tutorial touched on very small portions of the tremendous power that ODI has. This tutorial allowed an intermediate to advanced user to learn more about ODI knowledge modules, learn more about SalesForce.com, integrate the two via a data flat file and verify its success. This is the first part in a two part series. The next part in the series will demonstrate how to pull data from SalesForce.com into a local data source which is quite a bit more complex as it involves modifying the Knowledge Module itself. All technical issues and follow-up conversation can be found on the Art of Business Intelligence blog at http://www.artofbi.com.

References (http://www.oracle.com/technology/products/oracle-data-integrator/10.1.3/htdocs/documentation/oracledi_setup.pdf).

ABOUT THIS TUTORIAL4Preface4Objective4Software Dependencies4Projected Completion Time4Pre-Requisites4

INSTALL THE ODI 10.3.5.5 PATCH6Direction6

INSTALL THE JAVA JDK7Direction7File Download & Installation7

SETTING-UP THE SAMPLE DATA SOURCE8Direction8Tutorial File Download & Installation8

SETTING-UP SALESFORCE.COM 3RD PARTY SOFTWARE9Direction9Software Downloads9Install Tutorial Files / Data Procedure10Test SalesForce.com API Login Credentials10

CREATE WORK REPOSITORY DATABASE SCHEMA13Direction13Create Schema13

SETUP TOPOLOGY MANAGER14Direction14Create Repository Connection Login14Create Work Repository15Add SalesForce.com Technology17Create An Agent18Create a Context19Create a Logical Agent20

CONFIGURE THE AGENT22Direction22Configure Agent Batch File22Create Agent Start/Stop Batch Scripts23

SETUP DATA SERVERS & DATA STORES25Add File Data Server25Add SalesForce.com Technology Data Server26

PREPARE THE INTEGRATION IN DESIGNER28Direction28Initial Designer Connection28Create a Project29Import the Knowledge Modules29Create a File System Log Folder30Configure RKM Options30Configure IKM Options30Create the Models31

GO, GO INTEGRATION35Direction35Add File Datastore35Create the Interface36

VERIFY SALESFORCE.COM CONTACTS LOAD40Direction40Verify40

CONCLUSION42REFERENCES42

artofbi odi10g salesforce integration deep dive (1)

Documents

oracle data integrator

sample data source

party software

software dependencies

software downloads

art of bi http

christian screen

projected completion