odi advanced

Download ODI Advanced

Post on 14-Nov-2015

31 views

Category:

Documents

4 download

Embed Size (px)

DESCRIPTION

ODI

TRANSCRIPT

  • Advanced Topics in ODI-EE: Hands on Lab

    Advanced Topics in ODI-EE: Hands on Lab Page 1 of 26 7/9/2010 6:08:00 PM (version 1.0) Oracle, All rights reserved

    Advanced Topics in Oracle Data Integrator: Hands-on Lab

  • Advanced Topics in ODI-EE: Hands on Lab

    Advanced Topics in ODI-EE: Hands on Lab Page 2 of 26 7/9/2010 6:08:00 PM (version 1.0) Oracle, All rights reserved

    Supported Version: Oracle Data Integrator 11.1.1.3

    Objectives: This lab will highlight to Developers, DBAs and Architects some of the best practices for implementing techniques to improve performance for real-time data warehousing using Oracle Data Integrator Enterprise Edition (ODI-EE). In this lab participants will walk through the steps to enable change data capture on a source relational table, then deploy and manage these implementations to load and transform records to a relational target.

    Prerequisite: Basic knowledge of Oracle Data Integrator, Installed Oracle Data Integrator software and Oracle Database instance.

    S308461 - Using ODI-EE and Oracle BI for Real-Time Insight: Hands on Lab

  • Advanced Topics in ODI-EE: Hands on Lab

    Advanced Topics in ODI-EE: Hands on Lab Page 3 of 26 7/9/2010 6:08:00 PM (version 1.0) Oracle, All rights reserved

    Table of Contents Part 1: Configuring a Model for CDC ........................................................................... 4 Part 2: Creating a Package to Set Up CDC ................................................................. 8 Part 3: Creating the Integration Flows to Consume the Changes ............................... 13 Part 4: Run the CDC Set Up and Consumption Processes in Production ................... 22 Summary .................................................................................................................. 26

  • Advanced Topics in ODI-EE: Hands on Lab

    Advanced Topics in ODI-EE: Hands on Lab Page 4 of 26 7/9/2010 6:08:00 PM (version 1.0) Oracle, All rights reserved

    Part 1: Configuring a Model for CDC In this exercise you will configure an existing source Oracle data model to support ODI CDC feature. The CDC mode used will be Consistent-Set CDC. Oracle Log Miner will be used to track the changes.

    1. Double-click the icon Reset_ODI.bat from the desktop. This will reset the files for all Hands-on Labs to the initial state. Warning: If you have custom ODI projects in the repository on this machine, they will be deleted.

    2. Open the Oracle Data Integrator Studio using the Start Menu > Oracle > Oracle Data Integrator > ODI

    Studio menu.

    3. Go to the Designer Navigator and press Connect to Repository

    4. Select HOL Advanced from the drop-down list of logins and press OK.

    5. Open the Designer navigator menu and make sure that the Default Execution Context is set to the

    Development context (not Production).

  • Advanced Topics in ODI-EE: Hands on Lab

    Advanced Topics in ODI-EE: Hands on Lab Page 5 of 26 7/9/2010 6:08:00 PM (version 1.0) Oracle, All rights reserved

    6. Open the Models accordion in the Designer navigator by clicking on the label.

    7. Right-click on the Oracle CDC Source model and select Changed Data Capture > Add to CDC. Click Yes to add all the tables to the CDC.

    8. Double-click on the Oracle CDC Source model. The editor for this model opens.

  • Advanced Topics in ODI-EE: Hands on Lab

    Advanced Topics in ODI-EE: Hands on Lab Page 6 of 26 7/9/2010 6:08:00 PM (version 1.0) Oracle, All rights reserved

    9. Go to the Journalizing tab.

    10. Select the Consistent Set journalizing mode, click OK to close the popup window then the JKM Oracle 10g Consistent (Log Miner) knowledge module. Set the AUTO_CONFIGURATION KM option to Yes.

    11. Go to the Journalized Tables tab, select the SRC_REGION table and click on the Remove

    from CDC button.

    12. Click on the Reorganize button to order the tables according to their foreign-keys. Press yes on all confirmation dialogs. The datastores are automatically organized as shown below.

  • Advanced Topics in ODI-EE: Hands on Lab

    Advanced Topics in ODI-EE: Hands on Lab Page 7 of 26 7/9/2010 6:08:00 PM (version 1.0) Oracle, All rights reserved

    Note: Consistent Set Journalizing tracks changes to a group of the model's datastores, taking into account the referential integrity between these datastores. The group of datastores journalized in this mode is called a Consistent Set.

    13. Click Save( ) on the toolbar to save the Model.

  • Advanced Topics in ODI-EE: Hands on Lab

    Advanced Topics in ODI-EE: Hands on Lab Page 8 of 26 7/9/2010 6:08:00 PM (version 1.0) Oracle, All rights reserved

    Part 2: Creating a Package to Set Up CDC In this exercise you will create a package that sets up the CDC infrastructure in a given context.

    1. In the Designer navigator, Click on the Projects accordion.

    2. Expand the Advanced Topics in ODI-EE project and then expand the First Folder folder.

    3. Select the Packages node, right-click and select New Package. A package editor opens.

    4. Enter the following package name: Tutorial Part 2 - CDC Setup

    5. Go to the Diagram tab.

    6. Drag and drop into the diagram the Oracle CDC Source data model from the Designers Models tree view. A new step appears in the diagram, named after your data model.

  • Advanced Topics in ODI-EE: Hands on Lab

    Advanced Topics in ODI-EE: Hands on Lab Page 9 of 26 7/9/2010 6:08:00 PM (version 1.0) Oracle, All rights reserved

  • Advanced Topics in ODI-EE: Hands on Lab

    Advanced Topics in ODI-EE: Hands on Lab Page 10 of 26 7/9/2010 6:08:00 PM (version 1.0) Oracle, All rights reserved

    7. Click on this step. In the Properties panel: Select Journalizing Model in the Type drop-down list

    Click the Start and Add Subscribers checkboxes. Enter CONSUMER_1 in the Subscribers field, and then click Add. Enter CONSUMER_2 in the Subscribers field, and then click Add.

    8. Click Save( ) on the toolbar to save the package.

    9. Click on the Execute( ) button on the toolbar.

    10. In the Execution window, select Development for the context and then press OK.

  • Advanced Topics in ODI-EE: Hands on Lab

    Advanced Topics in ODI-EE: Hands on Lab Page 11 of 26 7/9/2010 6:08:00 PM (version 1.0) Oracle, All rights reserved

    11. Press OK when the Session Started window appears.

    12. Open the Operator navigator.

    13. In the Operator, select the Session List tab, and expand the All Executions node.

  • Advanced Topics in ODI-EE: Hands on Lab

    Advanced Topics in ODI-EE: Hands on Lab Page 12 of 26 7/9/2010 6:08:00 PM (version 1.0) Oracle, All rights reserved

    14. Check that the last session ran correctly. You can review the steps and tasks that have activated the CDC process.

  • Advanced Topics in ODI-EE: Hands on Lab

    Advanced Topics in ODI-EE: Hands on Lab Page 13 of 26 7/9/2010 6:08:00 PM (version 1.0) Oracle, All rights reserved

    Part 3: Creating the Integration Flows to Consume the Changes In this exercise you will create the integration flows that will consume the changes captured using CDC.

    1. In the Designer navigator, open the Projects accordion.

    2. Expand the Advanced Topics in ODI-EE project and then expand the First Folder folder.

    3. Expand the Procedures node.

    4. Right-click on the CDC Changes procedure and select Execute.

    5. In the Execution window, select Development for the context and then press OK.

    6. Press OK when the Session Started window appears.

    7. The procedure did the following: 1 insert in SRC_CITY. 1 insert in SRC_CUSTOMER. 1 update in SRC_CITY. 2 updates in SRC_CUSTOMER.

  • Advanced Topics in ODI-EE: Hands on Lab

    Advanced Topics in ODI-EE: Hands on Lab Page 14 of 26 7/9/2010 6:08:00 PM (version 1.0) Oracle, All rights reserved

    8. Expand the Interfaces node and double-click on Load SRC_CITY CDC.

    9. Go to the Mapping tab.

    10. Click on the source datastore SRC_CITY and check the Journalized Data only checkbox. This ensures that ODI will load only the changed data (latest inserts, updates or deletes) from the source instead of the entire source dataset.

    11. Click on the filter and modify the filter expression from JRN_SUBSCRIBER = 'SUNOPSIS' to JRN_SUBSCRIBER = 'CONSUMER_1'

  • Advanced Topics in ODI-EE: Hands on Lab

    Advanced Topics in ODI-EE: Hands on Lab Page 15 of 26 7/9/2010 6:08:00 PM (version 1.0) Oracle, All rights reserved

    12. Right-click on the filter and select Data to review the changed data.

    13. Save and close the interface.

    14. Double-click on Load SRC_CUSTOMER CDC in the Interfaces node.

    15. Go to the Mapping tab.

    16. Click on the source datastore SRC_CUSTOMER and check the Journalized Data only checkbox.

  • Advanced Topics in ODI-EE: Hands on Lab

    Advanced Topics in ODI-EE: Hands on Lab Page 16 of 26 7/9/2010 6:08:00 PM (version 1.0) Oracle, All rights reserved

    17. Click on the filter and modify the filter expression from JRN_SUBSCRIBER = 'SUNOPSIS' to JRN_SUBSCRIBER = 'CONSUMER_1'.

    18. Right-click on the filter and select Data to review the changed data.

    14. Click on the Models tab in the tree view.

  • Advanced Topics in ODI-EE: Hands on Lab

    Advanced Topics in ODI-EE: Hands on Lab Page 17 of 26 7/9/2010 6:08:00 PM (version 1.0) Oracle, All rights reserved

    19. Expand the Flat Files Sources model.

    20. Expand the Flat Files Sources model and drag and drop the SRC_SALES_PERSON.txt datastore f