odi overviewlatest

8/11/2019 ODI OverviewLatest

http://slidepdf.com/reader/full/odi-overviewlatest 1/51

Oracle Data Integrator

Kiran Kumar R, PMP



INTRODUCTION



5

Data Movement and Transformation from MultipleSources to Heterogeneous Targets

BENEFITS KEY DIFFERENTIATED FEATURES

1. Performance Heterogeneous “E-LT” 2. Flexibility Event-Driven Platform

3. Productivity Declarative Design

4. Open 100% Java and SOA Native

5. Hot-Pluggable Knowledge Modules

Customer Value PropositionStrategy: Best-of-Breed Core Architecture & Foundation



Previously Known as “Sunopsis”

• Sunopsis were an ETL tool vendor originally from France,

with offices in the USA and UK• Two major products

– Sunopsis Data Conductor (database integration)

– Sunopsis Active Integration Platform (event and serviced-based

integration)

• Sold on it’s platform independence, Java architecture and EAIcapabilities

• Acquired by Oracle in 2006



ODI Positioning

• Not a replacement for Oracle Warehouse Builder

– OWB has Oracle data warehouse modeling features

– OWB is the obvious choice when target is an Oracle database

• ODI supplements OWB in the following scenarios:

– Target database is non-Oracle (DB2, Teradata, XML, SQL Server etc)

– Sources include Web Services, XML, SOAP etc

– Data integration is complex / real-time

– Multi-platform CDC / SCD / Bulk load and insert / MDM and Data Hubs

– Batch windows require bad source data to be filtered out before loading

• Adds functionality to the Oracle DW ETL stack

– ODI and OWB likely to code-share/merge in future

– In short term, possible replacement for OWB Enterprise ETL Option



Market Position



Overview of ODI Studio

17

Navigatorsfor eachfunctionalarea.

ODI operations integrated inIDE: Menu, Gallery, etc.

Intuitive multi-editors supportclose all/save all, docking,etc.

Property inspectors,thumbnails and IDE windowsfully used as needed.

Accordionsfor eachsub-area.

Popup menusand treemarkers.

Interface Editor usingdiagraming.



Interface Editor / Quick Edit

18

Same InterfaceTwo views



Interface Editor

19

Diagram zoomand autolayout.

Datasets management

Lookup wizard

Errors and Autofix.

Lookup

Datasets appear here.

Propertiesinspector forselection.

Sortable andSelectable columns.

DnD from sources tomapping.

Thumbnail: bird’seye view of the

diagram.



22

Oracle Data Integrator

SOA Enabled


http://slidepdf.com/reader/full/odi-overviewlatest 23/51 3

SOA Enabled: Data Access Services

• Generate and share data

access services1. Generate and deploy data services

2. Test data services

3. Leverage data services in your SOA

infrastructure

SOA Infrastructure

Services

Data Access

Transform

Business

BusinessProcesses

ESB


http://slidepdf.com/reader/full/odi-overviewlatest 24/51 4

SOA Enabled: Data Flow Services

• Expose transformations as

Web Services1. Orchestrate data flows

2. Publish data flows as web services

in your SOA infrastructure

SOA Infrastructure

Services

Data Access

Transform

Business

Business

Processes

ESB

Bulk

Transf.

Extended Capabilities



Extended Capabilities



• Same philosophy as OWB – Use the Database as the ETL engine

– ODI supports heterogeneous databases, not just Oracle

• Built for SOA environments

– Support for Web Services etc.,

• Supports batch, event-based and real-time integration

• Data Integrity Controls create a data “firewall”

– Reduces data prep time by not processing erroneous data• Extensible through “Knowledge Modules”

– Change Data Capture

– Slowly Changing Dimensions

– Bulk load

Data Integrator Overview



Solution Talk – A Case Study

27




• Industry

• BFSI

• Customer Profile• Our customer is the largest BFSI company in India, providing a wide range of

banking operations and property and casualty insurance products together withaccidental injury insurance and short-term health insurance products

• With 31 sub-companies distributed across the country, this company has beendeveloping a complicated IT infrastructure and an analysis system with a hugevolume of data

• All the database systems are Informix• Business Problem

• KPI reports are significantly delayed due to poor system performance, leading tothe KPI system unreliable for decision-making

• The current system is too overloaded to provide internal users with stableservice based on existing business, not mentioning supporting new business

• Maintenance cost is augmenting due to sub-optimal DW development approach,which has become a manifest issue during the economic downturn

• Product Used

• Oracle Database Enterprise Edition 11.1.0.6 with RAC, Partitioning Options

• Oracle Data Integrator 10.1.3.4



Solution Talk – What Challenges

Were Responded to ??



Customer Challenges -Existing System

Architecture

• Distributed IT infrastructure

• 31 OLTP Informix DBs aredistributed across India, with a

central Informix DB

• Performance-thirsty

transformation

• Data is extracted (from 31 Informixinto ODS) and transformed (finally

into Report Data Layer) in real time

or periodically.

• Non-standardized

implementation• Data extraction and transformation

are implemented via code such as

C,SQL, etc, no ETL tool here.



Solution Talk – What did We

Achieve ??

Copyright © 2008, Oracle and/or its affiliates. All rights reserved. 31



Achievements – ODI Topology Manager

With ODI Topology Manager, wecan easily define all kinds ofdata sources and targettechnologies.

In our case, multiple Informixdatabases can be defined asdata sources thereby UI friendlydata consolidation is achieved.

Here we defineInformix as datasource



Achievements – ODI Designer

Here we designETL processing

ODI Designer is an integratedGUI for users to design ETLprocesses.

In our case, complicated

transformations can be definedin a high efficient manner. Highlyoptimized codes are generatedautomatically by tool.



Achievements – ODI Operator

With ODI Operator, we can

monitor and execute the ETLprocesses conveniently.

In our case, ODI Operator helpsimprove the maintenanceefficiency. For instance, userscan identify the ETL steps witherrors or warnings and pinpointthe problems efficiently.



Key Benefits - Delivered

Key Focus Areas Benefits

High Performance

No analytical report is delayed

The overall performance is dramatically improved and the

hardware resources are utilized with more efficiency

Additional resource is available for processing new business

ETL Efficiency

New business can be developed timely

Distributed data can be easily consolidated

Data processing is easily and flexibly implemented

Transformation code is optimized with high performance



Solution Talk – What did We

Propose ??



Oracle Proposed Solution – System Architecture

• Oracle is introduced as central DB

• As the most powerful DB, Oracle

DW approach helps our customerto solve current performance

problems and maximize system

availability.

• ODI is introduced as ETL tool

• As the most efficient ETL tool,

ODI integrates data from Informix

into Oracle, and subsequent datatransformation,with friendly UI

and efficient code.

• OBIEE would be further introduced

• Other Oracle DB for the

distributed sub-systems and BIEE

for reporting would be further

introduced, concerning that oursolution has successfully solved

current key problems.



Oracle Proposed Solution – How ODI Integrates Data from Informix

• ODI access Informix via JDBC using

Informix Driver. Connections are

managed within Topology, one

component of ODI.

• Complicated transformations are

defined within Designer(the design-time environment of ODI) ,

transformations such as extraction

from Informix(via JDBC), data

aggregation(via SQL).

– Transformations are executed from

Agent(the run-time environment of

ODI), which calls Java,SQL,etc., to

run ETL processes using workflow.

– Transformations are monitored from

Operator(the run-time logger of ODI),which traces each step of the

transformation.



Solution Talk – What Potential This

Solution Has ??




41



Copyright © 2008, Oracle and/or its affiliates. All rights reserved.4

iTech Solution Profile

Existing Pain

LL RTO’S

SCHEMAS

State

Consolidation

Register

Manual Data

Replication process

STATE LEVEL

RTO’S

SCHEMAS

State Register

Manual Data

Replication process

National Level

RTO Schema

Manual Data Replication

process for Huge volumes

of data

http://images.google.com.sg/imgres?imgurl=http://upload.wikimedia.org/wikipedia/commons/thumb/b/ba/Red_x.svg/600px-Red_x.svg.png&imgrefurl=http://commons.wikimedia.org/wiki/File:Red_x.svg&usg=__1zMSfBbyyvqzLnJn2owFGB4lafo=&h=600&w=600&sz=19&hl=en&start=2&um=1&tbnid=k_yipynTpf-8dM:&tbnh=135&tbnw=135&prev=/images%3Fq%3Dx%2Bimage%26hl%3Den%26rlz%3D1R2GGIT_en-GB%26um%3D1




43




Existing Data Flow @ HDFC Std Life



Problems with current data flow

• Performance is poor

• Multi step integration – CL + FTP + OWB• More Maintenance effort

• Difficult to incorporate development Best Practices

• No Unified monitoring and scheduling

Current Integration Flow

ODI can make this step as

straight through








• The proposed data integration solution is based on Oracle DataIntegrator (ODI) product.

• Oracle Data Integrator provided a unique data integration platform, thatwas designed for performance with a high degree of flexibility andmodularity.

Benef i ts

• Performance – Leveraged a high performance E-LT architecture.:

• Productivity- The in-built declarative design was utilised which in-turndrives reusability, accelerates development and maintenance.

• Flexibility – A single platform was presented that natively supportsseveral data integration styles and latencies.

• Modularity – Hot-pluggable knowledge modules were used to bettersupport application and database platform diversity.

Oracle’s Data Integration Solution



47

Advantages of using ODI

• High Performance – using Bulk set operations

• End-to-End integration within ODI

• Easy to maintain because of Repository based architecture and

Impact analysis features• Best Practices can be productized into Knowledge Modules

• End-to-End data lineage features to enable report to source data

lineage

• CDC mechanism for DB2/400, SQL Server and Oracle

Proposed Data Integration Flow




ODI Screenshots



Proof of concept: Results• Performance Improvement

• NOTE- Execution Times mentioned for OWB does not include the time taken

to extract file from DB2 and FTP the same to Oracle Data Warehouse Server.This currently takes 45 minutes for all mappings put together. ODI accesses

DB2/400 tables directly and does not require an FTP step.


No. ofRecords

ODI(Secs)

No. ofRecords

OWB(Secs)

%Improvement

ACTIVITY 82911 36 83561 118 ~ 320 %

POLICY_ACCOUNT_DD

533833 718 545028 1693 ~ 230%

S



Summary• Oracle Data Integrator is the new addition to Oracle’s Middleware toolset.

• ODI-EE is the only product that combines the best performance, flexibility

and cost-savings alternatives for data integration business needs. The E-

LT foundation provided by ODI-EE offers Oracle customers a way toleverage existing database

• (Oracle and non-Oracle) investments without sacrificing world-class

functionality that they would expect from a best-of-breed data integration

platform.

• Complements OWB by adding advanced data integration

Functions

– Works in the staging/integration layer

– Pre-built modules for CDC, bulk loading etc

– Supports event-based, service-based and real-time integration

• Adds the ability to target heterogeneous platforms



Demonstration

odi overviewlatest

Documents