Informatica Informatica Performance TuningPerformance Tuning

OBJECTIVEOBJECTIVE

• The goal of performance tuning is to optimize The goal of performance tuning is to optimize

session performance by eliminating performance session performance by eliminating performance

bottlenecksbottlenecks . .

General stepsGeneral steps

• The first step in performance tuning is to identify The first step in performance tuning is to identify the performance bottleneck .the performance bottleneck .

• Analyze the cause for the bottleneck.Analyze the cause for the bottleneck.

• Eliminate it.Eliminate it.

• Check the session performance and repeat the Check the session performance and repeat the above steps until it is satisfactory.above steps until it is satisfactory.

Performance bottlenecks can occur in Performance bottlenecks can occur in

• TargetTarget

• SourceSource

• MappingMapping

• sessionsession

• SystemSystem

Target bottleneckTarget bottleneckCauses: Causes: • Small check point intervals , small database Small check point intervals , small database

network packet size , problems during heavy network packet size , problems during heavy loading operationsloading operations . .

If Relational table as target:If Relational table as target:• Check by populating the records to a flat file.Check by populating the records to a flat file.

If Flat file as target:If Flat file as target:• check by populating the records to a flatfile in check by populating the records to a flatfile in

local power center serverlocal power center server..

Optimizing methods for relational Optimizing methods for relational targettarget::

1.1.Increasing Checkpoint Intervals Increasing Checkpoint Intervals

2. Use Bulk Loading .2. Use Bulk Loading .

3. Increasing Database Network Packet Size 3. Increasing Database Network Packet Size --for sybase & sql server increase from 8k-16k--for sybase & sql server increase from 8k-16k --for oracle increase in tnsnames.ora and listener.ora --for oracle increase in tnsnames.ora and listener.ora

4. Optimizing Oracle Target Databases 4. Optimizing Oracle Target Databases --by checking the storage clause, space allocation, --by checking the storage clause, space allocation, and rollback segments in appropriate table spaces. and rollback segments in appropriate table spaces.

5. Dropping Indexes and Key Constraints before loading 5. Dropping Indexes and Key Constraints before loading and recreate after session completion .and recreate after session completion .

Methods to identify Source Methods to identify Source bottlenecksbottlenecksFlat file source:Flat file source:

Using “Line Sequential Buffer Length setting” set Using “Line Sequential Buffer Length setting” set the no. of bytes the power center reads per line.the no. of bytes the power center reads per line.( default:1024 bytes).( default:1024 bytes).

Relational source:Relational source:• Use a filter transformation in the mapping to Use a filter transformation in the mapping to

measure the time it takes to read source data.measure the time it takes to read source data.

• Use a Read test session.Use a Read test session.

• Database query—execute the read query present Database query—execute the read query present

in sql override directly in database itself.in sql override directly in database itself.

Optimizing methods for relational Optimizing methods for relational

sourcesource::

• Optimize the query. Optimize the query.

• Create tempdb as an in-memory database to Create tempdb as an in-memory database to allocate sufficient memory–for Sybase or allocate sufficient memory–for Sybase or Microsoft SQL Server database Microsoft SQL Server database

• Use conditional filters (i.e., using filter condition Use conditional filters (i.e., using filter condition in sql override of source qualifier itself). in sql override of source qualifier itself).

• Increase database network packet size Increase database network packet size

--larger packets of data to cross the network at --larger packets of data to cross the network at

one time .one time .

Methods to identify Mapping Methods to identify Mapping bottlenecks:bottlenecks:

• Use a filter transformation before the target table Use a filter transformation before the target table and set the condition to false so that no data is and set the condition to false so that no data is loaded into the target.loaded into the target.

• Multiple lookups can slow down the performance.Multiple lookups can slow down the performance.

• Transformation errors impact session Transformation errors impact session performance so check the transformation errors performance so check the transformation errors in session log file.in session log file.

Mapping optimizationMapping optimization• Configure Single-Pass Reading --allows you to Configure Single-Pass Reading --allows you to

populate multiple targets with one source qualifier. populate multiple targets with one source qualifier.

• Avoid unnecessary data type conversions.Avoid unnecessary data type conversions.

• Reduce the number of transformations in the Reduce the number of transformations in the mapping.mapping.

• Minimize the amount of data moved by deleting Minimize the amount of data moved by deleting unnecessary links between transformations. unnecessary links between transformations.

• Limiting the number of connected input/output or Limiting the number of connected input/output or output ports reduces the amount of data the output ports reduces the amount of data the transformations store in the data cache. transformations store in the data cache.

SSingle Passingingle Passing

SourceSystem 1

SourceSystem 2

Target SourceQualifier1

SourceQualifier2

Target

SourceSystem 1

SourceSystem 2

SourceQualifier

Target

Single pass reading

Aggregator1

Aggregator2

Exp

Target

Exp1 Aggregator1

Exp2 Aggregator2

Look up Optimization:Look up Optimization:• Implement caching the lookup tableImplement caching the lookup table

• Reduce the Number of Cached Rows by using Reduce the Number of Cached Rows by using lookup sql override.lookup sql override.

• Always use equal ( = ) sign first in lookup condition Always use equal ( = ) sign first in lookup condition then use other signs such as <, >, <= ,>= , != then use other signs such as <, >, <= ,>= , != etc.,etc.,

• Use index in the lookup tableUse index in the lookup table. .

Filter Optimization:Filter Optimization:• filter rows in sql override in source qualifier filter rows in sql override in source qualifier

transformation itself.transformation itself.

• move the Filter transformation as close to the move the Filter transformation as close to the source qualifier.source qualifier.

• avoid using complex expressions in filter condition.avoid using complex expressions in filter condition.

• Use a Filter or Router transformation to drop Use a Filter or Router transformation to drop rejected rows from an Update Strategy rejected rows from an Update Strategy

transformationtransformation . .

Aggregator Transformation:Aggregator Transformation:• often slow performance because they must group often slow performance because they must group

data before processing it.data before processing it.

• It need additional memory to hold intermediate It need additional memory to hold intermediate group results. group results.

Optimizing Methods:Optimizing Methods:• Use simple columns i.e number instead of strings Use simple columns i.e number instead of strings

and dates for group by clause.and dates for group by clause.

• Use sorted input which decreases the use of Use sorted input which decreases the use of aggregate caches aggregate caches

• when changed rows < target rows then Use when changed rows < target rows then Use “incremental aggregation “ which present in the “incremental aggregation “ which present in the session properties.session properties.

Joiner: Joiner: • need additional space at run time to hold need additional space at run time to hold

intermediate results.intermediate results.• uses data cache to hold the master table rows uses data cache to hold the master table rows

and an index and an index cache to hold the join columns from the master cache to hold the join columns from the master

table.table.

Optimizing joiner transformation:Optimizing joiner transformation:• Ensure sufficient memory to hold the data cache Ensure sufficient memory to hold the data cache

and the and the index cache .index cache .• Use smaller table as a master table.Use smaller table as a master table.• Normal joins are faster than outer joins.Normal joins are faster than outer joins.• UUse database joins for homogenous sourcesse database joins for homogenous sources

Optimizing Sequence generator:Optimizing Sequence generator:

• Create single seq.generator transformation and Create single seq.generator transformation and use it for multiple pipeline in a single mapping use it for multiple pipeline in a single mapping instead of using different sequence generator for instead of using different sequence generator for each pipe line.each pipe line.

• configure the Number of Cached Values property configure the Number of Cached Values property approx.,>1000 but not too small.approx.,>1000 but not too small.

Optimize expressionsOptimize expressions:: • Remove expressions one-by-one to isolate the Remove expressions one-by-one to isolate the

slow expressions. slow expressions.

Steps to optimizeSteps to optimize::• Factoring Out Common Logic Factoring Out Common Logic

• Minimizing Aggregate Function Calls Minimizing Aggregate Function Calls

• Choosing DECODE versus LOOKUP Choosing DECODE versus LOOKUP

• Using Operators Instead of FunctionsUsing Operators Instead of Functions

Evaluating ExpressionsEvaluating Expressions::

If you are not sure which expressions slow If you are not sure which expressions slow performance ,thenperformance ,then

• Time the session with the original expressions. Time the session with the original expressions.

• Copy the mapping and replace half of the Copy the mapping and replace half of the complex expressions with a constant. complex expressions with a constant.

• Run and time the edited session. Run and time the edited session.

• Make another copy of the mapping and replace Make another copy of the mapping and replace the other half of the complex expressions with a the other half of the complex expressions with a constant. constant.

• Run and time the edited session. Run and time the edited session.

Session optimizingSession optimizing

• Increase the number of partitions. Increase the number of partitions.

• Reduce errors tracing. Reduce errors tracing.

• Increasing the Cache Sizes .Increasing the Cache Sizes .

• Increasing the Commit Interval .Increasing the Commit Interval .

• Remove staging areas. Remove staging areas.

System optimizingSystem optimizing Improve network speedImprove network speed : :• Slow network connections can slow sessionSlow network connections can slow session performanceperformance

Use multiple PowerCenter ServersUse multiple PowerCenter Servers : :• Using multiple Power Center Servers on separate Using multiple Power Center Servers on separate

systems might double or triple session systems might double or triple session performance.performance.

Use a server gridUse a server grid : :• Using a collection of Power Center Servers to Using a collection of Power Center Servers to

distribute and process the workload of a workflow.distribute and process the workload of a workflow. Improve CPU performance :Improve CPU performance :• Run the Power Center Server and related Run the Power Center Server and related machines on high performance CPUs, or configure machines on high performance CPUs, or configure your system to use additional CPUs. your system to use additional CPUs.

Thank you ! Thank you !