complete reference to informatica_ optimizing the bottleneck’s
DESCRIPTION
optimizing the bottlenecksTRANSCRIPT
5/31/2016 Complete reference to Informatica: Optimizing the Bottleneck’s
http://informaticatutorialsnaveen.blogspot.in/2011/04/optimizingbottlenecks.html 1/3
Home Data WareHousing Unix-Shell Scripts PL SQL Contact Us
PERFORMANCE TUNING
Necessity Of Performancetuning
Identification Of bottle Necks
Optimization of Bottle necks
Performance tuning of Lookup
Push Down Optimization
TESTING
Unit Testing
Integration Testing
UAT
Informatica Testing
Debugger
Constraint Based loading
Target Load Plan
INTERVIEW ZONE
Informatica Real Time Interview Questions
Informatica Experienced InterviewQuestions part1
Informatica Experienced InterviewQuestions part2
Informatica Experienced InterviewQuestions part3
Informatica Experienced InterviewQuestions part4
Data Warehousing concept Based InterviewQuestions
17
TOTAL VISITS
FEEDS
DISCUSSION FORUM
Discussion Forum
SUBSCRIBE TO LEARN INFORMATICA512 peoplelike this.
Like Share
Sunday, 17 April 2011
Optimizing the Bottleneck’s
1. If the source is a flat file, ensure that the flat file is local to the Informatica server. Ifsource is a relational table, then try not to use synonyms or aliases.
2. If the source is a flat file, reduce the number of bytes (By default it is 1024 bytesper line) the Informatica reads per line. If we do this, we can decrease the LineSequential Buffer Length setting of the session properties.
3. If possible, give a conditional query in the source qualifier so that the records arefiltered off as soon as possible in the process.
4. In the source qualifier, if the query has ORDER BY or GROUP BY, then create anindex on the source table and order by the index field of the source table.
PERFORMANCE TUNING OF TARGETS
If the target is a flat file, ensure that the flat file is local to the Informatica server. If targetis a relational table, then try not to use synonyms or aliases.
1. Use bulk load whenever possible.2. Increase the commit level.3. Drop constraints and indexes of the table before loading.
PERFORMANCE TUNING OF MAPPINGS
Mapping helps to channel the flow of data from source to target with all the transformationsin between. Mapping is the skeleton of Informatica loading process.
1. Avoid executing major sql queries from mapplets or mappings.2. Use optimized queries when we are using them.3. Reduce the number of transformations in the mapping. Active transformations likerank, joiner, filter, aggregator etc should be used as less as possible.
4. Remove all the unnecessary links between the transformations from mapping.5. If a single mapping contains many targets, then dividing them into separatemappings can improve performance.
6. If we need to use a single source more than once in a mapping, then keep only onesource and source qualifier in the mapping. Then create different data flows asrequired into different targets or same target.
7. If a session joins many source tables in one source qualifier, then an optimizingquery will improve performance.
8. In the sql query that Informatica generates, ORDERBY will be present. Remove theORDER BY clause if not needed or at least reduce the number of column names inthat list. For better performance it is best to order by the index field of that table.
9. Combine the mappings that use same set of source data.10. On a mapping, field with the same information should be given the same type and
length throughout the mapping. Otherwise time will be spent on field conversions.11. Instead of doing complex calculation in query, use an expression transformer and
do the calculation in the mapping.12. If data is passing through multiple staging areas, removing the staging area will
increase performance.13. Stored procedures reduce performance. Try to keep the stored procedures simple in
the mappings.14. Unnecessary data type conversions should be avoided since the data type
conversions impact performance.15. Transformation errors result in performance degradation. Try running the mapping
after removing all transformations. If it is taking significantly less time than with thetransformations, then we have to finetune the transformation.
INTRODUCTION
ETL Life Cycle
What Is Informatica
Client Components
Services Behind Scene
Try U R Hand's on AdminConsole
Difference Between 7.1and 8.6
Informatica 8.6Installation
ADVANCED CONCEPTS
Mapping Parameter's &Variable
Mapplets
Partitioning
Working with links
Scheduler
Types of Task's 1
Types of Task's 2
Indirect Method forLoading
SCD Type 1
SCD Type 2
SCD Type 3
Incremental Aggregation
Mapping Templates
Grid Processing
Frequently UsedFunctions
Work Flow Variables
TRANSFORMATION
Filter
Expression
Router
Sorter
Rank
Transaction Control
Source Qualifier
Stored Procedure
SQL Transformation
Normalizer
Sequence Generator
Aggregator
Union
Joiner
Update Strategy
Look Up
Completṭ referencṭ t蘐껵 Informatic㜹췙
5/31/2016 Complete reference to Informatica: Optimizing the Bottleneck’s
http://informaticatutorialsnaveen.blogspot.in/2011/04/optimizingbottlenecks.html 2/3
Enter your email address:
Subscribe
Delivered by FeedBurner
16. Keep database interactions as less as possible.
PERFORMANCE TUNING OF SESSIONS
A session specifies the location from where the data is to be taken, where thetransformations are done and where the data is to be loaded. It has various properties thathelp us to schedule and run the job in the way we want.
1. Partition the session: This creates many connections to the source and target,and loads data in parallel pipelines. Each pipeline will be independent of the other.But the performance of the session will not improve if the number of records is less.Also the performance will not improve if it does updates and deletes. So sessionpartitioning should be used only if the volume of data is huge and the job is mainlyinsertion of data.
2. Run the sessions in parallel rather than serial to gain time, if they are independent ofeach other.
3. Drop constraints and indexes before we run session. Rebuild them after the sessionrun completes. Dropping can be done in pre session script and Rebuilding in postsession script. But if data is too much, dropping indexes and then rebuilding themetc. will be not possible. In such cases, stage all data, precreate the index, use atransportable table space and then load into database.
4. Use bulk loading, external loading etc. Bulk loading can be used only if the tabledoes not have an index.
5. In a session we have options to ‘Treat rows as ‘Data Driven, Insert, Update andDelete’. If update strategies are used, then we have to keep it as ‘Data Driven’. Butwhen the session does only insertion of rows into target table, it has to be kept as‘Insert’ to improve performance.
6. Increase the database commit level (The point at which the Informatica server is setto commit data to the target table. For e.g. commit level can be set at every every50,000 records)
7. By avoiding built in functions as much as possible, we can improve theperformance. E.g. For concatenation, the operator ‘||’ is faster than the functionCONCAT (). So use operators instead of functions, where possible. The functionslike IS_SPACES (), IS_NUMBER (), IFF (), DECODE () etc. reduce the performanceto a big extent in this order. Preference should be in the opposite order.
8. String functions like substring, ltrim, and rtrim reduce the performance. In thesources, use delimited strings in case the source flat files or use varchar data type.
9. Manipulating high precision data types will slow down Informatica server. So disable‘high precision’.
10. Localize all source and target tables, stored procedures, views, sequences etc. Trynot to connect across synonyms. Synonyms and aliases slow down theperformance.
DATABASE OPTIMISATION
To gain the best Informatica performance, the database tables, stored procedures andqueries used in Informatica should be tuned well.
1. If the source and target are flat files, then they should be present in the system inwhich the Informatica server is present.
2. Increase the network packet size.3. The performance of the Informatica server is related to network connections.Datagenerally moves across a network at less than 1 MB per second, whereas a localdisk moves data five to twenty times faster. Thus network connections often affecton session performance. So avoid network connections.
4. Optimize target databases.
Posted by Naveen at Sunday, April 17, 2011
No comments :
Post a Comment
5/31/2016 Complete reference to Informatica: Optimizing the Bottleneck’s
http://informaticatutorialsnaveen.blogspot.in/2011/04/optimizingbottlenecks.html 3/3
Enter your comment...
Comment as: Google Account
Publish Preview
Links to this post
Create a Link