bw performance tuning
TRANSCRIPT
BW PerformanceTuning
Version 1.6
May 7, 2003
Dr. Thomas Becker
Alexander Peter
SAP AG
SAP AG 2002, BW Performance Tuning, Alex Peter, 2
Targets
How To Help Yourself Overview on most important performance issuesPerformance Tools in SAP Service Marketplace
Please do …Measure the progress of your own performance tuning
Measure before tuningTune the system according to the tips and via using provided toolsMeasure after tuning
Provide Feedback
SAP AG 2002, BW Performance Tuning, Alex Peter, 3
BW Architecture
SAP AG 2002, BW Performance Tuning, Alex Peter, 4
Contents
Modeling
Query Performance
General Settings and Checklist
Extraction / Data Load Performance
Analysis Methods
SAP AG 2002, BW Performance Tuning, Alex Peter, 5
Performance Guidelines
Modeling
Query Performance
General Settings and Checklist
Extraction / Data Load Performance
Analysis Methods
SAP AG 2002, BW Performance Tuning, Alex Peter, 6
Performance Tuning in BW
Database
Database
Performance Tuning
Application
Application
Performance Tuning
OLTP SystemsApplication Development and performance tuning separatedPerformance tuning by basis experts
Performance Tuning
BWPerformance Tuning as holistic process over application design and database configuration
BW
OLTP
SAP AG 2002, BW Performance Tuning, Alex Peter, 7
General Performance Rules
Do it parallel: Deploy parallelism on all Do it parallel: Deploy parallelism on all available levelsavailable levels
Be economic: Eliminate all unnecessary Be economic: Eliminate all unnecessary processesprocesses
Keep it small: Reduce the data volume to Keep it small: Reduce the data volume to be processedbe processed
Basic Rules for Performance Optimization:
Parallel processes are fully scalable
SAP AG 2002, BW Performance Tuning, Alex Peter, 8
Checklist – 1 –
Basis Parameter SAP Recommendations according to SAP note 192658Also consider process settings: SAP BW does not need an UP2 process and at most 2 UPD processes
Database ParametersCheck of most important parameters via transaction RSRV database parameters (not available for all platforms)ORACLE DB parameter recommendations: SAP note 180605Informix DB parameter recommendations: SAP note 181945IBM DB2/UDB DB parameter recommendations: SAP notes 302429 (BW 2.x), 546262 (BW 3.x)IBM DB2/OS/390 parameter recommendations: SAP note 390016IBM DB2/iSeries parameter recommendations: SAP notes 307077, 501572, 541508SQL Server DB recommendations: SAP notes 327494, 28667
SAP AG 2002, BW Performance Tuning, Alex Peter, 9
Checklist – 2 –
ORACLE-specific Database Tuning ORACLE 8.1.x-scripts for improved meta data access as described in SAP note 565075Locally managed tablespaces: if you run BW 2.x, be sure that SAP notes 359835 and 387946 are applied. Be sure that your BW 3.x temporary tablespace is locally managed.Database statistics: SAP notes 351163 and 428212
Database PartitioningUse DB partitioning, especially for large InfoCube tablesORACLE, Informix and DB2/OS390 support range partitioning, DB2/UDB supports hash partitioning
SAP AG 2002, BW Performance Tuning, Alex Peter, 10
Checklist – 3 –
Web Application Tuning in BW 3.xUse HTTP Compression to reduce network traffic: see SAP note 550669Use Client (browser) caching of images to reduce unnecessary network traffic: see SAP note 561792
BW 3.0B SP9 / BW 3.1 Content SP3 or higherWe strongly recommend these support packagesInternal measurements have proven an average improvement of over40% (compared to 3.0B SP4) in Web Application Performance
SizingDecisive Factors: Number and Speed of CPU’s, Main Memory, Disk Volume and Architecture (e.g. RAID), I/O-ControllerRough Sizing available in SAP Service Marketplace alias quicksizerIn cooperation with hardware partner
SAP AG 2002, BW Performance Tuning, Alex Peter, 11
Checklist – 4 –
Sound Data ModelThe Data Model is one of the most important issues
BW Reporting PerformanceAggregatesPre-calculated Web TemplatesOLAP Cache
Extraction / Data Load PerformanceRecommendations in SAP notes 130253 and 417307
SAP AG 2002, BW Performance Tuning, Alex Peter, 12
Performance Guidelines – General Aspects – 1 –
Database (or Physical) PartitioningDatabase tables are cut into smaller chunks (partitions)One logical database tableTransparent for userAvailable for the following database management systems
Range Partitioning: ORACLE, Informix, IBM DB2/390Hash Partitioning: IBM DB2/UDB
Year / Month Customer Sales2003/01 4711 102003/01 4712 152003/01 4713 202003/01 4714 232003/02 4711 442003/02 4713 322003/02 4714 17
DatabaseTable
(logical)
DatabasePartition
DatabasePartition
SAP AG 2002, BW Performance Tuning, Alex Peter, 13
Performance Guidelines – General Aspects – 2 –
Database (or Physical) PartitioningBenefits
Parallel accesses to partitionsRead smaller sets of dataFast Deletion of partitions (DROP PARTITION instead DELETE FROM WHERE)
Automatically Partitioned Database Tables (for Range Partitioning)InfoCube F-Fact table: partitioned by requestPSA table: partitioned by requestODS Change Log: similar to PSA table
User Defined Partitioning Criteria (for Range Partitioning)InfoCube E-Fact tablePartition Criteria: Time characteristics like month or fiscal periodNote: Both fact tables are extended by the SID of the chosen time characteristic
SAP AG 2002, BW Performance Tuning, Alex Peter, 14
Performance Guidelines – General Aspects – 3 –
Database (or Physical) PartitioningQuery Access (Partition Pruning)
Jan - MarApr - JunJul - SepOct - Dec
Package 2Package 5Package 7
month
Master Data
Time
Package
Dimension
Fact Table
„F“
„E“
Example:
Month = August 2003
Apply Restrictions to Dimensions Apply Restriction
to fact table
Month Customer SalesAug 03 4711 15Aug 03 4712 66
Discard irrelevant partitions
F table has to be read, too
SAP AG 2002, BW Performance Tuning, Alex Peter, 15
Performance Guidelines – General Aspects – 4 –
Distribution of workloadApplication Server groups / Load Balancing
Logon load balancing (via group login): this allows you to distribute the workload of multiple query/administration users across several application serversDistribution of web users across application servers can be configured in the BEx service in SICFODS Object Data Activation is definable for specific server groups (BW 3.x)Process Chains can be processed on specified server groups (BW 3.x)Extraction in the mySAP source system can be processed on specified server groups: RFC destination in BW to source system must be defined accordingly (transaction SM59)Data load in BW can be processed on specified server groups: RFC destination in source system to BW must be defined accordingly (transaction SM59)Data staging via XML over HTTP/SOAP can be processed on specified server groups (BW 3.x)
Operation modesDifferent modes for mainly query processing (many dialogue jobs) and data load (several batch jobs)
SAP AG 2002, BW Performance Tuning, Alex Peter, 16
Performance Guidelines – General Aspects – 5 –
Other AspectsReorganization of Log Tables
SLG2 (SAP note 195157)EDI40 (SAP note 179046)RSDDSTAT-tables
Check for unnecessary traces and logs that impact system performanceAuthorization log (RSSMTRACE)User Trace (RSRTRACE)SQL Trace (ST05)ABAP Trace (SE30)Statistics Trace
SAP AG 2002, BW Performance Tuning, Alex Peter, 17
Performance Guidelines – General Aspects – 6 –
Other AspectsArchiving (as of BW 3.x)
Reduction of number of records in InfoCube fact tables / ODS object tables
ODSobject
InfoCubes • OpticalStorage system
• HSMSystem
• Storage-system
Archive-Files
Archive-Files
BW Functionality with ADK 3rd party products
SAP AG 2002, BW Performance Tuning, Alex Peter, 18
Performance Guidelines
Modeling
General Settings and Checklist
Query Performance
Extraction / Data Load Performance
Analysis Methods
SAP AG 2002, BW Performance Tuning, Alex Peter, 19
Checklist
Design of Operational Store, Data Warehouse Layer and Multidimensional Model
Dimensions of InfoCubes
Logical (MultiProvider) Partitioning
Time-Dependent Master Data
Non-Cumulative Key Figures
SAP AG 2002, BW Performance Tuning, Alex Peter, 20
Operational Data Store and Data Warehouse Layer
Data WarehouseNon volatileGranularIntegratedHistorical foundationBuilt with ODS Objects
Operational Data StoreOperational Reporting Near Real-Time / VolatileGranular Built with ODS Objects
Multidimensional ModelsMultidimensional analysisAggregated viewIntegratedBuilt with InfoCubes
ODS Object ODS Object
SAP AG 2002, BW Performance Tuning, Alex Peter, 21
InfoCubes SAP-Extended Star Schema
ONE Normalization of DimensionsMaster Data is shared, not part of the InfoCube
Fact TableFact Table
DimensionDimension
DimensionDimension DimensionDimension
DimensionDimension
Dimension ID
MasterData
MasterData
MasterData
MasterData
MasterData
MasterData
MasterData
MasterData
Surrogate Key
SAP AG 2002, BW Performance Tuning, Alex Peter, 22
InfoCubes
Master DataHierarchiesAttributesTexts (Multi-Language Support)
Fact TableFact Table
DimensionDimension
MasterData
MasterDataSID TableSID Table
Attributes
Hierarchies
TextsPlant UKPlant GermanyPlant Australia
Customer Street Account
SAP AG 2002, BW Performance Tuning, Alex Peter, 23
Data Modeling – Line Item Dimensions – 1 –
Line Item DimensionsDirect Link from InfoCube Fact Table to Master Data SID Table – without dimension tableOnly one characteristic is possible in a line item dimensionRecommended for large dimensions (e.g. invoice number, order number, but also reasonable for large material or customer dimensions)
SID Dimension Fact Table Dimension SID SID Fact Table Dimension SID
Without line item dimension
With line item dimension
SAP AG 2002, BW Performance Tuning, Alex Peter, 24
Data Modeling – Line Item Dimensions – 2 –
Line Item DimensionsBenefits
Saves one table join at query runtimeSaves determination of dimension ID’s at data load time
DisadvantagesF4 – Help not possible on dimension values (only on master data values)
SAP AG 2002, BW Performance Tuning, Alex Peter, 25
Data Modeling – MultiProviders – 1 –
MultiProvider (or Logical) PartitioningPossible partitioning criteria: year, plan/actual, regions, business areaParallel sub-queries are started automatically to basic InfoCubesUse MultiProvider Partitioning to cut large amounts of data in chunks
MultiProvider
Basic InfoCubes Europe Asia USA
Consolidated view on all data
SAP AG 2002, BW Performance Tuning, Alex Peter, 26
Data Modeling – MultiProviders – 2 –
MultiProviders (BW 3.x)Definition
Combination of InfoCubes, ODS objects, InfoObjects and InfoSets
With
Dat
a
Master Data
Basic InfoCube
ODS Object
No
Dat
a
InfoSet
Virtual InfoCubeMulti
ProviderMulti
Provider
SAP AG 2002, BW Performance Tuning, Alex Peter, 27
Data Modeling – MultiProviders – 3 –
MultiProvidersBenefits
No additional data storageSingle InfoProviders smaller, less complex and less sparsely filled than one big InfoProvider(Parallel) Data load into individual InfoProvidersQueries are split automatically and distributed to InfoProviders (parallel execution)Transparent usage for ReportingLocal queries on each InfoProvider possibleArchiving of single basic InfoProvider is very easy
DisadvantagesAdminstration (with aggregates)Additional I/O
SAP AG 2002, BW Performance Tuning, Alex Peter, 28
Data Modeling – MultiProviders – 4 –
MultiProvidersExample for sparsely filled single InfoCube
….. and the distribution to single InfoCube with consolidating MultiProvider:
MultiProvider
MultiProvider
Order Number Customer Product Order Date Delivery Date Billing Date Order qty Delivery Qty Billing Qty1 Imunex Pen 2001 * * 5 0 02 Pasag Ruler 2001 * * 8 0 01 Imunex Pen * 2002 * 0 4 02 Pasag Ruler * 2002 * 0 8 01 Imunex Pen * * 2002 0 0 52 Pasag Ruler * * 2003 0 0 7
InfoCube Order Number Customer Product Order Date Order qtyOrder 1 Imunex Pen 2001 5
2 Pasag Ruler 2001 8
InfoCube Order Number Customer Product Delivery Date Delivery QtyDelivery 1 Imunex Pen 2002 4
2 Pasag Ruler 2002 8
InfoCube Order Number Customer Product Billing Date Billing QtyBilling 1 Imunex Pen 2002 5
2 Pasag Ruler 2003 7
SAP AG 2002, BW Performance Tuning, Alex Peter, 29
Data Modeling – Time-Dependent Master Data – 1 –
Time-dependent master dataMaster Data is assigned to a validity periodKey date in query defines the valid characteristic valuesExample:
Country Valid From Valid To Sales PersonUK 2002/01/01 2002/08/31 MillerUK 2002/09/01 2003/02/28 StevensonUSA 2002/01/01 2002/10/31 SmithUSA 2002/11/01 2003/02/28 MillerAustralia 2002/01/01 2002/04/30 LeaskAustralia 2002/05/01 2003/02/28 Cassidy
Master Data Table: Country
2002 2003Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb
UKUSA
Australia CassidyMiller
StevensonMillerSmith
Leask
Example: Keydate 2002/09/15 delivers sales persons Stevenson, Smith and Cassidy for UK, USA and Australia, respectively, for all records within the result set
SAP AG 2002, BW Performance Tuning, Alex Peter, 30
Data Modeling – Time-Dependent Master Data – 2 –
Time-dependent master dataNote that a query on time-dependent attributes does NOT perform a join between the validity period and the InfoCube time dimensionAggregates on time-dependent master data possible as of BW 3.xAdjustment Process necessary when key date changes
Usually expensiveSometimes it is reasonable to model time-dependent attributes ALSO as time-independent to achieve a good response time for „today is yesterday“ - reports
SAP AG 2002, BW Performance Tuning, Alex Peter, 31
Data Modeling – Non-Cumulative KFs – 1 –
Non-cumulative key figuresNon-cumulative key figures cannot be cumulated meaningfully over time, e.g. inventory, number of employeesStorage: Historical movements and reference point (in E fact table)Reference point is updated when InfoCube is compressed; if all requests are compressed, the reference point represents e.g. the current stockExample:
Month Material Plant Material flow Reference pointJan 02 4711 1000 10 0
10Feb 02 4711 1000 20 10Mar 02 4711 1000 5 10
35
Compressing
Compressing
Note: the reference points are only stored in the compressed E-table; they have the time value “infinity”, e.g. for day 31.12.9999
SAP AG 2002, BW Performance Tuning, Alex Peter, 32
Data Modeling – Non-Cumulative KFs – 2 –
Performance Tips & Tricks:Compress InfoCubes with non-cumulative key figures as soon as possibleVery granular characteristic value combinations generate lots ofreference points in the E table
Aggregate rebuild is significantly influencedDelete unused values (e.g. old seasonal material) from the InfoCube by selective deletion without timely restriction)
LAST- and FIRST- key figures can use aggregates; AVG-key figures cannot use them. Try to split these key figures in separate queries, if semantically possibleAt query time use tight restrictions on time characteristics; ideally request only current valuesSuppress sum lines if not neededDo not use partial time characteristics (e.g. FISCPER3) if not needed
SAP AG 2002, BW Performance Tuning, Alex Peter, 33
Data Modeling – Non-Cumulative KFs – 2 –
Validity ObjectsValidity Objects are characteristics for which a validity period can be definedThe validity is stored within the validity tableExample: If plant 1 is closed at the end of March, it should not show up any value (even no zero) in April and May
Every entry in the validity table generates an individual SELECT at query runtime.Use as few validity objects as possibleDo not misuse validity objects for termination (e.g. insurance contract)
Jan Feb Mar Apr May Reference pointPlant 1 +10 +20 +30 100Plant 2 +20 +10 +10 +20 +30 150
SAP AG 2002, BW Performance Tuning, Alex Peter, 34
Checklist – Query Design
Multi-dimensional Query Design
Inclusion / Exclusion
MultiProvider Query
Cell Calculation
Customer Exits
Query Read Mode
SAP AG 2002, BW Performance Tuning, Alex Peter, 35
Query Design – Multi-Dimensional Navigation
Multi-dimensional NavigationSAP BW provides very powerful multi-dimensional reporting capabilities like slice&dice, drill-down, drill-throughEvery Query should start with a relatively small result set; let the user drill down to more detailed informationDo not use ODS objects for multi-dimensional reporting
Not Recommended
as first result
SAP AG 2002, BW Performance Tuning, Alex Peter, 36
Query Design – Inclusion/Exclusion
Inclusion/ExclusionSelections can include and exclude characteristic valuesExclusion cannot access database indicesBetter use inclusion and avoid exclusion of char values
recommendedNot recommended
SAP AG 2002, BW Performance Tuning, Alex Peter, 37
Query Design – MultiProvider
Query on MultiProviderQueries on MultiProviders usually access all underlying InfoProviders, even if some cannot be hit as no key figures within the query definition are contained in this InfoProviderUse 0INFOPROVIDER to restrict the query to a selection of individual InfoProviders
SAP AG 2002, BW Performance Tuning, Alex Peter, 38
Query Design – Others
Other Query Design TipsCell calculation by means of the cell editor generates separate queries at query runtimeBe cautious with cell calculationsCustomer-specific code is necessary for virtual key figures and characteristicsCheck Code in Customer ExitsRead Mode For QueriesUsually “Read when navigating and expanding hierarchies” is recommended
SAP AG 2002, BW Performance Tuning, Alex Peter, 39
Performance Guidelines
Query Performance
General Settings and Checklist
Modeling
Extraction / Data Load Performance
Analysis Methods
SAP AG 2002, BW Performance Tuning, Alex Peter, 40
Different Types of Users in OLAP Reporting
high
low
Ana
lytic
al a
bilit
y re
quire
d
Report creationAd-hoc reporting
Multi-dimensional analysis
Self-explanatory, predefined navigational pathsPredefined data collectionsStatic reports
Consumer
Analyst
Author
Information
Consumer
Analyst
Power User
~70%
~20%
~8%
Stand-Alone Query Designer*~2%
*Only needed for
Global Report Creation
~98% Potential web reporting users
SAP AG 2002, BW Performance Tuning, Alex Peter, 41
Checklist – Query and Web Performance – Overview
ST03 / Technical ContentST03 / Technical Content
DatabaseDatabase
Which component contributes most?
OLAPOLAP FrontendFrontend
1. Data Model
2. Query Definition
3. Aggregates
4. OLAP Cache
5. Pre-Calculated Web Templates
6. Compressing
7. Indices
8. DB Statistics
9. DB and basis (Buffer) Parameter
1. Data Model
2. Query Definition
3. Aggregates
4. OLAP Cache
5. Pre-Calculated Web Templates
6. Compressing
7. Indices
8. DB Statistics
9. DB and basis (Buffer) Parameter
Check these points
1. Data Model
2. Query Definition (including OLAP features)
3. Aggregates
4. OLAP Cache
5. Virtual Key Figures / Characteristics
6. Authorizations
1. Data Model
2. Query Definition (including OLAP features)
3. Aggregates
4. OLAP Cache
5. Virtual Key Figures / Characteristics
6. Authorizations
1. Query Definition
2. Network
3. WAN and BEx
4. Client Hardware
5. VBA / Java
6. Documents
7. Formatting
8. ODBO / 3rd party
1. Query Definition
2. Network
3. WAN and BEx
4. Client Hardware
5. VBA / Java
6. Documents
7. Formatting
8. ODBO / 3rd party
SQL Trace (ST05)
RSRV
RSRT, RSRTRACE
SQL Trace (ST05)
RSRV
RSRT, RSRTRACE
RSRT, RSRTRACE
SQL Trace (ST05)
ABAP Trace (SE30)
RSRT, RSRTRACE
SQL Trace (ST05)
ABAP Trace (SE30)IEMON
RSRT, RSRTRACE
IEMON
RSRT, RSRTRACETools
SAP AG 2002, BW Performance Tuning, Alex Peter, 42
Checklist – Query and Web Performance – 1 –
ST03 / Technical ContentST03 / Technical Content
DatabaseDatabase
1. Data Model
2. Query Definition
3. Aggregates
4. OLAP Cache
5. Pre-Calculated Web Templates
6. Compressing
7. Indices
8. DB Statistics
9. DB Parameter, AppBuffer
1. Data Model
2. Query Definition
3. Aggregates
4. OLAP Cache
5. Pre-Calculated Web Templates
6. Compressing
7. Indices
8. DB Statistics
9. DB Parameter, AppBuffer
Query Monitor (RSRT)
Query Trace (RSRTRACE)
Query Monitor (RSRT)
Query Trace (RSRTRACE)
ToolsCheck these points
SQL Trace (ST05)
Analysis and repair of BW objects (RSRV)
DB / Buffer Monitors (ST04/ST02)
SQL Trace (ST05)
Analysis and repair of BW objects (RSRV)
DB / Buffer Monitors (ST04/ST02)
InfoCube MaintenanceInfoCube Maintenance
SAP AG 2002, BW Performance Tuning, Alex Peter, 43
Checklist – Query and Web Performance – 2 –
ST03 / Technical ContentST03 / Technical Content
OLAPOLAP
1. Data Model
2. Query Definition (including OLAP features)
3. Aggregates
4. OLAP Cache
5. Virtual Key Figures / Characteristics
6. Authorizations
1. Data Model
2. Query Definition (including OLAP features)
3. Aggregates
4. OLAP Cache
5. Virtual Key Figures / Characteristics
6. Authorizations
Query Monitor (RSRT)
Query Trace (RSRTRACE)
Query Monitor (RSRT)
Query Trace (RSRTRACE)
ABAP Trace (SE30)
SQL Trace (ST05)
ABAP Trace (SE30)
SQL Trace (ST05)
ToolsCheck these points
Authorization Trace (RSSMTRACE)Authorization Trace (RSSMTRACE)
SAP AG 2002, BW Performance Tuning, Alex Peter, 44
Checklist – Query and Web Performance – 3 –
ST03 / Technical ContentST03 / Technical Content
FrontendFrontend
1. Query Definition
2. Network
3. WAN and BEx
4. Client Hardware
5. VBA / Java
6. Documents
7. Formatting
8. ODBO / 3rd party
1. Query Definition
2. Network
3. WAN and BEx
4. Client Hardware
5. VBA / Java
6. Documents
7. Formatting
8. ODBO / 3rd party
OS Monitor (ST06)
IEMON
RSRTRACE (with “web ping”)
OS Monitor (ST06)
IEMON
RSRTRACE (with “web ping”)
ToolsCheck these points
Query Monitor (RSRT)Query Monitor (RSRT)
Frontend system monitorFrontend system monitor
IEMONIEMON
SAP AG 2002, BW Performance Tuning, Alex Peter, 45
Performance Layers
ReuseReuse
PerformancePerformance
Offline Analysis
Pre-Calculation
OLAP Cache
Aggregates
InfoCubes
Performance LayersFor specific Scenarios
SAP AG 2002, BW Performance Tuning, Alex Peter, 46
Aggregates – 1 –
Aggregate DefinitionMaterialization of aggregated subsets of InfoCube fact table dataIndependent structures where summary data is stored within separate, transparent InfoCubesTransparency: Users do not notice if aggregate is hit or notImproved query performance by reducing the amount of data to be read from DB
Aggregates can be createdOnly on top of basic InfoCubesFor dimension characteristicsFor navigational attributesOn hierarchy levelsUsing time-dependent navigational attributes (as of BW 3.x)Using hierarchy levels where the structure is time-dependent (as of BW 3.x)
Note: Aggregates cannot be built on MultiProviders, SAP Remote Cubes, Remote Cubes or ODS ObjectsAggregates can improve query performance considerably, but keep in mind that they also impact the load performance. For more information, see chapter on data load performance.
SAP AG 2002, BW Performance Tuning, Alex Peter, 47
Aggregates – 2 – Example
Example for flow with and without aggregates
Flow without aggregate
InfoCube Number of records read on the database Records
transferred to BW instance after
being summarized on the database
Month Material RevenueJuly Hammer 10July Nail 20August Hammer 10August Nail 20
Aggregate
Month RevenueJuly 30August 30
Month Material RevenueJuly Hammer 10July Nail 20August Hammer 10August Nail 20
Month RevenueJuly 30August 30
Month RevenueJuly 30August 30
Flow with aggregate
Database Selection OLAP Engine
SAP AG 2002, BW Performance Tuning, Alex Peter, 48
Aggregates – 3 – Example
Country Customer Sales
USAGermanyUSAAustriaAustriaGermanyUSA
Winsoft Inc.InternetworksFunny Duds Inc.InternetworksThor IndustriesFunny Duds Inc.Winsoft Inc.
10155
10102025
Fact Table: Sales DataFact Table: Sales Data Aggregate Tables: Sales DataAggregate Tables: Sales Data
Country *Customer Space
Country *Customer Space
Country Sales
403520
USAGermanyAustria
Summarize by characteristic customerAbbreviation: customer = space
SAP AG 2002, BW Performance Tuning, Alex Peter, 49
Aggregates – 4 – Example
Fixed Value (Germany) for country (subset of data)Abbreviation: country = F, Germany
Country Customer Sales
USAGermanyUSAAustriaAustriaGermanyUSA
Winsoft Inc.InternetworksFunny Duds Inc.InternetworksThor IndustriesFunny Duds Inc.Winsoft Inc.
10155
10102025
Fact Table: Sales DataFact Table: Sales Data Aggregate Tables: Sales DataAggregate Tables: Sales Data
Country F, GermanyCustomer *
Country F, GermanyCustomer *
SalesCustomerCountry
GermanyGermany
InternetworksFunny Duds Inc.
1520
SAP AG 2002, BW Performance Tuning, Alex Peter, 50
Aggregates – 5 – Example
Summarize by navigational attribute industryAbbreviation: customer__industry = *
Aggregate Tables: Sales DataAggregate Tables: Sales Data
Country SpaceCustomer__Industry *
Country SpaceCustomer__Industry *
Industry Sales
602510
TechnologyConsumer ProductsChemical
TechnologyConsumer ProductsTechnologyChemical
Navigational Attribute for Characteristic Customer
Customer Industry
Winsoft Inc.Funny Duds Inc.InternetworksThor Industries
Navigational Attribute for Characteristic Customer
Fact Table: Sales DataFact Table: Sales Data
Country Customer Sales
Winsoft Inc.InternetworksFunny Duds Inc.InternetworksThor IndustriesFunny Duds Inc.Winsoft Inc.
1015510102025
USAGermanyUSAAustriaAustriaGermanyUSA
SAP AG 2002, BW Performance Tuning, Alex Peter, 51
Aggregates – 6 – Example
Summarize on hierarchy level 2 of country hierarchyAbbreviation: country = H, Level 2
AllAll
EuropeEurope
Hierarchy for CountryHierarchy for Country
Time-independent hierarchies are stored outside the dimension, in a table called /BI0/ICOUNTRY
AmericaAmerica
USAUSA
Country Customer Sales
USAGermanyUSAAustriaAustriaGermanyUSA
Winsoft Inc.InternetworksFunny Duds Inc.InternetworksThor IndustriesFunny Duds Inc.Winsoft Inc.
1015510102025
Fact Table: Sales DataFact Table: Sales Data Aggregate Tables: Sales DataAggregate Tables: Sales Data
Country H, Level 2Customer Space
Country H, Level 2Customer Space
Country Sales
4055
AmericaEurope
GermanyGermany AustriaAustria
SAP AG 2002, BW Performance Tuning, Alex Peter, 52
Aggregates – 7 – Tips to Find Aggregates
Use Technical Content (=BW Statistics) or Workload Monitor (transaction ST03)
Based on query statistics (table RSDDSTAT)
BW Statistics
BW Workload Analysis – ST03
SAP AG 2002, BW Performance Tuning, Alex Peter, 53
Aggregates – 8 – Tips for Creating Aggregates
Do the following stepsCall query or InfoCube overview in technical content or ST03Sort by mean overall time to find queries/InfoCubes with highest runtimesCalculate the KPI ‘aggregation ratio’ = number of records read from DB / number of records transferred Check quota of database time to total runtimeAs a rule of thumb, an aggregate is reasonable and may be created, if:
Aggregation ratio > 10Aggregation ratio > 10, I.e. 10 times more records are read than are
displayed, ANDANDPercentage of DB time > 30%Percentage of DB time > 30%, I.e. the time spent on database is a substantial part of the whole query runtime
SAP AG 2002, BW Performance Tuning, Alex Peter, 54
How to build Aggregates – 1 –
Sort by mean overall time to find InfoCubes with queries having the highest runtimes
SAP AG 2002, BW Performance Tuning, Alex Peter, 55
How to build Aggregates – 2 –
Database time ~40% of total runtime (6771 of 17 025 seconds)
Ratio records selected / records transferred: 24
Aggregates will probably improve query performance
SAP AG 2002, BW Performance Tuning, Alex Peter, 56
How to build Aggregates – 3 –
Database time ~3,5% of total runtime (1011 of 28 540 seconds)
Ratio records selected / records transferred: 32
Aggregates will not improve query performance
SAP AG 2002, BW Performance Tuning, Alex Peter, 57
How to build Aggregates – 4 –
Database time ~90% of total runtime (1420 of 1572 seconds)
Aggregates will not improve query performance
Ratio records selected / records transferred: 1.1
SAP AG 2002, BW Performance Tuning, Alex Peter, 58
How to build Aggregates – 5 – (ST03)
Highest contributor in Total Run Time per
InfoCube
High % DB Time
Expert Mode!!
High ratio:Selected /
transferred records
SAP AG 2002, BW Performance Tuning, Alex Peter, 59
How to build Aggregates – 6 –
Utilizing OLAP per QuerySort by overall time to find queries that have the highest total runtime
SAP AG 2002, BW Performance Tuning, Alex Peter, 60
How to build Aggregates – 7 –
Utilizing OLAP per Query Database time ~60% of total runtime (2.798 of 4.685 seconds)
Ratio records selected / records transferred: 54
Aggregates will improve query performance
SAP AG 2002, BW Performance Tuning, Alex Peter, 61
How to build Aggregates – 8 –
Database time ~89% of total runtime (1064 of 1162 seconds)
Ratio records selected / records transferred: 2.1
Aggregates will not improve query performance
SAP AG 2002, BW Performance Tuning, Alex Peter, 62
PARALLEL Query on MultiCubes in BW 2.x
Queries on MultiCubes are split up over the different BasisCubeswith parallel access to the InfoCube or maximal one aggregate per Cube
Query on MultiCube => 40 sec runtime
BasisCubeI
BasisCubeII
Sub-Query I Sub-Query II Sub-Query III
AggregateBasisCube
III
15 sec 40 sec 2 sec
SAP AG 2002, BW Performance Tuning, Alex Peter, 63
NOPARALLEL Query on MultiCubes in BW 2.x
Using the NOPARALLEL–option in table RSADMIN:
the BasisCubes are accessed in serial, but more than one aggregate per Infocube
can be used
Query on MultiCube
Sub-Query IISub-Query I Sub-Query III
15 sec
1 sec 3 sec 2 sec
2 sec
BasisCubeI
Aggregate IBasisCube
II
Aggregate IIBasisCube
II
Aggregate IIIBasisCube
II
AggregateBasisCube
III
=>23 sec runtime
SAP AG 2002, BW Performance Tuning, Alex Peter, 64
Query on MultiProvider in BW 3.X
Query on MultiProvider
Sub-Query II Sub-Query III
15 sec
1 sec 3 sec 2 sec
2 sec
BasisCubeI
Sub-Query I
Aggregate IBasisCube
II
Aggregate IIBasisCube
II
Aggregate IIIBasisCube
II
AggregateBasisCube
III
=>15 sec runtime
SAP AG 2002, BW Performance Tuning, Alex Peter, 65
Aggregates Features
Further Performance Features in BW 3.xBlock Size
Aggregate Build for large InfoCubes are resource-consuming (e.g. temporary tablespace, memory)Data of large InfoCubes can be read in several blocks to prevent resource problems when filling an aggregate.The block size can be customized (system-wide) – optimum depends on system sizing and on underlying DBMSBlocks are internally distinguished by characteristic values
Flat AggregatesIf an aggregate has less then 15 components, each component is put into a separate dimension (“Flat Aggregates”)The dimensions (except the package and unit) are marked as “Line Item”Flat Aggregates are filled / rolled up without loading the data into the application server (performance gain)Line item / high cardinality dimensions already specified in the InfoCube are also used for aggregates for corresponding dimensions (also for non-flat aggregates)
SAP AG 2002, BW Performance Tuning, Alex Peter, 66
OLAP Cache – 1 –
Caching in BW 2.xOne Local Cache for each sessionNo cache access across sessionsNo possibility to set size of cache
Caching in BW 3.x OLAP CacheAdditional global cache which is accessible from all sessionsOLAP Cache is part of application buffer (Imp/Exp SHM) and can also be stored in a DB table or a fileThe Cache stores query results and navigation statuses as highly compressed cluster dataQuery Cache is used for equal queries or subsets of cached queriesBenefits: OLAP Cache reduces workload on database and application serverInvalidations:
Data Load into underlying DataTargetQuery Generation
SAP AG 2002, BW Performance Tuning, Alex Peter, 67
OLAP Cache – 2 –
OLAP CacheOLAP Cache size can be adjusted(In 3.x there is also an adjustable size for the local cache)Cache can be switched off entirely, for InfoCubes and for individual queriesLocal Cache is used, when OLAP Cache is inactive.Specific instance profile parameters:
rsdb/esm/buffersize_kb 4096 (kB) Size of exp/imp SHM buffer
rsdb/esm/max_objects 2000 Max. number of objects in the buffer
rsdb/esm/large_object_size 8192 (byte)Estimation for the size of the largest object
rsdb/esm/mutex_n 0 Number of mutexes in Exp/Imp SHM buffer
Buffer small by default!
SAP AG 2002, BW Performance Tuning, Alex Peter, 68
OLAP Cache – 3 –
OLAP Cache Usage
InfoCube
Front-end
Application ServerApplication Server
Database ServerDatabase Server
CurrentQuery View
AnalyzerDisplay
andManipulation
OLAP Processor
...
QueryDefinition (BEx)
Query Execution
Aggregates
Data Manager
OLAP Cache1st
2nd3rd
Memory
SAP AG 2002, BW Performance Tuning, Alex Peter, 69
OLAP Cache – 4 –
OLAP Cache MonitorAccessible via RSRTCheck status of cacheDelete cache entries
Delete Cache
SAP AG 2002, BW Performance Tuning, Alex Peter, 70
Pre-Calculated Web Templates – 1 –
Pre-Calculated Web TemplatesPre-calculation is a set of techniques where you can distribute the workload of running the report to off-peak hours, and have the report result set ready for very fast access to the dataData Pre-Calculation or HTML Pre-Calculation
HTML Pre-Calculation can be used for Offline AnalysisBenefits
Fast response timeSystem workload shifted to off-peak hoursRe-use data that goes to many queriesReports also available offline
SAP AG 2002, BW Performance Tuning, Alex Peter, 71
Pre-Calculated Web Templates – 2 –
ScenarioSupports “Newspaper” scenario
Quick access to many reportsReports are requested by many usersStatic reporting, usually little navigation requirementsActive Reporting Authorisations
RestrictionsNo invalidation when new data is loadedRestricted navigation: only filtering possibles
SAP AG 2002, BW Performance Tuning, Alex Peter, 72
Pre-Calculated Web Templates – 3 – Access Modes
BW Server
OLAPEngine
Reporting Agent
Log
Content Management Framework
Web Framework
SAP GUI
Web Application Server
BEx Web Service Doc Web Service
WebDAV Handler
Web Browser
http://myserver:myport/sap/bw/BEx?cmd=ldoc&template_id=myTemplate&data_mode=new
DATA MODE
NEW
STORED
HYBRID
STATIC
STATIC_HYBRID
InfoProvider RA Data Storage CMF Repository
SAP AG 2002, BW Performance Tuning, Alex Peter, 73
Pre-Calculated Web Templates – 4 – Access Modes
BW Server
OLAPEngine
Reporting Agent
Log
Content Management Framework
Web Framework
SAP GUI
Web Application Server
BEx Web Service Doc Web Service
WebDAV Handler
Web Browser
http://myserver:myport/sap/bw/BEx?cmd=ldoc&template_id=myTemplate&data_mode=stored
DATA MODE
NEW
STORED
HYBRID
STATIC
STATIC_HYBRID
InfoProvider RA Data Storage CMF Repository
SAP AG 2002, BW Performance Tuning, Alex Peter, 74
Pre-Calculated Web Templates – 5 – Access Modes
BW Server
OLAPEngine
Reporting Agent
Log
Content Management Framework
Web Framework
SAP GUI
Web Application Server
BEx Web Service Doc Web Service
WebDAV Handler
Web Browserhttp://myserver:myport/sap/bw/BEx?cmd=ldoc&template_id=myTemplate&data_mode=static
DATA MODE
NEW
STORED
HYBRID
STATIC
STATIC_HYBRID
http://myserver:myport/sap/bw/doc/tmpl/fldtmpl/“cmfpath/report.htm“
InfoProvider RA Data Storage CMF Repository
SAP AG 2002, BW Performance Tuning, Alex Peter, 75
Reporting Performance – Indices
IndicesIf you report on ODS objects with a restrictive selection key, check if indices are defined for this keyCheck if the InfoCube Indices exist
In ORACLE, you can choose between bitmap index and B-tree indexUse B-tree index, if dimension size exceeds 10% of the fact table size
If you select on navigational attributes, be sure that an appropriate index is available
SAP AG 2002, BW Performance Tuning, Alex Peter, 76
Reporting Performance – ODS Objects
Indexing ODS ObjectsIndices can be defined in the
3.x ODS object maintenance; in BW 2.x indices must be defined via SE11
Use indices on characteristics which you are accessing regularly (for reporting or DataMart interface)
Doc.No. Customer Sales4711 Miller 104712 Smith 204713 Miller 254714 Cassano 354715 Birch 124716 Smith 44
ODS Object
Index
Doc.No. Customer Sales4711 Miller 10 Miller4712 Smith 20 Smith4713 Miller 25 Cassano4714 Cassano 35 Birch4715 Birch 124716 Smith 44
ODS Object
Full
table
scan
Example: If you look for sales figures with customer Birch, a full table scan is necessary when there is no index (see picture above); if the index has been created, only two DB accesses are necessary (see picture on the left).
SAP AG 2002, BW Performance Tuning, Alex Peter, 77
Reporting Performance – Indices
Index types (selection)Bitmap Index
Well-suited for few characteristic valuesSeveral bitmap indexes can be used for one table (bitmap merge join)
B-Tree IndexGood for range queries and for high cardinality
rowidMiller 00110000111Smith 10000011100Trapp 01001100000
Miller
Leask Osbourne Smith
PageSmith
BrownLeask
MoloneyOsbourne TrappMiller
SAP AG 2002, BW Performance Tuning, Alex Peter, 78
Indexing in BW
Fact TableNon-Unique Index on every dimension ID
ORACLE: Bitmap (except for “high cardinality” dimensions)ORACLE: B-tree on F-fact table for transactional InfoCubes (guarantees parallel read&write)ORACLE: additional bitmap index on time characteristic SID for partitioned InfoCubesDB2/AS400: EVIOther DBMSs: B-TreeDB2/OS390: B-Tree not on package dimension ID
Non-unique B-Tree on all dimension IDs (Primary Key)Used for compressionORACLE/Informix: only on E-fact table
SAP AG 2002, BW Performance Tuning, Alex Peter, 79
Indexing in BW
DimensionsDimension ID (primary key)
Unique B-TreeMaster Data SIDs
Non-Unique B-TreeORACLE / DB2/AS400: not on first SID (this is included in the index on all SIDs)
On all master data SIDsNon-Unique B-Tree
Master Data X- and Y- tables (SIDs for all navigational attributes)
Unique B-Tree on <SID, Objectversion> (primary key)Additional indices could be defined for individual SID-columns to avoid full table scans (for selective accesses)
S-table (SID)Unique B-Tree on master data key (primary key)Unique B-Tree on SID
SAP AG 2002, BW Performance Tuning, Alex Peter, 80
Cost-Based Optimizer (CBO)
CBOCalculation of the optimal execution plan
E.g. full table scan vs. indexE.g. Index range scans vs. unique scan (only one result)E.g. nested loop vs. hash join (vs. sort merge join)
Based on table and index statisticsNumber of recordsNumber of different keys
Based on histograms (ORACLE 9i)Detailed information about data distributions which are non-uniform
Recalculate statistics when there is a major change in the table structure (e.g. data load)
SAP AG 2002, BW Performance Tuning, Alex Peter, 81
InfoCube – Compression – 1 – Double Fact Table
“F” TableRequest InformationUsually smallOptimised for Loading and Deleting
“E” TableOptimised for QueriesUsually largeUser-defined DB Partitioning
Fact TableFact Table
REQUEST No. Time Material Sales
F - Table
REQUEST No. Time Material Sales
E - TableCompression
Upload
As InfoPackages are added, F fact table partitions are created
SAP AG 2002, BW Performance Tuning, Alex Peter, 82
InfoCube – Compression – 2 –
CompressionWe recommend to compress as soon as possibleCompression usually reduces the number of records by combining records with the same key that has been loaded in separate requestsThe user-defined DB partitioning (depending on the DBMS) is only affecting the compressed E-tableAdvantage
Query performance is usually improved significantlyDisadvantage
Note that you lose the request information after the compression
SAP AG 2002, BW Performance Tuning, Alex Peter, 83
Aggregates and Compression
Request Handling within BW 3.x AggregatesIn BW 3.x, InfoCubes can be marked such that the request is kept in its aggregatesRequests which are not compressed can then be deleted out of theInfoCube and its aggregates, without completely rebuilding the aggregatesAggregates of marked InfoCubes are compressed together with the InfoCubeCompress aggregates as soon as possible to avoid query performance penalty
SAP AG 2002, BW Performance Tuning, Alex Peter, 84
Reporting Performance – Authorization
Reporting AuthorizationsReporting Authorizations restrict critical data to certain user/user groupsDifferent authorization levels:
InfoCubeQueryKey figuresCharacteristic valuesHierarchies
Complex authorization checks on large hierarchies including several hierarchy levels can slow down query performance
SAP AG 2002, BW Performance Tuning, Alex Peter, 85
Reporting Performance – Frontend / Network – 1 –
Web Applications vs. BEx AnalyzerAdvantages of Web Applications
HTTP Compression, MIME CompressionBrowser Caching of imagesSignificantly less roundtrips than BEx Analyzer
Browser-based web applications, using http, represents a significant gain in query response time – especially in a WAN!
SAP AG 2002, BW Performance Tuning, Alex Peter, 86
Reporting Performance – Frontend / Network – 2 –
DocumentsTransferring large documents (e.g. as document items) can cause significant network traffic
FormattingMany Different formatting definition (especially in BEx Analyzer) can impact the frontend performance of queries displaying many linesIf this is considered to be a problem, try to reduce sum lines (as they require a new format definition transfer) or switch off formatting completely
SAP AG 2002, BW Performance Tuning, Alex Peter, 87
Performance Guidelines
Extraction / Data Load Performance
General Settings and Checklist
Modeling
Query Performance
Analysis Methods
SAP AG 2002, BW Performance Tuning, Alex Peter, 88
Checklist – Data Load Performance – Overview 1 –
Technical Content, Data Load MonitorTechnical Content, Data Load Monitor
ExtractionExtraction
Which component contributes most?
TransferTransfer Load Into PSALoad Into PSA
1. Customer Exits
2. Resource Utilization
3. Load Balancing
4. Data Package Size
5. Indices on tables
6. Flat File format
7. Content vs. generic extractor
1. Customer Exits
2. Resource Utilization
3. Load Balancing
4. Data Package Size
5. Indices on tables
6. Flat File format
7. Content vs. generic extractor
Check these points
1. Resource Contraint
2. CPU / Memory Bottleneck
3. Network
4. Application Buffer Synchronization
1. Resource Contraint
2. CPU / Memory Bottleneck
3. Network
4. Application Buffer Synchronization
1. I/O Contention1. I/O Contention
Extractor Checker (RSA3),
ABAP Trace (SE30),
SQL Trace (ST05)
Extractor Checker (RSA3),
ABAP Trace (SE30),
SQL Trace (ST05)
SM50
SQL Trace (ST05)
OS Monitor (ST06)
SM50
SQL Trace (ST05)
OS Monitor (ST06)
ToolsOS Monitor (ST06)
DB Monitor (ST04)
OS Monitor (ST06)
DB Monitor (ST04)
SAP AG 2002, BW Performance Tuning, Alex Peter, 89
Checklist – Data Load Performance – Overview 2 –
Which component contributes most?
Technical Content, Data Load MonitorTechnical Content, Data Load Monitor
1. Transformation Rules / ABAP Coding
2. Transformation Library Formulas
1. Transformation Rules / ABAP Coding
2. Transformation Library Formulas
1. Roll-up
2. Change Run
3. Compression
4. Indices
5. Load Master Data before Transaction Data
6. Buffering Number Ranges
1. Roll-up
2. Change Run
3. Compression
4. Indices
5. Load Master Data before Transaction Data
6. Buffering Number Ranges
Debugger within Monitor
ABAP Trace (SE30),
SQL Trace (ST05)
Debugger within Monitor
ABAP Trace (SE30),
SQL Trace (ST05) SQL Trace (ST05)SQL Trace (ST05)Tools
Check these points
Transfer RulesTransfer Rules Load Into Data TargetsLoad Into Data Targets
Update RulesUpdate Rules
InfoCubesInfoCubes ODS ObjectsODS ObjectsMaster DataMaster Data
1. Parallel ODS activation
2. Unique Data Records
3. Flag BEx Reporting
4. Indices
1. Parallel ODS activation
2. Unique Data Records
3. Flag BEx Reporting
4. Indices
1. Buffering Number Ranges
2. Change Run
1. Buffering Number Ranges
2. Change Run
SAP AG 2002, BW Performance Tuning, Alex Peter, 90
Performance Guidelines – Extraction / Staging –
Typical BW Data Load Process
SourceSystem
Business InformationWarehouse
BWS-APIBW
S-API
PSA
LoadingProcess
Extractor
Extractor
InfoCube
ODS
BWS-APIBW
S-API
ALE
ALE
IDOC
ALE
IDOC
UpdateRules
TransferRules
ALE
Scheduler
tRFC
SAP AG 2002, BW Performance Tuning, Alex Peter, 91
Performance Guidelines – Extraction / Staging –
Determine the extraction time:
Data load Monitor :Transaction RSMO
SourceSystem
Business InformationWarehouse
PSA
LoadingProcess
Extractor
Extractor
InfoCube
ODS
ALE
ALE
IDOC
ALE
IDOC
ALEScheduler
tRFCUpdaterules
Transferrules
S-API
S-API
Extraction
Data Load Monitor :Transaction RSMO
SAP AG 2002, BW Performance Tuning, Alex Peter, 92
Checklist – Data Load Performance – Extraction –
Technical Content, Data Load MonitorTechnical Content, Data Load Monitor
ExtractionExtraction
1. Customer Exits
2. Resource Utilization
3. Load Balancing
4. Data Package Size
5. Indices on tables
6. Flat File format
7. Content vs. generic extractor
1. Customer Exits
2. Resource Utilization
3. Load Balancing
4. Data Package Size
5. Indices on tables
6. Flat File format
7. Content vs. generic extractor
Extractor Checker (RSA3),
ABAP Trace (SE30),
SQL Trace (ST05)
Extractor Checker (RSA3),
ABAP Trace (SE30),
SQL Trace (ST05)
Process Overview (SM50 / SM51)Process Overview (SM50 / SM51)
SQL Trace (ST05)SQL Trace (ST05)
Check these points Tools
SAP AG 2002, BW Performance Tuning, Alex Peter, 93
Data Extraction Performance – 1 –
Customer ExitsTake Recommendations of performance ABAP coding into consideration; see later
Resource UtilizationLook for resource competing processes in SM50/SM51
Load BalancingDistribute to different servers to avoid CPU / Memory Bottlenecks on one server and enable greater throughputConfigure ROIDOCPRMSLoad Balancing can be defined:
SAP source system – BW – Connection in SM59InfoPackages, Process ChainsXML Data Loads: HTTP/HTTPS processes can be allocated to specific server groups
SAP AG 2002, BW Performance Tuning, Alex Peter, 94
Data Extraction Performance – 2 –
Data Package SizeData Package size defines the number of DB commits
In resource constrained systems: small package sizeIn large systems: increase size to speed up collection
Set ROIDOCPRMS (which can be overwritten by InfoPackage) according to recommendations in OSS note 417307
Indices on Source TablesIf you decrease the size of data to be extracted by selection criteria in the InfoPackage, consider building indices on the source tables to avoid full table scans
SAP AG 2002, BW Performance Tuning, Alex Peter, 95
Data Extraction Performance – 3 –
Flat File FormatCSV must be internally converted into fixed length format
Use fixed-length ASCII format if you load huge volumes of data via flat filesMoreover, place the file on the application server rather than on the client; avoid large loads across slow networks, from tape or from I/O critical disks
Content vs. Generic ExtractorMost Content extractors support important features like delta capabilityIf a Content extractor meets your business requirements, do not build a generic extractorFor logistics, use V3 extractors rather than the old LIS extractors
SAP AG 2002, BW Performance Tuning, Alex Peter, 96
Data Extraction Performance – 4 –
Use parallel processesData is automatically packaged in the source system; so extraction is parallelLoad into PSA and data targets in parallelStart several InfoPackages
Use selection criteria to split dataSplit flat files
Use 3.x-Process Chains (or Event Chains or InfoPackageGroups
SAP AG 2002, BW Performance Tuning, Alex Peter, 97
Performance Guidelines – Transfer –
SourceSystem
Business InformationWarehouse
PSA
LoadingProcess
Extractor
Extractor
InfoCube
ODS
ALE
ALE
IDOC
ALE
IDOC
ALEScheduler
tRFCUpdaterules
Transferrules
S-API
S-API
Data Load Monitor :Transaction RSMO
Transfer
SAP AG 2002, BW Performance Tuning, Alex Peter, 98
Checklist – Data Load Performance – Transfer –
Technical Content, Data Load MonitorTechnical Content, Data Load Monitor
TransferTransfer
1. Resource Constraint
2. CPU / Memory Bottleneck
3. Network
4. Application Buffer Synchronization
1. Resource Constraint
2. CPU / Memory Bottleneck
3. Network
4. Application Buffer Synchronization
SM50SM50
OS Monitor (ST06)OS Monitor (ST06)
SQL Trace (ST05)SQL Trace (ST05)
Check these points Tools
SAP AG 2002, BW Performance Tuning, Alex Peter, 99
Data Transfer – 1 –
SM50 - ProcessesWith upload method “load PSA and Data Targets in Parallel”, the number of processes in BW is double as much as the extractor jobs in the OLTP system
Data packet new task / commit workData packet
new task / commit workData packet new task / commit workData packet
new task / commit workData packet new task / commit workData packet
new task / commit work
Data packet new task / commit workData packet
new task / commit workData packet new task / commit work
6 Dialog WPs 3 Dialog WPs
BW OLTP
SAP AG 2002, BW Performance Tuning, Alex Peter, 100
Data Transfer – 2 –
Resource ConstraintsMany parallel processes can compete for resourcesCPU / Memory / Network resources
Check ST06
Application Buffer SynchronizationDuring large data loads, swapping of buffers (where e.g. master data resides) can occur on a BW application serverSynchronization overhead of buffers for several application servers (check SQL trace for log table activities)If you run into these sync problems, shut down 2nd
application server and set rdisp/bufrefmode = “sendoff, exeauto” during the load phase
BW OLTP
BW App server 2
BW App server 1
2. Insert DDLOG
3. Select DDLOG
Database
1. Data packet new task / commit work
SAP AG 2002, BW Performance Tuning, Alex Peter, 101
Performance Guidelines – Load Into PSA –
SourceSystem
Business InformationWarehouse
PSA
LoadingProcess
Extractor
Extractor
InfoCube
ODS
ALE
ALE
IDOC
ALE
IDOC
ALEScheduler
tRFCUpdaterules
Transferrules
S-API
S-API
Data Load Monitor :Transaction RSMO
Load Into PSA
SAP AG 2002, BW Performance Tuning, Alex Peter, 102
Checklist – Data Load Performance – PSA Load –
Technical Content, Data Load MonitorTechnical Content, Data Load Monitor
Load into PSALoad into PSA
1. I/O Contention1. I/O Contention OS Monitor (ST06)OS Monitor (ST06)
Check these points Tools
SAP AG 2002, BW Performance Tuning, Alex Peter, 103
Load Into PSA
I / O - ContentionHigh number of DB writes during large data loadsDisk Layout and Striping
What is located on the same disk or tablespace/DB space etc.?
DB StatisticsDB statistics are usually not an issue, as PSA tables/partitions are normally read sequentially
Load MethodIf you select sequential loading from PSA to DataTargets, the total runtime of the load process is composed by the (parallel) loads of requests to PSA and the subsequent loading to DataTargets one by one
Extraction1234
PSA1 2
43
SAP AG 2002, BW Performance Tuning, Alex Peter, 104
Performance Guidelines – Transfer Rules –
SourceSystem
Business InformationWarehouse
PSA
LoadingProcess
Extractor
Extractor
InfoCube
ODS
ALE
ALE
IDOC
ALE
IDOC
ALEScheduler
tRFCUpdaterules
Transferrules
S-API
S-API
Data load Monitor :Transaction RSMO
Transfer Rules
SAP AG 2002, BW Performance Tuning, Alex Peter, 105
Performance Guidelines – Update Rules –
SourceSystem
Business InformationWarehouse
PSA
LoadingProcess
Extractor
Extractor
InfoCube
ODS
ALE
ALE
IDOC
ALE
IDOC
ALEScheduler
tRFCUpdaterules
Transferrules
S-API
S-API
Data load Monitor :Transaction RSMO
Update Rules
SAP AG 2002, BW Performance Tuning, Alex Peter, 106
Checklist – Data Load Performance – Transformation –
Technical Content, Data Load MonitorTechnical Content, Data Load Monitor
Transfer and Update RulesTransfer and Update Rules
1. Transformation Rules / ABAP Coding
2. Transformation Library Formulas
1. Transformation Rules / ABAP Coding
2. Transformation Library Formulas
Transformation Routine Debugger,
ABAP Trace (SE30),
SQL Trace (ST05)
Transformation Routine Debugger,
ABAP Trace (SE30),
SQL Trace (ST05)
ToolsCheck these points
ABAP Trace (SE30)ABAP Trace (SE30)
SAP AG 2002, BW Performance Tuning, Alex Peter, 107
Transformation Rules – 1 –
Identify expensive transfer/update rulesUse ABAP trace (SE30) or SQL trace (ST05) to pinpoint sub-optimal code and/or expensive DB statementsUse Debugging facilities within the Data Load Monitor
Simple tool for debugging of transfer or update rulesImproves error search and analysis – together with the enhanced error messagesDebug from one update rule to the next one for each InfoObject
SAP AG 2002, BW Performance Tuning, Alex Peter, 108
Transformation Rules – 2 –
Transformation Rules / ABAP CodingCheck recommendations at the end of the workshop (e.g. start routine)
Transformation Rules / Transformation LibraryBW 3.x provides powerful transformation rules that can be applied without any ABAP knowledge
Library transformations are interpreted at runtimeIf you notice interpreter overhead in transaction SE30, switch complicated rules to ABAP coding again
SAP AG 2002, BW Performance Tuning, Alex Peter, 109
Performance Guidelines – DataTargets –
SourceSystem
Business InformationWarehouse
PSA
LoadingProcess
Extractor
Extractor
InfoCube
ODS
ALE
ALE
IDOC
ALE
IDOC
ALEScheduler
tRFCUpdaterules
Transferrules
S-API
S-API
Load Into Data Targets
Data Load Monitor - RSMO
SAP AG 2002, BW Performance Tuning, Alex Peter, 110
Checklist – Data Load Performance – Master Data –
Technical Content, Data Load MonitorTechnical Content, Data Load Monitor
Data TargetsData Targets
Master Data
1. Buffering Number Ranges
2. Change Run
1. Buffering Number Ranges
2. Change Run
SQL Trace (ST05)SQL Trace (ST05)
ToolsCheck these points
Aggregate HierarchyAggregate Hierarchy
Master Data
SAP AG 2002, BW Performance Tuning, Alex Peter, 111
Loading Master Data – 1 –
Number RangesSID number range can be buffered instead of accessing the DB for each SIDIf you encounter massive accesses to DB table NRIV via SQL trace (ST05), increase the number range buffer in transaction SNROIf possible, reset the value to its original state after the load (to avoid unnecessary memory allocation)
SID Material Number1 47112 08153 4712
SID Buffer45678
New material: 1125Draw new SID from Buffer
SID
SAP AG 2002, BW Performance Tuning, Alex Peter, 112
Loading Master Data – 2 –
Change RunWhen master data changes, the changes of the navigational attributes/hierarchies must be applied to the depending aggregates; this process is called change run
Newly loaded master data is not active until the change run has been applied the changes to all aggregates
See more information in the “Load into InfoCube” chapter
SAP AG 2002, BW Performance Tuning, Alex Peter, 113
Checklist – Data Load Performance – InfoCubes –
Technical Content, Data Load MonitorTechnical Content, Data Load Monitor
Data TargetsData Targets
InfoCubesInfoCubes
1. Roll-up
2. Change Run
3. Compression
4. Indices
5. Load Master Data before Transaction Data
6. Buffering Number Ranges
1. Roll-up
2. Change Run
3. Compression
4. Indices
5. Load Master Data before Transaction Data
6. Buffering Number Ranges
Aggregate HierarchyAggregate Hierarchy
ToolsCheck these points
SQL Trace (ST05)SQL Trace (ST05)
SQL Trace (ST05)
InfoCube Maintenance
SQL Trace (ST05)
InfoCube Maintenance
SAP AG 2002, BW Performance Tuning, Alex Peter, 114
Roll-Up Performance – 1 –
Roll-upThe roll-up process populates all aggregates of an InfoCube with the newly loaded delta loadBasic Rule: InfoCube and all depending aggregates must be in-sync, i.e. you must see the same data no matter if it has been derived from the InfoCube or the aggregateNewly loaded data is not available for reporting until it has been rolled up into the aggregates
Aggregate HierarchyAggregates can be built out of other aggregates to reduce the amount of data to be read and, hence, to improve the roll-up performanceAggregate hierarchy is determined automaticallyGeneral Guideline: define few basis (large) aggregates and many small aggregates that can be built from the hierarchy level before
Display aggregate hierarchy
SAP AG 2002, BW Performance Tuning, Alex Peter, 115
Roll-Up Performance – 1 – Aggregate Hierarchy
Example: Optimized Aggregate Hierarchy
Basic InfoCube
{Material, Material Group, Customer, Day}
Example
Few largebasic aggregates
{Material Group, Customer, Day}
Many small aggregates
{Material Group, Month}
SAP AG 2002, BW Performance Tuning, Alex Peter, 116
Why Can Too Many Aggregates Be Harmful?
Faster QueriesTime for Rollup andChange Run
Balanced aggregates strategy
SAP AG 2002, BW Performance Tuning, Alex Peter, 117
Aggregate Design
Tips for Maintaining Good AggregatesRelatively small compared to parent InfoCubeTry for summarization ratios of 10 or higherFind good subsets of data (frequently accessed)Build on some hierarchy levels, not allNot too specific, not too general – should serve many different query navigationsShould be frequently used and used recently (except basis aggregates)
Avoid Aggregateso That are too big (compared to the parent InfoCube/Aggregate)o That are similar to too many other aggregateso That are old and that are not frequently usedo That are built for a specific query (rather build general ones)
SAP AG 2002, BW Performance Tuning, Alex Peter, 118
Change Run Performance – 1 –
Change RunAggregates can contain
Dimension CharacteristicsNavigational AttributesHierarchy Levels
When master data changes, the changes of the navigational attributes/hierarchies must be applied to the depending aggregates; this process is called change runNewly loaded master data is not active until the change run has been applied the changes to all aggregatesThreshold for delta and new build-up in customizingThe Change Run can be parallelized across InfoCubes; see SAP note 534630 for more detailsCheck aggregate hierarchy (see Roll-Up for more details)Try to build basis aggregates that are not affected by the change run, i.e. no navigational attributes nor hierarchy levelsThe following slides show details on the process itself …
SAP AG 2002, BW Performance Tuning, Alex Peter, 119
Change Run – 2 – Before Master Data Activation
Country Customer Sales
USAGermanyUSAAustriaAustriaGermanyUSA
Winsoft Inc.InternetworksFunny Duds Inc.InternetworksThor IndustriesFunny Duds Inc.Winsoft Inc.
1015510102025
Fact Table: Sales DataFact Table: Sales Data Aggregate Tables: Sales DataAggregate Tables: Sales Data
Country SpaceIndustry *
Country SpaceIndustry *
Industry Sales
602510
TechnologyConsumer ProductsChemical
TechnologyConsumer ProductsTechnologyChemicalConsumer Products
Industry
Navigational Attribute for Characteristic CustomerCustomer
Navigational Attribute for Characteristic Customer
Winsoft Inc.Funny Duds Inc.InternetworksThor IndustriesInternetworks
Changed master data not available for reporting
Old:
New:
SAP AG 2002, BW Performance Tuning, Alex Peter, 120
Change Run – 3 – After Master Data Activation
Country Customer Sales
USAGermanyUSAAustriaAustriaGermanyUSA
Winsoft Inc.InternetworksFunny Duds Inc.InternetworksThor IndustriesFunny Duds Inc.Winsoft Inc.
1015510102025
Fact Table: Sales DataFact Table: Sales Data Aggregate Tables: Sales DataAggregate Tables: Sales Data
Country SpaceIndustry *
Country Country SpaceSpaceIndustry Industry **
Industry Sales
355010
TechnologyConsumer ProductsChemical
TechnologyConsumer ProductsTechnologyChemicalConsumer Products
Industry
Navigational Attribute for Characteristic CustomerCustomer
Navigational Attribute for Characteristic Customer
Winsoft Inc.Funny Duds Inc.InternetworksThor IndustriesInternetworksNew:
Changed master data now available for reporting
Old:
SAP AG 2002, BW Performance Tuning, Alex Peter, 121
Indices – 1 –
IndicesIndices are necessary for reporting as they enable fast retrieving of records from DB tablesMaintaining index build during load time is expensive, thoughIf the (uncompressed) F fact table is small, it is usually faster to drop the secondary (bitmap) indices before the load and build them up after the loadThis feature is available in InfoCube maintenance and as Process Type
SAP AG 2002, BW Performance Tuning, Alex Peter, 122
Data Load Performance Into InfoCubes – Others
Load Master Data BEFORE transaction dataMaster Data Load generates SIDs and populates master data tablesAvoids expensive operations (SID creation) during transaction data
Number RangesDimension ID number range can be buffered instead of accessing the DB for each Dimension IDIf you encounter massive accesses to DB table NRIV via SQL trace (ST05), increase the number range bufferIf possible, reset the value to its original state after the load (to avoid unnecessary memory allocation)
SAP AG 2002, BW Performance Tuning, Alex Peter, 123
Checklist – Data Load Performance – ODS Objects –
Technical Content, Data Load MonitorTechnical Content, Data Load Monitor
Data TargetsData Targets
1. Parallel ODS activation
2. Unique Data Records
3. Flag BEx Reporting
4. Indices
1. Parallel ODS activation
2. Unique Data Records
3. Flag BEx Reporting
4. Indices
Process Monitor (SM50 / SM51)Process Monitor (SM50 / SM51)
ToolsCheck these points
ODS Object MaintenanceODS Object Maintenance
SQL Trace (ST05)SQL Trace (ST05)
ODS ObjectsODS Objects
SAP AG 2002, BW Performance Tuning, Alex Peter, 124
ODS Objects – Data Activation – 1 –
ODS ObjectsIn BW 3.x, data can be loaded and activated in parallelUse Parallel Upload and Activation in BW 3.x
Number of parallel processes
IMG (or RSCUSTA2)
Load Balancing to Server Groups
SAP AG 2002, BW Performance Tuning, Alex Peter, 125
ODS Objects – Data Activation in BW 2.x – 2 –
Staging EngineStaging Engine
Data Packets / Requests can not be loaded into an ODS object in parallel
overwriting functionalityLocking on Activation table
active data change log
New/modifieddata
Req1
Req2Req3
Activation
Doc-No.
Doc-No.
Sequential load
Req 3,1,2
SAP AG 2002, BW Performance Tuning, Alex Peter, 126
ODS Objects – Data Activation in BW 3.x – 3 –
New queuing mechanism replacing previous Maintenance (M)table
active data change log
New/modifieddata
Req1
Req2Req3
Activation Activation
Req2Req3
Req1
Activation queue
Req.ID I Pack.ID I Rec.No
Req.ID I Pack.ID I Rec.NoDoc-No.
Doc-No.
Parallel load
Staging EngineStaging Engine
SAP AG 2002, BW Performance Tuning, Alex Peter, 127
ODS Objects – Data Activation in BW 3.x – 4 –
Staging EngineStaging Engine
Active data
Activation queue
Req.ID I Pack.ID I Rec.No
Change log
Doc.No I ValueODSRx I P 1 I Rec.1I4711I 104711 I 10
Activation
ActivationDuring activation the data is sorted by the logical key of active data plus change log key. This guarantees the correct sequence of the records and allows inserts instead of table locks.
Before- and After ImageRequest ID in activation queue and change log differ from each other.After update, the data in the activation queue is deleted.
ODSRy I P 1 I Rec.1I4711I-10ODSRy I P 1 I Rec.2I4711I+30
4711 I 30
Req2Req3
Req1
REQU1 I P 1 I Rec.1I4711I 10REQU2 I P 1 I Rec.1I4711I 30
Upload to Activation queueData from different requests are uploaded in parallel to the activation queue
SAP AG 2002, BW Performance Tuning, Alex Peter, 128
ODS Objects – Data Load
Loading Unique KeysThe “Unique”-Flag accelerates data activation significantly
It directly inserts data to the active data table instead of checking if the key is already there (mass inserts)No sortingNo Before Image in change log
Only usable for documents that cannot be changed, e.g. sales documentsSet the “Unique”-Flag if you are sure that all your records are unique
Activation Queue
Doc.No. Custom Sales4711 Miller 104712 Smith 204713 Miller 254713 Miller -15Not unique
Activation Queue
Doc.No. Custom Sales4711 Miller 104712 Smith 204713 Miller 254714 Smith 21
Unique!
SAP AG 2002, BW Performance Tuning, Alex Peter, 129
ODS Objects – Reporting
ODS Object
InfoSet
InfoSet on top of ODS ObjectThe BEx Reporting flag forces
the system to draw SIDs for the master data; although they are determined in parallel, this slows down activation performance.
In BW 3.x, SIDs are drawn during activation, in BW 2.x during data load.
Do not use “BEx Reporting” flag, if not necessary
InfoSets in BW 3.x provide also fast selective access to individual records, if indices are available
Try to use InfoSet on top of ODS objects instead of BEx-enabled ODS objects
Reporting possible via InfoSet
BEx Reporting flag disabled!
SAP AG 2002, BW Performance Tuning, Alex Peter, 130
Performance Guidelines
Analysis Methods
General Settings and Checklist
Modeling
Query Performance
Extraction / Data Load Performance
SAP AG 2002, BW Performance Tuning, Alex Peter, 131
Analysis and Repair of BW Objects (RSRV) – 1 –
Analyze DB parameters and specific InfoCubesDB ParametersIndices & DB Statistics of InfoCubes (and Aggregates)
SAP AG 2002, BW Performance Tuning, Alex Peter, 132
Analysis and Repair of BW Objects (RSRV) – 2 –
Analyze DB parameters and specific InfoCubesDatabase information about the star schema tables of an InfoCubeCheck dimension sizes in relationship to the fact table size
SAP AG 2002, BW Performance Tuning, Alex Peter, 133
Query Monitor (RSRT) – 1 –
Analyze specific queries
Debug Option
SAP AG 2002, BW Performance Tuning, Alex Peter, 134
Query Monitor (RSRT) – 2 –
Analyze specific queriesCheck if the right aggregate(s) is(are) used by the query
Number of Database Access
Structure to be read
Suggested Aggregates
AvailableAggregates
* disaggregatedspace aggregated
F fixed valueH hierarchy level aggregation
SAP AG 2002, BW Performance Tuning, Alex Peter, 135
Query Trace (RSRTRACE)
Analyze specific queriesCheck all drill-downs and navigation steps
Record Trace
Replay trace with transactionRSRCATTTRACE
SAP AG 2002, BW Performance Tuning, Alex Peter, 136
Performance Analysis: SQL Trace (ST05) – 1 –
Analyze high database runtimeTrace specific process with SQL Trace
Activate SQL Trace Flag
Activate Trace
Execute process
Deactivate TraceAnalyze Trace
SAP AG 2002, BW Performance Tuning, Alex Peter, 137
Performance Analysis: SQL Trace (ST05) – 2 –
Total Num
ber of „statement-identical“ statem
ents
Percentage of „value-identical“ statements
Total Execution Time in m
icrosec (Prep, Open,
Fetch, Close)
Total number of row
s
Table name or D
B procedure
Explain
SAP AG 2002, BW Performance Tuning, Alex Peter, 138
Performance Analysis: SQL Trace – Explain
1. Find all bitmap streams for package restriction
3. Calculate “AND” of bitmap streams
2. Find all bitmap streams for fiscal periods
4. Access fact table partition by partition through local index
5. Join other requested dimensions to the result via hash join
SAP AG 2002, BW Performance Tuning, Alex Peter, 139
SQL Explain
Basic Help to Read the Execution PlanRead from inside out of the hierarchy
Read a set of operations at the same level from top to bottomImportant node names
Hash JoinSmaller Table is used to build up hash table on the join key (optimal: in memory)Then scanning the larger table with hash table
Nested LoopFor each relevant row in the first table, find all matching rows in the other table
Bitmap Merge (=Or) / Bitmap AndPartition Range Iterator
SAP AG 2002, BW Performance Tuning, Alex Peter, 140
ABAP Runtime Analysis (SE30) – 1 –
Analyze high ABAP RuntimeParticularly useful for Customer Exits
Trace transaction, program or function module ......
... or enable tracing in parallel session
Analyze results
SAP AG 2002, BW Performance Tuning, Alex Peter, 141
ABAP Runtime Analysis (SE30) – 2 –
Num
ber of Calls
SAP AG 2002, BW Performance Tuning, Alex Peter, 142
ABAP Runtime Analysis (SE30) – 3 –
Runtime of function modules:Gross Time
Includes times for sub-function callsNet Time
Excludes times for sub-function calls
FM1
FM2
FM3
SAP AG 2002, BW Performance Tuning, Alex Peter, 143
ABAP Tips – 1 –
Avoid unnecessary loops
Delete from itab where field1 = condition.
Loop at itab.If itab-field1 =
condition.Delete itab.
Endif.Endloop.
goodgood badbad
SAP AG 2002, BW Performance Tuning, Alex Peter, 144
ABAP Tips – 2 –
Avoid unnecessary selectsUse DB features instead of programming individual logic
SELECT SUM(amount) INTO Sum FROM tab WHERE year = ‘2002’.
Sum = 0.SELECT amount INTO
Add FROM tab WHERE year = ‘2002’.
Sum = Sum + Add.ENDSELECT.
goodgood badbad
SAP AG 2002, BW Performance Tuning, Alex Peter, 145
ABAP Tips – 3 –
Avoid unnecessary selectsTry to reuse data
SELECT f1,f2,f3 FROM table WHERE cond
SELECT f1, f2 FROM table WHERE cond
SELECT f1, f3 FROM table WHERE cond
goodgood badbad
SAP AG 2002, BW Performance Tuning, Alex Peter, 146
ABAP Tips – 4 –
Avoid unnecessary selectsAvoid nested SELECTsTip: If you discover several full table scans with “FOR ALL ENTRIES” in Basis 6.20, change rsdb/max_in_blocking_factor from –1 to 500.
SELECT * FROM knb1 FOR ALL ENTRIES IN I_kun WHEREkunnr = I_kun-kunnr.* do something
ENDSELECT.
LOOP AT I_kunSELECT * FROM knb1
WHERE kunnr = I_kun-kunnr.
* do somethingENDSELECT.
ENDLOOP.
goodgood badbad
SAP AG 2002, BW Performance Tuning, Alex Peter, 147
ABAP Tips – 5 –
Buffering DB tablesUse Start Routine in Transformation Rules to build up internal tables and use them in the rule definitionUse Array operations instead of SINGLE accessesDon’t forget to refresh the tables when necessary
SELECT * FROM knb1 INTO itab FOR ALL ENTRIES IN I_kun WHERE kunnr = I_kun-kunnr.
LOOP at itab.* do something
ENDLOOP.
LOOP AT I_kunSELECT * FROM knb1
WHERE kunnr = I_kun-kunnr.
* do somethingENDSELECT.
ENDLOOP.
goodgood badbad
SAP AG 2002, BW Performance Tuning, Alex Peter, 148
ABAP Tips – 6 –
Accessing internal tablesSort internal tables and use binary search on it ORHashed Tables
Accessing DB TablesFully buffered DB tables are transferred completely to the application buffer when accesses the first time (see SE11-settings of table)Specify whole key in WHERE clause, if possible – or specify key which is included in one of the secondary indicesTry to avoid “NOT” in WHERE clause, as there is no index support
SAP AG 2002, BW Performance Tuning, Alex Peter, 149
Database Performance Analysis (ST04) – 1 –
Analyze high database runtimeScreen and features differ for each DBMS
Data Buffer Size & Quality, buffer quality should be > 95%
Detail Menu
SAP AG 2002, BW Performance Tuning, Alex Peter, 150
Database Performance Analysis (ST04) – 2 –
Analyze high database runtimeScreen and features differ for each DBMS
Displays all DB processes
Check SQL statements from history
Check Table Buffers
Locks held by another process
SAP AG 2002, BW Performance Tuning, Alex Peter, 151
Database Tables and Index Monitor (DB02)
Analyze high database runtimeScreen and features differ for each DBMS
Refresh statistics for up-to-date information on indices and tables
Compare table/index size
Check missing indices
SAP AG 2002, BW Performance Tuning, Alex Peter, 152
Buffer / Memory Monitor (ST02) – 1 –
Analyze high database runtimeBuffers not used ?Buffer replacement expensive? Check free space
and free entriesHitratio > 95% Swaps?
Table Buffer
Exp/Imp SHM OLAP Cache
SAP AG 2002, BW Performance Tuning, Alex Peter, 153
Buffer / Memory Monitor (ST02) – 2 –
Buffers in BWGeneric Key Table Buffer
Most BW-specific control tablesRS*-Tables
Single Record BufferMaster Data Tables
Export/Import Shared MemoryOLAP Query Cache
SAP AG 2002, BW Performance Tuning, Alex Peter, 154
Operating System Monitor (ST06) – 1 –
Analyze high system runtime
LAN throughput (network problems)
Disk Utilization(I/O Bottleneck)
Memory Swapping (page out)
CPU Utilization
Detailed Analysis
SAP AG 2002, BW Performance Tuning, Alex Peter, 155
Operating System Monitor (ST06) – 2 –
Analyze high system runtimeDetail Analysis
snapshot analysis
24h analysis
SAP AG 2002, BW Performance Tuning, Alex Peter, 156
Web Query Analysis
Typical questions for (single) Web queries:
How do I measure the total response time of a Web Query?How do I analyze where most of the query executiontime is spent in BW?Is the time required for rendering the Web page on the front-end significant?
Typical questions for Portal pages:
How are multiple URLs processed on the time line, i.e. which URL is called at what time, and how long does ittake to come back?Which URLs dominate the overall response time of a Portal page?
SAP AG 2002, BW Performance Tuning, Alex Peter, 157
Web Query Analysis – Overview
2 different measuring points:
Web Browser
WAS / BW / DB
HTTPrequest
HTMLresponse
ICMOLAPINIT
DB
OLAP
Rendering
OLAP Cache
Monitoring of WAS / BW / DB internal activities
HTTP M
onitor
Frontend monitoringNetwork traffic monitoring
IEMonitor
„frontend“
„backend“
SAP AG 2002, BW Performance Tuning, Alex Peter, 158
Web Query Analysis – Frontend Monitoring
Key Performance Indicators for Frontend Monitoring:Query response time
Time to load HTML pageFrontend rendering time
Transferred data volumeNumber of round-trips between frontend and server
IEMon.exeSimple Web browser replacement with protocol logging facilitiesUsed to analyze load and frontend rendering time of an HTML pageEasy deployment
httpmon.exeSet up as a proxy serverRequires changes of IE settingsRequires environment variables to be set for configuration
SAP AG 2002, BW Performance Tuning, Alex Peter, 159
IEMon.exe – Usage
Simple Web browser replacement
No installation process, just copy and execute
Uses IE control of the Internet Explorer installed on workstation
Path to log file
View log file
View log as it is produced
Navigate button
Stop navigation
Address bar
Status line
SAP AG 2002, BW Performance Tuning, Alex Peter, 160
IEMon.exe – Logfile
Load time
Frontendrenderingtime
Millisecond time stamps
Status information
SAP AG 2002, BW Performance Tuning, Alex Peter, 161
IEMon.exe – Portal Page Example
IEMon log files can become very complex and lengthyHard to find matching events for load and render time
SAP AG 2002, BW Performance Tuning, Alex Peter, 162
IEMon.exe – Log File Analysis
IEMon log file contains information for lots of different events
BUT: only 3 types of events required to determine load and rendering time:
Before Navigate eventNavigate Complete eventDocument Complete event
Sequence of events in log file depends on structure of HTML code
Complex log files: analysis tool for information extraction
analyze_iemon.exetakes path to IEMon log file as command line parameterproduces comma separated list of values for start time, load time, rendering time, total response time, and URLOutput can be redirected into .csv file and conveniently be opened with MS Excel
SAP AG 2002, BW Performance Tuning, Alex Peter, 163
IEMon.exe – Log File Analysis: Output
SAP AG 2002, BW Performance Tuning, Alex Peter, 164
IEMon.exe – Log File Analysis in Excel
Output produced by
analyze_iemon
Graphical representation
with a few clicks
SAP AG 2002, BW Performance Tuning, Alex Peter, 165
httpmon.exe – Setup
Set up as a proxy server
Configuration by environment variables:HTTP_PROXY: proxy_server:proxy_port (e.g. pwdf0754:1080)NO_PROXY: list of web server domains which do not need proxy (e.g. *.sap.corp;*.sap-ag.de)
httpmon listens on port 8000 by defaultchange Internet Explorer settings
Web Browser Proxy server
proxy_port
httpmon
port 8000
otherWeb servers
(no prxy required)
SAP AG 2002, BW Performance Tuning, Alex Peter, 166
httpmon.exe – UsageBe sure that you delete the ICM Server Cache (TA SMICM Goto HTTP
Server Cache Invalidate Global) to get the compressed data volume!
BW server pwdf0754
port 1080
Path to log file
httpmonlisten port
httpmontarget proxy
and port
Transferred data volume
(compressed!)Set data volume
counters to 0
SAP AG 2002, BW Performance Tuning, Alex Peter, 167
Httpmon.exe – Use Case
First execution: download of MIME
objects
Second execution: MIME objects were read from
browser cache
SAP AG 2002, BW Performance Tuning, Alex Peter, 168
Frontend Monitoring – Summary
3 important tools for frontend monitoring of Web pages:IEmon.exe:Load timeFrontend rendering timeanalyze_iemon.exeProduce .csv file from IEMon log file to view results in Excelhttpmon.exe:Transferred data volumesNumber of round-trips between frontend and server
These tools are available to customers (without support) and can be downloaded from SAP Service Marketplace, alias /bw, section Performance
Recommendation: Do not trace more than one navigation step at a time. For multiple navigation steps use individual log files!
Open IEMon log file, use Save as ... to store in a different fileDelete contents of log file in editor, save back to original file
SAP AG 2002, BW Performance Tuning, Alex Peter, 169
Backend Monitoring
Open question: How to analyze query execution in BW backend?
Query analysis: trace of ABAP function calls (transaction SE30)
First solution: call SE30 in transaction RSRTDisplay result as HTMLBUT: MIME handling and other things are different from a „real“ http service connection
Second solution: switch on ABAP trace for each incoming http request
Configuration of http services (transaction SICF)Can trace only those service requests coming from a specific IP address no impact on other concurrently executing requests
SAP AG 2002, BW Performance Tuning, Alex Peter, 170
Backend Monitoring – Configuration in SICF
Restrict tracing on requests handled by
this service
Restrict tracing on requests coming from SAPGUI IP
address only
Idle time limit
SAP AG 2002, BW Performance Tuning, Alex Peter, 171
Backend Monitoring – Tracing a Query
After tracing has been activated in SICF:Start Web browser (Internet Explorer, IEMon, ...)IMPORTANT: the browser must be started AFTER switching on the trace. Otherwise no trace files will be written!Execute queryUse transaction SE30 to analyze ABAP trace
SAP AG 2002, BW Performance Tuning, Alex Peter, 172
Backend Monitoring – ABAP Trace Analysis
Sort descending by column „Net“ to
identify most time consuming function
modules
SAP AG 2002, BW Performance Tuning, Alex Peter, 173
Backend Monitoring – Summary
Use transaction SICF to switch on ABAP traces for incoming http service requests
Use transaction SE30 to analyze ABAP traces
Note:ABAP traces can have substantial impact on query response times!BUT: times reported in ABAP trace do not contain SE30 execution times
… don‘t forget to switch off ABAP tracing when you‘re finished ...
SAP AG 2002, BW Performance Tuning, Alex Peter, 174
Information on Performance
SAP Service Marketplace alias ‘BW’ PerformanceSAP Training BW360: Performance & AdministrationCollective performance notes BW 3.x
Note 567745: DB-specific SettingsNote 567746: Query & Web PerformanceNote 567747: Extraction & Loading