basis week4 (1)

SAP AG 1999

TABC20 Technical Consultant Training Week 4:ORACLETechnical Consultant TrainingORACLETechnical Consultant TrainingORACLE

TABC20 R/3 Release 4.6B 50039592

TABC20 R/3 Release 4.6B 50039592

Week 4Week 4

Oct-17-2000

SAP AG 1999

Copyright 2000 SAP AG. All rights reserved.

Neither this training manual nor any part thereof maybe copied or reproduced in any form or by any means,or translated into another language, without the priorconsent of SAP AG. The information contained in thisdocument is subject to change and supplement without priornotice.

All rights reserved.

Copyright

n Trademarks: n Microsoft ®, Windows ®, NT ®, PowerPoint ®, WinWord ®, Excel ®, Project ®, SQL-Server ®, Multimedia

Viewer ®, Video for Windows ®, Internet Explorer ®, NetShow ®, and HTML Help ® are registered trademarks of Microsoft Corporation.

n Lotus ScreenCam ® is a registered trademark of Lotus Development Corporation.

n Vivo ® and VivoActive ® are registered trademarks of RealNetworks, Inc. n ARIS Toolset ® is a registered Trademark of IDS Prof. Scheer GmbH, Saarbrücken

n Adobe ® and Acrobat ® are registered trademarks of Adobe Systems Inc.

n TouchSend Index ® is a registered trademark of TouchSend Corporation. n Visio ® is a registered trademark of Visio Corporation.

n IBM ®, OS/2 ®, DB2/6000 ® and AIX ® are a registered trademark of IBM Corporation.

n Indeo ® is a registered trademark of Intel Corporation. n Netscape Navigator ®, and Netscape Communicator ® are registered trademarks of Netscape Communications,

Inc. n OSF/Motif ® is a registered trademark of Open Software Foundation.

n ORACLE ® is a registered trademark of ORACLE Corporation, California, USA. n INFORMIX ®-OnLine for SAP is a registered trademark of Informix Software Incorporated.

n UNIX ® and X/Open ® are registered trademarks of SCO Santa Cruz Operation.

n ADABAS ® is a registered trademark of Software AG n The following are trademarks or registered trademarks of SAP AG; ABAP™, InterSAP, RIVA, R/2, R/3, R/3

Retail, SAP (Word), SAPaccess, SAPfile, SAPfind, SAPmail, SAPoffice, SAPscript, SAPtime, SAPtronic, SAP-EDI, SAP EarlyWatch, SAP ArchiveLink, SAP Business Workflow, and ALE/WEB. The SAP logo and all other SAP products, services, logos, or brand names included herein are also trademarks or registered trademarks of SAP AG.

n Other products, services, logos, or brand names included herein are trademarks or registered trademarks of their respective owners.

Contents

Course Overview......................................................................................................................................................................1 Target Group ........................................................................................................................................................................3 Course Prerequisites............................................................................................................................................................4 Course Goals ........................................................................................................................................................................5 Course Composition............................................................................................................................................................6 TABCUO - Sections ...........................................................................................................................................................7 TABCNS - Sections............................................................................................................................................................8 TABCNO - Sections ...........................................................................................................................................................9

Section: Database Administration - ORACLE .................................................................................................................11 Database Overview ...........................................................................................................................................................13

Database Overview.......................................................................................................................................................15 Review: Oracle Overview............................................................................................................................................16 Security: Operating System and Database Users.....................................................................................................17 Security: SAPR3 Password .........................................................................................................................................18 NET8 Basics ..................................................................................................................................................................19 Database Administration Tools ..................................................................................................................................20 Database Performance Monitoring (ST04)...............................................................................................................21 Database Space Monitoring (DB02) ..........................................................................................................................22 DBA Operations Monitor (DB24)..............................................................................................................................23 DBA Alert Monitor (RZ20) ........................................................................................................................................24 SAPDBA (OS Level) ...................................................................................................................................................25 Oracle - Database Architecture...................................................................................................................................26 Oracle - Starting and Stopping the Database............................................................................................................27 Oracle - Writing Data and Log Files..........................................................................................................................28 Oracle - Storage Management Concepts ...................................................................................................................29 Oracle - R/3 Naming Conventions.............................................................................................................................30 Oracle - Oracle Directory Structure in R/3...............................................................................................................31 Oracle - Oracle Directories/Environment Variables ...............................................................................................32 Oracle - Database Roles and Users ............................................................................................................................33 Oracle - Net8 Listener..................................................................................................................................................34 Oracle - Alert Monitoring Tree...................................................................................................................................35 Unit Summary ................................................................................................................................................................36 Unit Actions...................................................................................................................................................................37 Database Overview: Exercises....................................................................................................................................39 Database Overview: Solutions....................................................................................................................................41

Backup Strategy and Tape Management.......................................................................................................................43 Backup Strategy and Tape Management...................................................................................................................45 The Importance of Database Backups.......................................................................................................................46 Preventing and Handling Errors .................................................................................................................................47 Possible Causes of Data Loss (1) ...............................................................................................................................48 Possible Causes of Data Loss (2) ...............................................................................................................................49 Possible Causes of Data Loss (3) ...............................................................................................................................50 Backup Cycle Recommendations...............................................................................................................................51 SAP Database Backup Tools .......................................................................................................................................52 Backup Objects..............................................................................................................................................................53 Tape Pools ......................................................................................................................................................................54 Tape Initialization .........................................................................................................................................................55 Tape Label Contents and Tape Checks .....................................................................................................................56 Tape Locking .................................................................................................................................................................57 Scenario 1: Automatic Tape Selection ......................................................................................................................58 Scenario 2: Manual Tape Selection............................................................................................................................59 Scenario 3: Tape Selection by an External Tool......................................................................................................60 Tape Layout ...................................................................................................................................................................61 Unit Summary ................................................................................................................................................................62 Unit Actions...................................................................................................................................................................63 Backup Strategy and Tape Management: Exercises ...............................................................................................65 Backup Strategy and Tape Management: Solutions ...............................................................................................67

Scheduling, Performing, and Monitoring Backups .....................................................................................................69 Scheduling, Performing, and Monitoring Backups.................................................................................................71

How SAP Backup Tools Work Together..................................................................................................................72 Backup Profile Parameters ..........................................................................................................................................73 Profile init<SID>.sap Parameter tape_size ...............................................................................................................74 Hardware Compression................................................................................................................................................75 Scheduling and Performing a Normal Database Backup .......................................................................................76 Phases of a Whole Database Backup.........................................................................................................................77 Logical Verification of a Database Backup..............................................................................................................78 Physical Verification of Offline Database Backups................................................................................................79 Monitoring a Database Backup...................................................................................................................................80 Offline Redo Log Files: Status and Option -cds......................................................................................................81 Performing Offline Redo Log File Backups.............................................................................................................82 Monitoring Offline Redo Log File Backups.............................................................................................................83 Log File Cleanup...........................................................................................................................................................84 Freespace Problems in Directory saparch.................................................................................................................85 Unit Summary ................................................................................................................................................................86 Further Documentation ................................................................................................................................................87 Unit Actions...................................................................................................................................................................89 Performing Backups: Exercises..................................................................................................................................91 Performing Backups: Solutions..................................................................................................................................93

Restore and Recovery .......................................................................................................................................................97 Restore and Recovery...................................................................................................................................................99 Database Errors .......................................................................................................................................................... 100 Scenarios: Introduction ............................................................................................................................................. 101 Scenario: Partial Restore and Complete Recovery............................................................................................... 102 Scenario: Database Reset.......................................................................................................................................... 103 Scenario: Point in Time Recovery .......................................................................................................................... 104 How to Handle Problems .......................................................................................................................................... 105 Partial Restore and Complete Recovery (1) .......................................................................................................... 106 Partial Restore and Complete Recovery (2) .......................................................................................................... 107 Partial Restore and Complete Recovery Limitations........................................................................................... 108 Database Reset Using a Full Offline Backup........................................................................................................ 109 Database Reset Using a Consistent Online Backup ............................................................................................. 110 Full Restore and Recovery (1)................................................................................................................................. 111 Full Restore and Recovery (2) ................................................................................................................................. 112 Unit Summary ............................................................................................................................................................. 113 Further Documentation ............................................................................................................................................. 114 Unit Actions................................................................................................................................................................ 115 Restore and Recovery: Exercises ............................................................................................................................ 117 Restore and Recovery: Solutions............................................................................................................................. 119

Backup Strategies Using RMAN................................................................................................................................. 121 Backup Strategies Using RMAN............................................................................................................................. 123 Full Backup (Level 0) with RMAN and SAP Tools (1)...................................................................................... 124 Full Backup (Level 0) with RMAN and SAP Tools (2)...................................................................................... 125 Savesets........................................................................................................................................................................ 126 Preparation Run.......................................................................................................................................................... 127 Incremental (Level 1) Backup.................................................................................................................................. 128 Level 1 Backup: Important Considerations (1) ..................................................................................................... 129 Level 1 Backup: Important Considerations (2) ..................................................................................................... 130 Recovery Using Incremental Backup with sapdba............................................................................................... 131 Unit Summary ............................................................................................................................................................. 132 Further Documentation ............................................................................................................................................. 133 Unit Actions................................................................................................................................................................ 135 Backup Strategies Using RMAN: Exercises ......................................................................................................... 137 Backup Strategies Using RMAN: Solutions ......................................................................................................... 139

Advanced Backup Techniques..................................................................................................................................... 143 Advanced Backup Techniques................................................................................................................................. 145 Backup Requirements and Costs ............................................................................................................................. 146 BRBACKUP and BRARCHIVE: One-Run Strategy .......................................................................................... 147 Consistent Online Backups....................................................................................................................................... 148 Parallel Tape Support ................................................................................................................................................ 149 Partial Database Backups.......................................................................................................................................... 150 Backing Up Data Tablespaces Only ....................................................................................................................... 151 Two-Step Disk Backup ............................................................................................................................................. 152

Structure-Retaining Database Copy........................................................................................................................ 153 Split Mirror Disk Backups........................................................................................................................................ 154 SAP Tools and the Oracle Standby Database....................................................................................................... 156 External Backup Tools Using BC-BRI .................................................................................................................. 157 Unit Summary ............................................................................................................................................................. 158 Further Documentation ............................................................................................................................................. 159 Unit Actions................................................................................................................................................................ 161 Advanced Backup Techniques: Exercises ............................................................................................................. 163 Advanced Backup Techniques: Solutions.............................................................................................................. 165

Storage Management ..................................................................................................................................................... 167 Storage Management................................................................................................................................................. 169 Space Management: Review.................................................................................................................................... 170 Space Management: Fragmentation Types............................................................................................................ 171 Daily Monitoring: sapdba -check............................................................................................................................ 172 Configuring sapdba -check....................................................................................................................................... 173 Tablespace Extension................................................................................................................................................ 174 Storage Categories of SAP Database Objects ....................................................................................................... 175 Using sapdba -next ..................................................................................................................................................... 176 Daily Monitoring: Tables and Indexes ................................................................................................................... 177 Tables and Indexes: Important Reports.................................................................................................................. 178 Analyzing Internal Fragmentation .......................................................................................................................... 179 Reorganization: Basics .............................................................................................................................................. 180 Reorganization: Reasons........................................................................................................................................... 181 Reorganization: Phases and Types .......................................................................................................................... 182 Reorganization: Methods.......................................................................................................................................... 183 Reorganization: Options ........................................................................................................................................... 184 Unit Summary ............................................................................................................................................................. 185 Unit Actions................................................................................................................................................................ 187 Storage Management: Exercises.............................................................................................................................. 189 Storage Management: Solutions.............................................................................................................................. 191

Top 10 Problems ............................................................................................................................................................. 193 Top 10 Problems ........................................................................................................................................................ 195 Troubleshooting Steps............................................................................................................................................... 196 Top 10 Problems ........................................................................................................................................................ 197 Archiver Stuck Situation........................................................................................................................................... 198 Avoiding an Archiver Stuck Situation.................................................................................................................... 199 Incorrect Tape Size (Hardware Comp. Tape Drives)........................................................................................... 200 Missing......................................................................................................................................................................... 201 Tablespace Overflow................................................................................................................................................. 202 MaxExtents Limit is Reached.................................................................................................................................. 203 ORA-1555: Snapshot Too Old................................................................................................................................. 204 Net8 TCP/IP Delay .................................................................................................................................................... 205 ORA-1578: Data Block Corruption ........................................................................................................................ 206 ORA-600: Internal Database Error ......................................................................................................................... 207 Influence of the Cost-Based Optimizer.................................................................................................................. 208 Unit Summary ............................................................................................................................................................. 209

Section: SQL Cache Analysis - CBO - ORACLE......................................................................................................... 211 Introduction and Technical Background .................................................................................................................... 213

Unit: Introduction and Technical Background...................................................................................................... 215 Introduction................................................................................................................................................................. 216 Overview..................................................................................................................................................................... 217 ORACLE Architecture Review ............................................................................................................................... 218 ORACLE Architecture Review ............................................................................................................................... 219 Shared SQL Area ....................................................................................................................................................... 220 How Oracle Processes an SQL Statement ............................................................................................................. 221 Summary ...................................................................................................................................................................... 222

Introduction to the Shared SQL Area.......................................................................................................................... 223 Unit: Introduction to the Shared SQL Area ........................................................................................................... 225 Definitions................................................................................................................................................................... 226 Expensive SELECT Statements .............................................................................................................................. 227 Data Buffer Hit Rate.................................................................................................................................................. 228 Examine the Data Buffer Hit Rate .......................................................................................................................... 229

Shared SQL Area ....................................................................................................................................................... 230 Shared SQL Area ....................................................................................................................................................... 231 Buffer Gets for an SQL Statement .......................................................................................................................... 232 Different Statements in the Shared SQL Area ...................................................................................................... 233 Summary ...................................................................................................................................................................... 234

Analyzing SQL Statements........................................................................................................................................... 235 Unit: Analyzing SQL Statements ............................................................................................................................ 237 SQL Statements.......................................................................................................................................................... 238 Expensive SQL Statements ...................................................................................................................................... 239 Expensive SQL Statements ...................................................................................................................................... 240 Expensive SQL Statements ...................................................................................................................................... 241 Expensive SQL Statements ...................................................................................................................................... 242 Analyzing the Shared SQL Area ............................................................................................................................. 243 Buffer Gets .................................................................................................................................................................. 244 Buffer Gets per Execution ........................................................................................................................................ 245 Buffer Gets per Record ............................................................................................................................................. 246 Records per Execution............................................................................................................................................... 247 Disk Reads................................................................................................................................................................... 248 Statement Details ........................................................................................................................................................ 249 Display the Execution Plan for an SQL Statement .............................................................................................. 250 The ABAP Dictionary (Transaction SE12) ........................................................................................................... 251 Summary ...................................................................................................................................................................... 252

Update Statistics ............................................................................................................................................................. 253 Unit: Update Statistics............................................................................................................................................... 255 Update of Optimizer Statistics ................................................................................................................................. 256 Overview of the two-phase strategy........................................................................................................................ 257 Support of two-phase strategy by CCMS.............................................................................................................. 258 Table Statistics Date .................................................................................................................................................. 259 Table Statistics Date .................................................................................................................................................. 260 Accuracy of statistics................................................................................................................................................. 261 Accuracy of Table Statistics..................................................................................................................................... 262 Accuracy of Table Statistics..................................................................................................................................... 263 Control Table DBSTATC......................................................................................................................................... 264 Change the Optimization Mode............................................................................................................................... 265 Change the Optimization Mode............................................................................................................................... 266 Preferential Order of Possible Optimizations........................................................................................................ 267

Identify Coding............................................................................................................................................................... 269 Unit: Identify Coding ................................................................................................................................................ 271 Roadmap for Finding SQL Statements in Programs ............................................................................................ 272 Where-Used List......................................................................................................................................................... 273 Find the Program Developer.................................................................................................................................... 274 Statements not Contained in the Where-used List................................................................................................ 275 Using the Global Work Process Monitor............................................................................................................... 276 Using the Oracle Session Monitor .......................................................................................................................... 277 Differences in ABAP Open SQL and SQL Statements....................................................................................... 278 Statements using Internal Tables............................................................................................................................. 279 Internal Table Handling: FOR ALL ENTRIES .................................................................................................... 280 SQL Statements to Project Views ........................................................................................................................... 281 Find the Application Area for a Statement ............................................................................................................ 282 Summary ...................................................................................................................................................................... 283

Workflow and Reporting............................................................................................................................................... 285 Unit: Workflow and Reporting................................................................................................................................ 287 Workflow Overview.................................................................................................................................................. 288 Administrator's and Developer's Responsibilities ................................................................................................ 289 Statement Documentation and Logging................................................................................................................. 290 Workflow Overview.................................................................................................................................................. 291 OSS Call Template .................................................................................................................................................... 292 Workflow Overview.................................................................................................................................................. 293 Responsibilities Overview........................................................................................................................................ 294 Summary ...................................................................................................................................................................... 295

Index Utilization ............................................................................................................................................................. 297 Unit: Index Utilization .............................................................................................................................................. 299 Indexes ......................................................................................................................................................................... 300

Oracle Index Structure............................................................................................................................................... 301 Index Unique Scan..................................................................................................................................................... 302 Index Range Scan....................................................................................................................................................... 303 Order of Fields in the Index..................................................................................................................................... 304 Order of Fields in the Index..................................................................................................................................... 305 Full Table Scan........................................................................................................................................................... 306 Unselective Index Range Scan................................................................................................................................. 307 Important Execution Plans........................................................................................................................................ 308 Summary ...................................................................................................................................................................... 309

Cost Evaluation............................................................................................................................................................... 311 Unit: Cost Evaluation ................................................................................................................................................ 313 Database Cost Based Optimizer .............................................................................................................................. 314 Database Optimizer.................................................................................................................................................... 315 Number of Blocks read for a Full Table Scan....................................................................................................... 316 Costs for a Full Table Scan ...................................................................................................................................... 317 Costs for an Index Unique Scan .............................................................................................................................. 318 Costs for an Index Range Scan ................................................................................................................................ 319 Costs for a 'FOR ALL ENTRIES' Statement ........................................................................................................ 320 Costs for Operators: Between, Like, < and > ........................................................................................................ 321 Costs for two BETWEENS ...................................................................................................................................... 322 Parameter: dbs/ora/use_hints.................................................................................................................................... 323 Estimated Costs for Other Access Paths ................................................................................................................ 324 Optimizer Trace.......................................................................................................................................................... 325 Optimizer Problems ................................................................................................................................................... 326 Appendix: Table Statistics........................................................................................................................................ 327 Appendix: Table Statistics........................................................................................................................................ 328 Appendix: Index Statistics........................................................................................................................................ 329 Appendix: Index Statistics........................................................................................................................................ 330 Appendix: Database Parameters that Control Cost Calculation Functions...................................................... 331 Appendix: R/3 Parameters that Control SQL Statements ................................................................................... 332

Creating an Index........................................................................................................................................................... 333 Unit: Creating an Index............................................................................................................................................. 335 Missing Indexes.......................................................................................................................................................... 336 Check table statistics ................................................................................................................................................. 337 Rules for Creating Indexes ....................................................................................................................................... 338 Selectivity Analysis ................................................................................................................................................... 339 Selectivity Analysis ................................................................................................................................................... 340 Selectivity Analysis ................................................................................................................................................... 341 Selectivity Analysis ................................................................................................................................................... 342 Selectivity Analysis ................................................................................................................................................... 343 Selectivity Analysis ................................................................................................................................................... 344 Selectivity Analysis ................................................................................................................................................... 345 SQLPLUS.................................................................................................................................................................... 346 Preferential Order of Possible Optimizations........................................................................................................ 347 Summary ...................................................................................................................................................................... 348

Similar Statements.......................................................................................................................................................... 349 Unit: Similar Statements ........................................................................................................................................... 351 Similar Statements ..................................................................................................................................................... 352 How to Find Expensive Similar Statements .......................................................................................................... 353 Example ....................................................................................................................................................................... 354 Why Similar Statements Occur................................................................................................................................ 355 Possible Optimizations.............................................................................................................................................. 356 Summary ...................................................................................................................................................................... 357

View Processing............................................................................................................................................................. 359 Unit: View Processing............................................................................................................................................... 361 Views............................................................................................................................................................................ 362 View Processing......................................................................................................................................................... 363 View Processing......................................................................................................................................................... 364 View Processing......................................................................................................................................................... 365 Importance of the Table Access Order for a Nested Loop.................................................................................. 366 Importance of the Table Access Order for a Nested Loop.................................................................................. 367 Execution Plan for a View Statement..................................................................................................................... 368 Preferential Order of Possible Optimizations........................................................................................................ 369

Summary ...................................................................................................................................................................... 370 Joins.................................................................................................................................................................................. 371

Unit: Joins.................................................................................................................................................................... 373 SQL Statements for Joins ......................................................................................................................................... 374 Execution plan of a Join ............................................................................................................................................ 375 ABAP Statements for Joins...................................................................................................................................... 376 Preferential Order of Possible Optimizations........................................................................................................ 377 Summary ...................................................................................................................................................................... 378

Expensive Statements with a Suitable Access Path.................................................................................................. 379 Unit: Suitable Access path........................................................................................................................................ 381 Expensive Statements Using a Suitable Access Path........................................................................................... 382 Modularization in ABAP.......................................................................................................................................... 383 Driven SELECT Encapsulated in a Subroutine (FORM)................................................................................... 384 Navigation in ABAP Coding: Where-Used/Defined ........................................................................................... 385 Driven SELECT Encapsulated in Function Module ............................................................................................ 386 Why It May Be Difficult to Find the Driver.......................................................................................................... 387 Where Resolving Nested SELECTs is not Appropriate...................................................................................... 388 Case Study of a Nested Select.................................................................................................................................. 389 Case Study of a Nested Select.................................................................................................................................. 390 ABAP Coding from Customer................................................................................................................................. 391 Recommended Coding .............................................................................................................................................. 392 Statement Performance After Tuning..................................................................................................................... 393 Preferential Order of Possible Optimizations........................................................................................................ 394 Summary ...................................................................................................................................................................... 395

Appendix.......................................................................................................................................................................... 397 Exercises & Solutions - SQL Cache Analysis for Oracle ................................................................................... 399 Expensive Statements List – open problems ......................................................................................................... 402 SQL Cache Analysis for Oracle - OSS Call Template ........................................................................................ 405

Section: Performance Monitoring .................................................................................................................................... 407 Performance Monitoring ............................................................................................................................................... 409

Performance Monitoring........................................................................................................................................... 411 Database Related Performance Issues .................................................................................................................... 412 Cost-Based Optimizer ............................................................................................................................................... 413 Oracle Cost-Based Optimizer: Review.................................................................................................................. 414 Cost-Based Optimizer Performance Problems ...................................................................................................... 415 Refreshing the Object Statistics: Phase 1............................................................................................................... 416 Refreshing the Object Statistics: Phase 2............................................................................................................... 417 SAP Two-Phase Strategy.......................................................................................................................................... 418 Modifying the Standard Procedure ......................................................................................................................... 419 Using R/3 to Monitor Performance Problems ....................................................................................................... 420 Memory Configuration.............................................................................................................................................. 421 Data Buffer Utilization.............................................................................................................................................. 422 Identifying the Data Buffer Hit Ratio ..................................................................................................................... 423 Increasing the Size of the Data Buffer.................................................................................................................... 424 Identifying Usage of the Shared Pool..................................................................................................................... 425 Identifying the Efficiency of the Shared Pool....................................................................................................... 426 Increasing the Size of the Shared Pool................................................................................................................... 427 Application Design .................................................................................................................................................... 428 When a Lockwait Situation Occurs......................................................................................................................... 429 Using the Exclusive Lockwait Monitor.................................................................................................................. 430 Reducing Exclusive Lockwaits................................................................................................................................ 431 Identifying Expensive SQL Statements (1) ........................................................................................................... 432 Identifying Expensive SQL Statements (2) ........................................................................................................... 433 Running an Explain Plan .......................................................................................................................................... 434 Poorly Qualified SQL Statements ........................................................................................................................... 435 Analyzing Poorly Qualified SQL Statements ....................................................................................................... 436 Physical and Logical Layout.................................................................................................................................... 437 I/O Contention............................................................................................................................................................ 438 Identifying I/O Contention in the Database........................................................................................................... 439 Solving the I/O Contention Problem....................................................................................................................... 440 Checkpoint not Complete ......................................................................................................................................... 441 Rollback Segments..................................................................................................................................................... 442

Solving Rollback Segment Problems ..................................................................................................................... 443 Fragmented Indexes ................................................................................................................................................... 444 Identifying a Fragmented Index............................................................................................................................... 445 Unit Summary ............................................................................................................................................................. 446 Further Documentation ............................................................................................................................................. 447

SAP AG 1999

Course Overview

SAP AG 1999

Target Group

l Audience:

n future SAP R/3 Technical Consultants

n experienced System Administrators

n experienced Database Administrators

n with no or less R/3 knowledge

l Duration: 5 weeks

SAP AG 1999

Course Prerequisites

l In-depth knowledge of the UNIX or NT operating system,the ORACLE or MS SQL Server database and TCP/IPnetwork administration.

l Practical experience with these issues is essential forpassing the SAP R/3 Certified Technical Consultant exam.

SAP AG 1999

Course Goals

l Course participants receive comprehensive expert leveltraining in R/3 System management

l The course prepares participants for the “SAP R/3 CertifiedTechnical Consultant” exam.

SAP AG 1999

Course Composition

UNIX/ORACLE: TABCUO = TABC10 + TABC20 + TABC30

NT/ORACLE: TABCNO = TABC11 + TABC20 + TABC30

NT/MS SQL Server: TABCNS = TABC11 + TABC21 + TABC30

5 weeks = 3 weeks + 1 week + 1 week

SAP AG 1999

TABCUO - Sections

TABC10

Section SAP Basis Technology

Section Technical Core Competence - UNIX

Section Software Logistics

Section R/3 Technical Implementation and Operation Management

Section Advanced R/3 System Administration

Section Ready-to-Run

Section Technical Core Competence - Workplace

TABC20

Section Database Administration ORACLE

Section SQL Cache Analysis - CBO - ORACLE

TABC30

Section Workload Analysis

Section Technical Optimization of ALE Processing

SAP AG 1999

TABCNS - Sections

TABC11


Section Technical Core Competence - NT


Section R/3 Technical Implementation and Operation Management




TABC21

Section Database Administration MS SQL Server

Section SQL Cache Analysis - CBO - MS SQL Server

TABC30



SAP AG 1999

TABCNO - Sections

TABC11


Section Technical Core Competence - NT


Section R/3 Technical Implementation and OperationManagement




TABC20

Section Database Administration ORACLE

Section SQL Cache Analysis - CBO - ORACLE

TABC30



SAP AG 1999

Section: Database Administration - ORACLE

Database Overview Backup Strategies Using RMAN

Backup Strategy and Tape Management

AdvancedBackup Techniques

Scheduling, Performing,and Monitoring Backups Storage Management

Restore and Recovery Top 10 Problems

SAP AG 1999






Database Overview

SAP AG 1999

Database Overview

Contentsl Database componentsl Oracle Process architecturel Net8 Basicsl Database administration tools

ObjectivesAt the end of this unit, you will be able to:l Describe the different Oracle processes and their functionsl Explain the role of Net8l Name the important database administration tasks

n This course is designed to be operating system platform-independent. If you are using Windows NT, you must substitute certain terms and notations.

� Oracle on the Windows NT operation system uses the Windows NT implementation of threads. Windows NT threads are comparable to processes in a UNIX environment. In most cases, you can substitute the UNIX term process by the NT term thread. However, all Oracle processes (except the Oracle listener) form one Oracle process in the Windows NT environment. This process is started as a service called OracleService<SID>. To enable network communication, the Oracle listener process in the NT implementation of the Oracle database is started as a service called OracleTNSListener.

� The directory separator sign used here is the “/” sign. For Windows NT, the “\” sign is used.

� The current value of a UNIX environment variable is obtained using $VARIABLE_NAME. For Windows NT, the syntax %VARIABLE_NAME% is used to obtain the value of a variable.

� For Oracle on UNIX, the naming convention for the listener controller program is lsnrctl. For Oracle on NT, the naming convention depends on the release:

Oracle 8.0: lsnrctl80

Oracle 8.1 and above: lsnrctl

n When you have completed this training unit you will have refreshed your knowledge about the architecture of the Oracle database.

SAP AG 1999

Processes

Memory area

Database files

WP Reconnect:rsdb/reco_trials = 3rsdb/reco_sleep_time = 5

RDBMS

Controlfiles Data files

Online redo log files

Offline redolog files

Profile

SGASGA Database buffer pool Database buffer pool

Shared pool Shared pool

Redo log bufferRedo log bufferShared SQL Area

Configurable(init<SID>.ora)

Row cache

Shared processes

AR

CH

AR

CH

CK

PT

CK

PT

LG

WR

LG

WR

DB

WR

DB

WR

PM

ON

PM

ON

SM

ON

SM

ONOracle listener processOracle listener process

R/3 work processes

Sh

ado

wS

had

ow

Sha

dow

Sh

ado

wS

hado

wS

had

ow

1:1

Review: Oracle Overview

n When an Oracle database instance is started, several processes are created. The groups of processes are distinguished as follows:

� Dedicated shadow processes are created when a new user session on the database is established

� Shared processes perform the various tasks that are required for the database management system to function

n Database data is stored in 8 KB blocks in data files on disk. To accelerate read and write access to data, these data blocks are cached in the database buffer pool in main memory.

n Modifications to database data are logged in the online redo log files. This procedure ensures data security. To ensure fail-safe database operation, without using additional operating system utilities, the control files and the online redo log files of the database system should be mirrored.

n The Oracle database management system holds the executable SQL statements in the Shared SQL Area, which is part of the shared pool.

n Each R/3 work process:

� Connects to the database as one database user, SAPR3

� Handles database requests for the different R/3 System users

� Communicates with a corresponding shadow process on the database

� Can reconnect to the database

SAP AG 1999

R/3 in a Windows NT Environment

OS User OS Group Database User Database Privileges

<SID>adm ORA_<SID>_DBA INTERNAL (SYS), Full database administrationOPS$<SID>adm

SAPService<SID> ORA_<SID>_OPER OPS$SAPService<SID> Restricted database administration

R/3 in a Windows NT Environment


<SID>adm ORA_<SID>_DBA INTERNAL (SYS), Full database administrationOPS$<SID>adm

SAPService<SID> ORA_<SID>_OPER OPS$SAPService<SID> Restricted database administration

R/3 in a UNIX Environment


ora<SID> dba, oper INTERNAL (SYS) Full database administration

<SID>adm oper OPS$<SID>adm Restricted database administration

R/3 in a UNIX Environment


ora<SID> dba, oper INTERNAL (SYS) Full database administration

<SID>adm oper OPS$<SID>adm Restricted database administration

Profile init<SID>.oraOS_AUTHENT_PREFIX=OPS$

Privileges required for SAPDBAbackground actions such as backup,check, -checkopt,-analyze, -next� Restricted database administration

� Access to tables required for R/3database administration

Privileges required for SAPDBAbackground actions such as backup,check, -checkopt,-analyze, -next� Restricted database administration

� Access to tables required for R/3database administration

Security: Operating System and Database Users

n To ensure the security of your R/3 System, you must consider the security of the operating system and database users. Operating system users have certain privileges for accessing files and executing programs. Database users have different privileges for changing tables and indexes.

n Oracle mechanisms move the entire database security mechanism to the operating system level. If the user OPS$<user_name> is defined as identified externally at the database level, the operating system user <user_name> can connect to the database without authentication.

n The following OS and database users are available for Oracle administration in an R/3 System:

� UNIX: ora<SID> (no OPS$ user), <SID>adm, and OPS$<SID>adm

� Windows NT: <SID>adm, OPS$<SID>adm and SAPService<SID>, OPS$SAPService<SID>

n The following OS groups are important in an Oracle database environment (see also Appendix):

� dba (NT: ORA_<SID>_DBA): OS users of this group can connect to Oracle using CONNECT INTERNAL with full database administration privileges

� oper (NT: ORA_<SID>_OPER): OS users of this group can connect to Oracle using CONNECT INTERNAL with restricted database privileges, such as startup or shutdown database

n The database administration tool SAPDBA requires the restricted database administration privileges available in the group oper. SAPDBA only has access to the tables required for performing R/3 database administration in the background. These privileges are assigned during R/3 installation or upgrade.

SAP AG 1999

OracleOracle

R/3 workprocess

R/3 application server

Database server

TableSAPUSER

� Get password foruser SAPR3 fromtable SAPUSER

� Disconnect user OPS$� Connect as user SAPR3

OPS$ user:UNIX: OPS$<SID>admNT: OPS$SAPService<SID>

init<SID>.ora parameterREMOTE_OS_AUTHENT = TRUE

� Connect as user OPS$

Security: SAPR3 Password

n The following mechanism is used by an R/3 work process to connect to the database as user SAPR3:

� A connection to the database is made as user OPS$ (UNIX: OPS$<SID>adm, NT: OPS$SAPService<SID>) with very few privileges.

� User OPS$ is the owner of table SAPUSER. From this table, the password for user SAPR3 is retrieved and the database session for user OPS$ is terminated.

� The work process now connects as user SAPR3 with the password from table SAPUSER.

n To allow R/3 work processes to connect over the network using the OPS$ mechanism, the init<SID>.ora parameter REMOTE_OS_AUTHENT must be set to TRUE. This allows remote OS authentication for OS users with an OPS$ user on any computer in the network in which the database is accessible.

n To change the password of user SAPR3 , perform the following:

For UNIX: Use program SAPDBA to perform the required actions

For NT: Connect to Oracle as user OPS$SAPService<SID>, and:

� Change the password entry for user SAPR3 in table SAPUSER

� Change the password of user SAPR3

n As of version 4.5B, user SAPR3 is stored with an encrypted password in table SAPUSER.

SAP AG 1999

Net8Net8TCP/IPTCP/IP

R/3 work process

R/3 work process

R/3 work process

R/3 application server

ShadowShadow

tnsnames.orasqlnet.ora

ShadowShadowShadowShadow

ShadowShadow

R/3 work process

Net8 Net8 IPC IPC


listener.ora

Database server

OracleOracleListenerListener

NET8 Basics

n If an R/3 instance is running on a server other than the database server, R/3 work processes and their dedicated shadow processes communicate over a network. As communication protocol, TCP/IP is used. The work processes of an R/3 instance configured on the database server use the IPC protocol to communicate with dedicated shadow processes running on the same server.

n For Net8 to accept connections on the database server, the listener must be running. The Oracle utility lsnrctl is used to start and stop the listener and to check the status of Net8 connections. In a UNIX environment, the process tnslsnr is started. On NT, the service "OracleTNSListener" is started.

n Three operating system files are used in a NET8 configuration. You can find these files in the ORACLE_HOME subdirectory network/admin (NT: net80\admin) on each application server and on the database server:

� tnsnames.ora: Contains a list of service names for all databases that can be accessed in the network

� sqlnet.ora: Contains client side default domain information and optional diagnostic parameters used for client tracing and logging

� listener.ora :Only used on a database server machine. Contains Oracle system IDs for which the listener can receive connections, and various control parameters used by program lsnrctl

The default R/3 System profile should contain the entry dbs/ora/tnsname = <SID>

SAP AG 1999

BankBank

Database Administration Tools

LiveR/3 System

SAPDBA

T.D.

To fix the problem,use the monitoringtools and database

administration toolsprovided by SAP

ST04DB02

DB24

RZ20

SAPDBA

Poor performance

n To improve performance and to minimize system downtime, you must monitor the Oracle database daily. The Computing Center Management System provides the following monitors:

w The Database Performance Monitor (transaction ST04) displays an overview of the load and configuration of the database management system and the database.

w The Tables and Indexes Monitor (transaction DB02) displays an overview of the storage behavior of the database and the status of the database objects.

w The DBA Operations Monitor (transaction DB24) provides you with a central point from which you can check the status and logs of all database operations, including backup monitoring, updates of the optimizer statistics, and dba checks.

w The Database Alert Monitor (can be started from transaction RZ20) is a tool you can use to monitor all the preset alerts for different areas of the database.

SAP AG 1999

Database Performance Monitoring (ST04)

SharedSQL

quality

SharedSQL

quality

Oracledata

bufferquality

Oracledata

bufferquality

Usercalls

statistics

Usercalls

statistics

Reset Since reset Since DB start Detail analysis menu Previous days Summary

Data buffer

Shared Pool

Calls

Log buffer

Database QAS Database summary Day, Time 08/19/1999 16.57:46DB Server TWDFMX04 Start up at 08/19/1999 16.59:42Release 8.0.5.1.1 Elapsed since start (s) 86,284

Size kbQuality %

Size kb

reloads/pins %

DD-Cache quality %

pinratio %SQL Area getratio %

User calls commits rollbacks

103,424

Redo logbufferquality

Redo logbufferquality

145,920

64.8 90 94

0.046

99.3

236,764 3,072 171

Size kbEntriesAllocation retriesAlloc fault rate %Redo log wait sLog files (in use)

Recursive callsParses

User/Recursive callsReads/User calls

ReadsPhysical reads

writesBuffer busy waits

Buffer wait time s

36,204,681 246,629 10,641

61 1

328 71,969

1 0.0

0 8 (6)

n From the R/3 main screen, choose Tools → Administration → Monitor → Performance → Database → Activity (or use transaction ST04).

n The main screen of the SAP/Oracle Database Monitor shows the most important indicators of Oracle database performance, such as data buffer, shared pool buffer, user calls, and table scans/table fetch.

n The Detail analysis menu takes you to the detailed level of performance related analysis. Here, you can analyze database activity from the point of view of, for example, Oracle sessions, file system requests, and SQL requests to the database.

n You can also view the Oracle alert file, monitor any changes to the parameter file for Oracle, and monitor database lock-waits.

n The monitor takes a snapshot of the system at a freely selectable reference point (usually at database start). You can use the following to change this reference point:

� Reset sets the reference point to the current point in time

� Since DB start sets the reference point to the time of the database start

� Since reset sets the reference point back to the last reset point

SAP AG 1999

Database Space Monitoring (DB02)

Overallcheck for

thedatabase

Overallcheck for

thedatabase

Check attablespace

level

Check attablespace

level

Checkat object

level

Checkat object

level

Tables and Indexes

Tablespaces

Database system

Total numberTotal size/kb

More than 1 extent

Missing in database

Missing in R/3 DDIC Space-critical objects

Tables Indexes Detailed analysis

Missing indexes

Space critical objects

Space statistics

Space statistics

Current sizes

Freespace statistics

Total number

Total size/kb

Total free/kb Minimum free/kb

Max. autoextensible/kb

Database

NameDate/time of this analysis

Refresh Checks Space statistics

19,299

7,026,064

1,351

0

0

0

22,449

3,783,600

1,851

0

0

0

27

18,090,848

7,083,096

4,040

AutoExtend off

39%

ORACLE

QAS

08/12/1999 13:42:05

n From the R/3 main screen, choose Tools → Administration → Monitor → Performance → Database → Tables / Indexes. Alternatively, use transaction DB02.

n In the section Database system check , you can refresh the statistics, look at the database history (Space Statistics), or check the storage parameters for tablespace and tables/indexes. You can also check for the optimizer statistic runs.

n In the Tablespaces section, options are available for analyzing tablespaces. You can view a complete list of tablespaces with details of, for example, size, freespace, used space, number of objects, and number of extents. You can trace the growth of a tablespace over a particular time period, and track changes to size, extents, and so on.

n The Tables and indexes section displays the objects in this tablespace. The sizes of the objects (in kilobytes and blocks) are displayed, and the number of used extents and the value defined for the object for MAXEXTENTS are also displayed. In this section, you can monitor indexes that are defined in the ABAP Dictionary but are missing in the database (missing indexes) or indexes that are created in the database but are unknown to the ABAP Dictionary.

SAP AG 1999

DBA Operations Monitor (DB24)

Overview of all,running and

finished operationson database

Overview of all,running and

finished operationson database

Click to displayerrors only

Click to displayerrors only

Click to displayall operations

Click to displayall operations

Click to displaywarning only

Click to displaywarning only

Displays specificdatabase operations

chosen by thecorresponding pushbutton on the toolbar

Displays specificdatabase operations

chosen by thecorresponding pushbutton on the toolbar

All Running Finished

Total

Warning

Error

Overview

Backup/recovery Performance Memory structure Check sessions ConfigurationAll database operations

Setup

Status TimeDate FID Object Runtime Program Description

Refresh every

Delete after

View: the last

10 seconds

111 days

10 days

OK OK OK

WarningWarning

ErrorError

Total Total

COMPLETEDCOMPLETED

COMPLETEDCOMPLETED

COMPLETED

COMPLETED

COMPLETED

COMPLETED

COMPLETED

COMPLETED

COMPLETED

24.08.19 ...24.08.19 ...

24.08.19 ...22.08.19 ...

22.08.19 ...

21.08.19 ...

20.08.19 ...

20.08.19 ...

17.08.19 ...

17.08.19 ...

17.08.19 ...

23:08: ...16:24: ...

16:06: ...10:35: ...

02:01: ...

02:01: ...

13:09: ...

10:20: ...

19:22: ...

15:30: ...14:40: ...

anf

anf

svd

svdsvd

svd

svd

svd

svd

aly

opt

database

database

database log

database logdatabase log

database log

database log

database log

database log

DBSTATCO

PSAP%

01:49:46

00:09:34

00:14:07

00:17:44

00:02:02

00:10:24

02:14:46

00:08:04

00:13:06

00:01:41

00:05:00

BRBACKUP

BRBACKUP

BRARCHIVE

BRARCHIVE

BRARCHIVEBRARCHIVE

BRARCHIVE

BRARCHIVE

BRARCHIVE

SAPDBA

SAPDBA

whole online database backup using backint

whole online database backup using backint

first backup and deletion of archivelogs

first backup and deletion of archivelogsfirst backup and deletion of archivelogs





check need of new db-optimizer-statistics

create / refresh db-optimizer statistics

8

3

0

11

0

0

0

0

3

8

0

11

Double-click here to see more details

n Use the DBA operations monitor for online monitoring of database operations. You can also monitor the runtime and the remaining time of operations that are running. The DBA operations monitor provides historical as well as current (online) information about the following database operations:

w Backup/recovery (for example, backing up or recovering the database)

w Performance (for example, checking, creating, updating and deleting database statistics)

w The memory structure (for example, space information for database objects, reorganizing database objects, or extending and deleting database objects).

w Database checks (for example, checking the database for critical situations)

w Configuration (for example, configuring database parameters)

n To call the DBA operations monitor, choose Tools → CCMS → DB administration → Operations monitor. Alternatively, use transaction DB24.

n To display specific database operations (for example, backup/recovery operations), choose the corresponding button.

n To automatically refresh the display of database operations, choose Setup → Auto-refresh → Activate . To set the time interval for the refresh, choose Setup → Auto-refresh → Time Interval.

n For details about operations (for example, the remaining time for the operation, or the directory and name of the log file), double -click the table entry, or select the table entry and choose Display details on the application toolbar.

SAP AG 1999

DBA Alert Monitor (RZ20)

You can monitor :

l Tablespaces

l Performance

l Backups

l Database connections

l SQL statements

l R/3 System log

l Database host operatingsystem

Pre-definedtemplate forOracle databasemonitor in RZ20

space management

performance

backup/restore

R/3 consistency

health

SAP CCMS Monitor Templates (Database . . .

Database

Oracle

View: Current system status (08/12/1999 , 14:46:48

tablespacessegments

optimizerbufferscheckpoints

archivingbackup status

objects missing in the databaseunknown objects in ABAP Dictionaryinconsistent objectsother checksoptional indexes

running jobs

database files

Expert analysis , Node display off

Open alerts Properties

n The alert monitor in CCMS allows you to centrally monitor different parts of the Oracle database. You can configure analysis and data collection tools for different types of alerts. Use the following menu path: Tools → CCMS → Control/Monitoring → Alert monitor. Alternatively, use transaction RZ20. You can find the Database monitor under SAP CCMS Monitor Templates.

n You can monitor, for example, the freespace left in the tablespace, table/indexes with too few allocable extents, segments approaching MAXEXTENT, and the failure of rollback segment extensions.

n You can monitor the performance of the database system by looking at alerts for the optimizer, the Oracle buffer/cache area, and buffer gets of SQL statements.

n You can monitor the backup status and the status of the /oracle/<SID>/saparch directory.

SAP AG 1999

SAPDBA (OS Level)

h - Backup databasei - Backup offline redo logsj - Restore/Recoveryk - DB check/verificationl - Show/Cleanupm - User and Securityn - SAP Online Help Log info

and more

Versionnumber

CallBRARCHIVE

SAPDBA V4.6C - SAP Database Administration

ORACLE version: 8.0.5.1.0ORACLE_SID: TCCORACLE_HOME: /oracle/TCCDATABASE: openSAPR3: 46B, 25 times connected

Databasestate

CallBRBACKUP

a - Startup/Shutdown instanceb - Instance informationc - Tablespace administrationd - Reorganizatione - Export/importf - Archive modeg - Additional functions

q - Quit

Please select ==> I

Start or stopdatabase

Start HTML help

Perform recovery

Reorganize table or data file

Add space

n To start SAPDBA, run command sapdba at command line. SAPDBA includes the following components for administrating the Oracle database:

• Database backup, restore and recovery

• Space management

• Database system check

• Database reorganization

• Cost-based optimization of access

n You can call up the SAPDBA functions from an ASCII interface, or you can use a command option to configure and execute the functions individually. The administration tool SAPDBA for Oracle and its backup tools BRBACKUP, BRARCHIVE and BRRESTORE support the database administrator both in daily routine tasks and in less frequent, more complex tasks, such as recovering or reorganizing the database.

n You can use the SAPDBA functions from the CCMS, since it meets the R/3 System’s application-specific requirements. SAPDBA is delivered as standard with the R/3 System.

n If you call SAPDBA with any command options, the SAPDBA initial menu does not appear. Instead, you can perform operations that do not require interaction with the end user.

n To enable SAPDBA to function properly, you must configure init<SID>.dba file.

n Make sure you have the latest patch installed for SAPDBA. To check SAPDBA’s patch management concept, refer to SAP Notes 126769 and 141999.

SAP AG 1999

l Starting and Stopping the Database

l Writing Data and Log Files

l Storage Management Concepts

l R/3 Naming Conventions

l Oracle Directory Structure in R/3

l Oracle Directories/Environment Variables

l Database Roles and Users

l Net8 Listener

l Alert Monitoring Tree

Contents:

Oracle - Database Architecture

SAP AG 1999

Oracle Oracle processesprocesses::

Data files

Profile


Offline redo log files

Database buffer pool

Redo log buffer

AR

CH

AR

CH

CK

PT

CK

PT

LG

WR

LG

WR

DB

WR

DB

WRNomount

Mount

Open

Control files

Shared pool

PM

ON

PM

ON

SM

ON

SM

ON

Oracle listener processOracle listener process

SGA

Oracle - Starting and Stopping the Database

n When an Oracle database is started, it goes through 3 phases:

� In the No mount phase, the database instance is built up. Operating system resources are allocated using configuration information stored in the profile init<SID>.ora.

� In the Mount phase, the control files of the database are evaluated. The system reads the information about the file structure of the database. Data files and logs are not yet opened.

� In the Open phase, all files in the database system are opened. If required, an instance recovery is performed immediately after opening the database. Pending database transactions are ended.

n You can shut down the database using one of three commands:

� SHUTDOWN NORMAL: No new database logon possible. After all database user have logged off, the database is closed systematically: all files are closed, the database is dismounted, and the instance is shut down. The database is consistent after shutdown.

� SHUTDOWN IMMEDIATE: Only the current commands are executed. PMON ends all sessions and performs a rollback of the open transactions. The database is then closed systematically (as for a normal shutdown). The database is consistent after shutdown. Caution: DBWR and ARCH may require up to 1 hour post-processing time.

� SHUTDOWN ABORT: Emergency database shutdown. Users are not logged off, and a rollback of the open transactions is not performed. The database is not consistent after shutdown. An instance recovery is automatically performed at the next database startup.

SAP AG 1999

Data files

Profile


Redo log buffer

ARCHARCH

LGWRLGWR

Processes Processes andand memorymemory

Profile init<SID>.oralog_archive_start = TRUE log_archive_dest = ?/saparch/

CKPTCKPT

Control files


Offline redo log files ARCHIVELOG MODE: TRUE

DBWRDBWR

Oracle - Writing Data and Log Files

n An Oracle database system has three processes that write information from the Shared Global Area (SGA) to the appropriate files:

� During a checkpoint, the database writer (DBWR) asynchronously writes the changed blocks from the SGA to the database data files

� To speed up the writing of checkpoints, the checkpoint process (CKPT) is started

� The logwriter (LGWR) synchronously writes the change log from the SGA redo log buffer to the currently active online redo log file

n In a production database system, the database must always run in ARCHIVELOG mode and have the archiver process (ARCH) started (init<SID>.ora: log_archive_start = TRUE). ARCH archives a completed online redo log file into an offline redo log file in the archive directory.

n ARCH determines the archive directory from the init<SID>.ora parameter log_archive_dest (default: ?/saparch/) and determines the file name from the parameter log_archive_format.

n Once the offline redo log file has been successfully created, the corresponding online redo log file is released to be overwritten with new log information.

n If no freespace is available in the archive directory, the archiver does not archive the file. After a corresponding number of redo log switches, the database becomes "stuck". Database changes cannot be committed as long as this archiver stuck situation persists.

SAP AG 1999

Oracle - Storage Management Concepts

Extent

Data Block

Segment(Table/Index)

8K

200K 150K

350KData file

Tablespace

n A database is divided into logical storage units called tablespaces. Tablespaces are divided into logical units of storage called segments (tables/indexes). Segments are further divided into extents, which consist of contiguous data blocks. A data block (normally 8K) is the smallest unit of I/O used by a database.

n A tablespace in an Oracle database consists of one or more physical data files. A data file can be associated with only one tablespace. You can increase a tablespace in two ways:

� Add a data file to a tablespace. When you add another data file to an existing tablespace, you increase the amount of disk space allocated for the corresponding tablespace.

� Increase the size of a data file.

n Storage parameters such as INITIAL EXTENT, NEXT EXTENT and MAX EXTENT allow you to manage space allocated to a table.

n For performance reasons, operating system block size should be the same as Oracle data block size.

SAP AG 1999

Prefix Abbreviation ExtensionPSAP <TS_name> D (data) or I (index) Physical

layer

<TS_name>.data1btabd.data1

<TS_name>.data2btabd.data2

Logicallayer

<TS_name>_1btabd_1

Tablespacename

<TS_name>_2btabd_2

PSAP<TS_name>PSAPBTABD

Directorynames

Filesnames

Prefix Tablespace name Ext. Meaning Used bySYSTEM Oracle DDIC Oracle RDBMS

PSAP ROLL Rollback segments Oracle RDBMSPSAP TEMP Sort processes Oracle RDBMSPSAP EL<Release> D or I Development environment loads R/3 BasisPSAP ES<Release> D or I Development environment sources R/3 BasisPSAP LOAD D or I Screen and report loads (ABAP) R/3 BasisPSAP SOURCE D or I Screen and report sources (ABAP) R/3 BasisPSAP DDIC D or I ABAP Dictionary R/3 BasisPSAP PROT D or I Log-like tables (for example, spool) R/3 ApplicationsPSAP CLU D or I Cluster tables R/3 ApplicationsPSAP POOL D or I Pool tables (for example, ATAB) R/3 ApplicationsPSAP STAB D or I Master data and transparent tables R/3 ApplicationsPSAP BTAB D or I Transaction data, transparent tables R/3 ApplicationsPSAP DOCU D or I Documentation, SAPscript, SAPfind R/3 ApplicationsPSAP USER1 D or I Customer tables R/3 Applications

Oracle - R/3 Naming Conventions

n The Oracle database uses tablespaces. From a logical point of view, a tablespace is a container for database objects, such as tables and indexes. On disk, a tablespace consists of one or more data files. You can increase the capacity of a tablespace by adding files to it.

n The R/3 naming convention for tablespace names is defined as follows: PSAP<tablespace_name><extension>.

n The abbreviations in the tablespace name are part of the directory name and file name of each data file. Directories and data files are numbered.

n The objects located in the tablespaces SYSTEM, PSAPROLL, and PSAPTEMP belong either to the Oracle database users SYS or SYSTEM. Do not create any objects owned by other users in these tablespaces.

n The objects located in the other tablespaces belong to the R/3 database user SAPR3. R/3 System users do not have a database system user.

n The R/3 System and SAP tools, such as SAPDBA, require that the naming conventions be observed. The installed system constitutes a logical unit, which you should not change. In this way, SAP can ensure that you receive fast and efficient support.

SAP AG 1999

Directory Contains File name examples

dbs SAP and Oracle profiles, init<SID>.ora, init<SID>.dba, init<SID>.sap

bin Oracle executables

saptrace Background (Oracle alert file)usertrace

sapdata1 Datafiles /btabd1/btabd.data1, system.data1,

. ctrl<SID>.dbf, /btabi1/btabi.data1

.

sapdata<n>

sapbackup BRBACKUP, BRRESTORE logs

saparch BRARCHIVE logs, Oracle archive dir ctrl<SID>.dbf

sapcheck SAPDBA logs (-next, -check, -analyze)

sapreorg SAPDBA logs(default),default compression directory

origlogA Online redo log files log_g101m1.dbf, log_g103m1.dbf

origlogB Online redo log files log_g102m1.dbf, log_g104m1.dbf

mirrlogA Online redo log files log_g101m2.dbf, log_g103m2.dbf

mirrlogB Online redo log files log_g102m2.dbf, log_g104m2.dbf

. . .

Profile init<SID>.oralog_archive_format = %t_%s

Oracle - Oracle Directory Structure in R/3

n Directory and file names are standardized in the R/3 environment. We recommend that you use the following standards:

� Tablespace files reside in the sapdata<n> directories

� The online redo log files reside in the origlog and mirrlog directories

� The offline redo log files are written to the saparch directory

� There should be at least 3 copies of the Oracle control file on different disks

� The profile init<SID>.ora configures the Oracle instance, and resides in directory dbs (NT: database)

� The profile init<SID>.sap configures the backup tools brbackup and brarchive, and resides in directory dbs (NT: database)

� The profile init<SID>.dba configures the SAPDBA tool, and resides in directory dbs (NT: database)

� The Oracle alert file is written to directory saptrace/background

� Trace files of the Oracle shadow processes are written to the directory saptrace/usertrace

� During reorganization, export datasets are written to directory sapreorg

n The directories saparch, sapcheck , sapreorg, and sapbackup are used by the SAP database tools.

SAP AG 1999

Server site

dbs(NT:database) bin

network/admin(NT: net80\admin)

ORACLE_HOMEoriglogB

mirrlogB

sapdata<n>

sapbackup

sapreorg

...

sapdata1

origlogA

mirrlogA

saparch

sapcheck

saptrace

SAPDATA_HOME

Client site

ORACLE_HOME

network/admin(NT: net80\admin)

UNIX environment variables (client site)ORA_NLS: $ORACLE_HOME/ocommon/NLS_723/admin/data ORA_NLS32: $ORACLE_HOME/ocommon/NLS_733/admin/data ORA_NLS33: $ORACLE_HOME/ocommon/NLS_805/admin/data

Oracle - Oracle Directories/Environment Variables

n The Oracle database file tree structure on the database server site has two main branches:

� The Oracle binaries are located in the subdirectory bin of the ORACLE_HOME directory. The environment variable ORACLE_HOME points to this directory. The ORACLE_HOME directory is also required on each server with a database client

� The environment variables SAPDATA_HOME and SAPDATA<n> point to the directories containing database-specific files, such as data files, online redo log files, and offline redo log files

n The operating system users <SID>adm and ora<SID> (on the database server, not used in an NT environment) require the following environment variables:

� ORACLE_SID = <SID> (on the database server site)

� DBS_ORA_TNSNAME: set to the database identifier <SID> from tnsnames.ora

n In a UNIX environment, the following environment variables are set by R/3 configuration tools:

� ORA_NLS: $ORACLE_HOME/ocommon/NLS_723/admin/data (on each client site)

� ORA_NLS32 $ORACLE_HOME/ocommon/NLS_733/admin/data (on each client site)

� ORA_NLS33: $ORACLE_HOME/ocommon/NLS_805/admin/data (on each client site)

SAP AG 1999

Database Role Description

SYSDBA Can perform database administration,has privileges to access all database tables

SYSOPER Can perform database administration such as startup, shutdown, and backup,has no privileges to access database tables

SAPDBA Has privileges to access certain tables required for database administrationactions performed in background (-check, -checkopt, -analyze, -next, backup)

Database Role Description

SYSDBA Can perform database administration,has privileges to access all database tables

SYSOPER Can perform database administration such as startup, shutdown, and backup,has no privileges to access database tables

SAPDBA Has privileges to access certain tables required for database administrationactions performed in background (-check, -checkopt, -analyze, -next, backup)

Database User Description

SYS Database owner, can perform database administration,has privileges to access all database tables

SYSTEM Can perform database administration,has privileges to access database tables in read and write mode (but notOracle DDIC tables)

SAPR3 Owner of database tables and indexes used by the R/3 applications,has no privileges to perform database administration

Database User Description

SYS Database owner, can perform database administration,has privileges to access all database tables

SYSTEM Can perform database administration,has privileges to access database tables in read and write mode (but notOracle DDIC tables)

SAPR3 Owner of database tables and indexes used by the R/3 applications,has no privileges to perform database administration

Connect Request Description

INTERNAL CONNECT INTERNAL possible for OS user belonging to OS group DBAwith SYSDBA privileges and for OS user belonging to OS group OPERwith SYSOPER privileges

Connect Request Description

INTERNAL CONNECT INTERNAL possible for OS user belonging to OS group DBAwith SYSDBA privileges and for OS user belonging to OS group OPERwith SYSOPER privileges

Oracle - Database Roles and Users

n The following database roles are important for performing database administration tasks in the R/3 environment:

� SYSDBA: Privilege to access all database objects

� SYSOPER: Privilege to change the operation mode of the database. No privilege granted on tables

� SAPDBA: Privilege to access certain tables belonging to SAPR3 that are required to perform database administration tasks in the background

� The combined privileges of the SYSOPER and SAPDBA roles are sufficient to perform certain database administration tasks in the background

n OS users belonging to the OS groups DBA and OPER can connect to the Oracle database using the identification INTERNAL. They are assigned the privileges of the database roles SYSDBA or SYSOPER.

n The database users are as follows:

� SYS: Oracle default database user for database administration, owner of the database

� SYSTEM: Oracle default user who can access all database tables in read and write mode when the database is open. No privilege to change Oracle DDIC tables.

� SAPR3: All R/3 tables and indexes belong to this database user. Does not have privilege to perform administrative actions on the database

SAP AG 1999

> lsnrctl helpThe following operations are availableAn asterisk (*) denotes a modifier or extended command:

start stop status servicesversion reload trace spawndbsnmp_start dbsnmp_stop dbsnmp_status quitexit cancel* repeat* set*show*

> lsnrctl statusConnecting to (ADDRESS=(PROTOCOL=IPC)(KEY=TCC))STATUS of the LISTENER------------------------Alias LISTENERVersion TNSLSNR for HPUX: Version 8.0.5.0.0 - ProductionStart Date 13-MAY-99 14:11:35Uptime 20 days 23 hr. 52 min. 47 secTrace Level offSecurity OFFSNMP OFFListener Parameter File /usr/sap/trans/listener.oraListener Log File /oracle/TCC/network/log/listener.logServices Summary... TCC has 1 service handlersThe command completed successfully

Oracle - Net8 Listener

n The Oracle listener is controlled by command lsnrctl. (NT: lsnrctl80)

n Command lsnrctl status (lsnrctl80 status) displays information such as Net8 version, listener program start time, and the location of parameter and listener log files.

� To return a list of available commands, enter help at the lsnrctl command prompt.

n The file listener.ora is read when the listener program is started on the database server. The configuration information specified in this file determines Net8 settings such as the network protocol to be used, host name, port, and the default tracing information.

n Database server listener tracing can be enabled by setting trace level information in the file listener.ora or by turning it on through the program lsnrctl.

Valid options for listener tracing are:

OFF: No tracing (default)

USER: Limited level of tracing information

ADMIN: Detailed trace

Use tracing for diagnostic purposes only. Do not leave tracing on indefinitely in a production system.

SAP AG 1999

Oracle - Alert Monitoring Tree

Problem or ErrorWarningOKNo data

Monitoring attributes and current valuesMonitoring attributes and current values

Show most recentperformance

values

Show most recentperformance

values

Display all alerts ofthe selected item

Display all alerts ofthe selected item

Customize alerts andconfigure threshold

values

Customize alerts andconfigure threshold

values

Delete alerts fromopen alert list

Delete alerts fromopen alert list

Start analysis toolassociated with the alert

Start analysis toolassociated with the alert

Monitoring objectMonitoring object

Current status Display alerts Complete alerts Properties

Oracle

space management

performance

Database

View: Open alerts (08/18/1999 , 09:39:16)

optimizer

checkpoints

library cache

Expert analysis , Node display off

buffers

buffer cache

redo log buffer

[ 16 Alerts ] , 56% < 90%: Cache hit ratio below threshold

[ 25 Alerts ] , 115 < 4,000 redo entries per redo log space requests

Monitor identificationMonitor identification

Monitoring treeelement (MTE)

Monitoring treeelement (MTE)

Display detailed alertsDisplay detailed alerts

n The Alert Monitor monitors various component of your R/3 System. Use the menu path: Tools → CCMS → Control/Monitoring → Alert monitor. Alternatively, use transaction RZ20.

n The Open Alerts view shows what has happened in the system since it was last checked.

n The Current status view shows the most recent values.

n The Display Alert shows the history of the alert values.

n Any problems or errors are displayed in in red. Warnings are displayed in yellow. Green means that, according to the threshold values, there are no problems.

n You can use Properties to customize the threshold values for red and yellow alerts.

n To start the analysis tool, double -click the alert text that you want to analyze.

n To display details of certain type of alerts, set the checkbox next to the alert and then choose Display detailed alerts.

n The Complete Alert button resets the alerts displayed on the screen.

SAP AG 1999

Unit Summary

Now you are able to:

l Explain the basic concepts of the Oracle database

l Address security issues

l Explain Oracle communication over a network (NET8)

l Understand which tool to use for each part of the Oracledatabase

n The Oracle database system can be operated only when it is configured correctly. To make configuration changes, you need a basic knowledge of its components.

n The connection process and database administration are the two greatest security risks. You need to know how to address these risks.

n In the R/3 System environment, each individual database component is created following the standard naming convention. These conventions simplify database administration, because they are automatically used by the various R/3 database administration tools.

n NET8 is used to communicate with the Oracle database over the network. To ensure that network communication functions properly, you need to know the basic configuration files and their contents.

SAP AG 1999

l Exercises?

l Solutions

Unit Actions

Database Overview: Exercises

No. Exercise

1 Start your local database using SAPDBA. Check if your local database is running in ARCHIVELOG mode. If it is not, use the SAPDBA to switch to ARCHIVELOG mode.

2 Check if the passwords of users SYSTEM and SYS still have their default values in your local database.

3 Change the password of user SAPR3 in your local database.

4 Use the SAPDBA menu Tablespace administration to find a list of all tablespaces on your local database. What are the names of the files on the operating system level and in which directory do they exist?

5 Log on to the R/3 System and start the database monitor. Enter values for:

5.1 Data buffer

5.2 Log buffer

5.3 Shared pool

6 List the parameter values belonging to the following objects:

6.1 The size of the shared pool

6.2 The number of blocks in the data buffer

6.3 The size of the log buffer

6.4 The size of an ORACLE block

7 Look in the V$ tables V$LOG and V$LOGFILE

7.1 Find the names of the online redo log files

7.2 Find the current log sequence number

7.3 Which online redo logs have already been backed up by the ARCHIVER?

8 Start the Tables and Indexes Monitor and access the list of all tablespaces in the R/3 System:

8.1 Which is the largest tablespace?

8.2 Which tablespace has the smallest amount of free space?

8.3 Which tablespace contains the most tables or indexes?

8.4 How many data files are there in the tablespace PSAPBTABD?

9 (Optional) In the R/3 System, find out if the R/3 database is running in ARCHIVELOG mode.

Database Overview: Solutions

No. Solution

1 To find the necessary information in SAPDBA, select f - Archive mode. If noarchivelog is displayed by DATABASE LOG MODE, switch to archive log mode by using a - Toggle database log mode. NOTE: SAPDBA must recycle (stop and restart) the database for this operation.

2 Call SAPDBA, and choose m - User and Security → b - User information

3 Call SAPDBA, and choose m - User and Security → p – Change password → c – Change password. Change password of user SAPR3. Enter a new password. Confirm the new password.

4 From SAPDBA, select c - Tablespace administration → h - Display all tablespaces and data files. A list of all tablespaces and the related data files from your local database is displayed.

5 From the main R/3 menu choose Tools → CCMS → Control/Monitoring → Performance menu → Database → Activity. The values you need are displayed.

6 From the main R/3 menu choose Tools → CCMS → Control/Monitoring → Performance menu → Database → Activity → Detail analysis menu → Parameter changes. Then choose Active parameters. The parameters you need are:

6.1 SHARED_POOL_SIZE

6.2 DB_BLOCK_BUFFERS 6.3 LOG_BUFFER

6.4 DB_BLOCK_SIZE 7 From the main R/3 menu choose Tools → CCMS → Control/Monitoring

→ Performance menu → Database → Activity. Then choose Display V$Values

7.1 The names of the online redo log files are in table V$LOGFILE 7.2 The current log sequence number is the number of the redo log group with

the status current in table V$LOG 7.3 If yes is displayed for Archive, then the online redo log has already been

archived. 8 From the main R/3 menu, choose Tools → CCMS → Control/Monitoring

→ Performance menu → Database → Tables/Indexes. Then choose Current sizes.

8.1 Sort by Size(kb)

8.2 Sort by Free(kb) 8.3 Sort by Tab/Ind. 8.4 Place the cursor on PSAPBTABD, and then choose Data Files. A list of data

files belonging to PSAPBTABD is displayed. 9 The information is in table V$DATABASE. See exercise 5 for the menu

path.

SAP AG 2000







SAP AG 1999


Contentsl Backup strategyl Possible causes of data lossl Tape management functions provided by the SAP database

backup tools

ObjectivesAt the end of this unit, you will be able to:l Define a backup strategy that meets the needs of your

companyl Configure the tape management system for performing

database and offline redo log file backupsl Set up and manage tape pools

n Once you have completed this unit, you will be able to:

� Define a backup strategy that meets the needs of your company

� Configure the tape management system for performing database and offline redo log file backups

� Set up and manage tape pools

� Perform tape initia lization

� Describe the tape layout

� Describe the procedure of tape select by the tape management system

SAP AG 1999

A good database backup strategy prevents dataloss and minimizes system downtime

External factors(Such as fire orwater damage)

Physical errors(Such as hardware

failure)

DROP MARA

Logical errors(Such as a deleted

table)

Dataloss

The Importance of Database Backups

Procedure andEscalation Plan

n If you do not have a suitable backup strategy, external factors, physical errors, and logical errors can cause system downtime and may lead to data loss.

n If data is lost due to external factors, such as water damage to your hardware, or physical errors, such as hardware failure, you must recover the database up to the point in time when the database crashed. If a full recovery is possible, only the data of uncommitted transactions before the error will be lost.

n If data is lost due to logical errors, such as an unintentionally deleted table, you must recover the database up to a point in time shortly before the error occurred.

n Your backup strategy must be designed according to the needs of your company. To ensure the availability of your R/3 System, your backup strategy must be carefully tested before your R/3 System goes live, and after any changes to your backup strategy.

n When you set up your backup strategy, you must consider how long you can afford to shut down the R/3 System for each of the above scenarios.

n To ensure that the correct actions are performed for each of the scenarios, create a document containing organizational descriptions of procedures and an escalation plan. This document must be understood by the person who performs the database restore and recovery.

n You should evaluate and implement the most suitable backup type and method for your company. SAP provides tools that support different types of backups, such as full online, incremental backups with RMAN, and split mirror backups, which are discussed in the next units.

SAP AG 1999

Physicaldata check:

Verify backupon tape

Logicaldata check:

Verify databaseconsistency

Oracle data files

ORA-1578:Oracle data

blockcorrupted

Database backup

Preventing and Handling Errors

n Your backup strategy should include verifying the data to be backed up as well as the data on tape.

n To verify the consistency of the Oracle database, perform a logical data check.

� Corrupt Oracle blocks (error ORA-1578) can appear in your R/3 database as a result of operating system or hardware errors. Corrupt Oracle blocks may make a backup unusable.

� The existence of these blocks only becomes evident during the next read access attempt to a table within the database. Since this particular access attempt do not occur often, and corrupt Oracle blocks are not recognized during a database backup, these corrupt blocks may remain undetected in your system for a long time.

� Therefore, you should perform a logical data check at regular intervals. You can perform this check using brbackup -w use_dbv (see SAP Notes 155524 and 23345). For optimal performance, perform this check during periods of low system activity, such as weekends.

n To verify the tapes used for a database backup, perform a physical data check. To check the physical correctness of the data transferred, the tapes should be read after a successful backup.

SAP AG 1999

You must recover the complete database to a point in time before the error

DROP MARA

Logical errors(Such as a deleted

table)

Time when logicalerror occurred

Time when database isstopped for recovery

Lost information

Logical error recovery

Possible Causes of Data Loss (1)

12:50 4:00

12:50 - 4:00

n Logical errors can be caused by:

� Manually dropping database objects

� Manually deleting parts of a database objects, such as rows in a table

� Application errors

n If a logical database error occurs, you must recover the complete database since the data from different tables must be consistent.

n Because you must perform a point in time recovery to a point in time before the error, data changed between the time when the logical error occurred and the time the database is stopped for recovery is lost.

n Depending on the table, it may be possible to restore the database to a different machine and then export the table from that machine to your production R/3 System or to read the missing table rows from the restored table. This method avoids data loss. However, this method is difficult and requires expert knowledge of the application module that uses the table.

SAP AG 1999

If one offline redo log file is lost, none of the filesthat follow it can be used

Sequence of offline redo log files:

Forward recoveryA database backup is restored and you want torecover data from offline redo log files

Lost offline redo log file

Point in time ofthe database error

Intact but unusable offline redo log files

Lost information

Time


n When you perform a point in time recovery, you need all the offline redo log files from the point in time of the last database backup up to the point in time you want to restore to.

n If a file is missing from the chain of offline redo log files, then a restore of subsequent offline redo log files is not possible. Data will be lost from the point in time of your lost offline redo log file.

n Therefore, you should keep at least two copies of all offline redo log files on tape.

SAP AG 1999

Disaster recovery

External factors(Such as fire or water damage)

Physical errors(Such as hardware failure)

Only data saved ontape can be recovered

Only tapes stored in a safelocation can be recovered


n If a hardware failure occurs, such as a disk crash, you can only restore data that is stored on tape.

n If all of your disks or the complete hardware is lost, only the data available on tape can be recovered. Only the offline redo log files already stored on tape can be restored. Offline redo log file information that is not stored on tape will be lost.

n If data loss occurs due to external factors, such as fires or water damage, all tapes that are not stored in a safe location may be lost.

SAP AG 1999

Additional

Offline redo log file backup (x2)

28 days

Offline redo log file backup (x2)

Verify the database

Verify the backupOnline

Online

Verify the backupOffline

Backup Cycle Recommendations

n We recommend a backup cycle of 4 weeks.

n A pool of tapes for database and offline redo log file backups is required. Ensure that enough tapes are provided in each tape pool to span the entire backup cycle. We recommend having 30% more tapes than required to cover database growth and additional backups, for example after a database extension. Backup tapes can be reused at the end of a backup cycle (after 28 days).

n Perform a full online backup each workday. Perform a full offline backup at least once in the cycle.

n You must back up the offline redo log files each workday, as well as after every online and offline backup. Ensure that you back up the offline redo log files twice, on separate tapes, before they are deleted.

n To verify a backup, check the database for logical errors and the database backups for physical errors. You must perform backup verification at least once in the backup cycle. However, we recommend that you perform it once a week.

n Remove the last verified full offline backup of each cycle from the tape pool, and keep this backup in long-term storage. Replace the tapes, and initialize new ones.

n Changes to the file structure of the database affect the subsequent database restore. These changes occur when a data file is added, when a data file is moved to a different location, or when a tablespace and its data files are reorganized. Perform additional backups after each database structure modification or a system upgrade. Place these additional backups in long-term storage.

SAP AG 1999

Media

cpio/dd

Controlfile

Datafiles

Onlineredo log files

Offlineredo log files

Oracle database

Media

BRRESTORE BRARCHIVE

cpio/ddparallel

BRBACKUP

SAP Database Backup Tools

Detail logDetail logSummary Summary

loglog

Detail logDetail logSummary Summary

loglog

n In addition to the database administration tool SAPDBA, SAP provides you with the following tools for performing data backups:

� The program BRBACKUP backs up the data files, the control file, and the database redo log files where necessary.

� The program BRARCHIVE backs up the offline redo log files of the database.

n Both BRBACKUP and BRARCHIVE record the actions performed in log files. These log files can be used in the case of a database restore, and can be analyzed by the program BRRESTORE. This program can restore all files belonging to the database system from the backups.

n The database backup tools support standard backups, both to disk and to tape.

SAP AG 1999

Objects that need to be backed up

R/3 dataComputing center data

l R/3 interfacesl SAP executables

l Operating systeml Database executables

Database objects

Data files


Control file

Profiles


Offline redo logfile backup

Offline redo logfile backup

Database backupDatabase backup

BRBACKUP

BRARCHIVE

BRRESTORE

Backup Objects

n SAPDBA will backup all the business data, but your backup strategy must include backing up all objects. Exactly which objects these are depends on the organizational structure of your company. In the R/3 environment, the backup objects include the operating system and the files associated with R/3.

n The objects that need to be backed up include:

� R/3 data, such as:

R/3 archiving objects

R/3 interfaces

SAP executables

� Computing center data, such as:

The operating system

Third party programs connected to R/3

� Database objects

n A correctly implemented database backup strategy is the only effective protection against data loss in the database.

SAP AG 1999

BRBACKUP

BRARCHIVE

??Length of

backup cycle

Number of parallel backup devices

+ 30% Reserve

Frequency of backups

Databasesize

Number and size of redo log files in a backup cycle

Tape pools

Tape Pools

n To help manage your tapes, BRBACKUP and BRARCHIVE offer a tape management system that:

� Helps you find the tapes that are necessary to perform a backup

� Helps you find the appropriate tapes when you need to recover your database

� Provides the security that tapes are not overwritten accidentally

n You must initialize one pool of tapes for BRBACKUP and another pool of tapes for BRARCHIVE. Tapes that are initialized by BRBACKUP cannot be used by BRARCHIVE, and vice versa.

n The number of tapes you need for BRBACKUP depends on factors such as the length of your backup cycle, the size of your database, the number of tape devices to be used in parallel, the storage capacity of the tapes you use, and the frequency of database backup operations.

n The number of tapes you need for BRARCHIVE depends on the length of your backup cycle, the storage capacity of the tapes you use, the average number and the size of the redo log files created in a backup cycle, and the frequency of offline redo log file backup operations.

n In addition to the number of tapes you need based on your backup strategy, you should have a reserve of 30% more tapes in each tape pool. This is useful in the case of database growth, exceptionally high redo log volume, or if additional backups need to be performed.

SAP AG 1999

brbackup -i -v <tape name> or brarchive -i -v <tape name>

nn Initialize new tapes, non-SAP tapes, or locked tapes:

nn Rename non-locked tapes:

brbackup -i force or brarchive -i force

.tape.hdr0

nn Write the label to the tape that also contains the tape name

...volume_backup = (<SID>B01,<SID>B02...volume_archive = (<SID>A01,<SID>A02......

nn Profile init<SID>.sap contains the tape names:

Tape Initialization

n During tape initialization, an SAP-specific label is written on the tape as the first file (.tape.hdr0).

n To initialize the tapes, use the following BRBACKUP or BRARCHIVE commands:

� For locked tapes, or tapes that were used by another application

brbackup -i force or brarchive -i force

� For renaming tapes that are not locked

brbackup -i or brarchive -i

n You can also use SAPDBA to initialize tapes.

n You can specify the tape name that you want to initialize explicitly by using commands:

brbackup -i -v <tape name> or brarchive -i -v <tape name> respectively.

n If you do not specify the tape name explicitly , BRBACKUP or BRARCHIVE will automatically select the tape names from the pool of tape names specified in the configuration file init<SID>.sap by parameters volume_backup and volume_archive.

n Suggested naming conventions for your tapes are:

� Tapes used for BRBACKUP: <SID>B01,<SID>B02,..., <SID>BXX

� Tapes used for BRARCHIVE: <SID>A01,<SID>A02,..., <SID>AYY

SAP AG 1999

Tape labelcontents:

Tape checks: n Tape name

n Lock period

n Use count

n Tape name

n Database name

n Timestamp of last backup

n Number of backups performed

nn Show tape label contents:

brbackup -i show or brarchive -i show

...expir_period = 28tape_use_count = 100...

Profile init<SID>.sap:

Tape Label Contents and Tape Checks

ErrorError

WarningWarning

n The tape label contains the following information:

� The tape name

� The database name

� The timestamp of the last backup recorded on the tape

� The number of backups performed with the tape

n By default, BRBACKUP and BRARCHIVE read the tape label before they start writing to a tape in order to check:

� The tape name

� If the tape is locked

� The number of times the tape has already been used

n If the tape name is wrong or if the tape is locked, an error is reported and the tape is not used.

n If tape is used more often than the value set in parameter tape_use_count in file init<sid>.sap, a warning is generated but the tape is used.

SAP AG 1999

BRBACKUP

Tables SDBAHand SDBAD

Logical tape locking

List of tapes usedin expir_period

...C11B05,C11B06,...

Profile: init<SID>.sap...volume_backup = (C11...volume_archive = (C11......

Days1 2 28 29 30 31 32 33

C11B01 C11B02expir_period = 30expir_period = 30

Physical tape locking

.tape.hdr0

BRARCHIVE

Tape Locking

2828daysdays

n Before writing to tape, BRBACKUP and BRARCHIVE check that the tape is not locked. To prevent tapes from being overwritten too early, ensure parameter expir_period in file init<SID>.sap is set to at least the length of your backup cycle (in days).

n A tape is locked if the number of days passed since it was last used is less than value of parameter expir_period. There are two different types of locks for a tape:

� The physical lock is derived from the tape label. The timestamp of the last backup and the parameter expir_ period determine if a tape can be reused. If the last backup was performed too recently, then the tape is locked physically. The timestamp is written at the beginning of a backup.

� The logical lock is derived from the timestamp written to certain database tables. BRBACKUP and BRACHIVE also write information about the backup, including a timestamp, to the database tables SDBAH and SDBAD when the backup is successfully finished.

n To find the tapes that can be used for the next backup, BRBACKUP connects to the database and searches tables SDBAH and SDBAD for the tapes that were used in the lock period. The tapes used cannot be used for the next backup. This is the logical lock check for database backups.

n The logical lock check for the offline redo log file backups is performed by BRARCHIVE using information from the summary log. Therefore, offline redo log files can be backed up when the database is not available.

n The tape from the tape name list in profile init<SID>.sap that follows the tape that was last recently used, and is not contained in the list of tapes used in the lock period, is selected for the next backup.

SAP AG 1999

28 days

Offline

Additional

Online

Online

Archives (2x)

Archives (2x)

Tape management active,BRBACKUP finds tape names

+Tape label check active

=Only accepts tapes with requestednames and with expired lock period

C11B01 C11A01

BRBACKUPBRARCHIVE

Tables SDBAHand SDBAD

Scenario 1: Automatic Tape Selection

n The SAP tools provide three procedures for selecting a tape for the backup:

� Automatic tape selection by BRBACKUP or BRACHIVE

� Manual tape selection by the operator

� Tape selection by an external tool

n If you want BRBACKUP or BRACHIVE to select the tape(s) to be used for the next backup run automatically, you must define the parameters volume_backup and volume_archive in profile init<SID>.sap.

n Each time you start a backup, BRBACKUP or BRACHIVE requests the next unlocked tape in the order defined by parameters volume_backup and volume_archive. After the last medium on the list is used, the first medium on the list is requested again. This medium must not be locked at that time.

n To find out the name of the requested tape, call BRBACKUP or BRARCHIVE with the option -q. To check whether you have mounted the requested tape, call BRBACKUP or BRARCHIVE with the option -q check.

SAP AG 1999

C11B01 C11A01

BRBACKUPBRARCHIVE

Scenario 2: Manual Tape Selection

Tape lock expiration will be checked+

Tape management is not active=

To select any tape manually

Tape lock expiration will be checked+

Tape name is changed to thecurrently required tape name

=You can use this option to replace

tapes in your tape pool

Tape with label name SCRATCH

28 days

Offline

Additional

Online

Archives (2x)brbackup -v SCRATCH orbrarchive -v SCRATCH

n To select the tape that is used for the backup, use option SCRATCH.

n Option SCRATCH can be used in two ways:

� As an option for BRBACKUP or BRARCHIVE

� As a tape name

n If a tape is initialized with the name SCRATCH, it can be used for any backup regardless of the name of the tape that is required for the current backup. The tape name is changed to the name of the tape required. Use this option to replace tapes in your tape pool.

n If SCRATCH is used as an option for BRBACKUP or BRARCHIVE, that is you issue the command: brbackup -v SCRATCH or brarchive -v SCRATCH, any initialized tape with an expired lock period can be used for a data backup. However, in this case, the tape name will not be changed. You must ensure that the correct tape is inserted. This option can be used to select a tape manually, for example, for a month end backup if you do not want this backup to be performed on your tape pool tapes.

SAP AG 1999

C11B01 C11A01

BRBACKUPBRARCHIVE

Scenario 3: Tape Selection by an External Tool

Searchmechanism

Tape management isdeactivated

but tape name is checked+

Tape label check active=

Only accepts tapes - With the specified names- With expired lock period

brbackup -v C11M01,C11M02

28 days

Offline

Additional

Online

Archives (2x)

n To explicitly specify the tape(s) to be used by BRBACKUP or BRARCHIVE, use the option -v. This is useful when using a shell script or external tape management tool for determining the tapes to be used at the next backup run.

n BRBACKUP and BRARCHIVE check that the tapes specified by option -v have been mounted and that they are not locked. Locked tapes are refused.

SAP AG 1999

Offlineredo log n

Detaillog

Summarylog

BRBACKUP:

BRARCHIVE:

.tape.hdr0 init<sid>.orainit<sid>.dba

init<sid>.sap DB file 1

DB file nDetail

logSummary

logreorg.logstruct.log

reorg.logstruct.log

.tape.hdr0init<sid>.orainit<sid>.dba init<sid>.sap

Offlineredo log 1

Control file

Tape Layout

n The files written to tape by BRBACKUP and BRACHIVE are:

� .tape.hdr0: Tape label

� init<SID>.ora: Database configuration file

� init<SID>.dba: SAPDBA configuration file

� init<SID>.sap: BRBACKUP and BRARCHIVE configuration file

� reorg<SID>.log:: Information about the creation, extension, or reorganization of a tablespace, restoring and recovering the database (located in directory sapreorg)

� struct<SID>.log:: History of database structure changes (located in directory sapreorg)

� Detail log:: Complete output of the BRBACKUP or BRARCHIVE run (located in directories sapbackup or saparch)

� Summary log back<SID>.log:: List of all backups started with BRBACKUP (located in directory sapbackup)

� Summary log arch<SID>.log:: List of all offline redo log files backed up by BRARCHIVE (located in directory saparch)

SAP AG 1999

Unit Summary


l Configure the tape management system for runningdatabase and offline redo log backups

l Set up and manage tape pools

l Perform the tape initialization

l Describe the procedure of tape selection by the tapemanagement system

l Describe the tape layout

n The SAP backup tools offer a basic method of tape management. Used in conjunction with a suitable naming convention, tape management with SAP backup tools enables you to manage your backup media securely and in a comprehensible manner.

n The tapes that are required must be initialized before the backup cycle can be implemented. It is important that you allow sufficient reserve capacity when initializing the tapes.

SAP AG 1999

l Exercises?

l Solutions

Unit Actions

Backup Strategy and Tape Management: Exercises

No. Exercise

Backup Strategy

1 Evaluate the following backup strategy.

Technical specifications:

The planned size of the database is roughly 100 GB

A maximum of 50 online redo log files of 20 MB are expected to be written daily

Three tape devices are available and each can write or read up to 6 GB per hour

The tapes have a capacity of 40 GB

It takes on average three minutes to apply an offline redo log file during the recovery

Strategy:

An online backup is performed every night

Three tapes are reserved for every night

The database administrator performs a backup of the offline redo log files daily, and deletes the offline redo log files from disk afterwards

1.1 Is this a good backup strategy?

1.2 Can a full restore be performed in 8½ hours?

1.3 What is the significance for an instance recovery, if the error that led to the restore and recovery operation occurred during a long background processing job without a commit?

Tape Management

2 Internal tape selection

2.1 What happens if you use command brbackup -i force -v <tape name> and enter the same tape name to re-initialize a tape that has already been initialized under that name and used before in the backup cycle?

2.2 Can you use this method to release a locked tape?

3 Compare the BRBACKUP command option SCRATCH with the tape name SCRATCH.

3.1 What happens when BRBACKUP or BRACHIVE is started using the option -v SCRATCH and a tape with an arbitrary name is used? (NOTE: tape name is not SCRATCH.)

3.2 What happens if you start the SAP utilities BRBACKUP or BRACHIVE without entering a tape name (brbackup) and insert a tape initialized with the name SCRATCH (brbackup -i -v SCRATCH)?

4 Initialize a tape for BRBACKUP.

Backup Strategy and Tape Management: Solutions

No. Solution

Backup Strategy

1

1.1 The problem when using this backup strategy is that only one copy of the archived redo log files is written to tape before deletion. We recommend that at least two copies are written to different backup media. The data is distributed automatically by BRBACKUP across the tape devices so that the backup can be performed in unattended mode, even if the files are not compressed.

1.2 If the data volume is distributed over the three backup media, each tape will contain approximately 33 GB. At a read rate of 6 GB per hour, a restore operation would take approximately 5½ hours. 1 GB of offline redo log files can be restored in 10 minutes. It takes approximately three minutes to apply one redo log file to the database. Therefore, it would take approximately 150 minutes to apply all offline redo log files from one day.

To restore and recover the database would take up to 8 hours and 10 minutes. However, this time does not include the time to ana lyze and repair the error that led to the restore. Additionally, the time of the instance recovery that is performed at system startup is not accounted for in this calculation.

Due to the times not accounted for in the calculation, it is unlikely that the full restore and recovery can be performed in 8½ hours.

1.3 An uncommitted transaction has to be rolled back during instance recovery. Therefore, the database needs more time to complete the recovery.

Tape Management

2

2.1 When a tape that is already integrated into the backup cycle is re-initialized, the label of the tape label will be reset including the information about the number of times the tape has been used. Therefore, the tape may be used more often than the recommended number of times.

2.2 A tape will remain locked with a logical lock but not with a physical lock.

The information about which tape should be used is contained only in the database tables SDBAH and SDBAD. These database tables are not reset when you re-initialize the tape with the option -i force.

However, the label of the tape is reset. Therefore, there is no date recorded on the tape when it was last recently used and no the tape is not locked with a physical lock.

3

3.1 The lock expiration of the tape is checked.

If the tape is not locked, it is used regardless of its name.

The name of the tape will not be changed.

This option can be used for an unscheduled backup or if you want to define

the tape name using an external tool. When you use an external tool, for example, if you want the name to contain the correct day of the week, switch off the automatic tape administration. Do not use the same tape name twice in a backup cycle, and do not use the tape name SCRATCH.

3.2 The lock expiration of the tape is checked.

If the tape is not locked, it is used.

The name of the tape is changed to the name of the tape requested from the backup cycle by BRBACKUP or BRACHIVE.

The tape name SCRATCH can be used when a tape needs to be replaced, for example when the tape_use_count is exceeded or when the tape is defective. Note that you must remove the old tape from your tape pool to ensure that tape names are unique in the pool.

4 Copy the init<SID>.sap.tape file to init<SID>.sap

brbackup –i force –v <SID>B01

SAP AG 2000






Scheduling, Performing, and Monitoring Backups

SAP AG 1999

Scheduling, Performing, and Monitoring Backups

Contentsl Database backupsl Offline redo log file backups

ObjectivesAt the end of this unit, you will be able to:l Schedule and perform a backup using SAP toolsl Monitor and verify a backup runl Evaluate, implement and test the most suitable backup method

for your company

SAP AG 1999

BRBACKUP

Data files

BRBACKUP

Data files

Log filesLog files................ ............ ............

BRARCHIVE


BRARCHIVE


....

....

....

....

init<SID>.sapsapr3.SDBAH

sapr3.SDBAD


DBA Planning CalendarPlanning Goto Listing Help System

MONMON TUETUE WEDWED THUTHU FRIFRI SATSAT SUNSUN




BRBACKUP -t ...

or

BRARCHIVE ...

Command prompt- x

How SAP Backup Tools Work Together

n A backup can be called:

� From CCMS in R/3 – for periodic actions. To schedule periodic backups, use the DBA Planning Calendar (transaction DB13). In R/3, the required tapes can be determined, and the logs displayed.

� Using SAPDBA – for one-time actions and exceptional cases. Starting from the command prompt, you can use SAPDBA to administer the Oracle database. With this program, backups are started interactively.

� Using BRBACKUP or BRARCHIVE – for one-time actions and exceptional cases. You can call the SAP tools for database backups (BRBACKUP) and offline redo log file backups (BRARCHIVE) at the command prompt level. To schedule such backups, you can use operating system commands (UNIX: cron, NT: at).

n When a backup is scheduled by CCMS, or started up by SAPDBA, these tools call BRBACKUP and/or BRARCHIVE in order to back up the files to tape or disk, and to log backup actions in database tables SDBAH and SDBAD. Each tool also backs up a summary log and a detail log per action. Default values are read from the parameter file init<SID>.sap. However, with these backup alternatives, some of the default values can be overridden. For internal tape administration, BRBACKUP determines the required tapes from tables SDBAH and SDBAD, while BRARCHIVE does so on the basis of its summary log.

SAP AG 1999

????

?

?

....

....

....

....

init<SID>.sap

compresscompress = hardware= hardwarecompress_compress_cmdcmd = “compress -b 12 -c $ > $”= “compress -b 12 -c $ > $”compress_compress_dirdir = /oracle/<SID>/= /oracle/<SID>/sapreorgsapreorgtape_copy_tape_copy_cmdcmd = = dddddisk_copy_disk_copy_cmdcmd = = rmanrmanexec_parallelexec_parallel = 0= 0tape_addresstape_address = /= /devdev//rmtrmt/0mn/0mntape_address_tape_address_rewrew = /= /devdev//rmtrmt/0m/0mtape_address_archtape_address_arch = /= /devdev//rmtrmt/1mn/1mntape_address_tape_address_rewrew_arch_arch = //devdev//rmtrmt/1m/1mbackup_ modebackup_ mode = all= all

backup_ typebackup_ type = online [offline]= online [offline]volume_backupvolume_backup = = (<SID>B01, <SID>B02, ...)(<SID>B01, <SID>B02, ...)tape_ sizetape_ size = 32G= 32Gtape_use_counttape_use_count = 100= 100expirexpir_period_period = 28= 28backup_backup_devdev_type_type = tape= tapearchive_functionarchive_function = copy_delete_savevolume_archivevolume_archive = (<SID>A01, <SID>A02, ...)tape_size_archtape_size_arch = 6000M

Backup Profile Parameters

n To choose the tape drives for the tape stations used for database or offline redo log file backups, set parameters tape_address and tape_address_rew. The optional parameters tape_address_arch and tape_address_rew_arch are used to specify one (or two) tape drives for the tape station used for offline redo log file backups. When the offline redo log file backup parameters have been set, tape_address and tape_address_rew are only used for the database backup.

n The parameter tape_copy_cmd determines whether copy program cpio or dd is used to back up the data files to tape. Using dd, the required backup time can be reduced significantly. If you use dd, you must maintain the following parameters (for NT, “obs=nk” or “ibs=nk ” do not apply):

dd_flags = “obs = nk bs = nk” with n 16 (for DLT, for example n = 32 or n = 64)

dd_in_flags = “ibs = nk bs = nk” with n 16 (for example, dd_in_flags = “ibs = 64k bs = 64k ”)

n The parameter compress_dir indicates which directory is being used with verify or software compression during a backup.

n The parameters compress_cmd and exec_parallel are discussed on the slide “Hardware Compression.”

n The init<SID>.sap parameters shown in smaller print are not discussed here. These profile parameters are only examples; they may differ on your system. For further parameter definitions, see the R/3 Online Documentation.

SAP AG 1999

BRBACKUP/BRARCHIVE

cpio/dd

Data . ..

dd: Error

cpio: Error or cpio continuation

volume

Data Data

cpio/dd cpio/dd

.. .

BRBACKUP/BRARCHIVE

cpio/dd

Data ... Data

cpio/dd cpio/dd

. ..BRBACKUP:

SAP follow-up

tape

Correct

Parameter tape_size

Physical tape size

Parameter tape_size

Data

Physical tape size

Profile init<SID>.sap Parameter tape_size

Incorrect

n The data volume to be backed up is determined by BRBACKUP/BRARCHIVE, and distributed among SAP tapes, using parameter tape_size. If a tape change is required, use an SAP follow-up tape (for cpio, this is called a continuation volume). The files are not split; they are backed up to tape in one piece. During an offline redo log file backup, the maximum number of offline redo log files that can fit on this tape (as defined by tape_size or tape_size_arch) is backed up. An SAP follow-up tape is not used.

n If the value for tape_size is too large, the copy program (cpio or dd) may start backing up a file to tape, and may reach the physical end of the tape. The consequences depend on the copy program and the type of backup.

n The copy program dd always generates an error message when it reaches the end of the tape. The error message depends on the operating system. For Windows NT, it reads Physical end of tape has been reached. For UNIX, it reads I/O Error.

n During a serial database or offline redo log file backup, cpio requests a cpio (not an SAP) continuation volume. The database backup terminates successfully. Caution: Since SAP tools cannot request the cpio continuation volume directly, problems may arise during a restore from this database backup.

n During a parallel database or offline redo log file backup, cpio terminates with an error message, and the backup terminates with an error.

n Therefore, tape_size must be set to a value somewhat smaller than the physical tape capacity (allowing, for example, a 10% safety margin).

SAP AG 1999

400 MB 400 MB 400 MB 400 MB

BRBACKUP400 MB400 MB400 MB

400 MB

compresscompress = no= notape_sizetape_size = 1800M= 1800M

init<SID>.sap

400 MB

400 MB

Tape station withouthardware compression

200 MB

200 MB

200 MB

200 MB

200 MB

400 MB

BRBACKUP

200 MB

200 MB

200 MB

200 MB

400 MB400 MB

400 MB

400 MB400 MB

400 MB400 MB400 MB

compresscompress = hardware= hardwarecompress_compress_cmdcmd = “compress -b 12 -c $ > $”= “compress -b 12 -c $ > $”exec_parallelexec_parallel = 0= 0tape_sizetape_size = 1600M= 1600M

................

init<SID>.sap

Tape station withhardware compression

Once per cycle:Determine compression rate

400 MB 400 MB400 MB400 MB

Hardware Compression

n For tape stations with hardware compression, parameter tape_size is set to a smaller value (as an additional safety margin, since the compression rate changes in the course of a backup cycle) than for tape stations without hardware compression. For more information on setting tape_size for different tape stations, see SAP Note 8707.

n To be able to distribute files across the tapes, BRBACKUP requires information on how well the files to be saved are being compressed by the tape station. However, tape stations do not report a compression rate. You must therefore determine the compression rate once per backup cycle. Determine this rate by using SAPDBA, or by using the operating system command brbackup -k.

n To determine the compression rate, for UNIX, we recommend that you set parameter compress_cmd as shown above in order to obtain more precise values. For NT, the appropriate value for compress_cmd is given in the R/3 Online Documentation.

n If parameter exec_parallel has been set to 0 during compression rate determination, one process per logical volume is triggered in order to determine the compression rate. If parameter exec_parallel has been set to a value smaller than the number of logical volumes, the number of processes required to determine the compression rate is limited to the number indicated by the parameter. This reduces the CPU load on the database server.

SAP AG 1999






BRBACKUP -t online -d tape

-m all -c -u system

Command prompt- x

Scheduling and Performing a Normal Database Backup

tape_copy_tape_copy_cmdcmd = = dddd orortape_copy_tape_copy_cmdcmd == cpiocpio

................

init<SID>.sap

1 Calendar

Schedule an Action for Tue 05

18:00:00Start time

Period (weeks):

Full database offline + redo log backup

Full database offline backupFull database online + redo log backup

Full database online backupRedo log backup

Partial database offline backupPartial database online backupCheck optimizer statisticsAdapt next extentsCheck database

h - Backup databaseSAPDBA- x

c - Backup device type taped - Objects for backup alle - Backup type onlineg - Query only no

S - Start BRBACKUP

SAPDBA- x

n If possible, schedule all periodic database backups in your backup strategy using the DBA Planning Calendar (transaction DB13). For all further database backups, use SAPDBA, or call BRBACKUP using the command prompt.

n SAP recommends that you perform one online database backup each working day. Schedule this backup for periods with low system activity.

n To schedule a full backup with redo backup from CCMS, the “one-run” strategy is used. This strategy is discussed at the end of this unit.

n To start a database backup with SAPDBA, select h - Backup database. Here, you can temporarily override the parameters that have been preset for this backup by the parameter file (default: init<SID>.sap). The parameter file is not changed. For example, to select another type of backup such as offline or offline_force, select e - Backup type. You can then start the backup using s - Start BRBAKUP.

n For purposes of internal tape administration, to determine which tapes are required for the backup parameters that are currently set, select g - Query only (select the setting with/without tape check). The query begins with S - Start BRBAKUP.

n If you call BRBACKUP from the command prompt, you can temporarily override the parameters that have been preset by the parameter file (default: init<SID>.sap), by using the call parameters. The parameter file is not changed. To display the list of call parameters, use the command prompt brbackup -h | more, or check the R/3 Online Documentation.

SAP AG 1999

Phases of a Whole Database Backup

Save control file to diskSave control file to disk

Back up saved control fileBack up saved control file

For all tablespaces to be backed up:Begin tablespace backup modeBack up tablespace data filesEnd tablespace backup mode

For all tablespaces to be backed up:Begin tablespace backup modeBack up tablespace data filesEnd tablespace backup mode

Log file switchLog file switch

Back up control fileBack up control file

Start databaseStart database

Shut down databaseShut down database

Back up data filesBack up data files

*Back up online redo log files*Back up online redo log files

Offline*Start database*Start database

*Shut down database*Shut down database

Retrieve file names of data and online redo log files from databaseand retrieve names of control files from init<SID>.ora

Retrieve file names of data and online redo log files from databaseand retrieve names of control files from init<SID>.ora

Back up tape header, init<SID>.sap, init<SID>.dba, and init<SID>.oraBack up tape header, init<SID>.sap, init<SID>.dba, and init<SID>.ora

Back up reorg.log, struct.log, detail log, and summary logBack up reorg.log, struct.log, detail log, and summary log

Online

n Some, but not all, steps of offline and online backup procedures are identical.

n During an offline backup, the steps marked above with an asterisk do not necessarily have to be performed. In this way,

The DB is only started if it is shut down at the start of the offline backup.

The online redo log files are only backed up during a complete database backup.

The DB is only shut down if it was shut down at the start of the offline backup.

n During an online backup, the tablespaces are set one by one to Begin Backup Mode or End Backup Mode. Therefore, when a data file is backed up, the associated tablespace is in the backup mode. If all data files have been backed up, and all tablespaces have been reset to End Backup Mode, a log file switch is performed. During a subsequent offline redo log file backup, all offline redo log files required for consistency of the online backup can be backed up.

n During an online backup, the control file cannot be backed up to tape during normal database operation. Therefore, at the start of the backup, a consistent copy of this control file is made to disk. This copy is backed up to tape after all data files have been backed up.

n At the start of a backup, a tape header is written. By reading this tape header at the end of the backup, BRBACKUP checks whether the tape header was able to be written correctly to tape.

n This strategy does not use RMAN (Recovery Manager).

SAP AG 1999

Data files compress_dir

BRBACKUP -c -w use_dbv

AB

N..A ?

Corruption

Tape readable?


Oracle8-KBBlock=

?File

length

Logical Verification of a Database Backup

Once per week.Minimum: onceper cycle

n When the tape header is checked, the tape station also undergoes a minimal function check. Although problems occurring during a backup are revealed by the tape header check, these problems are not detected until the end of the next backup performed on this tape station, at the earliest. Systematic tape station errors cannot be detected in this way. To confirm tape readability, check backups regularly using the option verify or -w.

n Corrupt data blocks are only detected when Oracle processes access these data blocks. A database backup may therefore contain corrupt blocks. We recommend that you check a complete database backup for corrupt data blocks once each week, or at least once each backup cycle. During a database backup, use the option -verify use_dbv or -w use_dbv to:

� Ensure tape readability

� Detect corrupt data blocks

n The files are read from tape, and copied to the directory defined by the init<SID>.sap parameter compress_dir. BRBACKUP checks whether the file read from tape is the same length as the one that was backed up (file length is specified by the single backup logs). In addition, every data file is checked for corrupt data blocks, using the Oracle utility DB_VERIFY. If the BRBACKUP log reports corrupt data blocks, see SAP Note 99962.

n Caution: A backup performed using verify takes at least twice as long as a backup performed without verify. You can therefore defer a verify with BRRESTORE. You can run a deferred verify on the database server or another server.

SAP AG 1999

Data files compress_dir

BRBACKUP -t offline -c -w

AB

N A.. . ..

Checkdatabasebackup

If possible:Once each cycle

=?Offline(binary)

Physical Verification of Offline Database Backups

n During a database backup with the option -verify use_dbv, BRBACKUP checks whether the backup contains corrupt data blocks. However, it does not check whether the files read from tape are identical to the corresponding files in the database. (During an online database backup, these files may also differ.)

n During an offline database backup using the option -verify or -w, the restored files are compared at the binary level to the corresponding files in the database. If a database backup has been terminated using verify, and no error message has appeared, all files were readable, and were identical to the corresponding files in the database. Such a database backup takes at least twice as long as a backup performed without verify. If the required time window for the offline backup with verify is available, we recommend that you perform an offline database backup using verify once each backup cycle.

n A binary verify cannot be deferred using BRRESTORE.

SAP AG 1999

sapr3.SDBAHsapr3.SDBAD

Log filesLog files................

............ ............






cd /oracle/<SID>/sapbackup

cat back<SID>.log | more

cat b<timestamp>.and | more

Command prompt- x

L - Show/CleanupSAPDBA- x

a - Show log files / profilesSAPDBA- x

e - BRBACKUP log filesSAPDBA- x

Monitoring a Database Backup

Database OperationsMonitor (DB24)

DBA Calendar(DB13)

n BRBACKUP writes logs to SAP tables SDBAH and/or SDBAD, and to files in directory sapbackup. Therefore, by using SABDBA or file editors, you can check in R/3 whether the backup has been performed successfully.

n In R/3, you can monitor database backups using the DBA Planning Calendar (transaction DB13). To select the compressed data from tables SDBAH and SDBAD, choose the appropriate action. You can branch to the detail logs at the operating system level.

n For an overview of all database backups that have been performed, use the DBA Operations Monitor (call transaction DB24). In the DBA Operations Monitor, double-click the backup run, then choose Display action logs.

n Using SAPDBA, you can display logs from database backups. Select l - Show/Cleanup, a - Show log files / profiles, e - BRBACKUP log files.

n The BRBACKUP logs are at the operating system level in directory sapbackup. For every BRBACKUP action, the summary log back<SID>.log contains an entry with the date and name of the corresponding detail log. The detail logs b<time stamp>.<ext> contain a complete description of BRBACKUP activity. The file suffix <ext> depends on the BRBACKUP function selected. The summary log and detail logs can be viewed using commercial editors or operating system commands.

n With the option brbackup -o dist|time[,time|dist], additional information is entered into the detail log. See also the database handbook in the R/3 Online Documentation.

n Backups that have been terminated can be completed. In the unit “Advanced Backups,” see the slide “Partial Database Backups.”

SAP AG 1999

BRARCHIVE option -cds (copy, delete, save)Status of an offline redo log file

42

42

42

../saparch

<SID>A01<SID>A01

<SID>A02<SID>A02

ARCHIVED

DELETED

SAVED

COPIED

../saparch

<SID>A01<SID>A01

<SID>A02<SID>A02

43

42

44

46

45

<SID>A03<SID>A03

42

44

43

42

BRARCHIVE -cds

Mon Tue Wed

42

42

42

43

44

42

43

44

45

46

Offline Redo Log Files: Status and Option -cds

n After a log file switch, the Oracle process ARCH copies the online redo log file that was the current redo log file before the log file switch to directory saparch. An offline redo log file generated in this way can have various statuses for BRARCHIVE. These statuses are always listed and updated in summary log arch<SID>.log after a BRARCHIVE run.

n During a backup to tape, an offline redo log file, when generated, has the status ARCHIVE. (This status is not displayed until the offline redo log file is backed up for the first time.) At first save, the file status is SAVED; the second time, it is COPIED; and after deletion, it is DELETED.

n During a backup to disk, an offline redo log file, when generated, has the status DISK. (Again, this status is not displayed until the offline redo log file is backed up for the first time.) A second copy is not supported. The only statuses here are DISKSAV (first save to disk) and DISKDEL (deletion after a save to disk).

n Do not mix tape and disk backups.

n BRARCHIVE has a series of call options that determine how the offline redo log files are processed. SAP recommends the option -cds because: (1) If an offline redo log file has the status SAVED, it is saved to tape for a second time, and subsequently deleted from disk. This procedure is repeated until no offline redo log file with status SAVED is found. Next, all offline redo log files with status ARCHIVE are backed up to tape for the first time. (2) After the backup, all offline redo log files exist at two locations: either (a) in saparch and on tape, or (b) on two different tapes. Thus, you can achieve a high fail-safe rate without drastically increasing the tape requirement.

SAP AG 1999

DBAPlanning CalendarPlanning Goto Listing Help System





brarchive -cds -d tape

-c -u system

Command prompt- x

1 Calendar

Schedule an Action for Tue 05

18:00:00Start time

Period (weeks):




Partial database offline backupPartial database online backupCheck optimizer statisticsAdapt next extentsCheck database

i - Back up offline redo log files

SAPDBA- x

a - Archive function Copy, delete, andsave archive logs

c - Archive device type tape

s - Start BRARCHIVE

SAPDBA- x

Performing Offline Redo Log File Backups

DB13

n To schedule all periodic offline redo log file backups that are part of your backup strategy, use the DBA Planning Calendar (transaction DB13). Back up offline redo logs every day.

n You can start an offline redo log backup using SAPDBA by selecting i - Backup offline redo logs. To select another type of backup, such as Copy, delete and save offline redo logs, select a - Archive function. To start the backup, select s - Start BRARCHIVE.

n If you call BRARCHIVE using the command prompt, the parameters that have been preset using the parameter file (default: init<SID>.sap) can be temporarily overridden using call options. To obtain the list of call options, see the R/3 Online Documentation.

n To ensure that the tapes are readable, check backups regularly using the option -verify or -w.

n SAP recommends that you perform an offline redo log file backup using verify once per backup cycle. If the time window required for the offline redo log file backup using verify is available, the tape readability check should be performed during each offline redo log file backup.

n If you start BRARCHIVE by choosing -f (Fillup), all offline redo log files in saparch are initially backed up according to the selected backup function. BRARCHIVE then periodically checks newly generated offline redo logs and writes them to tape until the tape is full.

n To cancel brarchive -f, use the command brarchive -f stop only. Never use ‘Ctrl + C’ or the kill command (UNIX).

SAP AG 1999



............ ............






cd /oracle/<SID>/saparch

cat arch<SID>.log | more

cat a<timestamp>.cds | more

Command prompt- x

L - Show / Cleanup

SAPDBA- x

a - Show log files / profiles

SAPDBA- x

f - BRARCHIVE log files

SAPDBA- x

Monitoring Offline Redo Log File Backups

Database OperationsMonitor (DB24)

CCMS Calendar(DB13)

DB13

n BRARCHIVE writes logs to SAP tables SDBAH and/or SDBAD, and to files in directory saparch. Therefore, you can check in R/3 whether the offline redo log file backup was performed successfully, by using SAPDBA or file editors.

n In R/3, you can monitor offline redo log file backups using the DBA Planning Calendar (transaction DB13). To select the compressed data from tables SDBAH and SDBAD, choose the action you want. You can branch to the corresponding detail logs at the operating system level. In addition, transaction DB24 provides an overview of all executed offline redo log file backups.

n Using SAPDBA, you can display logs for the offline redo log file backups. Select

l - Show/Cleanup, a - Show log files / profiles, f - BRARCHIVE log files.

n The BRARCHIVE logs are located at the operating system level in directory saparch. The summary log arch<SID>.log specifies which offline redo log file was backed up using what action to which tape. The detail log a<time stamp>.<ext> contains a complete description of BRARCHIVE activity. The file suffix <ext> depends on the BRARCHIVE function selected. The summary log and detail logs can be viewed using commercial editors or the appropriate operating system commands.

n With the option brarchive -o dist|time[,time|dist], additional information is entered in the detail log. For more information, see the database handbook in the R/3 Online Documentation.

SAP AG 1999

sapr3.SDBAHsapr3.SDBADexpirexpir _period_SAPDBA_normal_period_SAPDBA_normal = 11= 11

expirexpir _period_daily_check_period_daily_check = 5= 5expirexpir _period_BRBACKUP_period_BRBACKUP = 30= 30expirexpir _period_BRARCHIVE_period_BRARCHIVE = 30= 30expirexpir _period_oracle_trace_period_oracle_trace = 1= 1

................

init<SID>.dba

delete b<timestamp>.and

Command prompt- x


............ ............

L - Show/Cleanup

SAPDBA- x

b - Cleanup log files / directories

SAPDBA- x

a - SAPDBA log files and dump directoriesb - SAPDBA daily check log files c - BRBACKUP log filesd - BRARCHIVE log files

f - ORACLE traces and audits

SAPDBA- x

Log File Cleanup

n The log files of these SAP tools are written to the corresponding files at operating system level and to the backup tapes. In case of damage to the database, SAPDBA takes repair actions based on these log files.

n However, you must delete log, trace, and audit files regularly, especially those that are generated by the database. If you do not do this, the file systems overflow. As a result of cryptic file names, logs that are still needed can easily be deleted accidentally. Therefore, you should not delete these files using operating system commands. To use the SAPDBA delete function, select l - Show/Cleanup and b - Cleanup log files / directories. This choice refers to logs in directories sapreorg, sapcheck (both SAPDBA logs), sapbackup (BRBACKUP), saparch (BRARCHIVE), saptrace/background, saptrace/usertrace, and <ORACLE_HOME>/rdbms/audit (all of which are Oracle). SAPDBA's default values for the minimum age of a log before deletion is permitted are derived from the SAPDBA configuration file init<SID>.dba. The parameters expir_period_* represent lower limits, from which deletion is permitted using SAPDBA. You should adapt these limits to the backup cycle used.

n SAPDBA simultaneously deletes the corresponding data in data table SDBAD. The entries in header table SDBAH are retained. BRBACKUP deletes entries older than 400 days from tables SDBAH and SDBAD. (The operating system logs are not deleted.)

n SAPDBA can be called in the command prompt mode using SAPDBA -cleanup. All directories are then cleaned up, according to the parameters in init<SID>.dba.

SAP AG 1999

Detecting an archiver stuckMonitoring the directory saparch

f - Archive mode

SAPDBA- x

c - Show all archive information

SAPDBA- x

FREE SPACE : ...SAPDBA- x

cd saparch

df -k .

Command prompt- x

../saparch

alert_<SID>.log:Thread <n> cannot allocate new log,All online logs needed archiving

________________________________________

SAPGUI- x.… .… ....

Resolving an archiver stuck

Removedummy files

and

Run BRARCHIVE

<SID>A01<SID>A01

Dummy

Freespace Problems in Directory saparch

RZ20 DB24

n In the ARCHIVELOG mode, the logwriter process can override an online redo log file only if it has been backed up successfully to directory saparch, using the archiver process. Therefore, for a database in the ARCHIVELOG mode, you must make sure that the archiver process is active, and that sufficient freespace is available in saparch. You must monitor freespace daily.

n To monitor the freespace in saparch, use transaction DB12. This transaction also shows an overview of the backup status of the offline redo log files. In SAPDBA, this freespace is displayed in the saparch directory under menu option f - Archive mode, c - Show all archive information. You can also check the freespace using operating system commands.

n If the logwriter process attempts to switch to the next online redo log file, and this online redo log file has not been backed up by the archiver process, the database waits. This situation is called archiver stuck (see also the unit “Top 10 Problems”). A possible cause is missing freespace in saparch. The database alert log then contains error messages. To resolve the archiver stuck , the offline redo log files must be backed up to tape using the standard copy function, and then deleted from directory saparch. To do so, use SAPDBA or, if SAPDBA can no longer log on to the database, use BRARCHIVE at the command prompt level.

n TIP: Place a dummy file in directory saparch. In case of an archiver stuck , the dummy file is deleted, the archiver again backs up offline redo log files, and the database keeps running for a short time. This interval is sufficient to enable you to start offline redo log file backups as usual using SAPDBA.

SAP AG 1999


l Describe the SAP backup tools

l Maintain the appropriate profile parameters

l Schedule, perform, monitor, and verifydatabase backups and offline redo log file backups

l Clean up the log files written during a backup

Unit Summary

SAP AG 1999

Further Documentation

l Knowledge Product CD

n SAP Database Administration Oracle

l Online Documentation

n Database Administration: Oracle

l SAPNet

n SAP Notes 19909, 8707, 33630, 99962,23345, 100400, 13446, 128726, 127395,126694, 106497, 121727

SAP AG 1999

l Exercises?

l Solutions

Unit Actions

Performing Backups: Exercises

No. Exercise 1 Database Backups

This exercise teaches you to use SAP database backup tools. The exercise refers to the play databases, and is limited to the operating system level.

1.1 Using SAPDBA, perform a backup that meets the following criteria: Whole Offline To disk Without software compression With a binary verify

1.2 Perform a backup that meets the following criteria: Whole Online To disk Without compression Without using verify Use either SAPDBA or BRBACKUP in the command prompt mode.

1.3 Confirm that your backup has been performed correctly by checking the log files at the operating system level, and by using SAPDBA. Where exactly has the data been backed up?

2 Scheduling Periodic Database Backups This exercise teaches you to use the DBA Planning Calendar in the R/3 System.

2.1 Where in R/3 can you schedule database backups periodically, and check a backup after it has been run? (Please do not schedule a backup!)

3 Backing Up Offline Redo Log Files This exercise teaches you to use SAP backup tools for offline redo log files. The exercise refers to the play databases, and is limited to the operating system level.

3.1 Using SAPDBA, perform an offline redo log file backup that meets the following criteria: To disk With a verify As an exception, use the copy function save&delete.

3.2 Confirm that your offline redo log file backup has been performed correctly by checking the log files at the operating system level, and by using SAPDBA. Where exactly has the data been backed up?

4 Optional Exercise: Logical Database Backup Check This exercise teaches you to check a logical database backup using the Oracle tool DB_VERIFY.

4.1 Check the backup you performed in exercise 1.1 for logical consistency using DB_VERIFY.

Performing Backups: Solutions

No. Solution

1

1.1 Start SAPDBA. To get to the backup menu, choose h - Backup database.

To start the backup, choose a - Backup function: Normal backup. Specify the backup device type by choosing c - Backup device type: local disk. To define which objects are to be backed up, choose d - Objects for backup: all. To define the backup type, choose e - Backup type offline. Under h - Special options, choose b – Compress: no c – Verification after backup: Binary (offline) or by size (online) To start the backup run, choose S - Start BRBACKUP. The backup run is extended significantly by a verify. Additional disk space is not required.

1.2 Solution using SAPDBA: Start SAPDBA. To get to the backup menu, choose h – Backup database. To start the backup, choose a – Backup function: Normal backup. To specify the backup device type, choose c - Backup device type: local disk. To define which objects are to be backed up, choose d – Objects for backup: all. To define the type of backup, choose e - Backup type online. Under h - Special options, choose b – Compress: no c – Verification after backup: no. To start the backup run, choose S - Start BRBACKUP. Solution using BRBACKUP: At the operating system level, enter: brbackup -c -d disk -t online -k no To set BRBACKUP to the non-monitored mode, use the option –c. Before performing this call, set the database user system password to manager (NOTE: This is only for the play database; do not use this setting at home). If

you have not done so, you must enter the password using [-u|-user [<user>[/<password>]].

1.3 The BRBACKUP logs are in subdirectory sapbackup of directory $HOME:

cd

cd sapbackup

The detail log is named b<time stamp>.<ext>, with <ext> = afd signifying full offline on disk, and <ext>=and signifying full online on disk. Using the time the log was created, select the appropriate log, and view it by entering (for example): more b<time stamp>.<ext>.

To view logs in SAPDBA, choose l - Show/Cleanup → a - Show log files / profiles e - BRBACKUP log files.

The files that have been backed up are located in subdirectories of sapbackup. The names of these subdirectories are identical to the corresponding names of the detail logs (without extensions).

2

2.1 You can schedule backups periodically from within R/3 by using the DBA Planning Calendar (transaction DB13).

To make an entry, double-click a free slot (that is, day). By entering a period (for example, 1 week) and going through the weekdays, a database administrator can quickly set up a backup strategy. For every backup entry, a corresponding background job is created. To display these background jobs, call transaction SM37.

The morning after the backup, the successful backup run should also be checked using transaction DB24 or DB13 (double-click the backup action).

3

3.1 Start SAPDBA. To get to the desired menu, choose

i - Backup offline redo logs.

To select the copy function, choose

a – Archive function → d - Save and delete archive logs.

To specify the backup device type, choose

c – Archive device type: local disk.

Under h - Special options, choose

c – Verification after backup: yes

To start the backup run, choose S - Start BRARCHIVE.

NOTE: The backed up offline redo log files cannot be checked at the binary level for all copy functions. For an overview of copy functions, see the R/3 online documentation.

3.2 The BRARCHIVE logs are in subdirectory saparch of directory $HOME:

cd

cd saparch

The detail log is named b<time stamp>.<ext>, with <ext> = svd signifying save & delete on disk. Using the time the log was created, select the appropriate log, and view it by entering (for example): more b<time stamp>.<ext>.

To view logs in SAPDBA, choose

l - Show/Cleanup → a - Show log files / profiles f - BRBACKUP log files.

The files that have been backed up are located in directory sapbackup.

4

4.1 You can perform a deferred logical check of a backup to tape, using SAPDBA, by choosing

h – Backup database → a – Backup function → f - Verify BRBACKUP tape.

The backup you ran for exercise 1.1 was performed to disk. Such backups cannot be checked from within SAPDBA; however, they can be checked using BRRESTORE at the command prompt level.

First, determine the name of the BRBACKUP detail log from exercise 1.1.

To check the log, enter

brrestore -d disk -k no -w use_dbv –b <BRBACKUP-detail log>.

NOTE: For backups to disk, DB_VERIFY can only be used for runs without software compression.

SAP AG 2000






Restore and Recovery

SAP AG 1999

Restore and Recovery

Contentsl Restore and recovery options using SAPDBA

l SAPDBA functions and their limitations

ObjectivesAt the end of this unit, you will be able to:l Analyze the physical database structure using SAPDBA

l Recover the database using the SAPDBA functionPartial Restore and Complete Recovery

l Reset the database using the SAPDBA function Reset Database

l Perform a point in time recovery using the SAPDBA functionFull Restore and Recovery

SAP AG 1999

Error types:

l Statement

l Process

l Instance

l User

l Media

Database Errors

“Most importantly, be prepared for disasters. Don’t think you will never

see a failure. Every DBA will experience a database failure. It’s just a matter of when...

Good luck.” Rama Velpuri

n In an Oracle database, various kinds of errors can occur that require the database administrator (DBA) to take action. The different types of database error are:

� Statement errors, such as an attempt to enter incorrect data in a table. Oracle terminates the statement with an error message, and performs a statement rollback.

� Process errors, such as termination of the connection between a work process and a server process. PMON rolls back the open transaction, and releases occupied resources.

� Instance errors, such as when a background process fails. The next time an instance is started, a consistent status is restored by means of automatic instance recovery.

� User errors, such as when a user accidentally enters the SQL statement drop table.

� Media errors, such as a head crash, or accidental deletion of a database file.

n If user and media errors occur, the DBA must take action. This unit describes both of these error types and the appropriate repair scenarios.

n To help you determine exactly when to apply Oracle tools, the SAPDBA functions and their limitations are discussed in detail in this chapter.

n For further information about database errors, see the R/3 Online Documentation on database administration.

SAP AG 1999

Control files10

1010

38

3838

Offline and online redo log files11 12 37 38. . .

Data files10

10

10

38

38

Full backup

10 9 36

38

Legend:Log sequence number 9

Scenarios: Introduction

n In an R/3 System with an Oracle database, most data files have the status online and read/write. For a functioning, consistent database, all data files as well as the control files must therefore be synchronous – that is, they must match in terms of time.

n Oracle creates synchronization data using time stamps. Time stamps are integers that are increased during certain database actions, and entered in all data and control file headers by the log writer or checkpointer process at the checkpoint.

n An example of synchronization data is the log sequence number (LSN), which is increased by 1 during every log switch. At a more sophisticated level, Oracle can define synchronization on transaction level using the system change number (SCN), which is increased, for example, after the COMMIT in a change transaction, or at the checkpoint.

n The scenario shown in this slide (and the following three slides) depicts a database that has been saved completely and error-free at time LSN=10. At time LSN=38, a media or user error damages the database so extensively that the database instance breaks down or the database is inconsistent. The offline and online redo log files that were created between the beginning of the backup and the occurrence of the error are available.

SAP AG 1999

38

3838

38

38

38

38

3838

10

38

10

Partial restore

10 11 37 38. . .

Complete recovery

39 . . .

mount open

Scenario: Partial Restore and Complete Recovery

Control files

Offline and online redo log files

Data files

n Scenario: Because of a head crash, data has been lost during business operation. The database is inconsistent, and is no longer running properly.

n A partial restore and complete recovery is performed to get the database running properly again, and to recover the database to its status just before the error occurred.

n During a restore, database files are copied from the backup medium back to the disk. During a partial restore and complete recovery, only the required minimum of data is copied. The database files that are to be copied back can be combined from different backups. Because the database files are no longer synchronous after a partial restore, the database is inconsistent and will not run properly after the copy-back procedure has terminated.

n To synchronize the files, the database evaluates the synchronization data that has been saved in the file headers. The database requests all offline redo log files that have accumulated since the “oldest” database file (in logical terms), in uninterrupted sequence. During a recovery, all data changes logged by these offline redo log files are replicated in the files that have been copied back. With a partial restore and complete recovery, all changes are performed again until all data files are at the same SCN (a procedure called media recovery). When the database is subsequently started up, during the instance recovery, all transactions that are not committed are taken back, using the rollback segments, which are likewise recovered. The database is now consistent, capable of running, and back to its data status just before the error occurred.

SAP AG 1999

Full restore

10

1010

10

10

10

910 11 37 38. . .

10 11 . . .

open

Scenario: Database Reset

Control files


Data files

n Scenario: During an upgrade, extensive software or hardware problems have arisen. As a result, the upgrade must be terminated. The database is inconsistent, and is no longer running properly. Fortunately, a full offline backup was performed immediately before the upgrade.

n A database reset is performed to get the database running properly again, and to reset the database to its status immediately before the upgrade.

n The database is reset using a full restore. With a full restore, all data files as well as the older online redo log files and control files are copied back from the backup medium. Since these files must originate from the same valid offline backup, the database is consistent and ready to run after the copy-back procedure has terminated. Therefore, a recovery is not required; the database can be started up at once.

n After the database startup, new offline redo log files are generated which, at the technical level, “fit” the full backup as precisely as the old offline redo log files. If an additional full restore is required, you risk making a recovery possible in two different logical directions.

n A database reset always results in data loss. The data that has been generated between the full backup and the problem situation is lost. Of course, the database as such does remain consistent.

SAP AG 1999

10

1010

10

10

10

10 11 25. . .1 2 . . .

26 37. . .

mount Open resetlogs

24

24

25

2525

25

25

25

1

11

1

1

1

Incompleterecovery

Scenario: Point in Time Recovery

Control files


Data files

Full restore

n Scenario: During an upgrade, a user accidentally enters the command drop table. As a result, the upgrade must be terminated. A full backup is available, but was not performed immediately before the upgrade process began.

n A point in time recovery is performed to get the database running properly again, and to reset the database to the status at a certain point in time before the upgrade. From that point on, the data is recovered up to a certain point, for example, up to the start of the upgrade, or up to the table drop.

n Initially, all data files are replaced by copies of online/offline backups, using a full restore. The termination point of the recovery determines whether the control files should also be replaced. All data files and online redo log files are entered in the control files with specified paths. The control files must reproduce this file structure at the operating system level according to the status of the structure at the end of the recovery procedure.

n During the recovery phase, the changes to the dataset are performed again. Incomplete recovery refers to the end point of recovery, which can be anywhere between the end of the copied backup and the last entry in the current online redo log. The recovery end point can be defined by the redo log file sequence number, or by specifying either a point in time or an SCN.

n After a point in time recovery, the database is normally opened using alter database open resetlogs, unless a complete recovery is performed. Since a recovery cannot be performed after using open resetlogs, a whole backup must be triggered immediately.

SAP AG 1999

How to Handle Problems

l Do not make any rash decisions

l Analyze the problem in detail

l Create a problem-solving strategy

l Before restoring any files, check:

n What is causing the problem

n Whether there is enough disk space to save and restore files

n Whether a hardware extension is necessary

n The file system and mount points

n The availability of backups

n The availability of offline redo log files

n If a database problem occurs, you must analyze the problem and create a problem-solving strategy.

n To analyze the database problem, check the database alert log and the trace files belonging to the background processes in directory $ORACLE_HOME/saptrace/background.

n Your problem-solving strategy depends on the answers to the following questions: What is the status of the database – available or not available? Is this a user error or a media error? Which files are corrupted? Which file types (data files, control files, online redo log files) are affected? Is software or hardware mirroring available?

n To be on the safe side (and time permitting), perform a full offline backup before the files are copied back, using BRBACKUP or operating system (OS) backup tools.

n In the event of a hard disk problem, such as a head crash, hardware must be replaced. In this unit, we assume that, at the OS level, a file system has been created and mounted at the old location.

n If you followed the backup cycle recommended by SAP, you will have a number of database backups and offline redo log file backups for a restore and recovery. Your problem-solving strategy will determine which backup and offline redo log files are copied back, and how they need to be applied.

n Do not make any rash decisions . If you make mistakes or act carelessly, you can drastically aggravate the restore and recovery situation. The costs incurred by a consulting session provided by SAP or an SAP partner are negligible compared to the business consequences of data loss, even for a single day of production operation.

SAP AG 1999

Detail logs

back<SID>.log arch<SID>.log Recoveryscript

saparch

Find offlineredo log files

Restoredata files

Recoverdatabase

Restore offlineredo log files

Findbackups

CheckDatabase

Database

Partial Restore and Complete Recovery (1)

n The SAPDBA function partial restore and complete recovery replaces lost data files by using appropriate backups, and subsequently recovers the restored data file status using redo log files. To be able to use this function, your online redo log files and control files must be valid. The partial restore and complete recovery procedure consists of six phases that are executed either manually or automatically, in a predetermined sequence – that is, a particular phase can only be selected after the previous one has been completed successfully (status: finished or not needed).

n In the Check Database phase, the status of all files in the database (that is, the control files, online redo log files, and data files) as well as the tablespace status (online/offline; online backup mode) are checked. Check Database can be executed regularly with the database running; thus, it provides an overview of the physical status of the database.

n In the Check Database phase, SAPDBA refers to entries in Oracles V$Views (such as V$DATAFILE, V$RECOVER_FILE). If an error is detected during this phase, a safe check must be performed – that is, the database must be shut down (initially using shutdown immediate; if this is unsuccessful, SAPDBA suggests shutdown abort). Next, to update the V$Views, the database is set to status mount. SAPDBA logs any recorded errors in data files in directory sapreorg with the suffix (rcv) for recovery. A safe check is a prerequisite for any subsequent restore and recovery activities.

n Missing sapdata directories are not created automatically; rather, they are mount points. However, missing subdirectories are created automatically.

SAP AG 1999

Detail logs

back<SID>.log arch<SID>.log Recoveryscript

CheckDatabase


Restore backup files

Recoverdatabase

Restore offlineredo log files

Findbackups

saparch

Database

Partial Restore and Complete Recovery (2)

n In the Find Backup Files phase, backups are determined using the entries in the BRBACKUP summary log file back<SID>.log (return code 0 or 1). The associated detail logs show whether the required data files were in the backup. The data files can be compiled from various backups. To minimize the subsequent recovery time, SAPDBA always suggests the most recent backup.

n In the Restore Backup Files phase, the data files are restored to their original location. If only index files are missing, SAPDBA can recreate and build up these files using Database Dictionary information.

n In the Find Offline Redo Log Files phase, the offline redo log files required for a complete recovery are determined. The BRARCHIVE summary log file arch<SID>.log lists the tapes where the offline redo log files have been saved. You can choose between a first or second backup (for example, when saved, with brarchive -cds). SAPDBA takes existing online redo log files and offline redo log files in saparch into consideration. After the appropriate backups have been found for all required offline redo log files, the Find Archive Files phase ends with the status finished.

n In the Restore Offline Redo Log Files phase, the offline redo log files that have been found are read (from tape) back to directory saparch.

n In the Recover Database phase, SAPDBA creates recovery scripts in a subdirectory of sapreorg. Using these scripts, a control file is saved, and a recover database statement (that is, a complete recovery) is transmitted to Oracle. The SAPDBA message Recover database terminated successfully indicates that the database has been recovered completely.

SAP AG 1999

Partial Restore and Complete Recovery Limitations

Logs

No data or offline redolog file backups available

No BRBACKUP/BRARCHIVElogs available

Control files damaged

Online redo log files damaged

init<SID>.* No init<SID>.*files available

Problem Solutions

l Perform a database resetl Perform a point in time

recovery

l Use the SAPDBA functionRestore individual files

l Restore these files fromtape using commandbrrestore -n init_ora

l Copy one of its mirrors

l See R/3 Online Help

n The SAPDBA function partial restore and complete recovery can be used to restore lost data and to handle the most frequently occurring database problems. In some cases, however, partial restore and complete recovery requires additional manual tasks, or the use of Oracle tools.

n If there are no appropriate data backups, or if all offline redo log files generated since the last backup are not available, you cannot run a partial restore and complete recovery. In this case, you must perform a database reset or a point in time recovery up to the last existing offline redo log file.

n If other database files are corrupted, in addition to data files, the partial restore and complete recovery function terminates and you must restart this function once the additional error has been resolved.

n If the BRBACKUP/BRARCHIVE logs cannot be found, you can restore them from the last backup using the SAPDBA function Restore individual files.

n If the files init<SID>.dba and init<SID>.ora cannot be found, you can restore them from tape. At the command prompt, enter brrestore -n init_ora.

n If init<SID>.sap has been lost, SAP tools can no longer access the tape drive. In this case, adapt a sample init<SID>.sap (directory SAP-EXE), or use OS command cpio to restore it from the third position on a BRBACKUP or BRARCHIVE tape.

n If a control file is damaged, you can copy one of its mirrors.

n If all the control files or online redo log files are lost, see R/3 Online Help, section DBA Oracle.

SAP AG 1999

Database

sapreorg

Save current online redo log files

and control file

Overwrite alldata files, control files,

and online redo log files

Find full offlinebackups

Detail logs

back<SID>.log

Databasemount

Databaseopen

Database Reset Using a Full Offline Backup

n When you perform a database reset, the database is reset to its previous consistent status – that is, its status at the time of a full backup. To determine the last possible full backup, SAPDBA is guided by the entries in the BRBACKUP summary log file back<SID>.log and the associated detail logs.

n Resetting the database always involves data loss. Therefore, SAP recommends performing a full offline backup before resetting the database. (If the database is running properly, use SAP tools; otherwise, use operating system tools.)

n The SAPDBA function Reset Database can be selected with a full offline backup (choose Restore database and startup open or Restore database and startup mount), or a full online consistent backup (choose Restore database using online consistent backups).

n Depending on the function chosen, SAPDBA sets the database ether to status open (that is, no reset logs) or to status mount. If the database has status mount, you can recover data using Oracle tools, such as the Server Manager. If the database has the status open, you cannot perform a retroactive recovery. For a retroactive recovery, use Restore database and startup mount instead of Restore database and startup open.

n With a database reset using a full offline backup, the data files, control files, and online redo log files are overwritten using the appropriate (taped) backups. For security reasons, these files are copied immediately to a subdirectory of sapreorg. (To enable these copies to be made, the database must have the status mount.)

SAP AG 1999

Recoveryscript

saparch

Database

sapreorg

Save onlineredo log files

and control file

Find Online_Cons

backups

Detail logs

back<SID>.log

Recoverdatabase

until cancel

Databaseopen

resetlogs

Database Reset Using a Consistent Online Backup

Overwrite all

Data files andcontrol files

Offlineredo log files

n When you perform a database reset using a full online consistent backup, the database is reset to a consistent status from the end (point in time) of the full backup.

n With a database reset using a full online consistent backup, data files, control files, and offline redo log files are overwritten by the appropriate (taped) backups. Therefore, you must save all offline redo log files in saparch using BRARCHIVE and perform a full backup before you reset the database using Online_Cons. During this process, note the messages displayed by SAPDBA.

n After a full restore, during a point in time recovery (recover database using backup controlfile until cancel), only the offline redo log files created during the online consistent backup are restored and applied. No other point in time can be chosen. The database is then started using option resetlogs. The online redo log files are newly initialized or newly created. Data cannot be recovered after opening the database with the option resetlogs, therefore, you must perform a backup. None of the backups performed before the database reset using online consistent backups can be used for a partial restore and complete recovery. Note: After a successful database reset, any offline redo log files that have been restored should be deleted manually from saparch.

n If SAP tools have been used, reworking is required after a database reset. Since log tables SDBAH and SDBAD are reset to an obsolete status, BRBACKUP may request tapes that have been released (in logical terms), but which are physically still locked. BRARCHIVE may not recognize the new offline redo log files as needing to be saved. For more details, see R/3 Online Help, chapter DBA Oracle .

n After a successful database reset, the data files can be searched for corrupt blocks, using the Oracle tool DB_VERIFY.

SAP AG 1999


Find full offline/online backups

Detail logs

back<SID>.log arch<SID>.log

Recoveruntil?

Status:allowed?

reorg<SID>.log

Database

Input:time

Not allowed if (for example):- No backup specified- No offline redo log files found- Recovery over tablespace reorg- Backup before open resetlogs

Full Restore and Recovery (1)

n With a full restore and recovery, the database is reset to a consistent status between the (end) point in time of the full backup, and the current point in time. This SAPDBA function corresponds to the point in time recovery.

n A full restore and recovery usually involves data loss. Therefore, SAP recommends that you perform a full offline backup before any full restore and recovery. (If the database is running properly, use SAP tools; otherwise, use operating system tools.) In addition, all offline redo log files in saparch should be saved using BRARCHIVE.

n A full restore and recovery can be performed using SAPDBA if the database can be set to status mount or open. First, a full online (or, if applicable, Online_Cons), or a full offline backup must be selected. Again, SAPDBA is guided by the BRBACKUP summary log file back<SID>.log and the corresponding detail logs. Next, enter the recovery end point. For a complete recovery, enter NOW. The offline redo log file backups required for this point in time recovery are determined using the entries in the BRARCHIVE summary log file arch<SID>.log.

n Under Show Status, SAPDBA indicates whether the intended recovery is allowed (status: allowed). A recovery may be rejected if:

� No full backup has been specified, or the required offline redo log files have not been found

� The recovery to be run contains a tablespace reorganization with data files

� The selected backup is dated before the last time the database was opened using the option resetlogs.

SAP AG 1999

Recoveryscript

saparchsapreorg

Save onlineredo log files

and control file

Recoverdatabase

(until time)

Database

Databaseopen

(resetlogs)

Overwrite allData files andcontrol files

(if necessary)Offline redo

log files

Full Restore and Recovery (2)

n For security reasons, the current online redo log files and a control file are copied to a subdirectory of sapreorg. All data files are restored from the backup medium (full restore). The control files may also be restored, depending on whether a tablespace was extended at the recovery time point.

n After a full restore, SAPDBA can replicate the tablespace extension in the database, using alter database add data file ... Information about file specifications is contained in directory sapreorg in the SAPDBA logs struct<SID>.log and reorg<SID>.log. After a tablespace extension, or after moving data files to another location, ensure you back up the newly changed structure.

n The offline redo log files required for the indicated recovery time point are restored to directory saparch. Using a recovery script, the database is recovered to the desired point in time (recover database until time xxyyzz [using backup controlfile]).

n If recovery was incomplete, the database must be opened using the option resetlogs. Using the Oracle tool DB_VERIFY, the database can be searched for corrupt data blocks. In addition, SAPDBA automatically goes to the backup menu, since the database has been opened using resetlogs.

n The copied offline redo log files should be deleted from directory saparch. If you have used SAP tools, reworking is required after an incomplete recovery. For more details, see R/3 Online Help, chapter DBA Oracle .

SAP AG 1999

Unit Summary


l Analyze the physical database structure usingSAPDBA

l Recover the database using the SAPDBA functionPartial Restore and Complete Recovery

l Reset the database using the SAPDBA functionReset Database

l Perform a point in time recovery using the SAPDBAfunction Full Restore and Recovery

SAP AG 1999




l R/3 Online Help

n Basis → Database interface → DBA Oracle

l SAP TechNet

n Information → Media Center → SystemManagement → CCMS

l R. Velpuri, A. Adkoli

n Oracle8 Backup and Recovery Handbook.Oracle Press, Osborne

SAP AG 1999

l Exercises?

l Solutions

Unit Actions

Restore and Recovery: Exercises

No. Exercise

1 Partial Restore and Complete Recovery Using SAPDBA

This exercise demonstrates database behavior after accidental loss of data, with the aim of familiarizing the participant with use of the SAPDBA function Partial restore and complete recovery, and of pointing out the limitations of this program.

1.1 Ensure that at least one valid full backup (online/offline) is available, and that your offline redo log file chain is backed up without any gaps.

1.2 Simulate a head crash by deleting the entire contents of directory sapdata3 (subdirectories protd_1, stabi_1, user1i_1).

Restore your database completely using one of the backups you performed for the unit "Scheduling, Performing, and Monitoring Backups". Use the SAPDBA function Partial restore and complete recovery.

1.3 Simulate a head crash by deleting directory sapdata2 (subdirectories proti_1, stabd_1, user1d_1, cntrl).

Restore your database completely using one of the backups you performed for the unit "Scheduling, Performing, and Monitoring Backups". Use the SAPDBA function Partial restore and complete recovery. Choose the option Partial restore and complete recovery a second time after you have eliminated the error.

Restore and Recovery: Solutions

No. Solution

1

1.2 Change to directory sapdata3, using cd /oracle/<SID>/sapdata3.

To confirm that only subdirectories protd_1, stabi_1, user1i_1 are located in sapdata3, enter ls –l. To delete these directories, enter rm –r protd_1, stabi_1, user1i_1.

Start SAPDBA. Under j: Restore/Recovery, choose a: Partial restore and complete recovery. To determine the function of each of the six phases, run through each of them manually and in sequence.

In the a: Check phase, the loss of data files is detected. Next, SAPDBA attempts to shut down the database, using shutdown immediate. Since a consistent data file status can no longer be achieved, this attempt fails. The DBA must subsequently confirm the shutdown abort. SAPDBA brings the database to the mount status, and detects the loss of a total of three data files.

In the b: Find Backup Files phase, under d: Select a backup run for restore, you can choose one of the backups you performed for the unit "Scheduling, Performing, and Monitoring Backups". SAPDBA automatically suggests the most recent backup. Under c: Select a backup file for restore, confirm that the backup you have chosen is being used for the restore.

After the desired data files have been successfully restored during the c: Restore Backup Files phase, the required offline redo log files are determined during the d: Find Archive Files phase.

If backups have been performed to several tapes (for example, brarchive -cds), the individual runs can be selected under e: Restore Archive files.

Recovery is started using f: Recover Database. The message Recover database terminated successfully indicates successful completion of a repair action.

1.3 Change to the Oracle home directory using cd /oracle/<SID>. Delete this directory by entering rm –r sapdata2.

As with exercise 1.2, change to the expert mode. Under j: Restore/Recovery, start the function a: Partial restore and complete recovery.

In this scenario, along with directory SAPDATA, a control file has also been lost in addition to data files. When this error occurs, the DBA must take manual action. During the check phase, Partial restore and complete recovery is aborted.

In file init<SID>.ora, check where the database expects control files, and enter:

cd /oracle/<SID>/dbs

more init<SID>.ora

Check the value set for parameter controlfiles.

Using SAPDBA, shut down the database, by choosing a: Startup/Shutdown instance → b: Shutdown → d: Shutdown abort

Create directory sapdata2 once again (normally, this is the mount point in the database system).

Shut down the database using SAPDBA with Shutdown abort. In file init<SID>.ora, confirm the location of your database control files, and their names (under parameter control_files). In directory sapdata2, create a subdirectory for the control file, and copy a mirror of the control file to this subdirectory. (NOTE: Copying a control file from one of your backups leads to an unnecessary recovery situation that can only be resolved by a full restore and full recovery, or by using Oracle commands.)

Next, the SAPDBA function partial restore and full recovery can be restarted. It then runs automatically.

In the Oracle home directory, create a new sapdata2 directory. For this directory, create a subdirectory for the control file by entering:

cd /oracle/<SID> mkdir sapdata2 cd sapdata2 mkdir cntrl

Copy a mirrored control file <source> to the new directory.

cd cntrl cp <source>

NOTE: Copying a control file from one of the backups leads to a recovery situation that can no longer be resolved using Partial restore and complete recovery.

As for exercise 1.2, in SAPDBA under j: Restore/Recovery, start the function a: Partial restore and complete recovery. The database repair is now processed as described in exercise 1.2.

SAP AG 2000






Backup Strategies Using RMAN

SAP AG 1999

Contentsl Backup strategies using RMAN

ObjectivesAt the end of this unit, you will be able to:l Explain the various backup strategies using RMANl Decide whether RMAN backup strategies fit the

needs of your company

Backup Strategies Using RMAN

n RMAN (Recovery Manager) is delivered with Oracle. This chapter describes the various RMAN backup options that are available for use with SAP tools.

SAP AG 1999

Full Backup (Level 0) with RMAN and SAP Tools (1)

cpio/dd

brbackup

RMAN

Control files

Database files

111

222

333

level 0:

backup_modebackup_mode = full= fulltape_tape_copy_cmdcopy_cmd = cpio|dd= cpio|dd

cpio/dd

n If you use SAP tools for a database backup with RMAN, you cannot perform a native backup with RMAN.

n Two types of backup using RMAN can be performed:

� Full backup (also called level 0 backup)

� Incremental backup (also called level 1 backup). An incremental backup is based on the last full backup, and will be discussed later in this chapter.

n For more information about full, incremental and whole backup types, see SAP Note 170013.

n A full backup is always performed with backup_mode = full. In a full backup, there are two ways of writing data to tape:

� Backing up data with RMAN

� Backing up data with OS tools

n If SAP tools are used, no recovery catalog is required. The backed up data files are cataloged in the control file.

n With full backup with OS tools , the tape_copy_cmd parameter is set to cpio or dd and the data files are saved to tape with the command specified. After that, brbackup starts RMAN. RMAN catalogs the backed up data files to the control file as level 0 backup. A control file is then backed up to tape with the OS tool specified (cpio or dd).

SAP AG 1999

Full Backup (Level 0) with RMAN and SAP Tools (2)

Oracleshadowprocess

222333

RMAN

level 0:

SBT Lib

backup_modebackup_mode = full= fulltape_copy_tape_copy_cmdcmd = rman= rman

222

Control files

Database files

cpio

brbackup

111

n With full backup with RMAN the tape_copy_cmd parameter is set to rman. Brbackup starts RMAN, which backs up the data files. RMAN reads all data file blocks, and only backs up those blocks that are no longer in initial status. Consequently, blocks from dropped tables are also backed up. The blocks are backed up by the Oracle shadow process direct to tape. A backup library for Oracle must therefore be installed (see SAP Note 142635). During the data file backup, RMAN catalogs the level 0 backup to the control file. After the data file backup, a control file (with all level 0 information) is saved to tape.

n Advantages of backing up with RMAN:

� All blocks are checked for block corruption

� The tablespaces are not set to begin/end backup mode. Thus, usually, fewer offline redo log files are created.

� Less data to be backed up

n Caution: A whole or partial backup with RMAN (tape_copy_cmd=rman, backup_mode = all) is possible, but is not a level 0 backup. However, all other advantages mentioned above apply.

n Full backups to disk can also be performed (backup_dev_type=disk). The parameter disk_copy_cmd is used instead of the parameter tape_copy_cmd, with the corresponding settings. The method differs only where RMAN is used: No backup library for Oracle is needed, and data files instead of savesets are saved (as is the case when using OS tools).

SAP AG 1999

saveset

Header

Trailer

Savesets

saveset_members = 1| 2| 3| 4 saveset_members = tsp saveset_members = all

saveset

Header

Trailer

saveset

Header

Trailer

btabd.data1

btabd.dataX

saveset

Header

Trailer

saveset

Header

Trailer

datafileA

datafileB

datafileC

datafileD

datafile1

datafileN

n In a backup using RMAN, the tape layout is the same as with a backup using OS tools. The difference is that savesets are backed up instead of data files.

n A saveset consists of a header, a trailer, and the blocks of at least one data file. Savesets are only used when the backup is performed with RMAN.

n The init<SID>.sap parameter saveset_members determines the number of savesets. This parameter can be overridden in sapdba or by calling up brbackup.

n The following settings are possible: 1, 2, 3, 4, tsp or all. For example:

� If saveset_members = 4, four data files are grouped together to form one saveset. In a complete database backup, several savesets are formed, each with the data from four data files. These savesets are backed up to tape.

� If saveset_members = tsp, a saveset is formed for every tablespace that is to be backed up. The saveset contains the data of all data files per tablespace.

� If saveset_members = all, only one saveset is formed. This saveset contains the data of all data files.

n If savesets are formed from more than one data file, RMAN reads the data in parallel from the appropriate data files.

� Advantage: Higher output to the tape station(s). Fast tape stations can be kept in streaming mode, thus reducing the time required for a backup.

� Disadvantage: In a restore/recovery situation, if the data from one data file is needed, the complete saveset must be read from the tape. If a disk that contains several data files is damaged, these data files must be restored from several savesets for the restore/recovery.

SAP AG 1999

Preparation Run

brbackup

111

222

RMAN

/dev/null

compressionrate

saveset_members = 1saveset_members = 1

saveset_members = 4saveset_members = 4saveset 1: compressratio xsaveset 1: compressratio x

datafileAdatafileAdatafileBdatafileBdatafileCdatafileCdatafileDdatafileD

saveset 2: compressratio ysaveset 2: compressratio y

saveset_members = tspsaveset_members = tsp

saveset_members = allsaveset_members = all

333

444

data file

Start preparationrun once per cycle

Start preparationrun once per cycle

OracleshadowprocessSBT Lib

brtools

n If tape stations with hardware compression, or savesets with more than one member, are used, you must perform a preparation run.

n In the preparation run, brbackup starts an RMAN backup of every data file to a saveset of its own. The data is compressed by brtools, and sent to /dev/null. Therefore, no additional space on the hard disk is required. The compression rate of every saveset with one member is verified by brtools, and sent to brbackup.

n At this point, brbackup determines how data files are allocated to savesets for every possible value of saveset_members, and calculates the compression rate of each saveset. The allocation cannot be controlled, and only changes (if necessary) when a further preparation run is performed. Therefore, between two preparation runs, if the saveset_members parameters are the same, the savesets contain the same data files.

n If data files exist that were not included in the preparation run (for example, because a data file was added), each one of these files is put in its own saveset.

n We recommend that you perform a preparation run once per backup cycle, or after major database changes, for example, reorganization, mass data transfer, or an SAP or database release upgrade.

SAP AG 1999

Incremental (Level 1) Backup

Level 0 backup

10

1010 25

2525

brbackup

Oracleshadowprocess

RMAN

Level 1 backup

111

222

333

level 0: # 10level 0: # 10Control files

Database files

SBT Lib

cpio/dd

10

10

10

25

25

25

n Incremental backup (also known as level 1 backup) is always based on the last level 0 backup (full backup). With SAP tools, only cumulative level 1 backup is supported as incremental backup. RMAN retrieves information about the last level 0 backup from the control files. An incremental backup is always a backup of the whole database, not of individual data files.

n In an incremental backup, all blocks of all data files are always read. However, only those blocks that have changed since the last level 0 backup are backed up. Therefore, if long backup runtime was caused by low throughput on the tape stations, incremental backup can reduce the backup time.

n Only one saveset (ending in .INCR) is created for an incremental backup The parameter saveset_members is set to all. Since only one saveset is created, the backup must fit on one tape. Follow-up tapes cannot be used. After the incremental backup is complete, a control file is saved to tape.

n If data files have been added to the database between the last level 0 backup and the level 1 backup, a level 0 backup is performed for these new data files before the level 1 backup starts. All new data is backed up to one saveset (ending in .FULL), even if the data was partially backed up.

SAP AG 1999

Level 1 Backup: Important Considerations (1)

Sat/Sun

Level 0backup

Level 1backup

Level 1backup

Level 1backup

Mon Tue Fri

. . .

. . .

Level 0backup

Mon Fri

Level 1backup

Level 1backup

. . .

. . .

data files data files

Partial restore and complete recovery with Partial restore and complete recovery with sapdbasapdba

recovery with offline redo log filesLevel 1 backup based on Level 0 backup

Sat/Sun

n With SAP tools, only cumulative incremental backups are supported at level 1. This means that incremental backups contain all blocks that have changed since the most recent level 0 backup (in relation to the time the level 1 backup was performed).

n If a restore/recovery is necessary (for example, due to a disk crash), a level 1 backup is not sufficient to repair the database. The level 0 backup of the damaged data files are always needed. These data files must be restored from the level 0 backup. Then the changed blocks from the level 1 backup (which must be based on this level 0 backup) can be imported to the data file. Now you only need to perform a recovery from the time the level 1 backup was made. If no level 1 backup is available for this level 0 backup, then you must perform a recovery based on the last available level 0 backup. This usually takes longer than using the level 1 backup as a basis.

n If the latest level 0 backup is damaged, then you must use the previous level 0 backup as a basis for recovering the data file. Only level 1 backups can be used that are based on this (that is, previous) backup. The level 1 backups that are based on the damaged level 0 backup cannot be used.

SAP AG 1999

Level 1 Backup: Important Considerations (2)

Recommended:1 level 0 backup

per week

Strongly recommended:4 level 0 backups

per cycle(if necessary, increase

backup cycle)

As basis for alevel 1 backup

n We recommend that you

� Perform at least one level 0 backup per week

n We strongly recommend that you

� Ensure that each backup cycle contains four level 0 backups. If necessary, increase the backup cycle. Otherwise, the whole backup strategy is dependent on one or two level 0 backups. If the most recent level 0 backup was not performed in the current backup cycle, a warning appears when the level 1 backup is performed. Caution: If the most recent level 0 backup was not performed in the current backup cycle, then it cannot normally be used for a restore because it has been overwritten. Therefore, if a disk error occurs, the result can be a complete loss of data.

n We recommend that you verify a level 0 backup at least once per backup cycle, but preferably once a week. A delayed verification with brrestore is only possible on the database host with an open or mounted database.

SAP AG 1999

Recovery Using Incremental Backup with sapdba

Database

Restoredata files

Findbackups

Detail logs

back<SID>.log

CheckRecoverdatabase

Recoveryscript

saparch

Findoffline

redo log files

arch<SID>.log

Restorelevel 1

backup

Detail logs

back<SID>.log

Findlevel 1

backups

Findlevel 0

backups

Restoreoffline redolog files

Restorelevel 0

backup

n A partial restore and complete recovery with level 0 and level 1 backup is only slightly different than a partial restore and complete recovery of a whole backup.

n The check and repair phase is performed as normal.

n In the find backup phase, the level 0 backup is selected. In the restore backup phase, the data file(s) is/are restored.

n In the find level 1 backup phase, a level 1 backup is selected that is based on the level 0 backup selceted. In the restore level 1 backup, the blocks that were backed up in the level 1 backup are written to the restored data files.

n The phases that follow are performed as normal.

SAP AG 1999

Unit Summary


l Explain the various backup strategies using RMAN

l Recognize the advantages and limitations of thesestrategies

l Decide whether RMAN backup strategies fit theneeds of your company

SAP AG 1999


l Knowledge Product: SAP DatabaseAdministration Oracle

l R/3 Online Documentation: BC →SAP Database Administration:Oracle

l SAPTechNet → DB Admin. Oracle →Knowledge Base

l SAP Note 170013

SAP AG 1999

l Exercises?

l Solutions

Unit Actions

Backup Strategies Using RMAN: Exercises

(Optional)

No. Exercise

1 Full Backup (Level 0)

In these exercises you perform a full backup with the SAP database backup tools. The exercise refers to the play databases, and is limited to the operating system level.

1.1 Maintain the standard settings for full backups in the file init<SID>.sap.

Decide whether to perform the full backup with OS tools or with RMAN and maintain the appropriate parameter. (Ensure that the backup is made to disk.)

1.2 Perform a full backup to disk with SAPDBA.

1.3 Confirm that your full backup has been performed correctly by checking the log files at the operating system level, and by using SAPDBA.

1.4 Optional: Force several log file switches (ca. 3)

2 Incremental Backup (Level 1)

2.1 Perform an incremental backup to disk with SAPDBA.

2.2 Confirm that your incremental backup has been performed correctly by checking the log files at the operating system level, and by using SAPDBA.

2.3 Which file contains the backed-up blocks?

Where can you find this file?

3 Partial Restore and Complete Recovery Using Incremental Backup with SAPDBA

3.1 Ensure that at least one valid full backup (Level 0, online/offline) is available, and that your offline redo log file chain is backed up without any gaps.

3.2 Simulate a head crash by deleting the entire contents of directory sapdata3 (subdirectories protd_1, stabi_1, user1i_1).

3.3 Repair your database using the full and incremental backups you performed in Exercises 1 and 2. Use the SAPDBA function Partial restore and complete recovery.

Backup Strategies Using RMAN: Solutions

No. Solution

1

1.1 Using the OS editor, in file /oracle/<SID>/dbs change the following parameters of file init<SID>.sap:

backup_mode = full

backup_dev_type = disk

disk_copy_cmd = copy

or dd or rman

1.2 Start SAPDBA. To get to the backup menu, choose

h - Backup database.

To start the backup, choose a - Backup function: Normal backup. Specify the backup device type by choosing

c - Backup device type: local disk.

To define which objects are to be backed up and which kind of backup is to be performed, choose

d - Objects for backup: full.

To define the backup type choose for example

e - Backup type online.

Under h - Special options, choose

b – Compress: no

To start the backup run, choose

S - Start BRBACKUP.

1.3 The BRBACKUP logs are in subdirectory sapbackup. cd /oracle/<SID>/sapbackup

The detail log is named b<time stamp>.<ext>, with <ext> = fnd signifying full online on disk, and <ext>=ffd signifying full offline on disk. Using the time the log was created, select the appropriate log, and view it by entering (for example)

more b<time stamp>.<ext>

To view logs in SAPDBA, choose l - Show/Cleanup → a - Show log files / profiles e – BRBACKUP log files.


1.4 Start the Oracle server manager using: svrmgrl

Dispatch the following commands in the order shown below:

connect internal;

alter system switch logfile;



exit;

2

2.1 Start SAPDBA. To get to the backup menu, choose

h - Backup database.

To start the backup, choose a - Backup function: Normal backup. Specify the backup device type by choosing

c - Backup device type: local disk.

To define which objects are to be backed up and which kind of backup to perform, choose

d - Objects for backup: incr.

To define the backup type, choose for example

e – Backup type online.

Under h – Special options, choose

b – Compress: no

To start the backup run, choose

S – Start BRBACKUP.

2.2 The BRBACKUP logs are in subdirectory sapbackup.

cd /oracle/<SID>/sapbackup

The detail log is named b<time stamp>.<ext>, with <ext> = ind signifying incremental online on disk, and <ext>=ifd signifying incremental offline on disk. Using the time the log was created, select the appropriate log, and view it by entering (for example) more b<time stamp>.<ext>

To view logs in SAPDBA, choose l - Show/Cleanup → a - Show log files / profiles e – BRBACKUP log files.


2.3 The blocks are backed up to a file named B<time stamp>.INCR. If new data files have been added to the database since the last full backup, then those blocks are backed up to a file named B<time stamp>.FULL.


3

3.1 See 1.3

3.2 Change to directory sapdata3, using cd /oracle/<SID>/sapdata3.

To confirm that only subdirectories protd_1, stabi_1, user1i_1 are located in sapdata3, enter ls –l. To delete these directories, enter rm –r protd_1, stabi_1, user1i_1.

3.3 Start SAPDBA. Under j: Restore/Recovery, choose a: Partial restore and complete recovery. To determine the function of each of the eight phases, run through each of them manually and in sequence.

In the a: Check phase, the loss of data files is detected. Next, SAPDBA attempts to shut down the database, using shutdown immediate. Since a consistent data file status can no longer be achieved, this attempt fails. The DBA must subsequently confirm the shutdown abort. SAPDBA brings the database to the mount status, and detects the loss of a total of three data files.

• In the b: Find Backup Files phase choose S: Start finding backup files, under d: Select a backup run for restore, you can choose one of the backups you performed in Exercise 1. SAPDBA automatically suggests the most recent backup.

• In the c: Select a backup file for restore phase, confirm that the backup you have chosen is being used for the restore.

• In the i: Find incr. backup phase, choose S: Find appropriate incremental backup runs, under a: Select an incr. backup run for restore, you can choose one of the incremental backups you performed in Exercise 2. SAPDBA automatically suggests the most recent backup.

• In the j: Restore incr. backup phase, confirm that the incremental backup you have chosen is being used for the restore.

• After the desired data have been successfully restored, the required offline redo log files are determined during the d: Find Archive Files phase. Choose S: Find offline redo logs

• in the e: Restore Archive files phase, confirm that the redo log files you have chosen are being used for the restore.

• Recovery is started using f: Recover Database. The message Recover database terminated successfully indicates successful completion of a repair action.

SAP AG 2000






Advanced Backup Techniques

SAP AG 1999

Advanced Backup Techniques

Contentsl Advanced Backup Techniques

ObjectivesAt the end of this unit, you will be able to:l Explain the various backup strategies supported by SAP

l Decide which strategies fit your needs

SAP AG 1999

Backup Requirements and Costs

Backup time

Recovery time

High availability

Training

Acquisition costs

Administrative workload

n Requirements. Depending on your demands, you may require one, or a combination of several, advanced backup strategies. When you define your backup strategy, you must consider:

� How long it takes to perform a backup

� The maximum time required for a complete database recovery

� To what extent your backup strategy provides additional security in case of a hardware failure

n Costs. Your backup strategy will also depend on:

� How much training the administrator requires

� The amount of database administration required for implementation and production operation

SAP AG 1999

... ... Logsinitfiles

Tapeheader

Data files Offline redo log filesin saparch

Databasebackup

Offline redo log filebackup

brbackup -m all -c -a -cds -c

BRBACKUP and BRARCHIVE: One-Run Strategy

n The advantage of the one -run strategy is that for a complete backup, BRBACKUP and BRARCHIVE are called together rather than individually. Only one tape pool (in this example volume_backup) is used. The offline redo log files are backed up to the tapes where the database files are backed up. As a result, tapes can be saved, and the administrative workload reduced.

n SAP recommends that you use the one-run strategy for BRBACKUP:

� BRBACKUP <database backup options> -a <offline redo log backup options>

With this procedure, BRBACKUP backs up all files (as usual) and then starts BRARCHIVE using the options entered after -a. BRARCHIVE first backs up the corresponding offline redo log files (as usual), and then backs up all logs (including BRBACKUP logs). When a complete backup is planned using CCMS, the recommended one-run strategy is used.

n With the one-run strategy, the maximum number of offline redo log files that can be backed up is the number that can still fit on the tape after the database backup. If more offline redo log files are generated daily than can be backed up, for example because the database has grown, or the number of offline redo log files is increasing, an archiver stuck occurs. Therefore, you must regularly check whether the tape capacity is sufficient. If necessary, you should use larger tapes, an extra tape station, or another backup strategy.

n The one-run strategy cannot be used to resolve an archiver stuck , since BRBACKUP attempts to connect to the database. If an archiver stuck is to be resolved using BRARCHIVE, tapes must be available in tape pool volume_archive (that is, the “emergency tape pool”).

SAP AG 1999

10

1010

Data files

25

2525

25

25

25

10

10

10

11 24. . .9 10

Control files

Offline redo log filesin saparch25

10 10initfiles ...

. . .

. . .

25 10 25... Logs

...begin backup ...end backup

brbackup -t online_cons

Legend: Log sequence number 9

Consistent Online Backups

n A consistent online backup is a database backup in the online mode, containing logically consistent data. This means that the offline redo log files generated during the backup are saved to the same volume as the database files that are backed up using BRBACKUP. This type of backup run is designed to provide an additional backup beyond the standard SAP backup cycle.

n A consistent online backup can be used to reset the database to its status at the end of the last backup. This is done by restoring data files and offline redo log files, and performing a point in time recovery. See also the unit “Restore and Recovery.”

n After backing up all data files online, BRBACKUP performs an online redo log switch using Oracle commands. BRBACKUP waits until the archiver process has finished copying the last redo log file into directory saparch, and then copies the offline redo log files created during the online backup to tape. The last files on tape are the BRBACKUP summary and detail log.

n The backup of the offline redo log files in a consistent online backup is controlled completely by BRBACKUP. Therefore, this run is independent of the BRARCHIVE backups, and does not affect these backups. In particular, no entries are processed in the BRARCHIVE summary log file arch<SID>.log.

n A consistent online backup can be performed by using SAPDBA, or by calling BRBACKUP directly, using the command prompt option -t online_cons.

SAP AG 1999

DDS

DDSDLT DLT

DLT

BRBACKUP

init<SID>.sapexec_parallel = 0tape_address = (dev1, dev2, dev3)archive_function = double_save_deletetape_address_arch = (dev4, dev5)

BRARCHIVE

Control files

Offline redo log filesin saparch

Parallel Tape Support

n To reduce the time required for backing up and restoring the data files and offline redo log files, the SAP backup tools support the parallel use of several tape stations.

n BRBACKUP uses all tape stations defined in parameter tape_address in file init<SID>.sap. Parameter exec_parallel should be set to 0, since this triggers a copy process (cpio, dd) for each tape station.

n To reduce the backup time, the database files are distributed across the tape stations. For tape stations with hardware compression, the backup time does not necessarily correlate directly with the data volume. Therefore, BRBACKUP refers to the times required by previous backup runs. If time optimization would result in an additional tape change, time optimization is not performed. To keep backup times to a minimum, the total tape station capacity should be significantly larger than the total volume of data to be backed up.

n BRARCHIVE supports parallel backups of offline redo log files on two separate tape stations, which is defined in parameter tape_address_arch. If this parameter is not set, BRARCHIVE uses the first two tape stations defined in tape_address. As backup options for archive_function, you can choose double_save or double_save_delete.

n If data must be restored from tape to disk, BRRESTORE also uses all tape stations defined in tape_address(_arch) in parallel, which minimizes the restore time.

SAP AG 1999

Data files. . .

Mon Tues Sun

brbackup-m PSAPBTABD

brbackup-m all -f 7

BRARCHIVE BRARCHIVE BRARCHIVE

brbackup-m PSAPSTABD

Partial Database Backups

n Instead of performing a complete backup, you can perform a partial backup – that is, you can back up only certain parts of the database. If you perform partial backups, the sum of individual backups must cover the entire database. For example, if you perform a partial backup once a week, you must perform four partial backups in one month.

n Both Oracle and SAP tools support the recovery of data files from different backup runs. For this type of recovery, these tools require all offline and online redo log files generated since the backup of the oldest data files.

n To perform a partial backup, you can either (a) use the DBA Planning Calendar (transaction DB13), (b) call SAPDBA (choose d - Objects for backup), or (c) run BRBACKUP directly (using the command prompt option -m <objects>). In the CCMS, only tablespaces can be selected. The latter two procedures allow you to select individual data files.

n Ensure that the complete database is backed up within the selected time interval. To complete a database backup, use SAPDBA (choose l - Make part. backups compl., and enter the number of days), or BRBACKUP (enter -f <days> ). This backup completes the partial backups performed for the previous few days (set parameter <days>).

n Note: This procedure can also be used to complete aborted backup runs. In this case, specify the log file associated with the backup run (for SAPDBA, choose k - Restart backup; for BRBACKUP, enter -f <log file(s)>).

SAP AG 1999

System, roll, andtemp tablespace

brbackup-m all_data

Datatablespaces

Pure indextablespaces

Backing Up Data Tablespaces Only

n Another method of reducing backup times is to limit backups to data tablespaces. If index tablespaces are lost, the indexes must be rebuilt using information from the Oracle Dictionary. To be able to use this procedure, the database administrator must have extensive background information.

n When choosing tablespaces, BRBACKUP does not refer to tablespace naming convention; instead it evaluates Oracle tables. Therefore, BRBACKUP ensures that only those tablespaces that are either empty or only contain indexes are excluded from the backup procedure. This ensures that tablespaces SYSTEM, PSAPROLL, and (unless empty) PSAPTEMP, which are required to run the database, are always backed up as well.

n You can perform a backup limited to data tablespaces by calling SAPDBA (choose d - Objects for backup, and enter all_data), or by running BRBACKUP directly (using the command prompt option -m all_data).

n If index tablespaces are affected by a database failure, SAPDBA rebuilds the data files and indexes. To do this, SQL scripts that contain the index definitions using the Oracle Dictionary are generated. The missing data files are created, and the indexes built up (similarly to the reorganization of index tablespaces and data files). See also the unit “Storage Management and Monitoring.”

n Note: When data files are restored, you can also limit the restore to data tablespaces. Using BRRESTORE, enter -m all_data.

SAP AG 1999

Data files

init<SID>.sapexec_parallel = 0tape_address = (dev1, dev2, dev3)backup_root_dir = (lvol1, lvol2)

dir1

dir2

Fast 1st stepbrbackup

-d disk -e 4

“Slow” 2nd stepbrbackup -b last -d tape

Two-Step Disk Backup

n You can also reduce the time required to perform a backup by using the two-step disk backup method. First, a backup is performed to disk, which is typically faster than a backup to tape. Second, the files are backed up from disk to tape. This method also reduces restore time (if the required files are still available on disk).

n The first step (backup to disk) can be performed using CCMS, SAPDBA, or BRBACKUP. If directories on different logical volumes (NT: logical partition) are used, you must first enter them in parameter backup_root_dir of init<SID>.sap. If parameter exec_parallel has been set to 0, one copy process is triggered per directory entry. You can manually define the number of copy processes to run in parallel by calling SAPDBA and choosing h - Special options → f - Level of parallel execution, or by running BRBACKUP and entering -e <number>.

n To start the second step (backup to tape), call SAPDBA (choose j - Backup from disk backup), or run BRBACKUP (at the command prompt, enter -b <log file> | last).

n For BRARCHIVE, perform the same two steps. Either call SAPDBA, and choose j - Use disk backup, or run BRARCHIVE, using the command prompt option -a .

n BRRESTORE resets the database files to the original directories, regardless of which backup type (disk or tape) is used.

n In the second phase of the backup, you can use BRBACKUP to automatically delete files. To integrate the automatic deletion of files, configure parameter backup_delete [<log_name>|last].

SAP AG 1999

$ORACLE_HOME

.

.

.

mirrlogA

sapbackup

sapcheck

saptrace

origlogA

dbs

sapdata<n>

sapreorg

mirrlogB

saparch

origlogB

sapdata1 btabd_1

NEW_DB_HOME

.

.

.

mirrlogA

sapbackup

origlogA

dbs

sapdata<n>

mirrlogB

origlogB

sapdata1 btabd_1

init<SID>.sapnew_db_home = /oracle/NEW

brbackup -d disk_copy

Structure-Retaining Database Copy

n The structure-retaining database copy is a backup to disk that retains the original directory structure. This type of backup can be used in combination with the two-step disk backup method in a normal backup cycle.

n In case of a disk failure, it may suffice to remount the file system mount points. This procedure is faster than copying the files. Additional uses of the structure-retaining database copy include (a) for building up new R/3 Systems from a database copy, (b) for an Oracle standby database, or (c) for moving the file system (for example, in order to move the data files from the file system to raw devices, or vice versa).

n The parameter setting for new_db_home in file init<SID>.sap defines the home directory of a database copy. This directory, in addition to directories dbs, sapdata<n>, origlogA/B, mirrlogA/B, and sapbackup, must first have been created by the database administrator. The subdirectories of these directories are created automatically during the copy process. Additional files in the database environment, such as executables and log files, are not copied during this process.

n A backup to remote disks can be performed without NFS mount. Ensure the following parameters are set:

� backup_dev_type = stage_copy

� stage_copy_cmd = rcp or ftp

� stage_db_home = <dir_name>, where: <dir_name> is the new ORACLE home on a remote disk

n BRRESTORE always writes database files back to the original directories.

SAP AG 1999

1. Split mirror2. Mount3. Backup4. Unmount5. Resync

brbackup-t online/offline_split

-d tape

Production server Backup server

Mirror

Split Mirror Disk Backups

n Split mirror disk backups can significantly reduce backup time. At the start of a backup, the disk mirror is broken up where the data files are located, by a predefined command. The first half (that is, one mirror) is backed up from a separate server, while the “production half” is still running, without impairing performance. Next, the disk mirror is resynchronized.

n A backup is performed as follows: � Online. 1. Bring the tablespaces into the backup mode. 2. If your mirror system has

problems with splitting a mirror while disk writes are occurring, issue the ALTER SYSTEM SUSPEND statement. 3. Break up the mirror. 4. Issue the ALTER SYSTEM RESUME statement to resume your database. 5. Stop the backup mode in the production half. 6. Perform an online backup of the mirror. 7. Resynchronize the mirror. � Offline. 1. Stop the database. 2. Break up the mirror. 3. Start the database in the production

half. 4. Perform an offline backup of the mirror. 5. Resynchronize the mirror.

n The following settings in init<SID>.sap apply to the configuration of a split mirror disk backup (For Windows NT refer to SAP Note 122363 ): � primary_db defines the server where the production database is running (for local disks).

� orig_db_home = <dir_name>, where: <dir_name> = original Oracle home directory of the productive database (for remote disks).

� split_cmd contains the commands for breaking up the mirror, and for mounting the file system on the backup server.

� resync_cmd has the analogous commands for unmounting and resynchronization (optional).

n The configuration must enable BRBACKUP to connect from the backup server to the database.

n During normal operation, disk mirroring protects against database failure. If such protection is also required during the backup procedure, an additional mirror is required for the production half.

SAP AG 1999

Production server

saparch

Database open


brarchive -sd-d disk -f -w

SAP Tools and the Oracle Standby Database

Standby server

Database mounted standbyor offline for backup

Recovery

brarchive -ssd-f -m 60

brbackup-t offline_standby

saparch

ftpftp

NFSNFSOR

n An Oracle standby database consists of two database servers. The production database has the status open. During normal operation, the standby database has the status mount, and is continually applying the offline redo log files from the production server. In case of a production server failure, the standby database can be opened, and can take on the role of the production database.

n The data files are saved to tape on the standby server, using either SAPDBA (choose e - Backup type, e - offline_standby) or BRBACKUP (enter -t offline_standby). These actions are logged on the production server (table entries in the database, and log files in directory sapbackup, which both servers have in common due to NFS mount).

n BRARCHIVE runs on both servers: From the production server, a continual backup to disk is performed (using verify, with option -w) through NFS mount in directory saparch on the standby server. On the standby server, the backup to tape is performed.

n The offline redo log files are applied on the standby database when the option -m <delay> has been entered. The optional entry delay determines whether the connection is “hot” (that is, replicated with no delay) or “warm” (that is, replicated with a delay). The latter makes it possible to stop applying offline redo log files before a user error is replicated on the standby server.

n You can perform backups without NFS mount on a remote hard disk with parameter backup_dev_type = stage_standby. The parameter stage_copy_cmd should be configured properly.

n Note: Several structural changes on the production database are not automatically replicated on the standby database. In this case, the recovery is stopped, and the database administrator must take action manually.

SAP AG 1999

BACKINT

External backup server

init<SID>.sapbrbackup

$ORACLE_HOME

...

saparch

sapdata<n>

sapdata1 btabd_1

SAPDBA

brarchive brrestore

Backup RestoreInquire

brbackup

init<SID>.utl

init<SID>.sapbackup_dev_type = util_file_onlineutil_par_file = init<SID>.utl

External Backup Tools Using BC-BRI

n Backups can also be performed using external tools. Communication with SAP tools takes place through an interface defined by SAP (BC-BRI).

n The backups must continue being started by SAP tools. This ensures that all actions are logged, and that backups can be monitored using the CCMS. In addition, this allows you to use the SAPDBA restore and recovery features.

n For interface configuration, in file init<SID>.sap, set parameter backup_dev_type to util_file or util_file_online. (In the latter case, only the tablespace to be backed up is set to the backup mode.) The setting util_par_file refers to the configuration file init<SID>.utl, which contains parameters for the interface program BACKINT.

n With R/3 Release 4.5B and higher, the Legato Storage Manager (LSM) and the implementation of the BACKINT interface is delivered free of charge by Oracle. But this is a limited version of the Legato NetWorker. The native BRBACKUP with cpio or dd, you can also use the BACKINT program from Legato for normal backups. Two methods are available for incremental backups:

� Using the SAP backup library, or

� Using the LSM backup library with BACKINT

n For more information about the Legato installation, see SAP Note 142635. This note describes the installation of the SAP backup library and LSM backup library.

n For an overview of certified providers, see SAPNet (Complementary Software Program).

SAP AG 1999

Unit Summary


l List the backup strategies that are supported by SAP

l Recognize the advantages and limitations of these

strategies

l Decide which strategies fit your needs

SAP AG 1999




l R/3 Online Documentation

n BC → SAP Database Administration: Oracle

l SAP TechNet

n DB Admin. Oracle → Knowledge Base

SAP AG 1999

l Exercises?

l Solutions

Unit Actions

Advanced Backup Techniques: Exercises

(Optional)

No. Exercise

1 Consistent online backup

1.1 Perform a backup of your local database that meets the following criteria:

Complete

Online_cons

To disk

With offline redo log files backed up by BRBACKUP

1.2 Using the log files, check whether the backup was successful.

2 Partial database backup


Partial: with customer tablespaces PSAPUSER1D and PSAPUSER1I backed up

Online

To disk


3 Optional: Backing up Data Tablespaces Only


Pure index tablespaces are excluded

Online or offline

To disk


Advanced Backup Techniques: Solutions

No. Solution

1

1.1 The parameters in file init<SID>.sap should have been maintained correctly, according to the instructions given in the preceding unit.

Solution using SAPDBA: Choose

h – Backup database → e – backup type → c – online (consistent).

Check the setting for

c – Backup device type.

To start the backup, choose


Solution using BRBACKUP: At the operating system level, enter brbackup –d disk –m all –t online_cons.

1.2 The detail log

b<timestamp>.and

is in directory sapbackup. In SAPDBA, choose

l – Show/Cleanup → a – Show log files/profiles → e – BRBACKUP log files.

2

2.1 Solution using SAPDBA: Choose

h – Backup database,

d – Objects for backup,

g – a tablespace name;

and enter PSAPUSER1D, PSAPUSER1I. Check the settings for

c – Backup device type and

e –backup type.



Solution using BRBACKUP: At the operating system level, enter

brbackup –d disk –m psapuser1d,psapuser1i –t online.

2.2 The detail log

b<timestamp>.pnd



3

3.1 Solution for SAPDBA: Choose

h – Backup database → d – Objects for backup.

Enter all_data. Check the settings for

c – Backup device type and

e – backup type.



Solution for BRBACKUP: At the operating system level, enter

brbackup –d disk –m all_data –t online.

3.2 The detail log

b<timestamp>.and



SAP AG 2000






Storage Management

SAP AG 1999

Storage Management

Contentsl Storage management basics

l Monitoring freespace and space critical objectsl Internal and external fragmentation

l Reorganization

ObjectivesAt the end of this unit, you will be able to:

l Adapt storage parameters of tables and indexes

l Run and analyze the results of sapdba -check and sapdba -nextl Determine if a reorganization should be performed

l Perform a reorganization

SAP AG 1999

Disks

è File system

Datafile1 Datafile2

. . .

Space Management: Review

8K8K8K8K

8K8K8K8K

8K8K8K8K

8K8K8K8K

8K8K8K8K

8K8K8K8K

Extent144K

Extent48K

Extent144K

Segment192K

1

Initial Extent

Segment(table/index)

Datablock

Table ABC

Table XYZ

Table DEF

Extent192K

Extent640K . . .

. . .

. . .

Extent192K

Extent224K

Extent80K

Extent256K

Datafile2Datafile2Datafile1Datafile1

Tablespace

NextExtent

2

NextExtent

n Each table and index is assigned to a tablespace, which consists of one or more data files at the operating system level. All table and index data is stored in the data files of the tablespace.

n Oracle stores tables and indexes in individual data blocks. In an R/3 installation, data blocks are 8 KB in size. When new storage space is required for a table or index, one or more contiguous data blocks of a data file are allocated to form an extent. If there is not enough contiguous freespace to allocate a new extent, the Oracle error message ORA-1653 (for a table) or ORA-1654 (for an index) occurs.

n Oracle data objects have several storage parameters that influence growth:

� The first extent (initial extent) should be large enough for the expected table or index size. If an extent of a data object becomes full during an insert or update operation, the Oracle storage management system attempts to allocate another extent in the tablespace.

� An object can allocate extents up to the limit MAXEXTENTS. If the maximum number of extents for each object is reached, the error message ORA-1631 (for a table) or ORA-1632 (for an index) is displayed. If this occurs, you must increase the parameter MAXEXTENTS and check the size of the table’s NEXT parameter.

� PCTFRE, PCTUSED, and PCTINCREASE are three additional storage parameters. Only change them under special circumstances and after consulting SAP for support.

SAP AG 1999

. . .00 11 22 33 44 55

Objects (tables/indexes) Tablespace

<tablespace>.data1

2

3

10

2 3

0

0

1

0

4

1

2

4

5

Gaps

Oracle block Internalfragmentation

Used

Free

35

1 4

Externalfragmentation

2

Space Management: Fragmentation Types

Criticalobject

Extents

n Depending on the size of each data record, several data records can be stored in an 8 KB data block. For fields of indexes and for long raw fields, Oracle compresses the contents of each data record. As a result, the data records of an index or a table with one or more long raw fields will usually differ in length.

n When a data record is deleted, a gap of unused storage space results within the corresponding data block. The existence of such gaps within data blocks is called internal fragmentation. Oracle can reorganize internally fragmented data blocks so that wasted storage space is re-used.

n The extents, which belong to the different data objects of a tablespace, allocate storage space within data files of this tablespace. When a database object is dropped, the extents are released. Gaps of unused storage space result. The existence of one or more of these gaps is called external fragmentation of a tablespace. Newly allocated extents can only be inserted into such a gap if they are smaller than or equal to the size of the gap.

n If the NEXT extent size of an object is larger than the largest free contiguous storage area of its tablespace, it is called a space critical object. The allocation of a new extent for this object will fail if you have not extended the tablespace.

SAP AG 1999

<Timestamp.chk> in/oracle/<SID>/sapcheck

DBA Planning CalendarR/3 xPlanning Goto Listing Help System

MONMONCheckDBCheckDB

TUETUECheckDBCheckDB

WEDWEDCheckDBCheckDB

THUTHUCheckDBCheckDB

FRIFRI SATSATCheckDBCheckDB

SUNSUNsapdbasapdba

-next-nextMONMON

CheckDBCheckDBTUETUE

CheckDBCheckDBWEDWED

CheckDBCheckDBTHUTHU

CheckDBCheckDBFRIFRI SATSAT

CheckDBCheckDBSUNSUN

sapdbasapdba-next-next






SUNSUN






SUNSUNsapdbasapdba

-next-next

Daily Monitoring: sapdba -check

Database tableDBMSGORA

1 Calendar

Schedule an Action for Tue 0518:00:00Start time

Period (weeks):


Full database offline backup

Full database online + redo log backupFull database online backup

Redo log backup

Partial database offline backupPartial database online backup

Check optimizer statisticsUpdate optimizer statistics

Adapt next extents

Check database

S ò0 rñ0! Start immediately

Start check

WarningsErrors

Total

Configure check Database operations monitor History StandardDatabase tableDBMSGORA

Refresh every

Delete after

View the last

Check results Settings

10 seconds inaktiv

100 days aktiv

10 days

History: All messages

6

5

11

Result Date Time Days Error type Error name Text

WEEE

08/21/1999 22:00:00 10 PROF LOG_SMALL_EN... LOG_SMALL_ENTRY_MAX_SIZE08/21/199908/21/199908/21/1999

22:00:0022:00:0022:00:00 10

1010 DBO

DBODBA

OPTALYNO_OPT_STATS

DB Operation opt never started or finished successfulDB Operation aly never started or finished successful6 INDEXE(S) DDXTF%0,DDXTT∃0, TATAF %0, ... WITHOU

Database Check: Overview of Results

sapdba -check

DB16

n Use the R/3 tool SAPDBA with the option -check to check the following:

� Extents of tables and indexes

� Tablespace filling

� Physical consistency of the database. That is, the consistency of the control files, redo log files, and data files

� Severe error messages in the alert log

� init<SID>.ora parameter settings

� Database problems specific to the R/3 environment

n Schedule SAPDBA -check to run daily during periods of low system activity, by using either the DBA Planning Calendar (transaction DB13) or by entering command sapdba -check at the command prompt.

n Command sapdba -check generates a log file called <DateTime>.chk. which is written to directory ../../sapcheck. The log information is also written to the database table DBMSGORA, and can be viewed using the R/3 Database System Check Monitor (transaction DB16). This log information should be monitored after each SAPDBA -check run.

n If a database or system error occurs, use command sapdba -check to check the log information.

n To monitor your sapdba -check run, you can also use transaction DB24.

SAP AG 1999

Configuration tableDBCHECKORA

Configuring sapdba -check

TypeParameterObject

Actv.

Condition

Description

Change Database Check Parameter

Database analysis tool (SAPDBA)ARCHIVE_STUCK

Yes

Error

ARCHIVE STUCK - FS SPACE #1 MORE THAN #2 FULL

greater than (old 80 Percentage

Repeat period

Corrective measure

Changed by

Duration

Type

User

Time Unit

Operation

Date

Program SAPDBA: BACKUP ARCHIVE LOGS

if

DB17Number of SAPDBA check parameters

ARCHIVE_STUCKDBA E PP80> SAPDBA: BACKUP ARCH ... ARCHIVE STUCK - FS SPAC...

TotalStatusProfile param.Oracle alerts

Operations

67192813 7

66 1

ActiveInaktiv

DBA

DBA

DBA

DBA

DBA

DBA

DBA

DBA

CONTROL_FILE_MISSING

CONTROL_MIRROR

CRITICAL_SEGS

DF_OFFLINE

FILE_MISMATCH

FILE_MISSING

FILE_TYPE_UNKNOWN

FS_FULL

E

E

E

E

E

E

W

W >

>

P

P

1

95

INIT<SID>.ORA

SAPDBA: TABLESPACE A...

DO A “CREATE CONTROL...

SAPDBA: RESTORE/RECO...

EXTEND FILESYSTEM OR...01 / 22 / 1999

P

E

CA...

CONTROL FILES ARE MISSI...

CONTROL FILE(S) ARE NOT...

SEGMENT(S) #1 WOULD CAU...

DATAFILE(S) #1 OFFLINE

FILE TYPE DOES NOT MATC...

DATA FILES ARE MISSING

FILE TYPE ( DATAFILE , RA...

# 1 FILE SYSTEM(S) # 2 ARE...

Typ Parameter O... Actv. S.. Op... Val. U... Per... Unit Date U... C DescriptionCorrMeasure

n To configure the checks performed by SAPDBA -check , choose Tools → CCMS → DB administration → Check → Configuration (transaction DB17).

n The system checks are identified by an error type and name (Err Type, Parameter ID):

� DBA: The checks that report these errors are programmed into SAPDBA. You can change the thresholds specified for these checks. You can activate and deactivate these checks.

� ORA: Oracle alert-log messages (important administrative and error messages) that the R/3 System check will report to you. You can add additional “ORA”-entries.

� PROF: Problems in the Oracle init<SID>.ora initialization profile. You can add additional parameters.

n The columns in the configuration table DBCHECKORA mean the following:

� Active: Activate (Y) or deactivate (N) the check for the problem

� Severity : Warning (W) or error (E). Errors require immediate attention.

� Check Oper, Check Val, Check Unit: The threshold value for triggering a warning are defined in these columns. For example, Check Oper >, Check Value 80, and Check Unit P indicates that a warning or message should be triggered when the database value exceeds 80 percent.

� CorrMeasure: This provides a quick, editable tip for solving the problem. If these fields are empty, first analyze the problem in detail, and then choose a measure to correct it.

SAP AG 1999

ADD A DATA FILE:File size depends on the estimated increase of thetablespace objects. Check for the number of datafiles in the database.

<tablespace>.data1

Critical object

Extents

New file<tablespace>.data2

Back up extendedtablespace and control files

Tablespace Extension

RESIZE THE DATA FILE:Extend the size of the data file dependingon the space available on the file systemand size of critical object.

<tablespace>.data1

Critical object

Original size After Resize

OR

n If the database tries to allocate another extent but finds that there is no more freespace in the corresponding tablespace, the SQL operation fails. The available storage space of a tablespace can be extended in online operation by adding another data file.

n To add a data file, use the SAPDBA. Select c -Tablespace administration.

� Specify the name of the tablespace to be extended in the submenu a - Tablespace

� In the submenu f - Alter tablespace <tablespace name> add data file default recommendations for file name and data file size are already given. Adapt them according to your requirements.

� Select s - Start to start the Add data file action. SAPDBA performs a check on the available storage space in the specified file system before the data file is added.

� SAPDBA continues with the backup menu and asks you to back up the extended tablespace. Backing up the extended tablespace ensures that the new state of the database can be recovered. SAPDBA stores the old version and the new version of the control file in directory sapreorg/<timestamp>. The action log <timestamp>.ext is written to directory sapreorg.

n To resize a data file, use SAPDBA. Select d - Reorganization.

� Then select option h - Resize data file of a tablespace.

� Specify the name of the tablespace to be extended in the submenu a - Tablespace

� Select s - Start to start the resiz ing process. SAPDBA gives a list of data files, from which you can select which data file you want to resize. In the submenu b - New size default recommendations for the data file size are already given. Adapt them according to your requirements and select s - Start and execute changes. SAPDBA performs a check on the available storage space in the specified file system before the data file is resized. It also writes a log file with <timestamp>.rrs name in the sapreorg directory.

SAP AG 1999

Table TGORA (storage parameters for R/3 tables)

Category INIT NEXT MINEXTENT MAXEXTENT0 16 40 1 3001 16 160 1 3002 16 640 1 3003 16 2560 1 3004 16 10240 1 3005 16 20480 1 3006 16 40960 1 3007 16 81920 1 3008 16 163840 1 3009 16 327680 1 300

10 16 655360 1 15011 16 1310720 1 15012 16 2621440 1 15013 16 5242880 1 15014 16 10485760 1 150

Table IGORA (storage parameters for R/3 indexes)

Category INIT NEXT MINEXTENT MAXEXTENT0 16 40 1 3001 16 80 1 3002 16 160 1 3003 16 640 1 3004 16 2560 1 3005 16 5120 1 3006 16 10240 1 3007 16 20480 1 3008 16 40960 1 3009 16 81920 1 300

10 16 163840 1 15011 16 327680 1 15012 16 655360 1 15013 16 1310720 1 15014 16 2621440 1 150

R/3 xABAP Dictionary: Display technical settings

Logical storage parameters

Name ZPROGRAM Transparent Table Short text Test Table for technical settings Last changed TND 24/08/1999 Status Active Saved

Data class APPL1 Transaction data. transparent tablesSize category 4 Data records expected: 39.000 to 3.100.000

Storage Categories of SAP Database Objects

n Default values are used for INITIAL, NEXT, and MAXEXTENT when creating an SAP table or index during installation of an R/3 System. These defaults are derived from the objects category.

n The category assignment for each R/3 table and index is a technical setting. You can access the technical settings of R/3 tables and indexes using the viewing (SE12) and editing (SE11) transactions for ABAP Dictionary objects.

n The INITIAL, NEXT, and MAXEXTENT values used for a specific category are defined in table TGORA for R/3 tables and in IGORA for R/3 indexes.

� NEXT sizes range from 40 KB, for tables in category 0, to 20971520 for tables in category 14

� The NEXT size for a category is either twice or four times the size of the NEXT size for the previous category. This helps to prevent external fragmentation.

� The MAXEXTENT for R/3 tables and indexes is usually set to 300. If the number of extents for a database object approaches 300, you must increase this parameter. As of Oracle release 7.3, you can set this parameter to UNLIMITED.

n Uncontrolled growth of the number of extents present in the database can increase the number of displacements in the Oracle shared pool. Because it is essential to limit the growth of the extents, we do not recommend setting the MAXEXTENT to UNLIMITED for all objects. The growth of an object, such as a table, index, or rollback segment is determined by the size specified in parameter NEXT.

SAP AG 1999

DBA Planning CalendarR/3 xPlanning Goto Listing Help System

MONMON TUETUE WEDWED THUTHU FRIFRI SATSAT SUNSUNsapdbasapdba-next-next



sapdbasapdba-next-next


Example sapdba -next:

Table size: 900 MB10 % value: 90000KBCurrent NEXT: 20480KBNEXT candidate: 90000KB

Next larger TGORA value: 163840

NEXT candidate: 90000Next smaller TGORA value: 81920

Current NEXT: 20480

Technical settings: Category 3: 2560

10485760

5242880

2621440

1310720

655360

327680

163840

81920

40960

20480

10240

2560

640

160

40

TGORAvalues (KB)

New value for NEXT: 163840

Using sapdba -next

Database tableDBMSGORA

1 Calendar

Schedule an Action for Tue 0518:00:00Start time

Period (weeks):




Partial database offline backupPartial database online backup

Check optimizer statistics

Update optimizer statisticsAdapt next extents

Check database

S ò0 rñ0! Start immediately

n The SAPDBA option -next allows you prevent uncontrolled extent growth for all database tables and indexes. The size of parameter NEXT for all R/3 database objects is calculated according to the following algorithm:

� The storage space allocated for the database objects (in KB) is determined and divided by 10. This value is compared to the current setting for NEXT. The larger of the two values, which is called the “NEXT candidate”, is used to perform the following comparisons:

� The NEXT value derived from the technical settings of the object is used if it is larger than the NEXT candidate

� If it is not, the next smaller value found in TGORA or IGORA is used if it differs by not more than 5 blocks from the NEXT candidate

� If it does not, the next larger value found in TGORA or IGORA is used

� If no larger value is found in TGORA or IGORA, parameter NEXT is set to the value of the NEXT candidate

n Schedule SAPDBA -next using the R/3 DBA Planning Calendar. Schedule SAPDBA -next to run at least once a week, and after major database changes. The action log is written to file <timestamp>.nxt in the directory ../../sapcheck.

n It may be the case that not enough contiguous storage space is available to allocate a new extent for tables that have an increased setting of parameter NEXT. After each SAPDBA -next run, you must check for space critical objects in the database.

SAP AG 1999

Daily Monitoring: Tables and Indexes

Database ORACLE Date/time of this analysis 08/24/1999 07:01:30Name TCC

Database System

RefreshRefresh ChecksChecks Space statisticsSpace statistics

Total number 27Total size/kb 12.123.016Total free/kb 3.050.320 25%Minimum free/kb 4.024Max. autoextensible/kb AutoExtend off

Tablespaces

Current sizesCurrent sizes

Freespace statisticsFreespace statistics

Space statisticsSpace statistics

Tables IndexesTotal number 13.064 15.305Total size/kb 5.915.664 2.979.432More than 1 extent 1.082 1.591Missing in database 0 1Missing in R/3 DDIC 0 0Space-critical objects 0 0

Tables and indexesDetailed analysisDetailed analysis

Space critical objectsSpace critical objects

Missing indexesMissing indexes

Space statisticsSpace statistics

Day Time M T W T F S S 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 ------------------------------------------------------------------------------- RSDBPREV 1 C

X:X:X:X:X:X:X X:X:X:X:X:X:X:X:X:X:X:X:X:X:X:X:X:X:X:X:X:X:X:X

RSORATDB 1 CX:X:X:X:X:X:X : : : : : : :X: : : : : : : : : : :X: : : : :

Maintain table: TCOLL

n To perform a detailed analysis of storage related issues, use the Tables and Indexes Monitor (transaction DB02).

n The Tables and Indexes Monitor has three levels of resolution: Database level, Tablespace level, and the Tables and Indexes level. In any one of the generated views, double -click the line of an object to display the elements of the object. From the lowest level, that is from the Tables and Indexes level, you can access a history view for this object. Navigating to higher levels of resolution works for most of the available views.

n The ABAP report RSORATDB collects the data required for the Tables and Indexes Monitor and stores it in the database table MONI. The contents of this table are evaluated when the Tables and Indexes Monitor is displayed. To refresh the values, choose Refresh.

n To set the frequency of the execution of report RSORATDB (and other reports relevant for monitoring) maintain table TCOLL using transaction SM31. Also schedule job COLLECTOR_FOR_PERFORMANCEMONITOR to run hourly within your production system. For further information, see SAP Notes 12103 and 16083.

n The Oracle autoextend option is supported in R/3 Release 4.5B and higher.

SAP AG 1999

Tables and Indexes: Important Reports

09/05/1999 10:40:26 DB_SERVERCritical table/index growth

Intervals: 08/06/1999 - 09/05/1999 Measurements: 29 Table Name Type Size(KB) NextExtSize Extents MaxExtents FirstExtent Tablespace Total Growth (Kbyte) Total Growth defined %used (Kbyte) ATAB~0 INDEX 72.720 10.880 160 168 68 300 56 45.936 PSAPPOOLISMENINTT~ INDEX 2.416 2.400 80 31 30 300 10 16 PSAPSTABISKAT~001 INDEX 9.960 2.080 80 89 26 300 30 2.904 PSAPSTABI

Critical growth of tables/indexes in the last 4 weeks

Table Space History09/05/1999 10:40:26 DB_SERVERIntervals: 08/06/1999 - 09/05/1999 Measurements: 30 Scale: DayScale: Day Size (KB) Free(KB) Used (KB) %-Used Tables/Indexes ExtentsTablespace Total Chg/day Total Total Chg/day Total Chg Total Chg/day Total Chg/dayPSAPPOOLD 747.512 0 38.944 708.568 2.415 94 0 6.619 0 8.692 22PSAPPOOLI 563.200 0 97.888 465.312 1.793 82 0 6.751 0 9.296 21PSAPSTABI 921.600 0 70.288 851.312 1.909 92 0 4.404 0 6.156 10PSAPSTABD 1.013.744 0 31.024 982.720 3.384 96 0 3.325 0 4.307 8PSAPROLL 307.200 0 182.392 124.808 3.084 40 1 15 0 120 3PSAPBTABD 735.216 0 231.336 503.880 1.249 68 0 2.344 0 2.471 1PSAPBTABI 409.600 0 125.768 283.832 157 69 0 3.222 0 3.542 1PSAPSOURCED 102.400 0 69.296 33.104 6 32 0 47 0 49 0PSAPSOURCEI 102.400 0 47.520 54.880 3 53 0 54 0 60 0PSAPTEMP 307.200 0 307.192 8 36 0 0 0 0 0 0PSAPUSER1D 8.192 0 4.024 4.168 36 50 0 4 0 4 0PSAPPROTI 33.792 0 16.592 17.200 52 50 0 114 0 131 0

n To display a list of objects with the strongest growth of extents or with more than 80% of MAXEXTENTS allocated, call transaction DB02, and choose Checks → Check next extent size. To display a history view for a single table or index, double -click a line describing an object and choose History per weeks.

n To monitor the size, free space, and usage of storage space for all tablespaces, choose Space statistics from the tablespace section. To display the history of a specific tablespace, double -click the tablespace line. Daily, weekly, and monthly histories are available.

n To display a detailed analysis of the current storage usage of a table or an index, choose Detailed Analysis from the tables and indexes section, and enter a name of a table or index. You can also specify the name of a tablespace to display a list of its elements.

n To analyze critical storage-related problems, choose Space critical objects.

n To display a list of indexes that are defined in the R/3 Data Dictionary but do not exist on the database, choose Missing indexes in the Tables and indexes section.

SAP AG 1999

Detail log: 9908221023.aly ***********************************************************************

(12690 tables analyzed - sorted by empty space in descending order)TABLESPACE_NAME TABLE_NAME EMPTY(kb) NEVER_BEEN_USED(kb) USED(kb)

PSAPSTABD E071K 42032 42032 5128PSAPEL40AD D010L 31112 31112 3168PSAPPOOLD ATAB 17112 17112 58368PSAPES40AD DOKCLU 13136 13136 323448PSAPES40AD D010S 10088 10088 774848***********************************************************************CHARTS OF 20 EMPTY INDEXES - USING: VALIDATE STRUCTURE(6751 indexes analyzed. sorted by empty space within BTREE =USED_BY_BTREE - USED)TABLESPACE_NAME INDEX_NAME TOTAL(kb) USED_BY_BTREE(kb) USED(kb) PSAPPOOLI ATAB~0 72720 72648 50563PSAPPOOLI T512T~0 7280 7200 4641PSAPPOOLI RTXTL~0 11920 10072 7745PSAPPOOLI T800Y~0 8800 8744 6726

SAPDBA Detail Log

sapdba -analyze

DB02 >> Detail Analysis

Data from DBA_SEGMENTS/DBA_INDEXESSpaceAllocated space..Kbyte 72.720 blocks...... 9.090 extents..... 168

Block structureBlocksize.........byte 8.192Pct_free.............. 10Transactions initial.. 2 maximum.. 255Header minimum....byte 159Data maximum......byte 7.230

Process freelists..... 1Freelist groups....... 1

Detailed analysis for Index ATAB~0

Analyzing Internal Fragmentation

n The SAPDBA option -analyze provides information about the storage allocation for a table or an index. You can use it to determine the degree of internal fragmentation of the database object, and to analyze a single table using SAPDBA.

n To display the logs of the SAPDBA actions performed to refresh the optimizer statistics, use the DBA Operation Monitor (transaction DB24). To display only the action logs of SAPDBA -analyze, choose Function IDs (function ID aly).

n USED space is storage space consumed by the table or index contents. EMPTY is the difference between storage space allocated for a table and USED space.

n NEVER_BEEN_USED space is storage space that has been allocated for a table but has not yet been used by the table. USED_BY_BTREE is the storage space allocated by the B*tree.

n In the sapdba -analyze detail log, check for:

� Tables that have a large difference between EMPTY and NEVER_BEEN_USED. This difference indicates internal fragmentation of a table.

� Indexes that have a large difference between USED_BY_BTREE and USED. This difference indicates internal fragmentation of an index.

n The Tables and Indexes Monitor (transaction DB02) provides detailed information about tables and indices. Call transaction DB02, choose Detailed Analysis and, in the dialog box displayed, specify the name of the object you want to analyze. Select the object and choose Detail Analysis. Block level details for that object are then displayed.

SAP AG 1999

Example for a table reorganization

1) Export table

2) Drop table

3) Import table

Recreate index

sapreorg

Data_1

001

2 21

Example for an index recreation

1) Drop index

2) Recreate indexPSAPTEMP

Additional storage space required

Additional storage space required

Data_1

001

2 21

Data_1

0

0

21

Data_1

01

02

Reorganization: Basics

n To eliminate storage-related problems, perform a reorganization. The objects that can be reorganized include tables, indexes, and data files of the database.

� After a table or index is reorganized, block usage for the reorganized object is optimal. Extents can be merged together to reduce the number of extents present in the database. Storage space required for the object can also be minimized.

� When data files of a tablespace are reorganized, some data files are merged together. This means the number of data files present in the database can be minimized.

n To perform a reorganization, additional storage space is required to store intermediate data. This storage space is either required in the database or in directories created for this purpose. Intermediate data can also be stored on tape. SAPDBA tries to forecast the amount of additional storage space required. When a reorganization is performed, bottlenecks can occur in the following areas:

� The data files of the tablespace where the reorganized object resides. Typical error: No data file has enough freespace to hold the new larger extents or a temporary second copy of the object.

� The directory sapreorg. Typical error: This directory is too small to hold the export sets

� The tablespace PSAPTEMP. Typical error: This tablespace is too small for an index recreation

n If an error occurs during a reorganization, data can be lost. If a valid and up-to-date data backup exists, the risk of data loss is minimized. After the data files of a tablespace have been reorganized successfully, you must immediately perform a backup.

SAP AG 1999

Disk_4

102 3

0

10

4

2

5

3

102 3

0

10

4

2

5

3

Disk_3

102 3

0

10

4

2

5

3

102 3

0

10

4

2

5

3Disk_2

102 3

0

10

4

2

5

3

102 3

0

10

4

2

5

3Disk_1

102 3

0

10

4

2

5

3

102 3

0

10

4

2

5

3

Disk "hot spots"

Dis

k ac

cess

es [

%]

Disk

Reorganization: Reasons

Detail log: 9909050715.alyCHARTS OF 20 EMPTY INDEXES - USING: VALIDATE STRUCTURE(6751 indexes analyzed. sorted by empty space within BTREE= USED_BY_BTREE - USED)TABLESPACE_NAME INDEX_NAME TOTAL(kb) USED_BY_BTREE(kb) USED(kb)

PSAPPOOLI ATAB~0 72720 72648 20563PSAPPOOLI T512T~0 7280 7200 4641PSAPPOOLI RTXTL~0 11920 10072 7745PSAPPOOLI T800Y~0 8800 8744 6726PSAPPOOLI T52C5~0 5480 5424 3760PSAPPOOLI USOBX_C~0 4160 4032 2627

SAPDBA Detail Log

FragmentedIndexes

. . .00 11 22 33 44 55

Oracle block

Internalfragmentation

Free

Used

n Reorganizing physical data objects is costly, both in terms of time and resources, and should be performed only in exceptional cases. To minimize the need for reorganizations, monitor the database regularly.

n The R/3 System is not available during a reorganization. Therefore, reorganize physical data objects for the following reasons only:

� Disk hot spots: Unfavorable physical distribution of tables, indexes, or data files results in non-uniform distribution of disk accesses on available disks

� Heavily fragmented indexes: Fragmented indexes can affect performance. Fragmented tables will only affect performance in special cases (check for SAP Notes indicating this problem).

n To avoid having to perform reorganizations:

� Run sapdba -next weekly to adapt the storage parameters of all tables and indexes. This prevents database objects from allocating a high number of extents. If a table has a high number of extents, change the parameters NEXT and MAXEXTENTS. Do not perform a reorganization.

� Try to estimate growth of a critical tablespace before extending it with a data file of an appropriate size. Use SAP archiving to archive obsolete data. This prevents the number of files in your database system from approaching the limit DB_FILES. If there is a high number of data files, try to increase the parameters MAXDATAFILES (number of files your OS can handle) and DB_FILES. Do not reorganize a tablespace.

SAP AG 1999

Moving / Renaming data files

Disk x

102 3

0

1

0

4

2

5

31 4

0

Disk y

102 3

0

10

4

2

5

31 4

0

Reorganization of a tablespace

Data file_1

10

2 3

0

1

0

4

2

5

3

Data file_2

76

8 9

1

5

4

10

6

11

7

Data file_2

0

Data file_1

0

0

0

Reorganization of a tablespace with data files

Data file_1

Data file_2

102 3

0

10

4 52

76

8 9

1

5

4

10 11

Data file_new

0

0

0 00

Reorganization of a singleobject or a list of objects

Phases Directories� Create script and restart file sapreorg/<timestamp>/<timestamp>.<extension>.

Check the free space sapreorg, PSAPTEMP.� Perform a reorganization PSAPROLL, objects tablespace

Phases Directories� Create script and restart file sapreorg/<timestamp>/<timestamp>.<extension>.

Check the free space sapreorg, PSAPTEMP.� Perform a reorganization PSAPROLL, objects tablespace

Data file_1

001

2 21

Data file_1

0

0

21

Disk "hot spots" Fragmented indexes

Internal fragmentationInternal and external fragmentation

Fragmented indexes

Disk "hot spots"

Reorganization: Phases and Types

n SAPDBA reorganizations are performed in two phases:

� In phase one, the SQL script for the reorganization process is created in the subdirectory <timestamp> of the working directory. The file restart.<extension> is created in this subdirectory to enable a restart of the reorganization (first eliminate the cause of the error). Reversible actions can be reset. SAPDBA checks if there is enough storage space available to perform the reorganization.

� In phase two, the reorganization script is executed. (Action log name: <timestamp>.<extension>)

n Reorganization of a single object (table or index): Use this function to eliminate internal fragmentation, to reduce the number of extents allocated for a table or index, and to move a table or index to another tablespace in order to eliminate a disk hot spot (extension rsi)

n Reorganization of a list of objects: Use this function to reorganize a group of several objects. An ASCII file must be created in the working directory with the required list of objects. (extension rli)

n Reorganization of a tablespace: Use this function to reorganize all objects belonging to this tablespace. The directory and file structure of the tablespace will remain unchanged. (extension rtc)

n Reorganization of a tablespace with data files: Use this function to change the data file structure and to reorganize the objects contained in the tablespace. The number of data files existing in the database can be minimized. (extension rtd)

n Moving and renaming data files: Use this function to move files to another disk. This is not classified as a reorganization because the action is performed on file level (extension rmv).

SAP AG 1999

Legend:++ = very good, + = good, o = average, NA = Not applicable

I = Parallel on index, T = Parallel on table. (Oracle PARALLEL clause)

P = Parallel on process level (SAPDBA creates several processes)

Reorganization of tables

Method Runtime Security Additional space R3Chop Parallel Compress Restrictions

Create table ++ ++ Objects tablespace NA T NA No tables withas select PSAPROLL LONG/RAW fields

SAPDBA unload. + + sapreorg NO P NO export < max_file_sizeSQL*loader PSAPTEMP (usually 2 Gbyte)

Oracle o + sapreorg YES P YES export/import PSAPTEMP

Reorganization of indexes

Method Runtime Security Additional space Parallel

Alter index ++ ++ Objects tablespace Irebuild PSAPTEMP

Index + + PSAPTEMP Irecreate

Reorganization: Methods

n There are two methods to export and import table data. In both cases, because tables are dropped before they are recreated, data may be lost :

� export / import uses Oracle EXPORT and IMPORT commands. Additional storage space for the export dump is required in directory sapreorg and in PSAPTEMP (index recreation).

� SAPDBA unload / load uses either the tool SAPDBA LOAD or the Oracle SQL*loader. Additional storage space for the LOAD file is required in directory sapreorg and in PSAPTEMP (index recreation).

n The following methods do not require the export and import of database data:

� Create table ... as select: SAPDBA first generates the table to be reorganized with the new parameters under a new name (by adding the character #). The data is copied directly from the old table to the new. The old table is dropped and the new table is renamed. This option cannot be used for tables with LONG columns or for reorganizing tablespaces with data files. Additional storage space is required as the table temporarily exists twice in the same tablespace. Enough space must be available in PSAPROLL to hold rollback information.

� Alter index / Rebuild: The index is first rebuilt in tablespace PSAPTEMP using the existing index. Then it is copied into the corresponding tablespace. The old index is dropped and the optimized index is activated. Table and index are locked during this process. Additionally required storage space can be up to twice the size of the index.

� Recreate index: The index is dropped and recreated. Storage space is required in PSAPTEMP.

SAP AG 1999

Compress extents:Yes No

0

0 1

1 2

0

0

2 3

1 4

0 1 2 3 4

Reduce object size:Yes No

0

0 1

1 2

0

0

2 3

1 4

0

Freespace

Chop (export dump):Yes No

sapreorg2 GB2 GB

sapreorg4 GB

Compress (export dump):Yes No

sapreorg1 GB

2 GB

sapreorg

2 GB

2 GB

Parallel export / import(Example: 2 processes in parallel)exp_imp_degree = 2

sapreorg1dump1

sapreorg2

dump2

Database

Process 1 Process 2

Both conditions required:exp_imp_degree = 2

and2 dump destinations

Possible?Possible?

Reorganization: Options

n To define a reorganization, you can specify the following options:

� Compress extents = Yes. SAPDBA merges the extents occupied by the table or index to one extent. If this option is not used, the extents are allocated using the current storage parameters of the object. Adjacent free storage fragments in the entire tablespace are merged.

� Reduce object size = Yes. SAPDBA automatically tries to reduce the size of the objects (allocated storage space) during the reorganization for an export or import. To determine the actual storage space occupied, SAPDBA uses the Oracle ANALYZE command. Script generation requires a specific amount of time and locks the tables to be reorganized.

� Chop = Yes. SAPDBA sends the exported data to the chop tool through a named pipe. The chop tool then splits the export data into several files. This option is only available if parameter chop_util_name is entered in the profile init<SID>.dba. This option is not available for Windows NT. If the export dump files are larger than the maximum file size (usually 2 GB) for the operating system, use this option.

� Compress = Yes. The export dump is compressed using OS facilities (not for dump to tape).

� Parallel export/import: The export and the import are distributed to several parallel processes. The init<SID>.sap parameter exp_imp_degree determines the maximum number of processes that are created for the reorganization. The number of directories and/or tape devices specified for the export dump files also limits the number of processes created.

� Oracle PARALLEL clause: The reorganization is accelerated with the help of the Oracle PARALLEL QUERY functionality.

SAP AG 1999

Unit Summary


l Explain the concepts of Oracle storage management

l Name the problems of data storage

l Monitor and avoid problem situations

l Solve storage related problems efficiently

n In an Oracle database, the way physical hard disk space is allocated for a table or index is controlled by the objects storage parameters. Storage parameters that are set incorrectly can lead to unwanted growth behavior.

n The following R/3 System tools provide you with an effective means of preventative monitoring, and should be used to avoid uncontrolled database growth:

� The SAPDBA checks the database for possible storage related problems

� You can extend the database using the SAPDBA

� The SAPDBA adapts storage parameters of growing objects to optimize growth behavior

� The SAPDBA provides comprehensive support for the reorganization of Oracle database objects

� You can monitor the current status of the database using the CCMS Database Monitors (DB02, ST04, DB24).

n You can limit growth of the database by archiving obsolete data. R/3 provides you with the required archiving functionality.

SAP AG 1999

l Exercises?

l Solutions

Unit Actions

Storage Management: Exercises

No. Exercise

1 Use sqlplus to run the script createtab.sql, which is located in directory scripts:

Log on as user ora<SID>. If the database is not open, open it using SAPDBA. Switch to directory scripts. Start sqlplus. Log on to the database as user SAPR3 with password SAP. Start the script createtab.sql by entering the sqlplus command line @createtab.sql

2 Run sapdba –check on your database.

2.1 Check the log file of sapdba –check for messages regarding table BC505CHECK and index BC505CHECK~0.

3 Adapt the storage parameters of table BC505CHECK using SAPDBA. Ensure that the new MAXEXTENTS value is larger than number of extents currently allocated + 30%. Do not use the SAPDBA recommendation for NEXT, but set it back to the current value.

3.1 Run sapdba –check again. Explain the difference in the log files of the sapdba –check runs before and after you adapted MAXEXTENTS of table BC505CHECK.

4 Use SAPDBA to automatically adapt the NEXT extent sizes for all SAP tablespaces.

4.1 Check for problems in the sapdba –next log.

4.2 Run sapdba –check again and check for problems related to table BC505CHECK in the log file. Why should you always run sapdba –check after a sapdba –next?

5 Solve the problems found for tablespace PSAPUSER1D by adding a new data file (1 MB in size).

5.1 Solve the problems found for tablespace PSAPUSER1I by adding a new data file (200 KB in size).

5.2 Restart sapdba –check and check for problems in the sapdba –check log. Explain what happened to the entries for table BC505CHECK and index BC505CHECK~0.

6 Why is it important to run a backup after you have changed the file structure?

7 Check for internal fragmentation of tablespace PSAPUSER1D

7.1 Using “estimate sample 10 percent”

7.2 Using “compute statistics / validate structure”

7.3 Check the log files of the analyze actions. What is the difference between both methods?

8 Reorganize the index tablespace PSAPUSER1I using Alter Index Rebuild, the storage options compress extent and reduce object size, and hide the table during reorganization.

8.1 Check the log file of the reorganization.

9 Reorganize tablespace PSAPUSER1D including the data files, using the SAPDBA unloader / ORACLE SQL*loader, with option reduce object size, reduce data file size (accept the default).


10 Optional: Reorganize tablespace PSAPUSER1D including the data files using the Oracle export/import, reduce object size and hide table during reorganization.


11 Optional: Run script extent.sql from SQL*Plus to create an extent problem in tablespace PSAPUSER1D. Now execute the script extent_run.sql. Analyze the error message.

11.1 Correct the error and try to successfully complete the script extent_run.sql.

12 Optional: Run script ts_over.sql from SQL*Plus. Analyze the error message.

12.1 Solve the problem.

Storage Management: Solutions

No. Solution

1 2 As user ora<SID>, issue command sapdba –check.

2.1 After SAPDBA has finished, switch to directory sapcheck and look for the file <timestamp>.chk. Edit the file (using command more <check file name>, and use the spacebar to go to the next page). The following messages are displayed: SEGMENT_MANY_EXTENTS : BC505CHECK TABLESPACE_FULL : PSAPUSER1I (This problem will be solved later.)

3 Start SAPDBA, and choose: d – Reorganization → b - Alter/show table or index storage parameters → b - specify table name BC505CHECK → s - Alter/show parameters → b - NEXT, enter current value → d – MAXEXTENTS, enter new value → s – commit.

3.1 After SAPDBA has finished, switch to directory sapcheck and look for the file <timestamp>.chk. Edit the file (using command more <check file name>). The following message should no longer be displayed: SEGMENT_MANY_EXTENTS : BC505CHECK

4 Run sapdba –next PSAP%. Then run sapdba –check again.

4.1 After SAPDBA has finished, switch to directory sapcheck and look for the file <timestamp>.nxt. NEXT for BC505CHECK has been changed by sapdba –next.

4.2 BC505CHECK is now reported as a critical object because PSAPUSER1D is too small to hold a NEXT extent of this table. As sapdba –next can increase the NEXT size of tables or indexes, the larger NEXT extents may not fit into the tablespace anymore. Run sapdba –check to detect these problems.

5 Start SAPADBA and choose: c - Tablespace administration → a – Tablespace , enter PSAPUSER1D → f – Alter tablespace PSAPUSER1D add data file → c – New size, enter 1M → s – Start.

You can skip the backup action recommended by SAPDBA.

5.1 Start SAPADBA and choose c - Tablespace administration → a – Tablespace, enter PSAPUSER1I → f – Alter tablespace PSAPUSER1I add data file → c – New size, enter 200KB → s – Start

You can skip the backup action recommended by SAPDBA 5.2 As PSAPUSER1D/PSAPUSER1I are now large enough to hold a NEXT

extent of BC505CHECK/ BC505CHECK~0, the critical object entries are no longer displayed.

6 You should always save the extended tablespace and the new version of the control file after the extension so that the complete recovery functionality of the SAPDBA is available (partial restore and complete recovery). To do this, you can use, for example, the SAP tool BRBACKUP. For this reason, after a tablespace extension, SAPDBA automatically branches to the backup database menu to enable you to

start the appropriate backup immediately. 7

7.1 Run command sapdba –analyze PSAPUSER1D -method E –option P10

7.2 Run command sapdba –analyze PSAPUSER1D -method CI

7.3 Log files are written in directory sapcheck. The log file name of an analyze action is <timestamp>.aly.

For the estimate method, empty and never_been_used always have the same value. For the compute statistics/validate structure method, the values for empty and never_been_used differ, as this method is more accurate.

8 Start SAPDBA and choose d – Reorganization → e – Reorganize tablespace → a – Tablespace , and enter PSAPUSER1I. Then choose g – Storage parameters → d – Reduce object size → q – return → q – return → h - Object handling → a – Hide tables during reorg → c – Rebuild indexes → q – return → s – Start. After generating a script, start the script immediately using option 1.

8.1 Check the file <timestamp>.rtc in directory sapreorg. 9 Start SAPDBA and choose d – Reorganization → f – Reorganize

tablespace and data files → a – Tablespace , and enter PSAPUSER1D. Then choose f – ORACLE exp/imp. Then choose b to toggle to Unload / load → q – return → g – Storage parameters → d – Reduce object size → q – return → h – Object handling → d – Reduce data file size (use 10% default) → q – return → s – Start. After generating a script, start the script immediately using option 1.

9.1 Check the file <timestamp>.rtd in directory sapreorg. 10 Start SAPDBA and choose d – Reorganization → f – Reorganize

tablespace and data files → a – Tablespace , and enter PSAPUSER1D. Then choose g - Storage parameters → d – Reduce object size → q – return → q – return → h - Object handling → a – Hide tables during reorg → q – return → s – Start. After generating a script, start the script immediately using option 1.

10.1 Check the file <timestamp>.rtc in directory sapreorg. 11 Change to directory /<ORACLE_HOME>/scripts, start SQLPLUS, log on

as user SAPR3, and run @extent.sql. Now execute the script extent_run.sql. A MAXEXTENT problem is displayed for table BC505CHECK.

11.1 Increase parameter MAXEXTENTS for table BC505CHECK (see exercise 3). 12 Change to directory /<ORACLE_HOME>/scripts, start SQLPLUS, log on

as user SAPR3, and run @ts_over.sql. Now execute the script ts_over_run.sql. A tablespace overflow will be displayed for tablespace PSAPUSER1D.

12.1 Add a data file to tablespace PSAPUSER1D (see exercise 5).

SAP AG 2000






Top 10 Problems

SAP AG 1999

Top 10 Problems

Contentsl The most frequent problems that occur when R/3 is installed on

an Oracle database

ObjectivesAt the end of this unit, you will be able to:l Recognize and solve these problemsl Prevent these problems from occurring

n This unit explains the ten problems that most frequently occur when R/3 is installed on an Oracle database.

n Once you have completed this unit you will be able to:

� Recognize and solve these problems

� Prevent many of these problems from occurring, before they become problem

SAP AG 1999

Run SAPDBA -checkRun SAPDBA -check

Check forerror messages

in:

Check forerror messages

in:

R/3 Syslog

SAP Note Search

Oracle backgroundprocess trace files

in directory:saptrace/background

Oracle backgroundprocess trace files

in directory:saptrace/background

Oracle user trace filesin directory:

saptrace/usertrace

Oracle user trace filesin directory:

saptrace/usertrace

R/3 startdb.login the home directory

of user <SID>adm

R/3 startdb.login the home directory

of user <SID>adm

Oracle alert log file:saptrace/background

/alert_<SID>.log

Oracle alert log file:saptrace/background

/alert_<SID>.log

Check SAPNetfor solutions

Troubleshooting Steps

n When a problem occurs, run sapdba -check and check the log file in directory sapcheck .

n To begin troubleshooting, you can check the R/3 System log or the following Oracle files:

� The Oracle alert log file ORACLE_HOME/saptrace/background/alert_<SID>.log, which often contains further information, such as the ORA-XXXX return code that may identify the problem

� The Oracle background process trace files in directory saptrace/background

� The Oracle user trace files in directory saptrace/usertrace

� The startdb.log file in the home directory of user <SID>adm, which contains error information in the case of database instance startup failure

n To check the R/3 System log, use transaction SM21.

n You should also search for related SAP Notes in SAPNet. If you cannot find an applicable SAP Note, create a customer message in SAPNet, and provide the SAP hotline with the following information:

� Syslog messages and the short dump (transaction ST22) related to the error

� Error messages from the Oracle alert log file

� Error messages from the Oracle background process and user trace files

� Error messages from the file startdb.log

� Log files of SAPDBA or BR* tools

SAP AG 1999

The most frequently occurring problems are:

l An archiver stuck situation

l The incorrect tape size on tape drives with hardware compression

l A missing "end backup"

l A tablespace overflow

l A table or index reaching the MAXEXTENTS limit

l Oracle error ORA-1555, snapshot too old

l Net8 TCP/IP delay

l Oracle error ORA-1578, data block corruption

l Oracle error ORA-600, internal database error

l Poor performance of the cost-based optimizer

Top 10 Problems

n The ten most frequent problems that occur when R/3 is installed on an Oracle database are:

� An archiver stuck situation

� The incorrect tape size on tape drives with hardware compression

� A missing “end backup”

� A tablespace overflow

� A table or index reaching its MAXEXTENTS limit

� ORA-1555, snapshot too old

� Net8 TCP/IP delay

� ORA-1578, data block corruption

� ORA-600, internal database error

� Poor performance of the cost-based optimizer

SAP AG 1999

SAPGUI is hangingDatabase in

ARCHIVELOGmode

Archiverstuck

Offline redo logdirectory saparch

Log fileswitches

Error

ORA-255 orORA-272

DBMS

Online redo logs

Archiver Stuck Situation

n The database of a production system should always be operated in ARCHIVELOG mode with an active Archiver process.

n When the database is in ARCHIVELOG mode, an online redo log file can only be reused after it has been completely archived to the offline redo log archive directory saparch. If the directory saparch is full, the online redo log files cannot be completely archived, and the database will enter the archiver stuck state. This means that no further changes are possible to the system.

n If this occurs, the Oracle error message ORA-255 or ORA-272 is written to the database alert log.

n The following is an example of the Oracle alert log file alert_<SID>.log:

Tue May 19 10:25:11 1998

ORA-00255: error archiving log 11 of thread 1, sequence # 1337

ORA-00312: online log 11 thread 1:'/oracle/TC1/origlogA/log_g11m1.dbf’

ORA-00312: online log 11 thread 1: '/oracle/TC1/mirrlogA/log_g11m2.dbf’

ORA-00334: archived log: '/oracle/TC1/saparch/TC1arch1_1337.dbf’

ORA-19502: write error on file "/oracle/TC1/saparch/TC1arch1_1337.dbf"

n When an archiver stuck situation occurs, the R/3 Systems hangs, and you cannot perform transactions until the online redo log file has been completely archived to directory saparch.

SAP AG 1999

Monitor directory saparch

Do not deleteor move

offline redolog files

Create a "dummy file"5 times the size of an

online redo log file

Offline redo logdirectory saparch

Reserve three times theamount of the spacerequired for the dailyoffline redo log files

DatabaseAdministration

log_archive_start = truein init<SID>.ora

Avoiding an Archiver Stuck Situation

n To avoid an archiver stuck situation:

� Reserve some disk space by creating a “dummy file” in the archive directory saparch. Make the size of the dummy file five times the size of the offline redo log file. If saparch becomes full, remove the dummy file and run BRARCHIVE immediately.

� If BRARCHIVE is scheduled to run daily, ensure that saparch has enough disk space to hold at least three times the amount of the daily offline redo log files.

� Monitor how full saparch is.

� Check the file init<SID>.ora to see if parameter log_archive_start = true.

� Check if user ora<SID> has write authorization to saparch.

n Warning: Do not delete or move files from the archive directory saparch.

SAP AG 1999

cpio or dd error:end of tape

reached

Redefine the parameter tape_sizein file init<SID>.sap

Recalculate the compression ratioonce per backup cycle

Redefine the parameter tape_sizein file init<SID>.sap

Recalculate the compression ratioonce per backup cycle

Error

Incorrect Tape Size (Hardware Comp. Tape Drives)

n The physical tape size (tape length * write density) is specified in the parameter tape_size in the file init<SID>.sap. The physical tape size is the total volume of data that can actually be written to a volume without compression (in MB or GB).

n If parameter tape_size is configured too large, cpio or dd reaches the physical end of the tape. This causes severe problems when database restoration is required.

n When using tape devices with hardware compression, always leave a reserve of approximately 200 MB to take account of any errors in the calculation of the compression rate.

n When hardware compression is used, BRBACKUP does not receive any information about the compression ratios for the individual files. You can obtain this information from BRBACKUP by running a dummy compression. To do this, run command brbackup -compress|-k only . Recalculate the dummy compression ratios at least once per backup cycle, and after a lot of data change activity in the database, such as release upgrades and inserting large amounts of data.

n To calculate the compression ratio, use the -b 12 option of the compression command for the corresponding init<SID>.sap parameter. For example, the compression command parameter should be defined as follows: compress_cmd = "compress -b 12 -c $ > $”.

n See also SAP Note 8707.

SAP AG 1999

Online tablespace backupORA-1149 or

ORA-1113Missing "end backup"

Restoring data filesis not necessary

alter tablespace...begin backup

...in backup mode...

alter tablespace...end backup

Use the SAPDBA to:l Check which tablespaces are

in Begin backup model Shutdown the database for

error ORA-1149l Recover the database for

error ORA-1113

Error

Missing "end backup"

n If you back up a tablespace online without RMAN, the tablespace must be in begin backup mode during the backup. This ensures that the Oracle data file headers of the data files belonging to the tablespace in the online backup mode are not updated.

n If a tablespace is in begin backup mode and the command shutdown immediate is issued in SAPDBA, SAPDBA puts every tablespace into end backup mode before the database is shutdown.

n If a tablespace remains in begin backup mode and the command shutdown immediate is issued in svrmgrl (or command stopsap db is issued), the database returns error ORA-1149 (missing end backup).

n To check which tablespaces are still in begin backup mode, use SAPDBA and choose DB Check verification → DB System Check. The system then prompts the user whether SAPDBA should set the end backup automatically.

n If the database crashes during an online backup, or is powered down, or is shutdown by using shutdown abort, some tablespaces may remain in the begin backup mode. In this situation, because the tablespaces are in online backup mode when you try to open the database, error ORA-1113 is returned.

n For ORA-1113, you must perform a database recovery. To perform the recovery, use SAPDBA and choose Partial restore and complete recovery database.

n If a missing end backup error occurs, you do not need to restore any data files.

n The missing end backup error does not occur if the backup is performed with RMAN.

SAP AG 1999

<tablespace>.data2

New disk

Add data file

Tablespace

<tablespace>.data1

File size dependson the estimatedincrease in thetablespaceobjects

Gaps

Notenoughfree spacefor thisextent

Data objects

Insertions

Tablespace

Perform a backupafter every change infile structure

ORA-1653 orORA-1654

Error

Tablespace Overflow

Extents

n If an extent of a given size is required in a tablespace, but there is not enough contiguous freespace available in the tablespace, Oracle returns error ORA-1653 (for tables) or ORA-1654 (for indexes).

n The error messages are displayed in both the Syslog file and the ABAP short dump.

n To solve this problem, use SAPDBA to extend the tablespace with one extra data file. From the SAPDBA, choose Tablespace administration → Alter tablespace → Add Data file. If you extend several tablespaces, make sure that you place the data files on different disks, otherwise there may be problems with the I/O speed of disk access.

n You must perform a backup after every change in the file structure.

SAP AG 1999

Adjust thenext extent?Check

300543210 . . . . . .

Object

Initialextent

Next extents

MaxExtents. . .

. . .

ORA-1631 orORA-1632

Error

Increase MaxExtents

by 50

+

Adjust thenext extent?

l Monitor the number ofextents regularly

l Run sapdba -next weeklyl Adjust the MaxExtents

parameters

MaxExtents Limit is Reached

n If the number of extents allocated to an object reaches the maximum number specified in the MaxExtents storage parameter, then this object cannot request any more extents. When this maximum value is reached, Oracle reports error ORA-1631 (for tables) and ORA-1632 (for indexes). The error messages are displayed in both the Syslog file and the ABAP short dump.

n To avoid this problem:

� Monitor the number of extents regularly

� Run sapdba -next once a week

� Adjust the MaxExtents parameters as required

n To change the values of the next parameter for tables and indexes, run sapdba -next.

n To change the values of the MaxExtents parameter for tables and indexes, use the SAPDBA and choose Reorganization → Alter/show table or index storage parameters → Alter/show parameters.

n If the storage parameter MaxExtents is set to unlimited, the maximum number of extents is actually 2,147,483,645.

n Do not change MaxExtents to unlimited for all segments in your database. Do not set the MaxExtents to unlimited on rollback segments.

SAP AG 1999

Update...

Data after update

SELECT... -> fetchENDSELECT

Long running query

SELECT...

Long processing time

1

Overwritten rollback segment

data

Update 3 records and commit

1

23

Tables

32

ORA-1555

1

Data before update

Rollback segments

Error

ORA-1555: Snapshot Too Old

n Oracle enforces statement-level read consistency. This ensures that the data returned by a single query is consistent with respect to the time when the query began. Therefore, a query never sees the data changes made by transactions that commit during the course of execution of the query. Usually, a long running query has error ORA-1555 if the data accessed by the query is updated and committed by another user or session after the query has been started.

n The most common reasons for error ORA-1555 (snapshot too old) are:

� A long running query due to a poorly qualified data access

� A high processing time between fetches of the same query

� Incorrect rollback segment setup

n A long run query can cause some activities in the ABAP program loop to take too long, even if the SQL statement is correct. For example:

SELECT ...

Fetch (a long running activity)

ENDSELECT

n Before you try to change the number or size of the rollback segments, check if the problem is caused by expensive queries, such as inefficient application design, wrong index design, or insufficient cost-based optimizer statistics. You can adjust the rollback segment setup by adding more rollback segments or making them larger.

SAP AG 1999


listener.oraprotocol.ora

ORACLE_HOME/network/admin

Net8 Net8 TCP/IPTCP/IP Database server

Listener process

Shadow process

Shadow process

Shadow process

Shadow process

Wait for 200ms tofill the Net8 TCP/IPpackets causing

poor performance

Net8 TCP/IP packet


protocol.ora

ORACLE_HOME/network/admin

Remoteapplication server

R/3 work processes

R/3 work processes

R/3 work processes

R/3 work processes

Delay

Net8 TCP/IP Delay

n Net8 is the Oracle communication layer between the client (application servers) and the database server. Locally, it uses inter process communication (IPC); remotely, it uses TCP/IP.

n When you configure Net8, you can choose to open the connection with delay or no-delay for TCP/IP. By default, Oracle opens the connection in delay mode, which allows the TCP/IP implementation on the operating system to delay sending half-empty TCP packets. The wait cycle is approximately 200ms, which slows down the communication speed in R/3.

n To solve this problem, perform the following steps:

� Download the file protocol.ora from sapservX

� As user ora<SID>, copy this file into the ORACLE_HOME/network/admin directory of the database server and every application server (even though TNS_ADMIN may point to a different directory)

� Give read permission for file protocol.ora to users <SID>adm and ora<SID>

� Restart the Net8 listener on the database server

� Stop and start all the application servers.

n For further information, see SAP Note 72638 (Unix) and SAP Note 132937 (Windows NT).

SAP AG 1999

Data fileSAP Note Search

BC-DB-ORAComponent

Search Criteria:

ORA-1578

- x

Oracleblock

ORA-1578

Extents

Error

ORA-1578: Data Block Corruption

n Error ORA-1578 indicates a data block corruption. Data block corruption usually occurs because of a hardware error, and in most cases, remains undetected until the corrupted information is required.

n For further information, see SAP Notes 13952 and 23345 or search for latest SAP Notes about error ORA-1578.

n Once the hardware problem that caused the data block corruption has been solved, you may need to restore and recover parts of the database. If an index block is corrupted, you can drop and re-create the index.

n When a normal backup is performed, a corrupted data block will remain undetected. To detect data block corruption early, perform a database verification at least once in your backup cycle.

n To detect the corrupted data block, do one of the following:

� Use command brbackup -verify|-w only_dbv|use_dbv during a normal backup, or use the SAPDBA and choose k - DB check/verification DB verification using DB VERIFY

� Use profile parameter disk_copy_cmd=rman to do the backup, because RMAN automatically does a verify when it copies the blocks.

SAP AG 1999

saptrace/background

alert_<SID>.log

SAP Note Search

BC-DB-ORAComponent

Search Criteria:

ORA-600

12700

- x

Mon May 11 13:45:01 1998Errors in file/oracle/BIN/rdbms/log/ora_1207.trc:ORA-00600: internal error code,arguments: [12700],[2215], [1073777814],[4], [], [], [], []

Mon May 11 13:45:01 1998Errors in file/oracle/BIN/rdbms/log/ora_1207.trc:ORA-00600: internal error code,arguments: [12700],[2215], [1073777814],[4], [], [], [], []

Create Original Message x

BC-DB-ORAComponent

Short Text

Argument

ORA-600 detected

12700

-

ORA-600

Error

ORA-600: Internal Database Error

n ORA-600 indicates an internal database error.

n Record the first argument of the ORA-600 error message. In this example, the first argument is 12700.

n Search for any SAP Notes related to ORA-600 and the first argument.

n If no SAP Notes correspond to the particular ORA-600 problem, create a customer message in SAPNet, and attach the related trace files, alert_<SID>.log, and R/3 Syslog.

n Error ORA-600 is often followed by a state dump in the trace files. These trace files are found in directory saptrace/background or saptrace/usertrace. The alert_<SID>.log is in the directory saptrace/background.

SAP AG 1999

Performance Memory StructureBackup/RecoveryAll database operations

All Performance operations ( )

Influence of the Cost-Based Optimizer

Old statistic information can cause serious performance

problems

n The cost-based optimizer determines the most efficient access path based on analyzed statistics for tables and indexes. Outdated or incorrect statistics can cause severe performance problems.

n To prevent the cost-based optimizer using an inefficient access path that could cause performance problems, you must update the statistics regularly.

n In case of general performance problems, check whether the statistics are up-to-date first. To do this, display the DB Operation Monitor (transaction DB24) and choose Performance.

SAP AG 1999

Unit Summary


l Recognize and prevent the problems that occur mostfrequently when R/3 is installed on an Oracle database

SAP AG 1999

Section: SQL Cache Analysis - CBO - ORACLE

Techical Background Cost Evaluation

The Shared SQL Area Creating an Index

Analyzing SQL Statements Similar Statements

Update Statistics View Processing

Identify Coding Joins

Workflow and Reporting Expensive Statements

Index Utilization

SAP AG 1999

Introduction and Technical Background

Techical Background

The Shared SQL Area

Analyzing SQL Statements

Update Statistics

Identify Coding

Workflow and Reporting

Index Utilization

Cost Evaluation

Creating an Index

Similar Statements

View Processing

Joins

Expensive Statements

SAP AG 1999

Unit: Introduction and Technical Background

Contents:• Oracle architecture review

• Processing of SQL statements

Objectives:At the end of this unit you will be able to:

• Explain why you should use the Shared SQL Area to analyzeexpensive SQL statements

• Describe how Oracle processes an SQL statement


� Explain why the Shared SQL Area should be used to find and analyze expensive statements

� Describe how SQL statements are processed by Oracle

SAP AG 1999

Introduction

• Expensive SQL statements affect all users

• The Oracle Shared SQL Area can be used to:

• Find and analyze expensive SQL statements

• Identify performance issues that are not related tohardware configuration or database parameter settings

• For optimizing parameter and hardware configuration seeWorkshop EWB11

• Performance issues must be reported to the appropriate people

n Database problems are displayed in the Oracle Shared SQL Area. To identify the expensive SQL statements in your system, you can analyze the statements in the Shared SQL Area. An expensive SQL statement does not only affect the user running this statement, it affects other programs that are running as well. Therefore, expensive SQL statements affect all users.

n With the Shared SQL Area, you can identify performance issues that are not related to hardware configurations or database parameter settings. This course does not cover hardware or parameter modification. For optimizing parameter and hardware configuration see Workshop EWB11

n Performance issues must be reported to the appropriate people or groups, such as:

� Developers (coding, table, and index design)

� Application Designers (customizing and user requirements)

� SAP (standard coding, standard index, and table design)

n Although specific transactions can be analyzed using other monitoring tools, such as SQL Trace, these tools are normally used to find specific problems that affect a small percentage of users. These other monitoring tools are not covered in this course.

SAP AG 1999

Overview

• Oracle architecture basics

• Shared SQL Area

• SQL statement analysis

• Workflow and reporting

n This workshop begins with a review of the Oracle architecture basics, and then moves on to analyzing SQL statements in the Shared SQL Area.

n Several optimization possibilities are discussed, such as:

� Creating, changing, and dropping indexes

� Changing database view definitions

� Changing the access to pooled and clustered tables

� Modifying Matchcodes

n This workshop provides templates for recording and reporting expensive SQL statements. These templates are to be used by the:

� System administrators who are monitoring the system

� Developers who design the data model and develop the programs

� Application department that customizes the application

� Users who are processing the reports

SAP AG 1999

Database

Controlfiles Data files

init<SID> .ora

......

RDBMS Memoryarea

Shared SQL Area

SGASGA

RECORECO

CKPTCKPT

LGWRLGWR

DBWRDBWR

ARCHARCH

SMONSMON

PMONPMON1:11:1

Database buffer poolDatabase buffer pool

Redo log bufferRedo log buffer

Shared poolShared pool

Parameter

Shared processesSAP workSAP work

processprocessShadowShadowprocessprocess

Dedicatedprocesses

Row cache

ORACLE Architecture Review



n An Oracle database consists of a set of files and the Oracle instance.

n The files are:

• init<SID>.ora file

• control files

• data files

• online redo log files

• offline redo log files

n The Oracle instance consists of the allocated shared memory resources and the following database background processes:

• DBWR = Database writer

• LGWR = Log writer

• CKPT = Checkpointer

• RECO = Recovery process

• PMON = Process monitor

• SMON = System monitor

• ARCH = Archive process

SAP AG 1999

ORACLE Architecture Review

Data files

Mirrored

Control files

Mirrored

Profile (init<SID>.ora)




Redo log buffer

ARCHARCH

CKPTCKPT

LGWRLGWR

DBWRDBWR

Processes and memory

n There are two basic writing background processes in the Oracle database system:

• Database writer writes the data blocks that have been changed from the system global area back to the data files asynchronously

• Log writer writes the logs of the changes that have been made to the data blocks to the redo log files synchronously

n The archive process copies the redo log files that are currently not in use.

n The checkpointer assists the internal writing process at a checkpoint.

SAP AG 1999

Shared SQL Area

Shared Global Area (SGA)

Shared SQL Area Data buffer

Log buffer

Oracle process Oracle process Oracle process

SELECT * FROM AUSP WHERE ...

l SQL statements are shared across all Oracle processes

l Cost statistics are collected for each statement

n The Shared SQL Area is a shared memory structure in the database system that contains the parsed SQL statements and execution plans.

n Each SQL statement is only parsed once and then shared across all Oracle processes.

n Oracle reuses parsed statements and collects cost statistics for each statement.

n Because all Oracle processes share the SQL statements you can monitor the global cost of statements using the Shared SQL Area .

SAP AG 1999

How Oracle Processes an SQL Statement

Oracle processes SQL statements in two phases:• The statement is parsed• Data is retrieved

SQLSQLin SCC?in SCC?

NoSQL Parse

Data retrieval

Data buffer

Data file

Yes

Parsing andParsing andstoring the SQLstoring the SQLstatement in thestatement in the

Shared SQL AreaShared SQL Area

OracleOracleshadowshadowprocessprocess

n Oracle processes SQL statements in two phases:

� During the Parse phase, an SQL statement is compared to other statements in the Shared SQL Area. If a match is found, the parsed statement is reused. If a match is not found, the SQL statement is stored in the Shared SQL Area with the access path.

� During the Data retrieval phase, Oracle locates the required tables and loads the appropriate data blocks into the data buffer. Oracle then performs the necessary operations on the data. The data retrieval phase consists of two steps: Open and Fetch.

n A single SQL statement may generate multiple fetch operations depending on the length of the row of data and the number of rows retrieved.

SAP AG 1999

Summary


• Explain why you should use the Shared SQLArea to analyze expensive SQL statements

• Describe how Oracle processes an SQLstatement

n Now you are able to:

� Explain why the Shared SQL Area should be used to find and analyze expensive statements

� Describe how SQL sta tements are processed by Oracle

SAP AG 1999

Introduction to the Shared SQL Area

Techical Background

The Shared SQL Area


Update Statistics

Identify Coding


Index Utilization

Cost Evaluation

Creating an Index

Similar Statements

View Processing

Joins


SAP AG 1999

Unit: Introduction to the Shared SQL Area

Contents:

• Database Overview Monitor (Transaction ST04)

Objectives:

At the end of this unit you will be able to:

• Explain the impact of an expensive statement

• Use the Database Overview Monitor to analyze the Shared SQLArea

• Determine if there are any expensive statements in the SharedSQL Area

• Determine where the different statements come from


� Explain the impact of an expensive statement

� Use the Database Overview Monitor to analyze the Shared SQL Area

� Determine if there are any expensive statements in the Shared SQL Area

� Determine where the different statements come from

SAP AG 1999

Definitions

• Quality:

• The hit ratio for the data buffer

• Reads:

• Total number of Oracle blocks accessed in the databuffer

• Physical Reads:

• Total number of Oracle blocks read from disk

• User Calls:

• The total number of calls made to the Oracle kernelsince instance startup

n The following fields are displayed in Database Overview Monitor:

� Quality is the cache hit rate for the data buffer. This rate is the percentage of data blocks already found in the buffer of the total number of blocks read.

� Reads are the total number of Oracle blocks found in the data buffer by all processes since instance startup.

� Physical Reads are the total number of Oracle blocks read from disk since instance startup. Block reads from the data buffer do not increase this amount.

� User Calls are the total number of calls made to the Oracle kernel since instance startup.

SAP AG 1999

Expensive SELECT Statements

Databasedisks

Physical I/Oload

DBMS DBMSprocesses Data buffer

Shared SQL Area SQL statements with statistical info...SELECT FROM T001 WHERE …...

Memoryconsumption

Consumption ofCPU resources

n The important resources on the database server that are limited are the CPU capacity, the amount of memory, and the number of physical I/O operations that are performed efficiently.

n Improper or unnecessary use of the database buffers results in displacements of other data blocks, and affects overall system performance.

n Displacements of data blocks that are needed frequently will result in high physical I/O activity and will decrease disk performance.

n While processing the requests to the buffer, CPU resources are also used.

n The typical ratio for User calls to reads from the data buffer for a well tuned R/3 System is approximately 1:15. This is the average number of buffer blocks that are read to process a statement. If this ratio is significantly higher, it is a good indicator that the Shared SQL Area needs to be carefully analyzed for expensive statements.

SAP AG 1999

Data Buffer Hit Rate

Database

RDBMSSGASGA Data bufferData buffer

Shared pool

parsed SQLstatements andexecution path

dc information about database

objects

Datafiles

... SystemPSAPBTABI PSAPSTABD

Memoryarea

Shared SQL Area DDIC cache

R/3 work process

SELECT * FROM MARA

WHERE ...

Shadowprocess

Reads ≥20

Physical reads 1

n The ratio of physical reads per total reads is the data buffer hit rate. This hit rate is displayed in the Database Overview Monitor (Transaction ST04), and should be at least 94%.

n NOTE: It is only useful to evaluate this rate after your system has been up for some time. In the first few hours after startup, the hit rates are low.

SAP AG 1999

l A data buffer hit rate of 94% or greater does not meanthe buffer is large enough

l Expensive SELECT statements can increase the databuffer hit rate because they significantly increase thenumber of successful buffer gets

l If the ratio of buffer gets to user calls is much greaterthan 15, the data buffer hit rate is not a usefulindicator

Examine the Data Buffer Hit Rate

ExpensiveSELECT on table VBBE

Other

Tables

Data Buffer

VBBE

n The hit rate of the data buffer depends on many factors. A hit rate of 94% or higher does not necessarily mean that the data buffer is large enough.

n Consider the following situation: A few expensive SELECT statements are executed often. The data blocks of the accessed tables are ‘pinned’ in the data buffer, if the statement is executed often. Additionally, the number of buffer gets is very high because it is an expensive SELECT statement. Therefore, the hit rate for this statement is close to 100%. This type of statement increases the overall ratio of buffer gets to disk reads (the data buffer hit rate).

n If this expensive statement is tuned and reads less data, the data buffer hit rate can drop.

n The data blocks accessed for such an expensive statement use a large fraction of the data buffer. Therefore, the size of the data buffer that can be used for all the other tables is significantly reduced. Disk accesses are performed to read data blocks for statements to the other tables, which could otherwise be read from the data buffer.

n If the expensive SELECT statement is tuned, significantly fewer data blocks are read from the buffer. This can cause the hit rate to drop. However, the space available for all tables increases and the performance increases for all statements as there are fewer disk accesses.

n The assessment of the data buffer hit rate is more reliable if you also consider the ratio of buffer gets to user calls. In a well-tuned system, this ratio is approximately 15. An expensive SELECT statement that increases the data buffer hit rate will increase this ratio as well. That is, if this ratio is much higher than 15 the data buffer hit rate cannot be used, and you must check for expensive SELECT statements.

SAP AG 1999

1

2

3

Shared SQL Area

n To display the Database Overview Monitor, use Transaction ST04 or choose Tools → Administration → Monitoring → Performance → Database → Activity.

n To display the Shared SQL Area, choose Detail analysis menu → SQL Request.

n To display all SQL cache statements, enter the value 0 in the fields Buffer Gets and Disk Reads.

SAP AG 1999

Shared SQL Area

n The following fields are displayed in the Shared SQL Area:

• Total Executions is the number of times a parsed SQL statement has been executed

• Disk Reads is the number of physical disk read requests issued by the SQL statement

• Buffer Gets is the number of blocks requested from the data buffer

• Records processed is the number of records processed for the statement

• Buffer Gets/Record is the average number of block requests that must be made per record

• SQL Sort Number of sort operations necessary per execution

n The value per execution for all of these fields is also displayed.

SAP AG 1999

Application server A

SELECT * FROM T001WHERE ...

R/3 database interface

ABAP program

DBMS DBMSprocesses

Database

buffer

= Buffer gets

Records

Database

Buffer Gets for an SQL Statement

n The SELECT statement specifies a set of records that must be read from the database. To find these records, the database scans the necessary data blocks.

n Blocks that are not yet in the data buffer have to be read from disk.

n A statement is efficient if the optimizer can use an access that limits the number of blocks that must be scanned. This reduces the number of buffer gets and disk reads.

n To improve the efficiency of a SQL statement, reduce the number of buffer gets.

SAP AG 1999

18.12.1997 12:33:58 Shared Cursor Cache (last reset at

Total / Gets/ SQL Execution Execution Text

612 1.001,0 SELECT "MANDT" , "BUKRS" , "VBELN", "VBELV" 6545 12,6 SELECT "MANDT" , "BUKRS" , "VBELN", "VBELV"

123425 65,0 select o.name, u.name from obj$ o, user$ uwhere 5239 683,0 SELECT "MANDT" , "BUKRS, "RCSTAT", "MCOD1"

5 24,4 SELECT SEQ_NR, S_ID, WR_TS, NOTEBOOK FROM 1 63 SELECT "MANDT", "WERKS" ,

1 4.027,0 SELECT TCODE , PGMNA , OBJCT, PAHNR, MAPTT, 1 63 SELECT MANDT, werks , Vbeln , POSNR , Stat ,

Different Statements in the Shared SQL Area

ABAP OPENSQL statement

Recursive call

SAP tool statement

Exec SQL statement

Third party statement

n ABAP OPEN SQL statements can be identified by uppercase letters with the table fields in quotes.

n Recursive calls occur when Oracle issues an SQL statement in addition to the SQL statement issued by a user process. The most common causes of recursive calls are data dictionary cache misses and dynamic storage extension. Recursive calls can reduce database performance and should be minimized. Recursive calls can be identified by lowercase letters without quotes that access system tables such as tables containing a "$".

n SAP Tool statements are executed from SAPDBA or other SAP tools. SAP tool statements can be identified by upper case letters without quotes. System tables are usually accessed by SAP tools.

n An Exec SQL statement (native SQL statement) bypasses the ABAP SQL statement check. An Exec SQL statement can be identified by uppercase letters without quotes. An Exec SQL statement can access any table and is normally used for database specific SQL statements.

n Third party statements are called from other vendor’s interface programs. Third party statements may have any structure, for example, mixed upper- and lowercase letters. Any statement that is not one of the above statements is a third party statement.

SAP AG 1999

Summary


• Explain the impact of an expensive statement

• Use the Database Overview Monitor to analyze theShared SQL Area

• Determine if there are any expensive statements in theShared SQL Area

• Determine where the different statements come from


� Explain the impact of an expensive statement

� Use the Database Overview Monitor to analyze the Shared SQL Area

� Determine if there are any expensive statements in the Shared SQL Area

� Determine where the different statements come from

SAP AG 1999


Techical Background

The Shared SQL Area


Update Statistics

Identify Coding


Index Utilization

Cost Evaluation

Creating an Index

Similar Statements

View Processing

Joins


SAP AG 1999

Unit: Analyzing SQL Statements

Contents:

• Recognizing expensive statements

• Analyzing the Shared SQL Area

• Using the Explain function

• Table statistics

• Using the ABAP Dictionary

Objectives:


• Determine if and why a statement is expensive, using the:

• Shared SQL Area

• Explain Function

• Use the ABAP Dictionary

n This unit describes how to:

� Recognize the different types of expensive SQL statements

� Use the Explain function

� Check table statistics

� Use the ABAP Dictionary

n Once you have completed this unit, you will be able to use the analysis tools to determine if and why statements are expensive.

SAP AG 1999

SQL Statements

SELECT MANDT, VBELN, OBJNR FROM ZVBAKWHERE MANDT = 001 AND ERNAM = ERNIE

Search area(the data that is

searched through forthe requested data)

Records specified by the WHERE clause

MANDT VBELN OBJNR ERNAM

001 000013245 654875454 ERNIE001 000013246 697935435 FRED001 000013247 673225747 JAMES001 000013248 216356543 GUSS001 000013249 687535435 ERNIE001 000013250 258674802 KURT001 000113245 983231365 MAX001 000113251 522112486 BERT001 000113252 325348654 BERT002 000013249 241567685 HUGO002 000013250 968351332 FRED002 000013251 874352321 MAX002 000013252 544546546 GUSS

n The WHERE clause specifies which records are searched from these tables. In the case of joins, these records are also restricted by the join condition. The fields specified in the WHERE clause also determine which indexes are used.

n The search area is the set of data (records) that are searched to evaluate the query. This set is not explicitly specified in the query. It is determined by the optimizer when the statement is evaluated, using the indexes of the tables.

n The goal of the database optimizer is to reduce the search area as much as possible. This can be achieved by SQL statement tuning or by technical tuning such as creating indexes or changing view definitions.

SAP AG 1999

Expensive SQL Statements

Type 1: The optimizer chooses an unsuitable access pathSymptom: Many buffer gets but only a few records per execution







n When you analyze a statement, classify the statement first, to find the possible tuning methods.

n There are two different types of statements:

Type 1 statements search through a lot of data for only a few qualified records. Here the optimizer chooses an unsuitable path to access the data.

Type 2 statements request a lot of records. Here the optimizer chooses a suitable access path.

n In this example, the application requests the records of table ZVBAK where MANDT (the client) equals 001 and the field ERNAM (the name of person who created the record or object) is 'ERNIE'. Many records are read, although only two are requested. Therefore, this statement is of Type 1.

SAP AG 1999


Type 1: The optimizer chooses an unsuitable access pathSymptom: Many buffer gets but only a few records per execution


Statements of type 1 may be accelerated by:

l Update of optimizer statistics

l Change of the ABAP code

l Optimizing user input

l Creating an index

l Extending an existing index

l Droping an existing index

n Statements of type 1 may be accelerated by one of the following:

• Update of optimizer statistics

• Changing the ABAP code, for example specify additional fields in the WHERE clause so that an existing index is chosen by the optimizer, read the data from a different table where an appropriate index exists, or avoid reading the data.

• Optimizing the user input

• Create a new index

• Extend an existing index

• Drop an existing index

Caution: The performance of other statements may suffer if an index is created, extended, or dropped.

SAP AG 1999


SELECT MANDT, VBELN, OBJNR FROM ZVBAKWHERE MANDT = 001






Type 2: The optimizer chooses a suitable access pathSymptom: Many buffer gets and many records per execution

n In this example, the application requests all records of table ZVBAK where MANDT = 001. Because the database returns all the records that are read, this statement is of type 2.

SAP AG 1999


Statements of type 2 may be accelerated by:

l Change of the ABAP code

l Tuning of the business process

l Optimizing user input

SELECT MANDT, VBELN, OBJNR FROM ZVBAKWHERE MANDT = 001

Type 2: The optimizer chooses a suitable access pathSymptom: Many buffer gets and many records per execution

n A statement of type 2, that returns many records, cannot be accelerated by using an index. To accelerate this type of statement, you can:

• Tune the ABAP report

• Tune the business process

• Optimize the user input

SAP AG 1999

Analyzing the Shared SQL Area

• Check database activity sincestartup

• Sort for the following criteria:

• Buffer getsExpensive statements foroverall performance

• Disk readsExpensive statementsfor I/O performance

• RecordsExpensive statements fordatabase and applicationserver performance

• Every statement has its own hitratio

DBMSprocesses

Databasebuffer

Buffer gets

Records

Database server

Applicationserver

Disk reads

n Before you analyze the Shared SQL Area, check that there have been well over a million user calls. Then the database has processed enough statements to make the analysis meaningful.

n A statement that causes a lot of buffer gets is expensive with respect to system resource consumption. Such statements can reduce overall database performance.

n A statement that causes a lot of disk reads can critically affect the system I/O performance and reduce overall system performance.

n A statement that reads a lot of records can reduce the performance of both the database and application servers. Statements of this type may be coded inefficiently.

n Every SQL statement has its own hit ratio.

n Terminology:

� Buffer gets are called reads in the Database Overview Monitor

� Disk reads are called physical reads in the Database Overview Monitor

SAP AG 1999

Buffer Gets

Check statements for which the number ofbuffer gets exceeds 5% of the total reads

Manyexecutions

Few executionswith many buffergets per execution

Buffer get Database request

Applicationserver

Requested record

Databasebuffer

Applicationserver

Databasebuffer

Note: The load onthe data bufferdepends only onthe number ofbuffer gets. It isindependent fromthe number ofexecutions

n To display the statements that cause the highest database load, sort by buffer gets.

n The statements at the top of the list cause the highest consumption of database resources. These statements are either executed very often, with a small number of buffer gets per execution, or they have a very high number of buffer gets per execution. These statements can reduce the global database performance.

n An R/3 System has expensive statements if:

� The number of buffer gets for the topmost statement exceeds 5% of the total number of reads.

� The ratio of reads to user calls is greater than 15.

n Check any statements for which the number of buffer gets exceeds 5% of the total reads.

n If the ratio of reads to user calls is much greater than 15, check at least the topmost statements.

n If there are many expensive statements, each statement has only a small fraction of the total number of reads. In this case, the ratio of user calls to reads is the best indicator of expensive statements.

SAP AG 1999

Buffer Gets per Execution

• Statements with many buffergets per execution

• Have high consumption ofsystem resources duringexecution

• Should have a low number ofexecutions

• Should not be executedduring peak hours

Buffer get Database requestRequested record

Many buffer getsper execution

Applicationserver

Databasebuffer

n The statements that cause a high database load while being executed have a high number of buffer gets per execution. These statements can reduce the performance of all other statements running at the same time. If these statements are are executed during background operation, when online users are not logged on to the system, they are less critical.

n To determine when the statement was executed for the first time, choose Next info from the Shared SQL Area and check column 1. Load Time.

SAP AG 1999

Buffer Gets per Record

• High number of buffer gets per record→ No appropriate index




searched throughfor the requested

data)

Applicationserver

Databasebuffer

n Statements with a high number of buffer gets per record (compared to the size of a record) use an unsuitable access path. This means that the search area is unnecessarily large.

n However, if the records returned by the statement are often zero, the number of buffer gets per record can be very large although the statement is executed with a suitable access path. Therefore, always compare the number of records processed to the number of executions. If the buffer gets per execution is low, this statement uses a suitable access path.

n Our experience shows that statements, which often return no records, can be avoided by coding changes.

SAP AG 1999

Records per Execution

• Statements with a high number of recordsper execution may use suboptimal logic

• Example:ABAP coding:

SELECT * FROM MARA.CHECK MARA-MATNR = 12345....ENDSELECT.

Applicationserver

Databasebuffer


n Statements with a high number of records per execution may be coded inefficiently.

n In this example, the complete table MARA is read from the database to the application server when only one record is required.

n Check if column SQL Sort is greater than 0. Sort operations must be avoided for statements that process many records.

SAP AG 1999

Disk Reads

• Symptoms:

• High number of disk reads→ Degraded I/O performance

• Hit rate for the statement is low

• Causes:

• Large tables

• Low number of rows per block

• Data is aged out of the buffer

• Many full table scans performed for this table

DBMS processes Database buffer

Database server

Disk reads


Check statements where the number of disk readsexceeds 2% of the total physical reads

n To display the statements that critically affect system I/O performance, sort by disk reads.

n If the statement cannot be tuned by any other means, it may be advantageous to create additional tablespaces for the tables that have expensive disk accesses and to locate these tablespaces on separate disks.

SAP AG 1999

Statement Details

Bind variableFull SQLStatement

n To display the full SQL statement, double -click the statement in the Shared SQL Area.

n Bind variables are passed from R/3 to Oracle during the parsing of the statement. The values for the statement are passed to the database at a later point in time when the data is retrieved. Using bind variables allows Oracle to reuse statements in the Shared SQL Area for queries with the same structure but different values. If the values of the query were passed to the database without using bind variables, the statement could not be reused.

SAP AG 1999

Display the Execution Plan for an SQL Statement

n To display the execution plan of an SQL statement, choose Explain.

n Estimated Costs indicates the costs calculated by the optimizer for this access path. The number represents either the expected number of buffer gets that must be performed, or the expected number of disk accesses. The CPU time necessary to process the statement does not greatly influence the estimated costs.

n Before you analyze a statement further, classify it:

Type 1 statements read records inefficiently. Before trying to tune these statements, check if the table statistics are current.

Type 2 statements return many records per execution. For these statements, you cannot tune the database access path. To tune them, you must change the ABAP reports.

SAP AG 1999

The ABAP Dictionary (Transaction SE12)

• Tables accessed by ABAP statementsare declared in the ABAP Dictionary

• The ABAP Dictionary displaysinformation about tables, views, andindexes

n To display table fields or indexes, use the ABAP Dictionary (Transaction SE12) or choose DDIC Info from the Shared SQL Area screen.

SAP AG 1999

Summary


• Recognize expensive statements

• Analyze the Shared SQL Area

• Use the Explain function

• Use the ABAP Dictionary


n Recognize expensive statements

n Analyze the Shared SQL Area

n Use the Explain function

n Use the ABAP Dictionary

SAP AG 1999

Techical Background

The Shared SQL Area


Update Statistics

Identify Coding


Index Utilization

Cost Evaluation

Creating an Index

Similar Statements

View Processing

Joins


Update Statistics

SAP AG 1999

Unit: Update Statistics

Contents:

• Table Statistics

• Two Phase Update Statistics Strategy

Objectives:

At the end of this unit you will be able to

• Describe the SAP recommended update statistics strategy

• Detect problems with optimizer statistics


� Describe the SAP recommend update statistics strategy

� Detect problems with optimizer statistics

SAP AG 1999

Two main problems occur with optimizer statistics

• Statistics are too old:

• Create new optimizer statistics at least once per week.

• Statistics are not accurate enough:

• Create optimizer statistics with sufficient accuracy

How to prevent most of these problems

• Use the two phase strategy using sapdba once a week

Update of Optimizer Statistics

n Correct table statistics are necessary to enable the optimizer to choose the correct access path to the data. If the table statistics change significantly either the index that is most suitable changes, or the access path might change between a full table scan and an index scan.

n In the table statistics the number of the distinct values for the indexed columns are recorded. This information is used by the optimizer to determine the selectivity of each index for a query.

n Table statistics can be estimated based on a certain sample of the table, or they can be computed, that means the complete table is read for the statistics update.

n You can prevent most of the problems associated with optimizer statistics when you use the two phase strategy of sapdba. Moreover, this strategy limits the system resources used for statistics update to a minimum.

SAP AG 1999

Phase I Phase II

- analyze

Determines demand fornew statistics

Updates or createsnew statistics

method, option

- checkopt

Uses:

Primary indexes(no. of distinct keys)

Uses:

Entries in DBSTATC

DBSTATC

(control table)

xtable E P10

method

option

SAPDBA SAPDBA

new statistics needed

Overview of the two-phase strategy

n Because you cannot refresh the statistics of every table with ultimate accuracy, sapdba provides the possibility to perform the update statistics in two phases and for each table with the necessary accuracy.

n In the first phase the sapdba determines the tables for which the statistics need to be updated. The command for this action is: sapdba - checkopt PSAP% to check for all tables if the statistics need to be refreshed.

n In the second phase the sapdba then updates the statistics for these tables. The command for this action is: sapdba -analyze DBSTATCO to actually update the table statistics for the tables determined in the first phase.

n Both phases of the update statistics should be scheduled at least once a week. It is necessary that the first phase is scheduled and completed before the second phase.

n The control table for the update statistics is table DBSTATC.

n During the first phase of the update statistics procedure, the tables for which new statistics need to be calculated are flagged in this table.

n During the second phase the table is read and for the flagged tables new statistics are determined.

n The table DBSTATC also contains information about the desired accuracy for the update statistics determination, or if a table should be excluded from rule based optimization. In the latter case the table statistics are deleted if existing.

SAP AG 1999

Schedule these actions fortimes of low system activity

Transaction DB13

Support of two-phase strategy by CCMS

n You can use transaction DB13 to schedule the two phases of the update statistics procedure.

n The first phase is scheduled via ‘Check optimizer statistics’. For this action we recommend to choose option ‘PSAP%’ to check for all R/3 tables if new statistics are required.

n The second phase is scheduled via ‘Create new optimizer/space statistics’. For this action we recommend to choose option ‘DBSTATC’ to actually perform the statistics update for the tables determined in the first phase.

n You should schedule these actions for times of low system activity because they can degrade system performance.

SAP AG 1999

Table Statistics Date

Check the Last statistics dateUpdate the table statistics ifthey are older than one weekfor an expensive statement

n You can use the explain function in the Shared SQL Area to display the current table statistics.

n With a double click on the table name in the execution plan you can display the current table and index statistics.

n If you find an expensive select statement, first check the date when the table is analyzed last. If the table statistics are not up to date, the optimizer may be misled and choose the wrong access path.

n If you find an expensive statement, check the update statistics date of all tables accessed in the query. If the statistics of one of these tables is older than one week, or if the table contents have changed significantly since the last update, update its statistics using sapdba:

sapdba -analyze <tablename>

This command will update the statistics with the default method for the table. Check if the access path changed after the statistic update.

n The table is not locked during statistic update. However, the complete table or a major part of it has to be read for this operation.

SAP AG 1999

Old Statistics:Full Table Scan

New Statistics:Concatenation of Index Unique Scans

Table Statistics Date

n In this example we show how the table statistics can influence the decision of the cost based optimizer.

n At the time of the first statistics update table VBUK contains only 100 rows of data. Then, for example due to a data import, a lot of records are inserted into the system. However, the optimizer statistics are not updated when data is inserted into the system but at the next statistics update.

n With the old optimizer statistics the costs calculated for a full table scan are 2. Although the table contains much more data now, optimizer still assumes that the table has the same size as at the time of the last statistics update. Therefore, the costs calculated and the access path does not change if you insert data into a table.

n After the statistics update the optimizer uses a new access path, a concatenation of several index range scans. The costs for this access are estimated now to 18. Because there are now 1891 blocks allocated instead of 6, a full table scan would have had much higher estimated costs.

SAP AG 1999

• Exact calculation

• considering all rows of a table

• time consuming: not applicable for all tables

• Estimation based on a sample

• considering only a sample of rows of a table

• fast but too inaccurate for certain tables

• No statistics maintained for

• Pools / Cluster

• Frequently changed tables, e.g. Update table VBDATA

necessary for certain tables

sufficient for most tables

Accuracy of statistics

n The accuracy used for for the table statistics determination can be selected for each table. For some tables it may be necessary that the statistics are calculated exactly. All the rows of the table are then read during the update statistics. However, for most tables it is sufficient if an estimation of the distinct values is performed using a sample of the table. This method is faster, but not accurate enough for some tables.The percentage that is used for the estimation depends on the size of the table and is determined by sapdba.

n For some tables no table statistics are maintained. The SQL statements for these tables are optimized using the rule based optimizer.

n Reasons for not maintaining table statistics are:

The tables are changed frequently. Hence, the table statistics cannot be up to date.

The access path to the data in the table is determined by the application logic. Only a specific access path is desired that is supported and chosen by the rule based optimizer. Maintaining table statistics is an overhead.

The cost based optimizer does not choose the correct access path to the table.

n The tables for which no statistics are maintained are for example:

SAP update tables (VBDATA, VBMOD, VBHDR)

Pooled tables

Clustered tables

SAP AG 1999

Accuracy of Table Statistics

n The update statistics accuracy is shown together with the table statistics. The different methods of table analysis are:

� Estimate: a part of the table is used for the analysis

� Compute: the complete table is used for the analysis

n For method Estimate either a percentage of the table blocks, or the a sample based on a certain number of rows is used for the analysis. This information is shown together with the Analyze Method.

n The default settings for method and accuracy are constantly enhanced with new sapdba releases. Therefore, you should always use the newest sapdba.

n Check if the correct access path is used for the table. If you find, that the distinct values determined by update statistics do not represent the actual data in the table, update the statistics with a higher accuracy.

n If you want to update the table statistics with an increased accuracy issue the command:

sapdba -analyze <tablename> -method E -option P<n>

where <n> is the percentage of blocks used for the statistics update or

sapdba -analyze <tablename> -method C

if you want to compute the optimizer statistics.

SAP AG 1999

Accuracy of Table Statistics

Estimated Statistics:Full Table Scan

Computed Statistics:Index Range Scan

n In this example the update statistics with a sample of 1064 rows underestimated the number of distinct values for fields VBTYP and TRVOG and overestimated the distinct values for VGBEL. The differences in the number of distinct values to the actual number of can lead to a wrong optimizer decision. Here the optimizer statistics created with lower precision lead to a full table scan and the computed optimizer statistics lead to an index range scan.

n Depending on the table size and hardware platform the time required for the statistic update can increase when you increase the accuracy.

n Check if the statements access path changed after the more accurate statistics update. If it does, you should always use this method for the statistics update. To do so create an entry in table DBSTATC using transaction DB21 and set the customer flag.

SAP AG 1999

Transaction DB21

Control Table DBSTATC

n You can use transaction DB21 to display and maintain table DBSTATC.

n Here the entries of DBSTATC for table MARA is displayed. The analysis method is to use an estimation (analysis method 'E') with a sample size of 10% of the table blocks (option 'P10').

n However, if a run of the second phase would be scheduled now, no new table statistics would be generated, because the TODO flog is not set. Either the first phase of the update statistics procedure was not scheduled yet, or during this run it was determined that no new statistics need to be determined for table MARA.

n If for 30 days no new statistics need to be created for table MARA, the entry for MARA is deleted in table DBSTATC. If later sapdba finds that MARA has to be analyzed again, a new entry for MARA is created in DBSTATC.

n If you want to specify your own settings for certain tables, such as an increased accuracy, you have to set the customer flag. Then the entry is never deleted from DBSTATC.

n Note: Do not insert a more than one record into DBSTATC for one table. This is currently not checked and can lead to problems at the statistics update.

SAP AG 1999

Change the Optimization Mode

n You can control the optimizer mode by deleting or creating statistics. If no statistics are maintained for all tables in the query, the rule based optimizer is used for the access. If at least one table accessed in the query has optimizer statistics maintained, the cost based optimizer is used.

n Sometimes the cost based optimizer chooses a wrong access path due to problems with the optimizers cost evaluation, where the rule based optimizer chooses the correct access path..

n To check if the rule based optimizer would choose the correct access path, you can explain the statement with a rule based optimizer hint. Click on 'Explain with hint' and type 'RULE' into the following popup. This will switch on the rule based optimizer for the explain function but not for processing the statement.

SAP AG 1999

Change the Optimization Mode

• What you need to check before you switch to therule based optimizer

• Is the statement expensive because of

• old statistics

• inaccurate statistics

• a problem that can be solved in the ABAP code

• a problem that can be solved with a better index design

• inaccurate parameter settings

• Are other statements going to be expensive due to problems associated to therule based optimizer

Using the rule based optimizer is only aworkaround but not a long term solution

n Before you switch to the rule based optimizer, you have to check the following:

� Does the cost based optimizer choose a wrong access path due to old statistics or statistics that are not accurate enough? In this case you should update the statistics but not switch to the rule based optimizer.

� Are any parameter settings required for the cost based optimizer incorrect? (See following chapter for details). In this case you should change the parameter settings but not switch to the rule based optimizer.

� Are there any statement statements that will become expensive because the optimizer is switched? To check this, display all statements to the table and check them using the Explain function with the rule based optimizer hint. If any of the other statements chooses an unsuitable access path with the RULE hint, you cannot switch to the rule based optimizer. Do not forget to check the statements for views that contain the table.

n To switch to the rule based optimizer delete the statistics of the table with the following command:

� sapdba -delete <tablename>

n After you have switched to the rule based optimizer:

� Check if any statement accessing the table has become expensive

� Create an entry for the table in the control table DBSTATC using transaction DB21 and set the customer flag

SAP AG 1999

• Table and Index Statistics:

• Follow the SAP recommended two phase strategy

• Update the optimizer statistics for the tables of theexpensive statement.

• Update the statistics with increased accuracy

• Switch between cost and rule based optimization for thetable(s)

Preferential Order of Possible Optimizations

n Follow the SAP recommended two phase strategy to update the optimizer statistics for all tables, i.e. schedule:

� sapdba -checkopt PSAP% weekly followed by

� sapdba -analyze DBSTATCO

� You can schedule these commands in the DBA planning calendar DB13.

n To create new optimizer statistics for the table of the expensive statement with the default settings of the sapdba issue the command:

� sapdba -analyze <tablename>

n If you want to update the table with an increased accuracy update the statistic s with the following command:

� sapdba -analyze <tablename> -method E -option P<n>

� or

� sapdba -analyze <tablename> -method C

n To switch between the cost and rule based optimizer, delete the table statistics:

� sapdba -delete <tablename>

n Note: This option will influence the behavior of all statements that access the table.

SAP AG 1999

Techical Background

The Shared SQL Area


Update Statistics

Identify Coding


Index Utilization

Cost Evaluation

Creating an Index

Similar Statements

View Processing

Joins


Identify Coding

SAP AG 1999

Unit: Identify Coding

Contents:

• Where-used lists

Objectives:

At the end of this unit you will be able to

• Identify the lines of code where a statement is issued

• Find programs using the Global Work Process Monitor and OracleSession Monitor

• Describe the differences between ABAP OPEN SQL statements andSQL statements


� Identify the lines of code where a statement is issued

� Find programs using the Global Work Process Monitor and the Oracle Session Monitor

� Describe the differences between ABAP OPEN SQL statements and SQL statements

SAP AG 1999

Roadmap for Finding SQL Statements in Programs

Yes

No

Identified SQL

statement

In Where-usedlist

Use Transaction SM66to find the process and

program used

Use the Oracle Session Monitorto verify that the expensive SQL statement is processed

Search the Where-used

list

Workflowcommunications

process

Identify statement

in code

n Use this roadmap to find the code responsible for an expensive SQL statement.

SAP AG 1999

Where-Used List

Shared SQL Area

Where-used listProgram listing

n Once an expensive SQL statement is located in the Shared SQL Area, the ABAP program needs to be analyzed. To find the ABAP program, use the ABAP Dictionary (Transaction SE11).

n To display a list of all the programs using a specific table, use Transaction SE12. Enter the name of the table, and choose Utilities à Where-used list.

n To display only the relevant locations, you can click on Search Area in the following action and specify select in the field for ABAP key words and a search string that specifies the statement more closely, for example a certain table field that is specified in the where clause, or, as in this example, the string order by if the database statement contains an order by.

n You can schedule the Where used list as a background job and view the generated spool list later on, or you can start the search online. If you start the search online you can navigate to the source code by a double click. However, for frequently used tables the where used list may take a while to complete.

SAP AG 1999

Find the Program Developer

l Use the ABAP Editor to display the programattributes

l The program attributes contain the user nameof who created and last changed a program

n To find the program developer, use ABAP Editor (Transaction SE38). Enter the program name, and choose Display à Goto à Attributes.

n If a program was created and last modified by SAP, search for OSS Notes containing the program or table name using the search terms:

<Table> OR <Program> OR <Transaction> AND Performance OR Index

n If a program was created or modified by one of your developers, contact the developer about optimization possibilities. Use the templates provided for the communication process.

SAP AG 1999

Statements not Contained in the Where-used List

• A statement cannot be found in the Where-used list, for example, if:

• The code is generated at program runtime

• The SQL statement is from an Native SQL program line

• Important fields in the WHERE clause are not specified becauseinternal tables used are empty

• Use the Global Work Process Monitor and the Oracle SessionMonitor instead

n A statement cannot be found in the Where-used list, for example, if:

� The code is generated at program runtime

� The SQL statement is from an Native SQL program line

� Important fields in the WHERE clause are not specified because internal tables used are empty

n Use the Global Work Process Monitor (Transaction SM66) and the Oracle Session Monitor to find the code. If the buffer gets are very high, the probability that the statement is executed while the system is monitored is high.

SAP AG 1999

Using the Global Work Process Monitor

n To monitor if the table for which the expensive statement is issued is currently accessed, use the Global Work Process Monitor (Transaction SM66). The Global Work Process Monitor displays the user that issues the statement, the report, and the transaction that is processed.

n To display only the work processes that are currently accessing a specific table, choose Select process, and enter the table name in the dialog box displayed.

n To display the table and program name, choose Settings and deselect Display only abbreviated information, avoid RFC.

n The Global Work Process Monitor displays only the table for which a SQL statement is issued. To verify that the expensive statement you are looking for is processed, use the Oracle Session Monitor.

SAP AG 1999

Using the Oracle Session Monitor

n To display the Oracle Session Monitor, use the Database Overview Monitor (Transaction ST04) and choose Detailed analysis menu à Oracle Session.

n This monitor will connect the PID of the work process to the SQL statement that is processed by the Oracle shadow process. This shows if the statement you are looking for is currently processed.

n To select only active sessions, choose Filter.

n If the statement currently processed is the expensive statement you are looking for, the report displayed in the Global Work Process Monitor must be analyzed.

n NOTE: The expensive statement you are looking for can be issued from several programs.

SAP AG 1999

Differences in ABAP Open SQL and SQL Statements

• ABAP Open SQL statement is different than the SQLstatement in the Shared SQL Area

• This happens, for example, when you use:

• Internal tables (IN ITAB)

• The command FOR ALL ENTRIES

• Projection views

n The ABAP OPEN SQL statement may not resemble the SQL statements displayed in the Shared SQL Area. This happens, for example, when:

� Internal tables are used to build the WHERE clause

� The ABAP command FOR ALL ENTRIES is used in the OPEN SQL statement

� Data is selected from projection views. If this occurs, the SELECT statement to the view is converted to a SELECT statement to the underlying table.

SAP AG 1999

Statements using Internal Tables

Selectionscreen

ABAP OPEN SQL:

SELECT * FROMBKPFWHERE BUKRS IN I1AND BELNR IN I2AND GJAHR IN I3AND BLART IN I4AND BLDAT IN I5AND BUDAT IN I6

SQL statement:

SELECT ...FROM "BKPF"WHERE "MANDT" = :A0AND "BUKRS" = :A1AND ( ( "BELNR" BETWEEN :A2 AND :A3

OR "BELNR" BETWEEN :A4 AND :A5 )OR "BELNR" IN ( :A6 , :A7 , :A8 )OR "BELNR" >= :A9OR NOT "BELNR" = :A10 )

Multiple range selection

n Ranges tables (internal tables with a special format) can be used to build the WHERE clause of an SQL statement. The internal table contains values and operators for the WHERE clause. Multiple values and operators for one table field are possible. These ranges tables can be declared in the ABAP code with the ABAP key words SELECT-OPTIONS or RANGES.

n If an internal table does not contain any values, the corresponding field will not be specified in the WHERE clause of the SQL statement, although it appears in the WHERE clause of the ABAP OPEN SQL statement.

n In this example, the WHERE clause of a statement to table BKPF is built by ranges tables. Only the ranges tables for the field BUKRS and BELNR are filled. Field BUKRS is specified with an EQUAL condition and field BELNR is specified with two BETWEEN conditions, an IN condition, a GREATER OR EQUAL condition, and a NOT condition linked with the OR operator. All the internal tables for the other fields in the OPEN SQL statement are empty and do not appear in the SQL statement.

SAP AG 1999

Internal Table Handling: FOR ALL ENTRIES

•A statement that appears in the Shared SQL Area as:

SELECT * FROM DBTAB

WHERE ( F1 = :A0000 AND F2 = :A0001 AND F3 = :A0002)

OR ( F1 = :A0003 AND F2 = :A0004 AND F3 = :A0005)

...

OR ( F1 = :A0057 AND F2 = :A0058 AND F3 = :A0059)

Is called from in the ABAP program as:

SELECT * FROM DBTAB FOR ALL ENTRIES IN ITAB

WHERE F1 = 'X' AND F2 = 'Y' AND F3 = ITAB-F.

n OPEN SQL statements using the command FOR ALL ENTRIES for an internal table generate a set of SQL statements with outer OR conditions.

n If you want to find SQL statements that have outer OR conditions in ABAP programs, look for OPEN SQL statements using the command FOR ALL ENTRIES.

SAP AG 1999

SQL Statements to Project Views

SELECT"MANDT" , "BEDID" , "BEDZL" , "CANUM" , "ARBID" , "TYPKZ" , "OBJKZ" ,"KAPID" ,"BEDKZ" , "KRUEREST" , "KBEAREST" , "KABRREST" , "KEINH" ,"FSTAD" , "FSTAU" ,"FENDD" , "FENDU" , "SSTAD" , "SSTAU" , "SENDD" ,"SENDU" , "KPVER" , "OTYPE" , "PERNR" , "CANUMF" , "SPLIT" , "BSTKZ" ,"PLSCN" , "FSSBD" , "FSSBZ" , "FSSAD" ,"FSSAZ" , "SSSBD" , "SSSBZ" ,"SSSAD", "SSSAZ" , "BEDZLF" , "PHASE_KZ"

FROM

”KBED"

WHERE "MANDT" = A0000: AND "KAPID" IN ( A0001: , A0002: ) AND"TYPKZ" IN ( A0003: , A0004: , A0005: ) AND "PLSCN"= A0006: AND"BSTKZ" IN ( A0007: , A0008: ) AND ( "FSTAD" >= A0009: AND "FSTAD" <=A0010: OR "SENDD" >= A0011: AND "SENDD" <= A0012: )

WHERE

SQL Statement

SQL call totable KBED

Fields selectedfor statement

Selection criteriafor access

View fields

n This examples shows an expensive statement to table KBED, which was found in the Shared SQL Area. Using the Where-used list for table KBED and reviewing all ABAP programs, no program could be found that contained this statement. The fields in the SELECT clause are a subset of the fie lds of table KBED. No statement was issued in any of the programs where these fields were specified in the SELECT clause.

n When there is a subset of the table fields specified in the SELECT clause of a statement, check if any projection views are associated with the table. If a statement to a projection view is processed, the database interface converts the OPEN SQL statement to the projection view to a SQL statement to the underlying table. In the SELECT clause of the SQL statement, only the fields contained in the projection view are specified.

n To check if a projection view with the fields of the SELECT clause exists, use the Where-used list for the table in views. Display the views, and compare the view fields with the SELECT clause.

n The statement to the view is only converted to a statement to the table if the view is defined in the ABAP Dictionary as a projection view and not as a database view.

n In this example, the SELECT clause of the SQL statement to table KBED matched the view fields of KBED_AVCHK, which is defined as a projection view. A Where-used list for the projection view KBED_AVCHK showed the program that caused the expensive SELECT statement.

SAP AG 1999

l The ABAP report RSSTATUS delivers the applicationarea for tables, transactions, programs, or dataelements

l Contact the development department for theapplication area

Find the Application Area for a Statement

Report RSSTATUS

n If you cannot find the statement, use the ABAP report RSSTATUS. This report shows the application area for tables, transactions, programs, or data elements.

n Contact the development department of the application area, and analyze the statement with the developer responsible for this area.

SAP AG 1999

Summary


• Identify the lines of code where a statement is issued

• Find programs using the Global Work Process Monitorand Oracle Session Monitor

• Describe the differences between ABAP OPEN SQLstatements and SQL statements

SAP AG 1999

Techical Background

The Shared SQL Area


Update Statistics

Identify Coding


Index Utilization

Cost Evaluation

Creating an Index

Similar Statements

View Processing

Joins



SAP AG 1999

Unit: Workflow and Reporting

Contents:

• Interdepartmental workflow and communication

Objectives:


• Record and report your analysis results


� Record and report your analysis results

SAP AG 1999

Workflow Overview

Development or application department

System administratorUsers

Performance view Functional view

Information needsSystem resources

Table, index, and query design

Analyze queriesOptimize coding,

indexes, andcustomizing

Optimize input

Index designrecommendations

n The system administrator monitors the performance of the R/3 System, and can see what impact expensive SQL statements actually have on the system. If the system is well tuned and properly sized, however, the administrator can only analyze the situation, not solve it.

n Users have an information need that has to be met. Users have a functional view of the business processes. That is, the information they require must be available at a certain frequency and at certain times. This requires a certain amount of system resources.

n Developers have both functional and performance views. They provide the programs that meet the information needs of the users and they are responsible for the data model design.

SAP AG 1999

Administrator's and Developer's Responsibilities

Table statistics

Developer

Administrator

Indexes

ZVBAK~0ZVBAK~1

Update statistics

Create

Index design

Communication

A cost based optimizer uses thetable statistics and the values

in the WHERE clause todetermine which index is used

System Monitoring

n We highly recommend, that a single person is clearly responsible for the monitoring, and pursuit of solutions for expensive SQL statements.

n The database administrator must ensure that the optimizer statistics are up-to-date and performed with sufficient accuracy.

n The program developer is responsible for the code of the expensive statement and the table and index design. Therefore, the developer must be contacted if the code has to be changed or if the table and index design has to be changed.

n The creation of an index locks the table. Therefore, indexes should be created, this means transported into the productive system, by the database administrator, because he or she knows best when the creation of an index does not disturb the online users.

SAP AG 1999

Statement Documentation and Logging

• Log the expensive statements and your efforts!

n It is important that you log the expensive SQL statements you found. Copy the expensive statement into the Excel sheet or Access Database provided and note important information like:

� the responsible person

� the report where the statement originates from

� important performance information

number of buffer gets

executions

records returned)

� possible solutions

� actions you performed

� Persons that were contacted and are involved in the solution of the problem

n For large installations you can easily find many expensive statements that have to be solved. If you do not log your efforts, they will probably be lost.

SAP AG 1999

Workflow Overview

R

OSSOpens an OSS Call ifnecessary

System administrator

Log the statementsand send a summaryto IT management ifrequested


• Administrator Responsibilities

Development or application department

Contact theresponsible developer

Tuning the ABAP code ispreferred over other tuningmethods.

n If an expensive statement is found, the administrator must contact the developer of the statement. If the query originates from a standard SAP report, and there is no OSS Note that solves the problem, an OSS Call should be opened.

n An expensive statement must be tuned. To do this, contact the developer of the statement and explain the possible tuning methods you found. Changing the ABAP code or application customizing is the preferred method of tuning statements because other technical tuning methods, such as creating indexes, may impair the performance of other statements.

n The system administrator can suggest other tuning methods, which have to be evaluated by the developer of the data model. If the code of the statement cannot be changed, the developer must determine which technical tuning methods should be used, for example creating an index.

n After the expensive statement is solved, you have to monitor the result. Check if the statement is not expensive anymore. In case the index, table, or view design has been changed, check also if other statements have become expensive.

n A summary of the expensive SQL statements can be sent to the IT Management periodically if required.

SAP AG 1999

OSS Call Template

l If the expensive statement originates from a standard SAPreport, check the OSS Notes

l If an appropriate Note does not exist, open an OSS callusing the OSS Call Template

n If the expensive statement originates from a standard SAP report, check the OSS Notes for possible solutions. Use the search terms <TABLE> or <Program> together with "performance" or "index”.

n If an appropriate Note does not exist, open an OSS call using the OSS Call Template.

n NOTE: Do not open an OSS call:

� For transactions such as SART or SE16, because user input and customizing are usually responsible for slow response times

� If user input is responsible for the expensive statement, such as for a field not specified in a selection screen (example:Matchcodes)

� For sapdb*programs called from your own reports. These programs are logical databases that may be inefficiently used in your coding.

� If the statement is caused by inaccurate or old optimizer statistics

� If an index is missing (check DB02)

� If the index is fragmented

SAP AG 1999

Workflow Overview

Users

• If User Input is responsible for the expensive Statementcontact the Users

System administrator

Discuss a possiblesolution with the userswho issue the statement

n The administrator can contact the users to see if the statement is really required and to find out when the statement needs to be processed. For example, an expensive statement is less critical if it can be processed in background or during periods of low system activity.

n Also, users might be able to optimize their input so that the statement is not expensive anymore, for example, when Matchcodes or selection screens are used inefficiently.

SAP AG 1999

Responsibilities Overview

Roles:- Coordination of problem resolution- Establish contacts between parties- Documentation of problem status- Support level agreement definition

UsersApplication Department(s)

Coordinator at Customer

Roles:- Train users- Solve problems in customizing- Redesign businessprocess

Development

Roles:- Monitoring and analysis of expensive SQLstatements

- Solve technical problems with DB- Find ABAP Source of problems- Find Users responsible for expensive input- Assist in SQL problem resolution- Document expensive statements- Open SAP problem messages for problems inSAP standard

- Monitor successSystem administrator

Roles:- Solve problems in in-house developments- Change coding- Design indexes to support in-house coding- Monitor own developments inproductive environment

- Verification of solutions

n For the monitoring and resolution of expensive SQL statements, different tasks have to be fulfilled.

n The system administrator monitors the system for expensive SQL statements, finds the source of the problem, which could be either the code or the user responsible for input that leads to the problems. He solves technical problems, for example outdated optimizer statistics, assists the developers in problem resolution, and opens problem SAP messages. The system administrator also monitors the success of the statement tuning and documents the expensive statements.

n If the system is administered by an outsourcing partner we recommend that a coordinator at the customer that owns the R/3 system co-ordinates and documents problem resolution and establishes contacts between the two parties outsourcing partner and the customer if necessary. Also support level agreements are defined by the coordinator at the customer.

n If the system is run by an in-house department the two parties 'Coordinator' and 'System Administrator' can be fulfilled by one person.

n The development department is responsible for solving problems with in-house solutions. This means to tune the coding as well as to design indexes to support in-house coding. If the developer is not familiar with designing indexes this may be done in close co-operation with the system administrator. We recommend that the developers have a user in the productive system to monitor their own programs and verify the success of their solutions to expensive SQL statements.

n The application department is responsible for end user training, solving problems in customizing settings and redesigning the business process if necessary.

SAP AG 1999

Summary


• Record and report expensive SQL statements to theapplication or development department

SAP AG 1999

Index Utilization

Techical Background

The Shared SQL Area


Update Statistics

Identify Coding


Index Utilization

Cost Evaluation

Creating an Index

Similar Statements

View Processing

Joins


SAP AG 1999

Unit: Index Utilization

Contents:

• Indexes

• Index structure

Objectives:

At the end of this unit you will be able to describe:

• What an index is

• How an Index is structured

• How data is accessed using an index

n Once you have completed this unit, you will know:

� What an index is

� How an index is structured

� How data is accessed using an index

� How the Oracle rule -based optimizer chooses an index

SAP AG 1999

Indexes

• An index can accelerate data access

• Different types of indexes are:

• Primary index

• Secondary index

• Unique secondary index

n An index supports data searches in the database.

n In the R/3 environment, there are two types of indexes:

� Primary indexes

� Secondary indexes

n All R/3 tables have a primary index, which consists of the key fields that the developer must define when creating an R/3 table. An index can be defined as unique if a maximum of one record exists for any combination of fields in the index. The primary index is always unique because the key fields must identify each table record uniquely.

n For queries where the primary index cannot be used to determine the results, for example, because none of the primary index fields were specified in the WHERE clause, the RDBMS scans through the whole table. To avoid this, you can define secondary indexes to significantly reduce the amount of data records searched to determine the result set.

n Secondary indexes can also be unique indexes.

n You cannot define a single index to match all field combinations used in queries. However, to keep unnecessary data searches to a minimum, ensure that the optimizer can always use an appropriate index.

SAP AG 1999

MANDT VBELN ROWID001 0000121 65487001 0000122 65754001 0000123 65872001 0000124 98756001 0000125 98321001 0000126 65686001 0000127 98321001 0000128 65321001 0000129 98321002 000000001 65325002 000000002 98654002 000000003 65987002 000000004 98332

Oracle Index Structure

Index B* tree

Leafblocks

Branchblocks

Root blocks

Index leaf block

ROWID MANDT VBELN ERDAT ERZET ERNAM ......65872 001 0000123 04051999 085123 ERNIE ...65873 002 0000546 01021998 210136 HUGO ......

Table block

Root and Branch blocks containpointers to the next index level

The ROWID is used as a pointer from the index to the table row

n An index consists of one or more fields of a table.

n Every row in the table is identified by the ROWID, which is also the link between the table and the indexes. The ROWID consists of the Oracle data file number, the number of the Oracle data block within the data file and the row number inside the data block.

n The rows are sorted in the index by the indexed columns to allow for a quick access. In the table the rows are not sorted.

n An index should not have too many fields. If an index has few very selective fields, its reusability increases and the possibility of a bad optimizer decision decreases.

n Index data is stored in a B* tree. The B* tree of an Oracle index in the R/3 environment normally has up to 4 levels.

n When the database searches for a row in an index, the Root block is read first, followed by blocks on the subsequent branch levels and the leaf level. The Root and Branch blocks contain pointers to the following level, sometimes called separators. These separators contain an abbreviated index key and allow for efficient access to the necessary index leaf block.

n The Leaf blocks contain the full index rows and the ROWID for the records. When you search for a row in the index, normally no more than 4 blocks have to be read, that is one block for every index level.

SAP AG 1999

Index Unique Scan

Table Data BlocksUnique index (Primary index)

SELECT * FROM ZVBAK WHERE MANDT = '001' AND VBELN = '0000123'

Table access by ROWID

~ 4 blocks accessed

MANDT VBELN ROWID001 0000121 65487001 0000122 65754001 0000123 65872001 0000124 98756001 0000125 98321001 0000126 65686001 0000127 98321001 0000128 65321001 0000129 98321002 000000001 65325002 000000002 98654002 000000003 65987002 000000004 98332

ROWID MANDT VBELN ERDAT ERZET ERNAM ......65872 001 0000123 04051999 085123 ERNIE ...65873 002 0000546 01021998 210136 HUGO ......

ROWID MANDT VBELN ERDAT ERZET ERNAM ......98321 001 0000132 04051999 085256 BERT ...98322 002 0000355 11041999 170235 KURT ......

n If all key fields of a unique index are specified in the WHERE clause of the SQL statement, the ROWID is found with an index unique scan.

n The ROWID is then used to access the row requested in the table. To find the row, approximately 4 blocks are accessed.

n In this example table ZVBAK has a unique index with the fields MANDT and VBELN, which are the client and document number. This index is used to find the row WHERE MANDT = '001' and VBELN = '0000123'. 3 blocks have to be read on the index and another block on the table.

SAP AG 1999

Index Range Scan

SELECT * FROM ZVBAP WHERE MANDT = '001' AND VBELN = '0000123'

MANDT VBELN POSNR ROWID001 0000120 0001 45487001 0000121 0001 45423001 0000122 0001 32555001 0000122 0002 32566001 0000122 0003 45328001 0000122 0004 45686001 0000122 0005 32651001 0000122 0006 32981001 0000122 0007 37321001 0000122 0008 37325001 0000123 0001 37254001 0000123 0002 37487001 0000123 0003 45332

MANDT VBELN POSNR ROWID001 0000123 0004 37987001 0000124 0001 48423...

ROWID MANDT VBELN POSNR MATNR ......37254 001 0000123 0001 085123 ......37487 001 0000123 0002 210136 ......37987 001 0000123 0004 006584 ...

ROWID MANDT VBELN POSNR MATNR ......45332 001 0000123 0003 000815 ......

1

2

3

4, 7

5

6

Block reads

n If not all the key fields of a unique index are specified with '=' in the WHERE clause of the SQL statement, or if a non unique index is used the ROWIDs are found with an index range scan. The ROWIDs are then used to access the rows requested in the table. If an index is defined as a secondary (non-unique) index, the ROWIDs are also found with an index range scan.

n In this example table ZVBAP has a unique index on fields MANDT, VBELN and POSNR. However, in the query only MANDT = '001' and VBELN = '0000123' are specified.

n For this criteria four records exist in the table with different values for the field POSNR. On the index these records are distributed across two index leaf blocks and two table blocks. For ROWIDs found on the first index leaf block, two table blocks have to be read. For the row found on the next index leaf block, the table block read for the previous index block has to be read again.

n In total 7 blocks have to be read. The root and branch block of the index, two leaf blocks of the index, and two blocks on the table, one of which is read twice, once for position number 1 and 2 and the second time for position number 4, which is contained in a different index block, but the same table block.

SAP AG 1999


MANDT VBELN POSNR ROWID001 0000120 0001 45487001 0000121 0001 45423001 0000122 0001 32555001 0000122 0002 32566001 0000122 0003 45328001 0000122 0004 45686001 0000122 0005 32651001 0000122 0006 32981001 0000122 0007 37321001 0000122 0008 37325001 0000123 0001 37254001 0000123 0002 37487001 0000123 0003 45332

MANDT VBELN POSNR ROWID001 0000123 0004 37987001 0000124 0001 48423...



1

2

3

4, 7

5

6

Block reads

Searched index string: 0010000123%

Order of Fields in the Index

n Why is the order of indexed fields important?

n The order of fields in an index is important for the efficiency of the index scan. In this example a query is send to the database which requests all the records of table ZVBAP where MANDT = '001' and VBELN = '0000123'.

n The data in the index is sorted by the criteria MANDT, VBELN , POSNR. That means the database only has to scan the index for entries where MANDT = '001' until VBELN > '0000123'. In other words: The searched string on the index is: 0010000123%. The block that is read first on the index leaf level is found by the separators in the index root and branch blocks.

n The index scan is efficient. Seven blocks have to be read in total to find the requested data.

SAP AG 1999




5, x+1

y

MANDT POSNR VBELN ROWID001 0001 0000001 45487001 0001 0000002 45423001 0001 0000003 32555

...

1

2

x

Block reads

MANDTPOSNR VBELN ROWID

...

001 0001 0000123 37254001 0001 0000124 45234...

... MANDT POSNR VBELN ROWID

...001 0002 0000122 45598001 0002 0000123 37487001 0002 0000124 46978...

...3 4 6

Searched index string: 001_____0000123

Order of Fields in the Index

n Why is the order of indexed fields important? (continued):

n In this example the index has a different order, but the SQL Statement is the same as on the previous page. The fields specified in the where clause are not the first fields in the index used. The position number is not specified in the query but is contained in the index on the second position between the client and the document number, which are both specified. The index is now sorted by the criteria: MANDT; POSNR; VBELN. All the index blocks for MANDT ='001' have to be read because it is unknown for which POSNR records exist in the table with VBELN = '0000123'. In other words: The searched index string is: 001____0000123. This index scan is inefficient. Many index blocks are read that do not contain data requested by the query.

n The fields that are specified sequentially (starting with the first field) are used to limit the access to index blocks. Any other fields that are specified for the index can be used to limit the access to data blocks. This is called a filter.

SAP AG 1999

Full Table Scan

SELECT * FROM ZVBAP WHERE MATNR = '000815'

Each data block is read onceduring a full table scan

No index is used duringa full table scan

Table ZVBAPIndex

Maximum Gets/Execution = number of table blocks



ROWID MANDT VBELN POSNR MATNR ......46894 002 00009658 0012 000123 ...46895 002 00009874 0007 000815 ......

...


n If a full table scan is used all data blocks of the table are read sequentially but no index blocks are read.

n For a full table scan, the data blocks read from disk are placed at the end of the data buffer Last Recently Used list (LRU list). Therefore, only a small amount of memory is used in the data buffer for full table scans.

n This access path is efficient if a large fraction of the table rows are requested by the query. However, in the example above only few table rows are requested. In this case a full table scan is very inefficient.

SAP AG 1999

Unselective Index Range Scan

SELECT * FROM ZVBAP WHERE MANDT = '001' and MATNR = '000815'

A large fraction of theindex is read

Index

Each table block can be read morethan once

Maximum Gets/Execution = Number of table rows + Number of Index blocks



ROWID MANDT VBELN POSNR MATNR ......46894 001 00009658 0012 000123 ...46895 001 00009874 0007 000815 ......

...


Table ZVBAP

n During an unselective index range scan, the number of gets per execution can be much higher than the number of table blocks and the number of index blocks. The same table block can be read multiple times, if its rows are requested from multiple index blocks. The maximum number of blocks that could be read is determined by the number of table rows, because every row contained in each index block could be read from a different table block.

n Each time an index or data block is accessed, the entire 8KB of data is read. If a large fraction of the index is read, a full table scan would more efficient. All data and index blocks that are read during an index range scan are placed at the beginning of the LRU list and are therefore held in the buffer. This can cause the database to swap a lot of other blocks out of the buffer. Therefore, this type of query can cause a lot of disk accesses for other statements.

n The reasons for choosing an unselective index could be outdated optimizer statistics or an optimizer bug.

n In this example, the field MANDT, which is very unselective, is specified in the WHERE clause of the SQL statement and is part of the index used. The selective field MATNR however, is not part of the index. Therefore, all the rows where MANDT = '001' have to be read on the table. Most of them have not the requested MATNR and are not passed back to the application.

SAP AG 1999

Important Execution Plans

• Index unique scan:

• Unique index is fully specified (0 or 1 records are returned)

• Index range scan:

• Index is not fully specified(possibly more than one record is returned)

• Full table scan:

• No index is accessed

• Concatenation:

• An index is accessed several times

• Nested loop:

• More than one table is accessed (for example, with views)

n The most important execution plans are:

� Index unique scan: A unique index is fully specified. For this type of access, either one or no row is returned. This type of access is inexpensive because it reads approximately 4 data blocks.

� Index range scan: An index is used for this type of access. It is not known how many rows will be returned. If the index is suitable, the index range scan can be inexpensive. However, it can also be much more expensive than performing a full table scan.

� Full table scan: The complete table is read sequentially. Each data block is read once.

� Concatenation: An index is accessed several times with different values, such as in OR and IN conditions. To ensure that a record is passed on to the application only once, a concatenation of the result is performed.

� Nested loop: If more than one table must be accessed to find the requested records, a nested loop must be performed. For example, if one table must be accessed to find the fields that are used for the access condition to a second table.

SAP AG 1999

Summary

Now you are able to describe:

• What an index is

• How an index is structured

• How data is accessed using an index

n Now you know:

� What an index is

� How an index is structured

� How data is accessed using an index

SAP AG 1999

Cost Evaluation

Techical Background

The Shared SQL Area


Update Statistics

Identify Coding


Index Utilization

Cost Evaluation

Creating an Index

Similar Statements

View Processing

Joins


SAP AG 1999

Unit: Cost Evaluation

Contents:

Objectives:

At the end of this unit you will be able to describe:

• The different factors that influence estimated cost

n Once you have completed this unit, you will know:

� The different factors that influence the costs estimated for an access path by the Oracle cost based optimizer

SAP AG 1999

Database Cost Based Optimizer

Table statistics

Costbasedoptimizer

. . .

. . ....

. . .

...

Full Table Scan

DB_FILE_MULTIBLOCK_READ_COUNT 8event = "10181 trace name context forever, level 10"...

Estimated Costs

Index Scan

Possible access paths

The estimated costs arederived from the blocksthat are expected to beread

Exact costs forthe execution cannot be calculatedbut estimated

Optimizer statistics must be up-to-date

Database and R/3 Parameter must be set correctly

SELECT * FROM ZVBAKWHEREMANDT = :A0 ANDVBELN = :A1

n In this chapter we will explain the main factors that are considered by the Oracle cost based optimizer to calculate the estimated costs for an access path. The exact cost calculation function varies depending on the Oracle release and is not disclosed by Oracle.

n The Oracle cost based optimizer estimates the costs for execution of a statement by the number of blocks to be read. Other information, like CPU time necessary to perform the access or the likelihood to find a block in the data buffer, is not evaluated.

n The number of blocks that have to be read cannot be calculated exactly before the statement is executed. Therefore, the cost based optimizer has to make assumptions about the blocks that have to be read using table statistics and a cost function. Additionally, the cost function Oracle uses is controlled by parameters.

n In order to find the best access path to the data the optimizer calculates an estimation of the costs for several possible access paths. The access path with the lowest costs is then used.

n Prerequisites for the determination of the optimal access path for a statement are up-to-date and accurate optimizer statistics and the correct setting of database and R/3 parameters that influence the optimizer.

SAP AG 1999

• Selecting the Cost or Rule based optimizer

Table statistics

SQL Statement:SELECT * FROM ZVBAK WHERE MANDT = :A0 AND VBELN =:A1

Costbasedoptimizer

Rulebasedoptimizer

Yes No

Options not supported by the Rule based optimizer

Database Optimizer

n If table statistics exist, the Oracle cost based optimizer is used.

n If no table statistics exist the Oracle rule based optimizer is used. However, if options are used that are not supported by the rule based optimizer, the cost based optimizer is used, although no statistics exist. The cost based optimizer then uses simple assumptions about the statement costs.

n Options not supported by the rule based optimizer are, for example parallel query on a table. These options could be switched on accidentally, for example by non-SAP tools on table or tablespace level during database reorganization.

n If the rule based optimizer is used you can see either 'estimated costs = 0' or no estimated costs at all displayed in the explain output. If the cost based optimizer is used, even if no statistics are available, the estimated costs for the selected access path are greater than zero.

SAP AG 1999

Estimated costs for a full table scan are proportional to the number of allocated blocks

Number of Blocks read for a Full Table Scan

n For a full table scan the number of 'blocks allocated' is used for the calculation of the estimated costs. Empty data blocks, that have never been used, don't have to be scanned and hence are not considered.

n In this example table ZVBUK first contains 1000 rows in 261 blocks. First the table has 21 allocated blocks and 240 empty blocks. The access path with the lowest estimated costs is a full table scan with estimated costs of 4.

n Then some table rows were inserted and subsequently deleted from the table. After a new optimizer statistics update the table still contains 1000 rows. However, now there are 191 allocated blocks and 70 empty blocks. The access path with the lowest estimated costs is now a concatenation of index range scans on the primary index with estimated costs of 15. The estimated costs for a full table scan are higher. Because of the Oracle architecture all allocated blocks could contain data and have to be read for a full table scan. Note that the average space per used block also increased.

n To artificially increase the number of allocated blocks you can insert dummy records into the table, which are subsequently deleted. However, you have to be careful not to overwrite or delete productive data with this operation. For a save set of records you could copy the existing data into a non existing client using database tools. This method can be used to force the optimizer to prefer index range scans over a full table scan, for example for a 'for all entries' statement to a small table, as shown in this example. On the other hand, sometimes it may be of advantage to perform full table scans to process a statement, for example if all the records of the table have to be read. If for such a statement the number of blocks allocated is high and the average free space per used block is high, you can reorganize the table to have as few blocks as possible filled with data.

SAP AG 1999

init<SID>.ora Parameter:DB_FILE_MULTIBLOCK_READ_COUNT = 16

init<SID>.ora Parameter:DB_FILE_MULTIBLOCK_READ_COUNT = 8

Estimated Costs ≈ Allocated Blocks / DB_FILE_MULTIBLOCK_READ_COUNT

Costs for a Full Table Scan

n The costs for a full table scan are estimated from the number of allocated blocks. Additionally, the number of blocks that can be read from disk by one disk read operation is taken into account, which is determined by the Oracle Parameter DB_FILE_MULTIBLOCK_READ_COUNT.

n In this example the estimated costs for the same statement, the same table contents and the same table statistics are calculated for two different settings of this parameter. With the settings:

� DB_FILE_MULTIBLOCK_READ_COUNT = 16 the costs are calculated to 535

� DB_FILE_MULTIBLOCK_READ_COUNT = 8 the costs are calculated to 844

n The higher the number of blocks that can be read by one disk read operation are, the faster the full table scan access gets. Therefore, if the parameter DB_FILE_MULTIBLOCK_READ_COUNT is increased the costs calculated are lower. That means, a full table scan could often be preferred over an index access.

n For an SAP R/3 system the most database accesses should be performed using an index scan. Therefore, the parameter DB_FILE_MULTIBLOCK_READ_COUNT is set to a small value, so that the optimizer does not select a full table scan too often.

n To our experience a good balance can be achieved by setting the parameter DB_FILE_MULTIBLOCK_READ_COUNT = 8.

SAP AG 1999

Estimated Costs = 1 for one index unique scanEstimated Costs = 1 * number of index unique scans

Costs for an Index Unique Scan

n For an Index Range Scan the costs calculated depend on several factors:

� The selectivity of the indexed fields

� The efficiency to find the data in the table (Average data blocks per key)

� The number of B*Tree Levels

� The number of Index range scans that have to be performed

� The operator used in the WHERE condition

n In this example the effect of the number of Index Scans that have to be performed is shown. In the first screen shot, only one index unique scan is performed. The costs estimated are 1. In the second screen a concatenation of three index unique scans is shown. Therefore, the estimated costs are 3, which is three times the costs estimated for one index unique scan.

n The costs for one index unique scan are calculated to 1. This number is derived from the number of blocks expected to be read on the index, which is 3, plus the blocks expected to be read on the table, which is one in this case. Because of the EVENT that is set on the database, the resulting number which is 4 is then divided by ten, which is 0,4 and then rounded to the next number which is one.

n The number of blocks that have to be read on the table and index is derived from the selectivity of the indexed fields. For a unique column combination, the number of table columns found for an index combination is always 0 or 1. Therefore, only the index root and branch blocks have to be read plus one index leaf block and one table block.

SAP AG 1999

Estimated Costs ≈ (Table Blocks + Index Blocks) / (Π Distinct values)10 (because of event 10181 or parameter optimizer_index_cost_adj)

Costs for an Index Range Scan

n For this statement an index range scan is used. Columns MANDT, VBTYP and TRVOG can be used on the index but not the field AUFNR. The costs calculated are 102.

n Although the cost function is not disclosed in detail by Oracle, here is a simplified explanation, how the costs are derived:

On the index the selectivity for the indexed fields is calculated by the number of possible combinations, which is 6 for the specified fields (1 MANDT * 3 VBTYP * 2 TRVOG = 6). One of these combination is specified. Therefore, 1/6 of the index and table has to be read approximately which is 1000. The EVENT set on the database forces the database to divide the costs by 10 for an index access, which results in the costs of approximately 100 for the access using an index range scan on ZVBAK~ZA.

SAP AG 1999

R/3 Parameter:rsdb/max_blocking_factor = 5R/3 Parameter:

rsdb/max_blocking_factor = 15

Estimated Costs = rsdb/max_blocking_factor * Costs for one index range scan

Costs for a 'FOR ALL ENTRIES' Statement

n When a number of records have to be read from a database table, the ABAP command 'for all entries' can be used. This command allows to read an amount of records in portions of a fixed number of records which are combined to the required set of records by the ABAP database interface. For this reason a number of database statements is generated that consists of a number of 'OR' terms. The number of 'OR' terms is controlled by the R/3 parameter rsdb/max_blocking_factor. The higher this parameter is, the more 'OR' terms are generated and the less statements are send to the database.

n For example: 300 values are specified in a 'for all entries' command. With the default setting of 15 for the parameter rsdb/max_blocking_factor, 20 statements are generated, each of which reads the records for 15 values. With a value of 5 for this parameter, 60 statements are generated, each of which reads the records for 5 of the values.

n The more 'OR' terms exist in a statement the higher the costs for an index access are. However, the costs calculated for a full table scan does not vary with the number of 'OR' terms. Most often in the R/3 system an index access is preferred over a full table scan. Therefore, the parameter rsdb/max_blocking_factor should be small enough that no full table scans occur. However, if the parameter is set to a small value, the number statements that have to be processed increases. Therefore, if this parameter is set to a low value and a full table scan is still performed, even more full table scans result.

n To our experience a good balance can be achieved by the default setting of the parameter rsdb/max_blocking_factor, which is 15.

SAP AG 1999

Estimated Costs ≈ (Table Blocks + Index Blocks) * 5%( for BETWEEN or * 10% for LIKE, <, >) 10 (because of event 10181 or parameter optimizer_index_cost_adj)

Costs for Operators: Between, Like, < and >

n The estimated costs of statements where fields are specified with the operators BETWEEN, LIKE, < and > are independent of the distinct values for the field. The optimizer assumes, that a fixed fraction of the table and index has to be read. For the operator BETWEEN a fraction of 5% is assumed, for LIKE, < and > a fraction of 10% is assumed.

n In these two examples the costs for a BETWEEN on two different fields is shown. The first example shows a between on the document number VBELN with 100.000 distinct values. The second example shows a BETWEEN on field VBTYP with only 4 distinct values. In the second example the estimated costs are slightly higher because the index has a few more leaf blocks.

n The cost function for a statement with a BETWEEN clause is approximately : Estimated costs ≈ (number of table blocks + index blocks)*5% / 10 (because of event 10181 or or parameter optimizer_index_cost_adj)

n The cost function for a statement with a LIKE, < and > clause is approximately : Estimated costs ≈ (number of table blocks + index blocks)*10% / 10 (because of event 10181 or or parameter optimizer_index_cost_adj)

SAP AG 1999

Estimated Costs ≈ (Table Blocks + Index Blocks) * 5%*5% 10 (because of event 10181 or parameter optimizer_index_cost_adj)

Costs for two BETWEENS

n For more than one of the operators BETWEEN, LIKE, < and > in one statement that can be used on the same index, the estimated fractions of table and indexes that are read are multiplied.

n In this example a statement with two BETWEEN clauses the cost function is approximately : Estimated costs ≈ (number of table blocks + index blocks)*5% *5% / 10 Estimated costs ≈ (6251 + 376)*5% *5% / 10 = 1.6 ≈ 5

n The discrepancy of 1.6 to 5 is probably to to truncation during the internal calculation.

SAP AG 1999

Parameter: dbs/ora/use_hints

n In this example a statement to Search Help M_VMVAA is shown twice, Once with and once without the hint /*+ first_rows */ . Without the hint the statement is processed using a sort merge join and the costs calculated are lower (8.534) than with the hint (12.737) where a nested loop is used. However, when using a sort merge join the time required to send the first records is much higher because large fractions of the tables VBUK, VBAK and VBKD have to be read and sorted prior to sending the first record to the application.

n This hint can be generated into the statement by the database interface when the R/3 parameter dbs/ora/use_hints is set and the statement contains a 'ROWNUM <=' clause. On average the number of records that were requested for this statement by the application is 1.7, whereas the estimated number of rows that match the WHERE clause is 13.866. Therefore, it makes sense to optimize the statement for the time of the delivery of the first records returned.

n Using the hint /*+ first_rows */ the optimizer optimizes the number of block reads required to generate the first fetch. For a nested loop, usually few blocks have to be read to find the first data records that can be returned to the application. For other access paths, for example a sort merge join, a lot of data has to be read before the first record can be returned. However, the time required to return the last record may be shorter for the sort merge join than for the nested loop.

n The settings for parameter dbs/ora/use_hints depends on the R/3 release (See also R/3 note 135048):

first ->1 (R/3 ≤ 4.5A)

upto ->1000 (R/3 ≥ 4.5B)

SAP AG 1999

Estimated Costs for Other Access Paths

n To analyze what the estimated costs are for other access paths you can explain the statement with a hint. The optimizer also calculates the cost associated with the access path displayed. Use this information to analyze why the cost based optimizer has selected a certain access path.

n When you type in index names you have to type double quotes around an object that contains non alphanumeric characters like ~ or _. When you type an incorrect hint, Oracle will treat it as a comment which is disregarded.

n The following hints are often useful:

� RULE: Use the rule based optimizer

� FULL(TABLE): Use a full table scan

� INDEX(TABLE): Use the index range scan with the lowest estimated costs

� INDEX(TABLE "INDEX"): Use the specified Index

� FIRST_ROWS: Use the access path that the returns the first rows as quick as possible

� ALL_ROWS: Use the access path that the returns all the rows as quick as possible

n Note that the hints are not actually used for the statement execution, but only for the explain function. To use a hint in the statement you have to write the hint into the ABAP statement. Hints will be available for OPEN SQL as of R/3 release 4.5 (See R/3 note 129385). Prior to that release EXEC SQL has to be used to include a hint into a statement.

SAP AG 1999

Optimizer Trace

n The optimizer trace is a tool that is used by Oracle support staff for analysis of optimizer problems. This trace shows the different access paths and the estimated costs associated to them. However, it is difficult to read and not suitable for a standard analysis.

SAP AG 1999

Optimizer Problems

n A cost based optimizer is a complicated set of logic, equations and table statistics. Here are some examples where the optimizer selects an access path that is not optimal because the costs are not estimated correctly:

� The calculation scheme for the event rounds the estimated costs off. This often causes problems if the costs for a concatenation of several index range scans are estimated.

� For operators other than '=' the distinct values are not taken into account when calculating the estimated costs. This often causes problems if unselective fields are specified with operators different from EQUAL and a more selective field from a different index is also specified.

� The data distribution is not uniform For example, a field is indexed which has two possible values. One of the values occurs very often and the other one only in exceptional cases. If the query searches for the exceptional cases, the index is advantageous. If the query searches for the value that occurs often, a full table scan is probably more efficient. However, the optimizer will always choose the same access path because bind variables are used. This problem can be fixed in release R/3 4.6 when hints and histograms will be available.

n For problems with the cost estimation the number of Gets per Execution is much higher than the 10*Estimated costs and a more suitable access path exists but is not chosen by the optimizer

n Please open a problem message for such problems in component BC-DB-ORA

SAP AG 1999

Table ZVBAK Last statistics date 10.01.1999 Analyze Method Estimate 10% Number of rows 100.000 Number of blocks allocated 5.556 Number of empty blocks 45 Average space 1.173 Chain count 0 Average row length 383

Show indexes of table ZVBAK

Appendix: Table Statistics

n Table statistics contain:

� The date of the last statistic update.

If the statistics are not maintained recently, the optimizer might be mislead by wrong statistical data, which might be the cause of the expensive statement. Update the optimizer statistics for the tables accessed by the statement if the statistics are older than one week and check if the statement remains expensive.

� Analyze Method

The analysis method is shown together with the option used. Here the analysis method was Estimate with option 10%.

� Number of rows

The number of rows in the table is compared to the number of distinct values of the different fields in the available indexes to find out the selectivity of the different indexes.

� Number of blocks allocated

This is the number of blocks that have to be read in case of a full table scan. This number is also called the High water mark of the table. Other blocks may be allocated in the extents of the table but have never been used before.

SAP AG 1999

Table ZVBAK Last statistics date 10.01.1999 Analyze Method Estimate 10% Number of rows 100.000 Number of blocks allocated 5.556 Number of empty blocks 45 Average space 1.173 Chain count 0 Average row length 383

Show indexes of table ZVBAK

Appendix: Table Statistics

n Table statistics contain (continued):

� Number of empty blocks

These blocks have never been used before. Other blocks may be empty but have been used before. The number of blocks allocated in the extents of the table is the Number of blocks allocated plus the Number of empty blocks.

� Average space

This is the average free space in a used block in bytes. Compare this number to the Oracle block size for R/3 of 8192 bytes. A full table scan is inefficient if a lot of blocks are read that contain a lot of free space.

� Chain count

This is the number of chained rows in the table.

� Average row length

This is the average length of a row in bytes.

SAP AG 1999

Show indexes of table ZVBAK UNIQUE Index ZVBAK~0 Column Name #Distinct MANDT 1 VBELN 100.000 Last statistics date 10.01.1999 Analyze Method Estimate 10% Levels of B-Tree 1 Number of leaf blocks 315 Number of distinct keys 100.485 Average leaf blocks per key 1 Average data blocks per key 1 Clustering factor 5.880

Index statisticsContinue

Appendix: Index Statistics

n Index statistics consist of the following values:

� Distinct values

For all indexed fields the number of different values is calculated. In this example there is one value for the field MANDT and 100.000 different values for the field VBELN in this table.

� Last statistics date

Table and index statistics can be maintained at different dates. Do not forget to check the table and index date of the last statistics update. If the statistics are older than one week for either the index that is used for the execution of the expensive statement or the index that should have been used instead, update the statistics of the table and indexes and check if the access path changes.

� Analyze Method

The analysis method for index and table statistics can be chosen independently. Check for the table and indexes involved in the expensive statement if the statistics were created with sufficient accuracy.

� Levels of the B-Tree

This is the number of index B* Tree levels above the leaf level. Therefore, the minimum number of blocks that have to be read on the index is 'Levels of B-Tree +1'. In this example the index has only the root block and the leaf level but no branch levels.

� Number of leaf blocks

These are the number of blocks on the lowest level of the Index B*Tree. In case of a Full Index Scan, this is the number of blocks that have to be read.

SAP AG 1999

Show indexes of table ZVBAK UNIQUE Index ZVBAK~0 Column Name #Distinct MANDT 1 VBELN 100.000 Last statistics date 10.01.1999 Analyze Method Estimate 10% Levels of B-Tree 1 Number of leaf blocks 315 Number of distinct keys 100.485 Average leaf blocks per key 1 Average data blocks per key 1 Clustering factor 5.880

Index statisticsContinue

Appendix: Index Statistics

n Index statistics consist of the following values (continued):

� Number of distinct keys

This is the number of different combinations that exist for the indexed fields. Compare this value to the number of table rows. If the index is selective, this number is close to the number of table rows.

� Average leaf blocks per key

These are the average number of leaf blocks that one combination of the index key occupies. An index is either fragmented, or unselective for at least some column combinations, if this number is much higher than one. A unique index allows only one record in the table per column combination. Therefore, for a unique index there should be an average of 1 leaf blocks per key, otherwise the index is fragmented.

� Average data blocks per key

This is the average number of data blocks that have to be read if the index is fully specified. For a selective index this number is close to 1.

� Clustering factor

This factor shows how well the table is ordered by the index criteria. If the table is perfectly ordered by the index criterion, the clustering factor equals the table size in blocks. Otherwise the clustering factor is higher.

SAP AG 1999

• DB_FILE_MULTIBLOCK_READ_COUNT

• Controls the number of blocks read with one diskaccess

• OPTIMIZER_INDEX_COST_ADJ

• Controls cost calculation functions for indexaccess paths

• EVENTS

• Control cost calculation functions

Appendix: Database Parameters that Control CostCalculation Functions

n Database parameters that influence the cost based optimizer are:

� db_file_multiblock_read_count

� This parameter sets the number of blocks that are read for one read request from disk. For a full table scan the costs are proportional to: Number of used blocks / db_file_multiblock_read_count

� Therefore, this parameter reduces the costs calculated for a full table scan

� optimizer_index_cost_adj (As of Oracle 8.0.5 and R/3 ≥ 4.0)

� this parameter has the same effect as the event formerly used:

� event = "10181 trace name context forever, level 10" (Up to Oracle 8.0.4. and R/3 4.x)

� Events can be used to modify the database behavior, for example to force the optimizer to use a different calculation scheme. Currently in R/3 we use this event or parameter to reduce the costs calculated for an index access to 10% of the original value. This is necessary to compare the costs calculated for an index range scan to the costs calculated for a full table scan, because the costs for a full table scan are reduced by the parameter db_file_multiblock_read_count.

SAP AG 1999

• RSDB/MAX_BLOCKING_FACTOR

• Controls the costs of a statement by the numberof OR terms generated for an ABAP FOR ALLENTRIES command

• DBS/ORA/USE_HINTS

• Adds a hint to statements of a certain type

Appendix: R/3 Parameters that Control SQL Statements

n R/3 Parameters that influence the cost based optimizer are:

� rsdb/max_blocking_factor

This parameter controls the number of OR terms in a FOR ALL ENTRIES Statement. For such a statement the optimizer performs either a full table scan or several index range scans to find the records. If index range scans are used, the costs calculated vary with the number of OR terms.

� dbs/ora/use_hints (See also R/3 note 135048)

first ->1 (R/3 ≤ 4.5A)

upto ->1000 (R/3 ≥ 4.5B)

Using this parameter you can enable the R/3 database interface to add a hint to the statement. Currently the hint /*+ first_rows */ can be used for statements with a 'ROWNUM ≤' clause. Such statements limit the number of rows that are returned by the database. This clause is issued by the ABAP code "up to <n> rows" and is also appended to the statements generated from Search Helps.

Because of the bind variable used for the ROWNUM, the optimizer gets no information about the number of records requested by the application and assumes that the majority of the records that match the WHERE clause of the SQL statement have to be returned. However, in the R/3 system often the opposite is true, only a small number of records is requested.

Other hints exist, but are currently not used.

SAP AG 1999

Creating an Index

Techical Background

The Shared SQL Area


Update Statistics

Identify Coding


Index Utilization

Cost Evaluation

Creating an Index

Similar Statements

View Processing

Joins


SAP AG 1999

Unit: Creating an Index

Contents:

• Missing indexes

• Guidelines for creating an index

Objectives:


• Determine if an index could be created

• Determine if an SQL statement is expensive


� Determine if an index could be created

� Determine if a statement is expensive by analyzing the Shared SQL Area

SAP AG 1999

Missing Indexes

l Before you create an index, check for missing indexes

n Before you create an index, check for missing indexes using Transaction DB02.

n Indexes are declared in the ABAP Dictionary and are created in the database. If an index does not exist in either the dictionary or the database, it is missing.

� If the index does not exist on the database, queries for which the index is intended will be inefficient and slow.

� If the index does not exist in the ABAP Dictionary you cannot see the index in the ABAP Dictionary. This may lead to confusion, for example, when creating an index or analyzing interference between indexes.

n When do missing indexes occur?

� When a table is reorganized, all the indexes are dropped. These indexes must be recreated after a table reorganization.

� Indexes created in the ABAP Dictionary must be activated and generated before they are available on the database. If the indexes are not activated and generated, they are missing.

SAP AG 1999

Check table statistics

l Before you create an index check the table statistics

n Before you create a new index check the table statistics of the existing index for the following criteria:

� Are the statistics current

� What is the table size now and how large was the table at the time of the last update statistics

SAP AG 1999

Rules for Creating Indexes

l Use only selective fields if possible

l Use as few fields as possible

l Use selective fields at the beginning if possible

l Do not create an unselective index

l Create as few indexes per table as possible

l Do not create an index that can be used unintentionally

l Do not change an index created by SAP

n When you create an index, use only selective fields if possible.

n Use as few fields as possible. Every field that is included in the index costs storage space and makes the index access less efficient.

n Selective fields should appear at the beginning of an index if possible.

n Do not create an unselective index.

n Never change an SAP created index, unless you are requested to do so by SAP.

SAP AG 1999

Selectivity Analysis

l Which Index Should Be Created?

l To find out the selectivity of an index, use:

n Transaction DB05Or

n SQLPLUS:

select Field1,Field2,Field3,count(*)from Tablegroup by Field1,Field2,Field3having count(*) > 100order by count(*);

This statement shows which combinations of Field1, Field2, andField3 occur more than a hundred times

l Selectivity analysis is expensive!

n Before you create an index, you must know how selective the index would be.

n To display the distribution of records for possible field and value combinations of the index, use Transaction DB05.

n You can also use sqlplus to determine the index selectivity. The SQL statement given above will return the combinations that occur more than a hundred times.

n Note: The selectivity analysis using either Transaction DB05 or SQLPLUS is expensive.

SAP AG 1999

Number of Records returned

Index:Field1Field2Field3

SELECT FROM TABLE WHERE FIELD1 = ‘A’AND FIELD2 = ‘B’AND FIELD3 = ‘C’

How many records will bereturned by the Index scan?

Num

ber

of d

iffer

ent

com

bina

tions


lHistogram of combinations

n An index range scan is cheap if only a few records are found using the index.

n To ensure that there are no combinations of values that would return many columns a histogram can be calculated. The table contents are scanned for the different possible combinations of the indexed fields. For each combination the number of records that are returned is recorded. In the histogram the number of different combinations that return a certain number of records is recorded.

SAP AG 1999

Number of Recordsreturned

Num

ber o

f diff

eren

tco

mbi

natio

ns



A Selective Index An index that is selectivefor most combinations ofindexed fields

An index that is not selectivefor most combinations ofindexed fields

Num

ber o

f diff

eren

tco

mbi

natio

ns

Num

ber o

f diff

eren

tco

mbi

natio

ns


Histograms of combinations

n The histogram on the left hand side shows an index that is selective for all of the possible field combinations.

n If the index is selective for most of the combinations of indexed fields a histogram similar to the one displayed in the middle will result. Most of the combination return only a few records and some combinations of fields exist that would return a lot of records. In this case you must check if a query could be issued by the application with the combinations of values for which a lot of records would be returned. This would cause an expensive unselective index range scan. Only the developers of the table and the programs that are selecting data from the table can decide if this could happen.

n For example: A spool table exists where all the print requests are stored. All requests have a status flag STATUS, that can be set to either ‘printed’ or ‘not printed’. The majority of the spool requests are usually in the status ‘printed’ and only a few are ‘not printed’. The histogram for an index with the fields PRINTER and STATUS will show some combinations for which only a few records would be returned by the index, that are the not yet printed print requests to a certain printer. Other combinations would return many records, these are all the print requests that are already printed on a printer. Only if it can be assured by the application and table logic, that the already printed print requests are not selected from the spool table using the flag STATUS the index could be created to accelerate the search of the not yet printed print requests.

n An index that usually returns many records will show a histogram similar to the histogram as displayed on the right hand side. Such an index should not be created.

SAP AG 1999

In this example, there are 18,607different combinations for all ofthe fields.Compare this number to thenumber of table rows.


n Transaction DB05 displays the selectivity of a field combination. Use this information to determine if an index should be created.

n The number of Distinct values shows you how many different combinations of fields exist. This number should be close to the number of table rows for the combination of all of the key fields of the index.

n In this example, there are:

� 4,506 different values for field USPOB

� 7,172 possible combinations of USPOB and GJAHR

� 9,195 possible combinations of USPOB, GJAHR, and WRTTP

� 18,607 possible combinations of all the fields.

� 635, 336 table rows

n When you compare the possible combination of all the fields to the number of table rows, you can determine that this index would return 30 rows on average.

SAP AG 1999

If the number of distinctvalues does not increasefor a field, the field isunselective.


n If the number of distinct values does not increase for a field, the field should not be included in the index.

SAP AG 1999

In this example, 16,587 combinationsof all the fields would return 1 to 10rows.


n This column display how many field combinations would return 1 to 10 rows.

n For a selective index, these numbers are close to the number listed in the column Distinct values.

SAP AG 1999

In this example, 8 combinations of all thefields would return up to100,000 rows.This index could cause an unselectiveindex range scan.


n In this example, 8 combinations of all the index fields have a hit set of between 10,001 and 100,000 records. If one of these combinations is specified, an expensive unselective range scan will occur.

n In general, it is not effective if an index returns many rows.

n To determine if you should create an index, even if unselective field combinations exist, you must know the table logic.

SAP AG 1999

SQLPLUS

n To display the combinations of index fields for which more than a hundred records exist, use the following SQL statement: select Field1,Field2,Field3,count(*) from Table group by Field1,Field2,Field3 having count(*) > 100 order by count(*);

n In this example, 80 combinations of the fields USPOB, GJAHR, WRTTP, and VERSN exist with more than a hundred records.

n The combination that returns the most records is the combination of the fields where USPOB is blank, the field GJAHR = 1997, the field WRTTP = 03 and VERSN = 000. This combination occurs 2898 times.

SAP AG 1999

• Check for missing indexes with DB02 and recreate them

• Run Update statistics

• Change the code so that existing indexes can be used

• Changing an index, so that it can be utilized for queries thatused it before as well as the expensive queries

• Create a new index to best satisfy the query

Never change an SAP Index without approvaland advisement of SAP


n Check for missing indexes with DB02 and recreate them

n Run update statistics in the mode that is necessary for the table

n Change the code so that existing indexes can be used

n Changing an index, so that it can be utilized for queries that used it before as well as the expensive queries

n Create a new index to best satisfy the query

SAP AG 1999

Summary


• Determine if an index should be created

• Determine if a statement is expensive


� Determine if an index should be created

� Determine if a statement is expensive by analyzing the Shared SQL Area

SAP AG 1999

Techical Background

The Shared SQL Area


Update Statistics

Identify Coding


Index Utilization

Cost Evaluation

Creating an Index

Similar Statements

View Processing

Joins


Similar Statements

SAP AG 1999

Unit: Similar Statements

Objectives:


l Find expensive similar statements in the SharedSQL Area


� Find expensive similar statements in the Shared SQL Area

SAP AG 1999

Similar Statements

l Use the same access path

l Have the same or similar WHERE clause

l Have different entries in the Shared SQL Area

l Can be accelerated by using one tuning method

l Might not be individually expensive, but the sum of similarstatements can be expensive

n Similar statements use the same access path. They may have the same or a similar WHERE clause. Similar statements, however, have different entries in the Shared SQL Area.

n Using one tuning method, such as creating, changing, or dropping an index could help to accelerate the similar statements.

n Each of the similar statements might not be expensive, but the sum of similar statements can be expensive.

SAP AG 1999

How to Find Expensive Similar Statements

l In the Shared SQL Area, sort by:

n Buffer gets/Execution

n Buffer gets/Record

l Check the top pages of the output for similar statements

n If similar statements are processed using the same access path, the number of Buffer gets/Execution is often equal. You can also sort by Buffer gets/Record. For similar statements that are using the same access path, this number is also often equal.

n To find similar statements in the Shared SQL Area, you can:

� Sort the statements by Buffer gets/Execution and Buffer gets/Record

� Check the top pages of the output for statements that cause more than 0,1% of the total Buffer gets or a significant amount of Buffer gets/Execution and Buffer gets/Record.

n To determine if the statements found are similar, compare the SQL statements.

n Add the Buffer gets for all similar statements. If the sum of all statements is expensive, treat these statements as a single expensive statement.

SAP AG 1999

Example

SELECT * FROM V_ANLSUM1 WHERE ... AND KOSTL IN (A, B)

SELECT * FROM V_ANLSUM1 WHERE ... AND KOSTL IN (A, B, C)

SELECT * FROM V_ANLSUM1 WHERE ... AND KOSTL IN (A, B, C, ... , Z)

.

.

n In this example, there are many statements that read the same number of blocks per execution. None of the statements has more than 5% of the total number of buffer gets and not one of the statements is executed often. Therefore, none of these statements are considered expensive.

n However, the statements all have the same structure and are all using the same access path. Only the IN condition consists of a different number of entries. The sum of the buffer gets for all statements is very high. Therefore, the set of similar statements is expensive.

n The similar statements in this example were caused by a different number of IN conditions. To avoid using IN conditions, use the ABAP command FOR ALL ENTRIES instead.

SAP AG 1999

Why Similar Statements Occur

l The Shared SQL Area treats statements as "different"statements if the character string is different

l A character string is different if:

n There is a different number of IN conditions

n There is a different number of OR conditions

n Different unselective fields are specified

n Different selected fields are specified

n It is written once in upper case and once in lower case (third partytools only)

n The Shared SQL Area treats statements as "different" statements if the character string is different. For example, if one statement is issued twice, once in lower case and once in upper case, it is stored twice in the Shared SQL Area. Even if only a single character is different, the statement is stored twice.

n Statements with the same structure but a different number of IN or OR conditions are also treated as different statements by the Shared SQL Area.

n Statements with a different number of selected table fields are also different, for example ’SELECT <COLUMNLIST> FROM TABLE’ is different than ’SELECT * FROM TABLE’.

SAP AG 1999

Possible Optimizations

l Change the ABAP code so that the statements matchexactly, where appropriate

l Use the ABAP command FOR ALL ENTRIES instead ofusing IN conditions

l Use the optimization possibilities for non-similar statements

n To optimize similar statements, you can:

� Change the ABAP code so that the statements match exactly, where appropriate

� Use the ABAP command FOR ALL ENTRIES instead of using IN conditions

� Use the optimization possibilities for non-similar statements

SAP AG 1999

Summary


• Find similar statements in the Shared SQL Area


� Find similar statements in the Shared SQL Area

SAP AG 1999

View Processing

Techical Background

The Shared SQL Area


Update Statistics

Identify Coding


Index Utilization

Cost Evaluation

Creating an Index

Similar Statements

View Processing

Joins


SAP AG 1999

Unit: View Processing

Objectives:


l Explain how views are processed by the Oracle database

l Analyze expensive statements to views


� Explain how views are processed by the Oracle database

� Analyze expensive statements to views

SAP AG 1999

Views

T1.K1 T1.F1 T1.F2

T1 K1 F1 F2

T2 K1 F3 F4

Database view

T2.F3 T2.F4

Applicationtransaction

Database interface

l A view is a logicalobject. No physicaldata is stored in thedatabase.

l The data is read fromthe appropriate tableduring runtime.

n A view is a logical object. That is, the data of a view is not physically stored on the database. Instead, the data of a view is read from one or more tables during runtime.

n The structure of a view is defined by the tables and fields it contains and a JOIN condition.

n A view can be used to summarize data that is distributed among several tables (database view).

n A view can also be used to reduce the number of fields read from a table, keeping interfaces to a minimum (projection view).

SAP AG 1999

View Processing

• Nested Loop:

1. The table access order is determined by the optimizer

The Join Conditions are analyzed to find the table from which thefields with join conditions are read from.

The Optimizer chooses the table that appears to return thesmallest number of rows first.

The sequence of the inner tables is chosen accordingly.

2. The data from the view is selected

n The database has different ways to process statements to a view:

� Nested Loop

� Sort Merge Join

� Hash Join.

n The access path with the lowest number of estimated block reads is used.

n For a Nested Loop, the number of blocks to be read depends critically on the number of rows read on the outer table. The steps of processing a view are as follows:

� 1. The access path for the view is determined. To do this, the appropriate sequence to access the different tables of the view is determined:

Join conditions are analyzed to determine from which table fields with join conditions are read.

The optimizer chooses the table that appears to return the smallest number of rows as the outer table.

The sequence of the inner tables is chosen accordingly

� 2. The data is selected from the tables of the view

The data from the outer table is selected first

The inner tables are accessed using the data that was selected from the outer tables and the JOIN conditions, until all the data is read from the view

SAP AG 1999

View Processing

• Sort Merge Join:

1. All appropriate data is read from the joined tables and sortedaccording to the join condition

2. The rows of the tables are merged

MANDT VBELN OBJNR...

001 0000120 0000110001 0000121 001 0000122 0000110001 0000123 0000777001 0000124 0000777001 0000125 0000777

...

MANDT VBELN CUOBJ ...001 0000121 1 ...001 0000122 2 ...001 0000123 3 ...001 0000123 4 ......

001 0000123 12345 ...001 0000456 12345 ...

ZVBAP ZVBAK ZVBUK

MANDT VBELN ...001 0000121 ...001 0000122 ...

001 0000123 ...

...

SELECT * FROM ZVIVEDA WHERE MANDT = '001' AND CUOBJ = '12345'AND OBJNR = '0000777'

Data read for the Sort Merge Join

Data that is containedin the joined view

Sort Merge Join Nested Loop

n For a Sort Merge Join the steps of processing a view are as follows:

� 1. The appropriate data of the different tables for the view is read and sorted according to the join conditions.

� 2. The data of the different tables is merged, that is the data rows and fields that belong to the view according to its definition are returned to the application.

� A sort merge join is only efficient if selective criteria are specified in the where clause to both of the tables in the jo in. If no selective join conditions exist on a joined view, this access path is more efficient than a nested loop. However, for most SAP Views the join conditions are very selective.

� If you find an expensive statement that is processed with a sort merge join and has a ROWNOM clause, please check note 135048.

n A sort merge join can be combined in the access path with a nested loop.

n In this example a sort merge join is performed for tables ZVBAP and ZVBAK. From ZVBAP all records for MANDT = '001' and CUOBJ = '12345' are read. From ZVBAK all records for MANDT = '001' and OBJNR = '0000777' are read. Both result sets are then sorted and merged according to the join conditions on MANDT and VBELN. Only the record with the VBELN = '0000123' is contained in both result sets. With a nested loop this record is then also read from table ZVBUK which is also part of the joined view ZVIVEDA.

SAP AG 1999

View Processing

• Hash Join:

Should be disabled for R/3 by the init<SID>.ora parameter:

HASH_JOIN_ENABLED = FALSE

Hash Join operations are only efficient if a major fraction of each ofthe view tables has to be read to satisfy the query

n Hash Join operations should be disabled by setting the init<SID>.ora parameter:

� HASH_JOIN_ENABLED = FALSE

� A hash join operation is also only efficient if the majority of the different table records are requested, this is not normally the case for R/3 operations.

SAP AG 1999

Importance of the Table Access Order for a Nested Loop

• Correct Access order

MANDT VBELN ...001 0000121 ...001 0000122 ...001 0000123 ...001 0000124 ...001 0000125 ...001 0000126 ...

...

001 1000815 ...

...

MANDT VBELN CUOBJ ...001 0000121 1 ...001 0000122 2 ...001 0000123 3 ...001 0000124 4 ...001 0000125 5 ...001 0000126 6 ...

...

001 1000815 12345 ......

SELECT * FROM ZVIVEDA WHERE MANDT = '001' AND CUOBJ = '12345'

ZVBAP ZVBAK ZVBUK

Join conditions on fields MANDT, VBELN for all tables

MANDT VBELN ...001 0000121 ...001 0000122 ...001 0000123 ...001 0000124 ...001 0000125 ...001 0000126 ...

...

001 1000815 ...

...

n In this example the view ZVIVEDA is created on three tables ZVBAP, ZVBAK and ZBUK with join conditions on the fields MANDT and VBELN. That means, only records of these tables are eligible for the view if records with the same values for these fields exist in all three tables.

n The query selects all records from the view where the fields MANDT = '001' and CUOBJ = '12345', which are both read from table ZVBAP.

n Only for the records that match the where condition on the outer table, the records from inner tables are read. For finding the records on inner tables the join conditions can be used. That means in this example, for table ZVBAK the records are read where MANDT = '001' and VBELN = '1000815'. Further fields specified from ZVBAK would have been used as well.

n Because of the join conditions on table ZVBUK, the record where MANDT = '001' and VBELN = '1000815' is read as well. In total 3 records are read in total.

n Because on the inner tables of a view the rows are often found by the jo in conditions, an appropriate index must exist on the join conditions to find the records efficiently.

SAP AG 1999

SELECT * FROM ZVIVEDA WHERE MANDT = '001' AND CUOBJ = '12345'

ZVBAPZVBUK

Join conditions on fields MANDT, VBELN for all tables

MANDT VBELN CUOBJ ...001 0000121 1 ...001 0000122 2 ...001 0000123 3 ...001 0000124 4 ...001 0000125 5 ...001 0000126 6 ...

...

001 1000815 12345 ......

ZVBAK

MANDT VBELN ...001 0000121 ...001 0000122 ...001 0000123 ...001 0000124 ...001 0000125 ...001 0000126 ...

...

001 1000815 ...

...

MANDT VBELN ...001 0000121 ...001 0000122 ...001 0000123 ...001 0000124 ...001 0000125 ...001 0000126 ...

...

001 1000815 ...

...

Importance of the Table Access Order for a Nested Loop

• Incorrect Access order

n In this example the view ZVIVEDA is processed with a different sequence of table accesses. First table ZVBAK is read. On this table the field CUOBJ does not exist. Therefore, all the records where MANDT = '001' are selected. For each of these records the appropriate records in the inner tables are read. To find these records the join conditions can be used, that is the records are searched with the criteria MANDT = '001' and VBELN = '0000121' then the record MANDT = '001' and VBELN = '0000122' etc. All the records of the inner tables where the search criteria of the outer table matches are found. However, when the last table VBAP is accessed, the condition CUOBJ = '12345' is checked and the database finds that the record is not valid for the select statements where condition in most of the cases.

n For views the order of table access paths is important for the statement costs. Thus, the statement costs for views are calculated for a different orders of table access. Also, costs for other view access strategies are evaluated, like hash join and sort merge join .

SAP AG 1999

ZVBAP~ZA--------------------MANDTCUOBJ

Show execution plan for SQL statement

SELECT *FROM ZVIVEDAWHERE MANDT = :A0 AND CUOBJ = :A1

Execution Plan

SELECT STATEMENT ( Estimated Costs = 108 )NESTED LOOPS

TABLE ACCESS BY ROWID ZVBAP INDEX RANGE SCAN ZVBAP~ZA INDEX UNIQUE SCAN ZVBUK~0

TABLE ACCESS BY ROWID ZVBAK INDEX UNIQUE SCAN ZVBAK~0

Execution Plan for a View Statement

Table Field Table FieldVBAK MANDT = ZVBUK MANDTVBAK VBELN = ZVBUK VBELNVBAP MANDT = ZVBUK MANDTVBAP VBELN = ZVBUK VBELN

ZVBUK~0--------------------MANDTVBELN

ZVBAK~0--------------------MANDTVBELN

Indexes

View field Table Field nameMANDT ZVBAK MANDTVBELN ZVBAK VBELNPOSNR ZVBAP POSNRCUOBJ ZVBAP CUOBJ

Join ConditionsView Fields

n In this example the select statement selects all records from view ZVIVEDA where MANDT = :A0 and CUOBJ = :A1.

n The optimizer chooses to perform a nested loop for the statement. First table ZVBAP is read with an index range scan. This is the outer table of the view. For all records found in this table and according to the join conditions, records from the next tables are selected. The next access is to index ZVBUK~0. Because all data needed for the view is contained in the index no table access is performed. An index unique scan can be performed because the fields MANDT and VBELN are known from the access to table ZVBAP. The last table accessed is ZVBAK. Here the join conditions can also be used for the access to the data. However, in contrast to table ZVBUK some fields contained in the view are not contained in the primary index. Therefore, a table access has to be performed.

n In this example the transitivity of the join conditions can be used to read the field MANDT from table ZVBAP although in the view definition this field is defined to be read from ZVBAK.

n The cost based optimizer is in general able to resolve the join conditions and make use of the transitivity of the join conditions. However, in some cases errors occur. If the optimizer chooses the wrong outer table, you can change the table from which a view field is read for all fields with join conditions. In this case you could change the table from which the fields MANDT and VBELN are read from to one of the tables ZVBAK, ZVBAP and ZVBUK.

SAP AG 1999

Never change an SAP View without approvaland advisement of SAP


l Run Update statistics

l Change the code so that an existing index of the outer tableof the view can be used

l Change the view definition

l Create an index to support access to the outer table of theview

l Change the code to a nested select instead of a view

n Before you analyze the expensive statement in detail run update statistics for all tables of the view. Check also if an increased precision of the update statistics leads to a better access path.

n Change the code so that an existing index of the outer table of the view can be used

n Change the view definition. You can change the table from which the fields with join condit ions are read from.

n Create an index to support access to the outer table of the view

n Change the code to a nested select instead of a view

SAP AG 1999

Summary


• Analyze expensive statements to views


� Analyze expensive statements to views

SAP AG 1999

Joins

Techical Background

The Shared SQL Area


Update Statistics

Identify Coding


Index Utilization

Cost Evaluation

Creating an Index

Similar Statements

View Processing

Joins


SAP AG 1999

Unit: Joins

Objectives:


l Explain how joins are processed by the Oracle database

l How joins are structured in the ABAP code

l Analyze expensive statements to joins


� Explain how joins are processed by the Oracle database

� Explain how joins are structured in the ABAP code

� Analyze expensive statements to joins

SAP AG 1999

SQL Statements for Joins

SELECTT_0 . "MATNR" , T_0 . "MEINS" , T_0 . "SPART" , T_1 . "MAKTX" , T_1 . "MATNR" , T_1 . "SPRAS" , T_2 . "KTGRM" , T_2 . "MATNR" , T_2 . "PRODH" , T_3 . "MATNR" ,T_3 . "TAXM1"FROM "MARA" T_0 , "MAKT" T_1 , "MVKE" T_2 , "MLAN" T_3WHERE ( T_0 . "MANDT" = :A0 AND T_1 . "MANDT" = :A1 AND T_1 . "MATNR" = T_0 . "MATNR" ) AND ( T_2 . "MANDT" = :A2 AND T_2 . "MATNR" = T_1 . "MATNR" ) AND( T_3 . "MANDT" = :A3 AND T_3 . "MATNR" = T_2 . "MATNR" ) AND

( T_0 . "MTART" BETWEEN :A4 AND :A5 AND T_0 . "SPART" BETWEEN :A6 AND :A7 )

Join Conditions

Selection Conditions

Join Fields

Joined Tables

n You can identify Joins in the shared SQL area by statements that contain variables of type "T_0". The first part of the statement defines the join fields that are read from the different joined tables that are defined in the from clause. The first blocks of the where clause define the join conditions. You can identify the join conditions by the comparison of two different table fields, for example T_2."MATNR" = T_1."MATNR". The last part of the Where clause are the selection conditions. The selection conditions are the conditions that can limit the number of rows returned from the outer table of the join.

n The field MANDT is specified for all tables part of the join.

n Joins are processed similar to joined views.

SAP AG 1999

Execution plan of a Join

n Joins are processed similar to joined views on the database.

n In this example the optimizer chooses a nested loop as access path. The access sequence is not necessarily the sequence of tables in the from clause. However, from the logic of joins in the ABAP, usually the best access sequence is the same as the sequence of tables in the from clause.

n When you analyze a join statement:

� Check if the table is read first in the access sequence, that returns the smallest number of rows. This is the outer table of the join.

� Check if for each table the correct index is used. Fields that can be used on index are for the outer table the selection conditions and for the inner tables the selection condition and the fields from join conditions to tables that were accessed before.

n In this example for table MARA the fields MANDT, MTART and SPART can be used for the table access. For table MAKT the fields MANDT and MATNR can be used on indexes, since field MANDT is specified and field MATNR is part of the join condition between table MAKT and MARA. Table MARA is read first. Therefore, the record found on MARA contains the MATNR which can be used to find the corresponding record of table MAKT.

SAP AG 1999

ABAP Statements for Joins

211 SELECT MARA~MATNR MARA~MEINS MARA~SPART MAKT~MAKTX212 MAKT~MATNR MAKT~SPRAS MVKE~KTGRM MVKE~MATNR213 MVKE~PRODH MLAN~MATNR MLAN~TAXM1214 INTO (MARA-MATNR , MARA-MEINS , MARA-SPART , MAKT-MAKTX 215 MAKT-MATNR , MAKT-SPRAS , MVKE-KTGRM , MVKE-MATNR216 MVKE-PRODH , MLAN-MATNR , MLAN-TAXM1 )217 FROM ( MARA 218 INNER JOIN MAKT 219 ON MAKT~MATNR = MARA~MATNR 220 INNER JOIN MVKE 221 ON MVKE~MATNR = MAKT~MATNR 222 INNER JOIN MLAN 223 ON MLAN~MATNR = MVKE~MATNR ) 224 WHERE MARA~MTART IN SP$00001 225 AND MARA~SPART IN SP$00002.

Join Conditions

SelectionConditions

Join Fields

Joined Tables

Internal Table Fields

To find a join statement in the ABAP coding use the whereused list for the first table, in this example MARA, the keyword ‘select’ and search string ‘join’.

n You can identify join statements in the ABAP code by the ABAP command 'inner join'.

n In a join you should always take care that you can access each table in the join statement by a suitable index. The database can use indexes for either the fields specified in the selection conditions or the join conditions. Normally you should use the table as the first table of the join that most limits the number of records. All subsequently accessed tables should have a selective join condition to one of the tables accessed above and an index on the join condition.

n To find the ABAP code for a join, start a where used list for the first table in the statement and the key word ‘select’ and the additional search string ‘join’.

n There also exists the possibility to use outer joins in the ABAP.

SAP AG 1999

Never change an SAP Join without approvaland advisement of SAP


l Run update statistics

l Change the table access sequence

l Change the join conditions to conditions that are supportedby an index

l Create a suitable index for the outer table of the join

n Run update statistics. Check also if an increased precision leads to a better access sequence of the tables.

n Change the table access sequence in the ABAP code

n Change the join conditions to conditions that are supported by an index

n Create a suitable index for the outer table of the join

SAP AG 1999

Summary


• Analyze expensive statements to joins


� Analyze expensive statements to joins

SAP AG 1999

Expensive Statements with a Suitable Access Path

Techical Background

The Shared SQL Area


Update Statistics

Identify Coding


Index Utilization

Cost Evaluation

Creating an Index

Similar Statements

View Processing

Joins


SAP AG 1999

Unit: Suitable Access path

Contents:

l ABAP Navigation

l Nested Selects

l Selects in Loops

Objectives:


l Explain when statements using a suitable access pathmay lead to performance problems

l Check if these statements can be solved technically inthe ABAP code


� Explain when statements processed with a suitable access path may lead to performance problems

� Give recommendations how to avoid nested selects

SAP AG 1999

Expensive Statements Using a Suitable AccessPath

Such Statements cannot be tuned on database level

Reasons why statements using a suitable access path can beexpensive:

l A very high number of records processed per execution

The developer has to check thoroughly if all the records are requiredby the application.

l A very high number of executions

In this case the statement is most often driven by a 'Nested Select' ora 'Loop'. This can also happen if the driven statement isencapsulated in an ABAP modularization unit.The developer has to check if all statements are required to be sentto the database.

n Expensive statements that are using a suitable access path either read a very high number of records per execution or are executed very often. These statements can not be tuned on the database. They must be tuned in the ABAP.

n A high number of records processed per execution indicates that the program is written with a simple logic. Often such programs read many records which are not processed or explicitly discarded in due course. The developer has to check thoroughly if all the records are required by the application.

n Statements with a high number of executions are often driven by a 'Nested Select' or a 'Loop'. This can also happen if the driven statement is encapsulated in an ABAP modularization unit. The developer has to check if all statements are required to be sent to the database.

n Statements that are executed frequently are often identical statements sent to the database. In this case the executions for the statement can be avoided by pulling the statement out of the loop or nested select. It is also possible to read the records into an internal table and read this internal table instead of the database table.

SAP AG 1999

Modularization in ABAP

Conventional

Object-oriented

Created withdefinition call

Program ProgramSUBMIT <prgname>

TransactionCALL TRANSACTION<trancode>

Transactioncode, menu,report tree

End user

local global

SubroutinesABAP EditorFORM ... ENDFORMPERFORM

Function modulesFunction builderFUNCTION ... ENDFUNCTIONCALL FUNCTION (incl. remotely)

Local classesABAP EditorCLASS ... ENDCLASSCREATE OBJECT

Global classesClass builderCLASS ... ENDCLASSCREATE OBJECT

Created withdefinition call

n To read ABAP programs you need to know about the ABAP modularization units. The basic modularization units within an ABAP program are subroutines, function modules, and classes:

� Subroutines

Are defined locally within a program

Are defined using FORM … ENDFORM and called using PERFORM

� Function Modules

Are defined globally in the Function Builder. In R/3 Release 4.0A, SAP delivered 735 function modules that had been released for customer use

Are defined using FUNCTION ... ENDFUNCTION and called using CALL FUNCTION

Can also be called in remote system by remote function calls (RFC)

� Classes

Are defined either locally within a program, or globally in the Class Builder

Are defined using CLASS … ENDCLASS. You create instances of classes using CREATE OBJECT.

n These modularization units have an interface for passing references or values. There is a difference between local data in the modularization unit and the global data of the main program.

SAP AG 1999

Driven SELECT Encapsulated in a Subroutine (FORM)

FORM ... ENDFORM defines asubroutine that is called withPERFORM

n In this example, the driven SELECT statement to table KNA1 is encapsulated in the subroutine READ_KNA1.

SAP AG 1999

Navigation in ABAP Coding: Where-Used/Defined

Where used Where defined

Double-click tonavigate Where-usedto Where-defined and

vice versa

For example:l From FORM to PERFORMl From FUNCTION to CALL

n To find the driver, double -click on READ_KNA1 next to the ABAP key word FORM.

SAP AG 1999

Driven SELECT Encapsulated in Function Module

FUNCTION ... ENDFUNCTION defines afunction module that is called withCALL FUNCTION

n In this example, the driven SELECT statement is encapsulated in the function module Z_KNA1_SINGLE_READ.

n To find the driver, double -click on Z_KNA1_SINGLE_READ, next to the ABAP key word FUNCTION.

SAP AG 1999

Why It May Be Difficult to Find the Driver

l A subroutine is used several times. The Where-used listpoints to several positions in the program. You mustdetermine which PERFORM or CALL FUNCTION called thesubroutine.

l A subroutine is calleddynamically. TheWhere-used listwill not find any calls.

n It can be difficult to find the driver of the expensive select statement if the FORM or FUNCTION is called either very often, or if their calls are dynamic.

n In the example above the call to FORM 'READ_KNA1' is dynamic. It is called via the variable l_form_name which has the value 'READ_KNA1'. However, during the navigation the value of the variable is not determined. Hence, the where used list would not find the call for this form.

n If the FORM or FUNCTION is called very often you have to go to every found location and check if the driver for the select statement is there. Note that there can be more than one level of modularization. For example a FORM can be called within a second FORM routine. You than have to look in the where used list of the second FORM for the driver of the select statement.

SAP AG 1999

Where Resolving Nested SELECTs is notAppropriate

l The driving SELECT is SAP code.

l One of the joined tables is a pool or clustertable

n Database join not possible

n max_blocking_factor does not apply forSELECT FOR ALL ENTRIES on pool andcluster tables

n Do not resolve a nested SELECT if:

� The driving SELECT is SAP code. However, if the driven select is SAP code but the driving select is customer code you should resolve the nested select.

� One of the tables in a nested select is a pool or cluster table. In this case you cannot solve the problem with a JOIN, because pooled and clustered tables are not known to the database and cannot be joined.

SAP AG 1999

Case Study of a Nested Select

A statement to table LIPS had extremely high Disk reads

Total Current Disk Reads/ Buffer Gets/ Records Records/ Bufgets/ SQL Execution Exec Reads Execution Gets Execution processed Execution record Sort

856 1 24,181,760 28,249.7 361,175,136 421,933.6 3,365,453 3,931.6 107.3 0

SELECT * FROM "LIPS"WHERE "MANDT" = :A0 AND "LGNUM" = :A1 AND "KOMKZ" = :A2 #

Table LIPS contains delivery document position informationTotal Rows: 500.000 + 50.000 each day

This statement reads all document positions from LIPS for a storagelocation where the "Indicator for picking control" is set to a certain valueNo Index exists that supports this query

Field Name Short description Distinct values

LGNUM Warehouse number/complex 26KOMKZ Indicator for picking control 2 (Blank and 'C')

10% of total disk reads 3,2 Gbytes readper execution

n In this example a statement to table LIPS reads a lot of records and causes 10% of the total disk reads.

n This statement is expensive because of several reasons:

� It is processed without index support

� It reads many records per execution (4.000 Records per Execution)

� It is executed fairly often (856 times in 2 Days)

n Creating an index for this query would not make sense because the only field that is specified, that is the warehouse complex, is only moderately selective.

n This statement reads every record in LIPS multiple times per day. This indicates that the logic of the program is wrong.

n Per execution the statement reads 421,000 blocks, which is 3,2 Gbytes.

n The customer was live since 10 days and the performance of the program that issued this statement got worse from day to day because table LIPS is growing by about 50.000 entries per day.

SAP AG 1999

Case Study of a Nested Select

Another statement to table VBUP had a very high numberof executions


3,868,347 0 1,158,760 0.3 15,473,388 4.0 8,123,528 2.1 2.0 0

SELECT * FROM "VBUP"WHERE "MANDT" = :A0 AND "VBELN" = :A1 AND "KOSTA" = :A2 AND "LVSTA" = :A3 #

5% of the total user calls

Table VBUP contains status information for sales documents (incl. delivery)

This statement reads the document positions from VBUP for one documentwhere two status are set

Field Name Short description

VBELN Document numberLVSTA Status of warehouse management activitiesKOSTA Picking status

n Another expensive statement was found on table VBUP that was executed very often but had a suitable access path.

SAP AG 1999

ABAP Coding from Customer

87 form start_of_selection. 88 89 select * from lips where lgnum eq p_lgnum 90 and vbeln in s_vbeln 91 and erdat in s_erdat 92 and komkz = con_komkz_c. 93 94 select * from vbup where vbeln eq lips-vbeln 95 and kosta eq 'A' 96 and lvsta eq 'A'. 97 98 lfn-lgnum = lips-lgnum. 99 lfn-vbeln = lips-vbeln. 100 lfn-ernam = lips-ernam. 101 collect lfn. 102 103 endselect. 104 endselect. 105 106 endform.

1) Only those records from LIPS are processed where a record inVBUP has the values KOSTA = 'A' and LVSTA = 'A'

2) No further processing of VBUP records3) This report was executed once for each warehouse (LGNUM)

every few hours

Driving Select on LIPS

Driven Select on VBUP

n The ABAP program that executed the two expensive statement contained a nested select, where the driving select is the expensive statement on table LIPS and the driven select is the expensive statement on table VBUP.

n This ABAP program was executed in background and the selection option for the document number ('vbeln in s_vbeln') and the selection option for the creation date ('erdat in s_erdat') are left blank in the variant for the execution. Most of the documents in LIPS had the status KOMKZ = 'C' which is specified.

n The report is executed every few hours once for each warehouse (LGNUM). Therefore, all records from LIPS are returned by the database for one warehouse where the indicator for picking control is set to 'C'.

n For each of these records a select statement to table VBUP is sent to the database in the nested select. However, only for those records, where a corresponding record exists in table VBUP with the requested status, some fields are saved in the internal table 'lfn'. The records of table VBUP are not processed in the ABAP. Only the status of the records of table LIPS is checked.

SAP AG 1999

Recommended Coding

106 select li~lgnum li~vbeln li~ernam into table lfn 107 from lips as li join vbup as vb 108 on li~vbeln = vb~vbeln and109 li~posnr = vb~posnr 110 where li~lgnum in s_lgnum 111 and li~vbeln in s_vbeln 112 and li~erdat in s_erdat 113 and li~komkz = con_komkz_c 114 and vb~kosta eq 'A' 115 and vb~lvsta eq 'A' 116 and vb~gbsta in ('A', 'B', 'C', ' ') 117 and vb~lfgsa eq ' ' 118 and vb~lfsta eq ' ' 119 and vb~absta eq ' ' 120 and vb~rfgsa eq ' ' 121 and vb~rfsta eq ' '.

1) An index on VBUP is created on fields (GBSTA, LFGSA, ..., LVSTA)2) The JOIN is processed by reading VBUP first, then only the positions for

which processing is required (~10.000) are read from LIPS3) The program is run once for all warehouses (LGNUM) every 4 hours

n Because the records of table VBUP are not processed in the ABAP program they don't need to be sent from the database to the application server. They are only needed in order to decide if the records in table LIPS are to be processed or not. This can be done by using a JOIN or joined view on tables LIPS and VBUP.

n During the investigation of this problem it turned out that the number of records that have to be processed by the program is about 10.000 if the program is run every 4 hours. The selective information if a record has to be processed or not is the status information stored in table VBUP. Therefore, an index was created on VBUP for the status information fields. This JOIN is processed by reading first table VBUP and then table LIPS for the records which have the requested status in table VBUP. Note that only the fields that are required by the program are sent over the network, these are LGNUM, VBELN and ERNAM which are all from table LIPS. No records are returned from table VBUP. This saves sending about 10.000 records across the network.

n The status information of a delivery document record is independent of the warehouse complex. Therefore this program is executed only once for all warehouses every 4 hours.

n In total the performance gain of the program is at least (50 * 26 * 2) = 2600 because the program is processing only 10.000 instead of 500.000 records (factor 50), it is executed only once instead of 26 times every 4 hours (factor 26), and only the records of LIPS instead of the records of LIPS and VBUP are sent across the network (factor 2). A further reduction of the runtime results in sending only the fields required instead of the complete record across the network and by using only index supported accesses to the tables of the JOIN.

SAP AG 1999

Statement Performance After Tuning

Total Current Disk Reads/ Buffer Gets/ Records Records/ Bufgets/ Execution Exec Reads Execution Gets Execution processed Execution record

1 0 18,262 18,262,0 66,170 66,170.0 15,956 15,956.0 4.1

SELECT T_0 . "LGNUM" , T_0 . "VBELN" , T_0 . "ERNAM"FROM "LIPS" T_0 , "VBUP" T_1WHERE ( T_0 . "MANDT" = :A0 AND T_1 . "MANDT" = :A1 AND T_0 . "VBELN" = T_1 . "VBELN" AND T_0 . "POSNR" = T_1 . "POSNR" ) AND ( T_0 . "KOMKZ" = :A4 AND T_1 . "KOSTA" = :A5 AND T_1 . "LVSTA" = :A6 AND T_1 . "GBSTA" IN (:A7 , :A8 , :A9 , :A10 ) AND T_1 . "LFGSA" = :A11 AND T_1 . "LFSTA" = :A12 AND T_1 . "ABSTA" = :A13 AND T_1 . "RFGSA" = :A14 AND T_1 . "RFSTA" = :A15 ) #

The statement processes now 15,956 Records per execution where thenumber of blocks read per record is 4.1, which indicates that the accesspath chosen is suitable. The total block reads from disk and memory arenow moderate.

n The performance of the statement after the changes were made, is shown above. 4.1 Blocks have to be read per record which indicates a suitable access path to the requested records. Furthermore, only those records are requested that have to be processed and the statement is executed only as many times as necessary.

n In comparison to the original statement to table LIPS now 66,120 Blocks are read to process all the records required. Previously 421,000 Blocks had to be read per execution and the statement had to be executed 26 times, that is once for each warehouse complex. Therefore, in total the statement read around 10,000,000 blocks for the processing of all records required on LIPS plus the blocks required to read the data from VBUP.

SAP AG 1999


l Process only those records that are required

l Pull the statement out of the Loop or nested select if thewhere clause of the driven statement does not change

l Process a Join or View instead of the nested select

n The possible optimizations for expensive statements with a suitable access path are in preferential order:

� Process only those records that are required

� Pull the statement out of the Loop or nested select if the WHERE clause of the driven statement does not change

� Process a Join or View instead of a nested select

SAP AG 1999

Summary


• Explain when statements processed with a suitableaccess path may lead to performance problems

• Give recommendations how to avoid nested selects


� Explain when statements processed with a suitable access path may lead to performance problems

� Give recommendations how to avoid expensive nested selects

SAP AG 1999

Appendix

Techical Background

The Shared SQL Area


Update Statistics

Identify Coding


Index Utilization

Cost Evaluation

Creating an Index

Similar Statements

View Processing

Joins


Exercises & Solutions - SQL Cache Analysis for Oracle

Exercises and Solutions for: Identify Coding

Number Exercise

1. Start transaction ZEWB and execute the simulation for this exercise.

There are two expensive statements to table ZVBAK with the field BSTNK specified in the WHERE clause. Find the coding.

Number Solution

1. Unit “Report ZEWB00A Line 13

Report ZEWB00B Line 13 access to projection view ZENT6050

Exercises and Solutions for: Workflow and Reporting

Number Exercise


For the most expensive statement in your system originating from one of your reports fill in the Expensive Statements Access database or Excel Sheet.

2. If you find an expensive statement originating from a standard SAP report use the OSS Call template to open an OSS call and fill in the Expensive statement Access database or Excel sheet.

Exercises and Solutions for: Creating an Index

Number Exercise


Search for the following SELECT statement in the Shared SQL Area:

SELECT ... FROM ZVBAK WHERE MANDT = AND VBTYP = AND TRVOG = AND BSTNK =

1.1 Can an index be created to accelerate the statement?

1.2 What else could be done to accelerate the statement?

Number Solution

1. In the SQL Cache screen, choose “Select Table” and enter “ZVBAK”. Scan through the list until you find the appropriate statement.

1.1 An index with the fields MANDT and BSTNK could accelerate the statement.

1.2 If the field VGBEL or VBELN could be specified in the WHERE clause, index ZVBAK___ZA or the primary index could be used.

Exercises and Solutions for: Similar statements

Number Exercise


Check if there are similar select statements in the Shared SQL Area.

1.1 Which indexes are used for the access path of the statements?

1.2 What could be done to accelerate the statements?

Number Solution

1.

1.1

1.2 Because the OBJNR is not specified, index ZVBAK____ZB cannot be used efficiently. If it would be possible to specify the OBJNR in the queries the statements would be processed efficiently. If this is not possible an index could be created containing the field AUFNR.

Exercises and Solutions for: View Processing Number Exercise


Analyze the expensive statement to view ZVIVEDA in the shared SQL Area.

1.1 Why is this statement expensive?


Number Solution

1.

1.1 The fields that are specified in the where clause are not indexed

1.2 Creating an index on table ZVBAP on field CUOBJ or try to change the code so that a selective field is specified preferably the document number VBELN.

Exercises and Solutions for: Pools and Cluster Number Exercise


Analyze the expensive statements to Pool KAPOL in the shared SQL Area.



2 Analyze the expensive statements to Cluster ZRFBLG in the shared SQL Area.



Number Solution

1.

1.1 Field KAPPL is not specified in the query

1.2 Specify field KAPPL in the query

2

2.1 Field BUKRS is not specified in the query

2.2 Specify field BUKRS in the query

Expensive Statements List

Expensive Statements List – open problems

Problem ID 1

Priority 1

Responsible Mister X

Table ZVBAK

Report / Transaction

TEST

SQL Statement (Copy Statement from Shared SQL Area)

Total Current Disk Reads/ Buffer Gets/ Records Records/ Bufgets/ SQL Execution Exec Reads Execution Gets Execution processed Execution record Sort 1 0 0 0,0 10.042 10.042,0 5 5,0 2.008,4 0 1 0 0 0,0 10.038 10.038,0 5 5,0 2.007,6 0 1 0 597 597,0 10.725 10.725,0 5 5,0 2.145,0 0 SELECT "MANDT" , "VBELN" , "AEDAT" , "AUART" , "AUDAT" , "AUGRU" , "AUTLF" , "BNAME" , "BSARK" , "BSTDK" , "BSTNK" , "BSTZD" , "ERDAT" , "ERNAM" , "ERZET" , "FAKSK" , "FKARA" , "GSBER" FROM "ZVBAK" WHERE "MANDT" = :A0 AND "AUTLF" = :A1 AND "BSTNK" = :A2 #

Problem Description

No suitable Index exists

Action / Solution

Select data from a different table or create index

Comment Developer contacted. He found a way to select data from a different table.

Status Log 1.1.99 open 2.1.99 Developer Mister X contacted 3.1.99 Report changed to be transported 4.1.99 Report transported

Detection Date 01.01.1999

Detected by JR Ewing

Expensive Statements List – solved problems

Priority

Table

Report / Transaction

SQL Statement

(Paste Statement from Shared SQL Area)

Problem Description

Action / Solution

Comment

Status Log

Detection Date

Detected by

SQL Cache Analysis for Oracle - OSS Call Template

OSS call template Use this template to open an OSS call if you found an expensive statement issued from a standard SAP report.

Do not open an OSS call for:

• Transactions SART or SE16, because user input or customizing are normally responsible for slow response times

• If user input is responsible for the expensive statement, such as for a field not specified in a selection screen (example: Matchcodes)

• For sapdb* programs called from your own reports. These programs are logical databases that might be inefficiently used in your coding.

If you think this problem is caused by a wrong decision of the optimizer,

OSS message text Short text:

Expensive SQL Statement on table <...>

Description:

We found an expensive SQL statement on table <...> causing <...>%

of the total database load. The statement is issued in transaction

<...> by the ABAP report <...>.

The last optimizer statistics update was performed on <date> for this table.

SQL statement:

<Expensive SQL statement in easy to read format:

SELECT <...>

FROM <...>

WHERE <...>

AND <...>

AND <...>

...

Access path chosen by the Optimizer:

<EXPLAIN plan>

Indexes defined:

<List of all indexes along with their fields>

Table statistics:

<List of index and table statistics>

The database is up for <...> <days/hours>. During that time the

above statement has been executed <...> times with an average of

<...> Buffer gets/Execution. The total number of Buffer gets

caused by the statement is <...>. The total number of database

reads is <...>.

The oracle parameter DB_FILE_MULTIBLOCK_READ_COUNT is set to <...> and

Note 114716 is implemented.

SAP AG 1999

Performance Monitoring

Section: Performance Monitoring

SAP AG 1999



SAP AG 1999


Contentsl Cost-based optimizerl Memory configurationl Application designl Physical and logical layout

ObjectivesAt the end of this unit, you will be able to:l Identify performance problemsl Refresh the statistics used by the cost-based optimizer

n When you have completed this unit, you will be able to:

� Identify performance problems by monitoring the:

Cost-based optimizer

Memory configuration

Application design

Physical and logical layout

� Refresh the statistics used by the cost-based optimizer

SAP AG 1999

Performance issuesPerformance issues Cost-based optimizerCost-based optimizer

Memory configurationMemory configuration

Application designApplication design

Physical and logical layout

Database Related Performance Issues

n Poor database performance can result from problems with the cost-based optimizer, database memory configuration, application design, or the physical and logical layout.

n This unit focuses on how to use R/3 to monitor and identify these performance problems.

SAP AG 1999

Cost-based optimizerCost-based optimizer

Modifying thestandard

procedure

Modifying thestandard

procedure

Refreshing theobject statisticsRefreshing the

object statistics

Reasons forperformance

problems

Reasons forperformance

problems

Cost-Based Optimizer

n The following section describes:

� What cost-based optimization means

� How to check for performance problems originating from the cost-based optimizer environment

� How to refresh the statistics used by the cost-based optimizer

� How to modify the standard procedure used for refreshing the optimizer statistics

SAP AG 1999

OPTIMIZEROPTIMIZER

Which is the optimal access path?Which is the optimal access path?

Databasetable

ADDR

Index A

Fulltablescan

Index BSELECT * FROM ADDR SELECT * FROM ADDR

WHERE name = 'miller'WHERE name = 'miller'ANDAND pnum pnum = 123 = 123

AND city = 'Houston'AND city = 'Houston'

init<SID>.oraOPTIMIZER_MODE = choose

$$$$

$$$$$$

$$$$$$$$

Possible access pathsPossible access paths

CostsCosts

Oracle Cost-Based Optimizer: Review

n The cost-based optimizer determines the most effective strategy for retrieving database data. The access strategy used depends on the information in the:

� Queried table (or tables, for a view or join)

� Fields specified in the WHERE clause of the SQL statement

� Indexes defined for the tables queried

n The cost-based optimizer computes the cost of several strategies for accessing the tables, and chooses the one that requires the smallest amount of data accesses. To calculate the cost of a strategy, the optimizer requires statistical information about the tables and indexes of the database, such as:

� Number of table or index rows and number of blocks allocated for the object

� Number of distinct values in each column of the table

n The statistical information for a table or index is stored in the Data Dictionary of the database. To collect the statistical information, use the Oracle SQL command analyze table.

SAP AG 1999

SYSTEM

Statisticinformation

l Old statistic information

l No precise statisticinformation

l Incorrect assumptions

l Not using SAP-tools torefresh the statistics

Cost-Based Optimizer Performance Problems

n Performance problems can occur if the cost-based optimizer uses:

� Old or non-existent statistic information

� No precise statistic information

� Incorrect assumptions, such as uniformly distributed data within the object

n You can solve performance problems regarding old, non-existent, or no precise statistic information by:

� Refreshing the object statistics by using the SAP two-phase strategy

� Increasing the precision by modifying the SAP standard procedure

n You can solve performance problems regarding incorrect optimizer assumptions by:

� Using optimizer hints or histograms (as of R/3 Release 4.5A). See SAP Notes 129385 and 130480

� Using the rule-based optimizer access for some tables by modifying the SAP standard procedure

� Changing the application access to the data

n Performance problems can also occur if SAP tools (SAPDBA or CCMS) are not used to refresh the object statistics. Statistic updates performed by non-SAP tools can create severe performance problems if the precision of the update is not set correctly.

SAP AG 1999

Phase 1

SAPDBA determines which objects in thedatabase need refreshing

Object needs to be refreshed if:

The number of rows in the tablefrom last the check

differ from

The actual number of rowsin the table

by more than the threshold

Table E P10 x

Control table(DBSTATC)

Object name

Option

New statisticsneeded

Method

Refreshing the Object Statistics: Phase 1

n To minimize the time and system overhead necessary to refresh the cost-based optimizer statistics, SAP provides the following two-phase strategy:

� Phase 1: SAPDBA determines which database objects need refreshing

� Phase 2: SAPDBA refreshes only the objects determined in phase 1

n The control table DBSTATC stores information about the objects that need to be refreshed. During phase 1, SAPDBA determines which objects need to be refreshed by the following rule:

� If the number of rows in a table found during the last check differs from the actual number of rows by more than a specific threshold number, then the object needs to be refreshed. By default, the threshold number is 10% for small tables and 100% for large tables.

n Rows for objects in the control table DBSTATC that need to be refreshed are either updated or new rows are created, then they are flagged as New statistics needed.

n The four important columns in control table DBSTATC for phase 1 are:

� Object name: The name of the object that needs to be refreshed

� Method: The method used to analyze the object. Either C (compute) or E (estimate). Default value: E (estimate)

� Option: Percentage (P<n>) or number of rows (R<n (* 1000)>) to be analyzed if method E (estimate) is chosen. Default value: 10 percent.

� New statistics needed: A flag that indicates that the statistics need to be refreshed

SAP AG 1999

SAPBDAl Reads entries in the control table

(DBSTATC)l Checks the flag for new statisticsneeded

Phase 2

SAPDBA refreshes the objects

Unflagged

Control table(DBSTATC)

Table E P10 x

Object name

Option

New statisticsneeded

Method

Refreshing the Object Statistics: Phase 2

n Only the statistics of tables that are flagged with new statistics needed in the control table DBSTATC are refreshed during phase 2.

n After the statistics are refreshed, the row remains in the control table DBSTATC, but the column New statistics needed is unflagged.

n The SAP standard procedure does not create statistics for pool and cluster tables and deletes any statistics that already exist. This ensures that the rule -based optimizer access is used for pool and cluster tables. If the Oracle parameter optimizer_mode is set to Choose and no statistics exist for any database object part of an SQL statement, Oracle chooses the rule -based optimizer access instead of the cost-based optimizer access.

SAP AG 1999

Control table DBSTATC

DB object TODO flag …

AUFK x …

EKPO …

KNVK x …

LIPS …

MKPF …

VBUK x …

RESB x …

SAP Two-Phase Strategy

sapdba -analyze DBSTATCO

SAPDBA- x

The statistics for thedatabase objects that are

marked in tableDBSTATC are updated

PHASE 2

sapdba -checkopt PSAP%

SAPDBA- x

Database objects requiring anupdate of optimizer statisticsare determined and marked in

table DBSTATC

PHASE 1

sapdba -statistics all

SAPDBA- xExecute both phases inone step,

once a week

ALTERNATIVELY

n Only up-to-date statistical information can ensure that the Oracle cost-based optimizer chooses the optimal access path. However, gathering optimizer statistics is expensive and reduces system performance.

n We recommend the following two-phase strategy:

� In the first phase, the SAP tools determine which tables require a statistical update. The command sapdba -checkopt PSAP% determines which database objects contain obsolete statistics, and modifies the control table DBSTATC accordingly.

� In the second phase, the statistics of the tables marked TODO in the control table DBSTATC are refreshed using command sapdba -analyze DBSTATCO.

n As of R/3 Release 4.6B, you can use the command sapdba -statistics. This command runs both of the phases in parallel. That is, you do not have to schedule the second phase separately. You should schedule command sapdba -statistics to be run once a week, during periods of low system activity.

SAP AG 1999

Modifying the Standard Procedure

...

New Entries: Details of Created Entries

Object informationDB object BC505TSDB object type 01OwnerDatabase

Default settingsUse 0 Analysis method

Analysis optionActive ADate changed

TODO settingsTODO flag x Customer ü

TODO date

Table view Edit Goto Selection criteria Utilities System Help

Act Short text

A Active (generating statistics)I IgnoreP Positive (active with priority)

U Forced (statistics are constantly updated)R Restrictive (only active for the appl. tab. monitor)

Control flag for analysis

E X

EP30

History Customer

Analysis

n To modify the standard procedure used for refreshing the optimizer statistics, use the CBO Control Panel (transaction DB21).

n There are two ways you can modify the standard procedure:

� Increase the precision of the statistics for a single table (including the index)

� Delete statistics for a single table (including the index)

n To increase the precision, you must enter the name of the DB object, the DB object type (01 for a table), the Type of usage (O for optimizer), Active (A for active or P for active with priority), the Analysis method (E for estimate, C for compute, EH for estimate with histograms or CH for compute with histograms), and the Option (percentage (P<n>) or number of rows (R<n (* 1000)>) to be analyzed if method estimate is chosen).

n For every customer defined entry in the control table DBSTATC, you must select the field Customer.

n If the field Customer is not selected, entries defined as Type of usage = O that are not changed for more than 30 days are removed from control table DBSTATC automatically.

n You can delete statistics for a single table so that the rule -based optimizer is used. Specify the value for Active as I (ignore) if you do not want the object to be analyzed, or specify the value as R (restrictive) to allow the object to be analyzed for space purposes.

n When you modify the standard procedure, select the TODO flag so that the modifications are used the next time the statistics are refreshed.

SAP AG 1999

...

Database Checks

Tablespaces

Tables/Indexes

Database performance: Tables and Indexes- x

Tables and Indexes Monitor

COST-BASED OPTIMIZER Actions- xEdit System Help

Begin of action End of action Fct Object RC 22.03.1999 02:01:22 22.03.1999 02:28:12 aly DBSTATCO 0002 21.03.1999 02:01:17 21.03.1999 02:24:54 opt PSAP% 0001 15.03.1999 02:01:18 15.03.1999 02:21:09 aly DBSTATCO 0000 14.03.1999 02:01:19 14.03.1999 02:27:17 opt PSAP% 0000

Select optionsSelect options SortSort

DBA Logging Monitor

SAP Note Search

Search Criteria:

<table name>

Performance

- x

SAP Notes

Use SAP tools only to refreshUse SAP tools only to refreshR/3 table statisticsR/3 table statistics

Using R/3 to Monitor Performance Problems

n For general performance problems, use the DBA Logging Monitor (transaction DB14) to check whether the statistics are up-to-date.

n If a specific SQL statement causes performance problems, you must check whether:

� The statistics of all the tables accessed by the SQL statement are up-to-date with the correct precision, using the Tables and Indexes Monitor (transaction DB02)

� There is an SAP Note related to the problem

� A statistical update is necessary for any of the tables accessed

� The rule-based optimizer would use a better access path

n If no solution can be found or if you have to switch back to the rule -based optimizer, create a customer message in SAPNet using the component BC-DB-ORA.

n Remember: When you refresh the statistics of R/3 tables, use SAP tools only. SAP tools ensure that the statistical update is performed using the method and option defined for the object in the control table DBSTATC. Statistical updates performed by non-SAP tools can create severe performance problems if the precision of the update is not set correctly.

SAP AG 1999

Memory ConfigurationMemory ConfigurationMemory Configuration

Data bufferData bufferData buffer

Shared poolShared poolShared pool

Memory Configuration

n The following section describes:

� The Oracle memory configuration

� What the data buffer is used for

� How the size of the data buffer is determined

� What the shared pool is used for

� How the size of the shared pool is determined

SAP AG 1999

The data buffer functions as a cache for the database

Database


Shared pool

parsed SQLstatements andexecution path

dc information about objects of the database

Datafiles

... SystemPSAPBTABIPSAPBTABI PSAPSTABD

Memoryarea

Shared SQL Area Row CacheR/3 work process

SELECT * FROM MARA

WHERE ...

Shadowprocess

LogicalLogicalreadsreads

Physical readsPhysical reads

Data Buffer Utilization

n A physical read must go to the disk to access the database data. When a physical read occurs, a copy of the data block is written to the data buffer and then read and analyzed by the shadow process.

n A logical read does not need to go to the disk to access the database data, instead, it reads the data block from the data buffer.

n Accessing the data buffer is 1000 times faster than accessing the disk. To minimize disk access, the data buffer must be tuned.

n When a database update occurs, the data blocks are updated in the data buffer immediately, and written to disk at later time.

SAP AG 1999

ST04: Database performance analysis: Oracle database overview

éé??

Refresh Detailed Analysis Menu

éééé

éééééé

ORACLE Monitor

Data Buffer

Shared Pool

Calls

SizeQuality

256,00097

Log buffer

ReadsPhysical reads

writes

389,42210,809

2,325

kb%

...

.........

≥ 94

Identifying the Data Buffer Hit Ratio

n The hit ratio (Quality) of a database is defined as the percentage of data blocks accessed (Reads) compared with the total number of data blocks read from disk (Physical reads). The Reads are the sum of the logical and physical reads.

n The hit ratio is displayed in the Database Performance Monitor (transaction ST04), and should be at least 94%.

n Since the hit ratio is poor in the first few hours after startup, you should only evaluate the hit ratio after your system has been up for some time. As a general rule, wait until the number of Reads exceeds 20,000,000.

n Before you increase the size of the database buffer, check for poorly qualified SQL statements. Problems in the application can cause poor hit ratios in even the largest of database buffers, for example, in the case of inefficient SQL coding, many blocks may be read into memory unnecessarily.

SAP AG 1999

DB_BLOCK_BUFFERSparameter in init<sid>.ora

Use the Operating System Monitorto analyze the

operating system paging statisticsbefore and after increasing

DB_BLOCK_BUFFERS

Hour Pages in Pages out Paged in Paged out /h /h [Kb/h] [Kb/h]

13 11 0 44 0 12 227 0 908 0 11 2,162 2,542 8,648 10,168 10 626 464 2,504 1,856 9 22 0 88 0 8 22 0 88 0 7 42 64 168 256 6 74 64 296 256 5 3,034 2,363 12,136 9,452 4 1,648 1,106 6,592 4,424 3 22 0 88 0 2 29 0 116 0 1 89 0 356 0 0 83 0 332 0 23 33 0 132 0 22 1,063 0 4,252 0 21 22 0 88 0 20 8,157 2,435 32,628 9,740 19 22 0 88 0 18 128 0 512 0 17 4,046 3,056 16,184 12,224 16 53 0 212 0 15 281 0 1,124 0 14 39 0 156 0

Increasing the Size of the Data Buffer

n If the hit ratio is lower than 94%, consider increasing the database buffer. But before you increase the size of the database buffer, check for poorly qualified SQL statements. Problems in the application can cause poor hit ratios in even the largest of database buffers, for example, in the case of inefficient SQL coding, many blocks may be read into memory unnecessarily.

n To increase the size of your database buffer, change the init<sid>.ora parameter DB_BLOCK_BUFFERS (which is specified in units of blocks). This parameter specifies the number of database blocks buffered in memory. Note: When you increase this parameter, you reduce the memory available to other processes in the system, which may cause OS paging and/or swapping to occur.

n To check the OS paging use the OS Monitor (transaction ST06 ) and choose Detailed Analysis → Previous Hours: Memory.

n Each hardware platform has an upper limit on the total amount of shared memory that can be allocated. The sum of the fixed and variable portions (data buffer cache, shared pool, and log buffer) of the System Global Area cannot exceed this amount.

n To monitor changes in any Oracle initialization parameters, use transaction DB03.

SAP AG 1999

The shared pool caches parsed SQL statements andData Dictionary information from the database

Database


Shared poolShared pool

Data files

Memoryarea

Shared SQL Area Row Cache

...

R/3 work process

Shadowprocess

SELECT * FROM MARA

WHERE ... User callsUser calls

Recursive callsRecursive calls

PSAPBTABI PSAPSTABD System

Parsed SQLstatements andexecution path

dc information about objects of the database

Identifying Usage of the Shared Pool

n The shared pool consists of the:

� Shared SQL Area, where parsed SQL statements are cached for shared access to all shadow processes

� Row Cache, which holds the Oracle Data Dictionary information, including the cost-based optimizer statistics. Note: With the cost-based optimizer, the Row Cache will have substantially more information to hold than with the rule -based optimizer

n A user call refers to a shadow process accessing the Shared SQL Area for parsed SQL statements.

n A recursive call refers to the Row Cache making a physical read to load Oracle Data Dictionary objects from the system tablespace.

SAP AG 1999

Identifying the Efficiency of the Shared Pool

ST04: Database performance analysis: Oracle database overview

éé??

Refresh

éééé

éééééé

ORACLE Monitor

Data Buffer

Shared Pool

Calls

Size kbDD-cache quality %...

User calls......

128,00097...96

18,131......

Log buffer

Recursive calls......

2,338......

User calls / Rec. calls

...pinratio %

...7.75

...≥ 2

0.03 reloads/pins % ≤0.04

> 80

≥ 95

Previous DaysDetailed Analysis Menu

n The ratio of user calls to recursive calls should be at least 2 to 1.

n The Data Dictionary cache quality should also be greater than 80%.

n The pin ratio should be larger or equal 95%.

n The ratio of reloads to pins should be at maximum 0.04.

n Since these ratios are poor in the first few hours after startup, you should only evaluate them after your system has been up for some time. As a general rule, you should wait until the number of Reads exceeds 20,000,000.

n Note: Creating new optimizer statistics is heavily based on recursive calls. You should therefore first make sure that creating statistics is not responsible for the values. To do this, check the history by choosing the Previous days button.

SAP AG 1999

SHARED_POOL_SIZEparameter in init<sid>.ora

Examine the operating system paging statistics using theOperating System Monitor

before and after increasing SHARED_POOL_SIZE

Hour Pages in Pages out Paged in Paged out /h /h [Kb/h] [Kb/h]

13 11 0 44 0 12 227 0 908 0 11 2,162 2,542 8,648 10,168 10 626 464 2,504 1,856 9 22 0 88 0 8 22 0 88 0 7 42 64 168 256 6 74 64 296 256 5 3,034 2,363 12,136 9,452 4 1,648 1,106 6,592 4,424 3 22 0 88 0 2 29 0 116 0 1 89 0 356 0 0 83 0 332 0 23 33 0 132 0 22 1,063 0 4,252 0 21 22 0 88 0 20 8,157 2,435 32,628 9,740 19 22 0 88 0 18 128 0 512 0 17 4,046 3,056 16,184 12,224 16 53 0 212 0 15 281 0 1,124 0 14 39 0 156 0

Increasing the Size of the Shared Pool

n If any of these ratios are lower then the threshold mentioned, increase the size of the shared pool.

n There are no specific Oracle parameters for increasing the size of the Row Cache or the size of the Shared SQL Area. By increasing the area for the entire shared pool, you also increase the amount of space available for both areas.

n To increase the entire shared pool, change the init.ora parameter SHARED_POOL_SIZE (which is specified in units of bytes). Do not cause excessive operating system paging by using too much memory.

SAP AG 1999

Application designApplication designApplication design

Lockwait situationsLockwait situationsLockwait situations

Expensive SQL statementsExpensive SQL statementsExpensive SQL statements

Poorly qualified statementsPoorly qualified statementsPoorly qualified statements

Application Design

n This section describes how to identify the following application design problems:

� Lockwait situations

� Expensive SQL statements

� Poorly qualified SQL statements

SAP AG 1999

WorkProcess 1

WorkProcess 3

WorkProcess 4

WorkProcess 2

AcquiresMARA Lock

A long periodof processing CommitUpdate

MARA

Working...WAITING!Update

MARARequests

MARA LockAcquires

MARA LockCommit

Working...WAITING!UpdateMARA

RequestsMARA Lock

AcquiresMARA Lock

WAITING ...UpdateMARA

RequestsMARA Lock

MARA Lockedby

WP 1 WP 2 WP 3

When a Lockwait Situation Occurs

n A lockwait situation occurs when numerous work processes request a lock on the same object. In order for the RDBMS to maintain transactional consistency, the object is locked by the process that requested it first.

n If a user starts a logical unit of work and updates an object, such as the most often used material number of the company, then all other users who want to update the same material must wait until the first user has committed the changes before they can get the record.

n A user holding a lock will occupy an R/3 work process. Other users trying to apply the same lock must wait and at the same time occupy their own R/3 work process. As the number of lockwaits increases, fewer and fewer R/3 user requests can be processed by available R/3 work processes. In a worst-case scenario, the lock holders and waiters would equal the number of R/3 work processes, and a small number of users could cause the entire R/3 system to “freeze”.

SAP AG 1999

Using the Exclusive Lockwait Monitor

Object Holder (Oracle-SID, -SPID; Client-Host, -PID) Level Time (s)

Waiters (Oracle-SID, -SPID; Client-Host, -PID) Row-ID

T100 9 , 9.687 ; hs5821 , 9.681 1 21 41

30.07.1999 14:45:58 TC1 hs5821 Exclusive session-lock situations

Level Time (s)

14 , 27730 ; hs5821 , 16.492 1 6

Double-click

Primary keyvalues

Oracle Session-ID 0Locked object MONILocked Row (ID) AAAA8mAALAAAAQkAAFValues:

RELID DBSRTFD TC1 snapshot1SRTF2

30.07.1999 14:59:00 TC1 hs5821Exclusive session-lock situations

n Use the Exclusive Lockwait Monitor (transaction DB01) to identify exclusive lockwait situations and to display information about the:

� Lock holder

� Number of lock waiters

� Primary key locked

SAP AG 1999

Reducing Exclusive Lockwaits

l Redesign the application to reduce the locking period by:

n Increasing the commit frequency in the application

n Not allowing a single process to hold a lock for a long period

n Locking the object as late as possible

l Adjust the job scheduling cycle so that lock situations donot occur

n If exclusive lockwait situations occur because a user holds the lock too long, analyze the application and determine whether:

� More commits can be safely built into the application

� The lock period can be shortened by acquiring the lock later

n If exclusive lockwait situations occur because many users want to access the same record in high-volume processing, you can schedule the jobs to be performed at different times.

SAP AG 1999

Total Reads / Buffer Gets ≥ 5 %

Identifying Expensive SQL Statements (1)

BufferGets

1,272,090 1,204,074 704,074 443,694

Shared SQL Area

6.1%

Sortedby

Database Performance Monitor

éé??

Refresh Detailed Analysis Menu

éééé

éééééé

ORACLE Monitor

Shared Pool Log buffer

ReadsPhysical reads

writes

20,853,934581,809

42,325... ...

Data Buffer

... ... Reads / User calls 7.8 < 20

n An indicator for having expensive SQL statements in the system is the ratio of Reads to User calls. This value should be less than 20. If this value is greater than 20, you must check the SQL statements in detail.

n Expensive SQL statements have a high number of Buffer gets compared to Total reads.

n If the ratio of Buffer gets to Total reads is greater or equal than 5%, the SQL statement is expensive.

n Once a statement has been identified as expensive in the Shared SQL Area, run an Explain plan on the statement.

n After you have run an Explain plan on an expensive SQL statement:

� Check the cost-based optimizer settings for the tables being used in the SQL statement

� Create a customer message in SAPNet if the statement is identified as part of the SAP code

� Redesign the application if the statement is identified as part of the customer code

� Check if the statement is poorly qualified

SAP AG 1999

ST04 - Detailed Analysis - SQL Request - Database Performance: Shared SQL

éé??

Sort Reset Since Reset Since DB Start

éééé

éééééé

Detail stats.

25.05.199916:54:36 Shared Cursor Cache (last reset at 25.05.1999 15:17:55 )


7,476 0 12,468 1.7 1,272,090 170.2 6,336 0.8 200.8 0 81 0 149,601 1,846.9 7,709,223 95,175.6 783 9.7 9,845.8 0 5 0 22,594 4,518.8 1,204,074 240,814.8 142,221 28,444.2 8.5 0 3 0 10,604 3,534.7 443,694 147,898.0 0 0.0 443,694.0 1

Number ofexecutions

Sorted by

Number ofbuffer getsper record

Identifying Expensive SQL Statements (2)

n To identify expensive SQL statements, use the Database Performance Monitor (transaction ST04) and choose Detailed analysis → SQL request.

n Analyze the Shared SQL Area statistics collected since the last startup:

� The column Total Execution refers to the number of times the SQL statement was executed

� The column Buffer Gets refers to the total number of buffers accessed by the statement

� The column Bufgets/record refers to the average number of buffers accessed per record retrieved

n Sort by the column Buffer Gets. Check for statements that are executed very often with a low number of Bufgets/record. These statements should be analyzed.

n To identify which program an SQL statement belongs to, you can check:

� The WHERE-USED list in the Data Dictionary (transaction SE12)

� The SystemWide Work Process Overview (transaction SM66) and the Oracle Session Monitor (transaction ST04 → Detailed analysis)

SAP AG 1999

Total Disk Reads/ Buffer Gets/ SQL

Execution Reads Execution Gets Execution Text

Total Disk Reads/ Buffer Gets/ SQL

Execution Reads Execution Gets Execution Text

57 554 9.7 5,200,089 91,229.6 SELECT "PGMID" , "OBJECT" , " DEVCL

Specify hint:For testing only!

No change to the SAP code

SQL Statement

SELECT *

FROM "TADIR" WHERE

"PGMID" = :A0 AND "OBJECT" = :A1 AND "DEVCLASS" = :A2 AND ROWNUM <= :A3 #

Execution Plan

SELECT STATEMENT ( Estimated Costs = 95 ) 5 COUNT STOPKEY 5 TABLE ACCESS BY INDEX ROWID TADIR

INDEX RANGE SCAN TADIR^1

Explain with hintExplain with hint Optimizer traceOptimizer trace

Specify Hint

Enter the hint without the commentsigns /*+ ... */ (for the syntax ofhints see the ORACLE tuning guide)

" CancelContinue

Running an Explain Plan

Double-click

n To run an Explain plan for an expensive SQL statement, double -click the appropriate line in the Shared SQL Area and choose Explain .

n The output of the Explain plan shows how the cost-based optimizer has decided to access the data.

n The option Explain with hint allows you to check a possible change in the access path using Oracle hints, such as checking the access path chosen by the rule -based optimizer (Hint RULE). When you use a hint, only a new explain plan is created, the actual SQL statement is not changed.

n For further information about Oracle hints, refer to Oracle documentation.

n Here are the definitions of some Explain plan output for SQL statements:

� INDEX RANGE SCAN: The database retrieves a number of records using an index to limit the result set before going to the data pages

� INDEX UNIQUE SCAN: The database retrieves a single row from an unique index

� INDEX NAME: The name of the index that the database uses for retrieving data

� CONCATENATION: The database unites a set of rows retrieved for the query

� NESTED LOOPS : The database joins one table to a second table, using the information found in the first table to check second table, and builds a result set out of both tables

� TABLE ACCESS FULL: The database retrieves all rows from the table to build the result set

� SORT: The database sorts the data before returning it

SAP AG 1999

Total Buffer Bufgets/

Execution Gets Record

Total Buffer Bufgets/

Execution Gets Record

28 1,333,625 666,812.5

Table MARAFull table scan Index scan

SELECT * FROM MARA WHERE MATERIAL = 10001

Index MARA~O

Poorly Qualified SQL Statements

n Poorly qualified SQL statements do not use an index correctly to access the data. If an index is used correctly, data access is more efficient.

n To identify poorly qualified SQL statements, check the Shared SQL Area for expensive statements with a high number of Bufgets/record. Poorly qualified statements usually occur if:

� No index is associated with the table being accessed

� A secondary index is required for the query being performed

� The incorrect index is being utilized

� An index is being used although a full table scan would be more effective, for example, in the case of small tables or a high number of records retrieved

� The index being used is defined incorrectly

n If a statement is identified as a poorly qualified SQL statement because of an SAP report, create a customer message in SAPNet.

n Note: Do not change the standard R/3 index design. This is considered an “expert” tuning measure. Contact SAP for support.

SAP AG 1999

SQL Explain Plan

No index being used

Full Table Scan

Execution Plan

SELECT STATEMENT ( Estimated Costs = 59 ) TABLE ACCESS FULL MARA

Table MARA

Last statistics date 05.06.1998Number of rows 73,181Number of blocks allocated 610Number of empty blocks 9Average space 1,133Chain count 0Average row length 56

UNIQUE Index MARA_____0

Column Name #Distinct

MANDT 1 MATNR 224,405

NONUNIQUE Index MARA___L

Column Name #Distinct

MANDT 1 MATKL 72

Continue Show statistics

Show Indexes of Table MARA

Analyzing Poorly Qualified SQL Statements

n To check if an index is being used to access data, run an Explain Plan on the SQL statement.

n To display the information about the index structure or the structure of the table and all indexes, double -click the index or table name. Information is also displayed about the statistics that the cost-based optimizer used to create the access, and the date the statistics for this table was last refreshed.

SAP AG 1999

Physical and logical layout I/O contention

Checkpoint not complete

Rollback statement problems

Fragmented indexes

Physical and Logical Layout

n This section describes the following physical and logical layout problems:

� I/O contention

� Checkpoint not complete (for online redo log files)

� ORA-1555 Snapshot too old (for rollback segments)

� Fragmented indexes

SAP AG 1999

Occurs when numerous shadow processes and thedatabase writer access the same disk at the same time

ShadowprocessShadow

processShadowprocessShadow

processShadowprocess

PSAPBTABD

PSAPSTABD PSAPPOOLD

PSAPUSER1D PSAPBTABI

PSAPSTABD

DatabasewriterDBWR

READ WRITE

I/O Contention

n I/O contention refers to the high I/O wait times for processes accessing the database. When numerous Oracle shadow processes and the database writer access the same disk at the same time, I/O contention is likely to occur.

n I/O contention occurs if:

� The application design is inefficient, due to expensive, unnecessary, or poorly qualified SQL statements

� The I/O is not evenly distributed across many disks

� The disks are not fast enough to handle the high I/O activity

� Heavily accessed tables or indexes are not distributed or striped across many disks

� The hardware configuration is incorrect (for example, many disks and not enough controllers)

n I/O contention is often caused by application design problems, therefore, check this first.

SAP AG 1999

I/O per path

Check for:

Average read times > 20ms

Deviations from the median value > 20%

11.08.1999 17:35:22 TC1 hs5821Statistics of physical accesses on Oracle database

Blk Reads Avg(ms) Avg(ms)

85,751 127,381 26,962,690 354 333,635 93,629 178,322 248,200 22,226

Filename

/oracle/TC1/sapdata1/oracle/TC1/sapdata10/oracle/TC1/sapdata2 /oracle/TC1/sapdata3 /oracle/TC1/sapdata4 /oracle/TC1/sapdata5 /oracle/TC1/sapdata6 /oracle/TC1/sapdata7/oracle/TC1/sapdata8/oracle/TC1/sapdata9

Reads

20,346 8,358 4,277,300 96 133,641 89,761 83,173 243,981 3,478 13,764

Writes

12,056 16 3,868 43 65,120 1,060 506 221 46 30 181,332

3 1 3 5 3 4 3 5 2 1

Blk Writes

12,056 16 3,868 43 493,064 1,060 506 221 46 30

7 6 6 15 5 6 5 5 6 3

Sorted by

Identifying I/O Contention in the Database

n To identify I/O contention in the database, use the Database Performance Monitor (transaction ST04) and choose Detailed analysis → File system requests → I/O per path .

n The I/O per path allows us to identify the sapdata mount points where I/O contention is occurring.

n Check the Average ms columns for block reads (BLK Reads) and block writes (BLK writes), and use these values to identify “hot disks”.

n If the total number of reads and writes is relatively low, there is no I/O contention.

n If the total number of reads and writes is high, check if the average read time is higher than 20 ms.

n You can also identify "hot disks" by checking for values that deviate by more than 20% from the median value of the average read or write times.

n Note: Due to the various hardware configurations and disk speeds, the actual values can differ significantly from system to system. Contact your hardware vendor for specific information.

n Because Oracle writes data blocks asynchronously, the average write time is not important. However,the average write time becomes important if Oracle is not able to handle the volume anymore. You can identify this situation by checking the write complete waits and the free buffer waits in view v$system_event.

SAP AG 1999

Total per device

Statistics of physical accesses on Oracle database

Filename Reads Writes Blk ReadsAvgms Blk Writes Avgms

docud_1 loadi_1 poold_1 poold_2 protd_1 roll_1

/oracle/TC1/sapdata1

es40bd_2


ddicd_2 docui_1 system_1


protd_2


btabd_1 clui_1

303 29 14,914 2,113 1,294 1,693

20,346

8,358

8,358

2,113 1,695 1,273,492

1,277,300

96

96

190

11 11 44 13 673 11,304

12,056

16

16

11 11 3,846

3,868

43

43

795 11

1,796 29 62,372 14,982 4,879 1,693

85,751

127,381

127,381

10,403 1,695 950,592

962,690

354

354

40,590 190

2 5 3 2 3 1

3

1

1

2 7 0

3

5

5

2 8

11 11 44 13 673 11,304

12,056

16

16

11 11 3,846

3,868

43

43

795 11

0 0 14 3 15 12

7

6

6

0 0 19

6

15

15

15 0

Solving the I/O Contention Problem

n To solve the problem of I/O contention, you can:

� Distribute the I/O evenly on the disks available

� Distribute the I/O evenly on more disk (then would be necessary to hold the database files)

� Purchase faster disks

� Move “hot spot” tables or indexes to their own tablespaces on their own disks (may be striped)

n To identify the tablespace and the data file that has a bottleneck, you can break down the total I/O requests per file system by choosing Total per device. The tablespace and the data file can then be moved to another physical disk in the system.

n Different hardware platforms may have bottlenecks in disk controller ports, motherboards, and back planes. Refer to your hardware vendor for I/O distribution guidelines.

SAP AG 1999

Online redolog files

2. DBRW writes data to disks

3. Redo log writerwrites to the onlineredo log in parallel

4. Last online redo log file is full but the checkpoint is not completed

Data files

Data buffer

Redo log buffer

saptrace/background

alert_<SID>.log

Checkpoint notcomplete

1. Online redo log file switch occurs

Checkpoint not Complete

n How the error Checkpoint not Complete occurs:

1. An online redo log switch occurs and the check point process tags the online redo log group being written to until the checkpoint triggered by the online redo log switch is complete.

2. A checkpoint is triggered. That means, dirty pages from the data buffer are being written to disk by the database writer (DBWR).

3. Transactions are still occurring and committing, so data is written from the redo log buffer to the online redo logs in parallel to the checkpoint.

4. Eventually, after some log switches, all the online redo logs are in use before the checkpoint triggered has finished. No data can be flushed from the redo log buffer to an online redo log file because all online redo log files are necessary for instance recovery.

n The Oracle RDBMS automatically handles this situation, no further changes are processed until the checkpoint is complete and the online redo log file can be overwritten.

n The message “Checkpoint not complete” is written to the Oracle alert log alert_<SID>.log. This problem generally occurs during high database activity and during peak hours.

n If the message appears frequently, increase the number of online redo log groups. Increasing the size of the online redo log files is not recommended, unless the time between the two log switches is less than three minutes.

n Note: This problem should only be of concern if it occurs frequently.

SAP AG 1999

Reading queryPers-No. Salary

1......... 2350

2......... 4362

8......... 3040

4......... 5827

9......... 4160

Table

Rows changed and committed

Rollback Segments

Transaction T1 (Update)

Transaction T2 (Select)

Commit

4320

5770

3010

4119

Rollback segment

Overwrite or

shrink

ORA1555

n The rollback segments are used by the Oracle RDBMS to:

� Save before images of uncommitted transactions

� Provide read consistency during the runtime of a query

n If rows of a table are modified, the before images of the data are copied to a rollback segment. After the commit, this information is no longer needed, and the rollback segment is freed by the process.

n If a table is modified between the time a query is issued and when the records are delivered by the query, the data is read from the rollback segments. However, rollback segments are overwritten or shrunk in a cycle and are not locked for queries. If a query is not finished until the rollback segment is overwritten or shrunk, the query does not receive the data.

n When this occurs, Oracle issues error 1555 Snapshot too old, and the query is aborted. A short dump then occurs, and an entry in the R/3 System log is generated.

SAP AG 1999

Solving Rollback Segment Problems


2. Long processing time between two data fetches

Application server

Large number ofbuffers used on the

database server


Application server

High processingtime on the

application server

Snapdump

ORA1555

ORA1555

1. Expensive query

n There are two main reasons why this error occurs:

� Expensive queries

� Long processing time between the fetches

n Because a SELECT does not normally read all the data at one time, long processing times between the fetches can cause error ORA-1555, even if the statement is not expensive. The next fetch is processed on the database when it is requested. Therefore, even if the statement is not expensive, the time it takes until the statement is finished is long.

n To avoid error ORA-1555:

� Decrease the runtime of the statement, by tuning the statement causing the Snapshot too old

� Decrease the processing time between two fetches of an SQL statement

� Schedule the reports and the updates at different times

� If none of these tuning methods are successful, increase the number or the size of rollback segments (preferably, increase the number)

n Rollback data files have the highest amount of write activity in the database. To reduce bottlenecks in the rollback segments, distribute the I/O for the data file evenly.

SAP AG 1999

...... ............ ...... ...... ......

...... ......

IndexIndex

............

...... ......

......

TableTable

... ... ... ...

2

1

3 4 5 6 7 8 9

Index with alow fill level

Many buffer gets perexecution although thecorrect index is used

Fragmented Indexes

n Fragmented indexes have a low fill level.

n Indexes can become fragmented:

� After data has been archived

� After many records have been deleted

� In highly dynamic tables

n An index that is fragmented consists of empty blocks or branch and leaf pages with only a few valid entries. If an index is fragmented, a query that scans a range in the index must read many index blocks. This affects the performance of the entire database, since data or index blocks of other tables are swapped out of the buffer by the newly read index blocks that contain only a few records.

n In this example, 9 index blocks and 5 data blocks are read in order to retrieve five rows of the table.

SAP AG 1999

Use the Tables and Indexes Monitor

Fill level

Tables and Indexes: Detailed Analysis of T100^0R/3 xDB Analysis Extents Table Columns Detailed Analysis History

14.05.1999 11:36:20 Analysis of B*-tree Data from INDEX_STATS Storage summary Performance summary Levels................ 3 Gets per access........ 4 Blocks allocated...... 1,315 Distinct keys.......... 517,152 Avail in tree.....byte 10,477,868 Most repeated key...... 1 Used in tree......byte 9,856,219 Rows per key........... 1 ...................% 95 without del. rows..% 94 B*-tree branch blocks B*-tree leaf blocks Branch blocks......... 4 Leaf blocks........... 1,309 entries........ 1,308 entries........... 517,152 size.......byte 19,406 size..........byte 9,836,813 size/block.byte 8,012 size/block....byte 7,980 deleted........... 6 del. size.....byte 114

Analyze Index Validate Structure >Storage QualityValidate Structure >Dialog

BackgroundDialog

Identifying a Fragmented Index

n To analyze the fill level of an index, use the Tables and Indexes Monitor (transaction DB02) and choose Detailed analysis.

n Indexes that have a fill level less than 50% should be reorganized if they are frequently accessed.

n To create or refresh the index statistics, choose Analyze index → Validate structure → Dialog or Background.

During the runtime of the Validate structure, the related table remains locked for changes.

n For further information about detecting and recreating fragmented indexes refer to the article about Reorganization in SAP TechNet.

SAP AG 1999

Unit Summary


l Define a strategy for refreshing the statistics used bythe cost-based optimizer

l Identify performance problems caused by the:

n Cost-based optimizer

n Memory configuration

n Application design

n Physical and logical layout

SAP AG 1999


l SAP TechNet

n Communication → SAP TechNet→ DB Admin.Oracle

l SAPNet

n Information → SAP Solutions → SAP R/3 →Basis Technology → System Management →CCMS - Computer Center ManagementSystem → Database Management

l SAP Notes 109034, 127715, 128221

basis week4 (1)

Education