im107 sap netweaver bi accelerator - new features

77
IM107 SAP NetWeaver BI Accelerator: New Features - Including High Availability, Statistics, Reorg Algorithm Josh Djupstrom, SAP NetWeaver RIG SAP US

Upload: mark-anthony-palma

Post on 02-Apr-2015

139 views

Category:

Documents


6 download

TRANSCRIPT

Page 1: IM107 SAP NetWeaver BI Accelerator - New Features

IM107 SAP NetWeaver BI Accelerator: New Features - Including High Availability, Statistics, Reorg Algorithm

Josh Djupstrom, SAP NetWeaver RIG SAP US

Page 2: IM107 SAP NetWeaver BI Accelerator - New Features

© SAP 2008 / SAP TechEd 08 / IM107 Page 2© SAP 2008 / SAP TechEd 08 / IM107 Page 2

Disclaimer

This presentation outlines our general product direction and should not be relied on in making a purchase decision. This presentation is not subject to your license agreement or any other agreement with SAP. SAP has no obligation to pursue any course of business outlined in this presentation or to develop or release any functionality mentioned in this presentation. This presentation and SAP's strategy and possible future developments are subject to change and may be changed by SAP at any time for any reason without notice. This document is provided without a warranty of any kind, either express or implied, including but not limited to, the implied warranties of merchantability, fitness for a particular purpose, or non-infringement. SAP assumes no responsibility for errors or omissions in this document, except if such damages were caused by SAP intentionally or grossly negligent.

Page 3: IM107 SAP NetWeaver BI Accelerator - New Features

© SAP 2008 / SAP TechEd 08 / IM107 Page 3© SAP 2008 / SAP TechEd 08 / IM107 Page 3

1. HA - Definition Of Terms In General 1.1. High Availability1.2. Downtime1.3. Disaster Recovery1.4. Data Recovery

2. HA - Understand The SAP Solutions2.1. High Availability2.2. Downtime2.3. Disaster Tolerance2.4. Backup And Recovery

3. HA - Solution Details3.1. Disaster Tolerance3.2. Backup And Recovery

Agenda

Page 4: IM107 SAP NetWeaver BI Accelerator - New Features

© SAP 2008 / SAP TechEd 08 / IM107 Page 4© SAP 2008 / SAP TechEd 08 / IM107 Page 4

4. HA - Outlook4.1. Current Status4.2. Documentation

5. Demo – Backup And Recovery6. BIA - Reorg

6.1. Reorg Basics6.2. Locality vs. Parallelism6.3 New functionalities

7. Demo – BIA - Reorg8. Sparse Compression9. Package Wise Read

Agenda

Page 5: IM107 SAP NetWeaver BI Accelerator - New Features

© SAP 2008 / SAP TechEd 08 / IM107 Page 5© SAP 2008 / SAP TechEd 08 / IM107 Page 5

1. HA - Definition Of Terms In General 1.1. High Availability1.2. Downtime1.3. Disaster Recovery1.4. Data Recovery

2. HA - Understand The SAP Solutions2.1. High Availability2.2. Downtime2.3. Disaster Tolerance2.4. Backup And Recovery

3. HA - Solution Details3.1. Disaster Tolerance3.2. Backup And Recovery

Agenda

Page 6: IM107 SAP NetWeaver BI Accelerator - New Features

© SAP 2008 / SAP TechEd 08 / IM107 Page 6© SAP 2008 / SAP TechEd 08 / IM107 Page 6

1. HA - Definition Of Terms In General

1.1. High Availability

High availability is a system design protocol and associated implementation that ensures a certain absolute degree of operational continuity during a given measurement period.

Availability refers to the ability of the user community to access the system, whether to submit new work, update or alter existing work, or collect the results of previous work. If a user cannot access the system, it is said to be unavailable. Generally, the term downtime is used to refer to periods when a system is unavailable.

Availability is usually expressed as a percentage of uptime in a given year. Common values are:

99.9% ≡ 43.8 minutes/month or 8.76 hours/year ("three nines") 99.99% ≡ 4.38 minutes/month or 52.6 minutes/year ("four nines") 99.999% ≡ 0.44 minutes/month or 5.26 minutes/year ("five nines")

Page 7: IM107 SAP NetWeaver BI Accelerator - New Features

© SAP 2008 / SAP TechEd 08 / IM107 Page 7© SAP 2008 / SAP TechEd 08 / IM107 Page 7

1. HA - Definition Of Terms In General

1.2. Downtime

Downtime or outage refers to a period of time or a percentage of a time span that a system is unavailable or offline. This is usually a result of the system failing to function because of an unplanned event, or because of routine maintenance.

A planned outage is the result of a planned activity by the system owner and/or by a service provider. Such activities can include changes or upgrades, and they are often scheduled as maintenance windows.

Unplanned downtime may be the result of a software bug, human error, equipment failure, malfunction, high bit error rate, power failure, overload due to exceeding the channel capacity, a cascading failure, etc.

Page 8: IM107 SAP NetWeaver BI Accelerator - New Features

© SAP 2008 / SAP TechEd 08 / IM107 Page 8© SAP 2008 / SAP TechEd 08 / IM107 Page 8

1. HA - Definition Of Terms In General

1.3. Disaster Recovery

Disaster recovery is the process, policies and procedures of restoring operations critical to the resumption of business, including regaining access to data (records, hardware, software, etc.), communications (incoming, outgoing, toll-free, fax, etc.), workspace, and other business processes after a natural or human-induced disaster.

Page 9: IM107 SAP NetWeaver BI Accelerator - New Features

© SAP 2008 / SAP TechEd 08 / IM107 Page 9© SAP 2008 / SAP TechEd 08 / IM107 Page 9

1. HA - Definition Of Terms In General

1.4. Data Recovery

Data recovery is the process of salvaging data from damaged, failed, corrupted, or inaccessible secondary storage media when it cannot be accessed normally. Often the data are being salvaged from storage media formats such as hard disk drives, storage tapes, CDs, DVDs, RAID, and other electronics. Recovery may be required due to physical damage to the storage device or logical damage to the file system that prevents it from being mounted by the host operating system.

Page 10: IM107 SAP NetWeaver BI Accelerator - New Features

© SAP 2008 / SAP TechEd 08 / IM107 Page 10© SAP 2008 / SAP TechEd 08 / IM107 Page 10

1. HA - Definition Of Terms In General1.1. High Availability1.2. Downtime1.3. Disaster Recovery1.4. Data Recovery

2. HA - Understand The SAP Solutions2.1. High Availability2.2. Downtime2.3. Disaster Tolerance2.4. Backup And Recovery

3. HA - Solution Details3.1. Disaster Tolerance3.2. Backup And Recovery

Agenda

Page 11: IM107 SAP NetWeaver BI Accelerator - New Features

© SAP 2008 / SAP TechEd 08 / IM107 Page 11© SAP 2008 / SAP TechEd 08 / IM107 Page 11

2. HA - Understand The SAP Solutions

2.1. High Availability @ SAP NetWeaver BI AcceleratorThe purpose of this lecture is to illustrate the capabilities of SAP NetWeaver BI Accelerator (BIA) from a software perspectiveSome facts

High available BIA solutions by SAP minimize or even avoid downtimesThere is no offer of tailor-made solutions by SAP (development due to individual requirements)For technical guidance contact your SAP consultant, HW supplier or the SAP Competence CenterHigh availability is not part of the BI Accelerator appliance (implementation on project basis)The HW partner and customer are responsible for high availability of the storage and the network infrastructureThe HW partner is responsible for the sizing and configuration of a highly-available landscapeSAP provides the concepts and the software

Page 12: IM107 SAP NetWeaver BI Accelerator - New Features

© SAP 2008 / SAP TechEd 08 / IM107 Page 12© SAP 2008 / SAP TechEd 08 / IM107 Page 12

2. HA - Understand The SAP Solutions

2.1. High Availability @ SAP NetWeaver BI AcceleratorTopical Delimitation

A disaster tolerant solution is needed to cover catastrophes such as power-off or hardware damageA disaster tolerant solution is used to guarantee special Service Level Agreements A disaster tolerant solution is not a solution for the event of data loss or data corruption Backup and recovery complements a disaster tolerant solution– If the BIA data is lost or corrupt – If the SAP NetWeaver BI database needs to be restored

Page 13: IM107 SAP NetWeaver BI Accelerator - New Features

© SAP 2008 / SAP TechEd 08 / IM107 Page 13© SAP 2008 / SAP TechEd 08 / IM107 Page 13

2. HA - Understand The SAP Solutions

2.2. Downtime @ SAP NetWeaver BI AcceleratorUnscheduled downtime

Crashes of certain processes on the bladesFailure of one or more hardware components

Scheduled downtimeHW maintenance of single blades

SolutionConcept of BIA backup blade (one or more backup blades that take over the load of the blades that are not available in the event of failure)Different backup modes available

Page 14: IM107 SAP NetWeaver BI Accelerator - New Features

© SAP 2008 / SAP TechEd 08 / IM107 Page 14© SAP 2008 / SAP TechEd 08 / IM107 Page 14

2. HA - Understand The SAP Solutions

2.3. Disaster Tolerance @ SAP NetWeaver BI AcceleratorA disaster tolerant system is able to work after catastrophic events like

Outage of all bladesOutage of complete local storageFailure of whole data center

Page 15: IM107 SAP NetWeaver BI Accelerator - New Features

© SAP 2008 / SAP TechEd 08 / IM107 Page 15© SAP 2008 / SAP TechEd 08 / IM107 Page 15

2. HA - Understand The SAP Solutions

2.4. Backup And Recovery @ SAP NetWeaver BI AcceleratorStrategy needed to minimize downtime and to recover as much data as possibleThe BI Accelerator backup is independent of the SAP NetWeaver BI database backupRecovery takes place in two stages

Import of snapshot on BI and BIABI delta computation is triggeredSynchronization is achieved via BI system only with re-indexing of data that has been changed after the sync point was written

Page 16: IM107 SAP NetWeaver BI Accelerator - New Features

© SAP 2008 / SAP TechEd 08 / IM107 Page 16© SAP 2008 / SAP TechEd 08 / IM107 Page 16

1. HA - Definition Of Terms In General1.1. High Availability1.2. Downtime1.3. Disaster Recovery1.4. Data Recovery

2. HA - Understand The SAP Solutions2.1. High Availability2.2. Downtime2.3. Disaster Tolerance2.4. Backup And Recovery

3. HA - Solution Details3.1. Disaster Tolerance3.2. Backup And Recovery

Agenda

Page 17: IM107 SAP NetWeaver BI Accelerator - New Features

© SAP 2008 / SAP TechEd 08 / IM107 Page 17© SAP 2008 / SAP TechEd 08 / IM107 Page 17

3. HA - Solutions Details

3.1. Disaster Tolerance @ SAP NetWeaver BI AcceleratorThe HP solution (software component view*)

SAP NetWeaver BI

BI Accelerator BI Accelerator

DC1 DC2

File System File System

Storage

switch over

* Disaster Tolerant Solution for BIA with HP regardless of the availability of the SAP NetWeaver BI system

StorageRead-only mode

Synchronous-write

Page 18: IM107 SAP NetWeaver BI Accelerator - New Features

© SAP 2008 / SAP TechEd 08 / IM107 Page 18© SAP 2008 / SAP TechEd 08 / IM107 Page 18

3. HA - Solutions Details

3.1. Disaster Tolerance @ SAP NetWeaver BI AcceleratorThe HP solution

BIA High Availability Disaster Switch-Over Solution The setup consists of two independent BIA installationsOnly one side is active (There is no phase in the switch-over process where both sides are active)Each BIA installation itself has a configuration with backup blades (in shared mode)– In case one blade fails inside one BIA installation there is no need to switch– Switch-over triggered manually (at present there is no self-monitoring available)BIA failure due to a disaster– In DC1: stop BI Accelerator, then unmount file system– Switch direction of storage replication (HP continuous access)– In DC2: mount file system, then start BI Accelerator

Page 19: IM107 SAP NetWeaver BI Accelerator - New Features

© SAP 2008 / SAP TechEd 08 / IM107 Page 19© SAP 2008 / SAP TechEd 08 / IM107 Page 19

3. HA - Solutions Details

3.1. Disaster Tolerance @ SAP NetWeaver BI AcceleratorThe HP solution

Data written by the active BIA is permanently mirrored by file system firmware used for data storing– Data storage of BIA is continuously mirrored from the active to the other data center – Synchronous writing is a prerequisite– Dedicated SAN network between the BIA storage systems of both data centers required– Solution relies on the HP data replication software ‘Continuous Access’ (CA) For details on the SAP software part of the solution see attachment of SAP note 1124514

Page 20: IM107 SAP NetWeaver BI Accelerator - New Features

© SAP 2008 / SAP TechEd 08 / IM107 Page 20© SAP 2008 / SAP TechEd 08 / IM107 Page 20

3. HA - Solutions Details

3.1. Disaster Tolerance @ SAP NetWeaver BI AcceleratorThe HP solution

Configuration with two identical installations of the SAP NetWeaver BI system– Attached to each BI system is an own BIA landscape– BI and BIA of one data center run productively– BIA of other data center is powered off as “cold standby”– BI system installations can be coupled in a cluster installation with virtual host name and

virtual IP address– While switching over, host name and IP address are moved from source host to the mirror host

– BI data is stored on a storage system with synchronous data replication

Page 21: IM107 SAP NetWeaver BI Accelerator - New Features

© SAP 2008 / SAP TechEd 08 / IM107 Page 21© SAP 2008 / SAP TechEd 08 / IM107 Page 21

3. HA - Solutions Details

3.1. Disaster Tolerance @ SAP NetWeaver BI AcceleratorPrerequisites for the HP solution

Installation of TREX BIA Revision 45 or higher on SLES Linux only, SLES 10 SP1 at the time of writingInstallation and configuration of Continuous Access (CA) software from Hewlett-Packard for synchronous data mirroring on HP StorageWorks EVABIA switchover requires that the BIA installation in both data centers is done with the same SID, instance number and user id (the user number of Linux)An auxiliary script named /opt/oem_bia/device_rescan.sh provided by HP for this DT solution, supporting the activation of the mirrored storage devices (EVA); it has to be installed on all hosts of both BIA landscapes.An integration of BIA switchover into a cluster management software with automatic switching over, or with switching over driven by scripts, activated by an administrator, should enforce, that the BI system comes up before the BIA is switched over. If the switchover is done fully manually, the guidance documentation for the administrator should also conduct to this starting order.The CA software has to be configured to run in the “Failsafe Mode”. A mode-changing into “not failsafe mode” has to be visible in some monitoring environment, so that resetting to “failsafe mode” cannot be omitted.

Page 22: IM107 SAP NetWeaver BI Accelerator - New Features

© SAP 2008 / SAP TechEd 08 / IM107 Page 22© SAP 2008 / SAP TechEd 08 / IM107 Page 22

3. HA - Solutions Details

3.1. Disaster Tolerance @ SAP NetWeaver BI AcceleratorDisadvantages of the HP solution

Manual or script-based switch-over necessaryTwo separate landscapes to maintain

Page 23: IM107 SAP NetWeaver BI Accelerator - New Features

© SAP 2008 / SAP TechEd 08 / IM107 Page 23© SAP 2008 / SAP TechEd 08 / IM107 Page 23

3. HA - Solutions Details

3.1. Disaster Tolerance @ SAP NetWeaver BI AcceleratorThe IBM solution (software component view*)

SAP NetWeaver BI

BI Accelerator

DC1 DC2

File System

Storage

BI Accelerator

Storage

* View of the Disaster Tolerant Solution for BIA with IBM regardless of the availability of the SAP NetWeaver BI system

Page 24: IM107 SAP NetWeaver BI Accelerator - New Features

© SAP 2008 / SAP TechEd 08 / IM107 Page 24© SAP 2008 / SAP TechEd 08 / IM107 Page 24

3. HA - Solutions Details

3.1. Disaster Tolerance @ SAP NetWeaver BI AcceleratorThe IBM solution

Integration into the infrastructure at customer siteRecommendation for a completely redundant infrastructureProtection against failure of bladesProtection against loss of the complete blade centerProtection against loss of primary storageBundling of SAP BIA failover capabilities with GPFS data availabilityThe setup comprises one BIA installation– One side consists of active master blades – Second side consists of active backup blades running in standby modeIn case of loss of a blade, a failover to the dedicated backup blade takes place to ensure application availability

Page 25: IM107 SAP NetWeaver BI Accelerator - New Features

© SAP 2008 / SAP TechEd 08 / IM107 Page 25© SAP 2008 / SAP TechEd 08 / IM107 Page 25

3. HA - Solutions Details

3.1. Disaster Tolerance @ SAP NetWeaver BI AcceleratorThe IBM solution

Due to GPFS data replication, continuous data availability without disruption or manual intervention providedClustering of BI system and virtualization to SAP BI Accelerator through usage of virtual host names and IP addresses

Page 26: IM107 SAP NetWeaver BI Accelerator - New Features

© SAP 2008 / SAP TechEd 08 / IM107 Page 26© SAP 2008 / SAP TechEd 08 / IM107 Page 26

3. HA - Solutions Details

3.1. Disaster Tolerance @ SAP NetWeaver BI AcceleratorPrerequisites for the IBM solution

Need for access of all blades from both sides belonging to the BIA cluster to a sufficient network connection to the BI systemAll blades on both sides configured in one single IP networkSufficient network bandwidth between the two blade center chassisFor further details on the IBM solution see SAP note 1178661

Page 27: IM107 SAP NetWeaver BI Accelerator - New Features

© SAP 2008 / SAP TechEd 08 / IM107 Page 27© SAP 2008 / SAP TechEd 08 / IM107 Page 27

3. HA - Solutions Details

3.1. Disaster Tolerance @ SAP NetWeaver BI AcceleratorLimitations of the IBM solution

SAN and IP network latency should be low between the data centers, e.g. 20-30 km are known to workIP network bandwidth between the blade centers should approximately match the cumulative bandwidth of half the blades of one sideWhen master takes back control all services are released on the backup except for the RFC server services restart of RFC server on backup blade necessary Index creation while all masters are down index parts not reassigned to the master side; when backup fails error: index not available workaround: reorganize landscape If an indexing procedure is running and a master blade goes down, the indexing procedure has to be restarted (task taken over by backup blade)

Page 28: IM107 SAP NetWeaver BI Accelerator - New Features

© SAP 2008 / SAP TechEd 08 / IM107 Page 28© SAP 2008 / SAP TechEd 08 / IM107 Page 28

3. HA - Solutions Details

3.1. Disaster Tolerance @ SAP NetWeaver BI AcceleratorCustomer experience with the IBM solution

Test cases– Initial indexing – Remove one blade on master side– Remove quorum blade on master side– Remove storage of master blade center– Power off blade rack of master blade center– Insert blades back and power on blade rack of master blade center

Rollup (failure in phases fill, prepare optimize, commit)Change run (failure in phases fill, prepare optimize, commit)Disaster Tolerance and Database Fallback of Queries

Page 29: IM107 SAP NetWeaver BI Accelerator - New Features

© SAP 2008 / SAP TechEd 08 / IM107 Page 29© SAP 2008 / SAP TechEd 08 / IM107 Page 29

3. HA - Solutions Details

3.1. Disaster Tolerance @ SAP NetWeaver BI AcceleratorCustomer experience with the IBM solution

Behavior in failure situations as expected– System reliability/stability for querying in failure situations– Immediate rescheduling of adjustment processes like e.g. rollup possible when masters

fail– Master blades take back over when available again automatically– Failure situations transparent to the BI/BIA administrator

Page 30: IM107 SAP NetWeaver BI Accelerator - New Features

© SAP 2008 / SAP TechEd 08 / IM107 Page 30© SAP 2008 / SAP TechEd 08 / IM107 Page 30

3. HA - Solutions Details

3.2. Backup And Recovery @ SAP NetWeaver BI AcceleratorReasons for loss of data

Human influences: Unintentional deletion of an index in the BIA appliance, stolen systemsNatural hazards: Failure due to fire, terrorism, natural disastersSoftware errors: Corrupt files, index destroyedHardware errors: Hard drive crash, power failureViruses: Infected system data may be destroyed

ScenariosSevere problem with the BI Accelerator that requires a recovery (fix problem with the BIA; start recovery) There are two data centers each having its own BIA (HP disaster tolerant solution e.g.); there is a problem with BIA1 that requires a switch to BIA2 (start recovery on BIA2)After database recovery of the BI system start BIA recovery

Page 31: IM107 SAP NetWeaver BI Accelerator - New Features

© SAP 2008 / SAP TechEd 08 / IM107 Page 31© SAP 2008 / SAP TechEd 08 / IM107 Page 31

3. HA - Solutions Details

3.2. Backup And Recovery @ SAP NetWeaver BI AcceleratorPrerequisites and enablement

SAP note 1077439 BIA Backup and Recovery SAP NetWeaver BI SPS15 (or SAP NetWeaver BI SPS14 and note 1178792)Further notes for backup and recovery

– 1172481 BIA B&R Admin-parameter 'B&R_ENABLED' – 1225748 BIA Schedule Deletion of Backup and Recovery Snapshot Further relevant BIA notes

– 1146849 BIA metadata change: Termination _CHECK_SID_LOCK – 1179307 BIA - Incorrect data after re-indexing in rollup– 1223284 BIA index set to inactive 'A' after InfoCube activation – 1228470 Retry when index is not found (SAP Basis note)BIA revision 45 – but always latest revision recommendedBIA note 1160182 (see attachments)

Page 32: IM107 SAP NetWeaver BI Accelerator - New Features

© SAP 2008 / SAP TechEd 08 / IM107 Page 32© SAP 2008 / SAP TechEd 08 / IM107 Page 32

3. HA - Solutions Details

3.2. Backup And Recovery @ SAP NetWeaver BI AcceleratorMethods

Default method: Backup by Index Export – Copy of whole index data 1:1 to a file system location designated by the configuration

of the TREX backup server– Usage of index export feature of the TREX index server– Method is not vendor-specific and easy to enableHardware-vendor-specific method: Backup by Storage Snapshots

– Fast method– Saves storage space– Uses the native snapshot feature of the underlying storage system or the block device– The BIA is set to read-only mode before starting a snapshotBackup data is persisted either on the BI Accelerator storage or on any other mounted external storageThe integration into the existing backup infrastructure of the customer is part of a customer project

Page 33: IM107 SAP NetWeaver BI Accelerator - New Features

© SAP 2008 / SAP TechEd 08 / IM107 Page 33© SAP 2008 / SAP TechEd 08 / IM107 Page 33

3. HA - Solutions Details

3.2. Backup And Recovery @ SAP NetWeaver BI AcceleratorSteps to backup I

Definition of time windows– Time with no or few business user activities– Indexing free time (All indexing processes are automatically on hold and wait until the

backup is completed)– Dependence on how often and how much data has been changedEnsure data consistency and availability of BIA– Check InfoCube data (rollup status up to date)– Check master data (attribute change runs executed)– Run BIA connection availability– Check results in BIA monitor and regularly scheduled check sets– Run predefined consistency checks (data consistency check center and RSTT see

documentation)

Page 34: IM107 SAP NetWeaver BI Accelerator - New Features

© SAP 2008 / SAP TechEd 08 / IM107 Page 34© SAP 2008 / SAP TechEd 08 / IM107 Page 34

3. HA - Solutions Details

3.2. Backup And Recovery @ SAP NetWeaver BI AcceleratorSteps to backup II

Creation of snapshot (ID and timestamp determined and stored in the SAP NetWeaver BI system)Checking the application log (Details and snapshot log)Deletion of old snapshots on a regular basis (by using program RSDDTZA_BNR_DELETE_SNAPSHOTS with a suitable variant)

Page 35: IM107 SAP NetWeaver BI Accelerator - New Features

© SAP 2008 / SAP TechEd 08 / IM107 Page 35© SAP 2008 / SAP TechEd 08 / IM107 Page 35

3. HA - Solutions Details

3.2. Backup And Recovery @ SAP NetWeaver BI AcceleratorSteps to recover

Simulation of recovery (helps to decide whether recovery or complete re-index is faster)Start of recovery (choose relevant snapshot ID)– Indexes are set to ‘inactive’ with technical status ‘B’– Reset landscape is executed (deletion of all indexes)– Import of snapshot is executed (index create, rollback, show index) – Recovery takes place (adjustment of all changes since the last snapshot; comparison of

index status and timestamps)– No data has been changed no action– Index of F-table (if not compressed or compresses before snapshot creation) request

information (adjusted)– Index of Dimension tables complete rebuild except for package dimension (adjusted)– Index of S/X/Y tables complete rebuild (delta mechanism in test phase)

Checking of job log and application logChecking of data consistency– Run predefined consistency checks (data consistency check center and RSTT see

documentation)

Page 36: IM107 SAP NetWeaver BI Accelerator - New Features

© SAP 2008 / SAP TechEd 08 / IM107 Page 36© SAP 2008 / SAP TechEd 08 / IM107 Page 36

3. HA - Solutions Details

3.2. Backup And Recovery @ SAP NetWeaver BI AcceleratorFactors that influence performance of backup and recovery

The age of the snapshotPerformance of the I/O subsystem on which the backups are stored

Finding out if recovery or initial indexing is fasterIf the snapshot is up to date, recovery should be fasterIn simulation mode the number of indexes and the recovery modes are shown

Pilot Customer experienceBackup by Index Export: – 33GB of index data was exported in 15 minutes– It took 30 minutes to recover (with only minor delta creations)– Initial Indexing took 3-4 hours

Page 37: IM107 SAP NetWeaver BI Accelerator - New Features

© SAP 2008 / SAP TechEd 08 / IM107 Page 37© SAP 2008 / SAP TechEd 08 / IM107 Page 37

3. HA - Solutions Details

3.2. Backup And Recovery @ SAP NetWeaver BI AcceleratorLimitations of the SAP solution

If the BIA installation binaries are corrupt a reinstallation is necessary Recovery procedure recovers up to the current SAP NetWeaver BI status (indexes deleted in the BI system will not be recovered by the recovery)Depending on the age of the snapshot and the changes that have taken place, a complete re-index of all InfoCubes could be fasterWhole BI indexes are recovered (it is not possible to recover only parts)Functionality can only be used for 1:1 relationships between BI and BIA (reset landscape executed on whole BIA installation)

Page 38: IM107 SAP NetWeaver BI Accelerator - New Features

© SAP 2008 / SAP TechEd 08 / IM107 Page 38© SAP 2008 / SAP TechEd 08 / IM107 Page 38

4. HA - Outlook4.1. Current Status4.2. Documentation

5. Demo – Backup And Recovery6. BIA - Reorg

6.1. Reorg Basics6.2. Locality vs. Parallelism6.3 New functionalities

7. Demo – BIA - Reorg8. Sparse Compression9. Package Wise Read

Agenda

Page 39: IM107 SAP NetWeaver BI Accelerator - New Features

© SAP 2008 / SAP TechEd 08 / IM107 Page 39© SAP 2008 / SAP TechEd 08 / IM107 Page 39

4. HA - Outlook

4.1. Current Status @ SAP NetWeaver BI AcceleratorDisaster Tolerance

HP on OCFS file system (address requests to your hardware partner contact)IBM on GPFS file system (address requests to your hardware partner contact)

Backup And RecoveryOpen to pilot customers see note 1077439 Method ‘Backup by Storage Snapshots’– IBM: solution available, GPFS-snapshots, block-copy management on file system level;

address requests to your hardware partner contact (competence center: [email protected])

– HP: in progress– FujitsuSiemens: solution available; address requests to your hardware partner contact

(competence center: [email protected])– Sun: not yet defined

Page 40: IM107 SAP NetWeaver BI Accelerator - New Features

© SAP 2008 / SAP TechEd 08 / IM107 Page 40© SAP 2008 / SAP TechEd 08 / IM107 Page 40

4. HA - Outlook

4.2. Documentation @ SAP NetWeaver BI AcceleratorDowntime/Backup Modes

Technical Operations Manual (pages 58-64)SAP Developer Network (SDN) www.sdn.sap.com/irj/sdn/bia →

Documentation about SAP NetWeaver BI Accelerator →

Technical Operations Manual for BI Accelerator 7.0 Disaster Tolerance

Technical Operations Manual (pages 60-62)SAP Developer Network (SDN) www.sdn.sap.com/irj/sdn/bia →

Documentation about SAP NetWeaver BI Accelerator →

Technical Operations Manual for BI Accelerator 7.0 SAP notes

– HP: 1124514– IBM: 1178661

Page 41: IM107 SAP NetWeaver BI Accelerator - New Features

© SAP 2008 / SAP TechEd 08 / IM107 Page 41© SAP 2008 / SAP TechEd 08 / IM107 Page 41

4. HA - Outlook

4.2. Documentation @ SAP NetWeaver BI AcceleratorBackup And Recovery

How to Guide SAP NetWeaver BI Accelerator Backup and Recovery SAP Developer Network (SDN) www.sdn.sap.com/irj/sdn/bia →

Documentation about SAP NetWeaver BI Accelerator →

How to Perform a SAP NetWeaver BI Accelerator Backup and Recovery

Technical Operations Manual (pages 40-42)SAP Developer Network (SDN) www.sdn.sap.com/irj/sdn/bia →

Documentation about SAP NetWeaver BI Accelerator →

Technical Operations Manual for BI Accelerator 7.0

You will find current information and detailed procedures for the BIA backup and restore by BIA index snapshots in the SAP Note 1160182 BIA 7.0: Backup & Restore for SAP NetWeaver BI Accelerator

Check of Data Consistency (Blog)https://www.sdn.sap.com/irj/sdn/weblogs?blog=/pub/wlg/10081

Page 42: IM107 SAP NetWeaver BI Accelerator - New Features

© SAP 2008 / SAP TechEd 08 / IM107 Page 42© SAP 2008 / SAP TechEd 08 / IM107 Page 42

4. HA - Outlook4.1. Current Status4.2. Documentation

5. Demo – Backup And Recovery6. BIA - Reorg

6.1. Reorg Basics6.2. Locality vs. Parallelism6.3 New functionalities

7. Demo – BIA - Reorg8. Sparse Compression9. Package Wise Read

Agenda

Page 43: IM107 SAP NetWeaver BI Accelerator - New Features

© SAP 2008 / SAP TechEd 08 / IM107 Page 43© SAP 2008 / SAP TechEd 08 / IM107 Page 43

5. Demo - Backup And Recovery

Where to enable?BI: RSDDTREXADMIN (report RSDDTREX_ADMIN_MAINTAIN)BIA: – Backup by Index Export (see note 1160182):

– TREXBackupServer.ini In section [backup] enter the path

– Backup by Storage Snapshots (see note 1160182):- Contact your hardware partner

Where to find the monitor on BI and BIA side?Snapshot creationRecovery SimulationRecovery executionLogs

Page 44: IM107 SAP NetWeaver BI Accelerator - New Features

© SAP 2008 / SAP TechEd 08 / IM107 Page 44© SAP 2008 / SAP TechEd 08 / IM107 Page 44

DEMO

Page 45: IM107 SAP NetWeaver BI Accelerator - New Features

© SAP 2008 / SAP TechEd 08 / IM107 Page 45

5. Demo - Backup And Recovery

Enablement in BI:

Enablement in BIA (Method Backup by Index Export):

Page 46: IM107 SAP NetWeaver BI Accelerator - New Features

© SAP 2008 / SAP TechEd 08 / IM107 Page 46

5. Demo - Backup And Recovery

Monitor in BI: RSDDBIAMON2 BI Accelerator Maintenance Functions Backup and Recovery

Export

Import

Page 47: IM107 SAP NetWeaver BI Accelerator - New Features

© SAP 2008 / SAP TechEd 08 / IM107 Page 47

5. Demo - Backup And Recovery

Monitor in BI: RSDDBIAMON2 BI Accelerator Maintenance Functions Backup and Recovery

Simulation

Continue to view the details.

Page 48: IM107 SAP NetWeaver BI Accelerator - New Features

© SAP 2008 / SAP TechEd 08 / IM107 Page 48

5. Demo - Backup And Recovery

Monitor in BIA

Page 49: IM107 SAP NetWeaver BI Accelerator - New Features

© SAP 2008 / SAP TechEd 08 / IM107 Page 49

4. HA - Outlook4.1. Current Status4.2. Documentation

5. Demo – Backup And Recovery6. BIA - Reorg

6.1. Reorg Basics6.2. Locality vs. Parallelism6.3 New functionalities

7. Demo – BIA - Reorg8. Sparse Compression9. Package Wise Read

Agenda

Page 50: IM107 SAP NetWeaver BI Accelerator - New Features

© SAP 2008 / SAP TechEd 08 / IM107 Page 50© SAP 2008 / SAP TechEd 08 / IM107 Page 50© SAP 2008 / SAP TechEd 08 / IM107 Page 50

6. Reorg: Why is it Even Necessary?

6.1 Reorg Basics

As existing BIA indexes are updated and new indexes are added, the distribution of indexes over the available hosts in the BIA landscape can become suboptimal

Status of necessary Reorg can be foundin tab Reorg of the TREXAdmintoolin transaction RSSDBIAMON2in E-Mails from the TREXAlertserver

Page 51: IM107 SAP NetWeaver BI Accelerator - New Features

© SAP 2008 / SAP TechEd 08 / IM107 Page 51

Reorg: BI Accelerator Horizontal Partitioning

Fact TableFact Table

Physical Index Server

Index Part 1

Physical Index Server

Index Part 2

Physical Index Server

Index Part N

Logical Index Server

Index Metadata

Storage

TREX Services

Index

Metadata

Part 1

Part 2

...

Part N

Fact Table

Part 1

Part 2

...

Part N

LargeTables

Page 52: IM107 SAP NetWeaver BI Accelerator - New Features

© SAP 2008 / SAP TechEd 08 / IM107 Page 52

Landscape Reorganization Locality vs. Parallelism

6.2 Locality vs. Parallelism

Two conflicting objectives

Locality avoids communication -> put everything in the same place

Parallelism speeds up computation -> distribute as much as possible

There are some limiting factors, e.g.:Available memory per host/index serverAvailable CPU power per host/index server…

Find some compromise between both objectives

Page 53: IM107 SAP NetWeaver BI Accelerator - New Features

© SAP 2008 / SAP TechEd 08 / IM107 Page 53

Landscape Reorganization Locality vs. Parallelism

Aggregation often is the dominating computational factor in queriesAggregation is done on F-Tables and F-Tables are usually largei.e. F-Tables consume the most memory, aggregation consumes the most CPUReturns small results does not consume much communication bandwidth

Joins and Search on S-,X-,Y-,D-TablesThey consume less memory, are often fast compared to aggregationE.g. Join suffers from communication overhead if tables are distributed

SolutionFor aggregation prefer parallelism to localityFor Joins and Search prefer locality to parallelism

Page 54: IM107 SAP NetWeaver BI Accelerator - New Features

© SAP 2008 / SAP TechEd 08 / IM107 Page 54

Landscape Reorganization

Find a distribution of indexes to the landscape such thatQuery performance is highUtilization of all available resources is highNo hotspots arise and amount of communication is low

Partition algorithm:Divide indexes into small and large

Split large indexes into as many parts as there are cores in the landscape

Distribute the large parts evenly to the landscape

Distribute the small parts by the number of small indexes on each hostFor OLAP and Join indexes calculate Join clustersPut all indexes in a cluster on the same host

Page 55: IM107 SAP NetWeaver BI Accelerator - New Features

© SAP 2008 / SAP TechEd 08 / IM107 Page 55

Landscape Reorganization New Functionalities

6.3 New Functionalities

Reorganize only selected indexesReorg algorithm gets a filter list containing the indexes to be reorganizedOLAP, Join, Logical indexes get expanded to include all their childrenMark all other indexes as fixed and calculate the resource usage per host with the fixed indexes only

Initial ReorgUses the functionality aboveReorg called after create index– split and distribute cube before it gets filled with data– only estimates from BI as index size still unknown

Page 56: IM107 SAP NetWeaver BI Accelerator - New Features

© SAP 2008 / SAP TechEd 08 / IM107 Page 56

Landscape Reorganization Heterogeneous landscapes

Customers want to extend their appliancesoriginal blades are no longer available, only the latest generation can be addedresults in a heterogeneous landscape

Previous Reorg assumed hosts with equal computational resourcesduring reorganization all hosts get equal loadBUT hosts with twice the computational resources are likely to be able to handle twice the load

smaller blades become a bottleneck and prevent the bigger ones from exposing their true potential

Example for the following landscape:2x 2*3.8GHz + HT and 8GB 1x 8*2.4GHz and 32GB1x 4*3.0GHz and 16GB

Page 57: IM107 SAP NetWeaver BI Accelerator - New Features

© SAP 2008 / SAP TechEd 08 / IM107 Page 57

Landscape Reorganization Heterogeneous landscapes

To each host assign a scale factorThe hosts with the fewest memory get factor 1.0For all other hosts scale factor = memory(host)/memory(smallestHost)

Resource capacity (num CPU) of each host is determined by new value Reorg capacityReorg capacity is determined by memory of each host

assumption: each core has 4GB availableE.g. host with 8 Cores and 16GB memory has capacity 4 (not 8)

There is also an overall Reorg multiplier new equivalent to parts per host

Page 58: IM107 SAP NetWeaver BI Accelerator - New Features

© SAP 2008 / SAP TechEd 08 / IM107 Page 58

4. HA - Outlook4.1. Current Status4.2. Documentation

5. Demo – Backup And Recovery6. BIA - Reorg

6.1. Reorg Basics6.2. Locality vs. Parallelism6.3 New functionalities

7. Demo – BIA - Reorg8. Sparse Compression9. Package Wise Read

Agenda

Page 59: IM107 SAP NetWeaver BI Accelerator - New Features

© SAP 2008 / SAP TechEd 08 / IM107 Page 59

DEMO

Page 60: IM107 SAP NetWeaver BI Accelerator - New Features

© SAP 2008 / SAP TechEd 08 / IM107 Page 60

Landscape Reorganization Heterogeneous landscapes

Page 61: IM107 SAP NetWeaver BI Accelerator - New Features

© SAP 2008 / SAP TechEd 08 / IM107 Page 61

Landscape Reorganization Heterogeneous landscapes

0

5

10

15

20

25

30

3.8GHz+HT,8GB

3.8GHz+HT,8GB

4*3.0GHz,16GB

8*2.4GHz,32GB

mem

ory

axis

0

10

20

30

40

50

60

3.8GHz+HT,8GB

3.8GHz+HT,8GB

4*3.0GHz,16GB

8*2.4GHz,32GB

mem

ory

axis

Before After

Page 62: IM107 SAP NetWeaver BI Accelerator - New Features

© SAP 2008 / SAP TechEd 08 / IM107 Page 62

Landscape Reorganization Heterogeneous landscapes

Page 63: IM107 SAP NetWeaver BI Accelerator - New Features

© SAP 2008 / SAP TechEd 08 / IM107 Page 63

Landscape Reorganization Heterogeneous landscapes

Page 64: IM107 SAP NetWeaver BI Accelerator - New Features

© SAP 2008 / SAP TechEd 08 / IM107 Page 64

Landscape Reorganization Future development: slave indexes

TCP/IP

Logical Index

Query Executor

QueryMediator

Plan Generator

Plan Optimizer

Physical Index

Planoperation

Planoperation

Planoperation

Query Executor

Physical Index

Planoperation

Planoperation

Planoperation

Query Executor

Index copy

Index copy

Index copy

Page 65: IM107 SAP NetWeaver BI Accelerator - New Features

© SAP 2008 / SAP TechEd 08 / IM107 Page 65

4. HA - Outlook4.1. Current Status4.2. Documentation

5. Demo – Backup And Recovery6. BIA - Reorg

6.1. Reorg Basics6.2. Locality vs. Parallelism6.3 New functionalities

7. Demo – BIA - Reorg8. Sparse Compression9. Package Wise Read

Agenda

Page 66: IM107 SAP NetWeaver BI Accelerator - New Features

© SAP 2008 / SAP TechEd 08 / IM107 Page 66

Sparse Compression Motivation

Data volumes are increasing significantlyMain memory is expensive and a big cost factor

Further compression of data is neededFour sparse attribute types implemented Using lightweight compression schemes to guarantee good

performance

Page 67: IM107 SAP NetWeaver BI Accelerator - New Features

© SAP 2008 / SAP TechEd 08 / IM107 Page 67

Sparse Compression Compression Types

Page 68: IM107 SAP NetWeaver BI Accelerator - New Features

© SAP 2008 / SAP TechEd 08 / IM107 Page 68

pos value pos value pos value0 A 1 0 4 11 B 2 0 5 02 C 3 0 6 03 D 4 1 7 2

5 0 8 36 07 28 3

Dictionary IndexvectorIndexvector(values)

example data: A, A, A, B, A, A, C, D

Sparse Compression Compression types: Prefixed

Page 69: IM107 SAP NetWeaver BI Accelerator - New Features

© SAP 2008 / SAP TechEd 08 / IM107 Page 69

pos value pos value pos value pos value0 A 1 0 4 1 4 11 B 2 0 7 2 5 02 C 3 0 8 3 6 03 D 4 1 7 1

5 0 8 16 07 28 3

Dictionary IndexvectorIndexvector(values)

Bitvector(not sparse)

Sparse value: 0

Sparse Compression Compression types: Sparse

Page 70: IM107 SAP NetWeaver BI Accelerator - New Features

© SAP 2008 / SAP TechEd 08 / IM107 Page 70

pos value pos value pos value cluster value0 A 1 0 3 0 0 01 B 2 0 4 1 1 12 C 3 0 5 0 2 03 D 4 1 7 2

5 0 8 36 07 28 3

Dictionary IndexvectorIndexvector(values)

Bitvector(is clustered)

Cluster size: 2

Sparse Compression Compression types: Clustered

Page 71: IM107 SAP NetWeaver BI Accelerator - New Features

© SAP 2008 / SAP TechEd 08 / IM107 Page 71

Integer(IV offset)

pos value pos value pos value pos value value0 A 1 0 1 0 0 0

1 B 2 0 4 1 1 0

2 C 3 0 5 0 2 0

3 D 4 1 6 0 3 1

5 0 7 2 0

6 0 8 3 1

7 2 28 3 3

Dictionary IndexvectorIndexvector(values)

Indexvector(cl. offsets)

0

2

Cluster size: 4

Sparse Compression Compression types: Indirect

Page 72: IM107 SAP NetWeaver BI Accelerator - New Features

© SAP 2008 / SAP TechEd 08 / IM107 Page 72

4. HA - Outlook4.1. Current Status4.2. Documentation

5. Demo – Backup And Recovery6. BIA - Reorg

6.1. Reorg Basics6.2. Locality vs. Parallelism6.3 New functionalities

7. Demo – BIA - Reorg8. Sparse Compression9. Package Wise Read

Agenda

Page 73: IM107 SAP NetWeaver BI Accelerator - New Features

© SAP 2008 / SAP TechEd 08 / IM107 Page 73© SAP 2008 / Page 73

Package Wise Read BIA architecture

Page 74: IM107 SAP NetWeaver BI Accelerator - New Features

© SAP 2008 / SAP TechEd 08 / IM107 Page 74© SAP 2008 / SAP TechEd 08 / IM107 Page 74

Package Wise Read

© SAP 2008 / Page 74

Motivation: no limitation on query result sizeQuery result transported in chunks

Indexserver

BIAEngine

Indexes

TREX

RFCServer

Application Server

chunksize

Page 75: IM107 SAP NetWeaver BI Accelerator - New Features

© SAP 2008 / SAP TechEd 08 / IM107 Page 75© SAP 2008 / SAP TechEd 08 / IM107 Page 75

4. HA - Outlook4.1. Current Status4.2. Documentation

5. Demo – Backup And Recovery6. BIA - Reorg

6.1. Reorg Basics6.2. Locality vs. Parallelism6.3 New functionalities

7. Demo – BIA - Reorg8. Sparse Compression9. Package Wise Read

Agenda

Page 76: IM107 SAP NetWeaver BI Accelerator - New Features

© SAP 2008 / SAP TechEd 08 / IM107 Page 76© SAP 2008 / SAP TechEd 08 / IM107 Page 76

Thank you!

Page 77: IM107 SAP NetWeaver BI Accelerator - New Features

© SAP 2008 / SAP TechEd 08 / IM107 Page 77© SAP 2008 / SAP TechEd 08 / IM107 Page 77

Please complete your session evaluation.Be courteous — deposit your trash,

and do not take the handouts for the following session.

Thank You !

Feedback