sap business objects planning and consolidaton, version for sap netweaver

W I N T E R C O R P O R A T I O N

WH

ITE

PA

PE

R

S P O N S O R E D R E S E A R C H P R O G R A M

Th

e L

a rg e

Sc a l e

Da t a M

a n a g e m e n t E x p e r t s

SAP BuSineSSOBjectS PlAnning And cOnSOlidAtiOn, VerSiOn fOr SAP netWeAVer

Performance and Scalability

March 2010

©2010 Winter Corporation, Cambridge, MA. All rights reserved.

245 FIRsT sTREET, suITE 1800CAMbRIdgE MA 02145

617-695-1800

v i s i t us a t www.wintercorp .com

W I N T E R C O R P O R A T I O N

SAP BuSineSSOBjectS PlAnning And cOnSOlidAtiOn, VerSiOn fOr SAP netWeAVer

Performance and Scalability

SAP BuSineSSOBjectS PlAnning And cOnSOlidAtiOn, VerSiOn fOr SAP netWeAVer: Performance and Scalability

A W I N T E R C O R P O R A T I O N W H I T E P A P E R

Copyright © 2010, WINTER CORPORATION, Cambridge, MA. All rights reserved.

3

Executive SummaryTHE uSE Of ENTERPRiSE PERfORMANCE MANAGEMENT (EPM) SOfTwARE for speeding an organization’s financial reporting and planning and budgeting can have a direct impact on the bottom line. Closing the books rapidly means that business decision makers at every level can immediately react to changes in the business environment as revealed by the financial results. Faster reaction can mean both more leverage from new opportunities and less money wasted on initiatives that don’t perform. Faster planning, and budgeting according to a “what-if” plan allows deeper investigation of alternative strategies, with a greater chance that the final strategy will be effective. Finally, today’s business environment often couples a complex, rapidly growing organization with a need for deeper financial analysis. This means that planning, budgeting, and consolidation software must perform and scale better with each version.SAP BusinessObjects™ Planning and Consolidation software, presently used at an sAP-estimated 1800 customers worldwide, aims to meet the needs of medium- and large-scale organizations for high-performance, scalable planning, budgeting, forecasting, and financial consolidations. The SAP BusinessObjects Planning and Consolidation, application version for SAP NetWeaver was introduced into the market in late 2008. To verify that SAP BusinessObjects Planning and Consolidation, version for SAP NetWeaver meets the needs of likely end users for performance and scalability, WinterCorp reviewed and monitored testing performed by sAP on a typical configuration. The testing involved a simulated high-workload use of SAP BusinessObjects Planning and Consolidation, version for SAP NetWeaver for planning, consolidation, and “combined” (simultaneous planning and consolidation activities) scenarios, using HP/Mercury LoadRunner for the test. Tests were performed for 50, 250, and 500 concurrent users.Highlights of our findings include:

The tests that sAP performed were realistic, as to the number of concurrent users, hardware •and software used, and the user scenario for each activity.The tests established to our satisfaction that the configuration specified exhibited better-than-•linear scalability in processor consumption as the number of concurrent users increased from 50 to 500. Even allowing for delays as users work offline during the processes, it is possible for some •organizations to complete a full consolidation process or “what-if” scenario with participation by lines of business, intermediary mid-level managers, and senior corporate finance staff—within a day. scalability from 50 to 500 concurrent users in key subprocesses was better-than-linear on average •in all scenarios. Maximum response time for these subprocesses was within a reasonable distance of average •response time. SAP BusinessObjects Planning and Consolidation, version for SAP NetWeaver demonstrated a high degree of predictability in its performance.Average response times, in all cases where the end user awaits a response (e.g., reports and •currency conversions), appeared satisfactory to excellent even in the 500-user case.

WinterCorp concludes that, when correctly configured, the SAP BusinessObjects Planning and Consolidation, version for SAP NetWeaver product should produce satisfactory to excellent performance, and excellent scalability, in typical business uses involving up to 500 concurrent users.




AcknowledgementssAP used IbM system x3950 M2 and x3850 M2 servers to achieve its performance goals for this project. IbM provided hardware, system software and support for the testing effort. The configurations used in this testing included eight, six, and two processor configurations. Hyde Park Partners, LLC contributed to the design of the planning application using expertise gained through their onsite engagements and supported the Consolidation application effort.

4




5

Table of ContentsExecutive summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

Table of Contents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

1 Introduction & Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

2 benchmark Overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

2.1 Overview of sAP businessObjects Planning and Consolidation, Version for sAP NetWeaver . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

2.2 The system Configuration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

2.3 scenarios Tested . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

3 Test Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

3.1 Findings – Resource scaling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

3.2 Findings - Response Time . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .10

4 Analysis of Test Results. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .11

5 Conclusions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .11




6

IntroductionToday’s business environment is exceptionally cost-constrained and risky, with rapid and unexpected external developments that often affect the bottom line of the business, such as changes in bank lending behavior. Reacting to these changes in the context of a business plan requires rapid generation of results, quick re-planning and re-budgeting, and coordinated, swift implementation of the new budget across highly complex organizations.ground zero for this effort is consolidation, planning, and budgeting software—an application of EPM software. Vendors such as sAP that offer this software are held to a higher standard: they must improve performance and scalability with each release, to meet the new demands; and they must handle all of the complexities of organizations with tens and hundreds of distinct lines of business. The aim of WinterCorp’s benchmark assessment is to see whether SAP BusinessObjects Planning and Consolidation, version for SAP NetWeaver is able to handle typical large/complex-organization consolidation/planning needs in the real world.

METHODOLOGYThis WinterCorp White Paper reviews the performance and scalability of mixed workloads in the SAP BusinessObjects Planning and Consolidation, version for SAP NetWeaver architecture. In developing this paper, WinterCorp operated as an independent industry expert, reviewing product documentation and critically reviewing test design, test process and benchmarking measurements in order to arrive at the descriptions and conclusions presented here. sAP was provided an opportunity to comment on the paper with respect to facts. WinterCorp has final editorial control over the content of this publication. This White Paper was sponsored by sAP.sAP completed the tests described below during the period of december 2009–January 2010. sAP then supplied WinterCorp with the raw LoadRunner statistics for the tests, as well as summary statistics generated from the tests. In preparing this report, WinterCorp reviewed the flow diagrams for the scenarios described below as well as the statistics generated by LoadRunner and sAP, and generated summary statistics that form the basis for our conclusions in this report.

Benchmark Overview2.1 OVERVIEW OF SAP BuSINESSOBjECTS PLANNING AND CONSOLIDATION, VERSION FOR SAP NETWEAVERSAP BusinessObjects Planning and Consolidation is software that enables organizational planning and consolidation processes. The aim of the product is to improve planning/consolidation frequency, accuracy, and effectiveness; reduce budget-cycle times and costs; and increase user confidence in the validity of financial results. Notable features include support for “self-service” financial reporting, intercompany reconciliation, legal consolidation, and collaboration. sAP says that 1800 customers use the application.The version of the application being tested is SAP BusinessObjects Planning and Consolidation, version for SAP NetWeaver. This version is available alongside the existing version for the Microsoft platform. It is built directly on the sAP NetWeaver technology platform (bW, AbAP Application

1

2




7

server). Whereas in the version for the Microsoft platform, Microsoft sQL server was the only database supported, the version for sAP NetWeaver works with any relational database that sAP NetWeaver supports. The architecture is a typical Web setup (see Figure 1 below), with remote clients issuing transactions accessing an OLAP-type data store including cubes via .Net server, sAP AbAP Application server, and the NetWeaver bW OLAP Engine.

Figure 1: SAP BusinessObjects Planning and Consolidation, version for SAP NetWeaver Architecture

source: sAP

2.2 THE SYSTEM CONFIGuRATION

The configuration of the test environment is 3 IbM servers for the .NET tier, and 3 IbM servers for the AbAP/NetWeaver tier. Remote clients are represented by processes generated by LoadRunner on the .NET tier. The .NET tier runs Windows 2003 Enterprise Edition, 32-bit version, and the AbAP/NetWeaver tier runs Windows 2003 Enterprise Edition, 64-bit version. Each .NET server feeds one AbAP server. One AbAP server runs a “central instance” of the AbAP/NetWeaver software and the Oracle database, with the other two running “dialog instances” that connect to the central instance.The test environment uses between 1.5 and 19 million records of generated data per test and 55 to 151 million records of total data. some of this data is pre-formatted as cubes, and then pre-loaded into the Oracle data store; the rest is “entered” during the test as “user input.” Likely areas of stress in the configuration during a test run include “write back” of data input during the test, query-type reads for reporting, and data Manager that will handle data loading and logic execution. The test configuration includes sAP NetWeaver business Warehouse (bW). An optional component of the product to improve performance is sAP NetWeaver business Warehouse Accelerator (bWA). The test configuration did not include this component.




8

2.3 SCENARIOS TESTEDsAP identified, modeled, and tested planning, consolidation, and combined scenarios. Combined, these scenarios involved more than 450 distinct transactions, or “unit operations.” Each scenario’s process included multiple “sub-processes” (e.g., gross Margin calculation) as shown in Figures 2 and 3 below. Each sub-process represented a sequential series of unit operations (e.g., Report Actuals) that are carried out by a particular type of user (entity-level manager, mid-level manager, or senior finance person). sub-processes can occur concurrently or sequentially.Each scenario was tested using 50, 250, and 500 simulated concurrent users. These numbers of concurrent users approximated an organization with up to 1200 lines of business that would feed their own planning and budgeting estimates or their results over the most recent time period into the system, have these estimates/results reconciled by mid-level managers, and have the results consolidated for the reporting use of senior finance personnel at corporate headquarters.The tests make a distinction between end users, concurrent users, and active users. For the purposes of this report, we define end users as all of the people using the system at one point or another in a scenario, although all may not be using the system at a given point in time. Concurrent users are those end users who are using the system at a given point in time. Active users are those who are not only using the system at a given point in time, but are actively running processes within the system, simultaneously. The tests measure scenarios in which 50 (or 250, or 500) concurrent users use the system during each sub-process, although not all are active at any one time.An overall flow diagram for the planning scenario is shown in Figure 2. The scenario simulated gross Margin and Human Capital Management (HCM) entry and computation in parallel, followed by overall financial analysis/reporting activities. HCM mid-level manager activities included head count, benefits, and salary input and computation by entity, period, and employee. gross Margin entity-level (e.g., line of business) activities included gross margin unit, unit x price, unit x cost, revenue by entity/customer, and expense detail input/computation/consolidation. gross Margin mid-level manager activities included gross margin comparison and identification of top customers. The senior finance calculations and reporting included expense and expense driver computations, income statement reporting, and revenue by entity/customer.

Figure 2: Planning Scenario Process Flow

source: sAP

An overall flow diagram for the consolidation scenario is shown in Figure 3. The scenario simulated entry of results by entity-level personnel, followed by reconciliation by mid-level managers and consolidation and analysis/report generation by senior finance. Entity-level inputs were entered grouped by asset/liability type, and end users also input/computed changes in liability, tax




9

statements, and validation. Mid-level manager activities included generation and validation of per-entity statements (balance sheet, income statement, cash flow statement, and equity statement). senior finance calculations and reporting included consolidation of each set of per-entity statements into an organization-wide statement.

Figure 3: Consolidation Scenario Process Flow

source: sAP

The combined scenario simply split the sets of concurrent users between planning and consolidation scenarios, and then tested these scenarios in parallel.In the opinion of WinterCorp, these scenarios represent a reasonable approximation of the core planning and consolidation activities of a typical mid-sized or large organization with many entities to include in overall projections or results.

Test Results3.1 FINDINGS – RESOuRCE SCALINGFigures 4 and 5 below present the scaling of processor utilization for the planning, consolidation, and combined scenarios. Figure 4 shows the average percent of processor utilization at peak load in each scenario, for 50, 250, and 500 users.

3

Figure 4: Processor utilization in Each Scenario

source: WinterCorp

Figure 5: Processor utilization Per user In Each Scenario

source: WinterCorp

Figure 5 shows the average percent of processor utilization per concurrent user in each scenario, for 50, 250, and 500 users (for easier understanding, the 50-user case is set to value “1”). Figure 5 shows that in each scenario, addition of concurrent users decreases processor utilization per




10

concurrent user. In other words, SAP BusinessObjects Planning and Consolidation, version for SAP NetWeaver delivers better than linear scalability of processor utilization for up to 500 concurrent users in all scenarios.sAP reports that they have observed much greater database resource requirements during the Planning scenario than during the Consolidation scenario, and much greater NetWeaver processing requirements during the Consolidation scenario than during the Planning scenario.There appears to be some “headroom” to scale beyond 500 concurrent users in the consolidation scenario. However, the planning and combined scenarios show peak average utilization rates of approximately 85% at the 500-user level, indicating that addition of more concurrent users may call for increased processor capacity.

3.2 FINDINGS - RESPONSE TIMEOf course, less-than-full processor utilization may not translate into flat response time when SAP BusinessObjects Planning and Consolidation, version for SAP NetWeaver is scaled from 50 to 500 concurrent users, if some sub-processes are I/O-bound or communications-bound. To determine if this is the case, we analyzed response times for unit operations. In particular, we sought to determine the likely user “wait time” when an end user requested a report, since “endless waits for a report” are the most frequent cause of user dissatisfaction.

Figures 6 and 7 show the average response times for report requests for each user type (line-of-business “entity” manager or mid-level manager/senior finance person) in the consolidation and planning scenarios for 50, 250, and 500 users. Combined scenario results are comparable. For the planning and consolidation scenarios, minimum response times per report request range from 1 to 19 seconds, average response times range from 1 to 41 seconds, and maximum response times range from 1 to 103 seconds. The overall average response time was approximately 5 seconds. These results do not include programmed “think times” (meant to simulate offline work by the user), or processing time for other parts of a unit operation.Figures 6 and 7 also show that as the number of users is doubled, report-request response times increase, on average, by 23%. Any increase of less than 100% indicates good scalability. similar results hold for “write-back” and “data manager” transactions, the other two kinds of transactions in a unit operation. We conclude that SAP BusinessObjects Planning and Consolidation, version for SAP NetWeaver demonstrates, on average, better-than-linear scalability to 500 concurrent users.

Figure 6: Average Response Times For Each user Type, Consolidation Scenario

source: WinterCorp

Figure 7: Average Response Times For Each user Type, Planning Scenario

source: WinterCorp




Figure 8 shows the average standard deviation of response times for each user type in the 500-user case. This figure can be used to understand the predictability of response times. Consistency in response times typically leads to increased user satisfaction with the product. since, 99% of the time, results fall within three standard deviations of the average, these results suggest that product users will only see report requests that take longer than 30 seconds to run, less than 1% of the time. Of course, your mileage may vary, depending on the architecture chosen. In other words, SAP BusinessObjects Planning and Consolidation, version for SAP NetWeaver demonstrates a high degree of predictability in its performance.

Analysis of Test ResultsThe results discussed above indicate that sAP businessObjects Planning and Consolidation, ver-sion for sAP NetWeaver achieves satisfactory to excellent performance and excellent scalability for realistic planning and consolidation scenarios that involve between 50 and 500 concurrent users.For situations in which yet faster system response is needed, customers may wish to consider the bWA option, which aims to accelerate SAP BusinessObjects Planning and Consolidation, version for SAP NetWeaver performance.One interesting alternative identified by the test results is what might be called “rolling” budget-ing. The present planning scenario models a process in which plans are made on the basis of a full set of “before-consolidation” or “after-consolidation” results, and are not altered “on the fly” by intermediate results captured during a simultaneous consolidation process. because of the speed of consolidation, customers could employ a practice in which mid-level managers are given some flexibility to alter the plans and perform “what-if” scenarios during the consolidation process and before senior finance has seen the results, based on variations of these results from plan. For example, a mid-level manager might adjust a line-of-business budget for the remainder of the year, on the basis of results reported for the current month. After reconciliation of results, corporate could then review the new mid-level budgets, adjust the overall budget, and go through a full follow-on planning process—all before the end of the day. because this process would not involve any greater peak loads than the ones modeled here, it appears that customers could employ SAP BusinessObjects Planning and Consolidation, version for SAP NetWeaver in this way and expect to see similar levels of performance.

ConclusionsTo meet the planning/budgeting/financials needs of medium- and large-sized organizations, SAP BusinessObjects Planning and Consolidation, version for SAP NetWeaver customers need to scale cost-effectively to reconcile hundreds of lines of business, and speedily, to allow “continuous” reporting and planning in a rapidly-changing business environment. In particular, SAP BusinessObjects Planning and Consolidation, version for SAP NetWeaver customers need rapid report generation, scaling via

11

Figure 8: Response Time Variation, Planning and Consolidation Scenarios

source: WinterCorp

4

5




proportional increments in system capacity as the business scales, and the ability to handle “mixed” workloads involving simultaneous planning and consolidation.The sAP tests monitored by WinterCorp show that:

SAP BusinessObjects Planning and Consolidation, version for SAP NetWeaver• , when properly configured and running with appropriate hardware capacity, scales approximately linearly with the amount of processor power used for realistic planning, budgeting, and combined scenarios involving 50 to 500 concurrent users at maximum load.Response times showed average better-than-linear scalability, and were quite predictable in all •cases, demonstrating end user report display within 30 seconds of request initiation, better than 99% of the time.Performance was adequate to excellent in all cases that we considered – for example, an •average of 8 seconds to wait for report display after request initiation in the planning and consolidation scenarios. Handling “mixed” processes and transaction loads during the combined scenario caused little •or no decrease in performance and scalability—as measured by processor utilization—compared to other scenarios in all cases. This indicates that SAP BusinessObjects Planning and Consolidation, version for SAP NetWeaver is tuned effectively for a wide range of customer planning/budgeting/financial use cases.

WinterCorp concludes that SAP BusinessObjects Planning and Consolidation, version for SAP NetWeaver, when properly configured and running with appropriate hardware capacity, can provide scalable business planning and consolidation support for at least 500 concurrent users in highly complex organizations, supporting “continuous” results reporting and deep “what-if” analysis for the agile business.

12

WinterCorp is an independent consulting firm that specializes in the performance and scalability of

terabyte- and petabyte-scale data management systems throughout their lifecycle.

Since our inception in 1992, we have architected many of the world’s largest and most challenging

databases in production today. Our consulting services help organizations define business-critical database solutions,

select their platforms, engineer their implementations, and manage their growth to optimize business value.

With decades of experience in large-scale database implementations and in-depth knowledge

of database products, we deliver unmatched insight into the issues that impede performance and

the technologies that enable success.

©2010 Winter Corporation, Cambridge, MA. All rights reserved.

Duplication only as authorized in writing by Winter Corporation.

245 FIRsT sTREET, suITE 1800CAMbRIdgE MA 02145

617-695-1800

v i s i t us a t www.wintercorp .com

sap business objects planning and consolidaton, version for sap netweaver

Technology

sap netweaverperformance

sap netweaver product

sap netweaver forplanning

consolidation software

planning application

faster planning

scalable planning

sapestimated1800 customers