SAP - Redefining software testing

Sanujeet Puhan

SAP Technical Architect

Overview

• Application softwares

• Enterprise applications

• Current trends

Test Approach

• Applications vs Processes

• Risk perspective

• Critical questions

Methods and Tools

• Business blueprint

• Technical Bill of Material

• Impact Analysis

• Test plan simulation

Agenda

Application softwares

Custom application, supports specific

functions/services

Standard application, can be combined to provide

specific functions/services

Enterprise application, provides range of standard

enterprise functions and services

Systems

Applications

Products

Enterprise Applications

A whole bunch of domains

FI,MM,SD…many unused

Tightly integrated

Change means side-effects

Vendor specific Data model

Client has limited view

SAP Testing situation

Source: ASUG Test influence council member survey 2010

Customer software vs Standard erp software 1(4)

Focus

Custom software Standard software

Thinking in terms of Applications Focus on - Bug-free operation, - Ergonomics of user interface, - Compact solutions

Thinking in terms of Business Processes Focus on - Quick deployment, - Robust business processes, - separation and standardization

How it matters for testing

Major tests are: - Unit testing (code quality) - Whitebox testing (coverage) - Exploratory testing (exception)

Most tests are: - Integration testing - Blackbox testing - Regression testing

Customer software vs Standard erp software 2(4)

Development

Custom software Standard software

Coding is a major activity. Typical cycle is:

Coding is only for enhancements. Typical cycle:

Design

Build

Test

Deploy

Design

Configure

Test

Deploy

How it matters for testing

Since coding is a major activity, Testing involves developers.

Needs good data quality. Testing requires business process experts.

Customer software vs Standard erp software 3(4)

Change control

Custom software Standard software

Changes triggered by - new user requirements - obsolescence

Changes triggered by - User requirements - Obsolescence - Vendor’s strategy

How it matters for testing

Since customer is in control, Frequency of testing is less.

As vendor updates must be applied, Regression Testing is frequently done.

Customer software vs Standard erp software 4(4)

Custom software Standard software

Self architected, meaning Dependencies between modules are usually well-known.

Predefined architecture, meaning Many dependencies maybe unclear.

How it matters for testing

Test scope can be decided with good accuracy.

Test scope can become large, to account for hidden dependencies.

Dependencies

P1

P4

P2

Px

P3

P5

But “Test all” feasible ?

Px

P1 P2

P3 P4

P5

Test

bench

Test everything, it is safest

A typical test scenario

C1

C2

C3

C5

C4

C6

- too vast - too intricate

P1

P4

P2

Px

P3

P5

Test

bench

Px

P1

P2

P5

P3

P4

Identify critical processes

Risk = Probability of failure * Cost of failure

But, what is the actual probability of failure ?

C1

C3 C4

C6

C2 C5

Risk based test approach

P1 P2 P5

P4 P3

Critical Questions

How to

find processes with actual risk, NOT assumed risk

adapt test plan to resource constraints

P1

P4

P2

Px

P3

P5

C1

C2

C3

C5

C4

C6

If process and code can be linked …

Then we know which processes are actually impacted

8 steps, 6 affected by code changes

C1

C2

C3

C5

C4

C6

Scenario Coverage (%) Effort (min) Choice

S1 83.3 180

S2 83.3 315

S3 66.6 210

S4 66.6 315

S1

Test

bench

Px

P2

P5

P4

P3

An optimization example

S2 S3 S4

P1 P2 P5

P4 P3

Px

Links also indicate test effort

1

How to:

Identify processes

with actual risk

Impact analysis

Adapt test plan to actual

resource constraints

Optimization

Test coverage

Test effort

process code /*

A real life scenario:

New SAP Enhancements are applied, which changes many objects

The need of a test strategy in SAP

Business Blueprint SAP’s approach to unify processes, applications and systems

Technical bill of materials Links process to underlying objects

Impact Analysis

Determining which processes are affected

Boundary setting Select a meaningful subset of packages

Dashboarding Status and other aids for project management

Test Scope Optimization

Optimization

based on:

Test coverage

Test effort

Business priority

Approach: All changed objects should be tested at least once.

Actual Optimization Simulations with Coverage and Test effort

Final checks

Identify / add anything missing

Benefits

NEXT STEP:

Generate test plan, automate, assign testers, create learning maps …

Precise insight of change impact

Better risk management for business

Safeguard test case obsolescence with TBOMs

Timely identification of gaps in test scope

Multivariate optimization

Reduced effort for a requirement based test plan

Thank you !