Orange County Convention Center

Orlando, Florida | June 3-5, 2014

Lessons from SAP Retail loyalty testing

Mathi Natarajan, Cognizant Tech Solutions

Ana Paula Pacheco, Loblaw Companies Ltd

Frequent Testing challenges in Today’s SAP Projects

TSAP Methodology and Solution Accelerators

RITE Methodology for Continuous Improvement

LEARNING POINTS

Today’s SAP Implementation Landscape

Implementation Model ASAP – Big Bang ASAP – Waves Agile, Iterative Continuous Improvement

Technology Model

Best Practice / Off the Shelf Wireframe SOA/Plug & Play Conventional

Delivery Model Software Factory Managed Services Packaged Services Semi-Packaged Services SAAS

Intelligent SAP loyalty program

Loyalty invite to Customer Customer Registers Online Promotional

Rewards

Customer walks into a store and makes purchases

Customer earns loyalty points

Point transactions aggregated and sent to predictive analysis

Personalized Rewards sent to Customers mobile

Customer Segmentation Personalized rewards

Customer calls call center to activate

The Big Picture – Integration with SAP Retail

Master Data

• Point transactions Posted

• POS Sales Audit • ECC Revenue and

Inventory Posting

• Store Orders to DC • DC Pick, Pack & Ships

the product

Point transactions aggregated and sent to predictive analysis

Article is assorted and Items/Prices sent to POS

Loyalty invite to Customer Customer Registers Online Customer calls call center to activate

Promotional

Rewards Customer walks into a store and makes purchases Store receives the order

Personalized Rewards sent to Customers mobile

Personalized rewards

Customer earns loyalty points

Execution Model - Case Study

SAP CRM Loyalty

SAP Call Center

SAP Retail

SAP BI

POS

Enterprise Integration

SAP Mobile

Web

Predictive Analytics

Email Management System

Bank Integration

Software Factory – Development, System Testing

Managed Services – Development, System Testing and Hosting

Packages Services – SoA Solution, System/Integration Testing, Hosting & Support

SAAS Services – Plug & Play Systems, No Development, Testing, & Support requirements

Semi-Packaged Services – Custom SoA Solution, System/Integration Testing, Hosting & Support

Test Approach

• TSAP

• RITE

TSAP – Suites

Tools

PM Pro

Universal Data Migrator

RISE

iAuto*

cTDMS

Solgen / Sol Access

RAPIDO / iFrame*

ADPART*

Strategy

Kanban Test Scheduling*

Spray Testing Strategy

Simulation Test Strategy*

Business Value Testing

Pattern Based Testing*

Analytics

Causal Analysis

Application Quality Index

Requirement Quality Index

Change Impact Analysis*

Trend Prediction*

EVM Dashboard*

QIS

Process Model

Modular Test Architecture*

12 Point Inspection System

RITE*

BA - QA

cTAAS

* Used in the current case study

RITE - comprises of lean and fast delivery strategy, just in-time scheduling and QA Analytics resulting in greater value generation

KANBAN Just In Time Scheduling

AGILITY Innovation & Automation

PBS Trend & Impact Analysis

BVT Quality & Change Control

RITE - Rapid & Intelligent Test Engineering

Pattern Analysis

0

20

40

60

80

100

120

140

160

180

AB

AP

BI-

FI

BI-

Me

rch

BI-

Sale

s A

ud

it

Ass

et

Acc

ou

nti

ng

Acc

ou

nts

Pay

able

Acc

ou

nts

Rec

ieva

ble

Gen

eral

Le

dge

r

SRM

/IP

MD

-Art

icle

s

Ass

ort

men

ts

Pu

rch

ase

Pri

cin

g

MD

-Cu

sto

me

r

Re

tail

Pri

cin

g

MD

-Sit

e

MD

-Ve

nd

or

MM

Pro

ject

Sys

tem

s

Re

al E

stat

e

MD

M

Sale

s A

ud

it

Jul

Aug

Sep

Oct

Nov

Dec

0

20

40

60

80

100

120

140

160

180

Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb

SAP ECC - Merch - Articles

SAP ECC - Merch - Pricing

Strategy realignment based on trend analysis

Just in Time Scheduling enabling flexible intake process as well as transparent tracking system

KANBAN

Case Study - Test Challenges & Mitigations

• Scope, Schedule & Budget

• Requirement & Design

• Environment & Release Management

• Test Data Management

• Delivery Model

• Technology Management

• Change Management

• Incident Management

• Problem Management & Continuous

Improvement

Challenges & Mitigation - 1

Scope, Schedule & Budget

o Delay in initial scope finalization & Frequent changes

in scope o Schedule over run due to delay in

design/development/unit testing o Budget constraint for regression testing

Mitigation

o Introduce Entry Exit Guardrails to limit scope

changes

o Leverage Semi-Agile Test Approach to mitigate build

delays

o Leverage Change Impact Analysis & iAUTO to

mitigate Regression testing constraints

o Build Common Regression cycles

Challenges & Mitigation - 2

Requirement & Design

o Requirement creep and frequent changes in design

o Inconsistent and incomplete requirements, which led

to multiple design changes

o No documentation or little documentation on Non

functional requirements

Mitigation

o Measure Requirement Stability Index to improve

requirement changes in future releases

o Perform Requirement assessment for Functional and

Non-Functional information's as part of Entry Gating

criteria

Challenges & Mitigation - 3

Environment & Release Management

o Saas, Hosted Services, etc impacting In-house QA

environment planning

o Challenges in Holistic release management for an

Enterprise implementation

Mitigation

o Introduce Integrated Environment planning and

scheduling process

o Use SPRAY testing approach for early detection of

environment issues

o Build / FUT Scope/dates by applications needs to

align with Semi-Agile SIT Scope/dates

o Collaborative Release and Test scheduling with

adequate contingencies

o Build Simulators to mitigate application / build

availability challenges

Challenges & Mitigation - 4

Test Data Management

o Master data quality issues due to incorrect conversion

o Sharing of master data among Marketing,

Merchandise, Supply Chain, Store, Finance, Reporting

work streams lead to huge co-ordination effort

o Data dependency across multiple applications

Mitigation

o Execute Data Validation Macros/Queries as part of

SIT/RT Entry Criteria

o Completion of Mock Conversion cycles with target

quality % before each QA cycle

o Build E2E Data Management Plan as part of E2E

Scenario creation with specific step wise callouts on

data and system dependencies

Challenges & Mitigation - 5

Delivery Model

o Highly customized SAP solution with incomplete

documentation

o Frequent churn resulting in knowledge gaps especially

for customized modules

o Geographically separated delivery teams

o Packaged vs Factory vs Managed Services model

introduces varying degrees of problems like Demand

management, Contractually diverse responsibility

matrix, Defect turnaround KPI’s, FUT Quality, etc

Mitigation

o E2E Architecture expanded to include test

requirements, dependencies and data flow variations

o Leverage technology tools, flexible schedules and

process templates to mitigate geographical

constraints.

o Centralized QA Entry Exit gating criteria and Process

Control measures

Challenges & Mitigation - 6

Technology Management

o Complex end to end testing involving diverse

technologies leading to testing tool challenges

o Convoluted Non-Functional capabilities across

applications/ interfaces

o Application, Device and Infrastructure testing in play

o High number of infrastructure variations leading to

QA coverage limitations and QA result extrapolation

challenges

Mitigation

o Leverage hybrid testing tools and accelerators to

mitigate test tool limitations.

o Build Visual E2E Non-functional matrix for

applications/interfaces to identify gaps

o Master Test strategies should cover device and

infrastructure parameters to enable holistic strategy

o QA result extrapolation models to be fine-tuned

based on Pilot Go-Lives

Challenges & Mitigation - 7

Change Management

o Multiple vendors and Diverse delivery models pose a

change management night mare.

o Application Support KPI’s and varying business needs

influence process deviations

o Expedited CRQ’s bypass all change control processes

Mitigation

o Leverage QA change control (Poka Yoke technique) to

mitigate production risk due to frequent changes

deployed into production

o Publish Weekly QA KPI’s measuring upstream and

downstream quality including Dev Quality/Schedule,

QA Quality/Schedule/Capacity, Environment

Availability, etc. This would increase stakeholder

awareness including schedule.

Challenges & Mitigation - 8

Incident Management

o Infrastructure and Application stability/ maintenance

activities would cause high impact production

incidents which require faster resolutions including

QA activities.

o Consumer focussed applications warrant emergency

responses due to Business sensitivity & qualitative

impact which in-turn requires faster testing timelines

o Security patches are governed by legal procedures

requiring faster resolution times including QA

Mitigation

o Build Automation Modules to execute on-demand

including test data prep and validation

o Build Simulators to expedite critical path testing

o Build automated regression suite specific for security

and infrastructure patches

o Leverage Impact Analysis tools for risk based testing

Challenges & Mitigation - 9

Problem Management & Continuous Improvement

o Consumer focussed applications warrant constant fine

tuning of functionalities leading to frequent KTLO

enhancements

o Heterogeneous application landscape requires

frequent patching / maintenance activities leading to

regular QA testing activities

o Frequent changes to production causes application /

infrastructure stability issues increasing incident

testing requests

Mitigation

o Leverage pattern based analysis to right size QA

readiness based on incident prediction

o Right size automation requirements to meet

predicted demand

o Cross Skill associates to expedite integration testing

o Increase innovation through Gamification

o Leverage Incident Build Synergy and QA Execution

synergies for handling more requests

Learn technology related challenges and it’s impact to SAP testing

How to apply hybrid testing techniques to fit changing needs of SAP

projects

Understand the RITE techniques like Pattern analysis, Kanban, etc for

warranty & maintenance phases

QA Analytics for successful SAP program management

KEY LEARNINGS

THANK YOU

THANK YOU FOR PARTICIPATING

Please provide feedback on this session by completing a short survey via the event mobile application.

SESSION CODE: 2511

For ongoing education on this area of focus, visit www.ASUG.com

FOLLOW US