1

Name of the Paper: Unlocking Business to IT Connect: Testing is the Key

Name of the Conference: Software Testing Conference in India - 2013

Author(s):

Abhijit Bora Sumeet Sawhney Organization:

SAP Global Delivery SAP India Private Limited 5th Floor, Wing A, Salarpuria Softzone Sarjapur Outer Ring Road, Bellandur Varthur Hobli, Bangalore – 560 103, India

2

Abstract:

Business Dynamics and IT Innovations are at the speed of light, but still there is a big gap

perceived by the business on IT support. IT organizations of the customers are trying hard

to ensure business is up and running with industry dynamism. Still at the end of the day

Business perceives the hard work from IT hasn’t paid in business sense and IT is slow to

adapt to the business requirement.

Testing is one of the key elements in IT operations and need to discover basic framework

that can connect business to IT and can faster validate the IT support with Quality

assurance. Testing needs to adapt frameworks, measurement and service levels that can

bring in expected business value.

At last Tester needs to understand the industry dynamics of the customer.

This paper aims at the basic rules and framework that IT and in particular testing

organizations should adapt to provide efficient and effective support to business. This

session in particular will touch upon:

• Business expectations from IT and how testing plays a key role.

• Models and Best Practices around innovative testing to support business strategy and

execution.

• IT Service Providers to understand cost is not the only competitive parameter.

Business expectations from IT and how testing plays a key role.

Seamless and uninterrupted business operations are the minimal expectation which the

business stakeholders expect from the IT department. Unfortunately inspite of a large

investment in IT in most of the companies, this minimal expectation is hard to be met and

leaves the stakeholders frustrated. IT Managers are always under tremendous pressure to

justify the investment and fail to make the right influence and maintain customer

satisfaction. One of the key attributes contributing to this is System Downtime.

Downtime falls into two primary categories planned and unplanned. Planned downtime

accounts for approximately 80 percent of all downtime. Planned downtime cannot be

avoided most of the time as more and more business operates 24/7. However the issue is

when planned downtime turns into unplanned one when we cannot recover back within the

schedules downtime or any changes in downtime results in system failures or outages.

Unplanned downtime is what keeps IT professionals, executives and business owners awake

at night.

Testing plays an important role in ensuring a good uptime for the business stakeholders and

end users. Any planned downtime involves a change to the IT systems, be it putting a

3

simple fix or doing a major upgrade. Effective regression testing is essential to ensure a

successful delivery of the fix or upgrade. It is important to do the impact analysis and do

the right amount of testing within the minimum time.

We would like to present an effective model of testing to ensure business operations with

minimum disruption. However before we go into detail, we would like to highlight few

examples of system outages across industries and how it impacts the end customers and

results in loss of revenue, reputation and loyality.

Monday 10 October 2011 16.34 BST

Blackberry Service Outage

Tens of millions of BlackBerry users in Europe, the Middle East and Africa have been unable

to receive or send emails and messages through their phones, following an outage at the

server systems of parent company Research In Motion (RIM) in Slough, Berkshire. September 8, 2011 08:07 GMT Google short on explanations after Docs goes down Google Docs, the company's cloud-based productivity suite, suffered an outage of almost an hour on Wednesday afternoon. The downtime began around 2. June 22, 2011 1:20 PM Microsoft's Cloud BPOS Suite Suffers Outage Again Microsoft's BPOS cloud-hosted communication and collaboration suite suffered an outage on Wednesday, the latest in a string of technical problems in recent months. The problem apparently lasted for more than three hours and involved a networking hardware problem that affected customers in North America. Jun 18, 2011 1:10 PM Computer Problem Grounds United Flights United Airlines had to cancel flights on Friday night after a computer breakdown affected key systems, including those for its website, flight departures and reservations. Although the five-hour outage was solved at around 1 a.m. U.S. Eastern Time on Saturday, the airline expects logistical problems to linger for the rest of the weekend. October 14, 2010 06:13pm EST Yahoo Acknowledges, Apologizes for Yahoo.com Outage Yahoo has acknowledged and apologized for a "brief" outage on Yahoo.com on Thursday afternoon. Yahoo went down at about 2:30 PM PT or so. and remained out for possibly 20 minutes. During that time, visitors to the main Yahoo.com page were greeted by the message "Server hangup" and a blank page. March 4, 2011 5:54am CST AT&T Suffering from 'Massive' International Outage AT&T Wireless is suffering from a major international network outage, representatives from the mobile giant have confirmed to Spetnik Solutions News. Early this morning, AT&T Wireless users currently overseas began experiencing connectivity issues with their devices. Affected users are unable to make calls, access the data network, and many see an SOS. The outage started with iPhones, then we began receiving reports about BlackBerry devices, and now it seems that all mobile phones are affected as well.

4

January 12, 2011 eBay Hosting Service ProStores Experiences Downtime

Customers of eBay’s hosting service ProStores have reportedly suffered downtime on

Tuesday. ProStores sellers reported their sites experienced about two hours starting at 3:30

p.m. EST on Tuesday. Issues with the hosting service began earlier that day with many

secure pages hosted on ProStores running into timeout errors.

Testing is one of the critical components to ensure successful business operations,

especially after a planned downtime. However due to both time and cost constraints, the

aspect of testing is always ignored without realizing the impact of not having it. The testing

fraternity has always highlighted the benefits of testing and had to fight a big struggle to

convince all stakeholders on the necessity and benefits of testing.

Explained below in detail is a very successful approach towards relating the business

benefits of right testing. This approach had helped in ensuring that testing is given the right

priority and business stakeholders are convinced about the benefits of testing. Using this

method, we were able to get the right amount of budget and time to do the requisite

amount of testing for any business need.

This approach consists of 3 (three) main steps for implementation:

• Identification of the most critical scenarios and associate business priority to the

same.

• Analyze the business impact due to failure of this scenarios against the following

impact parameters

o Business Disruption

o Loss of revenue ($/ hour)

o Employee productivity loss

o Reputation and Loyalty damage

• Simulation of the failure situations for each of these scenarios and ensure the

creation of the right test artifacts followed by a successful testing phase.

5

Figure 1: Three steps approach towards a business focused testing.

Below is an example of an implementation of this approach in the retail industry.

Scenario Impact Parameters

Business Disruption

Lost revenue ($ / Hr)

Loss of Employee Productivity ($ / Hr)

Reputation and Loyalty Damage

Point of Sales (POS) Scenario Helps retailer cost effectively run his stores and maximize customer value. Data Points: 50 stores across the city with each store having 10 POS doing a transaction of avg. $450 per hour

Point of Sales is not working. Sales Personnel not able to run transactions for end customers. Slow, inefficient handling of payments lead to customers abandons purchases.

(Transactions/hour) *(POS stations per store)*(# of stores) $450*10*50 = $225,000

(Employee Cost per hour)*(POS stations per store)*(# of stores) $55**10*50 =$27500 Assumption: Avg Employee cost is $55 /hr.

Customer defers purchases in the store for next few months. Intangible damage. Not quantified in $ value.

Table 1: Example of a business focused testing in retail industry

The above approach provides an innovative way to define testing beyond the normal

fundamentals of investigation done to satisfy customer requirements. Stating and clearly

documenting the business impact of a test case brings altogether a different perspective to

why and how testing is done.

Models and Best Practices around innovative testing to support

business strategy and execution.

We would also like to present two conceptual frameworks around testing which is vital and

support business strategy and execution.

• Technical and Business Impact analysis for any change before start of testing.

6

• Reverse engineering of software which have been developed and in use for a long time. Regression testing becomes a difficult task with lack of requirements and test cases. Reverse engineering helps in analyzing a software system, either in whole or

in part, to extract design and implementation information.

There are multiple tools available in market around these concepts. Some key examples

are:

Technical and Business Impact analysis

• HP Change Impact Testing module for SAP applications integrates SAP application

entities with Application Lifecycle Management. Using Change Impact Testing, you

can track the SAP application entities that are impacted by each change you make to

an SAP application. This enables you to run tests associated with the affected

application entities for each change made to an SAP application.

• Change Impact Analyzer from PeopleSoft helps determine the impact of specific

changes planned to be made during an application upgrade. It's an interactive

program where the relationships of PeopleSoft definitions in a hierarchical view can

be seen and analyzed.

• Microsoft Test Impact Analysis (TIA) helps in analysis of impact of development

on existing tests. Using TIA, developers know exactly which tests need to be verified

as a result of their code change. They can then verify the unit tests directly from

inside Visual Studio. Likewise, Testers know exactly which tests they need to verify

for each build.

• Panaya Change Impact Analysis for SAP ECC Version Upgrade and Oracle R12

Version Upgrade and Reimplementation. It helps in mapping all system

customizations, figuring out what will break when we upgrade, and what we need to

test.

• SAP Business Process change Analyzer is an application within SAP Solution

Manager which helps customers in performing a technical impact analysis of software

changes coming into their SAP landscapes.

Reverse engineering of software

• Imagix 4D: With Imagix 4D, software developers have a tool to understand,

document and improve complex, third party or legacy source code. Automate the

analysis of control flow and dependencies. Detect problems in data usage and task

interactions

• Microsoft Visual Studio: From within Visual Studio, you can reverse engineer your

source code into the Unified Modeling Language (UML); your project's class

definitions are used to generate a UML diagram in the Visio environment.

• SAP Reverse Business Process Documentation (RBPD) allows customers to

easily re-document their business processes in SAP Solution Manager. The re-

documentation is done by analyzing the actual usage of the system and is supported

by comprehensive analysis content pre-delivered from SAP.

7

We are choosing the tools from SAP to explain in detail these conceptual frameworks.

Business Process Change Analyser (BPCA) and Reverse Business Process Documentation

(RBPD) both provide the right inputs for defining a strategy for Risk Based Testing. The

steps involved are:

• RBPD is used to identify the process steps that are really in use and therefore have implications for testing requirements when updating a solution.

• The BPCA compare the existing object information (Technical Bill of Materials) against the objects of an incoming change (transport request, support package, or enhancement) and flag the processes that touch changed objects.

• Test Consultant take the process steps that have been flagged by the BPCA tool as being impacted and identify and collect the relevant test scripts for execution and reporting. SAP recommended test tools like SAP TAO (Test Acceleration and Optimization) and HP Quality Centre is used subsequently to carry forward the testing activities.

Figure 2: Risk Based Testing using Business Process Change Analyzer

8

Figure 3: Test Scope Optimization for SAP Support Packages and Enhancement Packs

Figure 4: Typical Lifecycle of Documentation and Requirement for Reverse Business Process

Documentation

9

IT Service Providers to understand cost is not the only competitive

parameter.

Testing has always been considered as a repeatable task that can be outsourced to the

service providers. Outsourcing is a good strategy but if analyzed with single dimension of

cost often leads to severe outages in IT providing support to business and leads to end user

frustration.

With IT departments of the business organizations working under tremendous pressure of

supporting agile business needs (newer and newer ways to compete, mergers, demergers

etc.), over and above managing the IT operations, Cost becomes a multidimensional aspect

with other parameters influencing it. Few of these parameters are highlighted below:

1. Processes and Approach : Cardinal sins that companies perform is either making

organizations around the new technologies / tools that can support testing or

procuring testing tools based on the organization they have internally or outsourced.

A great paradigm shift is required for all these organizations to focus on the

Processes and then identify the tools that are required to support the processes and

organization that can leverage the tools and support the processes.

Figure 5: Processes, Tools and Organization

2. Bring Business close to IT: IT needs to understand that the effectiveness and

efficiency in their services is directly linked to a framework that brings Business close

to IT and provide a transparency. Below depicted pyramid is one of the dimensions

that companies can think to put a framework. This pyramid talks about the following

two important aspects:

10

• Strategy: IT strategy is based on the way Business defines the strategy, IT

cannot create any approach without Business connect. Only with the right

business connect optimized and effective testing can be ensured.

• Governance: Governance with Business into it is imperative, this is relevant for

in-house / outsources testing too.

The topic of IT to Business connect is also detailed in a subsequent section on TCO

reduction.

Figure 6: Framework to bring Business close to IT

3. Support Efficient Business Process Improvement: To stay competitive,

companies need to continuously analyze, monitor and improve their existing

processes that are supporting business. With complexity and geographies multiplying

each day, companies badly needs least disruptions in current way of business and

still stimulating what can be the much efficient process to support business

acquisition. The answer is to implement a process enabled by the tool to support

Business Process Management (BPM), that proactively anticipates the problems and

thus avoid a cost infested Implementation and then hefty testing support to support

a not so efficient process. Companies look at IT to provide:

• Quantitative process analysis • Process efficiency analysis • Benchmarking • Business continuity analysis

11

4. Aligning Customers need to reduce the overall IT TCO (Total Cost of Ownership) with Testing: It is important to connect to customer’s business stakeholders beyond the IT department. IT Service providers should understand customer’s overall business strategy and how IT Strategy is aligned to the same. IT Service Providers need IT Architects to define and implement the connect between business and IT. These Architects will be the trusted advisers to the IT Managers and CIO and help

them in reducing the overall TCO by pulling the right levers. The key levers are

detailed below.

Table 2: TCO Levers.

Testing plays an important role in all these levers and once these aspects are

established, there is a universal acceptance for the need of testing.

Explaining below are the testing aspects for each of these levers:

• Reducing the complexity of Systems , Data and overall IT Environment

o Installation Qualification: Verifies the components detailed in the

infrastructure specification exist and are installed correctly after

consolidation and harmonization of landscapes.

o Extensive regression testing to ensure that refactoring of code and moving

towards standard does not impact business requirement and business

continuity is maintained.

12

o Successful movement of business from legacy systems to modern and

new systems with minimal disruption. Testing with the relevant business

data.

• Bringing in the right level of excellence in execution

o Enterprise wide Test Strategy to ensure an end to end testing across

business domains.

o Aligning with the overall ITIL Processes and guidelines. (IT Service

Management Strategy).

o Defined SLAs for testing with both IT and Business.

o Setting up the required Testing Centre of Excellence around Strategy,

Tools and Process, Organization and Execution.

• Optimizing the complete sourcing for IT

o Positioning the right level outsourcing and vendor strategy for the

customer.

o Being a trusted adviser and partner rather than an IT vendor and ensuring

the right value proposition around testing.

• Aligning business with IT

o Aligning the test scenarios with business impact of failure – Helps to show

the value proposition of testing and business connects well.

o Simulate business scenarios with real test data.

Based on the above parameters , we would like to present the below template which provides few key metrics to ensure a right value proposition of testing the IT Service providers can position to their customers. This is only an example with few metrics and can be expanded depending on the needs of the customer.

Metrics Details Value and Positioning

Benchmarking of IT Operations against it’s peers

The Benchmarking values are: • IT Spend (% of revenue) • # of Systems per IT Landscapes • Unplanned downtime (hrs/ year) • % of Critical Interfaces • Incidents/month ($) • Support Cost per End User ($) • # of Changes per 100 users • # of IT Vendors • Testing cost per Change ($)

These parameters are important to benchmark against “Industry Average” and “Peers – top 25%. Based on where the customer stands, the right testing approach and strategy could be defined and implemented.

Key Business KPIs

Need to identify and align test cases with Key business KPIs (Examples below) • Time To Close Annual Books (in days) • New Products Or. Services As % Of

Revenue (in %) • Order Fill Rate (in %)

These KPIs will help in understanding the business impact and key scenarios for testing.

Costs around Testing

• Internal headcount • Contracting /Consulting costs • Outsourcing Costs

These costs are vital to justify testing and bring in optimization in testing.

13

• Hardware Costs • Software Costs (License, Training)

It also gives transparency to the customer around testing.

Table 3: Metrics to position relevant testing aligned with the goal of reducing TCO

14

Reference Materials

Blackberry Service Outage http://www.citynews.ca/2011/09/16/rims-bbm-service-experiences-outage/ Google short on explanations after Docs goes down http://www.zdnet.com/google-short-on-explanations-after-docs-goes-down-4010024291/ Microsoft's Cloud BPOS Suite Suffers Outage Again http://www.pcworld.com/article/230903/article.html Computer Problem Grounds United Flights http://www.pcworld.com/article/230694/Computer_Problem_Grounds_United_Flights.html Yahoo Acknowledges, Apologizes for Yahoo.com Outage http://www.pcmag.com/article2/0,2817,2370807,00.asp AT&T Suffering from 'Massive' International Outage http://www.spetnik.com/a/news-story/id:7001111,story:confirmed_att_wireless_suffering_from_major_international_network_outage eBay Hosting Service ProStores Experiences Downtime http://www.thewhir.com/web-hosting-news/ebay-hosting-service-prostores-experiences-

downtime

Panaya Technical Fact Sheet http://panaya.com/images/stories/Datasheets/panaya_sap_upgrade_automation.pdf http://panaya.com/images/stories/Datasheets/panaya_oracle_upgrade_automation.pdf HP Change Impact Testing http://update.external.hp.com/qualitycenter/qc110/ent/cit/ChangeImpactTestingUG.pdf PeopleSoft Change Impact Analyzer http://docs.oracle.com/cd/E15743_01/psft/acrobat/pt850tcia-b0909.pdf Microsoft Test Impact Analysis http://msdn.microsoft.com/en-us/library/ff576128(v=vs.100).aspx SAP Reverse Business Process Documentation https://websmp107.sap-ag.de/~form/sapnet?_SCENARIO=01100035870000000202&_SHORTKEY=00200797470000094563 Imagix 4D http://www.imagix.com/index.html Microsoft Reverse Engineering http://office.microsoft.com/en-in/visio-help/about-reverse-engineering-code-to-the-uml-HP081550745.aspx

15

Author’s Biography Abhijit Bora is the Practice Head for operations services at SAP Global Delivery with a portfolio spanning across Application Life Cycle Management, Testing to Application Support Management. He holds a portfolio that is responsible for more than €20 Milo person days of delivery. Over the years he has led many positions in SAP product development and SAP consulting in operations services. With his vast experience in ALM services, Abhijit has been a key player in acquiring strategic customers and fore-fronted in delivering these projects. He is a thought leader in pioneering new testing methodologies and providing value proposition to customers. Sumeet Sawhney, BE & MBA has 12+ years of experience in SAP and Business functions. With SAP, he has 7 years of experience in Application life cycle management and Test management. In his role as Operations Services Manager he has extensive experience in Leading, Managing & Strategizing testing and operational services approaches for various customers using different tools. Given the flexibility to run Engagement for Test Services as an entrepreneur, was instrumental in creating innovative work packages for new services, Quick Wins & remodeled existing services that helped new customer acquisitions across the globe.

16

Appendix IT: Information Technology BPOS: Business Productivity Online Suite POS: Point of Sales TCO: Total Cost of Ownership ALM: Application Lifecycle Management KPI: Key Performance Indicators SLA: Service Level Agreements ITIL: Information Technology Infrastructure Library