09 solution testing- an approach for testing …minisites.qaiglobalservices.com/stc2012/paper_...
TRANSCRIPT
SOLUTION TESTING - AN APPROACH FOR
TESTING MULTIPLE ENTERPRISE INTEGRATED
PRODUCTS
STC-2012 Conference Author:Rashmi Mahajan, Saroj Patnaik
Confidentiality Statement
This document contains confidential and proprietary information for Fiserv
and may not be disclosed to others than those to whom it was originally
distributed. It must not be duplicated, published, or used for any other purpose than originally intended without the prior written permission of
Fiserv, Inc.
DisclaimerInformation described in this document is believed to be accurate and
© 2012 Fiserv, Inc. or its affiliates.
Information described in this document is believed to be accurate and reliable, and much care has been taken in its preparation. However, no
responsibility, financial or otherwise, is accepted for any consequences
arising out of the use or misuse of this material.
Fiserv is an equal opportunity employer.Fiserv is a trademark of Fiserv, Inc.Other brands and names are the property of their respective owners. Information subject to change.© Copyright 2010 Fiserv, Inc.All rights reserved.
Author Information
Rashmi MahajanTesting Manager – Testing Center of Excellence
Fiserv India
Mrs. Rashmi Mahajan is Manager (Testing) at Fiserv Global Services, Noida.
She is a PMP certified professional having more than 12 years of IT experience
in development and testing projects. She is currently working as a Testing
© 2012 Fiserv, Inc. or its affiliates.
3
in development and testing projects. She is currently working as a Testing process consultant in Solution Delivery Process group.
Co-Author Information
Saroj PatnaikGeneral Manager – Testing Center of Excellence
Fiserv India
Mr. Saroj is General Manager (Testing) at Fiserv Global Services, Noida. He is a PMI®,
Prince2® and CSQP® certified professional having more than 18 years of IT experience
in consulting Testing Projects and having extensive experience in implementing Test
Automation Framework using HP and IBM suite of product. On his recent engagement as
Testing Practice Manager with Fiserv Global Services, he has been developing many
© 2012 Fiserv, Inc. or its affiliates.
4
turnkey automated solutions for various customers of Fiserv. Some of his key
implementation includes ALM implementation using SpiraTeam/HP ALM Quality Center
11.0, End-to-End Service testing using HP STM/Soap UI Pro, Performance Testing best
practices for a Batch Process System of a large financial product, Applying Risk Based
Testing using HP Quality Center and many more.
Abstract
Enterprise Solution Testing
becoming extremely important
Enterprise Solution Testing
becoming extremely important
Lack of a defined process
for testing the entire Solution before delivery
Lack of a defined process
for testing the entire Solution before delivery
© 2012 Fiserv, Inc. or its affiliates.
Need an approach to Solution Delivery that can helps better turnover and engagement of implementations team from sales team
Need an approach to Solution Delivery that can helps better turnover and engagement of implementations team from sales team
The individual product teams are aligned towards the product delivery
The individual product teams are aligned towards the product delivery
Problems at go live
We never have time to plan in the beginning, but we always have time to fix
our mistakes at the end.
- Author Unknown
© 2012 Fiserv, Inc. or its affiliates.
Outcome
Penalties
Over Budget
© 2012 Fiserv, Inc. or its affiliates.
Not Fully Functional
Defective
Approach
A defined Solution Testing process within Solution Delivery
Process can help address this problem.
Solution Testing a consistent cohesive approach to test Multiple
Enterprise Integrated Products before delivering to the Client.
Solution Testing includes set of services, standards, processes,
© 2012 Fiserv, Inc. or its affiliates.
Solution Testing includes set of services, standards, processes,and tools for planning and executing Testing for engagements to
ensure high-quality testing of integrated products.
Solution Delivery Process
Initiation
PlanningTransition
Engagement Delivery Team begin to get
engaged
Deal shifts from Sales process to
being driven by the Delivery process
Leadership shifts from delivery to the
operational support team
© 2012 Fiserv, Inc. or its affiliates.
Execution
Verification
Production
Go
Live
�Configuration�Smoke Testing�System Integration Testing�Limited Regression �Performance Testing�Readiness Assessment�Data Conversion(if applicable)
�Data Verification (if applicable)�Implementation�Smoke Testing�User Acceptance Testing (UAT)�Mock Testing�Performance Testing�Readiness Assessment
Planning and Requirement Analysis
StrategiesProduct Knowledge
Test Design
Test DataProduct Configuration
Test Execution
Test Manageme
nt
Solution Testing
Planning and Strategy
• Process and methodology for Integration Testing
• Studying requirement from Solution Definition Document
• Identifying Integration Testing scope
• High Level Integration Testing Schedule
Strategies
• High Level Integration Testing Strategy and Plan
• Entry/Exit Criteria for Integration Testing
• High Level Environment Strategy including touch point’s environment details
• Defect Management Strategy
• Approach for sharing Integration defects with Product Teams
• High Level Regression
Product Knowledge
• Implementation Specific Knowledge from Product Teams to Testers
Test Design
• Creation of High Level Test Scenarios
• Creation of Integration test cases
• Including Batch transmission and Real Time Testing (Both) under Integration Test Cases
• Validation of Integration Test Cases with Product Teams
© 2012 Fiserv, Inc. or its affiliates.
• High Level Regression Testing Strategy
Test Data
• Creating Test Data Requirement in Sync with Other Product Teams
• What Data for Testing (Production/ Dummy/ Client)?
• Data Tweaking?
Product Configuration
• Implementation Specific Configuration Knowledge from Product Teams to Testers
Test Execution
• Smoke Testing • Integration Testing
• Defect Sharing with Product Teams
• Joint Defect Triage Meetings with Product Teams
• Prioritization of Integration Testing defects
• Defect Retesting• Checking of Exit Criteria for Integration Testing (before UAT)
• Regression Testing with Final Build before GO-LIVE
Test Management
• Test Management throughout the Solution Testing in Execution Phase
Engagement Delivery Team Structure
Delivery Liaison
Enterprise Business Analyst
Solution Architect
Engagement Training Manager
Engagement Deployment
Manager
Engagement Testing
Manager
© 2012 Fiserv, Inc. or its affiliates.
Engagement Director
ArchitectManager
Solution Definition
Focus
Engagement Implementation
Focus
Solution Delivery Team Structure
Solution Testing Tester
Solution Testing Business Analyst
Solution Testing Lead
Solution Testing Manager
© 2012 Fiserv, Inc. or its affiliates.
Engagement Testing Manager
Solution Testing Focus
Metrics
S. No. Metrics Name
1 Planned Vs Actual - Test Case Execution
2 Test Case Execution - Distribution by Status
3 Test Case Distribution by Integration Point
4 Weighted Defect Distribution by Product
5 Weighted Defect Distribution by Integration Points
© 2012 Fiserv, Inc. or its affiliates.
5 Weighted Defect Distribution by Integration Points
6 Defect Distribution by Severity/Priority*
7 Defects by State
8 Defect Aging
9 Pre-Delivery Defect Distribution by Severity
10 Post Delivery Defect Distribution by Severity
11 Defect Removal Efficiency
Level of Prescription
Unifyingframeworks
Alignment in design (What)
� Language� Definitions� Roles
Unity
QualityPerformance, Protection
Unifyingframeworks
Alignment in design (What)
� High-level processes� Deliverables� Performance assessment
� Language� Definitions� Roles
Clarity in execution (How)� Procedures / SOP� Templates
� Tools� Metrics
Unity
Uniformity
© 2012 Fiserv, Inc. or its affiliates.
QualityPerformance, Protection
Alignment in design (What)
� High-level processes� Deliverables� Performance assessment
Clarity in execution (How)
� Procedures / SOP� Templates� Tools� Metrics
Uniformity
Other Considerations
• End-to-end Testing
• Dedicated Environment for Solution Testing
• Performance Testing
• Competency building for future engagements
© 2012 Fiserv, Inc. or its affiliates.
Benefits of the Approach
Solution Testing helps deliver a defect free Products or Solutions
and attain Client Satisfaction
Setting up a standard process for testing enterprise level integrated
product helps execute the Solution Testing effectively and efficiently
A centralized team executing the Solution Testing can help organization
© 2012 Fiserv, Inc. or its affiliates.
A centralized team executing the Solution Testing can help organizationto avoid repeating mistakes and in saving cost of testing bysharing resources over engagements
A satisfied Client will help promote the products to other institutions
References
• Presentations from the Solution Delivery Process Team at Fiserv.
• Other resources and images on internet.
© 2012 Fiserv, Inc. or its affiliates.
Thank you
Rashmi MahajanTesting Manager – Testing Center of Excellence
Fiserv India
Saroj PatnaikGeneral Manager – Testing Center of ExcellenceFiserv India
© 2012 Fiserv, Inc. or its affiliates.