© 2015 IBM Corporation
Building an Enterprise QA Centre
of ExcellenceBest Practices Discussion
2 © 2015 IBM Corporation
Objectives
Pleased to take this opportunity to present the Enterprise QACoE
The objectives of the session are to:
1. Share our specialized Enterprise QA operations’ Best Practices Management
2. Framework to standing up a specialized Enterprise QACoE
3 © 2015 IBM Corporation
SDLC Quality Operational Quality
CIO CXOs
Categorize,
Monitor,
Improve
Project
Application
E2E Process Process
$M’s opportunity
Quality• Next level in the progression
• Covers IT & Business
Operations
• Focused on removing defects
instead of identifying them
Task
Activity
Test Less
Test Early
Reduce Defects
Testing:• Progressed from
project up to E2E
Process
• Focused on Defect
Identification
Quality Assurance Strategy extends the business value beyond
traditional quality initiatives
Enterprise
Integrated Quality
4 © 2015 IBM Corporation
Test DeliveryIn
take
Me
tric
s a
nd
Rep
ort
ing
Test Planning Test Design Test Execution
Test Operation Configuration Test Automation
Requirements Verification
Test Data Management Performance Testing
UAT Execution/Support
Test Delivery
Inta
ke
Me
tric
s a
nd
Rep
ort
ing
Test Planning Test Design Test Execution
Test Operation Configuration Test Automation
Requirements Verification
Test Data Management Performance Testing
UAT Support
There are many QACoE operating models, IBM believes a successful
model needs to balance quality with testing
Testing
Capabilities
Quality
Competency
Client/IBM
Business Goals, Standards
Test Delivery
Inta
ke
Me
tric
s a
nd
Rep
ort
ing
Test Planning Test Design Test Execution
Test Operation Configuration Test Automation
Requirements Verification
Test Data Management Performance Testing
UAT Execution/Support
Infrastructure ServicesOperations/Management
▪ Community Management /
Governance
▪ Measurement and Metrics
▪ Knowledge Management
Test Operation Tools
Test Environment Services
Quality Services
Factories Virtual services
Automation components
Testing as a Services Defect Analytics and Reduction
Code Analysis and Reporting
Performance Engineering
Test Case Optimization
QA Deployment Coaches Agile
Mobile
“Instill a culture of quality”
“Excellence in testing”
Best Practices, Assets
Portfolio 2 Portfolio 3Portfolio 1
5 © 2015 IBM Corporation
QA Centre of Excellence (QACoE) will bring leading test practices for
more efficient and cost effective testing
QA
Co
E
Centrally organized and governed QA group
Common approach to testing practices
Common approach to test automation tools
Provide the right testing skills at the right time
Continuous improvement of testing quality
Common approach to QA metrics
6 © 2015 IBM Corporation
QACoE will define the QA strategy and the direction for test delivery
Quality
Assurance
Center of
Excellence
Consistent
Approach – Roles
and Responsibilities
Methods, Tools,
Techniques, Best
Practices
Continuous
Process
Improvement
Engagement Work
•Functional Testing
•Automation, Regression
•Performance Testing
•Multi-Platform for Testing
•Test Services Delivery
•Program Management
•Risk Management
•Integrated QA Delivery
•Culture of Metrics
•Enhance operational processes
•Productivity Capability Baselines
CMMI
Processes
Learning
from
Projects
Specialized
skills
Ongoing
TrainingTools R&D
Build the foundation for the QA Center of Excellence (QACoE)
7 © 2015 IBM Corporation
QA and Testing Center Of Excellence mandate should be aligned to the
Enterprise Business Goals and Quality Objectives for highest ROI“Delivery Excellence through a culmination of Standard Framework, Processes and Tools strengthened by a strong Governance mechanism
and Industry Best Practices to provide a wide variety of Testing Service Spectrum”
Test
Planning &
Estimation
Test Design
&
Build
Test
Execution &
Management
Defect
Management
Test
Metrics
Management
Test
Program
Management
TestingCenter of
Excellence
Infrastructure &
Tools COE
Tool Maintenance &
Support
Test Automation
Tool Implementation
Tool Provision
Process COE
Process Maintenance
Process Definition
Metrics Program
Induction Program and
Test Process Training
Continuous Process
Improvement
Best Practice
implementation
> >
QA C e n t e r O f E x c e l l e n c e
Service Spectrum
System /
Development
Testing
System
Integration
testing
Regression
Testing
Test
AutomationUAT
Core – Flexible
resourcing
Standard process
& Frameworks
Common Tools,
Infrastructure &
Automation
Solution
Accelerations/
Best practices
Governance
structure
>
Test Data
Management
Knowledge
Management
8 © 2015 IBM Corporation
Enterprise Application Testing Terminologies
Activities
• Review & Approve Test plan and test cases
• Execute Integration test cases as planned
• Verify test coverage by updating the Requirements Traceability
and Verification Matrix
• Conduct defect analysis
• Complete a test report
Delivery Excellence
Rational Functional Tester, Rational ClearQuest Test Manager, Rational
ClearCase, Winrunner,, Silk, QArun,, App Virtualization Usage &
Procedure Logging [UPL], Business Process Change Analyzer [BPCA],
Test Scope Optimization [TSO], Transaction Bill Of Materials [TBOMs],
Component Based Test Automation [CBTA], Test Data Container [TDC]
How are the tools used?
Automation of Integration test cases, structured test case management,
defect management, traceability metrics Test analytics, accurate usage
depiction, test data collation and parameterization
Tools
Cost and schedule variance, defect, productivity, efficiency and test
coverage metrics
When/ Where measured?
Defect metrics and test case metrics will be measured during the execution.
Improvement metrics will be measured after integration testing is complete.
Measuring usage in Production to show actual business impact and
mitigate risks possible
Metrics:
Roles
Execution - ReportingDesignAssess - Planning
Unit testing Systems integration
Functional testing Regression testing
Integration testing User acceptance
System testing Operability testing
Basis & Security testing Performance testing
ABAP Code testing Volume testing
Portal testing Load testing
Management support processes
Test manager Defect specialist
Test architect Test tool specialist
Systems engineer Performance engineer
Business and technical tester Test environment specialist
Test automation specialist Performance test specialist
9 © 2015 IBM Corporation
Test Data Analysts
QACoE Champion
Performance
Engineers
Automation Type 1
Specialists
Automation Type 2
Specialists
Test Tool Specialist
Program
Control Office
Client resource
Vendor resource
Legend
Client / Vendor resource
Testers
Test Data Manager
Test LeadsTest Architect
Test Infrastructure
Prime
Service Virtualization
Configurator
Test Environment
Specialist
QACoE Metrics
QACoE Manager
Sample Proposed QACoE Organization Structure
Business Leads
Business SMEs