using tmmi to measure, improve, deliver higher quality ......the goal-question-metric (gqm) approach...
Post on 08-Jul-2020
6 Views
Preview:
TRANSCRIPT
Phone: +1-610-644-2856
Measure. Optimize. Deliver.
softwarevalue.com
Using TMMi to Measure, Improve,
Deliver Higher Quality Software
©2016 DCG Software Value
Topics for Discussion
• What is TMMi?
• The 5 levels of TMMi
• World class performance
• What should you measure?
• The GQM Paradigm
• Popular Metrics
• The missing measure
• A measurement framework
1
©2016 DCG Software Value
What is the TMMi?
2
What is the TMMi standard reference model?
• A staged assessment model
• Compliant with SPICE - ISO 15504
• Can be used to support ITIL and others
• Lower levels can be used in isolation
• Aligns with CMMI; follows similar assessment and ratings process as CMMI
Why use the TMMi?
• Substantial efforts made to
improve the quality of
products
• Despite quality improvement
approaches, the software
industry remains far from
effective and efficient
• Testing accounts for between
30-50% of total project costs
• Encourages defect
prevention over detection
©2016 DCG Software Value
Purpose of the TMMi
• Provide a standard reference model for all industries
• Identify testing strengths and best practices
• Improve integrated test effectiveness and efficiency
• Identify issues and risks
• Identify a program of test process improvements
• Provide public governance, measurement and accreditation
3
©2016 DCG Software Value
A Pathway to Improved Performance
4
Independent model
Integrated best practices derived from over 14 quality and test models
Underpinned by Risk Management and Monitoring & Control
Addresses all test levels
Focused on moving organisations from defect detection to defect prevention
Fit for purpose – appropriate process in the appropriate places
Is the de facto International Standard to measure test maturity
©2016 DCG Software Value
5 Steps to World Class Testing Performance
1. Know your testing performance goals
– Understand what is important to the organization
2. Assess current testing performance against your goals
– You need to know where you are before building a map to where you want to be
3. Develop a plan and close the gap
– Consider a framework/proven methods
– Identify repeatable processes
4. Continuously improve your process
– Determine what is working, what isn’t, and what could be working even better
5. Measure, measure, measure
– Decisions based on fact are always better than assumptions or guessing
5
©2016 DCG Software Value
What Are the Right Things to Measure?
6
Manual
•Test Case Productivity
•Test Execution Summary
•Defect Acceptance
•Defect Rejection
•Bad Fix Defect
•Test Execution Productivity
•Test Efficiency
•Defect Severity Index
Performance
•Performance Scripting Productivity
•Performance Execution Summary
•Performance Execution Data -Client Side
•Performance Execution Data -Server Side
•Performance Test Efficiency
•Performance Severity Index
Automation
•Automation Scripting Productivity
•Automation Test Execution Productivity
•Automation Coverage
•Cost Comparison
Common Metrics
•Effort Variance
•Schedule Variance
•Scope Change
Software Testing Metrics, Lokesh Gulechha, Cognizant, 2014
Software Testing Metrics, P.M. Venhatech Babu, Slideshare
http://www.slideshare.net/pmvenkateshbabu/testing-metrics
©2016 DCG Software Value
Identifying the Right Measures
The Goal-Question-Metric (GQM) approach is a method for developing
and maintaining a goal-driven and purposeful set of metrics.
Benefits of the GQM:
• Links to business/technical goals and needs
• Supports measurement frameworks
• Helps to focus on targeted improvement initiatives
7
©2016 DCG Software Value
The GQM Paradigm
8
©2016 DCG Software Value
The GQM Components
9
©2016 DCG Software Value
Additional Activities
10
©2016 DCG Software Value
A GQM Example
11
©2016 DCG Software Value
Some Popular Metrics
12
©2016 DCG Software Value
What is the Missing Measure?
• Mr. Smith is upset. He is the VP of the Store Systems Division.
• He has just seen the first quarter dashboard of results and his #1
priority project, Store Inventory, has the greatest number of defects.
• Here is what was reported to Mr. Smith:
• You are the development manager. How might you respond to Mr.
Smith? Do we have all the information we need to properly evaluate
these outcomes?
13
Project Delivery Cost
(000's)
Quality
Defects
PO Special On Time $500 12
Vendor Mods Late $760 18
Pricing Adj. Early $80 5
Store Inventory On Time $990 22
©2016 DCG Software Value
Tracking Software Quality
• Size (value) can serve as a normalizing metric.
• A cost-per-unit-of-work (rate) can now be calculated
• “Defect Density” for Mr. Smith’s project is, in fact, the lowest of all of his
projects
14
Project Delivery Cost
(000's)
Quality
Defects
Size
Value
Rate Density
PO Special On Time $500 12 250
$
2,000.00 0.048
Vendor Mods Late $760 18 765
$
993.46 0.024
Pricing Adj. Early $80 5 100
$
800.00 0.050
Store Inventory On Time $990 22 1498
$
660.88 0.015
* Defect Density is calculated as defects / size
©2016 DCG Software Value
Establish a Measurement Framework
15
Project X Project Y Project Z
. .
Process Management
Performance
Measures
Enterprise
Database
Project
Data
Process
MeasuresImprove
Control
Define
Measure
Exec
ute
Process
Measurement
Repository
Service Level
DashboardEngage
Analysis
Collection
Project Defect Status
0
100
200
300
400
500
600
700
800
900
1000
Jan'08
Feb'08
Mar
'08
Apr'0
8
May
'08
Jun'08
Jul'0
8
Aug'08
Sep'08
Oct'08
# o
f D
efe
cts
Total Defects Discovered Total Closed Defects
Requirements Growth and Stability
-50
0
50
100
150
200
Jan'08
Feb'08
Mar
'08
Apr
'08
May
'08
Jun'08
Jul'0
8
Aug
'08
Sep
'08
Oct'08
Nov
'08
Dec
'08
# o
f Requir
em
ents
Added Changed Deleted Total Reqs
Project Resource and Effort Status
0
200
400
600
800
1,000
1,200
1,400
1,600
Jan'08
Feb'08
Mar
'08
Apr'0
8
May
'08
Jun'08
Jul'0
8
Aug'08
Sep'08
Oct'08
Nov
'08
Dec
'08
Pro
ject
Resourc
es/H
ours
Cum Planned Effort Allocated Cum Actual Effort Spent
"Earned Value" Baseline Total Hours
Milestone Baseline Plan Actual
%
Var
Checkpoint A – Charter & Kickoff 1/10/2008 1/10/2008 1/10/2008 0%
Requirements Complete 1/28/2008 1/28/2008 1/28/2008 0%
Vendor Selection Complete 2/4/2008 2/4/2008 2/15/2008 7%
PMP/Schedule Complete 2/12/2008 2/12/2008 2/28/2008 11%
Checkpoint B– Planning & Reqs 2/28/2008 3/15/2008 11%
Design Complete 3/15/2008 4/15/2008 20%
Development Complete 4/15/2008 4/30/2008 10%
Checkpoint C– Midpoint 4/30/2008 5/15/2008 10%
Testing Complete 4/30/2008 5/15/2008 10%
Training Complete 5/10/2008 5/30/2008 13%
Go Live 5/30/2008 6/15/2008 11%
Lessons Learned/Cust Sat Survey Complete 6/1/2008 6/30/2008 19%
Checkpoint D – Deploy & Close 6/1/2008 6/30/2008 19%
Baseline
Performance
Outsource
Governance
©2016 DCG Software Value
Billing Process – Key Quality Measures
16
Billing
Key Performance
Indicators IT
Project
Delivery
Processes
IT Metrics
Program
Project Delivery
Requirements Volatility
Defect Density
Defect Detection
Effectiveness
Defect Removal Efficiency
IT Metrics
Program
Operations
Average daily Bill Input
Average Daily Bill Backlog
Average age of IT Backlog
• % of Accurate Bills
• Delayed Bills %
• Delayed Bills Avg.
Backlog #
• Delayed Bill Avg.
Backlog $$ ($M)
• % of Returned Mail
• Customer
Satisfaction Index
• Cost of Billing per
Customer
IT
Operations
Processes
IT
Measured by
Key Metrics
IT
Measured by
Key Metrics
Bil
lin
g P
rocess
IT Processes
Directly Impact
the Business
Billing
Process
Measured by
KPI’s IT performance
Improvements are
Correlated to improve
Business KPI’s
©2016 DCG Software Value
Contact Us
Email: info@softwarevalue.com
Phone: 610-644-2856
http://www.softwarevalue.com
17
@DCGSWValue
/DCGSoftwareValue
/company/DCG-Software-Valuesoftwarevalue.com
Measure. Optimize. Deliver.
top related