introduction to bugs measurement
DESCRIPTION
TRANSCRIPT
Introduction to Bugs Measurement
by V. Novostavsky
Yes, it’s about metrics
TOC
• What we’re talking about• Why it's good, and when it’s evil• “Easy way”
Definition
• Distance function– from “anything” to numbers :)
Metrics in life: School
Metrics in life: Traders
Metrics in life: dou.ua
… in testing
– Usually we measure bugs• because there’re a lot of bugs
We can measure barely anything in testing, but that’s another story…
… in testing
– Number of open bugs– Bugs per module– Bugs reopened– Bugs posted in a week– Bugs rejected / accepted – A lot of other stuff…
… in testing
– Number of open bugs– Bugs per module– Bugs reopened– Bugs posted in a week– Bugs rejected / accepted – A lot of other stuff…
Why it's important
• tester's job is info support for the project
Why it's important
• tester's job is info support for the project• more and diverse info -> better conclusions ->
easier decisions
Why it's important
• tester's job is info support for the project• more and diverse info -> better conclusions ->
easier decisions• metrics are facts, operate with facts, not blah-
blah-blah
Why it's important
• tester's job is info support for the project• more and diverse info -> better conclusions ->
easier decisions• metrics are facts, operate with facts, not blah-
blah-blah• example: “big” manager
Why it's important
• tester's job is info support for the project• more and diverse info -> better conclusions ->
easier decisions• metrics are facts, operate with facts, not blah-
blah-blah• example: “big” manager
• NOTE: don't judge too quickly!
Example
500 open bugs this week… A lot of?
Example
Example
More bugs in a module… Does it mean it’s worse?
Example
Example
Less bugs this week… Is it good?
Example
Top issues
Top issues
• time waste!
Top issues
• time waste!
• wrong conclusions and politics!
Top issues
• time waste! make it easier to generate metrics
• wrong conclusions and politics!
Top issues
• time waste! make it easier to generate metrics
• wrong conclusions and politics!make it clear what they mean
“Easy way”
• Step 1: choose correct metrics for you
“Easy way”
• Step 1: choose correct metrics for you– Not so easy!
“Easy way”
• Step 2: ensure everybody gets what they mean
“Easy way”
• Step 3: automate them (+ use SQL)
“Easy way”
• Step 4: be prepared for change requests
“Easy way”
• Step 5: use often
“Easy way”
• Step 6: publish online
“Easy way”
• Step 6: publish online– Publish charts!
“Easy way”
1. choose correct metrics for you2. ensure everybody gets what they mean3. automate them4. be prepared for change requests5. use regularly6. publish online
Example: my metrics
• No classic test cases & project is not big– Bug metrics only
• Custom bugtracker– No access to DB
• 2 metrics - weekly– Open bugs: Trend in time– Open bugs: By modules
Example: tools
• MS XLS –> MS SQL
Example: tools
• MS XLS –> MS SQL
• Google Docs
Example: my trender
Example: my rollup
Thanks!
Q & A