devtalks.ro - how do we measure software
TRANSCRIPT
How do we Measure Software?RUBEN-DARIUS MOLDOVAN13 MAY 2015 DEVTALKS
HDWM Software Questions• Who measures?
• Why measure?
• Where?
• When?
• How?
• What is ? Why would I invest in your product?To improveTo express qualitative factor by a numberTo estimate costs, schedule, complexity, effort requirements
HDWM Software Agenda – Definitions
Software Definitio
ns
QuestionsTypesComplexityStatus
Metrics
PlanningDeveloping
TestingSupport
Summary &
Questions
HDWM Software Type
Price List
Internal• Shared
HDWM Software Complexity• Business• No. of People working on it
• No. of Customers
• No. of Supported Releases
• Size?
• Technology• Lines of code
• Modules
• Languages
• Symbiosis
• Customization
• Size?
Weighted Methods?
HDWM Software Status• Value• Sales
• Customers
• Quality• Health
• Performance
• Direction• Market
• Competitors
HDWM Software Life• Calendar• Months
• Weeks
• Fiscal• Years
• Quarters
• Cycle• Releases
Planning Specificatio
ns
Development
Design
Testing Validation
SupportMaintenanc
e
HDWM Software Agenda – Metrics
Software Definitio
ns
QuestionsTypesComplexityStatus
Metrics
PlanningDeveloping
TestingSupport
Summary &
Questions
HDWM Software Metrics
Support
Incoming Cases
Days to Resolution
Elevations
Customer Satisfaction
Index
Solved without Help
Development
Requirements
Defects
Enhancements
Escalations
Patents
Lines of Code
Testing
Automation
Performance
Code Coverage
Defect Escape Rate
Security
Financial
Sales / Licenses
Costs
Customers
Net Promoter Score
Other
Ease to Run
Time to Value
Usage Statistics
Maintainability
Mismatch / Consistency
HDWM Software Meaningful• Deliverables
• Process Effectiveness
• Development Time
• Product Quality
HDWM Software Reports
HDWM Software Indicator ModelNo. Meaning
1 This is the metric's value.
2The background color of the cell indicates the status of the metric which can be: "red", "yellow" or "green" color, based on the Thresholds
3 A blue rectangle means that the status changed from red to yellow/green or vice versa.
4
This is the trend of the metric. The arrows can be colored in red, or green if the difference was higher than 20% of the previous quarter. The trend is up if the product improved over previous quarter from a business perspective, not from a mathematical point of view.
5This is the difference from the previous quarter's value in percent. The number is green if the metric improved, or red if it declined.
6 This is the absolute difference from the previous quarter's value.
HDWM Software Indicator ExampleLet's suppose a product has 8 Escalations for the current quarter and 5 Escalations for the previous quarter. The Incident Escalations cell would have the following values:
• 8 as the cell value (count), 2 is the grade • Background color will be red, because Esc grade is 2. • The blue border is present because last quarter the Esc Grade was 3.0 and was
yellow. • The trend arrow is down (current quarter value is worse than previous quarter),
but the difference is more than 20%, so red color is applied. • The percentage difference rounded up to integer number is 60%
and is colored in red because the metric is worse than previous quarter. • Finally, the absolute difference is +3 Incident Escalations
More detailed information is available on every number's mouse over tooltip: values for the selected timeframes, grades, including metric numerators and denominators and history charts.
HDWM Software Support• Incoming Cases• Increase / decrease why?
• Seasonal?
• Solved without help
• Days To Resolution• What cases do we analyze?
• Average over Time? Median?
• Elevations
• Customer Satisfaction Index• Top 2, Top x
• Overall, Product Quality, Support, Documentation, Ease to.. etc
How an improvement looks?How do we improve?What’s the cost of improvement?
HDWM Software Support - DTR• Days To Resolution (DTR)
DTRAvg =
• For Current Backlog = • Considering the Urgent / High severity
• Customer Interaction
• What would you show in a current month?
• High volume vs. Low volume
• Is this seasonal?
HDWM Software Planning• Agile Metrics• Burn Up / Downs
• Velocity
• Feature Status
• Traceability• Defects
• Features
• Tests
HDWM Software Development• Cost per Defect• Critical / Serious Defects• Customer Visible / Customer Encountered• Backlog of Defects / Enhancements
• Escalations
• Defect Escape Rate (Leakage)
• Code Analysis• Length (LOC)• Cost per 1k / Errors per 1k / LOC per engineer
• Complexity
• Re-usability
HDWM Software Testing• Automated Tests• 100% automated tests?
• Ease to (automated) test
• Analysis of Test Results• Performance
• HW/SW Requirements
• Security
HDWM Software Financial• Sales• Licenses
• Support
• Market Share• Net Promotor Score
• Double blind surveys
HDWM Software OtherProduct Management
• Traceability (between Requirements, Test and Defects)
Improvement Programs
• Time 2 Value (T2 Demo, POC, T2 Install, T2 See Results, T2 Profit of Usage, T2 Customization)
• Ease 2 Run (Upgrades, HC to Operate)
HR
• Attrition
• Current (T) Shape or Engineers, Trainings
HDWM Software Usage Statistics• Know the top Features / Modules• and their performance
• Know the top Users• their environment
• their roles
• Attrition
• Current (T) Shape or Engineers, Trainings
HDWM Software Agenda – Summary
Software Definitio
ns
QuestionsTypesComplexityStatus
Metrics
PlanningDeveloping
TestingSupport
Summary &
Questions
HDWM Software Summary• Pre-Release / Post-Release Metrics
• Process / Product - Project / Program Metrics• Size / Function Oriented Metrics
• Customer Metrics• Private and Public Metrics
• What is improvement?• How do we simulate a 5%-20% improvement?
• Usage of the Product Logs• Install base history
• Ecological (Predictive) Metrics• The substance to anchor into the future
HDWM Software Questions
?
HDWM Software
Thank
you!