A Comparison of Best Practices

8th Malaysian Software

Engineering Conference

(MySEC 2014)

Resort World Langkawi,

MalaysiaRelease Readiness Measurement

24-09-2014

Nico Koprowski, M. Firdaus Harun, Horst Lichter

(Universiti Teknologi Malaysia)

• Release Readiness

• Measurement for Release Readiness

Defect Tracking

Software Readiness Index

ShipIt

• Discussion

• Comparison

• Summary

Introduction

2

Schedule Cost

Software Readiness

When to release the software product?

Release Readiness

3Time

Rele

ase R

ead

iness

Time of release?

100% finished

Is a software product ready to be released?

Defect Tracking

4

Q: When is a software product ready to be released?

Defect Tracking: When the number of remaining defects is sufficently low!

Reliability:

The probability of executing a software system without failure for a

specified time period.

Defect Density (DD)

5

LOC: 100K LOC: 50K LOC: 100K

Defects: 700 Defects: 475 Defects: 600

History

Present

LOC: 140K

Defects / DD: ?

DD: 7/KLOC DD: 9.5/KLOC DD: 6/KLOC

V1.0 V2.0 V3.0

Defect Pooling

6

Team A Team B

Common findings

Unique findingsUnique findings

The more unique findings, the more defects remain

Defect A Defect B

Defect Seeding

7

Seed Detect

Team ATeam B

Idea: Remaining defects rate proportional to seeded defects detected rate

Software Product

Architectural Defect Tracking (ADF)

8

Presentation Tier

Business Tier

Data Access Tier

N-tier architecture: Parameters:

• # UIs

• # UI messages

• # Parents

• # Children

• Depth of Inheritances

• Coupling

• …

• # Selects

• # Insert/Updates

• # Deletes

• # Sub-queries

• …

User

Interfaces

Classes

SQL

Software Readiness Index (SRI)

9

Q: When is a software product ready to be released?

SRI: When it obtains the desired amount of quality!

QualityReliability

Functionality Efficiency

Usability

…

Quality

SRI Criteria

10Taken from: A.Asthana and J. Oliveri. Quantifying software reliability and readiness.

Thresholds

11

Green

Yellow

Red

Criterion

Measurement

0%

100%

Good

Ok, Sufficient

Bad, Not tolerable

ShipIt

12

Q: When is a software product ready to be released?

ShipIt: When the overall development progress is sufficiently advanced!

Requirements Design Testing

Time

Progress:The ratio of the already spent effort to the overall planned effort

ShipIt Criteria

13

Requirements

Coding

Testing

Quality

Documentation

Supervision

Support

Gathered, Analysed and Designed

Modules, Objects codedBuild Times

Test CoverageOpen issues

Zero Failure Test hours COCOMO

Requirements, Design, Code Test Plan, User Guide

Installation and Training

Beta Test Bugs

Discussion: Defect Tracking

14

Scope

Simplicity

AvailabilityUniversality

Concreteness

Defect Density

Defect Pooling/Seeding

Architectural DefectTracking

• Concrete

Methods

• No support with

decision making

ADF only for n-tier

architectures and OOE

Simple methods

and criteria

Only Reliability

When Testing

Discussion: SRI

15

Scope

Simplicity

AvailabilityUniversality

Concreteness

Criteria with thresholds

No restrictions When Coding

• Many criteria

• But: light-weight

version available

Quality including Reliability

Discussion: ShipIt

16

Scope

Simplicity

AvailabilityUniversality

Concreteness

• Some criteria hard

to measure

• Fuzzy about

quality

From beginning of

the project

Many criteria and

subcriteria

Whole Progress including

Quality and Reliability

Only suitable for

waterfall model

Comparison

17

Defect Tracking SRI ShipItFits in Fits in

• Least release

criteria

• Used in final steps

• ADT where

applicable

• Universally

applicable

• Most comprehensive

and concreteness

• Thresholds

• Broadest scope

• Good for progress

communication

• Less concrete

AvailablityUniversality

Concreteness Simplicity

Scope

Summary

18

Question: Is the software product ready to be released?

Suggestion: Quantify release readiness properties via metrics

(applicable for all software designs) and project progress (any

software development types) i.e. Holistic approach.

1. Reliability measurement with Defect Tracking: Least

criteria

2. Quality measurement with SRI: Most comprehensive

approach

3. Progress measurement with ShipIt: Most complex

approach

The End

19

Thanks for your attention!

Defect Tracking SRI ShipIt

• Least release

criteria

• Used in final steps

• ADT where

applicable

• Universally

applicable

• Most comprehensive

and concreteness

• Thresholds

• Broader scope

• Good for progress

communication

• Less concrete

AvailablityUniversality

Concreteness Simplicity

Scope