org and sw quality

Influence of Organizational Structure on Software Quality

March 18, 2010

Harmony Brenner

Restricted document

Explain concepts that affect software quality within the organizational structure.

Let’s memorize each component now…

Influence of Organizational Structure on Software Quality Introduction

Restricted document

Explain statistical measurements which affect software quality within the organization.

Let’s memorize each metric…


Restricted document

Just Kidding!


Restricted document

• Purpose

• Case Study References

• Case Study Concepts

• Case Study Lessons Learned

• Guiding Principles

Influence of Organizational Structure on Software Quality Agenda

Restricted document

Influence of Organizational Structure on Software Quality Purpose

Purpose

• Explore relationship between organizational structure and software quality:

• Eight (8) metrics define organizational complexity;

• Metrics empirically evaluate results; and,

• Results predict failure-prone areas.

Restricted document

Influence of Organizational Structure on Software Quality Case Study References

Published by Microsoft about Vista (co-authored):

Nachiappan NagappanMicrosoft ResearchRedmond, WA, [email protected]

Brendan MurphyMicrosoft ResearchCambridge, [email protected]

Victor R. BasiliUniversity of MarylandCollege Park, MD, USA

[email protected]

Restricted document

Influence of Organizational Structure on Software Quality Case Study Concepts

Eight (8) metrics used in study:

1. NOE – Number of engineers

2. NOEE – Number of ex-engineers

3. EF – Edit Frequency

4. DMO – Depth of Master Ownership

Restricted document


Eight (8) metrics used in study (continued):

5. PO – Percent Org contributing to development

6. OCO – Level of Organization Code Ownership

7. OOW – Overall Organization Ownership

8. OIF – Organization Intersection Factor

Restricted document


Metric Definitions

1. Number of engineers (NOE): number of unique engineers who touched code and are still employed by the organization.

2. Number of ex-engineers (NOEE): total number of unique engineers who touched code and left the organization as of release date.

Implications: communications / knowledge transfer

Restricted document


Metric Definitions (continued)

3. Edit Frequency (EF): total number times the source code is edited.

4. Depth of Master Ownership (DMO): edits determine level of ownership (org level of person whose engineers make 75+% of edits = DMO).

Implications: code stability / ownership focus

Restricted document



5. Percentage of Org contributing to development (PO): ratio of people reporting at the DMO level owner relative to the Master owner org size.

6. Level of Organizational Code Ownership (OCO): percent of edits from organization that contains code owner / who made majority of edits.

Implications: measures Organization code control

Restricted document



7. Overall Organizational Ownership (OOW): ratio of percentage of people at DMO level making code edits relative to total engineers making edits.

8. Organization Intersection Factor (OIF): measure of number of organizations contributing 10%+ of code edits (measured by overall org owners).

Implications: Organizational distance cohesion vs. misunderstandings

Restricted document

Influence of Organizational Structure on Software Quality Case Study Lessons Learned

Lessons Learned – Prior Comparisons

• Other prediction model metrics:

• Code churn (total, frequency, repeat freq.)

• Code complexity (LOC, coupling, depth)

• Dependencies (incoming vs. outgoing)

Restricted document



• Other prediction model metrics (continued):

• Code coverage (block / arc coverage)

• Pre-release defects (number of pre-release defects found to predict post-release defects)

Restricted document



Model Precision Recall

Organizational Structure 86.2% 84.0%

Code Coverage 83.8% 54.4%

Code Complexity 79.3% 66.0%

Code Churn 78.6% 79.9%

Dependencies 74.4% 69.9%

Pre-Release Defects 73.8% 62.9%

Restricted document


Lessons Learned (People):

• NOE (Number of Engineers) – more people who touch the code, the lower the quality.

• NOEE (Number of Ex-Engineers) – large team member loss affects knowledge retention / quality.

Restricted document


Lessons Learned (People) – continued:

• EF (Edit Frequency) – more edits to components, higher the instability / lower quality.

• DMO (Depth of Master Ownership) – lower level of the ownership, better is the quality.

Restricted document


Lessons Learned (Organization):

• PO (Percentage of Org contributing to dev) – more cohesive are the contributors (organizationally), higher the quality.

• OCO (Level of Org Code Ownership) – more cohesive contributions (edits), higher the quality.

Restricted document


Lessons Learned (Organization) – continued:

• OOW (Overall Org Ownership) – more diffusion of code contributions, lower the quality.

• OIF (Org Intersection Factor) – more diffusion of different groups contributing code, lower the quality.

Restricted document

Influence of Organizational Structure on Software Quality Guiding Principles

Intuitive Ways to Increase Quality:

• Control / reduce code edits fix errors earlier (before code creation).

• Strive to control personnel transition and retention.

• Build team spirit and foster ownership at the right organization levels.

Restricted document

Influence of Organizational Structure on Software Quality Questions

Questions

org and sw quality

Business