04/19/23 1

Metric Based Software Project Management

Ashkan Sami, Ph. D.

Department of CSE and IT

Shiraz University

asami@ieee.org

sami@shirazu.ac.ir

Concerned with activities involved in ensuring that software is delivered on time and on schedule and in accordance with the requirements of the organisations developing and procuring the software.

Project management is needed because software development is always subject to budget and schedule constraints that are set by the organisation developing the software.

Software project management

The product is intangible. The product is uniquely flexible. Software engineering is not recognized as an

engineering discipline with the sane status as

mechanical, electrical engineering, etc. The software development process is not

standardised. Many software projects are 'one-off' projects.

Software management distinctions

Proposal writing. Project planning and scheduling. Project costing. Project monitoring and reviews. Personnel selection and evaluation. Report writing and presentations.

Management activities

04/19/23 5

Software Project Management Start of the project Build the Macro Model Build the Detailed Project Plan Build the Project Team How to Run the Project on a Day-to-Day

Basis Monitoring and Controlling the Project

– Management style– Performance Management– Performance Analysis– Interfaces– Systems

Successfully Shutting Down the Project

Monitoring and Controlling the Project

To monitor and control effectively:• your management style;• management of project performance;• analysis of project performance;• how you will manage the project

interfaces;• what systems you are going to use.

04/19/23 6

MANAGEMENT STYLE Three basic styles of management:

– the dictatorial style, – the consensus style and – the laissez-faire style.

A good project manager should be able to recognize when a particular style is appropriate.

You should be able to adopt that style when it is required.

04/19/23 7

04/19/23 8

04/19/23 9

PERFORMANCE MANAGEMENT Performance and its control is one of the

areas on which you should spend a significant amount of time.

Sadly, measuring how successful a project is can be difficult, particularly the success of an advanced project.

It is difficult to measure success because advanced projects tend to be unique projects. This means that little in the way of previous performance measures exists.

04/19/23 10

Quality Management Software Quality Standards

– ISO– CMM

Quality Planning– Cost Benefit Analysis– Benchmarking– Cost Of Quality– Quality Management Plan

Quality Assurance– Metrics Measurements– Corrective Actions– Preventive Actions

Quality Control– Audits, Analysis– Checklist

04/19/23 11

04/19/23 12

Software Project Quality Management The Activities that Determine Quality POLICIES,

OBJECTIVES, RESPONSIBILITIES. It includes the PLANNING, ASSURANCE and

CONTROL of the POLICIES, PROCEDURES and PROCESSES

Forecast the End Result Quality during the Development Stages

Special Characteristic of Software:– Increasing Criticality and Importance of Software– Intangibility of software is a source for Unpredicted Errors– Software Errors are accumulating through the project life

cycle

04/19/23 13

Software Product Quality Factors Product Operation:

– Correctness (follow specifications), – Reliability (number of errors, fault tolerance, simplicity), – Efficient (optimize resources usage: storage, memory, CPU) – Integrity (access control & audit to processes and data)– Usability (ease of use/training, operation)

Product Revision (changes, enhancements)– Testing (effort required, regression tests)– Flexibility (effort required to modify the functionality)– Maintenance (effort required to locate and fix a bug:

modularity, no duplication of code, documentation) Product Flexibility (Transition)

– Portability (change h/w, operating system, compiler, etc)– Interoperability (integrate with other systems, flexible data and

interface structures)– Reusability (can be used as part of another application as a

whole or parts – platform free, modular design)

04/19/23 14

Software Quality Standards ISO 9001 (International Organization for Standardization)

– Defines the needs from a monitor & control system to check quality

– Certification of the Design, Development, Production, Installation an Service Processes

– Describes the fundamental features of Quality Management System (QMS)

– Useful in selecting a sub-contractor with “best practices” quality processes.

– Deals with quality of the development process, not with the quality of the product

04/19/23 15

ISO 9001 Principles– Determine the NEEDS and Expectations of the customer– Establish Quality Policy and imply the actual quality objectives– Design the project activities to include the quality objectives– Assign responsible parties to all the quality objectives– Allocate enough and knowledgeable resources– Implement methods to measure the quality objectives in each

process– Collect & Analyze the Measurements and Identify discrepancies– Define action items to eliminate the cause of the discrepancies– Documentation (follows quality manual) of the actual (updated)

operation of an activity that includes objectives, plans, procedures and records

– The QMS is well managed and knowledgeable resources are assigned to the Quality Management Process.

– Demonstrate that the Production Process is well defined, designed, recorded, communicated and measured.

– Sub-Contractors and Purchasing are managed in the same manners.

04/19/23 16

CMM – Capability Maturity Model Software Development Methods and Tools which are Likely to

produce Quality Software Software Companies are assigned a level ( 1 to 5 ) of Process

Maturity that indicates the quality of their software production practices

Level 1: Initial Level– Default level, no defined quality process throughout the

organization. Some projects may adopt some measures. Level 2: Repeatable Level

– Basic Project Management in place, not in the activities level Level 3: Defined Level

– Project Management Plan is well defined at all levels Level 4: Managed Level

– Products & Processes are Measured and Controlled Level 5: Optimizing Level

– Process Improvement is introduced based on documented data gathered from previous projects, processes.

17

Causal Analysis and ResolutionOrganizational Innovation and Deployment5 Optimizing

4 Quantitatively Managed

3 Defined

2 Managed

Quantitative Project ManagementOrganizational Process Performance

Requirements DevelopmentTechnical SolutionProduct IntegrationVerificationValidation Organizational Process FocusOrganizational Process DefinitionOrganizational Training Risk ManagementIntegrated Project Management (for IPPD*)Integrated Teaming*Integrated Supplier Management**

Decision Analysis and ResolutionOrganizational Environment for Integration*

Requirements Management Project PlanningProject Monitoring and ControlSupplier Agreement Management Measurement and AnalysisProcess and Product Quality AssuranceConfiguration Management

1 Performed

Process AreasLevel

Work proactively managed, organizational standard processes

Improving, addressing common causes of variation

Quantitatively managed, eliminating special causes of variation

Work planned and tracked (reactively managed)

Work performed, but in an ad hoc fashion

CMMI Levels

Slide 18 of 146

CMMI Staged Representation - 5 Maturity Levels

Level 5

Initial

Level 1

Processes are unpredictable, poorly controlled, reactive .

Managed

Level 2

Processes are planned, documented, performed, monitored, and controlled at the project level. Often reactive.

Defined

Level 3 Processes are well characterized and understood. Processes, standards, procedures, tools, etc. are defined at the organizational (Organization X ) level. Proactive.

Quantitatively Managed

Level 4

Processes are controlled using statistical and other quantitative techniques.

Optimizing

Proce

ss M

atur

ity

Process performance continually improved through incremental and innovative technological improvements .

Slide 19 of 146

Maturity Level 1 Initial

Maturity Level 1 deals with performed processes.

Processes are unpredictable, poorly controlled, reactive.

The process performance may not be stable and may not meet specific objectives such as quality, cost, and schedule, but useful work can be done.

Slide 20 of 146

Maturity Level 2Managed at the Project Level

Maturity Level 2 deals with managed processes. A managed process is a performed process that is also:

– Planned and executed in accordance with policy– Employs skilled people– Adequate resources are available– Controlled outputs are produced– Stakeholders are involved– The process is reviewed and evaluated for adherence to

requirements Processes are planned, documented, performed, monitored,

and controlled at the project level. Often reactive. The managed process comes closer to achieving the

specific objectives such as quality, cost, and schedule.

Slide 21 of 146

Maturity Level 3Defined at the Organization Level Maturity Level 3 deals with defined processes. A defined process is a managed process that:

– Well defined, understood, deployed and executed across the entire organization. Proactive.

– Processes, standards, procedures, tools, etc. are defined at the organizational (Organization X ) level. Project or local tailoring is allowed, however it must be based on the organization’s set of standard processes and defined per the organization’s tailoring guidelines.

Major portions of the organization cannot “opt out.”

Slide 22 of 146

Behaviors at the Five Levels

Initial

Managed

Defined

QuantitativelyManaged

Optimizing

Process is unpredictable,poorly controlled, and reactive

Process is characterized for projects and is oftenreactive

Process is characterizedfor the organization andis proactive

Process is measuredand controlled

Focus is on continuousquantitative improvement

Maturity Level Process Characteristics Behaviors

Focus on "fire prevention;"improvement anticipated and

desired, and impacts assessed.

Greater sense of teamwork and inter-dependencies

Reliance on defined process. People understand, support and follow the process.

Over reliance on experience of goodpeople – when they go, the processgoes. “Heroics”.

Focus on "fire fighting;"effectiveness low – frustration high.

Slide 23 of 146

CMMI Components

Within each of the 5 Maturity Levels, there are basic functions that need to be performed – these are called Process Areas (PAs).

For Maturity Level 2 there are 7 Process Areas that must be completely satisfied.

For Maturity Level 3 there are 11 Process Areas that must be completely satisfied.

Given the interactions and overlap, it becomes more efficient to work the Maturity Level 2 and 3 issues concurrently.

Within each PA there are Goals to be achieved and within each Goal there are Practices, work products, etc. to be followed that will support each of the Goals.

Slide 24 of 146

Maturity Level Project Managment Engineering Process Management Support5

OptimizingOrganizational Innovation & Deployment Causal Analysis & Resolution

4Quantitatively

Managed

Quantitative Project Mngt Organizational Process Performance

3Defined

Integrated Project MngtRisk Management

Requirements DevelopmentTechnical SolutionProduct IntegrationVerificationValidation

Organizational Process FocusOrganizational Process DefinitionOrganizational Training

Decision Analysis & Resolution

2Managed

Project PlanningProject Monitoring & ControlSupplier Agreement Mngt

Requirements Mngt Measurement & AnalysisProcess & Product Quality AssuranceConfiguration Mngt

1Initial

CMMI Process Areas

25

State of the Software Industry in 1995

Maturity Level Frequency

1. Initial 70%

2. Repeatable 15%

3. Defined < 10%

4. Managed < 5%

5. Optimizing < 1%

Source: Royce, Project Management, P. 364

04/19/23 26

با تشکر از توجه شما

تقدیم به روح پر فتوح یکی از اولین فارغ التحصیالن و اساتید

دانشگاه علوم پزشکی

دکتر منصور سامی

Quality Planning

04/19/23 27

Identify the Project Quality Standards Determine how to Satisfy the Standards Inputs:

– Enterprise Factors (law, regulations, standards)– Organizational Factors ( QA policies)– Scope Statement (objectives, thresholds, acceptance criteria,

performance criteria, etc.)– Project Management Plan

Tools & Techniques– Cost Benefit Analysis

• Cost Of Quality: Expenses associated with the Project Quality Management

• Benefit: Less Rework, higher productivity, lower cost, higher stakeholder satisfaction

– Benchmarking Compare the project process practices to other projects to get ideas for improvements

– Others: Brainstorming, Flowcharts

Quality Planning

04/19/23 28

Identify the Project Quality Standards Determine how to Satisfy the Standards Inputs:

– Enterprise Factors (law, regulations, standards)– Organizational Factors ( QA policies)– Scope Statement (objectives, thresholds, acceptance criteria,

performance criteria, etc.)– Project Management Plan

Tools & Techniques– Cost Benefit Analysis

• Cost Of Quality: Expenses associated with the Project Quality Management

• Benefit: Less Rework, higher productivity, lower cost, higher stakeholder satisfaction

– Benchmarking Compare the project process practices to other projects to get ideas for improvements

– Others: Brainstorming, Flowcharts

04/19/23 29

Quality Planning - Output Quality Management Plan

– How the project management will implement the quality policy– Quality Control Plan– Quality Assurance Plan– Continuous Improvement Plan

Quality Metrics– Detailed Plan of what will be measured, actual values

Quality Checklist– Structured tool. Process, Activity based. Templates can be

available from prior projects or QA organizations. Quality Baseline

– The quality OBJECTIVES of the project– The basis for measuring and reporting

Process Improvement Plan– Identification of WASTE and NON-VALUE ADDED activities– Set TARGETS for Improved performance

04/19/23 30

Quality Assurance - QA A QA Department in the Organization Observe that the project will employ all processes

needed to meet Quality Targets QA support & Knowledge Base Provides Umbrella for Continuous Process

Improvement Recommend Corrective Actions and Updates to the

Project Plan Update Organizational Process Assets

04/19/23 31

Software Metrics“You Can Not Control What You Can Not Measure” Retain Records (historical data) – Used to evaluate

future projects (sizing,schedule, cost, effort) Problem Reporting – Supply Information about

problem fixing (cycle, time, amount, frequency, etc.) Quality Assurance Plan – Define Quality Thresholds

(What is the definition of “Low System Complexity”) Corporate Quality Goals – Measure progress toward

the Goals Types of Software Metrics

– Directly Observed: Number of Test Cases, Project Cost– Predicted: Project Baseline Cost, Estimate To Complete– Calculated: Productivity=KLOC/Staff-Months

04/19/23 32

Useful Metrics

Precise or Defined with Clear Tolerance, Well Defined Methods of Data Collection Helps to Measure the Organization/Project Quality

Goals Simple & Easy to understand Inexpensive to Use Robust Consistent and Used Over Time Easy To Collect Easy to Access (online)

04/19/23 33

Software Metrics – Examples of Direct Observations

Control Chart - Upper/Lower Control Limit ( Usually the Corporate or Project Quality Targets)– Actual Measures per Module/Activity VS Corporate Goals.

The Average Project Measurement implies the overall project compliance. Deviation from the control limits indicate problem in a specific Module/Activity

– Typical Metric: Design Review Hours Invested per Module. Total Hours to investigate and resolve problem per

predefined problem types (interface, documentation, input data, computational, etc.)

Defects Discovered and Defect Fixed Over Time Mean Age of: Open Problems at the end of the Month

Closed Problems at Month End

04/19/23 34

Continuous Process Improvement Main Goals:

– Continuous Improvement in Resource Planning (increase the probability to deliver accurate estimates)

– Continuous Reduction in Waste/Non-Productive Project Activities.

– Continuous Improvement in Other Corporate Quality Goals Values in Software Activities

– Value Added: Design, Code, Test, Install, Etc. – Non Value Added but Essential: Acceptance Testing, Setup,

Training, Etc.– Non Value Added and Nonessential: Retest, Rework,

Management, Installing Software Tools, Delays, Etc. Value Metrics: Value/Total, Non-Value Essential/Non-

essential, Value+Essential/Total

04/19/23 35

Quality Control - QC Monitor SPECIFIC project results to determine its

compliance with the Quality Standards Identify ways to eliminate CAUSES of unsatisfactory

results Continuous Process Uses Sampling and Probability analysis Analysis Types

– Prevention & Inspection– Sampling (check conformance (Yes/No) to standards, goals,

Etc)– Variables Sampling – Level Of Conformance with

Standards/Goals– Special Causes/Unusual Events – Tolerance (results fall within/outside the control limits)

04/19/23 36

QC – Tools & Techniques Cause & Effect Diagram

– Per Major Defect (E.G: Users Interface Issues)– Cause: Guidelines not Followed

• Reasons: Did not read the guidelines(no time), Can not find the guidelines (No central Location)

– Cause: Lack of users feedback• Reasons: Lack of users resources, Prototype not clear

enough, No process to get early feedbacks Control Charts Flowcharting (activities, decision points, sequencing) – Can

point potential quality issues Histogram – Number of Defects per Problem Type Pareto Chart – 80/20 rule. Order Defects by Number of Defect

and focus on those that causes the majority

04/19/23 37

QC – Tools & Techniques (Cont.) Run Charts/Scatter Diagram (indicate trends over time)

Examples:– Failure rate (defect found per test hour) over Cumulative test

time– New Defects/Resolved defect per week over weeks

Statistical Sampling Inspection Defect Repair Reviews