1

Lecture 4.3a: Metrics Overview(SEF Ch 14)

Dr. John MacCarthyUMBC CMSC 615Fall, 2006

2

(Program) Metrics in Management

(Program) Metrics are (quantitative) measurements collected for the purpose of determining project progress and overall condition

Metrics are used to manage risk (technical, cost, and schedule) Metrics are collected and reported periodically (to management and

the customer) Metrics are measured over time so that changes (trends) can be

observed

Technical Management requires three types of Program Metrics: Product Metrics: track the development of the product Earned Value: tracks conformance to the planned schedule and costs Management Process Metrics: track management activities

DoD Policy requires the identification and tracking of TPMs

3

Product Metrics [1] Product Metrics: Demonstrate

progress toward meeting customer requirements

Three Types: Operational Performance Life-cycle Suitability Affordability

Technical Performance Measurements (TPMs):

A set of (5-20) key product performance metrics selected for tracking and periodic reporting to management.

Derived from MOPs Generally allocated to CIs

Metric Hierarchy: Measures of Effectiveness

(MOEs): high-level measures of a system’s ability to achieve operational mission success (See CDD KPPs)

Measures of Suitability (MOSs): high-level measures of the extent to which the system integrates will into the operational environment. They generally include supportability, human interface compatibility, and maintainability

Measures of Performance (MOPs): high-level physical or functional attributes of the system that relate (trace) to the MOEs and MOSs.

4

Product Metrics [2] Suitability Metrics:

Degree to which a system can be placed satisfactorily in field use:

Availability, Reliability, Maintainability (RAM)

Usage Rates Supportability & Logistics Human Factors, Manpower,

Training, & Safety Interoperability, compatibility,

transportability Environmental Impact Robustness, Survivability Producibility, Testability,

Simplicity …

TPMs often include suitability metrics

MTBF MTTR

Affordability Metrics: LCC drivers:

(Development Costs) Production Costs Operational Costs Maintenance Costs Disposal Costs

5

Earned Value (14.2) Earned Value is the standard metric

for tracking cost and schedule progress and risk

See Separate Briefing on Earned Value

6

Process Metrics (14.3) Management Process Metrics: measurements taken to track

process of developing, building and introducing the systems: Examples include:

Availability of resources (personnel) Number of new hires Items Completed Customer Satisfaction Trouble Tickets Lines of Code (developed) Complexity ECPs baselined Time to approve documents Number of High Risks Etc.

These are important for CMMI

7

Summary Points (14.4) Technical Management requires three types of metrics

Product Metrics Earned Value Management Process Metrics

Metrics should be reported on a periodic basis The measurement, evaluation, control, and reporting of

metrics should be planned to assure proper measurement, evaluation and dissemination

TPMs are performance-based product metrics (key technical parameters) that are used to track progress toward meeting the program’s operational requirements

Use of TPMs is required by DoDI 5000.2

8

Technical Performance Measurement (S14-A) [1]

See separate TPM briefing