copyright © 2001, software productivity consortium nfp, inc. softwar e productivity consortium...

Copyright © 2001, Software Productivity Consortium NFP, Inc.

SOFTWAREPRODUCTIVITYCONSORTIUM

Integrating Information Models:

the essential element of successful measurement

programs

COCOMO Forum October 2004

Chris MillerSoftware Productivity Consortium

2Copyright © 2004, Software Productivity Consortium NFP, Inc.

Integrating Information Models

Motivation

• Popular emphasis on goal-based measurement, but— Anyone can have goals— Goals often are not consistent— Organizational goals may be independent of

project-specific issues

• Measurement initiatives often implemented as disjoint collections of data

• Measurement systems across the enterprise must be aligned to provide a true enterprise perspective

• Quantification of casual relationships provide the basis for predictive analysis and forecasting performance



Sales

CustomerSatisfaction

Market

INNOVATION

TIME-TO-MARKET

PRICE

QUALITY

Information Needs

Enterprise

Process

Project

Product



Enterprise

Process

Project

Product

Types of Information Needs

Measurement should support business goals and objectives

Information needs flow downward

Data and information are passed upward

Business performance

Efficiency and effectiveness of production

Accomplishment of project budget and schedule

Satisfaction of customer requirements



Enterprise View

• Need to measure business performance—Focusing on strategic goals —Reporting profitability (program, business

unit)—Providing information for marketing

• Enterprise may ask questions such as:—Should we invest in a new product line?

• When present, an enterprise policy for managing and measuring drive measurement behavior and actions



Balanced ScorecardP E R F O R M A N C E I N DI C A T O R S

OU T C O M E ME A S U R E S

( lag indicators of mission success )

OU T P U T ME A S U R E S

( lead indicators of goal achievement )I M P R O V E M E N T

I N I T I A T I V E S

S T R A T E G I C

OB J E C T I V E S

( measures of CSFs & progresstoward ultimate ends )

( measures of progresstoward intermediate ends )

C U S T O M E R A N D O T H E R S T A K E H O L DE R

C1 – Responsiveness to trends andopportunities

Market presence & leadership Name recognition

Number of product imitators

Focus Groups

C2 – Customer retention and growth *

Satisfaction “after the sale” Reduction of complaints “Customers First”

I N T E R N A L B U S I N E S S P R O C E S S

I 1 – Organizational alignment Improved core processes Understanding of value added

Mission-driven technology inserts

Process Reengineering

I2 – Operational competency Ability to provide customer-centered information access

Trained Web developers Corporate Web Site

L E A R N I N G , I N N O V A T I O N , R E N E WA L , A N D M A T U R I T Y

L1 – Astute executive decisions Availability, usability ofstrategic information and aids

Competitive information andexpertise gathered and organized

KnowledgeManagement

L2 – Product/servicedistinctiveness

Innovations fielded Breadth and depth of expertise

Intraorganizational coordination

OrganizationalDevelopment

F I N A N C I A L

F1 – Economic sustainability Expansion into new markets

Exposure to risk

Consortium membership

Joint ventures launched

Business Alliances

F2 – Profitability Return on equity High-leverage-point investments ABM

* or market share

*Source: Robert S. Kaplan, and David P. Norton. “The Balanced Scorecard – Measures that Drive Performance,” Harvard Business Review, January 1992.



Process View

• Concurrent processes exist in a typical enterprise (systems, acquisition, development, operations, logistics, marketing, …)

• Focus on the process capability — Definition of processes

– Including measurement points and vehicles— Adherence to defined processes

– Self-audits— Improvement of processes

– Effectiveness– Efficiency



Product View

• The system itself is the primary product—Component quality characteristics—Need to assess impact of changes across the

system components—Customer satisfaction with product

• Products and deliverables—System architecture, specifications,

software, interface control documents, test cases & procedures, users’ guides, …



…Applied Across the Life Cycle

Critical process performance measured in process, during process execution

Product quality characteristics measured and addressed at product transition points

throughout the life cycle

De

fec

tsMaintenance PhasesDevelopment Phases



Project Views

• Focus on the activities and tasks as a project—Meeting product delivery dates—Planning effort —Monitoring and controlling effort

– Adherence to management plan—Customer satisfaction with performance



PSM Analysis Model

Technology Effectiveness

ProcessPerform ance

Product Size and Stability

Resourcesand Cost

Schedule and Progress

ProductQuality

Custom er Satisfaction



Integration of Views

EnterpriseInfo. Needs

ProcessInfo. Needs

ProjectInfo. Needs

ProductInfo. Needs

EnterpriseInfo. Models

ProcessInfo. Models

ProjectInfo. Models

ProductInfo. Models

Systems Eng.Software Eng.

SystemsSoftware

Senior Mgmt Commitment

Pla

n

An

alyz

e

Eva

luat

e

Automate

Common Process

Tailored Support

Level-Specific Information



Integration Activities

1. Each view maintains it’s own set information models for monitoring their own information needs

2. Senior management commitment — Buy-in is needed at the most senior level to

facilitate and drive an integrated measurement program (after all they will benefit the most)

3. Use common base measures— When planning to do measurement use a

standard— Standard consists of common ways to analyze

and evaluate the data and measurement process

4. Build automated tool support around the standard base measures and analysis techniques— Requires tailoring for each view



Step 1: Information Models by View Measurement Information Model

• Defines measurement product

• Links measurable entities to information needs

• Provides rigorous definitions of basic concepts

• Measurement construct consists of :—Base Measures—Derived Measures—Indicators

• Explains prevalence of three-level measurement models (e.g., Goal-Question-Metric [GQM])

Information Model

MeasurementConstruct

Entities

Information Needs

InformationProduct

MeasurableConcept

Attribute



INFORMATION NEEDS

ANALYSIS RESULTS

ANALYSIS RESULTS AND

PERFORMANCEMEASURES

IMPROVEMENTACTIONS

Adapted from ISO/IEC CD 15939, Information Technology – Software Measurement Process

USER FEEDBACK

EstablishCommitment Plan Evaluate

Technical & Management

Processes

Core Measurement Process

ExperienceBase

MEASUREMENTPLAN

Perform

Database

Measurement Process Model



Step2: Buy-in with Business Case

Engineering ManagementProject ManagementExecutive Management

Business Goals

QualityCustomer Satisfaction

Cycle TimeCost

Reduced Rework

Fewer Failures

Lower Price

Lower support costs

Productivity

Increased Capacity

Reduced Effort

Market ShareProfit

Shorter schedules

Time-to-market

Higher MarginLow-cost provider

Retention Referrals

Award fees Bonuses

Image

SPI Activities



Map Goals/Measures/Activities (Cycle Time Example)

Quality

Cycle Time

Reduced Rework

Productivity

Increased Capacity

Process Engineering

Defined Process

Defined Process

Efficient Process

Predictable Schedules

Better Estimates

Stable Requirements

Historical Data

Estimating Model

Measurement Program

Change Control

Requirements Management Defined

Process

Shorter Schedules

(Cust. Sat.)

Reduced Labor (Cost)

Early to Market

Process Mapping

Process Initiatives

Desired Results of Activities

Fewer Steps, Delays

Business Goals,

Objectives

Engineering Analysis, Response



Step 2 Exit Criteria

• Business case needs to:—Obtain senior management commitment —Show information flow down—Highlight key activities

• Provide business-oriented information for each view—Potentially new information needs—Highlight linkage to other views



Step 3: Unify Definitions

• Base measures need to remain consistent across all measurement activities:—Uniform base measures—Data formats—Identify common entities and attributes

• Analysis techniques are tailored to generate unique indicator base on individual information needs:—Define derived measures—Consistent analysis techniques—Identify common reporting standards and

graphic for related indicators



Uniform Base Measures

• Consistent use of —Data labels (e.g.,

SLOC)—Data formats

• Base measures are the lowest common denominator—Validate across all

information models

• Avoid redundant data collection methods

• Maintain integrity of scales and units

Base Measure NameRelease 01.03 Lines of code

AttributeC++ language statements

Measurement MethodCount semicolons in all program units

Type of Measurement MethodObjective

Scale (including precision)Whole numbers from 0 to infinity

Type of ScaleRatio

Unit of MeasureLine of code

Base Measure Definition/Template



Uniform Analysis

• Mitigate redundant naming conventions between indicators — (e.g., Productivity)

• Set standard analysis approach, techniques, and methodologies

• Ensure decision criteria in consistent across information models (and views)

• Use consistent graphic representations for similar data

IDCost Performance Index for Project XX

Information NeedEnter ID of information need

Measures UsedDerived Measure(s)Cost Performance Index (CPI)Base Measures1.Actual Cost of Work Performed2.Budgeted Cost of Work Performed

Function(s)

Calculate CPI:

For any given project, perform the following calculation:

Actual Cost of Work Performed (ACWP)Budgeted Cost of Work Performed (BCWP)

Analysis ModelSimple CPI calculation. Optimal performance is to keep actuals close to or at the center line value of 1.0.

Decision CriteriaExpected value is ‘1.0’. Decision needed if actual value is +/– 10%.

Graphical Representation.Standard CPI line graph with value of 1 represented as a horizontal line, positive values above and negative values below.

CPI =

Indicator Definition/Template



Step 4: Tool Support

• Automated tools support is required to handle the mass of data collection and analysis associated with an integrated measurement program across many views

• Base measure definitions and analysis techniques provide a basis for selection of tool suite

• Understanding the information flow in the business case helps during trade-off decisions among different tools



Summary• An effectively designed measurement program must

consider the interrelationships of the views

• An integrated measurement program will need to establish:— Uniform base measures— Common tools and databases

• Each view uses the base measures to:— Define indicators

• Allows for the flexibility for each view to address their information needs

• Integration is an impossible task without senior management support



Back-up Slides



Interpretation Estimate or evaluation that provides a basis for decision making

Indicator

Model

DerivedMeasure

Value resulting from applying the algorithm to two or more measures

Algorithm combining measures and decision criteria

DerivedMeasure

Operations mapping an attribute to a scaleMethod

BaseMeasure

Function Algorithm combining two or more base measures

Value resulting from applying the method to one attribute

AttributeAttributeProperty relevant to information needs Entities

Information NeedsInformation

Product

Source: Adapted from ISO/IEC 15939, Software Measurement Process Framework

Measurement Construct

Method

BaseMeasure



0.5 > Organization average of 0.3 defects per page Indicator

Model

DerivedMeasure

Compare defect density to SE organizational data

InterpretationCompare defect density to organizational average, perform causal analysis if > 1.0 defects per page

InformationProduct

Indicator summarized for all SE documents in monthly status

Measurement Construct Example

DerivedMeasure

FunctionDivide (defects per page)

0.5 defects per page

AttributeAttributeEntities

Count only bullets in minutes

Method

BaseMeasure

50 defects found

Defects per peer review

Method

BaseMeasure

Count only technical pages

100 pages reviewed

Pages reviewed

SE DocumentPeer Review

Information NeedsSystems Engineering performance – documentation quality?

Source: Adapted from ISO/IEC 15939, Software Measurement Process Framework

copyright © 2001, software productivity consortium nfp, inc. softwar e productivity consortium...

Documents

business goals

goals goals

marketing enterprise

forecasting performance

enterprise policy

measurement points

typical enterprise systems

product products