cosatmo, cocomo iii, and cosysmo: developing next-generation cost models

University of Southern California

Center for Systems and Software Engineering

COSATMO, COCOMO III, and COSYSMO:

Developing Next-GenerationCost Models

Jim Alstad, PhD Candidate

USC Center for Systems and Software Engineering

CS 510 Lecture

September 17, 2014

09/16 1



Next-Gen Cost Models

09/16 2

• Learning objectives for CS 510 lecture:– To provide a quick view of how an estimating model is

developed– To describe some of the current research in cost model

development



Next-Gen Cost Models Agenda

09/16 3

Agenda:•Learning goals for 510 lecture•A brief history of the COCOMO family of estimating models•COCOMO III•COSYSMO 3.0•COSATMO



COCOMO II and Some Derivatives

• 1995: one-size-fits-all model for 21st century software

• 1999: poor fit for schedule-optimized projects; CORADMO

• 2000: poor fit for COTS-intensive projects: COCOTS

• 2003: need model for product line investment: COPLIMO

• 2003: poor fit for agile projects: Agile COCOMO II (partial)

• 2012: poor fit for incremental development: COINCOMO

09/16 4



COQUALMO1998

COCOMO 811981

COPROMO1998

COSoSIMO2007

Legend:Model has been calibrated with historical project data and expert (Delphi) data

Model is derived from COCOMO IIModel has been calibrated with expert (Delphi) data

COCOTS2000

COSYSMO2005

CORADMO1999,2012

iDAVE2004

COPLIMO2003

COPSEMO1998

COCOMO II2000

DBA COCOMO2004

COINCOMO2004,2012

COSECMO 2004

Software Cost Models

Software Extensions

Other IndependentEstimation Models

Dates indicate the time that the first paper was published for the model

COTIPMO2011

AGILE C II2003

COCOMO Family of Cost Models

09/16 5



Determine Model Needs

Step 1

USC-CSSE Modeling Methodology

Analyze existing literature

Step 2Perform Behavioral analyses

Step 3Define relative significance,data, ratingsStep 4

Perform expert-judgment Delphi assessment, formulate a priori modelStep 5

Gather project data

Step 6

Determine Bayesian A-Posteriori modelStep 7 Gather more data;

refine model

Step 8

- concurrency and feedback implied

09/16 6




09/16 7


Key slides provided by Barry Boehm, Vu Nguyen, and Brad Clark (CSSE 2014 Annual Research Review)



COCOMO II Data by 5-Year Periods

09/16 8

COCOMO II Calibration

Add in new data when calibrating



COCOMO II Data: Productivity Trends

09/16 9COCOMO II Calibration

Calibrate to new productivity values



Application Experience (APEX)Rating Trends

09/16 10

COCOMO II Calibration

Very High is>= 6 years

Reduce VH (etc) definition to cover same % of projects



Super-Domains and AFCAA Productivity Types

09/1611

Super Domain Productivity Types

Real-Time

(RT)

1 Sensor Control and Signal Processing

2 Vehicle Control

3 Vehicle Payload

4 Real Time Embedded-Other

Engineering(ENG)

5 Mission Processing

6 Executive

7 Automation and Process Control

8 Scientific Systems

9 Telecommunications

Mission Support

(MS)

10 Planning Systems

11 Training

12 Software Tools

13 Test Software

Automated Information System

(AIS)

14 Intelligence and Information Systems

Software Services

Software Applications



More on Application Domains• COCOMO III plans to identify a set of application

domains, and identify and calibrate a submodel to each domain

• Allows user to identify domain, and get a group of accordingly-set cost driver values off the shelf

09/16 12



COCOMO III Directions• CSSE’s April Annual Research Review discussions

set these directions for COCOMO development:– 149 new data points have been added since the 161 data

points used in COCOMO II.2000.• Add these data points

– The average level of productivity appears to have increased• Therefore recalibrate the model’s parameters

– The average level of several cost drivers appears to have changed

• Consider changing the definitions of the high/medium/low levels for these cost drivers

– More data points might add to variability in the data• Consider adding more cost drivers to improve the fit

– Simplifying the model would make it easier to use• Define (calibrated) submodels for certain important application

domains that provide off-the-shelf values for some cost drivers• Planned domains: Real-time, Engineering & Scientific,

Command and Control, Automated Information Processing09/16 13




09/16 14




What Is Systems Engineering?

09/16 15

• Some clauses of a definition:– Considers the system as a whole– Starts from stakeholder needs, results in a technical

approach meeting those needs• Frequently develops system requirements from system needs• Technical approach is commonly stated as the system

architecture– Systems engineers typically need to have familiarity with all

the specific kinds of engineering involved in their type of system

– Systems engineers can usually be found working on systems that are large, critical, and/or important

– Participation in identifying all major risks and reducing/managing them (ICSM)

– Typically most system engineering is done in the Valuation phase, with major work also being done in the Exploration and Foundation phases (ICSM)

– See also ICSM book section 0.2.1• The cost of system engineering needs an estimation

model



History of COSYSMO Models

09/16 16

COSYSMO 1.0Valerdi, 2005

• Identifies form of model• Identifies basic cost drivers• Identifies Size measure

Req’ts VolatilePena, 2012

• Adds scale factor based on requirements volatility

With ReuseFortune, 2009

• Adds weights to Size elements, reducing net Size in the presence of reuse

For ReuseWang et al, 2014

• Adds weights to Size elements, reducing net Size when artifacts are only partially completed

Sys of SysLane et al, 2014

• Adds effort multiplier when in the presence of system-of-systems

COSYSMO 3.0Alstad, 2015?

• Combines features of previous models



Basic COSYSMO (1/2)• COSYSMO [2] starts by computing the “size” of a

system engineering project, in units of eReq (“equivalent nominal requirements”)

• These artifacts are considered in the size: system requirements, system interfaces, system-critical algorithms, and operational scenarios.

• Each artifact is evaluated as being easy, nominal, or difficult.

• Each artifact is looked up in this size table to get its number of eReq, and then these are summed to get the system size:

09/16 17

Artifact Type Easy Nominal Difficult

System Req’ts 0.5 1.0 5.0

System Interfaces 1.1 2.8 6.3

System Algs 2.2 4.1 11.5

Op Scenarios 6.2 14.4 30.0



Basic COSYSMO (2/2)

• Size is raised to an exponent, representing diseconomy of scale, and then multiplied by factors for 14 effort multipliers and a calibration constant.

• This results in the following equation for a COSYSMO estimate of effort in person-months:

09/16 18



What Is Systems Engineeringfor Reuse?

• Systems Engineering for Reuse produces artifacts intended for later reuse on projects. A completed SEFR artifact may (intentionally) not be completely developed, so that it will be in one of these SEFR states:– Conceptualized for Reuse (e.g., Concept of Operations

document)– Designed for Reuse (e.g., component detailed design)– Constructed for Reuse (e.g., integrated component)– Validated for Reuse (e.g., validated component)

09/16 19



COSYSMO For Reuse• A SEFR estimate adjusts each artifact’s size

contribution by considering its SEFR state according to this table:

09/16 20

SEFR State (Degree of Development) SEFR State Factor

Conceptualized for Reuse 36.98%

Designed for Reuse 58.02%

Constructed for Reuse 79.15%

Validated for Reuse 94.74%



What Is Systems Engineeringwith Reuse?

• Systems Engineering with Reuse is project development, with reusable artifacts being brought into the product– A special case: zero reusable artifacts

• Each reusable artifact is included in one of these SEWR states of maturity:– New (i.e., not reused)– Re-implemented (through requirements & architecture)– Adapted (through detailed design)– Adopted (through implementation)– Managed (through system verification & validation)

09/16 21



COSYSMO with Reuse• A SEWR estimate adjusts each artifact’s size

contribution by considering its SEWR state according to this table:

09/16 22

SEWR State (Maturity) SEWR State Factor

New 100.00%

Re-Implemented 66.73%

Adapted 56.27%

Adopted 38.80%

Managed 21.70%




09/16 23




The Problem

09/16 24

• How much will the total system cost?

• Is one phase being optimized while increasing total cost?

• Is the system affordable?

• Does the acquisition comply with the Better Buying Power intiatives (DoD)?



The Solution

09/16 25

Example acquisition process (DoDI 5000.02)

COSATMO assists acquirers and developers during these phases (highest payoff during early phases)

COSATMO estimates the cost for these phases



COSATMO Objective

09/16 26

• Context:– Current and future trends create challenges for full-system

cost estimation• Emergent requirements, rapid change, net-centric systems of

systems, COTS, clouds, apps, widgets, high assurance with agility, multi-mission systems

– Current development practices can minimize cost of one phase, such as development, while raising full-system cost

• The COSATMO project is developing a modern full-system cost model (first space systems, then other DoD domains)– “Constructive SATellite cost MOdel”– Current estimating models focus on one aspect, such as

system engineering– COSATMO will enable:

• System-level trades to be handled within a single model• Easy customer evaluation of full-system cost• Modern technologies to be covered



Segments of Satellite System Cost

• Total satellite system cost [tied to slide 25 phases] = System engineering cost [EMD]+Satellite software cost [EMD]+Satellite vehicle hardware development [EMD] and production [Prod] cost+Launch cost [Deploy]+Initial ground software cost [EMD]+Initial ground custom equipment cost [EMD]+Initial ground facility (buildings, communications, computers, COTS software) cost [EMD]+Operation & support cost [Deploy, O&S]

• Updated at GSAW (Feb 2014)• Model as sum of submodels is new structure in

COCOMO family09/16 27



COSATMO Segment Tentative Models

• System engineering: COSYSMO, perhaps with add-ons• Satellite vehicle hardware development and production: Current

Aerospace hardware cost model(s); exploring extensions of COSYSMO for hardware cost estimation

• Satellite vehicle, ground system software development: COCOMO II, COCOTS, perhaps with add-ons

• Launch model: similarity model, based on vehicle mass, size, orbit

• Ground system equipment, supplies: construction, unit-cost, services cost models

• Operation & support: labor-grade-based cost models, software maintenance models

09/16 28



Key Overall Satellite SystemCost Drivers

• Most Important:– Complexity, Architecture Understanding, Mass, Payload TRL

level/Technology Risk, and Requirements Understanding.

• Important:– Reliability, Pointing Accuracy, Number of Deployables, Number of

Key Sponsors, Data Rate, and Security Requirements for Communications.

• Determined at COCOMO Forum (Oct 2013)

09/16 29



Ground System Segment Development (1/2)

• Determined at GSAW (Feb 2014)• Ground system-wide cost drivers

– Most important: Accreditation (information assurance, etc), Required security

– Also important: # satellites*

• Initial software cost drivers– Required data throughput– Generally handled by COCOMO II, COCOTS, COPLIMO

09/16 30

*Indicates a size measure



Ground System Segment Development (2/2)

• Ground custom equipment cost drivers– Most important: Amount of new development required, # of

custom equipment sites*, Required site availability & reliability, Required site security

– Also important: # driving requirements*

• Ground facility cost drivers– Most important: # facilities*, location of facilities (especially

US vs foreign), # ground RF terminals*– Also important: Facility “reuse”

• Operation and support cost drivers– Most important: # years of operation*, # FTE staff (with

labor mix)*– Also important: Size of software maintained*, Leased line

cost*, level of automation

09/16 31*Indicates a size measure



COSATMO as a Research Umbrella• General direction:

– Develop a full-coverage satellite system cost estimating model

– Generalize that to additional applications

09/16 32

Specific current research initiatives:

– COSYSMO 3.0

– COCOMO III

Research vehicles:

– My thesis

– Other theses

– Other research



Bibliography1. “A Generalized Systems Engineering Reuse Framework and its Cost

Estimating Relationship”, Gan Wang, Garry J Roedler, Mauricio Pena, and Ricardo Valerdi, submitted for publication.

2. “The Constructive Systems Engineering Cost Model (COSYSMO)”, Ricardo Valerdi (PhD Dissertation), 2005.

3. “Estimating Systems Engineering Reuse with the Constructive Systems Engineering Cost Model (COSYSMO 2.0)”, Jared Fortune (PhD Dissertation), 2009.

4. “Quantifying the Impact of Requirements Volatility on Systems Engineering Effort”, Mauricio Pena (PhD Dissertation), 2012.

5. “Life Cycle Cost Modeling and Risk Assessment for 21st Century Enterprises”, Barry Boehm, Jo Ann Lane, Supannika Koolmanojwong, Richard Turner (presentation), April 29, 2014.

6. "System Interoperability Influence on System of Systems Engineering Effort", Jo Ann Lane, Ricardo Valerdi, unpublished.

7. “COSYSMO Extension as a Proxy Systems Cost Estimation” (presentation), Reggie Cole, Garry Roedler, October 23, 2013.

09/16 33

cosatmo, cocomo iii, and cosysmo: developing next-generation cost models

Documents