1

IEE

E C

on

necti

cu

t S

ecti

on

May 2

4,

2007

Ricardo Valerdi

rvalerdi@

mit.edu

2

Ro

ad

map

(1)Description of the m

odel;

(2)Explanation of its size and cost

drivers;

(3)Lim

itations;

(4)Recent developments;

3

•USC CSSE Corporate Affiliate Program

•LAI Consortium Members

•Boeing, LMCO, NGC, Raytheon, General

Dynamics, SAIC, BAE Systems

•Practical Software & Systems Measurement

•INCOSE

–Measurement Working Group

–Corporate Advisory Board

COSYSMO Development Support

All m

odels are wrong…

…but some of them are useful.

5

Esti

mati

on

Accu

racy

Feasibility

Plans/Rqts.

Design

Develop

and Test

Phases and M

ilestones

Relative

Size

Range

Operational

Concept

Life Cycle

Objectives

Life Cycle

Architecture

Initial

Operating

Capability

x

0.5x

0.25x

4x

2x

6

Key D

efi

nit

ion

s &

Co

ncep

tsC

ali

bra

tio

n:

the t

un

ing

of

para

mete

rs b

ased

on

pro

ject

data

CE

R:

a m

od

el th

at

rep

resen

ts t

he c

ost

esti

mati

ng

rela

tio

nsh

ips o

f fa

cto

rs

Co

st

Esti

mati

on

: p

red

icti

on

of

bo

th t

he

pers

on

-eff

ort

an

d e

lap

sed

tim

e o

f a p

roje

ct

Dri

ver:

A f

acto

r th

at

is h

igh

ly c

orr

ela

ted

to

th

e a

mo

un

t o

f

Syste

ms E

ng

ineeri

ng

eff

ort

Para

metr

ic:

an

eq

uati

on

or

mo

del th

at

is a

pp

roxim

ate

d b

y

a s

et

of

para

mete

rs

Rati

ng

Scale

: a r

an

ge o

f valu

es a

nd

defi

nit

ion

s f

or

a

part

icu

lar

dri

ver

Un

ders

tan

din

g:

an

in

div

idu

al’s s

ub

jecti

ve ju

dg

men

t o

f

their

level o

f co

mp

reh

en

sio

n

7

CO

CO

MO

II

•C

OC

OM

O i

s t

he

mo

st

wid

ely

us

ed

, th

oro

ug

hly

do

cu

me

nte

d a

nd

ca

lib

rate

d s

oft

wa

re c

os

t m

od

el

•C

OC

OM

O -

the

“C

On

str

uc

tiv

e C

Os

tM

Od

el”

–C

OC

OM

O I

I is

th

e u

pd

ate

to

CO

CO

MO

19

81

–o

ng

oin

g r

es

ea

rch

wit

h a

nn

ua

l c

ali

bra

tio

ns

ma

de

av

ail

ab

le

•O

rig

ina

lly

de

ve

lop

ed

by

Dr.

Ba

rry

Bo

eh

m a

nd

pu

bli

sh

ed

in

19

81

bo

ok

So

ftw

are

En

gin

ee

rin

g

Ec

on

om

ics

•C

OC

OM

O I

I d

es

cri

be

d i

n S

oft

wa

re C

os

t E

sti

ma

tio

n

wit

h C

OC

OM

O I

I (P

ren

tic

e H

all

20

00

)

8

Historical Overview of COCOMO Suite of Models

COQUALMO

1998

COCOMO 81

1981

COPROMO

1998

COSoSIM

O

2004

Legend:

Model has been calibrated with historical project data and expert (Delphi) data

Model is derived from COCOMO II

Model has been calibrated with expert (Delphi) data

COCOTS

2000

COSYSMO

2002

CORADMO

1999

iDAVE

2003

COPLIM

O

2003

COPSEMO

1998

COCOMO II

2000

DBA COCOMO

2004

COINCOMO

2004

Security

Extension 2004

Costing Secure

System 2004

Software Cost Models

Software Extensions

Other Independent

Estimation Models

Dates indicate the time that the first paper was published for the model

9

Sta

te o

f th

e P

racti

ce

•Capability to m

easure systems engineering is

unavailable in current cost models

•Possible approaches

–Heuristics/rules of thumb (Honour)

–Analogy

–% of SW or HW effort (COCOMOII, PRICE-H)

–% of total effort (Honour)

–A function of complexity (Ernstoff)

•Systems Engineering is evolving

–INCOSE (est. 1992)

–Standards (EIA/ANSI632, EIA/ANSI731, ISO/IEC15288)

–Academic degrees

•We can start where COCOMO left off…

10

CO

SY

SM

O S

co

pe

•A

dd

res

se

s f

irs

t fo

ur

ph

as

es

of

the

sy

ste

m

en

gin

ee

rin

g l

ife

cy

cle

(p

er

ISO

/IE

C 1

52

88

)

•C

on

sid

ers

sta

nd

ard

Sy

ste

ms

En

gin

ee

rin

g

Wo

rk B

rea

kd

ow

n S

tru

ctu

re t

as

ks

(p

er

EIA

/AN

SI

63

2)

Conceptualize

Develop

Op

er

Test

& E

val

Tra

nsit

ion

to

Op

era

tio

n

Op

era

te,

Main

tain

,

or

En

han

ce

Rep

lace

or

Dis

man

tle

11

CO

SY

SM

O

Siz

e

Dri

vers

Eff

ort

Mu

ltip

liers

Eff

ort

Cali

bra

tio

n

# R

eq

uir

em

en

ts

# In

terf

aces

# S

cen

ari

os

# A

lgo

rith

ms

+

3 A

dju

stm

en

t

Facto

rs

-A

pp

licati

on

facto

rs

-8 f

acto

rs

-T

eam

facto

rs

-6 f

acto

rs

-S

ch

ed

ule

dri

ver

(fu

ture

)W

BS

gu

ided

by

EIA

/AN

SI 632

CO

SY

SM

O O

pera

tio

nal

Co

nc

ep

t

12

Visualization of Cost Driver Im

pacts

1.00

1.00

1.15

1.41

1.41

1.63

2.31

2.66

2.39

2.10

2.10

2.29

2.50

2.25

2.25

0.50

1.00

1.50

2.00

2.50

3.00

3.50

RELY

CPLX

RUSEDOCUBAND

PVOLSCAP

DCAPPCON

APEX

PLEX

DTEX

LIFE

SITE

SCED

System

Size

Project

Cost

Nominal

Cost

13

Where

:

PM

NS

= e

ffort

in P

ers

on M

onth

s (

Nom

inal S

chedu

le)

A=

calib

ration c

onsta

nt

derive

d f

rom

his

torical pro

ject

data

k=

{R

EQ

, IF

, A

LG

, S

CN

}

wx

=

weig

ht

for

“easy”,

“nom

inal”,

or

“difficult”

siz

e d

river

= q

uantity

of

“k”

siz

e d

river

E= represents d

isecon

om

y o

f scale

EM

= e

ffort

multip

lier

for

the j

thcost

driver.

T

he g

eom

etr

ic p

roduct

results in a

n

overa

ll eff

ort

adju

stm

ent

facto

r to

the n

om

ina

l eff

ort

.

xΦ

Model Form

∏∑

=

⋅

Φ

+Φ

+Φ

⋅=

14 1

,,

,,

,,

)(

j

j

E

k

kd

kd

kn

kn

ke

ke

NS

EM

ww

wA

PM

COSYSMO Data Sources

Army Transformation (Orlando, FL)

Integrated Data Solutions & Analysis (McLean, VA)

SAIC

National Security Solutions/ISS (San Diego, CA)

Information & Electronic Warfare Systems (Nashua, NH)

BAE Systems

Maritime Digital Systems/AIS (Pittsfield, MA)

Surveillance & Reconnaissance Systems/AIS

(Bloomington, MN)

General Dynamics

Transportation & Security Solutions (Rockville,MD)

Integrated Systems & Solutions (Valley Forge, PA)

Systems Integration (Owego, NY)

Aeronautics (Marietta, GA)

Maritime Systems & Sensors (Manassas, VA;

Baltimore, MD; Syracuse, NY)

Lockheed Martin

Mission Systems (Redondo Beach, CA)

Northrop Grumman

Intelligence & Information Systems (Garland, TX)

Raytheon

15

4 S

ize D

rive

rs

1.

Nu

mb

er

of

Sy

ste

m R

eq

uir

em

en

ts

2.

Nu

mb

er

of

Sy

ste

m I

nte

rfa

ce

s

3.

Nu

mb

er

of

Sy

ste

m S

pe

cif

ic A

lgo

rith

ms

4.

Nu

mb

er

of

Op

era

tio

na

l S

ce

na

rio

s

Weighted by complexity, volatility, and degree of reuse

16

Counting Rules Example

COSYSMO example for sky, kite, sea,

and underw

ater levels where:

Sky level: Build an SE cost model

Kite level: Adopt EIA 632 as the WBS and ISO 15288 as the life

cycle standard

Sea level: Utilize size and cost drivers, definitions, and counting

rules

Underwater level: Perform statistical analysis of data with

software tools and implement model in Excel

Source: Cockburn 2001

17

14 C

ost

Dri

vers

1.

Req

uir

em

en

ts u

nd

ers

tan

din

g

2.

Arc

hit

ectu

re u

nd

ers

tan

din

g

3.

Level o

f serv

ice r

eq

uir

em

en

ts

4.

Mig

rati

on

co

mp

lexit

y

5.

Tech

no

log

y R

isk

6.

Do

cu

men

tati

on

Matc

h t

o L

ife C

ycle

Need

s

7.

# a

nd

Div

ers

ity o

f In

sta

llati

on

s/P

latf

orm

s

8.

# o

f R

ecu

rsiv

e L

evels

in

th

e D

esig

n

Ap

plicati

on

Facto

rs (

8)

18

14 C

ost

Dri

vers

(co

nt.

)

1.

Sta

ke

ho

lde

r te

am

co

he

sio

n

2.

Pe

rso

nn

el/

tea

m c

ap

ab

ilit

y

3.

Pe

rso

nn

el

ex

pe

rie

nc

e/c

on

tin

uit

y

4.

Pro

ce

ss

ca

pa

bil

ity

5.

Mu

ltis

ite

co

ord

ina

tio

n

6.

To

ol

su

pp

ort

Team

Facto

rs (

6)

47.4

22.8

9.8

# of Operational Scenarios

18.2

6.5

3.4

# of Critical Algorithms

9.8

4.3

1.7

# of Interfaces

5.0

1.00

0.5

# of System Requirements

Difficult

Nominal

Easy

Size Driver Weights

20

Cost Driver Rating Scales

1.93

0.72

0.85

1.00

1.18

1.39

Tool support

1.93

0.72

0.80

0.90

1.00

1.18

1.39

Multisite coordination

2.16

0.68

0.77

0.88

1.00

1.21

1.47

Process capability

2.21

0.67

0.82

1.00

1.22

1.48

Personnel experience/continuity

2.31

0.65

0.81

1.00

1.22

1.50

Personnel/team capability

2.31

0.65

0.81

1.00

1.22

1.50

Stakeholder team cohesion

1.93

1.47

1.21

1.00

0.87

0.76

# of recursive levels in the design

1.87

1.87

1.52

1.23

1.00

# and diversity of installations/platform

s

1.64

1.28

1.13

1.00

0.88

0.78

Documentation

2.61

1.75

1.32

1.00

0.82

0.67

Technology Risk

1.93

1.93

1.55

1.25

1.00

Migration Complexity

2.98

1.85

1.36

1.00

0.79

0.62

Level of Service Requirements

2.52

0.65

0.81

1.00

1.28

1.64

Architecture Understanding

3.12

0.60

0.77

1.00

1.37

1.87

Requirements Understanding

EMR

Extra

High

Very

High

High

Nominal

Low

Very

Low

21

22

ISO

/IE

C 1

5288

Co

nc

ep

tua

lize

Deve

lop

Tra

ns

itio

n t

o

Op

era

tio

n

Op

era

te,

Ma

inta

in,

or

En

ha

nc

e

Re

pla

ce

or

Dis

ma

ntl

eEIA/ANSI 632

Ac

qu

isit

ion

&

Su

pp

ly

Te

ch

nic

al

Ma

na

gem

en

t

Sys

tem

Des

ign

Pro

du

ct

Rea

liza

tio

n

Te

ch

nic

al

Eva

lua

tio

n

Op

era

tio

na

l

Te

st

&

Eva

lua

tio

n

Effort Profiling

23

Lim

ita

tio

ns o

f th

e m

od

el

1.

Mo

stl

y q

ua

lita

tiv

e d

riv

ers

2.

Va

ria

nc

e o

f D

elp

hi

res

po

ns

es

3.

Sm

all

sa

mp

le s

ize

4.

Ae

ros

pa

ce

-he

av

y

5.

Ca

lib

rati

on

is

bia

se

d b

y s

uc

ce

ss

ful

pro

jec

ts b

ec

au

se

su

cc

es

sfu

l p

roje

cts

s

ha

re d

ata

, b

ad

on

es

do

n’t

6.

Mo

de

l w

ill

no

t w

ork

ou

tsid

e o

f c

ali

bra

ted

ra

ng

e

7.

A f

oo

l w

ith

a t

oo

l is

sti

ll a

fo

ol

24

25

Contact

Ricardo Valerdi

MIT

rvalerdi@mit.edu

(617) 253-8583

www.valerdi.com/cosysmo