an analysis of changes in productivity and cocomo cost drivers over the years
DESCRIPTION
An Analysis of Changes in Productivity and COCOMO Cost Drivers over the Years. Vu Nguyen, Barry Boehm Center for Systems and Software Engineering (CSSE) 24 th Int’l COCOMO Forum Nov 3 rd , 2009. Outline. Objectives and Background. Productivity Changes. Cost Driver Changes. - PowerPoint PPT PresentationTRANSCRIPT
University of Southern California
Center for Systems and Software Engineering
© 2009, USC-CSSE 1
An Analysis of Changes in Productivity and COCOMO Cost
Drivers over the YearsVu Nguyen, Barry Boehm
Center for Systems and Software Engineering (CSSE)
24th Int’l COCOMO Forum
Nov 3rd, 2009
University of Southern California
Center for Systems and Software Engineering
© 2009, USC-CSSE 2
Outline
Objectives and Background
Productivity Changes
Discussions and Conclusions
Cost Driver Changes
University of Southern California
Center for Systems and Software Engineering
© 2009, USC-CSSE 3
Objectives and Background
University of Southern California
Center for Systems and Software Engineering
© 2009, USC-CSSE 4
Objectives
• Analysis of Productivity
– How the productivity of the COCOMO data projects has changed over the years?
– What caused the changes in productivity?
• Analysis of COCOMO cost drivers
– How cost driver ratings have changed over the periods?
– Are there any implications from these changes?
• Inviting further discussions
University of Southern California
Center for Systems and Software Engineering
© 2009, USC-CSSE 5
Estimation models need upgrading
• It has been 10 years since the release of COCOMO II.2000
– Data collected during 1970 – 1999
• Software engineering practices and technologies are changing
– Process: CMM CMMI, ICM, agile methods
– Tools are more sophisticated
– Advanced communication facility
• Improved storage and processing capability
University of Southern California
Center for Systems and Software Engineering
© 2009, USC-CSSE 6
COCOMO Data Projects Over the Five-year Periods
• Dataset has 255 projects completed between 1970 and 2009
– 161 used in COCOMO II 2000
– 87 collected since 2000
16
32
0
17 20
101
55
14
0
20
40
60
80
100
1970-1974
1975-1979
1980-1984
1985-1989
1990-1994
1995-1999
2000-2004
2005-2009
Periods
# o
f d
ata
pro
ject
s
16
32
0
17 20
101
55
14
0
20
40
60
80
100
1970-1974
1975-1979
1980-1984
1985-1989
1990-1994
1995-1999
2000-2004
2005-2009
Periods
# o
f d
ata
pro
ject
s
7 projects were collected before 2000 but not included in COCOMO II 2000
University of Southern California
Center for Systems and Software Engineering
© 2009, USC-CSSE 7
COCOMO II Formula
• Effort estimate (PM)
– COCOMO II 2000: A and B constants were calibrated using 161 data points with A = 2.94 and B = 0.91
• Productivity =
• Constant A is considered as the inverse of adjusted productivity
EMSizeAPM
SFB**
01.0
EMSize
PMA
SFB*
01.0
PM
Size
University of Southern California
Center for Systems and Software Engineering
© 2009, USC-CSSE 8
Changes in Productivity and Cost Drivers
University of Southern California
Center for Systems and Software Engineering
© 2009, USC-CSSE 9
Average productivity is increasing over the periods• Two productivity increasing trends exist: 1970 – 1994 and 1995 –
2009
1970-1974 1975-1979 1980-1984 1985-1989 1990-1994 1995-1999 2000-2004 2005-2009
Five-year Periods
SL
OC
per
PM
• 1970-1999 productivity trends largely explained by cost drivers and scale factors
• Post-2000 productivity trends not explained by cost drivers and scale factors
University of Southern California
Center for Systems and Software Engineering
© 2009, USC-CSSE 10
Effort Multipliers and Scale Factors
• EM’s and SF’s don’t change sharply as does the productivity over the periods
EA
F
1970- 1975- 1980- 1985- 1990- 1995- 2000- 2005-1974 1979 1984 1989 1994 1999 2004 2009
Su
m o
f S
cale
Fac
tors
1970- 1975- 1980- 1985- 1990- 1995- 2000- 2005-1974 1979 1984 1989 1994 1999 2004 2009
Effort Adjustment Factor (EAF) or ∏EM Sum of Scale Factors (SF)
University of Southern California
Center for Systems and Software Engineering
© 2009, USC-CSSE 11
Changes in Product Factors and Platform Factors
• Testing data (DATA) increases over the years, suggesting an increase in size of the testing data
Product Factors
0.8
0.9
0.9
1.0
1.0
1.1
1.1
1.2
1.2
1.3
0 1 2 3 4 5 6 7 8 9
EM
CPLX
DATA
RELY
• Storage constraint (STOR) is almost constant in the last three periods
• Storage constraint was important during 1970 – 1979, but not during the other periods
1970- 1975- 1980- 1985- 1990- 1995- 2000- 2005-1974 1979 1984 1989 1994 1999 2004 2009
Platform Factors
0.8
0.9
0.9
1.0
1.0
1.1
1.1
1.2
0 1 2 3 4 5 6 7 8 9
EM
STOR
TIME
PVOL
1970- 1975- 1980- 1985- 1990- 1995- 2000- 2005-1974 1979 1984 1989 1994 1999 2004 2009
University of Southern California
Center for Systems and Software Engineering
© 2009, USC-CSSE 12
Personnel Factors
0.6
0.7
0.8
0.9
1.0
1.1
0 1 2 3 4 5 6 7 8 9
EM
APEX
PLEX
LTEX
PCAP
ACAP
Changes in Product Factors and Platform Factors
• Programmer capability is low in the 70-74 period
• Language and platform experience seems improving
• Application experience seems declining
– Programmers change projects more often?
• Tool support gradually increases over the years
– 20% improvement in effort over 40 years
• Multisite development drops sharply in the last period
– Supported by technologies?
1970- 1975- 1980- 1985- 1990- 1995- 2000- 2005-1974 1979 1984 1989 1994 1999 2004 2009
Project Factors
0.6
0.7
0.8
0.9
1.0
1.1
1.2
0 1 2 3 4 5 6 7 8 9
EM
TOOL
SITE
SCED
1970- 1975- 1980- 1985- 1990- 1995- 2000- 2005-1974 1979 1984 1989 1994 1999 2004 2009
Personnel Factors
University of Southern California
Center for Systems and Software Engineering
© 2009, USC-CSSE 13
Constant A decreases over the periods
• Calibrate the constant A while stationing B = 0.91
• Constant A is the inverse of adjusted productivity
– adjusts the productivity with SF’s and EM’s
• Constant A decreases over the periods
0.0
0.5
1.0
1.5
2.0
2.5
3.0
3.5
0 1 2 3 4 5 6 7 8 9
Con
stan
t A
0.0
0.5
1.0
1.5
2.0
2.5
3.0
3.5
0 1 2 3 4 5 6 7 8 9
Con
stan
t A
1970- 1975- 1980- 1985- 1990- 1995- 2000- 2005- 1974 1979 1984 1989 1994 1999 2004 2009
EMSize
PMA
SFB*
01.0
Productivity is not fully characterized by SF’s and EM’s
50% decrease over the post-2000 period
What factors can explain the phenomenon?
University of Southern California
Center for Systems and Software Engineering
© 2009, USC-CSSE 14
Analysis of Productivity Ranges
• Productivity range is the ratio between the highest and lowest rating value for a cost driver
– Represents the degree of impact of the cost driver on effort
• Calibrated SF’s and EM’s by dividing the dataset into five sub-datasets
– 161 projects used in COCOMO II 2000
• completed and collected by 1999– 190 projects completed between 1990 – 2009
– 51 maintenance projects
• completed between 1989 – 2009– 149 new development projects
• completed between 1989 – 2009
University of Southern California
Center for Systems and Software Engineering
© 2009, USC-CSSE 15
PR of Cost Drivers of COCOMO II and all projects in 1990 - 2009
Major changes:
- Increase CPLX
- Decrease Θ DATA, ACAP, PMAT
1.28
1.21
1.21
1.26
1.31
1.74
1.35
2.78
1.73
1.44
1.61
1.37
1.43
1.10
1.29
1.67
1.35
1.48
1.62
1.34
1.52
1.42
1.00 1.50 2.00 2.50 3.00
PMAT
PREC
TEAM
FLEX
RESL
PCAP
RELY
CP LX
TIME
STOR
ACAP
PEXP
LTEX
DATA
RUSE
DOCU
PVOL
AEXP
PCON
TOOL
SITE
SCED
1.41
1.32
1.25
1.26
1.40
1.78
1.49
2.51
1.71
1.44
2.00
1.35
1.41
1.39
1.22
1.61
1.47
1.48
1.51
1.47
1.52
1.43
1.00 1.20 1.40 1.60 1.80 2.00 2.20 2.40 2.60
PMAT
PREC
TEAM
FLEX
RESL
PCAP
RELY
CP LX
TIME
STOR
ACAP
PEXP
LTEX
DATA
RUSE
DOCU
PVOL
AEXP
PCON
TOOL
SITE
SCED
1990 – 2009: including 97 projects used in COCOMO II.2000
COCOMO II.2000
1. Complexity is more influential in the more recent data
2. Analyst capability and process are less influential in the more recent data
3. Testing data is least influential
1. Complexity is more influential in the more recent data
2. Analyst capability and process are less influential in the more recent data
3. Testing data is least influential
University of Southern California
Center for Systems and Software Engineering
© 2009, USC-CSSE 16
1.36
1.30
1.24
1.26
1.28
1.77
1.37
2.71
1.72
1.50
1.67
1.37
1.42
1.16
1.29
1.54
1.39
1.48
1.67
1.34
1.52
1.42
1.00 1.20 1.40 1.60 1.80 2.00 2.20 2.40 2.60 2.80
PMAT
PREC
TEAM
FLEX
RESL
PCAP
RELY
CP LX
TIME
STOR
ACAP
PEXP
LTEX
DATA
RUSE
DOCU
PVOL
AEXP
PCON
TOOL
SITE
SCED
1.34
1.18
1.21
1.26
1.37
1.62
1.47
2.38
1.63
1.40
1.22
1.40
1.43
1.29
1.32
1.65
1.44
1.50
1.53
1.41
1.51
1.44
1.00 1.20 1.40 1.60 1.80 2.00 2.20 2.40 2.60 2.80
PMAT
PREC
TEAM
FLEX
RESL
PCAP
RELY
CPLX
TIME
STOR
ACAP
PEXP
LTEX
DATA
RUSE
DOCU
PVOL
AEXP
PCON
TOOL
SITE
SCED
PR of Cost Drivers of New Development and Maintenance
Major changes:
- Increase DATA, DOCU
- Decrease Θ PCON, ACAP, PCAP, CPLX, PREC
New Development
Maintenance
1. Testing data and documentation are more important in maintenance
2. Personnel capacity, complexity in maintenance is not as critical as in development
1. Testing data and documentation are more important in maintenance
2. Personnel capacity, complexity in maintenance is not as critical as in development
University of Southern California
Center for Systems and Software Engineering
© 2009, USC-CSSE 17
Discussions
• Productivity has doubled over the last 40 years
– But scale factors and effort multipliers did not fully characterize this increase
• Hypotheses/questions for explanation
– Is standard for rating personnel factors different among the organizations?
– Were automatically translated code reported as new code?
– Were reused code reported as new code?
– Are the ranges of some cost drivers not large enough?
• Improvement in tools (TOOL) only contributes to 20% reduction in effort
– Are more lightweight projects being reported?
• Documentation relative to life-cycle needs
University of Southern California
Center for Systems and Software Engineering
© 2009, USC-CSSE 18
Conclusions
• Productivity is generally increasing over the 40-year period
– SF’s and EM’s only partially explain this improvement
• The impact of some cost drivers on effort has changed when calibrating more recent data
– Increased impact of complexity (CPLX)
– Decreased impact of testing data (DATA), Analyst capability (ACAP), and Process capability (PMAT)
• Need for further analysis and calibration using recent data points