uksma 2005 project estimating – theory and practice – beyond talking paul goodman

UKSMA 2005

Project Estimating – Theory and Practice – Beyond Talking

Paul Goodman

© 2003 Gartner, Inc. and/or Gartner Holdings Ireland. All Rights Reserved.

consulting

Our Starting Point

“Our customers quite rightly complain.

It is not the fact that we get it (our estimates) wrong!

It is that we are always late, never early!”CIO’s recent comment on the state of estimating and the effect on customer satisfaction


consulting

Topics for Discussion

Why is estimating accurately important?

Where should we start our improvement from?

A Case Study Overview


consulting

Estimation accuracy is a critical success factor for most businesses

Getting it wrong can adversely affect your company bottom line

And Will certainly upset your customers

Basic principles:

The Importance of Accurate Estimating


consulting

The Importance of Accurate Estimating

We should not expect accurate estimating to be “natural” i.e. to just happen

But It is possible to estimate, after requirements specification, to within 5% of actuals

Accepting 200%+ over runs is not the way it has to be


consulting

Starting Improvement

Appreciate that this will not happen overnight

It will probably take at least one year elapsed to effect broad sustainable improvement

Get to grips with some basic principles

Some are facts of life but they still need to be clearly recognised and stated

Others need to be embodied within your estimating processes


consulting

Facts of Estimating Life

Estimation inaccuracy is the difference between the estimate and the actual

If reducing this difference is important to you, make it important to everyone with estimating responsibility

If it is not important to you – STOP NOW!

Estimates and actuals need to be recorded in order to enable and to show improvement

Appreciate that people are genuinely and almost universally optimistic

Optimism is dangerous when estimating!


consulting

The estimator has the last word Estimates are estimates - use bounds and a most

likely The estimate is dynamic - re-estimate Use a sanity check, more than one technique Keep information

Estimating Principles


consulting

Recording Information

InitialEstimation

Re-estimation

Size ChangeControl

FirstEstimate

RevisedEstimates

ProjectCompletion

Actuals

InitialSize

RevisedSize

ProjectCompletion

FinalSize

Metrics Database

Identify the WEDGE factor

Effort


consulting

The Wedge Factor

base line requirements

final requirements

70% growth!


consulting

Mark II Function Point Analysis IFPUG Function Point Analysis Work Breakdown Structure/Task Sizing Line of Code Sizing for Infrastructure Projects

Effective sizing at an early stage is a pre-requisite to accurate estimation

Common Sizing Techniques:

The Need for Sizing Requirements


consulting

Using Estimation Tools

There are many estimating tools available on the market

They can be very useful

They do need to be used with care

Be aware of the need to calibrate

x

For internal use of <FullCompanyName> only. © 2003 Gartner, Inc. and/or Gartner Holdings Ireland.

All Rights Reserved.

FullCompanyNamePresentationTitleEngagement: xxxxxxxxx—DayMonthYear

consulting

Estimating Case Study


consulting

Agenda

Introduction and Process

Sizing

Costing

Effort and timescales

Summary


consulting

Introduction

Meta Group UK was requested by XXX to provide size estimates for the YYY System (a GIS application)

META Group were further asked to provide indicative and comparative costs estimates based on industry benchmarks


consulting

Conduct of Assignment

Chosen size metric was Mark II Function Point Index (FPI)

This was to be estimated based on best available information provided as documentation by XXX

Clarification of issues on documentation was provided by XXX

Documentation was reasonable and facilitated the use of a number of FPA estimation approaches

(Fact of life – the more limited your information, the less able you are to estimate)


consulting

Lifecycle Coverage

Our estimates will include the following elements of a project: Requirements

Design

Build

Integrate

Test

Project Management

Quality Assurance

Excludes effort associated with: Architecture and infrastructure (plan, build or run)

Business input


consulting

Cost/Effort estimate

Function Points are an estimate of size, not cost

The cost of delivering a system will depend on many factors including:

Maturity of the development teams

Stability of requirements

Tools and processes used

Quality of code developed (speed of test/acceptance)


consulting

Findings

Size estimation using three techniques:

Structured Expert Judgement; (Confidence is +/- 40% but tends to underscore)

Data Model Approach; (Confidence is +/- 25%)

Functional Decomposition (calibrated through Data Model Approach); (Confidence is +/- 20%)

(These are in the public domain – see “Sizing and Estimating Software in Practice”, Treble and Douglas)


consulting

Size Estimation Results

Technique Lower Est Most Likely Upper Est Bounds

Expert Judgement 1,600 FPts 2,700 FPts 3,800 FPts +/- 40%

Data Model 2,500 FPts 3,300 FPts 4,100 FPts +/- 25%

Functional Decomposition

3,000 FPts 3,800 FPts 4,500 FPts +/- 20%

META recommendation is that most likely size estimate be taken as 3,500 FPts


consulting

Sizing by Contribution

Contribution to total size made by Enquiry/Report functionality appears low

18% of total business functions are Enquiry/Report type contributing 12% of total functionality

This may be perfectly valid but raises a flag – “Is XXX confident that reporting functionality has been adequately identified?”

It is stressed that this observation is based only on estimated counts.


consulting

Cost Estimate 1

Assumptions:

System size is 3500 Function Points

Average lifecycle delivery rate of 15 FP / Man Month (mid-point performance within META Group’s AD benchmark)

Note: GIS type systems tend to achieve lower productivity levels than some other system classes

Blended ‘daily rate’ of £650 per day (mid-point in META Group’s Labour Rate benchmark)

Total Function Points

3500

Total months of Effort

233

Cost per ‘man month’

£14,300

Total cost £3,340,000

Note: Based on 22 working days per month


consulting

Cost Estimate 2

Assumptions: System size is 3500 Function Points

Based on International Software Benchmark Standards Group median productivity of 0.14 FP per Person Hour

Blended ‘daily rate’ of £650 per day (mid-point in META Group’s Labour Rate benchmark

This is an ‘optimistic’ estimate as this benchmark tends to reflects ‘best in class’ development productivity


3500

Total Days of Effort 4200

Cost per Day £650


Note: Based on 22 working days per month and 6 effective hours per day


consulting

Cost Estimate 3

Assumptions: System size is 3500 Function Points Based on CCTA median productivity of 0.07 FP

per Person Hour Blended ‘daily rate’ of £650 per day (mid-point in

META Group’s Labour Rate benchmark This is an a more pessimistic estimate based on a

less mature development environment


3500

Total Days of Effort 8300

Cost per Day £650


Note: Based on 22 working days per month and 6 effective hours per day


consulting

Costing Summary

META Group propose that the yyy application, based on documentation provided, is approximately 3500 FP in size ( with a likely accuracy of +/- 20%)

All evidence suggests that the cost range is between £2.7m and £5.4m based on market blended rate of £650 per day

META Group have no appreciation of the development maturity of the potential suppliers but do note that this has a major impact on costs

Cost may be reduced by the use of pre-packaged software components and the effective use of code generation tools (if applicable)

Overall, our experience indicates that based on our labour rate the total cost is likely to be between £3.2m and £4.5m


consulting

Reality Check

At this point we were told that the latest in house estimate was in the area of £3M

Given the discussion about reporting functionality that was now considered “low”

So what about the duration estimates?


consulting

Delivery Timing

We looked at duration estimates based on the estimated size and the profile of the organisation/project (mainly using COCOMO 2000)

The indications were that delivery within a 12 month period (including test) is an extremely aggressive timescale

This is putting it politely

This was in line with the “gut feel” of the teams on supplier and client sides (but now they felt able to talk about it)


consulting

Summary

Our best estimate is that the application should cost between £3.2 and £4.5m based on market rates

Delivery in a 12 month period would require a team of 15-23 FTE (and considerable “good fortune”)

The supplier would need to focus strongly on its development performance to achieve these targets

We recommend the supplier use function point techniques through the process to assure XXX of best in class development performance and to control change


consulting

Conclusions

This exercise took approximately two weeks elapsed time

The approaches described and data used are mostly in the public domain (or available at relatively low cost)

The analysis provided an objective vehicle for discussion

Estimates were revised towards the upper bounds and these revisions were informed by the analysis

Duration estimates were relaxed somewhat

Nobody discounts the need to apply some judgement

The project is on track to deliver within budget

uksma 2005 project estimating – theory and practice – beyond talking paul goodman

Documents