the accuracy of pre-tender building cost estimates in australia

PLEASE SCROLL DOWN FOR ARTICLE

This article was downloaded by: [HEAL-Link Consortium]On: 4 January 2011Access details: Access Details: [subscription number 786636552]Publisher RoutledgeInforma Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK

Construction Management and EconomicsPublication details, including instructions for authors and subscription information:http://www.informaworld.com/smpp/title~content=t713664979

The accuracy of pre-tender building cost estimates in AustraliaAjibade Ayodeji Aibinua; Thomas Pascoa

a Faculty of Architecture, Building and Planning, University of Melbourne, Parkville, Melbourne, 3010Australia

To cite this Article Aibinu, Ajibade Ayodeji and Pasco, Thomas(2008) 'The accuracy of pre-tender building cost estimatesin Australia', Construction Management and Economics, 26: 12, 1257 — 1269To link to this Article: DOI: 10.1080/01446190802527514URL: http://dx.doi.org/10.1080/01446190802527514

Full terms and conditions of use: http://www.informaworld.com/terms-and-conditions-of-access.pdf

This article may be used for research, teaching and private study purposes. Any substantial orsystematic reproduction, re-distribution, re-selling, loan or sub-licensing, systematic supply ordistribution in any form to anyone is expressly forbidden.

The publisher does not give any warranty express or implied or make any representation that the contentswill be complete or accurate or up to date. The accuracy of any instructions, formulae and drug dosesshould be independently verified with primary sources. The publisher shall not be liable for any loss,actions, claims, proceedings, demand or costs or damages whatsoever or howsoever caused arising directlyor indirectly in connection with or arising out of the use of this material.

http://www.informaworld.com/smpp/title~content=t713664979

http://dx.doi.org/10.1080/01446190802527514

http://www.informaworld.com/terms-and-conditions-of-access.pdf

The accuracy of pre-tender building cost estimates inAustralia

AJIBADE AYODEJI AIBINU* and THOMAS PASCO

Faculty of Architecture, Building and Planning, University of Melbourne, Parkville, Melbourne, 3010 Australia

Received 22 January 2008; accepted 4 October 2008

A pre-tender building cost estimate is an important piece of information when making decisions at the project

planning and design stage. The important project characteristics influencing the accuracy of pre-tender

building cost estimates are examined and practical improvement for increasing the accuracy of estimates are

considered. A quantitative approach is used to address the research problem. Analysis of data from 56 projects

and from a postal questionnaire survey of 102 quantity surveying firms suggests that the accuracy of pre-tender

building cost estimates varies according to project size and principal structural material. When eight identified

project characteristics are controlled in a multiple regression analysis, the accuracy of estimates is influenced by

project size. The estimates of smaller projects are more biased than the estimates of larger projects. It was

discovered that pre-tender building costs are more often overestimated than are underestimated. Overestimated

forecasts are incorrect by a larger amount than underestimated forecasts. Data analysis also revealed that the

accuracy of pre-tender building cost estimates has not improved over time. The majority of the respondents are

somewhat dissatisfied with the accuracy of estimates in the industry. Probability estimation and simulation of

past estimates, reducing quantity surveying and cost engineering skill turnover, incorporating market

sentiments into estimates, early involvement of the quantity surveyor at the brief stage, and proper

documentation of experience gained in the estimation of projects should help firms increase the accuracy of

estimates for new projects.

Keywords: Australia, estimating accuracy, pre-tender estimates, quantity surveying, tendering.

Introduction

Pre-tender building cost estimates are susceptible to

inaccuracies (bias) because they are often prepared

within a limited timeframe, and without finalized

project scope. Pursuing an underestimated project

can lead to project failure. On the other hand,

overestimation of a project at the pre-tender stage can

lead to a viable project being dropped or re-tendered

when there is no bid close enough to permit project

award. Bias in the estimate of a project may arise from

two sources, namely, bias associated with the project

itself (will be the same regardless of the estimator) and

bias associated with the estimating technique used and

the environment (which would change depending on

the estimator). The only known published works

relating to accuracy of cost forecasts in an Australian

context are Mills (1997), which compared prediction of

building price movement by quantity surveyors with the

Australian Bureau of Statistics’ actual building price

movement, and Bromilow et al. (1988), which analysed

the variance between contract sum and final contract

sum. The objectives of this study are:

(1) to explore the frequency and size of inaccuracy

in pre-tender building cost estimates (i.e. the

variance between pre-tender cost estimate and

contract sum—accepted tender) using

Australian data;

(2) to explore project characteristics influencing the

accuracy of pre-tender building cost estimates;

(3) to assess whether the accuracy of pre-tender

building cost estimates has improved over time;

(4) to investigate what firms are doing to improve

the accuracy of cost estimates in practice, and

in that regard, evaluate the effectiveness of the

improvement methods.*Author for correspondence. E-mail: [email protected]

Construction Management and Economics (December 2008) 26, 1257–1269

Construction Management and EconomicsISSN 0144-6193 print/ISSN 1466-433X online # 2008 Taylor & Francis

http://www.tandf.co.uk/journalsDOI: 10.1080/01446190802527514

Downloaded By: [HEAL-Link Consortium] At: 12:01 4 January 2011

It is important to study the accuracy of building cost

estimates because a large part of a quantity surveyor’s

and cost engineer’s role in the construction industry is

to provide certainty of cost to clients. Thus the

knowledge developed in this study should help quantity

surveyors, estimators and cost engineers so that they

are aware of how their cost forecasts have performed

over time and what project characteristics need special

attention during cost estimation; and what practices

could increase the accuracy of their estimates.

Theoretical framework and scope of study

What is accuracy of an estimate?

Pre-tender cost estimation (or early stage cost

estimation) is the forecasting of the cost of a project

during the planning and design stage (Serpell, 2005).

Skitmore (1991) describes the accuracy of early stage

estimation as comprising two aspects, namely, bias

and consistency of the estimate when compared with

the contract or accepted tender price. Bias is

concerned with ‘the average of differences between

actual tender price and forecast’ while consistency of

estimates is concerned with ‘the degree of variation

around the average’.

Factors affecting estimating accuracy

An overview of previous studies suggests that a large

number of factors may influence the accuracy of an

estimate. Gunner and Skitmore (1999a) reviewed

previous studies and summarized the factors as

follows: building function, type of contract, condi-

tions of contract, contract sum, price intensity,

contract period, number of bidders, good/bad years,

procurement basis, project sector (public, private or

joint), number of priced items and number of

drawings. Gunner and Skitmore (1999a) analysed

the estimates of 181 projects in Singapore. They

found that a majority of the factors influenced the

accuracy of estimates. Using data from 42 projects in

Singapore Ling and Boo (2001) found similar results

when they compared five variables against Gunner

and Skitmore’s (1999a) work. Skitmore and Picken

(2000) studied the effect that four independent

factors (building type, project size, project sector

and year) had on estimating accuracy. They tested

the four factors using data from 217 projects in the

United States of America. They found that bias in the

estimate of the projects is influenced by project size

and year, while consistency in the estimates is

influenced by project type, size and year. In a study

of 67 process industry construction projects around

the world, Trost and Oberlender (2003) identified 45

factors contributing to the accuracy of early stage

estimates. They summarized the factors into 11

orthogonal elements. Of the 11 factors, the five most

important include: process design, team experience

and cost information, time allowed to prepare

estimates, site requirements, and bidding and labour

climate. All these studies suggest that there are a large

number of variables that may substantially influence

the accuracy of an early stage estimate.

According to Gunner (1997) the factors influencing

accuracy of estimates are intercorrelated so that the

true bias of one factor could be masked by one or

more factors. For example, Gunner and Skitmore

(1999b) theorize that ‘Price Intensity alone is both

necessary and sufficient to account for systematic bias

(inaccuracy) in building price forecasting’. Price

intensity is the total cost of a building divided by

the gross floor area. Price intensity theory states that

buildings with low unit rates (cost/m2 gross floor

area) would tend to be overestimated, while those

with high unit rates would tend to be underestimated.

In a study of 89 construction projects in Hong Kong,

Skitmore and Drew (2003) support the price intensity

theory.

In another study, Skitmore and Picken (2000)

using data from 217 projects in the United States

found that ‘year’ was the underlying variable respon-

sible for the bias and inconsistency in cost estimates,

after partialling out confounding effects of the four

factors put forward. The finding contrasts Gunner

and Skitmore’s (1999b) ‘price intensity’ theory.

However, their result supports Gunner’s (1997)

theory which states that intercorrelations among

variables cause confounding effects. It also supports

Gunner and Skitmore (1999a) in their suggestion that

a single underlying variable is the cause of bias and

consistency seen in estimates.

Study hypotheses

Based on a review of past studies, the following

hypotheses are set out:

Hypothesis 1: Systematic bias and inconsistency in pre-

tender building cost estimates are influenced by project

size (measured by project value, number of storeys and

gross floor area), location, project type, procurement

route, project sector, price intensity and principal

structural material.

Hypothesis 2: The accuracy of pre-tender building cost

estimates has not improved over time.

Hypothesis 3: Quantity surveying firms agree on the most

effective ways they believe will improve the accuracy of

pre-tender building cost estimates.

1258 Aibinu and Pasco


Research method

Data collection

Two methods of data collection were used to achieve

the study objectives. In order to address objectives 1, 2

and 3, data were collected from the files of 56

construction projects completed between 1999 and

2007. The data were obtained from the office of a

quantity surveying firm in Australia. Information

obtained in respect of each project included: project

value, the number of floors, gross floor area (GFA),

project location (central business district, metropolitan,

or rural), project type (residential, industrial or

commercial), procurement route used, project sector

(public, private or joint), price intensity (measured by

ratio of project value and gross floor area), and

principal structural material used (steel, concrete or

timber).

The projects were selected by a simple random

sampling process. A list of projects completed from

1999 to 2007 was drawn. Projects that were not

suitable for analysis, owing to incorrect job type or

lack of early stage estimate, were discarded. Thereafter,

each project was assigned a serial number sequentially

from one. Random numbers were then generated using

the Microsoft Excel program. The process yielded 85

random numbers. Numbers that were repeated were

deleted the second time they appeared. The process

produced 56 random numbers. The projects with serial

numbers corresponding with the 56 random numbers

were selected for data collection and analysis. As the

researchers were allowed first-hand access to all data,

no data were selected on the recommendation of the

quantity surveyor responsible for the estimates. Thus

there was no bias in the data collection.

In order to achieve objective 4, a structured ques-

tionnaire was designed for data collection. It comprised

questions regarding the profile of the respondents, the

profile of their company, the respondents’ satisfaction

with the current level of estimate accuracy in the

industry, and the views of the respondents regarding

acceptable level of estimate accuracy. The respondents

were also asked to rate 12 methods that could be used

for improving the accuracy of estimates. Depending on

the nature of the question, respondents were asked to

indicate their answers on a five-point Likert scale or a

categorical scale.

The questionnaires were mailed to 102 quantity

surveying firms in July/August 2007. The firms were

randomly selected from the list of about 166 firms

maintained by the Australian Institute of Quantity

Surveyors (AIQS) (AIQS, 2003).

Data analysis, results and discussion

Response rate and characteristic of sample from

questionnaire

The questionnaire survey yielded a response rate of

41%. Figure 1 shows the geographical distribution of

Figure 1 Geographical distribution of the survey and responses

Key

ACT – Act Capital Territory; NSW – New South Wales; NT – Northern Territory;

QLD – Queensland; VIC – Victoria; SA – South Australia; TAS – Tasmania; WA – Western Australia

Building cost estimates 1259


the responses. 29% of the responding firms have fewer

than 6 technical staff, 42% have between 6 and 15

technical staff, 22% have between 16 and 25 staff, and

7% have 26 or more technical staff.

90% of the questionnaires were completed by either

a director or an associate of the firms while 10%

were completed by a senior quantity surveyor or a

quantity surveyor. 81% of them have had over 15

years’ experience, 13% have between 10 and 15

years’ experience, and 6% have fewer than 10 years’

experience. Altogether, the data have come from the

highest echelon of quantity surveying professionals in

Australia, and their responses can be confidently relied

upon.

Respondents’ tolerance and satisfaction with

estimate accuracy in the industry

None of the respondents indicated that they were very

satisfied with the current level of estimate accuracy.

66% indicated they were very dissatisfied, dissatisfied

or neither satisfied nor dissatisfied, while 34% indicated

that they were satisfied. When early stage estimate is

compared with the lowest tender (contract sum), 24%

of respondents believed that a tolerance limit of within

¡5% is acceptable, 54% nominated a tolerance of

¡10%, 20% nominated a tolerance of ¡20%, and 2%

nominated a tolerance of ¡30%. Also, 70% of

respondents indicated that the accuracy of early stage

estimates hadn’t improved at all, very slightly or slightly

over the past 15 years while 25% believed that it has

improved almost adequately or adequately. 5% did not

respond.

Preliminary analysis of data from past

projects

Treatment of data

Data obtained from the 56 projects were analysed by

project size (measured by project value, gross floor area

and number of storeys), location, procurement route,

project type, principal structural material and price

intensity. Time of estimate was controlled by transform-

ing the pre-tender estimate and contract sum of each

project to the December 2006 price using the building

price index published by Rawlinsons (2006, 2007). Also,

differences in estimating processes and approach were

understood to have been controlled because projects

analysed were undertaken in the same company under

the same quality assurance procedure. Project sector

(whether private or public or joint) was also excluded

from the analysis because the 56 projects did not provide

a large enough spread of data to enable statistical analysis

of the impact of project sector. There were too few

samples of projects procured by the private sector. There

were also too few samples of projects procured jointly by

the public and private sectors.

Frequency and size of inaccuracies in pre-tender

building cost forecasts

Percentage cost overestimate or underestimate (esti-

mate error or bias) were estimated for each project by

using the following expression:

Estimate bias~

pre tender cost estimate{accepted tender sum

accepted tender sum|100

The mean estimate bias was also computed for the 56

projects using the following expression:

Mean estimate bias xð Þ~P

x

n

where x5estimate bias; n5number of projects.

A positive value of estimate bias implies an over-

estimation of cost while a negative value implies an

underestimation of cost. The analysis shows that in

about 7 out of every 10 projects, cost forecasts were

overestimated while in about 3 out of every 10 projects

cost forecasts were underestimated. This implies that

pre-tender cost forecasts were more often overesti-

mated than were underestimated. Bias in overestimated

costs ranges from +0.97% to +31.88% with a mean of

+10% while bias in underestimated cost ranges from

22.21% to 219.83% with a mean of 29%.

A one-sample t-test (Levine et al., 2005) was used to

test the following hypotheses: (1) The mean bias in

overestimated costs (+10%) is not different from zero;

(2) The mean bias in underestimated costs (29%) is

not different from zero. The results show that the mean

bias in overestimated costs (+10%) is significantly

different from 0% (p50.000, standard deviation57%)

which implies that the overestimated forecasts are truly

biased and not by chance. Similarly, the mean bias in

underestimated costs (29%) is significantly different

from zero (p50.000, standard deviation55%) which

also implies that underestimated forecasts are truly

biased and not by chance.

Analysis shows that the cost estimates for the 56

projects are generally biased and were overestimated

(mean524.29%) (see Table 1). Again, one-sample t-test

analysis shows that the mean (+4.29%) is significantly

different from zero (p50.004, standard devia-

tion510.61%) meaning that the estimates are biased

overall. However, when one considers the fact that early

stage estimates are prepared with little information, the

overall mean estimate bias of 24.29% may be acceptable.



In order to examine whether the error of under-

estimation is the same as the error of overestimation,

the Mann–Whitney test (non-parametric) (Levine et al,

2005) was used instead of a two-sample t-test because

normality of data could not be demonstrated. The test

shows that the mean bias in underestimated costs

(29%) is not the same as the mean bias in over-

estimated costs (+10%) (they are statistically and

significantly different: p50.000). This means that

pre-tender cost forecasts that were overestimated are

incorrect by a larger margin than pre-tender cost

forecasts that were underestimated.

Standard deviation (S) was computed for the 56

projects using the expression:

S~

ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiP

x{xð Þ2.

n

r

where x5estimate bias; x̄5mean estimate bias;

n5number of projects.

Thereafter, the consistency in the estimates was

determined by calculating the coefficient of variation

(CV) as follows:

CV~standard deviation

mean estimate error|100

The standard deviation and CV were also determined for

projects in the different groups of the eight factors

examined (Table 2). Coefficient of variation is a measure

of predictability of estimate bias. Large coefficient of

variation implies that estimate bias is volatile and

unpredictable. By visual inspection of the results

(Table 2), the estimates for the 56 projects (put together)

are inconsistent with a coefficient of variation of 10.17%.

It is assumed that a double digit coefficient of variation is

large. Thus the risk of estimation bias is not small. It also

suggests that firms have little control over the propensity

that estimates would be biased.

Table 1 Preliminary analysis of 56 projects

Project factors broken

into sub-groups

Number of

projects

Mean error (%)

(estimate bias)

Standard

deviation (%)

Coefficient of variation (%)

(estimate consistency)

*Project value ($)

1–5 000 000 20 8.95% 10.85% 9.96%

5 000 001–10 000 000 16 1.38% 7.79% 7.68%

10 000 000+ 20 1.95% 11.16% 10.95%

**GFA (m2)

1–3000 19 5.32% 12.32% 11.70%

3001–10 000 19 5.68% 8.54% 8.08%

Above 10 000 18 1.72% 10.76% 10.58%

Number of storeys

1–2 storeys 29 9.31% 9.32% 8.53%

3–7 storeys 17 21.59% 8.76% 8.90%

8+ storeys 10 20.30% 10.53% 10.56%

Location

***CBD 10 20.90% 12.71% 12.83%

Metropolitan 34 4.82% 9.42% 8.99%

Rural 12 7.08% 11.41% 10.66%

Procurement route

Traditional (lump sum) 38 2.66% 11.22% 10.93%

Design & construct 18 7.72% 8.48% 7.87%

Project type

Residential 18 2.17% 7.84% 7.67%

Industrial 9 8.11% 10.93% 10.11%

Commercial 29 4.41% 11.92% 11.42%

Principal structural material

Timber 9 9.78% 10.84% 9.87%

Steel 14 7.29% 7.41% 6.90%

Concrete 29 0.69% 10.54% 10.47%

Price intensity ($/m2 GFA)

1–1200 20 7.55% 8.63% 8.02%

1201–1700 18 3.00% 11.54% 11.20%

Above 1700 18 1.94% 11.30% 11.08%

Total 56 4.29% 10.61% 10.17%

Notes: *Project value is measured in Australian dollars ($); **GFA5gross floor area; ***CBD5central business district.



Projects estimate bias and consistency across

project factors

Estimate bias and estimate consistency for each group

in the eight project factors were examined. The mean

estimate bias and coefficient variation (estimate

consistency) were also determined for the projects in

each group of the factors shown in Table 1. The

results (Table 1) show that estimates of projects in the

three project value categories tend to be overesti-

mated. The estimates of projects with lower project

value are more biased than estimates of projects with

larger value. The same trend is observed for estimates

of projects in the different ‘GFA’ and ‘number of

storeys’ categories. Thus the estimates of smaller

projects were more biased than the estimates of larger

projects. The result for ‘GFA’ is consistent with

Skitmore and Drew (2003) and Ling and Boo (2001)

which both found that the estimates of projects of

smaller gross floor area tends to be more biased than

the estimates of projects of larger gross floor area.

However, for ‘project value’, the result is inconsistent

with Skitmore and Drew (2003) where estimates of

projects with a smaller value (less than $60m) are less

biased than estimates of projects with a higher value

(over $60m).

Further, the cost estimates for traditionally procured

projects tend to be more accurate than the cost

estimates of those procured using design and construct

method. Cost estimates of residential projects tend to

be the least biased, followed by estimates of commercial

and industrial projects. The findings could be adduced

to experience; for example, the quantity surveying firm

from which the sample projects were obtained has

handled many traditionally procured projects and

residential and commercial projects. This could be

the reason why the estimates for those projects are less

biased. Projects that were procured with design and

construct method, as well as industrial projects are less

common in the firm’s experience. This could explain

why estimates for those projects are more biased.

The preliminary analysis (Table 1) also suggests that

cost estimates of projects using concrete as the

principal structural material are by far the least biased

(overestimated by 0.7%), followed by steel with an

average overestimate of 7.2%. Estimates of projects

using timber as their primary structural material tend to

be the most biased, with an average overestimate of

9.8%. Again, this result could be explained by

experience. The sample projects indicate that the firm

from which the data were drawn has handled fewer

projects using timber when compared to projects

constructed with steel and concrete.

With regard to price intensity, the trend seems to be

negatively related. Estimates of projects with low $/m2

GFA tend to be more biased than estimates of projects

with high $/m2 GFA. This is contrary to Gunner and

Skitmore’s (1999b) price intensity theory. However,

because this is a preliminary analysis, it is not conclusive.

Turning to estimate consistency, Table 1 suggests that

the estimates of projects with lower value tend to be more

consistent than the estimates of projects with higher

value. ‘Number of storeys’ follows the same trend. The

estimates of projects with lower number of storeys are

more consistent than the estimates of projects with higher

number of storeys. In contrast, ‘GFA’ shows an opposite

trend such that the estimates of projects having lower

GFA are most inconsistent when compared with the

estimates of projects with larger GFA.

For location, the estimates of projects located in the

central business district (CBD) are the least consistent

when compared with the estimates of projects in the

metropolitan and rural areas. The estimates of projects

procured with traditional methods are less consistent

when compared with the estimates of design and

construct projects. Also, residential project estimates

are more consistent than industrial project estimates

while industrial project estimates are more consistent

than the estimates of commercial projects. The analysis

also revealed that the estimates of projects constructed

with concrete as the principal structural material are the

least consistent followed by the estimates of projects

Table 2 Result of ANOVA and Levene’s test for homogeneity of variance*

ANOVA (for estimate bias) Levene’s test (for estimate consistency)

Project factor F R Test statistic RProject value 3.27 0.046* 1.006 0.372

Gross floor area 0.77 0.466 1.202 0.309

No. of storeys 11.86 0.001* 1.387 0.244

Project location 1.70 0.193 0.123 0.885

Procurement route 2.88 0.096 0.577 0.451

Project type 0.94 0.395 0.921 0.404

Structural material 3.96 0.026* 0.677 0.570

Price intensity 1.55 0.222 0.729 0.487

Notes: *Significant relationships taken at the 5% level.



constructed with timber and steel. The estimates of

projects with higher price intensity tend to be more

inconsistent than the estimates of projects with lower

price intensity.

Put together, the preliminary analysis suggests that

the estimates of small projects are more biased but are

more consistent than the estimates of large projects.

This means that bias in the estimates of smaller projects

is larger and is consistent, whereas bias in the estimates

of large projects is smaller but inconsistent.

Hypotheses testing

The impact of project factors on estimate bias and

estimate consistency

The data from the 56 projects were analysed more closely

to investigate the trends uncovered by the preliminary

analysis (Table 1). Analysis of variance (ANOVA)

method was used to compare the mean estimate bias of

projects in the different groups of each of the eight

factors. The aim was to investigate whether bias in the

estimate of the projects varies according to each of the

eight factors in Table 1. The results (Table 2) show that

there are significant differences between the mean

estimate bias for projects in the three categories of

‘project value’ (F53.27, p50.046). Also, there are

differences in the mean estimate bias for projects in the

three categories of ‘number of storeys’ (F511.86,

p50.001). Similarly, the mean estimate bias of projects

in the three categories of ‘principal structural material’

are statistically different (F53.96, p50.026).

The ANOVA results indicate that estimate bias

varies according to project value, number of storeys

and type of principal structural material used.

After significant differences in the mean estimate

were found, the Tukey–Kramer procedure (Levine

et al., 2005) was used to determine which groups are

different. The Tukey–Kramer procedure enabled

simultaneous examination of comparisons between all

pairs of groups (Levine et al., 2005) for each of the

three significant factors (project value, number of

storeys and principal structural material). The analysis

revealed the following differences:

N There is a significant difference between the

mean estimate bias of projects having a value of

‘$1–$5 000 000’ and those having value of

‘$5 000 001 and above’. All the other pairwise

differences are small enough that they may be

due to chance.

N The mean estimate bias of projects with ‘1–2

storeys’ significantly differs from the mean estimate

bias of projects with ‘3 or more storeys’. Similar to

project value, all the other pairwise differences are

small enough that they may be due to chance.

N There is a significant difference between the

mean estimate bias of projects constructed with

timber and those constructed with concrete. As

before, all the other pairwise differences are small

enough that they may be due to chance.

Similar to the trend observed in the preliminary analysis

(Table 1), the estimates of smaller projects tend to be

more biased than the estimates of larger projects. An

explanation could lie in the delegation of estimating

work. Typically, estimates of smaller projects are under-

taken by junior quantity surveyors, while estimates of

larger and more complex projects are undertaken by

more experienced staff; when junior staff are involved,

they are supervised by senior quantity surveyors.

Levene’s test (Brown et al., 1974) was used to

investigate whether the consistency in the estimates

varies according to the eight factors. Levene’s test

enabled analysis of the homogeneity of variance

(Conover et al., 1981; Levine et al., 2005). The result

(Table 2) shows that the consistency in the estimates

did not vary according to any of the eight factors

examined (i.e. the variances were found to be homo-

geneous across the groups of each of the eight project

factors).

Regression modelling of project factors

influencing estimate bias

Project factors influencing estimate bias were further

investigated using the traditional multiple linear regres-

sion technique with the help of the Statistical Package

for Social Sciences software (SPSS). The independent/

predictor variables are the eight factors listed in Table 1

and the dependent variable is estimate bias (Y). The

factors were entered stepwise. Gross floor area (GFA),

project value and price intensity were included in the

regression model as continuous variables while other

factors were included as categorical variables according

to the groupings shown in Table 1. Thus the multiple

regression model developed to determine the impact of

the factors may be represented as follows:

Y~b0zb1GFAzb2ProjectValuezb3Price Intensity

zb4CBDzb5Metropolitanzb6Rural

zb7Traditionalzb8D&Czb9Residential

zb10Industrialzb11Commercialzb12Timber

zb13Steelzb14Concretezb151{2Storeys

zb163{7Storeyszb178Storeys and aboveze

where: Y5estimate bias;



b05is the Y intercept—it represents the average

estimate bias (Y) when all the independent factors in

the model are zero;

b1 to b17 are the slopes of Y associated with each

independent factor when other factors are held

constant;

e5random error in Y.

It is assumed the relationship between Y and the

independent variables can be approximated by a linear

model which provides best fit estimates of the model

parameters by minimizing the error of the model

(Draper and Smith, 1981).

The regression coefficients b0, b1, b2…b17 are

unknown parameters. The hypothesis of interest is

that: b15b25b3…5b1750. The predictive performance

of multiple linear regression may be judged by the value

of adjusted coefficient of determination (R2 adjusted).

The results are summarized in Table 3.

From the results (Table 3), bias in estimates of

projects is influenced by project value and number of

storeys. About 29% of the variation in the estimate bias

of projects can be attributed to project value and

number of storeys. Thus we may conclude that

significant variation in early stage estimate bias can be

explained by project size. The final regression equation

may be written as:

Y~0:5811{0:0318 project valueð Þ{0:0906

3{7storeysð Þ{0:0124 8storeys and aboveð Þ

The negative sign of the regression slopes (i.e.

20.0318, 20.0906 and 20.0124) implies that project

cost estimate bias tends to decrease as project size

increases. This further reinforces the findings from the

preliminary analysis (Table 1) and the Tukey–Kramer

procedure which suggest that estimates of smaller

projects tend to be more biased than estimates of larger

projects. As earlier suggested, this may be explained by

experience of estimators.

Lowe and Skitmore (2001) stated that estimators

prefer the use of individual data and experience.

Akintoye and Fitzgerald (2000) found that the three

main methods of cost estimating are: (1) standard

estimating procedure in which construction costs are

initially found and allowances for overheads and profit

are added; (2) comparisons with past projects based on

personal experience; and (3) comparisons with past

projects based on documented facts. They noted that

these three models are ‘experience based models’. Thus

there are reasons to believe that the higher level of bias

observed in smaller projects might be because such

estimates are prepared by junior and less experienced

staff.

Overall effects of price intensity and type of

principal structural material

In the multiple regression analysis process, price

intensity was entered first to remove any confounding

effects. However, price intensity had no effect in the

final model. Thus the data did not support the price

intensity theory (Gunner and Skitmore, 1999b).

The ANOVA analysis shows that estimate bias varies

according to the type of principal structural material

used (Table 2). However, when put together with the

other project factors in the multiple regression model,

‘principal structural material’ is not a significant

predictor of estimate bias. Perhaps the effect was

masked by project size (confounding effect) as postu-

lated by Gunner (1997).

Has pre-tender building cost estimate accuracy

improved over time?

A scatter plot was used to test Hypothesis 2 which states

that the accuracy of pre-tender cost estimate has not

improved over time. The ratio of estimate bias and project

value (bias ratio) was determined for each project.

Thereafter, the bias ratios of the 56 projects were arranged

chronologically according to the time that the estimates

were undertaken starting from 1999 to 2007. This yielded

a time series dataset. A scatter plot of the time series data

was then constructed (see Figure 2). Similarly, consis-

tency ratios were determined for the 56 projects and were

Table 3 Results of multiple regression analysis

Variable b Coefficient Standard error t value P value R2 (adjusted)

Project valuea 20.0318 0.1316 22.41 0.019 0.2838

Number of storeys

(3–7 storeys)

20.0906 0.0285 23.19 0.002 F58.27 p50.0001

Number of storeys

(8 storeys and above)

20.0124 0.0478 20.26 0.797

Constant 0.5811 0.2028 2.86 0.006

Notes: Variables are significant at the 5% level (P,0.05). a Continuous data Log transformed in order to normalize data.



arranged according to the time the estimates were

undertaken. A scatter plot of the time series data (for

consistency ratio) was constructed (Figure 3).

If the accuracy of estimates has improved over time,

the scatter plots should indicate a downward trend

towards zero on the Y axis as we move from 1999 to 2007

on the X axis in Figures 2 and 3. However, the scatter

plots suggest that the year of project estimate has no

effect on estimate bias and estimate consistency. The

trend lines are insignificant. Estimate bias is in the same

order of magnitude as it was in 1999, 2000, 2001 and all

through to 2007. The errors are random and incon-

sistent. Flyvbjerg et al. (2002) found similar results in a

study of the differences between estimated cost and

actual costs of 258 transportation infrastructure projects.

We may conclude that the accuracy of pre-tender

building cost estimates has not improved over time.

There may be four possible explanations for these results:

(1) Estimates of new projects are based on historical

cost data from past projects. Thus inaccuracies

are transmitted to new estimates over time.

Figure 2 Scatter plot of bias ratio vs. time

Figure 3 Scatter plot of consistency of estimate vs. time



(2) Firms do not monitor the performance of their

estimates in terms of accuracy and so are not

aware of any inconsistent error trend.

Knowledge of the trend in estimates error and

inconsistency in estimates should help firms

modify their estimating policy in reaction to the

observed inaccuracies and inconsistencies

(based on Morrison, 1984 findings).

(3) Estimating expertise and skills developed by

firms based on experience with past estimates

are lost, and so reflect the lack of reduction in

estimate bias over time. In Australia, a possible

explanation is that there is a shortage and high

turnover of quantity surveying and cost engi-

neering skills (reflected by the rating points

allocated to the profession by Australia’s

Department of Employment and Workplace

Relations (DEWR, 2008) Skilled Occupation

Lists for immigration purpose). Skill shortage

and turnover may affect the accuracy of

estimates when one considers that financial

management and cost engineering processes

involve assumed knowledge developed over

time. New employees, if recruited, will take

time to reach their full effectiveness.

(4) Other than technical factors such as the skill of

the estimating team and their experience, team

expertise, estimating techniques, or inadequate

data, human-related factors such as the esti-

mator’s attitude might significantly influence

the accuracy of estimates. For instance, esti-

mators are predisposed to increase their esti-

mate (overestimate) when prices are rising—

‘pessimism bias’; and when prices are falling

estimators are predisposed to reduce their

estimates—‘optimism bias’ (Mills, 1997). This

might be responsible for the lack of trend in the

errors (estimate bias) observed and the lack of

consistency in the estimates over time. Also the

inconsistencies in the estimates might be as a

result of wide variance in human judgement

suggesting that the human factor is critical

when attempting to increase the accuracy of

estimates.

Improving the accuracy of estimates

To address objective 3 of this study, the respondents

were presented with 12 methods that could be used to

improve the accuracy of estimates. The methods were

identified from the literature—particularly Ling and

Boo (2001). Respondents were asked to rank the

effectiveness of each method on a scale of 1 to 5

(where 15least effective and 55most effective). The

Relative Effectiveness Index (REI) for each method was

determined using the expression (adapted from

Kometa et al., 1994):

REI~A

B|C

where A5total score; B5highest response options;

C5total number of survey responses.

The REI was then used to rank the methods. From

the results (Table 4), Australian quantity surveyors

perceive ‘ensuring sufficient information is available at

the time of estimating’ as the most effective method of

improving estimating accuracy (first), followed by

‘increased cost planning and control during the design

phase’ (second) and ‘checking all assumptions with

clients and consultants during the estimating period’

(third).

The result is similar to that of Ling and Boo (2001),

who found that the most effective methods of improv-

ing accuracy according to Singapore quantity surveyors

were (a) ensuring design information is sufficient and

available for estimate preparation (M3) which ranked

first in this study; (b) checking all assumptions when

preparing the estimate (M4) which ranked third; and

(c) providing a realistic timeframe for estimating

activity (M6) which ranked seventh. The similarities

indicate an international sentiment regarding methods

of improving the accuracy of estimates. Simulation,

probability and utility function is considered by

respondents as the least effective method of improving

the accuracy of estimates (ranked twelfth).

Test of agreement among quantity surveyors

Fleiss’ kappa statistical measurement (k) (Fleiss, 1971)

was used to ascertain whether quantity surveyors in

Australia agree with the ranking of the 12 methods of

improving the accuracy of estimates. Fleiss’ kappa

measurement is a variant of Cohen’s kappa statistical

measure of inter-rater reliability. Cohen’s kappa is

suitable where there are only two raters whereas Fleiss’

kappa can help researchers to assess the reliability of

agreement between more than one number of raters on

a number of items. If a number of raters assign

numerical rating to a number of items then Fleiss’

kappa statistical measurement will give a measure of

how consistent the ratings are. It is a measure of the

degree of agreement that can be expected above chance

(Fleiss, 1971). Fleiss’ kappa statistical measurement

(k) can be expressed as:

k~P{Pe

1{Pe

where: 1{Pe is the degree of agreement that is



attainable above chance; P{Pe is the degree of

agreement actually achieved above chance.

The k statistics can take the values between 0 and 1.

According to Landis and Koch (1977) agreement

between respondents is Almost Perfect if k50.81–1.00;

Moderate if k50.41–0.60; Fair if k50.21–0.40; Slight if

k50.0–0.20 and Poor if k,0.00. In this study, the

measurement k is estimated at 0.045 indicating that

quantity surveyors only slightly agree with each other on

the relative effectiveness of the 12 methods of improving

the accuracy of estimates. A possible explanation may be

that respondents make use of the different methods to

achieve the same result. A one-tail t-test reinforces this

conclusion (Table 4). The mean effectiveness for all the

methods is statistically above the midpoint of 3 on a

Likert scale of 1 to 5 (p,0.05) indicating that quantity

surveyors consider all the methods as effective means of

improving the accuracy of cost estimates.

Reducing estimate inaccuracies in practice

Further, the respondents were asked to list internal

review mechanisms in place in their company aimed at

improving the accuracy of pre-tender building cost

estimates. Overall, 33 of 41 survey respondents (80%)

listed at least one review mechanism used in their firm.

In total, 77 review mechanisms were listed and content

analysis was performed on the 77 items. Similar

mechanisms were grouped together and named by the

researchers. Figure 4 shows the frequency of 13

mechanisms identified from the content analysis.

Overwhelmingly, ‘benchmarking’ is the most popular

method used by firms to improve the accuracy of

estimates, followed by ‘internal peer review’ and ‘com-

munication with the market’ which both have relatively

smaller frequencies when compared with ‘benchmark-

ing’. This suggests that estimates are largely based on

cost data from past projects rather than on internal peer

review and market research. This reinforces the explana-

tion that errors in estimates are transmitted from past

projects to new projects. This could be responsible for

the lack of improvement in estimate bias over time.

The content analysis revealed that the use of

computer estimating software (M5) is not frequently

mentioned as a method to improve the accuracy of

estimates (mentioned by only three out of 41 firms—

7%). Also, identification and incorporation of future

market trends into estimates (M4) was mentioned only

seven times. These results also agree with the ranking of

the effectiveness of 12 methods of improving the

accuracy of estimates (Table 4) which shows that

incorporation of market sentiments into estimates using

simulation, probability and utility function (IM8) is the

least effective method of improving the accuracy of

estimates according to the respondents. Probability

estimation or simulation to predict future cost trends or

to extrapolate new estimates from past estimates could

reduce bias in cost estimates for new projects.

However, the results suggest that it is scarcely used.

This indicates that there is potentially low uptake of

computerized statistical techniques such as cost mod-

elling in the industry.

Table 4 Effectiveness of mechanisms for improving estimating accuracy

Improvement

method

Total score Mean One-tailed t-test* (t.3) REI Rank

t-value p-value

IM1 157 4.13 6.85 0.0000 0.826 5

IM2 167 4.18 8.23 0.0000 0.835 4

IM3 180 4.62 14.98 0.0000 0.923 1

IM4 172 4.20 7.81 0.0000 0.839 3

IM5 146 3.65 4.11 0.0001 0.730 10

IM6 155 3.88 6.49 0.0000 0.775 7

IM7 134 3.44 2.74 0.0047 0.687 11

IM8 132 3.38 2.07 0.0230 0.660 12

IM9 149 3.82 5.14 0.0000 0.745 8

IM10 169 4.33 9.27 0.0000 0.845 2

IM11 145 3.82 6.57 0.0000 0.744 9

IM12 153 4.03 7.15 0.0000 0.785 6

Notes: REI5Relative Effectiveness Indices of improvement methods. *One-tailed t-test of mean. IM15Ensure proper design documentation.IM25Establish effective communication and co-ordination between members of the project team. IM35Ensure sufficient information isavailable for estimating. IM45Check all assumptions with clients and consultants. IM55Establish formal feedback for design and estimatingactivities. IM65Provide a realistic timeframe for estimating activity. IM75Use a more rigorous method of estimating. IM85Incorporate marketsentiments and economic conditions into the estimate by way of simulations, probability and utility functions. IM95Incorporate other marketsentiments and economic conditions into the estimate. IM105Increase cost planning and control activities during the design stage.IM115Improve methods of selection, adjustments and application of cost data. IM125Update cost database with new cost analyses and providefeedback for improving estimate accuracy.



Conclusion

Data analysis revealed that although bias in pre-tender

building cost estimates varies according to project size and

principal structural material used, when all factors are

controlled and combined in a multiple regression analysis,

bias in estimates of projects is significantly influenced by

project size. Estimates of smaller projects tend to be more

biased than estimates of larger projects. None of the eight

project factors studied significantly contributed to the

level of consistency observed in project estimates. Thus

Hypothesis 1 is partly supported.

Firms need to pay greater attention to smaller or less

complex projects. When cost estimates of less complex

projects are undertaken by less experienced members of

staff, they should be reviewed more rigorously by senior

and experienced quantity surveyors.

The preliminary analysis suggests that bias in the

estimates of large projects is smaller but the estimates

are inconsistent. As modern projects are becoming

more complex, previously used estimating techniques

may be inadequate and may not be as effective as they

were previously. Quantity surveyors and cost engineers

need to use suitable estimating techniques if cost

certainty on projects is to be assured.

This study supported Hypothesis 2 which states that

the accuracy of pre-tender building cost estimates have

not improved over time. Quantity surveyors only

slightly agree on the techniques for improving the

accuracy of estimates. Thus Hypothesis 3 is not

supported. Although firms are using benchmarking of

estimates of previous projects to improve the accuracy

on new projects (36% of all responses), the effective-

ness of such an approach would depend on how often

and how accurately cost databases are updated to

incorporate market sentiments and economic condi-

tions. It would also depend on firms understanding the

size and trend of inaccuracies in their past estimates,

the factors influencing the inaccuracies observed and

incorporating such knowledge into new estimates. The

use of probability estimation and simulation is a way

forward in this regard. However, there appears to be a

low uptake of this approach according to the respon-

dents. There is need to create awareness of the benefit

of computerized statistical techniques such as cost

modelling.

Figure 4 Frequency of mechanisms used by firms for improving the accuracy of estimates

Key& M1 Benchmarking& M2 Internal Peer review& M3 Communication with the market& M4 Identify and incorporate future market trends into the estimate& M5 Use of computer estimating software& M6 Bulk Checking / Self Checking procedures& M7 Ensuring proper communication / information flow on the project& M8 Use of external price information& M9 Review with final costs on projects& M10 Comparisons with received tenders for future estimates& M11 Elemental review& M12 Internal Quality Assurance procedures& M13 Identification of project specific needs or risks



Firms need to find means of retaining the knowledge

and experience gained on estimates of past projects.

Rigorous cost analysis and documented feedback from

estimates of projects could help firms to pass on the

knowledge gained when estimating new projects. Also,

firms need to find ways of retaining their staff.

Further, quantity surveyors and cost engineers need

to be directly involved during the client briefing at the

project inception in order that they might adequately

understand a client’s requirements, rather than depend

on relayed information from the project manager or

architect. In order to reap and maximize the benefits of

cost engineering and quantity surveying skills, clients

need to appoint quantity surveyors and cost engineers

from project inception.

This is one of the few studies on this subject in an

Australian context. Its contribution is in the approach

used, which involves analysis of real life data and survey

data from professionals across Australia. The limita-

tions are acknowledged. The number of projects used

for the empirical analysis (56) was not large enough

hence small sample groups resulted when projects were

divided and analysed according to the eight project

factors. This placed restrictions on the ability to detect

higher significant effects. However, the results provide

a plausible description of the size and the pattern in the

accuracy of estimates. The research approach and the

step-by-step analysis can serve as a model for others

who may wish to conduct similar studies elsewhere on

this topic and could facilitate international comparison.

References

AIQS (2003) Find Member, QS Firms, and Best Practice

Charter, available at www.aiqs.com.au (accessed June

2007).

Akintoye, A. and Fitzgerald, E. (2000) A survey of current

cost estimating practices in the UK. Construction

Management and Economics, 18, 161–72.

Bromilow, F.J., Hinds, M.F. and Moody, N.F. (1988) The

Time and Cost Performance of Building Contracts 1976–1986,

Australian Institute of Quantity Surveyors, Canberra.

Brown, M.B., Forsythe, A.B. and Alan, B. (1974) Robust

tests for equality of variances. Journal of the American

Statistical Association, 69, 364–7.

Conover, W.J., Johnson, M.E. and Johnson, M.M. (1981) A

comparative study of tests for homogeneity of variances,

with applications to the outer continental shelf bidding

data. Technometrics, 23(4), 351–61.

DEWR (Department of Employment and Workplace

Relations) (2008) Skilled Occupation Lists (2008), avail-

able at www.immi.gov.au/allforms/pdf/1121i.pdf (accessed

30 May 2008).

Draper, N.R. and Smith, H. (1981) Applied Regression

Analysis, 2nd edn, Wiley, New York.

Fleiss, J.L. (1971) Measuring nominal scale agreement

among many raters. Psychological Bulletin, 76(5), 378–82.

Flyvbjerg, B., Holm, M.S. and Buhl, S. (2002)

Underestimating works in public works projects: error or

lie? Journal of the American Planning Association, 68(3),

279–95.

Gunner, J. (1997) Accuracy of construction pre-bid forecasts,

PhD thesis, University of Salford.

Gunner, J. and Skitmore, M. (1999a) Comparative analysis of

pre-bid forecasting of building prices based on Singapore

data. Construction Management and Economics, 17, 635–46.

Gunner, J. and Skitmore, M. (1999b) Pre-bid building price

forecasting accuracy: price intensity theory. Engi-

neering, Construction and Architectural Management, 6(3),

267–75.

Kometa, S.T., Olomolaiye, P.O. and Harris, F.C. (1994)

Attributes of UK construction clients influencing project

consultants’ performance. Construction Management and

Economics, 12(5), 433–43.

Landis, J. and Koch, G. (1977) The measurement of observer

agreement for categorical data. Biometrics, 33(1), 159–74.

Levine, D.M., Stephan, D., Krehbiel, T.C. and

Berenson, M.L. (2005) Statistics for Managers, 4th edn,

Prentice Hall, Upper Saddle River, NJ.

Ling, Y.Y. and Boo, J.H.S. (2001) Improving the accuracy of

approximate estimates of building estimates. Building

Research & Information, 29(4), 312–18.

Lowe, D. and Skitmore, R.M. (2001) Human effects in

construction contract price forecasting: experience and

experimental learning styles. Journal of Construction

Engineering and Management, 127(6), 485–93.

Mills, A. (1997) Analysis of the accuracy of building price

forecasts. Australian Institute of Quantity Surveyors Refereed

Journal, 1(2), 31–8.

Morrison, N. (1984) The accuracy of quantity surveyors’ cost

estimating. Construction Management and Economics, 2,

57–75.

Rawlinsons (2006) Australian Construction Handbook 2006,

Rawlinsons, Perth.

Rawlinsons (2007) Australian Construction Handbook 2007,

Rawlinsons, Perth.

Serpell, A.F. (2005) Improving conceptual cost estimating

performance. AACE International Transactions, EST.13,

EST.13.1–13.6.

Skitmore, M. (1991) Early Stage Construction Price

Forecasting: a Review of Performance, RICS Occasional

Paper, London.

Skitmore, M. and Drew, D. (2003) The analysis of pre-tender

building price forecasting performance: a case study.

Engineering, Construction and Architectural Management,

10(1), 36–42.

Skitmore, R.M. and Picken, D. (2000) The accuracy of pre-

tender building price forecasts: an analysis of USA data.

Australian Institute of Quantity Surveyors Refereed Journal,

4(1), 33–9.

Trost, S.M. and Oberlender, G.D. (2003) Predicting

accuracy of early cost estimates using factor analysis and

multivariate analysis. Journal of Construction Engineering and

Management, 129(2), 198–204.



the accuracy of pre-tender building cost estimates in australia

Documents

skilled occupation lists

gross floor area

central business district

principal structural material

computerized statistical techniques

internal peer review

multiple regression analysis

price intensity theory