Download - [Norman R. Augustine] Augustine's Laws (1)
-
8/10/2019 [Norman R. Augustine] Augustine's Laws (1)
1/234
*-
%A F
D4
-
8/10/2019 [Norman R. Augustine] Augustine's Laws (1)
2/234
Augusfines
Laws
= A
-
8/10/2019 [Norman R. Augustine] Augustine's Laws (1)
3/234
NORMAN R.
AUGUSTINE
aw s
AND MAJOR SYSTEM DEVELOPMENT PROGRAMS
AIAA-M
American
Institute
of
Aeronautics and Astronautics
370
L'Enfant Promenade,
SW
Washington,
DC
20024-2518
-
8/10/2019 [Norman R. Augustine] Augustine's Laws (1)
4/234
American Institute of Aeronautics and Astronautics,
Inc.
Washington,
DC
Library of Congress
Cataoglwg
in Pubflcatlon
Data
Augustine
Norman R.
Augustine's Laws and major
system development
programs.
Revised
and
enlarged.
1. Industrial
project management-Anecdotes,
facetiae, satire, etc.
2.
Management-Anecdotes,
facetiae, satire, etc.
1.
Title:
Laws and
major
system development programs.
TL56.8.A93
1983 658.4
83-22409
ISBN 0-915928-81-7
Second
Edition
Sixth
Printing
Copyright C
1983 by Norman R.
Augustine.
Published by the
American Institute
of
Aeronautics
and
Astronautics,
Inc. with permission.
Viking
Penquin, Inc. (40 West
23 St., New York,
N.Y. 10010) has
Is-
sued a
revised and expanded
version
of
this edition
for the
general
mar-
ket.
All rights reserved. No
part
of
this book
may
be reproduced in
any
form or by any
means, electronic or mechanical,
including photocopy-
ing and
recording,
without permission
in writing
from the publisher.
-
8/10/2019 [Norman R. Augustine] Augustine's Laws (1)
5/234
To
those
many individuals,
in
government
and out, who through
sheer ability and dedication
have
achieved
so very
much; too
often
in
spite
of
the
system
intended
to
support them.
-
8/10/2019 [Norman R. Augustine] Augustine's Laws (1)
6/234
You
can
see
a lot by observing.
YOGI
BERRA
-
8/10/2019 [Norman R. Augustine] Augustine's Laws (1)
7/234
Preface
Insight into
the
problems of management
is sometimes found in
unexpected places. For
example,
A.
A.
Milne could well have
been
writing about the vicissitudes
of
managers
of large
system develop-
ment activities in the opening paragraph of Winnie-the-Pooh: "Here is
Edward
Bear,"
he
wrote,"coming downstairs
now, bump, bump,
bump, on the back of his
head,
behind Christopher Robin. It is, as far
as
he knows, the only
way
of coming downstairs, but sometimes
he
feels
that there
really is another way.
. . if
only
he
could
stop bumping
for a moment
and think
of
it!"
Like
bears,
all
too
seldom
do
managers
take the
time to learn
from
their everyday
experiences.
It is much as
the
evidence reported by the
newspaper, Midlands of
England,
regarding a problem whereby long
queues
of would-be
passengers
wishing to use
the
Bagnall to
Greenfields bus
service
were being passed
by
drivers in half-empty
busses. As
reported in
the above
newspaper, bus
company
officials
countered objections to this annoying practice by pointing out that "it
is impossible for
the
drivers to
keep
their
timetables if
they must stop
for
passengers."
This
brief treatise seeks to take a respite from the
pressures
of
everyday
schedules for a moment of introspection to see
if
in fact
there
might
not
be,
as Edward Bear suggests, a better
way.
Thomas
Edison
even assures "there is
a better way," and then counsels, "find
it "
And, indeed, there
is a better way, as innumerable highly suc-
cessful
programs have
demonstrated.
Still, there remains that
large set
of
much maligned projects which, were they
ever
to be documented
into
a
movie,
might
best be viewed with the
film
run backward in order
to insure
a happy ending.
It
is largely
from
this latter category of
programs
that Augustine's Laws have been formulated. The laws are
dedicated to the proposition that, with
a better
understanding
of the
history
of
past programs,
one
need
only selectively repeat history
in
the future. In Bismarck's words, "Fools you are... to say you learn by
your
experience. . . I
prefer
to
profit by
others'
mistakes
and
avoid the
price of
my
own." This is
in
keeping with Augustine's Zeroeth Law of
Aeronautics: "Never
fly
on
an
airplane
with a
tail
number
less
than
ten.."
Perhaps the
principle
dilemma posed in these pages is not to
managers
but rather to
librarians:
Should
the book
be categorized
as
comedy or as
tragedy?
Or perhaps science friction?
Many
of the tribulations
which
will
be encountered
in
these
pages
V
-
8/10/2019 [Norman R. Augustine] Augustine's Laws (1)
8/234
vi
are
by
no means unique
to
defense
programs
but will, unfortunately,
be
recognized
to have rather broad
applicability to
a variety
of fields
of
endeavor.
The
author
has
merely
elected
to
depart
from
the
tradition
of most Washington writers
and address
matters with
which
he
has
some
familiarity. . .
thus
the
concentration
on
the
defense acquisition
process.
Although
treated in a
sometimes tongue-in-cheek
manner,
the
problems
addressed
are
nonetheless
unmistakably both
real
and
deserving of attention. At
times
slightly
irreverent
toward
"the
system," the
author hopes
to
improve
upon that
system, which has,
in
spite
of
its
many
pratfalls, demonstrated truly enormous
inherent
strength
and
accomplishment
largely
because
of
the
dedication and
native ability of
those individuals
who make
it work.
The
author is
proud
to have
counted
those people as his associates, both
in
government and
out,
and
holds an abiding
respect
for their
courage,
their integrity
and
their
contribution.
The
present
volume comprises
the
fifth
printing
of
these
laws, with
the
earlier, less
complete versions having originally been referred to as
"The
Compleat
Augustine's Laws." Such
is the transitory
nature
of
immutable
laws.
The
author
would
like
to
express
his
appreciation to Paul Blumhardt
and James
Morrison
for
their
assistance in
collecting certain portions
of the
statistical
data con-
tained
herein,
to Pamela Seats for reviewing this manuscript and
to
Rhoda
Glaser
and
Glenda
McFarlin
for
their help in ts preparation.
-
8/10/2019 [Norman R. Augustine] Augustine's Laws (1)
9/234
Table of
Contents
Preface ..............................
v
Chapter 1 - Unbounded Enthusiasm ............................... 1
I Paper Airplanes ...................................... 3
11 Justice Deserts ....................................... 9
III Telling It Like It
Isn't
................................... 13
IV
On
Making
a Precise Guess ............................. 23
V
FYI
............................................... 29
VI
Replacing
Congress with
an Equation ......................
37
Chapter
2
- Minor Oversights ...................................
41
VIl Marginal Costs ...................................... 43
Vill
Costing
Enough
To
Be
Useful
........................... 49
IX
The High
Cost
of Buying ............................... 53
X Wait 'Til Last Year .................................... 61
Xl
The
Reliability of
Unreliability
............................
67
XII
A
Long
Day's Night
...................................
71
XIII Malice in Wonderland ................................. 75
Chapter 3 - The Gathering Storm ................................ 79
XIV Piled High
.......
81
XV
What Goes
Up
...
Stays
Up .............................
87
XVI All
Started by
a
Spark ................................. 93
XVII
Bit by Bit ........................................... 99
XVIII
Off
Again,
Off
Again
.................................
105
XIX Work and
the
Theory of Relativity ....................... 111
XX
Striving
To Be
Average
...............................
115
XXI
The
Amoeba
Instinct .................................
119
XXII Hail
on
the Chief ................................... 123
Chapter 4
-
Impending Disaster
................................
129
XXIII
The
Reality of the Fantasy Factor ........................
131
XXIV Certain Uncertainty .................................. 135
XXV Buying
Time.......................................139
XXVI So Simple It
Can't
Be Trusted .......................... 141
XXVII The
Law of
Diminished Returns .........................
145
XXVIII Seeking To Profit from One's Inexperience ................. 149
XXIX
The Manager
of
the
Year
............................. 155
XXX The Half-Life of
a
Manager
............................ 159
XXXI
Anonymity
by Committee
.............................
163
vii
-
8/10/2019 [Norman R. Augustine] Augustine's Laws (1)
10/234
viii
Chapter 5 - Disaster ..........................................
169
XXXII For
the
Want of a
Nail
................................
171
XXXIII
To
Work or Not To Work
.............................. 175
XXXIV Caveat Emptor ..................................... 179
XXXV
Too Late Smart
183
XXXVI
So
Old for
Its Age
................................... 189
XXXVII
Meetings Dismissed
.................................. 193
XXXVIII On Doing Less
with More
.............................. 199
XXXIX Going Nowhere,
but Making
Good
Time
.................. 203
Chapter 6 -
Disaster Revisited
..................................
207
XL
Watching
the
Watchers Watch
..........................
209
XLI
Much Ado About Nothing ............................. 213
XLII Growing Like a Regulation ............................. 217
XLIII Regulatory
Geriatrics
.................................
225
XLIV
Employer of Only Resort .............................. 229
XLV For What
It's
Worth, Save Your Money
...................
233
Epilogue
..................................................... 237
-
8/10/2019 [Norman R. Augustine] Augustine's Laws (1)
11/234
"Reproduced
from
JANE'S
HISTORICAL AIRCRAFT 1901-1916
by permission of
Jane's
Publishing
Company,
Limited,
238
City
Road, London
EC1V
2PU,
England."
A:Np,4
*
IN
.W
-N
. 111 v V1
"A
- - -- --OVA - ro
- I
-
8/10/2019 [Norman R. Augustine] Augustine's Laws (1)
12/234
Chapter 1
Unbounded
Enthusiasm
-
8/10/2019 [Norman R. Augustine] Augustine's Laws (1)
13/234
-
8/10/2019 [Norman R. Augustine] Augustine's Laws (1)
14/234
Paper
Airplanes
I
read
part
of
it
all the way through.
Samuel
Goldwyn
It
was
the twenty-fourth of December and the government
had
just
released the Request
for
Proposal to industry for
what many said would be a program the likes of which had
never before
been
seen. The government always releases
Requests for Proposalson the twenty-fourth of December.
That is in part why "RFP"
is
a four-letter word. The event
thus
was
not altogether unexpected. In
fact,
for
over
three
years industry
hadbeen hard at work preparing tself
for
this
day.
Operating divisions within
giant industrial
firms had
been forming teams with divisions from other giant in-
dustrial
firms
in order to compete for
the
impending con-
tract.
An alternative
would, of
course,
have been
to form
teams
with
sister-divisions
from their own companies;
however,
this is seldom done due to
the difficulty
of
cooperating
with
one's
competitors. But once
having
formed up into corporatecoalitions everyone
went
to work
putting
the finishing touches on the proposal,
with the
engineering department
struggling
to grind out a
market-
oriented sales document, the finance department
wrestling
with
the problem of
estimating
the
engineering manhours
that would be
needed
to develop a microprocessor, and the
marketing department
determining
the
program's cost
which
would
actually
be
shown in the
bid.
It
was
already
apparent
hat
this
would be
a challengingproject
because of
the urgent need
to
recover during the development
effort
the two-year delay
which had been incurred
by
the
government
in
this very
important
project while trying
to
decide
whether
to
begin it or not.
Thus
came
about the
makings of what would
prove to be,
truly, a crash project.
Modern alchemists
of
the aerospace
industry, having
presumably
despaired of turning
lead into gold, have
progressed
into taking what
used to
be
aluminum and turning it
into
paper. That they have done
so
with
considerable alacrity is indicated
by the fact that the most
critical
aerospace
material is no longer cobalt,
titanium
or chromium,
but
is now widely considered
to be
woodpulp. This situation has not
prevailed
since the halcyon days
of
Howard Hughes'
famed
plywood
aircraft,
the Spruce Goose.
In
fact, the
only
material
playing a
more
3
-
8/10/2019 [Norman R. Augustine] Augustine's Laws (1)
15/234
4
pivotal role than paper
in
aerospace today is celluloid, commonly
used in the
thousands
of viewgraphs which are required to counter
would-be opponents of proposed
projects by
briefing them into
submission.
Figure 1 relates the number of pages in typical
proposals
for
new
projects
to the
dollar
value
of the
programs they potentially
produce,
the latter
based
on the program
plan at the time
the
proposal was
submitted. The points above the trend-line often correspond to
programs deemed by the
contractor to have
significant "growth"
potential (of one kind or another) while those below the line may have
been page-limited
by
fiat
or discounted in value by
the competitors
due
to
the
likelihood of
eventual
program
cancellation.
An
important
underlying
measure of merit for proposals which
is
derivable from
the
data shown is called the "Load Factor" (often misused to represent
acceleration levels or passenger
occupancy
in commercial aircraft),
and is
found
by dividing the height of the pile of paper required to
compete for a project
by
the
dollar-value
of the project.
The empirically determined value
of
the single-copy Load Factor
for programs in the multimillion-dollar range is seen to be ap-
proximately
one
millimeter
per
million.
It
is
believed
to
be
significant
that this
factor
implies that the pile
of
paper required to compete for a
billion-dollar program, assuming the traditional 50 copies are sub-
mitted,
must equal
the "worth" of that program as represented by a
stack
of
"2,000-dollar" bills. The
fact that
there are no such bills
should not
be
viewed
as any
particular
deterrent, at least not in
comparison with much
of
the other material that
appears
in proposals.
Contractors are
firm adherents
to
the views of
Horace,
"Brevis esse
laboro,
obscurus
io,"
liberally
translated,
"When
I
struggle to
be
brief,
I become
unintelligible."
The problem is that in the
case
of most
contractors, when they don't struggle to be brief, they also become
unintelligible.
One enthusiastic proposal
manager
described the
end-game
in
the
following terms: "We
shipped more
than
32
cases of
proposals
to
the
customer. Stacked up, the content of these cases would have made a
pile
at least
75-feet
high. Everyone
really
pitched in
to meet
our
deadline
-
word
processors worked day
and
night, and
'reproduction' printed more than 284,000 pages."
The truly classic cases include
the TFX,
for
which
the total set of
copies for one bidder'sproposals submitted during the four rounds
of
competition
reached
a final height
of
211 feet, and
the
giant cargo
aircraft, the
C-5A. In
the
latter instance,
just one of the
three
bidders
submitted
a
total of 1,466,346
pages
weighing in at
24,927
lbs.
The
Request for
Proposal
issued by the government
itself
occupied
1,200
pages.
.
.
and
was later
supplemented
by a "Clarification
Document"
-
8/10/2019 [Norman R. Augustine] Augustine's Laws (1)
16/234
U
2
0. 0
.2
o
04-
-0 0.
-
0
*-*-
0
41 1 J
Co
,Su
o
so
0os
S
sC
~8 02
0~000wMH-O.
2
2
n.4- w : Q
O0
Z
.
C.) . O
.w
444
u>
O~
0404~,CoN .040 040440o
> 00
44z
4 C o o o zoC
=oCoCo2
0
20
C.)
01
0Il
02
4~d
u
IC
0
a-
0
a-
to
0
S
1-
I
-
8/10/2019 [Norman R. Augustine] Augustine's Laws (1)
17/234
6
of over 1,600
pages
A total of some 500 evaluators were
needed to
wade
through
the
material provided
by the
three bidders.
"Why
use
one word," as
the
saying
goes, "when
ten will
do?"
The degree
of
improvement wrought by the
growing
length
of
proposals
and contracts
as they
have
evolved
over
the
years is
suggested
in
the
tale of
two airplanes.
When
the Army
Signal
Corps
purchased the development of an aircraft from
the
Wright Brothers,
the Request
for
Proposal consumed
fully
one page; the entire contract
(a fixed-price incentive type)
comprised two
pages.
The latter was
the
result
of
a
40-day competition
among
41 bidders
which culminated
in
a
9-day evaluation
period
by the
government. An
award was
made*
and
the
aircraft
flew
successfully
some
six months
later.
The primitiveness
of this management
system
contrasts sharply
with
the more sophisticated
approach
used
today which,
in
the case
of the
giant C-5A transport, generated
contractor
proposals the
paper for
which
would have more
than
filled
the
C-5A itself. The recent Ad-
vanced Helicopter Improvement Program (AHIP) competition ex-
ceeded
even this
standard
of
achievement . . .
with
one contractor's
proposals (all required
copies)
exceeding
the takeoff
weight of
the
helicopter. Many
systems,
it
seems, are now
quite
literally
worth their
weight in paper.
In
fact, the
former
President of Vought
Aeronautics
estimated
in The Dallas Times Herald that each
time a
new
military
airplane flew
over the fence
at
their plant, 27
percent
of its cost was
attributed
to paper. A single copy of a winning
proposal
for a modern
aircraft
requires
a
document embodying
a preparation
cost
per pound
(including
the
contractor-sponsored effort
to develop
the
information)
about
400
times the
cost
per
pound of
the
aircraft itself.
But
the above data
include only
the
initial
proposal
and
not
all
the
resubmittals
which often double
or
triple
the
pile which must be
provided.
Nor
do
the
data include supporting documentation which
must eventually
be
generated. . . such as the
MX
Environmental
Impact Statement
which
ran
a
full 8,000 pages even
in its draft
form.
One recent government
publication on
the marketing of cabbage
contains,
according to one report,
26,941
words. It is noteworthy in
this regard
that the Gettysburg
Address contains
a
mere 279
words
while
the Lord's Prayer comprises but 67.
Fortunately,
the
United Nations
recognized
the
existence
of this
situation and established
a
Committee on the Reduction of Paper-
work.
The Committee has now released
a
219-page report con-
cluding
that
paperwork should indeed be reduced. Similarly, the U.S.
Commission
on
Federal Paperwork produced
a
widely distributed
74-
page
report,
which
was
unfortunately
soon surpassed
by a 113-page
*Without
protest.
-
8/10/2019 [Norman R. Augustine] Augustine's Laws (1)
18/234
7
report
on
the same
topic generated by the
succeeding
administration.
Not to be outdone, the Defense Department has commendably
taken
steps to page-limit proposals submitted
by
its would-be con-
tractors. One
of
the first
efforts to
accomplish
this
involved
limiting
proposals for
one particular
program
to
1,000
pages. . . a laudable
concept had
not
the government's
own Request
for
Proposal bulged
1,114
pages.
The
initial response of
the bidders,
who
for years had
been
complaining
about the
length
of proposals, was not
particularly
commendable either; they produced documents for which
the print
size rivaled
the
best products
of
their microelectronics
production
facilities and invented
"foldouts"
with margins seemingly
1 Angstrom
wide.
The blame for this
verbosity
is not. however, entirely assignable
to
the contractors. They.
too.
are often faced with a veritable flood of
vagaries to which
a specific and detailed
response
is
demanded.
In
fact, as viewed
through
the
eyes of competing contractors the source
selection proves often appears to be somewhat an enigma wrapped in
a mystery.
Reluctance
to state
specific
scoring
weights
to
the bidders.
for
example. results
in
statements such
as
the
following
taken
from
one
recent Request
for
Proposal:
"The
weighting
of
technical factors
exceeds
that of all
other
individual factors but is
less
in value
than
the
sum
of cost
and
schedule.
Technical
excellence is weighted
more
heavily than cost which in turn receives more value than schedule
which
is equal
in
value
to
the combination of
management and
past
performance.
No
one factor represents half of the
total weighted
value
to be
used in
the contractor selection "
A
story in
Government Executive magazine tells of the Un-
dersecretary
of
the
Army
pointing
out
to
his
contract
manager that the
length
of
a recent Army Electronics Command Request for Proposal,
coupled with
the alotted
30-day
response
time, would very likely pose
problems to industry.
Specifically,
he
noted,
if
one
were to work 24
hours a day, seven days a week, for the
entire month, only
ten
minutes would be available to read, digest, and prepare a response to
each page
of
the government's request
Recent efforts
to select contractors
for new items
of hardware
conducting
"flyoffs" of
actual
prototype hardware instead
of
paper
engagements
have
produced some
astonishing
results. In those
cases
where
the selection
has
been based upon
a conventional evaluation
of
paper proposals, the
incumbents,
defined as
the
builder of the
previous generation of
the
item of
equipment
to be replaced, won
about two-thirds
of the
competitions. In contrast, when flyoffs of
actual hardware were involved, the darkhorse (newcomer) exactly
reversed the
above odds, winning
two
times
out of three.
-
8/10/2019 [Norman R. Augustine] Augustine's Laws (1)
19/234
8
The
explanation of
this happenstance
has
its origins
in
biblical times:
"He multiplieth
words
without knowledge":Psalm
35:16. Or, in more
contemporary
parlance,
"I should have asked
the
question you an-
swered. "
The
overall law resulting
from
these
considerations
appropriately
has its title
derived
from
computer parlance
and
is
known as
the
Law
of the Core
Dump:
The thickness of the
proposal required
to win
a
multimillion dollar contract is about one millimeter
per
million.
If
all
the proposals
conforming
to
this standard
were
piled
one
on
top
of
the
other at
the bottom
of
the
Grand Canyon, it would probably be a good idea.
(LAW
NUMBER
I)
-
8/10/2019 [Norman R. Augustine] Augustine's Laws (1)
20/234
Justice
Deserts
Fool
me once, shame on you.
Fool
me
twice, shame
on
me.
American
Indian expression
To err is human. To forgive
is
against
procurement
policy.
Or is it? Having preparedthe requisite
mound
of paper and
promises,
the
hopeful
contractors
pass
from
the
stage
momentarily and the second portion of
the courtship
begins.
This phase belongs
to the source-selection
evaluators,
government
employees who are assigned the
unenviable
task
of
reviewing
the mountain of material
which
has been painstakingly
prepared
by the contenders.
The source-selection process reaches its culmination when
that
decisive
and divisive
step having such a great
sound of
finality is
reached:
the submission
by
each contractor of
what is
officially
known as a "Best and Final" offer but is
more
aptly
described
by
many in
industry as
"Last and
Worst." Unfortunately,
in
this
project,
as
in many others,
the process culminated several
times
-
as
no
fewer than
four successive "Best and Final"
offers
were solicited from
each competing contractor. The entire process had
somehow
degenerated
into one
of
"Do
unto
others
before
they do
unto
you." Worse yet, it was soon to
be
learned
that,
as
George Eliott long ago noted
in
Silas
Marner,
"Nothing
is
so
good as it seems beforehand.
Would any private consumer continue to patronize a
seller
who had
just charged
50
percent
more
for
an
item than had
been
indicated in
the
original agreement,
delivered
the
article
one-third later
than
promised, and capped
this
off by changing the management being
dealt
with three
times
during the
course
of the
purchase?
Not
likely.
Yet this
has
quite
literally
been
the
norm*
in
the
sophisticated
process
of acquiring major system developments
and
leads to what is in-
formally
referred
to as the
"Lemming
Law of
Government
Procurement."
That is
not
to suggest that
there
are
not
many ex-
tenuating circumstances from the perspective of both the
seller
and
*The
indiscreet use
of
this
word
has,
on
particularlyuninspired
occasions, led to these
laws
being
referred
to
as "Augustine's
Norms."
9
-
8/10/2019 [Norman R. Augustine] Augustine's Laws (1)
21/234
10
Memory
in
the Source Selection Process
9
ARMY
PROGRAMS
8
AIR FORCE PROGRAMS
4 NAVY PROGRAMS
-- BASED
ON
BUSINESS BASE (MAJOR PROGRAMS)
ZO
= 15
0 -
F
10
C 0
C
2
z
cm
& -
CC Z
UJ Z
a:
= 0
Z LL
< -5
-10
AT TIME OF AWARD
REWARD
POOR
PERFORMANCE
- * *
VB
* % Z REWARD
GOOD
* 0
%
PERFORMANCE
01 I,.1 0
0
-
0
I I I I I I I I
-10
-5 0 5 10 15
20 25
30
ANNUAL PERCENT
COST GROWTH
FOR INDIVIDUAL LOSING
CONTRACTORS
DATA SOURCE: DOD
SELECTED
ACQUISITION REPORTS, FIVE
YEAR
AVERAGE
CORRECTED FOR INFLATION & QUANTITY CHANGES
Figure
2
The process of
selecting acontractor to undertake a project generally gives
little formal
consideration
to
the past
records
of
the contending
firms.
Only
slight
correlation
is found
between
cost
control
on
previous
contracts and
the likelihood
of
winningfuture
awards.
the buyer. There are. But to perpetuate a procurement policy
founded, perhaps
even floundered, on the virtues of amnesia
would
seem to
be contrary
to
fundamentals
of
the Free
Enterprise
System.
Many contractors, explaining away
disappointment with their past
work, subscribe
to Frank Lloyd Wright's viewpoint. "When Iwas
90,"
said
Wright,
"I
was
asked
to
single
out
my
finest work.
My
answer
was
'My
next
one.' " Similiarly, if the
corporate vice president of
marketing
points
out that
"we have
not had
a major
disaster
since 1955," the
government project manager is unable to take much solace
when a
quick check of his watch reveals it is
then
only 2015.
Figure
2
presents
evidence that whatever
other
problems may be
attributable to
the contract
award process, it is
at
least free
from
discrimination. Free from discrimination,
that
is, in the sense
that
it
-
8/10/2019 [Norman R. Augustine] Augustine's Laws (1)
22/234
I1
treats good
performers
and
poor performers
with
equanimity. The
figure displays for a number
of major
source
selections the
relative
ranking of the winning and
losing
bidders in
terms
of
one
important
measure
of
past performance: the degree
of
cost
control exhibited
on
their in-being major programs
at the time
of
the subject new award.
Were
all the data
points
to cluster in
the
lower-right-hand
corner of
the
figure, one
might conclude that contractors
which performed well
in
the
past were more
likely
to win
new business in the future and,
likewise,
that poor performers
were
less
likely to
be
future
winners.
But, as seen from
the
lack
of any such correlation
in
the
data,
the
extant version of turning the
other cheek seems
merely to
be
to
turn
the
other
check.
In
essence, then,
in
the
source-selection
process
the
accumulation
of, say, twenty
years' experience more
accurately
corresponds
to accumulating one
year's experience twenty
times.
These
results agree with the findings
of
a companion qualitative
investigation which
rated
competing
contractors
on all aspects
of their
past record
(cost, schedule, performance.
. . ) rather
than
on cost
alone.
The
correlation
coefficient
between
winning in
source
selection
evaluations and past
performance ranking
among
the competitors
was
a
mere
0.
1
on
a
scale
where
zero
indicates
total
randomness
and
unity
indicates
perfect correlation.
As
Carl
Ajello, Attorney General
of
Connecticut,
puts it,
"History is important. If
you
don't know
where
you
have been,
you
damn
sure don't know
where
you
are going."
The
public
seems to have hit
upon a solution to this
selection
problem
in the important matter of electing
a President. The criterion
in use is to select
the
taller
of
the
two
principal
candidates. . . a
criterion
which has applied in 19
of the
last
20
presidential elections.
*
Those who would dismiss this
occurrence as
a
probabilistic quirk
should be
forewarned that
on
a statistical
basis
the odds against having
so few
such
instances
are
about 50,000 to 1.
None
of the above is to
suggest that the assessment
of past
per-
formance of
politicians
or contractors is easy
. . . only that
at
least the
latter is somehow done
millions of times every
day by housewives
shopping,
children
buying
candy
and, interestingly,
prime contractors
dealing
with
their vendors.
Real
problems certainly do
exist:
how
should a new
firm
with no
track record whatsoever, either good
or
bad,
be
rated?
How
should
a
satisfactorily
performing
firm
acquired
by
a
poor
performer
be
ranked?
How
should a
sister
division
of
a
notoriously
poor
performer
within
the
same
company be treated?
Or
how,
as
actually happened
in the air-launched
cruise
missile
com-
petition, should
past performance
be allocated when the president
of
*The single departure
from this rule was in
1976,
when
Jimmy Carter
pulled
a
three-
inch
upset.
-
8/10/2019 [Norman R. Augustine] Augustine's Laws (1)
23/234
12
one of the competing
firms
suddenly
becomes
president
of
the
other?
Further, changes in management
or
even
management emphasis may
more than offset
past
problems; thus,
the objective
must
always be
to
maximize
the
chances
of
success
in
the future
and
not merely
to
assure
vengeance
for
the
past. But the practice of
engaging
in a source
selection
process with
no
apparent memory of either past successes or
transgressions would seem to perpetuate the
belief
that each time at
bat is the beginning of a new
season. . . and
thereby
reap all
the
liabilities
of the
widespread
suspicions that the Law
of
the Phoenix
must indeed be operative:
The
source
selection process
is
based
on
a
system
of
rewards and penalties, distributed randomly.
(LAW
NUMBER II)
Under such circumstances, there
will inevitably be individuals who
adopt a
policy
that
"it is a
pleasure
to do
business
to
you."
This
can,
unfortunately, lead to many
truly
forgettable experiences.
-
8/10/2019 [Norman R. Augustine] Augustine's Laws (1)
24/234
Telling
It
Like
It
Isn't
The
cause
is hidden,
but the result is
well
known.
Ovid
The real villain
in
this
tale
begins
to
emerge.
Its
exact
identity is revealed
only as
time
passes,
but its
parenthood
is
already known
to be
optimism
and
enthusiasm and its exact heritage seems to have
something
to
do with cost-estimating. "This
is
the
time," according
to Bert
Fowler,
a vice-president
and
General-Manager
at The
MITRE
Corporation,
"when
grown men
gather
in
a
room
in each company and
spend
their
time
not
trying
to decide
what the cost
would really be,
but
trying to guess
what the
grown
men in the
other
rooms
are
going
to
guess."
It's
like
trying
to read
the
mind of
people
who
haven't
yet
made
up their
mind.
Each
contractor
evolved an
intricate strategy for winning. One, a Fortune 500
industrialgiant,
was seeking
special considerationas
a
"Disadvantaged
Small
Business
Firm"
-
on the
grounds
that if it
did
not win it might in fact become a
small
business. Another was
proposing
a low
risk,
modest
upgrade"
to an
already existing item
it had
been producing (the
upgrade
consisting of a factor of
twelve increase n weight, a factor of ten
in thrust,
and
a factor of
nine
in
volume.
. . ), a
practice known
as
"jacking up
the
nameplate"
in recognition
of
the
fact
that
the
name would indeed
be
preserved. Still
another was encouraging the government
to
adopt
contract terms
and
conditions that were so incredibly
risky
and onerous
that all the competition would
be
prudent enough
to
drop out. But
by
far the
most
innovative strategy in this fixed-priced contest
was
that
of
the
contractor which,
by spreading rumors of
submitting an inordinately low bid, was attempting to
drive
its
principal
competitor
to reduce
ITS price
to
a
winning
position
.
.in the
hopes that
the
disaster
which
would surely
ensue would
so
mortally wound
the hapless "winner" that it could then
be purchased
13
-
8/10/2019 [Norman R. Augustine] Augustine's Laws (1)
25/234
14
Cost Estimation
Track Record
inn
U
VUU
PROBABILITY
80
THAT ACTUAL
COST EXCEEDS
ESTIMATED
60
COST BY
NO
MORE
THAN
AMOUNT 40
SHOWN
(PERCENT)
20
0
-30
-20 -10 0 10 20 30
40 50 60 70
80
90 100
ERROR
IN COST ESTIMATE,
(PERCENT)
COST
ESTIMATE AT APPROVAL OF
DEVELOPMENT CORRECTED
FOR
QUANTITY CHANGES
CORRECTED
FOR INFLATION
INCLUDES
COST
OF CHANGED OBJECTIVES
Figure
3
Estimating
the
cost
of
research
and
development
efforts
inherently
involves
uncertainty.
This is a
significant
ingredient
in
the probabilistic character
of
cost
growth.
lock,
stock
and barrel for
less
investment than
would
be
required
simply to win the incipient
contract All the
competitors,
as
it happened, held the belief
that it
would be fortunate
if the winner's honeymoon
with
the
customer survived more
than a few hours beyond
the
victory
party.
The
contractor employees
couldn't
help
but think back
to
the dispute
which
had boiled
over
whether to participate n
so
intensely
competed a
fixed-price development
effort
in the
first place and
how
the marketing
department
had argued
for
a "no
bid"
decision
because of a concern
that
the
company
might
lose while
the
finance
department
argued
for a
"no
bid"
because
of a concern
that
they
might
win.
Certainly no
one
could argue that
it
had
been
other
than
a
vicious
competition
from the very outset.
There
had
been
allegationsof buy-ins
and bail-outs,
with
the
allegators in
turn
being accused
of benefiting from
leveling,
leaks, and love-ins. Such practices became
so flagrant
that
for
a considerable
period the possibility
existed
that
the
contract might
be
the
very
first to be
awardedposthumously.
ODUCTION
COSTS
00*
DATA
BASE:
81 MAJOR PROGRAMS
1950-1980
I
-
8/10/2019 [Norman R. Augustine] Augustine's Laws (1)
26/234
15
When it
comes to accuracy
of
cost-estimating and pricing for many
activities of
the recent
past,
we
were,
as
the saying
goes, apparently
expecting
very little, and
we
certainly were not disappointed. We
might
even
settle
for
mediocrity, if
we
could
find
it.
In
fact,
the
overruns
in most
development
programs,
due
largely
to
unforeseen
(as
opposed to unforeseeable) tasks, were so
large
that, if it weren't
for
the bad luck
embodied in those programs, they
might
have had no
luck at
all.
The
habit
of unjustifiably concluding,
early in
an
undertaking, that
one
is in
a better
position than
the
facts warrant has
been found to
have
spread
beyond the
matter
of estimating cost
status
of new
development
programs.
For
example,
a
press release
by
an
underdog
Army
team
just
before an
Army-Navy
game, noted: "The season
began well enough,
with
Army taking a
10-10 lead
into
the fourth
quarter
[of its first
game]."
Actually, though, this proved to be quite
prescient
as
in
the
Army-Navy game
that
ensued, the Army
gained
what almost all agreed
was a
3-3
victory.
Thus, numbers do not
tell
all. . .
especially
numbers pertaining to
the cost status
of
newly
initiated
technological programs
or
other activities having
a
similarly
high
emotional content.
Law
Number XXIV
will later indicate how to adjust typical (i.e.,
wrong) cost estimates at any perceived point in a development
program so that the correct
estimate
could, on the average, be
determined.
But what
if one
were not satisfied
with
merely being
correct on the average? Supposing one wanted to
be,
say
90 percent
certain
that
the cost estimate
for a given
program would not
be
ex-
ceeded? What then?
Figure
3
presents
historical
data, corrected
for
inflation,
concerning
a
large
number of completed programs
from which
one can determine
the chances of
an
overrun of any
given
magnitude occurring. It
will be
seen immediately that
only
10 percent of the
time were
programs
completed
within the original cost estimate.
Stated
differently,
contractors and their government overseers are
bidding
for
development projects (involving presumably cost-reimbursable
contracts)
at
about
a
ten-percent confidence level. How can
this
be?
Why don't such improbable estimates
stand out
like fur coats on a
grocery
list? Particularly when
programs
which were terminally ill (i.e.,
ultimately
cancelled prior to completion) have not even been included
in
these
statistics. Viewing
overruns
from this perspective
it
should
certainly be
no surprise that we are frequently surprised. With this type
of mathematics it can
readily
be seen that it
would
indeed
be
odd to
come out even.
The reason,
simply
stated, is that
the
cost-estimation process
as
it
-
8/10/2019 [Norman R. Augustine] Augustine's Laws (1)
27/234
16
now exists is fundamentally
unstable.
The
forces inherent in
it all
act
in
concert to
drive estimated
costs
downward to unreasonable levels...
with
seemingly no effective counterbalancing influence
in existence.
The contractor
quite naturally wants to submit
a
lower cost
bid
than
any of its competitors; the sponsoring government organization wants
to obtain approval for its
program
from
the
Congress;
the Congress
wants to appropriate
as
little money as possible
prior
to
the
next
election ;the government's contracting department wants to
demonstrate
that it
is
a
tough negotiator;and so on
throughout the
entire process.
It
is like sending lettuce by rabbit. And
it
is soon learned
by those on both
sides
of the
negotiating
table that
if you
will just go 60
percent
of
the
way,
the
other
side
will
go
the other 60 percent.
The
participants in the process
may
have
attended
different
schools
but
they seem somehow to
have
attended them
together.
It all
adds up -
or,
more accurately, subtracts down - to
a cost
estimate
for
the
"winning"
contractor
which is,
as has
been seen,
far
too low. The
cost
estimate has in
effect
become a
cost "desirement."
In
Euripides' words
this
would
be
categorized
as a
bad beginning making
a
bad ending.
But in
modern
industrial
parlance, it
would be
described
as giving
each
other the
business.
As
Sebastian
Brant, who
understood
this problem
as
early
as
1494,
put it,
"The world
wants
to be deceived"
-
and
according
to C. N. Bover,
"The
worst
deluded
are
the
self-deluded."
There are sixty-two ways to
underestimate
costs and
it
has been said
that bidders
have not
resorted
to
one
of them. Now
what
needs to be
done
is
to find out
what
that one is so
at
least
some semblance
of
safeguard can
be established.
It
is
thus
discovered
that, in mathematical terms,
cost
estimating
deals
with
truly
complex
numbers-each
having
a
real
and
an
imaginary part.
The competitive
bidding
process pursued by the
government for
developmental
work
is such that
after
an
initial submittal
each
con-
tractor-is given the
opportunity to
fix certain shortcomings identified
in
the
review sequence.
This
tends to
produce
a
"leveling"
whereby
the
technical acceptability
of all contractors
becomes
very
close, with
initial advantages
having been
neutralized and price thereby
escalated
in
perceived
importance. The
next step
is the
simultaneous
submittal
by all contractors of a Best and Final Offer, pronounced "bafo." The
real
difficulty stems
from
those cases which, for one reason or
another,
generate a
second, third,
or even
fourth "bafo."
These
are
called, appropriately, "barfo" (best
and
really final offer)
and "baarfo"
(best and absolutely really final offer).
But,
independent of what they
are called, their effect is clear; at each step along the way
each
contractor
increases
his optimism by 10
percent.
An
old-fashioned
-
8/10/2019 [Norman R. Augustine] Augustine's Laws (1)
28/234
17
auction results. Although such procurement
practices
are
avoided,
some
of their
properties
have a subtle manner of
creeping
into pur-
chases.
For
most
programs
which
employ the above
techniques
of
cost
estimating, matters generally deteriorate steadily
after
the initial
disastrous start. But this has not dissuaded a long line of managers
from
believing that
they,
unlike all
their predecessors, will
be able to
manage their
programs
such
that they
encounter
no
unforeseen
events-and complete the job for the specified cost.
Shakespeare
wrote
about
this kind
of manager in
Henry
IV, PartI
(Act III,
Scene
1):
Mortimer: I can call spirits from the
vasty
deep.
Hotspur:
Why, so can
1,
or so can
any
man;
but
will
they
come
when you do call them?
One
solution
to this dilemma, that of incentive contracting, has on
occasion
been traced to
the Wright Brothers' original
agreement
with
the U.S. Army. . . which was, as
has
been noted, an
incentive-type
contract. But
this
solution. . . and
the problem it addresses. . . were
extant
long before
the
Wright
Brothers were even
considering
diversifying their
bicycle
business. It
turns out, according to
Marcus
Vitrivious Pollio,
the
architect
and engineer,
that the
ancient ancestral
law in the Greek
city of
Ephesus demanded engineers to file a formal
cost
estimate with
the
magistrate prior to
initiating
work on a public
project. If the
work was
completed for the
specified amount,
the
engineer
was
rewarded
with
decrees
and
marks
of
honor.
An overrun
of up to one-fourth
was
financed by the treasury without the
im-
position of penalty.
But
excesses over one-fourth were drawn from
the
engineer's personal
property which
had to
be pledged as security
at
the
time
of
response
to
the
RFP (Request
for
Proposal).
As
seen
from Figure 3,
were such
a
practice to
be
reinstated some 25 centuries
later, approximately 55
percent of the
engineers would
be
spending
their nights on
the
steps
of the Parthenon.
It
is
important to note
that
in matters entailing such
enormous
inherent
uncertainty as
research
and development of sophisticated
systems no one
need
actually be
guilty
of equivocation; everyone
need simply be wildly optimistic. . . a disease which is highly
con-
tagious in the absence of any
effective
vaccine or
antidote.
In terms
that Jimmy the Greek (if, not, sadly, many cost estimators) would
most
assuredly understand,
to obtain
a fifty-fifty
chance
of
completing
a
prescribed
undertaking
within the
estimated cost,
the bid costs,
determined through traditional practices
of
the past two decades,
would
have
to
have
been
increased by 31 percent. To have
obtained
a
90 percent confidence
of
not exceeding the estimate, one would have
needed to increase the estimates by fully 148 percent. The
disparity
-
8/10/2019 [Norman R. Augustine] Augustine's Laws (1)
29/234
18
between
these
two
figures,
incidentally, lies
at
the core
of the reason
why
fixed-price
contracts are
simply not
suitable
instruments
for
research
and development tasks involving prescribed
end-items.
Although
defense
acquisition projects are generally
characterized
by
significant
technological challenge
on
the basis
of seeking to
gain an
edge
over
potential adversaries, cost-estimation
problems are by
no
means the
exclusive
province of
defense
programs. Dulles Airport
suffered
an "overrun"
by a factor
of
1.49
when it was
constructed,
and the Tennessee
Tombigbee Waterway
by
1. 76. The corresponding
figure for
the
Appalachian
Development
Highway
is 2.65; for
the
New Orleans Superdome,
3.22; and for the Trans-Alaska
Pipeline,
4.25.
Even
the
Canadian
Olympics,
scheduled
to
cost a
reported
$400 million,
wound
up costing nearly $2 billion.
Some of the all-time overrun
leaders have been identified
in data
collected
by Myron
Kayton,
a
consulting
engineer, in
his studies of
new-technology projects
in the
19th Century. He
reports
the suffer-
factor
for the Erie Canal as
12,000
percent;
for the Cincinnati-
Covington
bridge as 730
percent;
the Hoosac
Tunnel in
Western
Massachusetts as 2,500
percent; and the Brooklyn
Bridge as
a mere
85 percent.
.
.at least
on the
occasion
of
its
initial
sale.
It
would
seem
that
to have a dismal
record
would require considerable im-
provement.
No
question but
that misery
loves
company.
These
data, of
course,
all
relate
only
to
the past and
include
government costs
as
well as
contractor costs.
But in
spite of these
limitations,
there
seems
once
again
to
be
little reason to doubt George
Santayana's admonition
that
"those who cannot remember
the
past
are
condemned
to
repeat it."
And, in this
respect,
there
would
seem
to
be
many
alive
and
well
today,
particularly
in
the development
trenches,
who are
suffering through
at
least
their
third
reincarnation...
all the while fully confident that the
future will
not
include unhappy
surprises and oblivious
to
the fact that the
past
always
included
unhappy surprises.
And in this case, the past goes
all the way back to
the building of the Suez Canal (200 percent overrun), the
Panama
Canal
(70
percent)
and even the
Roman
Aquaduct
(100
percent).
Mercifully,
the pharaohs kept only sparse
records on the pyramids.
The above
assessment
indicates
what
cost
estimate should
be used
in order to insure that, say,
half the programs
undertaken are com-
pleted for less
than their
projected
cost (and the other
half
for
more). It
may be
of
greater significance to assure that the money saved on the
programs
which
do
in fact
underrun
is sufficient to compensate
for
the
losses
on
those
which suffer overruns. The
above
two statements are
obviously not equivalent
since
the
probability distribution
is skewed.
That
is, a
plethora
of programs endure overruns
of
100
percent. . .
-
8/10/2019 [Norman R. Augustine] Augustine's Laws (1)
30/234
19
but
there is a
noticeable
scarcity of programs which offset this growth
with
underruns
of 100 percent.
It is
thus seen that, as
might have been
expected, the odds are indeed odd.
This is,
of
course, indirectly
related
to
the
principle
that
causes
people
to
drown
in
streams
having
an average depth
of six
inches.
Or,
to the principle
which
governs the
airport pick-up busses
run
by
car
rental firms;
said busses arriving an
average
of
every five minutes
-in
bunches of four.
The
factor which, if applied in recent years, would have guaranteed
that the
house
breaks
even
for
the
overall set of programs pursued
can
be determined approximately from the data
used
to prepare Figure
3.
This factor equals at least 1.52
and is
known as the "Las Vegas Factor
of
Development Program Planning."
Normally
it
is
quoted
to
at
least
seven
significant figures; however, this
neglects
the
fact
that even this
metric is
subject to change depending upon the
amount of
risk
that
is
entailed
in
a specific
program.
Graphically, this
degree
of hazard is
represented
by
how flat
(risky) or steep
(certain) is
the slope
of
the
curve
in
Figure
3
for
the particular
program
addressed.
One
might
expect,
in keeping
with the fundamental
laws of
economics-not
to
mention official government
policies-that
ac-
tivities
which entail
the
greatest
risks
would
be
those
which
would
return
the
greatest
profits. Quite the
contrary, such
highly
volatile
endeavors as research and development carry the smallest realized
profit
margins; routine matters such
as the provision
of spare
parts
carry the largest,
and
intermediate risk-bearers, such as
serial
production, reside,
in terms
of profit
rate, somewhere in between.
All the above mathematical meandering
can
be distilled much more
straightforwardly
into
the
Law of
Apocalyptic Costing:
Ninety
percent
of
the time things
will
turn
out worse than
you
expect. The other ten percent
of
the time you
had no
right
to
expect so
much.
(LAW
NUMBER III)
As
the old saying
goes,
you
have
to kiss a lot of
frogs
to
find a
prince.
Those who find such statistics
implausible
must explain, say, why
90 percent
of
the
world's air traffic arrives and
departs
from
the 10
percent of
the gates farthest from the
terminal
building.
The choice is thus
straightforward: one can either
face
up to the
true
cost
of
an
undertaking at its outset or,
alternatively,
parcel out the
bad
news on the
installment plan.
There simply
are
no other options. And
in this
respect,
those
involved in
government development activities
rank among the all-time great
proponents of
what
is said, in
some less
-
8/10/2019 [Norman R. Augustine] Augustine's Laws (1)
31/234
20
sophisticated circles,
to be referred to as
the
"slow reveal."
"I'll
think
on
it tomorrow. . .I
can stand
it
then. Tomorrow
I'll think of
some
way,"
said
Scarlett O'Hara.
In
the
words
of
Peter
Hall,
in
the
concluding
sentence
of
his
work,
Great Planning Disasters, "There may
be
some excuses for great
planning disasters, but there
are
not nearly so many
as we think."
Figure 3, of course, deals
only
with
the matter of estimating costs. If
one makes
a
similar
plot showing schedule
outcomes
or
performance
(speed, range,
accuracy,
payload... .) outcomes,
it is
found
that
while
there
is
only a
10
percent
chance of
meeting
cost goals,
there is a
15
percent
chance
of meeting schedule goals
and
a
70 percent chance
of
satisfying
performance
goals.
The long-suspected
priority
hierarchy is
thereby mathematically
derived. Performance reigns
supreme. For
many years it
had appeared
that nothing could
ever make the
schedule-performance record look
good.
That
was
before the
cost-
performance record came along.
Some 2400 years ago Thucydides got to the root of the
cost-
overrun problem: "Their judgment
was
based more on
wishful
thinking
than
on sound calculation of
probabilities; for
the
usual
thing
among
men
is
that
when
they
want
something they
will,
without any
reflection, leave
that
to
hope, while they
will employ the
full
force of
reason in
rejecting
what they find unpalatable." It is not that con-
tractors do not
know
how to estimate
costs
conservatively.
Nor
is it
that they
do not
know how
to win competitions.
The
problem
resides
in
the fact
that
most do
not know how
to do
both
simultaneously.
With all
participants
in
the cost
estimating process motivated
more
by the law
of survival
than the law of
probability, what other outcomes
could
we
have expected?
How could
we
possibly be surprised?
To
quote astronaut Pete Conrad from Gemini
XI,
"We're on top of the
world.
You
can't
believe
it...
utterly
fantastic. The world is round."
The
fact
that the
government's managers
know they have "in-
centivized"
contractors to be optimistic in estimating costs
suggests
that the following conversation
from
Romanoff and Juliet
involving a
general
and
two
ambassadors
might
just
as well have
taken
place
among
a general and two contractors:
General:
. .
Incidentally,
they
know
your
code.
American Ambassador: We know they know our code...
We
only
give them things we
want
them to know.
General: Incidentally, they
know you know they know your
code.
-
8/10/2019 [Norman R. Augustine] Augustine's Laws (1)
32/234
21
Soviet
Ambassador:
... We
have known
for some time that
they knew
we
knew their code.
We
have acted
accordingly
-by
pretending to be duped.
General: . . .Incidentally, you know - they know you
know they know
you
know. ..
American Ambassador:
What?
Are
you
sure?
The problem is much as the one encountered by the businessman
who discovered a
card in
his
hotel room
stating, "If
you
have a
problem
with
alcoholism
and
need
help,
call
344-2920."
Upon
calling
the
number,
he found
that it
was a liquor store.
-
8/10/2019 [Norman R. Augustine] Augustine's Laws (1)
33/234
On Making a Precise
Guess
A
horse
that
can count
to ten
is a remarkable horse,
not a
remarkable mathematician.
Samuel Johnson
As Benjamin
Disraeli
pointed out long ago, there
are
in-
dividuals
who
use
statistics
as a
drunkard
uses a
lamppost:
for
support rather
han
for illumination.
This
proves
to
be
a
valuable
piece of
insight
for the promoters
of
our
newly
established project
who must justify its very existence
armed
with only the
most tenuous
of data.
The
search not
unexpectedly
turns
to
methods
of making
the
unknown,
indeed even
the unknowable, appear plausible
to
those
who
insist upon knowing. As luck
would
have it, a powerful
technique is soon discovered.
In the
late 1950's a well-known
Princeton
geology
professor
an-
swered
a
question which arose during
a
field trip
about the
age of
a
fossil
that had been found by stating it was two million and two years
old. Responding to
still another
question by
students
incredulous over
his
ability
to precisely date
so
old an
object, he explained
that another
group
had visited the
same
site
two
years earlier and had
been told by
a local
farmer
that the
fossil
was then
two million
years
old
As
reported
to
the Congress
at
the
time
development
was to
be
initiated, the total program
cost
for the Harpoon program was
said
to
be $1,031.8 million. For
the
A-10 program, the corresponding cost
was defined
as $2,489.7 million.
Not
$2,400 million;
not
even
$2,489 million. Rather, the cost would
be
two
thousand
four
hundred
eighty-nine pointseven million dollars.
This
great
degree of accuracy may perhaps be somewhat surprising
to the
uninitiated in
view
of
the
fact that history
shows the
first digit
of
past program
cost
estimates
to
have been
in
error,
on the average,
by
about 100
percent
The General Accounting Office, in its most recent
report on the topic,
for
example, states that for the Department of
Defense
acquisition programs
now
underway, 67
percent
are already
overrun by more
than
100
percent
(including
the
effects of inflation).
In the case of
the
F-18 program it was originally stated
to
the
Congress
that the
cost would
be
twelve billion
eight
hundred
seventy-
five point
three
million dollars. A
few years
later
the
same
report
defined
the probable
cost as
(not
altogether inconsistent with the
23
-
8/10/2019 [Norman R. Augustine] Augustine's Laws (1)
34/234
24
GAO's
findings)
twenty-four billion twenty-three
million.
. . and (still )
0.3 million
dollars. Perhaps encouragement should be derived simply
from the
fact
that, although the first
significant figure did double, it
was
possible to maintain
the
last
one unchanged.
Detailed
analyses
by
the
author show that
although
the initial
digit in program cost
estimates is
virtually never
correct,
the last digit does prove to be
correct
ten
percent of the time.
This is sometimes called
Augustine's Final
Law.
George Will, speaking of the
President's
tax bill, describes the
application
of this
technique in the
following
words; "He pretended
the
tax
bill wasn't really a
tax
increase -
odd,
considering it is
sup-
posed to siphon
in
$98.3 billion. (Note the precision -
'.3'
- from
people
who
have been
unable
to
guess the
deficit
within
forty-billion
dollars.) "
Other examples
of
the preservation of the
last
digit in cost
estimating? The
Sydney
Opera
House (to have been
built in
6 years
but took
16) was
to have
cost 7.2-million Australian dollars but ac-
tually
cost
102-million dollars. English
taxpayers were to have paid
150-million
pounds for the development of the Concorde
supersonic
transport
but
eventually were billed 12.20-billion pounds.
(Note
that
an
initial
estimate ending
in
"zero" can
assure the
success
of this
methodology.) To
prove
that
agony
knows no international
boun-
daries,
the Bay
Area Rapid Transit ("BART") in San
Francisco grew
from
an estimated cost of $0.6 billion to $1.6 billion.
The whole process
is
something
akin
to "getting the
last
word" -in
this
case, "getting the
last
digit."
But,
then, one must start
somewhere.
Nonetheless,
by examining
the
data in Figure
4,
it
is possible to
derive
the
logic
which
underlies
the
practice of
quoting
fundamentally
dubious numbers
with
a
very great
degree of
apparent
accuracy.
It is
seen
from
the
figure
that there is
indeed a
relationship between the
number of "significant
figures"
quoted and the true
precision of the
data at hand,
but
this
relationship
is just
the opposite of what one
might
expect. The
Law
of Definitive
Imprecision, which is
based on
a
substantive
collection
of
data
such
as that presented
in
Figure
4,
states:
The weaker the data
available
upon
which
to base
one's
position,
the greater the
precision which
should
be quoted
in order
to give
that
data
authenticity.
(LAW
NUMBER
IV)
The use of the above law is
fairly
widespread, with one recent
-
8/10/2019 [Norman R. Augustine] Augustine's Laws (1)
35/234
25
Relationship
of Implied Precision
to Actual Precision
NUMBER
OF
SIGNIFICANT
FIGURES
QUOTED
'77
-
PROBABILITY OF SPACESHUTTLE
FRAGMENT NJURING
6
PERSONON GROUND - 1 N 166 667
(SOURCE:GAO TESTIMONY TO CONGRESS)
- WEIGHTING FACTOR ASSIGNED TO VALUE OF
5
RELIABILITY
IN LIST OF 15 PARAMETERS
DESCRIBING
5
INTACS' SYSTEM. 66,2187
I- SOURCE: SYSTEMSPECIFICATION)
z
DEPARTURETIME. LAST
FLIGHT. FRIDAY P.M. ATLANTA
=
4
TO WASHINGTON.D.C.- 14 P.M.
D
- (SOURCE:OFFICIAL
AIRLINE GUIDE)
z
O - ESTIMATED
NFLATION RATE FOR YEAR ENDING
JUNE 1987
3
= 4g6
:SOURCE: ARMY REOUESTFOR PROPOSAL
ISSUED
IN 1979)
L -
PROBABILITY OF RAIN; SEATTLE, WASHINGTON
Z 2 MAY 11, 1978 9
-SOURCE:
U.S. WEATHER BUREAU)
NUMBER OF PROTONS N HYDROGEN ATOM
1
OURCE: HANDBOOK
OF CHEMISTRY AND PHYSICS)
Figure 4 The
precision
with which
any given
numerical
factor
is known should
presumably be
reflected in the
number of significant figures
with
which
it
is
quoted.
This is
n practice seldom the case
thereby leading to misinterpretation
of
the
con-
fidence
which isassignable to data.
example being
the Army
Development
and Readiness
Command's
official
estimate
of the inflation
index
which will prevail
for
procurement in the
16th
year
from
the time of the
estimate (i.e.,
in
1995). This
index
has
been
stated to
be 2.6719-an amazing feat of
prescience,
particularly
in view
of the Office of
Management
and
Budget's
near-simultaneous
adjustment
of the inflation rate for the
next 12
months
by about 3 percentage points
Similar
confidence
in
projecting future
threats
is
reflected
in
a
document
released during
1980
with
the marking prominently
displayed
on
its cover,
"Declassified
on
10
January 2000."
But,
fortunately,
as
Will Rogers reminded,
"Numbers don't
mean
nothin'.
It's
people that count."
An example from
the
civilian
sector
is
the U.S.
Trust
Company's
announcement that
there
are
574,342 people
in the
United States
-
8/10/2019 [Norman R. Augustine] Augustine's Laws (1)
36/234
26
with
assets worth $1 million or more. One
cannot help
but
wonder
how such an assessment could
be
made;
or how
the
marginal
qualifiers
fared
in
the
hour-by-hour
vagaries
of
the
stock
market; or
if
one of them might not have
dropped a
few grand
at
the race track and
failed
to
have promptly
informed the
U.
S.
Trust Company.
Or
how
the U.
S.
Census
Bureau knows
that
"in 1980
the
U.
S. labor
force
consisted of
exactly 104,449,817
workers";
particularly
when most
employers don't
have any idea how
many of their own employees are
actually workers. It may
be
that this is
all simply
a consequence
of
more widespread
use in financial circles
than
had heretofore been
realized of
O'Brien's
Principle,
also called
the $357.73 theory, which
states, "Auditors
always reject
any
expense
account with
a
bottom line
divisible by
5
or
10."
A problem
which
has
long
been faced
in
applying
Law
Number IV,
however,
has been
what
to do in those cases wherein the analyses
from which the numbers
were derived
provide only rather discrete
results, such as
$1 billion,
or 10
miles, or 1 ton.
The solution to this
dilemma has been astutely derived by Lieutenant General Glenn
A.
Kent (USAF, Ret.) in his reviews of a large number of
quantitative
analyses.
The solution
is
quite simple:
it
consists
of simply
converting
all
data from the English system of
measures
into the
metric
system
and
back again
A
derivative
of this English-to-metric technique wherein
39.4
inches
equates to
one meter accounts
for
such
phenomenal
accuracies
as
are
identified in a bulletin
recently carried
in the U.
S.
from a European
wire
service concerning a
citizen
whose
private airplane
was
reported
to
have missed crashing
into
the control tower
at
an airport
in
Europe
"by less
than
39.4
inches."
That such
undeserved
precision can be
hazardous, particularly
when
combined with
a law later to
be
promulgated concerning the
unreliability of electronics, is
made
abundantly
clear in
the
following
excerpt
from
a news
article carried by
the
Associated Press
regarding
"a fuel shortage which
caused a
Boeing 767 to make
an
emergency
landing. . The plane, with 61 passengers
and a crew
of
eight,
went
into a
powerless
glide from 39,930 feet to a bumpy landing on a
Gimli,
Manitoba airstrip...
Airline
workers
had
resorted
to
a
manual
fueling procedure when an
electronic system
on the aircraft. . failed.
The
fuel in
the craft
is measured in
centimeters
and converted
to
liters
before
departure.
That figure
is
converted to
pounds and then
to
kilograms
so that the pilot
can
calculate the flight
plan.
It
was
during
this procedure that the
error
was made, the airline spokesman
said."
A
related
approach appears
to
have
been
used in testimony
-
8/10/2019 [Norman R. Augustine] Augustine's Laws (1)
37/234
27
provided to