value of research about the safety effect of actions
TRANSCRIPT
1
THE VALUE OF RESEARCH ABOUT THE SAFETY
EFFECT OF ACTIONS
Ezra Hauer, University of Toronto (retired), 35 Merton Street, Apt. 1706, Toronto, ON,
M4S3G4. 416-483-4452, [email protected]
James A. Bonneson, Texas Transportation Institute, The Texas A&M University System,
College Station, TX 77843-3135, [email protected]
Forrest Council, UNC Highway Safety Research Center, 730 MLK Jr. Blvd, Chapel Hill, NC
27599-3430, Phone: 919-962-0454, Email: [email protected]
Raghavan Srinivasan, UNC Highway Safety Research Center, 730 MLK Jr. Blvd, Chapel Hill,
NC 27599-3430, Phone: 919-962-7418, Email: [email protected]
Geni Bahar, P. Eng., NAVIGATS Inc., Phone: 416-932-9272, Email: [email protected]
Word Count 6578
Figures 2
Tables 2
2
THE VALUE OF RESEARCH ABOUT THE SAFETY
EFFECT OF ACTIONS
ABSTRACT
Money ought to be spent on research that promises most value for the buck; but how is one to
estimate the value?. This paper suggests a logical and quantitative approach to estimating the $-
value for a class of proposed road safety research projects – research to estimate the safety effect
of various actions. The aim of such research is to improving our ability to predict. The more
accurately one can predict the safety effect of some action the fewer incorrect decisions will be
made. With more decisions correct investment is more cost-effective. This is what gives such
research its value. The task of assigning $-value to a proposed research is doable. The logic and
the computations are described. The estimated $-value, when coupled with the cost of research,
can used to prioritize proposed research projects.
INTRODUCTION
That research has value seems obvious. However, when one is asked whether a certain
research is of sufficient value to be funded, or which of two proposed research projects should be
undertaken first, the fog rolls in. On such matters opinions often differ, and consensus difficult to
reach. The purpose of this paper is to suggest a logical, and quantitative approach to estimating
the $-value for a class of road safety research projects – those research projects the aim of which
is to estimate the safety effect of various actions. Once the structure is standing it will be clear
what factors give such research its value. Moreover, it will make reasoned discourse about
competing research projects possible and enhance the value of funded research.
Road safety management amounts to decisions and actions: whether to implement some
countermeasure, whether to choose a certain road alignment, whether to adopt a standard, etc.
These have costs and benefits. Research about the safety effect of actions reduces the uncertainty
about their safety benefits. This is why the logical foundation for valuing such research comes
from „Decision Analysis‟. The better the information that is used for making decisions the larger
the proportion of correct decisions, the larger is the expected benefit of implemented decisions
3
per unit of investment. The amount by which the results of some research increases the expected
$-value of the decisions that use its results is the $-value of that research.
LITERATURE
The concept of value of information (VOI) has its foundation in Decision Analysis [see e.g. (1),
(2), (3) ,(4), (5), (6), (7)].VOI methods were used to set research priorities for food safety (8).
Their use in medicine is discussed in (9), (10), (11), (12). Applications of VOI to medical
research priorities are described in (13). In spite of the air of optimism in the literature, the
impression is that VOI methods are used more in the design of research studies than in
determination of research priority [(14), (15), (16), (17), (18), (19)]. Most applications are based
on the „expected value of perfect information‟ [(13), (20), (21), (22)]. The use of the value of
„imperfect information‟ is more limited [(23), (24) and (25)].
BUILDING BLOCKS
The logic and the machinery for estimating the value of road safety research are best explained
around a story. The story is that someone proposes to do research about the safety effect of
illuminating access-controlled roads. The question is how to attach a $-value to this proposed
research.
The first step is to be clear about what decisions might be improved by the information
which that research intends to produce. In our story the decisions are about whether to illuminate
various unlit access-controlled roads. Decisions of this kind should depend on the balance
between the expected benefit (the reduction in night-time crashes) and cost (the capital and
operating cost of illumination). The decision is: „Implement „, when
(1)
The „r‟ is the smallest acceptable benefit-to-cost ratio and reflects the opportunity cost of the
capital available for safety improvements. For road safety actions the benefit is the:
(2)
In this „μ‟ is the expected annual number of target crashes on the unit, „θ‟ is the „Crash
Modification Factor‟, and „a‟ is the $-value of an average target crash. A „unit‟ can be a mile of
road, an intersection, a median opening, etc. (A glossary of notation is provided at the end of the
paper.)
4
To illustrate, if on a certain unlit freeway section one expects 6.1 night-time
crashes/(mile-year), and when the current estimate is that illumination reduces night-time crashes
by 30% (i.e., that θ=0.7), and if the cost of an average night-time freeway crash is $20,000, then
the expected safety benefit is 6.1×(1-0.7)×20,000= $36,600 per mile-year. Furthermore, if the
annual cost of freeway illumination is $25,000/(mile-year) then the benefit-cost ratio is
36,600/25,000=1.5. When only actions with a benefit-cost ratio exceeding 2.5 are funded, the
decision should be: „Do not illuminate‟.
In such computations θ plays a key role. What we know about θ comes from past
research. Each past research study about the safety effect of some action yielded estimates θ of
the θ that prevailed at that time and in those circumstances. Since the safety effect of an action
always depends on the details and circumstances of its implementation, there is no reason to
think that the θs in past research studies were all the same. Similarly, because future actions of
the same kind will differ in their circumstances, the corresponding θs will not be identical. It is
therefore prudent to think of θ as a random variable with a probability density f(θ), a mean E{θ},
and a standard deviation σ{θ}. The θ from past research serve to estimate E{θ} and σ{θ}. How
this is done is explained in (26) and in (30). (The and in (26) correspond of the
estimates of E{θ} and σ(θ) here). It will be assumed that the f(θ) can be approximated by the
Gamma pdf:
θ θ θ
(3)
While the Gamma pdf is flexible in shape and suitable for random variables that are non-
negative, there are, at this time, no empirical grounds to justify this assumption. To illustrate, if
E{θ}= 0.7 and σ{θ}=0.3, then f(0.4)=1.12 and P(0.35<θ≤0.45)≈ 1.12×0.1=0.112.
Of special interest is the θ dividing the „implement‟ and the „do not implement‟
decisions; the „break-even θ‟ denoted by θBE, At θ=θBE the ratio of benefits and costs equals „r‟.
Let „c‟ denote the annual cost of implementing some action on a unit. From μ(1-θBE)a/c=r it
follows that:
(4)
5
To illustrate, when r=2.5, c=$25,000/(mile-year), μ=6.1 night-time crashes/(mile-year), and
a=$20,000 per target crash, then θBE=0.49. When θ<0.49 then the right decision would be to
„Illuminate‟; when θ>0.49 the right decision would be „Do not illuminate‟.
The key to understanding the value of research (VOR) about the safety effect of actions is
in the distinction between the θ expected at the time of decision-making and the θ that
materializes after implementation. At decision-making time the expected benefit is computed by
replacing θ in equation 2 by . However, the benefit that materializes after implementation
will be determined by the θ which will be some value from f(θ). If the differences |
tend to be small then the decisions based on are likely to be correct. However, when the
differences tend to be large then many of the θs that will materialize after
implementation will be quite different from the used to make the decision. If so, some
decisions will be incorrect and cause a net loss. The role of research is to reduce the
differences . This will reduce the chance of incurring a net loss, and thereby increase
the average net benefit of investments. This increase in net benefit is what gives research its $-
value.
Having prepared the building blocks, the procedure for estimating the VOR (value of
research) about the safety effect of some action can be described. To provide a sense of direction
for the next section it may help to sketch a roadmap. We begin by focusing on a future action on
one unit with specified values of „r‟, „μ‟, „a‟ and „c‟. These values determine which θs, should
they materialize after implementation, will lead to loss and determine the size of the loss. Next
we define the „expected loss‟ -- the product of the size of the loss and the probability of its
occurrence summed over all values of θ. Research can reduce the size of the expected loss and
this reduction will be deemed to be its $-value for this action and unit. The sum over all units on
which the action may be taken is the $-value of the proposed research.
THE VALUE OF RESEARCH FOR ONE UNIT
When the decision is: „Implement‟; it is the wrong decision if .
When the decision is: „Do Not Implement‟; it is the wrong decision if .
Wrong decisions carry a loss. Two distinct cases will be analysed.
6
The “Do Not Illuminate” case
Recall that in the earlier numerical examples θBE was 0.49, was 0.7, and the corresponding
decision for such a unit was: „Do Not Illuminate‟. Since σ{θ}=0.3, some θs that might
materialize after implementation could be quite different from 0.7. If θ turned out to be, say, 0.40
this would imply not saving 6.1×(1-0.4)=3.66 crashes/mile-year the value of which is
3.66×$20,000= $73,200. By not illuminating we would save the opportunity cost of
$25,000×2.5= $62,500, resulting in a net loss of $ 73,200 - $ 62,500 = $10,700/(mile-year). Such
losses when viewed as a function of θ will be called the „loss function‟ and denoted by L(θ). The
loss function for the „Do Not Illuminate‟ decision for a unit with r=2.5, μ=6.1 night-time
crashes/(mile-year), c=$25,000/(mile-year), and a=$20,000 per target crash is shown in Figure
1.
Figure 1. The Loss Function of the „Do Not Illuminate‟ decision.
The corresponding general algebraic expression is:
(5)
Using f(θ) we can calculate the expected loss a unit as:
(6)
The expected loss is easy to compute by numerical integration as shown in Table 1. Here the
integration region between 0 and 0.49 is divided into 100 small intervals. The values of f(θ) in
7
column 3 are based on E{θ}= 0.7 and σ{θ}=0.3. For the freeway section in the illustrative
example the expected loss is $3786/(mile-year).
Table 1. Expected Loss computations when the decision is „Do Not Illuminate‟
1
i
2
θi
3
f(θi)×0.49/100
4
L(θi)*
5=
3×4
1 0.0024 1.727E-11 $59203 $ 0
2 0.0073 2.194E-09 $58608 $ 0
…
30 0.1439 4.265E-04 $41,948 $18
31 0.1488 4.762E-04 $41,353 $20
32 0.1536 5.292E-04 $40,758 $22
…
99 0.4804 6.613E-03 $892 $ 6
100 0.4853 6.660E-03 $297 $ 2
Sum=Expected Loss= $3786
* r=2.5, c=$25,000/(mile-year), μ=6.1 night-time crashes/(mile-year), a=$20,000 per target crash
Viewed in a different light, the expected loss can be given a new meaning. Suppose that
an omniscient creature sells information about what will be the θ after the contemplated action
for a unit (defined by r, μ, c and a)is implemented. How much should one be willing to pay for
this information? If the creature will say that θ will be 0.9, the information will be worthless; the
decision-maker was going to make the „do not illuminate‟ decision anyway. However, if the
creature will say that θ will be 0.4 the decision maker would decide differently. The reward for
the ability to tailor the decision to the value of θ when it will be 0.4 is the $10,700; it is the most
the decision-maker should be willing to pay for knowing that θ will be 0.4. However, inasmuch
as decision-maker has to say how much he/she would be willing to pay before the creature
reveals what θ will be, the decision-maker has to weigh the magnitude of the reward by the
probability of θ. Therefore the most he/she should be willing to pay for the knowledge of what θ
will be is $3786. This is the „Expected Value of Perfect Information‟, EVPI. Because it is the
value as of now, before any new research was done, it will be denoted as EVPIb.
Nobody sells perfect information. However, research can generate information which,
while not perfect, can reduce the risk of decisions proving to be incorrect. Imperfect information
also has value. It is the Expected Value of Imperfect Information which, in the present context is
the value of research (VOR). Using the logic and computational scheme already developed, there
8
is only one step from EVPI toVOR. Recall that the story is about a proposal to research the
safety effect illumination on access-controlled roads. Suppose that the proposed research is
expected to reduce σ(θ) from 0.30 to 0.25. What would be the value of this research?
All remains as before except that the parameters of the Gamma distribution in equation 3
are now =(0.7/0.25)2 and =0.25
2/0/7. This changes the f(θi) in column 3 of Table 1 and
therefore the entries of column 5. Now the sum of column 5 is $2383. This is EVPIa, the
expected value of perfect information after the research was done. The research which reduced
σ(θ) from 0.3 to 0.25 reduced the expected annual loss associated with the decision “do not
illuminate” by $3786-$2383=$1403/mile. A decision-maker who has to decide whether to
illuminate this road should be willing to pay up to $1403/mile for this research; this then is its
VOR.
The “illuminate” case
Consider now the “illuminate or not to illuminate” decision for a different road section, one
where one expects 10.9 night-time crashes/(mile-year). Now θBE=1-(2.5×25,000)/ (20,000×10.9)
=0.71. As 0.71> 0.7 for this road the decision is: „Illuminate‟. With the change of decision comes
a change in the loss function. Now the loss function is:
(7)
and the expected loss is:
(8)
Proceeding again by numerical integration the expected annual loss associated with the decision
to illuminate one mile of a road with μ=10.9 night-time crashes/mile-year is $24,436. Suppose
again that the suggested research is expected to reduce σ(θ) from =0.3 to σ(θ)=0.25. With the
reduced standard deviation the expected loss is $20,231. For this road the VOR is $24,436-
$20,231=$4205/mile. It follows that the value of a proposed research depends, among other
things, on the annual number of target crashes of the unit.
THE VALUE OF THIS RESEARCH FOR ALL UNITS.
The proposed research promised was to be in aid of future decisions whether or not to illuminate
various access controlled roads. The question was how much a society should be willing to pay
for it.
9
For one mile of freeway with μ=6.1 target crashes/year we established that the proposed
research promised to save $1403/year; when μ=10.9 target crashes/year the savings were
$4205/year. Thus, to determine the value of some proposed research for many units, one has to
add up the values for all units which will be subject to decisions that will benefit from the
conduct of the proposed research. The values to be added up will usually depend on the μ and c
of each unit.
To illustrate, suppose that, as shown in column 2 of Table 2, in a certain society there are
1000 miles of such roads where 0.5 to1.5 night-time crash are expected, 950 miles where
1.5<μ<2.5, etc. Suppose further that the cost of illumination per mile is the same for all roads.
Illumination cannot pay for itself when μ<rc/a; here when μ< 2.5×25,000/20,000=3.1/mile-year.
This explains the zeros above the double line. The decision in column 4 is not to illuminate as
long as the θBE in column 3 is less than , which is 0.7 in this illustration. Using the logic and
computation already explained one can calculate the EVPIb σ=0.3 and EVPIa when σ=0.25.
These are shown in columns 5 and 6. Their difference in column 7 is the expected annual
value/mile of the research which reduces σ from 0.3 to 0.25. Multiplying these by the number of
un-illuminated miles in column 2 and summing over all rows shows that the annual value of this
research is $901,339.
Table 2. VOR Computation
1 2 3 4 5 6 7 8
μ. Mid-point
of expected
night-time
crashes/mile
Un-
illuminated
Miles
Break-
even θ,
θBE
Illuminate?
EVPIb
when
σ=0.3
EVPIa
when
σ=0.25
VOR/
Mile
VOR for
units with
μ
1.0 1,000 0.00 No $0 $0 $0 $0
2.0 950 0.00 No $0 $0 $0 $0
3.0 900 0.00 No $0 $0 $0 $0
4.0 800 0.22 No $56 $12 $45 $35,708
5.0 400 0.38 No $1000 $466 $534 $213,727
6.0 200 0.48 No $3465 $2,143 $1,323 $264,520
7.0 100 0.55 No $7150 $5,061 $2,090 $208,984
8.0 50 0.61 No $11676 $8,924 $2,752 $137,595
9.0 10 0.65 No $16744 $13,426 $3,319 $33,187
10.0 2 0.69 No $22202 $18,393 $3,809 $7,618
11.0 0 0.72 Yes $24416 $20,170 $4,246 $0
Sum=VOR $901,339
10
What should be considered when forming an opinion about research priority? Now the
answer is obvious: whatever is needed to compute the VOR. Whether one uses judgment or
computation the following must go into the mix:
1. How many units will be affected by the decision;
2. The cost of an average target accident;
3. The expected frequency of target accidents;
4. The annual cost of implementing the action;
5. The limiting benefit/cost ratio;
6. The mean and standard deviation of θ based on the existing knowledge;
7. What is likely to be the standard deviation of θ after the proposed research is completed.
This list was distilled from the computational procedure described in this section. Most
would come up with a similar list when thinking about what factors should be considered. This
reassures us that the computational procedure is sensible and that nothing important was missed.
The purpose of this and the preceding section was to determine where the VOR comes
from and how it can be computed. To summarize, the safety benefit of future actions depends on
the θ which will materialize in specific future circumstances while the decisions must be made
on the basis of the expected benefit which is computed using the current estimated mean of the
mean, . While θ and are not the same, research helps to reduce the average difference
between the two. This, in turn, reduces the chance of the decision to implement or not to
implement being wrong. The expected $-value associated with the reduced risk of making wrong
decision can be computed and attributed to the conduct of research.
FACTORS AND CONSIDERATIONS
For the proposed procedure to inspire confidence one must believe that it mixes the correct
considerations in the right proportions. Moreover, one should be able to judge without tedious
number crunching which proposed research is likely to have large (or small) value. In this
section the focus is on understanding. We will try to explain the proposed procedure in terms of
factors and considerations.
Some think that new research should be undertaken when it plugs gaps in knowledge.
This is fallible guidance as there are several other considerations. Thus, e.g., irrespective of how
uncertain is θ there is little value to research about θ when the expected number of target crashes
11
is small. Conversely, even if the topic is not a „gap‟ and safety effect of that action was studied
by many, uncertainty about θ may still be large and the consequent decision errors costly. Others
think that research is best done on actions where target crashes are many. This too is true in some
circumstances and not in others. It is not true, e.g., when the cost of the countermeasure is very
large and when crashes are not severe. Reliance on intuition in this complex setting is
insufficient. However, the logic and procedure described earlier provide a solid base for
understanding.
Disregarding detail, the bottom line Table 2 can be thought of as made up of three
components:
N The number of units that are subject to the decision in which the θ plays a role;
EVPIb The EVPI for a representative unit before the proposed research is done;
EVPIa The EVPI for the same unit after the proposed research is completed.
A „representative‟ unit is one with average values of „μ‟, „c‟ and „a‟. These three components
combine to make up the VOR about θ as:
VOR = N×(EVPIb-EVPIa) (9)
To illustrate, in Table 2 there are 1562 freeway miles where illumination could possibly
be of value (i.e. where μ>rc/a). The average annual number of target crashes/mile for these is
about 5. For such a mile the EVPIb/mile is $1000 and EVPIa/mile is $466. From such averages
an approximate value of this research is 1562× (1000-466) ≈ $830,000. (in Table 2
VOR=$901,339) .
Component „N‟ requires little comment. If the θ is for freeway illumination, N is the
number of miles of unlit freeways where μ>rc/a; if θ is for signalizing intersections N is the
count of unsignalized intersections with μ>rc/a, etc. What needs to be explained are components
„EVPIb‟ and „EVPIa‟; that is, what determines the EVPI for a unit, when is this value it large, and
by how much will it be reduced by the proposed research.
Understanding the EVPI for a unit
The EVPI for a unit depends on the f(θ), the θBE, and on the cost of the target accidents (μ×a).
All three elements are shown in Figure 2.
12
Figure 2. How the elements combine
The curve is the f(θ) with E{θ}=0.7 and σ{θ}= 0.3; the same pdf was used earlier. Also
as earlier, in case (a) μ=6.1 while in case (b) μ=10.9 target crashes/(mile-year). The θBE in case
(a) was 0.49 and, because it is smaller than E{θ}=0.7, the decision is: „Do not implement‟. This
decision would turn out to be incorrect should the θ that materializes after implementation be in
the darkly shaded part of the pdf in Figure 2 (a); the „Do not implement‟ decision is expected to
be incorrect for about a third of the units with θBE=0.49. In case (b), because E{θ}=0.7<
θBE=0.71, the decision is: „Implement‟. This decision will turn out to be incorrect when the θ that
13
materializes after implementation is in the darkly shaded part of the pdf in Figure 2 (b) where
θ>θBE.
It is the position of θBE vis-à-vis E{θ} that determines the probability of decisions to be
incorrect. What matters is the distance |E{θ}-θBE| when measured in units of σ{θ}. Define
(10)
The larger is D the lesser is the probability of decisions to be incorrect. When D is larger than
about 2 or 3, this probability is quite small. That is why D is useful for identifying research that
is of little value irrespective of any other consideration. To compute the D for some action and
unit one needs to have estimates of E{θ}, σ{θ} and θBE. Estimates of E{θ} and σ{θ} come from
past research. These can be obtained by methods described in (26) and ought to be available
from sources such as (27, 28, and 29). The estimate of θBE is based on the values of „r‟, „c‟, „μ‟
and „a‟ (see Equation 4 above).
For D to be large (and the VOR into the safety effect of that action small) the
denominator of equation 10 has to be small and the numerator large. The denominator is small
when the safety effect can be accurately predicted; there is little point in more research about
such actions. The numerator is large when θBE is either far to the left of E{θ} in case (a) or far to
the right of E{θ} in case (b). As can be gleaned from equation 4, the larger is the cost of
implementing the action and the lesser the cost of the target crashes on the unit (μ×a), the further
to the left will be θBE. It follows that there are two circumstances in which the VOR about the
safety effect of an action is likely to be small: (a) when for the representative unit the cost of the
action is large while the cost of the target accidents is small and (b) when the cost of the action is
small while the cost of target crashes is large. In both cases VOR is small as it is already clear
what is the right decision; „Do Not Implement‟ in case (a) and „Implement‟ in case (b).
By using the D>2 or 3 rule one can weed out research that is not likely to be of value.
However, because D does not measure the EVPI, it cannot be used to rank competing research
projects in order of priority. Computing the EVPI involves integration of the product f(θ)×L(θ)
over the shaded parts of Figure 2. While the numerical integration is easy to do, it is not
transparent. To promote understanding an approximation may help.
For case (a) the product f(θ)×L(θ) is 0 when θ=0,it rises in an approximately parabolic
fashion to a maximum at some intermediate value of θ, and declines to 0 again when θ=θBE .The
14
base of the quasi-parabola is θBE and its height is f(θintermediate)×L(θintermediate). The integral can be
approximated by:
(11)
To illustrate, suppose that for the representative freeway mile μ=5 night-time crashes/year and
θBE=0.38 (see row 5 in Table 2). With E{θ}=0.7 and σ{θ}=0.3, and p=0.5, f(0.5×0.38)=0.21 and
EVPIb≈ 5×20,000×0.382×0.21/3=$1010.
A similar approximation can be written for case (b). Now the quasi-parabola begins at
θBE, rises to a maximum at some intermediate θ, and comes close to 0 at θ=E{θ}+3σ{θ} or so.
Now the area is proportional to (E{θ}+3σ{θ}-θBE)× f(θintermediate)×μa(θintermediate-θBE).
Equation 11 shows that EVPI is directly proportional to the product of the number of
target crashes („μ‟) and to the cost of a target crash („a‟) or, taken together, to the cost of target
crashes (μ×a). It follows that research about the safety effect of actions aimed at units with many
and costly target crashes will have more value than research about actions aimed at the same
number of units but with fewer and less severe crashes. The next factor in equation 11 is θBE. As
θBE increases EVPI grows (more than) quadratically and rises to a sharp peak when θBE=E{θ}.
As θBE increases further the EVPI decreases from the sharp peak in a similar fashion but with a
longer tail.
Putting Humpty together
The components can now be made into a whole. By equation 9, the VOR about the θ of some
action is the product: N×(EVPIb-EVPIa) where the EVPIs are for a representative unit. For case
(a) this is, approximately,
(12)
In this equation fb(θ) denotes the pdf before the proposed research is carried out and fa(θ) what
the pdf is envisioned to be, if the proposed research is carried out. How these are estimated is
described in (26 and 30). The proportions pa and pb determine where θintermediate is before and after
the proposed research is carried out. To illustrate, the total annual number of target crashes in
Table 2 is 7610 and their annual cost is 20,000×7610= $152×106. If the representative unit has 5
15
target crashes/(mile-year), θBE=0.38. When pb pa 0.5 then, with σ{θ}=0.3 fb(0.19)=0.21 while
with σ{θ}=0.25 fa(0.19)=0.06. Therefore, 2/3×(152×106)× 0.38
2×0.5×(0.21-0.06)=$1.1×10
6.
Although equation 12 can be used for back-of-the-envelope calculations, its use is not for
computing; one can easily calculate the VOR for a representative unit by numerical integration,
without resorting to approximations. The merit of equation 12 is in its transparency. The Nμ in
equation 12 is the annual number of target crashes on the units that are subject to the decision in
which the θ plays a role; multiplied by „a‟, it is the annual cost of the target crashes. The same
factor is present in the VOR for case (b).It follows that the VOR about some θ is proportional to
the cost of those target crashes that would be affected by the decision in which that θ is used.
The second insight is in the quadratic increase of VOR with θBE as it approaches E{θ}
either from the left or from the right . It follows that the VOR tends to be large for those actions
for which the θBE is close to E{θ}. Conversely, the VOR falls off as the distance between E{θ}
and θBE increases. There is little value to research into actions for which θBE is far to the left or to
the right of E{θ} (which is where D is large).
SUMMARY
CMF research is about the safety effect of actions. Using past research results one can estimate
the mean and the standard deviation of θ. The estimate of the mean is used to predict what might
be the θ for a future action; the estimate of the standard deviation tells how close might be the
estimate of the mean to the θ that will materialize after implementation.
In a prosaic light, CMF research can be viewed as the production of information that will
help to better predict what will be safety consequences of future actions. The smaller the σ{θ}
the better will be the predictions; the better the predictions, the fewer decisions will prove to be a
bad investment. To the extent that CMF research reduces the σ{θ} it makes for fewer decisions
that will turn out to be a bad investment, for more efficient use of money, and therefore has
economic value.
The bulk of the paper is devoted to developing a computational procedure for estimating
the $-value of a proposed research about the safety effect of some action. There is little merit in
attempting to summarize this detail. However, the logic of assigning value to research, deserves
a brief restatement. The essence of the situation is that the decision has to be made using an
estimate of what the mean safety effect of the action was in the past (a current estimate of E{θ}),
16
while the θ that would materialize in the future is some presently unknown value of θ from f(θ).
The decision-maker knows that if that unknown θ will be sufficiently far from the current
estimate of E{θ}, there is a chance that decision will lead to a loss. For given values of „μ‟, „a‟,
„c‟ and „r‟ the expected $-value of this loss can be computed. If one could buy information about
what the future θ will be, a rational decision-maker should be willing to pay for it up to the
amount of this expected loss. This is the EVPIb-the expected value of perfect information for a
decision before more research is done. Someone suggests to conduct research that will reduce the
σ{θ} and thereby the expected loss to EVPIa. If that research was carried out it would reduce the
expected loss by EVPIb-EVPIa; this difference is the largest amount that the decision-maker
should be willing to pay for the execution of the proposed research. If the decision about that
kind of action is to be made N times, with the parameters „μ‟, „a‟ and „c‟ varying from one
decision to the next, the VOR is the sum of (EVPIb,i-EVPIa,i) over i=1, 2, …,N. This can be
approximated by N×( EVPIb,t – EVPIa,t) for „typical‟ values μt, at and ct.
The remainder of the paper is an attempt to explain the computational procedure in terms
of factors and considerations. The twin hope is that (a) doing so will show it to make good sense
and (b) the explanation will help to understand why one proposed research is likely to be of large
value while the value of another proposed research is likely to be small.
One of the key factors is D, the (scaled) distance between E{θ} and θBE. The larger is D
the lesser is the probability of the decision to be incorrect. D tends to be large when σ{θ} is small
and the safety consequences of action can already be accurately predicted . D also tends to be
large in two opposing circumstances: (a) when the cost of the action is large while the cost of the
target accidents is small, and (b) when the cost of the action is small while the cost of target
crashes is large. In both cases there is little doubt about what is the right decision. It follows that
the first step in assessing the merit of a proposed research is to compute its D. When D is larger
than 2 or 3 there is little to reason to spend money on doing more research.
D represents only the probability of loss, not its size; to assess the $-value of a proposed
research both probability and loss size have to be accounted for. When both are taken into
account the EVPI for a unit is shown to be approximately proportional to its cost of target
crashes (μa) and to [E{θ}-σ{θ}D]2 when θBE<E{θ}. That is, the VOR for a unit is largest for
those decisions and actions for which the θBE and E{θ} are similar. The VOR for a unit depends
17
on the extent to which the proposed research reduces σ{θ} and thereby the probability of making
incorrect decisions. This issue is treated separately in (26). In sum, three conditions combine to
give high value to a proposed research about θ:
1. There must be many units that are subject to the decision in which θ plays a role;
2. There EVPIb of a typical unit must be large;
3. There must be a much smaller EVPIa.
GLOSSARY
Acronyms
EVII Expected Value of Imperfect Information
EVPI Expected Value of Perfect Information
EVPIb The EVPI for a representative unit before the proposed research is done;
EVPIa The EVPI for the same unit after the proposed research is completed.
pdf Probability density function
VOR Value of research
Notation
^ Estimate of symbol below caret
a Average cost of one target crash
c Annual cost of countermeasure implementation
D
E{θ} Expected value of θ.
f(θ) Probability density function (pdf) of θ.
fa(θ) The pdf after the conduct of the proposed research
fb(θ) The pdf before the conduct of the proposed research
L(θ) Difference between the foregone benefit and the opportunity costs of
implementing the countermeasure.
N The number of units that are subject to the decision in which the θ plays a role;
p a number between 0 and 1
18
r how many $ of benefit can the last available $ generate; the limiting benefit/cost
ratio
Greek Letters
θ Crash Modification Factor (or Function)
θBE That θ at which the benefit-cost ratio=r, the limiting benefit/cost ratio
μ Expected (annual) number of target crashes/year.
σ{θ} Standard deviation of θ
REFERENCES
1. Schlaifer R. (1959) Probability and statistics for business decisions: an introduction to
managerial economics under uncertainty. McGraw-Hill, New York, USA.
2. Howard RA. (1966). Information value theory. IEEE Transactions on Systems, Science
and Cybernetics. SSC-2 (1): 22-26.
3. Winkler, R.L., (1972). An introduction to Bayesian inference and decision. Holt,
Rinehart and Winston, New York.
4. Berger JO. (1985). Statistical decision theory and Bayesian analysis. Springer Series in
Statistics, 2nd
.
5. Pratt, J.W., Raiffa, H., Schlaifer, R., (1995). Introduction to statistical decision theory.
MIT Press, Cambridge, USA.
6. Raiffa H. and Schlaifer R. (2000) Applied statistical decision theory. Wiley Classics
Library Series, Wiley-Interscience, New York, USA.
7. Parmigiani G. (2002) Modeling in medical decision making: a Bayesian approach.
Statistics in Practice Series, John Wiley & Sons Ltd., Chichester, UK.
8. Hammitt JK, Cave JAK. (1995): Research planning for food safety: a value of
information approach. RAND, Santa Monica, USA
9. Meltzer D (2001) Addressing uncertainty in medical cost–effectiveness analysis
Implications of expected utility maximization for methods to perform sensitivity analysis
and the use of cost–effectiveness analysis to set priorities for medical research. Journal of
Health Economics 20, 109–129.
19
10. Institute of Medicine (1998) Scientific Opportunities and Public Needs: Improving
Priority Setting and Public Input at NIH. National Academies. The National Academies
Press. Washington, D.C.
11. Fleurence RL and Torgerson DJ (2004) Setting priorities for research. Health Policy 69:
1–10
12. Meltzer DO, Basu Y, and Meltzer HY, (2009) Comparative Effectiveness Research For
Antipsychotic Medications: HowMuch Is Enough? If comparative effectiveness research
is to guide policy, then it must include evaluations of changes in policy. Health Affairs
28, no. 5 794–808.
13. Claxton KP, Sculpher MJ. (2006) Using value of information analysis to prioritise health
research: some lessons from recent UK experience. PharmacoEconomics. 24 (11): 1055-
1068.
14. Hauer, E. and Geva, I. (1984). A decision analytic approach to the design of experiments
in research on traffic safety. In: Yagar, S. (ed.), Transport Risk Assessment. University of
Waterloo Press. Waterloo
15. Thompson KM, Graham JD. (1996) Going beyond the single number: using probabilistic
risk assessment to improve risk management. Human and Ecological Risk Assessment.
2(4): 1008-1034.
16. Claxton KP, Posnett J. (1996) An economic approach to clinical trial design and
research priority setting. Health Economics. 5(6): 513-524.
17. Claxton KP. (1999) The irrelevance of inference: a decision-making approach to the
stochastic evaluation of health care technologies. Journal of Health Economics. 18: 341-
364
18. Yokota F, Thompson K. (2004) Value of information literature analysis: a review of
applications in health risk management. Medical Decision Making. 24: 287-298
19. Briggs A, Claxton KP., Sculpher MJ. (2006) Decision analytic modelling for the
evaluation of health technologies. Oxford University Press., Oxford
20. Philips Z, Claxton KP, Palmer SJ, Sculpher MJ. (2006) Priority setting for research in
health care: an application of value of information analysis to glycoprotein iib/iiia
antagonists in non-st elevation acute coronary syndrome. International Journal of
Technology Assessment in Health Care 22(3): 379-387.
20
21. Colbourn TE, Asseburg C, Bojke L, Philips Z, Welton NJ, Claxton KP, Ades AE, Gilbert
RE. (2007) Preventive strategies for group b streptococcal and other bacterial infections
in early infancy: cost effectiveness and value of information analyses. British Medical
Journal. Sep 29;335(7621):655.
22. Bojke L, Claxton K, Sculpher MJ, Palmer S, Philips, Z. (2008) The cost-effectiveness
and value of information associated with repeat screening for age related macular
degeneration. Medical Decision Making 28: 33-43.
23. Ades AE, Lu G, Claxton KP. (2004) Expected value of sample information calculations
in medical decision modeling. Medical Decision Making 24 (2): 207-227.
24. Brennan A, Kharroubi SA. (2007a) Efficient computation of partial expected value of
sample information using Bayesian approximation. Journal of Health Economics. 26:
122-148.
25. Willan AR, Pinto EM. (2005) The value of information and optimal clinical trial design.
Stat Med. 24:1791-806.
26. Hauer, E, Bonneson, J.A., Council, F., Srinivasan, R., Zegeer, C., (2011) Crash
Modification Factors: Foundational Issues. Paper submitted for publication in the
Transportation Research Record.
27. AASHTO, Highway Safety Manual, 2010
28. Elvik,R., Hoye, A., Vaa, T., Sorensen, M. The Handbook of Road Safety Measures.
Emerald Group Publishing, Bingley, UK, 2009
29. FHWA, Crash Modification Factor Clearinghouse, http://www.cmfclearinghouse.org/,
accesed on 06 April 2011.
30. Hauer, E., Estimating the Mean and Variance of CMFs and how they can be changed by
new research. „Second Report‟ prepared for the Highway Safety Research Center at the
University of North Carolina and Appendix D of Task 1 report, NCHRP Project 17-48 –
Development of a Strategic National Highway Infrastructure Safety Research Agenda.
2011.
.