a bayesian method for analyzing dependencies in precursor data

ELSEVIER International Journal of Forecasting 11 (1995) 25-41

A Bayesian method for analyzing dependencies in precursor data

Vicki M. Bier*, Woojune Yi Department of Industrial Engineering, University of Wisconsin, 1513 University Avenue, Madison, W1 53706, USA

Abstract

Past Bayesian methods for analyzing accident precursor data have rested on unreasonable simplifying assumptions; in particular, the assumption that successive stages of each accident sequence (e.g. successive system failures) are independent. However, obtaining information on intersystem dependencies is one of the greatest benefits of precursor analysis. With such dependencies, each system may have not one but several conditional failure probabilities; for example, one under normal or test conditions, and another during accident conditions (e.g. after other systems have already failed). These probabilities, while not identical, may be correlated, since the system will contain the same components (with the same inherent reliability levels) regardless of whether other systems have already failed. In this paper, extended natural conjugate distributions are used in a Bayesian method to analyze pairs of correlated probabilities. While motivated by applications to precursor analysis, the method is in fact quite general.

Keywords: Accident precursors; Bayesian analysis; Natural conjugate distributions; Dependence

I. Introduction

In estimating the frequency of rare events, the number of observed events (typically either 0 or 1) will generally be too small to support the development of accurate estimates by means of the usual statistical estimator, i.e. the number of events divided by the years of experience. Two alternative approaches have been suggested to overcome this problem: (1) probabilistic risk analysis (PRA), in which the accident frequency is estimated as a function of the failure rates of

* Corresponding author. Tel: (608) 262-2064; fax: (608) 262-8454.

individual components; and (2) the use of data on accident 'precursors' or 'near misses'. While PRA has been widely used in practice, concerns regarding its accuracy remain. In particular, PRA requires that any dependencies between individual components or systems be explicitly modeled by the analyst, so biases can easily arise if some components or systems are mistakenly assumed to be independent. By contrast, since precursors consist of event sequences rather than individual component failures, precursor data will automatically reflect the effects of any dependencies that may exist between components or systems that were challenged during the observed precursor. The analysis of precursor

0169-2070/95/$09.50 © 1995 Elsevier Science B.V. All rights reserved SSDI 0169-2070(94)02011-6

26 V.M. Bier, W. Yi / International Journal of Forecasting 11 (1995) 25-41

data has therefore attracted considerable attention as an approach for estimating the frequen- cies of severe accidents (Apostolakis and Mos- leh, 1979; Minarick and Kukielka, 1982; Cooke et al., 1987; Cooke and Goossens, 1990; Chow and Oliver, 1988; Chow et al., 1990; Oliver and Yang, 1990; Bier and Mosleh, 1990, 1991; Bier,

1993; Kaplan, 1992; Abramson, 1993). Unfortunately, accident precursor data are

difficult to analyze using classical statistical methods, for two reasons. First, while precursors will generally be substantially more frequent than actual accidents, even precursor data may be extremely sparse. Determining how much weight to give to a single severe precursor should in principle depend on how confident we were about the accident frequency before observing that event (Bier, 1993). Bayesian methods are clearly better suited than classical methods for handling problems where data sparsity makes it necessary or desirable to incorporate prior information.

Secondly, in complex systems, there may be multiple different precursors that can all lead to the same accident. For example, in Fig. 1, the observed accident precursors might well include events of types E, A, B, EA, EB, and AB, in addition to actual accidents (of type EAB). The earliest precursor-based estimators (Apostolakis and Mosleh, 1979; Minarick and Kukielka, 1982) essentially ignore the overlap between precursors of different types (say, EA and AB). Hence, these estimators yield inconsistent results, as shown by Cooke and Goossens (1990); as the

observation time gets large, they converge to a value significantly greater than the actual accident frequency.

While some subsequent estimators (Cooke and Goossens, 1990; Bier, 1993) are at least consistent, they achieve this consistency by discard- ing much of the available data and effectively restricting their attention to a single precursor type. Given the sparsity of precursor data, dis- carding data in this manner can be expected to yield highly noisy estimators. In principle, Bayesian methods should provide an avenue to overcome these problems. Several researchers (Chow and Oliver, 1988; Chow et al., 1990; Oliver and Yang, 1990; Bier and Mosleh, 1990, 1991; Kaplan, 1992) have attempted to develop and apply Bayesian models for analyzing precursor data. However, existing Bayesian methods to date have rested on unreasonable simplifying assumptions.

For example, Bier and Mosleh (1990, 1991) develop a simple Bayesian model for analyzing precursor data, and use this method to study the significance of precursors in practice. However, this model applies only to accidents with a single type of precursor, which obviously limits its applicability to more complex accident sequences. Oliver and Yang (1990) develop a method for Bayesian analysis of precursor data in more general event trees (i.e. with arbitrary numbers of accident sequences and precursor types), using influence diagrams. However, their method explicitly assumes that the various stages of each accident sequence (e.g. successive system

E A B

s u c c e s s ~

f a i l u r e 4,

s u c c e s s

success

s u c c e s s

a c c i d e n t [£B

Fig. 1. Event tree for initiating event E and safety functions A and B (Cooke and Goossens, 1990).

V.M. Bier, IV. Yi / International Journal of Forecasting 11 (1995) 25-41 27

failures) are independent. By ignoring intersystem dependencies, their method thus ignores some of the most valuable information contained in accident precursor data (Mosleh et al., 1987; Bier, 1993).

In fact, it initially appeared that Bayesian methods would become intractable when applied to problems with intersystem dependencies. However, through the use of extended natural conjugate distributions (Martin, 1967), it is possible to develop Bayesian models that are relatively tractable analytically, and also capable of handling more realistic problems.

2. Bayesian updating of event tree parameters

Perhaps the most general method for Bayesian analysis of precursor data to date is the event tree approach developed by Oliver and Yang (1990). This method is applicable to event trees with arbitrary numbers of accident sequences and arbitrary numbers of precursor types. Fig. 2 is a representative event tree for power plant accidents, in which m i is the number of times that accidents (or precursors) of type i were observed, and ~ is the failure probability of system j. (Where both ~ and ~'~ appear, 7rj is the

conditional failure probability of system j given I the success of the previous system; 7rj is the

conditional failure probability of system j given that the previous system has already failed.)

Oliver and Yang (1990) show how to use observed accident and precursor event counts (the m i in Fig. 2) to update the system failure probabilities, under the assumption that all systems are independent (i.e. the failure probability of system j does not depend on the performance

P of previous systems, so that 7r/= 7rj for all j). Thus, they assume that the failure probability of system 4 will remain constant regardless of whether system 3 has failed, and, similarly, system 5 is assumed to be independent of system 4. These are obviously restrictive assumptions, however. In fact, Mosleh et al. (1987) and Bier (1993) have argued that obtaining information on intersystem dependencies is one of the primary reasons for using precursor data.

If we wish to use precursor data to tell us about intersystem dependencies, we clearly cannot assume that the failure probabilities of systems 4 and 5 will remain constant regardless of the performance of other systems. Instead, we

t need to be able to assess both 7r 4 and 7r 4, and p

similarly both 7r 5 and ~r 5. While it is unreasonable for ~4 and ~4 to be identical, they will

lrm~Ung ~sytem Event I 2 ] .t

L"I m I

~4

I mT

x ,I m6

I if,15

X; m.l

m 1 3~t /ILl I

I 114 m.

X2 ml

Fig. 2. A representative event tree for power plant accidents (Oliver and Yang, 1990).

28 V.M. Bier, W. Yi / International Journal of Forecasting I1 (1995) 25-41

almost certainly be correlated, since system 4 will contain the same components (with the same inherent levels of reliability) regardless of whether system 3 has failed or not. Thus, when system 4 is independent of all previous systems, • r 4 and "r/" 4 will be identical (and hence perfectly correlated). When the failure probability of system 4 depends on the performance of previous systems (e.g. if the failure of system 3 creates a harsher operating environment for system 4), then ~r 4 and 7r 4 will not be equal, but they need not be independent. In fact, 7r 4 and ~r 4 will generally be partially correlated, since our estimate of rr 4 will generally be affected by learning the value of ~r 4.

Given the complexity of the influence diagram model presented by Oliver and Yang (1990) even under the assumption that ~'4 and 7r 4 are identical, it initially seemed infeasible to extend their approach to handle cases in which ~r 4 and ~'4 are partially correlated (Bier, 1993). However, the use of extended natural conjugate distributions (Martin, 1967) makes the analysis of partial correlations relatively tractable analytically, by providing a straightforward way of assessing and updating joint prior distributions over pairs of probabilities such as 4 and 7r 4. The next section introduces the idea of extended natural conjugate prior distributions, and discusses their application to precursor analysis.

3. Extended natural conjugate distributions

Martin (1967) shows that for any non-negative Borel function g, the family of distributions

h(rr, 7r ' )~ (Tr)~-l(1 - 7T)/3-1('/T') a ' -I

× (1 - rr')~'-lg(Tr, ~-') (1)

will be closed under consecutive binomial sam- pling. Observed data changes a, /3, a ' , and/3 ' , but not the function g. Thus, distributions of this form are extended natural conjugate distributions. When g(~r, 7#) = 1, ~- and 7r' are indepen- dently beta distributed of course. The coupling function g can be arbitrary (provided only that it is continuous at all but a finite number of

points). However, in order to exploit the computational benefits of natural conjugate distributions, we would like to choose g(~r, 7#) in such a way that the moments of the joint distribution h(Tr, ~") are analytically tractable. This will be true when g(1r, 7r') is a weighted sum of terms of the form

7ra(1 - 7r)b~"c(1 -- 7r') d . (2)

For convenience, in this paper we will restrict our attention to coupling functions that have binomial expansions in terms of powers of ~- and rr'. The use of binomial series makes it possible to specify a coupling function with a relatively small number of parameters. However, other choices are of course also possible, and may offer competing advantages.

Martin (1967) presents an example in which g(~r, 7r') = (It - ~r') 2. In this case, 7r and 7r' will be negatively correlated. In particular, the joint distribution h(Tr, 7r') will equal 0 along the diag- onal where 7r--zr', and the coupling function g(Tr, 7r') will take on the largest values when 7r and ~r' are far apart. More generally, negative correlations will be achieved for g(~r, It ') = (Tr - 7r') 2k for any positive integer k; the factor of 2 in the exponent is needed to ensure that g(Tr, ~") is non-negative for all 7r and 7#.

In our case, 7r and zr' are likely to be positively rather than negatively correlated, since they are failure probabilities for the same system. Positive correlations can be achieved by letting

g(rr, ~-') = 1 - (rr - It ') 2k . (3)

In this case, g(zr, ~r') will take on the largest value when 7r = rr', and decrease as the distance between 7r and It' grows. Eqs. (A1)-(A4) in the appendix give expressions for the moments of an extended natural conjugate distribution with this choice of g(~', ~r'). For this choice of coupling functions, the largest values of p(rr, rr') (the correlation coefficient of 7r and 1r') will be achieved when k = 1; in fact, p(Tr, zr') will approach 0 as k becomes large, because g(~r, ~r') converges to 1 (corresponding to independence between ~r and rr'). Unfortunately, even the largest correlations that are achievable for this

V.M. Bier, W. Yi / International Journal of Forecasting 11 (1995) 25-41 29

101

a=d=0.2 p=p'=l.0

e = d = 1 . 0 13 = 13' = 2 . 0 104 .

o ; ;o l's k

Fig. 3. p(~r, 7r') as a function of k.

choice of coupling function are generally not very substantial, as shown in Fig. 3.

Higher correlations can be achieved by taking powers, i.e. letting

g(Tr, ~r') = [1 - (~r - 7r')=k]" . (4)

In this case the coupling function g(1r, 7r') is steeper, decreasing more rapidly than Eq. (3) as the distance between ~r and zr' grows. Because of the larger correlations achieved for n > 1, o u r estimate of 7r' will generally be more sensitive to data regarding the value of 7r. The moments of 7r and It ' for this choice of g(Tr, 7r') are given in the appendix. Using this model, p(~-, ~r') is most sensitive to n when k = 1. Therefore , in the remainder of this paper we will set k = 1 and express correlation primarily through the choice of n. Choosing n sufficiently large allows the user to specify arbitrarily large correlations. More general models may still be desirable, however. In particular, since n is integer-valued, a continuous range of correlation coefficients cannot be achieved using this approach.

A simple modification allows the user to achieve any desired degree of correlation. In particular, if we let

g( r, = [ c - - ( 5 )

then increasing c while holding n constant will decrease p(~-, ~r'), making it possible to obtain any desired value for p(Tr, ~r'), rather than only a discrete set of values corresponding to the inte- gers n = 1, 2, 3, .... Note that we must have c t> 1 to ensure that g(1r, 7r') is non-negative when n is odd. The moments of ~" and ~" are again given in the appendix. As expected, p(Tr, ~-') increases in n and decreases in c.

Finally, while we may expect ~" and ~-' to be positively correlated, we may not expect them to be equal. For example, if ~r is the probability of system failure under normal conditions and ~r' is the failure probability of the same system under accident conditions, then it may not make sense for the coupling function to be maximized when

= ~' . In particular, if we expect ~r' to be a factor of q larger than ~- (for q > l ) , then g(zr, 1r') should perhaps be maximized when ~ = ~ ' / q . This can be achieved by letting

g ( ' l r , "tr ') = [c - - (Tr - - 7r '] q ) 2 ] n . (6)

In this case the coupling function g(~r, 7r') is asymmetric in ~r and 7r', and decreases with the


distance between 7r and 7r'/q. The moments of ~r and zr' are again given in the appendix; by symmetry, similar results can be established for the case where we expect zr' to be less than ~r.

In order to actually assess distributions such as these we need to know of course how the parameters a, 18, a ' , 18', n, c, and q affect the moments of the resulting joint distribution. Ide- ally, one would hope that our intuitions for beta distributions would remain largely valid for these more general distributions. For example, it would be nice if a/(a +18) were the primary determinant of E(~-), a + 18 affecting mainly the degree of spread of the distribution, and c and n affecting primarily p(~r, 7r'). However, due to the complexity of the expressions for the moments of these extended natural conjugate distributions, the validity of these hypothesized relationships must be investigated empirically. The next section presents numerical sensitivity analyses investigating these relationships.

4. Sensitivity analysis

If zr is a random variable in the interval [0,1], then its central tendency and spread, respectively, can be measured by E(~') and the normalized variance NV(~r), where

Var(~-) NV(Tr) = E(Tr)[1 - E(~')] " (7)

In particular, for 7r in [0,1], we have Var(Tr)~< E(Tr)[1- E(1r)], with equality achieved when ~r is restricted to equal either 0 or 1, so NV(Tr) gives Var(~r) as a fraction of its maximum value. More specifically, if ¢r is beta distributed with parameters a and 18, then we have

E(zr) = a / ( a +/3), (8)

1 NV(Tr) = a + 18 + 1 " (9)

Thus, for beta distributions, varying a/(a + 18) while holding a + 18 constant will affect E(zr) but not NV(Tr); conversely, varying a +18 while holding a/(a + 18) constant will leave E(1r) un- changed, but will affect NV(zr).

In assessing extended natural conjugate distributions, it would be convenient if the expressions a/(a + t8) and a + 18 were still the primary determinants of central tendency and spread, respectively. The parameters n and c in Eq. (5) above could then be adjusted to give any desired value of the correlation coefficient p(Tr, ~-'). The extent to which these relationships hold for extended natural conjugate distributions is investigated graphically below. However, these results are not definitive. The complexity of the expressions for the moments of these distributions prohibits analytical investigation of their behavior, and no attempt has been made to systematically explore the space of possible parameter values.

Fig. 4 illustrates the effect on E(Tr), NV(zr), and p(Tr, ¢r') of varying a/(a + 18) while holding a + 18 constant. As can be seen from Fig. 4, increasing a/(a +18) does increase E(Tr), al- though not quite linearly (as would be expected if zr were beta distributed). Unfortunately, both NV(Tr) and p(Tr, or') also vary with a/(a +18). Similarly, Fig. 5 shows the effect of varying a + 18 while holding a/(a + 18) constant. NV(Tr) increases as a +18 decreases, approaching a limiting value of 1 for a + 18 small (as we would expect if zr were beta distributed). However, changes in a +/3 also affect E(Tr) and p(1r, 7r').

Thus, unlike for beta distributions, there do not seem to be simple relationships between the parameters of extended natural conjugate distributions and their moments. Since changes in any one parameter will affect multiple moments of the resulting joint distribution, achieving distributions with specified means and variances is likely to involve trial and error. However, our intuitions for beta distributions are still useful as guidance, since E(Tr) is increasing in a/(a + fl), while NV(Tr) is decreasing in a +/3.

Figs. 6 and 7 show the effects of varying n and c, respectively, in Eq. (5), while holding both a and /3 constant. As can be seen, increases in n yield significant increases in p(~', ~"), and increases in c yield rapid decreases in p(Tr, ~r'), with relatively little effect on E(zr) and NV(Ir). Thus, for coupling functions of the form given in Eq. (5), once the desired mean and variance

V.M. Bier, W. Yi I International Journal of Forecasting 11 (1995) 25-41 31

2.0 - - - o - - E ( = )

NV( = ) - - ~ - - - p( = ,= ' )

1.5

1.O

0.5'

0 . 0 ' • ~ • , • • •

0.0 0.2 0.4 0.6 0.8 1.0

Fig. 4. E f fec t o f a / ( a + f l ) on moments o f ~" and ~r' when g(~-, 7r') = [1 - (~" - ~r')2] 1°, a = a ' , /3 = /3 ' , and a + /3 = 1.0.

ld

- - - o - - E ( = ) - - o - - NV(= ) • - - - ~ - - p ( = , = ' )

ld

101

10 2

103 10 3

• ' " ' ' ' " I ' " " " ' ' " I " ' " " ' ' " I ' ' ...... I . . . . . . . . I

l o 2 lO' 10 ~ lo' ld

~+13

Fig. 5. Effect of (a +/3) on moments of ~ and ~-' when g ( ~ r - ~ " ) = [ 1 - (~, ~r')2] '°, a/ (a +/3) = 0.1, a = a ' , /3 =/3' .


- - -o.-- - E(n )

- - - o - - NV(~ )

1.0" ~ p ( 11; ,T t ' )

0,5'

O ' " O - - - - - O 0 ' O ' 0 O - - - - - C .~ ~-~ '__ ~ , ( ] 0 0

0.0 , " " J ' ~ "~ 0 5 1~0 15 2 0 2 5 3 0

n

Fig. 6. Effect of n on m o m e n t s of ~ and zr' w h e n g o t , ~r ') -- [1 - (~r - 7r')2] ", (~ = ~ ' = 0 . l , a n d / 3 = / 3 ' = 0.9.

--0-- E(~ ) - -0- - - NV(~ ) , .--=.--p( ,c,,~ )

0.5

,

0 . 0 ~ ~ ' ' 0 2 4 6 8 tO

C

Fig. 7. Ef fec t o f c on m o m e n t s of ~ and ~" w h e n g ( ~ , 7r ' ) = [c - (*r - ~r')2] l°, tx = t~' = 0.1, a n d / 3 = / 3 ' = 0.9.


have been achieved, it is relatively straightforward to achieve any desired value of p(~r, 7r') by adjusting only n and c. Therefore , while simple relationships between the parameters in Eq. (5) and the resulting moments do not exist, one can find parameter values that yield a wide range of desired moments.

The effect of the parameter q in Eq. (6) is more complex. Fig. 8 shows the effect of q on the moments of 7r and ~-' for the case where a/(a +/3) = 0.1 and a'/(ot' +/3 ' ) = 0.3. (These parameter values were chosen to reflect the fact that values of q > 1 will generally be of interest only when there is an asymmetry between zr and zr'.) Increases in q yield decreases in E(~'), as shown in Fig. 8(a), and increases in E(zr'), as shown in Fig. 8(b). These results are more or less consistent with intuition. Unfortunately, however, changes in q also affect NV(~'), NV(Tr'), and p(zr, zr'), with p(Tr, 7r') decreasing rapidly in q. Therefore , while values of q > 1 allow us to represent the belief that ~-' is likely to exceed 7r, it may be difficult to achieve distributions with large correlation coefficients in this case.

5. Example

In this section the method we have developed is applied to a hypothetical example originally used by Oliver and Yang (1990). This example highlights the effect of Oliver and Yang's assumption that ~rj = 7r] for all j (i.e. that the failure probability of system j remains constant regardless of whether other systems have failed),

!

and hence p ( ~ , 7rj ) = 1. The example also allows us to explore the effect of varying p ( ~ , 1r]).

The data for this example are presented in Table 1 (taken from Oliver and Yang). This table gives the number of times (mi) that sequence i in Fig. 2 was observed for all i, as a function of the observation time T, for values of T up to 1500 years. Note that there are roughly 100 commer- cial nuclear power plants in the United States, so this hypothetical data set can be considered the equivalent of 15 calendar-years of observation of this industry. However , the values of the rni are

not representative of actual accident and precursor counts.

On the basis of Fig. 2, Oliver and Yang would update zr 4 using data of m 2 + m 4 + m 5 failures a n d m 3 + m 6 + m 7 successes of system 4, under the assumption that 7r 4 = zr 4. By contrast, using our approach, 7r 4 and zr 4 can take on different values. Thus, we can account for the fact that we observed m 2 failures and m 3 successes of system 4 when system 3 had failed, and m 4 + m 5 failures and m 6 + m 7 successes of system 4 following success of system 3. Figs. 9(a) and (b) compare our results with those of Oliver and Yang, assuming correlations of p(~r 4, zr'4)=0.30 and 0.66, respectively, at time T = 0.

The prior distributions used in Figs. 9(a) and (b) were developed by specifying the functional form of g(Trn, 7r4) , and then selecting a 4 and /34 by trial and error, to approximate the prior mean and variance assumed by Oliver and Yang. Since Oliver and Yang assume that there is no asymmetry between "/r 4 and 7'/'"4, w e let o1~ 4 = Odt4 and /34 = /3 t4 in our prior distributions. The fit to the prior moments assumed by Oliver and Yang is fairly good. In particular, Oliver and Yang let a 4 = 0.2 and /34 = 2, corresponding t o E ( T r 4 ) =

0.091 and Var(~'4)= 0.026 at T = 0; the prior moments assumed in Figs. 9(a) and (b) agree with these to within a few percent.

As shown in Fig. 9, even with moderately large prior correlation coefficients, our results give posterior values of E(~'4) that are substantially less than E(Tr4) for most times T. This is because few trials of system 4 also involve failure of system 3, so E(Tr'4) remains close to its prior value. Increasing p(Tr 4, q'rt4) from 0.30 to 0.66 does increase the influence of data obtained following success of system 3, but this effect is not overwhelming. By contrast, Oliver and Yang assume that 7r 4 = 7r'4, corresponding to the case

t p(zr 4, zr4) = 1. In this case, data obtained following success of system 3 is assumed to be fully relevant to zr 4. Thus, Oliver and Yang's results agree with ours for E(~-4), but not for E(~-4).

Fig. 10 shows the effects of correlation on the t normalized variances of ~4 and zr 4. Here again,

Oliver and Yang's results are close to ours for ~'4, but not for Ir~. In particular, since Oliver and

1.0-

0.5"

0.0 0

I I I I /

2 4 6 8 1 0

- - - -o - - E (n ) NV(n )

---,~-- p ( Ic ,~ ' )


q

(a)

1 . 0 - E(~') - - o - - NV(~) ~ p ( ~ , ~ ' )

0.s ~ ~ o o o o (b)

0 . 0 , , , ,

2 4 6 8 ;0

q

Fig. 8. Effect of q on m o m e n t s of (a) 7r and (b ) 7r' when g(~r, , r ' ) = [1 - (Tr - -Tr'/q)2] 10, a + /3 = or' + / 3 ' = 1.0, al(t~ + / 3 ) = 0.1, and a ' / (a ' + fl ') = 0 . 3 .

Yang assume that all observations of system 4 are equally relevant to 7r~, they effectively over- state the amount of relevant data. By contrast,

our results consider data obtained following success of system 3 only partially relevant to 7r~, so the effective amount of data is less than

V.M. Bier, W. Yi / International Journal of Forecasting 11 (1995) 25-41

Table 1 Counts of accident sequences over time (Oliver and Yang, 1990).

35

T ms(T ) mT(T ) m6(T ) ms(T ) m4(T ) m3(T ) m2(T ) m,(T)

100 80 1 0 0 0 0 0 0 200 130 1 1 0 0 0 0 0 300 250 2 1 1 0 0 0 0 400 360 2 2 1 0 0 0 0 500 390 2 2 1 0 0 0 0 600 500 3 2 1 1 0 0 0 700 580 3 2 2 1 0 0 0 800 650 3 2 2 1 0 0 0 900 750 3 2 2 2 0 0 0

1000 810 4 2 3 2 1 0 0 1100 890 4 2 3 2 1 0 0 1200 960 4 3 3 2 1 0 0 1300 1050 4 3 3 2 1 0 0 1400 1170 4 3 3 2 1 0 0 1500 1250 5 3 3 2 1 0 0

assumed by Oliver and Yang, and the normalized variance correspondingly greater.

Fig. 11 shows the effect of setting q =3 , corresponding to the prior belief that 7r 4 is likely to be larger than zr 4, on E(Tr4) and E(1r'4). All other parameter values remain the same as in Fig. 9(b), so the increase in q decreases p(~r 5, zr~) from 0.66 to 0.24 for T = 0 . In this case, our results are close to those of Oliver and Yang for small values of T. However, even in this case, as T becomes large, we eventually have E(~"4)< E(rr4). Thus, the data consistently indicate that

t ~4 < 7r4, a message that is missed if we assume

t 7]'4 = T/" 4 .

Figs. 12 and 13 give similar results for the mean and normalized variance, respectively, of ~'5 and 7r~, assuming p(zrs, ~r~) = 0.09 when T = 0. Once again, the moments of our joint prior distribution are within a few percent of those assumed by Oliver and Yang, i.e. E(Trs)= 0.25 and Var(rrs) = 0.014 (corresponding to a 5 = 3 and ~ 4 = 9 ) . In Fig. 12, Oliver and Yang's assumption that p(rrs, 7r~)= 1 gives posterior mean values close to those we obtained assuming p(zrs, 7r~) = 0.09, revealing little sensitivity to the choice of correlation coefficient. This is because, in Table 1, the observed data for system 5 following failure of system 4 (m 4 failures and m 5 successes) is similar to that following success of system 4 (m 6 failures a n d m 7 successes), and in any case

E(Trs) and E(~r~) both remain close to their prior values. However, Fig. 13 shows that Oliver and Yang's results for normalized variance are lower than ours for most values of T.

This example illustrates that assuming perfect correlation can yield misleading results if partial correlation in fact applies. In particular, for system 4, the assumption that 7r 4 = ~r 4 yields conservative results for E(Tr4); the conservatism of this assumption would be even greater if 7r 4 and ~r 4 had larger variances. Cases in which the error is in the nonconservative direction can also be constructed. For system 5, the assumption that rr 5 = ~r~ has little effect on posterior mean values; however, even in this case, the assumption of perfect correlation still overstates the amount of relevant data, and hence understates normalized variances, leading to overconfidence. By contrast, our approach allows the analyst to specify how relevant data from one context are, in another context, based on a prior judgment of correlation.

6. Conclusions

This paper has shown that rigorous Bayesian analysis of accident precursor data, taking into account intersystem dependencies, can be achieved through the use of extended natural

36 V..M. Bier, W. Yi / International Journal of Forecasting 11 (1995) 25-41

0.5 °

- - - o - - Oliver

0.4 ¸

0.3 ̧

x~°~02 U.J

0.1

0.0

(a)

l I | ~ 0 | I I I 200 400 600 1(300 1200 1400 1600

Time, T

0.5-

0.4

- - -o- - - Ol iver

0.3 = (b ) ca

~a-O. 2 W

0.1,

0.0 I I I I ! ! I I 0 200 400 600 800 I(300 1200 1400 1600

Time, T

Fig. 9. E ( I r , ) and E(Tr'4) vs. t ime when: (a ) p ( ' n ' 4 , .-;IT= 0)=0.30, g(*-4, or ' , )= [ 1 - ( T r , - *-4)2] 3, a , = a ; = 0.186, and /34 = /3' 4 = 1.187; (b) P(*-4, *-;I r = 0) = 0.66, g(*-4, *-4) = [1 - (*'4 - *-4)2] 1°, a4 = a~ = 0.210, and/34 =/34 = 0.940.

conjugate priors. In particular, such priors provide a framework for assessing system failure probabilities when data are sparse (e.g. under accident conditions), by using partially relevant

data collected under different conditions (e.g. tests of the same system under normal conditions). The analyst can specify the degree of relevance of such data by the selection of a


0.5-

0.4 ¢,

> o~

0.1

0.0

Oliver

i i ~ 0 ! i i i i 2oo 4oo 8oo 100o 120o- 1~o 160o

Time, T

(a)

; ----0--- Oliver j

i o~ (b) O2

Z

0.1

0.0 !

0 200 400 600 8(]0 1000 1200 1400 1600

Time, T

Fig. 10. NV(Ir , ) and NV(~'4) vs. t ime when: (a) p ( r r , , ~ ' ; I r : 0 ) = 0 . 3 0 , g ( I r , , l r 4 ) = [ 1 - ( T r , - T r 4 ) 2 1 3 , a 4 = c r ; = 0 . 1 8 6 , and /3, =/3'4 = 1.187; (b) p(~r 4, "rr'alT = 0) = 0.66, g(~4, lr4) = [1 - (Ir 4 - 1r~)2] TM, cq = a~ = 0.210, and/34 =/34 = 0.940.

correlation coefficient. However, research is needed to develop guidelines for selecting appro- priate correlation coefficients, as this is a difficult judgmental task.

We also showed, through a numerical example, that the assumption of perfect correlation can yield misleading results, and more generally that the assumed degree of correlation can have

a significant effect. In particular, the assumption of perfect correlation can give misleading expected values when the probabilities being estimated actually differ. Perhaps more importantly, even when the uncertain probabilities are close in expected value, assuming perfect correlation can still understate their normalized variances, leading to overconfidence.


o.s-, J ~ Oliver E(~.) Eln i )

0.4-

0.3

W

0.1

0 . 0 , , , , ~ , , ,

200 4(~ 600 800 1000 1200 1400 1600

Time, T

Fig. 11. E = (~'4) a n d E(~ '~) vs. t i m e w h e n p(1r 4, ~r4JT = 0) = 0.24, g(~'4, rr'4) = [1 - ("/T 4 - - 7r'4/3)2] TM, a 4 = 0 / 4 = 0.210, a n d /34 =

r /34 = 0 .940

While our interest in extended natural conjugate priors was originally motivated by the analysis of accident precursor data, such priors are potentially applicable whenever probabilities are correlated, a situation that arises frequently in practice. For example, the failure probability

of a newly designed system is likely to be correlated with the failure probabilities of other similar systems (Bier, 1989). More generally, probability estimates for different quantities provided by a single expert will be correlated if we are uncertain about the expert's degree of cali-

0.51 0.4

~ 0.3-

0.2. UJ

- . . . o - - Oliver

0.1.

Fig . 12. E(zrs) a n d E(~r~) vs. t i m e w h e n p ( ~ s , ~-;IT = 0) = 0.09, g(~r~, r r ; ) = [1 - (~'5 - 7r~)2] 3, ct5 = ~5 = 2 .79, and /35 = / 3 5 = 8.21.

0.0 | ! I I I I ! I

0 200 400 600 800 1000 1200 1400 1600

T i m e , T

V.M. Bier, W. Yi / International Journal o f Forecasting 11 (1995) 25-41 39

0 0 ¢-

>

o 0.05. N

0 Z

O.lO ~ Ol iver

! ! ! ! i i

400 600 800 1050 1200 1400

Time, T

0.00 0

Fig. 13. NV(~rs) and NV(Tr~) vs. t ime when p0rs , TraiT = 0) = 0.09, g(~rs, lr~) = [I - (~r5 8.21.

!

1600

- ~'5)' 213, as = a ~ =2 .79 , and f15 =/3~ =

bration (Harrison, 1977). Extended natural conjugate distributions can be used to model such situations, with the relevance of data from one context to probabilities in another context being specified through the choice of correlation coefficient.

Unfortunately, however, the extended natural conjugate priors developed here are rather cum- bersome to use. First, the expressions for the moments of these distributions are complex, so that desired values for these moments can generally be achieved only by trial and error. In addition, computational difficulties can be en- countered in computing these moments, requir- ing high levels of precision; this issue is discussed in more detail in the appendix. Finally, while the method proposed here can in principle be extended to model correlations among more than two probabilities, the difficulties will be even greater in that case. Thus, the method does not seem useful for modeling correlations among large numbers of probabilities, unless the problem has a great deal of special structure, e.g. the transition probability matrices of Markov chains, for which the method was originally proposed (Martin, 1967).

Further work could make extended natural

conjugate distributions easier to use in practice by developing alternative coupling functions. For example, the use of copulas (e.g. Schweizer and Sklar, 1983; Jouini and Clemen, 1994), suggested by an anonymous referee, could preserve the natural conjugate nature of the prior distributions, while facilitating selection of prior distributions with particular means and variances; however, the computation of posterior moments and correlation coefficients would probably re- quire numerical integration. Whilst further work is still needed, however, extended natural conjugate distributions can greatly facilitate the treatment of dependence in Bayesian analyses, and reduce the need for restrictive assumptions of perfect independence or perfect correlation.

Acknowledgements

This paper was prepared with the support of the U.S. Nuclear Regulatory Commission (NRC) under Award No. NRC-04-92-089. The opinions, findings, conclusions and recommendations ex- pressed herein are those of the authors and do not necessarily reflect the views of the NRC. We


are grateful to the referees and the Departmen- tal Editor for helpful comments.

Appendix: Moments of natural conjugate distributions

The moments of extended natural conjugate distributions of the forms discussed in this paper are given by Eqs. (A1)-(A4) below:

C(a,/3, a ' , /3 ' ) ') E0rla , /3 , a ' , / 30 - C(a + 1,/3, ~',/3 ' (A1)

C(a,/3, a ',/3') (A2) E(~r'l~,/3, a' , /30 = C(~,/3, ~' + 1,/3') '

Var(orlc~,/3, a' , /30 = E(orla,/3, a' , /30

x[E(~'lu + 1,/3, a ' , / 3 ' ) - E0rla,/3, a',/3')], (A3)

Cov(or, or'Is,/3, ~' , /39 = E(orla,/3, a' , /39 x [E0r'l,~ + 1,/3, a ' , /30 - E0r'l a,/3, a' , /30] ,

(A4)

where a,/3, a ' , and/3' are the parameters of the underlying beta distributions, as shown in Eq. (1). Note also that the expressions for the expressions for Var0r) and Var0r') are symmetric.

The function C(a,/3, a' , /3') in Eqs. (A1) and (A2) depends on the choice of coupling function. In particular, if we let

r(a)r(b) B ( a , b ) - r(a + b ) ' (A5)

then for g(or, ~r') = 1 - (or - ort)2k, we have

C(a,/3, a' , /3,)-1 = B(a, /3)B(a ' , /3 ' ) 2 k / 2 k \

x - ~_~o~ i ) ( - 1 ) ' B ( a + 2 k - i , / 3 ) B ( a ' + i , /30 .

(A6)

For g(or, or') = [1 - (Tr - , / ] . , ) 2 k ] n , C(o~,/3, o~ ', /3 ¢) is given by

1~=o ( n _ ~ ~(2k'/2ki)j ', = - i ) ' _ \ C(a,/3, a [3') -1 i / ( /=0

x( - 1)JB(a + 2i - j , / 3 ) B ( a ' + j , /3 ' ) . (A7)

If we let g(or, or') = [ c - (0r - or,)2]., then C(or,/3, a ' , /3 ' ) is given by

C ( o / , / 3 , off ' , /3 t ) -1 : cn-i( - 1) '

× ( - 1)JB(a + 2 i - j , / 3 )B(a '+ j , /3 ' ) . (A8)

Finally, for g(or, or') = [c - (or - ~-,/q)2]., the moments of ~" and ~r' can once again be com- puted according to Eqs. (A1)-(A4), but with C(a, /3, a ',/3 ') given by

" n • 2i / 2 i \

' z C ( o~ , /3 , ee , /3 ' ) = i = o ( ' - 1 ) ' ~ '

x( - 1/q)JB(a + 2i - j , /3)B(a' +], /3 ' ) .

(A9)

Care must be taken in evaluating C(a,/3, a' , /3 ') , since the alternating signs can lead to numerical instabilities. This is particularly true in Eqs. (A7)-(A9) when n is large. How- ever, this can be handled with suitable computational approaches. In particular, when using symbolic algebra software (e.g. Symbolics, Inc., 1988; Char et al., 1990), users can specify any desired number of significant figures to be used in the calculations. Thus, numerical instabilities that might be problematic using single or double precision can be avoided using this approach.

References

Abramson, L.R., 1993, A unified statistical approach to estimating core damage frequency based on accident sequence precursor conditional core damage probabilities (abstract), presented at PSAM-II, San Diego, California, 20-25 March 1994.

Apostolakis, G.E. and A. Mosleh, 1979, Expert opinion and statistical evidence: An application to reactor core melt frequency, Nuclear Science and Engineering 70, 133-149.

V..M. Bier, W. Yi / International Journal of Forecasting 11 (1995) 25-41 41

Bier, V.M., 1989, On the treatment of dependence in making decisions about risk, Transactions of the lOth International Conference on Structural Mechanics in Reactor Technolo- gy, Vol. P ('Probabilistic Safety Assessment') (American Association for Structural Mechanics in Reactor Technolo- gy) pp. 63-68.

Bier, V.M., 1993, Statistical methods for the use of accident precursor data in estimating the frequency of rare events, Reliability Engineering and System Safety 41, 267-280.

Bier, V.M. and A. Mosleh, 1990, The analysis of accident precursors and near misses: Implications for risk assessment and risk management, Reliability Engineering and System Safety 27, 91-101.

Bier, V.M. and A. Mosleh, 1991, An approach to the analysis of accident precursors, in: B.J. Garrick and W.C. Gekler, eds., The Analysis, Communication, and Perception of Risk (Plenum Press, New York).

Char, B.W., K.O. Geddes, G.H. Gonnet, M.B. Monagan and S.M. Watt, 1990, MAPLE Reference Manual (Water- loo Maple Publishing, Waterloo, Ontario).

Chow, T.C. and R.M. Oliver, 1988, Predicting nuclear incidents, Journal of Forecasting 7, 49-61.

Chow, T.-C., R.M. Oliver and G.A. Vignaux, 1990, A Bayesian escalation model to predict nuclear accidents and risk, Operations Research 38, 265-277.

Cooke, R. and L. Goossens, 1990, The Accident Sequence Precursor methodology for the European Post-Seveso era, Reliability Engineering and System Safety 27, 117-130.

Cooke, R.M., L. Goossens, A.R. Hale and J. Van der Horst, 1987, Accident Sequence Precursor Methodology: A Feasibility Study for the Chemical Process Industries (Tech- nische Universiteit Delft, Delft, Netherlands).

Harrison, J.M., 1977, Independence and calibration in decision analysis, Management Science 24, 320-328.

Jouini, M.N. and R.T. Clemen, 1994, Copula models for aggregating expert opinions, College of Business Adminis- tration, University of Oregon.

Kaplan, S., 1992, 'Expert information' versus 'expert opinions'. Another approach to the problem of eliciting/com- bining/using expert knowledge in PRA, Reliability En- gineering and System Safety 35, 61-72.

Martin, J.J., 1967, Bayesian Decision Problems and Markov Chains (Wiley, New York).

Minarick, J.W. and C.A. Kukielka, 1982, Precursors to Potential Severe Core Damage Accidents: 1969-1979, A Status Report, NUREG/CR-2497 (U.S. Nuclear Regula- tory Commission, Washington).

Mosleh, A., V.M. Bier and G. Apostolakis, 1987, Methods for the Elicitation and Use of Expert Opinion in Risk Assessment: Phase 1--A Critical Evaluation and Directions for Future Research, NUREG/CR-4962 (U.S. Nuclear Regulatory Commission, Washington).

Oliver, R.M. and H.J. Yang, 1990, Bayesian updating of event tree parameters to predict high risk incidents, in: R.M. Oliver and J.Q. Smith, eds., Influence Diagrams, Belief Nets and Decision Analysis (Wiley, Chichester, UK).

Schweizer, B. and A. Sklar, 1983, Probabilistic Metric Spaces (Elsevier, New York).

Symbolics, Inc., 1988, MACSYMA Reference Manual (Symbolics, Burlington, MA).

Biographies: Vicki M. BIER is an assistant professor in the Department of Industrial Engineering and the Department of Nuclear Engineering and Engineering Physics at the Uni- versity of Wisconsin-Madison. She received a B.S. in Mathe- matical Sciences from Stanford University and a Ph.D. in Operations Research from the Massachusetts Institute of Technology. Professor Bier's current research interests include risk analysis, decision analysis, and the treatment of uncertainty in decision-making. She has published numerous articles in Risk Analysis and Reliability Engineering and System Safety. Woojune YI is a doctoral student in the Department of Industrial Engineering at the University of Wisconsin- Madison.

a bayesian method for analyzing dependencies in precursor data

Documents