Reliability Engineering and System Safety 41 (1993) 267-280

Statistical methods for the use of accident precursor data in estimating the frequency of

rare events

Vicki M. Bier Department of Industrial Engineering, University of Wisconsin, 1513 University Avenue, Madison, Wisconsin 53706, USA

(Received 3 October 1992; accepted 7 April 1993)

In estimating the frequency of rare events, the number of observed events is usually too small to support the development of accurate estimates by means of the usual statistical estimator, i.e. the number of events divided by the years of experience. Data on accident 'precursors' can help in obtaining a reasonably accurate estimator. However, past attempts to use precursor data have been problematic. In the work described here, the authors assess the problems associated with various precursor-based estimators of rare event frequencies, and propose a new estimator that seems suitable for use in practice. However, the results suggest that the competing goals of achieving low noise and low bias are inherently incompatible. Therefore, the authors recommend possible directions for future work to explore this tradeoff in more depth.

I INTRODUCTION

In estimating the frequency ~. of a rare event (e.g. a core melt accident at a nuclear power plant), the number of observed events is usually too small to support the development of accurate estimates by means of the usual statistical estimator:

N T (1)

where N is the number of observed events and T is the number of years of experience. This estimator is both unbiased (i.e. its expectation is equal to the true accident frequency ~.), and also consistent (i.e. it converges to ,~ as T gets large). However , it has extremely large variance when T is small.

For instance, in estimating the frequency of core melt at US nuclear power plants, if our observation period is one year, then T will equal roughly 100 (the approximate number of reactors in the US). Therefore, ~* will equal 0 if no core melts have occurred during the year, and 0.01 or greater if one or more core melts have occurred. However , an estimate

Reliability Engineering and System Safety 0951-8320/93/$06.00 O 1993 Elsevier Science Publishers Ltd, England.

267

of 0 is certainly too low, and, based on the results of probabilistic risk analyses (which typically yield core melt frequency estimates on the order of 10-4), an estimate of 0.01 is almost certainly much too high. Thus, if the observation time T is short relative to the expected time until the first accident, none of the possible values of 3~* will be at all helpful.

While we certainly want an estimator that is consistent, we also want it to be reasonably accurate in the short term, not only as T becomes large. Two alternative approaches have been suggested to achieve this goal: (1) probabilistic risk analysis (PRA), in which the accident frequency is estimated as a function of the failure rates of individual components of the system; and (2) the use of data on accident 'precursors' or 'near misses.' Probabilistic risk analysis is likely to yield lower-variance estimators than ;~* when T is small, because the individual components in the reactor will generally have failure rates substan- tially larger than the core melt frequency, so larger amounts of data will be available (see Refs 1 and 2). Similarly, because accident precursors occur substan- tially more often than actual core melts, precursor- based estimators might be able to provide 'leading indicators' of risk that are more accurate than ;~* when T is small.

The adoption of either PRA or a precursor-based

268 V i c k i M . Bier

approach involves an implicit acceptance that the resulting estimator of accident frequency may be biased. In other words, we are willing to accept the possibility of increased bias in our estimator, in the hope of achieving reduced noise. Biases can arise in PRA if the analyst does not capture all relevant dependencies between components or systems. Simi- larly, one possible source of bias in the precursor approach is if the observed precursors do not fully reflect the dependencies that would be observed in an accident situation. Unfortunately, as pointed out by Cooke and Goossens, 3 both of these phenomena are likely to lead to underestimates rather than over- estimates of risk. This is problematic, since in addition to low noise and small bias, we might also prefer that errors be in the conservative rather than the non-conservative direction. For example, in analyzing the frequency of core melt at a nuclear power plant, the catastrophic nature of this event may lead us to desire some degree of conservatism in our estimator.

As pointed out by Mosleh et al.,4 the precursor approach is not fully distinct from PRA, since PRA techniques such as event trees and fault trees must be used to estimate the conditional likelihood of an accident given any particular precursor. However, the precursor approach does have some advantages over total reliance on PRA. In particular, both Mosleh et al. 4 and Cooke et al. 5 note that the precursor approach uses data on observed sequences of events (rather than individual component failures) whenever pos- sible. Therefore, this approach may be better able to capture dependencies in such sequences, e.g. cases in which failure of a particular system makes the failure of additional systems more likely. In fact, this is one of the most compelling reasons for using precursor data, since subtle dependencies can easily be overlooked in PRAs.

Thus, the use of accident precursor data is an appealing method for estimating the frequencies of rare events such as core melt. Precursor-based estimators take a more empirical approach to estimating the effects of inter-system dependencies than PRA-based estimators, while at the same time yielding less noise than would be obtained simply by counting the number of observed accidents. Unfortu- nately, however, attempts to use precursor data in practice have often been problematic. Section 2 below surveys the methods that have been proposed for the use of precursor data, and their theoretical strengths and weaknesses. Section 3 discusses several issues that must be resolved in order to implement these methods in practice. Section 4 applies two selected estimators to actual precursor data from a single year to assess their performance in practice, and discusses the reasons for the observed differences. Finally, Section 5 gives recommendations for use in practice and directions for future work.

2 REVIEW OF THE LITERATURE

2.1 Early precursor-based estimators

Apostolakis and Mosleh ~ suggested a precursor-based estimator for the average core melt frequency at US nuclear power plants:

Pi

~ A M - - T

(2)

where T is the cumulative number of reactor years of experience accrued at all plants combined, pi is an estimate of the conditional probability of core melt at the ith plant given the observed precursors at that plant, and n is the number of plants in the population.

If one or more core melts actually occurred at plant i during the observation period, then Pi is set equal to 1. If no core melt occurred at plant i, pg is an estimate of the conditional probability that at least one of the observed incidents (i.e. precursors or near misses) at that plant would have progressed to a core melt;

k~

Pi = 1 - ]-I (1 - P , ) (3) j - I

where P0 is the conditional probability of an accident given the j th precursor at plant i, and ki is the total number of observed precursors at plant i. As an example of how the pij may be determined, Apostolakis and Mosleh use the results of Kazarians and Apostolakis 7 to estimate that the conditional probability of a core melt given a precursor similar to the Browns Ferry fire is approximately 3 x 10 -2.

A variant of AAM was adopted in the early accident precursor studies performed by the US Nuclear Regulatory Commission (NRC). ~'~ In the precursor studies, however, the computation was simplified to neglect the subtraction of cross-products in eqn (3) above, yielding

k~

'=' J=' (4) Z',,Rc - T

Minarick and Kukielka ~ point out that this approach can lead to overestimation of accident frequencies. In essence, the problem is that system failures are counted as failures, while system successes are assigned a non-zero probability of failure. In fact, this problem applies to virtually all precursor-based estimators, not only to '~;,nc, suggesting that precursor-based methods will often be biased high. To minimize this source of bias, Minarick and Kukieika restrict eqn (4) to include terms for only those precursors associated with initiating events; precursors involving system failures during testing are used

The use of accident precursor data 269

solvely to estimate the conditional accident probabil- ities Pij associated with other precursors. However, for simplicity we will assume in this paper that precursor-based estimators are applied to all precur- sors; unlike in Ref. 8, precursors involving failures during testing will not be excluded from ~.'nnc.

Cooke and Goossens 3 also note that A'nRC will generally yield overestimates of accident frequencies. In particular, they observe that 2,'nR c cannot converge to 2, over time, because in general the estimator A',,RC includes not only true precursors (in which no accidents occur), but also actual accidents (for which pij will equal 1). Thus, ~.'~nc is equal to ~.* plus a precursor-based term. Therefore, Cooke and Goos- sens state that as T gets large, ~.',,RC 'must eventually lead to a gross overestimate of the generic accident probability.'

It is interesting to note that 2.'~r c in fact has constant bias regardless of T (as long as factors such as the number of plants in the population and the definition of a precursor remain constant over time). To see this, we need only note that the value of ;~'~nc when T is long is simply the average of its values for shorter time periods of equal length (say, one-year intervals), and if the process is stationary, the values of ;t'~RC for these shorter intervals will be identically distributed. Thus, a priori (i.e. without knowing the number of precursors actually observed), the expected value of ;~'~nc will always exceed the true accident frequency ~, not only as T gets large.

Cooke and Goossens attribute their criticism regarding lack of consistency not only to A'~nc, but also to ~-AM, neglecting the differences between these two estimators. Since Apostolakis and Mosleh set p~ equal to 1 for any plant where an accident has occurred, ;tAM does not double-count both accidents and precursors at the same plant, and hence will not 'eventually lead to a gross overestimate of the generic accident probability' as T gets large. Of course, 2.AM does double-count in the sense of including both accidents and precursors over the population of plants, and hence will tend to be biased high, at least as long as no plant has had more than one accident. This overestimation is only a transient phenomenon, and vanishes once all plants have had at least one accident. However, this is not much help in practice, since precursor-based estimators are intended prima- rily for use in the short term (i.e. before large numbers of accidents have been observed), when the double-counting associated with 2.AM could in principle be substantial.

Unlike ~',,RC, ~'AM will eventually revert to ~.* when all plants have experienced accidents,i" at least under

t Before that time, ~-a~ will combine data on accidents at some plants with non-accident precursor data from other plants, and hence will exceed ~*.

certain conditions. (In particular, one must assume that plants are shut down rather than repaired after a core melt, and that the observation time T includes only actual operating time, with no additional time accrued during post-core melt shutdowns.) Under these conditions, however, neither 2,* nor ~-AM is consistent (at least unless there are infinitely many plants in the population), since with a finite population, the total observation time T will never grow arbitrarily large. Note that AAM is also not consistent if a single plant can experience more than one accident (i.e. if accidents are not fatal to plant operation), since Apostolakis and Mosleh set Pi equal to 1 regardless of the number of accidents actually observed at plant i. Thus, if plants can continue to operate after an accident and the total number of plants in the population is held constant, 2.AM will eventually converge to n i T (where n is the total number of plants in the population), but this quantity will tend toward 0 as T gets large.

2.2 Alternative estimators

Cooke and Goossens propose an alternative precursor-based estimator:

1 ;~,.g = ~ max(Njpj) (5)

where N/ is the number of precursors of type j occurring at any plant, pj is the estimated probability that a precursor of type j will progress to an accident, and T is the number of plant years of experience. They assume that the precursor and the subsequent system failures are 'associated' in the sense used by Barlow and Proschan 1° (i.e. the occurrence of a precursor does not decrease, and may increase, the probability of subsequent failures), which is a reasonable assumption in risk assessment. In this case, if the pj are estimated using data on all observed failures (e.g. including failures during testing, which are not preceded by an initiating event or other system failures), then the expected value of pj will be less than or equal to the true conditional accident probability given a precursor of type j. Under these conditions, X,.g is consistent, since in the limit the various precursor-based estimators considered in ;~,x will converge to something less than or equal to the true accident frequency, and the right-hand side of eqn (5) will therefore converge to the accident-based estimator ).*.

While ~.cg is consistent, some other claims made about this estimator are unfortunately not valid. In particular, Cooke and Goossens consider a specific class of numerical examples, in which all events occur deterministically (i.e. at constant intervals based on their expected frequencies), and generalize from the

270 Vicki M. Bier

behavior they observe in that context. However , while the observations drawn by Cooke and Goossens are valid for deterministic events, they are in general not true when events occur randomly, as shown below.

First, Cooke and Goossens argue that, since the non-accident terms in eqn (5) are biased low under the assumption of association, therefore )~,x will underestimate the true accident frequency )~ when no accidents have been observed. This claim is not necessarily valid, however. The non-accident terms in eqn (5) are biased low only in expectation. The expected value of the maximum of several such terms will not necessarily be biased low; that will depend on the number of terms, and also on their variability (which will be a function of T). More generally, )~,~ always either equals or exceeds )~*, and when T is finite will exceed ~,* with non-zero probability (except in degenerate cases, e.g. when all non-accident precursors have frequencies of 0). Therefore, E(),,.~) > E()~*), but since X* is unbiased, )~.~, must be biased high. (This does not necessarily imply, of course, that ~.,.g will overestimate the true accident frequency when no accidents have been observed; for example, )~,.~ could in principle be biased high when accidents have occurred, and biased low otherwise.)

In addition, Cooke and Goossens state that, if accidents have been observed, then ~',u reduces to )~*. In fact, Cooke et al. -~ make the stronger claim that )~,.~. 'looks only at precursors of maximal length in the data base,' even when the maximal precursor is not an actual accident. This might be a desirable property if it were true, since precursors of maximal length entail the largest possible portion of the overall event tree, and hence provide information on many parts of the overall accident process. While these claims are valid for the deterministic cases studied by Cooke and Goossens, neither of them is true of )~,.~, in general. However, there is an estimator that satisfies these claims. This estimator (which is clearly suggested by the comments of Cooke and Goossens, even though not explicitly defined by them) simply takes the maximum over the pj themselves (i.e. the conditional accident probabilities for all observed precursor types), instead of over the product N~pi:

accident has occurred, while )~,~ may still use data from non-accident precursors of type j if the quantity Njp~ exceeds the number of actual accidents. Therefore, unlike )~,~,, ~.~ always reverts to 7~* as soon as accidents have been observed, and thus is trivially consistent. Finally, ~.[.~ is biased high, since ~.~.~ equals )~* when )~*> 0, but will frequently exceed )~* when )~* = I).

2.3 Comparison

t The estimators )~AM, }~;,R(', ~ ' , ' g , and )~,.g all avoid the problem of yielding estimates of 0 when no accidents have occurred, as long as one or more precursors have been observed. In that regard, they are all preferable to ~.*. In general, all of these estimators are also biased high, for the following reasons:

• )~,RC is equal tO )~* plus a precursor term; • the same is true for ~'AM (as long as no plant has

had more than one accident); • )t,.~, is equal to the maximum of 2.* and a precursor

term; and ¢ • ,~,.~ is equal to ~.* when 2.* > 0 , and equals a

precursor term when )~* = 0.

However, the degree of bias in these estimators varies in predictable ways. For example, we know that

r )~,,m'--)~AM, since ~;,R~" neglects to subtract our cross-products associated with multiple precursors occurring at the same plant. Similarly, ~.c~-~ ~[.g, since ~.[.~ looks only at the precursor of maximal severity in the data base, while ~.,.g selects the precursor that yields the largest estimate. Finally, hAM will frequently exceed )~,~, since )~-u uses data from only a single precursor type, while ~'AM considers precursors of all types.t Thus, we can generally expect to see ~,R(" ~ )~AM ~ l~cg ~ A[g, and if we wish to choose an estimator with minimum bias, ~.[.~, will generally be preferable. In addition, neither I~AM n o r ~-~,n~ is consistent, and hence are undesirable for that reason;

Nkpk t ;t,,~- T ' k =argmax(p j /

Nj > 0) (6)

Essentially, the estimator ~,[.~ looks only at the precursor of maximal severity (or ' length'), while Z,.~ selects the precursor that yields the largest estimate. Thus, Xcg->Z'~.. (For the specific class of examples considered by Cooke and Goossens, in which events happen deterministically according to their expected

t t frequencies, ~,cu--> 2.c~.) Note also that ~,,.u will always use accident rather than precursor data once an

+ Admittedly, )~AM can in some cases be less than X,~., since )tAM subtracts out cross-products between multiple precur- sors occurring at the same plant. However, these cross-products will usually be small, so )'AM is likely to dominate ~.,g, especially when precursors of many different types have been observed (in which case )~,~. will ignore much of the data that is included in ~'AM)- In fact, under some conditions, ~-AM will certainly equal or exceed )~,.~.. For example, when no plant has experienced more than one precursor of the same type, the events reflected in )~,~. (which by definition are all of the same type) will also be reflected in full in )tAM; any cross-products subtracted out will only be between these events and precursors of other types.

The use o f accident precursor data 271

r ! by contrast, both A,.g and ~,,.g are consistent (Acg trivially, and ~.~, under the assumption of association).

Despite their consistency and relatively small bias, however, Acg and ;~'~ are still potentially problematic. In particular, they each use data on only a single type of precursor. While this approach avoids the problem of double-counting associated with Z'nRC, it also ignores much of the information contained in precursor data, and therefore seems likely to yield noisy estimators. To reduce noise and provide greater confidence in precursor-based estimators as a guide to risk management, it might be desirable to develop estimators that use a larger fraction of the available information. For example, one possible approach might be weighted averaging of estimators based on precursors of differing degrees of severity.

In addition, while the estimators considered here all avoid the problem of estimating 0 when no accidents have occurred (as long as precursors have been observed), they may still have extremely large variance when T is small. In particular, for short observation times, the occurrence of even a single accident or severe precursor can lead to extremely large estimated accident frequencies. In some cases these results may legitimately indicate that prior risk estimates had been too low, or had failed to include some previously unrecognized accident sequences; in fact, identifying such omissions is an important purpose of precursor studies. ~l In other cases, however, large estimated accident frequencies may simply reflect chance occurrences of rare events. The problem of distinguishing between these cases can in principle be resolved through the use of Bayesian esitmation, to take into account not only prior judgments about realistic accident frequencies, but also the degree of confidence in those judgments. This approach is discussed below.

2.4 Bayesian estimation

Bier and Mosleh ~2'13 have developed a simple Bayesian model for analyzing precursor data, and use this method to study the significance of precursors in practice. However, their method applies only to situations with a single type of precursor, which limits its applicability in risk models with realistic degrees of complexity. Oliver and Yang ~4 propose using influence diagrams to facilitate Bayesian analysis of precursor data in more general event trees, with arbitrary numbers of accident sequences and precursors. However, their method explicity assumes that the various stages of each accident sequence (e.g. successive system failures) are independent. They thus ignore some of the most useful information contained in accident precursor data. Extending their method to allow for inter-system dependencies would likely

make the resulting influence diagram models intractable.

Furthermore, even if more general Bayesian methods could be developed, it is not clear that such methods would be well suited for routine application in precursor studies such as those performed by the NRC. s'9 Unless natural conjugate prior distributions are used, Bayesian estimation tends to be mathemati- cally more complex than simple counting-type estimators such as those discussed above, and the reliance on inherently subjective prior distributions may also make Bayesian methods undesirable for regulatory applications. Therefore, the remainder of this paper focuses on non-Bayesian methods.

2.5 Summary

None of the methods developed to data appear altogether desirable in practice. Existing Bayesian methods, while theoretically rigorous, rest on unreason- able simplifying assumptions, and rapidly become intractable for problems of realistic size and complexity. The other available estimators are either inconsistent (A'nRc and AAM), o r ignore much of the available information by focusing on a single precursor type (~,cg and ~,~:c,). In addition, non-Bayesian estimators may have unacceptably large variance when T is small, since they cannot take into account prior judgments about realistic ranges for accident frequencies. Clearly, if precursor-based estimators are to be confidently used in practice, they must be put on a firmer theoretical basis, and their behavior must be better understood.

3 IMPLEMENTATION ISSUES

In applying precursor-based estimators to realistic situations, a number of issues must be resolved that do not arise in idealized contexts. First, precursors may not fall into distinct 'types,' but rather reflect an overlapping continuum of types, requiring judgment on the part of the analyst to categorize observed precursors. Second, plants may not constitute a single homogeneous population, all subject to the same types of precursors, but may instead fall into distinct 'classes'; in this case, the analyst must decide whether to pool plants from different classes for purposes of analysis, or to treat different classes of plants separately. Finally, there may be multiple accident sequences that can arise from any given precursor, rather than a single well-defined sequence, again requiring judgments regarding the most appropriate level of analysis. Each of these issues is discussed briefly in this section, and the implications of various choices are discussed. However, this discussion should

272 Vicki M. Bier

not be regarded as definitive, since precursor analysis is not yet well understood.

3.1 Precursor types

Unlike ~.'nRc and hAM , which simply combine terms corresponding to all observed precursors, ~.,,~, and ;~'~ require the analyst to identify distinct precursor types, and then choose which of these to use in the analysis based on specified criteria. In the idealized numerical example considered by Cooke and Goossens, 3 the notion of precursor types is unambiguous. In this example, each precursor involves failure of certain distinct systems (or 'top events') in an event tree. Precursors involving different combinations of system failures are clearly of different types, while those involving failures of the same system(s) are clearly of the same type.

In practice, however, such distinctions may be difficult to make. First, in actual precursors, a given system (corresponding to a particular branch on an event tree) may be subject to partial failures, e.g. failure of only one train in a multiple-train system. In addition, even precursors involving identical failures may have different durations. Particularly if the precursors in question do not involve the occurrence of an initiating event, failures of differing durations can lead to differing conditional accident probabilities (and hence differing levels of 'severity'), since initiating events would be more likely to occur during longer failures. However, )t,.g and )~,'.g assume that all precursors of the same type are characterized by the same conditional accident probability.

A number of different approaches could be taken to resolve this issue. For example, precursors involving similar but not identical failures could be grouped together for purposes of analysis, and an 'average' conditional accident probability could be defined for each such category of failures. Alternatively, one could choose to define precursor types based exclusively on their conditional accident probabilities, rather than on qualitative aspects of the precursors (such as which systems failed in each event). Since few precursors will have exactly the same conditional accident probabilities, this approach will tend to yield a larger number of precursor types, and hence possibly a noisier estimation procedure (since under this approach, estimators such as ~,,~ and ~ will tend to be computed based on only one or two precursors, rather than multiple precursors with a coarser categorization scheme). On the other hand, this approach may be simpler to apply in practice, since it would not require additional analyst judgment above and beyond the estimation of conditional accident probabilities.

Further investigation of the implications of different categorization schemes would undoubtedly be worth-

while. However, in the sample application in Section 4, precursors will be categorized purely on the basis of conditional accident probabilities. Categorization of precursors based on qualitative characteristics such as their associated system failures would be more difficult, since the NRC precursor studies do not always provide sufficient detail to facilitate this process.

3.2 Plant classes

Another difficulty in the implementation of precursor- based estimators is that plants may not constitute a single homogeneous population, but may instead fall into a number of distinct classes. In fact, the NRC precursor studies j5 divide plants into eight categories based on their design characteristics (e.g. which safety systems are present). Applying precursor analysis to individual plant classes (many of which may have comparatively small populations) will increase the noisiness of the resulting estimates, and in particular the probability that no precursors will be observed at a particular class of plants (resulting in an estimated accident frequency of 0). This would seem to argue in favor of pooled analysis.

On the other hand, plants of different classes may have widely differing accident frequencies, in which case pooled analysis would ignore potentially sig- nificant plant-specific differences in risk. Furthermore, plants of different classes may have differing combinations of safety systems and hence be subject to entirely different types of precursors. For example, because of differences in mitigation systems, the 1988 NRC precursor study ~5 was not able to identify a common set of accident sequences applicable to all classes of boiling water reactors (BWRs). This may make pooled analysis problematic for estimators such as ~.,~ and ).,'.~, which require the definition of distinct precursor types. The relative merits of pooling data across plant classes, versus analyzing each class separately, will be compared in the next section.

3.3 Multiple accident sequences

Cooke and Goossens 3 address cases in which only one event sequence is capable of leading to an accident, as shown in Fig. 1. In this case, estimators such as A,.~, or ).,'.g are relatively straightforward to apply. For example, there are seven distinct precursor types for the event tree depicted in Fig. 1: E, A, B, EA, EB, AB and EAB (which of course is an actual accident), all of which lead to the same accident sequence, EAB. Typically, however, multiple event tree sequences can lead to accidents such as core damage or anticipated transients without scram, as shown in Fig. 2. Thus, it

¢

is not clear whether estimators such as ~.,.~ or ).c~ (which require the analyst to identify distinct

The use of accident precursor data 273

suet- :1 fa i lu re

A B

L Success

SUCCESS

SUCCESS

acc iden t EAB

Fig. 1. Event tree for initiating event E and safety functions A and B 3.

precursor types) should be applied to specific event sequences individually, or to all causes of accidents taken together. In other words, should we select a different precursor type for each event tree sequence leading to core melt, or a single precursor type for the entire analysis?

If we have already decided to pool plants of different classes in the analysis, then this decision may effectively be made for us. If plants of different classes are subject to different accident sequences (as is true for US BWRs), then pooling data across classes would make it virtually impossible to analyze precursor data at the event sequence level, necessitating that the analysis be performed at the overall accident level. However, if we are analyzing different plant classes separately (or if plants of different classes are subject to the same accident sequences), then applying precursor-based estimators at the event sequence level might be a reasonable option.

At least within the same event tree, different accident sequences by definition constitute mutually exclusive events. For example, in Fig. 2, sequence 11 involves success of high pressure injection and failure

of high pressure recirculation, while sequence 12 involves failure of high pressure injection. Therefore, one could simply estimate the frequency of each event tree sequence individually, and then sum these to obtain an estimate of the overall accident frequency. (The frequencies of event sequences in different event trees (corresponding to different initiating events) can also be summed, because different initiating events constitute arrivals in independent Poisson arrivals and hence cannot occur simultaneously.) In this approach, the same precursor(s) might of course be used for several different sequences. The resulting estimates would then be dependent, but that is not necessarily a problem as long as we desire only a point estimate of the accident frequency, not an estimate of the uncertainty about that quantity. Using the same precursor for several different sequences would also not constitute double-counting of that event, since the conditional accident probability for each sequence would reflect only the likelihood of the observed precursor leading to that particular sequence, not the overall probability of core melt given the observed precursor.

Thus, one method of estimating accident fre- quencies using precursor data would be to assess the frequency of each event sequence leading to the accident in question individually, and then sum the resulting values to obtain an estimate of the overall accident frequency. Alternatively, precursor-based estimators could also be applied at the overall accident frequency level directly. Unfortunately, it is not immediately clear which approach should be adopted.

On the one hand, estimating each sequence

IT ANsl TIAFw I FwlPO V'IPO V' Is v CHALL RESEAT

i L L

HPI

I I

HPR PORV OPEN

CSR SEQ END NO STATE

Fig. 2. Event tree for PWR class A nonspecific reactor trip. t5

_ _ 20 CD

11 CD

12 CO

I ~ 21 CO

13 CD

14 CD

22 CO

15 CD

16 CO

17 CD

18 ATWS

274 Vicki M. Bier

individually has intuitive appeal. In fact, given how conditional accident probabilities are computed in the NRC precursor studies, selecting only a single precursor type for all event sequences taken together would in some sense not even estimate the total accident frequency at all, but only the combined frequency o f those sequences that could result from the selected precursor type (the most severe

t precursor, for A~g, or the precursor that yields the largest estimate, for ~-cg)- No terms would be included to account for other accident sequences, an omission that could in principle be substantial. (For example, the NRC precursor studies define several dozen event sequences that could lead to core melt, but for most precursors there are only a small handful of sequences with significant conditional accident probabilities.) By contrast, applying either A,.g or Jl;.g at the event sequence level ensures that our estimator will include terms for all accident sequences, provided that at least one precursor associated with each sequence has been observed.

On the other hand, analyzing precursor data at the event sequence level would preclude pooling data across plants with substantially different design features, if common event sequences could not be identified. This would reduce the number of plant years of experience available for use in the analysis, and hence yield noisier estimates of accident frequencies. In addition, with the exception of X~,RC, applying precursor-based estimators at the event sequence level rather than the accident level will generally result in greater bias.

Because X~,nc is strictly additive (in other words, it simply sums terms associated with all observed precursors), this estimator will yield the same estimated accident frequency regardless of whether it is applied at the individual event sequence level or the overall accident level, and thus has constant bias regardless of how it is applied. The same is not true for other precursor-based estimators, however. For example, applying hAM at the event sequence level, and then summing the estimated sequence fre- quencies, will yield results that equal or exceed those obtained by applying ~'aM at the accident level directly, since the cross-products subtracted out will be smaller at the event sequence level. However, one would expect the numerical difference between these two approaches to be small, since cross-products are only a second-order effect.

Maximization-based estimators such as X,g or ,~;u are likely to be even more affected by the level of analysis. For example, if A,g is applied at the overall accident level, then the entire core melt frequency would be estimated using data from only one precursor type. By contrast, applying A,.u at the event sequence level would involve selecting a particular precursor for each event sequence, in which case data

from a larger number of precursor types would generally be included in the analysis. It is easy to show that applying ~,,~ at the event sequence level (and then summing the estimated sequence frequencies) will yield results that equal or exceed those obtained by applying A,.~ at the accident level directly:

= maxLJvipi 1 t i t = | " ' ~ "~ T m = I "

-> max p '''~ T i ,,,=

1 = - max(Njpj) = )~,~ (7)

T j

where ~(m) is an estimator of the frequency of accident sequence m, p~m) is an estimate of the conditional probability of accident sequence m given a precursor of type j, and M is the total number of (mutually exclusive) accident sequences. Thus, apply- ing X,.g at the event sequence level will increase the conservative bias associated with this estimator.

The same is not necessarily true for ~ . This estimator maximizes the conditional accident prob- ability without regard to the number of precursors. Therefore, if the most severe precursor for some particular event sequence has occurred less often than the precursor of maximal severity overall, applying ,Vg at the event sequence level could in some cases yield results less than those obtained at the accident level. However, if the precursor that is most severe overall has been observed only once (which is fairly likely, since severe precursors will tend to occur relatively infrequently), then applying ~[.g at the event sequence level will always yield results that equal or exceed those obtained at the overall accident level. This is because ~l[.~, looks only at precursors that have occurred at least once, and if the precursor of maximal severity overall has occurred only once, then the most severe precursor for any particular event sequence cannot have occurred less often.

The tendency of ~.AM, Jt,:~, and ,~[.~ to exhibit greater bias when applied at the event sequence level seems to argue in favor of applying them at the overall accident level, especially since all of these estimators are already biased high in any case. However, it is not clear that this is actually the best approach. Although all of these estimators are biased high, this is only because they yield non-zero estimates of accident frequency when accidents have not occurred; they are not necessarily biased high conditional on having observed no accidents, which is the situation we will most often be in. Thus, even though these estimators are biased high overall, applying them at the overall accident level could nonetheless yield gross under- estimates of accident frequency when accidents have not occurred, in which case the greater bias associated

The use o f accident precursor data 275

with an analysis at the event sequence level may in fact be worthwhile.

At present, there appears to be no compelling argument for applying precursor-based estimators at either the event sequence level or the accident level. On the one hand, analyzing precursor data at the level of individual event sequences will tend to result in greater bias, and may preclude pooling data across plants with significantly different design features. On the other hand, applying estimators such as ~',x or ;~.~ at the overall accident level (i.e. by selecting only a single precursor type for all event sequences taken together) in some sense does not even estimate the total accident frequency at all, but only the combined frequency of those event sequences that could potentially result from the selected precursor type. Therefore, both approaches will be compared in the next section.

4 APPLICATION OF PRECURSOR-BASED ESTIMATORS TO ACTUAL DATA

This section compares the performance of two precursor-based estimators (Z'.rc and ~.~.~) for actual precursor data for 1989. ~5 The primary reason for choosing ~.'u rather than ;t,.g is because ;~.~ is easier to apply (since ~,.~ requires us to compute the values of multiple precursor-based estimators before selecting the maximum). In addition, while both estimators are biased high, ;t,.~ will always equal or exceed

t t f ,~,.~, so Ace is less conservative. Similarly, Anne is easier to compute than )tAM, since we need not subtract our cross-products between multiple pre- cursors occurring at the same plant. (However, the choice of Jl'nn c rather than '~AM is insignifi- cant for the data set considered here; the 1989 data include no actual accidents and no precursors with conditional accident probabilities larger than 10 -3, SO )"nnc" and )~AM will yield virtually identical estimates for this data set.) Finally, the performance of/l',n~- and ~.[.~ should bound the performance of the other estimators, since we can generally expect to

!

have )t',nc >-- ~aM >- )~,.~ >- ,~,~.

4.1 Precursor data

The 1989 precursor report 15 includes data from 107 US operating commercial nuclear power plants. For modeling purposes, these plants are divided into eight categories (BWR A, B, C, and PWR A, B, D, G, and H) based on general features of plant design (e.g. which safety systems are present). (In addition, San Onofre 1 is not included in any of the eight categories, since it has no similar sister units.) For each plant class, three event trees are then developed, one for each of three major initiating event categories: reactor

trip (RT); loss of offsite power (LOOP) ; and small loss-of-coolant accident (LOCA). A representative event tree is shown in Fig. 2. In this figure, the numbered event sequences are those leading to core damage, or in some cases to an anticipated transient without scram (ATWS). For any given class of plants, the three event trees taken together define several dozen distinct event sequences that could lead to core melt.

In 1989, a total of 30 precursors was observed at all US plants. For each precursor, a PRA-type analysis was performed to identify the event sequences that could result from the observed precursor, and to quantify the conditional probabilities of those sequences given the observed precursor. The condi- tional probabilities of core damage for different sequences potentially resulting from the same precursor were then summed to give an overall conditional probability of core damage for that event. Representative results for a few precursors, identified by their Licensee Event Report (LER) number, are shown in Table 1.

To give some feeling for the data, for the most numerous plant class (PWR B, which includes 42 operating nuclear power plants), 12 precursors were observed. (One of these events affected both units of a two-unit plant, Sequoyah, and will be analyzed here as two events, for a total of 13.) These precursors mapped onto 16 event tree sequences. A few precursors were found to result in accident sequences other than those represented by the generic event trees (e.g. sequences initiated by a medium loss-of-coolant accident or steam generator tube rupture); the precursor study includes separate analyses for these cases. Some sequences were associated with only one of the observed precursors,

Table 1. Sample precursor data

Plant name LER number Sequence Core damage number probability

Point Beach 2 301/89-002 49 9-2 x 10 -~' 54 2.3 × 10 -4 55 1.2 x 10 -5

Cook 2 316/89-014 11 1-5 x 10 -s 12 1-1 x 10 -7 15 7-2 x 10 s 16 8.0 x 10 '~ 17 7.6 × 10 -s 48 2.6 x 10 -7

53 6.5 x 10 -6 54 1-5 x 10 -~' 55 6.8 X 10 -7

Sequoyah 2 327/89-033 11 2-7 X 10 -7

16 2.2 x 10 7 71 3.1 x 10 -('

Trojan 344/89-021 17 6.7 x 10 ~ 72 2.1 x 10 -6

276 VickiM. Bier

while other sequences could have resulted from as many as half a dozen precursor events.

4.2 Effects of pooled analysis versus plant class analysis

If we consider all plant types taken together, then )t~,R(. yields an estimated accident frequency of 2 .2x 10 -5 (obtained by summing the conditional accident probabilities associated with all observed precursors, and dividing by 107 plant years). (Note that all precursors were used in this calculation, not only those associated with initiating events. There- fore, the method agrees with eqn (4), but disagrees slightly with that adopted in the early precursor studies, s''*) By contrast, applying 3,~g to the same data (at the overall accident level, not to individual event sequences) yields an estimated accident frequency of 7-2x 10-" (the frequency of the most severe precursor, 7 .7x 10 4 divided by 107), roughly a factor of 3 lower than ,~,'mo Applying the same estimators to individual plant classes yields estimates that are in roughly the same range, as shown in Table 2. However, 3,~.u appears to give results that are closer to ,~mc when applied to individual plant classes.

The results of Table 2 also suggest that precursor analysis for individual plant classes is likely to be problematic, at least for classes that are represented by only a few plants. In particular, for some classes, no precursors at all were observed, yielding estimated accident frequencies of 0 in those cases. Precursor analysis could, of course, be effectively used for small plant classes if data were pooled for time periods longer than a year, or if precursors with conditional accident probabilities less than 10 " were considered. However, if we wish to apply precursor-based estimators to the current NRC annual data, then some type of pooled analysis across dissimilar plant classes will likely be needed (although limited categorization

Table 2. Comparison of pooled analysis versus separate plant classes for ).',Re and Z ~

Plant class # P lan t s # E v e n t s )~;,R~ )v', R a t i o

Poo led ana lys i s 107 30 2.2 x l l ) s 7 .2 x 1~1 ~' 3-1

P W R A 7 1 2 - 7 x Ill " 2 . 7 x l0 ~ 1.0 P W R B 42 13 3-1Ix Ill 5 1 . 8 x 10 " 1.7 P W R D 7 4 8 - 9 x 111 > 4 " / ) x 10 ~ 2 ' 2 P W R G 8 2 1'5 x ll) ~ 1"5 x ll) ~ 1-0 P W R H 6 1 7-8 × 10 ~, 7"8 x 10 " 1.0 San O n o f r e 1 1 0 0 (I N / A

B W R A 4 0 0 0 N / A B W R B 2 1 6 .2 × 10 ~' 6 .2 x Ill ~ 1-/1 B W R C 29 8 3-6 x Ill ~' 1.2 x 10 ~' 3-0

Note: In this t ab le , ,t.~x is app l i ed a t the ove ra l l a c c i d e n t level , no t to ind iv idua l even t s e q u e n c e s .

of plants may still be possible; for example, one might wish to analyze PWRs separately from BWRs).

Note, by the way, that the decision to pool data across classes of plants with different event sequences has practical implications for other aspects of the analysis. In particular, since BWRs of different classes have different sets of event sequences, pooling data across classes of BWRs would effectively preclude performing the analysis at the event sequence level. Furthermore, when using estimators such as ,~,, and )~ (which require definition of distinct precursor types) in a pooled analysis, precursor types would likely need to be defined based on conditional accident probabilities rather than qualitative charac- teristics of the various precursors, since different classes of BWRs would in general be subject to differing types of precursors.

4.3 Effects of analysis at the event sequence level

Beginning with plants of PWR class B for illustrative purposes, applying X[~ at the overall accident level yields an estimated accident frequency of 1-8 × 10 -5 per year (as shown in Table 2 above). By contrast, applying this same estimator at the event sequence level (and summing the estimated sequence fre- quencies) yields an estimated accident fre- quency of 2.8 x 10 -~ per year, roughly 60% higher (and much closer to U, Rc). Performing the same type of analysis for other plant types yields a range of results, as shown in Table 3. For some plant classes (especially those with only a single observed precursor), analyzing the data at the event sequence level will yield exactly the same results as at the overall accident level. In other cases, the two approaches diverge by larger amounts than in the PWR B example; in fact, applying )t~,~ at the individual event sequence level can yield results that are more than double those obtained at the accident level. However, even this discrepancy is not very large relative to the level of uncertainty regarding core melt frequencies.

Table 3. Comparison of ~,,~ at the event sequence versus accident levels

Level o f ana lys i s

P l an t class # P lan t s # E v e n t s S e q u e n c e A c c i d e n t R a t i o

P W R A 7 I 2 . 7 x 10 ~ 2 . 7 x 10 ~ 1.0 P W R B 42 13 2 . 8 x It) ~ l - 8 x 10 ~ 1.6 P W R D 7 4 7 .9 x 10 5 4-0 x Ill ~ 2.1) P W R G 8 2 1-5 x 1/) ~ 1.5 x Ill ~ 1.1) P W R H 6 1 7-8 x 10 " 7.8 x Ill ~' 1.ti San O n o f r e 1 1 0 0 0 N / A

B W R A 4 0 0 0 N / A B W R B 2 1 6 .2 x l l ) " 6 .2 x 10 " l.(I B W R C 29 8 2 .8 x Ill '~ 1-2 x 10 '~ 2.3

The use of accident precursor data 277

It was noted in the previous section that analysis at the event sequence level will generally be impossible when data are pooled across plant classes with different sets of accident sequences. Thus, in practice,

!

estimators such as ;~cg will likely need to be applied at the overall accident level (at least for BWRs). Based on the results shown in Table 3, this approach does not seem problematic, since in any case the two levels of analysis appear to yield results that are in reasonable agreement.

t 4.4 Behavior of ~g over time

The results in Tables 2 and 3 above suggest that A'nnc p

is not significantly more conservative than )~cu for annual data (particularly when the latter estimator is applied at the event sequence level). In fact, for our

¢

data set, when Acg is applied to event sequence data, it yields estimated accident frequencies never less than 70% of A',RC. Thus, in this context, it may not make much difference which estimator we choose. How- ever, as the length of the data collection period gets longer, these estimators can be expected to diverge by larger and larger margins, since ~'nRC has constant bias

¢

over time, while ~,cg will eventually converge to the true accident frequency.

t To see why the bias in ~.c~, will decrease as the r observation time grows, note that Zc~, selects one

precursor type from a pool of precursor-based estimators. However, a given precursor type will be included in this pool only if at least one precursor of that type has actually occurred. This means that the

r precursor-based estimators from which ~,c.u is selected are all conditioned on the occurrence of at least one precursor. Therefore, the estimated frequency cor- responding to precursors of type i will be biased high by an amount equal to 1/(1 - e exp(-~,iT), where ~.i is the actual frequency with which precursors of type i o c c u r .

There are a few things to note here. The first is that, for any given precursor type, the extent of bias will decrease rapidly with T, at least when T is small. Secondly, the extent of bias also decreases with ;ti. In other words, the bias will be greater for rare precursors than for frequent precursors (especially

¢

when T is small). However, since ~,,.u chooses the precursor with the highest conditional core damage probability, it will tend to favor extremely rare precursors, for which the bias in the estimated precursor frequency will be greatest. Therefore, when T is small (e.g. substantially less than 1/~,i), ~,,'.~ can overestimate the actual accident frequency by a large margin. Note also that the bias may not decrease as rapidly as one might expect as T increases. This is because, as the observation time increases, the pool of

t observed precursor types from which Acg is drawn will also grow. In particular, rare precursors are more

likely to be observed as T gets large. Therefore, the bias in ).~.g may decrease only slowly (if at all) until all precursor types have been observed.

t If ~,,.g can have extremely large bias for small T, and ).,.~, is even more conservative than U.~, that poses an interesting question: why do Cooke and Goossens propose such estimators, especially since one purpose of precursor analysis is precisely to provide reasonably accurate leading indicators of risk when T is small? One reason might be a preference for erring in the conservative rather than the non-conservative direction, especially given the catastrophic nature of the accidents being analyzed. Another factor supporting Cooke and Goossens' suggestion to 'look only at precursors of maximal length in the data base' (despite the resulting bias in estimated precursor frequencies) is the belief that use of less severe precursors will be associated with significant non-conservative bias in estimated condi- tional accident probabilities.

In particular, it seems reasonable to assume that system failure probabilities are associated, in the sense of Cooke and Goossens 5 and Barlow and Proschan. "~ Some causes of this association are easy to model in PRA (e.g. failure of electric power will clearly increase the failure probabilities of systems that depend on electric power). Other types of dependency may be more difficult to model; examples include increased human error probabilities due to stress associated with past system failures, and increased failure probabilities due to extreme environments (e.g. high temperatures, or moisture due to steam leaks). In addition, system failures may be indicative of problems with plant maintenance, and therefore correlate with higher failure probabilities for other systems simply due to the chance that poor maintenance may have affected other systems.

Thus, PRA models may systematically underestim- ate the true extent of association among system failure probabilities. In this case, PRA will tend to yield non-conservative estimates of conditional accident probabilities. In addition, the extent of this non- conservative bias can be expected to grow as the number of systems to be modeled increases, since this will generally increase the number of inter-system dependencies. Thus, we may be able to reduce the non-conservative bias in estimates of conditional accident probabilities (and hence in overall accident frequency estimates) by increasing the number of systems whose failure probabilities are estimated using precursor data, i.e. by basing our accident frequency estimates on the most severe precursors.

However, as seen above, reducing non-conservative bias in our estimates of conditional accident probabilities by this approach will simultaneously increase the conservative bias in our estimates of accident precursor frequencies, possibly by quite a

278 Vicki M. Bier

large margin. To borrow the concepts of 'reliability' and 'validity' from the social sciences, '6 severe precursors have high validity, since they are similar to the events whose frequency we are interested in estimating (namely, actual accidents), and provide information on a large portion of the overall event tree. However, they also have low reliability (especially for small T), since we will generally not have enough data to estimate their frequencies accurately. Therefore, it is not immediately clear that the approach reflected in ~,~:~ and )~g (i.e. favoring the selection of severe precursors) is necessarily the most appropriate for use in practice, and further work is needed to explore the properties of various precursor- based estimators. This and other directions for future research are discussed in the next section.

5 CONCLUSIONS AND DIRECTIONS FOR FUTURE WORK

In this paper, we first surveyed the literature on use of accident precursors in statistical estimation of accident frequencies. This literature review suggested that the methods developed to date may not yet be altogether satisfactory for use in practice. Bayesian methods, while theoretically rigorous, often contain un- reasonable simplifying assumptions, and rapidly become intractable for problems of realistic size and complexity. The other available estimators are problematic as well. Several of these estimators are demonstrably inconsistent. Others ignore much of the available information, and may therefore have extremely high variance. In addition, the statistical properties of precursor-based estimators have not been well understood to date.

The discussion in Section 2 provides at least a preliminary basis for selecting precursor-based es- timators. In particular, ),~,Rc is inconsistent, and both ~'AM and )~,u are difficult to compute. In addition, ~.[.~, will generally have the least bias of any of these estimators. Therefore, analysts seeking a practical precursor-based estimator for use at present might be best advised to use ~.',,, i.e. to base their estimates on the most severe precursor observed for each event tree sequence. In any case, the example given in Section 4 suggests that the discrepancy between ~.~. and )~',R~" will generally be small, at least for annual data. Therefore, the choice of estimators may not be critical.

Sections 3 and 4 discussed issues involved in actually implementing precursor-based estimation methods. (Although ~-'nRc had already been applied in NRC precursor studies, s'9 the work by Cooke and Goossens -~ involved only simple numerical examples, rather than actual data and realistic event trees.) Our

discussion of implementation issues suggested that pooled analysis across plant classes will frequently be necessary to avoid obtaining estimated accident frequencies of () when no precursors have been observed for some plant classes. This in turn will generally necessitate that precursor-based estimators be applied at the overall accident level rather than the event sequence level, since there does not exist a common set of sequences applicable to all BWRs. Finally, in applying estimators such as ~,.~ and )t~..~ (which require the definition of distinct precursor types), perhaps the simplest approach will be to define precursor types based directly on conditional accident probabilities, to minimize the need for analyst judgment.

5.1 Directions for further work

t While )~c~ appears to be reasonable estimator for use at present, further work is needed to better characterize its behavior. In particular, we noted that )t~g (and hence also )~,~ and )d,,Rc) will tend to be biased conservatively when the observation time is small; since )~[.~ conditions on occurrence of at least one precursor, the estimated frequency corresponding to precursors of type i will be biased high by a factor of 1/(1 - exp(-)~,T)), where ~.i is the actual frequency of precursors of type i. Some degree of conservative bias in estimated precursor frequencies may be acceptable, of course, both for its own sake and to counteract the likely nonconservative bias in estim- ated conditional accident probabilities pointed out by Cooke and Goossens. However, the extent of bias in )~.~ may be greater than we desire, and further studies to quantify the likely degree of bias would therefore be helpful.

In addition, several approaches may permit the development of improved estimators. First, noise could in principle be reduced by developing estimators that use data from multiple types of precursors (like ~'AM and ,~,Rc), but are still consistent (like ,~,g and ,t~.g). One way of doing this might be to take a weighted average of estimators based on several different precursor types, with differing levels of severity. The weights could be adjusted as a function of the observation time T, such that as T gets large the weighted-average estimator would converge to )~*. This approach would permit the use of data on multiple types of precursors (thereby using more of the information contained in the data base than either )~,.~ and ,~.~), while avoiding the problem of double-counting associated with ~.~,Rc.

Alternatively, it might be possible to adjust accident frequency estimates in a Bayesian or pseudo-Bayesian manner, to account for prior judgments about the likely range of accident or precursor frequencies. Rigorous Bayesian methods are often considered

The use o f accident precursor data 279

impractical, because they require subjective estimates of parameters that may not be readily measurable or observable. However, subjectivity may be less problematic if judgment is used only to bound the range of reasonable results from an otherwise 'objective' estimator such as ~.~. Thus, for example, it may be possible to use conservative estimates for parameters such as the mean and variance of the prior precursor frequency to compute bounds on the results that could reasonably be obtained from a Bayesian analysis of the observed precursor data. The resulting bounds could then provide a check on the credibility of the results obtained from non-Bayesian estimation procedures.

Finally, it may also be possible to explicitly assess the tradeoffs between reliability (i.e. low noise) and validity (i.e. low bias) for estimators based on precursors with varying degrees of severity. This type of analysis could then provide a basis for determining the 'optimal' length or severity of precursor to use in practice. In particular, such an analysis may make it possible to replace Cooke and Goossens' reasonable but ad hoc suggestion to 'look only at precursors of maximal length in the data base' with more rigorous guidance identifying those conditions under which use of the most severe precursor is no longer reasonable.

Of course, assessing the merits of these approaches will require a formal measure of error. In other words, we need some way of translating both the noise associated with data sparsity and the bias associated with use of PRA into common terms, so that they may be explicitly traded off against each other. Unfortunately, some of the most obvious measures (such as mean squared error) may not be very appropriate. For example, means squared error will weight an order of magnitude overestimate of core melt frequency more heavily than an order of magnitude underestimate, while in reality we are likely to be more concerned about the underestimate, because of the severity of an accident if one does occur. Logarithmic measures of error would weight these two errors more reasonably, but would likely be analytically intractable, thereby necessitating the use of numerical approaches such as simulation to study the properties of alternative estimators.

5.2 Summary

While future work may provide more suitable precursor-based estimators of accident frequency, in the meantime our results suggest that two competing goals of precursor analysis (namely, achieving low noise and taking full advantage of information on dependencies) may be inherently incompatible. Capturing the dependencies reflected in accident precursors will generally favour using data from the most severe precursors. By contrast, achieving low

noise in estimated precursor frequencies would argue for using more frequent (and hence less severe) precursors. While analysts seeking a practical precursor-based estimator might at present be best

!

advised to use ;tee (based on data from the most severe observed precursor), the tradeoff between the competing aims of low noise and low bias remains an important question.

ACKNOWLEDGEMENTS

This paper was prepared with the support of the US Nuclear Regulatory Commission (NRC) under Award No. NRC-04-92-089. The opinions, findings, conclusions and recommendations expressed herein are those of the author and do not necessarily reflect the views of the NRC.

REFERENCES

1. Ravinder, H. V., Kleinmuntz, D. N. & Dyer, J. S., The reliability of subjective probabilities obtained through decomposition. Management Science, 34 (1988) 186-99.

2. Mosleh, A. & Bier, V. M., On decomposition and aggregation error in estimation: Some basic principles and examples. Risk Analysis, 12 (1992) 203-14.

3. Cooke, R. & Goossens, L., The Accident Sequence Precursor methodology for the European Post-Seveso era. Reliability Engineering and System Safety, 27 (1990) 117-30.

4. Mosleh, A., Bier, V. M. & Apostolakis, G., Methods for the Elicitation and Use of Expert Opinion in Risk Assessment: Phase 1--,4 Critical Evaluation and Directions for Future Research. NUREG/CR-4962, US Nuclear Regulatory Commission, 1987.

5. Cooke, R. M., Goossens, L. H. J., Hale, A. R. & Van der Horst, J., Accident Sequence Precursor Method- ology: A Feasibility Study for the Chemical Process Industries. Technische Universiteit Delft, 1987.

6. Apostolakis, G. E. & Mosleh, A., Expert opinion and statistical evidence: An application to reactor core melt frequency. Nuclear Science and Engineering, 70 (1979) 133-49.

7. Kazarians, M. & Apostolakis, G., On the fire hazard in nuclear power plants. Nuclear Engineering and Design, 47 (1978) 157-68.

8. Minarick, J. W. & Kukielka, C. A., Precursors to Potential Severe Core Damage Accidents: 1969-1979, A Status Report. NUREG/CR-2497, US Nuclear Regu- latory Commission, 1982.

9. Cottrell, W. B., Minarick, J. W., Austin, P. N., Hagen, E. W. & Harris, J. D., Precursors to Potential Severe Core Damage Accidents: 1980-1981, A Status Report. NUREG/CR-3591, US Nuclear Regulatory Commis- sion, 1984.

10. Barlow, R. E. & Proschan, F., Statistical Theory of Reliability and Life Testing. Holt, Rinehart and Winston, New York, 1975.

11. Lewis, H. W., Budnitz, R. J., Kouts, H. J. C., Loewenstein, W. B., Rowe, W. D., von Hippel, F. & Zachariasen, F., Risk Assessment Review Group Report

280 Vicki M. Bier

to the U.S. Nuclear Regulatory Commission. N U R E G / CR-0400, US Nuclear Regulatory Commission, 1978.

12. Bier, V. M. & Mosleh, A., The analysis of accident precursors and near misses: Implications for risk assessment and risk management. Reliabili~ Engineer- ing and System Safe~, 27 (1990) 91-101.

13. Bier, V. M. & Mosleh, A. , An approach to the analysis of accident precursors. In The Analysis, Communication, and Perception of Risk, ed. B. J. Garrick & W. C. Gekler. Plenum Press, New York, 1991.

14. Oliver, R. M. & Yang, H. J., Bayesian updating of event tree parameters to predict high risk incidents. In

Influence Diagrams, Belief Nets and Decision Analysis. ed. R. M. Oliver & J. 0 . Smith. Wiley, Chichester, 199().

15. Minarick, J. W., Cletcher, J. W., Copinger, D. A. & Dolan, B. W., Precursors to Potential Severe Core Damage Accidents: 1989, A Status Report. N U R E G / CR-4674, Volumes 11-12, US Nuclear Regulatory Commision, 1990.

16. March, J. G., Sproull, L. S. & Tamuz, M., Learning from Samples of One or Fewer. Organization Science, 2 (1991) 1-13.