on the concept of perfect aggregation in bayesian estimation

11
ELSEVIER 0951-8320(94)00018-2 Reliability Engineering and System Safety 46 (1994)271-281 © 1995Elsevier Science Limited Printed in NorthernIreland.All rightsreserved 0951-8320/94/$7.00 On the concept of perfect aggregation in Bayesian estimation Vicki M. Bier Department of Industrial Engineering, University of Wisconsin, 1513 University Avenue, Madison, Wisconsin 53706, USA (Received 28 August 1993; accepted 9 May 1994) Perfect aggregation is here defined as consistency between the results of aggregate (i.e. system-level) and disaggregate (i.e. component-level) reliability analyses of the same system using Bayesian estimation. This paper identifies necessary and sufficient conditions for perfect aggregation in simple two- component systems, and discusses the implications of these results for more complex systems. Unfortunately, we show that even in systems with only two components, perfect aggregation is achieved only in very rare special cases, which will generally not be obtained in practice. Therefore, tools are needed to estimate the magnitude of aggregation error when perfect aggregation is not achieved. 1 INTRODUCTION Techniques for Bayesian estimation have been extensively studied. 1"2 However, most of this work has focused on the application of Bayesian estimation to the assessment of a single unknown quantity. Issues that arise when applying Bayesian methods in the context of disaggregate models, where there are many uncertain quantities to be assessed, have received less attention. Bayesian analysis is widely used to estimate component failure rates in reliability analyses, 2 and several researchers 3~ have investigated the situation where data are available at both the component and the system levels. When the component and system data come from non-overlapping periods of observa- tion (e.g. component data from bench tests, and system data from actual operation of the system), all data can be incorporated in a straightforward manner. First, prior distributions for the individual component failure rates can be updated with the component data; then the resulting posteriors can be propagated to provide an induced prior distribution at the system level; and finally, the prior distribution for the system failure rate can be updated with the available system data. Mastran 3 and Mastran and Singpurwalla 4 consider this problem when a prior distribution exists only for the system failure rate, and derive 'approximate' prior distributions for the component failure rates. Martz et al. 5 and Martz and Waller 6 extend this approach to the 271 case where subsystem data is available in addition to component and system data; their method integrates component, subsystem, and system prior distributions and data in a unified manner. However, when both component and system data are available from the same observation period, little guidance is available on how to proceed, and we generally have a range of choices available to us. At one extreme, we can first update the prior distributions for the individual component failure rates with disaggregate (i.e. component-level) data using Bayes' theorem, yielding a posterior distribution for each component, and then propagate the resulting posterior distributions through the model to obtain a distribution for the system failure rate. At the other extreme, we can first propagate the prior distributions for the individual component failure rates through the model, yielding a prior distribution for the system failure rate, and then update this prior with aggregate (i.e. system-level) data to obtain a posterior distribution. These two choices are illustrated schematically in Fig. 1. In practice, it will often be possible to perform the Bayesian estimation at intermediate levels of aggregation as well. Unfortunately, the results of the aggregate and disaggregate analyses generally will not agree. We will refer to this phenomenon as aggregation error, 7 and its absence as perfect aggregation. 8 Mosleh and Bier 7 showed that, when data are available at the component level, analyzing the data at the system level instead can lead to significant aggregation error

Upload: vicki-m-bier

Post on 26-Jun-2016

213 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: On the concept of perfect aggregation in Bayesian estimation

ELSEVIER 0951-8320(94)00018-2

Reliability Engineering and System Safety 46 (1994) 271-281 © 1995 Elsevier Science Limited

Printed in Northern Ireland. All rights reserved 0951-8320/94/$7.00

On the concept of perfect aggregation in Bayesian estimation

Vicki M. Bier

Department o f Industrial Engineering, University o f Wisconsin, 1513 University Avenue, Madison, Wisconsin 53706, USA

(Received 28 August 1993; accepted 9 May 1994)

Perfect aggregation is here defined as consistency between the results of aggregate (i.e. system-level) and disaggregate (i.e. component-level) reliability analyses of the same system using Bayesian estimation. This paper identifies necessary and sufficient conditions for perfect aggregation in simple two- component systems, and discusses the implications of these results for more complex systems. Unfortunately, we show that even in systems with only two components, perfect aggregation is achieved only in very rare special cases, which will generally not be obtained in practice. Therefore, tools are needed to estimate the magnitude of aggregation error when perfect aggregation is not achieved.

1 I N T R O D U C T I O N

Techniques for Bayesian estimation have been extensively studied. 1"2 However, most of this work has focused on the application of Bayesian estimation to the assessment of a single unknown quantity. Issues that arise when applying Bayesian methods in the context of disaggregate models, where there are many uncertain quantities to be assessed, have received less attention.

Bayesian analysis is widely used to estimate component failure rates in reliability analyses, 2 and several researchers 3~ have investigated the situation where data are available at both the component and the system levels. When the component and system data come from non-overlapping periods of observa- tion (e.g. component data from bench tests, and system data from actual operation of the system), all data can be incorporated in a straightforward manner. First, prior distributions for the individual component failure rates can be updated with the component data; then the resulting posteriors can be propagated to provide an induced prior distribution at the system level; and finally, the prior distribution for the system failure rate can be updated with the available system data.

Mastran 3 and Mastran and Singpurwalla 4 consider this problem when a prior distribution exists only for the system failure rate, and derive 'approximate' prior distributions for the component failure rates. Martz et al. 5 and Martz and Waller 6 extend this approach to the

271

case where subsystem data is available in addition to component and system data; their method integrates component, subsystem, and system prior distributions and data in a unified manner.

However, when both component and system data are available from the same observation period, little guidance is available on how to proceed, and we generally have a range of choices available to us. At one extreme, we can first update the prior distributions for the individual component failure rates with disaggregate (i.e. component-level) data using Bayes' theorem, yielding a posterior distribution for each component, and then propagate the resulting posterior distributions through the model to obtain a distribution for the system failure rate. At the other extreme, we can first propagate the prior distributions for the individual component failure rates through the model, yielding a prior distribution for the system failure rate, and then update this prior with aggregate (i.e. system-level) data to obtain a posterior distribution. These two choices are illustrated schematically in Fig. 1. In practice, it will often be possible to perform the Bayesian estimation at intermediate levels of aggregation as well.

Unfortunately, the results of the aggregate and disaggregate analyses generally will not agree. We will refer to this phenomenon as aggregation error, 7 and its absence as perfect aggregation. 8 Mosleh and Bier 7 showed that, when data are available at the component level, analyzing the data at the system level instead can lead to significant aggregation error

Page 2: On the concept of perfect aggregation in Bayesian estimation

272 V. M. Bier

i f

X D f(X)

p ? Fff) =

glf(X))

Fig. 1. Perfect aggregation in Bayesian estimation.

in both series and parallel systems; Philipson 9 provided another illustration of this phenomenon for series systems, and identified a similar phenomenon that can occur when aggregating data from different time periods. However, the problem is not widely recognized by Bayesian system analysts. Thus, for example, Martz and Waller 2 state that 'the assignment of the prior distr ibution(s) . . , may take place at either or both the component and system level," and do not point out that these two approaches will, in general, yield different results.

2 P R A C T I C A L S I G N I F I C A N C E OF T H E P R O B L E M

In some cases, the choice of an aggregate or a disaggregate analysis will be effectively dictated by considerations of data availability. For example, if data are collected only at the system level, then it is clearly impossible to perform a component-level analysis. However, when consistent data are available at both the component and system levels, then performing a Bayesian analysis at the aggregate level is likely to yield significant aggregation error.

For example, consider a standby system consisting of two pumps in parallel, with the second pump functioning as an installed spare for the first. Thus, when the system is required to operate, pump 1 receives a start signal; if pump 1 fails to start, pump 2 receives a backup start signal. Assume that the failure probabilities of the two pumps, P~ and /'2, are independent and have beta distributions with para- meters ai = 10, bi = 90. Thus, each pump has a prior mean failure probability of 0.1; assuming that the pumps are independent, the prior mean failure probability of the system as a whole will be 0.01. Assume also that the system has been tested 100 times, with pump 1 failing 50 times in 100 trials, and pump 2 failing five times in 50 trials. Thus, the system as a whole will have failed five times in 100 trials.

If we update P~ and 1'2 with the disaggregate data of 50 failures in 100 trials and 5 failures in 50 trials, respectively, then the posteriors for P~ and P2 will be beta distributed with a'~=60, b'~=140, a ' : - 15 , b" = 135. Convolving these two posterior distributions to obtain a distribution for the system failure probability, P = P~P2, yields the distribution labeled 'DISAGG' in Fig. 2, with a mean value of 0.03. By contrast, if we first convolve the prior distributions for P~ and P2, and then update with the aggregate data of five system failures in 100 trials, we obtain the distribution labeled ' A G G R E G ' in Fig. 2, with a mean value of only about 0-02. Since the two distributions do not agree, an aggregation error has occurred. Essentially, the aggregate distribution ignores some of the information available in the disaggregate data.

Comparing the two distributions in Fig. 2, we see that the disaggregate distribution is slightly narrower, and also weighted towards larger values. This makes sense, because although both the aggregate and the disaggregate data suggest that the system failure probability is larger than the prior mean value of 0.01, the disaggregate data provide stronger evidence of this, due to the large amount of data available for component 1.

Errors of the sort shown in Fig. 2 can be important in practice. For example, prior to the Challenger disaster, an aggregate analysis using only the system-level data of no accidents in 23 flights would have indicated a decreasing accident probability with each flight, based on the number of observed successes. However, a disaggregate analysis consider- ing both the frequency of O-ring erosion and also the conditional probability of an accident given such erosion would have accounted for the observed O-ring erosion on six of the 23 flights, and therefore would likely have yielded a substantial increase in estimated accident risk. In this case, an aggregate analysis would

Aggregate and disaggregate posteriors system failure probability

Distribution

/ - - DISAGG

/ I I

! .

/ /

/

P

le-004 le-003 le-002 le-001

Fig. 2. Aggregate and disaggregate posteriors in the case of aggregation error.

Page 3: On the concept of perfect aggregation in Bayesian estimation

Perfect aggregation in Bayesian estimation 273

have misestimated not only the magnitude of the effect associated with the observed shuttle perfor- mance, but also the direction of the effect. In fact, while a rigorous Bayesian analysis was not performed prior to the Challenger launch decision, it can be argued that the Challenger disaster stemmed in part from excessive reliance on the aggregate data of 23 successful shuttle flights, and insufficient attention to the disaggregate data on O-ring erosion.

Similar problems can arise in other contexts as well. For example, the U.S. Nuclear Regulatory Commis- sion has compiled data on initiating events that can lead to core melt at nuclear power plants; Idaho National Engineering Labotatory "~ presents data for 41 different categories of such events. However, risk analyses do not usually model each of these 41 initiating event categories separately, since many categories of events have virtually the same effect on power plant operation. Therefore, data analyses such as the one by PLG, Inc., ~ often consider such data in an aggregate manner. This avoids the need to assess numerous prior distributions, but will generally lead to aggregation error.

One can argue that in scenarios such as these, where both aggregate and disaggregate data are available, it is inappropriate to use only the aggregate data, since it does not reflect our complete state of knowledge. However, in practice it will often be impractical to completely avoid aggregation error. First, collecting and analyzing large quantities of disaggregate data can be expensive, and may not always be worthwhile when aggregation error is small. In addition, we may not be sure that our disaggregate model is correct, and hence may wish to use aggregate data in order to avoid modeling errors, such as omission of key failure modes. Disaggregate data can also be less accurate than aggregate data (for example, if errors occur in assigning observed system failures to specific components or failure causes), in which case we may prefer to use the aggregate data. Finally, even if accurate disaggregate data and models are available, it can be difficult to assess the large number of prior distributions sometimes needed at the disaggregate level.

Thus, analysts are unlikely to be able to completely avoid aggregation error in practice. In fact, Philipson 9 describes aggregation error as a 'fundamantal problem', suggesting that it may invalidate Bayesian system reliability analysis altogether. We take issue with this extreme view. Rather, we note that when accurate disaggregate data, prior distributions, and models are available, then concerns about aggregation error will favor a disaggregate rather than an aggregate analysis. However, since limits exist on the extent of decomposition that can reasonably be undertaken in practice, determining the appropriate level of decomposition is likely to involve trade-offs

among several competing sources of error, of which aggregation error is typically only one. This highlights the need for methods to estimate the magnitude of aggregation error incurred when perfect aggregation is not achieved. ~2

In this paper, we identify necessary and sufficient conditions for perfect aggregation in a variety of simple two-component reliability systems, and discuss the implications of these results in practice. Unfortun- ately, we show that perfect aggregation is achieved only in very rare special cases, which will generally not be obtained in practice. While these results pertain only to very simple two-component systems, they put the study of perfect aggregation on a rigorous basis, as a first step toward developing tools to measure aggregation error.

3 AGGREGATION ERROR IN OTHER FIELDS

The concept of aggregation error has been extensively studied in a variety of fields, including econometrics 13 ~6 ecological modeling, 8"17-21 and aut- omatic control theoryfl 2 The absence of aggregation error has been referred to in these fields variously as perfect aggregation, 8 total consistency, TM and exact aggregation. ~5 In the context of dynamic (i.e. time-varying) systems such as ecological models, Iwasa et al. 8 define perfect aggregation as follows: 'If the aggregated dynamics are consistent with the original dynamics in the sense that the macrovariables behave identically both in the full system and in the aggregation... , then we can use the aggregated dynamics in place of the detailed dynamics.'

Phenomena similar to perfect aggregation have also been noted in decision theory. For example, identifying the conditions for perfect aggregation is similar to the problem of choosing an appropriate small world for analysis, as discussed by Savage. 23 Perfect aggregation is also related to several proper- ties involved in the combination of expert opinions. These properties include marginalization, 24 26 in which the consensus probability of an event does not depend on how it is partitioned into disjoint subevents, and data independence 26 (also referred to as external Bayesianity 27 or prior-to-posterior coherence28), in which the consensus distribution remains unchanged regardless of whether expert opinions are combined before or after Bayesian updating. Finally, in an analog of Arrow's impossibility theorem, Dalkey 29-3° has shown that no axiomatic method for combining expert estimates of probabilities will yield the same results when applied at the system and component levels.

Thus, aggregation error and related problems have been extensively studied in fields as diverse as

Page 4: On the concept of perfect aggregation in Bayesian estimation

274 V. M. Bier

econometrics, ecological modeling, and decision theory. With few exceptions, 7'9 however, these issues have received little recognition in Bayesian reliability analysis.

4 CONDITIONS FOR PERFECT AGGREGATION IN SIMPLE TWO- COMPONENT SYSTEMS

In this section necessary and sufficient conditions for perfect aggregation are developed for simple two- element reliability models, under the assumption that the prior distributions for the component failure rates in those models are independent. Four such models will be considered: products of two probabilities: products of a probability and a frequency; sums of frequencies; and the so-called probabilistic sum, as given by P~ + P 2 - P~P2. Subsequently, Section 5 will discuss the implications of these results in practice.

4.1 Products of probabilities

Consider a system made up of two components in parallel, where component 1 has failure probability P~ on any given trial, and component 2 has conditional failure probability P2 given failure of component 1. (As in Section 2, component 1 could be a standby component, with component 2 functioning as an installed spare.) Therefore, the failure probability of the system is given by the product/'1/'2. Assume that P~ and P2 are independent, with prior probability distributions fp~(-) and fP2('), respectively. Now, let us observe ko trials of the system, and collect the following data: the total number of failures of component 1 (denoted by ki); and the number of failures of component 2 in the k~ trials on which component 1 failed (denoted by k2). In this case, the number of system failures in k0 trials will simply be k 2.

parameters ai and bi, respectively, and let a~ = a2 + b:. Then the product P~P2 will be beta distributed with parameters a2 and b~ + b2.

This result can be shown either by transformation of variables, or as a special case of a more general result for products of beta distributions. 3~ Therefore, the result obtained by updating the prior distribution for the product Pl/)2 with data of k 2 system failures in k. trials will be beta distributed with parameters a2 + k2 and bl + b2 + k o - k2.

If we instead first update the distributions for P~ and P2 individually, then the posterior distribution for P~ will be beta with parameters a'~ =a l + k~ = a2 ~ b2 + k~, b'l =bl + k o - k ~ . Similarly, the posterior for P~ will be beta with parameters a'2=a2+k2, b'2= b2 + k~ - k2. These parameters satisfy a'~ = a~ + b'. Therefore, by Lemma 1 above, the posterior distribution for the product P~P2 obtained by convolution of these two beta distributions will itself be beta with parameters a ' = a 2 +k2 and b'~ +b'2 b~ + b2 + k o - k2. However, this is exactly the result obtained by performing the convolution before the Bayesian updating. Since the order of these two operations is irrelevant, we therefore have perfect aggregation. The proof of necessity (i.e. that the must be beta distributed with a~ = a2 + b2 in order to have perfect aggregation) is given in Appendix A.

It is interesting to note, by the way, that the condition al = a z + b 2 is simply an analog of the observation process by which the data were collected. In other words, if we interpret ai as the equivalent number of failures of component i contained in our prior distribution, and a~ +bi as the equivalent number of trials, then the restriction a~ =a2 +b2 simply states that the number of trials of component 2 must be equal to the number of failures of component 1, which would be the case if component 2 were tested only when component 1 fails.

Theorem 1. In order to have perfect aggregation for the product ,°1/'2, it is necessary and sufficient that the fei(') be beta conjugate priors with parameters ai and by, where a~ = a2 + b2.

Proof. First we show that the stated condition is sufficient for perfect aggregation. By definition, perfect aggregation means that convolving the posterior distributions for P~ and 1)2 (updated with data of k t failures in ko trials and k2 failures in k~ trials, respectively) yields the same result as obtaining a prior distribution for the product P~ P2, and updating this prior with data of k2 system failures in ko trials. Thus, the results of the disaggregate analysis must depend only on k2, not on k~.

Lemma 1. Let PI and P2 be beta distributed with

4.2 Probabilistic sums

Consider a system made up of two components in series (rather than in parallel), where component 1 has failure probability P1 on any given trial, and component 2 has conditional failure probability P2 given success (rather than failure) of component 1. (For example, component 1 could be a diesel generator providing electric power to component 2, so that component 2 cannot be tested unless component 1 operates successfully.) As before, let .P~ and P2 be independent, with prior distributions fp~(-) and fp2('), respectively. Now, let us observe k, trials of component 1, and collect the following data: the total number of failures of component 1 (denoted by kl), and the total number of failures of component 2 in the k0 - k~ trials on which component 1 succeeds

Page 5: On the concept of perfect aggregation in Bayesian estimation

Perfect aggregation in Bayesian estimation 275

(denoted by k2). In this case, the total number of system failures in ko trials will be k~ + k2.

The failure probability of this system is given by the probabilistic sum; i.e. P~ + P2- PtPz. Note, however, that this is just equal to 1 - ( 1 - P 0 ( 1 - P2). Thus, perfect aggregation holds for the probabilistic sum of the P~ if and only if it holds for the product of their complements. If we let Qg = 1 - P/be beta distributed with parameters a~ and b~, then P~ will be beta with parameters a; = b~ and b; = a~. Of course, since perfect aggregation for the product Q~Q2 holds if and only if a~ = a2 + b2, we must similarly have b~ = a2 + b2.

4.3 Sums of frequencies

Now consider a system made up of two components in series, where components 1 and 2 fail according to independent Poisson processes with rates A~ and A2. (For example, components 1 and 2 could be normally operating components that fail randomly over time.) Let A1 and A2 be independently distributed with prior distributions fa~(-) and f~2('), respectively. Let us observe the system for a total of t time units, and collect data on the total number of failures of each component (denoted by k~ and k2, respectively). In this case, the number of system failures in t time units will be k~ + k2.

Theorem 2. In order to have perfect aggregation for the sum At + Az, it is necessary and sufficient that the fai(') be gamma conjugate priors with parameters a i

and bi, where b~ = b2.

Proof. First we show that the stated condition is sufficient for perfect aggregation. By definition, perfect aggregation means that convolving the posterior distributions for A1 and A2 (updated with data of k~ failures of component i in t time units) yields the same result as obtaining a prior distribution for the sum A~ + A2, and updating this prior with data of k~ + k2 system failures in t time units. Thus, the results of a disaggregate analysis must depend only on the sum k~ + k2, not on the individual ki.

Lemma 2. Let A~ and A2 be gamma distributed with parameters ai and b~, respectively, and let b~ = b2. In this case, the sum AI + A2 will be gamma distributed with parameters a~ + a2 and bt.

The most straightforward proof of this result is using characteristic functions. Therefore, the result obtained by updating the prior distribution for the sum A~ + A2 with data of kt + k2 system failures in t time units will be gamma distributed with parameters a~ +a2+k l +k2 and bl + t.

If we instead first update the distributions for At and )t2 individually, then the posterior distribution for )t~ will be gamma with parameters a ' t = a t + k t , b't=

bl + t. Similarly, the posterior for A2 will be gamma with parameters a~=a2+k2 , b ~ = b 2 + t = b l + t . These parameters still satisfy b't = b~. Therefore, using Lemma 2, the posterior distribution for the sum At +A: obtained by convolving these two gamma distributions will itself be gamma distributed, with parameters a't + a~ = at + a2 + kt + k2 and b'l = b~ -- bt + t. However, this is exactly the result we obtained by performing the convolution before the Bayesian updating. Since the order of these operations is irrelevant, we have perfect aggregation. The condition that the Ai be gamma distributed with bt = b2 is also necessary for perfect aggregation; a proof of necessity is outlined in Appendix B.

4.4 Products of a probability and a frequency

Consider a system made up of two components in parallel, where component 1 fails according to a Poisson process with rate A, and component 2 fails according to a Bernoulli process with conditional probability P given failure of component 1. (For example, component 1 could be a normally operating component, and component 2 a standby for component 1.) As before, let A and P be independently distributed with probability distribu- tions fa(-) and fp(.), respectively. Let us observe the system for a total of t time units, and collect data on the numbers of failures of components 1 and 2 (denoted by kt and k2, respectively). In this case, the total number of system failures in t time units will be k2.

Theorem 3. In order to have perfect aggregation for the product AP, it is necessary and sufficient that f~(.) be a gamma conjugate prior distribution with parameters a and b, and fp(-) be a beta conjugate prior with parameters c and d, where a -- c + d.

Proof. First we show that the stated condition is sufficient for perfect aggregation. By definition, perfect aggregation is equivalent to the statement that convolving the posterior distributions for h and P (updated with data of kl failures of component 1 in t time units, and k2 failures of component 2 in k~ trials) yields the same result as first obtaining a prior distribution for the product AP, and then updating this prior with data of k2 system failures in t time units. (In other words, the results of a disaggregate analysis must depend only on k2, not on kl.)

Lemma 3. Let A be gamma distributed with parameters a and b, and let P be beta distributed with parameters c and d, where a = c + d. In this case, the product AP will be gamma distributed with parameters c and b.

This can be shown either by transformation of

Page 6: On the concept of perfect aggregation in Bayesian estimation

276 V. M. Bier

variables, or as a special case of more general results for products of independent distributions) ~ Therefore, the result obtained by updating the prior distribution for the product AP with data of k2 system failures in t time units will be gamma distributed with parameters c + k2 and b + t.

If we instead first update the distributions for A and P individually, then the posterior distribution for A will be gamma with parameters a ' = a + k~, b ' = b + t. Similarly, the posterior for P will be beta with parameters c' = c + k2, d' = d + k~ - k2. These para- meters will satisfy a' = c' + d'. Therefore, by Lemma 3, the posterior distribution for the product AP obtained by convolving these two distributions will itself be gamma distributed, with parameters c ' = c +k2 and b ' = b +t . However, this is exactly the result obtained by performing the convolution before the Bayesian updating. Since the order of these two operations is irrelevant, we have perfect aggregation. The condition a = c + d is also a necessary condition for perfect aggregation. This can be shown using techniques similar to those in Appendices A and B: the proof is omitted for reasons of space.

4.5 S u m m a r y

Table 1 summarizes the results presented above for simple two-component systems.

5 I M P L I C A T I O N S O F R E S U L T S

Determining how detailed models should be is often an important quest ion--for instance, in analyzing the risks of complex technological systems and processes such as nuclear power plants, hazardous material storage facilities, or the space shuttle. Both overly detailed and overly simplistic models can lead to erroneous esimates, for different reasons. 7'32 33

We have shown that perfect aggregation is vanishingly rare, at least in simple two-component systems. This implies that aggregation error will frequently result from the use of overly aggregate data and models (e.g. system-level rather than component-

Table 1. Conditions for perfect aggregation

Type of sys tem

Q u a n t i t y be ing e s t ima ted Para l le l Ser ies

Probab i l i ty

Fa i lu re ra te

P,& e~+P2-P, e2 g - B e t a (a i, bi) C - Be ta (a i, b i )

a I = a 2 + b 2 b l = a 2 + b 2

AP A i + A2 A - G a m m a (a, b) A, - G a m m a (a i, bi) P ~ Be ta (c, d ) bl = b2 a = c + d

level data). Such errors can be important in practice: for example, in analyses of accident precursors 34 (such as the space shuttle flights with damaged O-rings prior to the Challenger disaster), and in developing initiating event data bases.l~

A numerical example 7 indicates that aggregation error can be substantial. Consider two initiating event frequencies A1 and A 2, and assume that we are interested in estimating A = A~ + A2. Let Aj be gamma distributed with parameters a~ = 1, b~ = 100, and let A2 be independently gamma distributed with parameters a2 = 1, b2 = 10. Thus, we have E(A~) = 0.01, E(Az) = 0.1, and E(A)= 0.11. If we now observe the system in question for 10 time units and observe a total of three initiating events of both types taken together, then the posterior mean resulting from an aggregate analysis will be given by

E ( a I k = 3, t -- 10)

k~l (k + l) V(al +i)r(a2+k + l - i ) ,:o i (b , + t y (b2 + t) k~ '

- = 0.20 ,~) ( k ) F(a~ + i)F(a2 + k - i)

• (b, T -70 E '

where k = k t + k2. This is close to the value we would obtain from a

disaggregate analysis if all of the events were of type 2:

E(a, I k, = 0, t - - 10) + E(a2 I k2 = 3, t = 1o)

a ) + k l a 2 + k 2 - - - ~ - - - 0.21

b~+t b , + t

However, if the events had all been of type 1, then a disaggregate analysis would yield

E(/~. l l kl ~--- 3, t = 10) + E(A2 ]k2 = 0, t = 10)

a~ + kl a2 + kz - + - - - 0"09

b~+t b2+t

(somewhat less than the prior mean value of 0.11, and tess than half the value obtained from the aggregate analysis). Thus, as noted in Section 2, an aggregate analysis can misestimate both the magnitude and the direction of the effect associated with observed data.

In fact, Philipson 9 suggests that the problem of aggregation error may invalidate Bayesian system reliability analysis altogether. We take issue with this extreme view. However, analysts clearly need to use care in determining the level at which to perform reliability analyses. In particular, when accurate disaggregate data, prior distributions, and models are available, then concerns about aggregation error will favor a disaggregate analysis. However, in practice, limits will exist on the extent of decomposition that can reasonably be undertaken. Not only can extensive decomposition be costly and computationally intrac- table, but such decomposition is likely to introduce

Page 7: On the concept of perfect aggregation in Bayesian estimation

Perfect aggregation in Bayesian estimation 277

errors of other types. 7 As reliability models are decomposed to finer and finer levels of detail, sooner or later either the disaggregate data, the correspond- ing prior distributions, or the models themselves are likely to become inaccurate.

Thus, determining the appropriate level of detail in a reliability model is likely to involve trade-offs among several competing sources of error, of which aggregation error is typically only one. This highlights the need for methods to estimate the error that will be incurred when perfect aggregation is not achieved. ~2

6 EXTENSIONS OF THIS WORK

The results presented in this paper offer an initial analysis of aggregation error. Although rigorous results are presented only for simple two-component reliability systems, these results suggest that perfect aggregation is likely to be rare in practice, and show that the subject is amenable to rigorous analysis. In addition, numerical examples presented here and elsewhere 7 have shown that the magnitude of aggregat ion error can be significant. For example, aggregate analysis may in some cases show a decrease in risk even when disaggregate analysis of the same data would yield an increased risk estimate.

While our results thus suggest that aggregation error is important in practice, this paper has addressed only the simplest of reliability systems. It is important to know whether the conditions for perfect aggrega- tion in models with multiple components are in fact as restrictive as those we have derived for two- component systems. Recent w o r k 35-36 demonstrates that this is the case, at least for fairly broad classes of reliability systems. Similarly, it would be helpful to know the conditions for perfect aggregation when prior distributions need not be independent. Hazen 37 has identified such conditions for two-component systems, and also for some systems with multiple components.

Finally, given that perfect aggregation appears unlikely to be achieved in practice, there is a clear need for measures of aggregation error. 12 Such measures will allow analysts to estimate the extent of error likely to be incurred when perfect aggregation is not achieved, as a necessary first step in trading off aggregation error against other competing sources of error.

ACKNOWLEDGEMENTS

The author is grateful to Professor Laurel Travis (Metropolitan State University, Minneapolis), Profes- sor Roger Cooke (Technical University of Delft, The Netherlands), and an anonymous referee for helpful

suggestions regarding the proofs in this paper. This material is based in part upon work supported by the National Science Foundation under Award No. SES-9210080.

REFERENCES

1. Berger, J. O., Statistical Decision Theory and Bayesian Analysis. Springer-Verlag, New York, 1985.

2. Martz, H. F. & Waller, R. A., Bayesian Reliability Analysis. Wiley, New York, 1982.

3. Mastran, D. V., Incorporating component and system test data into the same assessment: A Bayesian approach. Operations Res. 24 (1976) 491-9.

4. Mastran, D. V. & Singpurwalla, N. D., A Bayesian estimation of the reliability of coherent structures. Operations Res. 26 (1978) 663-72.

5. Martz, H. F., Waller, R. A. & Fickas, E. T., Bayesian reliability analysis of series systems of binomial subsystems and components. Technometrics 30 (1988) 143-54.

6. Martz, H. F. & Waller, R. A., Bayesian reliability analysis of complex series/parallel systems of binomial subsystems and components. Technometrics 32 (1990) 407-16.

7. Mosleh, A. & Bier, V. M., On decomposition and aggregation error in estimation: Some basic principles and examples. Risk Analysis 12 (1992) 203-14.

8. Iwasa, Y., Andreasen, V. & Levin, S., Aggregation in model ecosystems: I. Perfect aggregation. Ecological Modeling 37 (1987) 287-02.

9. Philipson, L. L., Anomalies in Bayesian launch range safety analysis. Submitted to Reliab. Engng System Safety (1993).

10. Idaho National Engineering Laboratory, Development of Transient Initiating Event Frequencies for Use in Probabilistic Risk Assessments. U.S. Nuclear Regula- tory Commission, NUREG/CR-3862, Washington, D.C., 1985.

11. PLG, Inc., Database for Probabilistic Risk Assessment of Light Water Nuclear Power Plants, Volume 6: PWR Initiators, Revision 1. PLG-0500, Newport Beach, California, 1989.

12. Azaiez, M. N. & Bier, V. M., Aggregation error in Bayesian estimation. Submitted to Management Sci., 1993.

13. Chipman, J. S., Optimal aggregation in large-scale econometric models. Sankhya: The Indian J. Statistics: Series C 37 (1975) 121-59.

14. Ijiri, Y., Fundamental queries in aggregation theory. J. Am. Statistical Association 66 (1971) 766-82.

15. Simon, H. A. & Ando, A., Aggregation of variables in dynamic systems. Econometrica 29 (1961) 111-38.

16. Theil, H., Linear Aggregation of Economic Relations. North-Holland, Amsterdam, 1954.

17. Cale, W. G. & Odell, P. L., Behavior of aggregate state variables in ecosystem models. Mathematical Biosci. 49 (1980) 121-37.

18. Cale, W. G., O'Neill, R. V. & Gardner, R. H., Aggregation error in nonlinear ecological models. J. Theoret. Biol. 100 (1983) 539-50.

19. Gardner, R. H., Cale, W. G. & O'Neill, R. V., Robust analysis of aggregation error. Ecology 63 (1982) 1771-9.

20. O'Neill, R. V. & Rust, B., Aggregation error in ecological models. Ecological Modelling 7 (1979) 91-105.

Page 8: On the concept of perfect aggregation in Bayesian estimation

278 V. M. Bier

21. Zeigler, B. P., The aggregation problem. In Systems Analysis and Simulation in Ecology, Vol. 4, ed. B. C. Patten. Academic Press, New York, 1976.

22. Sinha, N. K. & Kuszta, B., Modeling and Identification of Dynamic Systems. Van Nostrand Reinhold, New York, 1983.

23. Savage, L. J., The Foundations of Statistics. Dover, New York, 1972.

24. Cooke, R. M., Experts in Uncertainty: Opinion and Subjective Probability in Science. Oxford University Press, New York, 1991.

25. Genest, C. & Zidek, J., Combining probability distributions: A critique and an annotated bibliography. Statistical Sci. 1 (1986) 114-48.

26, McConway, K. J., The combination of experts' opinions in probability assessment: Some theoretical considera- tions. Doctoral dissertation, University College, Lon- don, England, 1978.

27. Madansky, A., Externally Bayesian groups. Unpubl- ished manuscript, University of Chicago, 1978.

28. Weerahandi, S. & Zidek, J. V., Pooling prior distributions. Institute of Applied Mathematics and Statistics, 78-34, University of British Columbia, 1978.

29. Dalkey, N. C., An Impossibility Theorem for Group Probability Functions. The Rand Corporation, P-4862, Santa Monica, California, 1972.

30. Dalkey, N. C., Group Decision Theory. School of Engineering and Applied Science, UCLA-ENG-7749, University of California, Los Angeles, 1977.

31. Springer, M. D., The Algebra of Random Variables. Wiley, New York, 1979.

32. Ravinder, H. V., Kleinmuntz, D. N. & Dyer, J. S., The reliability of subjective probabilities obtained through decomposition. Management Sci. 34 (1988) 186-99.

33. Rowsome, F. H., How finely should faults be resolved in fault tree analysis?. Trans. Am. Nucl. Soc. 23 (1976) 225 -6.

34. Bier, V. M. & Mosleh, A., The analysis of accident precursors and near misses: Implications for risk assessment and risk management. Reliab. Engng System Safety 27 (1990) 91-101.

35. Azaiez, M. N. & Bier, V. M., Perfect aggregation in Bayesian estimation for several realistic reliability models. In press, App. Math. Computation, 1994.

36. Azaiez, M. N., Perfect Aggregation in Reliability Models" with Bayesian Updating. Dissertation, University of Wisconsin-Madison, 1993.

37. Hazen, G. B., Perfect aggregation. Unpublished manuscript, Northwestern University, 1992.

38. Johnson, N. L. & Kotz, S., Continuous Univariate Distributions. Houghton Mifflin, New York, 1970.

39. Breiman, L., Probability. Addison-Wesley, Reading, Massachusetts, 1968.

A P P E N D I X A

Assume that component 1 fails with probability P,, and that component 2 has conditional failure probability P2 given failure of component 1, where the quantities P1 and/ '2 are independently distributed with distributions fro(') and fro(.). In this appendix, we show that in order to achieve perfect aggregation for the product P~P2, the fpi(.) must be beta conjugate priors with a, = a2 + b2. This is shown by induction on

the moments of the P~. In other words, we show by induction that all moments of the P~ must be of the same form as the moments of beta random variables with a, = a2 + b2. Since the distributions fPs(') are over the interval [0,1], which is compact, they must therefore be beta distributions with a, = a2 + b2.

To start off, let fro(') and fv2(') be arbitrary probability distributions over the interval [0, 1]. Let P~ have first and second moments given by E(P,) and E(P2~), respectively, and assume that Var(Ps)¢0, which also implies E(P~)~ 0 and E(Ps)~ 1. Then we can choose as > 0 and b, > 0 to satisfy the relationships

E(P~) - a, E(p2 )__ a~(a, + 1) (A.1) as + hi' (as + bi)(as + bs + 1 )

Thus, with the exception of the degenerate special cases where Var(Ps) --0 for some i, we have as and bs that could potentially be the parameters of beta distributions.

The next step is to determine whether they obey the relationship a, = a 2 + b 2 . For this purpose, let us assume that we have observed a single trial of the system (i.e. ko = 1), and that component 2 did not fail (i.e. k2 = 0). These observations are consistent with either k, = 1 (component 1 failed, corresponding to a single trial of component 2), or k, = 0 (component 1 succeeded, corresponding to no trials of component 2).

If perfect aggregation is to hold, we should get the same value for E(PiP2[ko, k l , k 2 ) regardless of whether k, is equal to zero or one. We can show that if P~ and P2 are mutually independent prior to observing any data, they will also be conditionally independent given the observed data. Thus, we have

E(P, P2 I ko, k,, k2) = E(P, ] ko, k,)E(P~ l k,, k2) (A.2)

E(P, P2 [ ko = 1, kl = 1, k 2 = O) --

E(e, ) - E ( e 2) E(P~P2[ ko = 1, k, = O, k2 = O) = 1 - E ( P 0 E(P2)

(A.5)

E ( P 2) E(P2) - E ( P 2)

E(P , ) 1 - e ( e 2 )

(A.4)

where

i' E(P~ ] ks ~,, k,) = ~ PifP,(Ps ] ks- ,. ks) dp,

£, k , , 1 fP,(Ps)Ps' ( - Ps) k' ' k, dps - -, ( A . 3 )

Jo fp,(ps)p~'(1 - ps) k' '-k, dp,

Substituting in values for ko, k(, and k2, we obtain

Page 9: On the concept of perfect aggregation in Bayesian estimation

Perfect aggregation in Bayesian estimation 279

Setting eqn (A.4) equal to eqn (A.5) yields

pe E(P~) E(Pe)- E(P 2) E - E ( l ) . . . . (A.6) E(P , ) 1 - E(P ) - E ( P , ) t trz)

Finally, substituting in the expressions for E(P~) and E(PZ~) from eqn (A.1) and solving for al, after some algebra we obtain

al = a: + b2 (A.7)

Thus, with the exception of the degenerate special cases noted above, we have shown that for arbitrary E(P/) and E(p2) , perfect aggregation implies that the kth moment of P, must equal the kth moment of a beta distribution with parameters a~ and b~ for k = 1, 2, where a, = a2 + be; this relationship is also trivially satisfied for k = 0.

Since we have shown that E(P~) satisfies the desired relationship for k = 0, 1, 2 and i = 1, 2, it remains to show by induction that the n + 1st moments of the P~ must also satisfy the desired relationship whenever the nth, n-lst , and n-2nd moments of the P~ do so. While the algebra gets rather complicated, the proof of this assertion follows the same line of argument as in eqns (A.4)-(A.7) above. Let us now assume that we have observed n trials of the system, with kz = n - 2 failures of component 2. These observations are consistent with k, = n, n - 1, or n - 2. If perfect aggregation is to hold, we should get the same value for E(P, P2 [ n, k l , k2) regardless of the value of kl. As before, the posterior distributions for Pl and P2 will be conditionally independent given k,. Thus, eqn (A.2) holds. Substituting in values for n, k, , and k2, we obtain

E(P1P2 In, k, = n - 2 , k e = n - 1)

_ E(P7 -~) - 2E(PT) + E(P? +I) E(P'~ -1)

- E ( P 7 e) _ 2E(PT-l ) + E(P'~) E(P'~ -2) (A.8)

E(PIP~ In, k l -- rl - - 1, k2 = n - 2)

E(P'~) - E(P7 +l) E(P'~ l) _ E(P'~)

E(P'}-') - E(P'~) E(P'~ -e) - E(P~ -l) (A.9)

Note that eqns (A.8) and (A.9) both depend only E p~+,-~ on E(P'~+I), not on ( ~ j. Therefore, by setting

them equal and substituting in the expressions for E(P~-Z), E ( P T - ' ) , and E(P'I) for a beta distribution, we should be able to solve for E(P]'+I). Thus, we wish to solve the equality

E(PT-') - 2E(P]') + E(P'~ +I) E(P'~-')_ E(P7 -2) - 2E(P7 -1) + E(P'~) E(P'~ -2)

E(P'~) - E(P'~ +') E(P~-') - E(P'J) E(P'~ -l) - E(P'~) E(P~ -2) - E(P~ -1) (A.10)

We begin with the expression

E(P~) - r (a , + k )r(a, + b,) r(a,)F(ai + bi + k)

for the kth moment of a beta yields the following relationships:

F(a2 + n - 1)F(a2 + b2)

r(a )r(a2 + + n - 1)

E(P~ -2) r(a2 + n - 2)F(ae + b2)

F(az)F(a2 + b2 + n - 2)

F(ae + n - 1)F(a2 + b2 + n - 2)

(A.11)

distribution. 38 This

a 2 + n - 2

r(a2 + n - 2)F(ae + b2 + n - 1) w

az + be + n - 2

(A.12)

and similarly (after some algebra)

E(P~ -1) - E(P'~) a2 + n - 2 (A.13)

E(P'J 2 ) _ E ( p ~ - , ) a2 + b2 + n - i

Substituting eqns (A.12) and (A.13) into eqn (A.10) yields

E(PT-') - 2E(PT) + E(P7 +') a 2 + n - 2

E(P7 2) _ 2E(PT-l) + E(PT) a2 + b2 + n - 2

E(P'~) - E(P'~ +') a2 + n - 2 (A.14)

-E(P7 -l) - E(PT) ae + be + n - I

Rearranging terms and expanding the denominators in terms of the parameters a, and b, yields

E(P'~ -l) - 2E(P]') + E(P~ +~) a2 + b2 + n - 1

F(al + bl)F(a, + n - 2)b1(b1 + 1) a2 + b2 + n - 2

F(al)F(a, + b, + n)

E(P?) - E(P7 +l) =F(al +b,)F(a, + n - l)b, (A.15)

F(a,)F(al + b, + n)

By cancelling out common terms and substituting in a, = a2 + be, we obtain

[E(P~ '-~) - 2E(P~') + E(P'~+')](a2 + be + n - 1)

= [ E ( P T ) - E(PT+')](bl + 1) (A.16)

and solving this simplified relationship for E(P'~ +~) yields

E(p~+, ) _ F(al + b0F(a l + n + 1) (A.17) F(a0F(a l + bl + n + 1)

However, this is simply the expression for the n + 1st moment of a beta-distributed random variable with parameters al and bl. Thus, with the exception of the degenerate special cases noted above, we have shown by induction that if E(P~) satisfies eqn (A.11) for k = n, n - 1, n - 2 and i = 1, 2, then E(P7 +~) will also satisfy eqn (A.11). The only part of the proof that

Page 10: On the concept of perfect aggregation in Bayesian estimation

280 V. M. Bier

remains is to demonstrate the same relationship for E(P~+~). This follows similarly, by assuming that we have observed n trials of the system and n - 2 failures of component 2, with k, = n or n - 1.

Thus, with the exception of cases in which one of the P~ has a degenerate distribution, we have shown by induction that if E(P~) satisfies eqn (A.I1) for k = n, n - 1, n - 2 and i = 1, 2, then E(P'j +~) will also satisfy eqn (A.11): i.e.

F(ai + n + 1)F(ai + b~) E(PT+') -F(ai )F(ai + bi + n + 1)

(A.18)

where a~ = 02 + b2. Thus, since all of the moments of the P, are of the same form as the moments of beta distributed random variables with a, = a2 + b2, and since the P, are distributed over the compact interval [0, 1], they must therefore be beta-distributed random variables with a, = a2 + b2.

A P P E N D I X B

Assume that components 1 and 2 fail according to independent Poisson processes with rates h, and h2, respectively, where the quantities hi and h 2 a r e

independently distributed with distributions fa,(.) and fA2('). In this appendix, we outline a proof to show that, in order to achieve perfect aggregation for the sum h~ + h2, the f~i(') must be gamma distributions with b~ =b2. This is shown by induction on the moments of the hi; i.e. we show by induction that all moments of the hi must be of the same form as the moments of gamma distributed random variables with b , - - b > (The algebra is omitted, as it is generally similar to that in Appendix A.)

To start off, let f~( . ) and f~2(') be arbitrary probability distributions over the interval [0, 2) . Let hi have first and second moments given by E(A~) and E(A~), respectively, and assume that Var(h~)#0, which also implies E ( h i ) # 0 . Then we can choose a~ > 0 and bi > 0 to satisfy the relationships

ai E(A~) ai(ai + 1) e ( h , ) = = (B.I)

Thus, with the exception of the degenerate special cases where Var(hi) = 0 for some i, we have ai and bi that could potentially be the parameters of gamma distributions.

The next step is to determine whether they obey the relationship b~ = bz. For this purpose, let us assume that we have observed the system for a time t, during which we have observed a single failure. This observation is consistent with either kl = 1 and k2 = 0 (component 1 failed), or k, = 0 and k2 = 1 (component 2 failed).

If perfect aggregation is to hold, we should get the same value for E(hl + h2 ] k,, k2 = 1 - k,, t) regardless of whether k, is equal to zero or one. Taking the limit as t goes to 0, where

E(h? + ') limo E(A, I ki, t) - E(hf,) (B.2)

yields

E(a2 E(a ) E(A~) + - - - - + E(A2) (B.3)

E(A2) E(A,)

Substituting in the expressions for E(Ai) and E(Ai 2) from eqn (B.1) and solving for b~, after some algebra we obtain b t = b2. Thus, with the exception of the degenerate special cases noted above, we have shown that for arbitrary E(Ai) and E(A2), perfect aggregation implies that the kth moment of Ai must equal the kth moment of a gamma distribution with parameters a, and bi for k = 1, 2, where b, = b2; this relationship is also trivially satisfied for k = 0.

It remains to show by induction that the n + 1st moments of the Ai must also satisfy the desired relationship whenever the first n moments of the hi do so. The proof of this assertion follows the same line of argument as above, but with larger values for the sum kl +k2. Let us assume that we have observed the system for a time t, during which we have observed a total of n failures. This observation is obviously consistent with a wide range of values for k~ and k2. However, if perfect aggregation is to hold, we should get the same value for E(AI + A2 ] kl, k2 = n - kl, t) regardless of the values of k~ and k2. In particular, this implies that

E(h, L k, = n, t) + E(h2 L = 0, t)

= E(/~ 1 I k, = n - I, t) + E(A2 I k2 = I, t) (B.4)

Taking the limit as t goes to 0 and applying eqn (B.2), we obtain the relationship

E(h'; ~ ) E(A~') E(h~) F E(A2) + - (B.5)

E(A]') E(A','-') E(h2)

Substituting in the expressions for the first n moments of gamma distributions 38 with parameters ai and bi

E(hf) - F(ai + k) (B.6) r(a,)b

and solving for E(A~ '+') yields

E ( h ] , + , ) = a l + n F ( a , + n ) F ( a t + n + l ) (B.71 b, F(a,)b7 F(a,)b7 +'

However, this is simply the expression for the n + 1st moment of a gamma-distributed random variable with parameters a~ and b L. Thus, with the exception of cases in which one of the hi has a degenerate distribution

Page 11: On the concept of perfect aggregation in Bayesian estimation

Perfect aggregation in Bayesian estimation 281

with zero variance, we have shown by induction that if E(A k) satisfies eqn (B.6) for k =0, 1 . . . . . n and i = 1, 2, then E(A~ '+') will also satisfy eqn (B.6). The same relationship for E(A~ +') follows straightfor- wardly from the equality

E(A, I kt = 0, t) + E(A2 [ k2 =n , t)

= E ( A , ] k , = l , t ) + E ( A 2 J k z = n - l , t ) (B.8)

Thus, wc have shown that all moments of the Ai must be equal to the moments of gamma distributions with parameters ai and bi with b, =b2. From Breiman, 3~ we have that the moments uniquely determine the distribution if they do not grow too fast;

in other words, if

_ _ IE(Ak)[ l/k lim < ~ (B.9)

k k In this case, we have

li---~ IE(Ak)I , / _ _ . . k _ lim "J=" k k k bik

< lira [(a~ + k)~l ''~ + k 1 - l i m a~ _ ( B . I O )

k bik k bik bi Therefore, the gamma distribution is the only non-degenerate distribution whose moments satisfy the desired relationships.