using expert advice on a skew judgmental distribution

Using Expert Advice on a Skew Judgmental DistributionAuthor(s): Dennis V. LindleySource: Operations Research, Vol. 35, No. 5 (Sep. - Oct., 1987), pp. 716-721Published by: INFORMSStable URL: http://www.jstor.org/stable/171222 .

Accessed: 08/05/2014 23:59

Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at .http://www.jstor.org/page/info/about/policies/terms.jsp

.JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range ofcontent in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new formsof scholarship. For more information about JSTOR, please contact [email protected].

.

INFORMS is collaborating with JSTOR to digitize, preserve and extend access to Operations Research.

http://www.jstor.org

This content downloaded from 169.229.32.137 on Thu, 8 May 2014 23:59:26 PMAll use subject to JSTOR Terms and Conditions

http://www.jstor.org/action/showPublisher?publisherCode=informs

http://www.jstor.org/stable/171222?origin=JSTOR-pdf

http://www.jstor.org/page/info/about/policies/terms.jsp


USING EXPERT ADVICE ON A SKEW JUDGMENTAL DISTRIBUTION

DENNIS V. LINDLEY Decision Science Consortium, Inc., Falls Church, Virginia

(Received July 1985; accepted December 1986)

A decision maker is interested in a quantity 0. He consults an expert who provides three fractiles of her probability distribution for 0. The problem discussed in this paper is how the decision maker should use the expert's fractiles to produce his distribution for 0. A key ingredient has to be the decision maker's opinion of the expert. Our analysis shows how the Bayesian approach clearly incorporates this feature. With three fractiles provided, information about the skewness of the distribution of 0 is available, in addition to location and scale. Our development uses the skew logistic to accommodate this additional feature, and much of the paper is concerned with skewness, extending the methods of Lindley for the symmetric case. The ideas easily extend to several experts if the unrealistic assumption is made that the decision maker regards them as independent.

1. The Data

A decision maker is interested in a quantity 0 ~about which he is uncertain. The reader may like to think of himself as the decision maker: we will notationally refer to him as DM. As an example of 0, consider the amount of radiation, measured in man- rems, resulting from a core melt in a nuclear power station. Being uncertain about 0, DM consults an expert and obtains an assessment of the value of 0. The expert will be referred to as a subject and denoted S. Typically, S will also be uncertain about 0 and, accordingly, ought to express her knowledge of 0 in the form of a probability distribution for 0. Often, she does not feel able to do this. In this paper, we suppose that S provides three fractiles of her distribution for 0, rather than the complete distribution. Denoting S's probabilities by ps, we suppose that, for a given, small probability a, S provides three fractiles, 1, m and u, defined by p5(0 < 1) = a, p5(O < m) = 1/2 and Ps(6 < u) = 1 - a. Thus, m is the median and / and u are, respectively, the lower and upper 100a% points for p,(0). In applications it is often assumed that a is 0.05. Often, it is not clear what value of a is being used, merely that / and u represent limits on 0 unlikely to be exceeded. In this case, it would be useful to carry out the analysis that follows for various small values of a. Pratt and Schlaifer (1985) give data with a = 1/4 and also with unspecified "astonish- ing" values.

We will suppose the distribution of 0 to be continuous, or discrete on very many values, so that

continuity is a reasonable approximation, as with the number 0 of casualties that might occur as a result of a core melt. For some cases the distribution is a mixture of discrete and continuous components, as with the number of casualties near a nuclear plant in a year. If the plant functions well, 0 = 0: if there is a serious accident, 0 is effectively continuous. In this case, we suggest assessing the two components sepa- rately. The discrete component can be discussed using the methods of Lindley (1982).

2. The Basic Principle

The subject S provides the triplet (1, m, u) and DM needs to calculate his probability distribution for 0 given S's opinion, namely p(0 I 1, m, u). Here p(.) has been used, as it will be throughout the paper, to denote DM's probability, omitting a suffix DM. Additional knowledge, beyond (1, m, u), possessed by DM is omitted from the notation since it occurs in all conditions and its explicit introduction would overcom- plicate the notation. By Bayes' theorem,

POI 1, m u) X p(l, m, u I O)p(0). (1)

There, p(O) is DM's opinion before consulting S and p(l, m, u 0 0) is DM's judgment of what S would be likely to say were the quantity to have the value 0. The latter expresses DM's judgment of how good an expert S is. For example, if p(l, m, u I 0) is high for small u - l and m near 0, then DM thinks S will likely give a distribution with small spread, u - 1, about a value m close to 0: DM has a high opinion of S. On

Subject classification: 91 combining opinions, 561 skew logistic distributions.

Operations Research 0030-364X/87/3505-0716 $01.25 Vol. 35, No. 5, September-October 1987 716 ? 1987 Operations Research Society of America



Using Expert Advice on a Skew Judgmental Distribution / 717

the contrary, if p(l, m, u I0) is still high for small u - 1, but nearly constant over a large range of values of m, then DM thinks that S will make a confident judgment (small u - 1) that may well be wide of the mark, having m far from 0. If the distribution peaks at a value of m far from 0, then DM thinks S is biased. It is a basic principle of the Bayesian argument, applied to expert opinion, that DM's change of view from p(0) to p(O I 1, m, u) should be achieved by multiplication of p(0) by the likelihood p(l, m, u I 0) incorporating DM's opinion of how good S is (1). Winkler (1986) and five other papers in the same issue of Management Science, collectively, give an extensive discussion. Genest and Zidek (1986) provide a bibliography and discussion. The argument has the merit of explicitly introducing DM's judgment of S's expertise, clearly an important factor, in an interpretable way. The basic problem is, therefore, DM's assessment p(l, m, u I 0), and it is to this issue that we now turn.

3. Skewness

The first difficulty is that p(l, m, u I0) is a trivariate distribution, for it is always hard to think about three quantities, especially when, as here, the quantities are ordered, 1 < m < u. An additional assumption enables it to be replaced by a family of univariate distributions, thus simplifying the assessment. In a previous paper, Lindley (1983) discussed the case in which S provided a measure of location, such as the median m, and one of spread, such as the range r = u - 1. This analysis made the assumption that r on its own provided no information about 0. (The precise meaning of this statement will be given below.) In the case discussed in this paper, S has, in addition to location m and spread r, provided a third measure. It is convenient to think of this as a measure of skewness. Indeed, in the earlier work, there was an implication that because only m and r had been provided, S's underlying distribution had no skewness: for example, normal or Student's t. In the example of radiation from a nuclear plant, everyone has a highly skew distribution with a long tail to the right and a short, or even nonexistent, one to the left. Such a distribution is said to be positively skew. It is this strongly felt skewness that persuades S to give, and DM to accept, three measures. A possible measure of skewness is q = (u - m)/ (m - 1). This measure is unaffected by changes of origin or scale, is one for a symmetric distribution, and exceeds (is less than) one for a positively (nega- tively) skew distribution.

It is therefore convenient to replace (1, m, u) by

(m, r, q) representing location, spread and skewness, respectively, and then, in extension of the assumption made earlier, to assume that r and q on their own provide no information about 0. In a manner to be shown in Section 6, this means that all DM has to assess is p(m I r, q, 0), the conditional distributions of the location measure, given those of spread and skewness, for each 0. This is much easier than assessing a trivariate distribution. However, it is clear that m, given r, q and 0, will typically have a skew distribution and that we therefore need a skew family to replace the suggested normals or t's of the symmetric case. Some other considerations enter into this assessment. S has already implied a skew distribution for her judgment of 0, so that two skew families are involved; one for DM, one for S. Also, we would like our analysis to allow DM to agree with S whenever DM respects the expert's opinion. This means we would like the fractiles of p(O I 1, m, u) to be related to 1, m and u. This objective can be achieved by relating the skew families of DM and S. The next section intro- duces two families that enable us to do this. We are then able to make the assumption about no information precise, and complete the calculations according to Bayes' theorem (1). We shall actually use different measures of location, spread and skewness from those described in this section, which merely outlines the principles.

4. The Skew Logistic Family

With the easy availability of computers, most families of distributions can be handled, but there is some merit still in simplicity as an aid to human apprecia- tion. This is particularly true in the present problem, in which both S and DM are making probabilistic judgments about real quantities like radiation dosages, and need to appreciate the statements they are making. There is therefore much merit in the logistic distribution, with its simple density and distribution functions. It has been neglected in comparison with the normal, which it closely resembles, because of its lack of sufficient statistics, but this feature is largely irrelevant in the present analysis where there are no random samples, and conjugate distributions are not needed.

A quantity X is (standard) logistic if

p(X < x) = (I + e-X)-l -oo < x < Co.

The (standard) skew logistic is obtained by raising this value to an arbitrary, positive power K, so that

p(X<x) = (1 + e-x)-K _oo <x< o. (2)



718 / LINDLEY

For probability a, the fractile I satisfies

a =(1 + e-1)-K

or

I = -log(a1" - 1). (3)

The density for the skew logistic is

Ke-x/(l + e-x)K+l, (4)

and graphs of this density are given in Figure 1 for various values of K. As will be seen, the family includes only a limited amount of skewness, positive for K > 1,

negative for K < 1, and symmetric at K = 1. Table I shows the value of the skewness measure (u -m) (m - 1) for various values of K when a = 0.05. In the Appendix, we show that (u - m)/(m - 1) tends to finite limits as K tends to 0 or infinity and always lies between them. For a = 0.05, these limits are 0.279 and 1.779, respectively.

To every skew logistic of a quantity X there is another distribution, reflected skew logistic, for Y= -X. This has distribution function

p(Y < x) p(X > -x) = 1 - p(X < -x)

= 1 - (1 + ex)-K (5)

and density

Ke-KX(6

(+ e-X)K+l (6)

These are simply reflections of the original skew logistic for X in the line X = 0.

It is immediately apparent from Figure 1 that changes in K alter not only the skewness, but also the

location and scale of the (standard) skew logistic. The mean and standard deviation are rather complicated, but (3) shows that the median is -log(211K- 1). The mode is simply log K (as easily follows from equating the derivative of the logarithm of the density (4) to zero). A convenient measure of scale is the reciprocal of the square root of minus the second derivative of this logarithm at the mode. Easy calculation shows this to be (1 + K-)'12.

Finally, the (general) skew logistic is obtained by effecting general shifts of location by X and scale by a to obtain from (4) the density

Ke-(x-,>/., (7)

oci + e-(xx)/1a]K+l (7

There is a similar (general) reflected, skew logistic from (6). Notice that, in view of the remarks in the last paragraph, X and a are not the mean and standard deviation of Xin (7). A reasonable measure of location is X + log K, and of scale, o(1 + K'-)'/2.

The usefulness of the skew logistics is increased by noting that other families can easily be transformed into them. For example, it sometimes happens when dealing with a nonnegative quantity 0 that its distribution is so skew that the mode is at the origin: this can happen in the nuclear situation mentioned above. The density K(l + 9)-(K+1) for 0 > 0 has this property and can be transformed to the skew logistic by x = -logo.

It should be emphasized that the methods described in the rest of the paper would work for any family of distributions dependent on a skewness parameter, and then generalized by location and scale shifts. The only merit of the logistic is the simplicity of the resulting expressions and resulting ease of understanding by DM and S.

5. Subject's Distribution

We now return to the original problem, in which DM is to use the fractiles (1, m, u) for 0 provided by S. Underlying S's statement is an implied distribution for 0. We propose fitting this distribution by a member of the skew logistic family (7). This family has three parameters X, a and K, and these can, subject to a condition later discussed, be determined from the fractiles provided. From (3) and two similar equations with a replaced by 1/2 and by 1 - a, the results are

(I - X)/cr = -log(ao -1 ) = 1K, say, {

(m - X)/u = -log(2'- lmK _I (8) and (u- X)/o = -log[(l - a)-1/K- 1] = UK. J

DENSITY .35

.30 -

.25-

* 20

.15

. 10-

.05- =

-10 -8 -6 -4 - 4 8 10

x

Figure 1. Densities of skew logistic distributions for various values of the parameter K.




Table I Skewness (u - m)(m- 1) as a Function of K for

the Skew Logistic with a = 0.05 K (U- m)/(m- l)

0.0 0.279 0.1 0.318 0.3 0.499 0.5 0.680 0.7 0.830 0.9 0.950 1.0 1.000 1.5 1.186 2.0 1.302 3.0 1.438 4.0 1.514 6.0 1.596 00 1.779

The density is Kex/(l + e-x)"+I; 1, u are the lower and upper 5% points, and m the median.

Clearly (u - m)(m -1), the measure of skewness discussed above, equals (UK - mj/(m, - 1K), a function of K. Table I gives numerical values of the latter for a = 0.05. Having found K, we can easily find the values of X and r from (8). Denote the values of (X, a, K) arising from this fitting of the skew logistic by (t, s, k), so that

Ps (0) = ke-(O-t)ls s[1 + e-(o-)/s]k+l

It is these measures (t, s, k) that will be used as location, spread, and skewness in the manner of Section 3. What this procedure has accomplished is the substi- tution of (t, s, k) for (1, m, u). In other words, S may be thought of as having provided (t, s, k), rather than (1, m, u). In any practical application of this procedure, it would be sensible to present S with the skew logistic with parameters (t, s, k) to see whether it reasonably reflected her opinion. If not, she may wish to change one or more of (1, m, u), though she could dismiss the skew logistic family completely. The methods here suggested would then fail.

The above method can break down. As noted in Section 4, (UK - mK)/(mK - 1K) is bounded above and below. If S's value (u - m)/(m - 1) lies outside this finite range, k cannot be found. A way out of this difficulty is to apply a transform to 0 before doing the fitting by a skew logistic. For example, a power of 0 or a logarithm may be substituted. The difficulty will arise only with extreme skewness, so the object of the transform would be to reduce the amount of skewness.

It is easy to show that the transformation log(0 -t), with t = (lu - m2)/(u + I - 2m), will produce upper

and lower fractiles that are symmetric about the median. This idea may be useful with a single subject, but is unavailable with several subjects (Section 8) who cannot all be reduced to symmetry simultane- ously.

In the subsequent analysis, we will suppose that, if necessary, the transformation has been made, although, to avoid cumbersome notation, we shall con- tinue to refer to 0 and its fractiles 1, m and u.

6. The Likelihood Function

We saw in (1) that the key element in DM's use of S's opinion was the likelihood p(l, m, u I 0). This may now be replaced by p(t, s, k I 0), from which it differs only by the Jacobian of the transformation from (1, m, u) to (t, s, k) which does not involve 0. From this the marginal distribution of (s, k) may be obtained and the assumption mentioned above can be made precise.

Assumption. p(s, k I 0) does not depend on 0.

In words, s and k, on their own provide no information about 0: or the measures of scale and skewness tell one nothing about the quantity. This is not exactly true, for there are circumstances in which a large mean is associated with a large spread. More realistically, s and k provide little information about 0. In the symmetric case, Lindley (1983) showed that this small amount had only a weak effect on the total knowledge of 0.

The assumption does not imply that the other measures of scale, r = u - 1, and skewness, q = (u - m)/ (m - 1), do not depend on 0, but the dependency will be small and they will provide little information about 0.

The assumption enables the basic result (1) to be written

p(OI 1, m, U) = p(O l t, s, k)

cc p(t, s, k I 0)p(0)

-p(t Is, k, O)p(s5 k I O)p(O)

c p(tI s, k, 0)p(0), (9)

so that only the conditional distribution of location t, given s, k, and 0 need be considered. Notice that the assumption does not mean that (s, k) can be discarded. Although, on their own, they provide no information about 0, used in conjunction with t in the conditional distribution they do provide information. In statistical language, (s, k) is ancillary.



720 / LINDLEY

7. Decision Maker's Distribution

DM now needs to assess univariate distributions of t for values of (s, k, 0). We now assume that these distributions are reflected skew logistic with parameters (X, a, K) that depend on (s, k, 0) in a way described below. The reason for using the reflected form is that S's original judgment can be expressed as (0 - t)/s = XK, where X, is a (standard) skew logistic. This expression may simply be rewritten as (t - 0)/s = -XK = YK,

where YK has the reflected form. And this translates back into t having the reflected form for fixed (s, k, 0). It is the familiar pivotal argument first employed by Fisher in fiducial probability.

It is, therefore, assumed that, using (6),

-K(te-)/1a p(t I s, k, 0) = a[l + e-(v-X)/l]K+l (10)

where X, o, and K depend on s, k, and 0. Consider first DM's skewness, K. We suppose this is related to S's skewness, k, by the relation K = 3k, where a is a value assessed by DM. For example, suppose DM feels that S is concerned with the serious consequences of large O and, as a result, increases u unduly, or equivalently diminishes a. Then this will cause S to increase her skewness, so that DM may wish to deflate k and select 3 < 1. In this example, it might be felt that DM should express this view directly in terms of u, not through k, but u is affected by 0 whereas k, by assumption, is not, and the assessment appears to be difficult. If DM trusted S's assessment of skewness, he would take

3 = 1. We next pass to DM's scale a. Again, we suppose

this is related to S's scale s by a = 'ys where -y is also assessed by DM. This assumption was made in the symmetric case also. If DM thinks S tends to be overconfident in her assessment of 0, DM will work to inflate S's stated scale and choose -y > 1. The roles of -y and 3 are similar. Notice that neither involves 0, just as the earlier assumption supposed s and k were free of 0.

It would be possible for DM to ignore S's s and k and insert his own values. We feel that the above approach, in which DM learns about appropriate levels of dispersion and skewness, but not about 0, from (s, k) provided by S is realistic.

Finally, we pass to DM's location X, which clearly will depend on 0. Again, a multiplicative constant can be introduced, as with a and K, but there is also the serious possibility of a bias of fixed amount. It is therefore realistic to suppose X = a + p0, where a and 3 are assessed by DM. The values a = 0, A3 1 correspond to no bias, either absolute or proportional.

Together with a = y = 1, they would lead DM to accept S's distribution and, hence, S's fractiles.

There is another point to be borne in mind. As was seen in Section 4, the (standard) skew logistic had mode at log K, and therefore the (general) reflected skew logistic has mode at X- log K. Suppose that DM's opinion is that, if the true value is 0, S will provide a mode, rather than a mean or median, at 0; then t + log k will be about 0, or t about 0 - log k. However, the mode of t will be about 0- log K, since we have a reflected distribution. Consequently, it follows that if K and k are unequal, DM may need to make some adjustment. Using K = 3k, the adjustment is simply log 6. There is a similar, possible correction for the scale. We anticipate that, in practice, these corrections will be small and they will therefore be ignored in the subsequent analysis. They could be appreciable only if DM and S were in substantial disagreement over the amount of skewness.

In summary, we suppose that DM judges t to be reflected skew logistic, (10), with

X = a + #0, a = -ys, and K= ek. (11)

This choice gives DM four values (a, [, -y, 3) with which to describe his judgment of S's expertise. It remains only to insert (10), (11) into (9) to find DM's judgment

P(O M, U) 0C ke-bkQt-oc-fl0V-s Po (2 Iys [1 + e-(t`- 60)/1sIbk+l p(0) (12)

If p(0) is uniform, or approximately so, the integral over 0 is straightforward; otherwise numerical integra- tion is necessary. With p(0) uniform and a = 0, [ = y = a = 1, DM and S will be in complete agreement.

8. Several Subjects

In many practical problems facing DM, including the nuclear example in Section 1, there would be several experts, or subjects, S., S2, . . , Sm, giving DM advice. It would therefore be desirable to extend the previous discussion from 1 to m subjects. It is easy to do this if the subjects are judged by DM to be, given 0, independent. If this is so, the likelihood contribution from each is reflected skew logistic with (ao, fli, yi, pi) (i = 1, 2, ..., m) for Si, who provided data (Ii, mi, ui) and, by the independence, these may be multiplied together to provide the overall likelihood from m subjects. One caveat is necessary: we saw in Section 4 that the skew logistic can incorporate only a limited amount of skewness, and to avoid this difficulty, we supposed, in Section 5, that a transformation of 0 had




been effected to reduce the skewness to the levels tolerated by the logistic. With several subjects, it will be necessary to suppose that there is a single transformation which, when applied to all subjects, gives each subject a value of (u - m)/(m- 1) with the region of the logistic. If the subjects differ violently in their assessments of skewness, this may not be possible and the methods suggested here fail.

If DM does not judge the subjects to be independent, then the above methods fail and it is necessary to assess m-variate distributions p(t I s, k, 0), where t = (t1, t2, ... . t.), and so forth. If all the skewnesses are equal, k1 = k2= -. = km, then a multivariate skew logistic is a possibility just as the multivariate normal may be appropriate in the symmetric case. The assumption of equal skewness is rather severe. The discussion in the preceding paragraph needed the experts only not to be violently different. There is another difficulty with multivariate distributions, including the normal, which is that it is very hard to think about those parameters which numerically described the dependencies-in the normal case, the correlations. As we saw in Section 7, even the location, scale and skewness present difficulties: the dependency parameters are much worse.

A possible way out of the difficulty is for DM to try to identify the reasons for the dependencies and to model in terms of these. For example, S. and S2 may be dependent because they are both using the same set of data, although their other experiences are essen- tially unique to themselves. Then DM can think about the data set and the two sets of experiences as a trio of independent results. The author is currently writing a paper that explores this idea in the symmetric, normal case where it appears to work well. It remains to be seen if it is capable of handling skewness.

Appendix: The Range of Skewness with the Skew Logistic

For the density Ke-x/(l + e-X)K+1 and fixed a,

u-m -log{(1 - a)1/K _ I + logt(1/2)l/K-

m I logt(1/2)l/K - 1 + logta - 1}

The numerator of this expression is the logarithm of t(1/2)V - I /{(1 - a)v 1}, with v =-K-l. The limit of this ratio as K 00, V -O 0 can be found using L'Hopital's rule. Differentiating numerator and denominator with respect to v gives the ratio I(Q/2)vlog 1/21/{(1 - a)vlog(1 - a)) with obvious limit log ?/2/log( 1- a). Applying the same technique to the denominator, we easily have

lim u -m logtlog(1 - a)/log 1/4 K-? m - I log(log ?/2/log a)

As K -O V 0 , v (1 -)- l/K is large in comparison with -1 and the latter can be ignored. The result is then obviously

iu-m log2(1- a) lim -- K->om- I log 2a

Since (u - m)/(m - 1) clearly increases with K, it is bounded by the limits at K = 0 and oo. For a = 0.05,

0.279 < (u - m)/(m - 1) < 1.779.

For small a, they are wider.

References

GENEST, C., AND J. V. ZIDEK. 1986. Combining Proba- bility Distributions: A Critique and An Annotated Bibliography (with discussion). Stat. Sci. 1, 114- 148.

LINDLEY, D. V. 1982. The Improvement of Probability Judgements. J. Roy. Stat. Soc. A 145, 117-126.

LINDLEY, D. V. 1983. Reconciliation of Probability Dis- tributions. Opns. Res. 31, 866-880.

PRATT, J. W., AND R. SCHLAIFER. 1985. Repetitive Assessment of Judgmental Probability Distribu- tions: A Case Study (with discussion). In Bayesian Statistics, 2, pp. 393-423, J. M. Bernardo et al. (eds.). North Holland, Amsterdam.

WINKLER, R. L. 1986. Expert Resolution. Mgmt. Sci. 32, 298-303. (The topic is continued in other contribu- tions on pp. 303-328.)



using expert advice on a skew judgmental distribution

Documents