Artificial Intelligence 59 (1993) 43-47 43 Elsevier

ARTINT 996

A method for managing evidential reasoning in a hierarchical hypothesis space: a retrospective

J e a n G o r d o n and E d w a r d H. Shor t l i f fe

Section on Medical Informatics, Departments of Medicine and of Computer Science, MSOB X-215, Stanford University School of Medicine, Stanford, CA 94305-5479, USA

A central issue in the design of expert systems is the representation and manipulation of uncertain and incomplete knowledge. "A method for managing evidential reasoning in a hierarchical hypothesis space" [4] was an attempt to apply a rigorous mathematical theory of evidence, the Dempster- Shafer (DS) theory, to this problem. In the 1970s, the application of artificial intelligence (AI) to the field of medicine had necessitated the development of ad hoc techniques for the management of uncertainty. Classical probability theory had been rejected due to difficulties in the assessment of conditional probabilities, to the complexity of calculations involved, and to the perceived need to assume conditional independence. Various ad hoc models were thus developed to handle uncertainty--for example, MYCIN's CF model [ 18 ], Internist's evoking-strength/frequency-weight model [ 12 ] and Casnet's causal-weighting model [20]. The ad hoc basis of these models, their lack of generality, and their inability to deal with the observed tendency of experts to reason about abstract entities before focusing on single hypotheses led us to explore the DS theory.

The appeal of this theory lay in its mathematical rigor and in its ability to model the narrowing of the hypothesis set with the gathering of evidence, a process characterizing expert reasoning. Since an expert uses evidence relevant to sets of hypotheses as well as to single hypotheses, the ability of

Correspondence to: E.H. Shortliffe, Section on Medical Informatics, Departments of Medicine and of Computer Science, MSOB X-215, Stanford University School of Medicine, Stanford, CA 94305-5479, USA. Telephone: (415) 723-6979. Fax: (415) 725-7944. E-mail: ehs@camis.stanford.edu.

0004-3702/93/$ 06.00 Q 1993 - - Elsevier Science Publishers B.V. All rights reserved

44 J. Gordon, E.H. Shortliffe

the DS theory to represent hierarchical relationships was very attractive. We felt it important to explicate and motivate this theory for the AI research community.

We had three major goals. We wished to present the DS theory in a simple and concise manner, avoiding the excessive mathematical notation which had perhaps deterred many from an appreciation of the theory. We then demonstrated its relevance to a familiar expert system, MYCIN. Finally, we derived a computationally efficient algorithm for utilizing the DS theory in a hierarchical hypothesis space.

We feel the paper made two very different intellectual contributions. The first was its use of formal combinatorial analysis to derive the algorithm. The second was to bring to the attention of the AI community a mathemat- ical model which captured the essence of the medical and general expert evidence-gathering process. Thus, we attempted to reach a broad and not necessarily mathematically sophisticated audience. Much of our paper fo- cused on detailed examples drawn from medical reasoning in an effort to motivate the DS theory.

Our paper spawned an almost immediate sequel by Shafer and Logan [ 16 ]. They proposed an algorithm which, like ours, was computationally efficient but which did not require the approximation employed in our method. Shafer and Logan found our approximation to be close in most cases but gave examples of non-intuitive results in others. Their algorithm also applied to slightly more general types of evidence--bearing on hypotheses outside the hierarchical hypothesis space--and computed degrees of belief for these hypotheses.

The DS approach was subsequently challenged in a research note by Pearl [13]. He argued that evidential reasoning in a hierarchical space could be conducted using a Bayesian approach. His key assumption was to identify belief in a hypothesis with the probability that the hypothesis was true given all previous evidence. To calculate the impact of new evidence on the belief of every hypothesis in the hierarchy, he proposed the following. Given a piece of evidence, e, bearing on a set of hypotheses, S, assign to each hypothesis in S the likelihood ratio of S, an estimation of the degree to which the evidence confirms or disconfirms S. The ratio is that of the conditional probabilities, P (e IS)/P (etnotS). Belief in each single hypothesis of S is updated by multiplying the original belief by this likelihood ratio and by a normalizing factor. A set of hypotheses, i.e., an intermediate-level hypothesis, is assigned the sum of the beliefs of its individual hypotheses. This process can be applied recursively, where the updated beliefs serve as prior beliefs for new evidence. Pearl also proposed an alternative process avoiding normalization and propagating beliefs up and down the hypothesis hierarchy tree to neighboring nodes.

Fundamental to Pearl's approach was the following probabilistic interpre-

Managing evidential reasoning in a hierarchical hypothesis space 45

tation of the statement whose essence we tried to capture (namely, "e bears directly on S but says nothing about the individual hypotheses in S"): the conditional probability of the evidence given S is independent of the iden- tity of a single element in S, i.e., P(elS, h) = P(eLS) for every hypothesis h in S. Thus, evidence bearing on a set contributes no information about the relative likelihood of individual hypotheses in the set. This assumption, together with Bayes' rule, allows the computation of belief in every subset, T, of the hierarchy given e, i.e., P(T le ) .

The appeal of this probabilistic interpretation lay in its clearly stated assumptions, distinction between partial confirmation and disconfirmation, and easily understood meaning and definition of the likelihood ratio. Experts were required to assess only one number, this ratio, which could possibly be taken from actual data. Experts also appeared to be more comfortable assessing the probability that a finding is present in a given disease, P(eIS) , versus the probability that a disease is present given a particular finding, P(Sle ) . Finally, a Bayesian approach easily adapted itself to methods for converting beliefs to decisions about cost-benefit and utility questions.

Recent and current research in our group at Stanford reflects Pearl's point of view in a return to the formal axioms of probability and decision theory in the representation and manipulation of uncertainty. Most successful has been the theory of probabilistic belief networks, developed by Howard and Matheson [7] and Pearl [14], which makes unnecessary the restriction of conditional independence among variables. A belief network is a graphi- cal knowledge representation of probabilistic relationships which facilitates communication between the expert and the model. Formally, it is a directed, acyclic graph whose nodes represent variables such as diseases and features of diseases and whose arcs reflect conditional dependencies. Probabilities are attached to nodes: each node without predecessors is assigned an uncon- ditional probability distribution and each node with predecessors is assigned a conditional probability distribution for each instance (possible value) of the conditioning nodes.

Heckerman and others utilized belief networks in Pathfinder, an expert system in the domain of lymph node pathology [6]. Instrumental to the success of this approach has been the similarity network, developed as a tool to elicit subjective probabilities from experts. This representation was developed to overcome difficulties they experienced due to conditional dependencies among some features of diseases. Heckerman proved formally that similarity networks enable the construction of large belief networks from subproblems comparing two diseases and their distinguishing features [5]. Another key contribution was that of partitions. This representation is a generalization of Pearl's Bayesian method for representing evidence relevant to sets of hypotheses, a major asset of DS theory.

Another application of belief networks has been made in Nestor, a hyper-

46 .1. Gordon, E.H. Shortliffe

calcemia expert system developed by Cooper [2]. The system utilizes belief networks to represent the pathophysiology of diseases causing hypercalcemia. Causal rules were acquired from medical texts and augmented with the sub- jective probabilities of an expert. Nestor has been mostly a research, rather than a clinical, tool as opposed to Pathfinder, which has undergone exten- sive clinical evaluation. Other medical expert systems utilizing probabilistic and decision-theoretic inference include the Glasgow Dyspepsia system for gastroenterology [ 19 ], the Neurex system for neurologic diagnosis [ 15 ], and the MUNIN system for diagnosis of muscular problems [1].

Several other researchers associated with our group have explored other aspects of uncertainty management for medical decision support [17]. For example, Lehmann's program, Thomas, helps physicians determine the clin- ical significance of a study from the clinical trials literature [11]. It incor- porates the user's prior beliefs and methodological concerns in generating a statistical model with updated probability distributions on the likely out- comes for the competing interventions. Thus, Thomas attempts a normative representation of the clinical trials literature problem [10].

Klein has developed a method, "interpretative value analysis", for repre- senting hierarchically organized value models for realms involving frequent tradeoffs between objectives [8]. This method has been implemented in Virtus, a modeling tool based on formal decision theory. In Virtus, multi- attribute models enable a dialogue with the user wishing to understand the basis of the program's advice or to adapt the value model to his own preference structures. Virtus has been used to develop RCTE (Randomized Clinical Trials Evaluator), a decision tool which assesses the strengths and weaknesses of a clinical trial and provides a net evaluation of its credi- bility [9]. Klein is currently exploring extensions of his methodology to multi-attribute decision making under uncertainty.

In conclusion, it appears that our paper spawned renewed interest in de- veloping formal normative models of reasoning with uncertainty. Clearly, an axiomatic approach is appealing in its explicit formulation of the assump- tions being made in a particular system or in the system's computational methods. We feel that the exposition of the DS theory was an important first step in this return to formal models based on probability and decision theory. For as Dempster writes: "[Such] 'beliefs' are intended for inter- pretation as subjective probabilities and the formal manipulations of the subjective theory are embedded in the theory of belief functions" [3].

References

[1] S. Andreassen, M. Woldbye, B. Falck and S. Andersen, MUNIN: a causal probabilistic network for interpretation for electromyographic findings, in: Proceedings HCAI-87, Milan, Italy (1987) 366-372.

Managing evidential reasoning in a hierarchical hypothesis space 47

[2] G.F. Cooper, NESTOR: a computer-based medical diagnostic aid that integrates causal and probabilistic knowledge, Ph.D. Thesis, Medical Information Sciences, Stanford University, Stanford, CA (1986).

[3] A. Dempster, Commentary on papers by Lane and Cooper, Appl. Stoch. Models Data Anal. 5 (1989) 77-81.

[4] J. Gordon and E.H. Shortliffe, A method for managing evidential reasoning in a hierarchical hypothesis space, Artif. Intell. 26 (1985) 323-357.

[5] D. Heckerman, Probabilistic similarity networks, Networks 20 (1990) 607-636. [6] D. Heckerman, E. Horvitz and B. Nathwani, Toward normative expert systems, Part I:

the Pathfinder project, Methods Inf. Med. 31 (1992) 90-105. [7] R.A. Howard and J.E. Matheson, Influence diagrams, in: R.A. Howard and J.E. Matheson,

eds., Readings on the Principles and Applications of Decision Analysis, Vol. II (Strategic Decision Group, Menlo Park, CA, 1981 ) 721-762,

[8] D. Klein, Interpretive value analysis, IBM Tech. Rept. RC 15278 (#68173), Ph.D. Dissertation, Computer and Information Sciences, University of Pennsylvania, Philadelphia, PA (1989).

[9] D. Klein, H. Lehmann and E.H. Shortliffe, A value-theoretic expert system for evaluating randomized clinical trials, in: Proceedings Fourteenth Annual Symposium on Computer Applications in Medical Care, Washington, DC (1990) 810-814.

[10] H. Lehmann, A Bayesian computer-based approach to the physician's use of the clinical research literature, Ph.D. Dissertation, Medical Information Sciences, Stanford University, Stanford, CA ( 1991 ).

[ 11 ] H. Lehmann and E.H. Shortliffe, THOMAS: building Bayesian statistical expert systems to aid in clinical decision making, in: Proceedings Fourteenth Annual Symposium on Computer Applications in Medical Care, Washington, DC (1990) 58-64.

[12] R. Miller, H. Pople and J. Myers, INTERNIST-I: an experimental computer-based diagnostic consultant for general internal medicine, New England J. Med. 307 (1982) 468-476.

[13] J. Pearl, On evidential reasoning in a hierarchy of hypotheses (Research Note), A rtif. Intell. 28 (1986) 9-15.

[14] J. Pearl, Fusion, propagation, and structuring in belief networks, Artif. Intell. 29 (1986) 241-288.

[15] J. Reggia and B. Perricone, Answer justification in medical decision support systems based on Bayesian classification, Comput. Biol. Med. 15 (1985) 161-167.

[16] G. Shafer and R. Logan, Implementing Dempster's rule for hierarchical evidence, Artif. Intell. 33 (1987) 271-298.

[17] E.H. Shortliffe, Medical informatics and clinical decision making: the science and the pragmatics, Med. Decision Making 11 (1991) $2-S14.

[18] E.H. Shortliffe and B. Buchanan, A model of inexact reasoning in medicine, Math. Biosci. 23 (1975) 351-379.

[19] D.J. Spiegelhalter, Probabilistic reasoning in predictive expert systems, in: L.N. Kanal and J.F. Lemmer, eds., Uncertainty in Artificial Intelligence (North-Holland, Amsterdam, 1986) 47-67.

[20] S. Weiss, C. Kulikowski, S. Amarel and A. Safir, A model-based method for computer- aided medical decision making, Artif. Intell. 11 ( 1978 ) 145-172.