current tools for assisting intelligent agents in real-time … · 2008-08-29 · current tools for...

Current Tools for Assisting Intelligent Agents in

Real-time Decision Making

Hakan L. Younes

Royal Institute of TechnologySchool of Electrical Engineering and Information Technology

December 1998

AbstractDecision making is something with which we are all familiar. Each day wefind ourselves in situations where we have more than one alternative tochoose from. Most of our everyday decisions require little or no reflection.Yet, other decisions are more complex and may require thorough analysis.The field of decision analysis studies the application of decision theory toactual decision problems. Decision theory is the mathematical foundationfor decision making, which combines probability theory and utility theoryin order to describe what constitutes a rational decision. Today there arenumerous tools for decision analysis that run on ordinary PCs. Thesemake it easier for human decision makers to structure a decision problem,and help visualize the effects of possible outcomes. However, not onlyhumans are decision makers. Intelligent agents are also faced with decisionproblems. This thesis investigates if any of the computerized tools fordecision analysis can be used by intelligent agents. Special attention isgiven to real-time domains, where data that decisions depend on varyrapidly. These domains require quick responses from the tools. Otherwisethe result of a decision analysis will be based on data that is no longeraccurate. The objective of the thesis is to provide results indicating ifany of the available tools for decision analysis are fast enough for use inreal-time domains.

This thesis corresponds to the effort of 20 full-time working weeks.

To Barb

Contents

1 Introduction 11.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.2 Objective . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21.3 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31.4 Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41.5 Structure of Thesis . . . . . . . . . . . . . . . . . . . . . . . . . . 4

2 Decision Analysis 52.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52.2 Modeling Decision Problems . . . . . . . . . . . . . . . . . . . . . 5

2.2.1 Decision Trees . . . . . . . . . . . . . . . . . . . . . . . . 62.2.2 Belief Networks . . . . . . . . . . . . . . . . . . . . . . . . 72.2.3 Influence Diagrams . . . . . . . . . . . . . . . . . . . . . . 82.2.4 Hierarchies of Criteria . . . . . . . . . . . . . . . . . . . . 9

2.3 Solving Decision Problems . . . . . . . . . . . . . . . . . . . . . . 92.4 Supersoft Decision Theory . . . . . . . . . . . . . . . . . . . . . . 102.5 Sensitivity Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . 11

3 Current Tools 133.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133.2 Decision Analysis for Humans . . . . . . . . . . . . . . . . . . . . 143.3 Decision Analysis for Intelligent Agents . . . . . . . . . . . . . . 143.4 A Closer Look at the Tools . . . . . . . . . . . . . . . . . . . . . 15

3.4.1 Tools Based on Probabilistic Inference . . . . . . . . . . . 163.4.2 Tools for Classical Decision Analysis . . . . . . . . . . . . 173.4.3 Tools for Supersoft Decision Analysis . . . . . . . . . . . . 183.4.4 Tools for Multi-criteria Decision Analysis . . . . . . . . . 18

3.5 Preliminary Findings . . . . . . . . . . . . . . . . . . . . . . . . . 18

4 Basing Pronouncers on Current Tools 214.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 214.2 Functionality of Pronouncers . . . . . . . . . . . . . . . . . . . . 21

4.2.1 Basic Functionality . . . . . . . . . . . . . . . . . . . . . . 214.2.2 Extended Functionality . . . . . . . . . . . . . . . . . . . 22

4.3 Problem Representation . . . . . . . . . . . . . . . . . . . . . . . 234.4 Situating the Pronouncer . . . . . . . . . . . . . . . . . . . . . . 23

i

ii CONTENTS

5 Implementation of Pronouncers 255.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 255.2 Elements of Implementation . . . . . . . . . . . . . . . . . . . . . 25

5.2.1 Communication Layer . . . . . . . . . . . . . . . . . . . . 265.2.2 Parsing and Dispatching Layer . . . . . . . . . . . . . . . 265.2.3 Decision Analysis Layer . . . . . . . . . . . . . . . . . . . 275.2.4 Decision Modules . . . . . . . . . . . . . . . . . . . . . . . 28

5.3 Advanced Implementation Issues . . . . . . . . . . . . . . . . . . 29

6 Performance of Pronouncers 316.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 316.2 Test Cases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

6.2.1 Scenario 1: Agent Cannot See Ball . . . . . . . . . . . . . 316.2.2 Scenario 2: Agent Has Ball . . . . . . . . . . . . . . . . . 32

6.3 Test Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . 336.4 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

7 Discussion 377.1 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 377.2 Future Research . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

Acknowledgements 39

Bibliography 41

A Internet Resources 43

B Glossary 45

Chapter 1

Introduction

1.1 Background

Decision making is something with which we are all familiar. Each day we findourselves in situations where we have more than one alternative to choose from.To act rationally, we should choose the alternative which gives the most gainaccording to our values. Most of our everyday decisions require little or no reflec-tion. Yet, other decisions are more complex and may require thorough analysis.In all cases we base our decisions on the information we have gathered aboutthe alternatives and the knowledge we have about our situation. The qualityof the knowledge can vary from precise and known to imprecise and believed[Boman and Verhagen, 1998]. In a complex decision problem the quantity ofdata can be large, the relations between data can be intricate, and our uncer-tainty about the data and the relations will complicate the task of making theright decision. In domains such as health care and business, decision problemsare often complex and the stakes can be very high, thus making it even moreimportant to choose the right alternative.

The field of decision analysis evolved in the 1950s and 1960s, and studiesthe application of decision theory to actual decision problems [Fishburn, 1982].Decision theory is the mathematical foundation for decision making, which com-bines probability theory and utility theory in order to describe what constitutesa rational decision. In the early 1980s, computers began to get a more sig-nificant role in decision making processes. At first, access to computer basedtools was mainly restricted to expert users, but as ordinary desktop computersbecame more powerful in the 1990s, such tools became accessible to practicallyeverybody. Today there are numerous tools for decision analysis that run onordinary PCs, and are well integrated with other desktop applications. Theymake it easier for the decision maker to structure a decision problem, and helpto visualize the effects of possible outcomes.

However, not only humans are decision makers. Imagine a robot sent outto collect geological samples in a volcano. It must make the right decisions inorder not to fall down into the crater and be destroyed. A human could work asdecision analyst, assisting the robot in making the right decisions based on datagathered by the robot’s sensors. It would probably work well if the volcanowas on Earth, but what if it was on Io—one of the planet Jupiter’s moons?

1

2 CHAPTER 1. INTRODUCTION

The data from the robot would be at least 30 minutes old when it reached theperson on Earth. Even if the decision is sent from Earth almost immediatelyafterwards, it would still take another 30 minutes before it reaches the robot.Since over an hour will have passed from the point in time when the robotscanned its environment until it can receive a suggested action based on thisdata from its human decision analyst, the situation might very well have changeddramatically, and the decision based on the old data can prove to be fatal inthe new situation. Furthermore, it is sometimes a waste of resources if a humanis always needed for supervision. In [Dorais et al., 1998], an example with aMars rover in an imagined manned mission to Mars is given. Although Mars iscloser to Earth than Jupiter, the round trip time for a radio signal from Marsto Earth can be up to 40 minutes making it infeasible to tele-operate the roverfrom Earth. During a manned mission a crew member could tele-operate therover, but if the rover could make decisions autonomously—in particular whentravelling long distances over relatively uninteresting terrain—the crew membercould attend more important matters. Still, we do not need to extend our scopeto outer space, in order to find domains for non-human decision makers. On theInternet, electronic market places are rapidly emerging, where intelligent agentsact as buyers and sellers on behalf of their human mandators (see e.g. [Chavezand Maes, 1996] or [Elofson and Robinson, 1998]).

In view of these examples, we may ask the following question: Can intelli-gent agents benefit from using current tools for decision analysis? First we mustof course convince ourselves that any of the current tools at all can be used byintelligent agents. This is by no means obvious. Existing tools for decision anal-ysis focus on human users. While humans benefit from extensive graphical userinterfaces, intelligent agents require other means to communicate with the de-cision analysis tool. Whereas this might place practical obstacles in the way forintelligent agents to make use of the tools, it is of little principal interest. Thereare several effective techniques for adding an interface for intelligent agents toa tool without altering its main functionality (see [Genesereth and Ketchpel,1994] or [Fou, 1997]).

Assuming that the communication issue can be solved (cf. [Boman, in press]),it seems reasonable to believe that current tools can be useful for intelligentagents—at least in situations illustrated by the last of the three examples above.However, the first two examples illustrate situations very much unlike deci-sion situations for which most (if not to say all) current decision analysis toolswhere designed to be used—situations which require real-time decision mak-ing. Whether any of the current tools are useful in such situations must beempirically tested.

1.2 Objective

The main objective of this thesis is to investigate if any of the currently availabletools for decision analysis can be used for assisting intelligent agents in real-time decision making. In order to achieve this, some of the available tools (seeChapter 3) will be selected and a separate pronouncer1 will be built based on

1This is a term introduced in [Boman and Verhagen, 1998], where it is also further describedand motivated. In e.g. [Boman, in press], decision module is used to denote the same type ofentity. See also, Chapter 4.

1.3. RELATED WORK 3

each of the selected tools.The primary measure for the pronouncers will be speed. The speed is im-

portant because the focus is on real-time applications. What will be measuredis time passed from when an agent sends a decision problem to a pronouncerto when the agent receives an answer. By running tests on decision problemsof varying complexity, it can be determined which of the available tools for de-cision analysis is the fastest. Depending on what is understood by “real-time”(which can vary between domains), it can then be determined whether any of thetools are fast enough for use in real-time domains. In this work, the RoboCupdomain2 is used as a reference domain.

Not only speed is important in evaluating the performance of a pronouncer.Finding a good utilization of pronouncers is equally important. This, however,is a whole research project by itself. Even if a pronouncer is found to be fastenough for a specific domain, the correct decision models must be created, andit must be decided when to make calls to the pronouncer. Without thoroughanalysis of the decision problems involved, using the pronouncer can be a burdeninstead of a gain. Some limited qualitative tests in the RoboCup domain wereintended to be performed as part of this work with the UBU team [Boman etal., 1998] developed by members of the DECIDE research group. Unfortunately,the UBU team has not yet reached a state where pronouncer calls can easilybe incorporated. The main deficiency is the lack of explicit modeling of theuncertainty in the domain. Currently the uncertainty is handled in an ad hocmanner, which is highly inappropriate if pronouncers are to be used. Due to this,qualitative tests falls outside the scope of this work. Thus, the objective of thethesis, more clearly stated, is to provide results indicating if any of the availabletools for decision analysis are fast enough for use in real-time domains—not howto make the best use of them.

1.3 Related Work

The usefulness of decision theory and decision analysis to software agents is in-vestigated in [Bovin and Hammarberg, 1998]. The authors present a frameworkfor using decision analysis in decision support systems for agents, but lack toolsfor implementing many of the features they propose. This work can fill that gapif any of the investigated tools are found useful.

In [Bovin and Hammarberg, 1998] real-time domains are not specificallyconsidered. The idea of utilizing decision analysis for real-time decision problemscan be found in [Boman, in press]. There, two provisos are made.

• Agents act in accordance with advice obtained from their decision module,with which they can communicate.

• The decision module contains algorithms for efficiently evaluating supersoftdecision data.

The first proviso mainly concerns the communication issue which was brieflydiscussed at the end of Section 1.1, and is not very controversial. The secondproviso, however, may be too restrictive since it rules out all but one of theavailable tools for decision analysis. Consequently, this work will not upholdthis proviso.

2See [Kitano, 1998] for a comprehensive introduction to the RoboCup domain.

4 CHAPTER 1. INTRODUCTION

1.4 Methodology

The initial phase of the thesis work was partly dedicated to studying relevantliterature on decision analysis and decision theory, in particular work concerningintelligent agents and multi-agent systems. In addition, a substantial part con-sisted in gathering information about available tools for decision analysis. Thiswas first done by compiling a list of known tools. Later, information on eachtool was extracted from the Internet, and each of the companies and academicresearch groups responsible for a tool were inquired for information of specificinterest to this thesis.

After the initial phase followed a more thorough study of the theoreticalfoundations of involved techniques for decision analysis. This was accomplishedby further literary studies, as well as practical training with some of the availabletools.

When the preparatory and theoretical part of the work was done, some toolswere selected for basing pronouncers on. Pronouncers were then implementedusing a general architecture. Some of the tools had accompanying tutorialswhich made the pronouncer implementation easy. Help with the tools was alsoacquired directly from people responsible for development of the tools.

In the final phase, the pronouncers were tested. Suitable test cases weredesigned in co-operation with members of the DECIDE research group involvedwith RoboCup.

1.5 Structure of Thesis

In Chapter 2, the fundamentals of decision analysis needed to understand thiswork is presented. Those already familiar with the field of decision analysis couldeither skim through this chapter just to capture the terminology used through-out the thesis, or skip the chapter altogether. Chapter 3 gives an overview ofcurrent tools for decision analysis, and provides a brief statement for each of thetools in what way (if at all) they could be useful to intelligent agents. Readingthis chapter should be sufficient for people only interested in getting a pictureof what is currently available, and some preliminary recommendation for whichtools may be interesting to consider basing a pronouncer on. Chapter 4 dealswith implementation issues for pronouncers. It elaborates on matters brieflytouched upon in Chapter 3. Basic functionality of a pronouncer is defined here,which is later used when actually implementing pronouncers. Chapter 5 de-scribes the implementation of three pronouncer in detail. A general frameworkfor pronouncer implementation is presented here. This is followed by Chapter 6,which describes the tests that have been performed with the implemented pro-nouncers. The final chapter contains a discussion on what has been achievedwith this work, as well as what can be done in the future.

Chapter 2

Decision Analysis

2.1 Introduction

Decision analysis is the application of decision theory to actual decision prob-lems. This chapter introduces the elements of decision analysis as well as deci-sion theory referred to and used in subsequent chapters. Various techniques formodeling and solving decision problems are presented. Methods for handlingvagueness in descriptions of decision situations are briefly discussed. A sectionabout sensitivity analysis is also included, although this is a subject that willbe given limited attention in this work.

2.2 Modeling Decision Problems

The first step in analysing a decision problem is to create an explicit model ofthe problem. There are several techniques for doing this, and those techniques ofrelevance to this thesis are presented here. First, the following concepts shouldbe introduced:

Probability Uncertainty in a decision problem can be modeled with proba-bilities. If A is an event, then the probability of A occurring is denotedby P (A). This is a value between 0 and 1 when it is uncertain whetherA will occur or not, it is 1 if A for certain will occur, and it is 0 if A forcertain will not occur. Probabilities can be inherent in the domain of thedecision situation, as in lotteries where the probability of winning is thenumber of winning tickets divided by the total number of lottery tickets.Probabilities can also be subjective, and are then only manifestations ofthe decision maker’s beliefs. The only restriction placed on the decisionmaker when assigning subjective probabilities is that they must conformto the axioms of probability (see e.g. [Russell and Norvig, 1995]). Condi-tional probabilities are used when information about the situation has beenacquired. If A and B are events, then P (A|B) denotes the probability ofA occurring given that B has occurred. Conditional probabilities are alsocalled posterior probabilities—then in contrast to prior probabilities, whichis an alternative name for unconditional probabilities.

Utility In a decision situation there are several possible final outcomes. In

5

6 CHAPTER 2. DECISION ANALYSIS

order to concretize how desirable an outcome is to the decision maker, avalue is assigned to each outcome. This value is referred to as the utilityof an outcome. Sometimes utilities represent values in the real world.For example, the utility of winning $1,000 on a lottery ticket bought for$10 could be set to 990, while the utility of not winning could be set to−10. However, in many situations there is no straightforward measure toadopt from the domain of the decision situation, and a subjective valuemust be used for the utility of an outcome. Even in cases where objectivemeasures do exist, it can be necessary to account for subjective judgmentswhen assigning utilities. Imagine a person with severe toothache. If dentalservice costs $100, then having $100 would enable the person to cure thetoothache, while having $90 would not be of much help to the person.

2.2.1 Decision Trees

In a decision tree, a decision problem is modeled chronologically. It starts witha decision node (depicted as a square) in the root of the tree. A decision nodedenotes a point in time where a decision has to be made. From a decisionnode, two or more branches emanate—each branch representing an alternativeavailable to the decision maker. Each of the branches lead to a new node. Thesenodes can be additional decision nodes. They can also be either chance nodes(circles) or result nodes (triangles). A chance node stands for a probabilisticevent in time, the outcome of which is not under the influence of the decisionmaker. Branches that emanate from a chance node represent possible outcomesof the event. Finally, a result node denotes an outcome of the entire decisionproblem. Consequently, no branches emanate from a result node, and all leafnodes in a decision tree must be result nodes. A path from the root of thedecision tree to one of the result nodes is often referred to as a scenario.

This only gives the structure of the tree. In order to make a completedecision tree, certain values must also be added. First, probabilities have to beassigned to each of the branches emanating from chance nodes. Probabilitiesfor branches emanating from the same chance node must sum to 1. This isbecause all possible outcomes of a probabilistic event should be included in themodel, and the outcomes should be mutually exclusive. Second, utilities must

ball becomes visible

0.63

ball still not visible

0.37

ball still not visible

0.50


0.50

move

wait

ball not visible

10

-10

15

0

Figure 2.1: A sample decision tree.

2.2. MODELING DECISION PROBLEMS 7

be assigned to all result nodes. Once these two types of values have been added,the decision tree is complete.

Figure 2.1 shows a complete decision tree. The decision maker—in this casea soccer playing intelligent agent that cannot see the ball—has two alternatives:move, and wait. Both alternatives lead to a chance node with two possibleoutcomes—either the ball becomes visible to the agent, or it remains out ofsight. The subjective probabilities for each of the outcomes are given in thetree. For example, the probability of the ball becoming visible if the agentmakes a move has been estimated by the agent to be 0.63. Utilities are givenat each result node.

2.2.2 Belief Networks

A belief network1 is used to represent dependencies between random variables.Each node in the network denotes a random variable, and a directed link betweentwo nodes represents dependencies between the two random variables denotedby the involved nodes. The node at the starting-point of a link is called aparent of the node at the end-point of the link. A conditional probability tableis associated with each node. The table quantifies the effects that the parentshave on the node. The network may not contain cycles.

As before, imagine a soccer playing intelligent agent that cannot see the ball.It wants to find out the probability of the ball entering its field of vision, givenits beliefs about the current movement of the ball—whether it is moving towardsor away from the agent. The belief network given in Figure 2.2 represents thissituation. Probabilities, representing the agent’s beliefs, are given in the tablesassociated with each of the nodes in the network.


movement of ball

P(visible)

0.40

0.05

0.7

movement

towards

away

P(movement = towards)

Figure 2.2: A sample belief network.

Belief networks are not primarily intended for modeling decision problems,since they do not provide explicit means to model alternatives and utilities.Numerically, a belief network is just a representation of the joint probabilitydistribution among the involved random variables [Russell and Norvig, 1995].An extension of belief networks that is better suited for modeling decision prob-lems is influence diagrams.

1Other names for this structure that often occur are Bayesian network, probabilistic net-work, and causal network.


2.2.3 Influence Diagrams

In addition to nodes for representing random variables provided by belief net-works, influence diagrams also provide node types for modeling alternatives andutilities. Chance nodes (depicted as ovals) denote random variables, and cor-respond to the only node type available in belief networks. Alternatives aremodeled with decision nodes (rectangles). A decision node indicates a decisionfacing the decision maker—similar to decision nodes in decision trees—and con-tains all alternatives available to the decision maker at that point. The thirdand final node type provided by influence diagrams is utility nodes (diamonds).Nodes of this type represent the utility function of the decision maker. In utilitynodes, utilities are associated with each of the possible outcomes of the decisionproblem modeled by the influence diagram.

Directed links between nodes represent influences. Links between two chancenodes have the same semantics as in belief networks. Other links in an influencediagram may also represent a temporal relation between the nodes involved. Forexample, a link from a decision node to a utility node not only indicates thatthe choice of action influences the utility, but also that the decision precedesthe outcome in time.

Influence diagrams are useful in bringing out the structure of a decisionproblem. While for example decision trees are more effective at presentingthe details of a decision problem, influence diagrams more clearly show factorsthat influence a decision. This makes influence diagrams a fairly compact—butnevertheless very powerful—way of modeling decision problems.

In Figure 2.3, the belief network from Figure 2.2 has been extended to ainfluence diagram by the addition of a decision node and a utility node. Asin the example from Section 2.2.1, the decision to make is whether to make amove or to wait2. The tables in the figure show probability distributions for therandom variables, and the utility function for the utility node.

movement of ball

ball becomes visible U

D: move or wait?

movement

towards

towards

away

away

D

move

wait

move

wait

P(visible)

0.50

0.40

0.20

0.05

P(movement = towards)

0.7

visible

T

T

F

F

D

move

wait

move

wait

10

15

-20

0

U

Figure 2.3: A sample influence diagram.

2Without the chance node “movement of ball”, the two decision problems would be struc-turally equivalent.

2.3. SOLVING DECISION PROBLEMS 9

2.2.4 Hierarchies of Criteria

Both decision trees and influence diagrams focus on the events that can affectthe outcome of a decision situation. An alternative approach is to focus on thecriteria affecting the decision, and build hierarchies of these criteria. At the toplevel of the hierarchy is the goal to attain by making the decision. The secondlevel of the hierarchy consists of criteria directly affecting the goal. Subsequentlevels of the hierarchy consist of subcriteria contributing to the assessment ofcriteria (and subcriteria) in levels above. At the very bottom of the hierarchyare the alternatives available to the decision maker.

Once a hierarchy has been constructed, each element (criterion or alterna-tive) in the hierarchy must be assigned weights. The weights of an elementdetermines its level of influence on criteria above it compared to other elementsat the same level in the hierarchy. Weights can be assigned by pairwise compari-son between elements at the same level. What must be considered when makingthe comparison is dominance and intensity. Dominance determines which of twoelements meets a criterion more. Intensity stands for how much one elementdominates another.

In Figure 2.4, the goal is to score on a penalty kick in a soccer game. Twoplayers are being considered for taking the penalty kick. In this very simpleexample there is only one level of criteria, which contains “self-control” and“skill”. Weights, on a scale from 0 to 100, are given next to each link. Forexample, it can be read that self-control is an extremely important criterion forsucceeding in scoring on a penalty kick, while skill is a criterion with slightlyless influence on the goal node.

75 100

100

5050

75

self-control skill

player 1 player 2

score on penalty goal

criteria level 1

alternatives

Figure 2.4: A sample decision problem modeled as a hierarchy of criteria.

2.3 Solving Decision Problems

There are many methods for solving decision problems. Which method to useis partly dependent on how the decision problem has been modeled. Findingthe solution often involves calculating the expected utility of each alternativeavailable in the problem. If the decision problem has been modeled as a decisiontree this is a fairly straightforward process, and can be accomplished with a


dynamic programming algorithm called averaging out and folding back [Raiffa,1968].

For influence diagrams, there are several approaches to calculating expectedutilities. Influence diagrams were first introduced in [Howard and Matheson,1984], and there a method for deriving a decision tree from an influence di-agram is presented. The decision tree can then be solved with conventionalmethods, but in general the tree is of exponential size. A direct method forsolving influence diagrams by reversing links and removing nodes was presentedin [Shachter, 1986]. Yet another approach is to transform the influence diagraminto a belief network [Jensen, 1996].

Inference in belief networks is based on Bayes’ theorem for conditional prob-ability and is called Bayesian updating or belief updating. The method amountsto calculating the posterior probability distribution for some random variablesgiven the exact value for other variables. In [Pearl, 1986], a message-passingscheme is given for updating probability distributions in singly connected net-works3 (also known as polytrees). An algorithm that works with general net-works is given in [Lauritzen and Spiegelhalter, 1988]. It transforms the beliefnetwork into a junction tree of cliques before performing probabilistic inference.

Multi-criteria decision analysis constitutes a large area of research withinthe field of decision analysis. A method for solving decision problems modeledas described in Section 2.2.4 is the Analytic Hierarchy Process (AHP) [Saaty,1990]. There is a potential drawback with AHP if alternatives are added to theinitial model later. Due to the relative nature of judgments in AHP, adding anew alternative—even if it is very poor—can reverse the relative ranking of pre-viously top-ranked alternatives. The Simple Multi-Attribute Rating Technique(SMART) is similar to AHP in that it works with hierarchies of criteria, butsince it does not use relative ratings of alternatives it is less sensitive to additionof alternatives at later stages.

2.4 Supersoft Decision Theory

In classical decision theory, probabilities and utilities are assigned exact values.The probability of a given event occurring can for example be set to 0.17. Thisoften gives a pretence of accuracy, as decision makers in most realistic situationslack foundation for preferring a certain value like 0.17 to other values close by.This problem is addressed in supersoft decision theory, which allows assessmentsof probabilities and utilities to be represented by vague and imprecise statements[Malmnas, 1995]. This could be statements like “Event A is very likely to occur”and “Outcome o1 is slightly more desirable than outcome o2, but both are quiteundesirable”. The former is an example of a qualitative statement, while thefirst part of the latter is an example of a comparative statement.

In order to handle vague and imprecise statements computationally, theymust be translated into a quantitative form. A number of translation rulesfrom linguistic statements to intervals of probabilities and utilities are given in[Malmnas, 1994]. In [Danielson, 1997], the Delta method for solving decisiontrees containing so-called supersoft decision data is presented. Since probabil-ities and utilities are given as intervals, the expected utilities will not take on

3In a singly connected network, there is at most one undirected path between any twonodes.

2.5. SENSITIVITY ANALYSIS 11

single values. Usually the possible expected utilities of alternatives overlap,making it non-trivial to compare them. The Delta method measures the rel-ative strength between two alternatives in order to determine dominance. Thedominance of one alternative over another can be either weak or strong, thusindicating how much better an alternative is compared to other alternatives.

2.5 Sensitivity Analysis

A sensitivity analysis can be performed in order to determine how variationsin one or more of the quantities involved in a decision problem affect the resultof a decision analysis. Since probabilities and utilities often are uncertain, asensitivity analysis yields insight into where more effort should be put in orderto make the decision more certain. Traditionally, a sensitivity analysis is carriedout by studying the effect of varying one or a few input variables at a time.The result of a one-way sensitivity analysis (changes in the value of a singleinput variable) is thresholds for the involved variable for when the optimalstrategy changes. When one strategy is better than other strategies over theentire interval included in the analysis, the decision problem is insensitive tochanges in the involved variable. A sensitivity analysis can also be multi-way(simultaneous changes to multiple input variables), in which case the resultingthresholds are hyperplanes.

Even in cases where supersoft decision data is used in the representationof the decision problem, a sensitivity analysis can be useful (c.f. [Boman, inpress]). Although intervals for probabilities and utilities are already used, anddominance gives a measure on the closeness between alternatives, varying theintervals slightly (e.g. by contraction or expansion) will reveal which factors thedominance depends on the most.

Chapter 3

Current Tools

3.1 Introduction

Decision analysis is a computationally demanding task. Until the early 1980s,computers were only used to a limited extent by decision makers. Along withthe rapid progress made by the computer hardware industry in the past fewdecades, this has changed. Today, practically every decision maker has accessto a computer. Consequently, a great number of tools have been developedto meet the demands of human decision makers. This chapter starts with anoverview of what these tools have to offer human decision makers. The overviewis followed by a discussion on aspects where intelligent agents differ from hu-man decision makers, and what impact this has on the demands for the tools.Attention is then given to tools that at a first glance appear to be useful forintelligent agents, and it is investigated how any of these tools could be used toassist intelligent agents in making decisions. The chapter concludes with somepreliminary findings.

Figure 3.1: Screenshot from DATA 3.0 showing a simple decision tree.

13

14 CHAPTER 3. CURRENT TOOLS

3.2 Decision Analysis for Humans

Available tools for decision analysis have been designed with human users inmind. They are well integrated with other desktop applications for PCs, andare equipped with intuitive graphical user interfaces (Figure 3.1). These allowthe decision maker to focus on the structure of a decision problem, and helpvisualize the effects of possible outcomes.

Several tools take the integration with other elements of the decision maker’sdesktop environment one step further. These are the so-called add-ins for ex-isting spreadsheet applications, which do not work as stand-alone applications.Instead they make functionality for decision analysis directly available in thespreadsheet application (Figure 3.2). In many decision situations (e.g. in busi-ness activity), data used in the model of the situation come from spreadsheets.Thus, add-ins allow decision makers to transparently build models where datafor the models is found.

Figure 3.2: Screenshot from Microsoft Excel with PrecisionTree as add-in.

3.3 Decision Analysis for Intelligent Agents

To an intelligent agent, an extensive graphical user interface (GUI) is of lit-tle use. Other means to interact with the decision analysis tool are needed.While this does not per se disqualify tools that work as add-ins for spread-sheet applications, there are other reasons for not including such tools in anyfurther discussion. As has been argued for, add-ins make perfect sense in asituation where data for the decision analysis is taken from a spreadsheet. Ingeneral, this will not be the case for the intelligent agents mainly concerned

3.4. A CLOSER LOOK AT THE TOOLS 15

build model load model set value type of interface

Ergo x x x C library

GeNIe/SMILE x x x C++ library

Hugin x x x C library

Netica x x x C library

Analytica x x x OLE

DATA x x DDE, OLE

DPL x x x DDE

DecisionPro x x x DDE, TCP/IP

Delta x x C library

Expert Choice (GUI only)

Criterium DecisionPlus x x OLE

Table 3.1: Functionality provided by interfaces suitable for intelligent agents.

in this thesis—agents in real-time applications. Furthermore, it seems safe toassume that stand-alone tools will perform at least equally well as any add-inswith equivalent functionality—this without adding the computational power ofa spreadsheet application, which in most cases will prove superfluous and some-times even a burden.

Most current tools for decision analysis do provide means of interaction otherthan through a GUI. The functionality provided through these alternative in-terfaces, compared to what is provided through the GUI, vary. Some tools (e.g.DATA) only provide an alternative interface whose main purpose is to allowsharing data with other applications (e.g. a spreadsheet application). Theseinterfaces are often limited to providing functionality for reading and modify-ing values in the model of the decision problem. For other tools, alternativeinterfaces exist that provide total control over model building and inference inmodels (e.g. Netica).

In order to build a basic pronouncer based on a tool for decision analysis, theminimum requirement is that values (probabilities, utilities, etc.) in an alreadybuilt model can be set by the agent using the pronouncer. A more advanced pro-nouncer could allow the agent to build the model from scratch, and modify thestructure of the model at later stages (e.g. by removing or adding alternatives).A more thorough discussion on implementation issues for pronouncers is givenin Chapter 4. Table 3.1 shows to what extent these functionalities are providedby alternative interfaces of current tools, and also what type of interfaces theseare. Evidently, all tools except Expert Choice do provide functionality so thatat least a basic pronouncer could be based on them.

3.4 A Closer Look at the Tools

The various stand-alone tools that exist for decision analysis are mainly basedon the different techniques for modeling and solving decision problems brieflyintroduced in Chapter 2. Table 3.2 shows which of these techniques are sup-ported by which tools. In the closer presentation of the tools below, they havebeen divided into groups roughly based on which techniques they support. Foreach tool listed, the name of the company or academic institution responsible


BN ID DT AHP SMART

Ergo x

GeNIe/SMILE x x

Hugin x x

Netica x x

Analytica x

DATA x x

DPL x x

DecisionPro x

Delta x

Expert Choice x

Criterium DecisionPlus x x

Table 3.2: Support for belief networks (BN), influence diagrams (ID), decisiontrees (DT), AHP, and SMART in stand-alone tools.

for the tool is given in parenthesis. A list of Internet URLs for the companiesand academic institutions is given in Appendix A.

3.4.1 Tools Based on Probabilistic Inference

These are tools that have belief networks as basic structure, and use probabilisticinference for solving decision problems.

Ergo (Noetic Systems Inc.)

In Ergo, decision problems can only be modeled as belief networks. As men-tioned in Section 2.2.2, belief networks do not provide any explicit means formodeling alternatives and utilities. Consequently, Ergo is a less interesting al-ternative for basing a pronouncer on. Nevertheless, it would be possible. A Cfunction library supplies an API with functionality virtually identical to that ofErgo.

GeNIe/SMILE (Decision Systems Laboratory, Univ. of Pittsburgh)

Graphical Network Interface—GeNIe for short—uses probabilistic inference tosolve decision problems. Belief networks and influence diagrams can be used tomodel the problems. In reality, GeNIe is just a graphical interface to SMILE(Structural Modeling, Inference, and Learning Engine). SMILE is a platformindependent library of C++ classes which contains functionality for building andmodifying decision models, and for making inferences in such models. SMILEimplements several algorithms for probabilistic inference. Both exact algorithmsmentioned in Section 2.3 are included, as well as a number of approximatealgorithms based on stochastic sampling.

Hugin (Hugin Expert A/S)

Hugin is another tool that works with both belief networks and influence dia-grams. Models are compiled into junction trees for fast probabilistic reasoning.

3.4. A CLOSER LOOK AT THE TOOLS 17

The inference engine is available as a platform independent C function library,which also provides complete control over model building.

Netica (Norsys Software Corp.)

Netica is very similar to Hugin. Belief networks and influence diagrams aresupported and—as with Hugin—these are compiled into junction trees for in-creased performance. Support for entering positive, negative, and likelihoodfindings is also provided. This can be used for probabilistic learning. NeticaAPI—a platform independent C function library—provides total access to thefunctionality of Netica.

3.4.2 Tools for Classical Decision Analysis

This group basically contains all those tools that do not fit into any of the othergroups. They all require probabilities and utilities to be given as exact values.All but one of these tools uses decision trees as basis for models of decisionproblems.

Analytica (Lumina Decision Systems)

Analytica only allows decision problems to be modeled using influence diagrams.Analytica Decision Engine (ADE) gives access to the functionality of Analyticathrough OLE Automation.

DATA (TreeAge Software, Inc.)

DATA, available for Windows and MacOS, works with decision trees and in-fluence diagrams. The Windows version includes a scripting language calledDATAScript, which enables external access to the program through DDE con-nections. DATAScript allows for probabilities and utilities to be modified, andanalysis results to be retrieved. However, models can only be created throughthe GUI.

An ActiveX control, called ActiveDATA, which contains the inference engineof DATA is currently under development. Through OLE Automation, modelscan be loaded (only decision trees), probabilities and utilities modified, andanalysis results retrieved. ActiveDATA is better suited for basing a pronounceron than DATA.

DPL (Applied Decision Analysis, Inc.)

DPL stands for Decision Programming Language. It is a tool based on a pro-gramming language for decision analysis and runs on Windows platforms. Bothdecision trees and influence diagrams can be created and controlled throughthe language. A special developer version of the tool is available, which en-ables external applications to utilize the decision engine in DPL through DDEconnections.


DecisionPro (Vanguard Software Corp.)

In a sense opposite to Analytica, DecisionPro works only with decision trees.DecisionPro allows for external access to the functionality of the tool throughDDE or TCP/IP network connections.

3.4.3 Tools for Supersoft Decision Analysis

This group consists of tools that support supersoft decision analysis (see Sec-tion 2.4). Only one of the surveyed tools falls into this category.

Delta (DECIDE, Stockholm University/KTH)

In Delta, decision problems are modeled as decision trees, with probabilitiesand utilities stated on interval form. Comparative statements are representedby links between consequences. Delta is the only tool which also allows partialinformation, i.e. not all probabilities and utilities have to be specified in themodel prior to carrying out a decision analysis. The functionality of Delta isavailable through a platform independent C function library called DeltaLib.

3.4.4 Tools for Multi-criteria Decision Analysis

Tools in this group are based on multi-criteria decision analysis.

Expert Choice (Expert Choice, Inc.)

Expert Choice could be considered as the standard tool for decision analysisusing AHP. Yet, since it does not provide any other means of interaction otherthan a GUI, it is of little interest for this work.

Criterium DecisionPlus (InfoHarvest, Inc.)

Criterium DecisionPlus supports both AHP and SMART. Model building canonly be performed through the graphical editor. However, an ActiveX controlcalled WDObj (Weighted Decision Object) allows models to be loaded, weightsto be altered, and alternatives to be ranked through OLE Automation.

3.5 Preliminary Findings

As has been noted in this chapter, most of the surveyed tools are likely to beuseful to base a pronouncer on. The most common ways making the decisionanalysis mechanisms available to other software entities are through a C/C++function library, or a dynamic link library accessible through OLE Automation.Both of these types of “interfaces” to the inference engines are fairly easy touse, and can either be utilized directly in the code of the agent or in a separatepronouncer called upon by the agents.

Ergo lacks explicit means for modeling decision problems and is thus leftout of further consideration. The other three probabilistic tools GeNIe/SMILE,Hugin, and Netica are all very similar in the functionality that they provide,and could all provide an excellent basis for a pronouncer. Empirical tests haveto be performed in order to determine if there are any significant differences

3.5. PRELIMINARY FINDINGS 19

in performance between them, and if any of them are fast enough for real-time decision making. Hugin has been used in a control system for unmannedunderwater vehicles [Lockheed Martin, 1996], but there the real-time reasoningwas handled by a rule-based system while Hugin was used for overall missionplanning. It is not evident whether Hugin was ever considered for use in thereal-time component, so the example does not necessarily rule out Hugin—orany of the other probabilistic tools—as useful for real-time decision making.

Delta is interesting because it is the only tool allowing supersoft decisiondata. Unfortunately the documentation of the API is almost non-existent—something which the commercial tools (and even GeNIe/SMILE) do not sufferfrom. This makes it harder to utilize the API, in particular since the decisionmodels are somewhat more complex than in most other tools. In [Boman andVerhagen, 1998], it is also indicated that Delta might be too slow for use inreal-time situations. A new version of the tool based on a refined version of theoriginal theory—more appropriate for real-time use—is planned, but not yetcompleted. Until then, it would still be interesting to obtain measures on theperformance of Delta.

Of the other tools, primarily DATA (in particular ActiveDATA) and Cri-terium DecisionPlus (WDObj) could be worth taking a closer look at. Thiswould give at least one candidate for a real-time pronouncer from each of thegroups as given in Section 3.4. The remaining tools—Analytica, DPL, andDecisionPro—are all good candidates as well. At this stage they are left outmainly because they are not as readily available as the other tools, but theyshould be taken into consideration again if all the other tools prove to haveinsufficient performance.

Chapter 4

Basing Pronouncers onCurrent Tools

4.1 Introduction

In [Boman and Verhagen, 1998], a pronouncer is defined to be an entity givingadvice to intelligent agents. It is distinguished from a decision module in that itsuggests an extrinsic entity, while a decision module suggests an entity intrinsicto an intelligent agent. Furthermore it suggests “that the advice given is formaland authoritative, giving the entity a normative status” (p. 3, ibid.).

In this chapter, general implementation issues for pronouncers based on cur-rent tools for decision analysis are discussed. In particular, it is discussed whatfunctionality a pronouncer needs to provide, and the term basic pronouncer isestablished. The question of which problem representation to use for intelligentagents using a pronouncer is also treated, as well as the question of whether thedecision analysis functionality should be made internal or external to agents.

4.2 Functionality of Pronouncers

What functionality should a pronouncer provide? An answer to that questionwas hinted in Section 3.3. Here, a more thorough treatment of the questionis given. What is referred to as basic functionality below could be seen asa minimum requirement for a pronouncer. A pronouncer implementing onlybasic functionality is called a basic pronouncer. When considering if a certainfunctionality should be categorized as basic, special attention is given to itspotential use for intelligent agents requiring real-time decision analysis, butalso what can actually be done with most current tools for decision analysis isincluded as a factor.

4.2.1 Basic Functionality

The purpose of a pronouncer is to give advice to intelligent agents in decisionsituations. The agents are likely to be situated in dynamic environments, so thefunctionality of the pronouncer should reflect this in order for the pronouncerto be of any essential interest. This can be provided for in different ways, with

21

22 CHAPTER 4. BASING PRONOUNCERS ON CURRENT TOOLS

varying expressibility for the agents as result. On one extreme, the agents couldbe allowed to formulate decision problems on the fly. This requires that thepronouncer is based on a tool that supports model building. In [Boman, inpress], it is noted that formulating a decision problem from scratch is extremelydifficult for an agent. Instead it is suggested that agents can use templatemodels developed in advance for decision situations that can be presumed tooccur. In real-time decision situations, using templates can be further motivatedby the gain in speed by having an already constructed model compared tohaving to build the model when faced with the decision problem. Given atemplate, an agent can then—to varying extent—be allowed to modify the modelbefore an advice is queried for. At the very least, agents must be allowed toset and modify values (e.g. probabilities and utilities) in the model. A moreadvanced pronouncer could allow for structural modifications of a model, butsince this rules out a number of otherwise interesting tools (see Table 3.1) it isnot considered as basic functionality.

To conclude this discussion, basic functionality is then restricted to comprisethe following:

• load template models

• set/modify values

• evaluate model, and answer with a single recommended alternative

As has been argued for, this functionality is an absolute minimum for any pro-nouncer. In many real-time decision situations it should also be sufficient. Inaddition, the restrictions made allows for most of the tools presented in Chap-ter 3 to constitute the basis for a basic pronouncer.

4.2.2 Extended Functionality

There is hardly a limit to what could be included in the concept of extendedfunctionality. Here, only functionality that is supported by some of the toolsfor decision analysis is concerned.

One obvious candidate is structural modification of models, which was dis-missed as basic functionality in the previous section. Its main use would perhapsbe to allow for alternatives to be added to or removed from a template model.The example in [Boman, in press], with robots collecting radioactive materialin a room, illustrates well how it could be useful. In the example, a robot ex-periences a malfunction in one of its legs. It then has to decide which robotsin the room that it should engage in repairing the leg. The template model forthe given decision situation yields two alternatives: for the robot to repair theleg itself, or for the robot to seek assistance from another robot in the room.In reality, the second alternative may comprise more than one alternative—viz.one for each robot that the damaged robot believes can perform the repair. Atone point in time it might believe that two other robots are able to perform therepair, while later it can have been brought to its attention that yet anotherrobot also is capable of doing the work. This kind of change would best beaddressed by a change in the structure of the decision model.

A basic pronouncer only returns a recommended alternative when givingadvice, without further motivating the choice or indicating how much better itis compared to other alternatives. A more advanced pronouncer could makeuse of sensitivity analysis (see Section 2.5), enabling it to give an indication of

4.3. PROBLEM REPRESENTATION 23

which factors the advice rely on the most. In some cases this indication couldgive the agent receiving the advice a hint on what kind of information it shouldput more effort into acquiring.

An interesting feature of tools based on probabilistic inference is that offindings (also called evidence). A pronouncer making use of this feature wouldallow agents to enter findings as they acquire knowledge about the world. Theagents could in this way gradually build up a more accurate model of a decisionsituation.

4.3 Problem Representation

One important issue when designing a pronouncer is how intelligent agentsshould represent their decision problems when communicating with the pro-nouncer. In Chapter 3 it was shown that there is no single representation ofdecision problems that all current tools for decision analysis can handle. In[Bovin and Hammarberg, 1998] it is suggested that the agents could use anyrepresentation they like, and that this representation could be translated by awrapper into a representation that the pronouncer understands. The authorsrecognize that the translation can be time-consuming, making this approachunattractive in domains where time is of essence. This is of course a most rel-evant observation to this work, since the focus is on real-time decision making.However, there is a more crucial reason why using a translation wrapper is notalways a feasible approach, viz. the fact that some representations of decisionproblems are not translatable into some of the other representations. Influencediagrams can be transformed into decision trees and vice versa, but neither ofthese can be transformed into the hierarchies of criteria used by tools for multi-criteria decision analysis. Furthermore, if the agent uses supersoft decision data,there is no evident method for translating these into data that tools based onclassical decision analysis can utilize without loosing fundamental information.

Consequently, when designing an agent that makes use of a pronouncer it isimportant to know what kind of models the pronouncer supports. In some casesit will be possible to switch pronouncer without having to modify the agent. Inparticular, this may be the case when switching between pronouncers usingthe same model representation. Although the pronouncers might use differentprotocols for communication, a wrapper could in such cases be used to translatebetween the protocols. Time will still be an issue, but this kind of translation islikely to be less time-consuming than translations between models, making theuse of a wrapper more feasible.

4.4 Situating the Pronouncer

A pronouncer can either be used as a decision module, local to each agent, orbe an external entity that querying agents call upon. In [Boman and Verhagen,1998], the latter alternative is chosen. The reason is that the pronouncer thencan account for social factors when evaluating a decision problem, thus helpingto achieve social intelligence. When using a pronouncer for real-time decisionmaking, the extra time it inevitably will take for an agent to communicate withan external entity must be taken under consideration. In cases where this is a

24 CHAPTER 4. BASING PRONOUNCERS ON CURRENT TOOLS

significant factor, an internal entity might be the only feasible choice.

Chapter 5

Implementation ofPronouncers

5.1 Introduction

Chapter 4 discussed the implementation of pronouncers based on tools for de-cision analysis in general terms. The plausibility of the design proposals forbasic pronouncers given there are investigated in this chapter. Basic pronounc-ers based on SMILE, Netica, and ActiveDATA have been implemented, and arepresented here together with a description of a general architecture for suchpronouncers. The chapter also includes a brief discussion on more advancedimplementation issues.

A brief discussion on why SMILE, Netica, and ActiveDATA should be se-lected for further testing was given in Section 3.5. There, Hugin, Delta andCriterium DecisionPlus (WDObj) were also suggested, but no pronouncers havebeen implemented based on these tools. Hugin and Netica are very similar prod-ucts, and testing one of them should be sufficient in an initial stage. Netica wasselected because it was easier to get hold of. Delta has already been judgedas too slow for real-time use, although there are no actual figures supportingthis judgment. Getting test results for Delta would be interesting, but due toproblems with the undocumented API and the lack of a function for loadingmodels, no pronouncer based on Delta was implemented. Finally, CriteriumDecisionPlus is fundamentally different from the other tools in that it requiresdecision problems to be modeled as hierarchies of criteria. It is not obvious howthis type of models could be used in the domain primarily aimed at with thesetests—the RoboCup domain. For that reason, Criterium DecisionPlus has beenleft out.

5.2 Elements of Implementation

When implementing the three pronouncers, as much as possible of the code wasshared. The main elements of the implementations are the following:

• communication with agents

• parsing and dispatching of incoming messages

25

26 CHAPTER 5. IMPLEMENTATION OF PRONOUNCERS

• decision analysis functionality

The list is ordered from the most general element of the implementations to themost specific. Each element can be viewed as a layer (Figure 5.1), which onlycommunicates with adjacent layers.

Communication Layer

Parsing and Dispatching Layer

Decision Analysis Layer

OLE Automation, link library API, etc.

Decision Analysis Tool

UDP/IP socket

Figure 5.1: Architecture of implemented pronouncers.

5.2.1 Communication Layer

The pronouncers receive messages from agents through a UDP/IP socket (thisis the type of communication used in the RoboCup simulation league). Thereare three types of commands that these messages can contain.

• load model

• set value

• get advice

These correspond to the three basic functionalities given in Section 4.2.1. Whena message is received, it is immediately passed down to the parsing and dispatch-ing layer for further processing. Once the message has been processed, a resultis returned to the communication layer. Either this can be an indication of anerror having occurred while parsing the message, or it can be a result that shouldbe forwarded to the agent that sent the message. The latter occurs when a “getadvice” message has been processed, in which case a recommended strategy isreturned through the UDP/IP socket to the calling agent.

This layer only provides general functionality for communication. For thatreason the implementation of the layer could be shared between all pronouncers.

5.2.2 Parsing and Dispatching Layer

In this layer, messages are parsed and then dispatched to the decision analy-sis layer. It works as a mediator between the decision analysis layer and the

5.2. ELEMENTS OF IMPLEMENTATION 27

communication layer. Although the structure of some messages (e.g. the “loadmodel” message) can be kept constant for all pronouncers, other messages (inparticular the “set value” message) must reflect the underlying representation ofdecision problems used by a specific pronouncer. This implies that there mustbe at least one implementation of this layer for each representation of decisionproblems. If the decision analysis layer acts as a wrapper in the sense sug-gested in Section 4.3, one implementation of this layer for each type of problemrepresentation is also sufficient.

5.2.3 Decision Analysis Layer

As stated above, the interface to this layer can be shared between pronouncersusing the same underlying representation of decision problems. The implemen-tation, however, must be specific to each pronouncer since each tool defines itsown interface to the decision analysis functionality. What follows is a brief state-ment about the implementations of this layer in each of the three pronouncers.

SMILE

All the necessary functionality for a basic pronouncer is available in SMILE.The API is well documented, although some parts of the documentation feelsa bit unorganized. The vast number of classes included in the API allows formany alternative ways of performing a certain task (e.g. setting probabilities ina chance node). This could be seen as a strength, but it also makes it harder todetermine which way is the fastest—something that is essential when aiming forreal-time domains. Moreover, the API is incongruent in the sense that utilitiesare set and retrieved in a slightly different way than probabilities.

The pronouncer based on SMILE was implemented for Solaris, and its sizewas about 678 kB. For Windows, SMILE is available as a dynamic link. This al-lows for substantial cuts in pronouncer size, since the largest part—the inferenceengine—is not statically linked with the pronouncer.

Netica

Netica API provides the three basic functionalities in a very straightforward way.The functions of the API are very easy to use, and exemplary documented. Afree demo version of Netica was used when implementing the pronouncer. Thisversion put a limit to the size of models, but was in other respects identical tothe full version.

Two pronouncers based on Netica was implemented—one for Solaris, andone for Windows. The size of the Solaris version was 477 kB, which is about 70percent the size of the SMILE-pronouncer. The Windows version was slightlysmaller—363 kB.

ActiveDATA

ActiveDATA is still in beta-state, and some of the functions do not work prop-erly. In the version used for this work, probabilities could for example notbe set immediately. There were, however, work-arounds for this and a basicpronouncer could be implemented. Instead of setting probabilities and utilitiesimmediately in a model, they are set through variables. This has its advantages,


and is not only a work-around for bugs. By putting all variable definitions inthe root node, probabilities and utilities can be set without traversing the tree.While this put constraints on how decision trees should be modeled, it can bejustified by potential gains in performance.

Since ActiveDATA is an ActiveX control, and thus external to the programusing it, the size of the pronouncer based on ActiveDATA was only 58 kB.

A drawback with ActiveDATA is that models from DATA cannot immedi-ately be used. First a package has to be produced from the model. This is donewith a special utility provided by TreeAge. When a package is produced, apackage number is generated by the utility. With this number, a license for thepackage can be purchased from TreeAge with an accompanying ticket string.The ticket string is needed when opening the package with ActiveDATA. Thissystem may be reasonable when a few large models are used, which probably ishow DATA is traditionally being used. If ActiveDATA is to be used for real-timedecision making, however, there will be many small models instead. Having topurchase tickets for all these models does not seem to be an attractive option.

5.2.4 Decision Modules

Figure 5.1 defines an architecture for pronouncers. In Section 4.4 it was statedthat making the decision analysis functionality internal to the agent can some-times be desirable. In order to achieve this, the agent can bypass the two upperlayers in the original pronouncer architecture and instead communicate immedi-ately with the Decision Analysis Layer. This alternative approach is depicted inFigure 5.2. The Decision Analysis Layer for all three implemented pronouncersprovides an API consisting of C functions, and can be be statically linked withthe agent as a decision module.

OLE Automation, link library API, etc.

Decision Analysis Tool

link library API

Decision Analysis Layer

Communication Layer

Parsing and Dispatching Layer

UDP/IP socket

Figure 5.2: Architecture for decision modules.

A drawback with statically linking a decision module with the agent is thatthe agent then becomes considerably larger. The size of the implemented pro-nouncers range from 58 kB to 678 kB. If each agent in a multi-agent system is

5.3. ADVANCED IMPLEMENTATION ISSUES 29

linked with an entity of that size, it can have a negative impact on the overallperformance of the system.

5.3 Advanced Implementation Issues

A pronouncer should be able to handle calls from several agents concurrently. Ifmodels are shared among agents, then an agent must be able to send compositecommands in one message. Otherwise their commands may come in conflict withone another. For example, agent A sets a probability in model M by sending a“set value” command, and then sends a “get advice” command to evaluate themodel. If these two commands are sent separately, agent B might set anotherprobability in model M—conflicting with A’s beliefs—by sending a “set value”command between the two commands sent by A. In order to assert that noinappropriate changes are made by others, A must send the two commands asone atomic message.

A further consequence of using shared models is that agents must set all val-ues in a model before evaluating it in order to assert that the model correspondsto the agent’s beliefs. Unless the agent’s beliefs change dramatically betweeneach evaluation of the model, this will lead to loss in speed. As in Section 5.2.4,one must weigh the loss in speed against the savings in space. The pronounc-ers that have been implemented as part of this thesis work support compositecommands. Currently, however, they only have one model loaded into memoryat a time. Loading a model is a relatively time-consuming activity (due to diskaccess, memory allocation, etc.), so there is no good support for multiple mod-els. It could easily be added by having some sort of cache for models that havebeen loaded, and this will probably be necessary if the pronouncers are to beused in any real applications.

Chapter 6

Performance ofPronouncers

6.1 Introduction

This chapter presents quantitative performance measures for the pronouncersthat have been implemented. Although qualitative tests also need to be runbefore the usefulness of the pronounces can be determined, these results givea first indication on whether any of the pronouncers are fast enough for real-time decision making, and also if there are any notable differences between thedifferent tools. First four test cases, used in the tests, are presented. This isfollowed by a description of the specific test procedure used, and the resultsfrom the tests.

6.2 Test Cases

For all the test cases used, the decision maker is an imagined soccer playingintelligent agent. When designing the decision problems, the RoboCup domainwas kept in mind. There are two scenarios, each with one simple and oneextended model of the decision problem. The models range from very simple (4probabilities and 4 utilities) to more complex (52 probabilities and 12 utilities).It is hard to tell how realistic the models are, since they have not yet been utilizedby any soccer playing agent. The models should, nevertheless, be sufficient toget a first indication on the performance (both relative and absolute) of thetools when used with models of varying complexity.

6.2.1 Scenario 1: Agent Cannot See Ball

In the first scenario, the agent must decide whether to move or to wait. Theprecondition is that the agent does not currently see the ball. The possibleoutcomes of the decision situation is that the ball either becomes visible to theagent, or remains out of sight.

31

32 CHAPTER 6. PERFORMANCE OF PRONOUNCERS

D: move or wait?


Figure 6.1: Influence diagram for decision problem “cantsee1”.

movement of ball D: move or wait?


Figure 6.2: Influence diagram for decision problem “cantsee2”.

Simple Model

In the simple model (Figure 6.1), there is only one chance node. This is thenode representing the two possible outcomes, which contains a 2 by 2 conditionalprobability table. The utility node contains 4 values. No other information isexplicitly modeled.

Extended Model

The extended model contains more explicit information about the decision sit-uation. A node representing the agent’s beliefs about the current movement ofthe ball relative to the agent has been added (Figure 6.2). The new chance nodehas two possible states—towards agent, and away from agent—and has a linkto the node representing the outcomes of the decision problem. Consequently,this latter node now contains a 2 by 4 conditional probability table (comparedto 2 by 2 in the simple model).

6.2.2 Scenario 2: Agent Has Ball

In the second scenario, the agent possesses the ball and needs to decide whatto do with it. It can move with the ball, pass the ball, or just wait. For eachaction, there are four possible outcomes: the agent remains in possession of theball, a fellow-player gets the ball, an opponent captures the ball, or no one getsthe ball.

Simple Model

As in the first scenario, the simple model contains no explicit information aboutthe decision situation other than the possible outcomes of the entire decisionproblem, the alternatives, and the utility function (Figure 6.3). The chance node

6.3. TEST PROCEDURE 33

D: what to do with ball

ball U

Figure 6.3: Influence diagram for decision problem “hasball1”.

D: what to do with ballfellow-player

opponent ball U

Figure 6.4: Influence diagram for decision problem “hasball2”.

representing the possible outcomes contains a 4 by 3 conditional probabilitytable, and the utility node has 12 entries.

Extended Model

The extended model adds two new chance nodes—both influencing the outcomenode. The new nodes represent the agent’s beliefs about the relative position ofa fellow-player and an opponent respectively, and have two possible states each:near agent, and far from agent. The additions has as effect that the conditionalprobability table in the outcome node grows to 4 by 12, i.e. it now contains 48entries.

6.3 Test Procedure

The performance measure used in all tests was the time it takes to first set allvalues in a model, and then get an advice from the pronouncer. For each of thefour test cases given above, 1000 set/evaluate-runs were performed and timed.All the probabilities and utilities were generated by a pseudo random numbergenerator. The same seed was used for all test series so that all tools got toevaluate the same 1000 models.

6.4 Results

ActiveDATA only works on Windows platforms, and due to some problems withthe Windows version of SMILE, the SMILE pronouncer was only implementedfor Solaris. Consequently, the tests had to be run on two different platforms.Although this precludes direct comparison between ActiveDATA and SMILE,they can both be compared with Netica and thus indirectly with each other.


Netica SMILEmean std.dev. mean std.dev.

cantsee1 1.89 8.67 13.32 24.79

cantsee2 2.75 10.27 16.71 27.82

hasball1 2.64 11.29 18.63 29.07

hasball2 4.34 13.20 28.23 35.33

Table 6.1: Means and standard deviations (in milliseconds) for 1000 runs ofeach problem on Solaris.

Netica ActiveDATAmean std.dev. mean std.dev.

cantsee1 5.38 14.84 9.18 13.99

cantsee2 8.09 11.24 19.99 15.07

hasball1 8.05 12.13 28.07 15.40

hasball2 16.51 9.99 97.15 23.27

Table 6.2: Means and standard deviations (in milliseconds) for 1000 runs ofeach problem on Windows.

Results from the tests on the Solaris platform (a 167 MHz Sparc Ultra Cre-ator) are given in Table 6.1. The Netica pronouncer outperforms the SMILEpronouncer by a wide margin, which can be seen more clearly in Figure 6.5.The ratio between the average running time for the Netica pronouncer and theaverage running time for the SMILE pronouncer stays almost constant for thefour test cases. The Netica pronouncer outperforms the SMILE pronouncer bya factor 6 to 7 throughout the tests, which strongly suggests that the Neticapronouncer is the better of the two.

Also in absolute measures does the Netica pronouncer perform well. In theRoboCup domain, for example, an agent has about 30 milliseconds at its disposalfor making a decision. The Netica pronouncer keeps below 5 milliseconds forall four test cases, and seems to adhere to the requirements of this particularreal-time domain. Although the SMILE pronouncer in average keeps below 30milliseconds for all test cases, it comes very close with the most complex model.Furthermore, the standard deviations suggest that the requirement is likely notto be met in a fair amount of the runs.

On the Windows platform (a Pentium 200 MHz), the Netica pronouncerkeeps the same high performance. The ActiveDATA pronouncer is consistentlyslower than the Netica pronouncer. Results from the test runs are given inTable 6.2. It is noteworthy that while the ratio between Netica and SMILEwas practically constant, ActiveDATA seems to have a greater performancedegradation with a growth in the decision model to evaluate (Figure 6.6). If thesame ratio between Netica and SMILE is assumed on the Windows platform ason the Solaris platform, ActiveDATA should outperform SMILE at least for thefirst three test cases. For the fourth ActiveDATA and SMILE should performabout equally well, and for more complex models, SMILE is likely to outperformActiveDATA.

6.4. RESULTS 35

0

5

10

15

20

25

30

cantsee1 cantsee2 hasball1 hasball2

mill

isec

onds

NeticaSMILE

Figure 6.5: Performance on Solaris.

0

10

20

30

40

50

60

70

80

90

100

cantsee1 cantsee2 hasball1 hasball2

mill

isec

onds

NeticaActiveDATA

Figure 6.6: Performance on Windows.

Chapter 7

Discussion

7.1 Conclusions

There is no doubt that many of the currently available tools for decision analysiscan be used for assisting intelligent agents in making decisions. The three so-called pronouncers, based on some of the tools, that have been implementedas part of this work show this emphatically. A framework for implementationof pronouncers based on an existing tool for decision analysis has also beenpresented. All this constitutes by itself a significant result, but the questionthen remains whether any of the tools are useful for decision analysis in real-time. The answer depends on what is understood by the term “real-time”. Inthe RoboCup simulation league, each player has about 30 milliseconds at itsdisposal for making a decision. Other domains can inflict weaker or strongertime-constraints. The results that have been presented in this work give a firstindication on how fast current tools are. Whether this is fast enough or notmust be determined separately for each domain.

One of the tested tools—Netica—performs notably better than the othertwo. Not all the tools mentioned in Chapter 3 have been tested, and thereare reasons to believe that at least Hugin’s performance would be similar tothat of Netica—perhaps even better. Still, in absolute measures, Netica seemssufficiently fast for use in the RoboCup domain. This alone is an importantstep towards applying formal decision theory to real-time decision problems.

7.2 Future Research

In [Russell and Norvig, 1995] (p. 843) it is stated that “as AI systems move intomore complex domains, all problems will become real-time because the agentwill never have long enough to solve the decision problem exactly”. If Netica(or any of the other tools for that matter) at least to some extent can be used toperform exact decision analyses in real-time, it will be a result of great interestto the AI community. Although the results given by this work are promising,much remains before it can be accomplished. Even if a pronouncer is found tobe fast enough in evaluating a model, a good utilization of the pronouncer isnecessary. The correct models must be created, and it must be decided whento make calls to the pronouncer. These are by no means trivial tasks. This is

37

38 CHAPTER 7. DISCUSSION

a definite area for future research. Furthermore, the pronouncers implementedso far have been restricted to very basic functionality, like setting values andevaluating models. Many of the tools support additional functionality, anda future research project could be to evaluate the usefulness of some of theextended functionality suggested in Section 4.2.2.

Despite the promising results, it is too early to say if any of the tools surveyedin this thesis will prove to be useful in real-time domains. More tests with realdecision situations have to be performed. What if all the tools turn out to beinsufficient for real-time use? Then we need to ask what the reasons for thefailures are. It might be that the techniques used are inherently slow, or itmight only be that the particular implementations of the techniques have notbeen optimized for real-time applications. In the former case we must lookfor alternatives. One alternative is to use anytime algorithms. In [Horsch andPoole, 1998] an anytime algorithm for decision making under uncertainty isgiven which shows promising results for complex decision networks. Whether itis useful for smaller decision problems is not evident. Another anytime algorithmis mentioned in [Boman, 1997], but no results are given. In the latter case wecan try to build tools for decision analysis more appropriate for real-time use.Work in this direction is currently being done with the Delta tool [Boman andVerhagen, 1998].

Acknowledgements

I would first of all like to thank my advisor Magnus Boman for indispensableguidance and support. He has been available for discussions throughout thecourse of my work, and has continuously read and given insightful commentson drafts of my report. Another person I feel urged to acknowledge is JeffreyBellsey at TreeAge Software Inc. Without his devotion in helping me with allof the weird problems I ran into when trying to implement a pronouncer basedActiveDATA, I would have given up on ever getting it done. Many other peoplehave also played a part in helping me accomplish my objectives, which onlyproves that alone we are mortal but together we become eternal.

39

40 Acknowledgements

Bibliography

[Boman and Verhagen, 1998] Magnus Boman and Harko Verhagen. Social in-telligence as norm adaptation. In K. Dautenhahn and B. Edmonds, edi-tors, SAB’98 Workshop on Socially Situated Intelligence, pages 17–25, Zurich,1998. Technical Report, Centre for Policy Modelling.

[Boman et al., 1998] Magnus Boman, Jens Andreasen, Mats Danielson, Carl-Gustaf Jansson, Johan Kummeneje, Johan Sikstrom, Harko Verhagen, andHakan Younes. Ubu: Pronouncers in robocup teams. In Hiroaki Kitano,Gerald Seet, and K. Jagannathan, editors, RoboCup Workshop, PRICAI’98,pages 117–122, National University of Singapore, November 1998.

[Boman, 1997] Magnus Boman. Norms as constraints on real-time autonomousagent action. In Magnus Boman and Walter Van de Velde, editors, Multi-Agent Rationality (Proc MAAMAW’97), pages 36–44, LNAI 1237, 1997.Springer-Verlag.

[Boman, in press] Magnus Boman. Norms in artificial decision making. AI andLaw, in press.

[Bovin and Hammarberg, 1998] Mats Bovin and Marcus Hammarberg. Deci-sion support for software agents. Master’s thesis, Stockholm University, 1998.

[Chavez and Maes, 1996] Anthony Chavez and Pattie Maes. Kasbah: An agentmarketplace for buying and selling goods. In PAAM 96 Conference, pages75–90, London, April 1996. The Practical Application Company.

[Danielson, 1997] Mats Danielson. Computational Decision Analysis. PhD the-sis, Royal Institute of Technology and Stockholm University, May 1997.

[Dorais et al., 1998] Gregory A. Dorais, R. Peter Bonasso, David Kortenkamp,Barney Pell, and Debra Schreckenghost. Adjustable autonomy for human-centered autonomous systems on mars. In Mars Society Conference, August1998.

[Elofson and Robinson, 1998] Greg Elofson and William N. Robinson. Creatinga custom mass-production channel on the internet. Communications of theACM, 41(3):56–62, March 1998.

[Fishburn, 1982] Peter C. Fishburn. The Foundations of Expected Utility. D.Reidel Publishing Company, Dordrecht, Holland, 1982.

[Fou, 1997] Foundation for Intelligent Physical Agents, Geneva, Switzerland.FIPA 97 Specification Part 3: Agent Software Integration, October 1997.

[Genesereth and Ketchpel, 1994] Michael R. Genesereth and Steven P. Ketch-pel. Software agents. Communications of the ACM, 37(7):48–53,147, July1994.

41

42 BIBLIOGRAPHY

[Horsch and Poole, 1998] Michael C. Horsch and David Poole. An anytime al-gorithm for decision making under uncertainty. In 14th Conference on Un-certainty in Artificial Intelligence, Madison, Wisconsin, July 1998.

[Howard and Matheson, 1984] Ronald A. Howard and James E. Matheson. In-fluence diagrams. In R. A. Howard and J. E. Matheson, editors, Readings onthe Principles and Applications of Decision Analysis, pages 721–762. Strate-gic Decisions Group, Menlo Park, California, 1984.

[Jensen, 1996] Finn V. Jensen. An Introduction to Bayesian Networks. UCLPress, London, 1996.

[Kitano, 1998] Hiroaki Kitano. RoboCup-97: Robot Soccer World Cup I.Springer-Verlag, Berlin, 1998.

[Lauritzen and Spiegelhalter, 1988] S. L. Lauritzen and D. J. Spiegelhalter. Lo-cal computations with probabilities on graphical structures and their applica-tion to expert systems. Journal of the Royal Statistical Society, B 50(2):157–224, 1988.

[Lockheed Martin, 1996] Lockheed martin autonomous control logic toguide unmanned underwater vehicle. http://lmms.external.lmco.com/newsbureau/pressreleases/1996/9604.html, April 1996.

[Malmnas, 1994] Per-Erik Malmnas. Towards a mechanization of real life deci-sions. In Dag Prawitz and Dag Westerstahl, editors, Logic and Philosophy ofScience in Uppsala. Kluwer Academic Publishers, 1994.

[Malmnas, 1995] Per-Erik Malmnas. Methods of evaluation in supersoft decisiontheory. Unpublished manuscript, available on the WWW using URL: http://www.dsv.su.se/DECIDE, 1995.

[Pearl, 1986] Judea Pearl. Fusion, propagation, and structuring in belief net-works. Artificial Intelligence, 29:241–288, 1986.

[Raiffa, 1968] Howard Raiffa. Decision Analysis. Addison-Wesley, Reading,Mass., 1968.

[Russell and Norvig, 1995] Stuart Russell and Peter Norvig. Artificial Intelli-gence: A Modern Approach. Prentice-Hall, New Jersey, 1995.

[Saaty, 1990] Thomas L. Saaty. Multicriteria Decision Making - The AnalyticHierarchy Process, volume I of the Analytic Hierarchy Process Series. RWSPublications, Pittsburgh, PA, second edition, 1990.

[Shachter, 1986] Ross D. Shachter. Evaluating influence diagrams. OperationsResearch, 34(6):871–882, 1986.

Appendix A

Internet Resources

Noetic Systems Inc.http://www.noeticsystems.com/

Decision Systems Laboratory, University of Pittsburghhttp://www.sis.pitt.edu/~dsl/

Hugin Expert A/Shttp://www.hugin.dk/

Norsys Software Corp.http://www.norsys.com/

TreeAge Software, Inc.http://www.treeage.com/

Applied Decision Analysis, Inc.http://www.adainc.com/

Lumina Decision Systemshttp://www.lumina.com/

Vanguard Software Corp.http://www.vanguardsw.com/

DECIDE, Stockholm University/KTHhttp://www.dsv.su.se/DECIDE

Expert Choice, Inc.http://www.expertchoice.com/

InfoHarvest, Inc.http://www.infoharvest.com/

43

44 APPENDIX A. INTERNET RESOURCES

Appendix B

Glossary

ActiveX control A Dynamic Link Library (DLL) providing functionality toother programs.

Anytime Algorithm An algorithm that can be halted at any time, and willthen return a suboptimal solution to a problem. The longer it is allowedto run, the closer it gets to the optimal solution.

DDE Dynamic Data Exchange allows information to be shared or communi-cated between programs.

OLE Provides a standard conceptual framework for creating, managing, andaccessing object-based components that provide services to other objectsand applications.

OLE Automation The Microsoft standard for communication between Win-dows applications. OLE Automation allows one Windows application tocontrol the objects exposed by another application.

45

current tools for assisting intelligent agents in real-time … · 2008-08-29 · current tools for...

Documents