rational choice with deontic constraints

Canadian Journal of Philosophy

Rational Choice with Deontic ConstraintsAuthor(s): Joseph HeathSource: Canadian Journal of Philosophy, Vol. 31, No. 3 (Sep., 2001), pp. 361-388Published by: Canadian Journal of PhilosophyStable URL: http://www.jstor.org/stable/40232122 .

Accessed: 15/06/2014 14:34

Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at .http://www.jstor.org/page/info/about/policies/terms.jsp

.JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range ofcontent in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new formsof scholarship. For more information about JSTOR, please contact [email protected].

.

Canadian Journal of Philosophy is collaborating with JSTOR to digitize, preserve and extend access toCanadian Journal of Philosophy.

http://www.jstor.org

This content downloaded from 185.2.32.141 on Sun, 15 Jun 2014 14:34:06 PMAll use subject to JSTOR Terms and Conditions

http://www.jstor.org/action/showPublisher?publisherCode=cjp

http://www.jstor.org/stable/40232122?origin=JSTOR-pdf

http://www.jstor.org/page/info/about/policies/terms.jsp


CANADIAN JOURNAL OF PHILOSOPHY 361 Volume 31, Number 3, September 2001, pp. 361-388

Rational Choice with Deontic Constraints

JOSEPH HEATH Universite de Montreal Montreal, QC Canada H3C 3J7

Anyone who has ever lived with roommates understands the Hobbesian state of nature implicitly. People sharing accommodations quickly discover that buying groceries, doing the dishes, sweeping the floor, and a thousand other household tasks, are all prisoner's dilemmas waiting to happen. For instance, if food is purchased communally, it gives everyone an incentive to overconsume (because the majority of the cost of anything anyone eats is borne by the others). Individuals also have an incentive to buy expensive items that the others are unlikely to want. As a result, everyone's food bill will be higher than it would be if everyone did their own shopping. Things are not much better when it comes to other aspects of household organization. Cleaning is a common sticking point. Once there are a certain number of people living in a house, cleanliness becomes a quasi-public good. If everyone 'pitched in' to clean up, then everyone would be happier. But there is a free-rider incentive - before cleaning, it's best to wait around a bit to see if someone else will do it. As a result, the dishes will stack up in the sink, the carpet will get grungy, and so on. Things will get cleaned less frequently than anyone would like, resulting in a suboptimal outcome for all.

But anyone who has lived with roommates also knows that the 'state of nature' is not so hard to escape. People can counteract their tendency to fall into suboptimal interaction patterns in a variety of ways. The most common is to make rules. Thus the usual solution to the problems of cohabitation is to draw up a list of household chores, and then assign responsibility for them to individuals. Setting rules allows people to decide what outcome they would like to achieve, then simply instruct individuals to perform the actions needed to bring it about. Of course,



362 Joseph Heath

everyone also knows that just telling people to do things does not automatically translate into a willingness on their part to do them. There is still an incentive problem that needs to be addressed. Thus it is important, in order to understand how rules function, to understand how this underlying incentive problem is resolved.

This is where the most important conundrum in contemporary action theory has arisen. The problem stems from a tension between the motivational psychology of rule-following and the instrumental conception of rationality. Following a rule requires, to at least some degree, that the agent disregard the consequences of his actions. But to the extent that the agent does so, he is then not obviously engaged in an instrumental form of practical deliberation. Confronted with this prima facie tension, many theorists have been inclined to look harder, hoping to uncover some indirect or hidden instrumental rationale for the rule. The major reason for this has been a rather diffuse sense that expanding the conception of practical rationality to include non-instrumental reasons for action would involve introducing a range of mysterious (not to mention mathematically intractable) mental states. In this paper, I will try to show that reasons for action derived from shared rules can be integrated directly into a formal model of practical deliberation without any of these adverse consequences. The mechanism for doing so has been over- looked, I will argue, because the usual von Neumann-Morgenstern derivation of utility functions significantly obscures the motivational psychology underlying the instrumental conception of rationality. Once the basic structure of this psychology is brought back into view, it can easily be seen how rule-following considerations can be introduced into the model of deliberation without positing any dubious intentional states.

I Internal and External Control

Hobbes realized that rules were the way out of the state of nature. But he was also acutely aware of the fact that getting people to agree to a rule is not the same thing as getting them to comply with the rule. Just because everyone agrees to take turns sweeping the floor does not, in itself, mean that anyone has an incentive to sweep it. The old free-rider incentive structure remains intact, so people can be expected to 'forget' that it is their turn, or simply to defect from the agreement when the time comes to perform. Of course, this is not what usually happens. Most people can be relied upon to keep up their end of the bargain in cases like this. The question then is how this compliance is secured. How are rules rendered motivationally effective!



Rational Choice with Deontic Constraints 363

In considering this question, the first thing that most people pick up on is the fact that the rules are enforced. Among roommates, for instance, 'forgetting' to do one's chores is usually subject to negative sanction -

ranging from a rude reminder to being asked to move out. Hobbes turned this commonplace observation into a general theory about how social order is maintained. 'Covenants,' he said (somewhat dramati- cally), 'without the sword, are but words.'1 He argued that people will obey the rules only if it is in their interest to do so, and the only way to make it in their interest to do so is to take away the benefits of free-riding. This can be done by instituting a sanctioning system that punishes those who break the rules.

However, the way that Hobbes tried to cash this view out into a theory of society was remarkably unsuccessful - so unsuccessful that many subsequent theorists have taken it as evidence that his analysis of social institutions could not be correct.2 The problem is that he was not able to

explain how an authority capable of punishing defectors could arise in the state of nature, i.e. out of purely strategic interaction. The problem can be seen very clearly from the simple observation that punishing people usually involves costs for the person who is doing the punishing. The question then is what incentive anyone has to carry out the punishment. Certainly it can be somewhat uncomfortable having to complain or harass one's roommates when they don't do their chores. This gives rise to a new free-rider problem: everyone may put off punishing the

person who isn't playing by the rules, in the hope that someone else will doit.

This happens all the time. Consider the familiar situation where someone cuts into line in the grocery store, or brings too many items to the express checkout. Often everyone else in the line will say nothing, but wish that someone else would confront him. As a result, the intruder is able to get away with it, because his actions are not sufficiently harmful to any one individual to make it worthwhile to sanction him. Punishing defectors and free-riders is a kind of social service, i.e., it generates a

positive externality for all those who are harmed by the defection. But because it usually must be carried out by one person, there is a problem motivating individuals to do the punishing.3 This means that people can

1 Thomas Hobbes, Leviathan, Richard Tuck, ed. (Cambridge: Cambridge University Press 1991), 117

2 See Talcott Parsons, The Structure of Social Action, 2 vols. (New York: Free Press 1968).

3 For a model that gets around this problem, see Drew Fudenberg and Eric Maskin

'The Folk Theorem in Repeated Games with Discounting or with Incomplete



364 Joseph Heath

get stuck in a suboptimal strategic equilibrium in which they make rules, but then no one punishes those who break them.

Thus there is a problem explaining how social order could be brought about through external sanctions alone. Many social theorists have taken this as evidence that some kind of internal control system is required in order to secure rule-following. The most common view is that people are socialized in such a way that they acquire a very general disposition to comply with social expectations. Properly socialized individuals are willing to play by the rules, because failure to do so generates feelings of guilt, remorse, or regret. This occurs because individuals internalize the sanctions associated with failure to comply.

According to this view, both external control and internal restraint are essential components of social order. Children are initially taught to conform to the rules by parents and teachers, who systematically reward compliant and punish non-compliant behavior. Children come to anticipate these sanctions, thereby acquiring a disposition to follow the specific rule. But they also engage in a process of generalization, through which they acquire a more abstract disposition to secure the approval of the parent or the teacher.4 This process of generalization continues through peer-group identification, and the child acquires an increasingly generalized desire to be a 'good boy/girl/ or a 'good student/ and later to 'fit in/ or 'play by the rules/ What is important about this disposition is that it is extremely formal in structure. At the extreme, the desire to 'do the right thing' has no content without a set of rules that specifies what the right thing to do is.

Naturally, this disposition often conflicts with our other desires, and no matter how well socialized people are, no one can resist temptation all of the time. This means that external sanctions are still required among fully socialized adults in order to secure general compliance with the rules.5 This can be done by making it a specific obligation to punish those who break the rules. More commonly, however, people experience moral outrage and indignation when they witness a transgression of the rules, and so spontaneously sanction the deviant behavior. This suggests that the disposition to follow the rules acquired through socialization

Information/ Econometrica 54 (1986) 533-54. However, it is doubtful that any existing sanctioning systems have such a structure.

4 Talcott Parsons, The Social System (New York: The Free Press 1951), 210-18

5 Some of the most insightful analysis of the way that everyday social interaction is supported by sanctions can be found in Harold Garfinkel, Studies in Ethnomethodol- ogy (Cambridge: Polity Press 1984).




includes a disposition to punish those who fail to conform (or at least to ensure that they are deprived of the benefits of their actions).6 As a result, agents are punished for engaging in motivated deviance from the ac- cepted set of social rules, and by internalizing this sanction, come to acquire an increasingly general disposition to conform to and enforce the prevailing social expectations.

Because the sanctions are internalized by agents, the internal and external components of the social control system are so tightly inter- woven that it is often difficult to say where one starts and the other stops. Agents often appear to act from motives that have a hybrid character. A rule that is followed through a combination of external control and internal restraint is generally referred to as a social norm. The analysis to be developed here suggests that, in many situations, Pareto-suboptimal outcomes can be avoided through the introduction of social norms that constrain the set of actions available to participants. The question then is what is going on in people's heads when they follow these norms. If

they are just acting the way they do because of past conditioning, this

suggests that norm-conformity is irrational. Yet even when it is pointed out to people that they would be better off breaking the rules, most will continue to play along. The key question, then, is what reason reflexive agents (i.e. agents who know what they are doing, as they do it) have to conform to social norms.

II Instrumental Rationality

To ask what reason a person has for acting in a particular way is to ask for an explanation of a certain form, viz. one that explains the action in terms of some set of underlying intentional states. The instrumental

conception of rationality explains action through reference to two such states: beliefs and desires. A reductive account of social norms attempts to explain rule-following in terms of these two states, while a non-reductive account would attempt to do so by positing some new kind of state. In order to assess the plausibility of these two strategies, it is therefore

helpful to look a bit more carefully at the connection between belief-desire psychology and the instrumental view.

Part of the attraction of the instrumental conception of rationality is the simplicity of the underlying psychology. The basic idea is that agents try to achieve the outcome that they desire most, and that they use beliefs

6 See Herbert Gintis, 'Strong Reciprocity and Human Sociality/ Journal of Theoretical

Biology 206 (2000) 169-79.



366 Joseph Heath

in order to determine how best to achieve this outcome. Following through on the logic of this psychological theory, one can regard the events that are relevant to the success of a decision as being partitioned into three categories: actions, states and outcomes.7

The goal of practical deliberation is to select an action. Agents draw upon their beliefs and desires in order to do this. First, they consider the outcomes that could possibly arise as a result of the actions that are available to them. They then rank the set of outcomes according to which are the more or less desirable, and consider which actions will bring about which outcomes. In order to decide this, they must determine which of the possible states is most likely to obtain. They will then make the decision by hooking up a desire for an outcome with a belief about the state in order to recommend a particular action. What makes this account 'instrumental' is that actions are not chosen for their own sake, but rather for the outcomes they are thought likely to produce. Another way of putting this is to say that there is no class of sui generis intentional states directly associated with actions. Beliefs are used to project desires from the set of outcomes back onto the set of actions.

This psychology is intuitively very appealing, and offers a parsimonious account of practical deliberation. It is important, however, not to let this sense of obviousness obscure the fact that it is a theoretical construct. In particular, it is important not to reify the associated intentional states. The idea that people have beliefs and desires is not an empirical discov- ery, it is part of a structure of explanation and deliberation that we use for primarily practical purposes. It is important not to think that agents come equipped with these psychological states prior to any social interaction. In particular, it is important not to let the non-cognitivism associated with the concept of 'desire' lead one into thinking of these as a set of primitive physiological drives. The type of intentional state at issue here is generally regarded as a propositional attitude, and so it cannot be directly associated with any underlying somatic condition without subscribing to the 'myth of the given.'8

7 See Leonard J. Savage, The Foundations of Statistics (New York: Dover Publications 1972); and Richard Jeffrey, The Logic of Decision (Chicago: University of Chicago Press 1983), 1. Jeffrey differs from Savage in that he allows the probability of states to be affected by the actions chosen. I follow Savage here, primarily because his framework is the one adopted by most game theorists. See Roger Myerson, Game

Theory (Cambridge, MA: Harvard University Press 1991), 5-12.

8 For the full argument to this effect, see Joseph Heath, Toundationalism and Practical Reason/ Mind 106 (1997) 451-73. The term 'myth of the given' is from Wilfrid Sellars, Empiricism and the Philosophy of Mind (Cambridge, MA: Harvard University Press




In order to keep this straight, it is helpful to think of these intentional states as deliberative constraints. Agents use these constraints as a kind of filter to throw out possible options. Following Georg Henrik von Wright, we can think of the agent as starting out with a decision tree that contains the full set of action-state-outcome permutations, then pruning this tree by eliminating impossible combinations of events and undesirable outcomes, using these constraints.9 Within this framework, beliefs and desires can be thought of as doxastic and desiderative constraints, respectively.

This kind of decision procedure is represented in Figure 1. The diagram shows a decision problem with three actions available to the agent \d\, 02, 03}, three possible states {si, s2, s3} and three possible outcomes [o\, 02, 03}. Things start off with nature 'choosing' which state will obtain. This puts the agent at one of her three decision nodes. However, without knowing which node she is at, the agent is unable to determine which action will produce which outcome. For instance, if the state is S\ (i.e. nature has 'chosen' branch Si), then action a\ will produce outcome o\. However, if the state is s2, choosing a\ will produce o2, and if the state is S3, fli will produce 03.

The agent's decision procedure can then be regarded as a process of

pruning the decision tree. The agent's beliefs and desires can be introduced into the diagram simply by cutting off some branches. Suppose that the agent is able to ascertain which state obtains, and so believes, for

example, that nature has 'chosen' s2. This allows him to cut the S\ and s3 branches off at nature's node. What is left of the decision tree after this 'doxastic pruning' is shown in Figure 2.

Once the problem is simplified in this way, the agent knows which node he is at, and so knows which actions will produce which outcome. As a result, he can simply choose the action that will lead to the outcome that he most desires. This can be thought of as a desiderative pruning, in which each branch leading to an outcome other than the one most desired is cut off. What is left after this final pruning is a single action, which the agent can then proceed to perform.

The situation is more complicated when the agent is less than certain about which state will obtain. In this case, she can only assign prob-

1998). On the same point, see Michael Smith, The Moral Problem (Oxford: Blackwell

1994), 107.

9 Georg Henrik von Wright, An Essay on Deontic Logic and the General Theory of Action

(Amsterdam: North-Holland Publishing 1968), 58-68. 'Impossible' is being used here in the sense of 'false at all possible worlds physically accessible to our own, i.e.

having the same laws of nature/



368 Joseph Heath

Figure 1. Decision tree

agent at^^^^

M^ a o3

°^3 /

T^°' >^ nature / >^

abilities to the occurrence of various states, and so will not know with certainty which action will bring about the desired outcome. As a result, she cannot simply 'prune' branches off the tree. Instead, she must select the action that gives her the best chance of getting her favored outcome. But even then, things are not simple. She may face a situation in which she must choose between an action that gives her a reasonable chance of getting her favored outcome, but also some chance of getting an outcome that is disastrous for her, and an action that gives her a guarantee of at least a mediocre outcome. In order to choose between these two options,

Figure 2. Doxastically pruned tree

nature agent a<^^

O •^- a °'




she will have to consider how much she really wants the best outcome, and whether pursuing it instead of the mediocre outcome is worth the risk of disaster. So just as beliefs will have to be given probabilities, her desires will have to be assigned intensities.

Figure 3 shows a decision tree for the agent that includes the probability with which the agent believes that each state will obtain between angle brackets <x>, and the intensity with which the agent desires that each outcome obtain between parentheses (x). Since one of the three states must obtain, the probabilities assigned to nature's move must add up to 1. On the other hand, whether the numbers used to represent the intensity of the agent's desires add up to 1 is not important, since the only thing that matters is the strength of these desires relative to one another. For simplicity, the most preferred outcome is assigned an intensity of 10, and the least preferred outcome a 0.

Figure 3. Decision tree with uncertainty



370 Joseph Heath

In order to decide what to do, the agent must calculate the expected utility associated with each action. This can be done by multiplying in the usual way:

U(fli) = (.2 * 10) + (.3 * 4) + (.5 * 0) = 3.2

U(a2) = (.2 * 0) + (.3 * 10) + (.5 * 4) = 5

U(«3) = (.2 * 4) + (.3 * 0) + (.5 * 10) = 5.8

Now, instead of simply selecting the action that produces the outcome that the agent most desires, she selects the action with the highest expected utility. The agent is therefore said to maximize expected utility. This shows how the conception of practical rationality as utility-maxi- mization follows very naturally from the application of belief-desire motivational psychology to probabilistic choice problems. (This is ob- scured by the usual von-Neumann-Morgenstern procedure, which places only an ordinal preference relation on outcomes, then uses a set of axiomatic constraints on choice behavior to impose a cardinal utility ranking on actions. The strategy of assigning intensities to desires is, I believe, more parsimonious, but in any case keeps the philosophical motivation for the theory closer to the surface. I have chosen to represent the decision problem using the so-called 'Savage trichotomy' of actions- states-outcomes for the same reason.)

Ill Social Norms

When the instrumental conception of rationality is mapped out in this fashion, it becomes much easier to see where the tension with rule-fol-

lowing originates. The key characteristic of rules is that they are directly associated with actions. This is the difference in structure that Kant tried to capture with his distinction between categorical and hypothetical imperatives - rules say 'do that/ not 'if you want this, do that.' (A chore list, for example, assigns specific actions to specific individuals, it does not merely recommend outcomes.)

This observation suggests a very direct way of integrating norms into the decision framework. Desires are intentional states associated with outcomes, beliefs are intentional states associated with states. Norms, or

principles, can be treated simply as intentional states associated directly with actions.10 According to this view, social norms function as deontic

10 One might be tempted to reserve the term 'principle' for the subjective intentional




constraints that are imposed in a manner quite similar to that of their doxastic and desiderative cousins. We start out by looking at the options that are actually available to us. This amounts to a doxastic pruning of the decision tree. We then consider which among these options are normatively permissible, i.e. we eliminate all the options that violate the prevailing set of social norms. This amounts to a deontic pruning of the decision tree. Once this is finished, we can select the action that gives us the most desired outcome (the final, desiderative pruning).

The analogy between doxastic and deontic constraints can be elaborated in the following way. Under conditions of epistemic and moral certainty, we can then think of the agent as qualifying a set of intentional states according to a set of modalities:11

Doxastic modalities: Deontic modalities: M= possible P = permitted N= necessary O = obligatory 1= impossible F = forbidden

In practical deliberation the agent begins by determining which of her actions are possible ways of attaining a desired outcome. The agent then decides which of her actions represent permissible ways of attaining that same outcome. She then makes a final decision among the options that remain as to which is the most desirable. The procedure is shown dia-

grammatically in Figure 4. (It does not matter in what sequence these constraints are applied. But since it is a waste of time to normatively regulate impossible states of affairs, there is no real point to applying the deontic constraints first.)

The style of reasoning illustrated in Figure 4 is again somewhat simplified. It assumes that states are known with certainty, and that actions are absolutely prohibited. But just as states can be known with

state, and use the term 'norm' only for reference to shared rules. My reasons for not

wanting to draw such a distinction stem from a set of background commitments that cannot be fully elaborated here. See Joseph Heath, Communicative Action and Rational Choice (Cambridge, MA: The MIT Press 2001); and Robert Brandom, Making it Explicit (Cambridge, MA: Harvard University Press 1994). Readers uncomfortable with this usage should feel free to substitute 'principle' for 'norm' throughout the remainder of this section.

11 As has been widely noted, the primary doxastic and deontic modalities are interde- finable in exactly the same way: Mp =df -N->p =df -.Ip, and Pp =df -.O-ip =# -.Fp. In words: it is possible that p means that it is not necessary that not p, and that it is not

impossible that p. In the same way, it is permissible to do p means that it is not

obligatory to not do p, and that it is not forbidden to do p.



372 Joseph Heath

Figure 4. Doxastic and deontic pruning




varying probabilities, actions can be more or less frowned upon. Break-

ing a promise, for example, is contrary to duty, but the norm can be overridden in some circumstances. For this reason, permissible actions can be ranked as more or less appropriate, in the same way that possible states are ranked as more or less probable. A forbidden action, according to this view, could be represented as having an appropriateness of, say, zero, and an obligatory action as having some high positive value.12 All other actions can be assigned some intermediate value that specifies their

appropriateness, relative to the others. These values can then be inte-

grated into the agent's utility function, so that they act as a kind of 'filter' that reduces the value of normatively inappropriate actions relative to more appropriate ones. These normative considerations can be added to the decision tree in the way shown in Figure 5. (Appropriateness factors are shown between square brackets [x], given here on a scale between 0 and 10.)

We now have a complete decision tree: beliefs associated with states, norms associated with actions, and desires associated with outcomes. The norms can then be integrated into the agent's decision function. One

way to do so would simply be to add the appropriateness of each action to the expected utility of that action:

V(fli) = 10 + Ufa) = 10 + [(.2 * 10) + (.3 * 4) + (.5 * 0)] = 13.2

Vfa) = 6 + Ufa) = 6 + [(.2 * 0) + (.3 * 10) + (.5 * 4)] = 11

Vfa) = 0 + Ufa) = 0 + [(.2 * 4) + (.3 * 0) + (.5 * 10)] = 5.8

(So 03, despite being utility-maximizing, is bumped down through the introduction of deontic constraints.) To avoid confusion, we can call the

utility function that results from the introduction of normative constraints the agent's value function. It should be noted that the appropriateness of norms and the intensity of desires are both represented here on a scale of 0 to 10. Just as in the case of desires, it does not matter

precisely what scale is used to represent the appropriateness of norms. What matters is simply that the numbers reflect the significance of the norms relative to one another. But if the appropriateness factors range from

12 The precise scale that is used makes no difference. One might be tempted to

represent forbidden actions as having negative appropriateness. Similarly, one often represents outcomes as having negative utility. All of these functions remain

unique through any positive linear transformation. The only thing that matters is that the scale used to represent the desirability of outcomes be correctly calibrated to the scale used to represent the appropriateness of actions. More on this below.



374 Joseph Heath

Figure 5. Decision tree with norms

0 to 100, and desires only from 0 to 10, this will mean that the agent considers the most important normative obligation to be ten times more important than her most preferred outcome. This may be an accurate reflection of how some people weigh these matters, but the precise balance between the two will presumably be specific to the agent in question. As a result, the agent's value function should make explicit the way the agent weighs normative against instrumental considerations. This can be done by using the same scale to represent both appropriateness and desirability, but then multiplying the appropriateness 'score' by some value k that reflects the weight the agent assigns these normative considerations relative to the instrumental ones.13 Thus, if A(a) gives the appropriateness of some action a, and U(a) gives its expected utility, the expected value of a is given by:

V(a) = kA(a) + U(a)




The k factor can be referred to as the agent's fundamental choice disposition. This fundamental choice disposition is precisely the sort of generalized 'norm-conf ormative' attitude that sociological theorist have long thought to be the central product of primary socialization.

IV FAQ

This method of constructing a value function is likely to generate a number of objections. My primary objective in this paper is simply to develop a clear presentation of a new strategy for integrating deontic considerations into the instrumental model of rationality. There is no way to adequately address all of the technical and philosophical issues that this strategy raises. Thus, in what follows, I hope to do little more than show the general direction that I would take in responding to some of the more important objections:

How can you add normative considerations to the agent's utility function, given that the units used to represent utility have been chosen arbitrarily?

It is a well-known feature of von Neumann-Morgenstern style utility functions that they do not support meaningful interpersonal compari- sons of utility. Because the 'scale' is arbitrarily chosen, one cannot simply add one agent's utility to that of the next in order to determine the social value of an outcome. One might mistakenly conclude, on this basis, that it is illegitimate to add anything at all to the agent's utility function, including a set of weighted appropriateness factors.

The reason that one can make such an addition is that the strength of the agent's preference between an outcome and an action can be ascer- tained in more-or-less the same way as her preferences between two outcomes. In the case of outcomes, one offers the agent a hypothetical choice between a particular outcome and a lottery between the most-preferred and the least-preferred outcomes. By fiddling around with the

probabilities assigned to the two outcomes in the lottery, one should eventually be able to find a point at which the agent becomes indifferent between the outcome and the lottery. In the case of actions, one can do

very much the same thing. One can offer the agent another hypothetical choice, this time trying to find a point at which the agent becomes

13 This of course raises a number of technical and methodological questions that form the natural subject of future research. Here I deal only with the most abstract theoretical features of the model.



376 Joseph Heath

indifferent between some particular action and, for example, some other less appropriate action that has a higher utility payoff. By determining how much utility must be added in order to compensate for a given decrease in appropriateness, it will be possible to determine how much that agent considers utility to be worth relative to appropriateness.

The procedure is one that should be familiar to us all. The way to ascertain someone's moral character is to offer her some temptation, and see how much it takes to weaken her resolve. Of course, there may be some people who assign absolute priority to their moral obligations - so that the wrongness of the action completely overshadows the desirability of the outcome. But this does not cause any special problems for the theory being presented here, since people with lexical preference orderings over outcomes pose precisely the same problem for standard decision theory.

Why add normative considerations to the agent's utility function? Why not

multiply the two, or do something else entirely?

There is no high-powered philosophical rationale for this decision. Whatever intuitive rationale there may be for adding appropriateness and expected utility is based on an analogy between the agent's fundamental choice disposition and the agent's discount rate.14 In repeated games, the expected utility of an action is determined by summing the expected utility of the action at each stage of the game. The agent's utility function for each stage of the game is perfectly unique, and these functions are, strictly speaking, non-comparable (since events occurring at different times are simply different events). They are rendered com- mensurable by introducing a discount rate, which specifies how to measure 'utility at time x' in 'utility at time y' units. The 'fundamental choice disposition' introduced above functions in much the same way as the discount rate, except that instead of mediating the comparison between events occurring at different times, it mediates the comparison between events that are classified as actions and events that are classified as outcomes. Since these are all just events - events that the agent cares about to varying degrees - the way to calculate the value of some set of

14 It is perhaps worth noting that, whatever the merits of addition, multiplication is completely implausible so long as the endpoints of the scale used to measure

appropriateness are chosen arbitrarily. If one were to multiply appropriateness by desirability, the difference between assigning an action an appropriateness of 0 and

assigning it an appropriateness of 1 would be enormous.




these events should involve adding up how much the agent cares about each event in the set.

In any case, the assumptions underlying both the discount rate and the fundamental choice disposition may remain controversial. Not all game theorists accept the idea that agents should maximize the sum of time-discounted expected utility either.15 The important point is simply that norms can been introduced into the decision procedure without introducing any technical apparatus that is not already used in standard game theory, and that the proposal to add appropriateness to expected utility is no more controversial than the existing game-theoretic strategy for handling repeated games.

All of this seems to hang upon the distinction between acts and outcomes. But isn't that kind of an ad hoc distinction?

This question gets us a little ahead of ourselves. The distinction between acts and outcomes is relatively unimportant in non-social choice problems. Thus the type of value function that I have been developing only has interesting consequences when we look at social interaction, in the next section.

However, a few words on the subject may be in order. If I choose to

go to my favorite cafe, there is no reason why 'going to my favorite cafe' cannot be described as an action. However, one could just as easily describe 'moving my legs' as the action, and 'going to my favorite cafe' as the outcome. Since nothing is ever entirely under my control, pretty much any action can be redescribed as some kind of outcome. But things become more interesting when another rational agent is introduced into the frame of reference. In contexts of 'interdependent choice/ the outcome is determined by the actions of two or more agents. Thus for each

agent, the distinction between action and outcome is determined by the

point at which the other agent's choice begins to have an effect. Within a game-theoretic framework, 'meeting my friend at a cafe' is necessarily an outcome because it depends upon what both of us choose. 'Going to

my favorite cafe' is an action, because I can do that regardless of what

my friend does. Thus in social interaction the act/outcome distinction must be drawn quite strictly, through triangulation of the causal chains initiated by the players. Treating the action as a part of the consequence makes it seems as though everything that the agent cares about in the

15 For overview of different proposals, see Myerson, Game Theory, 310-16.



378 Joseph Heath

interaction hinges upon the other agent's decision, when in fact this will often not be the case.

Isn't this value function just equivalent to the utility function that would be generated through the von Neumann-Morgenstern procedure?

Any agent who allows a direct preference for an action to enter into her deliberations will violate the von Neumann-Morgenstern axioms, and so will appear to be acting 'irrationally' in their narrow sense of the term. The standard von-Neumann-Morgenstern approach to utility simply defines actions as lotteries over possible outcomes, then stipulates that the agent must be indifferent between any two actions that generate the same probability distribution over outcomes (this is the idea underlying the 'reduction of compound lotteries' axiom).16 This definition of action, combined with the stipulation, essentially prohibits agents from liking any action for its intrinsic merits. As a result, norms cannot be accom- modated within the standard von Neumann-Morgenstern representation - unless of course the action is treated as a component of the outcome. We will see in the following section why this is not an attractive way of modeling the role that norms play in practical deliberation.

Okay, so these value functions are not the same as standard von-Neumann- Morgenstern utility functions. But what about a Jeffrey utility function? Isn't V(a) equivalent to des(a)?

The proposal that I have been making here bears some similarity to Richard Jeffrey's version of decision theory. Jeffrey also departs from a narrowly instrumental conception of rationality by allowing agents to associate desirabilities with both actions and outcomes.17 1 differ from Jeffrey in one major respect, which becomes relevant only in a game- theoretic context. I think it is very important to keep distinct the contri- bution that norms of action and desires for outcomes make to deliberation, because some social behavior becomes intelligible only when these two are treated separately. In a multiple-equilibrium game, agents are unable to assign probabilities to a certain range of states, because they are unable to anticipate which of the many equilibrium strategies their opponents will play. As a result, they are unable to

16 The exposition most people look to is R. Duncan Luce and Howard Raiffa, Games and Decisions (New York: Wiley 1957).

17 Jeffrey, The Logic of Decision, 82-4




'project' their desire-based ranking of outcomes back onto the set of available actions. In this situation, the instrumental component of their motive is effectively neutralized. But because norms are directly associated with particular actions, agents in strategically indeterminate choice situations may still be able to engage in a meaningful pruning of their decision trees. Unable to calculate the full consequences of their actions, they may choose to do what is 'appropriate' under the circumstances.

This goes a long way toward explaining how social norms can reduce the complexity and indeterminacy of social interactions.18 But the insight is in danger of being lost when the full structure of the agent's decision procedure is ignored. In particular, it is essential to this explanatory strategy that norms provide a direct ranking of actions, without any dependence upon beliefs about what the other agent intends to do. This is what allows 'normative reasoning' to escape the regress of anticipations that gives instrumental rationality its characteristic strategic form in contexts of social interaction. Thus any attempt to merge norms and desires into a unified utility function results in a loss of information that in turn renders certain aspects of social interaction needlessly obscure. Jeffrey's utility functions would therefore need to be decomposed into these two components in order to be put to use in game-theoretic contexts.

Isn't this claim that both norms and desires motivate action inconsistent with the doctrine of revealed preference?

Some theorists have claimed that agents will always act in accordance with their von Neumann-Morgenstern utility functions, because the latter is simply a mathematical short-hand used to represent how they actually act.19 Preferences are 'read off of agent behavior - hence the idea that preference is 'revealed' through choice. If this were true, then the suggestion that actions are produced through the complex interaction of two preference orderings - desires and norms - would be either

unsupported or underdetermined by behavioral evidence. This would be a problem, except that the doctrine of revealed prefer-

ence is an obvious absurdity. First of all, no one could get a von Neu-

18 In particular, it is able to provide deliberative microfoundations for a Schelling-style 'focal point' solution to the equilibrium-selection problem. For further discussion, see Joseph Heath, "The Structure of Normative Control/ Law and Philosophy 17 (1998) 419-42.

19 See Ken Binmore, Playing Fair: Game Theory and the Social Contract Vol. 1 . (Cambridge, MA: The MIT Press 1994), 104-7.



380 Joseph Heath

mann-Morgenstern utility function by observing behavior, since these functions can only be constructed by incorporating information about how the agent responds to hypothetical wagers (i.e. how the agent would act, not how she does act). Thus the procedure used to elicit the information needed about the agent's preferences is designed mainly to prove a conceptual point. Second, the idea that even ordinal preference is revealed through choice is an illusion. As Donald Davidson has observed, any action can be made consistent with any of set of preferences, simply by varying the beliefs that are ascribed to the agent.20 (I may drink the coffee because I like coffee, but I may drink it because I like tea, and think that it is tea.) Thus the ascription of a single preference ordering to the agent will always be massively underdetermined by behavioral evidence - adding a second ordering to the mix doesn't make it much worse. Third, most actions taken in a social context do not reveal anything about preference, for obvious reasons related to the indeterminacy of strategic interaction, but also because game-theoretic models presup- pose common knowledge of preferences (i.e. prior to the interaction). Finally, the doctrine of revealed preference commits the theorist to the absurd claim that agents have no introspective access to their own preferences, but must explain their actions through extrapolation from their own behavior. Thus it eliminates all first-person authority in desire reports.

Where do these norms come from?

Nothing has been said here about where the content of the deontic constraints that the agent imposes come from, just as standard decision theory says nothing about where beliefs and desires come from. The claim here is simply that these three types of constraints function as selection criteria. The content of these constraints is taken to be determined exogenously. Of course, use of the term 'norm' or 'deontic' to describe constraints directly associated with actions might be seen to prejudge the question of where they come from. (The term 'categorical preferences' might be more neutral, but it is also somewhat clunkier.) I would argue that agents pick up norms from a shared fund of culturally transmitted rules of conduct, in much the same way that they acquire many of their beliefs. However, it must be acknowledged that an agent might choose to perform an action simply because she has an urge to do

20 Donald Davidson, 'The Structure and Content of Truth/ Journal of Philosophy 87 (1990) 279-328. See also Daniel M. Hausman, 'Revealed Preference, Belief, and Game

Theory/ Economics and Philosophy 16 (2000) 99-115.




it, and not because some shared rule of conduct prescribes it. This kind of motive would still fall under the category of 'norm/ perhaps stretch- ing the use of the term. But this is not much different from standard decision theory, where it is assumed that the agent's preference ordering over outcomes will combine both simple desires and more sophisticated values or conceptions of the good acquired from the social milieu. If the agent's set of desiderative constraints can incorporate elements of shared culture, then the agent's deontic constraints may also incorporate purely idiosyncratic principles of action. The details of this interconnection, however, are left to a secondary level of theory, which would supple- ment the theory of practical rationality with an analysis of the mechanism through which agents develop or adopt doxastic, desiderative, and deontic constraints.21

If norms can come from anywhere, in what sense is this a model of rational action?

The reason that the model of action outlined above is a theory of rational action is that it provides an intentional mechanism through which social norms can be transmitted from the level of culture to that of action. A

strictly 'behaviorist' theory, which would explain normative constraint

purely through conditioning or socialization, has the effect of bypassing the agent's consciousness, and so fails to account for the reflexive character of much norm-conformative action.22 According to the model

presented here, agents do not follow norms simply because they have been socialized to do so. They have been socialized to give deontic constraints a certain deliberative weight, but the specific norms that they follow are applied in a rational and reflexive manner. In the same way, agents must be socialized to respect doxastic constraints and not engage in 'wishful thinking,' but this does not mean that they are socialized to follow any specific set of beliefs. Thus the model of action is rational because it provides a cognitive mechanism through which agents apply

21 An example of such a theory for deontic constraints would be the conception of

'practical discourse' developed by Jiirgen Habermas, Moral Consciousness and Com- municative Action (Cambridge, MA: The MIT Press 1991).

22 Here I have in mind so-called 'cultural dope' theories of social integration. The discussion in John Heritage, Garkinkel and Ethnomethodology (Cambridge: Polity 1983) provides a useful introduction. See also Dennis Wrong, 'The Oversocialized

Conception of Man in Modern Sociology,' American Sociological Review 26 (1961) 183-93. Much of this discussion is unfortunately marred by a deeply uncharitable

reading of Talcott Parsons.



382 Joseph Heath

norms, not because it provides a rational vindication of the more general structure of practical deliberation.

V Social Integration

The model developed so far shows how agents incorporate deontic constraints into their practical deliberations. The next step is to general- ize this model to give an account of social interaction. Since there is no room to work up a full account here, I will simply highlight two ways in which the model of action developed here differs from standard game theory.

Figure 6 below shows a classic social interaction pattern. Instead of having 'nature' choose the state and the 'agent' choose an action (as in Figure 1), here each player selects an action. The action that each one chooses will be a state for the other. This means that the probability associated with each state is part of the solution, not the decision problem. As a result, beliefs are not represented in this diagram, because they cannot be fixed exogenously. However, as before, each action has an associated appropriateness factor, shown in square brackets. And the outcomes are given as a vector of desirabilities (showing how much player 1 desires that outcome, how much player 2 desires it, etc.).

In order to resolve the decision problem, each player must calculate the expected value of each available action. This can be done by starting with the desirability of each outcome, and then factoring in the effects of one's norms (how appropriate are the actions leading to it?) and beliefs

Figure 6. Cooperative norm

.(3,3)

player 2 \^

C[0]/ ! <^\^

player 1 cT i

\- j ^(4,1)

^(2,2)




(what are the chances of getting it?). However, since neither player can decide on a set of beliefs before determining which action the other will perform, and since neither can decide what to do before deciding on a set of beliefs, a classic regress of anticipations arises.

In this problem, the regress can be resolved through dominance reasoning. It is important to observe, however, that the solution differs depending upon what sort of weight the two players assign to normative reasons for action. In this interaction, both players subscribe to a norm that designates d as an inappropriate action. Suppose, however, that both players assign normative reasons for action no deliberative weight (i.e. ki = k2 = 0). In this case, the value of each action will be equivalent to its desirability. The action d then strongly dominates c for both players, since it gives the player who chooses it an outcome of greater value regardless of what the other player does. As a result, each player has reason to believe that the other will select d, and is therefore able to calculate the expected value of each available action. This leads each

player to select d, resulting in the (2,2) outcome. (Thus the interaction has the structure of a standard prisoner's dilemma.)

The picture changes if players assign greater weight to normative considerations. If k\ = k2 = 1, then dominance reasoning generates a different equilibrium. The cost of performing the inappropriate action of

defecting is now sufficient to outweigh the desirability of the associated outcomes, and so both players are better off cooperating, regardless of what the other does.23 Again, this gives them each grounds to believe that the other will choose c, and so grounds for calculating the expected value of each action. Each will then choose c, resulting in an outcome with a value of 3 for both players.

This example shows how normative constraint can provide a 'moral' solution to the prisoner's dilemma. Players here rely upon their internal control system (represented by k) in order to avoid a suboptimal interaction pattern. Examples of norms that secure cooperation in this way

23 This proposal bears a certain resemblance to a proposal made by Robert Nozick in The Nature of Rationality (Princeton: Princeton University Press 1993), 56. However, Nozick retains an essentially consequentialist framework, according to which actions can acquire value either through probabilistic or symbolic connections to other events. He rejects the idea that an action might have 'the usual kind of utility' directly associated with it. In my view, this privileging of symbolic associations is unmotivated. Actions are just as likely to have 'symbolic value' as outcomes - all of this will be reflected in the agent's preferences. Thus the entire question of

symbolic value belongs in the portion of the theory that explains where norms and desires come from, and thus not in the theory of practical rationality strictly construed.



384 Joseph Heath

abound in everyday life. The norm that prohibits cutting into line is a typical example. Waiting to get through a grocery checkout, or a fire exit, is a classic multi-player prisoner's dilemma - everyone wants to rush to the front, but if everyone does this, the disorder will result in everyone getting through more slowly. The queuing norm corrects this simply by proscribing going through before those who arrived earlier. The norm doesn't change how much people want to escape the building; it simply imposes constraints on the range of actions that they want to perform in order to achieve that goal.

This example shows how norms can secure cooperation and help agents to eliminate collective action problems. This is the core component of the process that sociologists refer to as 'social integration.' But another circumstance in which norms may prove to be helpful is when purely instrumental interaction generates multiple equilibria. Consider the interaction shown in Figure 7.

This is a classic coordination problem. Two of the outcomes are equally desirable, yet different combinations of actions are needed in order to achieve either one. What each player wants to do therefore depends entirely upon what the other intends to do. But how is either to decide? The introduction of a norm can resolve this problem.

Dominance reasoning does not pick out a unique outcome here. Notice also that simply writing the desirability of actions W\ and gi into the payoffs do not help matters any - in fact, it confuses things. The fact remains that if one player intends to choose g, then the other is best off playing w, and vice versa. As a result, {w\,gi) and (gi, Wi) are both equilibrium strategy profiles of this game, regardless of whether (w\fg2)

Figure 7. Coordination Norm

.(0,0)

player 2 J^^

y/ \ ^(2,2) player 1 cT i

^(0,0)




is treated as having a payoff of (2,2) or (3,3). If, however, we keep in mind that both players care about both their actions and the set of possible outcomes, a solution become available. If, due to the regress of strategic expectations, player 1 is unable to determine whether player 2 will choose gi or wi, then he will be unable to calculate the expected value of any of his actions. More specifically, he will not be able to calculate the value of U(fl) in V(a) = kA(a) + U(a) for any of his actions. One response, then, would be to parametrize U(a), i.e. to assign it some unknown value x for all actions. Player 1 can then represent his options as:

V(gi) = kA(gl) + x = x

V(wi) = JfcA(wi) + x = 1 + x

This makes it clear that in the absence of any settled beliefs about what player 2 will do, action w\ has greater expected value than g\. This gives player 1 reason to choose wh and player 2 reason to believe that player 1 will choose W\. This in turn allows player 2 to calculate the expected utility associated with both of her actions, leading her to choose gi. Reasoning from this gives player 1 an instrumental reason to choose w\, and so makes (w\, gi) the only equilibrium value of this game. (A solution that uses normative dominance in this way can be referred to as a normative expectations equilibrium.2* The norm can be thought of as gen- erating a 'focal point' effect.)

This example presents an interaction in which the normative supe- riority of one action makes the associated outcome focal, and therefore the equilibrium of the interaction. Again, there are many real-life exam-

ples of norms of this type. Many of the norms that specify gender roles, for instance, have a coordinating function. The example given in Figure 7 can be used to model an interaction in which two people arrive

simultaneously at a door that only one can pass through at a time. If both

try to go forward, or both wait, it will be inconvenient for them both. The traditional gender norm specifies that if one agent is male and the other female, the man should allow the woman to pass through first. (Again, it is highly misleading to think that this norm makes the man desire the ultimate outcome - being in the building - any less. It governs his actions.)

The examples given in Figures 6 and 7 show how norms can be used to secure cooperation and coordination respectively. In both cases, the

24 I am borrowing this term from Robert Sugden, who uses it in a somewhat different sense.



386 Joseph Heath

introduction of a norm enables agents to implement an interaction pattern that straightforward instrumental reasoning would fail to achieve.

VI Trust and Sanctions

The analysis developed above provides a formal model of action that can explain how, in principle, agents are able to use rules to work themselves out of the 'state of nature.' In practice, however, things are a bit more complicated. In order for the sort of equilibrium solutions outlined here to work, players must start out with a lot of knowledge about each other. In standard game theory, it is assumed that all players' beliefs and desires are common knowledge at the start of the game (and in repeated games, that all players' discount rates are common knowledge). If one simply adds common knowledge of norms and of fundamental dispositions to this, then interactions with value functions can be handled using all of the standard game-theoretic solution concepts (Nash equilibrium, subgame perfection, etc.), and refined with the 'normative expectations' mechanism. Naturally, the specific solutions will be different from those that would be picked out using standard utility functions. The point is simply that the same apparatus can be used.

Cases where there is some uncertainty about any of these intentional states or dispositions can be handled using a standard 'Bayesian' transformation of the game.25 For example, suppose that player 1 must interact in a prisoner's dilemma-type interaction with an agent who is known only to have a k of either 0 or 1 . Player 1 can start out by assigning a prior probability to each value of k (based, perhaps, on the number of agents with k values of 0 and of 1 in the population). The interaction can then be modelled as one in which nature moves first, 'deciding' whether k2 is equal to 0 or 1 on the basis of this prior probability. This allows player 1 to calculate the expected utility of each of his actions, by developing probabilistic expectations about how player 2 will behave. For instance, if he decides that k2 = 1 with probability .5, this suggests that there is a .5 chance player 2 will choose d2, and a .5 chance he will choose c2. Player 1 can then calculate the expected value of his own actions accordingly.

If the players are involved in a sequence of interactions (i.e. a multi- stage game), this framework can be used to model the development of trust. Trust can be defined as simply a state in which everyone believes

25 Eric Rasmusen, Games and Information, 2nd. ed. (Oxford: Blackwell 1989), 48-57.




that everyone has a broadly norm-compliant disposition (i.e. assigns a sufficiently high value to k that their normative constraints outweigh their desires).26 The best way to achieve this state is if the parties are able to somehow reveal to each other that they have such a disposition. This can be done, for example, by establishing a known history of having foregone opportunities for desirable defection. For instance, if player 2 does select c2, this gives player 1 reason to 'update his prior/ i.e. to reevaluate his belief that the probability of k2 = 1 is .5, in light of this new information. Agents who start out without knowledge of each others' fundamental dispositions may therefore be able to reveal these to one another by performing actions that provide evidence for specific hypotheses.

This analysis suggests that social integration can be achieved through internal control alone under specific information conditions, i.e. only among agents who trust one another, or who interact in a 'high-trust' social milieu. This in turn provides an explanation for the role of external control - sanctions - in stabilizing cooperative interactions. In the absence of trust, people can generate confidence that they will comply with the prevailing set of norms by agreeing to be punished in the event that they defect. This is the central insight in Hobbes's view. People want to be able to trust one another, because they can benefit from avoiding collective action problems. This makes it in their interest to submit to some authority who will sanction them in the event of non-compliance. Hobbes's mistake was simply to think that this kind of a sanctioning system rendered trust superfluous. Trust is still required in order to maintain the integrity of the punishment mechanism. Thus if two individuals do not trust one another, but both trust some third party, then

they may arrange with that party to impose the sanctions. In this way, sanctions can be used to extend cooperative behavior beyond the imme- diate circle of people who trust one another, by giving individuals the confidence to enter into cooperative arrangements that they might oth- erwise avoid for fear of being 'suckered.'

These observations allow us to specify more clearly the role that sanctions play in social interaction. People are punished, not because

26 Here I am using the term trust in the very general sense in which it is used in

sociological theory (and especially the literature on social capital). Consider Francis

Fukuyama: Trust is the expectation that arises within a community of regular, honest, and cooperative behavior based on commonly shared norms, on the part of other members of the community' (Trust [London: Penguin 1995], 26). Naturally, more specific trust relationships can develop as agents acquire greater knowledge of the particular norms that individuals accept.



388 Joseph Heath

they have performed the wrong action, but because they have violated the trust of others. This trust is grounded in the expectation that agents have a fundamental disposition to accord deontic constraints a certain weight against desiderative constraints in their practical deliberations. It is this disposition that, at root, enables them to adopt rules to govern their conduct, which in turn allow them to avoid suboptimal interaction patterns. When the distinction between deontic, doxastic, and desiderative constraints is sharply drawn, the way that these rules are integrated into the agent's deliberations, along with the role of internal and external control in social integration, can be clearly specified.

VII Conclusion

An enormous amount of effort has been expended in recent years in an attempt to show that instrumentally rational agents can adopt commitments, special choice dispositions, or some other mechanism that will allow them to work their way out of suboptimal outcomes. The underlying motivation has been to explain how it is that we are able to avoid the extremes of uncooperative behavior that simple instrumental theories of rationality consistently predict. Over the years, the arguments advanced have become increasingly baroque.27 The model presented here is comparatively simple. It shows how agents are able to cooperate with one another, even when more desirable outcomes can be achieved through defection. By integrating preferences for actions into the agent's overall utility function, without thereby obscuring the distinction between preferences for actions and preferences for outcomes, the model is able to provide an extremely straightforward representation of the way that norms might function as deontic constraints in social choice. The fact that the model is also able to supply deliberative micro-foundations for a 'focal point'-style solution to coordination problems is an unexpected bonus. It functions as indirect support for the proposed analysis, however, insofar as it fulfills the traditional expectation, among sociological theorists, that a solution to the 'problem of order' will require an account of both cooperation and coordination.

Received: December, 2000 Revised: June, 2001

27 Compare David Gauthier, Morals by Agreement (Oxford: Clarendon Press 1986) with David Gauthier, 'Assure and Threaten/ Ethics 104 (1994) 690-721.



rational choice with deontic constraints

Documents