1

Economic Foundations and Game Theory

Peter Wurman

2

Presentation Overview

Economics Economics of Trading Agents Economic modeling General Equilibrium and its Limitations Mechanism design Introduction to Game Theory Pareto Efficiency and Dominant strategy Nash Equilibrium Mixed Strategies Extensive Form and Sub-game Analysis Advanced Topics in Game Theory

3

Economics

Study of the allocation of limited resources in a society of self-interested agents.

Essential features: Agents are rational; Decisions concern the use of resources; Prices significantly simplify the allocation

process.

Note: agents are not assumed to be software entities here.

4

Trading AgentsAgent: software to which we ascribe

Beliefs and knowledge; Rationality; Competence; Autonomy.

Trading agent: software that participates in an electronic market and Is governed in its decision-making by a set of

constraints (budget) and preferences; Obtains the above from a user; Acts in the world by making offers (bids) on the

user’s behalf.

5

Economics of Trading Agents

We will consider economics of trading agents as software entities.

Elements of an Economic Model Resources; Agents; Market Infrastructure.

6

Resources

Resources Limited; Consumed (private) or shared (public).

Formalization N is the number of resources types; xi is an amount of resource i; x is a N-vector of quantities.

7

Two Types of Agents

Consumers Derive value from owning/consuming

resources.Producers

Have technologies to transform resources; Goal is to make money (distributed to

shareholders).Both have private information.

8

Consumer Preferences

Preferences (>, ≥) Total preorder over all bundles x in X

x ≥ x’ or x’ ≥ x (completeness)

x ≥ x’ and x’ ≥ x” implies x ≥ x”(transitivity)

9

Consumer Preferences (2)

Often, we assume convexity For all in [0,1], x ≥ x” and x’ ≥ x”

and x ≠ x’ implies [x + (1-) x’] ≥ x”x1

x2

x

x’x”

10

Preferences Expressed as Utility

Generally, we express preferences as a utility function: uj(x) assigns a numeric value to all

bundlesOften, we assume that utility is

quasi-linear in one resource: uj(x) = vj(x) + m,

where m is money

11

Consumer Endowments

Consumers generally begin with some resources, denoted ej.

Often, these endowments do not maximize the agent’s utility.

Agents engage in economic activities.

12

Simple Exchange Economy

• Suppose all participants are consumers

• How do we determine resources to exchange?• What is a “good” allocation?

Agent 1 Agent 2

Agent 3

Agent 1 Agent 2

Agent 3

13

Price Systems

Associate a price pi with each resource iPrices specify resource exchange rates:

One unit of i can be exchanged for pi/ph units of h.

Present a common scale on which to measure resource value.

Very compact representation of value

14

Solutions

An allocation assigns quantities of each resource to each consumer

Feasible allocations satisfy Material balance which requires that,

for all i, xi,j = ei,j ; Other feasibility constraints.

15

Solution Quality

Pareto efficiency There is no other solution in which

one agent is strictly better off, andno agent is worse off.

Global efficiency (when utility is quasilinear) Corresponds to maximizing j uj(xj); Unique.

16

Equilibrium

General Definition A state from which no agent wishes to deviate.

Equilibrium concepts make assumptions about Agent knowledge; Agent behaviors.

Equilibrium questions Do equilibria exist? How many? Do they support efficient solutions?

17

Classic Agent Behavior

Competitive assumption Agents solve optimization problem:

Find a bundle that maximizes agent’s utility,xi* = argmaxx uj(x);

Subject to agent’s budget, piei,j ;Assuming prices are given.

Agents truthfully state their demand (supply) zi = xi* - ei .

18

General Equilibrium

Definition: A price vector and allocation such that All agents are maximizing their utility

with respect to the prices; No resource is over demanded.

Also called Competitive or Walrasian equilibrium.

19

General Equilibrium Existence

A competitive equilibrium exists in an exchange economy if There is a positive endowment of every

good; Preferences are continuous, strongly

convex, and strongly monotone.One sufficient condition for existence is

gross substitutability Raising the price of one good will not

decrease the demand of another.

20

Production Economies

We allow agents to transform resources from one type to another.

Competitive Equilibrium exist if Production technologies have convex or

constant returns to scale.

21

Fundamental Theorems

First Welfare Theorem Any competitive equilibrium is Pareto

efficient.Second Welfare Theorem

If preferences and technologies are convex, any feasible Pareto solution is a Competitive equilibrium for some price vector and set of endowments.

22

Limitations of G.E. Model

When are the assumptions violated? When agents have market power When prices are nonlinear When agent preferences have

Externalities;nonconvexities (discreteness);Complementarities.

23

G.E. Summary

General Equilibrium Theory provides Some conditions under which

competitive equilibria exist and are unique.

Justification for price systems.But...

We have said nothing about how to reach equilibrium

24

Tatonnement

Tatonnement is the iterative price adjustment scheme proposed by Leon Walras (1874) Auctioneer announces prices; Agents respond with demands; Auctioneer adjusts price of most overdemanded

resource.

Convergence of tatonnement iterative price adjustment guaranteed if gross substitutability holds.

25

Mechanism Design

General Definition An allocation mechanism is a set of

rules that defineAllowable agent actions;Information that is revealed.

Examples Tattonement; Auctions; Fixed pricing.

26

Protocols

A protocol is a combination of a mechanism and assumptions on the agents’ behavior; Tatonnement & competitive assumption =

Walrasian protocol.

Protocols allows us to analyze systems when General Equilibrium conditions do not hold; Competitive assumptions are violated; Perfect rationality is intractable.

27

Two Sides of the Same Coin

Given assumptions about the agents, how do we design an allocation mechanism?

Given an allocation mechanism, how do we design an agent to participate in it?

28

Game Theory

Game theory is a general tool for analyzing mechanisms synthesizing strategies

29

Summary

The design of trading agents should be informed by economics.

General Equilibrium is the foundation of modern economic theory.

Competitive behavior is a simple form of competence.

But there is much more to the story…

30

A Game Players Actions Payoffs Information Finite game: has finite number of players and

finite number of decision alternatives for each player. We will consider examples of two-person games.

Zero-sum game: the sum of players’ payoffs equals zero.

Two-person-zero-sum games: one player’s loss is the other player’s gain.

31

Example

Players: Red & BlueActions

Red: join or pass Blue: join or pass

Payoffs

Join PassJoin 1 3Pass 0 2

Join PassJoin 1 3Pass 0 2

Red’s payoffs Blue’s payoffs

32

Play the Game

Join PassJoin 1 3Pass 0 2

Red’s payoffs Blue’s payoffs

Join PassJoin 1 3Pass 0 2

33

Normal (Strategic) Form

Join Pass

Join 1,1 3,0

Pass 0,3 2,2

Join PassJoin 1 3Pass 0 2

Red’s payoffs Blue’s payoffs

“Prisoners’ Dilemma”

Join PassJoin 1 3Pass 0 2

34

Pareto Efficiency

Join Pass

Join 1,1 3,0

Pass 0,3 2,2

Pareto Efficiency: There is no other solution in which

An agent is strictly better off;No agent is worse off.

35

Pareto Efficiency

Join Pass

Join 1,1 3,0

Pass 0,3 2,2

Pareto Efficiency: There is no other solution in which

An agent is strictly better off;No agent is worse off.

36

Dominant Strategy

Join Pass

Join 1,1 3,0

Pass 0,3 2,2

Dominant Strategy: A strategy for which the payoffs are

better regardless of the other player’s choice.

37

Dominant Strategy Equilibrium

Join Pass

Join 1,1 3,0

Pass 0,3 2,2

Dominant Strategy: A strategy for which the payoffs are

better regardless of the other player’s choice;Red plays join;Blue plays join.

38

Iterated Strict Dominance

Repeatedly rule out strategies until only one remains

L M R

U 4, 3 5, 1 6, 2M 2, 1 8, 4 3, 6D 3, 0 9, 6 2, 8

39

Iterated Strict Dominance

Repeatedly rule out strategies until only one remains

L M R

U 4, 3 5, 1 6, 2M 2, 1 8, 4 3, 6D 3, 0 9, 6 2, 8

Dominates

40

Iterated Strict Dominance

Repeatedly rule out strategies until only one remains

L M R

U 4, 3 5, 1 6, 2M 2, 1 8, 4 3, 6D 3, 0 9, 6 2, 8

Dominates

41

Iterated Strict Dominance

Repeatedly rule out strategies until only one remains

L M R

U 4, 3 5, 1 6, 2M 2, 1 8, 4 3, 6D 3, 0 9, 6 2, 8

42

Dominant Strategy Evaluation

When they exist, they are conclusive (unique).

Often they don’t exist.

43

No Dominant Strategy equilibrium.

“Matching pennies”

Dominant strategy equilibrium does not exist for pure strategies. Zero-sum game.

Head Tail

Head -1,1 1,-1

Tail 1,-1 -1,1

A solution exists if the game is played repeatedly.

44

Nash Equilibrium

An outcome is a Nash equilibrium if each player’s strategy is an optimal response given the other players’ strategies.

F BB 0,0 2,1F 1,2 0,0

“Battle of the Sexes”

45

Nash Equilibrium

An outcome is a Nash equilibrium if each player’s strategy is an optimal response given the other players’ strategies.

If red plays B, blue should play B.

If blue plays B, red should play B.

F BB 0,0 2,1F 1,2 0,0

46

Nash Equilibrium

An outcome is a Nash equilibrium if each player’s strategy is an optimal response given the other players’ strategies.

If red plays F, blue should play F.

If blue plays F, red should play F.

F BB 0,0 2,1F 1,2 0,0

47

Strategies

Strategy space Si = {si

1, si2,…si

n}Pure strategy

A single action, sij

Mixed strategy A probability distribution over pure strategiesi = {(pi

1, si1), (pi

2, si2),…(pi

n, sin)}

where j pij = 1

Von Neumann’s Discovery: every two-person zero-sum game has a maximin solution, in pure or mixed strategies.

48

Mixed-Strategy Equilibrium

A mixed-strategy equilibrium Red plays {(1/3, F)(2/3, B)} Blue plays {(2/3, F)(1/3, B)} E(ured) = 2/3, E(ublue) = 2/3

No other combination of probabilities is a Nash equilibrium

F BB 0,0 2,1F 1,2 0,0

49

Mixed Strategy equilibrium

Every finite strategic-form game has a mixed-strategy equilibrium (Nash, 1950).

No pure-strategy equilibrium. Mixed-strategy equilibrium:

Red plays {(1/2, H)(1/2, T)};Blue plays {(1/2, H)(1/2, T)}.

H TH 1,-1 -1,1T -1,1 1,-1

“Matching Pennies”

50

Assumptions So Far

Complete information: Agents know each other’s strategy

space and payoffs.Common knowledge:

Moreover, each agent knows the other knows…

No communicationSingle round

51

Stage Games

Games in which the players “take turns”

Actions are observablePayoff received at the end of the

game

52

Example: Matchsticks

There are four matchsticksYou may take either one or two

matchsticks on your turnThe last person to take a matchstick

loses

53

Game Tree for 4-Matchsticks4

3

2

2

1

1

1

1

2

22

2

11

1

1,0 0,1 0,1 0,1 1,0

54

Sub-game Analysis4

3

2

2

1

1

1

1

2

22

2

11

1

1,0 0,1 0,1 0,1 1,0

55

Sub-game Analysis4

3

2

2

1

1

1

1

2

22

2

11

1

1,0 0,1 0,1 0,1 1,0

56

Extensive Form

Extensive form contains: The set of players The order of moves The choices at each decision point The payoffs as a function of the moves

made The information each agent has at the

decision point The probabilities associated with

exogenous events (chance)

57

Information

Information is imperfect when An agent can’t observe the other

agents’ moves. There are stochastic events that occur

“in nature”.

58

Hidden-Move Matchsticks4

3

2

2

1

1

1

1

2

22

2

11

1

1,0 0,1 0,1 0,1 1,0

Informationset

59

Flip-a-Coin Matchsticks

4

3

2

2

1

1

1

2

2

11

1

1,0 0,1 0,1

4

3 2

1

1 2

22

0,1 1,0

n

60

AI: Minimax search

Developed in the context of zero-sum games Chess, matchsticks, etc.

Equivalent to backward inductionCan be enhanced using an evaluation

function to represent estimations of terminal node values Allows heuristics to guide search Allows pruning of dominated nodes before

expansion

61

Advanced Topics in Game Theory

Equilibrium Selection How do we choose among multiple Nash

equilibria? Are some inherently more likely to be

chosen than others?

62

Advanced Topics in Game Theory

Repeated Games Reward is received after each round Future rewards are discounted Punishment is possible Learning is possible

63

Advanced Topics in Game Theory

Learning in repeated games When an agent’s knowledge of other

agent’s payoffs is incomplete When an agent doesn’t know its own

payoffs

64

Game Theory Uses

Models of Contract negotiation Social choice Business strategy Auctions Marriage ...

65

Game Theory Conclusions

+Provides a precise description of multiagent interactions.

+Useful solution concepts.+Extremely general.

– Often inconclusive.– Often assumes much knowledge.– Extremely general.