GAMES!!!!

Two people games• Solved games

– Tic-tac-toe– Four In A Line– Checkers

• Impressive games played by robots– Othello bot is much stronger than any human player– Computer chess beat the human world champions– TD-Gammon ranked among top 3 players in the backgammon

world • Future bot challenges to humans

– Poker bots play respectfully at world-class level– Computer bridge programs play competitively at national level– Go bots are getting more serious in the amateur ranking

Complete game tree for Nim-7• 7 coins are placed on a

table between the two opponents

• A move consists of dividing a pile of coins into two nonempty piles of different sizes

• For example, 6 coins can be divided into piles of 5 and 1 or 4 and 2, but not 3 and 3

• The first player who can no longer make a move loses the game

P1 starts

P2 replies

P1

P2

P1

P2 loses

P1 loses

P2 loses

Node score = 0 means MIN wins.

1 means MAX wins.

Bold edges indicate forced win for MAX, Player2.

moves first to minimize

to maximize

MAX wins

MIN wins

MIN vs. MAX in a Nim game

MIN wins

The best that MIN (Player1) can do is to lose unless Player2 makes a mistake.

Minimax to fixed ply depth• Instead of Nim-7, image the chess game tree.• Chess game tree is too deep.

– cannot expand the current node to terminating (leaf) nodes for checkmate.

• Use fixed ply depth look-ahead.– Search from current position to all possible positions that

are, e.g., 3-plies ahead.• Use heuristic to evaluate all these future positions.

– P=1, N=B=3, R=5, Q=9 – Assign certain weight to certain features of the position

(dominance of the center, mobility of the queen, etc.)– summarize these factors into a single number.

• Then propagating the scores back to the current node.

Minimax

• Create a utility function– Evaluation of board/game state to determine how

strong the position of player 1 is.– Player 1 wants to maximize the utility function– Player 2 wants to minimize the utility function

• Minimax tree– Generate a new level for each move– Levels alternate between “max” (player 1 moves)

and “min” (player 2 moves)

Minimax treeMax

Min

Max

Min

Minimax Tree Evaluation

• Assign utility values to leaves– Sometimes called “board evaluation function”– If leaf is a “final” state, assign the maximum or

minimum possible utility value (depending on who would win)

– If leaf is not a “final” state, must use some other heuristic, specific to the game, to evaluate how good/bad the state is at that point

Minimax treeMax

Min

Max

Min

100

-24-8-14-73-100-5-70-12-370412-3212823

Minimax Tree Evaluation

• At each min node, assign the minimum of all utility values at children– Player 2 chooses the best available move

• At each max node, assign the maximum of all utility values at children– Player 1 chooses best available move

• Push values from leaves to top of tree

Minimax treeMax

Min

Max

Min

100

-24-8-14-73-100-5-70-12-370412-3212823

28 -3 12 70 -3 -73 -14 -8

Minimax treeMax

Min

Max

Min

100

-24-8-14-73-100-5-70-12-470412-3212823

21 -3 12 70 -4 -73 -14 -8

-3 -4 -73

Minimax treeMax

Min

Max

Min

100

-24-8-14-73-100-5-70-12-470412-3212823

21 -3 12 70 -4 -73 -14 -8

-3 -4 -73

-3

Minimax Evaluation

• Given average branching factor b, and depth m:– A complete evaluation takes time bm

– A complete evaluation takes space bm• Usually, we cannot evaluate the complete

state, since it’s too big• Instead, we limit the depth based on various

factors, including time available.

Pruning the Minimax Tree

• Since we have limited time available, we want to avoid unnecessary computation in the minimax tree.

• Pruning: ways of determining that certain branches will not be useful

a Cuts

• If the current max value is greater than the successor’s min value, don’t explore that min subtree any more

a Cut example

10021 -3 12 70 -4 -73 -14

-3 -4

-3

-73

Max

Max

Min

a Cut example

• Depth first search along path 1

10021 -3 12 -70 -4 -73 -14

Max

Max

Min

a Cut example

• 21 is minimum so far (second level)• Can’t evaluate yet at top level

10021 -3 12 -70 -4 -73 -14

Max

Max

Min 21

a Cut example

• -3 is minimum so far (second level)• -3 is maximum so far (top level)

10021 -3 12 -70 -4 -73 -14

Max

Max

Min -3

-3

a Cut example

• 12 is minimum so far (second level)• -3 is still maximum (can’t use second node yet)

10021 -3 12 -70 -4 -73 -14

Max

Max

Min -3

-3

12

a Cut example

• -70 is now minimum so far (second level)• -3 is still maximum (can’t use second node yet)

10021 -3 12 -70 -4 -73 -14

Max

Max

Min -3

-3

-70

a Cut example

• Since second level node will never be > -70, it will never be chosen by the previous level

• We can stop exploring that node

10021 -3 12 -70 -4 -73 -14

Max

Max

Min -3

-3

-70

a Cut example

• Evaluation at second level is -73

10021 -3 12 -70 -4 -73 -14

Max

Max

Min -3

-3

-70 -73

a Cut example

• Again, can apply a cut since the second level node will never be > -73, and thus will never be chosen by the previous level

10021 -3 12 -70 -4 -73 -14

Max

Max

Min -3

-3

-70 -73

a Cut example

• As a result, we evaluated the Max node without evaluating several of the possible paths

10021 -3 12 -70 -4 -73 -14

Max

Max

Min -3

-3

-70 -73

b cuts

• Similar idea to a cuts, but the other way around

• If the current minimum is less than the successor’s max value, don’t look down that max tree any more

b Cut example

• Some subtrees at second level already have values > min from previous, so we can stop evaluating them.

10021 -3 12 70 -4 73 -14

Min

Min

Max 21

21

70 73

Tic tac Toe