Turn-Based Games Hctor Muoz-Avila sources:Wikipedia.org Russell & Norvig AI Book; Chapter 5 (and slides) My own Turn-Based Strategy Games Early strategy games was dominated by turn- based games Derivate from board games Chess The Battle for Normandy (1982) Nato Division Commanders (1985) Turn-based strategy: game flow is partitioned in turns or rounds. Turns separate analysis by the player from actions harvest, build, destroy in turns Two classes: Simultaneous Mini-turns Turn-Based Games Continues to be A Popular Game Genre At least 3 sub-styles are very popular: Civilization-style games Civilization IV came out last week Fantasy-style (RPG) Heroes of Might and Magic series Poker games Poker Academy Some Historical Highlights 1952 Turing design a chess algorithm. Around the same time Claude Shannon also develop a chess program 1956 Maniac versus Human 1970 Hamurabi. A game about building an economy for a kingdom The Battle for Normandy (1982) 1987 Pirates! 1990 Civilization 1995 HoMM 1996 Civilization II The best game ever? 2005 Civilization IV 2006 HoMM V Side-tracking: Game Design: Contradicting Principles Principle: All actions can be done from a single screen. Classical example: Civ IVCiv IV But: HoMM uses two interfaces: HoMM IVHoMM IV Coming back: How to Construct Good AI? Idea: Lets just use A* and define a good heuristic for the game Search space: a bipartite tree After all didnt we use it with the 9-puzzle game? Problems with this idea: Adversarial: we need to consider possible moves of our opponent (s) Time limit: (think Chess) Types of Adversarial TBGs (from AI perspective) Perfect information Imperfect information Deterministic Chance Chess, Go, rock- paper-scissors Battleships, Stratego Backgammon, monopoly Civilization, HoMM Bridge, Poker Game tree (2-player, deterministic, turns) Concepts: State: node in search space Operator: valid move Terminal test: game over Utility function: value for outcome of the game MAX: 1 st player, maximizing its own utility MIN: 2 nd player, minimizing Maxs utility Minimax Finding perfect play for deterministic games Idea: choose move to position with highest minimax value = best achievable payoff against best play E.g., 2-play game: Minimax algorithm Properties of minimax Complete? Optimal? Time complexity? b: branching factor m: # moves in a game Yes (if tree is finite) Yes (against an optimal opponent) O(b m ) For chess, b 35, m 100 for "reasonable" games Therefore, exact solution is infeasible Minimax algorithm with Imperfect Decisions evaluationFunction(state) Cutoff-test(state) Evaluation Function Is an estimate of the actual utility Typically represented as a linear function: EF(state) = w 1 f 1 (state) + w 2 f 2 (state) + + w n f n (state) Example: Chess weight: Piece Number (w 1 ) Pawn 1 (w 2 ) Knight 3 (w 3 ) Bishop 3 (w 4 ) Rook 5 (w 5 ) Queen 9 Function; state Number f 1 = #(pawns,b) #(pawns,w) f 2 = #(knight,b) #(knight,w) f 3 = #(bishop,b) #(bishop,w) f 4 = #(rook,b) #(rook,w) f 5 = #(knight,b) #(knight,w) Evaluation Function (2) Obviously, the quality of the AI player depends on the evaluation function Conditions for evaluation functions: If n is a terminal node, Computing EF should not take long EF should reflect chances of winning EF(n) = Utility(n) If EF(state) > 3 then is almost-certain that blacks win Cutting Off Search - pruning example Properties of - Pruning does not affect final result Good move ordering improves effectiveness of pruning With "perfect ordering," time complexity = O(b m/2 ) doubles depth of search A simple example of the value of reasoning about which computations are relevant (a form of metareasoning) Why is it called -? is the value of the best (i.e., highest-value) choice found so far at any choice point along the path for max If v is worse than , max will avoid it prune that branch Define similarly for min The - algorithm Resource limits Suppose we have 100 secs, explore 10 4 nodes/sec 10 6 nodes per move Standard approach: cutoff test: e.g., depth limit (perhaps add quiescence search) evaluation function = estimated desirability of position Evaluation functions For chess, typically linear weighted sum of features Eval(s) = w 1 f 1 (s) + w 2 f 2 (s) + + w n f n (s) e.g., w 1 = 9 with f 1 (s) = (number of white queens) (number of black queens), etc. Cutting off search MinimaxCutoff is identical to MinimaxValue except 1.Terminal? is replaced by Cutoff? 2.Utility is replaced by Eval Does it work in practice? b m = 10 6, b=35 m=4 4-ply lookahead is a hopeless chess player! 4-ply human novice 8-ply typical PC, human master 12-ply Deep Blue, Kasparov Deterministic games in practice Checkers: Chinook ended 40-year-reign of human world champion Marion Tinsley in Used a precomputed endgame database defining perfect play for all positions involving 8 or fewer pieces on the board, a total of 444 billion positions. Chess: Deep Blue defeated human world champion Garry Kasparov in a six-game match in Deep Blue searches 200 million positions per second, uses very sophisticated evaluation, and undisclosed methods for extending some lines of search up to 40 ply. Othello: human champions refuse to compete against computers, who are too good. Go: human champions refuse to compete against computers, who are too bad. In go, b > 300, so most programs use pattern knowledge bases to suggest plausible moves. Summary Games are fun to work on! They illustrate several important points about AI perfection is unattainable must approximate good idea to think about what to think about