![Page 1: Applied machine learning in game theory Dmitrijs Rutko Faculty of Computing University of Latvia Joint Estonian-Latvian Theory Days at Rakari, 2010](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56649f4a5503460f94c6c5a3/html5/thumbnails/1.jpg)
Applied machine learning in game theory
Dmitrijs RutkoFaculty of Computing
University of Latvia
Joint Estonian-Latvian Theory Days at Rakari, 2010
![Page 2: Applied machine learning in game theory Dmitrijs Rutko Faculty of Computing University of Latvia Joint Estonian-Latvian Theory Days at Rakari, 2010](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56649f4a5503460f94c6c5a3/html5/thumbnails/2.jpg)
Topic outline
Game theory Game Tree Search Fuzzy approach
Machine learning Heuristics Neural networks Adaptive / Reinforcement learning
Card games
![Page 3: Applied machine learning in game theory Dmitrijs Rutko Faculty of Computing University of Latvia Joint Estonian-Latvian Theory Days at Rakari, 2010](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56649f4a5503460f94c6c5a3/html5/thumbnails/3.jpg)
Research overview
Deterministic / stochastic games Perfect / imperfect information games
![Page 4: Applied machine learning in game theory Dmitrijs Rutko Faculty of Computing University of Latvia Joint Estonian-Latvian Theory Days at Rakari, 2010](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56649f4a5503460f94c6c5a3/html5/thumbnails/4.jpg)
Finite zero-sum games
deterministic chance
perfect information chess, checkers, go, othello
backgammon, monopoly, roulette
imperfect information
battleship, kriegspiel, rock-paper-scissors
bridge, poker, scrabble
![Page 5: Applied machine learning in game theory Dmitrijs Rutko Faculty of Computing University of Latvia Joint Estonian-Latvian Theory Days at Rakari, 2010](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56649f4a5503460f94c6c5a3/html5/thumbnails/5.jpg)
Topic outline
Game theory Game Tree Search Fuzzy approach
Machine learning Heuristics Neural networks Adaptive / Reinforcement learning
Card games
![Page 6: Applied machine learning in game theory Dmitrijs Rutko Faculty of Computing University of Latvia Joint Estonian-Latvian Theory Days at Rakari, 2010](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56649f4a5503460f94c6c5a3/html5/thumbnails/6.jpg)
Game trees
![Page 7: Applied machine learning in game theory Dmitrijs Rutko Faculty of Computing University of Latvia Joint Estonian-Latvian Theory Days at Rakari, 2010](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56649f4a5503460f94c6c5a3/html5/thumbnails/7.jpg)
Classical algorithms
MiniMax O(wd)
Alpha-Beta O(wd/2)
1 2 7 4 3 6 8 9 5 4
2 7 8 9
2 8
8
√ √ √ Χ Χ √ √ √ Χ Χ
max
min
max
![Page 8: Applied machine learning in game theory Dmitrijs Rutko Faculty of Computing University of Latvia Joint Estonian-Latvian Theory Days at Rakari, 2010](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56649f4a5503460f94c6c5a3/html5/thumbnails/8.jpg)
Advanced search techniques
Transposition tables Time efficiency / high cost of space
PVS Negascout NegaC* SSS* / DUAL* MTD(f)
![Page 9: Applied machine learning in game theory Dmitrijs Rutko Faculty of Computing University of Latvia Joint Estonian-Latvian Theory Days at Rakari, 2010](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56649f4a5503460f94c6c5a3/html5/thumbnails/9.jpg)
Fuzzy approach
O(wd/2) More cut-offs
1 2 7 4 3 6 8 9 5 4
<5 ? ≥5 ≥5
<5 ≥5
≥5
√ √ Χ Χ Χ √ Χ √ Χ Χ
max
min
max
![Page 10: Applied machine learning in game theory Dmitrijs Rutko Faculty of Computing University of Latvia Joint Estonian-Latvian Theory Days at Rakari, 2010](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56649f4a5503460f94c6c5a3/html5/thumbnails/10.jpg)
Geometric interpretation
1) X2 - successful separation
2) X1 or X3 - reduced search window
α β
2 8
X2
X1 X3
α = X1 β = X3
![Page 11: Applied machine learning in game theory Dmitrijs Rutko Faculty of Computing University of Latvia Joint Estonian-Latvian Theory Days at Rakari, 2010](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56649f4a5503460f94c6c5a3/html5/thumbnails/11.jpg)
BNS enhancement through self-training
Traditional statistical approach
Minimax value
Tree count
25 126 527 1128 3829 12430 20631 25232 18933 11134 4235 1436 7
1000
![Page 12: Applied machine learning in game theory Dmitrijs Rutko Faculty of Computing University of Latvia Joint Estonian-Latvian Theory Days at Rakari, 2010](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56649f4a5503460f94c6c5a3/html5/thumbnails/12.jpg)
Two dimensional game sub-tree distribution
23 24 25 26 27 28 29 30 31 32 33 34 35 36Tree
count23 0 024 0 0 025 0 1 0 126 0 0 2 3 527 0 0 5 3 3 1128 0 1 0 12 12 13 3829 0 0 2 10 35 43 34 12430 1 2 6 9 26 58 71 33 20631 0 0 6 10 27 41 78 57 33 25232 0 1 3 13 17 30 32 41 38 14 18933 0 0 1 2 8 12 26 28 21 11 2 11134 0 0 0 1 3 5 13 8 6 2 2 2 4235 0 0 0 0 0 2 4 3 2 3 0 0 0 1436 0 0 0 0 0 0 1 2 2 1 1 0 0 0 7
![Page 13: Applied machine learning in game theory Dmitrijs Rutko Faculty of Computing University of Latvia Joint Estonian-Latvian Theory Days at Rakari, 2010](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56649f4a5503460f94c6c5a3/html5/thumbnails/13.jpg)
Statistical sub-tree separation
Separation value
Tree count
23 024 125 626 3027 8828 20829 37430 50931 47532 32533 16734 6135 2136 7
2272
![Page 14: Applied machine learning in game theory Dmitrijs Rutko Faculty of Computing University of Latvia Joint Estonian-Latvian Theory Days at Rakari, 2010](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56649f4a5503460f94c6c5a3/html5/thumbnails/14.jpg)
Experimental results. 2-width trees
![Page 15: Applied machine learning in game theory Dmitrijs Rutko Faculty of Computing University of Latvia Joint Estonian-Latvian Theory Days at Rakari, 2010](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56649f4a5503460f94c6c5a3/html5/thumbnails/15.jpg)
Experimental results. 3-width trees
![Page 16: Applied machine learning in game theory Dmitrijs Rutko Faculty of Computing University of Latvia Joint Estonian-Latvian Theory Days at Rakari, 2010](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56649f4a5503460f94c6c5a3/html5/thumbnails/16.jpg)
Future research directions in game tree search
Multi-dimensional self-training Wider trees Real domain games
![Page 17: Applied machine learning in game theory Dmitrijs Rutko Faculty of Computing University of Latvia Joint Estonian-Latvian Theory Days at Rakari, 2010](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56649f4a5503460f94c6c5a3/html5/thumbnails/17.jpg)
Topic outline
Game theory Game Tree Search Fuzzy approach
Machine learning Heuristics Neural networks Adaptive / Reinforcement learning
Card games
![Page 18: Applied machine learning in game theory Dmitrijs Rutko Faculty of Computing University of Latvia Joint Estonian-Latvian Theory Days at Rakari, 2010](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56649f4a5503460f94c6c5a3/html5/thumbnails/18.jpg)
Games with element of chance
![Page 19: Applied machine learning in game theory Dmitrijs Rutko Faculty of Computing University of Latvia Joint Estonian-Latvian Theory Days at Rakari, 2010](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56649f4a5503460f94c6c5a3/html5/thumbnails/19.jpg)
Expectiminimax algorithm
Expectiminimax(n) = Utility(n)
• If n is a terminal state
Max s Successors(n) Expectiminimax(s)• if n is a max node
Min s Successors(n) Expectiminimax(s)• if n is a min node
s Successors(n) P(s) * Expectiminimax(s)• if n is a chance node
O(wdcd)
![Page 20: Applied machine learning in game theory Dmitrijs Rutko Faculty of Computing University of Latvia Joint Estonian-Latvian Theory Days at Rakari, 2010](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56649f4a5503460f94c6c5a3/html5/thumbnails/20.jpg)
Perfomance in Backgammon
*-Minimax Performance in Backgammon,Thomas Hauk, Michael Buro, and Jonathan Schaeer
![Page 21: Applied machine learning in game theory Dmitrijs Rutko Faculty of Computing University of Latvia Joint Estonian-Latvian Theory Days at Rakari, 2010](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56649f4a5503460f94c6c5a3/html5/thumbnails/21.jpg)
Backgammon
Evaluation methods Static – pip count Heuristic – key points Neural Networks
![Page 22: Applied machine learning in game theory Dmitrijs Rutko Faculty of Computing University of Latvia Joint Estonian-Latvian Theory Days at Rakari, 2010](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56649f4a5503460f94c6c5a3/html5/thumbnails/22.jpg)
Temporal difference (TD) learning
Reinforcement learning Prediction method
![Page 23: Applied machine learning in game theory Dmitrijs Rutko Faculty of Computing University of Latvia Joint Estonian-Latvian Theory Days at Rakari, 2010](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56649f4a5503460f94c6c5a3/html5/thumbnails/23.jpg)
Experimental setup
Multi-layer perceptron Representation encoding
Raw data (27 inputs) Unary (157 inputs) Extended unary (201 inputs) Binary (201 input)
Training game series – 400 000 games
![Page 24: Applied machine learning in game theory Dmitrijs Rutko Faculty of Computing University of Latvia Joint Estonian-Latvian Theory Days at Rakari, 2010](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56649f4a5503460f94c6c5a3/html5/thumbnails/24.jpg)
Learning results
![Page 25: Applied machine learning in game theory Dmitrijs Rutko Faculty of Computing University of Latvia Joint Estonian-Latvian Theory Days at Rakari, 2010](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56649f4a5503460f94c6c5a3/html5/thumbnails/25.jpg)
Program “DM Backgammon”
![Page 26: Applied machine learning in game theory Dmitrijs Rutko Faculty of Computing University of Latvia Joint Estonian-Latvian Theory Days at Rakari, 2010](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56649f4a5503460f94c6c5a3/html5/thumbnails/26.jpg)
Topic outline
Game theory Game Tree Search Fuzzy approach
Machine learning Heuristics Neural networks Adaptive / Reinforcement learning
Card games
![Page 27: Applied machine learning in game theory Dmitrijs Rutko Faculty of Computing University of Latvia Joint Estonian-Latvian Theory Days at Rakari, 2010](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56649f4a5503460f94c6c5a3/html5/thumbnails/27.jpg)
Artificial Intelligence and Poker*
* Joint work with Annija Rupeneite
AI Problems Poker problems
Imperfect information Hidden cards
Multiple agents Multiple human players
Risk management Bet strategy and outcome
Agent modeling Opponent(s) modeling
Misleading information Bluffing
Unreliable information Taking bluffing into account