challenges for implementing monte carlo tree search in commercial games
TRANSCRIPT
1
Click icon to add picture
Challenges for implementing Monte Carlo Tree Search in commercial games
Matthew Bedder
16/Sept/2016
216/Sept/2016
Disclaimer
This is pitched at people with little/no knowledge of the area
(Sorry, AI people)
3
“A game is a series of interesting choices”
16/Sept/2016
416/Sept/2016
5
Monte Carlo Tree Search
16/Sept/2016
616/Sept/2016
Monte Carlo Tree SearchOnly generate the bits of the game tree we need
716/Sept/2016
MCTS – the basics
Iteratively builds a decision tree
816/Sept/2016
MCTS Selection
916/Sept/2016
MCTS Expansion
1016/Sept/2016
MCTS Simulation
1116/Sept/2016
Monte Carlo evaluationsPerforming random actions gives us a (weak) approximation of state values
X O X OX
OX
O
X
1216/Sept/2016
MCTS Simulation
1316/Sept/2016
MCTS Backpropagation
1416/Sept/2016
MCTS – the basics
15
Applications of MCTS
16/Sept/2016
1616/Sept/2016
MCTS applications
• Amazons• Arimaa• Connect 4• Dou Di Zhu• Go variants• Havannah
• Hex• Morpion Solitaire• Othello• Settlers of Catan• Spades• ...
Many board and card games, including:
1716/Sept/2016
1816/Sept/2016
MCTS applicationsAll of the strongest AI players of Go use variants of MCTS
1916/Sept/2016
MCTS applicationsCurrently there are very limited applications to commercial video games or other domains
2016/Sept/2016
http://www.aifactory.co.uk/AIF_Games_Spades.htm
2116/Sept/2016
MCTS Successes
http://aigamedev.com/open/coverage/mcts-rome-ii/
2216/Sept/2016
http://gwaredd.github.io/nuclai_mcts
23
Challenges for MCTS
16/Sept/2016
2416/Sept/2016
Games are hard
There are as many states of Civilization II as there are atoms in the
observable universe 10,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,
000,000,000,000,000,000,000,000,000,000 times over
2516/Sept/2016
Computational costThis is actually important
Game AIs (usually) are limited to time sliced CPU
AlphaGo used• 1,920 CPUs and 280 GPUs• ~1MW
2616/Sept/2016
Monte Carlo Tree Search optimisationsGenerally either:
Domain specificor
Overly generic
2716/Sept/2016
Monte Carlo Tree Search optimisationsReplacing random simulations helps… usually
Adding prior knowledge helps… usually
2816/Sept/2016
Our approachDomain-specific optimisations for arbitrary domains
I’d love to talk about our work during/after coffee
