N-GRAM PREDICTIONAND
BAYES INFERENCE27/10/2014
AI for Games Seminarby Andrea Tucci
Slides by Andrea Tucci - @andreatux
OUTLINE
N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux
➔ Context➔ Action Prediction
◆ Techniques◆ N-Grams
● Window Size◆ Considerations
➔ Decision Learning◆ Techniques◆ Naive Bayes Classifiers◆ Bayes Networks
● Bayes Inference● Probability Propagation● Games Application
OUTLINE
OUTLINE
N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux
➔ Conclusion➔ Literature and Further Readings➔ Questions
OUTLINE - 2
OUTLINE
N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux
● Learning in games○ more dynamism, basing on the player○ online and offline○ build a bot from user data/decision○ difficult to debug
Uses● Parameter Modification● Action Prediction● Decisions● ...more
Context
ACTION PREDICTION
N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux
● Study the player’s behaviour, guess next move
● Adaptive AI: the player cannot use the same technique
● Same challenge, different experience
● A little bit of randomness is needed…○ otherwise the player will predict the bot!
Action Prediction
ACTION PREDICTION
N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux
● Probability○ store data of the player’s choices○ pick the most predictable
+ very easy to implement- the player can soon learn the mechanism
● String Matching○ a string is used to store the player’s decisions○ to predict the next move, the string is parsed○ search for identical situation in the string○ prediction is based on history of choices
Techniques
ACTION PREDICTION
N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux
Example:
● In a RPG context the player can perform a physical attack and 3 spells: Fire, Blizzard, Thunder.○ Encoding:
■ Physical Attack: “A”■ Fire : “F”■ Blizzard: “B”■ Thunder “T”
● During a battle we store his/her decisions○ “AAFBAFBAATTFBBAAFB”
Techniques - 2
ACTION PREDICTION
N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux
● Next move?
● Search in the string what the player did in the past, after performing a Fire spell followed by a Blizzard spell.
● AAFBAFBAATTFBBAAFB (?)○ A physical attack is more predictable
● Window size : how many moves should be looked up○ is 2 in the example above○ important for the performance of the prediction
Techniques - 3
ACTION PREDICTION
N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux
● Record the probabilities for each move given all combination of choices for the previous “N” moves.
[Millington]
● “N” = window size + 1○ 3-Gram keeps track the probabilities for a window
size of 2
● Update the list when new actions are performed
● Predict the one with higher probability
N-Grams
ACTION PREDICTION
N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux
Example: Left or Right?
LRRLLLRLLRLRL..(?)3-Gram predictor
N-Grams - 2
->R ->L
LL 2/3 1/3
LR 1/4 3/4
RL 1/3 2/3
RR 0/1 1/1
ACTION PREDICTION
N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux
Easy to predict things like...
N-Grams - 3
ACTION PREDICTION
N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux
● Determine the accuracy of the prediction
● Increasing the window size means better prediction○ ...until it get worse
● Optimal window size should not be too large○ Long sequences tend to have too much randomness
N-Grams - Window Size
ACTION PREDICTION
N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux
Accuracy of N-Grams for the Left-Right game.[Millington]
N-Grams - Window Size
ACTION PREDICTION
N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux
● The more the window increase in size, the more memory is required to store probabilities
● Best performance are achieved when sequences are “known”○ otherwise learning takes a while…
● Very suitable for combat games, computational linguistics, computational biology and in general pattern-based actions
Considerations
DECISION LEARNING
N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux
● Learn how to make decisions
● From a set of possible actions, AI learn which one apply basing on observation data
● Weak and strong supervision○ give feedback to the AI choices
● Huge amount of information in games
Decision Learning
DECISION LEARNING
N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux
● Decision Tree Learning
● Artificial Neural Networks
● Naive Bayes Classifiers
● Reinforcement Learning
Techniques
DECISION LEARNING
N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux
● Decide an action from previous observation○ can be other players decisions, in the same context
● Basing on evidence, gives the probability of “how good” is executing an action
● Bayes rule
Naive Bayes Classifiers
DECISION LEARNING
N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux
● Core concept: express the conditional probability of A given B, in terms of its inverse, B given A.○ the probability that it rained can be expressed as the
observation that the ground is wet
Example: in a soccer game, should we shoot at goal?
● Base our decision on a training set● Try to find decision patterns● Data in the example is labeled
○ this help to generalise similar context
Naive Bayes Classifiers - 2
DECISION LEARNING
N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux
Naive Bayes Classifiers - Example
Shoot? Distance Direction Markers
Yes Medium Central Low
Yes Large Central None
No Short Central High
No Large Angular High
Yes Short Angular None
No Medium Angular High
Yes Medium Central Medium
No Short Central High
Yes Medium Angular Low
[...] [...] [...] [...]
DECISION LEARNING
N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux
Should we shoot if...○ distance = medium○ direction = angular○ markers = none
P ( shoot? | dist, dir, mark) = P(shoot? | dist) * P(shoot? | dir) * P(shoot? | mark)
● Applying the bayes rule, we have an indication based on the data collected○ Google Docs
● Player should shoot!
Naive Bayes Classifiers - Example ctd.
DECISION LEARNING
N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux
● Reason under uncertainty○ AI reasons in a human like way, no cheats
● Arrange variables and their relationship in a “belief network”○ acyclic graph○ variables are encoded as node○ relationships are arcs○ variable probability distribution encoded in a table
● Inference using probabilistic reasoning○ Causal Inference: given A → B, we know A○ Diagnostic Inference: given A → B, we know B○ Intercausal Inference: given A → C and B → C, we know C
Bayes Networks
DECISION LEARNING
N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux
Bayes Networks - 2
Away from home, our neighborhood tell us that he heard our alarm rang.. What happened?
● P(A) = P(B) * P(E) * P(A|B,E)
DECISION LEARNING
N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux
● If the news inform us on an earthquake… the probability that a burglary occurred as well, are low!○ B → A, E → A. P(E) changed and P(B) changed as
well
Bayes Inference
P(B) P(E) P(A|B,E) P(A) N. P(A)
T=.001 T=.002 .95 .000002 .000795
T=.001 F=.998 .94 .000938 .372814
F=.999 T=.002 .29 .000579 .230127
F=.999 F=.998 0.001 .000997 .396264
.002516 1
DECISION LEARNING
N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux
Determining the probability of a burglary after the news
Bayes Inference - 2
P(B) P(E) P(A|B,E) P(A) N. P(A)
T=.001 T=1 .95 .00095 .003269
T=.001 F=0 .94 0 0
F=.999 T=1 .29 .28971 .996731
F=.999 F=0 .001 0 0
.29066 1
DECISION LEARNING
N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux
● Find the probability distribution over a variable X given the evidence e○ P(X | e) ?
● Inference can be exact or approximated
● Computing the probability of a node: summing the probabilities of the parent nodes○ chain rule
Bayes Inference - 3
DECISION LEARNING
N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux
● The belief of each node is updated by prior or posterior evidence updates
Probability Propagation
DECISION LEARNING
N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux
● Visibility○ “fog of war”
● Dependency graphs○ RTS technology graph
■ infer existence of some technologies by the presence of others
■ which dependencies a player is attaining■ etc
● Detect Intruder
Bayes Networks in games
DECISION LEARNING
N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux
● Learning can be used for many reasons in game development
● Main question.. do we need it?○ balance effort of building a learning system with
outcomes● Offline learning is often preferred
● Learning a whole new behaviour is not so easy
● Adaptive AI can be the future of gaming○ ideally no repetitive challenges
Conclusion
DECISION LEARNING
N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux
● Artificial Intelligence for Games [Ian Millington, John Funge]● AI Games Programming Wisdom [Steve Rabin]● Teaching Bayesian behaviours to video game characters [Ronan Le
Hy, et al]● MIT open courseware● PR-OWL● Bayesian Networks [Judea Pearl, Stuart Russell]● Design and Development of a Compound DSS for Laboratory
Research [Tomáš Hujer]● Bayesian Artificial Intelligence [Kevin B. Korb, Ann E. Nicholson]● Player Prediction techniques for AI in video games [Michael
Webersdorfer]● Adaptive AI for fighting games [Ricciardi, Thill
Literature and Further Readings
CONCLUSION
N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux
Questions?
That’s it!Thank you for your
attention