![Page 1: PLANNING (SEARCH IN GAMESbartak/planovani/lectures/Planning_in_gam… · PLANNING IN THE REAL WORLD WHAT IS NOT SIMPLE Multiple problems: Continous worlds Dynamicity Non-determinism](https://reader034.vdocuments.mx/reader034/viewer/2022052008/601c8624f311c24b8858e8fd/html5/thumbnails/1.jpg)
PLANNING (SEARCH) IN GAMES The Real and Virtual Worlds and a Not-so-optimistic
Account on Planning
Martin Černý [email protected]
Pla
nn
ing in
Ga
me
s. M
artin
Če
rný.
![Page 2: PLANNING (SEARCH IN GAMESbartak/planovani/lectures/Planning_in_gam… · PLANNING IN THE REAL WORLD WHAT IS NOT SIMPLE Multiple problems: Continous worlds Dynamicity Non-determinism](https://reader034.vdocuments.mx/reader034/viewer/2022052008/601c8624f311c24b8858e8fd/html5/thumbnails/2.jpg)
OUTLINE
Planning in the real world
What is NOT so simple
Using planners
What I learned using planners in practice
Planning as A*
HTN
Adversarial search
http://aigamedev.com/open/review/planning-in-
games/
Planning in Games. Martin Černý.
![Page 3: PLANNING (SEARCH IN GAMESbartak/planovani/lectures/Planning_in_gam… · PLANNING IN THE REAL WORLD WHAT IS NOT SIMPLE Multiple problems: Continous worlds Dynamicity Non-determinism](https://reader034.vdocuments.mx/reader034/viewer/2022052008/601c8624f311c24b8858e8fd/html5/thumbnails/3.jpg)
PLANNING IN THE REAL WORLD WHAT IS NOT SIMPLE
Multiple problems:
Continous worlds
Dynamicity
Non-determinism
Unknown domains
Partial observability
Goals
Reasoning about time
Speed of planning
Meta reasoning
Planning in Games. Martin Černý.
![Page 4: PLANNING (SEARCH IN GAMESbartak/planovani/lectures/Planning_in_gam… · PLANNING IN THE REAL WORLD WHAT IS NOT SIMPLE Multiple problems: Continous worlds Dynamicity Non-determinism](https://reader034.vdocuments.mx/reader034/viewer/2022052008/601c8624f311c24b8858e8fd/html5/thumbnails/4.jpg)
We are still in PSPACE!
IPC – limit 30min
Games can offer only fractions of seconds
Contemporary STRIPS planners can handle
approximately hundreds of predicates and/or action
within a second (using a whole core)
Anytime planning
But how do you do that?
Planning in Games. Martin Černý.
PLANNING IN THE REAL WORLD PLANNING SPEED
![Page 5: PLANNING (SEARCH IN GAMESbartak/planovani/lectures/Planning_in_gam… · PLANNING IN THE REAL WORLD WHAT IS NOT SIMPLE Multiple problems: Continous worlds Dynamicity Non-determinism](https://reader034.vdocuments.mx/reader034/viewer/2022052008/601c8624f311c24b8858e8fd/html5/thumbnails/5.jpg)
When to plan/replan
Opportunism
How much time do I have?
When to stop planning?
Optimal plans are usually unnecessary
Commitment
Requires tight integration of the planner
Planning in Games. Martin Černý.
PLANNING IN THE REAL WORLD METAREASONING
![Page 6: PLANNING (SEARCH IN GAMESbartak/planovani/lectures/Planning_in_gam… · PLANNING IN THE REAL WORLD WHAT IS NOT SIMPLE Multiple problems: Continous worlds Dynamicity Non-determinism](https://reader034.vdocuments.mx/reader034/viewer/2022052008/601c8624f311c24b8858e8fd/html5/thumbnails/6.jpg)
Reactive techniques still prevalent
FSM
Hierarchical FSMs
Behaviour trees
At some level, reactive techniques are
indispensable
Planning in Games. Martin Černý.
PLANNING IN GAMES WHAT DO THEY USE INSTEAD?
![Page 7: PLANNING (SEARCH IN GAMESbartak/planovani/lectures/Planning_in_gam… · PLANNING IN THE REAL WORLD WHAT IS NOT SIMPLE Multiple problems: Continous worlds Dynamicity Non-determinism](https://reader034.vdocuments.mx/reader034/viewer/2022052008/601c8624f311c24b8858e8fd/html5/thumbnails/7.jpg)
Modelling, modelling, modelling
There are bugs.
Typically caused by too simple or “nonsensical”
problems
Not so many (compared to other academical SW)
Using planners from Java: Planning4J
Universal API for IPC planners nad JSHOP2
http://code.google.com/p/planning4j/
Planning in Games. Martin Černý.
USING PLANNERS
![Page 8: PLANNING (SEARCH IN GAMESbartak/planovani/lectures/Planning_in_gam… · PLANNING IN THE REAL WORLD WHAT IS NOT SIMPLE Multiple problems: Continous worlds Dynamicity Non-determinism](https://reader034.vdocuments.mx/reader034/viewer/2022052008/601c8624f311c24b8858e8fd/html5/thumbnails/8.jpg)
Recent IPC are quite opposed to realtime planning
30 minute timeout
Fast Downward
Issues with PDDL:
Except for BlackBox all planners I ever used (and a
majority of planners in general) translate PDDL to state-
variables (and it takes time)
No possibility to alter the course of planning,
interrupt prematurely etc.
Planning in Games. Martin Černý.
USING PLANNERS OTHER NOTES
![Page 9: PLANNING (SEARCH IN GAMESbartak/planovani/lectures/Planning_in_gam… · PLANNING IN THE REAL WORLD WHAT IS NOT SIMPLE Multiple problems: Continous worlds Dynamicity Non-determinism](https://reader034.vdocuments.mx/reader034/viewer/2022052008/601c8624f311c24b8858e8fd/html5/thumbnails/9.jpg)
Easy to understand
State-of-the art IS heuristic forward search
Procedural effects, procedural preconditions
Gravity, shooting….
Planning in Games. Martin Černý.
PLANNING AS A*
![Page 10: PLANNING (SEARCH IN GAMESbartak/planovani/lectures/Planning_in_gam… · PLANNING IN THE REAL WORLD WHAT IS NOT SIMPLE Multiple problems: Continous worlds Dynamicity Non-determinism](https://reader034.vdocuments.mx/reader034/viewer/2022052008/601c8624f311c24b8858e8fd/html5/thumbnails/10.jpg)
Goal oriented action planning
Jeff Orkin, F.E.A.R. (2005)
The only documented system in games that uses the word “planning”
Used in other games as well S.T.A.L.K.E.R.: Shadow of Chernobyl
Fallout 3
Deus Ex
…
Positive reception by players
STRIPS-based
http://web.media.mit.edu/~jorkin/goap.html
Planning in Games. Martin Černý.
GOAP
![Page 11: PLANNING (SEARCH IN GAMESbartak/planovani/lectures/Planning_in_gam… · PLANNING IN THE REAL WORLD WHAT IS NOT SIMPLE Multiple problems: Continous worlds Dynamicity Non-determinism](https://reader034.vdocuments.mx/reader034/viewer/2022052008/601c8624f311c24b8858e8fd/html5/thumbnails/11.jpg)
The world is represented by state variables (a bit
like CSP formalism)
Variables may point directly to in-game entities
Procedural preconditions and effects
Action costs
A*
Separate system for goal selection
Planning in Games. Martin Černý.
GOAP THE BASICS
![Page 12: PLANNING (SEARCH IN GAMESbartak/planovani/lectures/Planning_in_gam… · PLANNING IN THE REAL WORLD WHAT IS NOT SIMPLE Multiple problems: Continous worlds Dynamicity Non-determinism](https://reader034.vdocuments.mx/reader034/viewer/2022052008/601c8624f311c24b8858e8fd/html5/thumbnails/12.jpg)
Advantages
Smarter AI
Separation of actions and goals from the implementation
Easier to maintain than FSM
Disadvantages
No direct control over agent behaviour – difficult debugging
Many details to tweak
Reportedly, HTN and behaviour trees are favored now over GOAP
More at http://aigamedev.com/open/review/planning-in-games/
Planning in Games. Martin Černý.
GOAP ADVANTAGES AND DISADVANTAGES
![Page 13: PLANNING (SEARCH IN GAMESbartak/planovani/lectures/Planning_in_gam… · PLANNING IN THE REAL WORLD WHAT IS NOT SIMPLE Multiple problems: Continous worlds Dynamicity Non-determinism](https://reader034.vdocuments.mx/reader034/viewer/2022052008/601c8624f311c24b8858e8fd/html5/thumbnails/13.jpg)
Sims interact with objects
Autonomously
User-directed
We do not want to stop interactions when not
necessary
Planning in Games. Martin Černý.
THE SIMS IV
![Page 14: PLANNING (SEARCH IN GAMESbartak/planovani/lectures/Planning_in_gam… · PLANNING IN THE REAL WORLD WHAT IS NOT SIMPLE Multiple problems: Continous worlds Dynamicity Non-determinism](https://reader034.vdocuments.mx/reader034/viewer/2022052008/601c8624f311c24b8858e8fd/html5/thumbnails/14.jpg)
Let’s skip to Peter Ingebretson’s slides from GDC
(slide 116)
http://www.gdcvault.com/play/1020190/Concurrent-
Interactions-in-The-Sims
Planning in Games. Martin Černý.
THE SIMS IV
![Page 15: PLANNING (SEARCH IN GAMESbartak/planovani/lectures/Planning_in_gam… · PLANNING IN THE REAL WORLD WHAT IS NOT SIMPLE Multiple problems: Continous worlds Dynamicity Non-determinism](https://reader034.vdocuments.mx/reader034/viewer/2022052008/601c8624f311c24b8858e8fd/html5/thumbnails/15.jpg)
Starcraft – BroodWar
Depth-first branch & bound (no PDDL model!)
Saves memory in comparison to A*
Depth-first is FAST
https://www.skatgame.net/mburo/ps/aiide11-bo.pdf
Planning in Games. Martin Černý.
BUILD-ORDER PLANNING
![Page 16: PLANNING (SEARCH IN GAMESbartak/planovani/lectures/Planning_in_gam… · PLANNING IN THE REAL WORLD WHAT IS NOT SIMPLE Multiple problems: Continous worlds Dynamicity Non-determinism](https://reader034.vdocuments.mx/reader034/viewer/2022052008/601c8624f311c24b8858e8fd/html5/thumbnails/16.jpg)
Planning in Games. Martin Černý.
HIERARCHICAL TASK NETWORKS (HTN)
Attack(from, to)
Run(from, to)
Is_flankable(from, flankpos)
Shoot(from, to)
Flank Attack directly
Run(from,flank_pos)
Shoot(flank_pos, to)
![Page 17: PLANNING (SEARCH IN GAMESbartak/planovani/lectures/Planning_in_gam… · PLANNING IN THE REAL WORLD WHAT IS NOT SIMPLE Multiple problems: Continous worlds Dynamicity Non-determinism](https://reader034.vdocuments.mx/reader034/viewer/2022052008/601c8624f311c24b8858e8fd/html5/thumbnails/17.jpg)
Suppose: Is_flankable(mala_strana, kampa)
Planning in Games. Martin Černý.
HTN – SEARCH
Attack(stare_mesto,mala_strana)
Is_flankable(mala_strana, kampa)
Attack(kampa, mala_strana)
Run(stare_mesto, kampa)
Flank
Flank
Is_flankable(kampa, ???)
FAIL
Attack directly
Run(kampa, mala_strana)
Shoot(kampa, mala_strana)
![Page 18: PLANNING (SEARCH IN GAMESbartak/planovani/lectures/Planning_in_gam… · PLANNING IN THE REAL WORLD WHAT IS NOT SIMPLE Multiple problems: Continous worlds Dynamicity Non-determinism](https://reader034.vdocuments.mx/reader034/viewer/2022052008/601c8624f311c24b8858e8fd/html5/thumbnails/18.jpg)
Variables + bindings…
Undecidable in the worst case!
May be considered an extension of POP
Better at modeling parallel actions
Often employed in team planning
Good at capturing domain-dependent knowledge
Tasks that are “puzzle-like” (e.g. Sudoku) are hard
to express
Planning in Games. Martin Černý.
HTN PROPERTIES
![Page 19: PLANNING (SEARCH IN GAMESbartak/planovani/lectures/Planning_in_gam… · PLANNING IN THE REAL WORLD WHAT IS NOT SIMPLE Multiple problems: Continous worlds Dynamicity Non-determinism](https://reader034.vdocuments.mx/reader034/viewer/2022052008/601c8624f311c24b8858e8fd/html5/thumbnails/19.jpg)
Killzone 2
http://www.slideshare.net/guerrillagames/killzone-2-
multiplayer-bots
Dark Souls, Max Payne 3, …
Non-industry
Teams of bots in UT
http://www.aaai.org/Papers/AIIDE/2005/AIIDE05-011.pdf
Winner of the AIGameDev Capture the flag tournament
http://aigamedev.com/insider/interview/ctf-win-secrets/
Planning in Games. Martin Černý.
HTN – APPLICATIONS
![Page 20: PLANNING (SEARCH IN GAMESbartak/planovani/lectures/Planning_in_gam… · PLANNING IN THE REAL WORLD WHAT IS NOT SIMPLE Multiple problems: Continous worlds Dynamicity Non-determinism](https://reader034.vdocuments.mx/reader034/viewer/2022052008/601c8624f311c24b8858e8fd/html5/thumbnails/20.jpg)
Planning is just not enough!
Rock-paper-scissors mechanics.
Planning in Games. Martin Černý.
ADVERSARIAL SEARCH
![Page 21: PLANNING (SEARCH IN GAMESbartak/planovani/lectures/Planning_in_gam… · PLANNING IN THE REAL WORLD WHAT IS NOT SIMPLE Multiple problems: Continous worlds Dynamicity Non-determinism](https://reader034.vdocuments.mx/reader034/viewer/2022052008/601c8624f311c24b8858e8fd/html5/thumbnails/21.jpg)
Simplified combat model
AlphaBeta considering durations
http://webdocs.cs.ualberta.ca/~cdavid/pdf/aiide12-
combat.pdf
Portfolio search
https://www.skatgame.net/mburo/ps/combat13.pdf
Hierarchical adversarial search
https://www.skatgame.net/mburo/ps/HierarchicalSearch-
AIIDE-2014.pdf
Planning in Games. Martin Černý.
ADVERSARIAL SEARCH IN STARCRAFT
![Page 22: PLANNING (SEARCH IN GAMESbartak/planovani/lectures/Planning_in_gam… · PLANNING IN THE REAL WORLD WHAT IS NOT SIMPLE Multiple problems: Continous worlds Dynamicity Non-determinism](https://reader034.vdocuments.mx/reader034/viewer/2022052008/601c8624f311c24b8858e8fd/html5/thumbnails/22.jpg)
Highly recommended read
http://en.wikipedia.org/wiki/Monte_Carlo_tree_search
Tested in RTS (Wargus) combat
http://www.aaai.org/ocs/index.php/IJCAI/IJCAI-
09/paper/%20viewPDFInterstitial/632/587
But outperformed by the approaches on previous slide
Used in Total War: Rome II (strategic level), Fable
Legends (real time)
Planning in Games. Martin Černý.
MONTE-CARLO TREE SEARCH
![Page 23: PLANNING (SEARCH IN GAMESbartak/planovani/lectures/Planning_in_gam… · PLANNING IN THE REAL WORLD WHAT IS NOT SIMPLE Multiple problems: Continous worlds Dynamicity Non-determinism](https://reader034.vdocuments.mx/reader034/viewer/2022052008/601c8624f311c24b8858e8fd/html5/thumbnails/23.jpg)
Situations – find suitable NPC combinations with
CSP
In 0.1ms…
Monte Carlo Tree Search for Combat
Planning in Games. Martin Černý.
OUR WORK IN KINGDOM COME:
DELIVERANCE
![Page 24: PLANNING (SEARCH IN GAMESbartak/planovani/lectures/Planning_in_gam… · PLANNING IN THE REAL WORLD WHAT IS NOT SIMPLE Multiple problems: Continous worlds Dynamicity Non-determinism](https://reader034.vdocuments.mx/reader034/viewer/2022052008/601c8624f311c24b8858e8fd/html5/thumbnails/24.jpg)
Planning is good, BUT….
IPC results do not correspond to performance in
real time
There are other techniques than STRIPS and they
have their strengths
In games you absolutely need procedural effects
Contact me
Theses, projects: http://pogamut.cuni.cz/prace
Planning in Games. Martin Černý.
SUMMARY