finite state machine for games fall 2012 ref: chenney, cs679 lectures ai game programming wisdom 2
TRANSCRIPT
Finite State Machine for Games
Fall 2012Ref: Chenney, CS679 lecturesAI Game Programming Wisdom 2
Fall 2012 2
Outline
AI and GameIntroduction/examplesDesign Intuition State-based
ImplementationExtending Stack-based Fuzzy-state machine
Fall 2012 3
What is AI? (and is NOT)
AI is the control of non-human entities in a game
The other cars in a car game The opponents and monsters in a shooter Your units, your enemy’s units and your enemy in a
RTS game
But, typically does not refer to passive things that just react to the player and never initiate action
That’s physics or game logic For example, the blocks in Tetris are not AI, nor is a
flag blowing in the wind
Fall 2012 4
AI in the Game Loop
AI is updated as part of the game loop, after user input, and before rendering
Fall 2012 5
AI Module
AI Update Step
The sensing phase determines the state of the world
May be very simple - state changes all come by messaging
Or complex - figure out what is visible, where your team is, etc
The thinking phase decides what to do given the world
The core of AI
The acting phase tells the animation what to do
Game Engine
Sensing
Thinking
Acting
Fall 2012 6
AI by Polling
Senses: looks to see what has been changed in the
world
then acts on it
Generally inefficient and complicated Different characters might require different
polling rate
AI gets called at a fixed rate
Event Driven AI
• Events triggered by a message • basically, a function gets called when
a message arrives, just like a user interface)
• Example messages: You have heard a sound Someone has entered your field of
view A certain amount of time has passed,
so update yourself
Fall 2012 7
does everything in response to events in the world
Fall 2012 8
AI Techniques in Games
Basic problem: Given the state of the world, what should I do?A wide range of solutions in games: Finite state machines, Decision trees, Rule
based systems, Neural networks, Fuzzy logic
A wider range of solutions in the academic world: Complex planning systems, logic
programming, genetic algorithms, Bayes-nets Typically, too slow for games
Fall 2012 9
Expressiveness
What behaviors can easily be defined, or defined at all?Propositional logic:
Statements about specific objects in the world – no variables
Jim is in room7, Jim has the rocket launcher, the rocket launcher does splash damage
Go to room8 if you are in room7 through door14Predicate Logic:
Allows general statement – using variables All rooms have doors All splash damage weapons can be used around corners All rocket launchers do splash damage Go to a room connected to the current room
Fall 2012 10
Finite State Machines (FSMs)
A set of states that the agent can be inConnected by transitions that are triggered by a change in the worldNormally represented as a directed graph, with the edges labeled with the transition eventUbiquitous in computer game AI
Fall 2012 11
Formal Definitions (N. Philips)
"An abstract machine consisting of a set of states (including the initial state), a set of input events, a set of output events, and a state transition function. The function takes the current state and an input event and returns the new set of output events and the next state. Some states may be designated as "terminal states".The state machine can also be viewed as a function which maps an ordered sequence of input events into a corresponding sequence of (sets of) output events.Finite State Automaton: the machine with no output
Fall 2012 12
FSM with Output: vending machines
OJ & AJ for 30 centsState table
Fall 2012 13
Vending Machine: state diagram
Fall 2012 14
FSM and Game
Game character behavior can be modeled (in most cases) as a sequence of different “mental state”, where change is driven by the actions of player/other characters, …Natural choice for defining AI in games
Fall 2012 15
FSM Examples
Fall 2012 16
Fall 2012 17
Fall 2012 18
PacMan State Machine
Fall 2012 19
PacMan eats a Power Pill
Timer <= 0
Collision with PacMan
Timer <= 0
Collision with GhostBox
Actions of States
Fall 2012 20
Roam;If PacMan gets close, PathTo (PacMan)
Timer--;PathAwayFrom (PacMan)
PathTo (GhostBox)Timer—Move back-and-forth
Fall 2012 21
Ex: predator vs. prey
Prey (laalaa)
Idle(stand,wave,…)
Flee(run)
Sees predator
No more threat
capturedDead
Fall 2012 22
Predator (Raptor)
Idle(stand)
Hungry(wander)
Pursuit(run)
Tidle > 5
Prey in sight
Tpursuit > 10
Dining
Prey captured
Tdining>5
Fall 2012 23
Idling LaaLaa
Stand
Wave
Wander(set random target)
50%
30%20%
Target arrived
Twave>2
RTstand>4
This page illustrates:hierarchical state,Non-deterministic state transition
This page illustrates:hierarchical state,Non-deterministic state transition
FSM Design
Fall 2012 25
Quake2 ExamplesQuake2 uses 9 different states:
standing, walking, running, dodging, attacking, melee, seeing the enemy, idle and searching.
Incomplete design
Intuitive thinking: model the events and state changes
Intuitive thinking: model the events and state changes
Fall 2012 26
Quake: Rocket
Fall 2012 27
Shambler monster
Fall 2012 28
FSM Advantages
Very fast – one array accessExpressive enough for simple behaviors or characters that are intended to be “dumb”
Can be compiled into compact data structure Dynamic memory: current state Static memory: state diagram – array implementation
Can create tools so non-programmer can build behaviorNon-deterministic FSM can make behavior unpredictable
Fall 2012 29
FSM Disadvantages
Number of states can grow very fast Exponentially with number of events: s=2e
Number of arcs can grow even faster: a=s2
Propositional representation Difficult to put in “pick up the better powerup”,
“attack the closest enemy” Expensive to count: Wait until the third time I see
enemy, then attack Need extra events: First time seen, second time seen,
and extra states to take care of counting
FSM Control System Implementation
Fall 2012 31
FSM Implementation
Fall 2012 32
Previous Example
Fall 2012 33
Code 1 Ad hoc implementation
Fall 2012 34
Code 1p
Fall 2012 35
Code 2Structure, Readable, maintainable
Fall 2012 36
Hierarchical …
FSM Extensions
Advanced Issues
Hierarchical FSMNon-deterministic FSMSwapping FSM
Fall 2012 38
Fall 2012 39
Hierarchical FSMs
What if there is no simple action for a state?Expand a state into its own FSM, which explains what to do if in that stateSome events move you around the same level in the hierarchy, some move you up a levelWhen entering a state, have to choose a state for its child in the hierarchy
Set a default, and always go to that Or, random choice Depends on the nature of the behavior
Fall 2012 40
Stack-based FSM
History stack: Remember previous state; create characters with a memory …
Pushdown automaton(PDA)
Fall 2012 41
Goal-based vs. State-based
There is also a slight derivative to the state-based engine, but it used in more complicated games like flight simulators and games like MechWarrior. They use goal -based engines - each entity within the game is assigned a certain goal, be it 'protect base', 'attack bridge', 'fly in circles'. As the game world changes, so do the goals of the various entities.
Fall 2012 42
Processing Models
Polling FSM update
frequency Easy to implement
and debug Inefficiency (Little
Red example)
Event-driven Publish-subscribe
messaging system Game engine sends
event messages to individual FSMs
An FSM subscribes only to the events that have the potential to change the current state
Higher efficiency, non-trivial implementation
Fall 2012 43
Efficiency and Optimization
In AI, FSM is the most efficient technology availableYet, there is always room for improvement Level of Detail: depending on the
condition (e.g., distance with player), use different FSM, or different update frequency
Fall 2012 44
References
Web references: www.gamasutra.com/features/19970601/build_brain
s_into_games.htm csr.uvic.ca/~mmania/machines/intro.htm www.erlang/se/documentation/doc-4.7.3/doc/design_
principles/fsm.html www.microconsultants.com/tips/fsm/fsmartcl.htm http://www.angelfire.com/dragon/letstry/tutorials/dfa/
Game Programming Gems Sections 3.0 & 3.1
It’s very very detailed, but also some cute programming
Well, the sad news …
Fall 2012 45
Behavior trees
Fall 2012 46
Additional Materials
Fall 2012 47
Fall 2012 48
Intuitive Design
Say, a simple teletube baby has three states: idle, run, and waveScenario: When an idle laalaa sees a butterfly,
it waves to it. When the butterfly flies away, it returns to idle
When an idle laalaa sees a mouse, it flees away. When the mouse is no longer in sight, it returns to idle
Fall 2012 49
Laalaa
idle
flee
wave
mouse
~mouse
~butterfly
butterfly
How to make sure the design complete? I.e., all states and transitions are considered
Is there any systematic way to develop an FSM?
Fall 2012 50
Quake Bot Example (first-cut)
Types of behavior to capture: Wander randomly if don’t see or hear an
enemy When see enemy, attack When not see enemy and hear an enemy,
chase enemy When die, re-spawn (new a bot from start)
Events: see enemy, hear enemy, dieStates: wander, attack, chase, spawn
Fall 2012 51
Remark
With 3 events, potentially there should be 23 states: (E,S,D)=(0,0,0),(1,0,0),(0,1,0), …,(1,1,1)
Some doesn’t make sense E.g., ESD = 111
Name and create a state for the ones that we want to consider Wander (ESD=000) Chase (ESD=010) Attack (ESD=1x0), x for dont-care Die (ESD=xx1)
Fall 2012 52
FSM (first-cut)
Events: E: see an enemy S: hear a sound D: die
States: E: enemy in sight S: sound audible D: dead
D
~E
ED
~S
S
D
E S D
Problem: Can’t go directly from attack to chase. Why not?
Spawnxx1
Wander000
Attack1x0
Chase010
start
Fall 2012 53
FSM (first-cut)
Events: E: see an enemy S: hear a sound D: die
States: E: enemy in sight S: sound audible D: dead
D
~E
ED
~S
S
D
E S DSpawn
xx1
Wander000
Attack100
Chase010
start E
Attack+S110
~E
~S
S
D
Extra state needs to be defined
Extra state needs to be defined
Fall 2012 54
Quake Bot Example (refined)
Types of behavior to capture: Wander randomly if don’t see or hear an enemy When see enemy, attack When not see enemy and hear an enemy, chase
enemy When die, respawn
Extensions: When health is low and see an enemy, retreat When see power-ups during wandering, collect
them [hierarchical FSM]
Fall 2012 55
Example FSM with Retreat
SpawnD
(-E,-S,-L)
Wander-E,-D,-S,-L
E
-SAttack-EE,-D,-S,-L
E
Chase-E,-D,S,-L
S
D
S
D
D
Retreat-EE,-D,-S,L
L
-E
Retreat-S-E,-D,S,L
Wander-L-E,-D,-S,L
Retreat-ESE,-D,S,L
Attack-ESE,-D,S,-L
E
E-E
-L
S
-S
L
-E E
L-L
-L
-L
L
D
• States:– E: enemy in
sight– S: sound
audible– D: dead– L: Low health
A lot more states got added
Fall 2012 56
Hierarchical FSM Example
Note: This is not a complete FSM All links between top level states still
exist Need more states for wander
StartTurn Right
Go-throughDoor
Pick-upPowerup
Wander Attack
Chase
Spawn
~E
E~S
SD
~E
Fall 2012 57
Non-Deterministic HierarchicalFSM (Markov Model)
Adds variety to actionsHave multiple transitions for the same eventLabel each with a probability that it will be takenRandomly choose a transition at run-timeMarkov Model: New state only depends on the previous state
Attack
Start
Approach
Aim & Jump &Shoot
Aim & Slide Left& Shoot
Aim & Slide Right
& Shoot .3.3
.4
.3.3
.4