11

Artificial Intelligence in Artificial Intelligence in GamesGames

Ryan Donnellydonnelry@uwplatt.edu

22

What is AI in Games?What is AI in Games? Techniques used in computer and video Techniques used in computer and video

games to produce the illusion of intelligence games to produce the illusion of intelligence in the behavior of non-player characters in the behavior of non-player characters

A game must ‘feel’ naturalA game must ‘feel’ natural– Obey laws of the gameObey laws of the game– Characters aware of the environmentCharacters aware of the environment– Path finding (A*)Path finding (A*)– Decision makingDecision making– PlanningPlanning

Game ‘bookkeeping’, scoringGame ‘bookkeeping’, scoring ~50% of project ~50% of project

time building AItime building AI

33

Computer Game TypesComputer Game Types Strategy gamesStrategy games

– Real Time Strategy (RTS)Real Time Strategy (RTS)– Helicopter viewHelicopter view

Role Playing Games (RPG)Role Playing Games (RPG) Action gamesAction games

– First person shooter (FPS)First person shooter (FPS)– Sports gamesSports games

44

Goals of Game AIGoals of Game AI Be ‘fun’Be ‘fun’

– Reasonable challenge with natural behaviorReasonable challenge with natural behavior No Cheating!No Cheating!

– AI has bonuses over human players such as:AI has bonuses over human players such as: Giving more damageGiving more damage Having more healthHaving more health Driving fasterDriving faster Etc.Etc.

– Used to increase difficultyUsed to increase difficulty– Draws away focus to program more human-like bots.Draws away focus to program more human-like bots.

Run fastRun fast Use minimal memoryUse minimal memory

55

Game AI History -1980Game AI History -1980 1960’s1960’s

– First computer games (SpaceWar)First computer games (SpaceWar)– Board games against the computer (Chess)Board games against the computer (Chess)

1970’s1970’s– Atari (1972)Atari (1972)

Nolan BushnellNolan Bushnell PongPong

– First AI implemented into gamesFirst AI implemented into games Stored patternsStored patterns Space Invaders (1978)Space Invaders (1978)

– Distinct moving patternsDistinct moving patterns Galaxian (1979)Galaxian (1979)

– More complex and varied enemy More complex and varied enemy movementsmovements

– 1-2% of CPU time spent on AI1-2% of CPU time spent on AI

66

Game AI History 1980-Game AI History 1980- 1980’s1980’s

– Fighting gamesFighting games Karate Champ (1984)Karate Champ (1984)

– AI defeated a human player in chess AI defeated a human player in chess for the first time (1983)for the first time (1983)

– Pac-Man (1980)Pac-Man (1980)

77

Game AI History 1980-Game AI History 1980- 1990’s1990’s

– Sports gamesSports games Madden FootballMadden Football

– FPS and RTS gamesFPS and RTS games RTS games had problemsRTS games had problems

– Path findingPath finding– DecisionsDecisions– Many moreMany more– Dune II: Enemy attacked in a Dune II: Enemy attacked in a

bee line and used cheatsbee line and used cheats RTS games did get betterRTS games did get better

– WarCraftWarCraft First game to implementFirst game to implement

path-finding at such apath-finding at such alarge scalelarge scale

88

Game AI History 1980-Game AI History 1980- 1990’s (cont.)1990’s (cont.)

– Finite state machinesFinite state machines– Neural networksNeural networks

Battlecruiser 3000AD (1996)Battlecruiser 3000AD (1996)– Deep Blue defeats chess champ Gary Kasparov (1997)Deep Blue defeats chess champ Gary Kasparov (1997)

Chess playing computer developed by IBMChess playing computer developed by IBM Inspires AI developersInspires AI developers http://www.research.ibm.com/deepblue/games/game6/html/http://www.research.ibm.com/deepblue/games/game6/html/c.2.shtmlc.2.shtml

– Graphic cards allowing for more CPU timeGraphic cards allowing for more CPU time– 10-35% of CPU time spent on AI10-35% of CPU time spent on AI

99

Game AI History 1980-Game AI History 1980- 2000’s2000’s

– More games using neural networksMore games using neural networks Black & White (2001)Black & White (2001) Collin McRae Rally 2 (2001)Collin McRae Rally 2 (2001)

– HyperthreadingHyperthreading More sophisticated AI engines while More sophisticated AI engines while

simultaneously creating a more realistic 3D simultaneously creating a more realistic 3D environmentenvironment

– Core DuoCore Duo Even more complex AI enginesEven more complex AI engines

1010

AI in Different Game AI in Different Game TypesTypes FPS & RPGFPS & RPG

– AI is in opponents, teammates, and AI is in opponents, teammates, and extra charactersextra characters

RTSRTS– AI on all sidesAI on all sides

Sports GamesSports Games– AI is in opponents and teammatesAI is in opponents and teammates

1111

AI in FPS-type GamesAI in FPS-type Games Layered AI StructureLayered AI Structure

– Bottom layers = trivialBottom layers = trivial Determine pathsDetermine paths

– Top layers = non-trivialTop layers = non-trivial Reasoning and behaviorReasoning and behavior

Event Driven EngineEvent Driven Engine– Action based on eventsAction based on events– Good Idea to use leakingGood Idea to use leaking

buckets to make morebuckets to make moreflexibleflexible

Leaking BucketsLeaking Buckets– Buckets leak contents over timeBuckets leak contents over time– Script with the most filled bucket gets executedScript with the most filled bucket gets executed

1212

AI in FPS-type Games AI in FPS-type Games Cont.Cont. Path-FindingPath-Finding

– Based on graphs describing the worldBased on graphs describing the world A(*)A(*)

– Most commonly usedMost commonly used– Guaranteed to find shortest pathGuaranteed to find shortest path

Animation SystemAnimation System– Play appropriate sequence of animation at the chosen Play appropriate sequence of animation at the chosen

speedspeed– Play different animation sequences for different body Play different animation sequences for different body

parts (i.e. run and aim, and shoot and reload weapon parts (i.e. run and aim, and shoot and reload weapon while still running)while still running)

– Inverted kinematicsInverted kinematics Process of computing the pose of a human body from a set of Process of computing the pose of a human body from a set of

constraintsconstraints– i.e. An IK animation system can appropriately calculate the i.e. An IK animation system can appropriately calculate the

parameters of arm positioning animation so that the hand can parameters of arm positioning animation so that the hand can grab an object located on a table or shelfgrab an object located on a table or shelf

1313

Representation of the Representation of the World in an FPS-type World in an FPS-type Game Game

1414

AI in RTS-type GamesAI in RTS-type Games Path-FindingPath-Finding

– Handle collisionsHandle collisions– A(*)A(*)

Event Driven EngineEvent Driven Engine Maps Represented by a Rectangular GridMaps Represented by a Rectangular Grid

– Module that analyzes the game map uses a goal Module that analyzes the game map uses a goal driven enginedriven engine

Take highest rank goal and process it.Take highest rank goal and process it. Smaller sub-goals are created as needed and are Smaller sub-goals are created as needed and are

processed until the goal has been fulfilled.processed until the goal has been fulfilled. Analyzes terrain and a settlement is built based on Analyzes terrain and a settlement is built based on

evaluation of the terrainevaluation of the terrain Decides when cities should be built and how Decides when cities should be built and how

reinforcements should be placedreinforcements should be placed

1515

AI in RTS-type Games AI in RTS-type Games Cont.Cont. Interaction between event driven and goal Interaction between event driven and goal

driven engine example:driven engine example:– A building gets blown up by an air strikeA building gets blown up by an air strike– This sparks the event based engine to give a new This sparks the event based engine to give a new

goal to the goal based engine to increase air goal to the goal based engine to increase air defensesdefenses

– The goal based engine responds by moving units The goal based engine responds by moving units that are capable of air defense into position.that are capable of air defense into position.

1616

Representation of the Representation of the World in an RTS-type World in an RTS-type GameGame

1717

AI in RPG-type GamesAI in RPG-type Games Little AILittle AI

– Random encountersRandom encounters More common in games where fighting More common in games where fighting

and gaining levels is more importantand gaining levels is more important– Scripted behaviorScripted behavior

Often coupled with some minor AIOften coupled with some minor AI Common in games depending more on Common in games depending more on

their story line than other thingstheir story line than other things– Often a combination of bothOften a combination of both

1818

AI in Sports GamesAI in Sports Games CheatingCheating Racing gamesRacing games

– SegmentationSegmentation Track gets split into small sectors.Track gets split into small sectors. Each element gets its length calculatedEach element gets its length calculated Fragments used to obtain characteristics of the road in the Fragments used to obtain characteristics of the road in the

vehicle’s closest vicinityvehicle’s closest vicinity In effect, the computer knows it should slow down because In effect, the computer knows it should slow down because

it’s approaching a curve or an intersectionit’s approaching a curve or an intersection– OptimizationOptimization

Two curves are marked on the trackTwo curves are marked on the track First represents the optimal driving trackFirst represents the optimal driving track Second represents the track used when overtaking Second represents the track used when overtaking

opponentsopponents– AI system must analyze terrainAI system must analyze terrain

Detect obstacles lying on the roadDetect obstacles lying on the road

1919

AI in Sports Games AI in Sports Games Cont.Cont. Racing games (cont.)Racing games (cont.)

– Strict co-operation with physics Strict co-operation with physics modulemodule Physics module provides information Physics module provides information

such as when the car is skiddingsuch as when the car is skidding The AI system, having received the The AI system, having received the

information that the car is skidding, information that the car is skidding, should react appropriately and try to get should react appropriately and try to get the vehicle’s traction back under controlthe vehicle’s traction back under control

2020

Popular AI Algorithms Popular AI Algorithms Used In Computer Used In Computer GamesGames A(*)A(*) Finite State MachinesFinite State Machines Artificial Neural NetworksArtificial Neural Networks

2121

A(*) AlgorithmA(*) Algorithm Goal: Find shortest pathGoal: Find shortest path PrerequisitesPrerequisites

– GraphGraph– Method to estimate distance between points Method to estimate distance between points

(heuristic)(heuristic) Basic MethodBasic Method

– Try all paths?Try all paths? Takes timeTakes time

– Orient search towards targetOrient search towards target Minimizes areas of the map to be examinedMinimizes areas of the map to be examined Uses heuristics that indicate the estimated cost of Uses heuristics that indicate the estimated cost of

getting to the destinationgetting to the destination Main advantageMain advantage

2222

A(*) AlgorithmA(*) Algorithm AlgorithmAlgorithm

– Open listOpen list Nodes that need to be considered as possible starts for further Nodes that need to be considered as possible starts for further

extensions of the pathextensions of the path– Closed listClosed list

Nodes that have had all their neighbors added to the open listNodes that have had all their neighbors added to the open list– G scoreG score

Contains the length or weight of the path from the current node Contains the length or weight of the path from the current node to the start nodeto the start node

Low lengths are betterLow lengths are better Every node has a G scoreEvery node has a G score

– H scoreH score HeuristicHeuristic Resembles G score except it represents an estimate of the Resembles G score except it represents an estimate of the

distance from the current node to the endpointdistance from the current node to the endpoint To find shortest path, this score must underestimate the To find shortest path, this score must underestimate the

distancedistance

2323

A(*) AlgorithmA(*) Algorithm Algorithm (cont.)Algorithm (cont.)

– Start with an empty closed list and Start with an empty closed list and just the starting point in the open listjust the starting point in the open list

– Every node has a G score and the Every node has a G score and the node that was used to arrive at this node that was used to arrive at this node (Parent node)node (Parent node)

2424

A(*) AlgorithmA(*) Algorithm Algorithm (cont.)Algorithm (cont.)

– Extend the pathExtend the path Calculate the H scores of the nodes in the open list using a Calculate the H scores of the nodes in the open list using a

heuristic method.heuristic method. Pick the node (P) in the open list for which the sum of the Pick the node (P) in the open list for which the sum of the

G and H scores is the lowest. Note: If the open list is G and H scores is the lowest. Note: If the open list is empty then no pathempty then no path

For every point adjacent to P not in the closed or open list, For every point adjacent to P not in the closed or open list, add it to the open list. The previous nodes for these new add it to the open list. The previous nodes for these new nodes is P, and their G score is the G score of P plus the nodes is P, and their G score is the G score of P plus the distance between the new node and P. If it was already in distance between the new node and P. If it was already in the open list, check it’s current G score, and if the new G the open list, check it’s current G score, and if the new G score would be less than the current one update the G score would be less than the current one update the G score and previous node, otherwise leave it alone.score and previous node, otherwise leave it alone.

If the new point is the destination point, you have found If the new point is the destination point, you have found your path.your path.

Move P to the closed list and start overMove P to the closed list and start over

2525

A(*) AlgorithmA(*) Algorithm Example:Example:

– Manhattan methodManhattan method Calculate total # of squares moved Calculate total # of squares moved

horizontally and vertically to reach horizontally and vertically to reach target, ignoring diagonal movement and target, ignoring diagonal movement and obstacles.obstacles.

2626

A(*) AlgorithmA(*) Algorithm Example (cont.):Example (cont.):

2727

A(*) AlgorithmA(*) Algorithm Example (cont.): Notice: 2 squares = 54Example (cont.): Notice: 2 squares = 54

– Can be faster to choose last one added to the open listCan be faster to choose last one added to the open list This biases the search in favor of squares that get found This biases the search in favor of squares that get found

later on in the search, when you have gotten closer to the later on in the search, when you have gotten closer to the targettarget

2828

A(*) AlgorithmA(*) Algorithm Example (cont.):Example (cont.):

2929

A(*) AlgorithmA(*) Algorithm Example (cont.):Example (cont.):

3030

A(*) AlgorithmA(*) Algorithm Example (cont.):Example (cont.):

3131

Finite State MachinesFinite State Machines Each object in a game can have a number of states during its Each object in a game can have a number of states during its

life.life.– i.e. patrolling, attacking, resting, etc.i.e. patrolling, attacking, resting, etc.

Model of behavior composed of:Model of behavior composed of:– StatesStates

Stores information about the past, i.e. it reflects the input changes Stores information about the past, i.e. it reflects the input changes from the system start to the present momentfrom the system start to the present moment

– TransitionsTransitions Indicates a state change and is described by a condition that would Indicates a state change and is described by a condition that would

need to be fulfilled to enable the transitionneed to be fulfilled to enable the transition– ActionsActions

Entry actionEntry action– executed when entering the stateexecuted when entering the state

Exit actionExit action– executed when exiting the stateexecuted when exiting the state

Input actionInput action– executed depending on present state and input conditionsexecuted depending on present state and input conditions

Transition actionTransition action– executed when performing a certain transitionexecuted when performing a certain transition

3232

Finite State Machines Finite State Machines Cont.Cont. Advantage: Can divide Advantage: Can divide

implementation of each games implementation of each games object’s behavior into smaller object’s behavior into smaller fragmentsfragments– Easier to debug and extendEasier to debug and extend

3333

Finite State MachinesFinite State Machines State DiagramState Diagram

3434

Finite State Machines Finite State Machines Example: Pacman Example: Pacman GhostGhost

3535

Artificial Neural Artificial Neural NetworksNetworks BrainBrain

– Receives inputReceives input– Processes inputProcesses input– Communicates the outputCommunicates the output

Relies on the cooperation of the individual neurons Relies on the cooperation of the individual neurons within the network to operatewithin the network to operate

– If some neurons are not functioning, the network can If some neurons are not functioning, the network can still perform its overall functionstill perform its overall function

TrainableTrainable– Learn to solve complex problems from a set of Learn to solve complex problems from a set of

examplesexamples– Generalizes the “acquired knowledge” to solve Generalizes the “acquired knowledge” to solve

unforseen problemsunforseen problems

3636

Artificial Neural Artificial Neural NetworksNetworks

3737

Artificial Neural Artificial Neural NetworksNetworks Neural Networks in Games?Neural Networks in Games?

– Trendy topic in the late 90’s into 00’sTrendy topic in the late 90’s into 00’s Huge potential in computer gamesHuge potential in computer games

– Collin McRae Rally 2 (2001)Collin McRae Rally 2 (2001) Total successTotal success The trained artificial neural network is responsible for The trained artificial neural network is responsible for

keeping the computer player’s car on the track while keeping the computer player’s car on the track while letting it negotiate the track as quickly as possibleletting it negotiate the track as quickly as possible

Input parameters: curvature of the road’s bend, Input parameters: curvature of the road’s bend, distance from the bend, type of surface, speed, or the distance from the bend, type of surface, speed, or the vehicles propertiesvehicles properties

Output: selected in a way so that the car travels and Output: selected in a way so that the car travels and negotiates obstacles or curves at a speed optimal for negotiates obstacles or curves at a speed optimal for the given conditions. the given conditions.

3838

Artificial Neural Artificial Neural NetworksNetworks Obstacles Limiting Neural Networks’ Obstacles Limiting Neural Networks’

Application in GamesApplication in Games– Problems choosing appropriate inputProblems choosing appropriate input– Neural network’s sensitivity to changes in a Neural network’s sensitivity to changes in a

game’s action logic, and the need for re-game’s action logic, and the need for re-training the network whenever such a training the network whenever such a situation occurssituation occurs

– Rather complicated theory, and difficulties Rather complicated theory, and difficulties with debugging in case of problemswith debugging in case of problems

– Time-consuming and complicated process Time-consuming and complicated process of training the networkof training the network

3939

Artificial Neural Artificial Neural NetworksNetworks How to take advantage of an artificial neural network How to take advantage of an artificial neural network

in a simple game?in a simple game?– Need to know what kinds of information the neural Need to know what kinds of information the neural

network should provide to help solve the problemnetwork should provide to help solve the problem– Choose input parametersChoose input parameters

Choose in a way that its different combinations will let the Choose in a way that its different combinations will let the neural network learn to solve problems which haven’t neural network learn to solve problems which haven’t appeared in the example set of signalsappeared in the example set of signals

Should represent as much information about the game world Should represent as much information about the game world as possibleas possible

– i.e. vectors of relative positions of the nearest obstacle or i.e. vectors of relative positions of the nearest obstacle or opponent, the enemies strength, etc.opponent, the enemies strength, etc.

– Acquire set of input data for trainingAcquire set of input data for training Significant effortSignificant effort

– Train the neural networkTrain the neural network Mixed with simultaneous testing to make sure the game is Mixed with simultaneous testing to make sure the game is

not too difficult, or if too easy and in need of further trainingnot too difficult, or if too easy and in need of further training

4040

Artificial Neural Artificial Neural NetworksNetworks Fuzzy logicFuzzy logic

– Often used with neural networksOften used with neural networks– Conversion from computer’s reasoning into Conversion from computer’s reasoning into

something more strongly resembling the something more strongly resembling the way a human thinksway a human thinks

– Usually in the form:Usually in the form: IF variable IS set THEN action IF variable IS set THEN action

– i.e.i.e. IF road IS dry THEN maintain normal speed IF road IS dry THEN maintain normal speed IF road IS wet THEN slow down IF road IS wet THEN slow down

4141

ConclusionConclusion Games With No AI?Games With No AI?

– Not possible!Not possible!– Every game with computer controlled Every game with computer controlled

characters/opponents uses some sore of AIcharacters/opponents uses some sore of AI Game AI has come a long way since Game AI has come a long way since

the 1970sthe 1970s Future looks brightFuture looks bright

– Neural networks are the future of Neural networks are the future of computer games and a future that is not computer games and a future that is not that distant anymorethat distant anymore

4242

ReferencesReferences1)1) Grzyb, Janusz. Grzyb, Janusz. Artificial Intelligence in Games.Artificial Intelligence in Games.

Software Developer’s Journal. June 2005.Software Developer’s Journal. June 2005.2)2) Game Artificial Intelligence.Game Artificial Intelligence. Wikipedia Ecyclopedia. Wikipedia Ecyclopedia.

September 7, 2006. September 7, 2006. http://http://en.wikipedia.org/wiki/Game_artificial_intelligenceen.wikipedia.org/wiki/Game_artificial_intelligence

3)3) Petersson, Anders. Petersson, Anders. Artificial Intelligence in Games.Artificial Intelligence in Games. WorldForge Newsletter. August 2001. WorldForge Newsletter. August 2001. http://worldforge.org/project/newsletters/August200http://worldforge.org/project/newsletters/August2001/AI/#SECTION000200000000000000001/AI/#SECTION00020000000000000000

4)4) Popovic, Zoran; Martin, Steven; Hertzmann, Aaron; Popovic, Zoran; Martin, Steven; Hertzmann, Aaron; Grochow, Keith. Grochow, Keith. Style-Based Inverse Kinematics.Style-Based Inverse Kinematics. 2004. 2004. http://http://grail.cs.washington.edu/projects/styleik/styleik.pdfgrail.cs.washington.edu/projects/styleik/styleik.pdf

5)5) A*.A*. The Game Programming Wiki. September 15, The Game Programming Wiki. September 15, 2006. 2006. http://http://gpwiki.org/index.php/A_stargpwiki.org/index.php/A_star