Team Member AI in a FPS

AI Game Programming Wisdom 2

Chapter 3.3

Goals

• Teammates that back the player up

• Teammates that follow instructions

• Teammates that survive until end of game

• Don’t want: teammates that get in the player’s way or wander off

Correct Positioning

• Player’s F.O.V. should not be obstructed

• Balance between various aspects:– Visible teammates add to feeling of the game– Consider accuracy of the player’s weapon

• Should always leave area between player and targets clear

Balancing NPC Movement

• NPC movement while in player’s F.O.V. should be smooth so as to not detract from game play

• Should move to clear player’s F.O.V., but should move as little as possible

• Another consideration: stance, could the teammates crouch or crawl past the player? (Again, take into account accuracy of player’s weapon, and lines of fire)

Correct Movement

• Multiple targets means a greater chance that at least one line of sight is obstructed

• Moving out of the way of one target could put a teammate in front of another target

• Solutions:1. Consider all targets & find the smaller angle

that would leave a clear view for player2. Consider the player’s heading and move

away from it

Correct Movement

Correct Movement

May result in a longer path for teammate, but ensures the teammate will not cross player’s line of sight

New Considerations

• Target Threat:– Proximity– Level of awareness– Readiness

• AI must be able to act to keep the target with the highest threat clearEx. Teammate might spot a patrol 500m away

while there is an enemy just 10m away

Full Spectrum Warrior!

Guidelines

• Maximum straying distance: don’t want the player to have to chase teammates

• AI should not take the fun from the player (e.g. Should not be the first around corners, through doors)

• If ordered to take the initiative, teammates should behave strategically

• Should move at same speed as player but lag behind slightly

Guidelines

• Context: teammates should always respect the context and be able to intelligently change between contexts.Ex. In a stealth game teammates would avoid

firing on targets unless they were instructed to do so, or unless they were engaged by the target

• Can set context for a individual teammates or entire team

Remove?

Correct Behavior

• NPC behavior can be defined in rule based terms, focusing on strategic aspects of individual and the entire team

• 3 behaviors common to FPSs1. Use of cover

2. Selective firing

3. Reloading

Correct Behavior – Use of Cover

• Use of cover keeps the teammates alive as long as the player Should favor use of cover in path finding algorithms

• When entering a new area, teammates should map out possible points of cover

• Must take into consideration maximum straying distance

Remove?

Correct Behavior – Selective Firing

• Should only fire when target is in range and in view

• Realistic games: fire in short bursts to improve accuracy

• Balance accuracy with response time– Low response time could result in AI that

appears unaware and less intelligent– Response time should be >= enemy response

time to ensure the team survives more often

Remove?

Correct Behavior - Reloading

• Teammates should count the number of rounds used & find safe cover to reload

• Staggering firing of teammates ensure only one will need to reload at a given time

• While one teammate is reloading, another should be able to keep up the attack

Remove?

Supporting the Player• At least one teammate should cover

player’s flanks– Report enemy sightings so player can take

action and to help guide player through game– If enemy sighted and team too busy,

teammate should engage autonomously– Should concentrate on targets other than the

player’s target– Should take into consideration threat level– Might chose target that is farthest from

player’s line of sight

Table of Actions

• A table of actions can help the team function more effectively and give a greater illusion of intelligence

Action Immediate action?

Whole team required?

Cover Me No No

I need backup

Yes No

Attack my target

Yes No

Close in No Yes

Spread out No Yes

Move to waypoint

No Yes

Hold Yes Yes

The Player is Most Important• Player should be allowed to play the

game, supported by the team – team should never hinder game play

• NPCs should chose weapon according to player’s weapon

• NPCs should defer objects to player– If an NPC does get an object, most needy one

should have first choice

• NPCs should have different reaction times for distant and near enemies, to allow the player to respond first

Rainbow Six Demo

Implementation

• Layered approach– Four layers– Lower layers take higher priority– Caters to player to improve game play

Implementation – Player Awareness

• Primary concern is to avoid player’s line of sight– Player’s position, orientation, current target,

and possible targets should all be taken into consideration

• Secondary concern is with avoiding other NPCs’ lines of sight– A hierarchical ordering of NPCs solves this

Implementation – Threat Awareness

• NPCs should be looking for targets using line of sight algorithms

• Enemies sighted should be reported

• Movement gets deferred to player awareness level when not searching or engaging enemies

Implementation - Environment Awareness

• When moving into a new area, teamates should identify cover and areas that are impermissible due to danger or exposure

• Environment can be used to reduce NPC straying – boundary restrictions

Implementation - Team Manager• Shared by all teammates

• Contains all the team’s data

• Helps coordinate the whole team

• NPCs query manager to find out where and how to move (sets context)

• Tell NPCs if they can pick up objects

• Consults environ. level to get stray dist.

• Consults threat level to control rate of fire

• Never consider the player (unpredictable)

Teammate AvailabilityN = # enemies in covering areaO = # enemies in rangeP = # enemies threatening teamQ = supporting another team member (1/0)(Multiply by infinity to make sure busy teammate is

never selected)

Availability = (1+N) (1+O) (1+P) + (Q*Inf.)

Lower numbers indicate greater availability.Set thresholds to determine if a teammate is busy

Fire Staggering

• Should happen naturally

• Teammates should not target and fire at the same enemy

• Team manager should monitor ammo levels– Could widen cover area of one NPC to reduce

another NPC’s cover area, allowing them to conserve ammo

– When one NPC runs out the other can then cover the whole area while the former reloads

Applying Goal-Oriented Action Planning to Games

Autonomous decisions to activate behaviors intended to accomplish most relative goals

AI Game Programming Wisdom

3.4

Goal Oriented Action Planning (GOAP)

• A decision making architecture that allows NPCs to decide what to do next, and how to do it

• Produces less repetitive, less predictable behavior– Can be adapted to fit the current situation

• Facilitates authoring, maintaining, and reuse of behaviors

• Active goal determines NPC’s behavior through a hard-coded sequence of state transitions

FEAR!

GOAP

• An agent uses a planner to formulate a sequence of actions that will satisfy some goal

• GOAP does not replace the need for FSMs, but greatly simplifies those required– Plan becomes a sequence of actions where

each action represents a state transition– Separating the state transition logic from

states greatly simplifies the underlying FSM

GOAP - Goals

• A goal is any condition that an agent wants to satisfy– Agent may have any number of goals– Only one goal is active at any given moment– A goal knows how to calculate its relevance– A goal knows when it has been satisfied– A goal does not include a plan, but merely

conditions that must be met

GOAP - Plan

• A plan satisfying a goal is any valid sequence of actions that move the agent from some starting state to some a state satisfying the goal– Could be multiple plans that satisfy some goal– Only need to find one

• The planner can be given hintsEx. Associate costs with actions, look for lowest

cost to reach goal

GOAP - Action

• An action is a single, atomic step that makes an agent do somethingEx. GoToPoint, ActivateObject

– Could be short or infinitely longEx. Reload, Attack

– An action knows its preconditions and effects• Allows for easy chaining of actionsEx. Reload and Attack

GOAP – Plan Formulation• An agent provides a goal to a handler called a

planner• The planner searches the space of actions for a

sequence that will take the agent from its starting state to a goal state (formulating the plan)

• If planner is successful, the agent follows the plan until completion or until another goal becomes more relevant

• If another goal activates or plan becomes invalid, agent aborts and formulates another plan

GOAP – Plan Formulation

Draw Weapon

Go to Cover

Sleep

Go to Point

Reload Weapon

Dodge

Activate Object

Idle

Attack

Start

Goal State

Action Space

Kill Enemy Goal

GOAP – Runtime Benefits

• An agent can custom fit actions to current surroundings

• Agent can dynamically find alternate solutions to problems– Ex. Soldier out of ammo, but laser near by

GOAP – Development Benefits

• Handling every possible situation for every agent can quickly become difficult to manage

• Adding design requirements in GOAP is simply a matter of adding actions and defining their preconditions and effectsEx. Adding TurnLightsOn action, precondition to

GoTo action

• GOAP helps prevent invalid plans to be formed, where human error could easily contribute to invalid plans in hand-written code

GOAP – Variety Benefits

• GOAP can be used to create a variety of characters simply by giving the planner different subsets of actions

• One character’s actions can easily be replaced with functionally similar, but aesthetically different actions for another characterEx. OpenDoor for regular soldier and brute

• Extension: add preconditions to certain actions for mood, energy level, etc.

GOAP – Implementation Considerations

• Challenges:

1. Determining best method for search the space of actions

2. Planner must be able to represent the game world in a clear, compact manner

GOAP – Planner Search

• Similar to path finding, so the same algorithm can be used: A*

• The planner need only implement its own classes for A*’s nodes, map, and goals

• A* involves the calculation of the cost of a node and the heuristic distance from a node to a goal

• Can search forward or backwards– Forward: exhaustive brute force search– Backwards: more efficient and intuitive

GOAP – Planner Search

• Node Cost Calculation: sum of costs of actions that take the world to the state represented by the node– A lower cost is preferable

• Hueristic Cost Calculation: sum of the number of unsatisfied properties of the goal state

GOAP – World Representation

• Planner must be able to apply preconditions and effects of action to realize when it has reached the goal state

• Only need to represent minimal number of properties of the world state that are relevant to the goal the planner is trying to satisfy

• As planner adds actions, goal state grows with the preconditions of actions added

GOAP – World Representation

• Can construct a list of world property structures for each action taken– Enumerated attribute key– A value– A handle to a subject (the character)

• Pass desired properties to the goal state

GOAP – World Representation

struct SWorldProperty {

GAME_OBJECT_ID hSubjectID;

WORLD_PROP_KEY eKey;

union value {

bool bValue;

float fValue;

int nValue;

}

}

Ex. KillEnemy goal

SWorldProperty Prop;Prop.hSubjectID = hShooterID;Prop.eKey = kTargetIsDead;Prop.bValue = true;

Remove?

GOAP – Final Thought• Context preconditions – something that

needs to be true for an action to take place, but the planner will never try to satisfy itEx. Attack action, must be in range and F.O.V.

• Planner must run two functions, one to make a plan and check all preconditions and another to verify all context preconditions are satisfied for the plan

GOAP - Optimization

• Possible optimizations for GOAP:– Optimizing the search algorithm (which takes

the general form of A* but is game specific)– Cache results from previous searches– Distributed plan formulation over several

updates– Use of context preconditions to prune search

trees.