target selection for ai companions in fps gamesjtremb59/papers/fdg2014_paper_32.pdf · rpg games....

Target Selection for AI Companions in FPS Games

Jonathan TremblaySchool of Computer ScienceMcGill University, Montréal

Québec, [email protected]

Christopher DragertSchool of Computer ScienceMcGill University, Montréal


Clark VerbruggeSchool of Computer ScienceMcGill University, Montréal


ABSTRACTNon-player Characters (NPCs) that accompany the playerenable a single player to participate in team-based experi-ences, improving immersion and allowing for more complexgameplay. In this context, an Artificial Intelligence (AI)teammate should make good combat decisions, supportingthe player and optimizing combat resolution. Here we inves-tigate the target selection problem, which consists of pickingthe optimal enemy as a target in a modern war game. Welook at how the companion’s different strategies can influ-ence the outcome of combat, and by analyzing a variety ofnon-trivial First Person Shooter (FPS) scenarios show thatan intuitively simple approach has good mathematical jus-tification, improves over other common strategies typicallyfound in games, and can even achieve results similar to muchmore expensive look-up tree approaches. This work has ap-plications in practical game design, verifying that simple,computationally efficient target selection can make an ex-cellent target selection heuristic.

KeywordsArtificial Intelligence, Video Games

1. INTRODUCTIONNPC companions in modern games are intended to im-

prove player experience by augmenting player capability andacting as a surrogate teammate or co-adventurer. Simplecompanion AIs, however, often result in the companion de-monstrating a poor degree of cooperation [1], failing to rec-ognize and comport with player intention. This can inducefrustration and break immersion, especially during time-intensive and stressful situations like combat, where the de-gree of cooperation can result in significant differences in theoutcome of the battle.

In this work we explore the challenges of cooperation interms of target selection for FPS games. Optimal combatin FPS games relies heavily on AI team members select-ing appropriate targets, matching or complementing player

choices. Although this is a complex problem in its fullgenerality, for simple situations the problem can be feasi-bly expressed and analyzed formally. Using a mathematicalmodel, we first show that an intuitive, “threat-based” selec-tion strategy has theoretical justification for being optimal,at least within a simplified context. We then compare theresults of various common heuristics for target selection ina number of non-trivial FPS test environments, showing therelative performance of using a closest, strongest, weakest, orhighest threat enemy, as well as the effect of mimicking theplayer’s strategy. This analysis demonstrates relative perfor-mance of these heuristics, and in conjunction with a furthercomparison of threat to the optimal results of a full look-up tree search, shows that the theoretical basis for a threatheuristic remains valid in more complex environments.

An efficient means of optimizing AI combat strategy isimportant in FPS games, as it enables companions to moreclosely model the behaviour of clever teammates, eliminatesthe need for extra-narrative control of companions duringcombat, and gives another means of controlling game diffi-culty. More specific contributions of our work include:

• We derive a simple but justified target selection strat-egy through mathematical analysis of a reduced prob-lem space. Our approach is similar to prior analyticalwork by Churchill et al. and Furtak et al. [3, 2], butconsiders scenarios more common to FPS and Role-Playing Games (RPG) contexts. This analysis verifiesthat a “threat ordering,” prioritizing enemies based ontwo dimensions (health and attack), is theoreticallyoptimal.

• Through experimental analysis of several representa-tive scenarios, we show that threat ordering outper-forms other commonly used target selection strate-gies, even when considering the additional complexityof real-time gameplay, probabilistic hit and damage,character movement, and physical occlusion.

2. BACKGROUND & RELATED WORKOur work is motivated by trying to improve the integra-

tion of NPCs within the player game experience, in whichcontext we specifically concentrate on the combat target se-lection problem. Below we introduce and discuss relatedwork in both areas.

Human Player Game Experience - Considerable pre-vious work exists in trying to understand the gaming ex-perience [11]. For the purpose of this work, human experi-ence is modelled in terms of maximizing an economic utility

function, defined as the total team health. This maximiz-ing function is translatable as finding the highest rewardingNash equilibrium [8] in terms of selecting strategies. Withrespect to our game set-up, the player has to work withan NPC as a companion and trust her to make the rightchoice. When human players do not trust each other theynever reach an optimal equilibrium [4], and thus to maxi-mize the value of a companion a player needs to be able totrust her to make appropriate decisions.

This problem of trust is found in many modern FPS andRPG games. In Skyrim, for instance, the NPC companionoften exhibits sub-optimal strategy—targeting inappropri-ate enemies, entering battle too quickly, and generally in-terfering with the player’s combat intentions. This resultsin less effective combat, or even the death of the compan-ion. After losing trust in the companion, the player insteadadopts a sub-optimal strategy, such as keeping the compan-ion out of the combat [5].

Target Selection Problem - The general target selec-tion problem consists of two teams attacking each other,with each entity selecting an opponent to fight. The goal isto maximize the total team health at the end of the combat.We find this problem in FPS, real-time strategy, adventure,and other games. Work by others has closely examined thecase of 1 player against n enemies, showing that the prob-lem of minimizing health loss for even a single player is NP-Hard [3]. In our case we are interested in the case of a playerand her companion in a FPS or RPG scenario, a (small)team vs. team approach, which has mainly been previouslyaddressed through look-up tree search.

Look-up tree search consists of exploring the reachablestate space of the game. The naıve way to do it wouldbe to explore every possible strategy at every state, reach-ing all possible end-game states [7]. From there the optimalchoice is the one propagating back from the leaf with thebest solution. Even for small scenarios, however, exponen-tial growth in the size of the state space makes such anexhaustive search unrealistic in practice, at least within thecontext of real-time commercial games.

Look-up tree search typically assumes that players play indiscrete turns. In a real-time environment this does not hold,as entities take time to perform actions and may do multipleactions between opponent moves, magnifying the branchingfactor in a tree search. Churchill et al. explored ways toreduce the space of exploration in real time strategy gamesby using an alpha-beta search variant [2]. They were ableto solve an 8 vs. 8 combat using a game abstraction modelthat reduces the search space by focusing on important gamefeatures. Although this was done in real-time (50ms), therelative cost of this approach is still expensive for the rapidlychanging context of FPS games, where high frame-rates areparamount, and CPU is limited.

Heuristic valued approaches offer a more efficient solu-tion by attempting to approximate enemy value through aweighted function of observed data. The enemy with thehighest aggregate score is then selected as a target. In thegame Uncharted 2, for instance, they used a target selectingsystem that computed, for each enemy, a weighted functionof distance, cover, whether the enemy shot the player lasttime, who the enemy is targeting, close-range radius, and soon [9]. In general, their NPC would try to target differentenemies by adding a negative weight when multiple entitiestarget the same enemies, while staying on that enemy us-

ing a sticking factor. This approach can be effective, but asa complex and highly heuristic measure it must be closelytuned to a given context, and does not necessarily result inoverall better combat results.

Finally, we note that many games offer some level of man-ual control over target selection. In Drakensang, a playermay override a companion’s target choice, and direct her to-ward a specific enemy. Such extra-narrative control avoidsthe need for optimal target selection, but requires invok-ing an out-of-game interface, and if frequently necessary re-duces companions from teammates to tools. Middle groundis found in games such as Dragon Age 2, which lets the playerchoose very high-level strategies for her companion(s), tog-gling between aggressive or defensive mode. This reducesthe interaction complexity by hiding detail, but also makes itless obvious to the player what the companion will do, with-out giving confidence that the best results will be achieved.

3. MOTIVATING ANALYSISThe target selection problem exists in general within (Ba-

sic) Attrition Games, games wherein two sides, players andenemies, seek to eliminate the other. It has been previ-ously shown that solving Basic Attrition Games is exponen-tial (i.e., BAGWIN ∈ EXPTIME) [3], while the decisionproblem is PSPACE-hard, and is therefore not feasible inreal-time. Rather than solve the general form of the prob-lem, we aim instead to explore the faster heuristics that caneasily be computed in polynomial time and which are typi-cally applied in a FPS setting.

The goal of such heuristics is to find an enemy attack orderthat maximizes total remaining player health without evalu-ating the entire combat tree (state space). A naıve heuristicmight be to have all players attack the enemy with the low-est health, or target the enemy with highest attack, or evenattack the enemy with the highest health. However, any ofthese obvious heuristics will fare poorly under different sce-narios. For instance, attacking the enemy with the lowesthealth is a poor choice when there is an enemy with onlyslightly greater health but much greater attack power. In-tuitively, we should target enemies that are easy to kill andwhich may cause lots of damage first, and enemies whichare hard to kill but induce low player damage last. Theformer represent high threat enemies, while the latter haveless priority. Below we demonstrate that this simple modelactually has a well justified mathematical basis, describingfirst a discrete time context, and then extending the resultto a more realistic real-time environment. Note that thisformulation builds on the mathematical analyses found inwork by others, but deviates in order to accommodate ourcontext and goals.

Discrete Time - The following combat scenario will beused to define our basic attrition game. We begin with a setP of players (1 human and some companions) that are fight-ing a set E of enemies, where |P | = n, and |E| = m. Eachentity p ∈ P and e ∈ E has attack a and health h wherea, h ∈ N+. Fighting occurs in rounds, where the players andenemies each select an opposing entity to attack. A player’sattack is resolved by deducting pa from an enemy’s healtheh. If this leaves eh ≤ 0, the enemy is killed. Players hitfirst, meaning that a defeated enemy will not attack duringthe round in which it is killed. Any attack exceeding thehealth of the target is wasted. The game ends when eitherall players or enemies have been killed. Enemies will choose

their targets randomly, and for convenience, ph � ea, sim-ulating role playing games where players typically have anadvantage in order to ensure continued gameplay.

An enemy will deal damage each round until it is dead, sohealth savings for the player are maximized when the enemyis killed as quickly as possible. We express the maximumhealth savings S(e) for an enemy e as

S(e) = ea · (Tactual − Tα(e)) (1)

where T is the length of combat, and Tα(e) is the minimumlength of time needed to kill e. Unfortunately, Tactual andTα(e) are variable based on target assignment and the de-gree of overkill (when pa > eh). We can lower-bound Tactualas

Tactual ≥⌈EhPa

⌉(2)

where Pa is the players total attack, and Eh is the enemiestotal attack. However, the possibility of overkill means theactual combat length may exceed T . For instance, considerthe sitation where n = 1. It will take at least m turns todefeat m enemies, regardless of Eh and Pa. Instead, weapproximate Tactual using

T '∑e∈E

⌈ehPa

⌉(3)

This provides a reasonable estimate since it accounts forthe number of enemies. It does allow for overestimationof Tactual (e.g., in the case where every player can kill anyenemy in a single attack and n ≥ m), but this overestima-tion turns out to be necessary. Consider the situation whereT = Tα(e) = 1. This means that S(e) = 0 for all e ∈ E.If there is overkill, Tactual could be greater than T , yet oursavings estimates are all zero, providing no guidance. Over-estimating guarantees that we maintain information aboutenemy attacks and thus can still differentiate targets evenin the presence of overkill.

In Eq. (1), we also need Tα(e), which is given by

Tα(e) ≥⌈

ehPe,a

⌉(4)

where Pe,a is the total attack of the subset of players target-ing e. We use this subset of attack values to reduce overkill.If pa > eh, then it would not make sense to consider allPa, so using this reduced attack value allows us to take intoaccount the effects of spreading out attacks among enemies.With values for T and Tα(e), we now expand Eq. (1) to getour final equation for savings

S(e) = ea ·

(∑e∈E

⌈ehPa

⌉−⌈

ehPe,a

⌉)(5)

Target selection proceeds by summing S(e) over all ene-mies for every possible pairing, C, of P on E, which has mn

possibilities since an enemy can be targeted by more thanone player:

maxc∈C

[∑e∈E

S(e)

](6)

The pairing c that gives us the maximum savings is ourtarget selection. Evaluating Eq. (6) takes O(mn), and re-quires no manipulation or transformation from the basic pa-rameters of the problem. As combat proceeds, we reevaluate

Figure 1: Plot of equation (7), showing threat order fordifferent combinations of enemy health and attack

each round to determine the optimal savings given that en-emies have had their health reduced and may have died.

Real-Time Problem - The real-time formulation allowsfor entities to evaluate the best target at every moment.This means that players can react to changes in game state,such as an enemy dying, and change their attack instead ofwasting it. By eliminating the possibility of overkill, Eq. (2)becomes exact. Thus, we can evaluate exactly which enemyoffers the highest savings. The priority of all targets de-creases in linear proportion to time, and so relative priorityranking remains constant over time. Eliminating targets inpriority order thus guarantees an optimal outcome in real-time scenarios. We reach the same conclusion as [6], and findthat changing targets is suboptimal as it guarantees that theoptimal savings will not be reached. In general, all playersshould always be attacking the same enemy.

Using this knowledge we can rewrite Eq. (5). Here, Pe,ais equal to Pa as the players will all pick the same enemy.Using that knowledge we can drop Pa and get

maxe∈E

[e.a · (dEh − ehe)] (7)

What this means is that targets combining low health andhigh attack are preferred. We call this strategy threat or-dering. Figure 1 plots Eq. (7) for varying combinations ofea and eh while Eh is kept constant. The scale on the rightshows relative threat order for different enemy statistics.

4. SIMULATIONA theoretical explication necessarily abstracts over many

details, such as variant firepower, entity movement, physicalocclusion (cover), and so forth. It is possible that in morecomplex contexts the threat-based heuristic we justified andfound mathematically optimal is not in fact much betterthan other common and even more trivial heuristics thatgreedily focus on enemy health, proximity, or other one-dimensional factors. We thus here explore the relative valueof these heuristics in practice by applying them to a gamecontext representative of typical FPS games, built using theUnity 3D game development framework. In this we consider4 common targeting strategies within 4 varied scenarios (setsof enemies), as well as the impact of imperfect informationdue to the existence of cover, and the result when companionand player strategies are perfectly aligned.

Simulation Set Up - The simulation consists of a ba-sic Third-Person Shooter game where the player has to ex-plore a level by reaching goal locations while eliminatingencountered enemies. The game is over when the player haseliminated all enemies and reached all goals. The player isaccompanied by a single companion; the companion followsthe player around and will engage combat when she sees anenemy. Every NPC in the game is given a health and attackvalue.

The human player in the game is played by an NPC in or-der to simplify testing. We are interested in the influence ofthe companion’s behaviour over the outcome of the game,and by simulating the human player with artificial intelli-gence we assure that her behaviour will not be evolving overtime and she will act in the same manner for all simulations.In the simulation the human player’s behaviour is describedusing a behaviour tree. She will explore the space to reachall goals and engage combat with every enemy she sees. En-emy detection is also facilitated by sound—when she hearsgun fire, she will investigate the situation by walking to-wards the sound. The level is designed in such way that allenemies are near goals to ensure the occurrence of combat.

The companion’s behaviour is supportive; she will closelyfollow the player during exploration. She will engage in com-bat with every enemy she sees and, like the player, is aware ofsounds. Enemy behaviour reflects game industry standards;if NPCs do not see any enemies, they will patrol aroundusing pre-determined waypoints. When they hear fire theywill move towards that position. If they see an enemy theywill engage in combat by firing upon the closest target. Anyagent in the simulation will shoot for 2-3 seconds and willthen move to the left or the right; this behaviour simulatesdodging.

Target Selection Strategies - In order to fully test NPCtargeting action, we implemented 4 different strategies in-spired by modern FPS games. In each case, the selection isconstrained to visible enemies, and the human player usesthe threat ordering strategy.

• Closest strategy will pick the closest enemy using Eu-clidean distance.

• Highest Attack strategy will pick the enemy with thehighest attack.

• Lowest health strategy will pick the enemy with thelowest health .

• Threat ordering strategy will pick the enemy that hasthe highest priority according to equation 7.

Note that the closest strategy is strongly affected by thelevel design. Through careful placement of enemies, a de-signer could set up the level in a way that the companionchoice will match other strategies, at least initially. We thusrandomize enemy starting positions, and so closest acts morelike a random selection.

Scenarios and Levels - Four scenarios inspired by ac-tual game situations were developed in order to compare thedifferent strategies.

• Uniform scenario is composed of six enemies with thesame attack and health value.

• Boss scenario is composed of five enemies with thesame attack and health value, and a boss with highattack and health.

• Medley scenario is composed of two enemies with lowhealth and high attack value, two enemies with highhealth and low attack, and one enemy with medium-high attack and high health.

• Tank scenario is composed of five enemies with thesame attack and health value, and an enemy with veryhigh health value but only slightly higher attack.

Each combination of scenario and strategy was tested intwo environments. Simple level: an obstacle-free, open fieldwith no geometry blocking NPC vision. This is an optimalsituation for our threat ordering strategy, as it was designedwith access to perfect information in mind. Pillar level:a high-occlusion environment, with pillars blocking vision.Vision constraints increase the problem complexity in waysnot accounted for in Eq. 7, limiting target choices (in ourexperiments entities just pick a new target when the losesight of their initial target), and making movement time asignificant cost.

Figure 2 shows a top-down view and in-game, playtimescreen-shots of the pillar level (simple level is the same withno pillars). Red circles represent the enemies, the blue circleis the human player and the green circle the companion.

Figure 2: Top-down view and in-action playtime screenshotof the pillar level

A final set of experiments was also done having the com-panion make the same target choices as the player (MimicBehaviour), as the player uses different target selectionstrategies in the simple level. This experiment gives insightsinto how the player’s behaviour could be destructive whenthe companion does not make her own decisions.

5. RESULTSFor each combination of strategy, scenario, and level, and

again for the mimicking situation, we ran 31 simulations.This was sufficient to show trends in the data, while still re-sulting in a feasible experimental approach. From the datawe plotted average final team health and standard devia-tion. Results can be seen in figures 3 to 14, where we plotplayer+companion health over the duration of combat (inseconds) for the different combinations.

Simple Level - Figure 3 shows results for all the strate-gies given the uniform enemy scenario. Since the enemiesare identical in terms of attack and health, any target isa good target. Therefore any strategy is good as long asthe players do not deviate between the enemies, giving us abaseline for variance and sanity check for our simulation. Itis interesting to point out that with respect to our theoreti-cal justification, all enemies would sit at one point in figure1, emphasizing the lack of need for specialized strategies.

0 10 20 30 40time

110

120

130

140

150

160

170

180

190

200Tota

lHealt

h

Threat OrderingClosestLowest HealthHighest Attack

Figure 3: Simple Level Uniform

0 5 10 15 20 25 30 35 40time

110

120

130

140

150

160

170

180

190

200

Tota

lHealt

h


Figure 4: Pillar Level Uniform

0 5 10 15 20 25 30 35 40time

110

120

130

140

150

160

170

180

190

200

Tota

lHealt

h


Figure 5: Mimic Level Uniform

0 10 20 30 40 50time

110

120

130

140

150

160

170

180

190

200

Tota

lHealt

h


Figure 6: Simple Level Boss

0 5 10 15 20 25 30 35 40time

110

120

130

140

150

160

170

180

190

200

Tota

lHealt

h


Figure 7: Pillar Level Boss

0 10 20 30 40time

110

120

130

140

150

160

170

180

190

200

Tota

lHealt

h


Figure 8: Mimic Level Boss

0 10 20 30 40 50 60time

110

120

130

140

150

160

170

180

190

200

Tota

lHealt

h


Figure 9: Simple Level Medley

0 10 20 30 40 50time

110

120

130

140

150

160

170

180

190

200

Tota

lHealt

h


Figure 10: Pillar Level Medley

0 10 20 30 40 50time

110

120

130

140

150

160

170

180

190

200

Tota

lHealt

h


Figure 11: Mimic Level Medley

0 5 10 15 20 25 30time

110

120

130

140

150

160

170

180

190

200

Tota

lHealt

h


Figure 12: Simple level Tank

0 5 10 15 20 25 30 35time

110

120

130

140

150

160

170

180

190

200

Tota

lHealt

h


Figure 13: Pillar Level Tank

0 5 10 15 20 25 30 35time

110

120

130

140

150

160

170

180

190

200

Tota

lHealt

h


Figure 14: Mimic Level Tank

In the boss and medley scenarios, figures 6 and 9, highestattack outperforms lowest health as a strategy, and matchesour threat ordering approach. This is not always an idealchoice, however, as shown in the tank scenario, figure 12.In this case highest attack is actually the worst strategy; itpicks the tank (high health enemy) first because it also has

slightly higher attack, and thus spends a long time receivingnon-negligible damage from the other enemies. Threat or-dering, as well as the lowest health strategy, do not fall intothe same trap, prioritizing instead enemies that are morequickly eliminated and thus reducing total health loss.

Pillars - The addition of pillars to the level design reduces

visibility, preventing entities from seeing all targets in gen-eral, and dynamically changing the set of available targetsas entities move.

In general having subsets of enemies adds more noise tothe simulation, and results are largely similar to the sim-ple level, but with larger variance. This is evident in theuniform scenario, figure 4, and especially in the medley ex-periment, figure 10. Most evident in the uniform scenario,however, is that total health tends to be higher in the pillarversions. This is due to the enemies queueing behind eachother because of the limited room between pillars, thus re-ducing their visibility and allowing only a subset to shootat the players. With less shots fired at them, overall playerhealth ends up greater, and this argument holds for everypillar scenario.

For the tank level, figure 13, we see a small but inter-esting change in the relative difference between strategies.The highest attack strategy is still the worst, but the gapbetween it and other strategies is not as big as in the simplelevel scenario. Repeated occlusion in the pillar level reducesthe ability of the companion to stay focused on their sub-optimal choice, ameliorating the otherwise negative impactof this strategy. This is further verified by measuring andcomparing the number of times the companion targets anenemy with respect to the number of times she would targetthe ideal target for her strategy; in this case the companionis able to choose her intended but sub-optimal target onlyaround 1/3 of the time.

Companion Mimicking the Player - Since optimaltheoretical solutions suggest that concentrating attacks onone enemy is optimal, a trivial strategy for companions is tosimply mimick whatever the human player does. The suc-cess of this approach, however, depends very much on thestrategy the player chooses. The medley scenario, figure 11betters our understanding of the context. With a more inde-pendent companion we had at least one player selecting anoptimal choice, decreasing the impact of any wrong choiceson part of the other player. With mimicking, however, awrong choice ends up multiplying the negative impact, andsub-optimal strategies such as lowest health and closest endup with dramatically lower team health values. This is alsoevident in the highest attack strategy of the tank scenario,figure 14.

Of course when the player is an expert at picking theright target, mimicking performs well, as both players coop-erate and use good strategies. However, given that this willalso occur if the companion makes an independent choice touse threat order, and that will still imply generally betteroutcomes if the human player is not an expert, mimickingseems like an overall poor approach. We note that this isnot necessarily the case for every game or game context, andmimicking has been explored and shown to an effective ap-proach in more complex contexts where learning from theplayer is worthwhile, such as in fighting combat games [10].

Look-up Tree Search - The heuristics we examine of-fer simplicity and efficiency advantages over look-up treesearches, but even the overall best, threat ordering, dueto its inherent abstraction is not always necessarily opti-mal. We thus also compare performance with search basedtargeting to see how far from optimal threat ordering endsup being. For this we used a non-graphical, discrete-timesimulation, allowing us to explore a large number of scenar-ios, and avoiding any concerns with perturbing the real-time

simulation. Note that even this reduced problem is still NP-hard as shown by Furtak et al. [3].

Figure 15: Cumulative histogram, showing how close to op-timal threat ordering performs (discrete context).

Figure 16: Cumulative histogram, showing how highest at-tack, lowest health, and random targeting fare in comparisonto threat ordering. Note that the x-axis scale differs fromfigure 15.

We ran 500000 simulations in the discrete world with 2players against 2 to 5 enemies. We compared a full look-up tree search with the discrete form of the threat orderingheuristic, given as equation 6. The players in this simula-tion have attack 3 and health 500 each, with enemy attackvarying from 1 to 10 and health from 1 to 9. Although theseare fairly arbitrary values, there exists 6 billion ways to ar-range the enemies with a large variety of difficulty. Resultsare shown in figure 15. We can see that threat orderingfinds the optimal solution 50% of the time, is usually withinaround 1% of optimal, and never results in a total teamhealth less than 8% of the optimal.

Figure 16 compares the behaviour of highest attack, low-est health, and random targeting strategies to threat order-ing at each trial; this experiment used 50000 simulations,

and 2–10 enemies (other parameters are the same). Notethat in this discrete simulation we do not represent geomet-ric constraints, and so random replaces closest. In no casesdid these two strategies exceed threat ordering, but highestattack is clearly the better of the two, matching threat order-ing about 25% of the time, and being within 50% of threatordering over 97% of the time. Lowest health, however, onlybarely improves over random.

These discrete, analytical results largely mirror the re-sults shown in the more complex, real-time data given infigures 3 to 14. In general, focusing on high attack ene-mies is most important, eliminating weaker individuals isnext, and a weighted combination of these results is closeto optimal prioritization. Physical proximity has relativelylittle relevance, although this is likely also an artifact of ourcombat simulation; as future work it would be interestingto see how close-combat versus distance weapons alter theweighting of proximity.

6. CONCLUSIONOptimal solutions to enemy target selection in combat

games are complex, and ideally solved through expensivestate-space searches that are not practical in game contexts.Designers thus frequently resort to simple, but fast and easyto compute heuristics such as choosing the closest enemy,strongest enemy, mimicking the player, and so forth. Inthis work we explored and compared several such commonheuristics, showing that a slight variant (threat ordering)can be mathematically justified, and also performs notablybetter than other simple heuristics in realistic simulation.We also compared this result to a simplified, but exhaustivestate space analysis to verify that this approach is not onlyrelatively better, but also demonstrably close to the theoret-ical optimum. Understanding and validating these kinds oftargeting heuristics is important in terms of building inter-esting and immersive gameplay where companions behavesanely and can perform effectively.

There are a number of interesting extensions to this work.Combat complexity, for instance, is often increased in RPGsby giving characters a variety of special attacks, greatly lim-ited attack resources (as with magic), or by introducing spe-cific enemy weaknesses or strengths. We are interested inseeing if threat ordering or other simple, perhaps higher di-mension heuristics would still perform well in such complexenvironments. In more long-lasting or difficult combats, theavailability of defensive cover and resource restoration addseven more factors that should be considered in optimizingcombat behaviour [12]. Our main interest, however, is inexploring how to improve the value of companions to theplayer by maximizing their utility and ensuring appropri-ate, human-like combat behaviours, as well as in using theflexibility of companion choices to help games adapt to dif-ferent player skills.

7. ACKNOWLEDGEMENTSThis research was supported by the Fonds de recherche du

Quebec - Nature et technologies, and the Natural Sciencesand Engineering Research Council of Canada.

8. REFERENCES[1] S. Bakkes, P. Spronck, and E. O. Postma.

Best-response Learning of Team Behaviour in Quake

III. In Workshop on Reasoning, Representation, andLearning in Computer Games, pages 13–18, 2005.

[2] D. Churchill, A. Saffidine, and M. Buro. Fast heuristicsearch for RTS game combat scenarios. In AAAIConference on Artificial Intelligence and InteractiveDigital Entertainment, 2012.

[3] T. Furtak and M. Buro. On the complexity oftwo-player attrition games played on graphs. In AAAIConference on Artificial Intelligence and InteractiveDigital Entertainment, 2010.

[4] J. K. Goeree and C. A. Holt. Ten little treasures ofgame theory and ten intuitive contradictions.American Economic Review, pages 1402–1422, 2001.

[5] F. W. P. Heckel and G. M. Youngblood. Multi-agentcoordination using dynamic behavior-basedsubsumption. In AAAI Conference on ArtificialIntelligence and Interactive Digital Entertainment,2010.

[6] A. Kovarsky and M. Buro. Heuristic search applied toabstract combat games. In Advances in ArtificialIntelligence, Lecture Notes in Computer Science,pages 66–78. Springer Berlin Heidelberg, 2005.

[7] I. Millington and J. Funge. Artificial Intelligence forGames. Morgan Kaufmann, second edition, 2009.

[8] N. Nisan, T. Roughgarden, E. Tardos, and V. V.Vazirani, editors. Algorithmic Game Theory.Cambridge University Press, 2007.

[9] B. Russell. The Secrets Of Enemy AI In Uncharted 2 .http://www.gamasutra.com/view/feature/134566/

the_secrets_of_enemy_ai_in_.php, 2010.

[10] S. Saini, P. Chung, and C. Dawson. Mimicking humanstrategies in fighting games using a data driven finitestate machine. In Information Technology andArtificial Intelligence Conference (ITAIC), 2011 6thIEEE Joint International, volume 2, pages 389–393,2011.

[11] K. Salen and E. Zimmerman. Rules of Play: GameDesign Fundamentals. The MIT Press, 2003.

[12] Y. Shi and R. Crawfis. Optimal cover placementagainst static enemy positions. In Proceedings of the8th International Conference on Foundations ofDigital Games, FDG 2013, pages 109–116, 2013.

target selection for ai companions in fps gamesjtremb59/papers/fdg2014_paper_32.pdf · rpg games....

Documents