promoting mmo player interaction through …...“reducing the world to mathematical equations!”...

“Reducing the world to mathematical equations!”

Cover Me!Promoting MMO Player Interaction

Through Advanced AI

Dave MarkPresident & Lead Designer

Intrinsic Algorithm LLC


Dave Mark• President & Lead Designer of

Intrinsic Algorithm LLC• Game Studio• AI Consulting Company

• Author of Behavioral Mathematics for Game AI

• Co-founder ofAI Game Programmers Guild

• Organizer and co-host of theAI Summit at GDC 2009


Premises/Disclaimers• WoW is a good game.• Not everyone wants to be like WoW.• Bad AI has its place in games.• Not everyone wants to have bad AI.• You are here because you are working on

an MMO.• You are here because you want to know:

– How good AI can make your game better– How to use AI in an MMO environment


What makes a game a game?

“A game isa series of interesting choices.”

- Sid Meier


In the beginning…• Early games were entirely

played against the computer.• AI sucked• Choices were rudimentary• Problem-solving was simplistic.


1-Player Games w/Multi-player Component

• People played against each other more than computer

• Game won’t sell without multi-player

• 1-player campaign often neglected

• “I love Halo 3… but I’ve never played the campaign.”


Multiplayer Co-Op• People prefer playing with other people

– Interactivity– Teamwork (AI allies still not up to par)– Solving problems together


The Draw of PvP• People prefer playing against other people

– Difficulty of opponents– Adaptability of opponents– Dynamicity (e.g. the unexpected)– Solving problems together Repla

yabilit

y!


The Necessity of PvE• Beyond Game Mechanics

– Environments – Weapons– Characters

• World Immersion– Who is going to role-play

the dragon?– The harmless little bunny? – The slime creature?


AI is Your World!

Boring AI = Boring WorldDeep AI = Immersive World

Repetitive AI = Monotonous WorldDynamic AI = Dynamic World


PvP vs. PvE

PvP• Difficult• Adaptable• Dynamic• Necessitates ongoing

problem-solving

PvE• Simple• Rigid• Predictable• “Solve Once” and repeat

PvE• Difficult• Adaptable• Dynamic• Necessitates ongoing

problem-solving

So how do we do it?


Making PvE feel like PvP

• Single-player games have been improving their AI to generate a PvP-like feel

• What techniques can we import from single-player games into MMOs?– What effect does that have on the player’s individual

experience?– What effect does that have on the players working

together?


AI Techniques

By Genre• Shooter• RPG• Strategy

By Concept• Behavioral• Tactical• Strategic• Simulation• Economic


AI Techniques

By Genre• Shooter• RPG• Strategy

By Concept• Behavioral• Tactical• Strategic• Simulation• EconomicMultiple vs. Multiple


Behavioral AI

• How does the NPC make decisions?– Idle behaviors– When to attack – Who to attack– How to attack


When to Attack

• Generating “Aggro”– Distance– Environment Threshold Triggers– Line of Sight

• Well documented by the public– Attack happens when the player(s) want it to– Removes sense of enemy’s autonomy


Who to Attack

• Nearest opponent• Strongest opponent• Well documented by the public

– Attack is against whom the player(s) expect– Attack is against whom the player(s) want– Removes sense of enemy’s autonomy


How to Attack

• Primary attack most of the time• After delay of x, use secondary attack• In given situation, use special attack• Well documented by the public

– Attack happens how the player(s) expect it to– Removes sense of enemy’s dynamicity


Effect on the Players• On the Individual Player

– Calculatable– Predictable– Boring

• On the Group– No need for interaction– “Read the script…”– “Play your part…”


What is your reaction?

• Want to pet• Want to meet• Just curious• Annoyed• Want to kick• Want to run screaming

Want to pet

Just curious Annoyed

Want to kick

Want to run screaming

Want to meet


• Want to pet• Want to meet• Just curious• Annoyed• Want to kick• Want to run screaming

Now what is your reaction?

Want to pet

Want to meet

Just curious

Annoyed Want to kick



Varieties of Reactions

Want to pet


Want to kick


Want to meet

Want to pet

Want to meet

Just curious

Annoyed Want to kick



Same Model for All Agents

Want to meet


Want to kick

Annoyed

Just curious

Want to pet Want to run screaming

Want to kick

Annoyed

Just curious

Want to meet

Want to pet

Extreme Reactions


Varieties of Reactions

Want to pet


Want to kick


Want to meet

• Differences Exist• Don’t need to know why• Need to simulate that difference

do exist– Not completely random selection– Must be reasonable– Can be simulated with weighted

randoms


Know when to walk away…

• Design Decision: “Enemies don’t always fight to the death”

• Enemies can sometimes retreat– Flat % chance

• Is random… therefore looks random• Not realistic

– Situational random• Based on circumstances• Circumstances are flexible and dynamic


Know when to walk away…How many on my side are still fighting?

8

How many of my enemies are still fighting?

5

1.6



PercentChance = (4 – Ratio)3 / (43)Percent Chance that an Individual will Retreat

0%

25%

50%

75%

100%

0 0.5 1 1.5 2 2.5 3 3.5 4



PercentChance = (4 – Ratio)4 / (44)

PercentChance = (4 – 1.6)4 / (44)

PercentChance = 13%


Know when to walk away…How many on my side are still fighting?

7

How many of my enemies are still fighting?

5

1.4



PercentChance = (4 – Ratio)4 / (44)

PercentChance = (4 – 1.4)4 / (44)

PercentChance = 18%



PercentChance = (4 × Ratio)4 / (44)Percent Chance that an Individual will Retreat

0%

25%

50%

75%

100%

0 0.5 1 1.5 2 2.5 3 3.5 4


Know when to walk away…PercentChance =

( ( MaxRatio – Ratio )k × MaxPct ) / (MaxRatio k )

80.50In Base60.75Near Base41.00In FieldkMaxPct


Percent Chance that an Individual will Retreat

0%

25%

50%

75%

100%

0 0.5 1 1.5 2 2.5 3 3.5 4


( ( 4 – Ratio )4 × 1.00 ) / ( 44 )( ( 4 – Ratio )4 × 1.00 ) / ( 44 )

( ( 4 – Ratio )6 × 0.75 ) / ( 46 )( ( 4 – Ratio )6 × 0.75 ) / ( 46 )

( ( 4 – Ratio )8 × 0.50 ) / ( 48 )( ( 4 – Ratio )8 × 0.50 ) / ( 48 )



• Factors to Consider– Number of allies– Number of enemies– Proximity to Base– Strength of allies – Strength of enemies– My own health– Proximity of my leader


Know when to walk away…Numberof Allies

Numberof Enemies

Threat Ratio

My HealthProximityto Leader

Strengthof Allies

Strengthof Enemies

Allied Strength Enemy Strength

My Morale

Retreat %

Proximityto Base

Urgency

“Compartmentalized Confidence”


How does this look to the players?

• Enemies aren’t completely fearless• They are slightly unpredictable• They are still reasonable• Curiosity: “Where is he going?”• “If we show force, they might break and run.”• “If we back them up, they are more aggressive.”• We have to react to their reactions.


More than “Fight or Flight”

• Fight normally• Flee• Charge


More than “Fight or Flight”Percent Chance that an Individual will Retreat or Charge

0%

25%

50%

75%

100%

0 0.5 1 1.5 2 2.5 3 3.5 4


More than “Fight or Flight”• Fight

– Melee weapon– Ranged weapon– Special rare weapon

• Advance– Press forward– Berserker charge

• Retreat– Fighting withdrawal– Find Cover– Organized pull back– Flee in abject terror– Surrender


More than “Fight or Flight”Special Press Forward

Berserk Charge

Withdrawal

Cover

Pull Back

Flee

Surrender

Melee

Ranged


More than “Fight or Flight”SpecialPress ForwardBerserk Charge

Withdrawal

Cover

Pull Back

FleeSurrender

Melee

Ranged


More than “Fight or Flight”

Special Press Forward

Berserk Charge

Withdrawal

Cover

Pull Back

Flee

Surrender

Melee

Ranged

“Normal” “Defensive”

SpecialPress ForwardBerserk Charge

Withdrawal

Cover

Pull Back

FleeSurrender

Melee

Ranged

21%42%



• Not predictable• Reasonable• Causes us to have to

react to his reaction Want to pet


Want to kick


Want to meet


Target Selection• Who should I attack?

– Closest enemy– Biggest threat (highest DPS)– Weakest link (least health)– Most valuable (healer helping the other enemies)


Shoot that @$&%#!!


Tactics

• Multiple NPCs working in a coordinated plan– Formations

• Protecting high-value units (fighters protecting a mage)• Adequate defense coverage of a point

– Coordinated movement• Flanking• Covering fire• Leapfrogging advance


Tactical Manager

Agent Agent Agent Agent

TacticalManager


Tactical Manager• Has Rudimentary Goals

– “Protect X”– “Attack Y”

• Situation Analysis• Reactive

– Responds to players’ actions– Responds to situation changes

• Proactive– Looks for opportunities– “Running a play”


Protecting High-Value Units/PointsTacticalManagerTacticalManager


Protecting High-Value Units/Points


Protecting High-Value Units/Points

Flanking!!!



• Enemies react to our positioning– “They’re moving to block us.”

• Enemies react to each others’ actions– “They’re pulling back and closing ranks!”

• Enemies dynamically allocate their own resources as a cohesive unit– “OK… he saw me coming around the side.”


Strategy

• The entire body of a certain type of enemy working together to accomplish an over-arching goal

• Super-set of tactics– Strategy to tactics = tactical to individual


Strategy


TacticalManager

OrderOrder Order

Order


3-tier Hierarchy


TacticalManager


TacticalManager


TacticalManager

StrategicManager


TacticalManager

Goal

Goal Goal

Goal


Using Influence Maps

• Store relevant data in underlying grid structure• Periodically update the data map• Propagate information about individual objects to

surrounding squares• Aggregate the data to yield a broad representation• Different influence maps can be used in conjunction

with one another


Strategic Disposition

A

B



• Requires communication among players:– “There are continual raids on the northern road.”– “We should take the southern road instead.”– “We should send patrols out along the northern road to

keep it open.”• Continually dynamic:

– “The orcs have figured out that we are using the southern road instead.”


Quest Givers

• Information can be kept on:– Movements of NPCs and monsters– Significant events in the world

• Quest Givers can use dynamic information from the world to notify players of…– Threats to avoid– Challenges to overcome



A

FortThe orcs have

gathered…west of town, near the river.



A

FortThe orcs have regrouped…

west of the fort, east of the river.



Fort

A

The orcs have moved…

north of town, near the fort.



• “There are groups of them moving methodically through the outlying villages.”

• “They have our city surrounded.”• “Their other groups are coming to the aid of the

army we just routed.”• “There is a group trying to cut off our retreat back to

the fort!”


Other Uses• Big Bad Dragon in the mountains

– Too many players hunting the dragon

– Moves away from the players to some hills

– Now feasting on Farmer Ted’s cows– Quest giver tells you to help Farmer

Ted

Influence MapInfluence Map


Event LogEvent Log



Other Uses• Living Areas

– Moving toward player areas

– Moving away from player areas

– Moving towards food prey

– Moving away from predators






Sooo… what have we learned?• People like playing against challenging opponents• Playing against challenging, dynamic, adaptive

opponents forces the players to communicate and work together

• AI can be designed to be challenging in similar ways to human opponents

• By using advanced AI techniques in MMOs, we can create more engaging environments for our players to thrive in


How does this change thingsfor our players?

• Identify differences in enemy behavior• Communicate what they see to each other• Communicate their personal needs (“Cover me!”)• Formulate cooperative plans • Execute plans together• Adapt those plans to account for changing

circumstances


Dave MarkPresident & Lead Designer

Intrinsic Algorithm LLC

[email protected]

promoting mmo player interaction through …...“reducing the world to mathematical equations!”...

Documents