Constructing Constructing Complex NPC Complex NPC
Behavior via Multi-Behavior via Multi-Objective Objective
NeuroevolutionNeuroevolutionJacob Schrum – Jacob Schrum – [email protected]@cs.utexas.edu
Risto Miikkulainen – Risto Miikkulainen – [email protected]@cs.utexas.edu
University of Texas at AustinUniversity of Texas at AustinDepartment of Computer SciencesDepartment of Computer Sciences
IntroductionIntroduction Goal: discover NPC behavior automaticallyGoal: discover NPC behavior automatically BenefitsBenefits
Save production time/effortSave production time/effort Learn counterintuitive behaviorsLearn counterintuitive behaviors Find weaknesses in static scriptsFind weaknesses in static scripts Tailor behavior to human playersTailor behavior to human players
ChallengesChallenges Games are complexGames are complex Multi-modal behavior is requiredMulti-modal behavior is required Success often depends on multiple objectivesSuccess often depends on multiple objectives
RL & Evolution popular approaches RL & Evolution popular approaches Current methods do not explicitly consider Current methods do not explicitly consider
multiple objectives!multiple objectives!
Multi-Objective Multi-Objective OptimizationOptimization
Exceptional performance in any objective is Exceptional performance in any objective is importantimportant
Imagine game with two Imagine game with two objectives: objectives: Damage DealtDamage Dealt Health RemainingHealth Remaining
AA dominates dominates BB iff iff AA is is strictly better in one strictly better in one objective and at least objective and at least as as good in othersgood in others
Population of points Population of points not dominated are best: not dominated are best: Pareto Front Pareto Front
High health but did not deal much damage
Dealt lot of damage,but lost lots of health
Tradeoff between objectives
NSGA-IINSGA-II Evolution: natural approach for finding optimal Evolution: natural approach for finding optimal
population population Non-Dominated Sorting Genetic Algorithm II*Non-Dominated Sorting Genetic Algorithm II*
Population P with size N; Evaluate PPopulation P with size N; Evaluate P Use mutation to get P´ size N; Evaluate P´Use mutation to get P´ size N; Evaluate P´ Calculate non-dominated fronts of {P Calculate non-dominated fronts of {P P´} size 2NP´} size 2N New population size N from highest fronts of {P New population size N from highest fronts of {P P´}P´}
*K. Deb et al. 2000
NeuroevolutionNeuroevolution Genetic Algorithms + Artificial Neural Genetic Algorithms + Artificial Neural
NetworksNetworks NNs good at generating behaviorNNs good at generating behavior GA creates new nets, evaluates themGA creates new nets, evaluates them Three types of mutations (no crossover used)Three types of mutations (no crossover used)
Perturb Weight Add Connection Add Node
NPC Fitness ObjectivesNPC Fitness Objectives NPCs have several goalsNPCs have several goals
Inflict damage as a group → Attack Assist Inflict damage as a group → Attack Assist BonusBonus
Avoid damage individually → Damage ReceivedAvoid damage individually → Damage Received Stay alive individually → Time AliveStay alive individually → Time Alive
Objectives are contradictory and distinctObjectives are contradictory and distinct
NPCs take damage from bat Player is knocked back by NPC
ExperimentExperiment Multi-objective vs. Single-objectiveMulti-objective vs. Single-objective
Single-objective uses weighted sum of z-Single-objective uses weighted sum of z-scoresscores
Evolve each method against Evolve each method against static static player strategiesplayer strategies Spinning strategySpinning strategy Alternating strategyAlternating strategy Chasing strategyChasing strategy
NPCs must beat NPCs must beat previous player previous player strategy to progress strategy to progress
Evolved NPC BehaviorsEvolved NPC Behaviors Single-ObjectiveSingle-Objective
Cannot find balance Cannot find balance between attack and between attack and defensedefense
Behavior is ineffectualBehavior is ineffectual
Multi-ObjectiveMulti-Objective Baiting BehaviorBaiting Behavior
One agent acts as baitOne agent acts as bait Others attack player on blind Others attack player on blind
sideside Charging BehaviorCharging Behavior
Try to dodge bat while Try to dodge bat while rushing inrushing in
Maximizes damage dealtMaximizes damage dealt
DiscussionDiscussion BenefitsBenefits
Multi-modal behaviorMulti-modal behavior Exploits dynamics of Exploits dynamics of
environmentenvironment Individuals can have Individuals can have
different behaviorsdifferent behaviors Future WorkFuture Work
Evolve against humansEvolve against humans Learn many behaviors Learn many behaviors
in advancein advance More complex games: More complex games:
Unreal TournamentUnreal Tournament
ConclusionConclusion
Multi-objective encourages multi-modal behaviorMulti-objective encourages multi-modal behavior Multi-objective better than single-objectiveMulti-objective better than single-objective
Deals with contradictory objectivesDeals with contradictory objectives Leads to several different but effective solutionsLeads to several different but effective solutions
Lead to multi-modal behavior in future gamesLead to multi-modal behavior in future games
Questions?Questions?
See movies at: See movies at: http://nn.cs.utexas.edu/?multiNPCs
E-mail: E-mail: [email protected]@cs.utexas.edu