Download - Human-Level AI for Computer Games
![Page 1: Human-Level AI for Computer Games](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56813a14550346895da1ef6f/html5/thumbnails/1.jpg)
Human-Level AI for Computer Games
John E. Laird
University of Michigan
[email protected]://ai.eecs.umich.edu/people/laird
![Page 2: Human-Level AI for Computer Games](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56813a14550346895da1ef6f/html5/thumbnails/2.jpg)
Why I’m Interested in Video Games• AI is going to be more important in games
• A great environment for developing human-level AI
• Academic AI has something to offer the game industry
![Page 3: Human-Level AI for Computer Games](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56813a14550346895da1ef6f/html5/thumbnails/3.jpg)
Applications of AI to Games• Synthetic Characters
• Sports games, FPS, RTS, …
• Dynamic Game Control• Modify play calling based on experience (coaching)• Directing characters for plot and story• Providing play-by-play, camera control, …
• Development of Games• Automatic testing
![Page 4: Human-Level AI for Computer Games](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56813a14550346895da1ef6f/html5/thumbnails/4.jpg)
Human-Level AI & Games
• FPS Enemies: Quake, Unreal, Half-Life, ...
• FPS Partners: Spec Ops, Rainbow Six
• Adventure Characters: Blade Runner
• RPG Characters: Baldur’s Gate
• Massively Multiplayer Games: Asheron’s Call, Everquest
![Page 5: Human-Level AI for Computer Games](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56813a14550346895da1ef6f/html5/thumbnails/5.jpg)
Human Behavior Modeling
Time Scales
Seconds
104 1 hour Game
103 10 min Level
102 1 min Engagement
101 10 sec Unit task
100 1 sec Operations
10-1 100 msec Deliberative Act
10-2 10 msec Neural Circuit
10-3 1 msec Neuron
10-4 100 sec Organelle
Rational Band
Cognitive Band
Neural Band
![Page 6: Human-Level AI for Computer Games](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56813a14550346895da1ef6f/html5/thumbnails/6.jpg)
Human Behavior As A Sequence Of Deliberative Actions
• Consensus:• Human task-level behavior (> 100 milliseconds) can be
model as a sequence of deliberate actions, each taking approximately 50 milliseconds.
• Basic Perception takes > 100 milliseconds.• Simplest action (key press) takes 70 milliseconds.
• “Press a key when the light goes on.”Perception Thinking Action Total100 msec. + 50 msec. + 70 msec. = 220
msec.We found reaction time (shooting or sidestepping) to enemy in Quake II to be .5-2.5 seconds
![Page 7: Human-Level AI for Computer Games](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56813a14550346895da1ef6f/html5/thumbnails/7.jpg)
Processing is Pipelined• Perception/Cognition/Motor can run in parallel
Perception
Cognition
Motor
![Page 8: Human-Level AI for Computer Games](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56813a14550346895da1ef6f/html5/thumbnails/8.jpg)
Dual Task: Tracking/Choice
![Page 9: Human-Level AI for Computer Games](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56813a14550346895da1ef6f/html5/thumbnails/9.jpg)
![Page 10: Human-Level AI for Computer Games](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56813a14550346895da1ef6f/html5/thumbnails/10.jpg)
Vertical separation between tracking and CRT tasks (degrees)
0.0 3.2 6.4 9.6 12.8 16.0 19.2 22.4 25.6 28.8 32.0 35.2
RM
S T
rack
ing
Err
or
0
5
10
15
20
25
30
35
40
45
50
Observed EPICLockout (novice)Lockout + prepare
![Page 11: Human-Level AI for Computer Games](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56813a14550346895da1ef6f/html5/thumbnails/11.jpg)
Vertical separation between tracking and CRT tasks (degrees)
0.0 3.2 6.4 9.6 12.8 16.0 19.2 22.4 25.6 28.8 32.0 35.2
RM
S T
rack
ing
Err
or
0
5
10
15
20
25
30
35
40
45
50
Observed EPICLockout + Promotions + Interleaving (expert)Interleaving + Promotions + Retrained (expert)
![Page 12: Human-Level AI for Computer Games](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56813a14550346895da1ef6f/html5/thumbnails/12.jpg)
Cognitive Architectures• ACT-R (John Anderson, Carnegie Mellon)
• Detailed models of behavior and learning• 10 milliseconds-10 seconds.
• Used by cognitive psychologists• Issues on performance and scalabiltiy
• EPIC (David Kieras, University of Michigan)• Detailed models of human-computer interaction (no learning)
• Emphasizes models of perception and action• 50 milliseconds-10 seconds
• Used by cognitive psychologist
• Soar (John Laird, University of Michigan)• Models of human behavior and learning
• 50 milliseconds-1 hour• Emphasis more on complex behavior and learning
• Used by AI researchers and cognitive psychologist
![Page 13: Human-Level AI for Computer Games](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56813a14550346895da1ef6f/html5/thumbnails/13.jpg)
Conflicting Goals for Human-level AI in Games
Goal Driven
Reactive
Human
CharacteristicsKnowledge
Intensive
Low CPU & Memory Usage
Fast & Easy Development
![Page 14: Human-Level AI for Computer Games](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56813a14550346895da1ef6f/html5/thumbnails/14.jpg)
• Build an AI Engine around the Soar AI architecture• Soar/Quake II• Soar/Descent 3• Soar/Half-life
• Build autonomous, unscripted AIs
Soar/Games Project
InterfaceDLL
Quake II
AI Engine(Soar)
KnowledgeFiles
Actions
Sensor Data
![Page 15: Human-Level AI for Computer Games](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56813a14550346895da1ef6f/html5/thumbnails/15.jpg)
Soar• AI engine to support multi-method problem solving
• Applied to wide variety of tasks and methods
• Proposed unified theory for modeling human cognition
• Models a wide variety of human behavior: language, HCI, ...
• Combines reactive and goal-directed symbolic processing
• Supports very large bodies of knowledge (>100,000 rules)
• Optimized implementation in ANSI C
• In the public domain
![Page 16: Human-Level AI for Computer Games](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56813a14550346895da1ef6f/html5/thumbnails/16.jpg)
Soar Quakebot• Goal: Human-like behavior
• Fun and challenging to play against• Not super-human• Currently plays as well as good player
• Completely implemented in Soar• > 750 rules• Spread across operators for exploring, wandering, chasing, …
• Sensing similar to human
• Runs on separate PC from game• Uses ~5-10% of 400 MHz Win-95/98/NT machine
![Page 17: Human-Level AI for Computer Games](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56813a14550346895da1ef6f/html5/thumbnails/17.jpg)
Execution Flow of an AI Engine
Sense
Think
Act
G
A
M
E
Simulated PerceptionSelf: orientation, position, healthSenses: vision, hearing, smell, taste, touchAugmented Senses: radar, night-vision, …> 80 sensors in Quakebot
Simulated ActionsMovement, weapons’ use, communication~20 actions in Quakebot
Obstacle
Field of Vision
![Page 18: Human-Level AI for Computer Games](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56813a14550346895da1ef6f/html5/thumbnails/18.jpg)
Execution Flow of an AI Engine
Sense
Think
Act
G
A
M
E
?
Finite-state machines
Subsumption
Neural nets
Rule-based systems
C code
Planning systems
![Page 19: Human-Level AI for Computer Games](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56813a14550346895da1ef6f/html5/thumbnails/19.jpg)
Rule-based Systems• Program = set of if-then rules, not sequential code.
• Use whatever knowledge that is relevant to current situation
Working Memorysensing, elaborations, persistent memories, goals, actions …
Rule MemoryTactics, Movement Knowledge, ...
[ ][ ] [ ][ ]
[ ]
[ ][ ][ ]
Match Changes
[ ] [ ][ ]
[ ] [ ][ ]
![Page 20: Human-Level AI for Computer Games](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56813a14550346895da1ef6f/html5/thumbnails/20.jpg)
Rules• Conditions contain variables and other first-order tests
• If sense a weapon and do not have that weapon, pickup the weapon.(p rule1 (sense weapon ?weapon1) (self weapon <> ?weapon1) (action pickup ?weapon1))
• A single rule can have multiple instantiations• A single condition can match multiple working memory
elements
• Compilers convert rules into discrimination networks• Only test changes to working memory.• Share tests across similar rules• Efficient matching
![Page 21: Human-Level AI for Computer Games](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56813a14550346895da1ef6f/html5/thumbnails/21.jpg)
Which Rule Should Fire?
• Matching finds which rules can fire in current situation.
• Conflict resolution picks which rules actually fire.• Based only on syntactic features of rules and data.• Must engineer rules so correct rule is selected each time.
RuleMemory
WorkingMemory
[ ] [ ] [ ] [ ][ ] [ ] [ ] [ ]
[ ] [ ]
[ ] [ ]
[ ] [ ]
[ ] [ ]
Rule Instantiations
?Conflict resolution
Match
![Page 22: Human-Level AI for Computer Games](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56813a14550346895da1ef6f/html5/thumbnails/22.jpg)
Weaknesses in Rules• Conflict resolution based on syntactic features
• Not knowledge of the task
• Rules combine three types of knowledge about actions• What is possible• What should be done (based on conflict resolution)• How to do itLeads to duplication if have multiplicity in one of the types
• No hierarchical organization of knowledge and goals
![Page 23: Human-Level AI for Computer Games](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56813a14550346895da1ef6f/html5/thumbnails/23.jpg)
Soar’s Approach• Organize knowledge as operators:
• Primitive & abstract actions• Turn, move, climb, goto-door, get-item, wander, attack, chase• Provides trace of behavior comparable to human decisions
• Use rules to propose, select, and apply operators• Rules fire in parallel - act as an associative memory• Provides fine-grain, situation-dependent behavior• Conditional selection and execution determined dynamically
Inpu
t
Elaboration(propose operators)
Decide(select operator)
Elaboration(apply operator)
Out
put
Inpu
t
Dec
ide
Out
put
Inpu
t
Dec
ide
Elaboration(terminate operator & propose)
![Page 24: Human-Level AI for Computer Games](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56813a14550346895da1ef6f/html5/thumbnails/24.jpg)
Execution Flow of Soar Engine
Elaboration
Propose Operators
Evaluate Proposed Operators
Select One Operator
Perform Operator Actions
Sense
Think
Act
G
A
M
E
Soar
![Page 25: Human-Level AI for Computer Games](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56813a14550346895da1ef6f/html5/thumbnails/25.jpg)
Get item
Go through door
Get item in room
Detect item missing
Face item Move to item
Stop moving to item
Exit room
Go to door
Align with door
Face door
Move to door
Record at door
Shoot Slide to door
Stop move to door
Stop slide to door
![Page 26: Human-Level AI for Computer Games](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56813a14550346895da1ef6f/html5/thumbnails/26.jpg)
Soar Quakebot Top OperatorsWanderAttack ExploreRetreat Record
EnemyRemove Enemy
Chase
![Page 27: Human-Level AI for Computer Games](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56813a14550346895da1ef6f/html5/thumbnails/27.jpg)
Soar Quakebot Top OperatorsWanderAttack ExploreRetreat Record
EnemyRemove Enemy
If enemy visible and my health is > very-low-health-value (20%) and his weapon is not much better than mineTHEN propose attack
Chase
If enemy visible and my health is < very-low-health-value (20%) or his weapon is much better than mineTHEN propose retreat
If no enemy visible or recorded and explored levelTHEN propose wander
If no enemy visible or recorded and not explored levelTHEN propose explore
![Page 28: Human-Level AI for Computer Games](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56813a14550346895da1ef6f/html5/thumbnails/28.jpg)
Soar Quakebot Top OperatorsWanderAttack ExploreRetreat Record
EnemyRemove Enemy
If enemy just became not visible and no enemy visible or recordedTHEN propose record-enemy
Chase
If record-enemy selectedTHEN record enemy’s last position and time + 20 seconds.
If hear enemy and no enemy visible or recorded THEN propose record-enemy
If time > recorded enemy’s saved timeTHEN propose remove-enemy
If remove-enemy is selectedTHEN remove recorded enemy structure
If there is a recorded enemy structureTHEN propose chase
![Page 29: Human-Level AI for Computer Games](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56813a14550346895da1ef6f/html5/thumbnails/29.jpg)
Example Rules
IF attacking an enemy and there is an object I need and that object is closer than attack-get-item-rangeTHEN propose get-item for that object
IF I see a weapon that is much better than any I haveTHEN I need that weapon
IF my health is less than low-health-value (40%)THEN I need any health object
IF two get-item operators are proposedTHEN prefer selecting the one for the closer object
Attack
Get-item Face-enemy Side-step Approach Shoot
![Page 30: Human-Level AI for Computer Games](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56813a14550346895da1ef6f/html5/thumbnails/30.jpg)
Get item
Go through door
Get item in room
Detect item missing
Face item Move to item
Stop moving to item
Exit room
Go to door
Align with door
Face door
Move to door
Record at door
Shoot Slide to door
Stop move to door
Stop slide to door
![Page 31: Human-Level AI for Computer Games](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56813a14550346895da1ef6f/html5/thumbnails/31.jpg)
Get item
Go through door
Get item in room
Detect item missing
Face item Move to item
Stop moving to item
Exit room
Go to door
Align with door
Face door
Move to door
Record at door
Shoot Slide to door
Stop move to door
Stop slide to door
![Page 32: Human-Level AI for Computer Games](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56813a14550346895da1ef6f/html5/thumbnails/32.jpg)
Get item
Go through door
Get item in room
Detect item missing
Face item Move to item
Stop moving to item
Exit room
Go to door
Align with door
Face door
Move to door
Record at door
Shoot Slide to door
Stop move to door
Stop slide to door
![Page 33: Human-Level AI for Computer Games](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56813a14550346895da1ef6f/html5/thumbnails/33.jpg)
Get item
Go through door
Get item in room
Detect item missing
Face item Move to item
Stop moving to item
Exit room
Go to door
Align with door
Face door
Move to door
Record at door
Shoot Slide to door
Stop move to door
Stop slide to door
![Page 34: Human-Level AI for Computer Games](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56813a14550346895da1ef6f/html5/thumbnails/34.jpg)
Get item
Go through door
Get item in room
Detect item missing
Face item Move to item
Stop moving to item
Exit room
Go to door
Align with door
Face door
Move to door
Record at door
Shoot Slide to door
Stop move to door
Stop slide to door
![Page 35: Human-Level AI for Computer Games](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56813a14550346895da1ef6f/html5/thumbnails/35.jpg)
Get item
Go through door
Get item in room
Detect item missing
Face item Move to item
Stop moving to item
Exit room
Go to door
Align with door
Face door
Move to door
Record at door
Shoot Slide to door
Stop move to door
Stop slide to door
![Page 36: Human-Level AI for Computer Games](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56813a14550346895da1ef6f/html5/thumbnails/36.jpg)
Get item
Go through door
Get item in room
Detect item missing
Face item Move to item
Stop moving to item
Exit room
Go to door
Align with door
Face door
Move to door
Record at door
Shoot Slide to door
Stop move to door
Stop slide to door
![Page 37: Human-Level AI for Computer Games](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56813a14550346895da1ef6f/html5/thumbnails/37.jpg)
Get item
Go through door
Get item in room
Detect item missing
Face item Move to item
Stop moving to item
Exit room
Go to door
Align with door
Face door
Move to door
Record at door
Shoot Slide to door
Stop move to door
Stop slide to door
![Page 38: Human-Level AI for Computer Games](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56813a14550346895da1ef6f/html5/thumbnails/38.jpg)
Get item
Go through door
Get item in room
Detect item missing
Face item Move to item
Stop moving to item
Exit room
Go to door
Align with door
Face door
Move to door
Record at door
Shoot Slide to door
Stop move to door
Stop slide to door
![Page 39: Human-Level AI for Computer Games](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56813a14550346895da1ef6f/html5/thumbnails/39.jpg)
Example Tactics• Collect-Powerups
• Pick up best weapons from spawn locations • Remember when missing items will respawn• Use shortest paths to get objects• Get health/armor if low on health/armor• Pickup up other good weapons/ammo if close by to deny enemy
• Attack • Use circle-strafe• Move to best distance for current weapon
• Chase enemy based on sound of running
• Ambush in corner that can’t be seen by enemy
• Hunt at nearest spawn room after killing enemy
![Page 40: Human-Level AI for Computer Games](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56813a14550346895da1ef6f/html5/thumbnails/40.jpg)
Mapping• Need information on location of walls, doors, etc.
• Many tactics require this information
• Built up through exploration of a level• Similar to a robot exploring with range sensors• Saved and reused when return to level
• Represented internally as a graph structure• Currently restricted to 2D rectangles
![Page 41: Human-Level AI for Computer Games](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56813a14550346895da1ef6f/html5/thumbnails/41.jpg)
![Page 42: Human-Level AI for Computer Games](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56813a14550346895da1ef6f/html5/thumbnails/42.jpg)
![Page 43: Human-Level AI for Computer Games](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56813a14550346895da1ef6f/html5/thumbnails/43.jpg)
Anticipation?Dennis (Thresh) Fong:
“Say my opponent walks into a room. I'm visualizing him walking
in, picking up the weapon. On his way out, I'm waiting at the
doorway and I fire a rocket two seconds before he even rounds the
corner. A lot of people rely strictly on aim, but everybody has their
bad aim days. So even if I'm having a bad day, I can still pull out a
win. That's why I've never lost a tournament.”Newsweek, 11/21/99
![Page 44: Human-Level AI for Computer Games](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56813a14550346895da1ef6f/html5/thumbnails/44.jpg)
Anticipation• Predicts enemy’s actions
• Creates internal model of enemy’s state
• Uses own knowledge to predict what enemy would do• Can be modulated by knowledge of enemy’s tactics
• Stops planning when it knows what to do or uncertainty in actions
• Uses prediction to select actions: • Ambush, Deny-Powerups
![Page 45: Human-Level AI for Computer Games](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56813a14550346895da1ef6f/html5/thumbnails/45.jpg)
?
![Page 46: Human-Level AI for Computer Games](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56813a14550346895da1ef6f/html5/thumbnails/46.jpg)
![Page 47: Human-Level AI for Computer Games](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56813a14550346895da1ef6f/html5/thumbnails/47.jpg)
His Distance: 1
My Distance: 1
![Page 48: Human-Level AI for Computer Games](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56813a14550346895da1ef6f/html5/thumbnails/48.jpg)
His Distance: 2
My Distance: 2
![Page 49: Human-Level AI for Computer Games](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56813a14550346895da1ef6f/html5/thumbnails/49.jpg)
His Distance: 2
My Distance: 2
![Page 50: Human-Level AI for Computer Games](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56813a14550346895da1ef6f/html5/thumbnails/50.jpg)
His Distance: 3
My Distance: 1 (but hall)
![Page 51: Human-Level AI for Computer Games](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56813a14550346895da1ef6f/html5/thumbnails/51.jpg)
His Distance: 4
My Distance: 0 Ambush!
![Page 52: Human-Level AI for Computer Games](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56813a14550346895da1ef6f/html5/thumbnails/52.jpg)
![Page 53: Human-Level AI for Computer Games](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56813a14550346895da1ef6f/html5/thumbnails/53.jpg)
![Page 54: Human-Level AI for Computer Games](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56813a14550346895da1ef6f/html5/thumbnails/54.jpg)
References• The Psychology of Human-Computer Interaction,
Card, Moran, and Newell, LEA, 1983.
• Unified Theories of Cognition, Newell, Harvard Press, 1990.
• Modeling Human and Organizational Behavior: Applications to Military Simulations, National Research Council, National Academy Press, 1998.
• Soar Homepage, http://ai.eecs.umich.edu/soar