using expectations to drive cognitive behavior
DESCRIPTION
Using Expectations to Drive Cognitive Behavior. Unmesh Kurup Christian Lebiere , Tony Stentz , Martial Hebert Carnegie Mellon University. Cognitive Decision Cycle. Cognition is driven by Expectations/Predictions. Prediction. Prediction. Calculate Mismatch. High-level Cognition. Action. - PowerPoint PPT PresentationTRANSCRIPT
Using Expectations to Drive Cognitive Behavior
Unmesh KurupChristian Lebiere, Tony Stentz, Martial Hebert
Carnegie Mellon University
Cognitive Decision Cycle
t+1
Calculate Mismatch
World
High-level Cognition
Retrieve Response WorldAction Action
Action
PredictionPrediction
Prediction
t-1 t
Cognition
• Cognition is driven by Expectations/Predictions.
Pedestrian Tracking & Behavior Classification
Goals:• Investigate use of expectations• Integrate with perception• Run both offline & real-time
Integrated System
Partial Matching & Blending
Chunk2 isa location-
chunkid person2 nextx 1010nexty 500
Chunk3isa location-
chunkid person3 nextx 187nexty 313
Chunk4 isa location-
chunkid person1 nextx 299nexty 100
+retrieval>isa location-chunkid person1nextx 300
Chunk1 isa location-
chunkid person1 nextx 255nexty 100
Chunk4 isa location-
chunkid person1 nextx 299nexty 100
Declarative Memory
Partial Matches
Chunk5isa location-
chunkid person1 nextx 293.91nexty 100
Blended result
Chunk1 isa location-
chunkid person2 nextx 255nexty 100
Using Expectations: TrackingChunk-type visual-location
id X Y Dx Dy Nextx Nexty
Foreach Object o: +blending>
isa visual-locationid o
compare to (x,y)s from perceptionpick thresholded closest match, calculate newdx, newdy, newx, newy+imaginal>
isa visual-locationid o…
Features
straight1 straight2 detour left straight3 veerFeatures:
Behavior Features
Normal – Straight straight1, straight2, straight3
Normal – Left straight1, straight2,left
Peek straight1, detour, left, no-chk-pt
Behavior Features
Detour straight1, detour, straight3, chk-pt
Veer straight1, straight2,left, veer, chk-pt
Walkback straight1, straight2, left, straight2, straight1, chk-pt
Using Expectations: Detecting Features from Data
Straight & Left
Deviation from expected location indicates a point of interest
Foreach location+blending>
isa visual-locationx =x y =y
compare to (x,y)s from perception
if path deviates more than threshold, mismatch!
+imaginal>isa visual-locationid o…
Cluster points into regions
Detected Features
Data
• Combined Arms Collective Training Facility(CACTF) at Fort Indiantown Gap, PA.
• 4 examples. 3/1 split.• Multiple behavior set– 10 behaviors.
Behaviors
Straight & Left
Peek
Detour
Veer
Walkback
Results
Hand-coded Model(Single Behavior Set)
Hand-coded Model(Multiple Behavior Set)
Made 99.3% Made 46.5%Correct 99.15% Correct 30.2%
Incorrect 0.15% Incorrect 16.3%
Learning Model(Single Behavior Set)
Learning Model(Multiple Behavior Set)
Made 86.1% Made 82.4%Correct 68% Correct 43.8%
Incorrect 18.1% Incorrect 38.6%
Future Work – Semantic Labels
Future Work – Using Semantic Labels
Behavior Features (Spatial) Features (Semantic)Normal – Straight straight1, straight2,
straight3Sidewalk, Pavement
Normal – Left straight1, straight2,left
Sidewalk
Peek straight1, detour, left, no-chk-pt
Pavement, Sidewalk
Detour straight1, detour, straight3, chk-pt
Pavement
Veer straight1, straight2,left, veer, chk-pt
Sidewalk, Pavement
Walkback straight1, straight2, left, straight2, straight1, chk-
pt
Sidewalk
Future Work
• Generic model of monitoring using expectations
• Learn expectations• Monitor for deviations from expectations– Signal failure– Provide for recovery
Collaborators
Max Bajracharya, JPLBob Dean, GDRS
Brad Stuart, GDRSFMS lab, CMU