1 automaticity development and decision making in complex, dynamic tasks dynamic decision making...
Post on 15-Jan-2016
224 views
TRANSCRIPT
1
Automaticity development and decision making in complex, dynamic tasks
Dynamic Decision Making Laboratorywww.cmu.edu/DDMLab
Social and Decision Sciences DepartmentCarnegie Mellon University
Cleotilde GonzalezRickey ThomasPolina Vanyukov
2
Complex and dynamic tasks
Executing a battle, driving, air traffic controlling, managing of a production plan, piloting, managing inventory in a production chain, etc.
• Demand real-time decisions (time constraints)
• Demand attentional control
• Require multi-tasking: they are composed of multiple and interrelated subtasks
• Demand the identification of ‘targets’ defined by multi-attributes
• Demand multiple and possibly changing responses
3
Automaticity in dynamic, complex tasks
• targets and distractors are often inconsistently mapped to stimuli and responses
• Often, we bring pre-learned categories and mappings to a task
stimulus - category category - responseL ------------- letter button --------- click
• Are decision makers in dynamic situations operating in controlled processing continuously?
4
Proposed model of automaticity in DDM
Cues Categories Responses
sub-task Structure (Mapping)
CM/VM
coupling
Cues Categories Responses
sub-task Structure (Mapping)
CM/VM
coupling
Goals (Relevancy)
Task switching (resource allocation)
Cues Categories Responses
sub-task Structure (Mapping)
CM/VM
coupling
Cues Categories Responses
sub-task Structure (Mapping)
CM/VM
coupling
5
Experiments
• Automaticity develops with consistently mapped stimuli to targets, even when targets move and time is limited (Experiment 1)
• The consistency of target to response mapping also determines automaticity development (Experiment 2)
• Automaticity of a task component frees-up time and resources for high level decision-making (Experiment 3)
• Automaticity develops differently with different degrees of pre-learned categories (Experiment 4)
6
The Radar Task
7
8
General method
• Independent variableso stimulus mapping (CM or VM)
• CM = Search for Numbers in Letters• VM = Search for Letters in Letters
o cognitive load• Memory set size (MSS): Number of possible targets to remember (1 or 4)• frame size (FS): Number of blips present on the screen at a given time (1
or 4)o target present/absent (a target was present 75% of the trials)
• Dependent variableso Accuracy: proportion of correct detections or decision-making
responseso Time: mean target detection or decision-making time in msec
• From 18 to 30 hours of practice, 3 hours per day 6 to 10 days
9
Experiment 1: Consistency of stimuli
• Replicate major findings from the dual-process theory (Schneider & Shiffrin, 1977) in a dynamic task
• Automaticity is acquired with practice in consistent mapping conditions, and automatic performance is unaffected by workload
10
Experiment 1: Method
o CM vs. VM
o Cognitive Load Variables• Memory Set Size• Frame Size
o Only one possible response: pressing spacebar when target is detected
11
Experiment 1: Accuracy
CM
VM
0.6
0.7
0.8
0.9
1
FS=1 FS=4
MSS = 1
Ac
cu
rac
y
CM
VM
FS=1 FS=4
MSS = 4
12
Experiment 1: Detect Time
CMVM
600
800
1000
1200
1400
1600
FS=1 FS=4
MSS = 1
Ms
ec
CM
VM
FS=1 FS=4
MSS = 4
13
Experiment 1: Summary
• Radar’s manipulations of cognitive load interact with stimulus mapping in ways that parallel Schneider & Shiffrin’s results
• Automaticity develops with extended practice and consistently mapped stimuli even when targets move and time is limited
• Radar task can be used to study automaticity in dynamic stimulus environments
14
• There is some evidence that response mapping is not critical for automaticity to develop (Fisk & Schneider, 1984; Kramer, Strayer, & Buckley, 1991)
• In complex tasks mapping of targets to responses can be inconsistent
o Resulting in large processing costs, even when stimuli are consistently mapped to targets
Experiment 2: Response Consistency
15
Experiment 2: Method
o Only consistently mapped stimuli
o Cognitive Load Variables• Memory Set Size• Frame Size
o Response consistency varied in four levels
16
Mapped to Stimuli Fully Mapped to interface
Partial Mapping to interface Random Mapping
T T
T T
Response Mapping Conditions
17
Experiment 2: Accuracy
0.6
0.7
0.8
0.9
1
Stimulus Full Partial Random
Ac
cu
rac
y
18
Experiment 2 : detect time
600
800
1000
1200
1400
1600
Stimulus Full Partial Random
Ms
ec
19
Experiment 2: Summary
• A consistent response reduces processing requirements
• Total task consistency (both, consistency of stimuli and consistency of responses) matters
o There are processing costs if responses are not consistently mapped, even when stimuli are
• Implicationso Interface design: interface influences processing of
responses• Response selection using track-up vs. north-up displays• Make response selection intuitive• Interface design, decision support tools, training
o We can now systematically manipulate Radar to elucidate the effects of automaticity on high-level dynamic decision-making
20
Experiment 3: Automatic detection & high-level decision making
• How would automatic detection of a component help decision-making?
• Decision-making component required operators to analyze a sensor array of detected aircraft
• Sensor and weapon information changed dynamically
21
Experiment 3: Method
• Sensor Reading Task
• Determine if Target is Hostileo Scan Sensorso > 13 (Hostile)o < 13 (Non-Hostile)
• Press Ignore (5-Key)
• Select Response (Weapon Systems)o Guns vs. Missileso > 10 Missiles (6-Key)o < 10 Guns (4-Key)
• Quiet Airspace Reporto No targets detectedo Click submit report with mouse key
22
Experiment 3: Detect Accuracy
CM
VM
0.6
0.7
0.8
0.9
1
FS=1 FS=4
MSS = 1
Ac
cu
rac
y
CM
VM
FS=1 FS=4
MSS = 4
23
Experiment 3: Decision-making Accuracy
CMVM
0.6
0.7
0.8
0.9
1
FS=1 FS=4
MSS = 1
Ac
cu
rac
y CM
VM
FS=1 FS=4
MSS = 4
24
Experiment 3: Detect Time
CM
VM
FS=1 FS=4
MSS = 4
CM
VM
600
800
1000
1200
1400
1600
FS=1 FS=4
MSS = 1
Ms
ec
25
Experiment 3: Decision-making Time
CMVM
FS=1 FS=4
MSS = 4
CM
VM
600
800
1000
1200
1400
1600
FS=1 FS=4
MSS = 1
Ms
ec
26
Experiment 3: Summary
• Consistent mapping of targets improved he accuracy of the decision-making of the task
• Detect time, detect accuracy, and whole-task performance are sensitive to workload manipulations
• Implicationso Consistent mapping actually improved whole-task
performance by freeing up time for the controlled sensor-reading tasks to run to completion
o Thus, processing speed-up associated with automatic detection can have a large impact on whole-task performance
27
But…?
• Is accuracy of decision-making improved simply because there is more time to process?
• Effect of detection on high-level decision-making in the presence of a dual-task
28
Experiment 3b: Method
• Secondary tone task: enter count of number of non-standard tones
o Calibrated to standard tone at beginning of session for each participant
o Non-standard tones higher/lower pitch than standard
29
Experiment 3b: results
• In fact the Radar task performance was the same with and without the tone task!
• Detect Timeo No Effect of secondary task
• Detect Accuracyo No Effect of secondary task
• Decision-Making Timeo No Effect of secondary task
• Decision-Making Accuracyo No Effect of secondary task
Performance on Tone Task
0.65
0.67
0.69
0.71
0.73
0.75
0.77
0.79
0.81
0.83
0.85
MSS = 1 MSS = 4
Ac
cu
rac
y
CM
VM
30
Experiment 3b: Implications
• No effect of dual task on RADAR performance
• Operators are allocating resources away from tone task to maintain RADAR performance
• Implicationso Finding supports the hypothesis that consistent
mapping improves decision-making performance by freeing up resources for other tasks
o Thus, processing speed-up and low resource requirement associated with consistent mapping can have a large impact on performance in complex task
31
Experiment 4: Categorization
• Since consistent mapping is the search for numbers in letters, it is possible that load-free processing is due to categorization (Cheng, 1985)
• Purpose of this experiment is to establish the presence of load-free processing without categorization
32
Experiment 4: Method
• Incorporate memory ensembles where no possible categorization can take place either a priori or with learning
• CM vs. VM with toneo CM = {C, G, H, M, Q, X, Z, R, S}o VM = {B, D, F, J, K, N, W, P, L}
• Memory ensembles were equatedo Angular {H,M,X,Z,F,K,N,W} vs. Round {C,B,D,G,Q,P,R,J}o Beginning {B,C,D,F,G,H,J,K} vs. End {M,N,P,Q,R,W,X,Z}
• Cognitive Load Variableso Memory Set Size (1 or 4)o Frame Size (1 or 4)
• Indicated detection of target by pressing spacebaro Detect Performanceo Detect Response Time
33
Experiment 4: Detect accuracy
CMVM
0.6
0.7
0.8
0.9
1
FS=1 FS=4
MSS = 1
Ac
cu
rac
y
CM
VM
FS=1 FS=4
MSS = 4
34
Experiment 4: Decision-making accuracy
CM
VM
0.6
0.7
0.8
0.9
1
FS=1 FS=4
MSS = 1
Ac
cu
rac
y
CM
VM
FS=1 FS=4
MSS = 4
35
Experiment 4: Detect time
CM
VM
FS=1 FS=4
MSS = 4
CMVM
600
800
1000
1200
1400
1600
FS=1 FS=4
MSS = 1
Ms
ec
36
Experiment 4: Decision-making time
CMVM
FS=1 FS=4
MSS = 4
CMVM
600
800
1000
1200
1400
1600
FS=1 FS=4
MSS = 1
Ms
ec
37
Experiment 4: Implications
• Varied mapped performance is more sensitive to load than consistently mapped performance
• Individuals performed better in the high-level decision-making component of Radar when stimulus mapping was consistently mapped
• Implicationso Categorization is NOT a necessary requirement
for automaticity developmento Consistent stimulus mapping is a necessary
condition for the development of automatic detection
38
Summary of accomplishments
• Developed Radar, a dynamic simulation where it is possible to study (i.e., to measure) automaticity
• In Radar it is possible to elucidate the effects of automaticity on high-level dynamic decision-making
• Established the usefulness and applications of the dual-process theory of automaticity
• Deepen our understanding of the implications of automaticity development for practical real-world tasks
• Brought together two main theories of automaticity: instance-based theory and dual-process theory
39
Future research
• Consistency of mapping and responding is relative to the categories (i.e., similarity) that a user can form
• Thus, consistent mapping can lead to automatic responses for high-level decision-making after extended practice
40
Looking towards applications
• Test these hypotheses in airport luggage screening
• Decide whether to hand search the luggage
• There is no consistency but rather just similarity (relative to a ‘knife’ category)