seeing things differently: better performance in resuscitation-based simulation … ·...
TRANSCRIPT
Seeing Things Differently: Better Performance in Resuscitation-Based Simulation OSCEs is Associated
With Specific Gaze Patterns
Adam Szulewski, Gerhard Dashi, Nick McGraw, Rylan Egan, Andrew Hall,
Damon Dagnone, Dan Howes
Seeing Things Differently: Better Performance in Resuscitation-Based Simulation OSCEs is Associated With Specific Gaze Patterns
Adam Szulewski BSc, MD, FRCPC, MHPE
Assistant Professor, Dept of Emergency Medicine, Queen’s University
PhD Candidate, Maastricht University
Emergency, Trauma and RACE Team Physician, KGH
I do not have an affiliation (financial or otherwise) with a
pharmaceutical, medical device or communications organization.
Je n’ai aucune affiliation (financière ou autre) avec une
entreprise pharmaceutique, un fabricant d’appareils
médicaux ou un cabinet de communication.
3
Why resuscitation medicine?
http://www.cedars-sinai.edu/Patients/Programs-and-Services/Trauma-Program/Trauma-Program-Continuum-of-Care.aspx
Why resuscitation medicine?
• The stakes are high
• It is a complex environment
Numerous medical decisions
Team management, leadership and other elements of crisis resource
management (CRM) are key
• Little is known about expertise development in this setting
• Expert resususcitationists are easy to identify, but it is very
difficult to explain what exactly makes them expert
http://www.cedars-sinai.edu/Patients/Programs-and-Services/Trauma-Program/Trauma-Program-Continuum-of-Care.aspx
6
Why simulation?
7
• Common and effective method to teach CRM in resuscitation
medicine
• Allows for feedback, practice, curriculum integration
Why simulation?
Assessment challenges
• The milestones for the
development of competence (and
expertise) in resuscitation medicine
are not fully known.
• Expertise is tacit
8
http://www.innerbody.com/careers-in-health/psychology-degrees.html
Gaze tracking and expertise
• Getting inside the expert’s head
• There are measurable visual
differences between novices and
experts in numerous domains
(sports, driving) that can be
quantified using eye-tracking
9
Source: Frits Ahlefeldt / Wikimedia Commons
Basic premise: Assessing competence and expertise
expertise development
http://scrubs.wikia.com/wiki/My_First_Day https://dontfierme.files.wordpress.com/2008/01/jdiz.jpg
competence expertise
Question
• In medical simulation resuscitation-based OSCEs, how do
residents’ gaze patterns differ with respect to expert-rated
performance?
• A pilot study
Methods
12
84 cases
August OSCE 2 cases
February OSCE 2 cases
QSAT score by blinded external
assessor
Gaze-tracking metrics
QSAT
13
Gaze tracking
14
Diabetic ketoacidosis case
The amount of time it took to first check vital signs was negatively correlated with performance
The number of times the participant looked at the respiratory therapist was negatively correlated with
performance
High performers seem to rapidly recognize that the RT did not have much new (or relevant) information to
provide and check the vital signs early.
These high performers tend to treat the DKA algorithmically (the treatment of DKA is well described). As a
result, they don’t tend to focus their visual attention on the RT (as they have no relevant information to
provide).
Variables 1 2 3 4
Diabetic ketoacidosis (N = 11)
1. Average Score -
2. Number of RT views (First 60 Seconds) -.65** -
3. Time until participant checked patient’s
Vitals
-.70** .48 -
Ruptured abdominal aortic aneurysm case
The total time spent looking at the lab values and the ECG were negatively
correlated with performance
The number of times a participant looked at the RN was negatively correlated
with performance
The key to this scenario was quick diagnosis and then focusing attention on
getting the patient to an operating room.
The lab values, ECG and RN did not provide extra clues to the high performers.
Ruptured abdominal aortic aneurysm
1. Average Score -
2. Total time spent looking at lab values (First
60 Seconds)
-.50** -
3. Total time spent looking at ECG (First 60
Seconds)
-.39** .16 -
4. Number of RN views (First 60 Seconds) -.46** .54*** .117 -
Variables 1 2 3 4
Ethylene glycol toxicity
Ethylene glycol toxicity
1. Average Score -
2. Number of medication list views -.48** -
17
The number of medication list views was negatively correlated with performance The medication list did not provide any relevant information as to the eventual diagnosis in
this scenario. High performers looked at this list, analyzed it once, and then discarded it. They focused on
basic resuscitation priorities for an undifferentiated patient with altered mentation Poor performers seemed to come back to these pieces of information to try to determine
what was happening in the scenario as the diagnosis of ethylene glycol poisoning was not immediately apparent (reaching for clues?)
Variables 1 2 3 4
Conclusions
• Specific gaze patterns were correlated with objective
performance.
• These gaze behaviours were different from scenario to
scenario.
We hypothesize that this is because each scenario had different
“relevant” vs. “irrelevant” cues (or visual endpoints)
18
Conclusions
• Overall, higher performers were able to decide what was
relevant or irrelevant efficiently.
Irrelevant information was appropriately discarded and relevant
information was focused upon by the high performers.
Poor performers seemed to have difficulty deciding how to
appropriately divide their attention between relevant and
irrelevant information.
19
Next steps
• This was a pilot study
• Systematic (and partially automated) data analysis method
• Creating new scenarios with appropriate “task relevant” and
“task redundant” stimuli to test the hypotheses generated by
this pilot study.
20
expertise development
http://scrubs.wikia.com/wiki/My_First_Day https://dontfierme.files.wordpress.com/2008/01/jdiz.jpg
competence expertise
Assessing competence and expertise
Concluding thoughts
• This data provides a new way to think about our learners’
progress on the novice/expert continuum.
• Can we intervene earlier if we identify learners whose
expertise development is not progressing as expected?
• Implications for CBME
Would an examiner/program director/Royal College committee
want to know whether a candidate is novice-like or expert-like in
their cognitive and gaze behaviour characteristics (beyond
traditional assessment metrics)?
22
http://www.google.ca/url?sa=i&rct=j&q=&esrc=s&source=images&cd=&cad=rja&uact=8&docid=lY7CX_2xV5n8-M&tbnid=cMAEykDATlx7lM:&ved=0CAQQjB0&url=http%3A%2F%2Fwww.jeffkorhan.com%2F2011%2F08%2Fask-your-customers-the-tough-questions.html&ei=mb5NU7e7CpLW2wX_n4GAAg&psig=AFQjCNG_RWJqyPaBUpl-VS0jkMACUJVD8w&ust=1397690285007845
• Download the ICRE App,
• Visit the evaluation area in the Main Lobby, near Registration, or
• Go to: http://www.royalcollege.ca/icre-evaluations to complete the session evaluation.
Help us improve. Your input matters.
• Téléchargez l’application de la CIFR
• Visitez la zone d’évaluation dans le hall principal, près du comptoir d’inscription, ou
• Visitez le http://www.collegeroyal.ca/evaluations-
cifr afin de remplir une évaluation de la séance.
Aidez-nous à nous améliorer. Votre opinion compte!
You could be entered to win 1 of 3 $100 gift cards.
Vous courrez la chance de gagner l’un des trois chèques-cadeaux d’une valeur de 100.
24