Teaching and Assessing Critical Reasoning in the Era of

Competency-based Medical Education, Milestones and

Entrustment

Preventing Diagnostic Error

“We’re pretty sure it’s the West Nile virus.”

Clinical Reasoning: A Primer

Patient/situation

characteristicsPrior knowledge

Problem Representation

Information GatheringContext

Evaluation Action

Gruppen and Frohna, International Handbook on Research, 2002

Clinical Reasoning

Internal process Trainees and faculty need ways to

externalize and teach this process Programs need assessment methods that

document growth and competency in this skill

Critical Thinking Good Thinking

– Sensitivity• Interest in gaining

more information• Seeking

alternatives– Inclination

• Willing to invest energy in thinking the matter through

– Ability• Possess the

cognitive ability

“Bad Thinking” – “cognitive misers”– Chose to take

mental shortcuts, engage in heuristic thinking,

– without interest in “good thinking”

Krupat 2011

 

Adverse Events and Clinical Reasoning

Graber¹ adverse event study:– Most errors combination of individual and

systems factors– Average 5.9 system +/- cognitive error per case

“Cognitive factors” – 320 cognitive factors in 74 cases– 45 due to faulty data gathering– 264 due to faulty synthesis (problem

representation – clinical reasoning)

¹ Arch Intern Med. 2005; 165: 1493.

The Process of Clinical Reasoning

How can clinical reasoning be taught and assessed in a

competency-based system to reduce diagnostic error?

CBME - a new paradigm

You must truly know the trainee has demonstrated competence and is ready to progress to the next stage of their training or career:• Requires clear definition of expected outcomes

(“good thinking” with decrease in diagnostic errors)

• Requires assessment and evaluation systems capable of demonstrating that these things are done consistently and within the contextual needs of the clinical environment

Requirements

– The content of “good thinking”• Define the K/S/A of “good thinking” and frame as

an entrustment– An assessment and evaluation system– Faculty Development – creating a shared

“mental model” or understanding of “good thinking” and how it is assessed and evaluated

The Content - Twelve Tips “to prevent diagnostic error”

Understand heuristics Use “diagnostic timeouts” Think “worst-case scenario medicine” Systematic approach to common problems Ask why Teach/emphasize physical exam Teach Bayesian theory Acknowledge your emotions Identify what doesn’t fit Embrace zebras “Slow down” Admit mistakes

Trowbridge Medical Teacher 2008

The “Twelve Tips” and the Internal Medicine Curricular Milestones

142 discrete milestones published in 2009 Describe developmentally the discrete K/S/A

needed for competency in the six ACGME General Competencies

Cross walking the 12 tips against the milestones identifies at least 28 milestones that capture the knowledge, skills or attitudes that could be used to teach and assess critical reasoning.

Patient Care

ACGMECompetency

Developmental Milestones InformingACGME Competencies

ApproximateTime FrameTrainee to Achieve Stage

Assessment Methods/Tools

Clinical skills and reasoning

Manages patients using clinical skills of interviewing and physical examination

Historical Data Gathering1. Acquire accurate and relevant history

from the patient in an efficiently customized, prioritized, and hypothesis driven fashion

2. Seek and obtain appropriate, verified, and prioritized data from secondary sources (e.g. family, records, pharmacy)

3. Obtain relevant historical subtleties that inform and prioritize both differential diagnoses and diagnostic plans, including sensitive, complicated, and detailed information that may not often be volunteered by the patient

6 months

9 months

18 months

Standardized patient

Direct Observation

Simulation

Sub-competency

“Entrustment in Medical Education”

Focused assessments around what faculty and training programs “entrust” trainees to do?

Think critically to minimize error

Reflects the most important outcome of training: a trainee’s readiness to bear professional responsibility”

http://www.im.org/AcademicAffairs/milestones/Pages/default.aspx

How do I develop an assessment?

Step 1 – Describe the activity. What tasks constitute the entrustment.

Step 2 – Identify the Curricular Milestones (142) that will help you assess a resident performing this activity.

Step 3 – Identify specific assessment methods / tools to which you can apply the chosen Curricular Milestones.

Clinical Reasoning

Step 1 – Describe the activity. What tasks are required for you to entrust this activity to a resident?

The “Good Thinker” as described by the twelve tips cross walked to the Internal Medicine

Curricular Milestones.

Twelve Tips “to prevent diagnostic error” – The “Good Thinker” Entrustment

Understand heuristics Use “diagnostic timeouts” Think “worst-case scenario medicine” Systematic approach to common problems Ask why Teach/emphasize physical exam Teach Bayesian theory Acknowledge your emotions Identify what doesn’t fit Embrace zebras “Slow down” Admit mistakes

Trowbridge Medical Teacher 2008

Clinical ReasoningStep 2 – Identify the Curricular Milestones (142) that will help you assess a resident performing this activity

Key Considerations:• What Curricular Milestones are best assessed in this

setting? … in this context?• You don’t have to choose all milestones, only those

that will help you to “see” competence in the trainee.• Crosswalk the twelve tips and the 142 curricular

milestones.

Understand heurisitcs 1. MK -1- Demonstrate sufficient knowledge to diagnose and treat undifferentiated and

emergent conditions Use “diagnostic timeouts”

1. PC 3 - Modify differential diagnosis and care plan based upon clinical course and data as appropriate

2. PBLI – 6 - Actively seek feedback from all members of the health care team 3. PBLI – 6 - Calibrate self-assessment with feedback and other external data 4. PBLI – 6 - Reflect on feedback in developing plans for improvement

Think “worst-case scenario medicine” 1. MK 1 - Understand the relevant pathophysiology and basic science for uncommon or

complex medical conditions 2. MK 1 - Demonstrate sufficient knowledge to evaluate complex or rare medical conditions

and multiple coexistent conditions 3. PC 6 - Manage complex or rare medical conditions

Systematic approach to common problems 1. PC 3 - Synthesize all available data, including interview, physical examination, and

preliminary laboratory data, to define each patient’s central clinical problem 2. MK 1 – Demonstrate sufficient knowledge to evaluate common ambulatory conditions 3. MK 1 - Demonstrate sufficient knowledge to diagnose and treat common conditions that

require hospitalization Ask why

1. PBLI 2 - Classify and precisely articulate clinical questions 2. PBLI 2 - Develop a system to track, pursue, and reflect on clinical questions

Teach/emphasize physical exam 1. PC 1 - Obtain relevant historical subtleties that inform and prioritize both differential

diagnoses and diagnostic plans, including sensitive, complicated, and detailed information that may not often be volunteered by the patient

2. PC 2 - Routinely identify subtle or unusual physical findings that may influence clinical decision making, using advanced maneuvers where applicable

Teach Bayesian theory 1. MK 2 - Understand prior probability and test performance characteristics 2. SBP 5 - Minimize unnecessary care including tests, procedures, therapies and ambulatory or

hospital encounters 3. SPB 4 - - Understand how cost-benefit analysis is applied to patient care (i.e. via principles of

screening tests and the development of clinical guidelines) Acknowledge your emotions

1. P 6 - Recognize and address personal, psychological, and physical limitations that may affect professional performance

2. P 9 - Recognize and manage conflict when patient values differ from their own 3. P 5 - Recognize and manage subtler conflicts of interest

Identify what doesn’t fit 1. PBLI 5 - Determine if clinical evidence can be generalized to an individual patient 2. Patient care 3 - Recognize disease presentations that deviate from common patterns and

that require complex decision making Embrace zebras

1. MK 1 - Understand the relevant pathophysiology and basic science for uncommon or complex medical conditions

2. PK 6 - Manage complex or rare medical conditions “Slow down”

1. Maintain awareness of the situation in the moment, and respond to meet situational needs 2. Reflect (in action) when surprised, applies new insights to future clinical scenarios, and

reflects (on action) back on the process Admit mistakes

1. P 1 - Accept personal errors and honestly acknowledge them

Clinical ReasoningStep 3 – Identify specific assessment methods / tools to which you can apply the chosen Curricular Milestones.

Key Considerations:• What Curricular Milestones are best assessed in this

setting? … in this context?• You don’t have to choose all milestones, only those

that will help you to “see” competence in the trainee

Methods Portfolio – with required defense by learner

– Case log– Focused narrative writing– Admit mistakes

Chart stimulated recall– Structured questioning regarding the twelve tips

milestones Bedside rounds

– One minute preceptor– Time out

Please check ONLY ONE box per statement Y N NA

Clear chief complaint

Delineation of sick vs non-sick

Appropriate history

Appropriate physical

Appropriate analysis of lab data

Appropriate differential diagnosis

Appropriate thought process for differential diagnosis

Treatment appropriate for diagnosis

Appropriate thought process for treatment plan

Overall Note:Clarity____________________________________Organization__________________________________Internal consistency____________________________________Documentation____________________________________

The One Minute Preceptor – A Strategy For Busy Clinicians

Clinical teaching strategy 5 microskills

• Get a commitment• Probe for supporting evidence• Teach a general rule• Reinforce what was done right• Correct mistakes• “Create time for reflection”

Neher, Gordon, Meyer, Stevens. J Am Board Fam Pract 1992; 5:419-24.

“The System”

The Donobedian Framework – Schematic representation of a system

Every system has a structure and a process that processes that produce an outcome

S + P = OS(the when/where) + P (the teaching and

assessing) = O (“Good thinkers”)