by: martin warters, ma, medscape education,...

PB 1

In order to meet the needs of an evolving healthcare system, effective continuing education for healthcare professionals should be competency-based, interactive, experiential, and continuous. Virtual patient simulation (VPS) is a proven educational modality that supports competency-based learning and is grounded in the science of mastery learning. Mastery learning emphasizes the importance of establishing a baseline diagnostic performance assessment; setting well-defined learning objectives and outcomes measures; sequencing tasks and decisions across increasing levels of difficulty; and providing feedback.

Educational research shows that learning and performance outcomes improve when feedback entails specific

guidance and is targeted to learner needs. In VPS-based activities, feedback on learner decisions and their potential consequences reinforces good practice and helps learners to identify paths to improvement in order to close gaps between actual and desired clinical performance.

However, feedback is a complex concept with many dimensions that contribute to its effectiveness. By developing a greater understanding of what kind of feedback to use, when to use it, and how to deliver feedback for optimal effect, continuing education stakeholders will be better able to design and deliver educationally sound learning in virtual environments that enables clinical learners to move toward mastery.

Travis is a 70-year-old Caucasian man recently admitted to hospital with chest pain, shortness of breath and a cough. Dr. Jain examines Travis and learns that he has rheumatoid arthritis (RA), for which he takes methotrexate (MTX). Travis tells Dr. Jain that he doesn’t understand why he hasn’t had his RA medicine today. Dr. Jain is puzzled that Travis hasn’t been prescribed MTX, and so writes him up for 2.5mg oral.

The attending, Dr. Smith, arrives at that point and after introducing himself to Travis, reviews his chart. Noting the addition of MTX, Dr. Smith says sharply, “Dr. Jain, you might bother to read patient notes once in a while. If you did, you’d see that Travis has pneumonitis. That’s why we stopped the MTX!”

Dr. Jain looks at Travis and immediately blushes and feels foolish for not checking the chart. The attending’s real-time clinical feedback was ineffective and did not provide Dr. Jain with an opportunity to reflect on his practice and identify a plan for improvement. The content was evaluative, the tone judgmental, and the intention was directive, rather than facilitative. These features undermine the potential for feedback in learning situations.

Vignette 1

LOUDand CLEAR

What makes effective feedback in virtual patient simulation?

BY: MARTIN WARTERS, MA, MEDSCAPE EDUCATION, LLC.

INTRODUCTION

OVERVIEW

LOUD and CLEAR — What makes effective feedback in virtual patient simulation?

2 3

VPS-Based Education and Mastery Learning

In order to meet the needs of an evolving healthcare system, effective continuing education for healthcare professionals should be competency-based, interactive, experiential, and continuous. To this end, virtual patient simulation (VPS) has emerged as a proven educational modality that provides opportunities for clinicians to build competency by executing clinical tasks, making patient management decisions, and testing out newly acquired knowledge in complex, authentic scenarios similar to those they can anticipate in clinical settings. Indeed, four decades of healthcare education research show that simulation technology such as VPS can have large and sustained effects on healthcare provider (HCP) knowledge and skills acquisition and maintenance.1

VPS-based activities are grounded in the cognitive principles of mastery learning, which emphasizes the importance of establishing a baseline diagnostic performance assessment; setting well-defined learning objectives and outcomes measures; sequencing tasks and decisions across increasing levels of difficulty; and providing feedback to learners about their task performance, actions, or decisions.1,2

But what do educators mean when they talk about feedback?

Feedback in Clinical Education

Feedback is important for learners in any education context, and is broadly defined as a mechanism that provides information in response to learner actions or decisions that learners can use to make adjustments in reaching a goal, or to change the methods and patterns of their performance.3,4 Although feedback in everyday life is often naturalistic or implicit (such as feeling physical pain when one touches a hot surface), in clinical education, feedback typically comprises explicit confirmatory or corrective information delivered by an external source (eg, a clinical educator). External feedback has been a cornerstone of clinical learning since antiquity.5 Yet, while Galen of Pergamon and other Greek scholars delivered feedback as ‘kindly meant advice,’ modern clinical feedback has evolved into a structured mechanism with the capacity to incentivize learners to maintain interest in learning, and to continuously improve their performance.6

Contemporary educational scholarship defines learning as a highly social activity that involves continuous interaction between cognitive, behavioral, and environmental influences7 and is based on cognitive principles such as reflection-on-practice and self-appraisal.8,9 In this social model of learning, feedback is a mechanism that:

1) Enables learners to reconstruct their knowledge and become—or remain—motivated for future learning;10

2) Encourages learners to modify behaviors (or reinforces effective behaviors);4,11 and

3) Helps learners to integrate new information that supports changes in thinking, behavior, practice, or performance.11,12

Additionally, in this social model of learning, educators become facilitators who encourage learners to critique their own actions, recognize their learning deficits, identify ways to improve their performance, and implement and evaluate plans for success.11,13

Feedback is broadly defined as a mechanism

that provides information in response to learner

actions or decisions that learners can use to make adjustments in reaching a goal, or to change the

methods and patterns of their performance.

1

23

MOVING TOWARD MASTERY: FEEDBACK IN MEDICAL EDUCATION


2 3

Barriers to Providing Effective Feedback in Clinical Education

There are multiple barriers to the effective design and delivery of feedback in clinical education. First, clinical educators often use evaluation and feedback interchangeably, although these are discrete concepts. Whereas evaluation confers judgment and elicits an emotional response from learners, feedback is neutral, and provides sufficiently detailed information about the learner’s actions and the consequences of those actions.14 Evaluation—or negative feedback—is often associated with indirect or ambiguous language that can lead to defensiveness and demotivation among learners, limit opportunities for learners to self-correct, or reinforce learning by trial and error at patient expense.3,11 Second, the source credibility of feedback is key to the recipient’s interpretation and use of information in the learning process.15,16 If learners do not feel that feedback is delivered from ‘a position of beneficence,’ then learning can be undermined.17

These and other barriers limit not only feedback opportunities in clinical practice, but also the potential for behavior change and performance improvement in health care.5 Accordingly, continuing healthcare education provides an important vehicle for providing feedback that enables learners to not only calibrate their practice in education settings, but also identify paths to improvement that close gaps between actual and desired performance in clinical settings.3,18


4 5

Education and social science scholarship shows that feedback in continuing healthcare education—whether in actual or simulated clinical settings—is both necessary and effective in engaging HCPs in learning and in improving performance.19,20,21 Broadly, learning and performance outcomes improve when feedback is timely and entails specific guidance that is delivered in response to learner actions in a one-to-one actual or simulated environment.22

Effective feedback includes the following specific components.

COMPONENTS OF EFFECTIVE FEEDBACK IN CONTINUING HEALTHCARE

Content

Feedback content has different functions, but is most effective in supporting learning and performance improvement when it is formative. This formative aspect of feedback is derived from a mastery model of learning that places considerable emphasis on deliberate practice—itself is a key ingredient in information processing and skills acquisition. Formative feedback reinforces positive behaviors, allows learners to correct errors, provides direction for improvement, and helps learners to develop expertise and build mastery.2,23 Formative feedback typically contains verifying information that offers insight into what learners did well and about the consequences of their choices/actions, as well as elaborating information to guide the learner toward an end goal.3,24

Components of Effective Feedback6

• Descriptive vs evaluative• Focus on behavior vs personality• Specific vs general• Sharing information vs giving advice• Digestible chunks of information to

avoid cognitive overload


4 5

Language

To avoid defensiveness and embarrassment, feedback needs to be delivered in a non-evaluative way, using descriptive, constructive language.3,23 Direct, focused language that addresses learner tasks or choices rather than learner intention diffuses defensiveness, and minimizes negative emotional responses from learners.11,23 Further, learners are best able to make course corrections or plan for improvement when feedback is concrete, rather than abstract or general, and focuses on specific behaviors and decisions that immediately precede the feedback. Effective concrete feedback is delivered in small, digestible chunks and clearly labelled as feedback, in order to supply learners with important connections between different kinds of information and reduce cognitive load.6,11,22,25

Timing

The timing of effective feedback matters. Immediate feedback delivered in real time accompanied by encouragement to reflect on choices and actions furnishes learners with opportunities to make any necessary course corrections during activities.14 Delayed, or summative, feedback provides measures of learning outcomes and whether learners have achieved the learning goals that are required before advancing to the next level.1,11

Sources

The effectiveness of feedback is also enhanced when it is normalized and repeated throughout the learning process, and is derived from credible or authoritative actual or virtual sources.4 Research suggests that learners will discount feedback from sources that they feel lack clinical knowledge or expertise.12,24 Effective feedback also needs to be facilitative, and given in a way that motivates learners and shows them how to plan for improvement and apply new insights in practice without being directive.6

Effective Feedback Delivery Strategies5,11

• Provide feedback in a sequence of steps (Pendleton's Rules)

• Include both reinforcing statements and corrective guidance (feedback sandwich)

• Use formative and summative feedback at different times

• Offer individualized feedback• Make feedback memorable• Ground feedback in comparisons between

learner performance and expert decision


6 7

REFERENCES1. McGaghie WC, Issenberg SB, Barsuk JH, Wayne DB. A critical review of

simulation-based mastery learning with translational outcomes. Med Ed. 2014;48:375-385.

2. Ericsson KA, Krampe RT, Tesch-Römer C. The role of deliberate practice in the acquisition of expert performance. Psychol Rev 1993;100:363.

3. Ende J. Feedback in Clinical Education JAMA. 1983;250:777-781.

4. Veloski J, Boex JR, Grasberger MJ, Evans A, Wolfson DB. Systematic review of the literature on assessment, feedback and physicians’ clinical performance: BEME Guide No. 7. Med Teach. 2006;28:117-128.

5. Telio S, Ajjawi R, Regehr G. The “educational alliance” as a framework for reconceptualizing feedback in medical education. Acad Med. 2015;90:609-614.

6. Richardson BK. Feedback. Acad Emerg Med. 2004;11:e1-5.

7. Bandura A. Human agency in social cognitive theory. Am Psych. 1989;44(9):1175-1184.

8. Schön DA. Educating the reflective practitioner. San Francisco: Jossey-Bass. 1987.

9. Vygotsky LS. Interaction between learning and development. In: Cole M, John-Steiner V, Scribner S, Souberman E, editors. Mind in society: The development of higher psychological processes. Cambridge, MA: Harvard University Press; 1978. pp. 79–91.

10. Ertmer PA, Newby TJ. Behaviorism, cognitivism, constructivism: comparing critical features from an instructional design perspective. Performance Improvement Q 1993;6:50-70.

11. Cantillon P, Sargeant J. Giving feedback in clinical settings. BMJ. 2008;337:a1961.

12. Bing-You RG TR. Why Medical Educators May Be Failing at Feedback. JAMA. 2009;302:1330-1331.

13. Benedict N SK, Mcgee J. Promotion of Self-directed Learning Using Virtual Patient Cases. American J Pharm Ed. 2013;77:151.

14. Sender Liberman A LM, Steinhert Y, et al. . Surgery residents and attending surgeons have different perceptions of feedback. Med Teach. 2005;27:470-472.

15. Sargeant J, Mann K, van der Vleuten C, Metsemakers J. “Directed” self-assessment: Practice and feedback within a social context. J Contin Educ Health Prof. 2008;28:47–54.

LOUD AND CLEAR: EFFECTIVE FEEDBACK IN VPS-BASED EDUCATION

Education research increasingly shows that simulation-based learning—such as VPS—that is designed within a mastery learning framework can contribute to downstream effects that include improvements in procedural and communication skills; reductions in patient complications, readmission, hospital length of stay; and lower healthcare costs.1 Notably, a recent synthesis of the accumulated evidence over 34 years in educational science also identified feedback as the most important feature of simulation-based medical education—including VPS-based activities—for yielding better learning outcomes.20,21 Inclusion of the following features can help continuing education stakeholders design effective VPS-based activities.

Timeliness of Feedback

Real-time feedback or clinical guidance in VPS-based activities can boost learner self-confidence and perseverance.18 Formative feedback is best delivered via personalized, constructive, diagnostic debriefs that outline the key steps learners need to take to achieve a particular goal.26 Instant debriefs offer immediate commentary to reinforce key concepts at the point of learning and offer corrective guidance about the suitability of selections

made, error alerts, and alternative options. Learners can use what they have learned from this feedback to make another decision in the context of a particular learning activity or scenario. This approach allows learners to repeatedly test and actively explore the implications of feedback as they move through activities; to review the consequences of their choices or actions; and, if necessary, to make real-time clinical course corrections.27

In VPS-based activities, learner actions are also recorded and evaluated to create delayed, summative feedback. Summative feedback provides insights into overall performance and needed areas of education, allows learners to view and compare their personal achievements with peer-to-peer rankings, and details whether learners have achieved the learning goals that are required before advancing to the next level.1,18

Language

VPS-based feedback provides clinical guidance that focuses solely on remediable actions and choices, rather than on learner intentions. This design neutralizes connotations of good or bad in feedback content and eliminates the potential for learner embarrassment.12

6


6 7

16. Sargeant J, Armson H, Chesluk B, et al. The processes and dimensions of informed self- assessment: A conceptual model. Acad Med. 2010;85:1212–1220.

17. Eva KW, Armson H, Holmboe E, et al. Factors influencing responsiveness to feedback: On the interplay between fear, confidence, and reasoning processes. Adv Health Sci Educ Theory Pract. 2012;17:15–26.

18. Issenberg SB ea. Features and uses of high-fidelity medical simulations that lead to effective learning: A BEME systematic review. Med Teach. 2005;27:10–28.

19. Cervero RM, Gains JM. Effectiveness of Continuing Medical Education: Updated Synthesis of Systematic Reviews. Chicago, IL: Accreditation Council for Continuing Medical Education 2014. http://www.accme.org/sites/default/files/2014_Effectiveness_of_Continuing_Medical_Education_Cervero_and_Gaines_0.pdf. Accessed November 15, 2015.

20. Cook DA, Hatala R, Brydges R, et al. Technology-enhanced simulation for health professions education: a systematic review and meta-analysis. JAMA. 2011;306:978-988.

21. Cook DA, Brydges R, Zendejas B, Hamstra SJ, Hatala R. Mastery learning for health professionals using technology-enhanced simulation: a systematic review and meta-analysis. Acad Med. 2013;88:1178-1186.

22. Billings D. Adaptive feedback in simulation-based training. Unpublished Doctor of Philosophy thesis, Department of Psychology, College of Sciences University of Central Florida. http://etd.fcla.edu/CF/CFE0003225/Billings_Deborah_R_201008_PhD.pdf. Accessed November 15, 2015.

23. Bernard AW, Kman NE, Khandelwal S. Feedback in the emergency medicine clerkship. West J Emerg Med. 2011;12:537-542.

24. McGaghie WC, Issenberg SB, Petrusa ER, Scalese RJ. A critical review of simulation-based medical education research: 2003-2009. Med Ed. 2010;44:50-63.

25. Branch WT Jr, Paranjape A. Feedback and reflection: teaching methods for clinical settings. Acad Med. 2002;77:1185-1188.

26. Downing SM, & Yudkowsky, R. Assessment in health professions education. New York, NY: Routledge; 2009.

27. Croskerry P. The Feedback Sanction. Acad Emerg Med. 2000;7:1232-1238.

Language focused on actions rather than on intent also increases psychological distance, allowing learners to engage in dispassionate critique of their performance,3 and avoids the potential for learners to use cognitive mechanisms (eg, denial, distorting information,

discounting) to protect themselves from information they do not want to hear.12 Importantly, VPS-based feedback parses information to learners into small, manageable chunks to avoid cognitive overload.

7

VPS feedback is typically structured around pre-determined rules derived from current standards of care that depend on learners’ actual performance during the virtual interaction.


8 PB

MedSims is Medscape Education’s proprietary simulation-based education architecture. MedSims virtual patient simulations enhance learning via visually appealing and immersive experiences that target multiple learning styles and incorporate formative assessments with powerful metrics and clinical guidance on performance for a given scenario. MedSims reinforces key concepts at the point of learning via timely, individualized, and authoritative feedback that is developed by expert clinicians, and also provides insights into learners’ overall performance and areas of need for further education.

Refine mastery learning in virtual environments. To learn more, visit Medscape.org/vision or contact Chris Hoffman at [email protected].

Dr. Jain participates in a VPS-based activity featuring Travis, a 70-year-old Caucasian man with RA who was recently admitted to hospital with chest pain, shortness of breath, and a cough. Travis takes daily MTX 2.5mg for his RA, which was stopped on admission to hospital.

On review of Travis’s history and test results Dr. Jain makes a diagnosis of pneumonitis. When prompted to make a decision about medications, he selects restarting Travis on MTX 1mg.

Following Dr. Jain’s selection, a clinical guidance pop-up indicates that he has made a prescribing error. The clinical guidance states that restarting MTX should not be considered in a patient with a recent episode of MTX-induced lung disease—such as pneumonitis. The activity returns Dr. Jain to the original medication decision point and he selects an alternative drug to avoid further lung injury.

This real-time immediate debrief allowed the learner to return to the initial selection and make a course correction based on clinical guidance—with better patient outcomes.

Vignette 2

As continuing healthcare education evolves to support the delivery of evidence-based, cost-effective, quality-driven healthcare, VPS-based activities are uniquely positioned to build expertise or mastery that learners can transfer to real clinical settings.

CONCLUSION

The author would like to gratefully acknowledge the skillful writing assistance of Alexandra Howson, MA, PhD, CHCP, of Thistle Editorial, LLC.

by: martin warters, ma, medscape education,...

Documents