evaluation: making it work borrowed from the “mahec” preceptor program. e-learning_tools.asp m...

EVALUATION:EVALUATION:Making it WorkMaking it Work

Borrowed from the Borrowed from the “Mahec” preceptor “Mahec” preceptor

program. program. http://www.mahec.net/pdhttp://www.mahec.net/pdp/e-Learning_Tools.aspp/e-Learning_Tools.asp

M Modified for the UBC, odified for the UBC, Family Practice, Family Practice,

Postgraduate ProgramPostgraduate Program

http://www.mahec.net/pdp/e-Learning_Tools.asp

http://www.mahec.net/pdp/e-Learning_Tools.asp

Presentation based on materials Presentation based on materials from the:from the:

Produced by :Produced by :

Supported by HRSA Family Medicine Supported by HRSA Family Medicine Training Training Grant # 1 D15 Grant # 1 D15

PE50119-01PE50119-01

EvaluationEvaluation: Making it : Making it WorkWork

Goals:Goals:1. Review characteristics of 1. Review characteristics of

evaluation.evaluation.

2. Discuss pitfalls in evaluation.2. Discuss pitfalls in evaluation.

3. Outline a system for effective 3. Outline a system for effective evaluation. evaluation.

What is the difference What is the difference between feedback and between feedback and

evaluation?evaluation? Feedback Feedback

Take 3 minutes to Take 3 minutes to discuss.discuss.

EvaluationEvaluation

Evaluation: What is it?Evaluation: What is it?

FeedbackFeedback vs. vs. EvaluationEvaluation



Timing:Timing: TimelyTimely Scheduled Scheduled




Setting:Setting: InformalInformal Formal Formal





Basis:Basis: Observation Observation ObservationObservation





Basis:Basis: Observation Observation ObservationObservation

Content:Content: ObjectiveObjective Objective Objective





Basis:Basis: Observation Observation Observation Observation

Content:Content: ObjectiveObjective Objective Objective

Scope:Scope: Specific Global Specific Global Action Action Performance Performance


FeedbackFeedback vs. vs. EvaluationEvaluationTiming:Timing: TimelyTimely Scheduled ScheduledSetting:Setting: InformalInformal Formal FormalBasis:Basis: Observation Observation

ObservationObservationContent:Content: ObjectiveObjective Objective ObjectiveScope:Scope: Specific Global Specific Global

Action Action Performance PerformancePurpose:Purpose: ImprovementImprovement “Grading” “Grading”

Evaluation: What is Evaluation: What is it?it?

Characteristics of EvaluationCharacteristics of Evaluation ScheduledScheduled FormalFormal ObjectiveObjective Observation BasedObservation Based Global PerformanceGlobal Performance Required for gradingRequired for grading

EvaluationEvaluation

Why is it Why is it important?important?

Evaluation: why is it Evaluation: why is it important? important?

A Requirement of the A Requirement of the Residency ProgramResidency Program

Professional ResponsibilityProfessional Responsibility A Teaching/Learning A Teaching/Learning

OpportunityOpportunity

Evaluation: Why is it Evaluation: Why is it difficultdifficult

What do you find difficult or What do you find difficult or stressful about doing an stressful about doing an evaluation?evaluation? In groups of 2 or 3 discuss the In groups of 2 or 3 discuss the

aspects of evaluation which aspects of evaluation which give problems. give problems.

Why is Evaluation Why is Evaluation difficultdifficult

TimeTime Damage to relationship between the Damage to relationship between the

teacher and the learnerteacher and the learner Implications of a negative evaluation Implications of a negative evaluation

on the learneron the learner Unsure of the “Benchmark of Unsure of the “Benchmark of

comparison”comparison” OTHER SUGGESTIONS?OTHER SUGGESTIONS?

PitfallsPitfalls

““Halo Effect”Halo Effect”““Oops”-Insufficient EvidenceOops”-Insufficient Evidence““You never told me that!”You never told me that!”““But I NEED Honors!”But I NEED Honors!”““Uh-Oh”-Should they pass?Uh-Oh”-Should they pass?““Lake Wobegon” EffectLake Wobegon” Effect

Evaluation: Making it Evaluation: Making it WorkWork

STRATEGYSTRATEGY G …Get ReadyG …Get Ready R … Review Expectations with R … Review Expectations with

LearnerLearner A … Assess: Observe and RecordA … Assess: Observe and Record D … Discuss Evaluation at Mid-PointD … Discuss Evaluation at Mid-Point E … End with a “Grade”E … End with a “Grade”

Evaluation: Making It Evaluation: Making It WorkWork

Get ReadyGet Ready Review the objectives the resident has for Review the objectives the resident has for

the rotation. the rotation. Review the Evaluation Form for your Review the Evaluation Form for your

rotation rotation http://www.familypractice.ubc.ca/residency/http://www.familypractice.ubc.ca/residency/forms/forms/

Consider unique Opportunities/Challenges Consider unique Opportunities/Challenges of your siteof your site

What are your Expectations for the learner?What are your Expectations for the learner?


Review Expectations with the LearnerReview Expectations with the Learner Schedule a meeting with your resident Schedule a meeting with your resident

before the rotation begins to discuss before the rotation begins to discuss objectives.objectives.

Review the skill level expected of the Review the skill level expected of the resident at his or her level of training: resident at his or her level of training: http://www.familymed.ubc.ca/residency/fahttp://www.familymed.ubc.ca/residency/facultydevelopment/BordageModel.htmcultydevelopment/BordageModel.htm

Review your expectations. Review your expectations. Describe the Evaluation ProcessDescribe the Evaluation Process


AssessAssessObserve and record. Observe and record.

http://http://www.familypractice.ubc.ca/www.familypractice.ubc.ca/residency/formsresidency/forms

Provide Feedback RegularlyProvide Feedback Regularly

Self AssessmentSelf Assessment

Encourage the learner to self assess:Encourage the learner to self assess: How did that encounter feel to you?How did that encounter feel to you? What did you do well, what did you What did you do well, what did you

have trouble with?have trouble with? How do you think you could change?How do you think you could change? What would you do differently next What would you do differently next

time?time?

Mid-rotation EvaluationMid-rotation Evaluation

In groups of two or three In groups of two or three come up with at least 3 come up with at least 3 reasons why you think mid-reasons why you think mid-rotation evaluations are rotation evaluations are importantimportant. .


Mid- rotation evaluationsMid- rotation evaluations Formal MeetingFormal Meeting Learner and evaluator fill out Learner and evaluator fill out

form in advanceform in advance Compare evaluations together.Compare evaluations together. Discuss differences and how Discuss differences and how

expectations are being metexpectations are being met Plan for Remainder of RotationPlan for Remainder of Rotation


Complete evaluation in advanceComplete evaluation in advanceSchedule sufficient time.Schedule sufficient time.Support your evaluation with Support your evaluation with

examplesexamplesHighlight items that can be Highlight items that can be

worked on in the future.worked on in the future.

GradingGrading

The resident:The resident: Meets expectationsMeets expectations Is Borderline or Is Borderline or Fails to meet expectations.Fails to meet expectations.

Take 5 minutes to discuss Take 5 minutes to discuss what these categories mean what these categories mean to you.to you.

UBC Family Practice Residency ProgramPage 1

Resident Evaluation – Family Practice Rotation

Resident: ___________________________________ Faculty Supervisor: ______________________________ Rotation: ___________________________________ Dates on Service: ________________________________

A. Discussion of Program Objectives (see program website: http://www.familypractice.ubc.ca) Date: _________

B. Resident’s personal objectives for this rotation: Were they met/completed? 1. ________________________________________

2. ________________________________________

3. ________________________________________

4. ________________________________________

C. Mid-Rotation Evaluation Date: ______________________________ Comments:

D. Has Resident been observed (please circle):Taking a History

Yes

No

Using Technical Skills

Yes

NoDoing a Physical Exam

Yes

No

Giving Oral Presentation

Yes

No

UBC Family Practice Residency Program Page 2

Resident Evaluation – Family Practice Rotation

E. The resident has demonstrated a level of competence appropriate to level of training in the following areas:

Unableto

Judge

Needs Improvement

Average

AboveAverage

Exceptional Comments

1 2 3 4 5 6

7 8 9 10

Clinical Skills1.Has knowledge appropriate to level of training

1.Uses knowledge effectively

1.Performs appropriate history/physician assessment

1.Demonstrates skilled focused clinical exam

1.Successfully integrates knowledge and assessment

1.Plans appropriate investigation

1.Formulates appropriate management plan

1.Demonstrates awareness of own clinical limitations

1.Demonstrates appropriate procedural skills

RememberRemember

If you have any difficulty with web-If you have any difficulty with web-eval or with completing your eval or with completing your evaluations contact your Site evaluations contact your Site Director or your Site Faculty for Director or your Site Faculty for Evaluation;Evaluation;

Yours site faculty can be found at: Yours site faculty can be found at: http://www.familymed.ubc.ca/residehttp://www.familymed.ubc.ca/residency/about/facstaff1.htmncy/about/facstaff1.htm

Be Prepared to Explain your Be Prepared to Explain your AssessmentAssessment

Review evaluation forms in advanceReview evaluation forms in advance

RememberRemember

Written comments are a Written comments are a very valuable part of the very valuable part of the evaluation and very helpful evaluation and very helpful to the Programto the Program

Cite specific examples of Cite specific examples of areas of weakness or areas of weakness or concern that need to be concern that need to be addressed. addressed.

Evaluation: Making it Evaluation: Making it WorkWork

G …Get ReadyG …Get Ready Before the rotation starts.Before the rotation starts.

R … Review Expectations and objectives with R … Review Expectations and objectives with Learner at the beginning and throughout the Learner at the beginning and throughout the rotationrotation

A … AssessA … Assess Observe and Record.Observe and Record.

D … Discuss Evaluation at Mid-PointD … Discuss Evaluation at Mid-Point Avoid surprises at the end.Avoid surprises at the end.

E … End with fair and “useful” gradeE … End with fair and “useful” grade Objective and Future-Oriented.Objective and Future-Oriented.

evaluation: making it work borrowed from the “mahec” preceptor program. e-learning_tools.asp m...

Documents