Slide 1

Assessment Tools

Slide 2

Slide 3

Contents Overview Objectives What makes for good assessment? Assessment methods/Tools Conclusions

Slide 4

Educational Triangle Needs / Objectives/Outcomes & Contents Strategies Assessment Curriculum

Slide 5

Assessment Formative v/s Summative

Slide 6

Formative vs Summative It is continues assessmentAt the end of a course/ program Feedback is an essential partUsually NO feedback session Usually NO gradsMarks OR grades are essential There is no pass-fail decisionThere should be a pass-fail decision

Slide 7

Assessment V/S Evaluation

Slide 8

Evaluation: is more focused on making a judgment or determination concerning the quality of a performance, work product or use of skills against a set of standards. Assessment is more focused on measuring a performance, work product, or skill in order to offer feedback to document strengths and growth and to provide directives for improving future performance. Assessments are nonjudgmental and are designed and intended to be helpful to produce improvement.

Slide 9

Miller GE. The assessment of clinical skills/competence/performance. Academic Medicine (Supplement) 1990; 65: S63-S7. Knows Shows how Knows how Does Cognition/ Knowledge Skills & Behaviour Assessment

Slide 10

Miller GE. The assessment of clinical skills/competence/performance. Academic Medicine (Supplement) 1990; 65: S63-S7. Knows Shows how Knows how Does Knowledge Competence Performance Action Assessment

Slide 11

One method cant do it all One measure is no measure We need a mixture of methods to cover the entire pyramid We can choose different assessment tools

Slide 12

Criteria for different assessment methods 1.Validity: 2.Reliability 3.Educational impact 4.Cost effectiveness 5.Acceptability/ Practicability

Slide 13

Criteria / Requirements 1.Validity:Does it measure what it is supposed to measure? 2.Reliability:Does it produce consistent results? 3.Practicability:Is it practical in term of times & resources.

Slide 14

Educational Impact Assessment may have facilitative or adversive effect on learning Emphasizes the important learning outcomes Provide feedback about the success of study methods

Slide 15

What Should Be Tested ? Important topics are weighted heavily The sample of items reflect the instructional goals and the process.

Slide 16

Assessment

Slide 17

Written Exam

Slide 18

Written Written tests can assess knowledge acquisition and reasoning ability, They cannot so easily measure skills

Slide 19

all students from (all classes) are submitted to the same test Progress test

Slide 20

End of course assessment Multiple Choice Open ended questions Combination MCQ/OEQ Modified Essay Questions Computer assessment Assignments Construct a patient information folder Design a Poster Integration essay Prepare a forum discussion Write a research proposal

Slide 21

Clinical 1.Short cases 2.Long cases 3.OSCE 4.Simulated Clinics 5.Video consultation

Slide 22

Clinical (cont) 6. Slides Interpretation 7. Lab Data Interpretation 8. Portfolios 9.SOE (simulated oral exam) 10.Oral

Slide 23

Computerized Case-based Testing Computer is used to increase effectiveness in: Cost Test administration Scoring Authenticity (e.g. by multimedia) Written, practical, clinical,.

Slide 24

Slide 25

Long case" and Short case" The basic requirements for reliability and validity have not always been achieved in traditional long & short cases

Slide 26

Observing candidates' interaction with patients

Slide 27

OSCE Objective structured clinical examination

Slide 28

Slide 29

Observing Candidates' Interaction with Patients

Slide 30

Slide 31

Slide 32

Slide 33

Oral

Slide 34

Slide 35

Work Place-Based Assessment The oral examination is traditionally an unstructured face to face session with the examiners Direct observation: Clinical work sampling measures: Mini-CEX, Mini-Clinical Evaluation Exercise DOPS, Directly Observed Procedural Skills Directly ObservedDirectly Observed P-MEX, The Professionalism Mini-Evaluation Exercise Global performance measures Multi-Source Feedback (MSF or 360) Aggregation and reflection measures Logbook Portfolio

Slide 36

observable behaviour behaviouroutside valuesnormsinsideattitude professional behaviour= Assessment of professional behavior

Slide 37

PEER EVALUATION The oral examination is traditionally an unstructured face to face session with the examiners Peer Assessment & Self Assessment

Slide 38

Regular feedback of performance For all the students Helping students to grow as a professional Starting from the beginning of the studies Assessment of professional behavior

Slide 39

Professional behaviour consists of three aspects Dealing with workDealing with othersDealing with oneself Assessment of professional behavior

Slide 40

Student gathers evidence about aspects of his competence of his competence Criteria to meet the different aspects of competence are described by faculty results of assessment (longtudinal or tailored) reports, video projects peer assessment staff assessment articles, presenting papers Student reflects on the different aspects of competence (written statements) Portfolio

Slide 41

Student proves his competence On several aspects described by faculty Role as expert results of assessment reports projects peer assessment staff assessment Student reflects on the different aspects of competence (written statements) Student discusses his competence and reflection on it with the mentor Mentor = coach Portfolio assessment

Slide 42

Slide 43

1. Contineous Assessment 40% Assignment 2. End of the Course Assessment TOTAL a. MCQ- 10 20% b. Written Short Answer Questions 40% 100%

Slide 44

Conclusion

Slide 45

The oral examination is traditionally an unstructured face to face session with the examiners Comprehensive Assessment Different methods to be applied Standardized / structured Not to depend only on few methods

Slide 46

Knows Shows how Knows how Does Knows Knows how MCQ, SAQ, Short essy, MEQs.Simple recall questions

Slide 47

Knows Shows how Knows how Does Knows howShows how Simulations requiring behavioural performance OSCE, OSPE, Simulated Clinics,

Slide 48

Knows Shows how Knows how Does Shows how Does Assessing performance Work-based assessment: Clinical work-sampling, Mini-CEX, Portfolio, practice visits, case orals.

Slide 49

Knows Shows how Knows how Does Self-, peer, co-assessment, portfolio, multisource feedback, learning process evaluations

Slide 50