WHO?

WHAT?

Analysis

Application

Comprehension

Recall

Evaluation

Synthesis

Bloom’s categorization Associated thinking skills

Knowledge acquisition Memorization

Comprehension Question, discuss, explanation,

Abstraction and transfer

Application Doing a task

Analysis Categorizing, characterization,

comparison, contrast

Synthesis Collating, creating

Evaluation Relevancy, ordering, prioritizing,

judgment

Knows

Shows how

Knows how

Does

Knows Factual tests:MCQ, essay type, oral…..

Knows how(Clinical) Context based tests:

MCQ, essay type, oral…..

Shows howPerformance assessment in vitro:

OSCE, SP-based test…..

DoesPerformance assessment in vivo:Undercover SPs, Video, Logs…..

Knowledgeassessment

Problem-solvingassessment

Clinical skillsassessment

Practiceassessment

Professional or clinical

Authenticity

1960 2000

(adapted from van der Vleuten 2000)

What is a key features problem?

What are the conceptual and measurementissues underlying the design of key featureproblems?

What is the process for writing a key featuresproblem?

A clinical problem, with age & clinical situation specified (e.g., Severe, life-threatening respiratory distress in an infant)

A case scenario typically followed by 2 or 3 questions

Questions assess clinical decisions and actions (not underlying knowledge or reasoning)

Questions are asked only in relation to the most important steps/challenges in the resolution of the problem – i.e., the problem‟s “key features”

Case Specificity

(Arthritis , Anemia, Diabetes) =Key Features (KFs)

Problem solving for

each problem not a general skill

The critical or essential steps in the

resolution of the problem.

Steps, actions most likely to lead to error.

Most difficult aspects of problem

identification and management in practice.

Assess only the key steps/decisions (2- 4), in the resolution of a clinical problem (not reasoning)

…best discriminators

Assess effectiveness, not thoroughness

Permit wide sampling of problems to address “case specificity”

Reliability – “Focused" problems -- better sampling, more accurate assessment

Content Validity – assessing the most important clinical decisions within (a representative sample of) problems

“Bottom-up” thinking – assessing knowledge application and what clinicians do in real life!

What about “fidelity/authenticity”?

Higher “Fidelity” increased discriminating power More effective identification of weaker candidates

1.Select a Clinical Problem

2.Select a Clinical Situation

3.Define the problem's Key Features

4.Select a Case to represent the Problem

5. Develop Questions to test only the key features

6.Select the Format for the questions

7.Prepare Scoring Keys

Only score responses that relate to the Key Features

Each key feature should be scored out of „1‟. Partial scores can be assigned to multiple correct answers (e.g., .5 and .5). Key feature scores within a problem should be averaged to produce a problem score

A key feature score of „0‟ can be assigned if a student chooses too many options or harmful actions (e.g., doing a catheterization when uncalled for)

مه کو ره یافتم بو گنج حسه بی پایان دوست

صد گدای ىمچو خود را بعد از ایه قارون کنم