Teaching technology:Assessment

Dr Naresh T Chauhan

Objective

• To define evaluation and identify the steps of evaluation • Enumerate the aims of evaluation• Enumerate the characteristics of evaluation tools• Understands the role of essay type questions, short answer questions

and multiple choice question as evaluation tools• Formulate the above three evaluation tools

Education system

PROCESS(Training)

(Evaluation)

INPUT(Students)

OUTPUT(Doctors)

FEEDBACK

WHAT IS EDUCATION ?

• Education is process of bringing changes in behavioral pattern of the learner in the desired direction• Such changes can be at level of• Thinking (cognitive)• Feeling (affective)• Acting (psychomotor) can be brought about by

• Specifying educational objectives• Providing suitable learning experiences to the learners• Evaluation of learner

What is evaluation ?

• Evaluation is the process of determining whether predetermined educational objectives has been achieved

Aims

• For selection of a few students from large no of students• Preassessment of need of a learner (pre-test)• For continued monitoring of learning activities for giving a feedback( formative evaluation)For competence to complete a course (summative evaluation)

Characteristics of evaluation

• Relevance:• Validity:• Objectivity:• Feasibility :

Steps of evaluation:Define

objectives

Teaching learning

experience

Select measuring instrument

Administer the test

Decide marking

Score test

Analyze result

Take decision

Types of evaluation:

• Essays• Short answer questions• Objective items• Simulated patient management

problems• Assignments• Practical• Clinical

• OSPE• OSCE• Rating scales• Checklist • Questionnaires

Essay type questions

• “A test item which requires a response composed by the examinee, usually in the form of one or more sentences, of a nature that no single response or pattern of responses can be listed as correct and the accuracy and quality of which can be judged subjectively only by skilled or informed in the subject”

-- John M Stalnaker (1951)

• Permits the examinee to write in his own words a free and extended response to the problem presented and thus reveals information regarding the students mental process• EQ evaluate mainly the cognitive domains• It measures the students ability not only to recall knowledge but also

to use appropriate expression to record it and further his ability to analyze, synthesize and apply the knowledge to solve the problem presented

Advantages EQ

• Free and effective expression• Test their knowledge• Test ability to recognize ideas• Minimize guess work• Easy to frame

Disadvantage

• Lack objectivity• Limited curriculum tested• Grading is inconsistent, difficult and time consuming• No feedback to teachers• Advantage to those with better expression power• Student may interpret it differently from the examiner

Guidelines for framing good EQ

• Question should be clear and specific and exact limit should be set • Avoid what do you think ?

what is in your opinion ?Discuss, Describe ? (Use specific)

• Instead use : enlist, enumerate, define,compare and contrast, state your reasons

• Question should be in language familiar to students• Do not economize on words at the cost of clarity• No option as better comparison but if option are given, they should be

from same topic and of similar difficulty level

Example:-

• A 18 year old boy complains of fever with rigors since 3 days, body ache and headache. Physical examination shows temp. 102 °F. (2+4+6)

a) What is the most likely diagnosis of this patient ?b) Enlist the investigation which should be advised ?c) How will you manage this patient ?

Evaluation of EQ

How to evaluate EQ ?

• Prepare model answers / elements of good answer• Subdivide scores• Scoring of each subdivision before checking second to avoid carry

over effect• Sample a few questions before start marking• Have adequate time and proper place for evaluation• Mark one question in all paper at a time• When grading decision is crucial, have two or more reader score the

essay independently

Modified essay question

• Each candidate is presented with progressive bit of information on clinical problem, in a predetermined sequence. At successive stages candidate responds to information and is asked to make decisions and to explain and /or justify them. Answer are not suggested.• example

Criteria for EQ

• Validity• Reliability• Objectivity• Feasibility• Credibility• Importance

Short answer questions

• Short answer question are a system of evaluation where the response is by writing in one word/ one phrase/ one sentence

• Advantages :• Precisely framed • Covers wide subject area• No guess-like MCQ• Less cumbersome to attempt• Scoring is specific

Disadvantages

• Not suitable for testing complex learning outcome• Time consuming as compared to MCQ• Marking manually only

Types

• Completion type• Unique answer type • Open ended questions

Completion type• Incomplete statement • Missing words• Fill in the blanks• Labelling diagrams• Completing diagrams• Example:

• _________heart of biomedical waste management• Acronym DOTS mean_____• Delivery before _________weeks gestation is called preterm delivery• Ice berg phenomenon diagram ( identify)

Unique answer questions

• Interpretation of data• Simple e.g what is the DOC in Amoebic dysentery ?

Open ended short answer question

• Similar to unique answer question but one or two lined • Solve variation in the nature of answer • What is limiting amino acid ? • What is supplementary action of protein ?• What is placebo ?

Guidelines to frame SAQs

• Determine purpose and content of the item• Write stem• Choose the style• Use simple language• Be concise• Clear cut instruction and value of item- marks

Some clues:

• Word each item in specific terms with clear meanings so that the intended answer is the only one possible• Word each item so that the blank or answer space is towards the end

of the sentence• Avoid copying sentence verbatim from text or classroom materials• Omit important rather than trivial words• Avoid butchered or mutilated sentences

Thank you for your kind attention !

Tell me and I forgetTeach me and I remember

Involve me and I learn