Prepared by:

Walaa Ahmed

Eman Shabaan

Eman Antar

Fatma yousef

Mohamed Mansour

Mohamed Abd el aziz

Hesham Mohamed

Amr Omran

Gamal Mohamed

At the end of lecture, we should know the following:

What is evaluation?

Formative and certifying evaluation

Aims of student evaluation

Advantageous & disadvantages of different types of tests.

Qualities of a measuring instrument

How to organize an evaluation system

Introduction

To best study evaluation, we must know the psychology and physiology of evaluation:

The psychology of evaluation

For evaluation activities to be most effective, they should consist of the best possible techniques

Readiness is important for learning.

A student is ready when he understands and accepts the values and objectives involved.

The type of evaluation device used determines to great extent, the type of learning activity in which students will engage in class room

Individuals l earn better if they are constantly appraised in a meaningful manner as to how they are doing

Motivation is the most important and difficult to handle sometimes. If student is motivated =greater performance than if he lacks motivation

Learning is most efficient when there is activity on the part of learner.

Philosophy of evaluation:

1. Each individual should receive the education that most fully allows for his potential development

2. Each individual should be placed that he contributes to society and he receives personal satisfaction in doing so.

3. Recognize individual's individuality for his fully development.

4. The judgments required in assessing an individual's personality are complex in composition difficult to make and full of errors

5. Such errors can't be fully eliminated so evaluation can never be final.

6. Every form of appraisal will have criticism which is a spur to change and improvement

Definition of evaluation : A continuous process based upon criteria cooperatively developed concerned with measurement of performance of learners, the effectiveness of teachers and quality of the programme.

There is two types

1-Formative evaluation or diagnostic evaluation.

2-Certifying evaluation or summative evaluation.

Continuous evaluation ;formative & certifying evaluation

Formative evaluation :

Designed to inform the student about the amount he still has to learn before achieving his educational objectives.

Measures the progress made by students from the moment he begins the programme until the time he has completed it

Enables learning objectives to be adjusted in accordance with progress made or lack of it.

Very useful in guiding students and prompting him to ask for help.

Results should not appear in any official report

It is carried out frequently as often as student feel necessary

It is not to be used by the teacher in anyway to make certifying judgment. A coding system makes it possible to follow the progress of individuals and groups while preserving anonymty.

Provide teacher with qualitative and quantitative data for modification of his teaching

Certifying evaluation :

It is traditionally used for placing students in order of merit and justifying decisions as to whether they should be promoted to the next class or be rewarded a diploma

Provide a mean to protect society by preventing incompetent students from participating

It is cumulative, less frequently used than formative evaluation but be used at least once at the end of course

Elements needed for the construction of an evaluation system:1. Planning the evaluationof situation analysis and the identification of priority health problems(context);

concerned with the initial decisions of importance for the educational programme.

it is linked to situation analysis where all information of importance for the programme is available.

if the information available is not satisfactory,it may be necessary to collect further information in order to arrive at the right educational decisions.

This may include analysis of factors in learners potential job environment ,selection of various job descriptions and employees opinions on performance of earlier student in their jobs.

2. Planning the evaluation of the human and material resource to be used and elements to be included in the programme (the inputs):

At all stages of the learning process there are educational decisions to be taken by teachers.

It is therefore important to make sure that teachers are competent and comfortable with the teaching methodology to be used (problem-based education).

3. Planning the monitoring of implementation (the educational process)

An evaluation system must also plan how the implementation of the programme is to be monitored. this should detect the need for modification or replacement of any of the teaching /learning activities in the programme.

4. Planning the evaluation of learners (the output)

At this stage of planning , decisions must be made on the:

i. establishment of an evaluation committee

ii. identification of persons to prepare instruments of evaluation

iii. the various administrative arrangements to be made for the evaluation of the learners performance.

Aims of student evaluation Incentive to learn (motivation).

Feedback to student / teacher

Modification of learning activities.

Selection of students

Success or failure.

School public relation.

Protection of society(certification of competence)---by appropriate measuring techniques.

4 steps in student evaluation:

1. Once you are satisfied with the quality of criteria of the educational objectives.

2. Develop and use measurement instrument

3. Interpret measurement data

4. Formulate judgments and formulate corrective action

Common methodology for student evaluation:

Evaluation of practical skills.

Evaluation of communication skills.

Evaluation of practical skills.

1)Make a list of observable types of behavior showing that the objectives pursued has been reached.

2) Make a list of observable types of behavior showing that the objectives pursued hasnt been reached.

3)Determine the essential features of behavior in both lists.

4)assign a positive or negative weight to the items in both lists.

5)Decide on the acceptable performance score

For the last 3 stages obtain the agreement of several experts

Example reassure the mother of a child admitted to hospitalattitude-2-1012

Explain clearly what has been done to the childOften uses medical terms and never explains what they meanOften uses medical terms and rarely explains what they meanRarely uses medical terms and sometimes explain what they meanRarely uses medical terms and always explain what they meanUses only terms suited to mother vocabulary

Evaluation methodology according to domains to be evaluated

In real situations

In simulated conditions

Qualities of a test:

1. Directly related to educational objectives

2. Realistic and practical

3. Concerned with important and useful matters

4. Comprehensive but clear

5. Precise and clear

Consideration of the type of competence a test purports to measure

The form of the question doesn't necessarily determine the nature of the intellectual process required to answer it.

Question difficult and complexity of instructions aren't necessarily related to the nature of intellectual process being tested

A test that purports to measure student's clinical judgment and his ability to solve clinical problems must stimulate reality as closely as possible by presenting him with specific constellations of data that are unique and new to him as well.

Common defects of examination Triviality

triviality of the question asked. It is essential for each question to be important and useful as in examination questions only represent a small sample of all those to be asked

Outright error

outright errors in phrasing the question or in case of MCQ in phrasing the directors and the correct response

Ambiguity

Ambiguity in the use of language leads the student to spend more time in trying to understand the question than in answering it, may even lead to irrelevant answer

Obsolescence

Forcing the student to answer in terms of outmoded ideas of the examiner

Bias

Requesting the student to answer in terms of the personal preference of the examiner when several equally correct options are available

Complexity

lead to difficult search for answers

Unintended cues;

In the formulation of the questions that make the correct answer obvious

outside factors to be avoided

a. Complicated instructions( ability to understand instructions )

b. Over-elaborate style( ability to avoid traps)

c. Trap questions ( ability to use words )

d. Test-wise ( ability to predict questions )

Comparison of advantageous & disadvantages of different types of test

1. oral examinationsadvantages disadvantages

provide direct personal contact with candidates provide opportunity to take mitigating factors into account provide flexibility in moving from candidate's strong points to weak areas requires candidate to formulate his own replies without cues opportunity to question candidate on how he arrived at answer for simultaneous assessment by 2examiners lack standardization lack objectivity and reproducibility of results permits favouritism and possible abuse of physical contact suffer from undue influence of irrelevant factors

suffer from shortage of trained examiners to administer test costly in terms of professional time in relation to limited value of information

2. Practical examinations, projects

advantagesDisadvantages

Opportunity to test skills involving all senses while the examiner observes and checks performance Opportunity to confront the candidate with problems he has not met before, to test his investigative ability against his ability to apply ready made recipes opportunity to observe and test attitudes and responsiveness to complex situations. opportunity to test the ability to communicate under pressure, to discri minate between important and trivial issues,to arrange the data in final form Lack standardized conditions in laboratory experiments using animals Lack objectivity and suffer from intrusion or irrelevant factors are of limited feasibility for large groups Entail difficulties in arranging for examiners to observe candidates demonstrating the skills to be tested

3. Essay examinations

AdvantagesDisadvantages

Provide candidate with opportunity to demonstrate his knowledge & his ability to organize ideas & express them effectively

Limit severely the area of the student's total work that can be sampled Lack objectivity provide little useful feedback Take a long time to score

4. Multiple choice questionsAdvantagesDisadvantages

Ensure objectivity ,reliability and validity; preparation of questions with colleagues provides constructive criticism Increase the range and variety of facts that can be sampled in a given time Are easy and rapid to score Provide precise , unambiguous measurement of higher intellectual processes

Provide detailed feedback for both student and teachers

Take a long time to construct to avoid arbitrary and ambiguous questions Require careful preparation to avoid preponderance of questions testing only recall Provide cues that dont exist in practice Are costly when number of students are small

Evaluation in education

Definitions Education: is a process developed for bringing about changes in the student's behaviour. At the end of a given learning period desirable behaviour should appear and undesirable should disappear The educational objectives are the desired types of behaviour. The teacher should provide good environment for the student's acquisition of them Evaluation in education is a systematic process which enables to measure the extent to which the student has attained the educational objectives Measuring instruments must be available to make measurements In education, Measuring instruments are generally referred to as tests Qualities of a measuring instruments:

4 Essential qualities are essential among the qualities of a test: validity, reliability, objectivity and practicability Others are also essential ,but they contribute in some degree to the qualities of validity and reliability Validity:

The degree to which the instrument /test really measures what it is intended to measure No outside factors should be allowed to interfere with the manner in which the evaluation is carried out The notion of validity is a very relative one .It implies a concept of degree, I.e.,one may speak of very valid, moderately valid or not very valid results

The concept of validity is always specific for. a particular subject content validity is determined by the following question: will the test measure or has it measured the matter and behaviour that is intended to measure? predictive validity is determined when the results of a test are to be used for predicting the performance of a student in another domain /situation Reliability:

Reliability is the consistency with which an instrument measures a given variable. Reliability is always connected with a particular type of consistency: the consistency of the results in time; consistency of results according to questions; consistency of the results according to examiner Reliability is necessary but not a sufficient condition for validity. In other words valid results are reliable, but reliable results are not necessarily valid.

Reliability can also be defined as the degree of confidence that can be placed in the results of an examination. It is the consistency with which a test gives the results expected. Objectivity:

The extent to which several independent and competent examiners agree on what constitutes an acceptable level of performance Practicability: The overall simplicity of use of a test, both forty test constructor and for students.

Other qualities of a measuring instrument:

Relevance; this is the degree to which the criteria established for selecting questions (items )so that they conform to the aims of the measuring instrument are respected This notion is almost identical to the one of content validity; and the 2qualities are established in a similar manner

Equilibrium; achievement of the correct proportions allocated to each of the objectives Equity; extent to which the question set in the examination correspond to the teaching content Specify;quality of a measuring instrument whereby an intelligent student who hasn't followed the teaching on the basis of which the instrument has been constructed will get a result equivalent to that expected by pure chance Discrimination; quality of each element advantages a measuring instrument which makes it possible to distinguish between good and poor students in relation to given variable Efficiency; quality of a measuring instrument which insures the greatest possible number of independent answers per unit of time Time; When. the time allowed is too short, the measuring instrument will be less reliable leading to the introduction of non relevant factors (guessing, taking risks or chances, etc.) Length; the reliability of a measuring instrument can be increased almost indefinitely by the addition of new questions equivalent to those constituting the original instrument Relationship between characteristics of an examination Validity &Reliability To-be valid a measuring instrument /test must be both relevant and reliable a test can often, if not always, be more valid if it's reliability is increased Validity & Relevance By definition, a question is more relevant if it adds to the validity of the instrument, and an instrument is valid if it respects the specifications established during its preparation. Relevance &Equilibrium The concept of equilibrium is only a sub-category of the concept of relevance Relevance & Equity If the instrument is constructed on the basis of a content itself then it will by relevant by definition.=equitable If this is not done, then the instrument will not be relevant =non-equitable, not valid However, an examination can be equitable without being relevant /valid if the instrument not adequately derived from objectives Equity, Specificity and reliability A test can't be equitable If not firstly specific. Specificity just like equity will affect the reliability of results Reliability, Discrimination, length, homogeneity (of questions) and heterogeneity (of students) Reliability is influenced by the extent to which the questions (items) clearly distinguish competent from incompetent students, the number of items, the similarity of items as regards their power to measure a given skill and the extent to which students are dissimilar with respect to that skillRelationship between characteristics of an examination

Validity

Reliability Relevance

Objectivity DiscriminationEquilibrium

Equity

DifficulityHomogenity

(exam)

Specifity

EfficiencyHeterogenity

(students)

Duration Length

(time)(number of questions)

Organizational diagram showing relationships between cirriculum,committee,evaluation committee and teaching units

Each unit prepares the specific objectives and test items

Referance

Gilbert's educational hand book for health personnel 6th edition

Domain

Intellectual skills

Communication skills

Practical skills

Indirect methods

Direct observation

Practical tests

Carrying out of projects

Written tests

-questionnaire

-objective type-essays

-simulation

Oral tests

Practical tests

-research reports

-bibliography,etc

Evaluation committee prepares comprehensive test

The second year coordinator transmits the intermediate objectives

The first year coordinator transmits the intermediate objectives

The MH year coordinator transmits the intermediate objectives

Curriculum committee prepares specification tables

School administration transmits institutional objectives

Educational consultant