lecture3_testing and evaluation.pdf

7/27/2019 Lecture3_TESTING AND EVALUATION.pdf

1/45

LECTURE 3: TESTING ANDEVALUATION


2/45

PRINCIPLES OF LANGUAGE

TESTING AND EVALUATION A test is used to measure the performance of a

student.

Some tests are given under examinationconditions, with invigilators and special

conditions- formal test. Continuous assessment- often made by

teachers all the time in which they are makingjudgements about the pupils performance .

- listening to pupils- observing pupils working individually, in pairsor in groups

- marking their written work


3/45

Tests serve many of the needs of teaching,

as we teach we need to monitor progress. Testing and teaching are very closely

interrelated but they do not have the samefocus.

Tests focus on assessing the products oflearning and teaching concentrates onenabling students to succeed in the processof learning.


4/45

Tests try to assess a students previous learning. A teacher is to ensure that learning takes place

and tests are one of the tools to help a teacher todo so.

A test is only a sample of what the student issuppose to know as it picks out the mostimportant aspects of the skills that have beentaught.

The performance of the given tests is a way ofmeasuring a students mastery of the languageareas he is tested on.


5/45

A test is often used as a guide on what would

be the most important things to teach and itinfluences what is taught (backwash effect of a

test).

In practice, a test is not always the best guideas it often leaves out important skills because

of practical constraints. For example, the UPSR

& PMR examination do not have the Listening

and Speaking component. Therefore, theteachers do not pay sufficient attention to

these important skills.


6/45

Some teachers use the formats used in testingfor teaching purposes so as to help students topass examination. This is wrong as what istaught should be decided by reference to how

important the skill is to a students present andfuture life.

A test usually seeks the most economical way ofdetermining what the student already knows.

Therefore leaving out many important contentwhich are used for communication and normallanguage used.


7/45

Purpose of

Testing A teacher needs a test in order to:

- put students into groups according to their abilities.

- to find out whether a student has learnt what he hasbeen taught.

- to discover if students seem to be able to cope withthe curriculum and to evaluate the effectiveness of

the methodology

- to compare the performance of one student againstthe performance of their peers


8/45

Types of Test

Tests fall into 2 categories:- Summative Tests

- Formative Tests


9/45

Formative Tests

Formative Tests are used during a

course to monitor the progress of the students. The most common formative tests are

informal. Formal test are also sometimesdesigned.

Type of Formative Test1. Topic/ Progress Tests

- to test how well pupils have learntwhat has been taught

- to test at the end of each topic to be

covered. For example: at the end of eachunit or module to find out if students havemastered what was taught.


10/45

.....Types of Test: Formative

Purpose of Topic/Progress Tests

- To evaluate effectiveness of teaching and teachingmaterial.

- To obtain information about individual pupils level of

mastery of skills taught.

2. Diagnostic Tests

- attempts to diagnose areas of difficulty

- help teachers in deciding for whom and

what areas of remedial work needs to

be given.


11/45

.....Types of Test:

Formative

Purpose of Diagnostic Tests

- to obtain information about pupils

strengths and weakness in a specific area.- This information would be an important

input for decisions on material design,provision of remedial classes, etc.


12/45

.....Types of Test: Summative

Are those that test all that has been taughtin a course.

It usually comes at the end of the course orat the end of each year of the course.

End-of-year examination is a summativetest.

Type of Summative Test

1. Achievement tests- which tests what has been taught

throughout the course/year.


13/45

.....Types of Test: Summative

Purpose of Achievement Test

- to evaluate effectiveness of teaching andteaching materials and methods.

2. Placement Tests

- are used to help group the studentsaccording to overall entry-level proficiency

of students.- to assess all that a student knows at the

point he begins a new course of study


14/45

.Types of Test: Summative

Purpose of Placement Test- to help place pupils into groups

according to ability

3. Proficiency Tests- it is important but not often used in schools.- it is not pegged to any syllabus or course.- it attempts to assess a students future

performance in a specific area of learning orwork.

- it is intended for students from all kind ofbackground


15/45

Types of Tests

Norm-Referenced Tests- if the scores are used to compare a studentsperformance with peers.

- a student is considered average when his

results are compared to results of excellentstudents, and appear to be excellent whencompared to another group that is lessproficient.

- good and poor are relative concepts inthis kind of evaluation.


16/45

Types of Tests

Criterion-Referenced Test

- when a student is required to achieve apre-determined level of proficiency indoing a specific set of tasks, hisperformance is measured not against theperformance of his peers but in terms ofwhether he is able or not to achieve the

stipulated, fixed standard


17/45

Types of Tests

Discrete Point Tests

- relate to how this test is constructed

- it aims at testing one and only one point ofgrammar, phonology, vocabulary, or etc. at a time.

Integrative Tests- seek to test global proficiency

- may isolate discrete features like pronunciation,fluency and so on.

- dictation, cloze and all tests based on

communicative tasks


18/45

Types of Tests

Communicative Tests

- are based on activities which a personwould recognize as relevant to the use ofthe target language

- giving directions in English is a

communicative test ( would have to do

outside a classroom)


19/45

What makes a good test?

Valid

Reliable

Practical


20/45

Criteria of A Good Test

A good test must be valid, reliable andpractical.1. Validity

- a test is said to have validity when it tests whatit claims to be testing.

- is concerned with the issue of whether a testmeasures what the tester wants to know aboutthe performance of the target students.

- there are 5 types of validity:i. Face

ii. Contentiii. Predictiveiv. Concurrentv. Construct

( )


21/45

Validity summarized by Davies (1968)

Type ofValidity

Test

Face Looks like a good one to learner/layman

Content Accurately reflects the syllabus it isbased on

Predictive Accurately predicts futureperformance

Concurre

nt

Gives similar results to already

validated tests or other immediateexternal criteria (eg: teacherssubjective assessment)

Construct Reflects closely a valid theory offoreign language learning that ittakes as its model


22/45

........Criteria of A Good Test

2. Reliability

- is concerned with the precision withwhich the test measures what the

tester wants to know.

- a reliable test must yield the sameresults on the same sample of students

whenever the test is administered and

whoever administers and scores it.

- when several markers are required to mark

candidates performance, reliability is often

low which is caused by inter-variability

i i f d


23/45

..Criteria of A Good Test

....Reliability

- Unreliability is minimized by developingperformance bands which describe asaccurately as possible, the language

behaviour at each performance band.

- unreliability is also sometimesdiscovered in the marks awarded by the

same examiner which is called intra-

rater variability.

- it is often caused by factors such as fatique after longhours of marking, long intervals between periods ofmarking, personality factors such as mood of examinerat different periods in the marking, etc

C it i f G d t


24/45

........Criteria of A Good Test

3. Practicality

- some test may be ideal in theory,become difficult is practice, eg mass oral testing.

- testing oral proficiency is very important at all stages

in the learning process.- it is very expensive to conduct, very time-

consuming and its reliability is usually rather

low because of inter-rater variability.

A ti it


25/45

Activity

Comment on the validity, reliabilty and practicality of the

following tests.1. A group of waiters being assessed while they are working

in a restaurant after they have completed a course on

English for Waiters.

2. A national level essay test.

3. An oral component in monthly tests at school.

4. Reading aloud as a test of oral proficiency

5. National level testing of oral proficiency using languagelaboratories for candidates to record their oral responses

to taped questions


26/45

Test Specifications

What to test?

Why you need to test? How you want to test?

You need to spell out the details of your test:

- what kind of sub-tests and how many- what formats you want to use, etc

- what skills will be tested (eg: reading,

writing/ listening)- what sub-skills will be focused on

T t S ifi ti


27/45

Test Specifications

Section

SkillSub-skills/ Elements

A Reading Guessing meanings of words usingcontextual clues;

Distinguishing main point fromsupporting ideas;

inference

B Grammar Conjunction

Past Tense

Formal and Informal ExpressionsC Vocabulary Words related to animals,

occupations and family

D Writing Descriptions of people


28/45

Writing Test Specifications

Tasks

- need to be carefully chosen to give agood indication of the skills you want

to test.

- For example, to test on speaking

spontaneously, you cannot give a

reading aloud test. You need to

prepare a test : oral interview/

discussion situation- Typical tasks answering comprehension

questions, writing guided compositions, filling

in blanks, etc

W iti T t S ifi ti


29/45


2. Types of Text

- The KBSR and KBSM spell out a number of types oftexts: letters, stories, advertisements, labels,poems,etc.

- In specifying texts, you would need to specify length,eg read a story of 250 words; write a letter of 100

words.- Specify level of difficulty- it is difficult to specify levelof difficulty. One way that testers use is to specify aknown level, for eg the same level of texts as found in

the KBSR school textbook.- Number of texts also need to be specified, forexample, one text of 200 words/ 2 texts, each about300-350 words.


30/45


3. Topics

- select the topics for your test from

syllabus specifications;

= monthly test : achievement/progress

test

- you can also choose any topic within

the maturational level of students. For

example, you can choose a text on

People for a Year 4 students but not on

life insurance

W iti T t S ifi ti


31/45


4. Format

- Same skill can usually be tested in more than one way.- For example: reading proficiency can be tested usingMCQ questions, T/F questions, cloze passage, open-ended

questions, etc.

- The format that you would be using in testing should be

specified.

- The number of items in each sub-test need to be spelt

out.- The format should be familiar to the students. If not, use

and introduce in the teaching first.


32/45


Component Format No. ofItems

Reading Rational Cloze

Open-ended

15 blanks

5questions

Grammar Cloze

ErrorIdentification

10 blanks

15 items

W iti T t S ifi ti


33/45


5. Weightage

- All the skills in a test are equally important, butnot often.

- So, you need to allocate different marks todifferent sections of your test/ different items in

the same section.- Allocate marks according to the importance ofeach section/item in relation to the test as a wholeand what you want to know from the test results..

- The weightage of each item/section need to bespecified.

Writing Test Spe ifi ations


34/45


6. Time Allocation

- How much time will be given for each sub-test willalso need to be specified.

- 2 things to bear in mind:

(i) how important the skill is

(generally, the more important skillshould be given more time);

(ii) how long the task would take (eg: writing

a free composition would take a longertime than a guided composition/ filling in

blanks in already written composition)


35/45

Types of Test Items

1. Close-ended and open ended test items

- it is called according to how much freedom the

testee has in deciding on the scope and quality of

the answer.- a close-ended item has a predetermined answer or

set of answers which are usually short.

- open-ended items- an essay test, an oral interview,a reading or listening comprehension question that

expects students to answer using their own words

f


36/45

Types of Test Items

Close-ended items are of 2 types:

1. objective

2. structured

Objective items- option to choose the correct answer from the

alternatives provided (MCQ)

- pure cloze only one answer is accepted

Structured Items

- the structure of the answer is determined by theexaminer. For example: guided composition

Activity 2


37/45

Activity 2State whether the items on the test are

objective, structured or open-ended.

1. Fill in the blanks in each of the followingsentences.

Ali jumped .......... The river.

2. Write an composition on the topic , My Mother.

3. Combine the following sentences using the word in brackets.

4. Read the following text and answer the comprehensionquestions that follow:

a. The cat was weak and thin. (True or False)

b. Do you think the cat in the story is well-cared for?

Give reasons for your answer.

c. We know the cat was hungry from the fact that it ...

Types of Test Items


38/45

Types of Test Items

Subjective and Objective Tests

- All tests are subjective as it involves sampling.- Sampling means selection

- Selection is usually based on personal preference and

personal judgement.- The method of scoring distinguishes objective from

subjective test.


39/45

Types of Test Items

Subjective Test

- Each examiner uses his own judgments in evaluating

performance and awarding marks.

- Inter-rater as well as intra-rater variability are probable. Objective Tests

- All items have a predetermined answer or set of answers.

- Whoever marks this section, same person or different, theresult will be same.

Types of Test Items Subjective &


40/45

Types of Test Items.... Subjective &

Objective Tests

Objective Test

Strengths- Easy to mark.

- Can be used for mass testing because computermarking is possible.

- High reliability- Item banking for recycling is possible

Weaknesses

- Difficult to construct.

- Low validity

- Sometimes skills and areas tested because theyare testable not because they are important

Types of Test Items Subjective &


41/45

Types of Test Items.... Subjective &

Objective Tests

Subjective Test

Strengths

- Easy to set

- High validity

- Can assess affective and interpretiveaspects of language skills

Weaknesses

- Marking is time-consuming

- Reliability is low


42/45

Please refer to Nesamalar Chitravelus ELTMethodology Principle & Practice

TOPIC : Testing and Evaluation

Page 330 349 for your further

reference and preparation for your

Tutorial & Practical Presentation.


43/45

TUTORIAL

In pairs, analyse test items in sample UPSRexam papers.

Practical


44/45

Practical

Teaching and Assessing Listening and Speaking

Skills

Workshop: Testing and Evaluation Using UPSR exam papers as samples, prepare

test specifications.

Design the following to assess listening and

speaking skills:- Multiple Choice Questions

- Blank Filling

- Text Completion

- Structured Questions

Present the workshop materials on testing andevaluation.


45/45

THANK YOU

lecture3_testing and evaluation.pdf

Documents