thesis of ma linguistics

8/2/2019 Thesis of MA Linguistics

1/60

TESTING ENGLISH GRAMMAR AT

INTERMEDIATE LEVEL AND ITS EFFECT

ON TEACHING AND LEARNING.

By:

PIR SUHAIL AHMED SARHANDI

THESIS

Presented to the department of English university of Karachi

In partial fulfillment of the requirement

For the degree of Masters in Linguistics

UNIVERSITY OF KARACHI

JULY 2005

This document was created by the trial version of Print2PDF. !

Once Print2PDF is registered, this message will disappear.!

Purchase Print2PDF at http://www.software602.com
http://www.software602.com/http://www.software602.com/


2/60

ABSTRACT

This research study is conducted in order to know how testing of English grammar at

Intermediate level effects the teaching and learning process. After discussing the test and

its types thoroughly in section two, I think I have provided the readers a well

understanding of testing and our testing system especially grammar testing at

Intermediate level. The question grammatical items in papers (I & II) of English

compulsory at Intermediate level were evaluated and analyzed with their impact on

learning and teaching. The results show that the testing system of grammar is not so

effective because they are devised for testing not for learning that lacks their validity.

And at last I have proposed sample items that are devised by me on the communicative

approach of testing grammar. The sample test items, if introduced, in the testing system

of grammar at Intermediate level would help to achieve the beneficial backwash effect on

teaching and learning.



http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/


3/60

AKNOWLEDGEMENT

I would like to thank Mr. Muhammad Ali Khan, my research supervisor, to the

core of my heart who has supported me and encouraged me throughout my research

work. Because of his kind supervision and beneficial suggestions, I became able to

complete my thesis.

I am also thankful to my grammar teachers: Ms.Dilshat Bano, Mr.Ghulam

Hussain Manganhar and Mr.Ahsan Ahmed Alavi, who taught me grammar in such a way

that paved me the way to work on my research on grammar testing.

I am also grateful to my teacher as well as a sincere friend Mr.Zubair Ahmed

Mangrio for his well wishes and precious suggestions throughout my linguistics

program.

At last but not least, I am extremely grateful and obliged to my family members

especially to my father, Pir Khalil Ahmed Sarhandi who always encourages me

throughout my education.



http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/


4/60

CONTENTS

ABSTRACT

ACKNOWLEDGMENTS

Page No.

SECTION ONE: Introduction to Research 1

1.1 Introduction to My Research 1

1.2 Proposed Research Methodology 3

1.3 My Research Methodology 4

1.4 Purpose of My research 4

1.5 Literature Review 5

SECTION TWO: Introduction to Testing 8

2.1 An overview of Testing 8

2.2 What is a good Test? 8





5/60

2.3 Why do we need to Test? 9

2.4 Why do we need to Test Grammar? 11

2.5 Approaches to Language Testing 11

2.6 Types and purposes of Tests 14

2.7 Test Classification 17

(a) Kinds of Testing 19

2.8 Subjective and Objective Testing 22

2.9 Criteria for the evaluation of a Test 23

SECTION THREE: Backwash effect 27

3.1 Backwash Effect on Teaching and Learning 27

3.2 Harmful backwash effect 28

SECTION FOUR: Evaluation of English Compulsory Papers (I & II)

and some suggestions. 29

4.1 Evaluation of Grammatical items. 29

4.2 Advantages of Proposed Grammatical items 38

4.3 Conclusion 39



http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/


6/60

REFERENCES 40

BIBLIOGRAPHY 41

APPENDICES



http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/


7/60

SECTION ONE: Introduction to Research

1.1 Introduction to my Research

The thing that encouraged me to work on the Testing Grammar at Intermediate

Level was the inability of our students who after being in touch with English since sixth

standard or even theres a good number of those learners who have been taught English

since their Montessori class. The reason to work on only this topic was to know what is

lacking that our students are still unable to communicate in English. After pondering over

and discussing this problem with my friends and teachers I came to know that our testing

system is no so effective to produce the communicative competence in the learners.

According to Arthur Hughes:

Too often language tests have harmful effect on teaching and learning; and

too often they fail to measure accurately whatever it is they are intended to measure.

(2003:1)

One of the major problems, faced in English language teaching and learning is

the inappropriate administered tests at the end of the course. In our education

system, we have not been taught the language for the sake learning but for

testing.



http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/


8/60

We are now in a communicative era, a time when we emphasize evaluation of

language use rather than language form. I have tried my level best to expose those

reasons that are in the way to communicative learning.

The reason to choose the topic on grammar testing is that in our society there is a

n impression that grammar has no concern in communicative language so it should not be

taught and tested. This idea has overwhelmed our society. I think the major reason behind

this idea is ineffective teaching and learning system in our society.

.testing has been one of the greatest single beneficial forces in changing the

direction of the greatest single beneficial forces in changing the direction of language in

many areas and in encouraging the more responsive teachers to examine not only their

own teaching methods but also the language they are teaching. Heaton (1988: 171)

The above words of Heaton inspired me to work on a research that deals with

testing in order to bring a change in the way grammar items are taught and learnt.

Teachers dont teach the grammatical items according to their function and use in real life

communication so they dont test these items accordingly.





9/60

Widdowson describes the importance of grammar in these words:

A communicative approach, properly conceived, doesnt involve a recognition

of its central mediating role in the use and learning of language. (1990:88)

Little-Wood (1986) while stressing the importance of grammar writes:

Communicative language use is only possible by virtue of the grammatical

system and its creative potential. ( Little Wood 1986 quoted in Lock, 1996:66)

1.2 Proposed Research Methodology

Methodology is a general approach studying a research topic. It establishes how

one will go about studying any phenomenon. In social research, examples of

methodologies are positivism (which seeks to discover laws using quantitative methods)

and, of course, qualitative methodology (which is often concerned with including

hypotheses from field research). Like theories, methodologies cannot be true or false,

only more or less useful. (Interpreting Qualitative Data by David Silverman 1993)



http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/


10/60

Definition

Qualitative is the collection and analysis of extensive data in order to gain insightinto a situation of interest. It involves intensive data collection that is the collection of

extensive data on many variables over an extended period of time in a naturalistic setting.

It is much broader in scope.

Qualitative research assumes that all knowledge is relative, that there is a

subjective element to all knowledge and research, and that holistic ungeneralisable

studies are justifiable (an ungeneralisable study is one in which the insights and outcomes

generated by the research cannot be applied to contexts or situations beyond those in

which the data were collected. Reinhardt and Cook (1979)



http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/


11/60

1.3 My Research Methodology

My methodology for conducting this research is Qualitative because I think thebest to do the research on my topic Testing English Grammar at Intermediate Level and

its impact on Teaching and Learning is to evaluate and observe the data extensively in

informal way.

For this research study I have selected the Intermediate level in which I will

analyze and evaluate the papers (I & II) and the impact of testing grammatical items onteaching and learning. The purpose of choosing the Intermediate level for my research

work is that, in my opinion, these two years of study (first year and Inter) are the

foundation stones for a learners coming up professional life. As English is the utmost

need for a bright future in our country and the major problem in learning this language is

mostly caused by grammar so I have decided to evaluate the grammatical items and try to

give some effective suggestions to improve testing system of grammar which would

automatically change the teaching system as well.

For this study only those grammatical items are chosen which are found

constantly in the Intermediate papers of English for some years. These items include:

articles, preposition, English tense system, idioms, phrases and translation.



http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/


12/60

1.4 Purpose of my Research:

The main purpose of my research work is to discuss the possible implications

testing grammar may have for the teaching and learning of English grammar at

Intermediate Level. In Intermediate examinations, grammar is tested in isolated

sentences, which are not connected to each other. They are mechanical in a sense that the

learners can do them without really understanding the meaning of the sentences.

J.B.Heaton defines the purpose of tests of grammar and usage that

these tests measure students ability to recognize appropriate grammatical

forms and to manipulate structures. (1988:9)

The testers of grammar at Intermediate level here are quite contrary to the

purpose of these tests defined above. It is my fundamental aim to analyze the

Intermediate grammar papers in the light of the above definition and derive the attention

of teachers to the usage of grammar in English language.

My research study would help the testers to set the effective grammatical items

so that the testees focus would remain on the practical usage of grammar.



http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/


13/60

1.5 Literature Review

Testing has been a burning question since long different writers have different

opinions about its purpose.

John Bright Heaton defines the purpose of testing as

devices to reinforce learning and to motivate students or primarily as a means

of assessing. (1988:7)

In his book Writing English Language Tests (1988), J.B.Heaton has given a

brief introduction to testing. He discusses almost all the language skills to be tested in

English language. The book analyses the methods and techniques used in assessing the

English tests. Criteria and types are also given importance in the book. Heaton has

also provided proposals for the better testing in all skills.

Harold.S.Madsen in his book Techniques in Testing has noticed the other

purpose of testing and says:

Good tests can sustain or enhance class morale and aid learning. (1983:4)

In his book he gives a thorough introduction to testing and its types very clearly

explained with numerous examples. The book contains a good number of activities and

techniques that help the teachers to focus on teaching and evaluating in the class.



http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/


14/60

The newly re-edited book Testing for Language Teachers by Arthur Hughes

(2003) may be said to be the most current publication on Testing.

In his book Hughes analyses and differentiates what makes a good test. This

book considers the different purposes of testing, discusses how testing encourages

good and effective teaching and learning.

In my opinion, testing cannot be fully assessed without focusing on the teaching

process. Both, testing and teaching are interrelated fields, which cannot be separated

from each other.

Heaton defines the purpose of tests of grammar and usage that

these tests measure students ability to recognize appropriate grammatical

forms and manipulate structures. (1988:9)

Properly made English tests can help create positive attitude toward instruction

by giving students a sense of accomplishment and a feeling that the teachers evaluation

of them matches what s/he has taught them. Heaton (1988:4)

In the light of the above statements I would analyze and observe the impact of

Testing English grammar at Intermediate level on teaching and learning.



http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/


15/60

SECTION TWO: AN OVERVIEW OF TESTING

2.1 What is Testing?

Testing is not an easily definable term. Since long, it has been a burning

question if Teaching and Testing are two different fields. Its obvious that we cannot

separate these terms from each other. Both are interrelated fields for without focusing on

teaching process or methods we cannot think of proper or better assessment.

Testing is a multifaceted process so one cannot stick with its only one

definition. WE have to go through many definitions given by different scholars in order

to grasp this term. Carroll (1968) defined the term test, cited in Bachman (1990:20) as;

A psychologist or educational test is a procedure designed to elicit certain

behavior from which one can make inferences about certain characteristics of an

individual. (Carroll 1968:46)

2.2 What is a good Test?

Hughes has given the characteristics of a good test, which facilitates our

understanding of the nature of the test. They can be enumerated as followed:





16/60

1. It should attempt to measure students actual ability to communicate in language,

pointing out that a good test is never a mechanical test only.

2. It should have a beneficial influence on syllabuses and teaching strategies. In other

words, the test should enable the teacher to locate which part of the course has been

found difficult and problematic by the class. In this way the teacher can evaluate the

effectiveness of the course as well as his teaching strategies. (Arthur Hughes 2003: 8)

Harold.S.Madsen says:

Good tests can sustain or enhance class morale and aid learning. (1983:04)

Considering the above points, I will measure the efficacy of Intermediate Test

papers. There is no denying the fact that teaching and testing are related and this is best

explained by the term BACKWASH EFFECT. This term is further defined in details in

Section Three. Things start becoming clear when we identify different purposes and types

of tests.

2.3 Why do we need to test?

Testing as already discussed is a great source of teaching and learning especially

because of its backwash effect. It is a tool which encourages the teachers and up to date

them with their teaching process. For students it acts as an achievement of their struggle

while learning.





17/60

According to Madsen:

Properly made tests can help create positive attitude towards instructions by

giving students a sense of accomplishment and a feeling that the teachers evaluation of

them matches what he has taught them. Good English tests also can help the students

learn the language by requiring them to study hard, emphasizing course objectives and

showing them where they need to improve. (1983:5)

John Bright Heaton defines the purposes of testing as:

devices to reinforce learning and to motivate students or primarily as a means

of assessing students performance in the language. (1988: 06)

It is essential to give a list of the aims and purpose of the tests before conducting the test.

Language testing both serves and is served by research in language

acquisition and in language teaching. Language tests, for example, are frequently

used as criterion measures of language abilities in second language acquisition

research. Similarly language tests can be valuable sources of information about the

effectiveness of learning and teaching.

Bachman (1990: 2- 3)





18/60

According to Hughes:

The first thing that testers have to be clear about is the purpose of testing in

any particular situation. Different purposes will usually require different kinds of tests.

This may seem obvious but it is something which seems not always be recognized.

(Hughes 2003:7)



http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/


19/60

2.4 Why do we need to test grammar?

These tests measure students ability to recognize appropriate grammaticalforms and to manipulate structures: says Heaton (1988:9)

Arther Hughes (1989) while discussing the testing of grammatical items he says

that if it is language skill which is to be tested then they should be tested directly instead

of the abilities that seem to underlie them. Firstly because by underlying skills one cannot

accurately predict the mastery of a skill and secondly, the backwash effect of directtesting would be preferable to that of indirect testing.

As it is essential to have a good command of particular grammatical structures to

communicate, thats why its also necessary top test these grammatical items.

2.5 Approaches to Language Testing

Language tests may be classified according to four main approaches to testing:

a. The essay-translation approach;

b. The structuralist approach;

c. The integrative approach;

d. The communicative approach.





20/60

A useful test will generally have features of several of these approaches. Actually,

a test may have certain innate weaknesses simply because it is limited to one approach,

however attractive that approach may appear.

a) The Essay-translation approach:

This approach is commonly referred to as the pre-scientific stage of language

testing. No special skill in testing is required. Tests usually consist of essay writing,

translation, and grammatical analysis.

b) The Structuralist approach:

This approach is based on the view that language learning is chiefly concerned

with the systematic acquisition of a set of habits. All the four skills are separated from

one another as much possible because it is considered essential to test one thing at a time.

c) The Integrative approach:

This approach involves the testing of language in context and is thus concerned

primarily with meaning and the total communicative effect of discourse. This approach of

testing tests the language skills integratedly. The language is tested in context, chiefly





21/60

concerned. Integrative tests do not seek to separate language skills into neat divisions in

order to improve test reliability. Oral interviews, translation and essay writing are also

included in many integrative tests.

d) The Communicative approach:

According to Heaton

This approach is sometimes linked to the integrative approach. However, both

approaches emphasise the importance of the meaning of the utterances rather than their

form and structure, there are nevertheless fundamental differences between the two

approaches. Communicative tests are concerned primarily (if not totally) with how

language is used in communication. (1988:19)

This approach focuses on the function and the meaning of the utterances rather

than their form and function.

B.J.Caroll says:

The ultimate criterion of language mastery is therefore the teachers

effectiveness in communication for the setting he finds himself in.

Defining in a single sentence I say: Communicative approach is mainly

concerned with how language is used in real life communication.





22/60

2.6 Kinds of tests and their purposes.

Lyle F. Bachman has suggested five features to classify tests

.language tests can be classified according to five distinctive features: the

purpose, or use, for which they are intended; the content upon which they are based; the

frame of reference within which their result are to interpreted; the way in which they are

scored, and the specific technique or method they employ.

Bachman (1990:70)

There is some confusion regarding the terminology used to denote the different

types of language tests in use. Most test specialists, however, agree on the following

types of test divisions:

a) Achievement/ attainment tests,

b) Proficiency tests,

c) Aptitude tests, and

d) Diagnostic tests.



http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/


23/60

a) Achievement tests:

This group can be further subdivided into progress achievement tests and

(standardized) achievement tests.


Progress achievement tests, as their name suggests, are intended to measure

the progress that students are making. (2003:14)

(Standardised) Final achievement test are administered at the end of the course

indicating the extent to which an individual has mastered in the specific skills.

Achievement tests are directly related to language courses, their purpose being

to establish how successful individual, groups of students or the courses themselves have

been in achieving objectives.

Hughes (2003:13)





24/60

b) Proficiency tests:

Madsen defines it as:

Proficiency tests can measure overall mastery of English or how well prepared

one is to use English in a particular setting. (1983:9)

The proficiency test is concerned simply with measuring a students control of the

language in the light of what s/he will be expected to do with it in future performance of a

particular task.

According to Arthur Hughes:

Proficiency tests are designed to measure peoples ability in a language, regardless of any

training they may have had in that language. (2003:11)

c) Aptitude tests:

Heaton defines it as:

A language aptitude test is designed to measure the students probable

performance in a foreign language, which s/he has not started to learn: i.e. it assesses

aptitude for learning a language. (1988:173)

Aptitude tests try to tell us how well a person will do in learning a foreign

language: says Madsen (1983:189)



http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/


25/60

d) Diagnostic tests:

The teachers in the classroom conduct these tests in order to know what is stilllacking in teaching and learning.

Diagnostic tests are used to identify learners strengths and weaknesses. They

are intended primarily to ascertain what learning still needs to take place . Hughes

(2003:15)

2.7 Test Classification:

Here are some of the ways tests can be classified given by Harold.S.Madsen in his

book Techniques in Testing.

Table 1. CONTRASTING CATEGORIES OF ESL TESTS

Knowledge Tests Performance (or Sills) Tests

Subjective TestsObjective Tests

Productive TestsReceptive Tests

Language Subskill TestsCommunication Skills Tests

Norm-referenced Tests.Criterion-referenced Tests

Discrete-point Tests. Integrative Tests





26/60

Proficiency Tests..Achievement Tests

Review of the contrasts shown in Table 1.

1. Tests of Knowledge are used in various school subjects, from math and

geography to literature and language. While ESL knowledge exams show how well

students know facts about the language, ESL performance exams show how well a

student can use language.

2. Subjective tests, like translation and essay, have the advantage of measuring

language skills naturally, almost the way English is used in real life. By contrast

Objective tests such as multiple-choice or matching tests , can be scored quickly and

consistently.

3. Productive test measures, like speaking exams, requires active or creative

answers, while receptive testmeasures, like multiple-choice reading tests, tends to rely on

recognition, with students simply choosing the best answer.

4. Tests of language subskills measure the separate components of English, such as;

vocabulary, grammar, and pronunciation. Communicative skills tests, on the other hand,

show how well the language in actually exchanging ideas and information.5. Norm-referenced tests compare each student with his/her classmates but criterion-

referenced exams rate students against certain standards, regardless of how other students

do.

6. In discrete-point exams, each item tests something very specific as a preposition

or a vocabulary item. Integrative tests are those like dictation that combine various

language subskills much the way we do when we communicate in real life.

7. The final classification isproficiency and achievement tests. Proficiency tests can

measure overall mastery of English or how well prepared one is to use English in a

particular setting, such as; an auto mechanics course or a university. Achievement tests,

on the other hand, simply measure progress-gains for example, in mastery of count-

noncount noun, use or mastery of the skills presented in an entire language text or course.



http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/


27/60

It is obvious from the above discussion that several labels can be applied to any one test.

But more often, we apply only one pair of labels at a time.

(The above words under the title Test Classification are the abstract of the

topic Test Classification from the book Techniques in Testing 2003: 8 by Harold .S.

Madsen)

2.7 (a) Kinds of Testing.

(i) Direct versus Indirect Testing.

According to Hughes,

Testing is said to be direct when it requires the candidates to perform precisely

the skill that we wish to measure. If we want to know how well candidates can write

compositions, we get them to write compositions. If we want to know how well they

pronounce a language, we get them to speak. The tasks and the texts that are used should

be as authentic as possible.

Direct testing is easier to carry out when it is intended to measure theproductive skills of speaking and writing.

Direct testing has a number of attractions. First provided that we are clear about

just what abilities we want to assess, it is relatively straightforward to create the





28/60

conditions, which will elicit the behavior on which to base, our judgments. Secondly, at

least in the case of the productive skills, the assessment and interpretation of students

performance is also quite straightforward. Thirdly, since practice for the test involves

practice of the skills that we wish to foster, there is likely to be a helpful backwash

effect.

Hughes (2003:17-18)

Hughes also goes on defining indirect testing,

Indirect testing attempts to measure the abilities that underlie the skills in

which we are interested.

The main problem with indirect tests is that the relationship between

performance on them and performance of the skills in which we are usually more

interested tends to be rather weak in strength and uncertain in nature. Hughes (2003:18)

(ii) Discrete Point versus Integrative Testing.

Definition given by Hughes would best define it, before him :

Discrete point testing refers to the testing of: one element at a time; item by

item. This might, for example, take the form of a series of items, each testing a particular

grammatical structure. Integrative testing, by contrast, requires the candidates to combine

many language elements in a completion of a task. This might involve writing a

composition, making notes while listening to a lecture, taking a dictation, or completing a

close passage. Discrete point tests will almost always be indirect, while integrative tests





29/60

will tend to be direct. However, some integrative testing methods, such as the cloze

procedure, are indirect.

Hughes (2003:19)

(iii) Norm-referenced versus criterion-referenced testing.

According to Bachman (1990).

Norm-referenced (NR) tests are designed to enable the test user to make

normative interpretations of test results. That is, test results are interpreted with

reference to the performance of a given group, or norm. The norm group is typically a

large group of individuals who are similar to the individuals for whom the test is

designed. Bachman (1990:72)

Criterion-referenced (CR) tests are designed to enable the test user to interpret

a test score with reference to a criterion level of ability or domain of content. An example

would be the case in which students are evaluated in terms of their relative degree of

mastery of course content, rather than with respect to their relative ranking in the class.

Bachman (1990:74)





30/60

(iv) Objective testing versus subjective testing.

It would be better to define this topic according to Hughes, he writes:

The distinction here is between methods of scoring, and nothing

else. If no judgment is required on the part of the scorer, then the scoring

is objective. A multiple-choice test, with the correct responses

unambiguously identified, would be a case in point. If judgment is called

for, the scoring is said to be subjective. Hughes (2003:22)

2.8 Subjective and Objective Testing:

Subjective and objective are the terms used to refer to the scoring of tests.

Students are always required to exercise a subjective judgment while attempting a test

paper irrespective of how the tests are devised. In an essay test, a candidate is to think

what to say and in which manner s/he expresses the ideas; in a multiple-choice test, a

candidate has to be rationale in selecting the proper answer from the various alternatives

given.

Hughes defines:

The distinction here is between methods of scoring, and nothing else. If no

judgment on the part of the scorer, then the scoring is objective. A multiple-choice test,

with the correct responses unambiguously identified, would be a case in point. If

judgment is called for, the scoring is said to subjective. (2003:22)





31/60

It is only the scoring of the test that can be described as objective because all the

tests are devised subjectively by the test developer, who decides which areas of language

to test and how to test those particular areas.

Unfortunately, in our education system the students are taught by a person who

follows the syllabus and an unknown person, who is not in touch with the teaching and

learning process throughout the course, is the test developer and the third one is required

to check it. How can it be expected to achieve the beneficial backwash effect on teaching

and learning in such a situation? This issue will be discussed later on in Section Three.

Effects of subjective and objective testing:

In his book

Writing English Language Tests (1988) Heaton writes two main points of

criticism on Objective testing:

1. Objective tests are frequently criticized on the grounds that they are simpler to

answer than subjective tests.

2. Objective tests of the multiple-choice type encourage guessing.





32/60

In subjective testing, according to J.B.Heaton (1988) the tests are not reliable

because the scoring depends on the personal impressions of the teachers, while there is no

personal impression or favour on the part of teachers in objective tests so they are more

reliable.

2.9 Criteria for the Evaluation of a Test]

Issues of validity:

It is essential for a good test to respect the issues of validity. The test must aim to

provide a true measure of the particular skill, which it is intended to measure: to the

extent that it measures external knowledge and other skills at the same time, it will not

be a valid test.

Madsen defines:

A valid test is one that in fact measures what it claims to be measuring.

(1983:178)

According to Heaton:





33/60

The test must cover an adequate and representative section of those areas and

skills it is desired to test. (1988:12)

Content validity:

A test is said to have content validity if its content constitutes representative

sample of language skills, structure, etc with which it is meant to be concerned.

Hughes (2003:22)

What is the importance of content validity? First, the greater the tests content

validity, the more likely it is to be an accurate measure of what it is supposed to measure.

i.e. to have construct validity. Secondly, a test, lacking in content validity, is likely to

have harmful backwash effect. Areas that are not tested are likely to become areas

ignored in teaching and learning. Hughes (2003:26-27)

Face validity:

Heaton defines it as:

If a test item looks right to other testers, teachers, moderators, and testees, it

can be described as having at least face validity. (1988:159)





34/60

Construct validity:

This term can be well defined in the words of Heaton. He defines:

It (construct validity) is capable of measuring certain specific characteristics in

accordance with a theory of language behaviour and learning. (1988:161)

Empirical validity:


This validity is obtained as a result of comparing the results of the test with the

results of some criterion measure. (1988:161)

Issues of Reliability:

When we construct, administer and score tests in such a way that scores

actually obtained on a test on a particular occasion are likely to be very similar to those

which would have been obtained if it had administered to same students with the same





35/60

ability, but at different time. The more similar the scores would have the reliable, the

reliable the test is said to be. Hughes (2003:28)

For a reliable test what we have to do is construct, administer and score tests in

such a way that the scores actually obtained on a test on a particular occasion are likely to

be very similar to those which would have been obtained if it had been administered to

the same students with the same ability, but at a different time. The more similar the

scores would have been, the more reliable the test is said to be. Hughes (2003:36)

Madsen defines it as:

A reliable test is one that produces essentially the same results consistently on

different occasions when the conditions of the test remain the same. (1983:179)

Practicality:

A test being reliable and valid is not positive or effective if it creates problems for

the administration and practicability of performance required by learner.

A test must be practicable: in other words, it must be fairly straight forward to

administer. Heaton (1988:167)





36/60

SECTION THREE: Backwash Effect

3.1 Backwash Effect on Teaching And Learning:

The effect of testing on teaching and learning is known as backwash, and can

be harmful and beneficial. If a test is regarded as important, if the stakes are high,

preparation for it can come to dominate all teaching and learning activities. And if the test

contents and testing techniques are at variance with the objectives of the course there is

likely to be a harmful backwash. Hughes (2003:1)

The term is used to account for how the tests effects on teaching and learning

process. It is required that the test techniques and the test content should meet the

objectives of the course, only then the beneficial backwash effect can be achieved.

Arthur Hughes defines the backwash effect in the following words:

The effect of testing on the teaching and learning is known as backwash.

(2003:1)

That is the reason why the national examinations or tests of much high value for

the learners have a major impact on the teaching. And teachers often teach to the test as

phrased by James Popham (1989)





37/60

Teaching to the testmeans teaching only what is to be tested. It is quite clear that

in order to bring about the changes in teaching one must bring about the changes in

testing.


Davies (1968) has said that the good test is an obedient servant since it

follows and apes the teaching. I find it difficult to agree. The proper relationship

between testing and teaching is surely that of partnership. (2003:2)

Anjum Saleemi says:

Good backwash means that the test and the course of instruction influence

each other reciprocally, and that testing is beneficial to teaching-learning process. The

test objectives and the course objectives, ideally speaking, should be the same, or at least

should correspond very closely. (1988:261)

3.2 Harmful Backwash Effect:


It should hardly be necessary to point out that where a test that is important to

students is multiple-choice in nature, there is a danger that practice for the test will have a





38/60

harmful effect on learning and teaching. Practice at multiple-choice (especially when-as-

happen-as much attention is to paid to improving ones educated guessing as to the

content of items) will not usually be the best way for students to improve their command

of a language. (2003:78)



http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/


39/60

SECTION FOUR: Evaluation of English Compulsory Papers (I & II) and some

suggestions.

4.1 Evaluation Of Grammatical items:

Grammatical items are tested in isolated and decontextualised sentences in

Intermediate English compulsory papers. The sentences, which are given in grammatical

items, are not connected with each other and form an artificial use of language. The items

that are tested are mechanical. They are mechanical in the sense that the students can do

them with understanding semantically and sometimes they do the activities without

understanding meaning. The grammatical items are tested on the basis of rote learning as

well as teaching.

Hughes criticizes the idea of rote learning as:

In the view of some testers, the content of a final achievement test should be

based directly on a detailed course syllabus or on the books and other material used. This

has been referred to as the syllabus content approach. The disadvantage is that if the

syllabus is badly designed, or the books and other materials are badly chosen, the result

of a test can be very misleading. Hughes (2003: 13)

After going through the Intermediate papers(I&II), I have come to know that the

paper developers have set the paper not on the modern approach of testing grammar but

they still stick to the traditional approach of grammar testing. However, they are using



http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/


40/60

modern techniques of testing grammar but the purpose of testing grammar is not achieved

because of their approach to testing.

Heaton defines the purpose of grammar tests in the way that:

These tests (grammar and usage) measure students ability to recognize

appropriate grammatical forms and to manipulate structures. (1988:9)

Marianne and Hilles (1988) suggested that grammatical items should be tested in

such a way that learners would have to prove that how the grammatical structure is used

meaningfully in the context.

Unfortunately in our system grammar is taught in isolated, unconnected sentences

and that is how it is tested which gives a fragmented, unrealistic picture of English and

makes it difficult for students to apply what they have learnt.

Madsen (1983) says that grammatical items should be tested in such that they

would result in achieving positive backwash.

According to my observation of some papers of Intermediate level, all the

grammatical are being tested with the support of disconnected, isolated and

decontextualised sentences, which with the change of context would have meant

something entirely different.





41/60

Heaton says:

Note that context is of the utmost importance in all tests. Decontextualised

multiple-choice items can do considerable harm by conveying the impression that

language can be learnt and used free of any context. Both linguistic context and

situational context are essential in using language. Isolated sentences in a multiple-choice

test simply add to artificiality of the test situation and give rise to ambiguity and

confusion. (1988:28)

Hence in the case of articles and prepositions it is required to have the knowledge

of when and why of their use which is out of question in these papers. The grammatical

items of these papers are discussed below one by one.

Question on the use of correct tense form of verbs:

In the Intermediate paper (I) there is a question that requires the testees to put theverbs given in the brackets in their correct tense forms.

In these types of questions are required the appropriate forms of verbs and the

sentences were provided with the neutral form. These types of questions only provide the

students to transform the tense in a sentence but the actual purpose of the grammar in a

language (its usage and function according to the context) is lacking. Another drawback

of these are, in appropriate test techniques there is a possibility of more than one possible

answer. E.g., In English paper I (2003) (see appendices paper (1) 2003 P.no)





42/60

Questions on the correct use of prepositions and articles:

The questions on the appropriate use of articles and prepositions are imperative inthe Intermediate English compulsory paper I. The candidates are asked to fill in the

blanks with articles, where necessary and fill in the blanks with appropriate prepositions.

E.g.; In Intermediate compulsory paper (I) 2004 (see appendices paper I 2004 P.no ii)

Q.6 a) Insert a, an, the in the following blanks:-

i) ______ honest man is _____ noblest work of God.

ii) He is _____ only person I trust.

iii) ______ copper is ____ useful metal.

Context is very important in the case of article, for many of the cases it is context,which determines whether to use the indefinite article (a/an) or the definite article (the)

and the absence of context makes them totally form focused.



http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/


43/60


The context provides students with enough background knowledge and details

to avoid ambiguity and alternative interpretations.(1988:36)

This type of test also shows invalidity so it would be better to avoid practicing

these tests in the test papers.

Q.6 b) Fill in the blanks with appropriate prepositions:-

i) I came _____ an old friend yesterday.

ii) He was observed_______ deep thinking.

iii) He went _____ college ____ noon.

iv) I brought a gift ______ my father.

To use the prepositions in the sentences in the absence of context, the potential

knowledge of meaning remain unrealized but as students are to use them in the sentences

so they memorize them because prepositions, that are questioned in the papers, are

limited in numbers.





44/60

Q.6 (c) Put the appropriate forms of the verbs in the following sentences.

(ii) If they (miss) the flight, they (not reach) Islamabad to attend the seminar.

Possible answers:

(i) If they will miss the flight, they would not reach Islamabad. OR

(ii) If they miss the flight, they would not reach Islamabad. OR

(iii) If they have missed the flight, they would not have reached Islamabad.

By evaluating the above questions it is quite clear that these questions lack

content validity and face validity.

The greater the tests content validity, the more likely it is to be an accurate

measure of what it is supposed to measure. Hughes (2003:27)



http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/http://www.software602.com/


45/60

Question On Translation:

The Intermediate English compulsory paper I contains a question that is:

Q.8 Translate the following into English:- OR

Translate the following passage into Urdu or Sindhi.

(See appendices Intermediate paper I on p.no.3)

The question on translation directly drives my mind to the essay-translation

approach (defined in section two), which is now referred to as the pre-scientific stage of

language testing.

Tests of translation, however tend to be unreliable because of the complex

nature of the various skills involved and the methods of scoring. In too many instances,

the unrealistic expectations of examiners result in the setting of highly artificial sentences

and literary texts for translation. Students are expected to display an ability to make fine

syntactical judgments and appropriate lexical distinctions- an ability which can only beacquired after achieving a high degree of proficiency not only in English and the mother

tongue but also in comparative stylistics and translation methods. Heaton (1988:18)





46/60

This traditional approach in language teaching and testing has its limitations,

because we use the sentences/utterances to perform a function and purpose in real life,

not just merely complete, make a translate the sentences.


Translation attempts to test multi items at the same time which is not effective

and suggestive in learning and teaching a language. (1988:44)

Question on the use of Idioms and Phrase:

Idioms and phrases are also given in the Intermediate English Compulsory Paper I

&II, the candidates are asked to make sentences using them.

(See appendices Intermediate English Compulsory Paper I&II.)

The purpose of these tests is to test students vocabulary and their ability to use

them in writing correctly. But this test item fails to serve its objective because the idioms

e.g. at sixes and sevens, to look down upon, crocodile tears, etc are repeated so often that

the students can almost guess the ones they will be required to make sentences in the

papers. Hence, this test item only tests candidates memory i.e. how well they can utilize

it in exams. So these test items cause a negative backwash effect. I think it would be





47/60

suitable to apply other testing techniques like; asking the students to match the idioms

with their meanings or the idioms should be used in the sentences and students would be

required to paraphrase or rewrite the sentences in simple words, etc. These techniques

would enable the learners to infer the meaning of the idioms and phrases from

meaningful contexts.

Question on Narration:

Question on narration only appears in Intermediate English Compulsory Paper II.

Q.6 Change the following into indirect speech:-

Certainly, cried little Hans, I take quite as a compliment your coming to me,

and I will start off at once. But lend me your lantern, as the night is so dark that I amafraid I might fall into the ditch. (See appendices Intermediate English Paper I 2004.)

I think this is the only question that needs no change. The question Change the

following into indirect speech is according to communicative approach of testing. This

item is not in isolated or unconnected sentences. Context is used in order to develop

semantics and pragmatics sense in the candidate. In my opinion, this technique of testing

narration is a fit in the Intermediate English Compulsory Paper II.

In support to this item I must quote the Madsen here. He says:





48/60

The best exams are those that combine various skills as we do when

exchanging ideas orally or in written. (1983:7)

After analyzing and evaluating the papers I here recommend that the testing

system on the traditional approach should be replaced with the communicative language

testing approach which requires that the grammar be tested in a meaningful context,

preferably using authentic material (newspaper articles etc). I design a sample test on the

communicative approach that would, I think, better serve to test the grammatical

competence of the students in general and especially to the college students.

4.2 Advantages of my proposed grammatical items:

The technique that I have chosen to test grammatical items is Cloze Procedure.

The reason to choose this vary technique of testing is that it provides ease to the paper

developers in preparing test and to checkers in scoring it. It is based on integrative testing

(

thesis of ma linguistics

Documents