TSL 3123LANGUAGE

ASSESSMENT

SCORING, GRADING AND ASSESSMENT

CRITERIA(3 hours)

1

LECTURE’S OBJECTIVES

• Identify and differentiate the different approaches used in scoringObjectiveHolisticAnalytic

• Use the different approaches used in scoring in assessing language

2

OBJECTIVE APPROACH

3

OBJECTIVE SCORING

• relies on quantified methods of evaluating students’ writing

• analyzing occurrences of linguistic features. • The objective scoring used were categorized as

word and sentence counts, syntactic complexity with no attention to error, and syntactic complexity with attention to error.

• objective scoring that took error into account were able to discriminate between three essay quality levels.

4

OBJECTIVE SCORINGA sample of how objective scoring is conducted is given by Bailey (1999) •Establish standardization by limiting the length of the assessment: Count the first 250 words of the essay.•Identify the elements to be assessed: Go through the essay up to the 250th word underlining every mistake – from spelling and mechanics through verb tenses, morphology, vocabulary, etc. Include every error that a literate reader might note.•Operationalise the assessment: Assign a weight score to each error, from 3 to 1. A score of 3 is a severe distortion of readability or flow of ideas; 2 is a moderate distortion; and 1 is a minor error that does not affect readability in any significant way.•Quantify the assessment: Calculate the essay Correctness Score by using 250 words as the numerator of a fraction, and the sum of error scores as the denominator: The denominator is the sum of all the error scores.

5

HOLISTIC SCORING

6

HOLISTIC SCORING• A rater makes an overall judgement about the quality

of the performance (to rate the overall proficiency level)• A single score is awarded to the writing• Normally this score is on a scale of 1 to 4, or 1 to 6, or

even 1 to 10.(Bailey, 1998 : 187).• Each score on the scale will be accompanied with

general descriptors of ability. • The example below is of a holistic scoring scheme

based on a 6 point scale.• It includes broad descriptors of what a student’s essay

reflects for each band.

7

HOLISTIC SCORING METHOD

8

HOLISTIC SCORING

• Another type of scoring related to the holistic approach is primary trait scoring (Bailey, 1998).

• A particular functional focus is selected based on the purpose of the writing and grading is based on how well the student is able to express that function.

• For example, if the function is to persuade, scoring would be on how well the author has been able to persuade the grader rather than how well organised the ideas were, or how grammatical the structures in the essay were.

• This technique emphasises functional and communicative ability rather than discrete linguistic ability and accuracy.

9

HOLISTIC SCORING

• It is usually used for large scale assessment because raters can be trained and scores can be computed more quickly (Bauer, 1981).

• The major disadvantage emerges from the limitations of the single score, which gives no details.

• Does not allow raters to distinguish between various aspects of writing such as control of syntax, depth of vocabulary, organization, and so on.

• Problematic for second-language writers since different aspects of writing ability may develop at different rates for different L2 learner.

10

ANALYTIC SCORING

11

ANALYTIC SCORING

• A rater assigns a score to each of the component for the task.• Example of analytical scoring approach:

• The points assigned to each component reflect the importance of each of the components.

• Analytic scoring is useful in the classroom because the result can help teachers diagnose students strengths and learning needs.

12

Components Weight

Content 30 points

Organisation 20 points

Vocabulary 20 points

Language Used 25 points

Mechanics 5 points

ANALYTICAL SCORING METHOD

13

ANALYTIC SCORING

• it provides more useful diagnostic information about students’ writing abilities.

• it is easier to train raters to use analytic scoring schemes• The major disadvantage of analytic scoring is that it takes

longer than holistic scoring

14

COMPARING THE THREE APPROACHES

Scoring Approach

Advantages Disadvantages

 Holistic   

Quickly graded Provide a public standard that is

understood by the teachers and students alike

Relatively higher degree of rater reliability

Applicable to the assessment of many different topics

Emphasise the students’ strengths rather than their weaknesses.

The single score may actually mask differences

across individual compositions. Does not provide a lot of diagnostic

feedback

 Analytical

It provides clear guidelines in grading in the form of the various components.

Allows the graders to consciously address important aspects of writing.

Writing ability is unnaturally split up into components.

 Objective

Emphasises the students’ strengths rather than their weaknesses.

Still some degree of subjectivity involved.

Accentuates negative aspects of the learner’s

writing without giving credit for what they can do well.

15

DESIGNING SCORING RUBRICS FOR YOUR CLASSROOM

• Rubrics are rating scales - as opposed to checklists-that are used with performance assessments.

• They are formally defined as scoring guides, consisting of specific pre-established performance criteria, used in evaluating student work on performance assessments.

16

DESIGNING SCORING RUBRICS FOR YOUR CLASSROOM

• Prior to designing a specific rubric, a teacher must decide whether the performance or product will be scored holistically or analytically (Airasian, 2000 & 2001).

• specific performance criteria and observable indicators must be identified as an initial step to development.

• the use of a holistic or analytic approach to scoring has several possible implications. teachers must consider first how they intend to use the results. If an

overall, summative score is desired, a holistic scoring approach would be more desirable. In contrast, if formative feedback is the goal, an analytic scoring rubric should be used2001).

Other implications include the time requirements, the nature of the task itself, and the specific performance criteria being observed.

17

HOLISTIC RUBRICS

• Holistic rubrics are customarily utilized when errors in some part of the process can be tolerated provided the overall quality is high (Chase, 1999).

• Nitko (2001) states that use of holistic rubrics is probably more appropriate when performance tasks require students to create some sort of response and where there is no definitive correct answer.

• Holistic rubrics are also used when the purpose of the performance assessment is summative in nature.

• A template for holistic scoring rubrics is presented in Table 1.

18

HOLISTIC RUBRICS

19

ANALYTIC RUBRICS

• Analytic rubrics are usually preferred when a fairly focused type of response is required (Nitko, 2001)

• For performance tasks in which there may be one or two acceptable responses and creativity is not an essential feature of the students' responses.

• A general rule of thumb is that an individual's work should be examined a separate time for each of the specific performance tasks or scoring criteria (Mertler, 2001)

20

ANALYTIC RUBRICS

21

RUBRICS TO GRADES

• It is not a good idea to think of rubrics in terms of percentages (Trice, 2000).

• For example, if a rubric has six levels (or “points”), a score of 3 should not be equated to 50% (an “F” in most letter grading systems).

• The converting process is more a process of logic than it is a mathematical one.

• Trice (2000) suggests that in a rubric scoring system, there are typically more scores at the average and above average categories (i.e., equating to grades of “C” or better) than there are below average categories. 22

RUBRICS TO GRADES• if a rubric consisted of nine score categories, the

equivalent grades and categories might look like this:

23

STEPS IN THE DESIGN OF SCORING RUBRICS

24

TUTORIAL TASKS

• Based on a stimulus, construct the scoring rubrics in the marking of essay questions to enhance test objectivity.

ISL TASK

Do some background reading on scoring rubrics.

25

• Brown, H. D., & Abeywickrama, P. (2010). Language Assessment: Principles and Classroom Practices.New York, NY: Pearson Education.

• Brown, H.D. (2007). Teaching by principles: An interactive approach to language pedagogy. Englewood Cliffs, NJ: Prentice Hall Regents.

• Chitravelu, Nesamalar, 2005. ELT Methodology: Principles and Practice. Penerbit Fajar Bakti, Sdn, Bhd.

• Malaysia Education Blueprint 2013-2025.

REFERENCES

26