The 10th WSEAS International Conference on EDUCATION & TECHNOLOGY(ICEET '11)

Penang, Malaysia, Oct.3-5, 2011

Zamalia, M.Fac. of Computer Science & MathematicsUniversity Teknologi MARA, Shah Alam

Saidfudin, M; Azrilah, A.A, Azami, Z., Center of Eng’g Education, Fac. Built Env. & Eng’gUniversity Kebangsaan Malaysia

Overview of Grading System Rasch Model What is Standard Setting ? Findings and Discussion◦ Intro to Objective Standard Setting (OSS)◦ Item Measure : µ-essentials ◦ Person Grading

Conclusion

Marks has been the basis of the grading system; Rawscore: A;80-100, B;70-79, C;60 ……. Point : A ;3.67-4.00, B;3.33-3.66…. However, marks raw score gives only a rank

order; an indicator of achievement, NOT ability. Points are derived from raw score too.

Engineering Accreditation Council has been challenged on validity of assessment in accredited schools.

Students graduate with an ‘A’ but What’s an ‘A’ ?

Are we sure the correct instrument is used for the specified purpose; measuring ability ?

How do we set the threshold where the cut scores are; between A and B, B and C and so forth ?

What is the correct method of such data analysis; in this case Objective Standard Setting?

Nedelsky (1954), Angoff (1971) has been used but more for MCQ and contained large error.

Bookmark standard setting was developed by Green et. al. (1996) based on IRT. Problems:

- Choice of 3PL response probability 0.67 has no sound statistical basis.

- items simply sorted by difficulty level Rasch d Model resolved all the above issues and

utmost important include essential unidimensionality and local independence with better precision.

Rasch provides a ruler for measurement...

A student  will always have the chance of 50:50 of success when attempting  a  given  question.  His  chance  will  glide  along  a continuum  and  finally;  a  wrong  =1/99  or  a  right  =  99/1.;  in logarithm expressed as 10-2 , 100 and  10+2  as shown below.

In Rasch Model, a turn of event is seen as a chance; a likelihood of happenings hence a ratio data.(Steven, 1946) 

1090

10-2

-2

3070

6040

5050

991

199

100 102

0 2-1 1

exp

logit

Now, we already have a ruler with logit  SCALE

Identify essential items, NE ; noting the importance

of content validity by several subject experts. - Determine the level of mastery.- it gives the level of decision confidence

Then, determine the number of grade separation; say NS =four (4) separation of grades,

i.e. A ǀ B ǀ C ǀ D ǀ F Divide the total number of essential items, ΣNE and

number of separation grades; ΣNE/NS; this gives the number of items per grade...

Group 1

Group 2

Group 3

Group 4

1234

Rasch analysis offers a better method of evaluation where the decision of cut-off score is more objective in approach. This process is iterative until a ‘socially right’ answer is met.

However, whatever cutscore that is finally agreed upon shall not be done at the expense of the definition standard.

A standard should have meaning; at the least understand what a ‘Pass’ and ‘Fail’ means. It shall be able to assure that students end up in the right place.

1. Developed the difficulty ‘ruler’◦ Grading is based on ‘essentiality of knowledge’◦ Measure item or tasks difficulty

2. Standard for Measurement of Ability◦ Definition of each grade becomes meaningful;

we know what is ‘A’, ‘B’ and so forth.

3. Validate Decision Making◦ Better reflect measure of students’ ability◦ Precision and Accuracy of measurement.

Rasch probabilistic model gives the maximum likelihood estimate (MLE) of an event outcome.

Rasch Objective Standard Setting method offers a better method to verify the validity of measurement construct hence precision in deciding cutscores.

Hence, Rasch offers a new paradigm in engineering education longitudinal research; clearer to read, easy to understand in Standard Setting.

aidfudin@gmail.com

60 12240 2821

For more details contact:

University Kebangsaan Malaysia48000 Bangi, Selangor, MALAYSIA