![Page 1: The use of Dual-language test for bilingual students in ... · PDF fileThe use of Dual-language test for bilingual students in ... (STPM) All students are ... In 2003, the Ministry](https://reader033.vdocuments.mx/reader033/viewer/2022052708/5a7293087f8b9aa2538dbbc2/html5/thumbnails/1.jpg)
The use of Dual-language test for bilingual students in Science Assessment
Ong Saw [email protected]
School of Educational StudiesMalaysia Science University
1
![Page 2: The use of Dual-language test for bilingual students in ... · PDF fileThe use of Dual-language test for bilingual students in ... (STPM) All students are ... In 2003, the Ministry](https://reader033.vdocuments.mx/reader033/viewer/2022052708/5a7293087f8b9aa2538dbbc2/html5/thumbnails/2.jpg)
Malaysian Education System
Multilingual education system Three types of primary school primary medium
of instruction : National type Chinese primary school National type Tamil primary school National primary school (Malay being the national
language) Two types of Secondary school:
National Secondary school National type secondary school (Chinese
language learn as a subject) 2
![Page 3: The use of Dual-language test for bilingual students in ... · PDF fileThe use of Dual-language test for bilingual students in ... (STPM) All students are ... In 2003, the Ministry](https://reader033.vdocuments.mx/reader033/viewer/2022052708/5a7293087f8b9aa2538dbbc2/html5/thumbnails/3.jpg)
Centralised National Examinations Primary School Achievement Test (UPSR) Lower Secondary Examination (PMR) Malaysian Certificate of Education (SPM) Malaysian Higher School Certificate Examination
(STPM) All students are mandated to sit for the national
examinations Language of assessment:
UPSR: Malay, Chinese & Tamil PMR, SPM, STPM: Malay only
3
![Page 4: The use of Dual-language test for bilingual students in ... · PDF fileThe use of Dual-language test for bilingual students in ... (STPM) All students are ... In 2003, the Ministry](https://reader033.vdocuments.mx/reader033/viewer/2022052708/5a7293087f8b9aa2538dbbc2/html5/thumbnails/4.jpg)
Centralised Examinations
High stake examinations used for making important decisions
UPSR – selection of candidates into fully residential schools (for Malay students only), premiere secondary schools
PMR – selection into science schools (for Malay students only)
SPM – entry into pre-university colleges, Form 6, teacher training program, government scholarship
STPM –entry into university program 4
![Page 5: The use of Dual-language test for bilingual students in ... · PDF fileThe use of Dual-language test for bilingual students in ... (STPM) All students are ... In 2003, the Ministry](https://reader033.vdocuments.mx/reader033/viewer/2022052708/5a7293087f8b9aa2538dbbc2/html5/thumbnails/5.jpg)
Language Policy
Malay language has been the medium of instruction for the national secondary schools (1980) & national primary schools (1968)
In 2003, the Ministry of Education implemented the policy of teaching science and mathematics in English
The national examinations for science and mathematics are administered in dual-language, e.g. Malay and English for the secondary level national examinations.
5
![Page 6: The use of Dual-language test for bilingual students in ... · PDF fileThe use of Dual-language test for bilingual students in ... (STPM) All students are ... In 2003, the Ministry](https://reader033.vdocuments.mx/reader033/viewer/2022052708/5a7293087f8b9aa2538dbbc2/html5/thumbnails/6.jpg)
Language used in PMR Maths
In 2006, 24.1% of the candidates chose to answer in English, 46.2% still answered in Malay while 29.7% answered in both languages (Star, 2006).
Among the candidates who answered in Malay, they were however using the mathematical terms in English
6
![Page 7: The use of Dual-language test for bilingual students in ... · PDF fileThe use of Dual-language test for bilingual students in ... (STPM) All students are ... In 2003, the Ministry](https://reader033.vdocuments.mx/reader033/viewer/2022052708/5a7293087f8b9aa2538dbbc2/html5/thumbnails/7.jpg)
Reversal of Language Policy
The Education Minister has recently announced the reversal of the language policy to mother tongue; Malay, Chinese and Tamil for the teaching of science and mathematics in the primary schools beginning with Standard 1 in the year 2012
(The Star, July 2009)
7
![Page 8: The use of Dual-language test for bilingual students in ... · PDF fileThe use of Dual-language test for bilingual students in ... (STPM) All students are ... In 2003, the Ministry](https://reader033.vdocuments.mx/reader033/viewer/2022052708/5a7293087f8b9aa2538dbbc2/html5/thumbnails/8.jpg)
Validity Issue
• Question on comparability of the standardized achievement tests that use different language of assessment.
• The use of dual-language test for science and mathematics: Does this eliminate the language factor?
8
![Page 9: The use of Dual-language test for bilingual students in ... · PDF fileThe use of Dual-language test for bilingual students in ... (STPM) All students are ... In 2003, the Ministry](https://reader033.vdocuments.mx/reader033/viewer/2022052708/5a7293087f8b9aa2538dbbc2/html5/thumbnails/9.jpg)
Language Proficiency & Testing
Students who are still in the process of learning English when tested in English, their proficiency in English will also be tested (AERA, APA, & NCME, 1999)
For students with limited English proficiency, the language of test items can prevent them from demonstrating their knowledge in the area of science and mathematics (Abedi, Lord, & Plummer, 1997)
9
![Page 10: The use of Dual-language test for bilingual students in ... · PDF fileThe use of Dual-language test for bilingual students in ... (STPM) All students are ... In 2003, the Ministry](https://reader033.vdocuments.mx/reader033/viewer/2022052708/5a7293087f8b9aa2538dbbc2/html5/thumbnails/10.jpg)
Linguistic Accommodation in Content Areas Test
Content-based standardized achievement tests aim to measure students’ knowledge of specific content areas.
Reduce the English language load by making test more accessible to limited English proficient students (Liu et al., 1999).
Types of linguistic accommodation Use of native language (Rivera, 2006), dual-language test (Garcia, 2000), dictionary, bilingual glossary (Abedi, Courtney, & Leon
2003), linguistic simplified test (Abedi, 2001, Riverra &
Stansfielld, 2003/4 ) and use of figures and diagrams (Fairbairn, 2006).
10
![Page 11: The use of Dual-language test for bilingual students in ... · PDF fileThe use of Dual-language test for bilingual students in ... (STPM) All students are ... In 2003, the Ministry](https://reader033.vdocuments.mx/reader033/viewer/2022052708/5a7293087f8b9aa2538dbbc2/html5/thumbnails/11.jpg)
Reason for Dual-language Test in Malaysia
Malaysian students most likely operate in two languages as the primary medium of instruction is Malay but the language of instruction for science is English.
Candidates can gain additional information and comprehend the question better in one of the languages they are proficient in, or utilise both language versions
11
![Page 12: The use of Dual-language test for bilingual students in ... · PDF fileThe use of Dual-language test for bilingual students in ... (STPM) All students are ... In 2003, the Ministry](https://reader033.vdocuments.mx/reader033/viewer/2022052708/5a7293087f8b9aa2538dbbc2/html5/thumbnails/12.jpg)
Dual-language Test
Adopted by the Malaysian Examination Syndicate (MES), the provider of the national level assessments
Dual-language booklet is used for all examinees in the national level science and mathematics examinations
UPSR Malay and English Chinese & English Tamil & English
PMR, SPM & STPM Malay & English
Dual-language test is an additional burden, for school level assessment, teachers used only English.
12
![Page 13: The use of Dual-language test for bilingual students in ... · PDF fileThe use of Dual-language test for bilingual students in ... (STPM) All students are ... In 2003, the Ministry](https://reader033.vdocuments.mx/reader033/viewer/2022052708/5a7293087f8b9aa2538dbbc2/html5/thumbnails/13.jpg)
Is Linguistics Accommodation Useful?
Translating math tests into the native language of LEP students was found to be not helpful if math instruction took place in English (Abedi, Lord, & Hofstetter,1998).
Garcia et al.’s (2000) results showed that the dual-language maths test booklet assists those students who were less proficient in English.
Students in Garcia et al’s study perceived the math dual-language test booklet as beneficial although they tended to use and answer in one language (English).
13
![Page 14: The use of Dual-language test for bilingual students in ... · PDF fileThe use of Dual-language test for bilingual students in ... (STPM) All students are ... In 2003, the Ministry](https://reader033.vdocuments.mx/reader033/viewer/2022052708/5a7293087f8b9aa2538dbbc2/html5/thumbnails/14.jpg)
Aim of study
1) To investigate the effect of the use of dual-language test booklet in addressing the linguistic needs of Malaysian students.
2) To what extent the dual-language test will raise the science achievement by improving students’ comprehension of test items.
14
![Page 15: The use of Dual-language test for bilingual students in ... · PDF fileThe use of Dual-language test for bilingual students in ... (STPM) All students are ... In 2003, the Ministry](https://reader033.vdocuments.mx/reader033/viewer/2022052708/5a7293087f8b9aa2538dbbc2/html5/thumbnails/15.jpg)
Science Test 40 items aligned with the Malaysian science
curriculum were selected from the 1999 and 2003 TIMSS eight-grade public release science items.
Three bilingual science teachers translate the items into Malay
Two separate science test booklets were created. The English-only science items. The English and Malay science items .
The test is administered in 40 minutes
15
![Page 16: The use of Dual-language test for bilingual students in ... · PDF fileThe use of Dual-language test for bilingual students in ... (STPM) All students are ... In 2003, the Ministry](https://reader033.vdocuments.mx/reader033/viewer/2022052708/5a7293087f8b9aa2538dbbc2/html5/thumbnails/16.jpg)
Participants
1,720 eight-grade students from 26 secondary schools
While in primary schools, science were taught to these students in either Malay, Chinese or Tamil language
When they enter secondary school, the medium of instruction for science changed to English
At the time of administrating the test, students have undergone 2 years of science learning in English (Form I & II)
16
![Page 17: The use of Dual-language test for bilingual students in ... · PDF fileThe use of Dual-language test for bilingual students in ... (STPM) All students are ... In 2003, the Ministry](https://reader033.vdocuments.mx/reader033/viewer/2022052708/5a7293087f8b9aa2538dbbc2/html5/thumbnails/17.jpg)
Data Collection
876 students were administered the dual-language science test
844 students took the English-only science test The test was administered to the participants at the
end of the school year, October 2008. The English-only test and the dual-language test was
assigned randomly to students
17
![Page 18: The use of Dual-language test for bilingual students in ... · PDF fileThe use of Dual-language test for bilingual students in ... (STPM) All students are ... In 2003, the Ministry](https://reader033.vdocuments.mx/reader033/viewer/2022052708/5a7293087f8b9aa2538dbbc2/html5/thumbnails/18.jpg)
Language used by science teacher N
(%)Dual-language Group
English Group
Malay only 1.7%(15)
1.3%(11)
English only 19.9(174)
21.3(180)
English & Malay equally 42.4%(371)
36.3%(306)
Malay more than English 11.5%(101)
13.0%(110)
English more than Malay 24.0%(210)
27.5%(232)
![Page 19: The use of Dual-language test for bilingual students in ... · PDF fileThe use of Dual-language test for bilingual students in ... (STPM) All students are ... In 2003, the Ministry](https://reader033.vdocuments.mx/reader033/viewer/2022052708/5a7293087f8b9aa2538dbbc2/html5/thumbnails/19.jpg)
Use of Dual-language Test by participant
Frequency PercentUse only Malay 118 13.5Use only English 207 23.6Use Malay more than English 179 20.4
Use English more than Malay 149 17.0
Use English & Malay equally 176 20.1
Omit response 47 5.4Total 876 100.0
19
![Page 20: The use of Dual-language test for bilingual students in ... · PDF fileThe use of Dual-language test for bilingual students in ... (STPM) All students are ... In 2003, the Ministry](https://reader033.vdocuments.mx/reader033/viewer/2022052708/5a7293087f8b9aa2538dbbc2/html5/thumbnails/20.jpg)
Data Analysis
Comparing science achievement Score distribution plots Boxplots Moments Test equating
Comparing characteristics of tests Item parameters Differential Item Functioning
Software: Lertap20
![Page 21: The use of Dual-language test for bilingual students in ... · PDF fileThe use of Dual-language test for bilingual students in ... (STPM) All students are ... In 2003, the Ministry](https://reader033.vdocuments.mx/reader033/viewer/2022052708/5a7293087f8b9aa2538dbbc2/html5/thumbnails/21.jpg)
Mean Item Parameter
21
Bilingual Item
English Item
Mean Item Difficulty 0.500 0.499Mean Item Discrimination
0.398 0.424
Mean Point Biserial 0.354 0.367Mean Adj. Point Biserial 0.292 0.308KR20 (Alpha) 0.824 0.837KR21 0.798 0.817SEM (from KR20) 2.816 2.827
![Page 22: The use of Dual-language test for bilingual students in ... · PDF fileThe use of Dual-language test for bilingual students in ... (STPM) All students are ... In 2003, the Ministry](https://reader033.vdocuments.mx/reader033/viewer/2022052708/5a7293087f8b9aa2538dbbc2/html5/thumbnails/22.jpg)
Item Difficulty (Proportion Correct)
22
Comparing Item Difficulty
00.10.20.30.40.50.60.70.80.9
1
1 4 7 10 13 16 19 22 25 28 31 34 37 40
Item Number
Prop
ortio
n C
orre
English AverageBilingual Average
![Page 23: The use of Dual-language test for bilingual students in ... · PDF fileThe use of Dual-language test for bilingual students in ... (STPM) All students are ... In 2003, the Ministry](https://reader033.vdocuments.mx/reader033/viewer/2022052708/5a7293087f8b9aa2538dbbc2/html5/thumbnails/23.jpg)
Item Difficulty
13 items seem to be slighlt easier when presented in dual-language format.
Two items, Item 15 & Item 35, become very much easier in the dual-language format.
23
![Page 24: The use of Dual-language test for bilingual students in ... · PDF fileThe use of Dual-language test for bilingual students in ... (STPM) All students are ... In 2003, the Ministry](https://reader033.vdocuments.mx/reader033/viewer/2022052708/5a7293087f8b9aa2538dbbc2/html5/thumbnails/24.jpg)
Comparing item difficulty (Proportional Correct)
24
Item Number
English Bilingual Difference
15 .58 .74 -.16
35 .48 .68 -.20
Mean 0.499 0.500
![Page 25: The use of Dual-language test for bilingual students in ... · PDF fileThe use of Dual-language test for bilingual students in ... (STPM) All students are ... In 2003, the Ministry](https://reader033.vdocuments.mx/reader033/viewer/2022052708/5a7293087f8b9aa2538dbbc2/html5/thumbnails/25.jpg)
Item 15
The burning of fossil fuels has increased the carbon dioxide content of the atmosphere. What is a possible effect that the increased amount of carbon dioxide is likely to have on our planet?
a) A warmer climate b) A cooler climate c) Lower relative humidity d) More ozone in the
atmosphere25
Pembakaran bahan api fosil menambahkan kandungan karbon dioksida dalam atmosfera. Apakah kesan yang mungkin berlaku akibat penambahan karbon dioksida pada planet kita?a) cuaca yang lebih panasb) cuaca yang lebih sejukc) kurangkan kelembapan relatif d) lebih banyak ozon pada atmosfera
![Page 26: The use of Dual-language test for bilingual students in ... · PDF fileThe use of Dual-language test for bilingual students in ... (STPM) All students are ... In 2003, the Ministry](https://reader033.vdocuments.mx/reader033/viewer/2022052708/5a7293087f8b9aa2538dbbc2/html5/thumbnails/26.jpg)
Why dual-language Item 15 is easier?
When presented in a language that most students are not proficient, the item stem is too wordy
Bilingual students are put off by the long text used to describe the stimulus
26
![Page 27: The use of Dual-language test for bilingual students in ... · PDF fileThe use of Dual-language test for bilingual students in ... (STPM) All students are ... In 2003, the Ministry](https://reader033.vdocuments.mx/reader033/viewer/2022052708/5a7293087f8b9aa2538dbbc2/html5/thumbnails/27.jpg)
Options selection (Item 15)
27
0%
10%
20%
30%
40%
50%
60%
70%
80%
A B C D Other
I15
E
B
I15 A B C DE 58% 7% 14% 20%B 74% 5% 8% 12%
![Page 28: The use of Dual-language test for bilingual students in ... · PDF fileThe use of Dual-language test for bilingual students in ... (STPM) All students are ... In 2003, the Ministry](https://reader033.vdocuments.mx/reader033/viewer/2022052708/5a7293087f8b9aa2538dbbc2/html5/thumbnails/28.jpg)
Item 35
Overgrazing of land by livestock contributes to a major problem.
That problem isa) depletion of ground
waterb) increased pollutionc) erosion of soild) acid rain
28
Meragut rumput di kawasan tanah secara berlebihan oleh ternakan menyumbang kepada masalah berikut:a) pengurangan air tanahb) peningkatan pencemaran c) hakisan tanahd) hujan asid
![Page 29: The use of Dual-language test for bilingual students in ... · PDF fileThe use of Dual-language test for bilingual students in ... (STPM) All students are ... In 2003, the Ministry](https://reader033.vdocuments.mx/reader033/viewer/2022052708/5a7293087f8b9aa2538dbbc2/html5/thumbnails/29.jpg)
Why Item 35 is difficult?
The terms ‘overgrazing’ and ‘livestock’ are uncommon words among Malaysian students
The translation to Malay probably enable them to understand the question better
29
![Page 30: The use of Dual-language test for bilingual students in ... · PDF fileThe use of Dual-language test for bilingual students in ... (STPM) All students are ... In 2003, the Ministry](https://reader033.vdocuments.mx/reader033/viewer/2022052708/5a7293087f8b9aa2538dbbc2/html5/thumbnails/30.jpg)
Options Selection (Item 35)
30
0%
10%
20%
30%
40%
50%
60%
70%
80%
A B C D Other
I35
EB
I35 A B C D OtherE 15% 24% 48% 11% 2%B 14% 9% 68% 8% 1%
![Page 31: The use of Dual-language test for bilingual students in ... · PDF fileThe use of Dual-language test for bilingual students in ... (STPM) All students are ... In 2003, the Ministry](https://reader033.vdocuments.mx/reader033/viewer/2022052708/5a7293087f8b9aa2538dbbc2/html5/thumbnails/31.jpg)
Score Distribution
31
Dual-language Test English-only TestTotal possible score 40 40Minimum score 4 4Maximum score 39 39Median score 19 19.5Mean score 19.99 19.95Standard deviation 6.72 7.01Variance 45.12 49.13Skewness 0.27 0.28Kurtosis -0.60 -0.63Number of examinees 876 844
![Page 32: The use of Dual-language test for bilingual students in ... · PDF fileThe use of Dual-language test for bilingual students in ... (STPM) All students are ... In 2003, the Ministry](https://reader033.vdocuments.mx/reader033/viewer/2022052708/5a7293087f8b9aa2538dbbc2/html5/thumbnails/32.jpg)
Score Distribution
32
Score Distribution
010203040506070
1 5 9 13 17 21 25 29 33 37
Raw Score
Freq
uenc
BilingualEnglish
![Page 33: The use of Dual-language test for bilingual students in ... · PDF fileThe use of Dual-language test for bilingual students in ... (STPM) All students are ... In 2003, the Ministry](https://reader033.vdocuments.mx/reader033/viewer/2022052708/5a7293087f8b9aa2538dbbc2/html5/thumbnails/33.jpg)
Score distribution with Boxplot
33
Simulated Boxplot for 'BilEng'
4
9
14
19
24
29
34
39
B E
![Page 34: The use of Dual-language test for bilingual students in ... · PDF fileThe use of Dual-language test for bilingual students in ... (STPM) All students are ... In 2003, the Ministry](https://reader033.vdocuments.mx/reader033/viewer/2022052708/5a7293087f8b9aa2538dbbc2/html5/thumbnails/34.jpg)
MH DIF plots (Item 35)
MH alpha=.41, MH d-DIF=2.12, ETS level (C) large DIF
Dual-languageis easier (from score range 9-26)
34
Red line: Dual-language
![Page 35: The use of Dual-language test for bilingual students in ... · PDF fileThe use of Dual-language test for bilingual students in ... (STPM) All students are ... In 2003, the Ministry](https://reader033.vdocuments.mx/reader033/viewer/2022052708/5a7293087f8b9aa2538dbbc2/html5/thumbnails/35.jpg)
MH- DIF plots (Item 15)
Item 15: MH alpha=.42,MH d-DIF=2.04, ETS level (C) large DIF
Dual-language is easier,from score range (9-24)
![Page 36: The use of Dual-language test for bilingual students in ... · PDF fileThe use of Dual-language test for bilingual students in ... (STPM) All students are ... In 2003, the Ministry](https://reader033.vdocuments.mx/reader033/viewer/2022052708/5a7293087f8b9aa2538dbbc2/html5/thumbnails/36.jpg)
MH-DIF plot (Item 29)
MH alpha =1.57, MH D-DIF = -1.06, ETS level; B (moderate)
Item in Englishis easier than dual-language(for score rangefrom 9-22)
36
![Page 37: The use of Dual-language test for bilingual students in ... · PDF fileThe use of Dual-language test for bilingual students in ... (STPM) All students are ... In 2003, the Ministry](https://reader033.vdocuments.mx/reader033/viewer/2022052708/5a7293087f8b9aa2538dbbc2/html5/thumbnails/37.jpg)
MH-DIF plot (Item 36) MH alpha =.60, MH D-DIF =1.21, ETS level; B
(moderate) Dual-language is Easier (only for score range from12-19)
37
![Page 38: The use of Dual-language test for bilingual students in ... · PDF fileThe use of Dual-language test for bilingual students in ... (STPM) All students are ... In 2003, the Ministry](https://reader033.vdocuments.mx/reader033/viewer/2022052708/5a7293087f8b9aa2538dbbc2/html5/thumbnails/38.jpg)
Use of Dual-language Test
Most students (those given the dual-language test) used both Malay and English when answering the science test
Students perceived the dual-language test booklets as useful
38
![Page 39: The use of Dual-language test for bilingual students in ... · PDF fileThe use of Dual-language test for bilingual students in ... (STPM) All students are ... In 2003, the Ministry](https://reader033.vdocuments.mx/reader033/viewer/2022052708/5a7293087f8b9aa2538dbbc2/html5/thumbnails/39.jpg)
Students Science Achievement
Similar score distribution pattern obtained From graphical plot of score distribution Boxplot Skewness & Kurtosis
Overall, students science achievement is not improved with the use of dual-language test
39
![Page 40: The use of Dual-language test for bilingual students in ... · PDF fileThe use of Dual-language test for bilingual students in ... (STPM) All students are ... In 2003, the Ministry](https://reader033.vdocuments.mx/reader033/viewer/2022052708/5a7293087f8b9aa2538dbbc2/html5/thumbnails/40.jpg)
Dual-language Item Characteristics
Less than half of the multiple-choice test items were found to be slightly easier when presented in dual-language
Only two items become much easier significantly with the presence of the additional language and is said to exhibit large differential item functioning
40
![Page 41: The use of Dual-language test for bilingual students in ... · PDF fileThe use of Dual-language test for bilingual students in ... (STPM) All students are ... In 2003, the Ministry](https://reader033.vdocuments.mx/reader033/viewer/2022052708/5a7293087f8b9aa2538dbbc2/html5/thumbnails/41.jpg)
Conclusion
The extra language version of the science test did not provide additional advantage to the students, there were no difference in the overall performance
Only two items become very much easier with the additional language version
![Page 42: The use of Dual-language test for bilingual students in ... · PDF fileThe use of Dual-language test for bilingual students in ... (STPM) All students are ... In 2003, the Ministry](https://reader033.vdocuments.mx/reader033/viewer/2022052708/5a7293087f8b9aa2538dbbc2/html5/thumbnails/42.jpg)
Implication
Other form of accommodation, e.g. use of diagrams may be more useful for these bilingual students
Content knowledge may be the more important factor that determines students’ achievement in science
More emphasis should be on the teaching of science content
Results may be different when open-ended items are used
42
![Page 43: The use of Dual-language test for bilingual students in ... · PDF fileThe use of Dual-language test for bilingual students in ... (STPM) All students are ... In 2003, the Ministry](https://reader033.vdocuments.mx/reader033/viewer/2022052708/5a7293087f8b9aa2538dbbc2/html5/thumbnails/43.jpg)
43