Assessments used in teaching English as a Assessments used in teaching English as a foreign language at elementary schools in Asia: foreign language at elementary schools in Asia:

Cases from Korea, Taiwan, and JapanCases from Korea, Taiwan, and Japan

Yuko Goto Butler Yuko Goto Butler (University of Pennsylvania, USA)(University of Pennsylvania, USA)

Chinfen Chen Chinfen Chen (National Taipei University of Education, Taiwan)(National Taipei University of Education, Taiwan)

Won-Key Lee Won-Key Lee (Seoul National University of Education, South Korea)(Seoul National University of Education, South Korea)

QuestionsQuestions

(1) What are the political, social and educational (1) What are the political, social and educational contexts into which English FLES (contexts into which English FLES (FForeign oreign LLanguage at anguage at EElementary lementary SSchools) is being chools) is being introduced in each of these countries? introduced in each of these countries?

(2) What types of assessments have been (2) What types of assessments have been administered thus far? administered thus far?

(3) What challenges do policymakers face in (3) What challenges do policymakers face in evaluating the effectiveness of English FLES? evaluating the effectiveness of English FLES?

(4) What results have been obtained so far in (4) What results have been obtained so far in terms of students’ English learning? terms of students’ English learning?

English at Elementary Schools English at Elementary Schools in Korea, Taiwan, Japanin Korea, Taiwan, Japan

KoreaKorea TaiwanTaiwan JapanJapan

Dates Dates officially officially introducedintroduced

1997; 1997; academic academic subjectsubject

2001; 2001; academic academic subjectsubject

20022002

Government Government oversightoversight

Strong Strong central gov. central gov. initiativeinitiative

General General guidelinesguidelines

Based on Based on local choicelocal choice

Grades that Grades that are taughtare taught

33rdrd and and beyond; beyond;

33rdrd and and beyond, but beyond, but some start some start from 1from 1stst

Varies greatlyVaries greatly

Japan: Japan: English assessments and diversityEnglish assessments and diversity

Yuko Goto ButlerYuko Goto Butler

University of PennsylvaniaUniversity of Pennsylvania

Current policy contexts in JapanCurrent policy contexts in Japan

In 2002, MEXT allowed local governments and individual In 2002, MEXT allowed local governments and individual schools to conduct “foreign language activities” (which schools to conduct “foreign language activities” (which are overwhelmingly English activities) based on their are overwhelmingly English activities) based on their own choice own choice Introduced as a means of fostering international Introduced as a means of fostering international understandingunderstandingIn 2006, a sub-panel on foreign language education for In 2006, a sub-panel on foreign language education for MEXT’s Central Council for Education proposed that MEXT’s Central Council for Education proposed that English should be compulsory for the 5English should be compulsory for the 5thth and 6 and 6thth grade grade levelslevelsMEXT has not yet made English compulsory at MEXT has not yet made English compulsory at elementary schoolselementary schoolsTremendous diversity in practice across schools and Tremendous diversity in practice across schools and local governmentslocal governments

Assessments for “English Activities” Assessments for “English Activities”

Assessments are not required by government Assessments are not required by government policiespoliciesOne can see the effects of English activities One can see the effects of English activities based on children’s self-assessments and based on children’s self-assessments and teacher observations in select school reports.teacher observations in select school reports.The results of such effects vary from school to The results of such effects vary from school to school.school.Strong need for systematic assessmentStrong need for systematic assessment

Butler & Takeuchi (2006)Butler & Takeuchi (2006)

Background Background Substantial variability in practiceSubstantial variability in practice Heated discussions regarding whether or not Heated discussions regarding whether or not

the central government’s policy is necessarythe central government’s policy is necessary

AimsAims To examine communicative skills in English by To examine communicative skills in English by

grade levels among students in select schoolsgrade levels among students in select schools To examine the relationship between policy-To examine the relationship between policy-

related factors and students’ communicative related factors and students’ communicative skills in Englishskills in English

Major policy related issuesMajor policy related issues

At which grade level should English be At which grade level should English be introduced? introduced? How many hours of instruction should children How many hours of instruction should children receive?receive?What goals should be set for English at the What goals should be set for English at the elementary school level?elementary school level?Should foreign teachers teach English?Should foreign teachers teach English?To what extent do the English lessons that To what extent do the English lessons that students receive outside of their schools students receive outside of their schools influence their performance?influence their performance?

ParticipantsParticipants

6,5416,541 elementary school students in Japan elementary school students in Japan 697 3697 3rdrd grade students grade students 1,666 41,666 4thth grade students grade students 2,337 52,337 5thth grade students grade students 1,814 61,814 6thth grade students grade students

Enrolled in 28 schools (24 public and 4 private)Enrolled in 28 schools (24 public and 4 private) School sizes: ranging from schools with less than 50 School sizes: ranging from schools with less than 50

students to schools with more than 800 studentsstudents to schools with more than 800 students Various types of schools, including pilot schoolsVarious types of schools, including pilot schools Instruction ranges from twice per week to once per Instruction ranges from twice per week to once per

monthmonth

MeasurementsMeasurements

The Junior STEP Silver TestThe Junior STEP Silver Test Developed by STEP (originally established as part of Developed by STEP (originally established as part of

the MOE’s English language education policy)the MOE’s English language education policy) Designed to measure basic oral communicative abilities Designed to measure basic oral communicative abilities

among young learners of EFLamong young learners of EFL The Junior STEP Tests have three levels: Bronze, The Junior STEP Tests have three levels: Bronze,

Silver, and GoldSilver, and Gold The Silver Test was chosen based on a pilot study The Silver Test was chosen based on a pilot study

conducted in 2005 among 5,087 studentsconducted in 2005 among 5,087 students The Silver Test measures basic listening skills and word The Silver Test measures basic listening skills and word

recognition skills among young learners who have recognition skills among young learners who have undergone oral-based instruction at school for at least 2 undergone oral-based instruction at school for at least 2 yearsyears

Examining students’ self-assessments and their Examining students’ self-assessments and their attitudes towards English activities based on a attitudes towards English activities based on a surveysurvey

Result 1Result 1

How do students’ communicative skills in How do students’ communicative skills in English differ by grade level?English differ by grade level?

Performance by Grade LevelPerformance by Grade Level

0

20

40

60

80

100

Vocab

ulary

Conve

rsat

ion

Sente

nce

Wor

d re

cogn

ition

Total

Ave

rage

per

cent

age

corr

ect

3rd

4th

5th

6th

Result 2Result 2

What are the relationships between policy-What are the relationships between policy-related factors and students’ related factors and students’ communicative skills in English?communicative skills in English? Grade levelGrade level Goals of English activities set by schoolsGoals of English activities set by schools Total hours of English instruction at schoolTotal hours of English instruction at school Frequencies of instruction by foreign teachers Frequencies of instruction by foreign teachers

at schoolat school Extra lessons that students receive outside of Extra lessons that students receive outside of

their schoolstheir schools

Multiple regression analysis of variables Multiple regression analysis of variables for predicting Silver Test scoresfor predicting Silver Test scores

BB SE BSE B β β tt

GradeGrade 1.511.51 .08.08 .23.23 18.05*18.05*

Goals set by the schoolGoals set by the school .81.81 .08.08 .13.13 10.28 *10.28 *

Total hours of English Total hours of English instruction at schoolinstruction at school

.01.01 .001.001 .06.06 4.67 *4.67 *

Frequencies of instruction Frequencies of instruction by foreign teachers at by foreign teachers at schoolschool

-.40-.40 .05.05 -.09-.09 -7.36 *-7.36 *

Extra lessons received Extra lessons received outside of schooloutside of school

1.381.38 .05.05 .31.31 25.49 *25.49 *

* p < .01

Correlations between Silver Test scores Correlations between Silver Test scores and instructional hoursand instructional hours

GradeGrade No. of No. of studentsstudents

Total hours of Total hours of instructioninstruction

Hours of instruction Hours of instruction per weekper week

33 604604 -.08-.08 -.09-.09

44 1,4481,448 -.13-.13** -.11-.11**

55 2,0912,091 .19.19** .19.19**

66 1,5061,506 .13.13** .10.10**

ConclusionsConclusions

The older the students, the higher their Silver The older the students, the higher their Silver Test scores becameTest scores becameThe biggest performance gap across grade level The biggest performance gap across grade level was found in the subsection on Word was found in the subsection on Word RecognitionRecognitionIn addition to grade level, the number of extra In addition to grade level, the number of extra English lessons that students received outside of English lessons that students received outside of their school was an important predictortheir school was an important predictorTotal English instructional hours at school and Total English instructional hours at school and frequencies of instruction by foreign teachers at frequencies of instruction by foreign teachers at school was not very influential variables. school was not very influential variables.