students' and parents' views of sba in hong kong - citeseerx

Corresponding author:Liying Cheng, A213, Faculty of Education, Queen’s University, Kingston, ON K7L 3N6, Canada. Email: [email protected]

Impact and consequences of school-based assessment (SBA): Students’ and parents’ views of SBA in Hong Kong

Liying Cheng Queen’s University, Canada

Stephen Andrews and Ying YuUniversity of Hong Kong, People’s Republic of China

AbstractSchool-based assessment (SBA) has recently been introduced into the Hong Kong Certificate of Education Examinations (HKCEE) in English. The present study was conducted within the context of this assessment change to investigate students’ and parents’ perceptions of the impact of SBA. Two surveys were employed to explore students’ and parents’ perceptions of SBA and their perceptions of the impact of SBA on learning. The results from the student survey demonstrated a relationship between students’ perceptions of SBA-related learning activities and their perceptions of their own language competence. The results also showed significant differences between students’ perceptions of the learning activities they had taken part in during the previous school year and those they were currently engaging in. In addition, parents’ perceptions of SBA and the opportunities for them to know about SBA significantly and positively predicted their support for their children’s SBA learning. Parents’ education level and the amount of time they spent with their child daily also predicted their support for their children’s SBA learning, though to a lesser extent. Further, parents’ perceptions about the SBA are directly and significantly related to their children’s perceptions about SBA. Taking the results from both students’ and parents’ surveys together, we have gained a better understanding of the complexity of the impact of SBA within the Hong Kong educational context, as perceived by students and their parents.

Keywordsimpact and consequences, school-based assessment (SBA), students and parents

As part of the worldwide movement to combine assessment of learning with assessment for learning in order to promote students’ learning, standards-referenced school-based assessment (SBA) has recently been introduced into the Hong Kong Certificate of

Language Testing XX(X) 1–29

© The Author(s) 2010Reprints and permission:

sagepub.co.uk/journalsPermissions.navDOI: 10.1177/0265532210384253

http://ltj.sagepub.com

Article

at PENNSYLVANIA STATE UNIV on May 10, 2016ltj.sagepub.comDownloaded from

http://ltj.sagepub.com/

2 Language Testing XX(X)

Education Examinations (HKCEE) in English, taken at the end of Secondary 5 (the fifth of secondary education after six years of primary education: Ages 16–17). The HKCEE SBA component in English focuses specifically on the assessment of students’ spoken English, and is currently conducted over two academic years, Secondary 4 and 5. Second-ary 1–3 constitute the junior secondary stage; Secondary 4–5 form the first two years of senior secondary schooling. This SBA innovation, which was introduced into Secondary 4 classrooms in 2005–2006, has already caused reverberations within the Hong Kong education system (Davison, 2007). The introduction of SBA means that for the first time Hong Kong secondary school English teachers’ evaluation of their own students is counted towards their students’ final marks on a high-stakes exam, in this case the HKCEE. The addition of this SBA component has significant implications for both teaching and learn-ing, and presents challenges to definitions of key components of assessment such as validity, reliability, and fairness at the philosophical, theoretical, and practical levels.

The SBA component in English focuses on the assessment of students’ oral perfor-mance, linked to a reading/viewing programme running through the Secondary 4 and Secondary 5 academic years. During that two-year period, students are required to read/view at least three texts, each from a different category of text among the following four: print fiction, print non-fiction, non-print fiction, and non-print non-fiction. They are expected to record their personal reflections on what they have read/viewed, to take part in discussions with classmates about their reading/viewing, and to make individual pre-sentations on aspects of their books or films. The class English teacher conducts two assessments, one per year, and each summer the school’s scores are reported to the Hong Kong Examinations and Assessment Authority (HKEAA). The assessments may be either of students’ individual presentation or group interaction skills.

The SBA component has a weighting of 15% in the HKCEE English examination. It is intended to be integrated into the normal curriculum, rather than being treated as a separate exam paper, with the assessment information being used formatively by both teachers and students to improve teaching and learning. Students’ active involvement in learning is promoted in a number of ways. For instance, they are encouraged to keep copies of their assessment records, so that they can regularly review their own progress. They are also encouraged to develop skills in self- as well as peer-assessment. The SBA component aims to improve students’ oral English proficiency and to promote extensive reading as its long term goals (HKEAA, 2006).

Although students are the ultimate stakeholders in any assessment innovation (includ-ing SBA), in the sense that they are the ones who are assessed, they have tended to be researched less compared with other stakeholders in previous impact/washback studies (Cheng, 2008; Hamp-Lyons, 1997). Also, in spite of the evidence from previous research that parents often play an important role in shaping teaching and learning in schools (Linn, 1993) and that there is a relationship between parents’ involvement and their chil-dren’s learning (Mulvenon, Stegman, & Ritter, 2005), little attention has been paid to parents also in previous impact/washback studies. Furthermore, studies that have linked both stakeholders’ perceptions are non-existent. The purposes of the present study are therefore twofold: 1) to understand students’ and parents’ views about SBA in relation to its impact on aspects of the teaching and learning of English in Hong Kong secondary schools; 2) to explore the relationship between the perceptions of these two stakeholders: students and their parents.



Cheng et al. 3

Test impact: students and parents

Today, due to the power of tests, many educational systems around the world use high-stakes tests as an engine to induce desirable pedagogical changes, with improved student learning as the ultimate goal (Andrews, 2004; Chapman & Snyder, 2000; Cheng, 2005; James, 2000; Linn, 2000; Qi, 2005, 2007). Because high-stakes assessment innovations tend to be top-down in nature, students are often considered as recipients and potential beneficiaries of an innovation rather than as key participants in the change process (Rich-ards, 2001). However, as Henrichsen (1989) rightly points out, the success of any educa-tional innovation is dictated to some extent by learner-relevant factors, for example, students’ perceptual, cognitive, and affective characteristics.

According to Hughes (1993), a test may first influence its participants’ perceptions and attitudes, which may then affect what the participants do (the processes) in order to achieve improved learning outcomes, that is, the products. In his washback model, par-ticipants include teachers, learners, material developers, and publishers. As mentioned above, the perceptions and practices of teachers have been relatively widely researched in comparison with those of other stakeholders (e.g. Alderson & Hamp-Lyons, 1996; Cheng, 1999; Shohamy, Donitsa-Schmidt, & Ferman, 1996; Watanabe, 1996). However, much remains unknown about the washback effects of tests on various aspects relating to the learners and their learning processes. Of the 15 washback hypotheses proposed by Alderson and Wall (Alderson & Wall, 1993), eight are related to learners but very few of them have been empirically tested. Meanwhile, empirical studies of testing effects on parents are almost non-existent.

In some previous washback studies, students are included as part of the research. For instance, in a recent investigation of stakeholder perceptions of test impact on learners in the primary school context in the UK, Scott (2007) found that the degree of the test impact varied in different grades. The higher the grades, the more intensive testing effects were felt among the students. In another recent study, Qi (2007) examined students’ per-ceptions of writing compared with those of the test constructors embodied in the writing task of the national matriculation English test in China. Although there was evidence of some shared views between the two groups of stakeholders, a mismatch was identified in relation to perceptions concerning writing. The test constructors’ intention was that, as reflected in the input of the writing tasks, students would be encouraged to learn to write for communicative purposes. However, students were found to focus only on those aspects of writing that they believed would help achieve better scores, while neglecting the development of the ability to write communicatively in real-life situations. Though not discussed in the article, this finding seems to suggest that students may not always fully understand the construct of a test. This may happen especially in the public exam context, where test-related information may not be directly accessible to students. In addition, students’ perceptions of tests are likely to be shaped by the school context, for example, by their teachers and peers. Therefore, it is important to examine not only what students understand about a test, but also how they obtain such knowledge. Cheng’s study (1998) includes, probably for the first time in the washback literature, the investi-gation of both students’ perceptions and their practices in relation to exam change. The study identified a refocus of students’ learning activities following the changes in the




HKCEE examination. However, students’ motivation and learning strategies were found to remain largely unchanged.

In recent years, more attention has been paid to test impact on students’ learning prac-tices or test preparation activities. Ferman (2004), in a study investigating the washback of an oral examination on teaching and learning, found that intensive learning for the test is prevalent among students, and that low ability students tend to learn for the test more inten-sively than their high ability peers, believing that cramming can help them achieve a better score. Also, not surprisingly, more students with low language ability than with high lan-guage ability turn to private tutors for help. Such findings have resonance in Gosa’s study (2004), which reports that students feel a strong need to practise exam tasks intensively and actually do so in their personal learning environment. Stoneman’s study (2006) provides further evidence that students faced with a high-stakes exam tend to choose activities mainly intended for test orientation or test-specific coaching. A new contribution of her study to our knowledge about washback on learners is that students’ past learning and test taking experi-ence has a major influence on their choice of the types of test preparation activities.

Meanwhile, in the Hong Kong context where major examination changes are often intended to bring about positive washback on teaching and learning, research thus far has shown that examinations influence students both positively and negatively in terms of student learning. For example, the introduction of the Use of English1 oral examination as a requirement for university admission appeared to lead to general improvements in students’ spoken performance; however, some students’ inappropriate use of transitional words and discourse markers seems to indicate a rote-learning of exam-targeted strate-gies and formulaic phrases rather than meaningful internalization (Andrews, Fullilove, & Wong, 2002). What needs further research in the Hong Kong context is the reason why students think memorization can help them to cope with a speaking exam. In other words, it is important for us to know how students understand test demands (Green, 2007) and whether/how the understanding is related to the test preparation practices they undertake.

The studies discussed above suggest that test washback on learners does not seem to be any simpler than that on teachers. Tests used as an agent to promote desirable changes in learners and their learning may not necessarily be efficient tools for change and may not have the predicted consequences (Qi, 2007; Stoneman, 2006). The unpredictability of test washback on learners may be due to our lack of understanding about learners’ beliefs and expectations in a testing situation (Stoneman, 2006).

Another area that lacks empirical research is test washback on parents. The limited research in the literature shows that very often parents see the evaluative and normative values in the test results more than anything else (James, 2000; Scott, 2007). According to Scott (2007), most parents have very little understanding of what tests usually entail and what the test information they receive actually means. James (2000) argues that the function of tests in reporting to parents about their child’s learning progress remains unfulfilled. There are reports from teachers that parents do not seem to be interested in knowing about assessment or in being involved (Desgorges, Hughes, & Holden, 1994; Gipps, McCallum, & McAlister, 1995). However, contrary to what teachers believe, par-ents have been found to have interest in knowing more about assessments (Desgorges et al., 1994), and to feel responsible for helping their children prepare for exams (Mulvenon, Stegman, & Ritter, 2005).



Cheng et al. 5

The relationship between parental involvement and student progress and development has been widely noted in education (Bogenschneider, 1997; Ho & Douglas Willms, 1996; Hoover-Dempsey et al., 2005). In the Hong Kong context, it has been found that students’ better performance in reading literacy is associated with higher levels of social and educational communication at home and greater parental investment in cultural and educational resources (HKPISA Centre, 2008). Meanwhile, family involvement in their children’s education in Hong Kong is found to be related with parents’ socioeconomic status, measured by parental occupation and education (Ho, 2006). Based on the research discussed above, it is worth exploring the relationships among parents’ exam-related knowledge, their socioeconomic status, parental involvement, and student learning within an examination context.

Given that the benefits of parental involvement for student progress and development have been widely noted in the literature (Bogenschneider, 1997; Ho & Willms, 1996; Hoover-Dempsey et al., 2005), this is hardly surprising. Educators should therefore think about questions such as what role parents can play in preparing their children for high-stakes testing, and what can be done to help parents to understand what tests mean (Ediger, 2000).

Conceptualized within the existing literature, the present study was designed to exam-ine the influence of SBA on both students and parents, and whether an interaction pattern exists between these two stakeholders as far as test washback is concerned. A student and a parent survey were conducted. In addressing the specific research purposes noted ear-lier on the study was guided by the following four research questions:

1. To what extent did SBA impact on students a. with different levels of self-reported language competence? b. at different stages of their secondary schooling?2. To what extent did views of SBA and external examinations differ among students a. as a whole group? b. with different levels of self-reported language competence?3. To what extent was the amount of SBA-related support provided by parents to

their children (DV²) associated with a. aspects of the parents’ social/educational background (IVs)? b. the parents’ views about SBA (IVs)?4. To what extent were parents’ views about SBA (three factors) and the SBA-

related support they provide to their children (one factor) associated with a. their children’s views about SBA? b. their children’s SBA-related English learning activities?

Question 3a focused specifically on parents’ level of education, the type of housing they lived in, and the amount of time they spent with their children daily. In addressing ques-tion 3b, the following aspects of parents’ perceptions/views of SBA were investigated: their perceived knowledge about SBA; their perceived opportunities to know about SBA; their perceptions of SBA; their perceptions of the impact of SBA on their children’s skill development and motivation to learn; and their perceptions of negative effects on their children.




Method

Participants

The participants of the two surveys were from two Hong Kong secondary schools. One school is an aided co-educational school located in Kowloon. English is used as the medium of instruction.³ The school has mostly a Band 14 student intake. Most ofthe students are from low-middle or lower class families. The other school, located in the New Territories, is also aided and co-educational. The medium of instruction used in this school, however, is Chinese. The student intake is mostly Band 2 or Band 3, from low-middle or lower class families. 389 Secondary 4 students (n = 389) and their parents (n = 315) completed the questionnaires. As noted earlier, Secondary 4 is the grade where the students start to be involved with SBA. This involvement continues until they reach the end of Secondary 5 where they take the Hong Kong Certificate of Education exami-nation (HKCEE) in English. SBA comprises of 15% of the HKCEE final mark.

InstrumentsTwo questionnaires were designed for this study. Both questionnaires were designed and administrated in English and Chinese. The students’ questionnaire (SQ) consisted of six sections with 76 questions. Almost all questions were designed on a 6-point Likert-scale of agreement or frequency apart from the demographic information. The main sections of the SQ are as follows:

• gender, type of housing, parents’ level of education, family employment of domestic helper;

• students perceptions of their own SBA-related English competence;• students’ views about SBA;• SBA-related English activities conducted outside and inside class;• students’ perceptions about the challenges of SBA; and• students’ views about SBA and external examinations.

The parents’ questionnaire (PQ) consisted of four sections with 30 questions also on a 6-point Likert-scale. They are as follows:

• gender, age, level of education, type of housing, time spent with their children;• parents’ views about SBA;• parents’ support to their children in SBA; and • parents’ perceptions of the effects of SBA on their children.

Data collection and analysisThe students’ questionnaires were completed in class with the support of their English teachers with a return rate of 100%. The parents’ questionnaires were taken home by the



Cheng et al. 7

students, completed by their parents, and handed back to students’ teachers with a return rate of 82%. Both questionnaires were issued in November 2007, towards the end of the students’ first term of Secondary 4. The students’ and parents’ questionnaires were coded so the data of the two surveys could be linked and analyzed.

To answer the research questions of the study, three data analyses were conducted. First, descriptive analyses of the demographic variables were conducted for both ques-tionnaires. Second, exploratory factor analyses were conducted to plot the major con-structs of the two questionnaires. Principal component analyses with varimax rotation were conducted. Third, independent sample and paired sample t-tests and multiple regressions were used to explore the relationships within and across the two surveys. Sequential step-wise regression procedures were used controlling for demographic variables. For all of our statistical analyses, we used a critical value of a = .05.

Results

Descriptive analyses

A total of 389 students completed the SQ. Among them, 50.1% were male and 49.9% were female. 67.2% of the participants lived in public housing, that is, government-subsided housing, while 24.2% of them lived in privately owned housing. 95.6% of the participants mentioned that their family did not have domestic helpers. These last two results indicate that our sample was drawn from schools whose students mainly come from a low socioeconomic background. According to the SQ, 52.5% and 46.9% respec-tively of the students’ mothers and fathers graduated from secondary schools. However, a number of the students reported that they did not know about their parents’ education: 15% did not know about their mother’s education and 20.5% about their father’s educa-tion. As to students’ self-reported language competence, 47.6% of them perceived them-selves as having low English competence; and 52.4% high English competence.

A total of 315 parents completed the PQ. Among them, 28.3% were male and 71.7% were female. As with other similar surveys of parents in educational research, it was the mothers (71.7%) in our study who mostly answered the survey. 71.4 % of respondents were aged 40 or above. 69.5% of them reported living in public housing. This result was very similar to that reported by the students. Among the parent respondents, 61% reported hav-ing completed secondary education; and 32.4% primary education. 44.4% of these parents mentioned spending about an hour every day with their children. Surprisingly, 42.2% of the parent respondents claimed to spend less than 30 minutes with their children every day.

Factor analysisIndependent exploratory factor analyses using principal component analyses with var-imax rotation were conducted on both questionnaires. Principal component factoring and the varimax method were used in the factor analysis because they maximize vari-ance and help the interpretation of constructs deduced (Gorsuch, 1983). The varimax solution seeks ‘the rotation position where the variance is maximized across all factors




in the matrix’ (Gorsuch, 1983, p. 185). Furthermore, given the arbitrary nature of the question of how many factors to extract, the Kaiser Criterion, that is, eigenvalues greater than 1 (Crocker & Algina, 1986) and the scree plot were used to explore and ensure meaningful interpretability of the solutions and compare the relative model fit-ness and construct representation. Factor loadings lower than .3 were deleted and were not counted toward any factors. Meaningful interpretations were sought when double loadings occurred.

The factor structure of SQ with Cronbach’s a value and the number of items was as follows (see Appendix 1 for all questionnaire items and factor loadings):

• Perceived language competence (SQ section II) – single factor (Cronbach’s α = .86).

• Students’ views about SBA (SQ section III) – six factors:

1. perceived impact of SBA (α = .88) n = 7 with two sub-dimensions: a. perceived impact of SBA on motivation to learn (α = .87); b. perceived impact of SBA on the promotion of independent learners

(α = .79); 2. knowledge about SBA (α = .90) n = 6; 3. negative impact of SBA (α = .65) n = 3; 4. advantages of SBA (α = .55) n = 3; 5. impact of SBA on skill development (α = .87) n = 7; 6. nature of SBA in theory and practice (α = .63) n = 4.

• SBA-related English activities outside and inside class (SQ section IV) – three factors for activities outside class and five factors for activities inside class:

1. Reading, Writing, and Listening (α = .87) n = 8; 2. Speaking (α = .77) n = 5; 3. Watching TV and movies (α = .83) n = 2; 4. Secondary 3 activities (α = .88) n = 7; 5. Secondary 4 activities (α = .88) n = 7; 6. Secondary 3 feedback (α = .95) n = 4; 7. Secondary 4 feedback (α = .95) n = 4; 8. Secondary 3/4 teaching in English (α = .59) n = 2.

• Their perceived challenges of SBA (SQ section V) – one single factor (α = .64) n = 4.

• Their views about SBA and external examinations (SQ section VI) – two factors:

1. views on SBA (α = .83) n = 8; 2. views on external examinations (α = .79) n = 6.



Cheng et al. 9

Table 1. Six-factor rotated component matrix for SQ component 2: Students’ views about SBA (see Appendix 1 for actual items)

Item Factor Communality

1 2 3 4 5 6

13 .71 .6414 .82 .7415 .82 .7211 .33 .3818 .62 .5820 .72 .6821 .63 .571 .54 .352a .84 .782b .86 .792c .87 .812d .85 .783 .79 .6522 .66 .4523 .77 .6224 .75 .608 .85 .789 .82 .7317 .30 .405 .63 .556 .66 .6010 .57 .4916a .49 .6616b .63 .7316c .61 .7316d .58 .664 .49 .557 .52 .4712 .51 .4519 .55 .53Eigenvalue 9.73 2.86 2.09 1.55 1.20 1.04Variance explained (%) 32.43 9.55 6.98 5.26 4.01 3.46Accumulated variance explained (%) 32.43 41.98 48.96 54.13 58.14 61.61

Table 2. 3-factor rotated component matrix for SQ component 3: SBA-related activities outside the class (see Appendix 1 for actual items)


1 2 3

25a .56 .5725b .73 .6625c .63 .5626a .85 .7626b .86 .76

(Continued)




Table 2. (Continued)


1 2 3

26c .67 .5026d .47 .4227a .68 .6227b .66 .6227c .71 .6028 .45 .3829 .57 .4930 .80 .6931 .79 .6832 .58 .47Eigenvalue 6.18 1.55 1.03Variance explained (%) 41.25 10.39 6.91Accumulated variance explained (%) 41.25 51.64 58.55

Table 3. Five-factor rotated component matrix for SQ component 3: SBA-related English activities inside class (see Appendix 1 for actual items)


1 2 3 4 5

33a .60 .6033b .60 .6136 .61 .6937 .63 .6738 .69 .5739 .77 .7140 .69 .6041a .76 .6941b .82 .7544 .55 .6045 .54 .5646 .72 .6047 .75 .7248 .72 .6535a .84 .8235b .79 .8335c .84 .8435d .82 .8143a .88 .8643b .85 .8543c .88 .8643d .87 .8534 .77 .6542 .71 .64Eigenvalue 8.80 3.94 2.19 1.29 1.09Variance explained (%) 36.67 16.40 8.35 4.98 4.56Accumulated variance explained (%) 36.67 53.07 61.42 66.41 70.96



Cheng et al. 11

The factor structure for PQ is as follows (see Appendix 2).

• Their views about SBA (PQ section II) – three factors: 1. knowledge about SBA (α = .96) n = 6; 2. opportunities to know about SBA (α =.92) n = 4; 3. perceptions of SBA (α = .79) n = 5.• Their support for their children in SBA (PQ section III) – one single factor

(α = .87) n = 7.• Their perceptions of the effects of SBA on their children (PQ section IV) – three

factors: 1. impact on skill development (α = .97) n = 4; 2. impact on motivation to learn (α = .90) n = 5; 3. negative effects on their children (α = .82) n = 4.

Students’ perceived competence and aspects of SBA (RQs 1 and 2)It is known in the washback literature that tests may have washback effects for some learners and some teachers, but not for others (Alderson & Wall, 1993) and that tests may affect different learners differently (Cheng, Klinger, & Zheng, 2007).

With this in mind, we first used independent sample t-tests to explore the differ-ences of those with low and high perceived language competence – in terms of their views about SBA, the activities they reported engaging in inside and outside class, their perceptions of the challenges of SBA, and their views about SBA and external examinations (four major sections of the questionnaire: SQ III, IV, V, and VI). We

Table 4. Two-factor rotated component matrix for SQ component 5: Students’ views about SBA and external examinations (see Appendix 1 for actual items)


1 2

56 .65 .4058 .75 .6060 .78 .6162 .74 .5463 .54 .3765 .51 .3366 .63 .4968 .72 .5455 .65 .4057 .76 .6159 .67 .6161 .68 .4964 .64 .45Eigenvalue 4.95 1.95Variance explained (%) 35.37 13.98Accumulated variance explained (%) 35.37 49.36




conducted t-tests by the four components of SQ according to the results of factor analysis reported above (18 factors in total). To reduce type I error, we adjusted the alpha value divided by the number of factors for each component, for example component 2 p = .007

Table 5. Three-factor rotated component matrix for PQ component 1: Parents’ views about SBA (see Appendix 2 for actual items)


1 2 3

1 .73 .652a .84 .792b .85 .832c .86 .832d .85 .823 .76 .664 .81 .725 .80 .696 .80 .687 .74 .628 .74 .739 .84 .7910 .88 .8211 .87 .8212 .83 .78Eigenvalue 8.51 1.58 1.14Variance explained (%) 56.75 10.53 7.59Accumulated variance explained (%) 56.75 67.29 74.87

Table 6. Three-factor rotated component matrix for PQ component 3: Parents’ perceptions of the effects of SBA on their children (see Appendix 2 for actual items)


1 2 3

20a .87 .8620b .87 .8520c .90 .8720d .90 .8921 .80 .7122 .82 .7423 .81 .7624 .71 .6725 .58 .6926 .69 .5127 .85 .7528 .87 .7729 .78 .64Eigenvalue 6.06 2.47 1.20Variance explained (%) 46.58 18.97 9.22Accumulated variance explained (%) 46.58 65.54 74.76



Cheng et al. 13

(.05/7), component 3 p = .006 (.05/8), and component 5 p = .025 (.05/2). We did not adjust component 4 as there was only one factor.

Students with higher perceived competence, compared with students with lower com-petence, held stronger views about SBA with one exception on the advantages of SBA (see Table 7); did more English activities inside and outside class; felt more strongly about external examinations than SBA, but felt less concerned about the perceived chal-lenges of SBA. Significant differences were found in 15 out of the 18 factors listed above. The non-significant factors were negative impact of SBA, advantage of SBA, and the nature of SBA in theory and practice – all related with students’ view about SBA.

Table 7. T-test results on perceived competence and aspects of SBA

Variables Group N Mean SD F-value p-value

Perceived impact of SBA on motivation to learn

low 185 3.33 .92high 204 3.67 .92 .239 .002

Perceived impact of SBA on the promotion of independent learners

low 185 3.33 .87high 204 3.65 .90 .002 .000

Knowledge about SBA low 185 2.88 .91high 204 3.52 .93 .152 .000

Negative impact of SBA low 185 3.73 1.04high 204 3.84 .91 3.09 .267

Advantages of SBA low 185 3.94 1.05high 204 3.67 .93 3.92 .008

Impact of SBA on skill development low 185 3.38 .85high 204 3.73 .79 2.45 .000

Nature of SBA in theory and practice

low 185 3.86 .84high 204 4.07 .82 .000 .016

Reading, writing, and listening low 185 2.15 .66high 204 2.90 .84 13.35 .000

Speaking low 185 1.92 .66high 204 2.68 .91 17.71 .000

Watching TV and movies low 185 3.65 1.17high 204 4.32 1.07 2.01 .000

Secondary 3 activities low 185 2.98 .91high 204 3.72 .91 .47 .000

Secondary 4 activities low 185 2.99 .92high 204 3.46 .97 .089 .000

Secondary 3 feedback low 185 3.20 1.23high 204 3.77 1.17 1.39 .000

Secondary 4 feedback low 185 3.80 1.10high 204 4.12 1.11 .086 .006

Secondary 3/4 teaching in English low 185 4.67 1.14high 204 5.03 .99 3.45 .001

Perceived challenges of SBA low 185 3.56 .83high 204 3.41 .70 4.80 .045

Views on SBA low 185 3.44 .90high 204 3.64 .82 .58 .021

Views on external examinations low 185 3.37 .92high 204 3.81 .90 .03 .000




Students with lower competence perceived the advantages of SBA more compared with students with higher competence, but the result was not significant.

To explore the impact of SBA on students at different stages of their schooling, our analysis focused specifically on whether students did more SBA-related activities inside and outside class the closer they came to the time of SBA. Two paired sample t-tests were conducted with the alpha value adjusted to p = .025 (.05/2). Results indicated that there were significant differences relating to the activities conducted in English in two forms (S3 and S4). Although students’ perceptions of these activities were correlated (r = .43 for S4; r = .36 for S3), SBA related activities were reported to be more frequently conducted in S4 than in S3 – indicating a potential impact (washback) closer to the assessment event (SBA in this case). This is not surprising and lends support to the research question.

In order to explore whether students see the SBA and traditional external examina-tions differently (RQ 2), a paired sample t-test was run to see how the students perceived the external examinations and SBA with the alpha value adjusted to p = .025 (.05/2). Overall, students (on the whole, regardless of their perceived language competence) agreed more with the items relating to external examinations (Mean = 3.60) than the items relating to SBA (Mean = 3.54). The two factors, that is, external examinations and SBA, also share a significant and moderate correlation (r = .42, p = .000) but the mean comparison of the two constructs did not show significant differences (p = .229) (see Table 9). In other words, students’ perceptions of SBA are significantly correlated with their perceptions of external examinations, but their perceptions of the two are not significantly different. This may suggest that for these students SBA is simply another exam that they have to prepare for.

Interestingly, however, when the results were examined in order to compare students’ views according to their own perceived language competence, differences emerged. The responses to external examinations and SBA among students with high perceived

Table 8. T-test results on perceived activities and their stages of schooling

Variables N Mean SD t p-value

Pair 1 Secondary 3 activities 389 3.24 .97 −2.51 .013Secondary 4 activities 389 3.37 .98

Pair 2 Secondary 3 feedback 389 3.50 1.24 −6.95 .000Secondary 4 feedback 389 3.97 1.12

Table 9. T-test results on students’ views to SBA and external examinations

Variables N Mean SD t p-value

Whole Views about SBA 389 3.54 .87 −1.21 .229Views about external examinations 389 3.60 .94

Low Views about SBA 185 3.44 .90 .87 .385Views about external examinations 3.37 .92

High Views about SBA 204 3.63 .82 −2.61 .010Views about external examinations 3.81 .90



Cheng et al. 15

language competence were significantly different (m = −.17, t = −2.61, p = .010), but this was not the case for students with low perceived language competence (m = .064, t = .87, p = .385). Moreover, the students with low perceived language competence responded more positively to the items relating to SBA (Mean = 3.44) than the items on external examinations (Mean = 3.37). By contrast, the students with high perceived language competence responded more positively to the items on external examinations (Mean = 3.81) than to the items relating to SBA (Mean = 3.63).

Parents’ personal characteristics and aspects of SBA (RQ 3)Research has demonstrated that parents’ involvement plays a beneficial role in their chil-dren’s school progress and development. In order to answer research question 3a – the association between the amount of SBA-related support provided by parents to their children (DV) and parents’ level of education, their type of housing, and the time they spend with their child daily (IVs) – multiple regressions were conducted. It turned out that parents’ level of education (ß = .27) and the time they spend with their children (ß = .20) made significant contributions to parents’ support for their children in SBA, explaining 11.6% of the variance. This indicates that parents’ level of education and the time they spend with their child daily (not the type of housing they live in) play a signifi-cant role in their support for their children in SBA.

We then examined research question 3b, exploring first of all the association between the amount of SBA-related support provided by parents to their children (DV) and parents’ perceived knowledge about SBA, perceived opportunity to know about SBA, and perceptions of SBA (IVs). The results showed that all three aspects of parents’ views about SBA made a significant contribution to parents’ support for their children to a varying degree (ß = .24; .27; .12) – explaining 31.4% of the variance. This indicates that what parents know about the school reform/innovation, in this case, SBA, contributed to their support for their children in schools. The opportunity for the parents to know about SBA played the most important role among the above three variables.

We then focused on the association between the amount of SBA-related support pro-vided by parents to their children (DV) and parents’ perceptions of the effects of SBA on their children’s skill development, on their children’s motivation to learn, and their per-ception of negative effects of SBA on their children (IVs). The results showed that only parents’ perceptions of the effects of SBA on their children’s motivation to learn (ß =. 43) made a significant contribution to parents’ support for their children – explaining 18.5% of the variance. This indicates that when the parents perceived that SBA could motivate their children to learn, they would give more support to their children in doing SBA. Given the widespread scepticism in Hong Kong about the appropriateness of SBA when it was first introduced, as reported in South China Morning Post (e.g. J. Ho, 2005, 7 May; Pang, 2006, 14 January), it is surprising that parents’ perceptions of negative effects of SBA on their children did not even enter the regression model in the presence of the other two variables.

The results above support the findings in the research literature that parents play a ben-eficial role in their children’s education. In the context of SBA in Hong Kong, this suggests




the importance of fully informing parents about this assessment innovation in order to enhance the success of its implementation and eventually support student learning.

Relationship between the students’ and parents’ surveys (RQ 4)Many impact/washback studies have pointed out the importance of understanding the role that different stakeholders play in an assessment event (Desgorges et al., 1994). However, few have studied students and parents together and investigated the relation-ship between the two key stakeholders within an assessment context. In order to achieve this research purpose, we explored two research questions by analysing the student and parent surveys together (by linking a parent’s and their child’s responses together).

Research question 4a investigated the extent to which parents’ views about SBA (PQII) (three factors, i.e. parents’ perceptions of SBA, perceived impact on (their chil-dren’s) skill development, and perceived impact on (their children’s) motivation to learn), and parents’ support for their children regarding SBA (PQIII) (one factor) are associated with their children’s (students’) views about SBA. We conducted three sets of regression analyses on students’ views about SBA (SQIII): their perceptions of the impact of SBA 1) on motivation to learn; 2) on the promotion of independent learners; and 3) on their (English language) skill development.

First, after controlling the demographic variables (i.e. parents’ education and student gender), how much could parents’ views about SBA and parents’ support in SBA (4 IVs) explain students’ perceived impact of SBA as motivation to learn (DV)? Only parents’ perceptions of the effects of SBA on students’ motivation to learn entered the model suc-cessfully. The model explains 15.6% variance of students’ perceived impact of SBA as motivation to learn. This indicates that parents’ perceptions of the effects of SBA on students’ motivation to learn have a significant and direct relationship with their chil-dren’s perceived impact of SBA on their motivation to learn.

Second, after controlling the same demographic variables, how much could parents’ views about SBA and parents’ support in SBA (4 IVs) explain students’ perceived impact of SBA on the promotion of independent learners (DV)? Parents’ perceptions of the effects of SBA on student’s motivation to learn and on their skill development both entered the model successfully. The model explains 12.9% variance of students’ per-ceived impact of SBA on the promotion of independent learners. In this sense, both of the parents’ perceived effects of SBA were related with their children’s views of the impact of SBA on their being independent learners. In other words, when the parents believe SBA to be beneficial for their children’s motivation to learn and their skill development, their children will hold similar beliefs.

Third, after controlling the same demographic variables, how much could parents’ views about SBA and parents’ support in SBA (4 IVs) explain students’ perceived impact of SBA on their skill development (DV)? Parents’ perceptions of the effects of SBA on student’s motivation to learn and on their skill development both entered the model suc-cessfully. The model explains 17.8% variance of students’ perceptions of the impact of SBA on their skill development. Again parents’ views in these two aspects contributed to their children’s perceptions of the impact of SBA on their skill development. This is the model that explained the most variance among the above three. In short, aspects of parents’



Cheng et al. 17

views about SBA are directly associated with their children’s (students’) views about SBA, to a varying degree: on students’ perceptions of the impact of SBA on their motivation to learning, on the promotion of independent learners, and on their skill development.

Then we addressed research question 4b, investigating the extent to which the par-ents’ views about SBA (the same three factors as in RQ 4a) and parents’ support to their children in SBA (the same factor as in RQ 4a) are associated with students’ SBA-related English learning activities, specifically: 1) students’ reading, writing and listening activi-ties; 2) speaking activities; and 3) TV and movie watching activities in English. In other words, this time we explored whether parents’ perceptions are associated with what their children do (their reported practices inside and outside class).

First, after controlling the same demographic variables, how much could parents’ views about SBA and parents’ support to their children in SBA explain students’ SBA reading, writing and listening activities in English? Parents’ perceptions of the effects of SBA on student’s motivation to learn, skill development, and parental support were selected into the final model successfully. The whole model was able to explain 11.6% variance of the students’ reading, writing, and listening activities.

Second, after controlling the same demographic variables, how much could parents’ view about SBA and parents’ support to their children in SBA explain students’ SBA speaking activities in English? Parents’ perceptions of the effects of SBA on student’s motivation and parental support were selected into the final model successfully. The whole model was able to explain 10.2% variance of the students’ SBA speaking activities.

Third, after controlling the same demographic variables, how much could parents’ views about SBA and parents’ support for their children in SBA explain students’ SBA-related TV and movie watching activities in English? Only parents’ perceptions of SBA were selected into the final model successfully. The whole model explained 2.1% vari-ance of the students’ TV and movie watching activities, which is very little.

It is worth pointing out that the factor variable of parents’ support to their children in SBA had entered the model in two out of three sets of analyses regarding research ques-tion 4b, but not in any of the three models above regarding research question 4a, when we explored the association between parents’ views about SBA and their children’s views about SBA. This result indicates there could be a unique relationship in parents’ perceptions and what their children say they do. In addition, results from the above six models show that more variance was explained for the relationship between parents’ views and their children’s views than for the relationship between parents’ views and what their children say they do.

DiscussionThe impact/washback of examinations on their stakeholders (especially students) is not a new phenomenon and was documented as early as 1877 (Latham, 1877). By compari-son, empirical research in this area is fairly recent (Wall & Alderson, 1993). This study focused on researching the two major stakeholders (students and parents) in a recent assessment innovation – the introduction of school-based assessment into a major large-scale high-stakes high school leaving examination in English – the HKCEE – within the Hong Kong education context. The study also investigated students’ and parents’




responses separately as well as the relationship between students’ and parents’ responses to the school-based assessment.

Impact of SBA on studentsThe study first of all tested the hypotheses raised by Alderson and Wall (1993) that a test will influence learning; a test will influence how learners learn; tests will have washback effects on all learners and teachers; and tests will have washback effects for some learners and some teachers, but not for others. The results of our study showed that different students – those with self-reported high and low English competence – perceived SBA differently. The results are more evident in terms of the activities students did inside and outside class and perceived challenges of SBA but less so regarding their views about SBA with three out of seven factors non-significant. The students with higher perceived competence, compared to students with lower compe-tence, held stronger views about SBA; did more English activities inside and outside class; felt more strongly about external examinations and SBA, but felt less concerned about perceived challenges of SBA. This result suggests that testing (in this case SBA) influences students with different levels of competency differently – at least in terms of their perceptions. This result is supported by research in another context (Cheng, Klinger, & Zheng, 2007) where different groups of students (students who speak English as L1 and L2) have differential performance on certain test tasks and formats. The differential testing influence on different students may well suggest to us as educa-tors that, when planning a testing event or an assessment innovation as in this research context of SBA, we need to address the issue of students at various levels of educational achievement and realize that the event or innovation may affect certain group(s) more than the others.

The differential effects on different students are only partially supported by our next set of results when we explored how students viewed SBA and external examinations. Although the rationale behind SBA is to move toward assessment for learning and to use summative assessment to support learning (Davison, 2007), our results showed that stu-dents viewed SBA and external examinations similarly. The difference in their percep-tions about SBA and external examinations was not significant. Further, students responded more positively to the items about the external examinations than the parallel set of items relating to SBA. The reason for a non-significant result could be that the study was conducted at the beginning of SBA and with Secondary 4 students when they had recently started SBA in their class. When we explored students’ perceptions about the SBA and external examinations further by comparing the views of students with high/low perceived language competence, there was a statistically significant difference in how students with high perceived language competence viewed SBA and external examinations, but this was not the case for students with low perceived language com-petence. This might suggest greater sensitivity toward the assessment change among students with high perceived language competence. However, students with high per-ceived language competence responded more positively to the items relating to the external examinations while students with low perceived language competence responded more positively to the items relating to SBA. These results are not altogether



Cheng et al. 19

surprising, as SBA is designed to help students with low perceived language competence to maximise their achievement as is the case with any assessment for learning attempt (Black & Wiliam, 1998).

Previous washback research has demonstrated that the closer the students are to the time of the examination, the more impact will be seen – seasonality of washback (Bailey, 1999; Cheng, 2005). Our results support this, showing that activities inside and outside class conducted in English in two forms (Secondary 3 and 4) were significantly different but significantly correlated. More SBA related activities were frequently conducted in Secondary 4 than in Secondary 3. This result has implications for researchers when deciding the best time to conduct impact/washback related research.

Impact of SBA on parents The findings from the parent survey indicate that parents’ education (ß = .27) and the time they spend with their children (ß = .20) play a role in the amount of support they provide for their children in SBA – explaining 11.6% of the variance. The contribution is small yet can be potentially important. Further, this statistical positive significant association is consistent with other research evidence (Bogenschneider, 1997, Ho & Willms, 1996, Hoover-Dempsey et al., 2005). Within the Hong Kong context where time is extremely highly valued, the result of a correlation between the time spent with their children and the amount of support parents give to their children is not surprising, but affirming. In addition, the findings show parents’ knowledge about SBA, opportu-nity to know about SBA, and perceptions of SBA were also associated with the amount of support parents provide for their children in relation to SBA. The three aspects of parents’ views about SBA made significant contribution to parents’ support to their children – explaining 31.4% of the variance. Further, the results demonstrate parents’ perceptions of the impact of SBA on their children’s motivation to learn (in the presence of two other non-significant variables: of their children’s skill development and of negative effects on their children) were associated with the amount of support parents provide for their children in doing SBA – explaining 18.5% of the variance. This indi-cates parents’ knowledge about the SBA has the strongest association with their support for their children.

Relationship between parents and studentsAs mentioned earlier, previous research has examined stakeholders separately within a testing and assessment context, but few have looked at the relationship of stakeholders within the same context. The present study examined the relationship of two groups of key stakeholders – students and their parents by linking the two survey data sets. The results show parents’ perceptions about the SBA are directly and significantly related to their children’s perceptions about SBA. In other words, if the parents view SBA to a certain degree, their children will view it proportionally as well. This result clearly indi-cates the necessity of communicating what happens in school to parents so they can best support their children. The result shows that parents’ perceptions also have similar asso-ciation with what their children do inside and outside class though to a lesser extent.




Conclusions

Taking all the results from both the students’ and parents’ surveys together, we have gained a better understanding of the complexity of the impact of SBA as perceived by students and their parents within the Hong Kong context.

A few limitations of the study need to be addressed to support the accurate interpreta-tions of the results derived from this study. First, our data is self-reported data via two surveys, that is, perception data of the two stakeholders on what they see, what they understand, what they do, and what they think their language competence is. Second, the study was conducted at the beginning of the introduction of school-based assessment in HKCEE English in Hong Kong. Considering the long tradition of large-scale high stakes external examinations in Hong Kong (this SBA being the first time when teachers’ marks are counted toward a student’s secondary school graduation marks in English), the results stand at the time of the research6 and need to be interpreted with caution. Third, the data were obtained from two out of more than 400 secondary schools in Hong Kong, so the results offer us a snapshot of these two schools. Fourth, while the t-tests may show that a statistical difference exists, it does not tell us how important the difference may be. In a number of cases, the means were close and the standard deviations were relatively large. This indicates that our data again need to be interpreted with caution and within the context of this study.

Despite its limitations, the study is one of few studies that have provided separate and combined (and linked) results from two major stakeholders – students and parents – on the impact of school-based assessment. This study is also timely in education given the worldwide movement to combine assessment of learning with assessment for learning. The closeness of the relationship between the perceptions of these two stakeholders within the Hong Kong assessment context suggests that this is an educational phenomenon that merits further investigation within other contexts worldwide.

Acknowledgements

The present paper is part of the project entitled ‘An Impact Study of a High-Stakes ESL Assess-ment Innovation in Hong Kong Secondary Schools’ (HKU7483/06H). The authors would like to thank the Research Grant Council (RGC) of Hong Kong for their funding support. We also acknowledge that some of the questionnaire items in this study were adapted from the question-naire used in the project entitled ‘A longitudinal study of the implementation of the school-based assessment component in the 2007 HKCE English Language Examination’ commissioned by HKEAA, with the agreement of Professors Chris Davison and Liz Hamp-Lyons, the Primary Investigators of that project. Our appreciation also extends to Sophia Ling Li for her help with the analysis of the survey data, and to Wendy Leung for her support in this project.

Notes

1 The Use of English is the English language examination of the Hong Kong Advanced Level Examination (HKALE), or more commonly known as the A-level, conducted by the HKEAA, and is normally taken at the end of senior secondary schooling, i.e. Secondary 7.

2 DV refers to dependent variable and IVs refer to independent variables. 3 According to the Medium of Instruction Guidance for Secondary Schools issued in 1997,

most schools should use Chinese for teaching all academic subjects. Schools that wish to use



Cheng et al. 21

English as medium of instruction (MOI) should demonstrate to the Education Bureau that the school meets the prescribed requirements.

4 The Secondary School Placement Allocation (SSPA) system at the end of primary schooling determines the ‘band’ of a students’ ability. Altogether there are three bands, each representing 1/3 of the ability range (Education and Manpower Bureau, 2010). Students are placed in different secondary schools based on their ability range. Schools that mainly enrol Band 1 students tend to be labelled as ‘high-banding’ or ‘top-banding’ schools.

5 It is common in Hong Kong that families with professional parents (or those who can afford it) employ domestic helper(s) to take care of their child(ren) and the apartment. Whether a family employs domestic helper(s) is used in this study as a social-economic status variable.

6 This study is part of a larger study in which the students’ and parents’ surveys were issued twice – once when students were at the beginning of Secondary 4 (the data reported in the present paper ) and again near the end of Secondary 5, the data of which is to be reported in a separate paper.

7 There are a number of single factor component structures in the SQ and PQ, i.e. Components 1 and 4 of SQ and Component 2 of PQ. Thus factor loadings are not reported since one factor component was extracted at the initial loading and the solution could not be rotated. In this case, only questionnaire items are provided.

References

Alderson, J. C., & Hamp-Lyons, L. (1996). TOEFL preparation courses: A study of washback. Language Testing, 13(280–297).

Alderson, J. C., & Wall, D. (1993). Does washback exist? Applied Linguistics, 14(2), 115–129.Andrews, S. J. (2004). Washback and curriculum innovation In Y. J. W. L. Cheng, with A. Curtis

(Ed.), Washback in language testing: Research contexts and methods (pp. 37–52). Mahwah, NJ: Lawrence Erlbaum.

Andrews, S. J., Fullilove, J., & Wong, Y. (2002). Targeting washback: A case study. System, 30, 207–233.Bailey, K. M. (1999). Washback in language testing. TOEFL Monograph 15. Princeton, NJ:

Educational Testing Service.Black, P., & Wiliam, D. (1998). Assessment and classroom learning. Assessment in Education:

Principles, Policy & Practice, 5(1), 7–74.Bogenschneider, K. (1997). Parental involvement in adolescent schooling: A proximal process

with transcontextual validity. Journal of Marriage and the Family, 59(3), 718–733.Chapman, D. W., & Snyder, C. W. (2000). Can high stakes national testing improve instruction:

Reexamining conventional wisdom. International Journal of Educational Development, 20, 457–474.

Cheng, L. (1998). Impact of a public English examination change on students’ perceptions and attitudes toward their English learning. Studies in Educational Evaluation, 24(3), 279–301.

Cheng, L. (1999). Changing assessment: Washback on teacher perceptions and actions. Teaching and Teacher Education, 15(253–271).

Cheng, L. (2005). Changing language teaching through language testing. Cambridge: Cambridge University Press.

Cheng, L. (2008). Washback, impact and consequences. In E. Shohamy & N. H. Hornberger (Eds.), Encyclopedia of language and education, Vol. 7: Language testing and assessment (2nd edn, pp. 349–364). Berlin: Springer Science + Business Media.




Cheng, L., Klinger, D., & Zheng, Y. (2007). The challenges of the Ontario Secondary School Literacy Test for second language students. Language Testing, 24(2), 185–208.

Crocker, L., & Algina, J. (1986). Introduction to classical and modern test theory. New York: Saunders College Publishing.

Davison, C. (2007). Views from the chalkface: School-based assessment in Hong Kong. Language Assessment Quarterly, 4(1), 37–68.

Desgorges, C., Hughes, M., & Holden, C. (1994). Parents’ and teachers’ perceptions of assess-ment at Key Stage 1. In M. Hughes (Ed.), Perceptions of teaching and learning. Clevedon: Multilingual Matters.

Ediger, M. (2000). High-stakes testing and student achievement. Retrieved 15 December 2008, from ERIC: http://eric.ed.gov/ERICWebPortal/custom/portlets/recordDetails/detailmini.jsp?_ nfpb=true&_&ERICExtSearch_SearchValue_0=ED447193&ERICExtSearch_SearchType_0= no&accno=ED447193

Education and Manpower Bureau (2010). Information Leaflet on Secondary School Places Alloca-tion System 2008/2010 Cycle. Retrieved 27 August 2010 from www.edb.gov.hk/FileManager/EN/Content_1579/bilingual_e_5paged.pdf

Ferman, I. (2004). The washback of an EFL national oral matriculation test to teaching and learning. In L. Cheng, Y. Watanabe, with A. Curtis (Ed.), Washback in language testing: Research con-tents and methods (pp. 190–210). Mahwah, NJ: Lawrence Erlbaum.

Gipps, C., McCallum, B., & McAlister, S. (1995). Intuition or evidence? Teachers and national assessment of seven-year-olds. Buckingham: Open University Press.

Gosa, C. M. C. (2004). Investigating washback: A case study using student diaries. Unpublished PhD dissertation, Lancaster University.

Gorsuch, R. L. (1983). Factor analysis. Hillsdale, NJ: Lawrence Erlbaum.Green, A. (2007). Washback to learning outcomes: A comparative study of IELTS preparation and

university pre-sessional language courses. Assessment in Education, 14(1), 75–97.Hamp-Lyons, L. (1997). Washback, impact and validity: Ethical concerns. Language Testing,

14(3), 295–303.Henrichsen, L. E. (1989). Diffusion of innovations in English language teaching: The ELEC effort

in Japan, 1956–1968. New York: Greenwood Press.HKEAA (2006). HKCE English language examination: Introduction to the school-based assess-

ment component. Hong Kong: Hong Kong Examinations and Assessment Authority.HKPISA Centre (2008). The third HKPISA report PISA 2006 executive summary. Hong Kong:

HKPISA Centre, Chinese University of Hong Kong.Ho, E. S. (2006). Social disparity of family involvement in Hong Kong. School Community Journal,

16(2), 7–26.Ho, J. (2005, 7 May). We didn’t dare argue with the teachers. South China Morning Post, p. EDU 4. Ho, E. S., & Willms, J. D. (1996). Effects of parental involvement on eighth-grade achievement.

Sociology of Education, 69(2), 126–141.Hoover-Dempsey, V. K., Walker, J. M. T., Sandler, H. M., Whetsel, D., Green, L. C., Wilkins, A.

S., et al. (2005). Why do parents become involved? Research findings and implications. The Elementary School Journal, 106(2), 105–130.

Hughes, A. (1993). Backwash and TOEFL 2000. Unpublished manuscript, University of Reading, UK.



Cheng et al. 23

James, M. (2000). Measured lives: The rise of assessment as the engine of change in English schools. The Curriculum Journal, 11(3), 343–364.

Latham, H. (1877). On the action of examinations considered as a means of selection. Cambridge: Deighton, Bell.

Linn, R. (1993). Educational assessment: Expanded expectations and challenges. Educational Evaluation and Policy Analysis, 15(1), 1–16.

Linn, R. (2000). Assessment and accountability. Educational Researcher, 29, 4–16.Mulvenon, S. W., Stegman, C. E., & Ritter, G. (2005). Test anxiety: A multifaceted study on the

perceptions of teachers, principals, counselors, students, and parents. International Journal of Testing, 5(1), 37–61.

Pang, P. (2006, 14 January). Teachers groping in the dark on reform. South China Morning Post, p. EDU4.

Qi, L. (2005). Stakeholders’ conflicting aims undermine the washback function of a high-stakes test. Language Testing, 22(2), 142–173.

Qi, L. (2007). Is testing an efficient agent for pedagogical change? Examining the intended washback of the writing task in a high-stakes English test in China. Assessment in Education, 14(1), 51–74.

Richards, J. C. (2001). Curriculum development in language teaching. Cambridge: Cambridge University Press.

Scott, C. (2007). Stakeholder perceptions of test impact. Assessment in Education, 14(1), 27–49.Shohamy, E., Donitsa-Schmidt, S., & Ferman, I. (1996). Test impact revisited: Washback effect

over time. Language Testing, 13(3), 298–317.Stoneman, B. W. H. (2006). The impact of an exit English test on Hong Kong undergraduates: A

study investigating the effects of test status on students’ test preparation behaviors. Unpub-lished PhD dissertation, Hong Kong Polytechnic University, Hong Kong SAR.

Wall, D. & Alderson, J. C. (1993). Examining washback: The Sri Lankan impact study. Language Testing, 10, 41–69.

Watanabe, Y. (1996). Does grammar translation come from the entrance examination? Preliminary findings from classroom-based research. Language Testing, 13(3), 318–333.




Appendix 1. Component structure of student questionnaire (SQ)

Components SQ items and item loadings (in italics)

Component 1Perceived SBA-related language competence (SQ Section II) – single factor7 (Cronbach's a = .86)

How would you rate your ability to do the following in English? Please respond to the following statements and circle your answers on the rating scale of 1 to 4, where 1 = very weak, and 4 = very good.

• My ability to read English books is• My ability to understand English movies without

Chinese subtitles is• My ability to make an individual oral presentation

in English is • My ability to take part in a group discussion in

English is Component 2Students’ views about SBA (SQ section III) – six factors1. Perceived impact of SBA

(a =.88) n = 7 with two sub-dimensions: perceived impact of SBA on motivation to learn (a = .87) and Perceived impact of SBA on the promotion of independent learners (a = .79)

Item #13. I think SBA encourages me to read more English books (.79).

14. I think SBA encourages me to watch more English movies or TV programs (.88).

15. I find learning English for SBA is motivating as I can read novels and watch movies (.84).

11. I think self-assessment enhances my learning (.87).18. I think SBA helps students to become more

independent learners (.56).20. I think SBA motivates me to improve my English (.53).21. I think SBA helps me to identify my strengths and

weaknesses in speaking (.59).2. Knowledge about SBA (a

= .90) n = 6 1. I understand why SBA is being added to the HKCE

English exam (.54). 2. I have a good understanding of…

a) the requirements of SBA (.84) b) the procedures of SBA (.86)c) the marking criteria of SBA (.87)d) the moderation system of SBA (.85)

3. I have had sufficient opportunity to discuss SBA with my classmates (.77).

3. Negative impact of SBA (a = .65) n = 3

22. I think SBA has a negative effect on students’ attitudes to studying English (.66).

23. I think SBA increases my study workload excessively (.77).

24. SBA makes me nervous (.75).4. Advantages of SBA

(a = .55) n = 3 8. I think SBA disadvantages students with lower

language skills (.85). 9. I think SBA mainly benefits students with higher

language skills (.82).17. I think I perform better orally in English when talking

to my friends than to complete strangers (.30).



Cheng et al. 25


5. Impact of SBA on skill development (a = .87) n = 7

5. I think SBA results in fair judgment of each student’s level of speaking skills (.63).

6. I think SBA gives a fairer assessment of students’ speaking ability than the current HKCE English oral exam (.66).

10. I think peer-assessment enhances my learning (.57).16. I think SBA helps improve my

a) speaking skills (.49)b) reading skills (.63)c) listening skills (.61)d) writing skills (.58)

6. Nature of SBA in theory and practice (a =.63) n = 4

4. I think my English speaking skills can be assessed better by my teacher than by external examiners (.69).

7. I think it is good that students have more than one opportunity to be assessed (.52).

12. I think detailed feedback from my teacher can help me improve my language skills (.51).

19. I think SBA helps shy or nervous students to gain higher marks than the current HKCE English oral exam (.55).

Component 3SBA-related English activities outside and inside class (SQ section IV) SBA-related English activities outside class – three factors1. Reading, Writing, and

Listening (α = .87) n = 825. I read the following outside school:

a) English books (.56) b) English magazines (.73)c) English newspapers (.63)

26. I spend time on English entertainment outside school, for example:c) listening to English radio programs (.67) d) surfing English websites (.47)

27. I do the following activities outside school:a) writing letters in English (.68) b) writing emails in English (.66) c) writing diaries in English (.71)

2. Speaking (α = .77) n = 5 28. I talk to my friends in English (.45). 29. I talk to members of my household (e.g. family members,

domestic helper) in English (.57).30. I talk to the native English teacher(s) in English outside class (.80).31. I take part in English extra-curricular activities (e.g. English drama,

inter-class English debates, English corner/room activities) (.79).32. I make use of the resources in the school library to improve

my English in my free time (e.g. audio tapes, video tapes, VCD, references, etc) (.58).

Appendix 1. (Continued)

(Continued)






3. Watching TV and movies (α = .83)n = 2

26. I spend time on English entertainment outside school, for example:a) watching English TV programs (.85) b) watching English movies (.86)

SBA-related English activities inside class – five factors4. Secondary 3

activities (α= .88)n = 7

33. In Form 3 English lessons, we had the following activities:a) group discussions (.60)b) individual presentations (.60)

36. In Form 3 English lessons, we did self-assessment in various language tasks, (e.g. speaking, writing, etc.) (.61)

37. In Form 3 English lessons, we did peer-assessment in various language tasks (e.g. speaking, writing, etc.) (.63)

38. In Form 3 English lessons, we used a variety of materials(e.g. films, TV, news, recordings, songs, etc.) which were not from the textbooks (.69).

39. In Form 3, we had follow-up activities after reading the ERS books (e.g. story sharing, book reports) (.77).

40. In Form 3, we kept a journal/log book about our reading/viewing for ERS (.69).

5. Secondary 4 activities (α = .88) n = 7

41. In Form 4 English lessons, we have the following activities: a) group discussions (.76) b) individual presentations (.82)

44. In Form 4 English lessons, we do self-assessment in various language tasks, for example, speaking, and writing (.55)

45. In Form 4 English lessons, we do peer-assessment in various language tasks, for example, speaking, and writing (.54).

46. In Form 4 English lessons, we use a variety of materials (e.g. films, TV, news, recordings, songs, etc.) which are not from the textbooks (.72).

47. In Form 4 English lessons, we have follow-up activities after reading the ERS /SBA books (e.g. story sharing, book reports) (75).

48. In Form 4, we keep a journal/log book about our reading/viewing for ERS/SBA (.72).

6. Secondary 3 feedback (α = .95) n = 4

35. Our Form 3 English teacher gave us feedback on how to improve the following language skills

a) reading skills (.84) b) speaking skills (.79) c) writing skills (.84) d) listening skills (.82)

7. Secondary 4 feedback (α = .95) n = 4

43. Our Form 4 teacher gave us feedback on how to improve the following language skills:

a) reading skills (.88) b) speaking skills (.85) c) writing skills (.88) d) listening skills (.87)



Cheng et al. 27



8. Secondary 3/4 teaching in English (α = .59) n = 2

34. Our Form 3 English lessons were conducted almost entirely in English (.77).

42. Our Form 4 lessons are conducted almost entirely in English (.71).

Component 4Their perceived challenges of SBA (SQ section V) – one single factor (α = .64) n = 6

49. Our school does not have sufficient books and VCD/DVDs for SBA.

50. We find it difficult to understand the content of the SBA books/movies.

51. Students may memorize responses/performances and reproduce them for the SBA.

52. Teachers have the knowledge and skills to implement SBA.

53. Students and parents may not trust teachers’ assessment in SBA.

54. We do not have adequate class time for carrying out SBA tasks.

Component 5Their views about SBA and external examinations (SQ section VI) – two factors1. Views on SBA (α = .83) n = 8

56. Students like SBA (.62).58. Doing SBA is a valuable learning experience (.76).60. SBA pushes students to study harder (.76).62. A student’s score on SBA is a good indication of how

well she or he will be able to apply what has been learned (.72).

63. External examinations should NOT be used as a sole determinant of student grades (.55).

65. Students perform better in SBA than in the Paper 3 speaking part of the current HKCEE (.54).

66. I think that doing SBA will help me do better on the Paper 3 speaking part of the HKCEE (.62).

68. All students work hard to achieve their best in SBA (.72).

2. Views on external examinations (α = .79) n = 6

55. Students like external examinations (.63). 57. Taking external examinations is a valuable learning

experience (.73).59. External examinations push students to study

harder (. 71).61. A student’s score on an external examination is a good

indication of how well she or he will be able to apply what has been learned (. 65).

64. Students perform better in an external exam situation than in a normal teaching situation (. 67).

67. All students work hard to achieve their best in external examinations (.62).

(Continued)




Appendix 2. Component structure of parent questionnaire (PQ)

Components PQ items and factor loadings (in italics)

Component 1Their views about SBA (PQ section II) – three factors1. Knowledge about SBA

(α = .96) n = 6Item #1. I understand why SBA is being added to the HKCE

English exam (.64). 2. I have a good understanding of

a) the requirements of SBA (.73).b) the procedures of SBA (.84).c) the marking criteria of SBA (.86).d) the moderation system of SBA (.85).

3. I know the weighting of the SBA component in HKCEE (.76).

2. Opportunities to know about SBA (α = .92) n = 4

4. I have had sufficient opportunities to discuss SBA with my child (.82).

5. I have had sufficient opportunities to discuss SBA with my child’s English teacher (.80).

6. I have had sufficient opportunities to discuss SBA with other parents (.80).

7. I have had sufficient opportunities to read/view HKEAA support materials for parents (.74).

3. Perceptions of SBA (α = .79) n = 5

8. I think SBA results in fair judgment of each student’s level of speaking skills (.74).

9. I think SBA can help shy or nervous children to gain higher marks (.84).

10. I think SBA disadvantages students with lower language skills (.89).

11. I think SBA mainly benefits students with higher language skills (.87).

12. I think SBA gives a fairer assessment of students’ speaking ability than the current HKCE English oral exam (Paper 3) (.83) .

Component 2Their support for their children in SBA (PQ section III) – one single factor (a = .87) n = 7

13. I talk to my child in English. 14. I talk to my child about the SBA books he/she has read.15. I talk to my child about the SBA films he/she has

watched.16. I encourage my child to read more English books or

magazines at home.17. I try to watch some English TV programs or movies with

my child.18. I buy extra English books or magazines for my child in

addition to those required by the school. 19. I help my child to prepare for the SBA assessment tasks

(for example, going through his/her scripts, rehearsing with him/her).



Cheng et al. 29


Components PQ items and factor loadings (in italics)

Component 3Their perceptions of the effects of SBA on their children (PQ section IV) – three factors1. Impact on skill

development (a = .97) n = 4

20. School-based Assessment helps improve my child’s a) speaking skills (.87) b) reading skills (.87) c) listening skills (.90) d) writing skills (.90)

2. Impact on motivation to learn (a = .90) n = 5

21. My child reads more English books in S4 than before (.80).

22. My child watches more English programs in S4 than before (.82).

23. My child is becoming more interested in English in S4 than before (.81).

24. I find my child motivated to prepare for the SBA (.71).25. I think SBA helps my child to know better about his/her

strengths and weaknesses in speaking (.56). 3. Negative effects on their

children (a = .82) n = 426. I think SBA has a negative effect on my child’s attitudes

to studying English (.69). 27. I think SBA has increased my child’s study workload

excessively (.85).28. I think SBA makes my child nervous (.87).29. I think SBA makes me nervous (.78).



students' and parents' views of sba in hong kong - citeseerx

Documents