Full Terms & Conditions of access and use can be found athttp://www.tandfonline.com/action/journalInformation?journalCode=ncal20

Download by: [Orta Dogu Teknik Universitesi] Date: 15 November 2015, At: 02:18

Computer Assisted Language Learning

ISSN: 0958-8221 (Print) 1744-3210 (Online) Journal homepage: http://www.tandfonline.com/loi/ncal20

A washback study on e-portfolio assessment in anEnglish as a Foreign Language teacher preparationprogram

Shao-Ting Alan Hung

To cite this article: Shao-Ting Alan Hung (2012) A washback study on e-portfolio assessment inan English as a Foreign Language teacher preparation program, Computer Assisted LanguageLearning, 25:1, 21-36, DOI: 10.1080/09588221.2010.551756

To link to this article: http://dx.doi.org/10.1080/09588221.2010.551756

Published online: 17 Jan 2012.

Submit your article to this journal

Article views: 992

View related articles

A washback study on e-portfolio assessment in an English as a Foreign

Language teacher preparation program

Shao-Ting Alan Hung*

Department of English, National Kaohsiung First University of Science and Technology,Kaohsiung, Taiwan

Washback refers to both positive and negative influences of testing on teachingand learning. While washback studies abound in the literature, most of them dealwith high-stakes, standardized tests or large-scale entrance examinations. Scantefforts have been made to uncover washback effects in alternative assessments.This study implemented an e-portfolio project as an alternative assessmenttechnique in a language teacher preparation content course and explored thepositive and negative washback effects that e-portfolio assessments produced onlearning. Eighteen English as a Foreign Language (EFL) student teachers in agraduate course of a Master’s program in Teaching English to Speakers of OtherLanguages participated in this assessment project. Data were collected throughmultiple instruments, including interviews, observations, document analysis, andreflective journals. The findings suggest that e-portfolio assessments generatepositive washback effects on learning, including building a community of practice,facilitating peer learning, enhancing learning of content knowledge, promotingprofessional development, and cultivating critical thinking. However, e-portfolioassessments also bring some negative washback effects, such as learning anxietyderiving from larger audiences, and resistance to technology. Finally, it isconcluded that EFL professionals make judicious use of e-portfolio assessmentsto balance classroom assessments and to facilitate the learning of contentknowledge.

Keywords: e-portfolio; washback; classroom assessment; teacher education

Background

Washback refers to ‘‘the effect of testing on teaching and learning’’ (Hughes,2003, p.1). More specifically, it is generally known as the positive or negativeinfluences tests have on teachers’ instruction and students’ learning. A number ofstudies have explored washback effects that standardized exams have brought tolanguage learning and teaching. For instance, Watanabe (1996) examined theeffect of the university entrance examination on the use of the grammar-translation method in Japan and found that the entrance examination failed toplay any significant role in the choice of teaching methodology. Rather, it wasteacher factors that dictated how the course would be taught. Cheng, Klinger &Zheng (2007) also conducted a washback study to investigate the impact of a

*Email: alanhung123@gmail.com

Computer Assisted Language Learning

Vol. 25, No. 1, February 2012, 21–36

ISSN 0958-8221 print/ISSN 1744-3210 online

� 2012 Taylor & Francis

http://dx.doi.org/10.1080/09588221.2010.551756

http://www.tandfonline.com

Dow

nloa

ded

by [

Ort

a D

ogu

Tek

nik

Uni

vers

itesi

] at

02:

18 1

5 N

ovem

ber

2015

large-scale literacy test on second language (L2) students in Canada. The resultsindicated that the reading test formats, text types, skills, strategies, and writingtasks impacted L2 and L1 learners differently and significantly. Finally, Aldersonand Hamp-Lyons (1996) discovered that the Test of English as a ForeignLanguage (TOEFL) affected language teachers on the content of instruction andteaching strategies. In sum, a number of washback studies have been conductedto investigate the influences of testing on teachers, teaching, and textbooks(Cheng, 2005; Read & Hayes, 2003; Qi, 2005).

However, these washback studies all targeted large-scale, standardized tests, suchas entrance exams and proficiency tests; none of them probed the washback of small-scale, classroom-based, alternative assessments, such as portfolio assessments.Cheng and Curtis (2004) described the current trends in assessment as experiencing aprimary paradigm shift from standardized testing to alternative assessment as auseful means to evaluate students’ abilities in accomplishing specific language tasks.With this being the case, the study attempted to bridge the gap by exploring whetherwashback effects exist for such assessment practices.

E-portfolios in teacher training programs

Portfolios refer to a purposeful collection of students’ work that documents theirprogress over time (Hancock, 1994; O’Malley & Valdez Pierce, 1996). The artifactsin the portfolios range from writing samples, reading logs, reflections, and peers’comments to teachers’ feedback. Portfolios provide an opportunity for learners tomonitor their own progress and take responsibility for meeting goals. In recent years,with the advent of technology in language teaching and learning, portfolios havemoved from manila folders to an online environment to make distributing studentwork relatively easy. That is to say, electronic portfolios (e-portfolios) are morereadily available and more portable than paper-based portfolios. According toBarrett (2000), an e-portfolio incorporates electronic technologies that help theportfolio developers to collect and organize artifacts in a variety of formats.Furthermore, it is argued that the multimedia possibilities in the e-portfolio systemsmake student work seem more sophisticated, so students see their learning as moreimportant and take greater pride in it (Pullman, 2002). Specifically, when studentsare engaged in the processes of deciding on the portfolios’ purposes, analyzing theaudience, as well as examining and selecting artifacts to be included, they are usuallyinvolved in metacognitive goal setting and reflective self-assessment practices.

E-portfolios have been widely used in teacher training programs to help teacherscollect and organize artifacts and demonstrate professional growth. As defined byMacDonald, Liu, Lowell, Tsai, and Lohr (2004, p. 1), electronic portfolios serve as‘‘multimedia environments that display artifacts and reflections documentingprofessional growth and competencies’’. Similarly, they are also shown to encouragepersonal self-reflection and facilitate pedagogical knowledge and teaching practice(Darling-Hammond & Snyder, 2000; Sung, Chang, Yu, & Chang, 2009). Moreover,a number of benefits of e-portfolios for learning have been proposed. For example,e-portfolios offer opportunities to assess both the writing process and the product, aswell as opportunities for reflection, revision, and collaboration (Pullman, 2002).Furthermore, since an electronic portfolio expands writing to include creative workin sound, images, and hypertexts, it presents the possibility of a new literacy,‘‘e-literacy’’ (Hawisher & Selfe, 1997).

22 S.-T.A. Hung

Dow

nloa

ded

by [

Ort

a D

ogu

Tek

nik

Uni

vers

itesi

] at

02:

18 1

5 N

ovem

ber

2015

Several efforts have been made to research how e-portfolios work as a learning toolin teacher training programs. MacDonald et al. (2004) carried out a case study,investigating graduate-level preservice teachers’ perspectives on the development ofelectronic portfolios. The findings suggest that the most commonly reported discoverywas that the sharing and peer review portion of the class led to the most learning. Inother words, seeing other people’s work was noted as a continuous process for revision,reselection of artifacts, and interface design. Briefly put, participants went through agreat deal of reflection and spent much time in redesign and reselection of portfolioentries. Yang (2009) investigated the use of blogs as a reflective platform in the Englishas a Foreign Language (EFL) student teachers training program. The results indicatedthat student teachers were active in discussing language teaching theories and how thesetheories can be used in real classrooms. Student teachers also critically reflected on theirlearning and made valuable comments. Next, Sung et al. (2009) uncovered theeffectiveness of digital teaching portfolios in an in-service teacher training program. Theresults showed that most teachers demonstrated moderate levels of reflection, but onlyone-third of them showed the highest level of reflection. They also found that e-portfolios with self-assessment, peer assessment, discussion, and journal writing mightenhance the professional growth of teachers. Finally, van Olphen’s (2007) studyfocused on language teacher candidates’ views on developing digital portfolios andconcluded that digital portfolios could document evidence of students’ progress andoffer an opportunity for reflective thinking. In summary, these studies used e-portfoliosas learning tools to enhance teachers’ professional development. However, usinge-portfolios as an assessment tool still remained under-explored.

Using portfolios as an assessment tool

Hirvela and Pierson (2000) note that to improve the field of assessment, educatorsand researchers should search for ways to measure student achievement and seek amodel of assessment that would build on students’ strengths rather than highlighttheir weaknesses. Thus, portfolio assessment has become an appropriate form ofevaluation (Camp & Levine, 1991). Today, portfolios are at the center of manydiscussions on classroom pedagogy and assessment. As Hancock (1994, p. 3) posits,‘‘portfolio assessment is an ongoing process involving the student and teacher inselecting samples of student work for inclusion in a collection’’. Hence, it becamecentral to involve students in decisions about which pieces of their work to assess.

Portfolio-based assessment, by nature, stands for an alternative approach tolanguage assessment. Specifically, it combines learning and assessment. According toHirvela and Pierson (2000), portfolio assessment ‘‘enlarges and reshapes the wholenotion of what language assessment can and should do’’ (p. 107). Murphy (1994)also pinpoints that portfolios provide us with the opportunities to make assessmentprocess a learning process, something very different from the conventionalassessment scenarios. Hence, portfolio is perceived both as a learning tool and asan assessment tool (Ok & Erdogan, 2010). On one hand, it functions as a learningtool since it offers feedback for students about their growth and teachers abouttheir classroom practices. On the other hand, it serves as an assessment toolsince it reflects students’ performance and teachers’ self-monitoring on theinstruction. In summary, in the portfolio contexts, assessment and learning areinterwoven to bring more holistic insights into students’ and teachers’professional development.

Computer Assisted Language Learning 23

Dow

nloa

ded

by [

Ort

a D

ogu

Tek

nik

Uni

vers

itesi

] at

02:

18 1

5 N

ovem

ber

2015

Blogs as e-portfolios

Because of the increasing interest in Web 2.0 technologies, blogs have experiencedphenomenal growth in recent years (Churchill, 2009; Godwin-Jones, 2006, 2008).Serving as one type of e-portfolios, blogs enable users to interact with and have theirwork viewed by others inside and outside the classroom (Richardson, 2006) and tomove from being observers to participants. In other words, participatory practice isgreatly emphasized in public blog domains, also called the blogosphere (Bloch,2007). Moreover, Godwin-Jones (2003) asserts that language learners could use apersonal blog as an electronic portfolio to show development over time. Learners aregiven a chance to write for audiences beyond classmates, which encourages them tobe more thoughtful and responsible for what they write. Finally, the archiving ofblog entries enhances learners’ reflection and cultivates metacognitive strategies formonitoring the learning process (Richardson, 2006).

Purpose of the study

Although efforts have been made to research e-portfolio assessment in teachertraining programs, few empirical studies have been conducted in EFL teachertraining contexts (Yang, 2009). Furthermore, most washback studies have addressedhigh stakes, standardized tests, or entrance examinations; there is little explorationof washback effects of classroom-based, alternative assessment, e-portfolio assess-ment, in particular, on EFL student teachers’ professional development. Therefore,this study set out to probe the washback effects of e-portfolio assessment onprospective EFL teachers’ learning of content knowledge.

Research questions

(1) Does the implementation of e-portfolio assessment produce any positivewashback on content learning?

(2) Does the implementation of e-portfolio assessment produce any negativewashback on content learning?

(3) What pedagogical implications can be drawn?

Method

The setting and participants

The e-portfolio assessment project was implemented in a graduate-level content coursein a national university located in the southern part of Taiwan. The content course,Language Assessment, aimed to provide prospective EFL teachers with an overview,theories, and practices of language assessment and to enhance their skills of developingeffective language tests and assessment instruments. The course met for one three-hoursession each week, for 18 weeks. The participants were 18 full-time first-year graduatestudents in a Master’s program with a concentration on Teaching English to Speakersof Other Languages (TESOL). Among them, five were males and 13 were females. Theages ranged from 24 to 35 years. All of the participants had formal or informalexperiences of teaching EFL to children, teenagers, or adults for 1 to 10 years.

24 S.-T.A. Hung

Dow

nloa

ded

by [

Ort

a D

ogu

Tek

nik

Uni

vers

itesi

] at

02:

18 1

5 N

ovem

ber

2015

The e-portfolio website

The e-portfolio website, Wretch, was chosen for the current study for threereasons: (1) ease of use, (2) popularity, and (3) familiarity. First, Wretch is aready-made site that requires no high-tech skills and allows users to uploadartifacts in multiple formats, such as texts, graphics, sound, and video files. It isalso equipped with functions not merely for users to organize and categorize theirentries based on various themes and topics but also for visitors to postindividualized feedback as a medium for asynchronous communication with users.Second, Wretch is a well-known blog site among Taiwanese students. Accordingto statistics by Alexa Internet, Inc., a subsidiary company of Amazon.com knownfor its browsing behavior analyses and web traffic reporting, Wretch is ranked thesecond most visited website among the top 100 sites in Taiwan. Lastly, most ofparticipants in the current study had created their own e-portfolios on Wretch inother classes or in their leisure time. Hence, these three reasons supportedthe selection of Wretch as a platform for participants to maintain theire-portfolios.

Implementation

Table 1 describes the detailed procedures of the e-portfolio project.

Stage 1: project orientation and preparation

In the first two weeks, the prospective EFL teachers set up their e-portfolios atWretch, available at www.wretch.cc. At this stage, participants also learned thefundamental concepts of e-portfolio assessment, including purposes, outcomes,processes, students’ and teacher’s roles, among others. Moreover, to allow forefficient peer feedback, they also formed groups of two to three persons.

Stage 2: implementation

At the second stage, spanning from the third week to the end of the semester,participants worked on their required assignments. The participants were required tocomplete six types of assignments, which are explained as follows:

Table 1. Procedures for implementing e-portfolio assessment project.

Stage Task

I. Project orientation and preparation Setting up e-portfolios at www.wretch.cc(1st to 2nd week) Learning concepts of e-portfolio assessment

Forming portfolio groups of 2–3 persons forpeer commenting

II. Implementation (3rd to 18th week) Working on the assignmentsGiving peer feedback for each assignmentTeacher commenting on the participants’entries

Revising the assignments according to thecomments

Computer Assisted Language Learning 25

Dow

nloa

ded

by [

Ort

a D

ogu

Tek

nik

Uni

vers

itesi

] at

02:

18 1

5 N

ovem

ber

2015

(1) Annotated journal articles: Students read either assigned or self-chosenjournal articles related to the course content, Language Assessment,summarized the articles, reflected on the issues, and proposed classroomimplications in written forms.

(2) Critical responses: The instructor occasionally posed some pedagogical scenariosfor discussion. For instance, one scenario was ‘‘The teacher feels that his studentsrevise what they write only minimally, and he is eager that they learn the value ofrevision. How can be use portfolios to achieve this?’’ The participants respondedto the scenarios and commented on each other’s responses.

(3) Group assessment project: Students worked with their group members andconstructed an assessment project on specified skills, such as listening,speaking, reading, and writing. They applied theories they learned from classinto the projects. The final project consisted of a rationale, target test-takers,test specifications, test items, and grading criteria.

(4) Conference notes: Students attended two assessment-related paper presenta-tions at a conference and wrote one-page conference notes on arguments andfindings of presentations.

(5) Self- and peer-assessment questionnaire: After each annotated journal articlewas completed, participants filled out an open-ended self- and peer-assessment questionnaire, analyzing their own and peers’ strengths, weak-nesses, and areas for future improvement. (See Appendix 1 for self- and peer-assessment questionnaire.)

(6) Reflective journal: At the end of the semester, each student wrote a reflectivejournal, discussing what they learned in this content course. (See Appendix 2for a description of the reflective journal assignment.)

In addition to completing each assignment, participants were required to give peerfeedback on their group members’ assignments. Then, each participant responded tothe feedback or revised the assignments according to the teacher’s and peers’ feedback.After revising each required assignment, they posted their revised assignments ontotheir e-portfolios and named them as revised work to allow for comparison.

In terms of assessment, student teachers’ e-portfolios were constantly reviewedby both peers and the instructor during the semester and rated at the end of thesemester. Announced to all the pre-service teachers at the beginning of the semester,the criteria Included: (1) content of each assignment, (2) contribution of peerfeedback, (3) fulfilment of requirements and (4) timely submission. The grades on e-portfolios accounted for 50% of participants’ total scores.

Instruments

The current study collected data via interviews, observations, and documentanalysis. First, interviewing is an effective means of eliciting information when theresearcher cannot observe participants’ behavior and other visible cues as to theirfeelings or how they interpret the world around them. It is also important to collectdata through interviews when the researchers are interested in past events andexperiences that are impossible to replicate (Merriam, 1998). Therefore, aiming toinvestigate washback effects of e-portfolio assessment on EFL student teachers’learning of content knowledge, the researcher and research assistants conductedthree 30-minute semi-structured interviews with each of the students (see Appendix 3

26 S.-T.A. Hung

Dow

nloa

ded

by [

Ort

a D

ogu

Tek

nik

Uni

vers

itesi

] at

02:

18 1

5 N

ovem

ber

2015

for detailed interview questions). Second, participant observations also allowed theresearcher to learn firsthand how the actions of the participants corresponded totheir words and to observe patterns of behavior. Third, document analysis wasperformed because documents ‘‘corroborate the interviews and thus make thefindings more trustworthy. Beyond corroboration, they may raise questions aboutthe researchers’ hunches and thereby shape new directions for interviews’’ (Glesne,1999, p. 58). In the current study, the documents in students’ e-portfolios includedrequired assignments (i.e., annotated journal articles, critical responses, self- andpeer-assessment questionnaire, conference notes, assessment project, and reflectivejournal), self-chosen entries, peers’ responses, and teacher’s feedback.

Data analysis

The data analysis procedure followed a series of steps in an effort to generate themescapable of providing triangulation information. First, all the qualitative data werereviewed by the researcher and another qualified researcher who holds a doctorate inTESOL. This peer debriefing process served to establish the credibility of theinterpretation of the data. While reviewing the data, a number of codes weregenerated. Second, the researchers re-read the entire data set closely and labelled allthe data with the generated codes. Next, they carefully reviewed the codes and assuch combined the relevant ones into seven themes that represented positive andnegative washback effects. The detailed coding scheme that includes themes, codesand examples is presented in Table 2.

Findings and discussion

Synthesizing all the data sources, the study uncovered a number of washback effectsof e-portfolio assessment on EFL student teachers’ learning of content knowledge.Figure 1 presents all of the positive and negative washback effects derived from thecurrent study.

Positive washback

According to Pearson (1988), assessments’ washback effects will be positive if theyare beneficial and encourage desired changes on teaching, learning, and curriculum.Similarly, Cheng and Curtis (2004) argue that positive washback effects will begenerated when teachers and learners have a positive attitude toward theassessment and work willingly and collaboratively toward assessment and learningobjectives. In the present study, a number of positive washback effects from e-portfolioassessment were discovered, such as building a community of practice, facilitating peerlearning, enhancing learning of content knowledge, promoting professional develop-ment, and cultivating critical thinking. Corresponding to what Cheng and Curtis(2004) posited, these effects increased learners’ willingness to collaborate with peersand provoked desired changes in learning content knowledge.

Building a community of practice

E-portfolios built a small community in which these language teachers could interactthrough reading each other’s entries, discussing ideas, commenting on issues, and

Computer Assisted Language Learning 27

Dow

nloa

ded

by [

Ort

a D

ogu

Tek

nik

Uni

vers

itesi

] at

02:

18 1

5 N

ovem

ber

2015

Table 2. Coding scheme.

Themes Codes Examples

Positive washbackBuilding communityof practice

Learning environment/space

Constant engagement

‘‘E-portfolio is just likea learningspace for us to get to knowmore about this subject. Everyweek we share informationabout class readings,assignments, and projects.’’

Facilitating peer learning Peer interactionPeer feedbackCollaborative learning

‘‘I think e-portfolio is one of theuseful tools for me to learnfrom others. I could readother students’ assignments. Icould learn from their ideasand their writing styles.’’

Enhancing learning ofcontent knowledge

Development ofknowledge onlanguage assessment

Documentation oflearning process

‘‘This approach (portfolio)helped me review my learningprocess on languageassessment. It can show whatstudents have learned aboutthe subject.’’

Promoting professionaldevelopment

Theories into practicesLinkage to classroom

practices

‘‘When I am teaching, I canalways connect theories withteaching practices and try tofind some ideas from thosedocuments.’’

Cultivating criticalthinking

Reflective thinkingThinking from different

perspectives

‘‘I usually reflected on the ideasin my postings and thoughtabout what questions myclassmates would ask andhow to answer theirquestions.’’

Negative washbackAnxiety Audience outside of the class

Worries about writingentries online

‘‘I don’t likeotherpeople to readmy postings because I thinkmy English is poor.’’

Frustration withtechnology

Technical problems ‘‘I feel frustrated when there aresome troubles on the Internetbecause I don’t know how tosolve the problems.’’

exchanging information. This e-portfolio-based community not only enhancedpeer interaction through peer feedback but also exposed learners to differentperspectives. These 18 language teachers reported that e-portfolios created a virtualclassroom for them to discuss language teaching and assessment theories, review andcomment on peers’ assignments and thus take responsibility for their professionaldevelopment through constant engagement. The following excerpts from theinterview and reflective journal illustrate learners’ perceptions of the e-portfoliocommunity:

E-portfolios created a learning space for the class members. In addition to classroomlearning, we were given one more opportunity to exchange information and discusspedagogical issues.

28 S.-T.A. Hung

Dow

nloa

ded

by [

Ort

a D

ogu

Tek

nik

Uni

vers

itesi

] at

02:

18 1

5 N

ovem

ber

2015

Figure 1. Washback of e-portfolio assessment.

E-portfolios were like a place where we could view others’ entries to get some differentviewpoints on the same topics and learn from different perspectives. Sometimes when Iview other peers’ entries, and then I tell myself ‘wow, why didn’t I think of this idea?’

It is clear that this community serves as an open space for language teachers toexplore and describe their ideas. Similarly, it allows language teachers toexperiment with content knowledge and express themselves in a relaxedenvironment. Therefore, as indicated by Wenger (1998), the communityencourages these language teachers to take responsibility for information sharingand problem solving, to develop their personal identities in the community, andto foster unification of the community.

Facilitating peer learning

In the current study, a number of participants revealed that unlike other standardizedtesting where they were not allowed to discuss and collaborate with peers, portfolioassessment permitted them to use peers as resources to facilitate learning. Peerfeedback on participants’ entries was highly regarded and thus encouraged them toimprove the quality of their work. The following excerpts from student interviewsshow participants’ reactions to peer learning in e-portfolio assessment:

I think e-portfolio is one of the useful tools for me to learn from others. I could readother students’ assignments. I could learn from their ideas and their writing styles sothat I can improve my assignments.

In this portfolio approach, I benefited a lot from others’ feedback. For instance, I foundseveral constructive suggestions on my annotated journal articles from classmates. So,when I give feedback, I also need to be more responsible and serious, not justcommenting on others’ assignments carelessly.

Computer Assisted Language Learning 29

Dow

nloa

ded

by [

Ort

a D

ogu

Tek

nik

Uni

vers

itesi

] at

02:

18 1

5 N

ovem

ber

2015

Since e-portfolios empowered participants to give and receive more immediatefeedback from peers to resolve professional issues, they enabled collaborativelearning among peers to take place.

Enhancing learning of content knowledge

The findings also revealed that e-portfolios do not merely enhance peer interaction asmentioned above, but they also enhance participants’ learning of content knowledge.In this graduate course, Language Assessment, participants were introduced to typesof language assessments, principles of constructing language tests, and theories andpractices of large-scaled and classroom-based assessments. After the systematiccollection of assignments in the portfolios, these EFL student teachers found it easierto document their development of content knowledge. The following quotes wereextracted from reflective journal, self- and peer-assessment questionnaires andstudent interviews:

I know more about the content knowledge after a semester of constructing e-portfolio.When I write entries in my portfolio, I have to review the principles of languageassessment mentioned in class and in the textbook. I think my e-portfolio documents mylearning progress in this content course.

Because e-portfolio documents all my entries, it helps me review what I have learned inthis class.

I think my e-portfolio helped me organize my learning in this content course. Forexample, in my e-portfolio, I included the summaries and reflections of some journalarticles under the topic of self-assessment. Every time when I write my assignment, I caneasily review my previous work and the feedback from peers and instructor.

Therefore, it is evident that with the benefit of documentation the e-portfolioassessment project trained the prospective teachers to organize their contentlearning systematically and thus encouraged them to be reflective learners.

Promoting professional development

In addition to content knowledge, e-portfolios also provided EFL teachers with achance to link theories to their classroom practices. A number of them reported thatsince they had collected the learning products in their portfolios, they were able torefer to them when teaching and assessing their students. In other words, theconstruction process of e-portfolios strengthened EFL teachers’ professionalknowledge and development, which can also be found in other assessmenttechniques such as collaborative projects and reflective journals but are hardlyfound in one-shot, decontextualized tests. The following excerpts from reflectivejournals illustrate the participants’ professional development in e-portfolioassessment:

During the learning, I can see my improvement by looking at those documents that Icollected. When I am teaching, I can always connect theories with teaching practices andtry to find some ideas from those documents.

With all the entries in my portfolio, it became easier to review principles and theories oflanguage assessment, like reliability and validity. When I design tests for my Englishclasses, I can go back to my portfolio and improve the tests.

30 S.-T.A. Hung

Dow

nloa

ded

by [

Ort

a D

ogu

Tek

nik

Uni

vers

itesi

] at

02:

18 1

5 N

ovem

ber

2015

Acting as a platform, e-portfolios paved a way for professional development,enabling prospective language teachers to translate knowledge into practice andheighten the level of pedagogical reflection. As noted by Sung et al. (2009), whenteachers observe, review, and discuss the content of portfolios, they learn to clarifytheir previous conceptions about their own practices and think deeper on the meritsand drawbacks of their own work.

Moreover, some of these teacher trainees projected their future use of e-portfolios intheir EFL classrooms. For instance, one noted that she would use e-portfolios in herchildren’s English classes to help pupils collect their learning products such as Englishjournal entries and self-created glossaries. Another described his future plan of using e-portfolios as an assessment tool in a college writing class, saying that he would rateeach of his students’ writing samples as a formative assessment and grade the entire e-portfolio at the end of semester as a summative assessment. Hence, it became clear thatsome teacher trainees were already planning and evaluating how e-portfolios could beintegrated in their future classes.

Cultivating critical thinking

Unlike conventional paper-and-pencil tests that emphasize rote memorization andattainment of discrete skills, e-portfolio assessment cultivates language teachers’critical thinking ability. When giving feedback, they tended to approach the issuesfrom more than one perspective. Meanwhile, when responding to peer feedback onclassroom practices, they did not merely link practices to theories but also discussedforeseeable challenges and raised pedagogical implications. The following excerptsfrom interviews and self- and peer-assessment questionnaires explain how e-portfolioassessment cultivated their critical thinking skills:

Using e-portfolios to evaluate our learning is very different from using tests. E-portfolioassessment encouraged me to think deeper on some learning issues and challenge bothmy own and peer’s perspectives.

In this type of assessment, I usually re-evaluate my assignments in other people’s eyes. Iask myself some questions and try to answer them from different angles.

Negative washback

In the traditional testing context, negative washback effects refer to the undesirableeffects on teaching and learning, such as overemphasis on memorization, practicingexam techniques rather than language learning tasks, unnecessary test anxiety, andfailure to promote general understanding (Alderson & Wall, 1993; Cheng & Curtis,2004). On the other hand, in alternative assessments context, particularly e-portfolioassessment in the current study, negative washback was explored and described asfollows.

Anxiety

While e-portfolio assessment produced positive washback, it also generated somenegative impacts on learning. The first negative washback was anxiety derived fromthe presence of the audience. Since e-portfolios created onWretch are public domain,

Computer Assisted Language Learning 31

Dow

nloa

ded

by [

Ort

a D

ogu

Tek

nik

Uni

vers

itesi

] at

02:

18 1

5 N

ovem

ber

2015

they were visited by not only class participants but also unknown users beyond theclassroom. These language teachers were somewhat concerned about their writtenperformance being observed by people other than the class instructor. Hence,although giving and receiving feedback could enhance interaction and promotecritical thinking, it provoked learning anxiety. The following excerpts frominterviews illustrated language teachers’ increased anxiety:

I know my classmates will always read my postings. I am a little bit worried about thequality of my assignments and the feedback I give to others. I sometimes keep quiet notbecause I don’t have any ideas but because I don’t want all of the members to read mypostings.

I became anxious every time when I post my entries because I don’t know how myclassmates will look at them and how they will evaluate them.

In the traditional tests, learners’ performance is judged only by the teachers; however,in e-portfolio assessment, their performance is observed and evaluated by largerpopulations. When the audience increases, anxiety also increases. In Ward’s (2004)blog-writing study, students reported that they felt that they had to write really goodtopics because everyone would read their works. Worried about a similar issue, the pre-service teachers in the present study experienced a certain level of apprehension.

Fustration with technology

Another negative washback pertained to fustration. Stemming from the technicalchallenges they encountered while maintaining their e-portfolios, pre-service teachers’fustration seemed to lead to resistance against technology. Although they were familiarwith the e-portfolio system, some occasionally encountered technical problems, such asinstability of the Internet, uploading failure and system maintenance. For some learners,these problems seemed to provoke some resistance against technology, which in turnreduced their learning motivation and interest in using e-portfolios in their futurelearning. A few participants indicated their resistance to technology in the interviews:

Sometimes I can’t post my feedback on others’ entries. I tried several times but stillfailed. I had to wait until the next day to dispatch my messages. It’s inconvenient.

The e-portfolio system sometimes can’t display the full text of my assignments properly.It cut out words to the right of every line. I had to do the formatting again and again orspend a lot of time adjusting the formats.

Compared with paper-based assignments, online assignments created more troubles. Ifthe system is down, my assignments can’t be posted on time.

Corresponding to van Olphen’s (2007) argument that technology was not infallible,the current study found that although technology played an important role in theportfolio development process, it could create some frustration for some studentteachers. Hence, solving technical difficulties would need to be addressed before e-portfolio assessment could become effective in a language teacher education program.

Pedagogical implications

Based on the findings, two pedagogical implications are proposed to maximize theefficacy of e-portfolio assessment. First, with all the positive washback effects,

32 S.-T.A. Hung

Dow

nloa

ded

by [

Ort

a D

ogu

Tek

nik

Uni

vers

itesi

] at

02:

18 1

5 N

ovem

ber

2015

e-portfolio assessment is recommended to supplement conventional paper-and-penciltesting in language teacher preparation courses. Since traditional testing solelyevaluates the learning outcome and neglects the process of professional development,employing e-portfolio assessment that encompasses various entries of languageteachers’ growth may lead to a more balanced assessment approach that places equalweight on the process and product of professional development.

Second, the negative washback on anxiety from larger audiences may derive fromparticipants’ discomfort of revealing personal competence on the Internet. In otherwords, these EFL student teachers were worried that their entries might fail to meetpeers’ expectations or their postings may jeopardize their friendship. Hence, to tackletheir concern about overt performance, guidance should be designed to sustaincollaboration and socialization throughout the course. Alternatively, helping thesestudent teachers realize peers as learning resources rather than judges becomescrucial.

Conclusion and limitations

As washback research continues to grow, a number of efforts have investigated howlarge-scale standardized tests impact EFL students’ learning. What seems to belacking pertains to washback effects of classroom-based alternative assessment onlearning. The present study bridged the gap by examining if and what washbackexisted in one particular form of alternative assessments – e-portfolio assessment ina language teacher preparation program. The findings suggest that e-portfolioassessment produces some positive washback effects on learning, such as building acommunity of practice, facilitating peer learning, enhancing learning of contentknowledge, promoting professional development, and cultivating critical thinking.Conversely, e-portfolio assessment also brought some negative washback effects onlearning, such as anxiety from larger audiences and resistance to technology. Basedon the findings, TESOL educators are suggested to make judicious use of e-portfolio assessment to balance classroom assessment and foster socialization ofonline collaboration.

One of the limitations of the study lies in the small number of participants(N ¼ 18). Exploring washback effects of e-portfolio assessment on EFL studentteachers’ learning, the present study focused on a purposeful sample ofone graduate-level class of 18 participants. Although the findings revealedcontextualized meaning in a specific context, they could not necessarily begeneralized to all EFL teachers who have experiences with e-portfolio assessment.Larger sample sizes are needed for future investigation. Another limitationpertains to the compulsory nature of the project. Every EFL student teacher wasrequired to construct his e-portfolio and upload assigned entries in order to fulfillthe requirements of the course. The findings may be different if the projectbecomes voluntary since Sung et al. (2009) state that the effects of using e-portfolios on a voluntary basis will elicit more information on the feasibility andsuitability of using e-portfolios to increase teachers’ interaction in a learningcommunity.

Acknowledgement

My appreciation goes to anonymous reviewers for their comments and suggestions. The projectwas sponsored by National Science Council, Taiwan (Project no. NSC 97-2410-H-327-029).

Computer Assisted Language Learning 33

Dow

nloa

ded

by [

Ort

a D

ogu

Tek

nik

Uni

vers

itesi

] at

02:

18 1

5 N

ovem

ber

2015

Notes on contributor

Dr. Shao-Ting Alan Hung is an assistant professor in the Department of English and co-ordinator of the Foreign Language Program at National Kaohsiung First University ofScience and Technology, Taiwan. His research interests include CALL, L2 writing pedagogyand language assessment.

References

Alexa Internet, & Inc. (2010). Top sites in Taiwan. Retrieved February 2, 2010, from http://www.alexa.com/topsites/countries/TW/

Alderson, J.C., & Hamp-Lyons, L. (1996). TOEFL preparation courses: A case study.Language Testing, 13, 280–297.

Alderson, J., & Wall, D. (1993). Does washback exist? Applied Linguistics, 14, 115–129.Barrett, H. (2000). Create your own electronic portfolio. Learning and Leading with

Technology, 27(7), 14–21.Bloch, J. (2007). Abdullah’s blogging: A generation 1.5 student enters the blogosphere.

Language Learning & Technology, 11(2), 128–141.Camp, R., & Levine, D.S. (1991). Portfolios evolving. In P. Belanoff & M. Dickson (Eds.),

Portfolios: Process and product (pp. 194–205). Portsmouth, NH: Boynton/Cook.Cheng, L. (2005). Changing language teaching through language testing: A washback study.

Studies in Language Testing 21. Cambridge: Cambridge University Press.Cheng, L., & Curtis, A. (2004). Washback or backwash: A review of the impact of testing on

teaching and learning. In L. Cheng, Y. Watanabe, & A. Curtis (Eds.),Washback in languagetesting: Research contexts and methods (pp. 3–18). Mahwah, NJ: Lawrence ErlbaumAssociates.

Cheng, L., Klinger, D., & Zheng, Y. (2007). The challenges of the Ontario Secondary SchoolLiteracy Test for second language students. Language Testing, 24, 185–208.

Churchill, D. (2009). Educational applications of Web 2.0: Using blogs to support teachingand learning. British Journal of Educational Technology, 40(1), 179–183.

Darling-Hammond, L., & Snyder, J. (2000). Authentic assessment of teaching in context.Teaching and Teacher Education, 16, 523–545.

Glesne, C. (1999). Becoming qualitative researchers. New York: Longman.Godwin-Jones, R. (2003). Emerging technologies. Blogs and wikis: Environments for on-line

collaboration. Language Learning & Technology, 17(2), 12–16.Godwin-Jones, R. (2006). Emerging technologies. Tag clouds in the blogosphere: Electronic

literacy and social networking. Language Learning & Technology, 10(2), 8–15.Godwin-Jones, R. (2008). Emerging technologies. Web-writing 2.0: Enabling, documenting,

and assessing writing online. Language Learning & Technology, 12(2), 7–13.Hancock, C.R. (1994). Alternative assessment and second language study: What and why?

Eric Digest, ED376695.Hawisher, G.E., & Selfe, C.L. (1997). Wedding the technologies of writing portfolios and

computers: The challenges of the electronic classrooms. In K.B. Yancey & I. Weiser(Eds.), Situating portfolios: For perspectives (p. 305). Utah State: UP.

Hirvela, A., & Pierson, H. (2000). Portfolios: Vehicles for authentic self-assessment. In G.Ekbatani & H. Pierson (Eds.), Learner-directed assessment in ESL. Mahwah, NJ:Lawrence Erlbaum Associates.

Hughes, A. (2003). Testing for language teachers. New York: Cambridge University Press.MacDonald, L., Liu, P., Lowell, K., Tsai, H., & Lohr, L. (2004). Graduate student

perspectives on the development of electronic portfolios. TechTrends, 48(3), 52–55.Murphy, A. (1994). Writing portfolios in K-12 schools: Implications for linguistically diverse

students. In L. Black, D.A. Daiker, J. Sommers, & G. Stygall (Eds.), New directions inportfolio assessment (pp. 140–156). Portsmouth, NH: Boynton/Cook.

O’Malley, M.J., & Valdes Pierce, L. (1996). Authentic assessment for English language learners:Practical approaches for teachers. White Plains, NY: Addison-Wesley.

Ok, A., & Erdogan, M. (2010). Prospective teachers’ perceptions on different aspects ofportfolio. Asia Pacific Education Review, 11, 301–310.

Pearson, I. (1988). Tests as levers for change. In D. Chamberlain & R.J. Baumgardner (Eds.),ESP in the classroom: Practice and evaluation (pp. 98–107). London: Modern English.

34 S.-T.A. Hung

Dow

nloa

ded

by [

Ort

a D

ogu

Tek

nik

Uni

vers

itesi

] at

02:

18 1

5 N

ovem

ber

2015

Pullman, G. (2002). Electronic portfolios revisited: The efolios project. Computers andComposition, 19, 151–169.

Qi, L. (2005). Stakeholders’ conflicting aims undermine the washback function of a high-stakes test. Language Testing, 22, 142–173.

Read, J., & Hayes, B. (2003). The impact of IELTS on preparation for academic study in NewZealand. In R. Tulloh (Ed.), International English language testing system research reports,4. Canberra, Australia: ITLTS.

Richardson, W. (2006). Blogs, wikis, podcasts, and other powerful Web tools for classrooms.Thousand Oaks, CA: Corwin Press.

Sung, Y., Chang, K., Yu, W., & Chang, T. (2009). Supporting teachers’ reflection and learningthrough structured digital teaching portfolios. Journal of Computer Assisted Learning, 25, 375–385.

van Olphen, M. (2007). Digital portfolios: Balancing the academic and professional needs ofworld language teacher candidates. In M. Kassen, R. Lavine, K. Murphy-Judy, & M.Peter (Eds.), Preparing and developing technolgy-proficient L2 teachers (pp. 265–294). SanMarcos, TX: CALICO.

Ward, J.M. (2004). Blogging assisted language learning (BALL): Push button publishing forthe pupils. TEFL Web Journal, 3(1), 1–16.

Watanabe, Y. (1996). Does grammar translation come from the entrance examination?Preliminary findings from classroom-based research. Language Testing, 13, 318–333.

Wenger, E. (1998). Communities of practice: Learning, meaning, and identity. New York, NY:Cambridge University Press.

Yang, S.H. (2009). Using blogs to enhance critical reflection and community of practice.Educational Technology & Society, 12(2), 11–21.

Appendix 1. Self- and peer-assessment form (for the required assignments)

Name:Peer:

Self assessment

1. What are your strengths in this assignment? Please critically analyze your strengths bypresenting specific examples.

My strengths:

2. What are your weaknesses in this assignment? Please critically analyze your weaknesses bypresenting specific examples.

My weaknesses:

3. How would you make the assignment better? Discuss some suggestions.

Peer assessment Peer’s name:

4. Carefully read your peer’s assignment and point out the strengths and weaknesses. Whatimprovement can be made?

5. Other comments?

Computer Assisted Language Learning 35

Dow

nloa

ded

by [

Ort

a D

ogu

Tek

nik

Uni

vers

itesi

] at

02:

18 1

5 N

ovem

ber

2015

Appendix 2. Reflective journal

Please write a two-paged double-spaced reflective journal to reflect on your overall e-portfolioexperiences. Some guiding questions are suggested but not required, such as (1) What haveyou learned from constructing your portfolio? (2) Which parts of your portfolio did you enjoydoing the most? (3) Which parts didn’t you like to do? (4) Do you think constructing theportfolio helped you? If it did, in what ways? (5) What were the disadvantages? (6) Whatdifficulties did you encounter?, and (7) Do you like this kind of assessment as compared topaper-and-pencil tests?

Appendix 3. Sample interview questions

Sample questions for the first interview (warm-up)

(1) What is your motivation in taking this content course?(2) What are your goals in this content course?(3) How do you improve your learning in content course?(4) Do you use any strategy to help you learn content knowledge?(5) What can you do to improve your content learning?

Sample questions for the second interview

(1) What do you like about creating your portfolio? Why?(2) What do you dislike about creating your portfolio?(3) Do you think if portfolio can document your learning progress in content course? If it

can, in what ways?(4) What do you think about the feedback from your classmates? Are they helpful? Why/

why not?(5) Do you think if portfolios improve peer interaction?(6) Do you prefer being assessed by portfolio or traditional paper-and-pencil tests? Or

both? Why?(7) Do you want to see the use of portfolios in your future content course?(8) Do you think your portfolio help organize your content learning?

Sample questions for the third interview

(1) In your experience, what are some advantages of using the Internet to present yourportfolio?

(2) In your experience, what are some drawbacks of using the Internet to present yourportfolio?

(3) What are some challenges you encountered during this project?(4) Do you often view the artifacts in your portfolio? Can you identify what you can do to

improve your portfolio development?(5) What suggestions can you give to enhance Web-based portfolio assessment?

36 S.-T.A. Hung

Dow

nloa

ded

by [

Ort

a D

ogu

Tek

nik

Uni

vers

itesi

] at

02:

18 1

5 N

ovem

ber

2015