e-portfolio assessment in an online english language course

This article was downloaded by: [Universidad Autonoma de Barcelona]On: 04 November 2014, At: 22:36Publisher: RoutledgeInforma Ltd Registered in England and Wales Registered Number: 1072954 Registeredoffice: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK

Computer Assisted Language LearningPublication details, including instructions for authors andsubscription information:http://www.tandfonline.com/loi/ncal20

E-portfolio assessment in an onlineEnglish language courseMeltem Huri Baturay a & Ayşegül Daloğlu b

a Institute of Informatics, Gazi University , Teknikokullar, Ankara,Turkeyb Department of Foreign Language Education , Middle EastTechnical University , Ankara, TurkeyPublished online: 15 Nov 2010.

To cite this article: Meltem Huri Baturay & Ayşegül Daloğlu (2010) E-portfolio assessment inan online English language course, Computer Assisted Language Learning, 23:5, 413-428, DOI:10.1080/09588221.2010.520671

To link to this article: http://dx.doi.org/10.1080/09588221.2010.520671

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all the information (the“Content”) contained in the publications on our platform. However, Taylor & Francis,our agents, and our licensors make no representations or warranties whatsoever as tothe accuracy, completeness, or suitability for any purpose of the Content. Any opinionsand views expressed in this publication are the opinions and views of the authors,and are not the views of or endorsed by Taylor & Francis. The accuracy of the Contentshould not be relied upon and should be independently verified with primary sourcesof information. Taylor and Francis shall not be liable for any losses, actions, claims,proceedings, demands, costs, expenses, damages, and other liabilities whatsoever orhowsoever caused arising directly or indirectly in connection with, in relation to or arisingout of the use of the Content.

This article may be used for research, teaching, and private study purposes. Anysubstantial or systematic reproduction, redistribution, reselling, loan, sub-licensing,systematic supply, or distribution in any form to anyone is expressly forbidden. Terms &Conditions of access and use can be found at http://www.tandfonline.com/page/terms-and-conditions

http://www.tandfonline.com/loi/ncal20

http://www.tandfonline.com/action/showCitFormats?doi=10.1080/09588221.2010.520671

http://dx.doi.org/10.1080/09588221.2010.520671

http://www.tandfonline.com/page/terms-and-conditions

http://www.tandfonline.com/page/terms-and-conditions

E-portfolio assessment in an online English language course

Meltem Huri Baturaya* and Aysegul Daloglub

aInstitute of Informatics, Gazi University, Teknikokullar, Ankara, Turkey; bDepartmentof Foreign Language Education, Middle East Technical University, Ankara, Turkey

Many teachers have the tendency to look at only the standardized test scores oftheir students while ignoring how or why various dimensions of languageproficiency has improved or not improved. Portfolio, however, reveals a clearpicture of the student’s growth and development. This study reflects thattraditional approaches to assessment of student progress in EFL writing lackindicators of students’ development of the skill and aims to answer the followingquestions: what are the learning gains of students who keep an e-portfolio as partof an elementary-level English course and what are student perceptions about theonline course, using an e-portfolio and their learning gains? For this aim, theresearchers worked with two groups of students: the e-portfolio group who keptan electronic portfolio in the online elementary level English language course andthe traditional assessment group who did not keep an e-portfolio. Although therewere not significant differences between the post-test scores of the two groups,there were significant learning gains in both groups. The perceptions of thestudents in the e-portfolio group reflected that they benefited from and enjoyedkeeping a portfolio.

Keywords: e-portfolio; English language; writing assessment; e-learning

Introduction

Today’s educators favor active involvement of learners in the learning process andstudent-centered ways of teaching, which necessitate the use of alternativeassessment techniques. The constructivist approach to learning and teaching focuseson learners by putting them at the center of the learning process and by giving themthe opportunity to evaluate themselves. Evaluation in constructivism requiresauthentic assessment methods such as portfolio, a technique that reflects studentperformance and provides accurate information about student competency invarious domains of learning.

Portfolio, which is viewed as a practical alternative to standardized testing, is oneof the contemporary techniques as it reflects accurate information about a student’scapabilities within various domains of learning. It creates an atmosphere for student-centered learning, which requires active student involvement. Although a singlemeasurement is incapable of verifying students’ progress on the diversity of skills,

*Corresponding author. Email: [email protected]

Computer Assisted Language Learning

Vol. 23, No. 5, December 2010, 413–428

ISSN 0958-8221 print/ISSN 1744-3210 online

� 2010 Taylor & Francis

DOI: 10.1080/09588221.2010.520671

http://www.informaworld.com

Dow

nloa

ded

by [

Uni

vers

idad

Aut

onom

a de

Bar

celo

na]

at 2

2:36

04

Nov

embe

r 20

14

knowledge, processes, and strategies (Flood & Lapp, 1989), many teachers have thetendency to look at only the standardized test scores of their students by ignoringhow or why each student has improved or has not improved (Yoshida, 2001). Intheir study, Taylor and Walton (1997) question the adequacy of using norm-referenced tests with multiple choice items especially in schools where aconstructivist curriculum is employed. According to them, assessment practicesshould keep pace with the changes in curriculum and instruction.

The literature views the portfolio as a practical alternative to standardized testing(Hiebert & Calfee, 1989; Moya & O’Malley, 1994), which is considered to focus onfactual content rather than real-life application, problem-solving, and creativity(Reckase, 1997). Portfolios capitalize on students’ natural tendency to save work andto take a second look and think about how they could improve future work. Thismethod, therefore, is viewed to be a departure from the other writing methods wherefirst drafts were considered the final products (Zhang, 2009). Standardized tests arenot believed to indicate adequately the competence of a learner in one field whileportfolios assess learners’ performances and/or abilities in different domains of a field.Portfolio assessment is important for EFL learners, because the outcomes of languageproficiency can be assessed effectively, and the observable behaviors gathered throughit provide evidence of students’ acquisition of skills. Confirming this, Apple andShimo (2004) stated that in situations where there are varying proficiency levels ofEnglish language learners, use of a single standardized test is not appropriate to assessindividual learners’ progress in the target language. Moreover, for teaching amultifaceted subject matter like language, it is difficult to represent the skill areas witha single measurement item. As a remedy, portfolio assessment supports the use ofmultiple measures and better reveals the clear picture of the students’ growth anddevelopment (Moya & O’Malley, 1994). In addition to this, Haney and Madaus(1989) pointed out that the portfolio might overcome the limitation of theidentification of learners’ strengths and weaknesses in the target language.

Darling-Hammond (1994) recommended that students should perform real-world tasks and should be evaluated according to their actual performance. Thismight be done effectively with forms of assessment used to monitor student progressand help them work actively toward their own goals (Yoshida, 2001). The portfolioprovides the students with a chance to reflect on their learning gains from the course.It exhibits a learner’s development of problem-solving and critical thinking skills.Learners are actively involved in the decision-making process while the instructor isnegotiating with them the contents of the portfolio. However, a portfolio assessmentis not easy because it requires a paradigm shift regarding the role of teachers,students, and assessment criteria (French, 1992).

As defined by Apple and Shimo (2004), there are three types of portfolios:

(1) Documentation (collection) portfolio: includes all the works of a studentthrough one course.

(2) Assessment portfolio: students systematically select works for assessmentaccording to criteria given by the instructor. Rather than being viewed as afile containing all of a student’s work, a portfolio should, as Moya andO’Malley (1994) point out, include only high-priority information selected bythe student for the purpose of exhibiting to students, parents and otherteachers specific examples of a student’s work, which is particularlyimportant in the assessment of ESL performance.

414 M.H. Baturay and A. Daloglu

Dow

nloa

ded

by [

Uni

vers

idad

Aut

onom

a de

Bar

celo

na]

at 2

2:36

04

Nov

embe

r 20

14

(3) Showcase portfolios: students select only their best work for inclusion in theirportfolios.

Some researchers, such as Hamp-Lyons and Condon (2000), state that thesuccessful portfolios should exhibit collection, selection, and reflection. To them, aportfolio should include a sample of student work, chosen by the student asassessment portfolios defined above. However, whatever the type of portfolio is,learners should be informed about the required contents of their assigned portfoliosin clear statements (Fenwick & Parsons, 1999).

In an online course English course, use of an electronic portfolio is practical anduseful. According to Barret (2000), an electronic portfolio includes the use ofelectronic technologies that allow the teachers and the students to collect andorganize artifacts in many different formats. Furthermore, since electronic portfoliosare not constrained by time, they provide a stimulating environment for teacher andpeer feedback. Therefore, electronic portfolios provide more advantages than theregular portfolios (Hung, 2008).

This study was undertaken based on the observations that the currentapproaches on evaluating student progress in EFL do not include sufficientindicators of student learning with regard to skills. For the most part, assessment isbased on standardized tests that focus on grammar and vocabulary competences, butare unable to provide sufficient indication of a learner’s progress. More than just anexamination score is needed as students should be made aware of their own progressover time so that they, or their instructors, can work on strengthening the weakestparts in the learning chain. In view of the above, by focusing on portfolio assessmentin EFL writing, the study attempted to identify the learning gains of students whokeep an e-portfolio and their perceptions about the portfolio. It aimed to answer thefollowing questions:

. What are the learning gains of students who keep an e-portfolio as part of anelementary-level English course? More specifically,. What are the similarities and differences in the grammar and vocabulary

development of elementary-level EFL students who keep an e-portfolio andthose who do not?

. How do elementary-level EFL students who keep an e-portfolio developtheir writing skills?

. What are student perceptions about the online course, using an e-portfolio andtheir learning gains?

Although there is a wide body of theoretical research regarding the use ofportfolios in EFL classrooms (Lam & Lee, 2010; Nunes, 2004; Zhang, 2009), thereare very few studies on how it has been used by teachers and learners for EFLwriting. This study is believed to reveal how effectively the electronic portfolio worksfor assessment of EFL writing.

The method

Participants

This study was conducted during the 2008–2009 fall semester at a distance educationschool of a public university in Turkey with 58 students, who volunteered to

Computer Assisted Language Learning 415

Dow

nloa

ded

by [

Uni

vers

idad

Aut

onom

a de

Bar

celo

na]

at 2

2:36

04

Nov

embe

r 20

14

participate in the study out of the 75 students enrolled in the online elementary levelEnglish course. There were two groups as e-portfolio (G1) and the traditionalassessment group (G2) in the study. The demographics of two groups of students arepresented in Table 1.

The total online course included 10 units. At the beginning of the course, allparticipants were asked to take a pre-test that aimed to identify their languageproficiency level. The researchers used convenience sampling in the identification ofthe portfolio and the traditional assessment groups and the participants in the e-portfolio group were the students who were ‘‘available and could be easily recruited’’(Johnson & Christensen, 2004, p. 214).

Instruments

Students’ progress and achievement in the course were assessed through pre- andpost-test results and the achievement test scores of both groups and three drafts of a

Table 1. Demographic characteristics of the participants.

G1 G2 Total

GenderFemale 10 15 25Male 17 16 33

Age518 1 2 319–25 19 24 4326–35 6 5 11436 1 – 1

EmploymentUnemployed 13 14 27Full time employed 14 17 31

High school of graduationGeneral high school 3 3 6Foreign language high school 1 – 1Industrial vocational high school 8 11 19Commerce high school 7 5 12Anatolian high school 2 4 6Science high school 1 – 1Other 5 8 13

Level of computer useBeginner – – –Medium 13 14 27Advanced 14 17 31

Year of computer use51 1 – 11–3 4 1 54–7 9 16 2548 13 14 27

The place of Internet accessAt home 25 27 52At work 2 3 5Somewhere else – 1 1


Dow

nloa

ded

by [

Uni

vers

idad

Aut

onom

a de

Bar

celo

na]

at 2

2:36

04

Nov

embe

r 20

14

writing task, which the e-portfolio group students included in their portfolios. Inaddition, in order to get students’ perceptions about the course and e-portfolio use, acourse evaluation survey, students’ foreign language checklist, and self-assessmentform were used. E-portfolio group students were given the writing task aftercompleting three units of the online course. The online post-test included the samequestions as the pre-test and was given to both groups of students at the end of thestudy. The data collection tools are presented in Table 2.

Below are the instruments that were used to collect data in the study.

Data collection tools for both groups

Questionnaire. Demographic information, such as age, sex, the type of high schoolthey graduated from, and computer literacy levels was obtained through aquestionnaire at the beginning of the study.Pre- and post-tests. In order to assess students’ grammar and vocabulary proficiency,the course instructor prepared an online achievement test of 50 multiple-choiceitems.Achievement test. A face-to-face achievement test covered the grammar andvocabulary studied throughout the e-course. This was a face-to-face examadministered on campus at the end of the semester. The items of the test wereprepared by one of the researchers.

Data collection tools for the e-portfolio group

Writing tasks. Students in the e-portfolio group (G1) were asked to submit a draft ofthe writing task once in three units. An informational note about the requiredcontents of the e-portfolio (corrected writing tasks, checklist, self-assessment form,and the course evaluation survey) and the rubric that would be used for assessingstudents’ portfolios was placed on the webpage of the course. Students were free tochoose the topic for their tasks from the ones the instructor presented.

The scoring rubric was adapted from Moya’s holistic scoring criteria whichfocused on the communicative nature of writing (1990, cited in Valdez Pierce &O’Malley, 1992). The scoring range which was 0 to 40 in Moya’s original rubric wasconverted to 0 to 100 for ease of interpretation by the students.

The students’ individual performances were recorded as in a time-series researchdesign. Fraenkel and Wallen (2000) explain that ‘‘a time-series design involvesrepeated measures or observations over a period of time both before and aftertreatment’’ (p. 296). The students’ scores on the writing task drafts were comparedwith their previous scores.

Table 2. Data collection tools.

E-portfolio group Traditional assessment group

Online pre- and post-test Online pre- and post-testFace-to-face achievement test Face-to-face achievement testThree drafts of the writing taskThe course evaluation surveyStudents’ foreign language checklistSelf-assessment form


Dow

nloa

ded

by [

Uni

vers

idad

Aut

onom

a de

Bar

celo

na]

at 2

2:36

04

Nov

embe

r 20

14

The course evaluation survey. Student satisfaction with the online course wasevaluated using a survey which was originally developed by the College ofAgriculture and Life Sciences (CALS) at North Carolina State University (NCSU)to be administered to NCSU students (Lucas, 2007). The survey scale was translatedinto Turkish and piloted on Turkish university students (n ¼ 88) by the researcherbefore using it. A Cronbach’s a value of 0.92 indicated high internal consistency andreliability.Student’s foreign language checklist. This instrument was adapted from theStudent’s Foreign Language Standards Checklist prepared by the National CapitalLanguage Resource Center (NCLRC), a consortium of Georgetown University,George Washington University, and the Center for Applied Linguistics. Thischecklist measures which English language competences the students improve byseeking their perceptions regarding e-portfolio use in the course.Student self-assessment form. This instrument was a combined version oftwo scales developed from Apple and Shimo (2004) and Gagliano and Swiatek(1999). The scale by Gagliano and Swiatek (1999) which was a four-pointLikert scale was taken as the basis for the one used in this study. The aim of thisinstrument was to determine if students found keeping an e-portfolio to beuseful for their language development. This 17-item survey with two open-ended questions was used as an end of course survey. The survey wastranslated into Turkish and its reliability was satisfactory with a Cronbach’s avalue of 0.86.

Design of the e-portfolio and its assessment

As suggested by Driessen, Vleuten, Schuwirth, Tartwijk, and Vermunt (2005), fivequalitative research strategies were used to achieve credibility and reliability of the e-portfolio assessment: triangulation by obtaining information from different sources;member-checking by testing and discussing the data with the students from whichthey were collected; prolonged engagement by having the instructor provide guidanceand feedback on the e-portfolio components; dependability audit by having anexternal auditor also to assess the tasks, and audit trail by documenting theassessment process and instruments using a rubric.

The e-portfolio usage was designed, developed, and implemented based onMoya and O’Malley’s (1994) Portfolio Assessment Model, which was originallydesigned for use in face-to-face teaching and adapted to be used as an electronicportfolio in the online course. The model consisted of six interrelated levels ofactivities:

(1) Identifying purpose and focus of e-portfolio. The basic aim of e-portfolio wasto improve the writing skill of the students. The course instructor examinedthe online English course tutorial, identified the components of thee-portfolio, and informed the students about them at the beginning of thestudy. She explained what to include in the e-portfolio, the assessment ofindividual pieces of work, rubric to be used for evaluation of writing taskdrafts, and where to store their e-portfolios. A portfolio assessmentcommittee that included the course instructor and another instructor fromthe same institution was assigned for the assessment of writing task drafts inthe e-portfolio.


Dow

nloa

ded

by [

Uni

vers

idad

Aut

onom

a de

Bar

celo

na]

at 2

2:36

04

Nov

embe

r 20

14

(2) Planning e-portfolio components. There were required and optional entriesfor the e-portfolios. The required ones were three writing task drafts, pre-post English proficiency test, and achievement test results. The optionalones were the checklist, the survey, and the form. Students were offeredsome topics one of which they would choose and they were asked to writeon it. As suggested by Valdez Pierce and O’Malley (1992), familiar andinteresting topics which did not depend on a student’s knowledge of acontent area were given. The topics that were offered were description ofthe place they lived, description of a famous person or their friends, writinga CV/formal/informal letter, and retelling a book/story/film/memory. Also,some controversial topics to arouse their interest, such as violence at thefootball matches, dating, etc. were added to the list of topics. Students wereinformed about the due date of their drafts, the word limit (50 words), andthe resources they could benefit from (dictionaries and other online andprinted materials).

(3) Designing e-portfolio analysis. The writing task in the e-portfolio waschosen parallel to the learning goals of the course and students wrote threedrafts of this task. The mastery of the writing skills was assessed based onstudents’ individual performance across time and their relative groupstanding.

(4) Preparing for instruction. Students were encouraged to produce the mosteffective e-portfolio by studying, arranging, and submitting its componentsby the specified deadline. That is, the instructor expected the students to beself-organized and self-regulated learners. Self-regulation includes the processof implementing, monitoring, controlling, and directing one’s cognitiveactivities, motivation, and behaviors for the purpose of knowledge growth(Garcia & Pintrich, 1994). Students were informed about their progressthrough the feedback given.

(5) Planning verification of procedures. As mentioned previously, in order toeradicate inconsistency in interpretation and scoring of the components of e-portfolios and to enhance inter-rater reliability, the portfolio committeedecided on a specific criterion to reach judgments concerning students’progress (Moya & O’Malley, 1994) and used a rubric. Moreover, other datacollection items (optional entries) were checked for validity and reliabilitybefore their implementation.

(6) Implementing the model. In the first chat session, the instructor outlinedthe components and the purpose of the e-portfolio, and the assessmentrubric.

Although the course was run by one instructor, the writing task drafts wereevaluated by two instructors through the rubric and returned to the students to berevised by the next due date. Students were expected to improve their draftsaccording to given feedback. At the second writing draft submission time, studentssent their revised work to their instructor and this cycle was repeated three timesuntil end of the semester. Thus, the students were able to see their own progress inwriting by improving their previous draft. In this process, the course instructor andthe students worked in collaboration for writing the perfect paragraphs. Alle-portfolio components were submitted electronically to the instructor at the end ofthe semester.


Dow

nloa

ded

by [

Uni

vers

idad

Aut

onom

a de

Bar

celo

na]

at 2

2:36

04

Nov

embe

r 20

14

Results

Learning gains of students who keep an e-portfolio in the elementary level Englishcourse

Writing scores of the e-portfolio group were analyzed in a time-series design. Bothgroups’ learning gains were measured by using pre- and post-test and achievementtest scores. Paired samples t-tests were used to compare pre- and post-test scores ofthe e-portfolio and traditional assessment groups, and independent samples t-testswere used to compare differences between these groups.

Table 3 shows the mean pre- and post-test scores of students in the e-portfoliogroup (G1). The results indicated that the mean score for post-test (M ¼ 64.67,SD ¼ 25.1) was greater than the mean score for pre-test (M ¼ 47.70, SD ¼ 31.6)and there was a statistically significant difference between pre- and post-test scores ofthe e-portfolio group students T(26) ¼ 73.12, p 5 0.05.

Table 4 shows the mean pre- and post-test scores of students in the traditionalassessment group (G2). The results indicated that the mean score for post-test(M ¼ 67.19, SD ¼ 23.1) was greater than the mean score for pre-test (M ¼ 53.48,SD ¼ 31.8) and there was a statistically significant difference between pre- and post-test scores of the traditional assessment group students T(26) ¼ 74.02, p 5 0.05.

Table 5 compares the means of pre- and post-test scores of the e-portfolio andtraditional assessment groups. The independent samples t-test was not significant for

Table 3. E-portfolio group (G1) pre- and post-test scores.

Mean N Standard deviation

Pair 1Pre-test 47.70 27 31.60Post-test 64.67 27 25.09

N Correlation Sig. (p)Pair 1Pre-test and post-test 27 0.52 0.00

T df Sig. (two-tailed) (p)Pair 1Pre-test and post-test 73.119 26 0.00

Table 4. Traditional assessment group (G2) pre- and post-test scores.

Mean N Standard deviation

Pair 1Pre-test 53.48 27 31.77Post-test 67.19 27 23.03

N Correlation Sig. (p)Pair 1Pre-test and post-test 27 0.84 0.00

T df Sig. (two-tailed) (p)Pair 1Pre-test and post-test 74.020 26 0.00


Dow

nloa

ded

by [

Uni

vers

idad

Aut

onom

a de

Bar

celo

na]

at 2

2:36

04

Nov

embe

r 20

14

the pre-test scores differences, T(56) ¼ 70.33, p 4 0.05. Similarly, the independentsamples t-test was not significant for the post-test scores differences, T(52) ¼ 70.38,p 4 0.05. Two groups of students were found to have no difference between themregarding their pre- and post-test scores.

In order to examine the effects of e-portfolio use on both groups of students’achievement test scores, independent samples t-test was used (Table 6). The resultsindicated that the mean score for the traditional assessment group (G2) (M ¼ 58.32,SD ¼ 18.2) was greater than the mean score for the portfolio group (G1)(M ¼ 47.56, SD ¼ 20.8) and there was a statistically significant difference betweenachievement test scores of the both groups of students T(56) ¼ 72.1, p 5 0.05.There was, therefore, a significant difference in the achievement test scores of thee-portfolio and traditional assessment group students.

In order to examine the effects of keeping an e-portfolio on students’ writingskills, writing scores of students in the e-portfolio group (G1) were also evaluated ina time-series design (Table 7). E-portfolio group (G1) students’ three drafts wereevaluated by two instructors indicated as a1, a2, a3 and b1, b2, b3. The difference inscores in time throughout the semester was analyzed. Table 5 shows the mean foreach of the two sets of scores. As indicated, there were significant increase in meansfrom task 1(a1) to task 2(a2); from task 1(a1) to task 3(a3); and from task 2(a2) totask 3(a3). This was the same for b1, b2, b3 writing task scores (Table 5).

Table 5. Comparison of pre- and post-test scores of e-portfolio and traditional assessmentgroups.

Group N Mean Standard deviation

Pre-testG1 27 47.70 31.60G2 31 50.45 31.91

Post-testG1 27 64.67 25.09G2 27 67.19 23.03

t-test for equality of meansT df Sig. (two-tailed)

Pre-test 70.329 56 0.74Post-test 70.384 52 0.70

Table 6. Comparison of achievement test scores of e-portfolio and traditional assessmentgroups.

Group N Mean Standard deviation

Achievement testG1 27 47.56 20.82G2 31 58.32 18.21

t-test for equality of meansT df Sig. (two-tailed) (p)

Achievement test 72.101 56 0.04


Dow

nloa

ded

by [

Uni

vers

idad

Aut

onom

a de

Bar

celo

na]

at 2

2:36

04

Nov

embe

r 20

14

Although there was an increase in all of the mean scores for writing, there was astatistically significant difference between a1 and a2 T(27) ¼ 73.97 p 5 0.05, a1and a3 T(27) ¼ 73.55 p 5 0.05, and b1 and b2 T(27) ¼ 72.65 p 5 0.05. In otherwords, there were significant differences in some scores of the e-portfolio groupstudents in time.

Students’ perceptions about e-portfolio use and learning gains

Students in both e-portfolio and traditional assessment groups were asked tocomplete the course evaluation survey indicating their levels of satisfaction regardingthe online course. An independent samples t-test was used to determine students’satisfaction of the course. As Table 8 shows, the mean for the satisfaction scores ofthe traditional assessment group students was higher than that of the e-portfolio

Table 7. Comparison of e-portfolio group (G1) writing task scores.

Mean N Std. deviation Std. error mean

Pair 1a1 47.32 28 23.90 4.52a2 64.29 28 18.55 3.51

Pair 2a1 47.32 28 23.90 4.52a3 70.54 28 36.04 6.81

Pair 3a2 64.29 28 18.55 3.51a3 70.54 28 36.04 6.81

Pair 4b1 52.68 28 21.88 4.14b2 62.50 28 15.96 3.02

Pair 5b1 52.68 28 21.88 4.14b3 66.07 28 34.84 6.58

Pair 6b2 62.50 28 15.96 3.02b3 66.07 28 34.84 6.58

T df Sig. (two-tailed) (p)Pair 1a1–a2 73.968 27 0.00

Pair 2a1–a3 73.545 27 0.00

Pair 3a2–a3 70.960 27 0.35

Pair 4b1–b2 72.645 27 0.01

Pair 5b1–b3 71.952 27 0.06

Pair 6b2–b3 70.570 27 0.57


Dow

nloa

ded

by [

Uni

vers

idad

Aut

onom

a de

Bar

celo

na]

at 2

2:36

04

Nov

embe

r 20

14

group students; however, this difference was not statistically significant (p 4 0.05).The independent samples t-test indicated no significant difference in satisfactionscores between the groups (p 4 0.05).

Students were also asked to rate their perceptions regarding keeping ane-portfolio using a Likert scale ranging from 1 ‘‘strongly disagree’’ to 4 ‘‘stronglyagree’’. Responses of the students are given in Table 9.

The highest mean scores were obtained for the measures ‘‘I enjoyed conferencingwith my teacher about my e-portfolio,’’ followed by ‘‘I think my e-portfolio showsmy strengths and weaknesses better than a test score.’’ High mean scores were alsoobtained for the measures ‘‘I had opportunities to consider my strong and weakpoints’’ and ‘‘My e-portfolio helps me to see my academic progress,’’ indicating thatkeeping a e-portfolio differed from other assessment tools, such as tests and affectedstudents’ English learning in general. In addition, high mean scores were obtainedfor the measures ‘‘I evaluated how I improved my English competence’’ and‘‘keeping the e-portfolio led me to become more active in the learning process,’’

Table 8. Comparison of e-portfolio and traditional assessment groups’ satisfaction scores.

Group N Mean Std. deviation

Course evaluation surveyG1 27 112.11 33.43G2 31 116.45 28.80

t-test for equality of meansT df Sig. (two-tailed) (p)

Course evaluation survey 70.531 56 0.60

Table 9. Students’ perceptions about keeping an e-portfolio.

Measures N M SD

1. I had opportunities to consider my strong and weak points. 26 3.4 0.52. I had opportunities to establish reachable goals and objectives. 26 3.1 0.83. I planned my own learning process to achieve my objectives. 26 3.2 0.74. I chose my own materials, activities, and class contents. 26 3.4 0.65. I worked without being supervised by another person. 26 3.4 0.86. I evaluated how I improved my English competence. 26 3.3 0.77. Keeping the e-portfolio led me to become more active in the

learning process.26 3.3 0.6

8. Keeping an e-portfolio encouraged me to become a moreautonomous learner.

26 2.9 0.7

9. I enjoyed keeping an e-portfolio this semester. 26 3.3 0.810. I enjoyed conferencing with my teacher about my e-portfolio. 26 3.6 0.511. I enjoyed conferencing with a peer about my e-portfolio through

the forum.26 2.6 0.7

12. My e-portfolio helps me to see my academic progress. 26 3.4 0.713. I can self-evaluate my work. 26 3.2 0.714. I am proud of the work in my e-portfolio. 26 3.2 0.915. I think my e-portfolio shows my strengths and weaknesses better

than a test. score.26 3.5 0.5

16. I liked creating my own academic goals. 26 3.1 0.717. I feel I can achieve the goals I set. 26 3.1 0.7


Dow

nloa

ded

by [

Uni

vers

idad

Aut

onom

a de

Bar

celo

na]

at 2

2:36

04

Nov

embe

r 20

14

indicating that students actively learned and observed their progress during thelearning process.

Besides all, students in the e-portfolio group were asked to provide self-assessments of their learning through keeping an e-portfolio. They filled in thestudent’s foreign language checklist and sent it back to their instructor as an e-mailattachment. As Table 10 shows, students most frequently stated that keepinge-portfolios helped them to use English language to give their opinions (85%), toread (85%), to write (85%), to talk with their instructors (73%), to participate inonline courses (73%), and in their communities (58%).

Discussion

The results of the pre- and post-test showed that there were significant learninggains in both e-portfolio and traditional assessment groups. That is, both groups’post-test scores were significantly higher than their pre-test scores. However, whenpost-test scores of the two groups were compared, it was observed that that therewas not a significant difference. Although the scores of both groups of studentspointed out an increase over time, there was not any significant difference betweenthese groups’ scores indicating their grammar and vocabulary improvement.Although students observed an improvement in their language proficiency andreported having positive attitude towards portfolio use, this was not reflected intheir scores.

Table 10. Students’ self-assessments regarding the e-portfolio.

f %

By keeping the e-portfolio, I learned something about . . .1. the English language. 25 962. English culture. 5 193. about the places where English is spoken. 10 384. other things. 6 23

I used the language I am studying to . . .5. talk with my teacher. 19 736. talk with my classmate(s) online. 13 507. read. 22 858. give a presentation about something. 10 389. write. 22 8510. communicate with someone by e-mail. 15 5811. communicate with someone by e-mail. 6 2312. communicate with someone by telephone. 7 27

I used the language I am studying . . .13. in online courses. 19 7314. at home. 13 5015. in my community. 15 58

I gave some information on . . .16. my opinion. 22 8517. my feelings. 14 54

I received some information about . . .18. someone else’s opinion. 14 5419. someone else’s feelings. 10 38


Dow

nloa

ded

by [

Uni

vers

idad

Aut

onom

a de

Bar

celo

na]

at 2

2:36

04

Nov

embe

r 20

14

When the results of the face-to-face achievement test were analyzed, it was foundthat the traditional assessment group scored significantly higher than the e-portfoliogroup. This finding might be because the achievement test was comprised of onlymultiple-choice items, which measured grammar and vocabulary proficiency of thestudents. Although the e-portfolio group stated that they believed their overalllanguage proficiency had improved in their checklists and self-assessment forms, andthe analysis of their writing scores indicated an apparent language proficiencydevelopment over time, these language gains were not reflected in their testperformance. This finding demonstrates that writing competence improvement of thee-portfolio group did not affect their grammar and vocabulary proficiency measuredin the post-test and the achievement test. However, it is possible that their non-measurable gains made up for the fact that their scores were lower.

The e-portfolio group students reported that they benefited from keeping theportfolio. Moreover, they believed that keeping an e-portfolio provided them withmore information about their strengths and weaknesses by reflecting their academicprogress in English language more than a test score. Specifically, students indicatedthat by keeping an e-portfolio, they learned how to read, write, express theiropinions, and speak to their instructor in English, which students usually hesitateand feel uncomfortable because of their fear of making mistakes. When both groups’satisfaction with the course was investigated, a significant difference between thegroups was not observed. Although students in the e-portfolio group were highlysatisfied with and confident about keeping an e-portfolio and its positive effect ontheir English language competence, the survey results of the two groups did notsignificantly differ.

To recap, it is clear that a great majority of students were satisfied with keepingan e-portfolio for their English language improvement and with the ongoing teacherfeedback they received. This finding is supported with the finding of Chen’s (2006)study which reported that most of the students favored keeping a portfolio inlearning English, considered portfolios to be good tools for examining learningprocesses and augmenting learning methods, perceived self-progress and self-imagein English learning, felt it boosts their English confidence, and became aware of theirstrengths and weaknesses. In other studies it was reported that portfolio helpedlearners to become more autonomous (Lu, 2005; Zhang, 2009).

As for the effect of studying EFL writing through portfolios, Ozturk and Cecen(2007) found that most students believed that keeping a portfolio improved theirwriting skills. In this study, students appreciated formative feedback from theirinstructors which helped them to see their academic progress and stated that theyliked working collaboratively with their instructor. This finding was parallel to whatLam and Lee (2009) found in their study. Ozturk and Cecen (2007) additionallyclaimed that portfolios are not primarily used as a means of practicing grammar andmore importance is given to the expression of ideas both by the teachers and studentswhich might explain the lack of significant difference between the groups regardingtheir grammar improvement in the current study.

The scores given by the two instructors demonstrated that there were significantgains in different drafts. More specifically, instructor ‘‘a’’s scores showed that therewere significant learning gains between the first and the second drafts, and the firstand the third drafts. However, instructor ‘‘b’’s scores showed that there weresignificant learning gains only between the first and the second drafts. Although bothinstructors scored the drafts using the same rubric, there might have been some


Dow

nloa

ded

by [

Uni

vers

idad

Aut

onom

a de

Bar

celo

na]

at 2

2:36

04

Nov

embe

r 20

14

differences between their assessments. Overall, this finding shows that there wasdevelopment of the writing skill and the perceptions of the students also confirm this.

Conclusion

This study reflects that an e-portfolio can be regarded a practical alternative tostandardized testing (Hiebert & Calfee, 1989; Moya & O’Malley, 1994). Parallel toconclusions reached by Reckase (1997), the e-portfolio, similar to the benefits of aregular portfolio, enabled students to focus on real-life applications of language,problem-solving, and developing creativity. In addition to this, the e-portfolio gavethe students a chance to self-assess their language development and to think abouthow they could improve their future work. The ongoing feedback provided by theinstructor added to the benefits of the e-portfolio, making it a stronger alternative tostandardized testing. The assessment portfolio defined by Apple and Shimo (2004)was especially suitable for the purpose of promoting self-assessment because studentswere encouraged to review their productions and analyze their language developmentcritically in the process of reflecting their best performance. In other words, keepingan e-portfolio can be regarded as the initial step in developing the skills of becomingself-regulated learners which are often neglected in the Turkish high schools butexpected in tertiary education institutions, especially in online courses.

The lack of significant difference in the learning gains of e-portfolio andtraditional assessment groups can be because this was the students’ first experiencewith keeping a portfolio and being assessed through a portfolio. Standardized testare, however, commonly used in all levels of the Turkish educational system.Students are used to being assessed through such tests and are very familiar withmultiple choice test items. Therefore, it is possible to say that the e-portfolio group,while developing their writing skills, did not perform significantly better than thetraditional assessment group in the knowledge domains of vocabulary and grammaras they were tested through a different method. In addition to this, proficiency gainsin knowledge domains develop at a different pace than skills development.

Verification of the current study results with some other students or samples mightbe worthy of consideration in future studies. The same study might be repeated with alarger group of students in order reflect a clearer picture regarding students’ perceptionsof e-portfolio use and to enable generalization of the findings. Besides, e-portfolio canbe used in another course and the students’ learning gains and the development of thewriting skill can be compared with the findings of this course. Furthermore, alongitudinal study is needed to explore long-term achievement factors.

Notes on contributors

Meltem Huri Baturay studied Computer Education and Instructional Technology at MiddleEast Technical University, Turkey. She did her MA degree in Teaching English as aForeign Language from Gazi University, Ankara. Currently, she works as the AcademicAffairs Coordinator of Distance Education at Gazi University. She has particular interest instudent perceptions and achievement in e-learning environments and web-based languagelearning.

Aysegul Dalo�glu received her doctorate degree in Curriculum and Instruction from MiddleEast Technical University, Turkey. She did her MA degree in Teaching English as a ForeignLanguage from Bilkent University, Ankara, and her BA degree in Economics and Psychologyfrom the University of Michigan, Ann Arbor. Her areas of professional interest include pre-service teacher education, curriculum design and evaluation in English language teaching,


Dow

nloa

ded

by [

Uni

vers

idad

Aut

onom

a de

Bar

celo

na]

at 2

2:36

04

Nov

embe

r 20

14

teaching English to young learners, standards in language education, and teaching foreignlanguages with technology.

References

Apple, M., & Shimo, E. (2004). Learners to teacher: Portfolios, please! Perceptions ofportfolio assessment in EFL classrooms. In The Interface between Interlanguage,Pragmatics and Assessment: Proceedings of the 3rd Annual JALT Pan-SIG conference(pp. 53–58). Tokyo, Japan: Tokyo Keizai University.

Barret, H. (2000). Create your own electronic portfolio. Learning and Leading withTechnology, 27(7), 14–21.

Chen, Y.M. (2006). EFL instruction and assessment with portfolios: A case study in Taiwan.Asian EFL Journal, 8(1), 69–96. Retrieved May 20, 2010, from http://www.asian-efl-journal.com/March_06_ymc.php

Darling-Hammond, L. (1994). Performance-based assessment and education equity. HarvardEducational Review, 64(1), 5–30.

Driessen, E., Van der Vleuten, C., Schuwirth, L., Van Tartwijk, J., & Vermunt, J. (2005). Theuse of qualitative research criteria for portfolio assessment as an alternative to reliabilityevaluation: A case study. Medical Education, 39, 214–220.

Fenwick, T.J., & Parsons, J. (1999). A note on using portfolios to assess learning. CanadianSocial Studies, 33(3), 90–92.

Flood, J., & Lapp, D. (1989). Reporting reading progress: A comparison portfolio for parents.The Reading Teacher, 42, 508–514.

Fraenkel, J.R., & Wallen, N.E. (2000). How to design and evaluate research in education (3rded.). New York: McGraw-Hill.

French, R.L. (1992). Portfolio assessment and LEP students. In Proceedings of the SecondNational Symposium on limited English proficient students’ issues: Focus on evaluation andmeasurement (pp. 249–272). Washington, DC: US Department of Education.

Gagliano, K., & Swiatek, L. (1999). Improving student assessment through the implementationof portfolios in language art (Master’s thesis). Chicago: IL: Saint Xavier University andSkyLight Professional Development. Available from ERIC Document ReproductionService (ED436002).

Garcia, T., & Pintrich, P.R. (1994). Regulating motivation and cognition in the classroom:The role of self-schemas and self-regulatory strategies. In D.H. Schunk & B.J. Zimmerman(Eds.), Self-regulation of learning and performance: Issues and educational applications (pp.127–154). Hillsdale, NJ: Erlbaum.

Hamp-Lyons, L., & Condon, W. (2000). Assessing the portfolio: Principles for practice, theory,and research. Cresskill: Hampton Press.

Haney, W., & Madaus, G. (1989). Searching for alternatives to standardized tests: Whys,whats, and whithers. Phi Delta Kappan, 70, 683–687.

Hiebert, E.H., & Calfee, R.C. (1989). Advancing academic literacy through teachers’assessments. Educational Leadership, 46(7), 50–54.

Hung, S.T. (2008). Promoting self-assessment strategies: An electronic portfolio approach.Asian EFL Journal, 11(2), 129–146.

Johnson, B., & Christensen, L. (2004). Educational research: Quantitative, qualitative, andmixed approaches (2nd ed.). Boston, MA: Pearson Education.

Lam, R., & Lee, I. (2010). Balancing the dual functions of portfolio assessment. ELT Journal,64(1), 54–64. doi:10.1093/elt/ccp024.

Lu, Z.D. (2005). Developing portfolios with young English learners, Beijing: English teachingand research. Notes, 184, 52–55.

Lucas, J.W. (2007). Personality type (MBTI) relationship to performance and satisfaction inweb-based instruction (WBI) (Doctoral dissertation). Available from Proquest database.(AAT 3269423).

Moya, S., & O’Malley, M. (1994). A portfolio assessment model for ESL. The Journal ofEducational Issues of Language Minority Students, 13, 13–36.

Nunes, A. (2004). Portfolios in the EFL classroom disclosing an informed practice. ELTJournal, 58(4), 327–335.

Ozturk, H., & Cecen, S. (2007). The effects of portfolio keeping on writing anxiety of EFLstudents. Journal of Language and Linguistic Studies, 3(2), 218–236.


Dow

nloa

ded

by [

Uni

vers

idad

Aut

onom

a de

Bar

celo

na]

at 2

2:36

04

Nov

embe

r 20

14

http://www.asian-efl-journal.com/March_06_ymc.php

http://www.asian-efl-journal.com/March_06_ymc.php

Reckase, M.D. (1997, March). Constructs assessed by portfolios: How do they differ from thoseassessed by other educational test? Paper presented at the annual meeting of the AmericanEducational Research Association, Chicago, IL.

Student’s Foreign Language Standards Checklist (n.d.). Retrieved July 10, 2008, from http://www.nclrc.org/portfolio/formStudentsForeignLanguageStandards.html

Taylor, K., & Walton, S. (1997). Co-opting standardized tests in the service of learning. PhiDelta Kappan, 67, 66–70.

Valdez Pierce, L., & O’Malley, J.M. (1992). Performance and portfolio assessment for languageminority students. NCBE program information guide series, No: 9. Washington, DC:National Clearinghouse for Bilingual Education.

Yoshida, Y. (2001). Authentic progress assessment of oral language: Oral portfolios. Availablefrom ERIC Document Reproduction Service (ED453674).

Zhang, S. (2009). Has portfolio assessment become common practice in EFL classrooms?Empirical studies from China. English Language Teaching, 2(2), 98–118.


Dow

nloa

ded

by [

Uni

vers

idad

Aut

onom

a de

Bar

celo

na]

at 2

2:36

04

Nov

embe

r 20

14

http://www.nclrc.org/portfolio/formStudentsForeignLanguageStandards.html

http://www.nclrc.org/portfolio/formStudentsForeignLanguageStandards.html

e-portfolio assessment in an online english language course

Documents