integrating communities of practice in e-portfolio assessment: effects and experiences of mutual...

5
Integrating communities of practice in e-portfolio assessment: Effects and experiences of mutual assessment in an online course Ling Wang Nova Southeastern University, Graduate School of Computer and Information Sciences, 3301 College Ave. Fort Lauderdale, FL 33324, USA abstract article info Article history: Accepted 14 July 2010 Keywords: Communities of practice Collaborative learning Mutual assessment Formative evaluation Online education This study investigated the effects and experiences of a mutual assessment framework (CoPf) in an online graduate course at a mid-west university. CoPf was integrated into the course structure as an innovative application of the standard e-portfolio assessment tool. Using a mixed method, the study rst explored the effects of CoPf compared to the standard e-portfolio in relation to the promotion of revisions to students' work, students' nal course grades, and interactions both between the students and the instructor and among students. Qualitative analysis was then conducted to inquire the students' experiences in the CoPf course and how they perceived these experiences. Findings from the data analysis were presented and the contributions/implications of the study were discussed. © 2010 Elsevier Inc. All rights reserved. 1. Background e-portfolios have recently played a key role in online learning evaluation/assessment (Black & William, 1998; Gordin, Grueneberg, Laff, Martinez, & Lam, 2004). The four pillars of the e-portfolio (metacognition, authentic tasks, contextual feedback, and student responsibility) have attributed to the evidence for its effectiveness, but the classic e-portfolio concept and practice display a number of signicant shortcomings (Barbera, 2009). While there are some specic initiatives that consider the social aspect of the e-portfolios, their format is most frequently based on the individual creation of work, and the collective value that other students may bring to the work is not taken into account. Moreover, the e-portfolio is not a complete work because it is not submitted as a single large text but as pieces of work that are very interesting but unconnected to each other (Barbera). Each shortcoming, the individuality and the disconnection, are factors that the emerging netfolio concept aimed to overcome (Barbera, 2009). As the word reects, the netfolio is congured through a set of e-portfolios produced by different students who, at a given time and through online communication, provide the other students with new content and different perspectives. This collabo- rative approach in the form of a network offers students the ability to compare e-portfolios as another stage in the construction of their own personal e-portfolios (Barbera). The netfolio concept is well aligned with the situated learning theory postulated by Lave and Wenger (1991). Situated learning is a theory of knowledge acquisition. Lave and Wenger suggest that learning requires social interaction and collaboration. Social interac- tion is a key component of situated learning, as well as a necessary component in building Communities of Practice (CoPs). Online Communities of Practice (CoPs) offer learners a range of benets, including cognitive development, opportunities for growth as independent learners, opportunities to practice newly acquired knowledge in a supportive environment with peers (Kayler & Weller, 2007). To further support the postulation, previous studies (e.g., Cole, Ryan & Kirk, 1995; Hunt & Pellegrino, 2002; Olina & Sullivan, 2004; Riedinger, 2006; Svinicki, 2001) have argued that every learning action should focus on the collaborative construction of knowledge between students and teacher and among the students themselves. Rovai, Ponton, Derrick, and Davis (2006) indicated that the feedback process in online education needs to be more explicit than in face-to- face education in order to achieve similar educational effects in online education such as enhanced learning experiences, and more in-depth learning. It is expected that such effects can be achieved through integrating the CoP into the online e-portfolio assessment framework (called CoPf for short); the mutual assessment processes among peers have the potential to incorporate in-depth learning and to elicit positive effects on students' learning outcomes, as well as enhanced learning experiences. 2. Purpose of the study The purpose of the study was to evaluate the contribution and reinforcement effect introduced by CoPf on learning decisions concerning knowledge acquisition. This effect was reected in the increase in revision of students' work both on an individual level and Internet and Higher Education 13 (2010) 267271 Tel.: +1 954 262 2020; fax: +1 954 262 3915. E-mail address: [email protected]. 1096-7516/$ see front matter © 2010 Elsevier Inc. All rights reserved. doi:10.1016/j.iheduc.2010.07.002 Contents lists available at ScienceDirect Internet and Higher Education

Upload: ling-wang

Post on 05-Sep-2016

212 views

Category:

Documents


0 download

TRANSCRIPT

Internet and Higher Education 13 (2010) 267–271

Contents lists available at ScienceDirect

Internet and Higher Education

Integrating communities of practice in e-portfolio assessment: Effects andexperiences of mutual assessment in an online course

Ling Wang ⁎Nova Southeastern University, Graduate School of Computer and Information Sciences, 3301 College Ave. Fort Lauderdale, FL 33324, USA

⁎ Tel.: +1 954 262 2020; fax: +1 954 262 3915.E-mail address: [email protected].

1096-7516/$ – see front matter © 2010 Elsevier Inc. Aldoi:10.1016/j.iheduc.2010.07.002

a b s t r a c t

a r t i c l e i n f o

Article history:Accepted 14 July 2010

Keywords:Communities of practiceCollaborative learningMutual assessmentFormative evaluationOnline education

This study investigated the effects and experiences of a mutual assessment framework (CoPf) in an onlinegraduate course at a mid-west university. CoPf was integrated into the course structure as an innovativeapplication of the standard e-portfolio assessment tool. Using a mixed method, the study first explored theeffects of CoPf compared to the standard e-portfolio in relation to the promotion of revisions to students'work, students' final course grades, and interactions both between the students and the instructor andamong students. Qualitative analysis was then conducted to inquire the students' experiences in the CoPfcourse and how they perceived these experiences. Findings from the data analysis were presented and thecontributions/implications of the study were discussed.

l rights reserved.

© 2010 Elsevier Inc. All rights reserved.

1. Background

e-portfolios have recently played a key role in online learningevaluation/assessment (Black & William, 1998; Gordin, Grueneberg,Laff, Martinez, & Lam, 2004). The four pillars of the e-portfolio(metacognition, authentic tasks, contextual feedback, and studentresponsibility) have attributed to the evidence for its effectiveness,but the classic e-portfolio concept and practice display a number ofsignificant shortcomings (Barbera, 2009). While there are somespecific initiatives that consider the social aspect of the e-portfolios,their format is most frequently based on the individual creation ofwork, and the collective value that other students may bring to thework is not taken into account. Moreover, the e-portfolio is not acomplete work because it is not submitted as a single large text but aspieces of work that are very interesting but unconnected to each other(Barbera).

Each shortcoming, the individuality and the disconnection, arefactors that the emerging netfolio concept aimed to overcome(Barbera, 2009). As the word reflects, the netfolio is configuredthrough a set of e-portfolios produced by different students who, at agiven time and through online communication, provide the otherstudents with new content and different perspectives. This collabo-rative approach in the form of a network offers students the ability tocompare e-portfolios as another stage in the construction of their ownpersonal e-portfolios (Barbera).

The netfolio concept is well aligned with the situated learningtheory postulated by Lave and Wenger (1991). Situated learning is a

theory of knowledge acquisition. Lave and Wenger suggest thatlearning requires social interaction and collaboration. Social interac-tion is a key component of situated learning, as well as a necessarycomponent in building Communities of Practice (CoPs). OnlineCommunities of Practice (CoPs) offer learners a range of benefits,including cognitive development, opportunities for growth asindependent learners, opportunities to practice newly acquiredknowledge in a supportive environment with peers (Kayler & Weller,2007).

To further support the postulation, previous studies (e.g., Cole,Ryan & Kirk, 1995; Hunt & Pellegrino, 2002; Olina & Sullivan, 2004;Riedinger, 2006; Svinicki, 2001) have argued that every learningaction should focus on the collaborative construction of knowledgebetween students and teacher and among the students themselves.Rovai, Ponton, Derrick, and Davis (2006) indicated that the feedbackprocess in online education needs to be more explicit than in face-to-face education in order to achieve similar educational effects in onlineeducation such as enhanced learning experiences, and more in-depthlearning. It is expected that such effects can be achieved throughintegrating the CoP into the online e-portfolio assessment framework(called CoPf for short); the mutual assessment processes among peershave the potential to incorporate in-depth learning and to elicitpositive effects on students' learning outcomes, as well as enhancedlearning experiences.

2. Purpose of the study

The purpose of the study was to evaluate the contribution andreinforcement effect introduced by CoPf on learning decisionsconcerning knowledge acquisition. This effect was reflected in theincrease in revision of students' work both on an individual level and

268 L. Wang / Internet and Higher Education 13 (2010) 267–271

on a group level. Individual level (self-revision) requires the capacityto incorporate the contributions of others in a meaningful way. Grouplevel (mutual revision) requires the capacity to offer relevantfeedback on other's work, based on the understanding that morereflections upon and revisions of their work leads to more significantand improved learning (Galley, 2000; Heath, 2002). The study alsoexamined the expected enhancement of the students' final workproducts as well as perceptions of their learning experiences, asdemonstrated by in-depth reflections.

3. Context and participants of the study

As an exploratory study, the CoPf framework was applied to agraduate course on Foundations of Educational Research offeredonline in the Fall semester of 2009 to a group of graduate students(n1=29) in the program of Educational Leadership at a mid-westernuniversity. The course has previously been offered for a number ofsemesters using the standard e-portfolio assessment tool. To form acomparison group, the academic records of the students wereobtained from the immediate previous semester (n2=26) taught bythe same instructor. The university IRB approval was obtained andparticipants' consent was collected. In terms of the general profile ofthe participants, the majority (75% to 80%) were female, they wereyoung-to-middle aged (between 24 to 45), and they used technologyand internet for educational purpose on a daily basis because theywere either school teachers (about 3 quarters) or full-time students.The two groups were equivalent in respect to this general profile.

The objectives of this course are to develop basic researchcompetencies. The competencies are organized in five categories:defining the research question, formulating hypotheses, collectingdata, presenting and interpreting research results, discussing implica-tions and conclusions. To pass the course, the students are required tocomplete a portfolio that contains a minimum of two pieces ofevidence for each of the five competencies. Each evidence accounts for10 points in the final grade, and the 10 evidences add up to 100 totalpoints. The instructor has a set of criteria to assess whether theevidence sufficiently indicates the students' accomplishments of eachof the five competencies. After grading the evidences, the instructoruses the published scale to convert the grade points to letter grades.

The e-portfolio has been integrated into the university's e-learningplatform and has been designed technically and pedagogically toprovide learning support to students online. Prior to the Fall 2009semester, the e-portfolio tailored to the requirements of this coursewas structured in three sections (laid out in three horizontal tabs)visible to the students at all times: presentation, competencies, andmonitoring. The presentation included in the foreground a personalmotto and a photo, both of which were related to the student butnecessarily also to the specific contents of the course. The competen-cies section included the students' submission of the competenciesand the evidence required (once a competence was selected, the partof the rubric corresponding to that competence would be displayed),and the instructor's qualitative assessment of the students' portfolio.In the monitoring section, the complete rubric of the course wasdisplayed as well as an automatic alert system that informed them ofthe completion status of each of the evidences corresponding to eachcompetence (not started, in progress, in revision, and closed).

Specifically for this study, the e-portfolio structure was enhancedand transformed to the CoPf structure where the competencies sectionwas altered to such functions: Once a competence has been selected,an exclusive place (discussion forum)was provided to attach it, justifyit, reflect on it, and assess it. This forum was aimed at theimprovement of the evidence itself and the competence related tothe evidence, which was always open to the comments of theinstructor and also open to the opinions and assessments of otherstudents in the course.

In both structures, students could improve the evidences thatshowed the achievement of these competencies by submittingunlimited versions of the evidences for each of the five competencies.However, before the CoPf structure was integrated, the studentsmainly relied on the instructor's feedback to improve their work. Theyhad the option to view other students' portfolios by submitting arequest obtaining the permission of the portfolio's owner. With theCoPf structure, feedbackwas provided by other students first and thenby the instructor, and students had free access to other students'submitted evidences at all times. Therefore, the main differencesbetween the e-portfolio and CoPf lied in the nature of the interactionbetween students and with the instructor. Access-at-all-time com-munication exchanges took place in the CoPf where mutual feedbackand comments were stored and helped solidify the students' sense ofbelongings to the community.

4. Research design

A mixed method was adopted. A quantitative analysis of thedocumentation and interactions in the courses using the e-portfoliostructure and the CoPf structure respectively was conducted tocompare and detect the effect of the two structures on students'learning. Then a qualitative analysis was conducted of the datacollected from the students within the CoPf structure who were askedto provide a short essay to describe their learning experiences and theperceptions of these experiences with CoPf.

5. Data collection

The following data were collected from the students' in the CoPfgroup: the revisions made to the evidence of each of the fivecompetencies (such as how many of the students made revisions?how many revisions were made per evidence? how many revisionsled to improved competencies?); the final grades of the students; andthe interaction messages posted within the CoPf structure (instruc-tor's feedback messages, students' feedback messages, messages thatshowed improvements, and messages with high-level comments,etc.) In this study, the high-level comments were defined as suchwhen the messages demonstrated discussions related to analysis,synthesis, and evaluation, following Bloom's cognitive taxonomy(1956). On the other hand, when the messages only demonstrateddiscussions related to knowledge, comprehension, and application,they were categorized as low-level comments. In addition, qualitativedata were also collected from these students using an essay formatasking the students to discuss their learning experiences within theCoPf structure.

From the students who used the e-portfolio assessment in theprevious semester, some corresponding data were also collected: finalgrades were obtained from students' archived records; the revisionsand messages were extracted from the archived course e-portfoliostoo. However, due to the fact that the e-portfolio groupwas formed bythe students from the previous semester, when this study wasconducted, it was not possible to gain access to that group any morefor the collection of the qualitative data. This became a limitation ofthe study due to the fact that the study adopted a design and aprocedure like this.

6. Results and discussion

6.1. Comparison between the e-Portfolio and CoPf

The following table presented the data analysis results obtainedfrom the content analysis of the exchanges of messages that occurredin the e-portfolio and CoPf structures respectively with regard torevisions to the students' evidences, final grades of the students'portfolios, and the interaction among the students and the instructor.

269L. Wang / Internet and Higher Education 13 (2010) 267–271

The unit of analysis of the messages was at the idea level, i.e., for eachevidence, a distinctive idea discussed or presented in the messageswas counted as one unit of analysis (Table 1).

In the Revision section, over ninety percent of the students in bothgroups submitted revisions of their portfolios (90.8% of the e-portfoliogroup and 100% of the CoPf group). However, significant differenceswere detected in the final results of these revisions: while fewer thanhalf of the students (42.04%) in the e-portfolio group improved theirresults, in the CoPf group, over three quarters of the students (77.4%)improved their products. Another remarkable aspect in this section isthe difference in the average number of revisions for each of the 10required evidences made between the e-portfolio group and the CoPfgroup: a significantly greater number of revisions (pb .001) was madein the CoPf group (with an average of 3.11) than in the e-portfoliogroup (with an average of 0.57).

To account for such differences, it is feasible to consider the specificrequirement of the CoPf course as causes or factors: The CoPf courserequired students' mutual feedback/assessment and therefore en-hanced the visibility of the revision processes in the course. However,what makes the detected differences meaningful and relevant is theobserved difference in the learning outcomes of the students from thetwo groups. In the Final Grades section, notable difference was in factdetected in favor of the group using the CoPf structure: 96.5% of theCoPf students obtained an A or a B, while these grades only made up61.5% in the e-portfolio group. It was also shown in the results that theCoPf group was overwhelmingly concentrated (96.5%) in the A and Bgrades while the e-portfolio group had amajority concentration (73%)in B and C grades. This constitutes the evidence of higher achievementof the CoPf students, which is reinforced by the fact that both groupswere taught by the same instructor and worked with the samelearning content.

When examining the results in the Interaction section, it isinteresting to observe that, even though the absolute number ofinstructor feedback messages showed a difference between the twogroups (CoPf group greater than the e-portfolio group), the ratio of theinstructor feedbackmessages to the totalmessages in the forumactuallydemonstrated a result in the opposite direction (CoPf group—12.7% ande-portfolio group — 18.9%). On the other hand, an expected significantrise (pb .001) in the quantity of the feedbackmessages that the students

Table 1Results of the comparison between the e-portfolio group and the CoPf Group.

Dimensions e-portfolio group(n=26)

CoPf group(n=29)

Significance

Revision• Number of students who haverevised a competence

90.8% 100% p= .3914

• Number of students who haveimproved a competence

42.04% 77.4% p= .0678

• Average number of revisions foreach evidence

0.57 3.11 pb .001

Final gradesa A 19.2% A 44.8% –

B 42.3% B 51.7%C 30.7% C 3.45%D 7.7% D 0%

InteractionTotal messages in the forum 572 1768• Number of instructor feedbackmessages

108 (18.9%) 224(12.7%)

p= .2036

• Messages with improved versionsor improvement comments fromthe students

84 (14.7%) 576(32.6%)

pb .001

• Messages between students 26 (4.5%) 531 (30%) pb .001• Student messages with high-levelcomments to the instructor

155 (27%) 295(16.7%)

p= .0201

• Student messages with high-levelcomments to other students

11 (1.9%) 365(20.6%)

pb .001

a Grade Scale: A: 90-100 B: 80-89 C: 70-79 D: 0-69.

exchanged among themselves was detected as the CoPf group'smessages reached 531 (30%) in contrast to the meager number of 26(4.5%) messages in the e-portfolio group. It is also observed, althoughthe high-level comments to the instructor in the e-portfolio groupshowed a slightly higher ratio to the total number of messages than theCoPf group, thehigh-level comments to other students in theCoPf groupshowed a significantly higher (pb .001) ratio to the total number ofmessages than the e-portfolio group. Lookingmore closely, 11 out of 26messages (42.3%) that were exchanged between students qualified forhigh-level comments in the e-portfolio group whereas the counterpartin the CoPf group turned out to be 365 out of 531 messages (68.7%).

6.2. The CoPf experiences

Following the quantitative data analysis, a qualitative analysisusing the constant comparison method was conducted on the datacollected from the CoPf group. Three dominant themes emerged fromthe data.

6.2.1. Theme 1: fostering independent learningThe majority of students reported they enjoyed learning about

other students' experiences/expertise. They indicated they also usedtheir own experiences/expertise to support their ideas within theirpostings. The following three examples demonstrated how theyencouraged and supported each other:

• “I offered suggestions and/or different approaches for my fellowstudent when I found he/she was grappling with a concept or aproblem that had similarly troubled me previously.”

• “I want my classmates to know they are not facing the challengealone. As frequently as I could, I incorporated into my postings mypersonal experiences and my professional expertise to help withexplaining the concept under discussion.”

• “I (also) posted messages to seek advice from other students. Manyof my postings have led to other interesting ideas and opinionswhich were helpful with understanding the concepts.”

In these examples, students served not as problem solvers but asresources for each other. Communities of practice (CoP) weredeveloped when students discovered that they faced similar issuesand opportunities.

The data supportedWenger (1998, 2005)'s belief that CoP fulfills anumber of functions in the creation, accumulation, and diffusion ofknowledge and the exchange of interpretation of information. There isspecific knowledge that the community develops, shares, andmaintains (Wenger, McDermott, & Snyder, 2002). Just as indicatedby Stage, Muller, Kinzie, and Simmons (1998), “from a constructiveperspective, knowledge cannot simply be given to students; studentsmust construct their own meanings” (p. 35), the students in the CoPfcourse worked together but they independently created knowledge.The CoPf learning environment provided authentic ways for studentsto develop their expertise of practice through independent learningand knowledge acquisition.

6.2.2. Theme 2: enabling meaningful learningIt is commonly acknowledged that an important value of teaching

and learning is that students will become capable of makingconnections between their classroom practices and the concepts/theories/models they encounter in the course readings. However, it isinteresting to observe that not many students referred to the coursereadings in their feedback postings. Many students reported theystruggled with explicitly referring to the readings. At the same time,those students who did use a quote or two from the course readings tosupport their postings reported how they found it easier to respond toothers' postings when information/quotes from a reading wereincluded. In particular, one student admitted: “I often find it hard toconnect to the text readings myself, but the other students' cites or

270 L. Wang / Internet and Higher Education 13 (2010) 267–271

references of a specific reading or chapter or even a paragraph arevery helpful for me to interpret the concept and expandmy thoughts.”

Apparently, through the use of course readings, students madesense of the concepts or theories and socially constructed under-standings. This further supports the notion that the online CoP cansupport learners engaged in deep and meaningful interpersonalinteraction (Dabbagh & Bannan-Ritland, 2005).

Looking specifically into the experiences of those students whowere not able to use the readings in their postings, it was detectedthey also actively participated in generating discussions and offeringalternative perspectives to one another. In many cases, they did so byplaying the role of devil's advocate or by asking thought-provokingquestions, as one student admitted in his reflections: “It was usuallythe questions that are typically controversial that led to discussionsand articulations of different perspectives/views, which in turnbrought about newly gained insights into the concepts.”

Overall, through grappling with course readings, students engagedin discussions and gained alternative perspectives. Clearly thestudents using the CoPf structure shared similar challenges andsupported each other in enhancing their ability to articulate thechallenges and address them. By doing so, the students transformedtheir learning to higher-order thinking which made them bothindependent and meaningful learners (Wink, 2000).

6.2.3. Theme 3: promoting understandings through self and mutualassessment

The formative assessment processes within the CoPf structurepromoted students to be self-reflective regarding their contributionsto the development of the learning community (i.e., CoPf). Theprocesses of self andmutual assessment allowed students to reflect onmany aspects of the development of the learning community: theirdegree of participation, the quality of their contributions, and thestrengths and areas to improve upon. This important feature of theCoPf course is well in line with what Weimer (2002) postulated: “Theliterature on self-directed learning also underscores the importance ofassessment, only in this case it is the ability of students to self-assessaccurately. Sophisticated learners know when they do or do notunderstand something. They can review a performance and identifywhat needs improvement. They know when their lack of objectivitynecessitates their soliciting external feedback” (p. 17).

Data from students' reflections indicated that the studentsdeveloped awareness that their individual contributions were ofvalue to the learning community. Students also identified ways toenhance the feedback discussions and areas for improvement. Self-reflection allowed students to evaluate their work and use theirpostings as evidence to identify patterns of participation which couldenhance the larger learning community. The following are examplesof students' overview of their contributions:

• “Overall I feel I was on the right track in my feedback postings. Nowlooking back at the exchanged messages, I realized how much Ilearned from the discussions. There are areas to improve upon, suchas skills to critique, but isn't that one of the objectives of the course,to continuously improve one's skills and capabilities?”

• I definitely know now there are still questions that my postingscould have asked for the sake to enhance the discussions. But what Ifelt good about my contributions to the discussions was that Ialways left room for further dialogue toward the end of mypostings.”

While these overviews illustrated that students' contributionshelped formulate a positive experience with the learning community,the students also mentioned typical areas of difficulty, such as timecommitment (it is not easy to find time to post regularly to formulatean ongoing discussion), mismatch or connection failure of students'professional experiences/expertise, calibers of writing, and learningstyles, issues with the textbook and other course readings, etc.

Nonetheless, students may have had either positive or negativeexperiences in the CoPf course, but they unanimously acknowledgedthat lack of participation and lack of depth in individual contributionswould adversely impact the learning experience. Thus, the self-reflections of the students on their CoPf experience clearly helpedpromote students' understandings of the functioning (and malfunc-tioning) and the implications of this learning environment.

7. Conclusions

In summary of the study results, the use of the CoPf structure led tomore revisions both by the students, of their own work, and amongstudents, of each other's work, and these in turn led to better finalgrade results. This is in line with the direct relationship commonlyestablished between interaction processes and cognitive change,particularly when the exchanges are geared towards more profoundand in-depth learning. In terms of students' learning experiences, asdemonstrated by students' reflective data, students engaged inmeaningful learning by making sense of concepts/theories/models,they supported and challenged each other to become independentlearners, and they worked together to deepen their knowledge base,construct knowledge, as well as create new understandings.

Within the CoPf structure, it was observed that the role of theinstructor took a somewhat unexpected direction. Although CoPfincreased the students' revisions, the number of the instructor'sfeedback messages to the students in the CoPf course was actuallylower than that in the e-portfolio course in terms of its ratio to thetotal number of messages in the course. However, by assuming a roleof “observant assessor,” the instructor was demanded to bringtogether a more complex network of interactions between thestudents and intervene when necessary. The instructor did notintervene directly, and this nonintervention became a positive assent,i.e., she reaffirmed, with a silent but constant presence, the resolutionof the evaluator student in the sense that the work is being developedand assessed correctly. This kind of teaching model could be achallenge to course instructors, but it was integral to the CoPfframework, allowing students to have ownership, to develop theirvoices, and to construct knowledge.

The findings of this study reinforced the notion that educationalresearch is more importantly tasked with finding out what is insidethe “black box” (i.e., formative evaluation) than reporting the finalgrade point differences (i.e., summative evaluation). Black andWilliam (1998) called attention to the dangers of understanding aclass as a “black box” into which certain inputs are inserted and givenoutputs extracted. This study aimed to know more about what takesplace inside the box. As formative evaluation was recognized asimportant and significant for high quality teaching and learning, thestudy and its findings re-affirmed that mutual feedback among peersshould not be an accidental practice, but an essential, integratedcomponent of the evaluation system (e.g., a CoPf structure).

The findings of the study echoed the proposed social presenceenhancement in online inquiry communities (Rourke, Anderson,Garrison, & Archer, 2004) through the building of social networks(Wellman, 1999). The contribution of the study from this perspectivealso reinforced the effects of the structured and scaffolded feedbackreferred to by Whitelock (2006), who proposed to take the advantageof the responsiveness and immediacy of feedback offered by thetechnology to help students to be more reflexive. This aim wasachieved to a certain degree by the CoPf context employed in thisstudy. Within the CoPf, the active cognitive process of mutualinterpretation that students underwent among themselves werecaptured by the technology in a more visible way to give moremeaning to what was going on inside the black box, and therefore tothe process of teaching and learning as well.

Some limitations associated with the findings of the study have tobe considered. In reviewing the reflections of the students in the CoPf

271L. Wang / Internet and Higher Education 13 (2010) 267–271

course, it was suspected that, to an unknown extent, positive ornegative experiences with the CoPf framework could be based uponlearning style preferences. Therefore, for future studies, the role oflearning style preferences in students' participation rates and percep-tions of the CoPf experiences can be explored. Besides this, for futureresearch investigating the students' achievements and learning out-comes using the CoPf structure, it will be interesting to examine thestudents in two categories, i.e., those who are high performers andthose who are low performers at the beginning of the use of the CoPfstructure, and to compare the different effects, if any, the CoPfexperience has on the achievements and learning outcomes of thesetwodifferent groups. Some additional areas for future research includebut are not limited to: instructor's affective role inmonitoring the CoPfframework when there is limited to no student interaction, strategiesto scaffold higher-order thinking to promote a deeper level ofunderstanding of course content among students with differing levelsof expertise and knowledge, and shared positioning and co-construc-tion of meanings of the students in relation to the task.

References

Barbera, E. (2009). Mutual feedback in e-portfolio assessment: an approach to thenetfolio system. British Journal of Educational Technology, 40(2), 342−357.

Black, P., & William, D. (1998). Inside the white box: raising standards throughclassroom assessment. Phi Delta Kappa, 80(2), 139−148.

Bloom, B. S. (1956). Taxonomy of Educational Objectives. Handbook 1: Cognitive domain,2nd ed. Addison Wesley Publishing Company.

Cole, D. J., Ryan, C. W., & Kirk, E. (1995). Portfolios Across the Curriculum and Beyond.Thousand Oaks, CA: Corwin Press.

Dabbagh, N., & Bannan-Ritland, B. (2005). Online Learning: Concepts, Strategies, andApplication. New Jersey: Pearson Education.

Galley, S. (2000). Portfolio as mirror: student and teacher learning reflected throughthe standards. Language Arts, 78(2), 121−127.

Gordin, D., Grueneberg, K., Laff, M., Martinez, S., & Lam, R. (2004). Using CollaborativeEportfolios to Reinvent Teacher Education. In C. Crawford (Ed.), Proceedings of

Society for Information Technology and Teacher Education International Conference2004 (pp. 100−105). Chesapeake, VA: AACE.

Heath, M. (2002). Electronic portfolios for reflective self-assessment. Teacher Librarian,30(1), 19−23.

Hunt, E., & Pellegrino, J. W. (2002). Issues, examples, and challenges in formativeassessment. New directions in Teaching and Learning, 89, 73−86.

Kayler, M., & Weller, K. (2007). Pedagogy, self-assessment, and online discussiongroups. Educational Technology & Society, 10(1), 136−147.

Lave, J., & Wenger, E. (1991). Situated Learning: Legitimate Peripheral Participation.Cambridge England, New York: Cambridge University Press.

Olina, Z., & Sullivan, H. J. (2004). Student self-evaluation, teacher evaluation, andlearner performance. Journal of Educational Technology Research and Development,52, 5−22.

Riedinger, B. (2006). Mining for Meaning: Teaching Students How To Reflect. In A.Jafari & C. Kaufman (Eds.), Handbook of Research on ePortfolios (pp. 90−101).Hershey, PA: Idea Group Reference.

Rourke, L., Anderson, T., Garrison, D. R., & Archer, W. (2004). Assessing social presencein an asynchronous text-based, computer conferencing. Journal of DistanceEducation, 14, 51−70.

Rovai, A. P., Ponton, M. K., Derrick, M. G., & Davis, J. M. (2006). Student evaluation ofteaching in the virtual and traditional classrooms: a comparative analysis. Internetand Higher Education, 9, 23−35.

Stage, F. K., Muller, P., Kinzie, J., & Simmons, A. (1998). Creating Learning-CenteredClassrooms: What Does Learning Theory have to Say? Washington D.C.: GeorgeWashington Univ., Graduate School of Education and Human Development.

Svinicki, M. D. (2001). Encouraging your students to give feedback. New Directions inTeaching and Learning, 87, 17−24.

Weimer, M. E. (2002). Learner-Centered Teaching: Five Key Changes to Practice. SanFrancisco: Jossey-Bass.

Wellman, B. (1999). Networks in the Global Village. Boulder, CO: Westview Press.Wenger, E. (1998). Communities of Practice: Learning, Meaning and Identity. New York:

Cambridge University Press.Wenger, E. (2005). Communities of Practice: Learning, Meaning and Identity. New York:

Cambridge University Press.Wenger, E., McDermott, R., & Snyder, W. M. (2002). Cultivating Communities of Practice:

A guide to Managing Knowledge. Boston: Harvard Business School Press.Whitelock, D. (2006). Electronic Assessment: Marking, Monitoring and Mediating

Learning. In P. McAndrew, & A. Jones (Eds.), Interactions, objects and outcomes inlearning. Special Issue, Vol. 2(2/3). (pp. 264−276).

Wink, J. (2000). Critical Pedagogy: Notes from the Real World. Boston: Allyn and Bacon.