e-Portfolioassessmentinnetworkedlearningbasedcommunities

BarryAvery

July2016

ThisthesisissubmittedinpartialfulfilmentoftherequirementsforthedegreeofDoctorofPhilosophy.

DepartmentofEducationalResearch,LancasterUniversity,UK.

ThisthesiswascompletedaspartoftheDoctoralProgrammeine-Research&TechnologyEnhancedLearning.

This thesis results entirely from my own work and has not been offeredpreviouslyforanyotherdegreeordiploma.Signature........................................................

1

BarryAvery,BSc(Hons),PGDip,Cert.Ed.,SFHEAe-PortfolioassessmentinnetworkedlearningbasedcommunitiesDoctorofPhilosophy,July2016

AbstractThere is a substantial body of research suggesting the advantages of using e-Portfoliosinhighereducationassessment,whereworkiscollatedbyindividualsto record their learning. The use of learning communities in this context is anunder-researched area, despite the number of e-Portfolios that implement asocialcomponent.

This work develops an alternative e-Portfolio approach by using a networkedlearningbasedpedagogy,whichbringsricherdescriptionsofbothartifactsandthe structure of the underlying community. Action research and free/opensource development principles have been aligned over two cycles, wherestudents have participated as both co-researchers and co-developers. Evolvingthe nature and presentation of assessment artifacts, participants havedeterminedhowthesearebestsharedandreused,andthewaysinwhichlargercontextualinformationaboutthecommunitycanimproveboththelearningandtheknowledgeofthelearningtakingplace.

Amulti-methodresearchframeworkisusedtoshowwhatartifactsarecreated,whoisinteractingwithwhomandwhyparticipantsactastheydo.Datahasbeencollectedusinginterviews,focusgroupsandfromanalyticsfromthee-Portfolioitself.

Thefindingssuggestthatthetypesofartifactscreatedareinfluencedbyboththecommunity and by the nature of thematerial being learnt. Artifacts reveal thesources that students use for their work and although participants can bereluctant to reveal incorrector incompletework to thecommunity, thiscanbeencouragedby a carefully constructed induction, reinforcing the importanceofthe role of teacher as tutor. Expertise is quickly associated with someparticipants by the quality and regularity of their artifact construction, whobecome more central and influential to the community, with their workbecomingincreasinglyvisiblethroughsearchactivities.

Thiswork presents the framework, an analysis of the results, conclusions andrecommendationsalongwithareferenceimplementation.

2

Abstract.................................................................................................................................1

Acknowledgements...........................................................................................................5

Listofabbreviations.........................................................................................................6

Softwareusedinthiswork.............................................................................................7

Listoffiguresandtables.................................................................................................8

Chapter1 Introductionandbackground...........................................................111.1Contextandpurpose...........................................................................................11Thewidercontext...................................................................................................................11Researchpurpose...................................................................................................................13

1.2Domain.....................................................................................................................16Abouttheresearchsite.........................................................................................................16Pedagogicpracticesandtheparticipantsavailableforthestudy.....................18

1.3Theframeworkandresearchquestions......................................................20Underlyingtheoreticalframework..................................................................................20Researchquestions................................................................................................................22

1.4Researchapproachandtheintendedaudience........................................22Usingactionresearch............................................................................................................22Intendedaudience..................................................................................................................23

1.5Overviewofthethesis........................................................................................24

Chapter2 Literaturereview...................................................................................252.1Assessment,technologyande-Portfolios....................................................25Thedefinitionandoriginsofe-Portfolios....................................................................29Typesofe-Portfolios..............................................................................................................31

2.2Pedagogyandprocessine-Portfolios...........................................................34Theimportanceofreflectionandfeedback.................................................................36

2.3Learningcommunitiesandnetworkedlearning.......................................39Typesoflearningcommunities.........................................................................................42Networkedlearning...............................................................................................................43Frameworksusedinnetworkedlearning....................................................................44ApplyingCOItoe-Portfolios...............................................................................................47

2.4Artifactsandtutorrolesinacollaborativeenvironments....................49Theskillofartifactcuration...............................................................................................51Thecomplexityofteacher,tutorandfacilitatorrolesinane-Portfoliocommunity.................................................................................................................................52Criticismsofcommunitybasedmodelsandthedarkside...................................54

2.5Theresearchgapandtheresearchquestions............................................55

Chapter3 Researchdesign.....................................................................................573.1Researchmethodology.......................................................................................57Theactionresearchprocessinthiswork.....................................................................58Casestudy..................................................................................................................................61

3.2Myroleandphilosophicalstanceinthisresearch...................................62

3

Researcher’sexperienceandperspectiveonteachingandlearning...............623.3Theparticipants...................................................................................................643.4Researchmethods,techniquesandprocedure..........................................65Methodologyandmethodsincycleoneandcycletwo..........................................65Datavisualisationusingsocialnetworkanalysis......................................................69Interviews,codingandinterpretativeanalysis.........................................................71Codingportfolioentries.......................................................................................................73

3.5Quality,validityandreliability........................................................................743.6Ethicalconsiderations........................................................................................76

Chapter4 Cycleone...................................................................................................794.1Introductiontothefirstcycle..........................................................................794.2Designsofthelearningcommunityande-Portfolio.................................80Thedesignofthee-Portfoliolearningcommunity...................................................80Thedesignofthee-Portfolio..............................................................................................81Socialnetworkandactivityanalysis..............................................................................86Contentanalysisfromartifacts,commentsandnotes............................................88Contextanalysisfromquestionnairesandlogs.........................................................89

4.3Thecycle..................................................................................................................90Weekonetoweekthree.......................................................................................................90Weekfourandfive.................................................................................................................93Weeksixtoweekeight.........................................................................................................94

4.4Results......................................................................................................................94Activityinthecycle................................................................................................................95Analysisofartifactsandcomments.............................................................................103Questionnaireresponsesandobservationsfromthecycle...............................110

Chapter5 Discussionandreflectiononcycleone.......................................1155.1Discussion............................................................................................................115Researchquestion1:Whatassessmentartifactsemergefromco-operatingparticipantsinalearningcommunity?.......................................................................115Researchquestion2:Howareartifactsshared,usedandreusedbythecommunity?............................................................................................................................120Researchquestion3:Whatistheroleofthetutorandtheformofthecommunity?............................................................................................................................125

5.2Reflectionandfeedingforward....................................................................129Theinductionprocess........................................................................................................129Thee-Portfoliodesign........................................................................................................130Datacollectionmethodsandanalysis.........................................................................131

Chapter6 Cycletwo................................................................................................1336.1Introductiontothecontextofthesecondcycle......................................1336.2Changesforcycletwo.......................................................................................134Ane-moderationframeworkfore-Portfolios.........................................................134Recommendersystems.....................................................................................................136

4

Dataanalytics.........................................................................................................................1376.3Methodology,methodsandimplementationchanges..........................138Methodsandtheiruse.......................................................................................................138Thedesignofthee-Portfoliolearningcommunity................................................139Thee-Portfoliomoderationframework.....................................................................139Therecommendersystemandthelandingpage...................................................143

6.4Thecycle...............................................................................................................144Weekonetoweekthree....................................................................................................144Weekfourandfive..............................................................................................................146Weeksixtoeight..................................................................................................................146

6.5Results...................................................................................................................146Activityinthecycle.............................................................................................................147Analysisofartifactsandcomments.............................................................................154Interviewresponses...........................................................................................................163

Chapter7 Discussionandreflectiononcycletwo.......................................1707.1Discussion............................................................................................................170Researchquestion1:Whatassessmentartifactsemergefromco-operatingparticipantsinalearningcommunity?.......................................................................170Researchquestion2:Howareartifactsshared,usedandreusedbythecommunity?............................................................................................................................173Researchquestion3:Whatistheroleofthetutorandtheformofthecommunity?...............................................................................................................179

7.2Reflection.............................................................................................................185

Chapter8 Discussion,conclusionsandfurtherwork.................................1888.1Addressingtheresearchquestions.............................................................1888.2Implicationsforpractice................................................................................1918.3Reflectionsontheresearchandlimitationsoftheapproach............1928.4Contributionandsuggestionsoffurtherresearch................................195

References......................................................................................................................197

5

AcknowledgementsThisworkwouldnothavebeenpossiblewithoutthesupportofalargenumberofpeople. Iwould liketodedicatethisworktomyteachers,mycolleagues,myfriendsandfamily.Thankstothestudentswhotookpartinthisproject.Iwould like to thankcohortone, for the incredible levelsof support theyhaveofferedover the last six years.Debbie, Jane,Brian, John,Hans, Shuanna, Justin,Chryssa,Enda,Hans,AlexandMarkareanextraordinarygroupofpeopleandIcan’tbelievethegoodfortuneIhadwhenIfellinwiththisgang.Special thanks to my supervisor Dr Julie-Ann Sime and the original LancasterteamofDavid,Gale,MariaandAlice.DeborahAnderson,Dan,PhilandChriswhoencouragedmetoattemptthisinthefirstplace.JohnV.,BrianW.andBobK.,whoweretheinspirationalteacherswhostartedmeonthispathmanyyearsago.ThisworkisdedicatedtomywifeMinandmytwosonsJacobandNick.

6

ListofabbreviationsAPL AccreditationofPriorLearningAR ActionResearchBIS BusinessInformationSystemstaughtmoduleBIT BusinessInformationTechnologyCA ContentAnalysisCMC ComputerMediatedCommunicationcMOOC ConnectivistMOOCCOI CommunityofInquiryCOP CommunityofPracticeCSCL ComputerSupportedCollaborativeLearningCSS CascadingStyleSheetsCxA ContextAnalysisDI DirectInstructionFLOSS FreeLibreOpenSourceSoftwareFSF FreeSoftwareFoundationGNUGPL GNUNotUnixGeneralPublicLicenseHTML HypertextMarkupLanguageICT InformationCommunicationsTechnologyJISC JointInformationSystemsCommitteeLMS LearningManagementSystemMBIT MScBusinessInformationTechnologyMOOC MassiveOpenOnlineCourseNL NetworkedLearningOSDM OpenSourceDevelopmentMethodologyOSI OpenSystemsInterconnectionPLE PersonalLearningEnvironmentSNA SocialNetworkAnalysisSQL StructuredQueryLanguageTEL TechnologyEnhancedLearningTP TeachingPresenceVLE VirtualLearningEnvironmentWSA WebScriptingforApplicationtaughtmoduleWYSIWYG WhatYouSeeIsWhatYouGetXML ExtensibleMarkupLanguagexMOOC InstructivistorconstructivistMOOCZPD ZoneProximalDevelopment

7

SoftwareusedinthisworkSoftwareusedincodingandforgeneralstatisticsExcelATLAS.tiSoftware,languagesandlibrariesusedinthedevelopmentoftheportfolioApacheCSSEditJQuerymySQLmySQLWorkbenchNetvibesIntegratedDevelopmentEnvironmentPHPSMARTYTextmateSoftwaredevelopedinthisprojectisavailableundertheGNUGPLfromwww.foliocube.com/softwareAuthor’snoteonArtifactvsArtefactThis work uses the American spelling of artifact rather than the Europeanspelling artefact. As many of the e-Portfolio implementations are produced inAmerica this spelling dominates the technical literature; it is also common inWenger’s work on the Community of Practice model. Rather than switchingbetweenthetwodependingonthecontext,theUScentricversionhasbeenusedhere.

8

ListoffiguresandtablesFigure1.1–Kolb’sexperientialcycle.......................................................................................20Figure1.2–JISCe-Portfoliolearningprocess......................................................................20Figure2.1-e-Portfoliotypes......................................................................................................32Figure2.2-e-Portfolioimplementations...............................................................................33Figure2.3-Kolb’slearningcycle...............................................................................................37Figure2.4–JISCmodifiedlearningcycle...............................................................................38Figure3.1–DeLaat’sresearchframework...........................................................................65Figure3.2-Sociogram....................................................................................................................70Figure3.3-BespokeRscriptforsociogramproduction.................................................70Figure3.4-Sociogramexample.................................................................................................71Figure4.1-Anexampleartifactrepresentation.................................................................83Figure4.2-Exampleoverviewofnineartifacts..................................................................83Figure4.3-Exampleartifact........................................................................................................84Figure4.4-Examplethumbnailgrid........................................................................................85Figure4.5-Assignedactivitycodes.........................................................................................86Figure4.6-Snapshotofactivityrecordedintheactivitytable....................................87Figure4.7-Activityforweekonebycategory....................................................................87Figure4.8–Exampleindividualactivity................................................................................87Figure4.9-Interactionbetweenparticipantsinaweek.................................................88Figure4.10-Exampleofinitialtutorfeedback...................................................................91Figure4.11-Redesignedfrontpage........................................................................................92Figure4.12-Sociogramsforweeksonetofour...............................................................101Figure4.13-Sociogramsforweeksfivetoeight.............................................................102Figure4.14–Waris’artifact......................................................................................................103Figure4.15–Elizabeth’sartifacts..........................................................................................103Figure4.16-EileenandWaris’useoftags.........................................................................104Figure4.17–Robert’srequestforhelp...............................................................................104Figure 4.18 - An interactionwith a “past” artifact,with comments added three

weeksaftercreation...........................................................................................................107Figure4.19-Temporalbasedtags.........................................................................................108Figure4.20-Codedteachercomments...............................................................................109Figure 5.1 - Sociogram with artifact construction and participation for week

three...........................................................................................................................................121Figure5.2-Artifactsdemonstratingknowledgecascade............................................123Figure6.1–Salmon’sfivestagee-moderationmodel...................................................135Figure6.2-e-PortfoliomoderatorframeworkderivedfromSalmon(2003)....141Figure6.3-Recommendersystemalgorithm..................................................................142

9

Figure6.4–Theredesignedfrontpanel..............................................................................143Figure6.5–Exampleartifactandcomments....................................................................145Figure6.6-Sociogramsforweeksonetofour.................................................................152Figure6.7-Sociogramsforweeksfivetoeight................................................................153Figure6.8-Tagsforweekone.................................................................................................154Figure6.9-Curationartifactsinweektwo........................................................................155Figure6.10-Smalltasksolutionsasartifacts...................................................................156Figure6.11-Curationartifactwithcomments................................................................157Figure6.12-Tagsforweekfour.............................................................................................158Figure6.13-Susancreatingartifactsusingbrowsertests..........................................159Figure6.14-Tagcloudforweeksix......................................................................................160Figure6.15-Artifactframedwithhints..............................................................................160Figure6.16-Artifactseekinghelp.........................................................................................161Figure6.17-Activitybyweek,foranindividualparticipant.....................................163Figure7.1-Richerfolksonomytags......................................................................................172Figure7.2–Sociogramforweekthreewithoverlaidartifactcreationlevels....175Figure7.3–Artifactcreationforsubjectsovertime......................................................179Table3.1-Cyclesintheproject..................................................................................................59Table3.2–Datacollectionincycleone..................................................................................67Table3.3–Datacollectionincycletwo..................................................................................68Table4.1-Gradeprofileforparticipants...............................................................................80Table4.2-Methods,meansandsample.................................................................................86Table4.3-Questionsusedforcontextanalysis..................................................................89Table4.4-Ownandothers’useofartifacts..........................................................................96Table4.5-Generalgraphmeasures.........................................................................................96Table4.6-Activitysummaryfortheeightweeksofcycleone....................................98Table4.7-Degreevaluesforparticipants.............................................................................99Table4.8-Degreedistributionforparticipants.................................................................99Table4.9-Betweennessfiguresforparticipants............................................................100Table4.10-Eigenvectorcentralityforparticipants......................................................100Table4.11-Artifactandcommentcreation......................................................................103Table4.12-Activityforstudentsnotcreatingartifactsinweeksonetothree..105Table4.13–Catchupartifactscreatedinweekfour.....................................................106Table4.14-Participantreportedresources......................................................................111Table4.15-Participantreporteduseofownandothers’work...............................111Table5.1-Artifactuploadcomparedtouseoftheoverviewmechanism...........117Table5.2-Artifactcreationbyweeknumber...................................................................119

10

Table5.3-Viewingandinteractingwithothers’work.................................................121Table5.4-Thumbnailandfolksonomyusage..................................................................121Table5.5-Activitiesofparticipantsontheperiphery..................................................122Table5.6–Artifactcreationbytimeandperiod.............................................................124Table6.1-Methods,meansandsample..............................................................................139Table6.2–Generalstatistics....................................................................................................147Table6.3-Generalgraphmeasures......................................................................................148Table6.4-Activitysummaryfortheeightweeksofcycletwo.................................149Table6.5-Degreevaluesforparticipants..........................................................................150Table6.6-Degreedistributionforparticipants..............................................................150Table6.7-Betweennessfiguresforparticipants............................................................151Table6.8-Eigenvectorcentralityforparticipants.........................................................151Table6.9-Artifactcreationbyweek....................................................................................154Table6.10-Artifactcreationbyday.....................................................................................156Table6.11-Artifactcreationbyday.....................................................................................159Table6.12-Artifactcreationforweekseven...................................................................161Table6.13-Codedteachingpresenceforweeksonetothree..................................162Table6.14-Codedteachingpresenceforweeksfourandfive.................................162Table6.15-Codedteachingpresenceforweeksfourandfive.................................163Table7.1-Curationartifactproductionincycletwo....................................................171Table7.2–Density,reciprocityvalueswithartifactcreation....................................173Table7.3–Activityandartifactscreatedincycleoneandtwo................................174Table7.4–Interactionwithothers’work..........................................................................175Table7.5–Artifactcreationlevelsforparticipantsontheperiphery...................177Table7.6–Activityoftheparticipantsbyband...............................................................178Table7.7–Searchpopularityvsperipherality.................................................................178

11

Chapter1 IntroductionandbackgroundThischapterintroducesthewidercontextforthiswork,describingthewaysinwhich recent technological changes and research into community basedpedagogies has not seen an accompanying advance in e-Portfolio assessmentpractices. It then details the purpose, domain and participants used in thisresearch; undergraduate and postgraduate students on business informationtechnologycoursesinauniversitybasedinouterLondon.Theunderlyingtheoreticalframeworkofnetworkedlearningisdescribedalongwith the way that action research and the open source developmentmethodology have been interleaved in this work. The research questions arethen detailed and the audience of researchers interested in the practices ofassessment in learning communities, or lecturers seeking more sophisticatedassessmentpracticesisthenidentified.Thechapterconcludeswithanoverviewofthestructureofthiswork.

1.1Contextandpurpose

ThewidercontextTechnological advancements, massification and an appreciation of socialconstructionist based pedagogies have combined to offer opportunities forradical change in the way that higher education functions. Popular mobiledevices havemore computing power and connectivity than what was used topowermultinational corporations twenty years ago and it is commonplace foreachinherentlytechnology-savvycohort(Margaryan,Littlejohn,&Vojt,2011)tohavebroughttheirowndeviceswithgreatercomputingpowerthanthatofferedbytheinstitutiontheyarejoining(Johnsonetal.,2013).

WesternEurope’sshifttoknowledge-basedeconomieshassignalledadesiretoincreasegraduatenumbersalthoughthishasbeenparadoxicallyaccompaniedbya reduced resource allocation (Altbach, Reisberg, & Rumbley, 2010). Highereducation institutions have seen the cheapening and increasing availability oftechnologyinaseriesofrevolutionarymovements,eachpromisingsolutionstothe issues brought about by globalisation and massification (Selwyn, 2007).Blendedlearningandtechnologyenhancedlearning(TEL)havebeenseenasthelatest “silver bullet” approaches (Watson, 2006) that could slay this crisis,improvingstudentaccess,engagementandparticipationinthelearningprocess(Garrison& Kanuka, 2004). Formost institutions this is implemented throughtheuseofalearningmanagementsystem,suchasBlackboardortheopensource

12

softwareMoodle.After the rise, and in some cases fall, of technologies suchasMyspace,BeboandFacebook, learningmanagementsystemshaveattemptedtorespond to thedemand for social interactivity (Dalsgaard, 2006)bybolting oncommunity forums, blogs, social networking and collaborative tagging oflearningartifactscreatingseparatedistinctlearningspaces.Thesimultaneousresurgenceofresearchintocommunitybasedpedagogiessuchascommunitiesofpractice,learningcommunitiesandthecurrentin-vogueideaof massive open online courses (MOOCs) have suggested a greateracknowledgementof thepower that can come from learning in a group.Thesesocialbasedpedagogiesofferfoundationalsupporttothetechnologicalsolutions,assertingthatpeerbasedlearningandfeedback,enhancedthroughappropriatetechnology can support engagement, enculturalisation and better forms oflearningthroughcollaborativeactivitiesanddialogue(JISC,2010).Theycanalsoempower, allowing greater student autonomy (Beetham & Sharpe, 2007),aligningwiththeprinciplesofsustainablelifelonglearning(Knapper&Cropley,2000).Despite theseadvances, thenatureofassessmentpractices in theacademyhasnotdramaticallychanged–theoverridingassessmentmethodologyisstillbuiltaround marking and measurement rather than on learning (Price, Carroll,O'Donovan,&Rust,2011).Thelackofinterestinmoresophisticatedassessmentishamperedbynarrowlegacypractices(Delandshere,2001);exacerbatedbythecosts of massification (Gibbs & Simpson, 2004) and the lure of cheap pre-packaged cartridges of material, which come with aligned multiple choicequestions, reigniting the debate about the applicability of such tests whenassessing the demonstration of higher order approaches to learning (Scouller,1998). Knight (2002) highlights a number of critical issues in assessmentpractices, including non-transferability of results, limitations in criteria-referencing and poor reusability of the results in future career guidance.Studentscanreadilyaccessopeneducationalresources,share,reshapeandmix-upmaterialsfreelyonline,inwayswhichoftenconflictwithtraditionalnotionsofauthorship,ownershipandplagiarism.Findingawaytointegratethesepracticesintoassessmentchallengestheideaofatutordesignedassessmentandtheroleofassessorasfinalarbiter.E-Portfolios offer a solution to many of these issues, using technology toempowerlearnersindecidingwhatevidencetheypresentandallowingthistobe

13

placed in a social context, promoting reflection and peer collaboration(Stevenson, 2006). They can be used for career preparation, and credentialdocumentation(Lorenzo&Ittelson,2005);asworkspaceandshowcase(Barrett,2010); as sonnet,mirror ormap (Diez, 1994). The swift uptake of e-Portfolioshas seen a proliferation of commercial offerings becoming available, oftensupplied as additions to institutional learningmanagement systems. These arecomplimentedbynumerousopensourceimplementations,whicharedevelopedandmaintainedbythecodingcommunityatlarge.E-Portfoliopedagogysitsinaconstructivistphilosophy,but theadditionof socialnetworking tomanyof theimplementations has not seen a revision in the underlying theory as it hasoutpaced by practice (Stefani, Mason, & Pegler, 2007). Extensions to Kolb’sreflective cycle are often suggested as away to include social activities in theartifact constructive cycle (Gray, 2008), but these lack reference to anappropriate community basedpedagogy.Much of the e-Portfolio literature hasbeen criticised in lacking veracity or depth, leading to a call formore rigorousresearch(Rhodes,Chen,Watson,&Garrison,2014).Networked learning (NL)offers a framework to groundan e-Portfolio learningcommunity, situated in social constructivism. NL advocates supportingconnectionsbetweenlearnersandtheelectronicresourcesavailable(Goodyear,Jones, Asensio, Hodgson, & Steeples, 2004) and differentiates itself byacknowledging the importance of weak ties along with the non-privileging ofparticulartypesofrelationships(Jones&Esnault,2004;Ryberg&Larsen,2008),which distinguishes it from other community based models where strongrelationships and human-human relations are emphasised. The richness ofrelationshipspossiblethroughnetworks(Granovetter,1973)areacknowledgedthrough the variability of tie-features, which include emotional intensity,measuresoftime,intimacyandreciprocalservices(Jones,Ferreday,&Hodgson,2008).Ane-PortfoliolearningcommunitybasedonNLwouldbenefitfromboththese characteristics and the rich theoretical underpinnings that have beenvalidatedoverthelastdecade.

ResearchpurposeTheoverallpurposeofthisstudyistoinvestigatethewaysinwhichnetworkedlearning canbeused as theunderlying communitymodel for the collaborativeactivities that occur during the use of an e-Portfolio. Despite the breadth ofresearchinlearningcommunitiesand(separately)inassessmentpractices,thereis a gap in the literature in how the practices andwork constructed during acommunities’lifecyclecouldbeused,bothforthegrowthofthecommunityand

14

asevidenceforassessmentinsidethee-Portfolioitself.Wellplannedhighqualityassessment is vital for academic learning communities, providing feedbackopportunities forevaluationand improvement(Matthews,Smith,&MacGregor,2012;Moule,2007).Principleslongheldinnetworkedlearningsuchasdialogue,reflection and the nature of power relationships are noted as becomingincreasingly important in e-assessment (McConnell, 2006; Whitelock, 2009).Combininge-Portfoliosandnetworkedlearninginthiswayshouldreinforcetheuseofboth.E-Portfolio assessment emphasises the importance of an individual taking agreater control in the choice of work to be presented, compiled using the e-Portfolioarchitectureforanagreedpurposeandaudience.Bestpracticeadvisesthat each participant is given the opportunity to take part in reflective cycles(afterDewey,KolbandSchön), to improve the createdartifactsover time.Thepossibilities of community participation to allow for deeper learning(Ehiyazaryan-White, 2012; Tosh,Werdmuller, Chen, Light, & Haywood, 2006),suggestsincreasingtheuseoffeedbackfrombothtutorandpeers,aprocessthathasbeenenabledbymanymanufacturersboltingsocialnetworkingfeaturesintoe-Portfolios implementations.E-Portfoliousehas increasedintheacademy,buttheyarefrequentlyusedforlittlemorethanassessmentandreflection(Schwier,2001; Sherman, 2006). More recent analysis of the potential of e-Portfoliossuggests thatabettereducationalexperiencecanresult fromtheconsiderationand embedding of the roles of artifact creation and goal setting, with anacknowledgement of the advantages that would come from the promotion ofimprovedinteraction(Chang&Tseng,2009;Jones&Peachey,2005).Becauseofalackofclarityintheapplicationofanappropriatesocialpedagogyinthe e-Portfolio literature, the nature of the participation, the philosophy of thecommunity and the way in which such contributions should be shown areunclear.The researchpresentedherewill suggest thatnetworked learning canprovideanappropriatepedagogyfore-Portfolios,wheretheartifactscreatedbyindividuals in the NL context can be classified as resources that are shared,reusedandcollaboratedonandwith.Despite its history, NL has to an extent been side-lined by the popularity ofcertain technologies and the claims of the connectivist community. Personallearning environments (PLEs) were initially a rebuttal to the monolithicinstitutional LMS, which was seen as controlled, archaic and unresponsive to

15

change. Social technologies such as Twitter and LinkedIn, include socialinteractionandextendthePLEconceptintopersonallearningnetworks(PLNs),but it could be questionedwhether these technologies promote individualisedlearning, rather than community and mutual engagement (Çimer, 2011;Dirckinck-Holmfeld, Hodgson, &McConnell, 2011). MOOCs currently dominatethe e-learning debate eitherwith an underlying, as yet unverified, connectivistapproach (cMOOCs), ormore commonly asmore open versions of commerciallearning management systems with little obvious pedagogy (xMOOCs)(Liyanagunawardena & Adams, 2013). This work will seek to reaffirm thevalidityofnetworkedlearningintheuseofe-Portfolios.PowerrelationsinassessmentThereisaconflictbetweenthenecessityofdemonstratingindividuallearningtosatisfy institutional requirements, and thenature of collaborative learning in acommunity. In subjects where there are many small repetitive skills or smallsubsets of knowledge, the ability for students to evidence their individuallearning is small – in programming, for example, declaring variables orperformingsimplemathematicaltasksbecometrivialduplicatetasks.Itisintheapplicationofthisknowledgeandtheseskillsinmediumtolongertasks,withastudent initiated focus, that better assessment becomes possible. E-Portfoliosallowforthenatureoftheassessmentartifactstobedecidedbythecommunityitself.Theunderlyingprinciplesofpeerbased learningcommunitiesalignwiththeempowermentthatcomesfromallowinglearnerstodictatethenatureofthematerialcreated,curatedandpresented.Thisshouldacttorebalancethetutor-student relationship, allowing the participants to alter the power relationshipsimplicit in traditional assessment practices (Huot & Williamson, 1997). Thiswork will also address the call for further research on peer-review in e-assessment(DeLaat,Lally,Lipponen,&Simons,2006a;Stödberg,2012).Whatthisworkwilldo–thecontributionDespite the increase in the availability of technology in education and themovement towards the read/write web (Web 2.0), Dirckinck (2011), suggeststhatitisunclearwhetherthepotentialandpromiseofnetworkedlearningposedbyHarasim(1995)hasbeenachieved.This work will construct a framework for e-Portfolio assessment practices innetworked learning based communities. This should serve to provide atheoretical approach, practical examples of the process, and a reference

16

implementation to enable an assessment process that supports collaboration.Using the notion of a learning community based on networked learningprincipleswhereco-operatingpeersareencouragedtocreateandshareartifacts,thisworkattempts todeviseaway for theparticipant’sartifacts, thepath theyfollowthroughtheirlearningandtheconnectionstheymaketoformanoverallpicture that can be used for assessment purposes. Rather than being tutordirected,theshape,pathsandthenatureoftheworkconstructedshouldemergefromthepeersthemselves.PersonalreasonsforapproachAs an action research (AR) project, amajor aim of this work is to changemypractice, allowing the opportunity for reflection and improvement in theassessmentprocesses that areused inmy teaching.AligningARwith theopensourcedevelopmentmethodology(OSDM)ensuresthatanysoftwaregeneratedin the project can be used elsewhere and that the participants will learn theprinciplesofbothARandOSDM,whichisavaluableoutcomeinitself.Awiderdiscussionof learningcommunities,networkedlearningandtheuseofe-PortfoliosinassessmentwillbeexploredintheliteraturereviewinChapter2.The followingsectionexplores the localdomain inwhich thiswork is situated,alongwithdetailontheparticipantsavailableforthestudy.

1.2Domain

AbouttheresearchsiteKingston University is amedium sized andmiddle ranked university 10milesoutsideLondon.Ithasastudentbasewithadiverseethnicity,alargenumberofstudents who are the first entrants into higher education, and a wideningparticipation agenda that is reinforced by the links it maintains with furthereducationcollegesinthelocalarea(McDuff&Marcelline,2012).Whenapolytechnic,theinstitutionrecruitedstudentswithlowerAlevelgradesandcompensatedforthiswithmorestaffcontacttime,alowerstaffstudentratioandafocusonavarietyofauthenticassessments,usingcloselinkswithbusinesstosituatelearninginrealisticsettings.In 1992, the polytechnic became a university and followed the government’smassification policy to dramatically increase its intake – a policy aimed atincreasingparticipationinhighereducationto50%forall18-24yearolds.Asa

17

result, thenumberof full timestudents increased from10400 in1997tomorethan 18500. This placed tensions on a stretched resource base and wascounteredwithmodularisationandanembrace in theuseofblended learning.Modularisation has been a favoured approach in dealing with an upsurge innumbers - finding common subjects across degrees, and unifying them into asingle deliverymodel can have apparent cost and efficiency gains. Common inAmerica, it isalsoclaimedthatmodularstructurescanprovidemorechoiceforstudents,allowingthemtopersonalisetheirdegree(Goldschmid&Goldschmid,1973).The business faculty is one of the largest in the university, with 5000undergraduate and postgraduate students on general business courses andspecialised degrees such as Accounting, Marketing and Business InformationTechnology. Inthefacultytheconsequenceofmassificationandmodularisationhasbeenmuchlargerclasssizes,particularlyintheearlieryearsofthecourseswheremodules can have up to 750 students enrolled on them. There is somedegree of course identity loss, as studentsmixwithmore students from otherdegree courses, rarely repeatedly only interacting with students on their owndegree.Attendanceformostfirstyearlecturesinthebusinessfacultyistypically50-70%, which indicates that the ‘churn’ rate for physical attendance is high,with many students not attending every session and using the learningmanagement system for missed materials. Contact time and opportunities forassessment has been reduced, but these are aligned with offerings fromcompetitorinstitutionsoperatinginthesamesector.TheFacultyofBusinessatKingstonUniversityhasadistinctphilosophytowardsintegrating technical and theoretical practices. It has specific courses,modulesandstaffdeliveringcomputingandinformationtechnologysubjectstobusinessstudents tocreatewhatwereoriginallycalledhybridmanagers,graduatesableto use, create andmanage technology in business processes (Palmer & Ottley,1990).Despite the termhybridmanager falling out of popular use, there havebeen recent calls to re-engage with the concept (Brackley, 2013) and thephilosophyremainsintheundergraduateBScBusinessInformationTechnologycourse and theMScBusiness InformationTechnologypostgraduate conversioncourse. The modularisation structure means that specialist and non-specialiststudents,atbothundergraduateandatpostgraduatelevel,cantakemodulesthatcover systems analysis and design, along with implementation skills such asprogramminginserversidescriptinganddatabaselanguages.

18

PedagogicpracticesandtheparticipantsavailableforthestudyKingstonhasembeddedtechnologyenhancedlearningpracticesthroughouttheinstitution, investing in learning management systems (Heaton Shrestha,Edirisingha, Burke, & Linsey, 2005) and advocating a hybrid blended learningapproach(Garrison&Kanuka,2004;Martyn,2003;Singh,2003)withe-learningtechnologists situated in each department (Ooms, Burke, Linsey, & HeatonShrestha, 2008). Despite an increase in the use of the system for distributinglecturematerials, theuse of themore sophisticated collaborative tools such associal media, blogs and wikis remains low. Like most institutions, LMS usagestatistics brokendownby feature are not in the public domain, butKingston’sfiguresalignwiththoseinstitutionsthatdopublishthesedetails(Ashford-Rowe,2013;UniversityofQueensland,2013).AssessmenttechnologiesintheLMSareautomatedmultiplechoicetesting,alongwith online short and long answer interfaces. The facilities for short and longanswerquestions,alongwiththedigitaldropbox,areelectronicmanifestationsofexistingassessmentpracticeswhereissuesofalignment,authenticity,validityandsustainabilityapply. The increasinguseofautomaticallymarkedmultiple-choicetestshasreignitedanolddebateabouttheusefulness(andconsequences)of their use. Generally though, the assessment features are particularlyunderutilised,whichisoneofthecorereasonsfortheinitiationofthisproject.Forthisstudy,undergraduateandpostgraduatestudentsontwotaughtmoduleswillbeavailabletoparticipate:

• WebScriptingforApplications,and• BusinessInternetSystems.

Web Scripting for Applications is a second year undergraduate module, withpredominantly technically capable students. Typically 75-80% of the studentstend to be on a BSc Business Information Technology (BIT) course, with theremainder coming from the other general or specialised degrees. Studentselectingto takethemodule fromoutsidetheBITcoursehavetodemonstratealevel of technical aptitude, either in their first year module choices or fromprevious experience. Class sizes are between 30-50. In its original form, thismoduleusedanassignmentandexamforassessment.

19

BusinessInternetSystemsisapostgraduatemoduleofferedonMlevelcourses,such as the MSc in Business Information Technology and the MSc AccountingInformationSystems.Boththesecoursesareconversiondegrees,withclasssizesbetween 13-20. Themodule is placed in the second half of the academic year,afterafirstsemestertechnologymoduleensuresthattherequisitetechnicallevelhasbeenachieved.Assessmentonthismodulewastraditionallybygroupworkandexam.Bothmoduleshaveablendeddeliverypattern,withface-to-facetaughtsessionsinpracticalandtheoreticalexercises in laboratorysettings,deliveredovera12week period embedded in a semesterised pattern. Likemost higher educationinstitutions, there are no formal attendance requirements, although somemodulesmonitorattendanceforformativefeedbackprocesses.Inbothmodules,thereisevidencethatstudentshavevariableengagement,withmoreintensityofactivity,effortandattendanceoccurringaroundassessments.Thiswork presents two phases of an on-going project, in two action researchcycles:

• CycleoneintheundergraduateWebScriptingforApplicationmoduleforeightweeks.

• CycletwointhepostgraduateBusinessInternetSystemsmodulefor

eightweeks.The placement of the cycles has been designed to fit with the pattern of theacademicyear, theavailabilityof thestudents, toallowgaps fordevelopmentalworkandspaceforreflectiononeachactionresearchcycle.Cycleonerunsfromthe midpoint of the second year module, whilst cycle two starts from thebeginningofthepostgraduatemodule.

20

1.3Theframeworkandresearchquestions

UnderlyingtheoreticalframeworkThisstudyissituatedininterpretivism,withitsacknowledgementofsubjectiveperceptions of truth and the consequences of this for research methodology.Phenomena are considered in their natural environment and this interventionaffectstherealityofthestudy,whichinitselfformspart of the interpretation.Action researchwill beusedfortheinterventionhere,aimingfortheorytobe intertwined with practical emancipatoryoutcomes.

E-PortfoliopedagogyisbasedonanexperientialcyclesituatedinDewey’sworkon learning, experience and transformation into action. A four step reflectivelearningcycleismostfrequentlycited(figure1.1,fromKolb,1984),whichKolbderived using models constructed from Dewey, Lewin and Piaget’s learningapproaches.Attemptsatreframingtheexperientialcycleforportfoliostoincludean interaction element bolt on additional phases to include collaboration andfeedback(JISC,2008),buttheunderlyingprocessisnotclear(figure1.2).

Thisworkusesanetworked learningbasedcommunity toclarify thenatureofthecollaborationandcommunitydrivenfeedbackthatshouldbeused.Learningcommunities vary in definition, underlying pedagogy and technology, withdevelopment occurring in both practical settings and in theoretical discourse.Networked learning has differentiated itself by emphasising the peer-basednature of the learning community used, although this is not explicit in the NLdefinition:

Figure1.1–Kolb’sexperientialcycle

Figure1.2–JISCe-Portfoliolearningprocess

21

“’networked learning’ is learning in which information andcommunications technology is used to promote connections:between one learner and other learners, between learners andtutors;betweenalearningcommunityanditslearningresources”(Jones,Asensio,&Goodyear,2000,p.18).

Aligning the networked learning definition with an appropriate e-Portfoliodesignallowstherepositoryofparticipantgeneratedartifactstoactaslearningresources tobe sharedbetween the communicatingpeers. In this contextboththetutorandthee-Portfoliosystemitselfwillactastheagentsundertakingthepromotion of these connections. The literature on learning communities alsoprovidesagreatertheoreticalunderpinningonthenatureofsharedartifacts.Thepopularityintheuseoflearningcommunitieshascomefromtheemergenceofcommunitiesofpractice(Clow,2013;Wenger,1999;2002),therediscoveryofVygotsky’swork on zones of proximal development (ZPDs) and the increasingavailability of social based technologies. The overarching philosophy in sociallearning systems is that deeper knowledge and understanding comes from co-constructed learning, acknowledging that cultural knowledge is situated in asocial cultural context. Meaningful learning has participation and reification(makingintoanobject),

“theprocessofcreatingproducephysicalandconceptualartifacts—words, tools, concepts, methods, stories, documents, links toresources, and other forms of reification—that reflect our sharedexperience and around which we organize our participation”(Wenger,2010,p.1).

Numerous frameworks and analytical approaches have been applied innetworked learning (Conole,2010; Siemens&Long,2011).Thiswork seeks toimprove practice and as such uses action research and a thematic analyticalframeworkwithanunderlyingmixedmethodsapproach,atransformativestudy,whereatriangulationofquantitativeandqualitativedatawillbethebestwaytoconverge the information (Creswell, 2013; Kennedy, Ioannou, Zhou, Bailey, &O'Leary, 2013). Mixed methods offers the possibilities of cross validation andcorroboration, but also an increase in required effort and the possibilities ofdiscrepancies (p. 217). Thematic analysis will be used here to qualitativelyinterpret and organise the data, using analytics, social network analysis and

22

interviews, an approach that is suggested as appropriate for the complexity oftheactivitytakingplace(Campbell,DeBlois,&Oblinger,2007;DeLaat,2005;DeLaat&Lally,2003;2004;DeLaat,Lally,Lipponen,&Simons,2006a;2007).

ResearchquestionsResearch into the use of e-Portfolios technology commonly focuses on theindividual learner, with the notion of the community either missing from thediscussionorbrieflymentionedaspartofadesigntosatisfypeerappraisal.Thiswork flips this, using a networked learning community as the core, withparticipant’sartifactsplacedinasharedcommunalspaceforpeerappraisal.Theresearchquestionsfocusonthenatureofartifactsthatarecreated,howtheyareusedinthecommunityandtheformofthecommunityitself:

Researchquestion1: Whatassessmentartifactsemergefromco-operatingparticipantsinalearningcommunity?

Researchquestion2: How are artifacts shared, used and reused by the

community?Researchquestion3: What is the role of the tutor and the form of the

community?

1.4Researchapproachandtheintendedaudience

UsingactionresearchThisworkwill use action research for anumberof reasons - it iswork that isinformedbyadegreeofpragmatisminthatitseekstoimprovemyownpracticeandthebeliefthatthechangeprocesscanbeemancipatoryfortheparticipants.Thiswillresultinbetterwaysofintegratingtechnology,learningprocessesandassessment practices. There is a long history in the use of action research ineducational projects as a consequence of the teacher as researchermovement,which advocated that curriculum research and development ought to be theremitoftheteacher(Clow,2012;Elliott,1991;Stenhouse,1975).Actionresearchconcernsanimprovementofpractice,intheunderstandingofpracticeandinthesituationinwhichpracticetakesplace(Carr&Kemmis,2003).TheemancipatorynatureofARcorrelateswiththenetworkedlearningpeercommunityphilosophyand there aremany example of its use in NL projects (Clow, 2013; Dirckinck-Holmfeld&Jones,2009;McConnell,2006).

23

The open source development methodology (OSDM) has principles andprocesses that naturally alignwith action research, including cycles of activity,peer like relationships between participants (participant as researcher in AR,participantasco-developer inOSDM)andacriticalagenda inthe freesoftwaremovements freedoms enshrined in the principles of the GNU general publiclicense(GNUGPL).Theuseofactionresearchandtheopensourcedevelopmentmethodology in this project are discussed in more detail in chapter 3, whichdealswiththeresearchdesign.

IntendedaudienceThe researchmayproveuseful to educators in promoting learning communityuseforassessment.Despitetheavailabilityofforumsande-Portfoliotechnologyin learning management systems, usage remains low. By demonstrating theadvantagesthatarrivefromcombiningthese, it ishopedthattherecouldbeanincrease in use, along with an acknowledgment that more sophisticatedframeworks and implementations are required. The e-Portfolio moderationmodel suggested here may be useful in other settings and the applicationdevelopedduringthisprojectwillbeplacedunderafreesoftwarelicense,whichmeans that the code is available for free for any further researcher to use,developorintegrateinfurtherprojects.Emphasisingtheuseof thenetworkedlearningcommunitybasedmodel isalsoimportant. NL has a rich history demonstrated by conferences, research,analytical frameworks and practitioners and it is hoped that this work willcontributetothatfield.

24

1.5OverviewofthethesisThisdocumenthasthefollowingstructure:Chapter2 Contains a literature review concerning learning

communities,networkedlearningande-Portfolios.

Chapter3 Containstheresearchdesign,themethodologyandmethodsusedfordatacollection,alongwiththepedagogyandethicalapproachesused.

Chapter4 Explores the first action research cycle, with second yearundergraduate students. Data is collected through the e-Portfolio artifacts, posts and activity in the learningcommunity,alongwithapostcyclequestionnaire.

Chapter5 Contains the discussion of the data from the first cycle, areflectionontheprocessandchangestobefedforwardintothesecondcycle.

Chapter6 Exploresthesecondactionresearchcycle,withpostgraduatestudents. Data is collected through the e-Portfolio artifacts,postsandactivityinthelearningcommunity,alongwithpostcycleinterviews.

Chapter7 Containsthediscussionoftheresultsfromthesecondcycle,a reflection and suggested improvements thatwould carryforward.

Chapter8

Conclusions, answering the research questions, reflections,implicationsandsuggestionsforfurtherwork.

References

25

Chapter2 LiteraturereviewThis chapter reviews the literature that informs my research. I begin byexploring themajormovements in assessment to derive the characteristics ofbetter assessment practices. I then trace the three distinct phases of portfoliodevelopmentthroughanalogue,digitalandonlinearchitectures,andthewaysinwhich they are categorised and used. I follow thiswith a consideration of theunderlying pedagogy for e-Portfolios, showing how the extension of Web 2.0social technologies intoportfolioshas resulted inpoorormissingexplanationsforboththenatureoftheartifactsandtheunderlyingcommunitythatexists inthiscontext.Iargue that the learningcommunity literature, specificallynetworked learning,canofferanapplicabletheorythatcanbeusedtoaddressthisgapandtosituatethelearningandassessmentthatoccursaroundportfolios.Frameworksusedintheanalysisofnetworkedlearningcommunities,suchasCommunityofPracticeand Community of Inquiry, are shown to provide a better description of an e-Portfolio artifact and a model for the analysis of the community interactionthroughwhichartifactsarecreated.Learning communities are not without difficulties, and a discussion of theinconsistencies and possible dark sides of community-think are explored. Theliterature review concludes with a discussion of the contributionmy researchmakestotheunderstandingandimplementationofe-Portfoliosbasedaroundasocialpedagogy.

2.1Assessment,technologyande-PortfoliosBoud(1995)suggeststhattherehavebeenfourdistincterasinthedevelopmentof assessment practices, which he delineates as traditional summativeassessment;assessmentasmeasure;competencyorauthenticassessment;andaperiodofabroaderholisticapproach.Theearliestformsofassessmentderivedfromthescientificmodel,wherepost-teachingtestingoccurredtoseeifnecessaryfactsandskillshadbeenacquired.Aconsequenceofthespaceraceinthe1960’swasthattheperceivedinadequaciesof the educational process came under higher levels of scrutiny. Assessmentresultsbecameanobvious,butblunt,quantifiablemeasureofthequalityoftheteaching process and an indicator of institutional achievement (Eisner, 2003).Assessmentfellintotwobroadtypesandlanguagewasdevelopedtodistinguishbetweenthem;formativeevaluationwherethefeedbackwasusedreflectivelyto

26

improve performance and summative evaluation, typically in the form ofterminal assessment, to decide or sort (Scriven, 1966). The constructivealignment movement can be seen as a response to this. If assessment is anindicatorofthequalityofaprocess,closelyaligningthelearningobjectivesandoutcomes to the assessmentprocesswaspredicted to raise thequality of both(Biggs, 1996). In practice, the possibility of overly prescriptive, subjectivelearning outcomes can fail to represent the complexity of the learning takingplace(Entwistle,2005;Hussey&Smith,2008).Biggs’approach,where‘studentsareentrappedinthiswebofconsistency,optimisingthelikelihoodthattheywillengage theappropriate learningactivities’ (Biggs&Tang,2010,p.54)canalsoremovetheopportunitiesforself-direction(Beetham&Sharpe,2007)or,asevenBiggs(2014)suggests,supportamanagerialapproach.The increasing influence of the employability agenda can be seen in theintroduction of competency measurement and authentic assessment. Rubricsindicatingkindsofworktobeperformedalongwithlevelstobeacquired,areamorenatural indictorofworkplaceprogress than learningoutcomes(Andrade,1997;Bean,2005).Authenticassessments,whereproblemsorrealistictasksarewrapped in real world simulations and a context that resembles professionalpractice, are seen as ensuring the appropriateness and transferability ofknowledgeandskills(Wiggins,1999).Suchsimulationsmustbeauthenticandbeperceived to be authentic to succeed (Gulikers, Bastiaens, Kirschner, & Kester,2006). The consequential validity of assessment, specifically the intended andunintendedeffectsonthelearningprocess,canbeseenintheideaofsustainableassessment. This addresses the on-going education of the student after thecoursecompletes(Boud,2000),emphasisingadualitywhereassessmentisusedbothduringtheteachingprocessandalsolaterinlifetoincreasethefacilityforself-assessmentandreflection(Boud&Walker,1998).Acknowledging that students have differing initial skills and requirements,accreditationforpriorlearning(APL),hasbeenpopularintertiaryorvocationaltraining, traditionally through portfolio building and rubrics. There are highereducationmodels inuse,suchastheprior learningassessmentandrecognitionmovement inCanada (Wong, 2008) and theEuropeanBologna scheme (Adam,2002). Despite the possibility of credit movement based on APL and QAAguidelinesadvocatingitsuse(2004),therehasnotbeenasignificanttakeupofthisapproachinUKhighereducation.

27

Theoveremphasisonsummativeresultsledtoaresurgenceinsupportiveworkon formative feedback in the assessment for learning movement. Here,assessment is part of a continuous process, which focuses on the future pathlearners are to follow (Wiliam, 2006),with verbal orwritten feedback feedingforward, indicating the progression and direction for the learner (Black &Wiliam,1998).Currentdefinitionsemphasisetheappropriatenessoftheforwardpath, suggesting that this directionwill be better than that whichwould havebeen taken without the elicited evidence (Black & Wiliam, 2009a). Formativeassessment processes are less concerned with reliability, but by rapid andfrequent feedbackwhich suggests validity through improved action (Harlen &James,1997).Formativeassessment tends tobecriterionbasedandoccursduringa learningactivitywiththeaimofactivatingstudentssothattheybecomeownersoftheirown learning (Wiliam, 2011), providing a richer picture compared torepresentations possible through summative measures (Yorke, 2005). Thetraditionalroleoftheteacherasassessorcanbere-evaluated,seeninthemanyexamples of formative feedback processes advocating reflection, co-operationandcollaborationusingself,peerandco-reviewofwork(Bostock,2000;Boud,Cohen, & Sampson, 2001; Dochy, Segers, & Sluijsmans, 1999; Gielen, Dochy, &Onghena,2011).Inthiscontextpeerandcommunityreviewcantackleissuesofplagiarism, by suggesting that it is a cultural practice that involves socialrelationships, identity,valuesandattitudes(Valentine,2006).Solutions involveasking the community to self-police its own work, embodying a culture ofhonestyandintegrity(Kenny,2007).The lifelong learning movement can be seen as a counter to the alignment,competencyandauthenticassessmentstrategies.Thispromotesaholisticviewofeducationbasedonbalance, inclusionandconnectionwherestudentcentredlearningisintegratedintoawholeworldcontextofgrowth,processandpersonaldevelopment (Miller, 2007). Despite a focus on learning how to learn in thiscontext, there is frequently a failure to acknowledge the requirement for thecapacity to be developed of assessment; of formatively determining what hasbeen learnt and then planning future actions accordingly (Boud & Falchikov,2006).Thefullconsequencesintheadvancesofwebtechnologiestoassessmentarestillbeingfelt.Theclosestmodelstotheread/writewebareapproachesthatinvolve

28

the students in a collaborative constructivist process such as in discussionforums, setting assessment goals (Rust, Price, & OʼDonovan, 2003) andparticipating in self and peer marking processes (Orsmond, Merry, & Reiling,2000). Thesework best when embedded in a processwith a distinct learningpedagogy,suchasresource-basedlearning,problem-basedlearningoroneofthelearning communitymodels. Assigning individualmarks can be problematic inthese rich environments, particularly if the role of assessor has beendecentralised.Someattemptsatmeasuringamountsordegreesofparticipationhavebeenattemptedusingportfolios,dataminingorcontentanalysis(Blignaut,Blignaut,&Trollip,2003;Dringus&Ellis,2005;Littlejohn&Pegler,2004).Mostportfolio style approaches require students to present artifacts resulting fromtheir participation, along with justifications for their inclusion to add context(Macdonald&Twining,2002;McConnell,2000).Virtual learningenvironments (VLEs)are themain institutionalmechanism fordeliveringtechnologyenhancedlearning,whoseintroductioncoincidedwiththepopularity of the blended learning movement. Initially providing simple filemanagement capabilities, they have transitioned by adding features such asassessment,studentmanagement,collaborationandcommunication.Therecentadditionoftechnologiestypicallyassociatedwiththeweb2.0movementsuchasblogs,wikisandpodcastshavesuggestedamovementaway fromtransmissionbased pedagogies towards more constructivist approaches. Despite theseadvances,VLEstypicallyofferassessmentmechanismswhichareonlineversionsof familiar analogue methods such as short answer and multiple choice tests.Longerreportsorassignmentscanbesubmittedthroughdigitaldropboxes,andscannedbyplagiarismsystems.Thereareafewexamplesofnon-traditionaluseof these (Draaijer & van Boxel, 2006; Ledwith & Rísquez, 2008), but mostpractices replicate the traditional tutor-ledmarking processes. Themajority ofthe blended learning texts focus purely on learning and teachingmechanisms,assuming that the assessment tools supplied by the VLE systems will beappropriate. The poor use of such tools coupled with the perception of a lowconsequential validity can induce a negative backwash effect (Gielen,Dochy,&Dierick,2003)orreignitetheargumentsovertheassessmentofknowledgeandskillsatthehigherendofBloom’staxonomy(Andersonetal.,2000)inmultiplechoicetests(Scouller,1998;Woodford&Bancroft,2005).Blogs,wikisandpodcastinginassessmenttypicallyusesocialconstructivistandcommunity based pedagogies (Bruns & Humphreys, 2005; Williams & Jacobs,

29

2004). Although initially difficult to construct and integrate into othertechnologies, the arrival of sophisticated web toolkits has allowed for thebuilding of more complex interactive and responsive based learning andassessmentapplications.PerhapsbecauseofthelackofsuchsophisticationintheVLE tools, there has been a rise in the use of personal learning environments(Beauvoir,2010)andpersonallearningnetworks.Thesearealoosecollectionofweb based tools bound together by the user, typically consisting of a blog,bookmark manager, wikis for collaborative work, themed social networkingusingproductssuchasNING(2014)andapersonalisedhomepage.TheseexistoutsidetheinstitutionalVLEdomainandtheclosestthingtoassessmenthereistheuse of public personal learning journals, frequently becoming theplace forparticipatoryreflection(Attwell,2008).The wide variety of types, technologies and underlying theories results inassessment being used in numerous ways, ranging from temperature taking,gate-keeping, assessment of course objectives, feedback for teachers and forassessment of the quality of the educational process (Eisner, 2003). Thesemultipleusesledsometodecrythewholesummativemodelasbeingindisarray(Knight, 2002), particularly as studies indicate that the knowledge and use ofmanyofthemoreadvancedpracticesarelowamongststaffinhighereducation(Taras, 2008). The idea of a hidden curriculum,where the actual learning thattakesplace isseparateanddistinct fromthe lecturehallorclassroomactivities(Snyder,1971),suggeststhatassessmenthasbeenandwillremainakeytoolintheinteractionbetweenstaffandstudent.

Thedefinitionandoriginsofe-PortfoliosPortfolioshavebeenanacceptedformforpresentingstudentworkfordecades,withanorigininarteducationthroughtotheirmorerecentuseincompetencybasedvocationalqualifications.Perhapsbecauseofthebroaduseofportfoliosina variety of different fields and implementations, there is no single agreeddefinition for the term. Some focus on the nature of the collated work, “aportfolio is a placewhere a student’s selectedwork is kept, ... [any] containerdesignedorcreatedbythestudenttoholdhisorherartifacts”(Graves,1994,p.171),“apurposefulcollectionofstudent’sworkthatillustratesefforts,progressand achievement” (Barrett, 1998, p. 6). Other definitions focus on how thecollation process itself can reveal a view of student performance in context(Paulson, Paulson, & Meyer, 1991), where portfolios make learning visible(Johnsen, 2012). This variability in definition led Barrett to call for the

30

overloaded term to be qualified to indicatewhether used as a showcase or to“illuminatecapabilitiesnotcoveredbystandardassessment”(2005,p.2).Portfoliosofferawayofaddressinggoodpracticeinassessment:

• Reflection–assessmentsshouldincreasetheopportunitiesforself-assessment(Boud&Walker,1998;Schon,1983).

• Sustainability–assessmentsshouldaddresstheon-goingeducationofthestudentinthecontextoflifelonglearning(Boud,2000).

• Authenticity–problemsortasksshouldberealisticanduseacontextthatresemblesprofessionalpractice(Gulikers,Bastiaens,&Kirschner,2004).

The transition from portfolio to e-Portfolio occurred two decades ago, withdigital artifacts and containers becoming available in the classroom. Firstgeneration digital portfolios used a proliferation of different implementationstrategies from authoring of multimedia CDs and DVDs, through to hypertextbasedweb pages containing blog style entries usingweb servers. The claimededucational advantages of portfolios such as inducing deeper learning,emphasisinglearnerautonomyandpromotingreflectionhaveallcarriedforwardin this transformation. Driven by national and international acceptance of e-Portfolios as a preferred assessment approach, they survived the inevitabledescription as the next in vogue practice thatwould transform assessment, tobecome a leading technology in the assessment for learning movement,promotingmorevaried typesofassessmentandricher recordsofachievement(Zeichner&Wray,2001).Shulmansuggeststhatthey:

• permitthetrackinganddocumentationoflongerepisodesofteachingandlearning;

• allowreconnectionbetweenprocessandproduct;• institutionalisenormsofcollaboration,reflection,anddiscussion;• introducestructuretoafieldexperience;and• shiftagency,empoweringthelearner

(Shulman,1998,p.24).

It was a natural progression for second-generation e-Portfolios to switch tointernettechnologies,usingmanyofthetechniquesdevelopedinthemultimedia,hypertext and database based portfolio processes. These rapid technologicalchangeshaverenderedavarietyofterms,platformsandtechnologiesredundant,

31

although themovement towardsdigital artifacts in teachingandeducationhasmadethecollationprocesseasier(Batson,2002).Initscurrentusage,theterme-Portfolioreferstodatabase-driven,dynamicwebsites,ratherthanstatic,HTML-driven constructions (p. 3). The flexibility in implementation, linked to the e-Portfolioontology,bringswithittensionsbetweenflexibilityandrigidity.Eitherthesystemsallowcompletecustomisationrequiringahigherdegreeoftechnicalability in authoringwebpagesoroffer fixed templateswhichareeasier tousebutrestrictchoiceanddesign.Bothtypesprovideopportunitiesforfeedbackonperformance and reflection (Steeples, Jones, & Goodyear, 2002), but poorimplementations may give rise to entries that “lack purpose, offer limitedintegration of knowledge and weak connections between evidence and actualpracticeinvolvinggrowth”(Pitts&Ruggirello,2012,p.49).

Typesofe-PortfoliosThere are a variety of e-Portfolio implementations, with little compatibilitybetweenthesystemsortheworkingprocessesexplicitintheiruse.E-Portfolioscan be differentiated by theway they are used rather than by type, indicatingtheir use for developmental, presentation or assessment purposes (Mason,Pegler,&Weller,2004).What goes into the portfolio depends on the purposes of student, teacher orinstitution(Graves,1994).Portfoliosarepositionedintobroaduseswherethey:

• demonstrate individualcompetence; todevelop,demonstrateandreflecton pedagogical practice, show their attitudes, knowledge and skills(Sherry&Bartlett,2005);

• showcasequalificationsandcompetencies,aswellasforcriticalreflection

and learning purposes (Lorenzo & Ittelson, 2005), and as a vehicle forinstitution-widereflection,learningandimprovement;

• demonstrateinstitutionalaccountabilityoutwards,tomakeaccreditation

processesmorevisible,andtoshowcollectivestudentprogress(p.6).Showcase portfolios typically have artifacts which are the ‘best’ examples ofcompleted work, with writing or reflective analysis that place the work incontext. Working portfolios show in-progress artifacts, where growth andimprovement are detailed over time, so that the collected work indicates apathway throughadevelopmentalprocess.Portfolios thatemphasisereflection

32

throughfacetsorphasesoflearningfallintotheprocessportfoliocategory.Thesecategoriesmayoverlap to someextent, dependingon the e-Portfoliopedagogyand implementation in use.With such variability in features and use, aligningpedagogy with the technology is vital, through appropriate guidance andscaffolding(Yancey,2009a).A further way to categorise e-Portfolio implementations is by the degree oflearner choice in artifact creation, and whether the learner loosely collectsexternalartifactstogether,orisallowedtoweaveastorytogethersuggestingthelearningtakingplaceovertime.

Figure2.1-e-Portfoliotypes

Someimplementationsbuildprinciplesofconstructivealignmentintotheartifactcreation process, shown as the vertical axis in figure 2.1. These portfolios aretutorled,withparticipantsaligningartifactstolearningoutcomesembeddedintheinterface,asevidenceoftheirprogression.Thisdifferentiationcansuggestadivide between summative and formative assessment (Barrett, 2006). Suchalignment can be regarded as being problematic, when the artifacts aremeasured against “some high-stakes purpose”, as the deeper learning possiblethroughreflectiveprocessescouldbeatjeopardy(Barrett&Carney,2005,p.4).PaulsonandPaulson(1994)aligntheparticipant/facilitatoraxiswithparticularparadigms,suggesting that the facilitatoralignedmodel ispositivistic, in that ithas participants match against distinct agreed outcomes. The constructivist

33

portfolio allows greater freedom in artifact choice, so that the portfolio is amirrorofeachindividual’suniquelearningpath.Thehorizontal axis indicateswhere the artifacts are created and their context.Scrapbooking style portfolios allow for the loose collation of external artifacts,typically as attacheddigitalobjects laidout ina file/folder system.Storytellingportfoliosrequire the learners toarticulateanarrative, showing their flowandprogressionovertime,typicallyintheformofmark-upbasedwebpages,whicharecreatedinsidethesystemitself(Barrett,2008).

Figure2.2-e-Portfolioimplementations

Implementationsvary,butcanbeplacedaccording to thedegreeofparticipantfreedom and theway that the artifacts are presented (figure 2.2). e-PortfoliossuchasDIGIcationorEducauseacuratedwebpageapproach.WYSIWYGenginesallow users to customise pages to look like CVs, reports or descriptions ofactivities.MaharaandElggallowartifactmanagement;resourcescanbecreatedoutside the system and uploaded to the users library. There are then variouswaysofpresentinguploadedartifacts,suchasmicrobloggingaroundeachitemor by using templates for viewing. Portfolios such as Foliotek or Taskstreamallow artifacts to be uploaded aligned to tutor described requirements; tutorscan then see progression through which artifacts have been pinned to whichoutcome.Thisistheapproachthatmanyofthenewgenerationofcloudbasede-Portfoliosuse,suchasOpenSchool(OpenSchool,n.d.).

Stor

ytel

ling

DIGIcation

E-scape

Participant Directed

Facilitator AlignedSc

rapb

ooki

ng

Wordpress

RCampusEduca

MaharaElgg

foliotekDesire2learn

learningassitant

taskstream

pass-port

Pebblepads

foliofor.me

34

Theplacementofportfoliosalongtheseaxesisopentovariation,assomeofthesystemsallowadd-onorplug-inarchitecturesthatchangethefunctionality.Itisalsopossibleforportfoliostobeusedinalternativeways,forexampleaportfoliowhichdoesnotenforcetutor ledoutcomesthroughthe interfacecouldbeusedwith an ‘external’ list of tutor suggested artifacts enforcing what should becreated.There is no agreed standard single representation for an artifact, with XMLdescriptionsencapsulatingthewidevarietyofformsthatanartifactcantakebyrepurposing theATOMXML specification (Grant, 2009) or by focussing on theinteroperability of artifacts (Cambridge, 2006). The use of images has alwaysbeenintegraltoe-Portfolioconstruction,eitherthroughtheiruseinstorytellingtype structures in mark-up or through collation in album catalogues. Thetechnical advantages of using images to represent evidence are that they canovercome issues such as file type support, the variety of proprietary softwarethatusersmayhaveand theunpredictabilityof the rapidly changingnatureofsoftware,whichcombinetocreatewidevariationsinplatforms,devicesandusecases.Most e-Portfoliosacknowledge the importanceof reflective text toboth situatethe artifact in context, assessment purposes and to promote the deeperunderstandingthatisclaimedfromreflection.Situatingthetextisalsoimportantwhere the artifacts are being placed in a social context. Many acknowledge aconstructivepedagogyandclaimasocialconstructiveelementthroughtheuseofdiscussionforumsthatcanattachedtopublicartifacts.There is littlepedagogicdetailon theway inwhichthissharing is formalised,orhowcollaborationandcooperationispromoted.Mostsitesthatplaceartifactsinasocialcontextallowcataloguing to aid searching, sorting and filtering to take place (YouTube,Delicious, Pinterest). Educational taxonomies (Bloom, Biggs) and folksonomiescan be used for this purpose, but the evidence for their use in artifactrepresentationisscant.

2.2Pedagogyandprocessine-PortfoliosTheactivitiesoflearnersusingane-Portfolioaremostfrequentlypresentedasaseriesof stagesorprocesses, either in termsofwholeportfoliousageor as aninteractive process used repeatedly to create, present and then reflect onartifacts.Danielson(1997)stylestheportfoliocreationprocessasfivestagesinasequence-conception,collection,selection,reflection,andconnection(togoals).Mostauthorsplacetheprocessinthecognitiveprocessingtradition,suggesting

35

thateachartifactiscreatedusingareflectivecyclederivedfromDeweyandKolb.e-Portfoliosofferthebestexampleofthiskindofprocess,containingindividuallyconstructedartifactsandreflectionsoneachpieceofevidence.Typicalelementsincludedcouldbeeducationhistory,certificates,work-samples,awards,personalvalues,photos,videosandobservation.Workhereis“configured…asacontextforlearning”whichcanallowthedemonstrationoflearningoutcomesthatmaybedifficult topresentusingmore traditional assessment techniques (Sherman,2006,p.2).Theartifactsthatarecollectedcanbefromanauthenticpracticeorsimulationandwhengatheredattheendoftheprocess,formavaluablerecordofthereificationprocess.Askinglearnerstoselectandcreatetheirownartifactsinducesamorelearning-orientedviewofassessment.Aslearnersbecomeskilledtheyaremorelikelytobecome autonomous and fluent in the collecting evidence process (Smith &Tillema,1998).Chen (2002;2005)uses the term ‘folio thinking’ to suggest theembeddingofthecollectionandreflectionprocessintothelearningthatoccurs.Increasing the agency of the learner requires flexibility in the nature of theteacher role, requiring facilitation, tutoring and project management skills.Strategicapproaches suchas instructional scaffolding (Acosta&Lui,2006) canenhancetheteachingand learningbychangingthetraditionalroleofassessorsfrom authoritative gate keeper to collaborative guide. Successful e-Portfoliotutors steer students through the process, providing continuous and promptfeedback and promoting student self-reflection through reflective comments(Çimer,2011).Thelevelofguidancegivenonthenatureofartifactstobecreatediskey,withbalancesrequiredonthespecificityoftheitemstobecollated.Moreprescriptivespecificdetailonthenatureoftheartifactscansupplymetacognitivescaffolding,butmayruncountertotheprincipleofencouragingparticipants insettingtheirownlearninggoals(Sherman,2006).Moste-Portfolio implementations introduce thisproceduraldisconnect (Pitts&Ruggirello,2012),by insisting thatartifactsareuploaded tosatisfyaparticularrubric generally set by the teacher. A post reflection process may then berequired on the experiences and material presented, similar to Schön’sreflection-on-action, “re-establishing a logical connection by synthesizing andinterpreting” (p. 51). Pitt’s alternative approach suggests that each artifactshouldinitselfbeviewedasanopportunityforreflectionandbeusedasaunitofanalysis for reflection-in-action. How frequently the tutor provides feedback

36

doesappeartoaffecttheartifactcreationcycle,withmorefrequentinterventionslikelytoaddclarity,coherenceandlackofambiguity(Steeplesetal.,2002),butreduce the participants agency and time to reflect. Ultimately, tutors have toassume the gatekeeper role to validate artifacts as student’s authentic work(Lorenzo & Ittelson, 2005). Despite this difficulty, portfolios provide a way toactualiseashiftinthelocusofcontroltoemphasisestudentcentriclearningandtodevelopstudents’socialcapital(Acosta&Lui,2006;Batson,2011).

TheimportanceofreflectionandfeedbackNearly all explanations for the underlying portfolio pedagogy emphasise theimportanceofthefeedbackandreflectionpartoftheprocess,whereparticipantscan monitor their own development from continuous feedback, revealdiscrepanciesbetweenselfperceptionsandactualcompetency,andenabletheirperformancetobedocumented(Smith&Tillema,1998).Theopportunityfortheuse of reflection amongst the participants is seen as the key advantage ofportfolio use overmore traditional forms of assessment (Barrett, 2005; Cross,2012;Pitts&Ruggirello,2012)anditsusehasbeensuggestedasaspearheadinthetransformationtowardssituatedlearning(Batson,2011).Dewey(1910)originatedtheconceptofreflectionasaholisticdisciplinedwayofthinking where meaning making is achieved through “active, persistent andcarefulconsiderationofanybelieforsupposedformofknowledgeinthelightofthe grounds that support it, and the further conclusions towhich it tends” (p.118). Boud (1985) describes it in terms of recapturing experience involvingthinking, mulling and evaluating in a process of continuous learning. Variousauthors have attempted to further define, refine or categorise the concept,particularly in teacher trainingwhere theacceptanceof the reflectivemodel ineducationhasrequiredmoredetailonthewaysinwhichreflectivethinkingcanbe induced. Schön (1990) critiques the positivistic model and learningtransmissionprocess, indicatingthat they fail toaddressthenatureofproblemsolvinginscenarios involvingrealworldcomplexity.Hismodelofreflection-in-action and reflection-on-action is used frequently, and adds procedural detailaddressingthecontextofprofessionaldevelopment,wherereflectionisseenasakeysolutiontothefailuresoftechnicalrationality.

37

Despiteitswideacceptance,Schön’smodelofreflectionmaybehardertouseinpractice (Boud &Walker, 1998) and canbe criticised for failing to more fullyacknowledge the social context ofreflection – even though the inherentdialogical nature of reflective feedback isaddressed, the aspects of wider socialinteraction are not fully tackled (Kotzee,2012) despite being critical (Rodgers,2002). Eraut (1995) critiques the divideof reflection into the in-action and on-actionmodel and although supportive ofon-action, suggests that thenatureof timeandspeedof cognitionarenot fullyacknowledged(p.19).The portfolio literature uses the reflective cycle as its core pedagogy, mostfrequently citing Kolb’s experiential learning cycle (Kolb, 2005). This modelderived from Dewey, Lewin and Piaget enhances the detail of the reflectiveprocess by describing four activities in the reflective cycle (figure 2.3). Theconstructive stages of collection, selection, projection and presentationencourage students to adopt amore reflective approach to learning (Sherman,2006),whichallowsstudentstoactuponthefeedbackandreviseorredraftworkintegrating theadvice from faculty, advisors andothers (Pachler&Daly,2011;Walz,2006).Makingconnectionsbetweentheartifacts,learningself-assessmentskills and developing the stance of the reflective practitioner are all claimedadvantages(Yancey,2009b).DespitethefacttheKolbhimselfmovedawayfromthe successive cyclemodel inhis laterwork (Illeris, 2007), it is still frequentlyadvocated,forexampleinJISCpublications(Gray,2008).

There can be issues with e-Portfolio use. Shifting the locus of control to thelearnerallowsforthepossibilityof“lamination”,wheretheportfoliobecomesanexhibition or self-advertisement, used for superficially showing off (Shulman,1998). Decidingwhat should or should not be represented as an artifact is anacquiredskill,so it iscommonforparticipantsto initiallymisjudgethis,addingtrivial artifactswith limited reflection (p. 24). It is also possible that the “bestwork” represented may not describe the typical true picture of competency,although this canhappen in other assessmentpractices (Delandshere&Arens,2003). Institutional requirements formarking schemes and outcome guidanceact against the idea of learner set objectives. Attempts at using constructive

Figure2.3-Kolb’slearningcycle

38

alignmentmayresultinanoverlydetaileddescriptionofoutcomes,regimentingtheartifactschosenandreducingthee-Portfoliotolittlemorethanatraditionalassignment(Shulman,1998).

Withthewideuptakeofsocialnetworksandtheinclusionofsocialfacilitiesintolearning management systems, it has been a natural progression for similartechnologies to be added to both commercial and open source e-Portfoliosystems.Oneofthesignificantindicatorsforportfoliomaturationisthedegreetowhich social interaction is supported (Love, McKean, & Gathercoal, 2004).Unfortunately most of the e-Portfolio architectures available do not supplyeffective facilities that support collaboration,withmany of them not revealingwhatpeersarelearning(Hartnell-Young,2007).Withtheswitchtocloudbasedimplementations, the e-Portfolio cycle has been extended to includecollaborating peers. This may increase the opportunity for plagiarism, oruniformity in artifact reification, although better portfolio architecturesmakesthisvisible(Dalziel,2008).

Howanartifactisconstructed,improvedandpresentedwithinasocialcontextisproblematic, as there are gaps in the e-Portfolio literature. Most discussionssituateartifactcreationinaconstructiveepistemology,offeringlittleguidanceonwhata socially constructedartifact shouldbe. JISCoffersamodifiedversionofKolb’s learningcyclewithanattachedsocialcomponent(figure3.4),wherethecollaboration,sharingandsocialfeedbackaredisjointfromthemaincycle(Gray,2008).

Althoughe-Portfolioscanbeusedforbothformativeandsummativeprocesses,thereflectivecycleisclaimedtosupportsuperiorformativeassessment(Barrett,2010)whensituated inaconstructivepedagogy(Barrett&Carney,2005).This

Figure2.4–JISCmodifiedlearningcycle

39

transparentspacemakestheteachingandlearningvisible(Parkes,2013)andifthe nature of the practice is clear and well defined (Strivens, Baume, Owen,Grant,&Ward,2009)theopennesscanbeusedtoovercomeissuesinformativeassessmentprocesses(Stefanietal.,2007).The reflective cycle is still key in e-Portfolios architectures that support theenhancedsocial aspect.Portfolioprototypeswith these featuresare starting toappear, suggesting that the emphasis on the social component motivates andfosters “authentic studentvoices and facilitates student-centred social content”(Klein, 2013, p. 71). Seeing others’ content improved their own work andincreasedtheirperceivedownershipofthematerialsbyputtingthem“inchargeoftheirwork”(Garrett,Thoms,Alrushiedat,&Ryan,2009,p.205).The nature of the supporting portfolio community is generally unclear andtypically sidesteps discussions on the nature of the underlying pedagogy thatshouldbeapplied.Mostoftheliteratureagreeswiththeimportanceandvalueofpeerassessment,suggesting thatpromotingcollaborativepeer topeer learningincreases social awareness through community interactions (Acosta & Lui,2006). Deeper, more meaningful learning is created through social contextualexperiences (Carmean & Christie, 2006), but there is a failure to provide anyguidanceonhowtoachievethis inaportfoliosystem, forexample,BarrettandCarney (2005). There are occasional examples that attempt to clarify thisprocess, for example a learning portfolio using a secondary layering of peerassessment throughablogwithanonymousreviewsituated inacommunityofpractice (Stevenson, 2006). Skills such as learning in a network, collection,aggregation and forming ‘connectedness’ are declared essential (Cambridge,2009)butwithoutclarityinhowtheyshouldbeencouraged.Thenext section explores the recenthistoryof learning communities and thensuggestshowthecommunitymodelsassociatedwithnetworkedlearningcanbeapplicabletoe-Portfolios.

2.3LearningcommunitiesandnetworkedlearningThe increasing popularity of the learning community in the academy has acomplex setof roots, from investigations into theapparent collapseofwesternsocialcapitalism,therediscoveryofthesovietphilosophers,thepracticalitiesofAmerican college education in the 1980’s, through to community pedagogypopularisationandshiftstowardsinclusiveconnectingandmobiletechnologies.The historical development of the learning community can be difficult to

40

evidence as communities of learners may not be oriented to documentation(Hugo, 2002), and cohesive groupings may be empirically unobservable(Brookfield,1983).

Smith (2001) suggests three historical phases in the development of learningcommunities in the American system, startingwithMeiklejohn's ExperimentalCollege at the University of Wisconsin. Running from 1927 to 1932, a crosssection of students participated in a living-learning community within aresidential social experience, which promoted active learning, initiatedfacilitationrolesandbrokesubjectsilos.Curriculaandco-curriculaactivitywereintertwined, professors were re-designated as advisors and many of thetraditionaleducationalstructuressuchasrequiredattendance,courses,electives,anddepartmentsabandoned.Participantsusedactiveandexperiential learningwith a variety of assessments to provide structure and accountability (Smith,MacGregor,Matthews,&Gabelnick,2004).Despitethesuccessofthegraduates,theprogrammeended in1932,due toorganisational incompatibilities, internalpolitics, interference from the faculty and conflicts between participants andotherstudents(Meiklejohn,1932).

Many of these ideas continued in community college innovations during the1960s,suchasinprogrammesattheUniversityofCaliforniaandSanJoseStateCollege,where theyexperimentedwithstructure,educational roles, curriculumcontent and pedagogy. Although modest, many of the initiatives failed due toorganisational incompatibilities with issues of scale and cost, similar to thosethathadaffected theExperimentalCollege.Thepositive consequencesof theseprogrammeswere thatmany of the innovations developed during the processenteredmainstream practice, such as student-centred learning, active learningandinterdisciplinaryrelevantcurricula(Smith,2001).

Despitetheseprojects,thetransmissionmodelofteachingandlearningwasstilldominant in the1970’sand80’s,withanunderlyingbehaviouralandcognitivemodel.Designatinglearningasachangeinbehaviour,suggestedthatreinforcingpractice strategies would serve as a foundation for teaching practice whereknowledgewas transmitted toand thenacquiredbya learner. ItwasnotuntilthesuccessoftheEvergreenStateCollegeprojectthatlearningcommunitieshadawidelyacknowledgedsuccess(Jones,1981),alongwithresearchadvocatingitsadvantagessuchasengagement,retentionandenrichment(Tinto,1995;2000).Using a constructivist meaning making approach, the Evergreen project washeavilyinfluencedbytheBerkeleyprogrammeattheUniversityofCaliforniaand

41

usedyear longco-ordinatedstudies,organisedaround interdisciplinarythemes(Smith et al., 2004). It also successfully addressed the scaling, cost andadministrative integration problems that had stymied earlier attempts (Kuh,1991). Evergreen became a leading advocate for learning communities, to theextent that the approachwasdescribed as amaturingmovement twelve yearslater(Matthewsetal.,2012).Acknowledging the value or knowledge that can come from collaborativerelationships achieved contemporary significance through the promotion ofsocialcapitaltheoryintheworksofBourdieu,ColemanandPutnam,andthere-emergence,andexponentialgrowthofinterestinVygotskyandLeont’ev’sworkon the Zone of Proximal Development (ZPD) and Activity Theory (Johnston,2004; Roth & Lee, 2007). Vygotsky’s Zone of Proximal Development (1980)emphasisesthatinteractionwithknowledgeablepeersisafundamentallearningprocess and that the roots of cognition are social. Vygotsky’s work is the keytheoretical underpinning for social constructivism, the learning that can comeabout because of learning in a group. Dialogue, guidance, feedback and socialinteractions are drivers for transforming potential development into actualability. The zone of proximal development is “the distance between the actualdevelopmentallevelasdeterminedbyindependentproblemsolvingandthelevelof potential development as determined through problem solving under adultguidance,orincollaborationwithmorecapablepeers”(Vygotsky,1980,p.86).Inpractice themostcommonrelated implementationofZPD isscaffolding,wheretutorssettaskswhicharejustbeyondalearners’capacity,guidingthemthrough,graduallyproviding lessassistanceover time to fade the levelof supportaway(Wood,Bruner,&Ross,1976).Wenger’s community of practice model was a turning point before whichPiagetian, constructivist, and information processing paradigms were pushedaside by the idea that “knowing and knowledgeability are better thought of ascultural practices that are exhibited by practitioners belonging to variouscommunities”(Roth&Lee,2006,p.27).Describedintwoseminaltexts(Lave&Wenger, 1991; Wenger, 1999), situated learning and communities of practicewere popularised in the organisation management literature and dramaticallychanged educational research through the learning community ideal (Brown&Duguid, 1991). Although the notion of assessment is not explicitly addressed,Romer (2002) sees an implicit evaluation in the negotiation - participationprocess.BoudandFalchikov (2006)attempt toconceptualise thisparticipation

42

in practice as self-assessment, where fostering reflexivity ‘prompts self-monitoringandjudgingprogressiontowardsgoals’(p.409).Thecommunityofpracticemodelisfrequentlyusedasananalyticalframeworkinnetworkedlearning,andisdiscussedinthiscontextinthelaterdiscussion.

TypesoflearningcommunitiesTheincreasingpopularityoflearningcommunitieshashighlightedtheircomplexontology.Waystocategorisethemtypicallyuseanumberoffeaturessuchas

• characteristicsoftheparticipants;• intentionality,orpurposeofthelearningcommunity;• typeorstrengthofconnection;• domainorsubjectareathatthecommunityconcerns;• physicallocationofthelearningcommunity;and• abilityforthelearningcommunitytotransformovertime,adoptingnew

customsorpractices.Intentionalityisanindicatorofwhetherthecommunitywasdeliberatelyformedor emerged over time.Membership could be formal or informal,with fixed orfuzzy boundaries allowing or denying the entrance of new participants.Communitiescouldbeonline,virtualandremote,or theycouldbe face-to-face,situatedinasingleworkplacewithparticipantsassignedtoaparticularproject.All the models have feelings of belonging, shared resources, influence andemotional connections (McMillan & Chavis, 1986). Schwier’s ten elements ofcommunity (2001) emphasise the importance of historicity, identity andmutuality in the community design, and the consequences these have incommunityinitiationandgrowth.Henri (2003) seeks to distinguish between learning communities by using thegathering intentionality, the strength of the social bond, the goal of thecommunity and how the intention emerges.He also includes themethods thatwere used to create the group, if any, and the evolution of the goals andmembership over time. He derives four types; communities of practice (COP),communities of inquiry (COI), generic learning communities and goal-orientedcommunities, which are close to the COP model but have a differentintentionality of learning, where knowledge construction is for collective userather than for appropriation of new practices. McConnell (2006) dividescommunitiesintothreetypesderivedfromtheintentionalityoftheparticipants

43

andsituation,separatingthemintogeneral learningcommunities,communitiesofpracticeandknowledgebuildingcommunities.Generallearningcommunitiesembodyacultureoflearningwheremembersareinvolvedinacollectiveeffortofunderstanding. This differentiates them from the COP model where membersfocusonthedevelopmentofprofessionalpractice,typicallysituatedinashared,perhaps physical domain. McConnell’s final category is that of knowledgebuilding communities (Bereiter, 2005), which focuses on the advancement ofknowledgeratherthanontasksandprojects.

NetworkedlearningNetworked learning(NL) is learning inwhich informationandcommunicationstechnology is used to promote connections: between one learner and otherlearners, between learners and tutors; between a learning community and itslearning resources (Jones et al., 2000). The definition of networked learningemergedfromworkoncomputermediatedcommunication(CMC)andcomputersupportedcollaborativelearning(CSCL)wheretheconvergenceoftechnologiesandtelecommunicationsopenedupanewrangeofeducationaldesignsbreakingdown the “barriers of time and space” (Steeples & Jones, 2002). Networkedlearning seeks to establish connections, rather than the shared practicepromotedincommunitiesofpractice(Ryberg&Larsen,2008).Thedefinitionhasbeen remarkably robust despite rapid technological changes in the last fifteenyears(Dirckinck-Holmfeld&Jones,2009).Networked learning sits in a socio-cultural tradition, where participation isachieved through communities of learners wheremeaning is both “negotiatedandcreatedthroughcollaborativedialogue”(Dirckinck-Holmfeldetal.,2011,p.293), and that knowledge emerges or is constructed in relational dialogue orcollaborativeinteraction.Initsearlyconception,networkedlearningwasofferedasanewparadigm(Jonesetal.,2000),awayofincreasingcapacity(Hopkins&Jackson,2009)andasasuggestiononhowtoorganisestudentstoovercomethetensionbetweenorganisationalrequirementsfortightstructureswiththelooserstructuresusedtopromotelearnerindependence(Jonesetal.,2000).Focussingontheconnectivityofparticipantshasbeenshowntoincreaseaccesstolearning,people, diverse resources and artifacts (Haythornthwaite&De Laat, 2010). Itcan also explain meaning making and the complexity of identities, which arebeing continuously constructed through connections, where differing ties areusedfordifferentpurposes.NLacknowledgestheimportanceofweaktiesalongwith the non-privileging of particular types of relationships (Jones & Esnault,2004;Ryberg&Larsen,2008),distinguishingnetworkedlearningfromCSCLand

44

communitiesofpractice,wherestrongrelationshipsandhuman-humanrelationsareemphasised(Jonesetal.,2008).Thevariabilityoftie-featuressuchasamountof time, emotional intensity, intimacy and reciprocal services (p. 91), broughtabout the realisation that pre-digital depictions of strong links failed toencapsulate the richness of relationships possible through networks(Granovetter,1973).There have been a variety of research approaches in NL. Much of the earlyresearchworkusedactionresearchwithmultimethodsandcontentanalysisofmessagesandcodingschemes(DeLaat&Lally,2004),orethnographythroughcase study and survey (Goodyear et al., 2004;Hodgson&Watland, 2004). Theunderlyingframeworksusedinnetworkedlearninghaveincludedcommunitiesofpractice(COP),communitiesofinquiry(COI),culturalhistoricalactivitytheory(CHAT) and actor network theory (ANT).Until recently COP andCOIwere themost frequently used, either through COP’s three dimensions or COI’s threepresences. The three dimensions of Lave andWenger’s community of practice(mutual engagement, joint enterprise and a shared repertoire of actions), areoftenusedfortheanalysisofthecommunity(Guldberg&Pilkington,2006),withWenger’sintense,activeandperipherallevelsofparticipation(2002)providinggroupingsforanalysis(Henri&Pudelko,2003).Inmanyinstancesearlyworkfocussedonlevelsofinvolvementandreasonsforsuccess or failure using textual analysis. The well-defined nature of the COIinstrumentsandthemanyinstancesofitsuse,resultedinthecognitive,teaching,and social presence being applied in networked learning. Acknowledging thepossibleshortfallsofmissingcontext,workafterthispointshiftedtothenatureof learning interactions and whether they fit with tutors’ beliefs about ‘good’learning (Goodyear, Jones, Asensio, & Hodgson, 2005). Ryberg argues thatdetermining a unit of analysis is difficult, particularly where there may beinteracting networks, multi-memberships and boundary crossing (Ryberg &Larsen, 2008). He suggests that understanding themeaning-making process iscentral to “identifying theparametersneeded to judgewhether relationscountas weak or strong; and to unravelling the types of relations existing in thenetwork”(p.113).BoththeCOIandCOPmodelareexplainedinfurtherdetailinthenextsection.

FrameworksusedinnetworkedlearningCommunities of practice represent a more pragmatic version of community,comparedtothatinanidealisedlearningcommunity(Quinn,2010).Observedby

45

Wenger in anthropological studies in authentic real world situations, its ideasand concepts came from the situated learning that was taking place inapprenticeships,asthemembersbecameestablishedparticipantsofagroup.Initsoriginalform,LaveandWengerdescribedhowlegitimisationwasachievedthrough participation (Lave et al., 1991). New entrants (novices), use simpletasks and peripheral activities in basic roles to become aware of the customs,practicesandtasksinthecommunity,wherethereisasharedpracticerequiringbothspaceandtimetocollaborate(Jones&Esnault,2004).Astheyobservetheactivityof theexperts, they self-evaluate, gradually takeonmore complex taskandrolesandbecomemoreinfluentialtothefunctioningofthecommunity.Newmembersmovefromperipheralparticipanttocorememberthroughthisprocessof enculturation. Lave and Wenger (1991) saw legitimate peripheralparticipationasbeingkey to communitymembershipandas three inseparableaspects, of legitimate versus illegitimate, peripheral versus central andparticipation versus non-participation (p. 35) Wenger uses societalinterpretationsofVygotsky’sZPDconcept,placingmoreemphasison“issuesofsocioculturaltransformation…inthecontextofachangingsharedpractice”(p.49). Communities of practices emerge and evolve from ill-structuredproblemssituatedinauthenticsettings,differentiatingthemfromthesimplifiedproblemsoften seen in educational environments. Solving problems in this context is ashared goal, which when achieved, may cause the membership of the COP toevolvetotackleneweremergingproblems.

In the later work Wenger (1998) moved away from the idea of legitimateperipheral participation and used sets of dualities “inseparable and mutuallyconstitutive elements whose inherent tension and complementarity give theconcept richness anddynamism” (p. 66).Wenger identifies fourteen indicatorsthatrevealthepresenceofacommunityofpractice,alignedtofourinterrelateddualities, Participation-reification, Designed-emergent, Identification-negotiability and Local-global, the broader context in which the practice issituated.

The participation-reification duality has particular relevance in knowledgemanagement, where it is seen as key to solving the representation of implicitknowledge (Paul & Kimble, 2002). Wenger’s concept of reification, whereabstractrepresentationsaregivenformtoenablesharing,arekeytothenatureofe-Portfolioartifactsinacommunitysetting,andthisisexplainedinsection2.4.

46

Thethreefundamentalcharacteristicsofacommunityofpracticeare:

• adomainofknowledgeorjointenterprisewhichdefinesasetofissues,groundandidentity;

• mutualengagementthroughinteractionsandrelationshipsbasedonrespectandtrust;and

• asharedrepertoireanddevelopingpracticewithaccompanyingtools,ideas,languageanddocuments.

'Mutual engagement' is the interaction between individuals leading to thecreation of shared meaning on issues or a problem, where participants mustengage the community to enhance their membership (Schwier, 2001). 'Jointenterprise' is the process in which people are engaged and working togethertoward a common goal and 'shared repertoire' are the common resources andjargonthatmembersusetonegotiatemeaningandfacilitatelearningwithinthegroup(Wenger,1999).

The usefulness of the COP model, for example in achieving competitiveadvantages (Liedtka, 1999), led to Wenger’s evolution of the concept fromindividualised learning to a way for an organisation to manage knowledge(Wenger, 2002). The version introduces significant differences, such as theability for an organisation to engineer a COP rather than for it to emerge; theintroductionofaleaderorchampiontypicallyinamanagementrole;afacilitatorwho administers the group, and a redefinition of the three characteristics intodomain,communityandpractice.

Using a community of practice in the classroom has been attempted at bothprimary(Brown,1992)andsecondarylevels(Galbraith,Renshaw,&Goos,1999;Goos, Galbraith, & Renshaw, 2003), socialising students into an emergentpractice or situated learning (Pitri, 2004). Problems such as the lack of ‘old-timers’cancauseissuesintheapplicationofthemodel(Chang,Chen,&Li,2008)and the misapplication of the framework outside of a practical domain, fromwhere it emerged, are a frequent criticism (Gourlay, 1999). To counter this,classroom studies have investigated specific characteristics such as peripheralparticipation, or created derivatives, such as Boylan’s notion of ecologies ofparticipation(2010a;2010b).

The community of inquiry framework (COI) evolved out of research into theprocessesandpresencesapparentincomputer-mediatedcommunication(CMC)byGarrisonandAnderson(2003)andfurtherbyGarrisonandArbaugh(2007).

47

Throughasizableliterature,COIhasdevelopedintoapopularwayofjudgingthequalityandnatureofcriticaldiscourse(Garrison,Anderson,&Archer,2001),inpartdue to thewelldefinedaccompanyingmethodologyandsetofmethods. Itserves as a tool for conceptualising learning processes, that can be used bothduringacommunity’slifebyeducatorsinoptimisingthedialoguetakingplacetopromote good practice and retrospectively afterwards by researchers toinvestigatethegrowth,useandsuccess factors inCMC.Garrisonsituates it inacollaborativeconstructivistviewofteachingandlearning(2011),usingDewey’sidea of transactional communication where information is constructed asknowledgewithpersonalapplicationandvalue.This isthenconfirmedthroughcollaborationwithinacommunityoflearners.

COI uses the notion of three presences definedwith categories and indicators.Garrison(2001)seescognitivepresenceasansignaloftheactuallearningtakingplace using construction and meaning making through sustained discourse,reflection and discourse. Social presence is a measure of group identificationalong with how successfully personal and affective relationships progress. Hedirectlylinksittoenhancingcognitivepresencebutacknowledgesthechallengesimplicit in attempting to encourage it in a synchronous text based medium(Garrison, 2011). Teacher presence is indicated by facilitating discourse;instructionaldesignandorganisation;anddirectinstruction

ApplyingCOItoe-PortfoliosInane-Portfolio,thenotionofsocialandcognitivepresenceareembeddedintheartifactsandcommentary,whichcanmaketheCOIframeworkdifficulttoapply.Artifact representation varies by implementation, so coding video, audio andimagery falls outside the scope of the base COI model, which uses transcriptanalysisusingamessageastheunitofanalysis(DeWever,Schellens,Valcke,&Van Keer, 2006). There have been investigations into using teaching presenceseparately from the other presences (Arbaugh&Hwang, 2006; Pawan, Paulus,Yalcin,&Chang,2003;Shea,Pickett,&Pelz,2003;Shea,SauLi,&Pickett,2006;Shea, 2010), and evidence that teachingpresence canbe evaluated around theuse of an e-Portfolio, on the blogs and reflective statements surrounding theartifacts(Torras&Mayordomo,2011).

The concept of teaching presence (TP) encapsulates the idea that a teacher’sresponsibilityistofacilitatelearningthathaspurposeandisfocusedonessentialconcepts andworthwhile goals bydesigning, facilitating anddirecting learningonline(Anderson,Liam,Garrison,&Archer,2001).ThethreeTPcharacteristicsoffacilitatingdiscourse,instructionaldesignanddirectinstructionwerederived

48

from literature reviews and student surveys and these combined with theindicatorsthatsignaleachlevelserveto“understand,measureandimprove”theteachingtakingplace(p.15).Theimportanceofteachingpresenceine-Portfolioshasbeenshowntobemoreimportantthantheeducationalintentionalityofthetechnology(Torras&Mayordomo,2011).

Twooftheteachingpresencecharacteristicsaredirectlyapplicabletotraditionale-Portfolios.Instructionaldesignindicatorsare:

• settingcurriculum;designingmethods;

• establishingtimeparametersandnetiquette;• utilizingmediumeffectively;and• establishingnetiquette

(Andersonetal.,2001).

Thedirect instruction indicators are applicable if ane-Portfolio system isusedfor assessment and instruction, where a tutor provides regular feedback,directionandfocusthroughtheircommentary(p.10):

• presentcontentandquestions;• focusthediscussiononspecificissues;• confirmunderstandingexplanatoryfeedback;• diagnosemisconceptions;• inject knowledge from diverse sources, e.g., textbook, articles, internet,

personalexperiences(includespointerstoresources);and• respondingtotechnicalconcerns.

Facilitating discourse is difficult to apply in portfolios based on the individualKolb reflective learning cycle, but architectures that allow and promotediscussionsaroundartifactsmakeitapplicable.Ifnetworkedlearningisusedinan e-Portfolio context, tutors can use the artifacts and reflective statements inwaysthatwouldalignwiththeindicators(p.8):

• identifyingareasofagreementordisagreement;• seekingtoreachconsensusorunderstanding;• encouraging,acknowledging,studentcontributions;• settingclimateforlearning;• reinforcing;and

49

• drawinginparticipants,promptingdiscussion.

The networked learning definition emphasises the use of ICT to promoteconnectionsbetweenlearners,betweenlearnersandtutors;betweenalearningcommunity and its learning resources. An appropriate e-Portfolio designprovidesanappropriatearchitecturefortheartifactscreatedbyparticipantstoserveinadualrole,asbothlearningresourcesfortheirpeersandasassessmentartifacts.

2.4ArtifactsandtutorrolesinacollaborativeenvironmentsThe transition from analogue to online digital e-Portfolio has not resulted in achangeinthegeneraldescriptionofartifactsintheportfolioliterature–theyareexamples of work (Stevenson, 2006), collected for a particular audience andpurpose(Diez,1994).Artifactsaredescribed in termsofuse, suchasanobjectcreatedandthendesignedforpresentation(Flanigan&Amirian,2006),intermsof the surrounding container, such as in objects stored in a e-Portfolio (Walz,2006),orbytheactionsofthelearner,asadesignationfortheitemscreatedandcollectedwhichareorganised,displayedandthenconnected(Gibson&Barrett,2002).Inanyconfiguration, reflection isacknowledgedaskey,with thedescriptionofanartifactservingtoexplainitssignificance(Walz,2006),andthatstudentsneedto learn to constructively reflect upon andwrite about artifacts, looking in the‘mirror’toseetheirownprogressandthenmappingoutareasfordevelopment(Diez,1994).

The rubrics used to guide artifact construction have been used to situateportfolio use in a particular paradigm, where rigid external criteria preciselydescribing the nature of the work to be produced has placed it in positivism(Lowenthal,White, & Cooley, 2011; Ring & Ramirez, 2012). Allowing learnerschoice in artifact design can be constructivist (Barrett & Carney, 2005) orinterpretivistic, where approaches that suggest that truth is a matter ofconsensus allow a greater flexibility in the artifacts chosen and presented(Johnston, 2004). Barrett (2006) argues that this design choice can have asignificantmotivatingeffect.The movement towards online e-Portfolios with extra abilities for sharing(Gibson & Barrett, 2002), should have ignited a discussion on the nature ofshared artifacts or their conception, however the lack of community based

50

pedagogy isstill implicit inmanyof the implementationsoranalysis.Thereareindicationsthatartifactscanbeusedincollaborativeways,suchaswhencreated,taggedandsharedtoprovidecontextallowingfordiscussion(Toshetal.,2006).Perhapsbecauseof therelianceonportfoliosas individualassessmentdevices,there is anoverarching assumption that artifacts are individualpiecesofwork(Barrett&Carney,2005),usedinanpersonalreflectivelearningcycle.A solution lies in the community literaturewhere a broader discussion of thenature of artifacts provides richer depictions. Bereiter and Scardamalia’sknowledge-building communities and Wenger’s community of practice modelboth address the nature of an artifact, situated in constructivism and social-constructivism respectively. Bereiter and Scardamalia’s knowledge-buildingcommunities are learning communities whose goal is specifically knowledgeconstruction. Contributions to a community knowledge base serve to createshared intellectual property (Scardamalia & Bereiter, 2003), using collectivepractices and activities using improvable artifacts (Bereiter & Scardamalia,2005).Theyplaceartifacts inPopper’s thirdworld,whichcontainsproductsofthe humanmind (Popper, 1979), although Bereiter and Scardamalia call themconceptual artifacts to acknowledge criticisms of the third world concept.Artifactsarekeytotheenculturationprocess“joiningtheranksofthosewhoarefamiliarwith,understand,create,andworkwiththeconceptualartifactsoftheirculture”(Bereiter,2005,p.237).Conceptualartifactsaredifferentfromculturalormaterial artifactsby the relationshipsbetween them,asoneartifactmaybederivable fromorbepartofanother–but theymayalsocontradictorsupportanother(Bereiter,2002).ForWenger,artifactsarekey to thedualityofparticipationandreificationand“tend toperpetuate the repertoiresof practicesbeyond the circumstances thatshaped them in the first place” (Wenger, 1999, p. 89). Reification, which hedescribes as the process of giving form to experience, describes the sharedcreationofartifactswhichmayormaynotbematerialobjects.Thevisibilityofartifactsarecomplex,whichisusedtodescribethedegreesofaccessibilityofanartifact and also the level of encoding; what is revealed by the artifactrepresentation (Lave et al., 1991). The creation of shared artifacts can induceengagement and alignment, implying “sustained intensity and relations ofmutuality”(Wenger,1999,p.184),andarealsothekeyto theprogressionofacommunity as a whole, where the historical traces of artifacts continue thelifecycleofacommunityandwhenusedinaculturalpracticecarrythepractice's

51

heritage(Laveetal.,1991).

TheskillofartifactcurationWith the rise of Web 2.0 technologies, information abundance has led to theacknowledgment of the new digital skill curation, for the creation andmanagement of artifacts (Beagrie, 2006). Artifacts here are a sharablerepresentation of practice (Goodyear & Steeples, 2008), which throughdistributed cloud storage come in diverse tacit forms such as links, snippets,imagesorblogposts.Creating,annotatingandlinkingsuchartifactsisseenasanew digital pedagogy (Sharples et al., 2013). Digital artifacts can enhance thereputationoftheparticipantsintheircommunitywhentheyareseentopubliclycreatevaluableartifactsthatareofusetomembersofthegroup(Klammaetal.,2007).Placing theseat theheartofnetworked learning,Seitzinger(2014)callsforthetermsocialcurationtobeappliedtotheprocessofconnectingtolearningresources, managing information flow and information gathering, sharing,taggingandaggregation.Theskilloftaggingusingafolksonomyiscommonwherethereareshareddigitalartifacts,asseenonYouTubeandsitesthatarchivecontentsuchasDeliciousorEvernote.Usersattachsubjectdescriptorkeywordstocontent,creatingnewtagsor reusing ones suggested by participants in the community that have alreadyaccessed the item (Smith,2008).Theemergentvocabulary tends tobedomainspecific and lowers cost and barriers to entry for new participants (Mathes,2004).Extendingtheartifactcreationprocesstoincludesocialcuration,acknowledgesanewdigitalliteracy,wheretherolesofproducersandusersbecomesblurredandknowledgeproductioninvolvesreuseandremixing(Pachler&Daly,2011),usingthe affordances of digital technologies. The common view of digital literacyfocuses on individual skills or competences, one of encoding or decoding aparticular artifact. Digital literacy can be viewed as an expanded conceptinvolving social and cultural practices (Lankshear&Knobel, 2008),where it issimilartoCOPsconceptionoflearningthroughparticipation,asthemimicry,useand production of artifacts is part of the enculturation process that novicesexperience as they become experts. Social and cultural definitions of digitalliteraciesinclude:

• Working collaboratively in a multidisciplinary team to create useful,practicaltools.

52

• Supporting learning communities to work collaboratively in problemsolvingandtheco-constructionofknowledge

(Gillen&Barton,2010).

Some artifact creation processes may include reusing others’ artifacts in aprocess similar to assemblage, remixing video and audio. This originated fromthepracticeofcreatingtextsbuiltprimarilyfromexistingtexts(Johnson-Eilola&Selber, 2007), which has been put forward as a valid practice alongsidemoretraditional creative processes. There are positive correlations between anincreaseintheuseoftechnologyineducationandplagiarism,typicallythroughmakingiteasier(Harper,2006)andthebeliefthatplagiarismislowrisk(Szabo&Underwood, 2004). The remix culture and the academies traditional viewofplagiarismconflict,whereoriginalityofstudentworkisregardedaskey.

Thecomplexityofteacher,tutorandfacilitatorrolesinane-PortfoliocommunityIn a traditional e-Portfolio architecturewithout a community aspect, the tutorfacilitates the learning taking place by assisting participants in developingreflective skills (Doig, Illsley, McLuckie, & Parsons, 2006), using co-operativecollaborationtosupportstudentsintasksandassessments(Kirkhametal.,2009;Lopez-Fernandez&Rodriguez-Illera,2009).Theartifactsproducedbyalearnerareavisiblerepresentationofastudent’sprogression(Pachler&Daly,2011).Theattitudeoftheperson“receiving,assessingor introducingthe ...e-Portfoliosuchasatutor,anassessororalinemanager”(Curant,2009,p.27)iskeytothesuccessofane-Portfolioastutorsareprovidedwithanytime,anyplaceaccesstosubmissions(Lawson,Kiegaldie,&Jolly,2006).Thelevelandregularityoftutorfeedback is a vital measurement of effectiveness (Butler, 2006; Mason et al.,2004) with tutor availability and engagement a high priority for e-Portfoliolearners (Gray, 2008). This formative feedback is typically categorised intoverificationandelaboration,whereverificationistypicallyasimplecommentonthe validity of the work (Shute, 2008). Elaboration is more sophisticated andsuggests the tutor addresses topics more fully, providing multi-layeredresponsesandguidance.Moving beyond Kolb’s learning cycle to include a social component throughcommunitycomplicatesthetutorrole,becauseofthemultitudeofrolesthataresimultaneouslyrequired.

53

Networked learning, communities of inquiry and communities of practicehavedifferent conceptions of the role, depending on both the underlying pedagogyand desired outcomes of the community. Networked learning’s definition oflearning communityhas allowed fluidity in the roleof the lecturer– as leader,facilitator or guide. In a peer based community, the roles of the individualschangeandemergeovertime(DeLaat&Lally,2004),sosomeofthetraditionaltutor roles may be shared amongst the participants (De Laat & Lally, 2003).Tutorscaninitiallyprovidescaffolding,setupinitialstructuressuchaslearningsets, introduce subjects, but should also be participant (McConnell, 2006). Inmanyof thepeerbasedcommunitymodelsused, the teachermoves towardsafacilitationrole,asaguideontheside(DeLaat&Lally,2004)orinananimatorrole (Jones & Esnault, 2004), where theymanage issues of power, inhibit andmobilisefeaturesforbalance,provideoverviews,integrationanddistribution(p.6).Theimplicit“riddleofliberatingstructures”isthatthetraditionaltutorroleisinitially required to push for more equality in the participants’ roles (Pedler,1981,p.77).The earlier versions of the community of practice model suggest that rolesemerge, rather than being deliberately created. In these, the notion of expertsuggests a facilitation role,where initial instruction is equivalent tomirroring.Novices transition by association with experts, as the complexity of the tasksdemonstrated increases over time, moving them from newcomer to old-timer(Lave et al., 1991). Old-timers induct novices, which introduces generationaldifferences that thenpropels thepractice forward (Wenger,1999). In the thirditeration,WengersuggeststhatCOPscanbebuiltratherthanemerge,typicallyina professional, commercial context. This requires the introduction of moreformal roles such as leader, champion and facilitator (Wenger, 2002), whichbecauseofthecontextsuggeststhattheleaderorchampionroleislikelytobeina more senior management position, with a separate facilitator acting as co-ordinator.Theleader-championisintendedtoassertWenger’ssevenprinciplesofsuccessfulcommunitydesign(p.69).HelinksthesuccessorfailureoftheCOPdirectlytotheseroles.IntheCOIframework,thesignificantroleofthetutoristofacilitatethegrowthofthe community, through encouragingdiscourse, instructional design anddirectinstruction, with practices such as “identifying agreement and disagreement,sharingmeaning,andseekingtoreachconsensus”(Garrison&Arbaugh,2007,p.164). The facilitator role is fulfilled by an instructor, teaching assistant or by

54

peers,whichwhenfacilitatingdiscoursearemorethansimplyaguideontheside(Garrison,2011),as theyencourageappropriateandrelevantresponses,modelcritical discourse and encourage participation (p. 58). Student interaction intheseenvironmentscanvarydependingonthelevelofpresenceofthefacilitator,whereexcessivepostingcaninhibitlevelsofinteraction(An,Shin,&Lim,2009;Dennen,2005). Incaseswhere thepresenceof the instructorcouldbedeemedintimidating, some studies suggest sharing the responsibilities of the rolebetweenstudentandinstructor(Seo,2007).

CriticismsofcommunitybasedmodelsandthedarksideCommunity basedmodels have been criticised, throughmisapplication, lack ofdefinitionorfortheirabilitytodiscouragedissentorencourageconformity.Theidealrepresentationsofcollaborativeparticipationinlearningcommunitiesmaymaskthedarksideofinteraction,wherethetyrannyofthedominantmayinduceoppression and control or a failure to acknowledge the pressures to conform(Ferreday & Hodgson, 2008). Ferreday (2010) identifies themes fromparticipants such as feeling unworthy to participate, incremental strugglewithnewmodes of thought, a sense of lost innocence and possibilities of isolationexclusion. COPs can suffer from similar self-protective mechanisms, becomingrepressive and exclusionary as their pragmatic practical nature may make“counterargumentsandpractices[to]becomeunthinkable”(Quinn,2010,p.50).

Eraut(2002)suggeststhattheCOPmodelisanunrealisticideal,andthatactualworkingpracticesarenotaddressed,whereitfailstoacknowledgethevarietyofrolesapersonperformsinasinglejob.Healsohighlightsissuesassociatedwiththe way in which part time and temporary staff move in and out of the COPboundary, the scope and demand for inventiveness in roles, along with thedegree to which the structures allow staff to perform at their level ofcompetence. Contu (2003) attacks the consensual connotation implicit in thelanguageusedtodescribeCOPs,suggestingthatithidesthechallengesthatarisefromunfriendlyorunsociablerelationships.TheCOPliteratureisstillexploringthewideracknowledgmentof issuesofboundarycrossingwhere theremaybeindividuals situated in overlapping COPs, or where there are poorly definedcommunityedges(Fuller,2013).The evolution of ideas in COPs from legitimate peripheral participation totensions,fromevolutionarygrowthtospecificcreatedpractices,hasresultedinsome authors suggesting that there are significant deficiencies in the wholemodel.TheCOPdefinitionchangesfromversiontoversion,whichhasledcritics

55

to suggest that the definitions are unclear (Li et al., 2009) or that these aredistinct, different models, with the earliest concerning identity, self-empowerment and participation, the later amanagement tool,with a focus onmanagingknowledgeinorganisations(Cox,2005).Thedefinitionchangesfromagroup that coheres through mutual engagement on an indigenous enterprise,intoagroupwhodeepentheirknowledgeandexpertiseinanareabyinteracting.Thisfinaldefinitionsuggestsachangewhere“thepurposeisspecificallytolearnandshareknowledge,nottogetthejobdone”(p.534).GarrisonandArbaughself-critiquedCOIin2007,suggestingthatitneededmorequantitatively-oriented cross-disciplinary studies and further evidence tosuggest the relationship between the framework's components and courseoutcomes (Garrison & Arbaugh, 2007). Others followed, including Rourke andKanuka(2009)whousedaliteraturereviewtosuggestthattheindicatorsdonotcapture deep andmeaningful learning, and that as a consequence only surfacelearning ispromoted.Thepresences themselveshavebeencriticisedashavingweak interconnections (Annand, 2011) and by Krejins (2014), who criticisedsocialpresencesuggestingthatitmayactuallybetwointertwinedindicatorsinagreaterlevelofcomplexitythansuggested.

Thecriticismsthattheunderlyingtheoreticalfoundationslackedclarity(Jézégou,2010),arerebuttedbyGarrison(2011)inthelaterversion,whichmaintainsthesamepresencesandcharacteristics,butattemptstoelevatetheCOItobecomeacrediblemodel for e-learning. Despite these criticisms, COIs have remained aninfluentialmechanism for researching andpromoting community growth in anonline context, with over 1700 journal articles citing the work published in2014/15.

2.5TheresearchgapandtheresearchquestionsAn appropriate portfolio pedagogy can promote best practices in assessment,suchasauthenticity,self-assessmentandreflection,showcasingstudents’growth(Hansen, Stith, & Tesdell, 2011), particularly in programming (Carter, 1999).Portfolios are away for students to demonstrate authentic learning processesovertime,reducingtheopportunitiesforplagiarism(Blair,2011),“acculturatingstudentsintoappropriateacademicprocesses”(DeVoss&Rosati,2002,p.201).Unfortunately many of the existing studies lack rigour in the researchmethodology,with littlemore than an analysis of the student reflectionon theprocess (Bryant & Chittum, 2013). The recent call for more mixed methodsresearchintoe-Portfoliosacknowledgestheseweaknesses(Rhodesetal.,2014).

56

The use of a reflective cycle has been promoted as the optimal underlying e-Portfoliopedagogy, but as social technologyhasbeenadded to allowpeer andcommunitysupport(Bhattacharya&Hartnett,2007;Fitch,Peet,Reed,&Tolman,2010; Stevenson, 2006), the details of the process and underlying architectureremainunclear (Chau, 2010). This, combinedwith the fact thatmuchof the e-PortfoliosoftwarecurrentlyinusehasnotsignificantlyaddressedthecalltothetransitiontoWeb2.0collaborativetechnologies(Clark&Eynon,2009),indicatesagapintheliteratureonthenatureofartifactsandthecommunitythatcanexistintheseenvironments.

Thelearningcommunityliteraturepointstoabetterexplanationofhowartifactscan be constructed in a social context, along with richer descriptions of thenature of the supporting community. Networked learning suggests promotingconnections between learners and the learning resources in the community,which in an e-Portfolio, would be peer created artifacts. Both the e-Portfoliosoftware and the tutor would act to promote connections between theparticipants and the resources, so the model proposed here would have theartifacts inadualroleasassessmentartifactsandas learningresourcesforthecollaboratingpeers.

Theresearchquestionsaskwhatartifactswouldemergeinsuchanenvironment,and how these artifacts would be used, shared and reused. Communities ofpracticeandknowledgebasedcommunitiesprovideabetterexplanationforthenatureofanartifactcreatedinthissharedspace,whilstbothnetworkedlearningandthecommunitiesof inquirymodelhavewelldefinedanalytical frameworksto study the development, growth and actions of the participants and learningcommunity,asdiscussedinthenextchapter.

Thefinalresearchquestionasksaboutthenatureofthetutorroleandtheshapeofthecommunityformed.ThelearningcommunityproposedisbasedontheCOPmodel,butacknowledgesthattherearedifficultiesinusingthisintheclassroom.TheextendedversionusedherefollowsintheNLtraditionofacknowledgingthevarietyoflinkstrengththatcanoccur(Ryberg&Larsen,2008)andthatthegoalofapeerbasedcommunityallowsfortheexpertroletotransitionfromtutortostudent participant as the community progresses, as in Pedler’s equifinalitymodel(1981).

The next chapter details the research design, explaining how the researchquestions are explored in two action research cycles, using a multi methodanalyticalframeworkderivedfromDeLaat(2006b).

57

Chapter3 ResearchdesignThischapterexplainstheresearchmethodologyusedinthisproject,inparticularthe ways in which two action research cycles have been used to create an e-Portfolio learning community alongside prototype e-Portfolio software. Afterexplaining theontological andepistemologicalpositions taken in thework, thecomplexityofthemixedmethodsusedareexplored,withdetailonthecollectionandanalysisprocess.Finally, thevaliditycriteriaandethicalconsiderationsaredetailed.

3.1ResearchmethodologyThis work is an action research project, in that it seeks to explore the use ofapplicable theory inpractice,withchangesbeing incorporated intosubsequentcycles.ActionResearch(AR)seekstotransformaspectsofworkandresearchtheprocessof thischange(Kemmis,1993),bridgingthegapbetweenresearchandpractice (Somekh,1995).Thisduality is core to themethodology– suggestingonewithouttheotherisnotsufficient(Lewin,1946).Lewiniswidelycreditedastheoriginatorofactionresearchinitscriticalemancipatoryforminhisseminalpaperonintergrouprelations.Aftersomeinitialsuccessesitspopularitywanedin the1960sdue to thesurge inpositivisticeducationalapproachesduring the“space race”. The teacher as researcher movement in the United Kingdomreinvigorated action research (Elliott, 1991; Stenhouse, 1975), spinning off anemancipatory branch in Australia under Kemmis (1993). The popularity andsignificanceawardedtoitintheresearchcommunityhasfluctuated,butwithanincreasingacknowledgmentofsituatedresearchandmixedmethods,itsusehasincreasedagain(Dick,2011).Criticalphilosophicalstancesonactionresearchsuggestthatitisundertakentomaximisesocial justice (Carr&Kemmis,2003), improveone’sownandothers’identities (Kemmis, 1993; McNiff & Whitehead, 2002), or improve socialconditions (Grundy, 1987; Kemmis, 1993; Somekh, 1995). This work isparticipatory action research,where the aim is for participants acting in a co-operativeinquirytoconsiderthenatureofassessmentbyconstructingtheirownartifacts in a peer based learning community. Moving towards democratisingassessmentalignswiththeemancipatoryformsofactionresearch,buttheroleofthe academy and tutor as gatekeeper has to be acknowledged, as this placesboundary conditions on possible outcomes. Ultimately, participants have tosatisfy institutional conditions on the final assessment representations forinternalprocessessuchasmoderation,examboardsandexternalinspection.

58

Ihave chosenaction research foranumberof reasons.Thebusiness facultyatKingston University shares a campus and close links with the School ofEducation,whichhasastrongrecordofusingARinmanyofitsresearchprojects.This has informed my interpretation of AR, which aligns with my personalontological and epistemological views that there are individual perceptions oftruthandthatknowledgeisrevealedbyactingintheworld.Myoriginalreasonfor beginning thisworkwas a desire for an improvement inmy ownpersonalpracticewhenusingtechnologyinmyteaching,andARofferstheopportunityforembedding direct action into my day to day teaching activities. Lucas (1992)urgeseducatorstoallowtheuseofe-Portfoliostorespondnaturallytostudents’needs, curiosities and abilities and an action researchmethodology is the bestwayforthisgrowthtooccur.

TheactionresearchprocessinthisworkDespite thesplinteringofactionresearch intodifferent types, thereare sharedcharacteristics:

• Action takes the form of activity, consisting of cycles, with reflectionfeedingforwardintosubsequentcycles.

• Participants have a common goal or shared ethical background, whichmaybeemancipatoryorparticipatory(Boog,2003;Lewin,1946).

• Knowledgeisgainedthroughaprocessofmutualundertaking.Actionresearchistypicallydepictedasaseriesofcycles,wheretheoryisderivedfromreflection-on-actionrepeatedlyattemptingtoimprovesomepractice.Initsmost common form, participant’s work in a peer based relationship sharingbehaviourandpractices,withreflectiveprocessesfeedingforward.Thetimeandstudentsavailableforthisprojectallowfortwocycles.

59

Table3.1-Cyclesintheproject

CycleOne CycleTwoPurpose Creatingane-Portfoliocommunity

Levelofstudentsinvolved

UndergraduatesinaWebScriptingclass

PostgraduatesinaBusinessInternetSystems

classNoofParticipants 15participants 17participants

When Fivemonthsintoproject Ninemonthsintoproject

Period 8Weeks 8WeeksPriortothebeginningofthefirstcycle,Ihadtheopportunitytorunasmallpilotwhichtrialledvariousrepresentationsofonlineartifactswithsecondyear, finalyear and postgraduate students as they approached the end of their academicyear. Thiswas also an opportunity to see if theworkwas practicalwithin theconfines of the academy, my own technical abilities, and to see if this wassomethingthatwouldengagepossibleparticipants.Althoughtheintentionwasforcycleonetorunforanentireteachingperiod,thecomplexitiesinvolvedinbothinitiatingthetechnologyandchanginginstitutionalpractices whilst teaching, “Designing the plane whilst flying it” (Elliott, 1991;Herr&Anderson,2005;Stenhouse,1975),meantthatthestartwasdelayedtillhalf way through the semester. Roughly half of the group volunteered toparticipate, requiring an alternative assessment process to be run in parallel.Feedbackfromtheparticipantsandmyownreflectionswerethenfedintocycletwo,whereallthemembersofapostgraduateclassvolunteered.Participants in these cycles acted as co-researchers in a networked learningbasedcommunity,butalsoactedasco-developersforthesoftwarethatemergedfrom the process, created using the skills and tools that the students werethemselveslearning.Theintentherewasthatbymakingtheparticipantspartoftheopensourcedevelopmentproject, theresultingcommunitywouldhave theopportunitytofullyengagewiththeresearch,thecurriculumandthenatureofsoftwaredevelopment,which in itself is oneof the learningoutcomes forbothclassesincycleoneandtwo.FLOSS(FreeLibreOpenSourceSoftware)advocatessuggest that software developed during open source development is moreclosely aligned with the feature set and facilities demanded by its users – asparticipantswill be learning the tools and technologies that areused to create

60

theportfoliosystemitself,theywillbeabletodirectlyinfluencethefeaturesandfunctionality of the software, either as users during the initial parts of themodule, or as developers as their skill levels progress. Any software, tools orcodeproducedduringthisprocesswillbeplacedundertheGNUGeneralPublicLicenseforgeneralreuseattheendoftheproject.Amajorityof freesoftwareprojectsuse this license,whichoriginated fromtheFree Software Foundation (FSF), a not-for-profit organisation, founded byRichard Stallman in 1985. It creates, distributes and advocates free software,licensedunder theGNUGeneral Public License (GNUGPL),which enforces thefoundation’spoliticalbeliefinfreesoftware.Theseareenshrinedinauser’sfouressentialfreedoms-(0)torunaprogram,(1)tostudyandchangeaprograminsourcecodeform,(2)toredistributeexactcopies,and(3)todistributemodifiedversions (Stallman & Gay, 2009). The license ensures the continuity ofdevelopment, in that any changes to a GPL program must also be similarlylicensedandplacedinthepublicdomainforotherstouse,changeordistribute.Theintroductionofthelicenseiswidelyregardedasapivotalmomentinthefreesoftwaremovement.The Open Source Initiative (OSI) has similar goals in that it advocates thedevelopment of freely available software and source code, but concerns itselfwithdevelopmentalprocessesratherthanpoliticalaims.Itwascreatedin1998,anddifferentiateditselfbyseeingsoftwarefreedomasapracticalratherthananideologicalmatter(Dick,2011;OSI,2013).FreesoftwareadvocatesdecrytheOSIforabandoningpoliticalaims(Carr&Kemmis,2003;Stallman&Gay,2009),butthedifferencesbetweenthetwoarefrequentlyoverlookedbymostusers,sothenameFreeLibreOpenSourceSoftware(FLOSS)hasdevelopedasablankettermfor software from either camp. This work will use the term FLOSS, butacknowledges that there are significant political and ideological differencesbetweenfreeandopensourcesoftware.It is claimed that FLOSS has advantages over themore typical closed softwaredevelopmentmodel–thecharacteristicsofsuchsoftwareare:

• afrequentreleasecycle–releaseearly,releaseoften;• anengageduserbase;and• afeaturesetalignedwiththedemandoftheusers

(Raymond,2008).

61

A more critical perspective of open source suggests that the softwaredevelopment process may only develop software that developers have anintrinsicself-interestin,thatthetraditionalpowerrolesareassertedthroughthepopularity and use of the software, and that it has an underlying ideologicalagenda espoused by opposite ends of the political spectrum (Elliott, 1991;Selwyn,2013;Stenhouse,1975).There are analogies between action research and the FLOSS developmentmethodology,whichforthisparticularprojectmaketheconnectionbetweenthetwouseful.BothARandFLOSSdevelopmentusecyclesofdevelopment,ARwithAnalysis-Planning-Acting-Reflecting (Lewin’s original 4 step cycle), FLOSSwithrapid software release cycles in the traditional waterfall steps of Analysis –Design–Testing–Implementation.Eachcycleisrepeated,integratingreflectionsandimprovementsintothenextcycle.EmancipatoryARquestionsthenatureofpowerandroles,theFreeSoftwareFoundationadvocatestherightsofausertouse software in anymanner they wish, suggesting an equality between users,developers and corporations. OSDM promotes the idea that users of softwareshouldbetreatedasco-developers,eveniftheydonothavethetechnicalskillstocode themselves. The rapid release cycle is used to ensure their engagementthroughincorporationoftheirexperiencesandchangerequests,whichissimilartotheactionresearchprocessofaskingparticipantstobeco-researchers,usingtheir reflections to change the environment, power relations or situations insubsequentcycles.There are thousands of orphaned educational software projects, where theoriginal author or company has abandoned selling and supporting software.Multiplewebsitessuggestwaysinwhichthisabandonwarecancontinuetobeofuse ineducation(Dube,2010).Assoftware fromthisprojectwillbedevelopedusing the FLOSS principles, the source code will be free and widely available,with the possibilities of continual development if it finds a supportive base ofusers.

CasestudyManyactionresearchprojectsarepresentedascasestudies(Koshy,2009).Thecase study is used where a specific instance is designed to illustrate a moregeneralprinciple(Nisbet&Watt,1984).Theytypicallyacceptandrecognisethattherearemanyvariablesanduseavarietyofmechanismsfordatacollectionandsources of data (Cohen, Manion, & Morrison, 2008). The non-privileging ofspecific methods in AR or case study research is coherent with both the

62

philosophicalstanceandapproachtakeninthiswork.Thenextsectionexplainstheroleoftheresearcherinactionresearch,alongwithmyphilosophicalstanceanditseffectontheapproach.

3.2MyroleandphilosophicalstanceinthisresearchIn qualitative research the researcher has to explain their ontological andepistemological position, which has consequences for choices of methodologyandmethod.Thisisalsoimportantinactionresearch,wheremultipletraditionshaveevolvedwithdifferingphilosophicalstances.The role of the researcher in an action researchproject situates thework in acontinuumfrominsider tooutsiderresearchandassociates itwithaparticulartradition and validity criteria (Herr & Anderson, 2005). This work sits in theinsider and insider in collaboration with other insider position, “the second-person, inter-subjective inquiries of groups and communities of co-researchersengaged together incritical researchandpractice” (Reason&Torbert,2001,p.2),buthastoacknowledgethecontradiction insupportingpeerbased learningcommunities inaneducationalsettingwherethereare implicitpowerrelationsinthetutorstudentrelationshipwherethereisagatekeeperrole(McNiff,2013).

In action research, epistemology and ontology are intertwined; the nature ofrealityandhow thatnature isuncoveredare combinedbyaction in theworld.Acknowledging that individuals create their own identities, values and truths,meansthattheactionresearcherhastofindwaystoaccommodatethemultiplevaluesperspective(McNiff,2013).Someauthorssuggestthatactionresearchhasan objective ontology and subjective epistemology (Coghlan&Brannick, 2009;Sikes&Potts,2008),butthisworkissituatedininterpretivism,inthatrealityistheproductofconsciousnessandthereareindividualconceptionsoftruthwhichare uncovered by action in the world. Knowledge is socially constructed, butinformed by a pragmatism that suggests action through advocacy andparticipatoryapproaches-“Actionresearchersseeknowledgeassomethingtheydo,alivingprocess”(McNiff,2013,p.18).As this is insider action research, the philosophical position of the researcherdirectlyinfluencesthesituationanditisimportantthatthisalignswithboththeapproach and the situation being explored. Denscombe (2010) suggests thatmakingthis‘publicaccount’isanimportantaspectofAR.

Researcher’sexperienceandperspectiveonteachingandlearningWhenIstartedteachingtwenty-fiveyearsago,Iwasoneofthefewstafftoholda

63

teachingqualificationwhichhada specific focuson lecturing in theFEandHEsectors, from Garnett College, part of the University of Greenwich. The coursehad a profound effect onmy teaching philosophy, as it emphasised a range ofpedagogicpracticesthatwerelargelyunusedinfurtherandhighereducationatthe time. For the second part of the course I was simultaneously student andteacher,adualperspectivethathasalwaysstayedwithme.

During this time I taughtprogrammingand informationsystems toBTECHNC,HNDandundergraduatedegreestudents.ThevocationalprogressivisminBTECspedagogicpractices(Fisher,2003)influencedmyteaching;integrationoftheoryinto practice, learning objectives and learning outcomes characterised myteachingattheoutset.ForalongwhileIregardedBigg’sconstructivealignmentinassessment(2010)tobeakeyapproachinmyteaching.

TheincreasingpopularityoftechnologyenhancedlearningresultedinKingstoninvestingheavilyintheBlackboardlearningmanagementsystem(LMS)andtheblendedlearningapproach.DespitetheresourcesallocatedtotheLMS,manyofthemoreadvancedfeatureshavenotbeenwidelyusedanduntilrecentlyitdidnot address social componentsof learning.As it is auniversity-based resourcestructuredaroundsubjectunits(modulesorclasses),itlacksneitherthecoursecontinuity,nor theability for someof theadministrative tasks tobe sharedbyparticipatingpeers.

This and a dissatisfactionwith the predominant lecturing pedagogy acted as atriggerformyregisteringforthePhDatLancaster.Thiscoursehassignificantlytransformed the way I regard the use of technology in education and hasemphasisedtheimportanceofsocialconstructivistbasedpedagogiesinmywork,whereIviewthesocialaspectincollaborativelearningenvironmentsasvital.Insmaller, initial projects, I experimented with creating online learningcommunities using the social networking site NING (2014) and mobiletechnologies.This shift inpedagogy is evidenced inmypractice today,where Iuse exercises, tasks and communities to promote peer appraisal and reflectiveprocessesinlecturehalls,tutorialgroups,andpracticallabs.

Mycurrentteachingusesalearningcommunitybasedonthenetworkedlearningphilosophytopromoteanequifinalitymodelwithcollaboratingpeers,wherethelecturergraduallytransformsintoaguideoverthelifeofthecommunity(Pedler,1981). In this work I have used personal learning environment and personallearning network (PLN) techniques, both of which are particularly suited toalways available working practices on mobile devices (van Harmelen, 2006).

64

Using various online tools, students are taught how to create customisedhomepages,usesearchtoolsandgeneratefeedsaroundsubjectsofinterest,suchasinternetmarketing,retailmarketingorprojectmanagement.Amicrobloggingsite is used for the PLN concept, where students are shown how to use theservice to create a professional online identity, following, sharing andcollaboratingwithprofessionalsinaparticularareaofinterest.

3.3TheparticipantsThisthesiswilldocumentthefirsttwocyclesoftheproject,whichusealternateundergraduate and postgraduate students as the cycles progress. Despite thedifference in levels suggested, there aremany similarities in the nature of thestudents and the material covered. Kingston University allows for lowerundergraduate entry scores through a widening participation agenda; atpostgraduate level lower IELTS grades are accepted. A focus on improvingacademic study skills is integrated into both programme structures and isachieved throughdistinctmodules anddrop in support sessionsoffered in thelearningresourcecentrewithdedicatedacademicstaff.Participants for both the second year undergraduate web scripting forapplications class (WSA) and the postgraduate business internet systems class(BIS) are technically capable students, with varying levels of ICT andprogramming ability at the beginning of each module. Both sets of studentsattend full-time, with between 12-16 hours of contact time depending uponsubjectchoice.ThemajorityofstudentstakingWSAandBIShavethesubjectasacore,withsmallerpercentageselectingtotakethecoursefromarangeofoptionchoices.Theundergraduatestudentsaretypically20-21yearsold,withtheWSAclassasonefromsixclassestakenduringtheyear.BISstudentsareolder,24-29yearsoldforthiscohortandaretakingthreeothersubjectssimultaneously.Bothmodules coverweb technologies, programming languages and the use ofdatabases,whicharethencombinedtogethertoenabletheconstructionofwebbasedapplications.TheBISmoduleisanintroductorymodule,sodespitebeingapostgraduate level coursemuchof thecurriculaare similar,with theexceptionthatMBITstudentshaveamoredetailedintroductiontodatabasetechnologies.Bytheendofthemoduleitisexpectedthatstudentswillhavesufficienttechnicalandpracticalexperiencetobeabletoeitherbuild,ormanagethebuildingofwebapplication. Further detailed information about the participants is in chaptersfourandsix,whichcovercycleoneandtworespectively.

65

Thenextsectionexplainsthewaysinwhichmixedmethodswereusedtoanalyseandunderstandtheuseofe-Portfoliosinthelearningcommunitiesthatemerged.

3.4Researchmethods,techniquesandprocedureMixedmethodshavebecomeincreasinglypopular,“aresearchdesign…inwhichthe researcher collects, analyses, and mixes (integrates or connects) bothquantitative and qualitative data in a single study or amultiphase program ofinquiry” (Cresswell in Johnson, Onwuegbuzie, & Turner, 2007, p. 119). Itsclaimedadvantagesinclude:

• onemethod’sstrengthsbalancingagainstanother’sweaknesses(Johnson&Turner,2003);

• investigatingissuesfromdifferentpositionsandthenconvergingtheresults(Creswell,2013);and

• illumination, precision and the investigation ofmore complex problems(Greene,Caracelli,&Graham,1989).

Theuseofmixedmethodsstrategieshasbeenjustifiedinnetworkedlearning.DeLaat (2007; 2003; 2004; 2006a) argues for a rich range of data collectionmethods that reflect the complexnatureof thepraxis that exists innetworkedlearningbasedcommunities,“Moreofthiskindoftriangulationworkneedstobedone beforewe can claim to have a rounded picture of networked learning inhighereducation”(Goodyearetal.,2005,p.505).Thereareavarietyofdatacollectionmethodsusedinthisproject,withamulti-method framework,usingsocialnetworkanalysis,contentanalysisandcontextanalysisincyclesoneandtwo.

MethodologyandmethodsincycleoneandcycletwoUsing a wider variety of methods adds complications for the researcher, withextensive data collection and complexity in analysis requiring technical,qualitativeandquantitativeskills.Themethodsusedneedbejustified(Dick,2011;Koshy,2009)andhowtheywill bemixed should bemade clear (Creswell,2013).In a number of papers De Laat et al. developed amulti-method research framework to studynetworked learningprocesses,usingSocialNetworkAnalysis (SNA), Content Analysis (CA) and Context

Figure 3.1 – De Laat’s researchframework

66

Analysis(CxA)tofindout‘whoistalkingtowhom...whattheyaretalkingabout’,and‘whytheyaretalkingastheydo’(DeLaat,Lally,Lipponen,&Simons,2006b,p. 398). The three methods are combined (figure 3.1) to “triangulate andcontextualise our findings and to stay close or connected to the first-handexperiencesoftheparticipantsthemselves”(p.399).Amodifiedversionapplicabletoassessmentartifactsinane-Portfoliobasedonnetworkedlearningprinciplesisusedhere,with:

• SNA to see who is connecting to whom and the overall shape of thelearningcommunity,usingactivitytablesandsociograms.

• CAtoseewhatartifactsarebeingcreated,codedusingthematicanalysis.

• CxAtoseewhytheparticipantsbehavedastheydoduringthecycle,using

resultsfromquestionnaires,codedinterviewsandcommentsfromwithinthee-Portfoliosystem.

Context Analysis was further enhanced with information from weekly logs,emailsandin-cycleinteractionswiththeparticipants.In itsoriginal context,DeLaatusesSNA toseewho is talking towhom. In thiscontext SNA will reveal the path that participants take through the portfoliolearning community, showing activities in terms of using, commenting andviewingothers’artifacts.Contentanalysisappliedtoforumtextmessageshasalongprovenhistory,forexampleinthecommunityofinquirymodel,butherethenatureofanartifact’srepresentationaddscomplexity.Thisanalysisisformedbyreviewingtheimage,reflectivetextandassociatedcommentsinplace.The next section details the data collectionmethods used in each cycle, and isfollowedbymoreinformationontheanalyticalprocessesused.CycleoneCycleonebeginsfivemonthsintotheproject,with15studentsparticipatingforaneight-weekperiod.Duringthis time, theparticipant’susedthee-Portfolio tocreate artifacts, interacted with each other in the e-Portfolio community andcontributedtoartifactdiscussions.Participantswereinvitedtoprovidefeedbackontheprocesseveryweekand ina finalonlinequestionnaireat theendof the

67

eight-weekcycle.Datacollectedduringthecyclewasusedtobothtransformtheprocessasitprogressedandforapostcyclereflectionthatwasthenfedforwardintothesecondcycle(table3.2).

Table3.2–Datacollectionincycleone

Description Method Means Date Sample

Reflections

inthecycle

Weeklyinclass

feedback/logbook

Collatedin

Word

Weeklyforthe

eightweeks

Activityin

theportfolio

SNAsocialnetwork

analysisoveractivity

records

Analysisusing

Excel

Duringtheeight

weekcycle

e-Portfolioactivity

foreachweek

Reflections

onthecycle

CxAfinalquestionnaire Online Atendofcycle 11participants

Artifact

analysis

CAthematicanalysis

overartifactsand

discussions

In-vivocoding

inExcelfrom

reflective

statementand

tags

During/postthe

eightweekcycle

381artifacts

Comments

analysis

CAcodingusingtheCOI

teachingpresence

Analysisusing

Excel

Postcycle

DeLaatexaminesthenatureoftheteachingpresenceoverthetimeofalearningcommunity by using the COI teaching presence indicators (De Laat, Lally,Lipponen,&Simons,2006a).Herethishasbeenappliedto thetutorcommentsattachedtotheartifacts.Asparticipantsusetheportfolio,theiractivitywasautomaticallyrecordedinanactivitydatabasetable,whichrecordedthenameoftheparticipants;thedateandtime;and thedetailsof theiractivity.Thearchitectureof thee-PortfolioallowsfortheactivitydatatobeeasilyprocessedwithcustomiseddatabasequeriesintheprogramminglanguageSQL,exportedandthenpostprocessedinExcelandthedataanalysissoftwareR.Theartifacts,dataanalyticsandfeedbackfromtheparticipantswereusedbothinsidethecycleinthetutorroleandinthereflectivephase.At the end of the process participants were asked to complete an onlinequestionnaire, which is fully detailed in chapter four. Topics covered included

68

howtheydecidedtocreateanartifact,howtheyre-usedother’sworkandtheiroverallexperiencesofthelearningcommunity.CycletwoCycle twooccurredninemonths into theproject,with17studentsusing thee-Portfolio for an eight-week period. As with cycle one, participants createdartifacts and participated in the e-Portfolio learning community. Participantswere invited to feedback every week and in final individual interviews. Datacollected during the cycle was used to both transform the process as itprogressedandforapostcyclereflectionthatwasthenfedintotheinterviews.Table3.3–Datacollectionincycletwo

Description Method Means Date Sample

Reflectionsin

thecycle

Weeklyinclass

feedback/logbook

Collatedin

Word

Weeklyfor

theeight

weeks

Activityinthee-

Portfolio

SNAsocialnetwork

analysisover

activity

Analysisusing

ExcelandR

Weeklyfor

theeight

weeks

Alle-Portfolio

activityforeach

week

Reflectionson

thecycle

CxASemi-structured

interviews

Facetoface Postcycle 11participants

Artifactanalysis CAthematicanalysis

overartifactsand

discussions

In-vivocoding

inExcelfrom

reflective

statementand

tags

During/post

theeight

weekcycle

1647artifacts

Comments

analysis

CAcodingusingthe

COIteaching

presence

Analysisusing

Excel

Postcycle

Aswellascontinuingmanyofthein-processmethodsfromcycleone,therewasan opportunity for amore thorough examination of the participant’s views. Attheendofthecycle,11oftheparticipantswereinterviewedinsemi-structuredsessions, with graphical representations of their activity used with theparticipantsfortriangulation.Howsocialnetworkanalysis,semi-structuredinterviewsandcodingtechniqueswereusedincycletwoisexploredinmoredetailinthenextsection.

69

DatavisualisationusingsocialnetworkanalysisIt is easy to generate large amounts of activity data from online systems, butfindingappropriateanalysis techniques isastillanemergingfield.Datamininghas been demonstrated as a technique that has applicability for assessingparticipation (Dringus & Ellis, 2005) and for making class interaction visible(Baggs & Wu, 2010). In particular, social network analysis (SNA) has beenapplied to fieldswhere the social connectionsbetweenagents canbe capturedand measured. This can reveal both the individual connections and a wholenetworkview,aperspectivewhich“providesaviewoftheentirestructure,andthus of the ‘character’ of the network to which an individual belongs”(Haythornthwaite&DeLaat,2010,p.189).SNAhasbeenused to study thenatureof interactionof learners innetworkedlearning fordualpurposes; foreducators to improveconnectionsand to fostercollaboration between learners, and for researchers investigating learningrelationships and their value (Cambridge & Perez-Lopez, 2012; Schreurs,Teplovs,Ferguson,DeLaat,&Shum,2013).DeLaat(2006b)arguesfortheuseofSNA tomeasure interactionpatternsover time; ithasalsobeensuggested thatSNAmayprovideaneasiermeasureofsocialpresenceinacommunityofinquiry,overthemoretraditionalcodingandcontentanalysisapproach(Choi&Strobel,2012;Mika,2007;Sheaetal.,2010).Social network diagrams allow for the connections and relative placement ofactorsinanetworktobecomevisibleinsociogramdiagramsderivedfromdataingraphtheoreticnotation(Wasserman&Faust,1994),providingbettervisualrepresentationsoftheactorsandrelationships.Generatingsociogramsfromlogfiles has been successfully used to examine the relationships between artifactsand participants in networked learning based communities (Suthers & Rosen,2011).SNAusesthefollowingterminology(Wasserman&Faust,1994):

• Anactorornodeiseachsocialentityrepresented.• Arelationaltieoredgeisadirectedlinkbetweenactors.• Dyadsarepairsofactorsandtherelationaltiebetweenthem.• RelationsencompassthenatureofthetieinagroupofDyads.

70

The sociograms represent each actor with a circle with either the number ofartifacts created over a learning period, orwith a participant identifier (figure3.2).Arelationaltie,(theconnectingline)indicatesaparticipantinteractingwithanotheruserthroughbrowsing,commentingorreplying.Thesearevalueddirectedgraphs (Wasserman&Faust,1994),where

• A->BindicatesAinteractswithB’sartifacts.• B->AindicatesBinteractswithA’sartifacts.• A<->Bindicatesinteractioninbothdirections.

Asingle linebetweentwoactorsrepresentsthestrengthoftherelationshipinbothdirections,witharrowheadsindicatingwhetheritwasa single ormulti direction relationship. All activities in the system have equalweight, so for example searching for a user, viewing another’s work, orcommentingareallvaluedequally.The Fruchterman-Reingold algorithm (1990) is the most frequently usedmechanism for deriving the optimal placement of actors. Actor placement isderivedfrominteractionactivity,i.e.actorsplacedmorecentrallyinthediagramhavestrongerrelationalties,havinginteractedmorefrequently.Actorsplacedattheedgeofthediagramhaveinteractedlessfrequently.Here,actorplacementisderivedfromeachuser’sactivityhistory,asparticipantsmovingfromactivitytoactivitycreaterecordsintheactivitytable.Ratherthanusethestrongandweakbinarydivide, relational ties in thesociogramsuseadarker line to representamore frequent number of interactions i.e. a stronger relational tie, as seen inDoran(2011)andMazur(2010).The sociogram diagrams used inthis analysis were created withbespoke R scripts, after the cyclehad completed. Sociograms wereconstructed using R Studio(http://www.rstudio.com/) andthe graph-sociogram library(http://igraph.sourceforge.net)with a bespoke script written to

Figure3.2-Sociogram

WSA Week 7

0

0

7

0

0

0

0

0

0

10

0

29

10

2

0

10

Figure3.3-BespokeRscriptforsociogramproduction

71

generate coloured relational ties (figure 3.3). The strongest relational tie isdepictedasblackandtheweakestasalightgrey.Tiessittingbetweenthesetwoboundaries are then assigned colours derived from a value calculated from aratiobetweenthetwoouterpoints.Theexamplesociogram(figure3.4),wouldindicatethatparticipantsontheouteredges have created few artifacts, indicated by the zero values. They haveinteracted less online, which can be deduced by the placement and by thelightness of the connecting relationships. A more active core group haveuploadedmoreartifacts{8,3,9,10,5},haveahighercentralityandweremoreactivewitheachother,withthestrongestnumberofinteractionsbetween{10}->{9}and{1}->{8}.Various quantitative methods are available inSNA,althoughmanyofthemarelessapplicabletovalueddirectedgraphs.Herethefocuswillbeon sociogram use to strengthen theunderstanding of the network evolution overtime. Rather than use the strong and weakbinary divide, relational ties in the sociogramsuseadarker line torepresentamore frequentnumberofinteractionsi.e.astrongerrelationaltie,asseeninDoran(2011)andMazur(2010).The applicability of the statistical measuresavailable to represent overall and individualactivityinasocialnetworkareexploredinfurtherdetailincycleoneandtwo.Amoredetailedexplanationofthedatathatiscollatedinthee-Portfolioactivitytablesisinsection4.2

Interviews,codingandinterpretativeanalysisThere are a number of data collection opportunities that lend themselves to acodingprocessinthisproject,includingdatafromartifacts,commentsinsidethee-Portfolio,questionnaireresults fromcycleoneandsemi-structured interviewtextfromcycletwo.Asparticipantsfromthefirstcyclewereduetoleavecampusalmostimmediatelyafter the sixth week, feedback was collected through an online questionnaire.Thisinformationhasbeencombinedwiththeweeklylogs,emails,commentsand

WSA Week 3

1

0

0

0

9

0

8 3

0

5

0

0

0

10

0

0

Figure3.4-Sociogramexample

72

suggestionsnotedthroughoutthecycle.ATLAS.tiwasusedtocollatethevariousdocumentsandfeedbacktogether.Cycleoneresultswereanalysedbetweenthetwo cycles, so that feedback from the first cycle could feed into the second,although sociogram representations were created after both cycles hadcompleted.Semi-structured individual interviews were conducted with eleven of theparticipants from cycle two, and were analysed with thematic analysis. Thesevariedinlengthsfrom23to45minutesandwererecordedandthentranscribedby a third-party. This has been combined with weekly feedback from theparticipants, logs,emailsandnotesmade fromin-situconversationsduring thecycle.Thequestionsaredetailedinchaptersix,butinsummarytheyaskedaboutthe participant’s individual experiences using the e-Portfolio, the interactionstheyhadwithothersandtheirperceptionofthelearningcommunityasawhole.Thematicanalysisasamethodfor identifying,analysingandreportingpatternsiswidelyusedbutisperhapsilldefined(Braun&Clarke,2006)ornotexplicitlynamedinmanystudies(Boyatzis,1998).Tocounterthis,Braunsuggestsmakingtheassumptionsandprocessesusedexplicit, including thenatureof codes, thethemes and the methods used. This work uses an thematic analytical processderived fromBraunandClarke (2006)with recursive stagesof familiarisation,generationofinitialcodes,themeabstraction,reviewandfurtherdefinitionusingthe 15-point checklist for ‘good’ coding. Codes represent a level of repeatingpattern inductively determined across the whole dataset, but as these arederivedfromsemistructuredinterviews,theroleandpre-conceivedresearcherpositionmustbeacknowledged.IappliedBraunandClarke’scycleasfollows:

1. Familiarisationwiththedata–adegreeoffamiliarisationwaspresentasIwas thepersonconducting the interviews.The interviewswere listenedtowiththetranscriptstoreducetranscriptionerrorsandtomakegeneralnotes, thoughts and ideas. ATLAS.ti allows audio and text to be linkedtogether,whichallowed theaudio tobe labelled in5minute chunks forquotestobeheardincontext.

2. Generating initial codes – an inductive approach was used to generate

codes, with a name and descriptive comment appended to parts of the

73

text.Afterthethirdinterviewwascoded,codesfromthefirst interviewscould have introduced bias into the further coding processes, so toovercome thisnotes andbreakswere taken.Each interviewwasparsedagain to revalidate the coding process. Codes were identified usingtechniquessuggestedinRyan(2003).

3. Searchingforthemes–thecodeswerethenorganisedintothemesusing

the family functionality in ATLAS.ti. The codes, comments and quoteswere then exported to a spreadsheet for ease of sorting, searching andmanipulating.

4. Reviewing themes – themes were refined, reviewed and (if necessary)

collapsed,withadjustmentsmadeifrequired.

5. Defining and naming themes – themes were aligned with the researchquestionsandthencategorisedintosubdomainareas

6. Producing the report – the themes and linked codes were used to find

appropriatesignifyingcommentsandevidencethatwerethenembeddedinthefindingsforcycletwo.

Steps 3, 4 and 5were repeated refining the themes and names as the processcontinued. As validation is an important part of the process, I discussed theprocesswithmyPhDcohortandreviewedthecodingwithacriticalfriendwhoworksatKingstonUniversity.Forbothcycleoneandcycle two, thecommentsassociatedwithartifactswereextractedusingaSQLqueryandthenpassedintoATLAS.tiandExcelforcoding.Thematicanalysiswasused tocode theresponses,whichcouldbeanalysedbytime,participantandbyartifact.

CodingportfolioentriesThe variety of portfolio implementations has had a direct effect on the datacollectionmethodusedinresearchstudies.Althoughthereisalargeamountofresearchintothetypes,usesandeffectsofportfolios,therearefewstudiesthatdirectly analyse the content of the artifacts, perhaps because of the variety instyle and representations of artifacts, along with differences in the domainknowledgecovered.

74

Wherethearchitecturesupportsanarrativeflowofartifacts,thenananalysisofthe portfolio content has been used to reveal the rich picture in thetransformationofthelearnerintertwinedinwiththeportfoliouse.Architectureswhere the artifacts are presented in more discrete units have been analysedusingcontentanalysis,withcodingmechanismsaimedatrevealingthenatureofthe learning taking place through taxonomic categories. De Laat uses contentanalysis with SNA (De Laat, Lally, Lipponen, & Simons, 2006b) to explore thechangingnatureofrolesinanetworkedlearningbasedcommunitybyusingthecommunityofinquiry(COI)codingscheme(Andersonetal.,2001).UsingtheCOIteachingpresencewithe-Portfolioshasbeensuccessfullyusedtoshowthelevelsof teacher interaction from emails and reflective blog entries (Torras &Mayordomo,2011), although thee-Portfolioarchitecture in theTorres study isdifferenttothatsuggestedhere.Determining the levelofcoding inane-Portfolio isdifficult, similar to issuesofcodingaforummessageinCOI.Ane-Portfolioartifactwithcommentscouldfallinmanycodingcategories,soanassumptionhastobemadeabouttheatomicityof the coding that canbe applied. Existing e-Portfolios coding schemes tend toavoid the social activity inside the portfolio, focussing on the artifactsthemselves.Thisworkwill use thematic analysis to reveal the general themes in cycle oneand cycle twoby reviewing the artifacts inplace, through the image, reflectivetextandtagsassociatedwitheachitem.MorespecificcodingofthecommentsisusedtorevealthenatureoftheteachingpresenceintheartifactsformedusingCOI.

3.5Quality,validityandreliabilityInactionresearch,participantsareboththeobjectofstudy,initiatorsofchangeandresearchers.Researcherasparticipanthas implications forvalidity criteria(Heikkinen,Huttunen,&Syrjälä,2007),whichisoneofthefrequentcriticismsofthemethodology,alongwiththesuggestionthattheoutcomesareneithervalidnor generalisable. To rebut this, validity and reliability in the project contextneed to be clear (Koshy, 2009). Thisworkwill uses the term “validity”whichdespite its positivistic roots can be used in action research projects todifferentiatefromtheconceptoftrustworthinesswhichdominatesinnaturalisticinquiries(Herr&Anderson,2005).

75

Heikkinenet al’s quality indicators (2007)onARposeanumberof principlesthatneedtobeaddressed:Principleofhistoricalcontinuityandevocativeness�- how is the evolution of theaction researchevolvedhistorically and in context? Somewhat similar toHerrandAnderson’soutcomeandprocessvalidity(2005),thisarguesforanarrativethatevokesboththehistoricalcontextandtheprocess itself,which ithasbeensuggestedcanbeusedtosuggestinterpretavisticvalidity.Herethisisattemptedbytheinitialcontextdescriptionandthededicationofsuccessivesectionstothein-cycle and post-cycle reflective stages. Influenced byDavis (2007), thisworkdoesnotabandonthetraditionallinearapproachforthewholethesisstructure,but it does frame the presentation and analysis sections around the actionresearchcyclesthattookplaceinthisproject.Principle of reflexivity, dialectics and evocativeness – are the nature of therelations,theclarityoftheontologyandepistemologyclear? Theontologyandepistemologyofthisworkareexplainedearlierinthischapter,withtherelationsevokedthroughtheuseofparticipantnarrativesfromtheinterviewsandintheuseofSNA.

Principle of workability�- does the research create workable practices, withappropriateethics,empowerment?Theethicalpracticesdeployedinthisprojectarediscussed in the subsequent sectionand it is hoped thatboth the softwareand practices developed in this process are shown to be a catalyst for futurework(afterHerr’scatalyticvalidity).

In action research, validity is achieved by sound and robust data collection(Koshy, 2009), but these also serve as indicators of reliability. Kemmis,McTaggart and Nixon (2013) argue that ‘to do action research is to plan, act,observe and reflect more carefully, more systematically, and more rigorouslythan one usually does in everyday life’ (p. 210) and demonstrating theseprocessesisneededtoaddtotheauthenticityoftheaccount.

Thereliabilityinthegenerationofthesocialnetworkanalysiswasperformedbymatchingeachgridanddiagramagainstrawvaluesgeneratedfromtheactivitytable. Any inconsistency in diagrams or activitywas clear from the number ofartifacts produced by each participant each week. A backup routine wasdevelopedtoarchivetheentiree-Portfolioasasnapshottwiceaweek,ensuringthatifanyfaultoccurred,theportfoliocouldberestored.Thisbackupwasnever

76

required.

Thereliabilityof the interviewsandquestionnairewasensuredbyusingpilotsand consistency in question type and approach. Interviews were recorded onmultipledevicesandtheaudiowaslistenedtomultipletimesduringthecodingprocess.Participantswereasked forpermission for followupemailsaspartoftheethical agreement, shouldany issuesof clarificationarise.The reliability inthecodingofinterviewsorofinformationfromtheportfoliosystem(commentsand artifacts), was ensured by applying pre-existing schemes or by asking acriticalfriendtovalidatetheappropriatenessofthecodingbeingapplied.

3.6EthicalconsiderationsActionresearchraisescomplexethicalissues(Nolen&VanderPutten,2007)dueto thedualobserver-participantrole,particularly ineducationalsettingswhereparticipation, or non-participation may be perceived as having side affects on“grades, access to resources and enriching experiences” (p. 402). Herr (2005)argues that there is no guaranteedway to avoid ethical dilemmas in an actionresearchproject,andthattherefore“muchisaskedoftheactionresearchers interms of continuously exercising professional judgment” (p. 112). Theimportance of addressing possible ethical issues is also an important validityindicatorforAR(Heikkinenetal.,2007).Ethicalissuesforthisprojectwerecomplicatedbythenatureoftheassessmentcommunity the studentswere toparticipate in.All participantswere given thechoicetoparticipatewithnon-prejudicingalternativesforthosestudentswhodonotwishtotakepart.Incycleoneapproximatelyhalfthestudentsdidnotwishto take part and were given an alternative assessment that covered the samelearningoutcomespresentingtheirworkinamoretraditionalmanner,insidetheinstitutional learning management system. This ensured that the workloadamongst students in and outside the project remained consistent and wasvalidatedbyKingston’sinternalqualityprocess,whichconsistsofmoduleleaderreports, feedback processes from module leaders to boards of study, studentfeedbackandexternalexaminerreports.Noissueswerereported.Allparticipantsavailableforcycletwoelectedtotakepartintheprocess,whichmay be due tomore clarity being available onwhat participationwould entailand the steps taken to introduce the nature of the community, as detailed inchaptersix.Kingston’sinternalqualityprocessforpostgraduatemoduleshasthe

77

sameprocessasusedwith theundergraduates in cycleoneand therewerenoissuesreported.Participants for both cycles were asked to sign informed consent andconfidentialityletters,whichcoveredthat:

• every effort would be made to preserve the confidentiality of theparticipants,withpseudonymsused foranywork, activitiesor feedbackreportedinthiswork;

• participants could withdraw at any time, with alternative assessments

available to coveranymissingopportunities todemonstrate their skills;and

• contactdetails for staff (bothatKingstonandLancaster) in theeventof

anyissuesthatcouldnotberesolvedintheprocessitself.Studentswho participated in the questionnaires and interviewswere asked tosignspecificethics statements,with textexplaining thecontextandpurposeoftheparticulardatacollectionmethod. Intervieweesweregiven theopportunitytoreviewthetranscriptsfromtherecordedsession.Online activity canbringout anew set of ethical issues (Anderson&Simpson,2007),suchas:

• thenatureofhierarchicalpowerrelationsinonlinecourses,• thevalueascribedtoparticipation,and• surveillance.

Participantsusingthelearningcommunityweresacrificingalevelofprivacy,asthe peer-to-peer nature of the work necessitated their online identities to beknowntoeachother.Thiswasmadeclearduringthesignupphase,duringtheintroductiontotheprojectandbeforeeachstudentbeganparticipatinginthee-Portfolio.This initially seemed tosatisfy theparticipants incycleone,until thesurveillanceaspectbecameanissuewithoneoftheparticipants,asdescribedintheanalysisofcycleone.Participantsincycletwodidnotraisetheseissues.

78

Anyexamplesorresultsfromthisprojecthavebeencarefullyselectedtoensurethat the identity of the participants is protected. Aliases chosen for the workpresentedhereare random,although theydo reflect thegenderof theoriginalparticipant. Material in this work has been presented in such a way that noidentifyinginformationisavailable,whichhasrequiredsomeslightalterationinartifact images where there may have been identifying information, such asparticipant names inside screenshots or embedded in comments. Participantsagreed to the inclusion of artifacts created during this process in the ethicsagreement.EthicalprocesseshavebeenapprovedbybothLancasterUniversityandKingstonUniversity.

79

Chapter4 Cycleone

4.1IntroductiontothefirstcycleThemodulechosenforthefirstactionresearchcyclerunsinthefirstsemesterofanacademicyear,whereassessmentelementshavetobecompletedbeforethesecond semester starts. Students are expected to attend for an 11week term,withlaterassessmentsallowedtorunpasttheendofthetaughtperiodforfourweeks. The second year undergraduate students taking this module have allworkedtogetherinpreviouscoursesandinthefirstfiveweeksofthemodule.Theundergraduatesinthisstudyaretypically20-21,andwillhavearangeofITknowledgeandskillsasaresultoftheirentryqualificationsandthedifferingfirstyearITmodulesavailable.Someofthestudentsmayhavecodingexperience;inthis cohort three of the students have previously studied a differentprogramminglanguagetotheoneusedhere.Roughlyhalfofthestudentstakingthemodulehavetotakethesubjectasacorerequirementfortheirdegree,withtheremainingopting intothesubjectasasecondyearchoice.Theretendtobevariablelevelsofattendanceforundergraduatelevelfivestudents,whichhasinthepasthadnegativeconsequencesforthestudentoutcomes.Following student feedback fromprevious cohorts, the delivery of thismodulehasrecentlychangedtotwo2-hourlabsusingdemonstrations,guidedtutorials,examplesandexercises.Studentsareexpectedtoattendbothsessions,whichforthisacademicyeararetimetabledonthesameday,earlyinthemorningandthenlater in the afternoon. For the fiveweeksbefore the cyclehadbegun, studentshad covered subjects common to these modules such as an introduction toprogramming, thebasicsofusingvariables, formprocessing, control structuresandwaystousefunctionswitharrays.Assessmentinthefirstpartofthemoduleconsistedofasinglereportcontainingexercises, tasks and activities that had been attempted during the first fiveweeks. Aswell as acting as a natural breakpoint for the introduction of the e-Portfolio, it also had the aim of introducing students to the practicalities ofcollecting evidence as they progressed. There were few opportunities forcollaborationinthisinitialassessment,althoughstudentsdidconfirmthattherewasinformaldiscussionbetweenthemaboutthecontent,suchasthenumberofitems,theirnatureandhowtheworkshouldbepresented.

80

Thecyclebeganinthesixthweekoftermandranforaneight-weekperiod,withthe last threeweeks occurring after the formal attendance period. Following abriefingsessiononthenatureoftheproject,studentswereallowedtoopt-intothe use of the e-Portfolio system. Those students wishing to sit outside theprojectwereaskedtocontinuegatheringandcollatingartifacts,whichwouldbepresentedinasimilarstyletotheworkfromthefirsthalfofthemodule.

From the29 studentson themodule,17 students initially chose toparticipate, although two completedthe initial sign-up and then did notcontribute in the e-Portfolio or in thedata collection processes. Nine malestudentsandsixfemalestudentschosetoparticipate.Thegradesawardedforwork in the first part of the modulesuggested that the volunteers werethose with a higher level of ability

(table4.1);amorereflectivesampleof theclassasawholewouldrequire twomorestudentsinthe50-59%range.

4.2Designsofthelearningcommunityande-PortfolioAsdiscussedintheresearchdesignchapter,amodifiedversionofDeLaat’smultimethodresearchframeworkforstudyingnetworkedlearningprocesses isusedhere, applied to the study of e-Portfolio artifacts in a community. Both theartifactsand theactivityarecollected through theuseofanetworked learningbasedcommunitywithabespokee-Portfoliodesigned to facilitate the creationandcollationofartifacts.

Thedesignofthee-PortfoliolearningcommunityA key characteristic of more successful learning communities is the carefulconsideration of the term community along with a design that supports thepedagogy,makingexplicittheembodiedvalues(McConnell,2006).Asindicatedintheliteraturereview,thefeaturesthatarecommonlyusedtocategorisethemvary,buttypicallyinclude:

• characteristicsoftheparticipants;• intentionality,orpurposeofthelearningcommunity;• typeorstrengthofconnection;

Previous

gradesNoofstudents

inclassNoofstudents

inthestudy

70+ 8 5

60-69 13 8

50-59 6 1

40-49 2 1Table4.1-Gradeprofileforparticipants

81

• thedomain,andlocationofthelearningcommunity;and• the degree to which the learning community can transform over time,

adoptingnewcustomsorpractices.Wenger’sthirdcommunityofpracticemodelsuggeststhat“mappingthedomainandspecifyingtheirconceptandscopeisanart”(Wenger,2002,p.70).Herethedomainiswellspecifiedandthecommunityforcycleonewillbeformedfromasubset of the availablemodule cohort. Practice and the shared knowledge arealsowelldefinedbythecurriculum,butlettingparticipantsformtheirownpathsallowsforflexibleboundaries.Thenetworkedlearningphilosophyentailspurposivelypromotinglinksbetweenthe learners and the artifacts inside the community. Here the artifacts will becreated by the participants themselves andwill be in a variety of forms;workthey have created, links to resources they have used; discussions of their ownandothers’artifactsinascrapbookstyle.Networkedlearningacknowledgesthestrengthinbothweakandstrongties(Ryberg&Larsen,2008),althoughthiswillnotbeexplicitamongsttheparticipantsor intheportfoliodesign.The learningcommunity will be peer basedwith an implicit equality ideologywhere everyparticipant, includingthetutor,hasequal ‘weight’,whichmovesbeyondsimpleportfoliomodelsthatstratifyusers,suchasthosesuggestedbyLove’smaturationtable(2004).Curated artifacts and information about participationwill be collected in an e-Portfolioandwillformtheevidencefortheassessmentattheendoftheprocess.Asthelearningcommunityissituatedinanactionresearchcycle,feedbackfromtheparticipantswillbeusedtotransformthecommunityovertime,bothduringandafterthecycle.

Thedesignofthee-PortfolioThereviewoftheexistingportfolioimplementationssuggestedthatnoneoftheexisting architectures would allow for the collation, sharing, discussion andpromotionofartifacts inanetworked learningstyle.Thedecisionwastakentoimplementanewdesignwithallnewcode,withtheadvantagethatthesystemcould then be rapidly changed during and between the cycles, reflecting thefeedback and successful processes that occurred.Writing this turnedout tobenon-trivial and took fivemonths. The softwarewas created using open sourcetechnologies(PHP,MySQL),withaninternetfacinginterface,whichstudentscan

82

access through anyweb browser ormobile device. The choice of technologiesused here aligns with those that the students will be learning in the cycle,attemptingtoensurethepossibilityoftheirparticipationinthedevelopmentandfor the code to be used in classroom examples and activities. The name‘foliocube’waschosenforthedomainandprojectname.Thee-Portfolioliteratureemphasisestheimportanceofattractiveandrelativelysimple interface designs that require little or no training (Jafari, 2004). Thisphilosophy underpins the design and functionality of the e-Portfolio, so noknowledge of mark-up languages or requirements for users to design theoverarching context for artifacts is required. Through its structure, thearchitecture of an e-Portfolio signalswhat is important (Yancey, 2009b). Herethe primary activity is the representation and collation of artifacts, a visiblerepresentationof the learningtakingplace.Thesystemdoesnot includeanyofthepersonalisationfeaturesseenonsocialnetworkingsites.The core of the application is the representationof an artifact,whichhere, forimplementation simplicity are images, consisting of screenshots of usergenerated content, snapshots of web pages, eBooks, links or other forms ofinformationthattheparticipantshaveusedandwishtoshareinthecommunity.The name of the artifact creator, date of creation and date of last update areautomaticallyattachedtotheartifact.Eachartifactisaccompaniedbyanumberofattributestoenhancetheirusetoothers:

• areflectivecommentary;• adiscussionthread;• categorisationusingfolksonomytags,taxonomy;and• aURL,forexternalresources.

Anonlinerepresentationofanartifactisshowninfigure4.1.

83

Figure4.1-Anexampleartifactrepresentation

Figure4.2-Exampleoverviewofnineartifacts

84

Whennavigatingthroughmanyartifactstheportfoliosystemusesanoverview,withninesmaller views of the artifacts arranged in athree by three grid (figure 4.2). Thesesmallerviews(figure4.3)canbeorderedbycreation date, update date, number ofcomments or bynumberof views.Artifactscanalsobeviewedinthumbnailspresentedthroughataxonomygrid.Artifacts, commentary and participation‘inside’ the portfolio can only be seen byauthenticated users and are protected byusername and password. All artifacts andcomments are visible to all participants,which was highlighted in the ethic statement and made clear before sign-upprocess. The principal reason for this is to promote equality and restrict thepossibilitiesofstratificationanddependence(Wenger,2002),whichmayhappenifparticipantselect toshareworkamongstsubsetsofusers,whichwouldhaveconsequencesonthestructureofthecommunityasawhole.Promotingtheconnectionsbetween learnersandartifactswillbeperformed inthreeways:

• bythetutor,• byembeddedco-operationandcollaborationactivities thataredesigned

fortheparticipantsbythetutor,and• inthedesignoftheportfolioitself.

Connectionsbetweenusersandartifactswillbeembedded inthe interface inanumberofways.Whenaparticipant’snameisshowninanyartifact,commentorsummary page, it is a clickable link that will take the viewer to the artifactsassociatedwith thatparticipant. Clickingon tagsor categories reveals artifactsthathavebeensimilarlytagged,eitherfromthesameuserorfromthecontextofthesub-selectionatthattime.There have been attempts at developing classification systems for thedescriptors ofmultimedia objects used in a community of practice, where the

Figure4.3-Exampleartifact

85

sharable representation was aimed at creating discussion and reflection(Steeples&Goodyear,1999).Despitethelackofcorrelationshowninthepilot,itwasdecided to see if assigningartifacts todomains ina taxonomymaystillbeusefulwhen createdover time in a learning community.To support theuseofthissystem,thethumbnailoverviewpageusesatilemetaphor(figure4.4),withsmall summary pictures of artifacts in piles pegged to each domain. These areclickableandthenrevealfullerdetailsoftheartifactsusingeithertheoverviewpages,orthefullartifactdetails.The landing page (dashboard) is the first place shown upon login, and asexplainedinthecyclesummaryanddiscussion,changesmanytimesduringtheprocess because of its importance as a signal of the amount of activity that istakingplaceandasawaytopromoteconnections.Thisisalsothepagethatusersreturn to after using other features on the site, so its use is a signal of overallactivity.Duringtheresearchcyclesthee-Portfoliocodewillbesharedandusedwiththeparticipants, initiallyas inclassexamplesand laterasactualsharedcode.Aftertheprojectcompletesthee-Portfoliowillbeplacedunderanopensourcelicense,anapproachtakenwithmanyopenprojects,toensurethatenoughoftheprojectworks before releasing it into the public domain. There are overheads tomanaging a fully public project, which are too costly to attempt whilstsimultaneouslymanagingthiswork.

Figure4.4-Examplethumbnailgrid

86

Becauseofthecomplexityinherentinusingdifferentmethodsofdatacollection,theuseof eachmethod, themeans, analysisperiod and sample is explained indetailinthenextsection.Method,sampleandanalysisThereareanumberofdatacollectionopportunities,detailedintable4.2.Table4.2-Methods,meansandsample

Description Method Means Date Sample

Activityinthe

portfolio

SNASocialNetwork

Analysisover

activityrecords

Analysisusing

Excel

Duringthe

eightweek

cycle

Derivedfrom

allactivity

shownin

activitytables

Artifactanalysis CAThematic

analysisover

artifactsand

discussions

In-vivocoding

inExcel

During/post

theeight

weekcycle

Allartifactsin

thee-

Portfolio

Reflectionsonthe

cycle

CxAFinal

questionnaire

Online Atendof

cycle

11students

Reflectionsinthe

cycle

Weeklyinclass

feedback/logbook

Collatedin

Word

Weeklyfor

theeight

weeks

SocialnetworkandactivityanalysisAsparticipantsinteractwiththesystem,information about their activities isrecordedinanactivitytableusingacodeindicating the possible actions, such ascreating an artifact, commenting,viewing others’ work or visiting thedashboard(figure4.5).Theactivitytablerecordsdateandtime;

the owner of the artifact that is in use;categorisation tags; and artifactidentifiers(figure4.6).Informationfromthistableformsthecoreofthenetworkand activity analysis used both during the learning community and in thesubsequentanalysisafterthecyclecompleted.

Figure4.5-Assignedactivitycodes

87

Figure4.6-Snapshotofactivityrecordedintheactivitytable

The architecture of the e-Portfolio allows for the activity data to be easilyprocessedwithcustomiseddatabasequeriesintheprogramminglanguageSQL,exportedandthenpostprocessedinExcelandthedataanalysissoftwareR.Theartifacts,dataanalyticsandfeedbackfromtheparticipantswereusedbothinsidethecycleinthetutorroleandinthereflectivephase.Threederivedtableswereusedduringthecycle:

• ananalysisof activitiesbythecommunityasawhole,

• ananalysisof activitiesbyindividuals,and

• a grid showing theconnections betweenparticipants.

The whole class activity gridrecords e-Portfolio activities, alongwiththenumberoftimeseachactivitywas performed each week. The weekone grid (figure 4.7) reveals that thecommunity created 16 artifacts,participated in nine comment threadsandviewedothers’artifacts106times.Theindividualactivitygridbreaksthisdownforeachindividualparticipantby

Figure4.7-Activityforweekonebycategory

Figure4.8–Exampleindividualactivity

88

week.For theexampleshown(figure4.8) thisparticipantdidnot interactwithothers’workinweekone,butlookedatanoverviewofothers’artifactstentimesandindividualartifactsfivetimesinweektwo.The connections grid shows the connections between participants, with eachnumber indicating the number of times a participant interacts with anotherparticipantforthatweek–ahighernumberindicatesmorefrequentinteractionafterthebinarymatrixofDaniel,McCallaandSchweir(2008).This is triggeredby:

• viewinganother’swork,• commentingorreplyingtoacommentonanother’swork,• replyingtosomeone’scommentontheirownartifact,or• searchingforanindividual.

In theexamplegrid (figure4.9),Derek (DWa) interactedwithElizabeth’swork(EBl) 13 times,whilst Elizabeth did not reciprocate, interactingwith hisworkzerotimes.Arowofnullvaluesindicatesnointeractionfortheweek.Social network analysis statistical measures and the sociogram diagrams aregenerated from the activity tables. Results for the different SNAmeasures areshowninsection4.4andtheappropriatenessofeachmeasureisdiscussedinthepostdiscussionreflection.

Contentanalysisfromartifacts,commentsandnotesTheartifactscreatedduringtheprocesswereanalysedbycreationdateandthenbyuserinsidethee-Portfoliosystemitself.Reflectivestatementsandcommentswereprocessedinsideaspreadsheet.

Figure4.9-Interactionbetweenparticipantsinaweek

89

Eachartifactwasprocessedaccordingtothefollowingcriteria:• thenatureoftheimage,collationorcuration;• referenceto‘outside’resources;and• the subject of the artifact – what subject area, the level of reflective

comment,whetherarequestforhelp.Further information for the content analysis was derived from in class notestakenduringthecycle.

ContextanalysisfromquestionnairesandlogsThe context analysis (CxA) is generated from a questionnaire and notes takenduring the process. 11 of the 13 active participants responded to thequestionnaire,whichcontainedamixofopenandclosedquestions(table4.3).Table4.3-Questionsusedforcontextanalysis

Question Type Noofresponses

Howdidyoufindusingtheportfolio? Open 10

Howdidyoudecidewhatshouldbeanartifact?

Open 8

Whatresourcesdoyouuseforlearningonthiscourse?

Choice 11

Intheportfoliosystem,howoftendidyoulookbackonyourownwork?

5pointLikertscale

11

Whydidyoulookbackatyourownwork? Open 9

Howoftendidyouuseothers’work? 5pointLikertscale

9

Howdoyouthinkusingtheportfoliochangedthegroup?

Open 7

Whydidyoulookatothers’work? Open 7

Howdidyoufindtaggingandusingthethumbnailsystem?

Open 10

Othercomments Open 4

90

The analysis of responses for the questionnaire was performed in Excel, withstatisticalorthematicanalysisasdescribedinchapterthree.Forthepurposesofdescribingtheactivityinthecycleanumberofassumptionshavebeentaken,whichenablethedivisionofthecycleintoanalyticalsections.Aweekhasbeendefinedtobetheperiodrunningfromthestartofthefirstsessionwhere new taughtmaterial was introduced, for the next seven days. Studentsagreedwith thisdefinition, asmanyof them indicated that therewasa sharedexpectation that content, exercises and discussion should take place betweensessions.In the following discussion of activities inside the cycle, the eight weeks havebeen divided into three, signifying the beginning,middle and end of the cycle,after divisions used by De Laat (2007; 2006a; 2006b). The first three weekscovertheintroductionofthee-Portfolioandthestepstakentoincreaseitsuse.The next twoweeks run from there to the pointwhere students leave for theChristmas break; the final three weeks occur with students collaborating offcampus.The following discussions leave out John andDavid,whoneither attendednorparticipated in the cycle after signing on at the beginning. Neither supplied aresponsetothequestionnaire.

4.3ThecycleThis sectionprovides anoverviewof the activityoverdistinctphases, dividingthe cycle into three. This provides the context for the specific data collectedduringthecycle,whichispresentedinthesubsequentsectionsandanalysedinchapterfive.

WeekonetoweekthreeThe induction session introduced the aims of the learning community, anoverviewofe-Portfolios,ademonstrationoftheplatformalongwithadiscussionon the nature of artifacts and how they are created and shared. Exercises,examplesandotherresourceswereshownaspossiblesourcesforartifacts.For the first threeweeks of portfolio use, a combination of programming anddatabase techniquesweredemonstrated in class,with accompanying examplesshowing how to set up a connection between a web and database server.Examples and exercises asked students to create their own databases, write

91

queries and to display data extracted from the connection in a web page.Subsequent classes covered building a larger web application, using databaseandprogrammingtechniquesfromtheearliersessions.Each student’s first artifact wasacknowledged with a comment toencourage further participation.Artifacts with missing reflectivestatements, tags or taxonomyplacement were commented toreinforce the advantages in usingthese in the process. There are anumberoffactorsthatinfluencehowlong a facilitator should wait beforeresponding to an unanswered request for help. Answering too quickly candiscourage others from participating (Garrison & Cleveland-Innes, 2005;Vonderwell, 2003) leaving questions unanswered for too long could reduceconfidenceintheabilityofthecommunitytosupplyanswers,particularlywhenthevisibleparticipationlevelislower.DuetolowlevelsofinitialparticipationIrepliedwithin24-36hoursforthefirsttwoweeks(figure4.10).In the first twoweeks, eight of the participants had explored the system,withonlyfourcreatingartifacts.Despitegeneralencouragementinclass,participationlevels in theportfolioonly increaseda little fromweekone to two,withmanystudentsyettocreateanartifactortoparticipateintheonlinecommunity.AttheendofthesecondweekIdecidedtoexplorethereasonswhythiswasoccurring.InterventionParticipantfeedbackattheendofweektwosuggestedthatmanyofthestudentshad not explored beyond the front page, which only contained a welcomemessage and no immediate visual indicators of activity. This had led them toconclude that little portfolio use was taking place. A demonstration in classconfirmedthis.Six of the students agreed to attenda small focus group,which suggested fourreasonsforlowparticipation:

Figure4.10-Exampleofinitialtutorfeedback

92

• Confirmation that therewas a lack of awareness of the level of activityoccurringintheonlinecommunity.

• Awarinessofthedifferent‘new’approach.• Ananxietyoversharingofartifacts.• Ahesitationduetothe lackofclarityoverthenatureandshapeofwhat

participantsshouldupload.Feedback from the sessiondivided into two categories,with immediate designchanges for the e-Portfolio system that couldmore clearly signal the levels ofactivity occurring and suggestions for improving the initial induction,which isdiscussedinthepostcyclereflection.A new front design was brainstormed with the group, who suggested theinclusionofthefollowingelements:

• Snapshotsofthelastfiveartifactscreated.• Alistoftheloggedinparticipant’srecentactivityandcomments.• Aneasierwaytoseerecentartifacts.

Thedesignsforthenewfrontpagewereimplementedforthesubsequentclass(figure4.11).

Figure4.11-Redesignedfrontpage

93

During the discussion, it became apparent that the activity table could serve adualpurpose.Originallydesignedtogeneratesocialnetworkanalysisdiagramsand tables for this research, the activity table tracks every interaction andactivity that each participant performs as theymove through the system. Therealisation that it could also be used to reflect recent activity back to theparticipants on the dashboard was significant. It also forms the basis of therecommendationsystemdevelopedincycletwo.Theanalyticliteraturereferstothese as process traces, generated from log files, used to reveal recent activity(Suthers&Rosen,2011).At the startof the thirdweek, the frontpageand someof theartifacts createdwere demonstrated to the whole group, along with a re-emphasis of the peernatureofthecommunity.Followingthesechangesanddemonstration,therewasan increase in activity for week three with double the number of createdartifacts,andmanymoreinteractionsinsidethee-Portfolio.Participantswhowere first using the system fromweeks three onwardswererespondedtoinasimilarfashiontothoseinthefirstweeks.Iftheinitialartifactsfrom a user lacked reflective statements and tags, a note highlighting theadvantagesoftheirusetendedtoencouragestudentsinusingthemsoonafter.

WeekfourandfiveWeek fourand fivewere the last twowith specific timetabledclasses. Subjectscoveredincludewritingauthenticationpageswithscriptstomakeasitesecureandanintroductiontoobjectorientedprogramming.Bythisweek,studentshadcoveredmany of the techniques used in the creation of the portfolio itself, socodefromtheauthenticationsystemandartifactobjectswereusedinclassandforexercises.Oneoftheparticipants,David,hadnotloggedintothesystematallandhadnotattendedclasssincethefirstweek,soanemailwassentaskingforinformation,andhis casewaspassed into theUniversity tutor systemwhichofferspastoralsupport.As participants started commenting onwork from fourweeks earlier, studentfeedback in class suggested it became unfeasible to check all old artifacts forcomments. A more visible indicator was scripted for weeks six to eight,highlightingpopularcommentthreadsandartifactsonthedashboard.

94

Weekfivewasthefinalweekofattendanceandalsofellonthefinaldayofterm.Attendance in classwas low, asmanyof theparticipantshadalready returnedhome for their Christmas break. There was a significant amount of browsingaroundthesystembytheparticipants,withallofthemlogginginandlookingatexistingwork.Feedbackfromthosepresentinclasssuggestedthattheywantedtomakesure that theycouldaccess thee-Portfoliobefore they left thecampus.Despitehigherlevelsofinteractioninthesystem,withquestions,commentsandviews,onlyfourparticipantscreatednewartifacts.

WeeksixtoweekeightWeeksixspannedtheChristmasholidayweekandtherewasadrop inactivitywithasinglestudentcreatingartifacts,withviewing,commentingandsearchingonartifactsbyeightothers.Theoveralllevelofusewassignificantlylowerthaninearlierweeks.Weeksevenwasthepenultimateweekofthecycleandseestheparticipantsfallintothreecategories:

• Three participants continuing their regular patterns of activity andcreatingartifactsfrommaterialfromweekfourandfive.

• Three participants with irregular or little activity, starting to createartifacts.

• Participants interactingviacomments,orexploringothers’workbutnotcreatinganyartifacts.

Forthefinalweekparticipantsfallintoslightlydifferentcategories:

• Four students who continue participating but don’t create any newartifacts.

• Four participants who had irregular or little activity so far, starting tocreateartifacts.

• Twostudentscreatingcoveringmaterialfrommanydifferentweekswithlittleinteractionwiththeothers.

• Twostudentswhocontinuedwithlittleornoparticipation.

4.4ResultsThis section summarises the results from the first cycle, in three sections. Thefirstdetailsinformationgeneratedfromtheactivitytablesandthesocialnetworkanalysis.Thesecondconcerns thecontentanalysisover theartifactdetailsand

95

the third comes from a questionnaire. These are then discussed in the nextchapter.

ActivityinthecycleThe summary activity table (table4.6) is generated from the e-Portfoliowhichrecordstheactionseveryparticipantperformsinsidethesystem.Activityinthee-Portfoliohasbeengroupedintosixcategories:

• viewinghelppages,• creatingorviewingone’sownwork,• interactingwithothers’work,• viewingthumbnailsthroughthetaxonomygrid,• usingthedashboard,and• participatingincomments.

The“viewdashboard/recentactivity”numberisageneralindicatorofuse,asitisthe homepage that is shown after every requested activity is completed.Numbersinthetableindicatehowmanytimestheseeventsoccurineachweekforallusersofthesystem,forexampleinweekone:

• therewere16artifactscreated,and• participantslookedatothers’work106times.

Thelowlevelofactivityinthefirsttwoweeksissuggestedbyboththenumberofartifactscreated(16inweekone,18inweektwo),andbythelevelofinteractionwithothers’work (323and301 inweeksone and two respectively).After theinterventionandredesignofthehomepagetosignalactivitytakingplace, thesenumbersincreaseforweekthreewithdoublethenumberofartifactscreatedandthree times the level of interactionwith others’ work compared to week two.After this surge in activity, the number of artifacts created per week remainssteady (36, 33, 46 in weeks three, four and five), whilst the interaction withothers’ work settles at 460-500 times a week. The lull in activity over theChristmasweek(weeksix),isfollowedbyasurgeinactivityinweekssevenandeight,withalargeincreaseinbothartifactcreation,andinteractionwithothers’work.Participantsuploadedatotalof381artifactsovertheeightweekperiod,givingthefollowingstatistics-onaverage,aparticipantcreatedoneartifacteverytwo

96

days, looked at their ownwork three times a day and looked at others’ workseventimesaday(table4.4).Table4.4-Ownandothers’useofartifacts

Table4.5-Generalgraphmeasures

Thereareanumberofgeneralgraphmeasuresthatcanbecalculatedoveralltheactivityinthecommunity,onaweekbyweekbasis(table4.5).Numberofedges isameasureofactivitybetweenparticipants,wheremultipleinteractions between two nodes count as a single edge. A higher number isbetter,indicatingsuccessinpromotingconnectionsbetweenparticipants.Asthecycle progresses the number of connections steadily increases, apart fromChristmasweek(weeksix).Islandsaredisconnectedclusters,which inacommunitysetting isbetterwhensmalleraslargervaluessuggestnon-interactionorcliques.Duringthecycle,thisnumber steadily decreases, although the activity tables and sociograms revealthatsomeindividualsincycleonemovedinandoutofparticipationrepeatedly.Graphdensityindicatestheproportionofconnectionsandiscalculatedfromtheratioofthenumberofedgestothetotalnumberpossible.Highervaluessuggestmore connections between nodes, with zero meaning no connections and onesuggesting all nodes are connected to each other. The density has peaks and

97

troughs, with peaks corresponding to a surge in activity after week three,towardstheendofformalattendanceandattheendofthecycle.Reciprocitydefinestheproportionofmutualconnections,inadirectedgraph.Itisdefinedas theprobability that theoppositecounterpartofadirectededge isincluded and has values between zero and one. Zero indicates no mutualconnections whilst one indicates a balance between in and out connections.Highernumbersarebetter,indicatingmorebalancebetweenusingothers’workandaparticipant’sworkbeingused.Thereisapeakinweekthree,whichdropsawayuntilweeksevenwhereitstartstoincrease.

Table4.6-Activitysummaryfortheeightweeksofcycleone

99

Fordegree,betweennessandeigenvectorthevaluesinthetop20%(green)andbottom20%(red)havebeenhighlighted(table4.7,4.9and4.10).Table4.7-Degreevaluesforparticipants

Degreeisthenumberofin/outlinksfromavertex(table4.7).Highernumbersarebetterastheysuggestmoreconnectionsareoccurring.Thismeasuredoesnotcountrepeatlinks(i.e.multipleverticestothesamenodecountasone).Figuresinredareinthelowest20%,figuresingreenareinthehighest20%onaweek-by-weekbasis.Thetrendoversubsequentweeksisforthedegreetoincrease.Table4.8-Degreedistributionforparticipants

Degreedistributionsuggeststhenumberofeachlinksasaproportionofthetotal.Forweekone56%oftheparticipantshadnoconnections,6%had1,2and3links(table4.8).Highernumbersarebetter,indicatingmoreconnections.Asthecycleprogressestherearepeakconnectionsinweeks3,5and8.

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20Week.1 56% 6% 6% 6% 6% 13% 6%Week.2 44% 13% 13% 6% 6% 6% 6% 6%Week.3 31% 6% 6% 6% 6% 6% 6% 6% 13% 6% 6%Week.4 44% 6% 6% 13% 13% 6% 13%Week.5 13% 6% 6% 6% 6% 6% 6% 13% 6% 6% 6% 6% 13%Week.6 38% 19% 13% 13% 13% 6%Week.7 6% 13% 13% 6% 6% 13% 19% 6% 6% 6% 6%Week.8 6% 6% 13% 6% 6% 13% 6% 6% 6% 6% 6% 6% 6% 6%

Number.of.links

100

Table4.9-Betweennessfiguresforparticipants

Betweennesscentralitygivesahigherscoretoanodethatsitsontheshortestpathofothernodepairsandsuggeststhosethatare often found at the intersections ofmore densely connected network communities. A higher value suggests a participantperformingabrokeringroleacrossclusters,connectingotherwisedisconnectedpeople.Figuresinredarethelowest20%,thoseingreenareinthehighest20%,onaweekbyweekbasis(table4.9).Eileen(EWa)hasaconsistentlyhighscore,butthereareweekswhereothershavehighervalues–Waris(WHu)inweeksthreeandfour,Douglas(DCa)inweekfiveandRobert/Derek(RSl/DWa)inweekeight.Table4.10-Eigenvectorcentralityforparticipants

Eigenvectorcentralitygivesahigherscoretoanodeifitconnectstomanyhighscorenodesandsuggestshigherinfluencerswhodisseminateinformationquickly.Theydonotalwayshavethegreatestlocalinfluenceandmayhavelimitedbrokeringpotentiali.e.alowerbetweennessvalue.RSlhasaconsistentlyhighvalueandthereareotherswithoccasionalhighssuchasDouglas(DCa)inweekfive/six,Edith(EBl)inweekfourandNorman(NLo)inweekseven(table4.10).

SNAdiagramsThesociogramsforthefirstfourweeksshowasmallcoregroupofparticipantsinteracting (figure4.12),withpeakactivity inweek three after the redesignofthee-Portfoliofrontpage.Weekone Weektwo

Weekthree Weekfour

Figure4.12-Sociogramsforweeksonetofour

WSA Week 1

BAv

BNe

BSa

DCa

DWa

DWh

EBl

EWaGSh

HLa

JHo

MHe

NLoRSl

TLe

WHu

WSA Week 2

BAv

BNe

BSa

DCa

DWa

DWh

EBl

EWa

GSh

HLa

JHo

MHe

NLoRSl

TLe

WHu

WSA Week 3

BAv

BNe

BSa

DCa

DWa

DWh

EBl EWa

GSh

HLa

JHo

MHe

NLo

RSl

TLe

WHu

WSA Week 4

BAv

BNe

BSa

DCa DWa

DWh

EBl

EWa

GSh

HLa

JHo

MHe

NLo

RSl

TLe

WHu

102

Weekfive Weeksix

Weekseven Weekeight

Figure4.13-Sociogramsforweeksfivetoeight

Thesociogramsforweeksfivetoeightshowfluctuatingpatternsofactivity,withalullinweeksix,andhigherlevelsofactivitytowardstheendofthecycle(figure4.13).

WSA Week 5

BAv

BNeBSa

DCa

DWa

DWh

EBl

EWa

GSh

HLa

JHo

MHe

NLo

RSl

TLe

WHu

WSA Week 6

BAv

BNe

BSa

DCa

DWa

DWh EBl

EWa

GSh

HLaJHo

MHe

NLo

RSl

TLe

WHu

WSA Week 7

BAv

BNe

BSa

DCa

DWa

DWh

EBl

EWa

GSh

HLa

JHo

MHe

NLo

RSl

TLe

WHu

WSA Week 8

BAv

BNe

BSa

DCa

DWa

DWh

EBl

EWa

GShHLa

JHo

MHeNLoRSl

TLe

WHu

103

AnalysisofartifactsandcommentsOnlyfourartifactscouldbeclassifiedascuration,containinglinkstoonlinePHPtutorialsitesorYouTubevideos.Thesewerecreatedbythosewithmoreregularparticipation.Table4.11-Artifactandcommentcreation

Participantswholeftartifactcreationuntilweekssevenandeighttendedtousetagswhichlackedasemanticcontext,butinsteadindicatedtheweekorexercisenumber to suggest progression and completion. Participants naturally addedsupportingcommentstotheirartifacts,butdidnotinitiallytagorpositioninthetaxonomy.Nudgestylecommentsinweektwoandthreestartedtheuseoftags,butfailedtoincreasetheuseofthetaxonomy.WeeksonetothreeSix students used the system in thefirst week, with an even dividebetween those uploading work andthoseexploringthefeaturesavailable.Waris,ElizabethandRobertuploaded

16artifactsintotalandlookedatan overview of their own workapproximately 70 times each.They reviewed each individualartifact roughly three times.Waris’ first artifact was arequestforhelp,whichheinfactsolved almost immediately, posting a solutionwithout assistance (figure 4.14).DuringthefirstweekWarisdidnotlookatanyoneelse’swork.Robert and Elizabeth’s uploads were demonstrations of completed work andadvice forothers (figure4.15),and thesepromptedquestions frombothDerekandElaineon locationsofsamplecodeandsuggestionsofapproaches.Noneof

Figure4.15–Elizabeth’sartifacts

Figure4.14–Waris’artifact

104

the students used the folksonomy or taxonomy system, but they did addreflectivetexttotheirartifacts.The second week’s class covered building a larger web application, usingdatabaseandprogrammingtechniquesfromthepreviousweek.Despitegeneralencouragement in class, participation levels in the portfolio only increased alittle,witheightof thestudents interacting.Threeof theseparticipantscreatedartifacts.

Each student’s first artifact was acknowledged with a tutor comment toencouragefurtherparticipation.Artifactswithmissingreflectivestatements,tagsor taxonomy were commented to reinforce the advantages of these to theprocess.Waris’ artifactshowshimusingthetaggingsystem.Eileenusestags,butleavesoutthereflectivestatements(figure4.16).At the startof the thirdweek, the frontpageand someof theartifacts createdwere demonstrated to the whole group, along with a re-emphasis of the peernature of the community. Following the changes and another demonstration,therewasanincreaseinactivityforweekthree.The number of artifact uploaded in the week doubled, with many moreinteractions insidetheportfolio.Therewasasignificantsurge inexplorationofthe artifacts with three times asmany searches and views compared to weektwo.Fiveof theparticipantshadstillnotusedthesystem,but theothershadagreaterlevelofparticipation,withfiveparticipantscreatingartifactsandanotherfiveexploring,viewingorcommenting.

Figure4.16-EileenandWaris’useoftags Figure4.17–Robert’srequestforhelp

105

Robert was the most prolific contributor creating ten artifacts over the threeweeks. He, Derek and Elizabeth all demonstrated their willingness to sharemistakes and to ask and answer questions (figure 4.17). Of the 70 artifactsposted,49wereofvalidcode,with21showingcodewitherrorsandrequestsforhelp. Elizabeth and Robert uploaded and interacted with others’ work, to agreater extent than Derek, Eileen and Norman, where interaction indicatesviewinganartifact,commentingorusinganoverviewofothers’artifactsthroughanyoftheavailablesearchmechanisms.Maureen,DouglasandNormandidnotcreateanyartifacts,butstartedexploring thesystemduringthethirdweek,eitherthroughtheoverviewmechanismorbysearching (table 4.12). They viewed specific artifacts many times and addedcommentstoothers’artifactswithquestionsaboutthecodedemonstrated.BerylandWarisexploredlittle,butdidviewtheoccasionalartifact.Table4.12-Activityforstudentsnotcreatingartifactsinweeksonetothree

Despite participants being allowed to create artifacts on any relevant subject,nearlyalltheartifactscreatedwerealignedtoexamples,tasksoractivitiesfromclass,indicatinglittlecurationandsuggestinglittleuseof‘outside’ resources.WeekfourandfiveWeek fourand fivewere the last twowith specific timetabledclasses. Subjectscoveredincludewritingauthenticationpageswithscriptstomakeasitesecureandanintroductiontoobjectorientedprogramming.Fiveoftheparticipants(John,Tony,Barry,GeoffandDavid)continuedtheirlowlevelofonlineparticipation.Allof thesestudentshadattended the face-to-facesessionsandwereworkinginclass,butdespiteencouragementdidnotturnanyoftheirworkintoartifactstoshare.Elisabeth,Waris,Robert,DouglasandDerekuploaded33artifacts,withWarisandDouglasrespondingtoencouragementandtheirownexplorationoftheexistingartifactsinweekthree.

106

Table4.13–Catchupartifactscreatedinweekfour

For the33artifactsuploaded inweekfour,onlyeightwereonthematerialsdiscussed in class that week. Ananalysis of the artifacts producedduring this week indicated many of

themwereonprevioustopics,withonestudentattemptingworkfromthefirstweek (table 4.13). All the students were creating artifacts on material frommultipleweeks,indicatingacleardragexistingbetweentheintroductionofnewconceptsandthetimeittakesforstudentstoworkwithitoutsidetheclassroom.Weekfivewasthefinalweekofattendance.Despitehigherlevelsof interactionin the system, with questions, comments and views, only four participantscreatednewartifacts(Tony,Maureen,Howard,Douglas),withTonyandDouglashavingsignificantuploads.Douglascontinuedworkinghiswaythroughthesubjects,uploading17artifactsonsubjects fromtheprevious threeweeks.Tonyalsocreated19artifactsafternotparticipatingatalluptothispoint.Because of the lag in created work, many of the students who had notparticipatedinthefirstthreeweeksstartedtocommentonolderartifacts(figure4.18).

107

Bythisstage,theparticipantshadprogressedintheclassmaterialtoseethatthetechniquesusedinthelabswerethesameasthatusedinthedevelopmentoftheportfolio itself. Students responded to seeing the portfolio code, with twostudentsusingsamplesofthecodeinartifactsthemselves.WeeksixtoweekeightWeeksixspannedtheChristmasholiday,sotherewasadropinactivity.Douglaswas continuing his work from week five, creating four artifacts on materialcovered in the fourthweek,and therewasviewing, commentingandsearchingonartifactsbyeightoftheothers.Weeksevenwasthepenultimateweekoftheprojectandseesthestudentsfallintothreecategories:

• Participants continuing their regular patterns of activity and creatingartifactsfrommaterialfromclass,Waris,Robert,Howard.

• Participants who have had irregular or little activity so far, starting tocreateartifacts,Maureen,BerylandNorman.

Figure4.18-Aninteractionwitha“past” artifact,withcommentsaddedthreeweeksaftercreation

108

• Participants interactingviacomments,orexploringothers’workbutnotcreatinganyartifacts.

Waris, Howard and Robert created artifacts on material from week five,continuingtousetagsandreflectivestatements.Maureen, Beryl andNorman posted artifacts onmaterial fromweek one,withBeryl uploading pictures without accompanying text. The tags used on theirartifactsusedthefolksonomysystemtosuggesttheweektheypeggedtheworkto(week_xorweekx_tasky),ratherthanusingcontextspecifictags(figure4.19).Forthefinalweekparticipants’fallintothesefourcategories,whichareslightlydifferentfromweekseven:

• Students who participate but don’t create any new artifacts, indicatingthat they perceived that they had finished, Waris, Howard, Maureen,Elizabeth.

• Participants who have had irregular or little activity so far, starting tocreateartifacts coveringmaterial fromall fiveweeksTony,Barry,Eileen,Geoff.

• Studentswhoeitherhadparticipatedlittleornotatall,creatingartifactsonsimpleexamplesfromthefirstweek,Derek,John.

• Studentscreatingcatch-upartifactswithlittleinteractionwiththeothers,BerylandTony.

Beryluploaded19artifacts,continuingthestylesheadoptedinweekseven,withnotextandarestrictedsetoftags.Tonycreatedseven,whichlikeBeryl wereonmaterialfromweekthreetofive.Neitheroftheminteractedwithothers’workinthisperiod.

Figure4.19-Temporalbasedtags

109

CodingcommentsforteachingpresenceTheteacher’scommentsattachedtoartifactshavebeencodedusingtheteachingpresencecodingschemederivedfromthecommunityofinquirymodel.Asintheotheranalysis,thecyclehasbeendividedintothree(figure4.20).Each participant received comments aimed towards Instructional Design andorganisation and Facilitating discourse, as would be expected in the initialestablishing part of a community. Statements encouraging more meaningfulreflective statements and tags have been designated as utilising the mediumeffectively,asthereflectivestatementswouldbeusefulforboththeparticipantsandothers’reflection.Advisingabouttheuseofthesystemmoregenerallyandasking for blank fields to be filled has been placed under the more generalcategoryofnetiquette.Facilitatingdiscourseinthiscontextmostlyentailsdrawinginparticipantswithfurtherquestionsandthepromotionoflinksbetweenthestudents,encouragingthemtolookatandcommentonothers’artifacts.Fewercommentsfall intotheencouragingorreinforcingcategory.Weeksonetothree

Weeksfourandfive

Weekssixtoeight

Figure4.20-Codedteachercomments

110

QuestionnaireresponsesandobservationsfromthecycleResponses in this section are generated from the questionnaire sent out toparticipantsaftertheendofthecycle.Additionalinformationhasbeenincludedfromfieldnotestakeninclassandinthefocusgrouporganisedafterthesecondweek.Generalcommentsabouttheprocesssplitbetweenthoseparticipantssuggestingtheadvantagestheysawintheinteractionandthosewhowantedspecificdetailonthenatureoftheartifactsthattheyperceivedasbeingrequired:

“I could see what code needed improving and what code was nicelycoded”-Robert“Icouldgetquickerfeedback”-Geoff“Instantaneousfeedback,andinteractionwithotherstudents”-Norman“Allowed me to see where I went wrong and attain feedback andconstructivecriticism.Alsohelpingothers"-Waris

Thosethatwantedmoreclarityrequestedmoreinformationatthebeginningoftheprocess,specificallyemphasisingtheroleofthetutorindescribingthenatureof the artifacts to be created. Participants suggested that they took everysuggestedexerciseascompulsoryorusedtheartifactsofotherparticipantsasatodolist:

“Iworkedthroughtheexercisesweekbyweek”–Robert.“Ilookedattheothers’answersandcreatedsimilarsolutions”-ElizabethB.

Mostparticipant’sartifactswererepresentationsofcompletedwork.Thetriggerforpostingwasaself-determinedcompletionpoint:

“whenI’dfinishedoneofthetasksIpostedit” –Derek.Fewerstudentsfeltcomfortablewithpostingincompleteorincorrectwork:

111

“itneveroccurredtomethatIcouldpostworkthatwasn’tright”–Geoff.Participantsreportedalackofengagementwithresourcesoutsidetutorcreatedteachingmaterials.Twoofthestudentsindicatedthattheyusedbothvideosandmaterials fromotherweb sites,with seven reporting the use of only the tutorcreatedteachingmaterials(table4.14).Table4.14-Participantreportedresources

Theuseoftheoverviewmechanismiscommonafteragapinactivityandallowstheparticipantstoplacethemselvesatapointonapath:

“itenabledmetorememberwhatIhaddone” -Robert“… whereIhadgotto” -Elizabeth.

Participants reported lookingbackon theirownwork two to four timesaday,butlowerlevelsinacknowledgingtheuseofothers’work(table4.15).Table4.15-Participantreporteduseofownandothers’work

Manydifferentreasonsaregivenforbrowsingothers’work, fromdeducingthemechanicsofwhatshouldbeincludedthroughtotheenrichmentpossibilitiesofmotivation,conversationandlearning:

“bylookingat[others’]pictureIcouldseewhatneededtobeincluded“–Robert.“Allowed tocomparemywork toothers’differentcodewhichproducedthesameoutput”–Norman.

112

“I could see different problem solving approaches of different people” –Douglas.

“Itactedasamotivator,Icouldseeotherpeopledoingwork,soIthoughti'd better do the same :) Thatmeant I was spendingmore time on theassignmentthanIusuallywould,hencelearntmorestuff”–Derek.“Ilikedgettingandgivingcomments”–Maureen.

“readthecommentstoresolvetheissue”–Elizabeth.“I think it just changed how we look at each others’ work. Withoutfoliocubeweusuallyjustshoweachother,butwithfoliocubeyoucanlookinmoredetailandactually learn.Asopposedto justseeingifyourworklookssimilar”–Waris.

“Yesbyseeingotherpeopleswork,questionsaboutthecontentofscreenshots could be compared with everyone else so that I could upload asuitableone.”–Howard.

Reasons reported for the lack of use of the taxonomy ranged from anunwillingness to engagewith an unfamiliar terminology to a reluctance to usesomething which provided no immediate use to the participants, particularlywhen compared to the folksonomy system which they regarded as havingimmediatesearchingandcategorisingfacilities.Beryl and Eileen are participants with high attendance but low onlineparticipation, creating artifacts from theirwork at the end. Eileen has a socialnetworking account, but posts infrequently. Drawing parallels to her lack ofonline social presence and engagement in Facebook, she decided to use theprojectasifatraditionalportfolio,presentingherworkattheend.Berylismoreengagedwithsocialnetworking,but:

“alwaysleave[my]worktotheveryend”

113

She had mirrored her experience in submitting coursework in other classes,which she leaves until very close to the submission date. Both students rarelycommentedoraskedforcomments.Berylacknowledgedthis:

“IthinkifIwasdoingitagainIwouldtryandspacetheworkoutandinteractabitmore” “Ididn’tseethepointinworkingwithothers,butitwould’vemademylifeeasiernotleavingeverythinguntiltheend”

Eileendidnotseeanyneedtochangeherprocess:

“that’sthewayIwork”Neither student responded to tutor suggestions about creating artifacts on amoreregularbasisduringtheprocess.Derek(andtwoothers),suggestedthattheirsurgeinactivityattheendwasduetoacomparisonprocess, lookingforanoverallpictureofothers’worktoseeiftheirownwassimilar:

“Iwantedtoseeifmyworkwasliketheothers”.In an in class feedback session, Derek indicated that after the fourthweek, hestopped searching and interactingwith others’ artifacts as he realised that thesystem (and the tutor) could seewhat hewas doing could be tracked.Despitereassurances from the beginning of the process indicating that this interactionwasbeingpromoted,hedidnotchangehisstance.Whenaskedaboutthishesaidthathehadcalculatedthatnon-interactionwouldhavenodetrimentaleffects.Howard suggested that the focus group at the end of week two altered theparticipation levels by making the purpose of the process clearer, which is aproblemthatcouldbeaddressedinabetter inductionor introductoryphaseofthe next cycle. Eileen suggested using some actual examples in the inductionprocess.BothsheandDerekwouldallowtheirworktobeusedinthisway.This chapter has discussed cycle one; the design and implementation of the e-Portfolioandlearningcommunity;theresultsofcycleone;andananalysisofthe

114

questionnaire. The next chapter discusses these results, examines theappropriatenessof thedatamethodsappliedand reflectson the changes tobemadegoingforwardintocycletwo.

115

Chapter5 DiscussionandreflectiononcycleoneThis chapter discusses the first cycle in the context of the three researchquestionsandthenreflectsonthechangesthatcanbemadeforthenextcycle.

5.1Discussion

Research question 1: What assessment artifacts emerge from co-operatingparticipantsinalearningcommunity?Thecontentandnumberofartifactscreatedduringtheprocesswaslefttoeachparticipanttodetermine,animportantdifferentiatorinconstructivistportfolios(Barrett&Carney,2005;Paulson&Paulson,1994).Despitethewidevarietyofevidence that could have been used, the majority of artifacts were created inresponsetoexercisesoractivitiessuggestedbythetutor,withaperceptionthatthiswasworksetby the tutor,aviewreinforcedby theparticipants lookingatthenatureofothers’artifacts:

“once I saw that what the others were doing I started postingsolutionsaswell”–Douglas.

Given free choice of artifact content, there are examples of participantsattemptingalltheworkpossible(Masonetal.,2004).Similarlyhere,participantstookanyhintofataskoractivityassomethingthathadtobecompleted,whichwhen finished were posted. The participants extrapolated learning outcomesfrom the lecturer and teachingmaterials and thenproducedwork that alignedwiththeirperceptionsoftheseoutcomes,astudentinitiatedformofconstructivealignment(Biggs,1996).In the subject areas covered in thesemodules, the initial small low level tasksand activities frequently have a commonality to the solutions available. Themedium and larger activities that occurred after the third week allowed fordifferentiation in the nature of artifacts produced, although most were stillresponsestotheperceivednotionofworkthathadtobecompleted.Participantsresponded positively to the use of code from the authentication system andartifact objects in class and for exercises, which aimed to add authenticitythroughsuggestingresemblancetoprofessionalpractice(Gulikers,2006).Fewof theartifacts indicatedbrokencodeorwereasking forhelp,despite thisbeing suggested in the induction as a significant advantage of an e-Portfolio

116

community.Fewstudentsfeltcomfortablewithpostingincompleteor incorrectwork,orevenconsideredit:

“itneveroccurredtomethatIcouldpostworkthatwasn’tright”–Geoff.Thelowlevelofrequestsforhelpconnectswiththeemergentthemeofartifactsas completed work. Showing incomplete or incorrect work can be personallyrevealing (Carney, 2002) and the nature of artifacts is influenced by theperceivedaudienceofthework(Gibson,2006).Dereksuggestedthathisimpulseforartifactcreationwasthecompletionofataskoractivity“whenIfinishedoneofthetasksIpostedit”,aprocessthatwouldbypassanyopportunityforaskingforhelpfromthecommunity.There is a clear lack of engagement with resources outside the tutor createdteachingmaterials,signalledbyboththecontentanalysisofartifactsandbythequestionnaire responses. Only four artifacts could be classified as curation,containing links to online PHP tutorial sites or YouTube videos. These werecreated by those participating regularly online. Two of the students indicatedthattheyusedvideosandmaterials fromotherwebsites,withsevenreportingtheuseofonlythetutorcreatedslidesandhand-outs.There is little literature linking curation practices with formative assessment,although the use of e-Portfolios in demonstrations of curation is an emergingfield. By its nature, curation requires accessing, compiling and reproducingmaterials (Seitzinger, 2014) and there were a very low number of artifactsindicating this skill. Digital curation can be a signal of critical inquiry andengagement(Mihailidis&Cohen,2013)sothelimiteddemonstrationofthisisaconcern.Theoverviewmechanism,showingparticipant’sartifactsinathreebythreegrid,wascommonlyusedafteragapinactivityandallowedplacementatapointonapath (table 5.1). Thismonitoring of their own development is part of the self-reflectionprocess (Smith&Tillema,1998) “it enabledme to rememberwhat Ihaddone” -Robert ; “whereIhadgot to” -Elizabeth.Thispositioningofwherethe learner is now and where they are going is a key aspect in formativeassessment(Black&Wiliam,2009b),emphasisingtheactivationofthelearnerasanagentmakingdecisionsonfutureactions(p.10).

117

Table5.1-Artifactuploadcomparedtouseoftheoverviewmechanism

Artifactsdidserveasaninitiatorofdiscussion,whereparticipantswouldsearchforartifactscoveringsimilarareasandappendquestionstoit.Beforetheproject,participantshadrarelycreatedtags,buthadseentheiruseinbookmarking sites and YouTube. Students rapidly learned how to capture thecontext of their artifactswith semantically appropriate keywords and phrases,eitherontheirownorafternudgestylecommentsdesignedtoencouragetheiruse. Folksonomy systems lower barriers to co-operation and suggest anemergent vocabulary (Mathes, 2004). Here, as seen in section 4.4, the tagscreated by the participants regularly posting used context appropriate wordsandsuggestedaconsistencyintheterms,whichwerethenusedforsearching.Participantswhouploadedlargevolumesofworkattheendoftheprocessusedweek and exercise numbers as tags, suggesting that their lack of earliercommunity participation had resulted in themmissing the use of tagging as apractice that could add value.When seen in reflective statements, mechanicallabellinginthisstylecanbeasignalofshallowlearning(Jenson,2011),butthishastobeappliedwithcaution,astheparticipantwhohoardedherworkandthenpublished it all towards the end of the cycle would be wrongly categorised.Participantsdidnotseeanyadvantagetopinningartifacts inataxonomy.Mostparticipants used it initially; the activity table reveals that theusedroppedoffrapidly.Questionnaireresponsessuggestedtheysawlittleimmediatevalueinitsuse,comparedtothefolksonomytags,whichwereseenasaddingcontextalongwithvalueinthesearchingandsortingfacilities.Thelanguageinthetaxonomywasdescribedasoff-putting.

118

Twooftheparticipants,RobertandElizabethcreatedanduploadedworkasthecycle progressed, with a short time period between the labs and creation ofartifacts representing this work. Both these students fully integrated foliothinking(Feng,2006)intotheirworkingpractices.Otherparticipantstendedtohaveirregularpatternsofactivity,dippinginorout.Thetimestampedartifactsmakevisiblethedifferingworkingpatternswithparticipantgroupingspossibledependent upon the amount of time elapsing between new material beingintroducedinclassandhowlongittakestoappearintheportfolio.Thisgives:

• Participants who are triggered by new material and upload artifactstypicallyinthesameweek,forexampleRobertandElizabeth.

• Participantswhocreateartifactssomeperiodbehind,whichcanvarybytwotothreeweeks.

• Participantswholeftartifactcreationuntiltheendoftheinclassactivities(week five)or theendof thecycle (weekeight), forexampleEileenandBeryl.

Manyoftheparticipantsdidnottransitionawayfromthecustomarypracticeofsubmitting work at an end point, even if it had been completed many weeksearlier(table5.2).Thisendpointwaseitherattheendofface-to-faceclasses(e.g.Tony),orattheendofthecycle.Artifact creation has to be combined with activity to form a richer picture ofbehaviour,asmanyof theparticipantshadperipheralparticipationbyviewing,commenting or searching on existing artifacts, which would not be apparentfromartifactcreationfiguresalone.

119

Table5.2-Artifactcreationbyweeknumber

Studentsactivelyparticipatingapproachedcreatingartifacts ina linear fashion,following the direction of the material in class. Students creating significantnumbers of artifacts either inweek five orweek eight tended to have amorejumbledorderingtotheirwork,suggestingabulkuploadingprocessratherthananintegrationofartifactcreationintotheirworkingprocesses.Eileen and Beryl treated the e-Portfolio as summative assessment, reporting aresistancetoonlineparticipationandleavingartifactcreationtilltheend.Eileenuploaded all the work she had been collating privately; Beryl created a largenumber of artifacts in the last three weeks, some from the work she hadperformedearlier,therestbyasurgeinactivitytowardstheendoftheprocess.For these students, the focus audience for their work was the tutor (Gibson,

120

2006), so they considered little consequence in missing out on reflective andpeerappraisalopportunities.Priorexperiencesofassessmentstructurescandirectlyinfluencethenatureandprocess of artifact creation (Smith&Tillema, 1998) and these participants didnotseekonlinefeedback,withEileencitingherdislikeofgroupworkandBerylstatingthatshealwaysleftworkingonassessmentsuntilshortlybeforetheduedate. On reflection, Beryl said that she could see the value in collaborativeactivities,butthiswasaftertheprocesshadcompleted.

Researchquestion2:Howareartifactsshared,usedandreusedbythecommunity?There is a close relationship between using overviews of others’ work andclicking through to see one of their individual artifacts. Participantsunderestimatehowfrequentlytheylookbackontheirownandothers’work,assuggestedbythedifferencebetweenwhattheyreportandthefiguresrecordedby the analytics. Student’s fear of plagiarism (Ashworth, Bannister, & Thorne,1997; Gullifer & Tyson, 2013) aligns with these results, where they look atothers’worktwiceasoftenastheirown,yetfailtoacknowledgeit.Thisneedstobeexplicitlyaddressedinboththelearningmaterialdesignandtheinductionforthenext cycle, asproperly functioning learning communitieswith co-operativeandcollaborativelearningencourageasenseofresponsibility;makeplagiarismvisible (Palloff & Pratt, 1999) and reduce ways of “playing” the system(McConnell,2002).Derek’schangeinbehaviourinweekfour,wherehecitedhisrealisationthatthetracking system could reveal how he was using others’ work is tied to a big-brotherstyleconcern(Campbelletal.,2007)andsuggestsafailingintheinitialinduction in making the process clear, advocating the advantages of theapproach.Others’workisinteractedwithtwiceasfrequentlyasparticipant’sownwork;itprovidedsolutions,suggestionsonthenatureoftheartifactstobeincludedandtheopportunityfordiscussionoutofclass.Participantssuggestedthattheycouldnotinitiallyperceivethelevelofactivityofthegroup,duetotheopaquenatureofthe front page, the gateway into the community. After the redesign anddemonstrationsattheendofweektwo,theusageincreases(table5.3).

121

Table5.3-Viewingandinteractingwithothers’work

Thenumberofedgesandnumberof islandsare inan inverserelationship– intheweekswheretherearemoreinteractions,thenumberofparticipantswithnoactivitydecreases. Thegraphdensityandreciprocity followthesuggestionsofactivity levels from the sociogram, with a peak in week three followed by anincreaseinweekssixtoeight.Bothsuggesttherelationshipbetweennodesovertime,withthegraphdensitypeakingat0.3,indicatingtheoccurrenceofathirdofthe theoretical connections possible between participants. Reciprocity is ameasureofthemutualnessofconnections,thebalancebetweeninteractingwithothersandbeinginteractedwith,oratabasic level,viewingorcommentingonothers’workinaratiotohowmuchtheirownworkwasviewedorcommentedonbyothers.Althoughindividualoptimalvaluesforthisaredifficulttocommenton, comparisons between it and the reciprocity reveal that there are weekswheretheremaybemanyconnectionswithanimbalanceinthemutualness,forexampleinweekseven.Table5.4-Thumbnailandfolksonomyusage

Participants found the folksonomy system usefulforexploringothers’artifacts,particularlyafterthethirdweekwherealargernumberofartifactsweretaggedandavailableforsearching–suggestedhereby the viewing of a ‘subset’ of others’work (table5.4).WengerandLave(1991)suggestthatperipheralityisnotphysical,butthesociogramalgorithmmakesarepresentationofthispossible.WhenthenumberofartifactscreatedineachweekareoverlaidontheSNAdiagrams,arecurringpattern is indicated inthat there is a relationship between artifact

WSA Week 3

1

0

0

0

9

0

8 3

0

5

0

0

0

10

0

0

Figure 5.1 - Sociogramwith artifactconstruction and participation forweekthree

122

creationandoverallactivity,whereactivityisabroadercategoryencompassingviewing,reviewingandcommenting.Participantswhoregularlyconstructworkalso navigate and participate more frequently, which results in a highercentralityvalue,morecentralplacement inthediagramandechoes findingsonstrong relationships and shared resources (Dawson, 2008). Those that are notcreatingworkparticipateless,typicallyreviewingothers’workandoccasionallycommenting,representedbyaringofouterzerodigits(figure5.1).

For participants on the periphery, who are not creating artifacts, the mostcommon activities are looking at overviews of others’ work and then clickingthrough to seeothers’ individual artifacts (table5.5). In this view, artifacts areseen in a gridwithin the context of other artifacts,with possible searching onsubsetsbytag,keywordordate.Inthecommunityofpracticemodel,peripheralparticipation suggests newcomers or new entrants observing and mimickingexperts (Lave et al., 1991). Here this role is transitory and emergent, asmorefrequentreificationincreasesthevisibilityoftheparticipant’sartifactsinthee-Portfolio.Robert,ElizabethandDouglas’workwas themostoftensearched forandviewed,aligningwitheigenvectorcentralityscoresthatsuggesttheirstatusasdemonstratingperceivedexpertise.Table5.5-Activitiesofparticipantsontheperiphery

The placement of the circles is derived from overall interaction with others’artifacts,soitispossibleforaparticipanttocomment,interactandviewothers’work many times, giving them a more central placement without havingaccompanyingartifactuploads.Thisistypicallyasignalofthetutorrole,the‘1’infigure5.1.

123

Asparticipantsmadetheirworkvisible,thesmallerinitialexamplestendedtobesimilar across artifacts with more difficult concepts allowing for greatervariability and individuality. Combining the analytics data with the artifactsreveal where demonstrations of techniques cascade through participants, forexample,thetransitionofthePersonclasscodebetweenparticipantsovertimeshows how Robert, who was the first to tackle this concept, influenced laterparticipants. Robert’s original code was enhanced by Derek to include extrafeatures threeweeks later.Maureentries toreuseDerek’scode,but introduceserrors(figure5.2).

Artifactfromweekfour,byRobertS.

Artifactfromweekseven,byDerekW

Artifactfromweekeight,byMaureenH.

Figure 5.2 - Artifacts demonstrating knowledge

cascade

A post by Elizabeth was the first to talk about style in the context ofprogramming,anissuethatisdifficulttoteachasthenormsandvaluesimplicitinprogramming style canbe subjective (Pieterse,2008)andbestpracticesarefrequently ignored by students (Li & Prasad, 2005). Good stylewas positively

124

reinforcedthroughcommentsandsuggestions,andwasthesubjectofanonlinediscussion, echoing suggestions that peer review and collaborative interactioncan be a better approach for learning programming style (Li & Prasad, 2005;Robins,Rountree,&Rountree,2003;Roth,1980).In this cycle, participantswere allowed to determinewhen they should createartifacts, alongwith thenatureof theartifacts themselves.Fiveof thestudentsengagedwith embedding folio thinking into theirworkingpractices, suggestedbyalinearprogressioninthesubjectsoftheartifactsandtheirpresenceonlineandinclass.Table5.6–Artifactcreationbytimeandperiod

Participantswith a higher level of engagement created artifacts over a greaterspreadoftimeandtheanalysis(table5.6)revealsthatnoartifactswerecreatedinthefirsttaughtsession.Students with a less systematic activity tended to create artifacts on material frommultipleweeks,withacleardragexistingbetweentheintroductionofnewconceptsintheclassandthetimeittookforartifactstoappearontheseareasintheportfolio.Although this temporal flexibility ispresentedasanadvantage tothelearnerine-Learning(Macpherson,Elliot,Harris,&Homan,2004),thelackof

125

rhythm to their participation harms community growth (Wenger, 2001) andreducestheopportunityforformativefeedback.Thislaginparticipationwasalsoapparentwhenmanystartedtocommentonolder artifacts,which, if theywerequestionson theartifact contentweremore likely togounanswered.Thiswasparticularly true for those studentswhohad lower levels of participationuntiltheend,whothenuploadedlargevolumesofworkinthefinalweek.

Researchquestion3:Whatistheroleofthetutorandtheformofthecommunity?Tutor commenting on artifacts inside the e-Portfolio fall into the elaborationcategory of formative feedback (Shute, 2008) which tended to discuss errors,provide guidance (p.158) and promote connections between users / artifacts.Theteachingpresenceindicatorssuggest lowlevelsfortheinstructionaldesignand organisation role which can be accounted for by the higher levels of thisactivity that occurred off-line in class, especially during the initial period offragmented participant activity. There were higher levels of organising andinstructing theparticipants in the first thirdof thecycle,with initialcommentson netiquette and utilising the medium focussing on using tags and writingeffectivereflectivestatements.Duringtheintroduction,thenatureandprocessofartifactcreationwasdemonstrated,butdespitethismanyoftheinitialartifactslackedthemeta-datathatwouldenabletheirreuse.Astheprincipalmechanismsfor findingartifacts in the systemusesa search system, thoseartifactsmissingthis information, and through these, the participant’s activity would be lessvisible. Nudge style comments were attempted both online and in class,improvingtheuseofthefolksonomybutfailingtoigniteinterestintheuseofthetaxonomysystem.

Inthecycle,theboundaryofthecurriculumwaslooselydefinedbythenatureoftheteachingmaterialsandtheinclassactivities,althoughthecontentofartifactswas left to the students who could include anything thought relevant. Assuggestedinthefirstresearchquestiondiscussion,themajorityofartifactswerecreatedinresponsetoteachingmaterialswiththeperceptionthattheseweresetby the tutor and required solutions. The folksonomy tags emerging from thecommunities’artifactsprovidedsimpleabstractionsthatmadevisiblethenatureof the material being learnt to the tutor and the rest of the community – forexample the term ‘authentication’ was rapidly associated with artifacts thatcoveredmaterialonbuildingalogginginsystem.

Artifactsandpostswereautomaticallylabelledwithadatestamp,makingvisibleto their peers and the tutor each participant’s level and pattern of work.

126

Participantswereallowed todecide thenatureand frequencyof theirpostingswith the tutor suggesting that interactingmoreregularlyand frequentlywouldbebetter.Despitethis, therewerevaryinglevelsofactivityoverthecycle,withthecontentandtagsselectedinweeksfourtosevensignallingdifferingtemporallagsamongsttheparticipants.Theanalyticsusedtoinformtheactionsofthefacilitatorrequirethecycletobedivided into periods, which in this instance have been aligned to seven daysequences.Thiscanbeproblematicassomestudentsexhibiteddragsorflurriesof activity, interacting with artifacts out of the weekly sequence. Difficultiesemergedwhenstudentswithirregularactivitystartedpostingonartifactsfrommanyweekspreviously-thesecommentswereunlikelytogainreplies,requiringthetutortointervene.The‘life’ ofanartifactcontinuesbeyondtheperiodof itsinitial creation, which can create a reluctance for the original author to beengagediftheyarerequiredtorevisititmanyweekslater.Furtheranalysisofthetime and date stamps suggested that no artifactswere created in themorningsession, indicating that therewas toomuch tutordirection in thisearlier class,withfewerexplicitartifactcreationopportunities.In the facilitation role, the traditional COI model suggests promoting thecommunity and influencing the behaviour of the participants through a socialrole, seeking consensus, identifying agreement and setting the climate forlearning;thishadmixedresultsininfluencingthebehaviouroftheparticipants.According to the COI analysis, over the three cycles, there was a higherfacilitationactivityintheinitialpartsofthecyclewhichistobeexpectedduringtheformationofthecommunity.Thisdecreasedoverthenexttwothirdsofthecycle. The social form of online tutor interaction comes through comments onothers’work,whichrestrictsthiswhenparticipantshaveareducedorirregularonline activity. After the initial signup, participants had a low level ofparticipationforthefirsttwoweeks,despitesuccessivedemonstrationsinclass.Becauseofthislowlevelandthelackofvisibleindicatorsofuse,thestudentsdidnotengageuntildemonstrationsofactualartifactsand the search facilitywereshown in the redesign of the front page at the end of week two. Studentsresponded positively to the use of authentic data in the redesign and thesefactors together increased participation for approximately half of the group.Thosetreatingthee-Portfolioasatraditionalassessment,havelittleopportunityforthistotakeplace,makingitdifficulttodrawparticipantsin.

127

The lengthof timeleftbeforeatutorresponseappearswascomplicatedbythelowactivityinthefirsttwoweeks,whichisaconcernas,ifleftfortoolong,couldlimit the growth of the learning community. At the other extreme, excessivepostingcanshutdownstudentinteractionandinhibithigherlevelsofcognitivepresence (An et al., 2009; Dennen, 2005;Mazzolini &Maddison, 2007). Thereweretwoapproachesused:

• Increasing the time before responding over the life of the communitydependantuponthevolumeofpreviouspostingbytheindividual.

• A strategy of suggesting solutions throughpeer to peer communication,afterthenetworklearningphilosophy.

Formanythetriggerforartifactcreationwastheperceptionofacompletedtask,whichreducedthenumberofartifactsthatwereincompleteordemonstratedaparticipant seeking assistance. This is difficult to role model through tutorcreatedartifacts,butwasencouragedthroughthefacilitatingdiscourseroleandby activities in class. Inducing the integration of artifact production into thestudent’slearningprocessiskeytothis,butdespitedifferingapproachesusedtoencouragethis,halftheparticipantsdidnotengagewiththisidea.Student enquiry is encouraged through activity and feedback, but thepracticalities of this were complicated by the nature of the community. Theoriginalconceptionofthepeerbasedcommunityinthisworkwasforthetutor’sartifacts and comments to have the same weight and validity as the otherparticipants,andthesewouldalsoserveasawaytointroducedirectinstruction.Unfortunately, thenumberof tutorcreatedartifactswas lower,duetothetimededicated to moderating comment threads and monitoring others’ artifacts,particularlyinthemiddleandfinalthirdpartsofthecycle.Thosetutorartifactscreatedwere indicators of correct code and curation style artifacts suggestingfurtherreadingandareasthatcouldbeexplored.DirectInstructionpeaksatthestartandendofthecycle;atthebeginningduringtheinitialconstructionofthecommunity it was important to engage with questions from the participants,balancingtutorresponseswithappropriatedelaystoencourageparticipation.Inthe final third of the cycle, participantswere out of class, so there is a naturalincreaseinthevolumeoftutordirectinstructiontosupplementthelackofface-to-facecontact.Theimbalanceinparticipationlevelsacrosstheeightweeks,withan increase in activity in the final part of the cycle resulted in orphanedcomments, which the tutor had to answer due to their technical nature.

128

Garrison’sindicatorof“knowledgeinjection”isperformedbyartifactreification,doneherethroughbothtutorartifactsandtheincreasingprominenceofartifactscreatedbyparticipantsdemonstratingemergingexpertise.Despite indicating the idea of community support in the induction, there wasreluctanceamongst theparticipants to share incorrectwork,or tocreatemorecurationstyleartifacts.Postingincorrectworkwasencouragedbothinclassandonline, but at the end of the process some participants still reported a lack ofenthusiasm for posting incorrect work, or surprise that they could. Twoparticipantswhowereattendingeveryweekandhadlittleonlineactivityaskedquestions by email, including code for comment. These were encouraged tocreateartifactsortosearchforanartifactcoveringthesamematerialandtoaskquestions online. Confirming understanding and diagnosing misconceptions isperformedbyattachingcomments toartifacts,which isdependantonboth theregularityandnatureoftheartifactsbeingpublished.Participantswhoregularlypostwithfoliothinkingembeddedintheirworkingpracticesareclearlyabletoenjoy more interactions, receiving the advantages of formative feedback frompeers and the tutor (Lin, 2008;Rickards,Diez,Ehley,&Guilbault, 2008;Wang,2009). There was some success in facilitating the sharing out of the tutorfeedback role amongst participants by using links and directions to otherartifacts.Theteachingpresencedirectinstructorrole(DI)washighinallpartsofthecycleand alignswith the typical indicators such as presenting questions, diagnosingmisconceptions and introducing knowledge from diverse sources (Garrison,2011).Intheroleofteacher,newmaterialwasintroducedduringthefacetofacesession everyweek. Themajority of artifacts created in response to thisweredemonstrations of small extracts of codeor screen shots of programs running,with fewexamplesofcurationoraccessingexternalmaterials.Oncethis themeemerged, the nature of the teaching material was changed, so that greaterdifferentiationwaspossible,withmoreexplicituseofexternalresources.Examplesincluded:

• ViewaYouTubevideo,andthenuseittocreateafunctionwhich...• Usethehelpsystemtofinddefinitionsofthreebuilt-infunctionsandthen

writesmallprogramstodemonstratetheiruse.

129

Thissucceededingeneratingawiderdiversityofartifacts,butdidnot increasethereporteduseofexternalresources.Although satisfying two of Wenger’s characteristics of community, both thedomain of knowledge and a shared repertoire developing practice, levels ofmutual engagement fall short.Despite tutorencouragement, allowing completefreedom in both artifact choice and frequency of activity resulted in half thecommunityhavinglowparticipationlevelsoraskewinactivitytowardstheendofthecycle.Makingthereificationprocessvisibledoesallowforthemoreactiveparticipantstodemonstrateemergingexpertise,astheirartifactsarefrequentlyviewed and attract questions. This can act to address one of the frequentcritiques of the COPmodel in classroombased activities,which is the possiblelackofexperts.The emergence of this expertise can be assisted by the tutor,who can suggestconnectionsbetweenparticipantsandrelevantartifacts,ratherthanusingdirectinstruction. Without artifact construction, it becomes difficult for the tutor todifferentiate lurking from legitimate peripheral participation, and those thatremove themselves from mutual engagement can find it difficult to attractparticipationlaterinthecycle.

5.2ReflectionandfeedingforwardThissectionreflectsoncycleoneandsuggestschangesgoingforwardintocycletwo,whichincludeimprovementstotheinductionprocessalongwithchangestothee-Portfoliodesignandthedatacollectionmethods.

TheinductionprocessParticipantssuggestedthatoneofthereasonsfortheinitiallowlevelofactivitywasthelackofformalguidanceonartifactproductionandsharing.Ratherthanthe approach to sharing evolving over time, it was suggested that the way tointroduce these as community norms was through a more detailed, specificinduction with demonstrations and explicit opportunities for artifactconstruction and commenting. The teaching presence indicator from thecommunityofinquirymodelisperhapslessusefulwherepromotinganequality,participatory approach, as the aimof peerparticipation results in the teachingrole being distributed amongst cooperating peers. It does reinforce theimportance of an initialmoderator role, such as that suggested in Salmon’s e-moderatormodel.Thisandthefeedbackfromtheparticipantssuggeststheneedfor a stronger inductionwithmore direction on artifact creation and curation,

130

which is key to making the formative feedback practice clearer and betterdefined(Strivensetal.,2009).For the small proportion of students who don’t use, or wish to use socialnetworkingsites,aspecialeffortshouldbemadetoemphasisetheadvantagesofthe learningcommunityapproachduringthisprocess.Therecanbeparticipantanxietyoversharingandreceivingresponsesfrompeers(p.13),butthiscanbeaddressedbyemphasisingtheprocessofreificationandtheadvantagesinherentinparticipation.‘Loworder’ activitiesthattypicallyfeaturedemonstrationsofrepetitivelearningpushstudents towardsthe ideathat therearesinglesolutionstoactivities,andthatartifactsaresolutions to tutordesignedactivities that reinforce thepowerrelationineachrole.Startingwiththeseandthenbroadeningtheactivitytypesover time, along with more vivid demonstrations of what can be used todemonstrate learning taking place, should increase the variety of artifactscreated.Thevalueofmakingvisibleworkinprogressandcommunityassistanceshouldalsobedemonstratedwithactualexercisesduringthisprocess.Curationcould be encouragedwith specific activities, andmore emphasis in the use ofpersonallearningenvironmentsandpersonallearningnetworksshouldbroadenthesourcesthatareusedbytheparticipants.Better immediateuseof theanalytics in real timemayencourageparticipationlevels, which would require changes in the interface. This, and a betterdescription in the induction ofwhat and how data is being collected from thesystemshouldaddresssomeofthesurveillanceconcernsraised.

Thee-PortfoliodesignThe analytics prove a rich source of raw data, but finding better ways ofrepresenting the information to make it more accessible for both participantsandthetutorshouldbepossible.Duringacycle,therearethreekeyindicatorsofparticipation,whichare:

• theuseofthedashboard;• thelevelofartifactcreation,viewandreviewofownwork;and• thelevelofinteractionwithothers,encompassingsearching,viewingand

commentingonothers’work.

131

Thedashboardisthemainpagewhichisdisplayedafterloginandisreturnedtoafter each activity, which serves as an indicator of the general level ofparticipation.Thereareopportunitiesforusingbothartifactsandactivitytracesin the dashboard to increase links between the participants, using threemechanisms:

• usingthemetadataassociatedwitheachartifact,forexamplethetags;• usinganalyticsinrealtime;and• usingpreviousartifactstorecommendartifactsthatmaybeofinterest.

Tagsattachedtotheartifactssignalbothanemergingvocabularyandthetypesofactivitiesthatarebeingattempted,soitshouldbepossibletoreflectthisbacktothecommunityandthetutor.Similarly,activitydatashouldbeabletobeusedtosummarisewhataparticipanthasachieved,therhythmofthecommunityasawholeandtosuggestartifactsthatwouldbeoffutureinterest.Thesechangescanallbeimplementedonthedashboard.

DatacollectionmethodsandanalysisGraphmeasuressuchasnumberofedges,islandsandgraphdensitycanbeusedduringthecycleasappropriatesignalsofthecommunityperformanceovertime,signallingpeaksand troughs in class andonlineactivity.Degreedistribution iscalculatedbyparticipant,but isalsomoreusefulasameasureofactivityinthecommunity, working as a measure of the number of connections being made.Although the use of densitymeasures has been questionedwhen the networksizebecomeslarger(Toikkanen&Lipponen,2011),itisausefulmeasurehereasasignal tothetutorof theproportionofconnectionsbeingmadeonaweekbyweekbasis.For the measures that are calculated by participant such as degree value,betweenness and eigenvector centrality, highlighting the top and bottom 20%are useful in identifying individuals that are underperforming or “highflying”.These work less well in instances where the number of active participants islower but does signal where the tutor and/or portfolio is failing to promoteconnections. Eigenvector centrality is a signal of dissemination, which isapplicable toartifact creationand reusebyothers,whichwhenevaluatedoverlonger periods may be suggestive of emerging expertise. Betweenness is lessexpressive in thecontextofassessmentartifacts,as there isno immediatewayforbrokeringacrossthenetworktobemeaningfulasthesuccessfulapplicationofthenetworkedlearningphilosophyshouldmakethisredundant.

132

Sociogram diagrams provided an immediate visualisation of non-participationandthedegreeofcentralitybyactivity,butcanbetimeconsumingtocreateandrequire temporal boundaries to be set, whichmay not alignwith participant’sactual practices.Amoremeaningful analysis of thebehaviourof closerpackedactors requires an analysis of artifact content to see the nature of thecollaborationor reuse that is takingplace; cliquedetection iseasier to identifythrough this representationor by calculation, although it hasnot been appliedherebecauseofthesmallersamplesize.Althoughvalid,theuseofanalyticshastobeusedcautiouslyas individualsmayhaveactivitypatterns thatdon’t alignwith the measures. Here for example, the statistics would report that twoparticipants were not engaged at all, when in reality they had not integratedartifact production into their working practices so the work they wereperformingwasnotregistered.Thenextchapter takes thisanalysis forwardby implementing thechanges inasecondcyclewithpostgraduatestudentsonanequivalentmodule.

133

Chapter6 CycletwoThischapterdetailsthesecondcycle,whichfollowsontwomonthsafterthefirst.Participants were postgraduate students studying a business internet systemsmodule.After introducing the context, the changes suggested from cycle one arediscussed, followed by implementation details and a return to the overarchingmethodologyandmethods.Asummationofthedatacollectedduringthecycleisthen presented. The three principle changes in cycle two concern; a refinedinductionprocessusing an e-moderationmodel; an increase in theuseofdataanalytics to reflect activity back to the participants; and a recommendationsystemforartifacts.

6.1IntroductiontothecontextofthesecondcycleAlthoughthecycletwoparticipantsaretypicallyolderthantheundergraduatesin cycle one, the international nature of the class results in amixture of entryqualifications with a variability in their academic skills. The majority of thestudents have IELTS5.5 or higher, and arrivewith differing levels of technicalability. In this cohort, two of the students have previous programmingexperience, but neitherdescribes themselves asproficient in theprogramminglanguage and database technology used here. Five of the others describethemselvesasbeingtechnicallycapableinwebtechnologies.Lower IELTS levels suggest a modest to competent level of English (“IELTSscoringindetail”2016)andthisalignswiththeabilityofthestudyskillsthatthestudentsdemonstrate,whichistypicallyequivalenttothatofalevelfiveorlevelsix undergraduate student. Themajority of the students have English as theirsecond language. Following an assessment during the induction process, thoserequiring extra assistance are asked to follow specific postgraduate add-onmodules,coveringacademicwriting,researchandpresentationskills.The postgraduate business internet systems module is taught over a twelve-weekperiod.Eightweekstaughtmaterialarefollowedbyfourweeksdedicatedto cross module project work. Delivery consists of two sessions; a two-hourlaboratory followed by a two-hour tutorial on a subsequent day. Thepostgraduate course has a higher expectation in the level of work studentsperform out of class in their self-study time. Most laboratory sessions coverpracticalmaterialwithsomedemonstrations,examplesandexerciseswhicharecompletedbothinthesessionandinself-studytime.Thetutorialclassisusedfor

134

moretheoreticalmaterial,alongwithdemonstrationsanddiscussions.Followingthepatternincycleone,theinitiallaboratoryisusedasthestartofa‘new’week,with the expectation that newmaterial thatwill be covered in the subsequentseven days. Postgraduate students tend to attendmore regularly compared totheundergraduatestudentsdescribedincycleone.Forcycletwo,Iwaspleasedtodiscoverthatall17studentswishedtotakepartin the research, hence no alternative portfolio scheme was required.Demonstrating thenatureof theprocesswithanonymousexamples fromcycleoneduring the signup encouragedparticipation.Only oneof the studentshadprior experience of a portfolio and this was with a teacher-prescribed set ofartifactsthathepresentedthroughasinglewordprocesseddocument.

6.2ChangesforcycletwoThe threeprinciple changes for the second cycle are abetter induction for theprocess, a recommender system and more meaningful use of the analyticsgenerated during the process. Each of these is discussed here, withimplementationdetailinthefollowingsection.

Ane-moderationframeworkfore-PortfoliosFeedback from the students in the first cycle suggested there should bemoreclarity at the beginning, with more guidance on the nature of artifacts thatparticipants could create. Salmon’s five stagemodel (Salmon,2003)providesapopular structure for the e-moderation of discussion style forums, routed insocialconstructivism(Salmon,2007).Overaseriesofstagesdescribedasladderrungs, the complexity of the participant’s interactions increase, through an e-moderator’sguidanceandscaffolding(figure6.1).

135

Figure6.1–Salmon’sfivestagee-moderationmodel

It has been criticised for being rigidly applied (Lisewski & Joyce, 2003), nottakingintoaccountmixedpedagogiesandfailingtoacknowledgetheeffectthatco-locationofthelearnersmayhaveontheearliersocialisationstages(Jones&Peachey, 2005).Despite this, it has been applied to create communitieswith avariety of different technologies such as podcasting,wikis or in virtualworlds(Salmon,2011)anditspopularityhasspawnedmultiplederivations,forexampleMoule(2007).Whentheoriginalformisusedwithe-Portfolios,therecanbeproblemswiththeplacementof the fifth-rung,asparticipantsreachagreaterdegreeofautonomyandhigherlevelsofreflectivethinkingearlierintheprocess(Ehiyazaryan-White,2012). Traditional e-Portfolio implementations direct themoderator to set out“the tone of the community, attract and welcome new members to thecommunity, and lay out the purpose and guidelines for participation with thegroup as it forms” (Schwier, 2001, p. 3), providing continuous and promptfeedback and promoting self reflection through reflective comments (Çimer,2011). In this instance the use of the networked learning community modelsuggests that peer comments replace the necessity for the moderator to actthroughouttheprocess.Initiallytherewillalsoneedtobemoredirectiononthenature of participation as the community aims to move from an equifinalitymodel(Pedler,1981)toapeerbasedlearningcommunity.Theversionappliedherewilladjust for thisandadditionallyprovidestructureandguidanceonthenatureofartifactstopromotecollaboration.Theframeworkandteachingmaterialsusedwillinitiallyprescribetheartifactstobecreated,but

Reflecting on personal experience, creating links

Development5

Collaborating through building and manipulating artifacts, interacting and contributing towards a common goal

Knowledge construction4

Offering information to others and helping other to achieve individual goals

Information exchange3

Personalising avatars, developing identities, connecting through artifacts and activities

Online socialization2

Downloading software, creating avatars and learning basic technical skills

Access and motivation1

Incr

easin

g in

tera

ctivi

ty

136

will gradually suggest the possibilities of greater diversity in artifact choicethroughopenendedexercisesandcollaboration.Therewillalsobeanemphasison ensuring that participants engage with learning activities and resourcesoutsidethosesuppliedbytheinstitutionthroughacurationprocess.

RecommendersystemsOne of the consequences of the data gathered during cycle one was therealisation that the system put in place to provide data for an analysis of thecommunity’s growth over time could be additionally used inside the systemitself,inrealtime,sothatparticipantscouldseetheirworkandworkloadinthecontext of the whole learning community. Every page, click and interaction isrecorded (Clow, 2013) and is an opportunity to create actionable intelligence(Campbell et al., 2007). This data can be used in twoways – to create artifactrecommendations based on previous activity and to reflect analytic data onperformancebacktotheparticipantsduringtheprocess.Networked learning (NL) advocates growinga learning communityby creatinglinks between users and resources, a role which is traditionally built into thedesignoflearningactivitiesandisimplicitintheNLtutorrole.Inthee-Portfoliocommunity here, resources are distinct artifacts, which allows for links to becreated through tutor recommendations. Rather than rely on theserecommendationsalone,theactivitydatacanbeusedtoautomatethesuggestionoflinks,increasingthepossibilityofconnections.To encourage the growth of community, Neilson (2010) suggests makingparticipation ina recommendationsystemasideeffectofactivityanduses theexample of Amazon gathering information about the books bought to suggestfurtherpurchases.Therecommendationsystemtrialledhereisacontentbasedcollaborative system,where “items are recommended that are similar to itemsusers preferred in the past” (Adomavicius & Tuzhilin, 2005, p. 5). There aredisadvantagestosuchanapproach,inthatmetainformationassociatedwithanartifactmay not capture the rich aspects of its content and over-specialisationmay occur, where only artifacts that align with previous searches may besuggested (Balabanovic & Shoham, 1997). A richer systemwould be based oncollaborative recommendations, but this would require participants to gradeeach others’ work and this has proved difficult to implement because of thevolumeofworkrequired“anoneroustask”(p.67).

137

Therecommendationsystemtrialledhereallowsforlinksinthee-Portfoliotobederived from a recommendation system which uses existing artifacts,participant’sbrowsinghabitsandsearchactivities.Whencombinedthisshouldenablesuggestionsofartifactstobepartiallyautomated,increasingthevisibilityofartifacts.

DataanalyticsData analytics is a relatively new field,with a definition suggested by the firstinternationalconferenceonlearninganalyticsandknowledgeofanalyticsas:

“themeasurement,collection,analysisandreportingofdataaboutlearners and their contexts, for purposes of understanding andoptimising learning and the environments in which it occurs”(Siemens&Long,2011,p.3).

SuccessiveHorizonreportshavesuggestedthattheapplicationofdataanalyticsin education has moved from an emerging field to a mature practice, aspractitioners and administrators realised the wealth of data that was beinginadvertently collected as students interacted with electronic resources(Kennedy et al., 2013). There have been initiatives to use analytics to identifypotentialatriskstudentsalthoughthesetypicallyusegradesandachievementsrather thanparticipation for evidence (Campbell et al., 2007). This use of dataanalyticsdoesnothaveanarticulatedepistemologyofitsown(Clow,2013)butisajigsawofdifferenttechniques,toolsandmethodologies.Examplessuggestedinclude:

• Predictivemodelling,generatingstatisticalprobabilitiesonsuccessrates,forexample,thecoursesignalssystematPurdueUniversity(PurdueResearchFoundation,2013).

• Socialnetworkanalysisandsociogramsdepictingactivitybetween

participantstypicallygeneratedfromLMSdata.

• Usagetracking,recordingwhatfeaturesorfunctionsofsoftwarearebeingusedovertime.

• Contentanalysis,usingnaturallanguageprocessingorsemanticanalysis

toseehowconceptsareusedandarise.

138

• Recommendationengines,althoughthereislimitedusageatthemoment.The metrics generated in such systems tend to be used in two ways, withstudentstakingactioninthelightoftheirownactivitycomparedtothatoftheirpeers or through a teacher initiated process identifying students requiringadditionalassistance(Clow,2013).Theprocess istypicallydescribedasacycle(Campbell et al., 2007)withanemphasison reflection suchas inKolb’smodel(Clow,2012).

Thereareimplicationsandconcernsovertheuseofanalytics.Surveillancecouldreinforceexistingpowerrelationstothedetrimentof the learner(Clow,2012),learnersmaybeuncomfortablewith themistakes thatbecomevisiblewith theopennessofthedataanditispossiblethatmisclassificationmayoccur(Campbellet al., 2007). Caution is also advised as students may become “more data-orientedabouttheirlearningprocess”(Bader-Natal&Lotze,2011,p.185).Here, an appropriate use of real-time analytics derived from the activity tableshould encourage self reflection and increase self responsibility, as long as themetrics are carefully selected to optimise learning (Clow, 2012). Revealing theactivityofindividualsinthecontextofthegroupwillmakeoverallparticipationlevelsvisible,whichcanbedifficulttoperceiveonline.Theintentionhereisthatthiswillalsoenableparticipantstoseetheimmediatevaluepossibleintheuseofthestatisticsgatheredduringtheirday-to-dayusageofthesystem.

6.3Methodology,methodsandimplementationchangesAs discussed in chapter three and section 4.2, amodified version of De Laat’smultimethodresearchframeworkforstudyingnetworkedlearningprocessesisusedhere(DeLaat,Lally,Lipponen,&Simons,2006b),appliedtothestudyofe-Portfolio artifacts in a community,where context analysis is performed to seewhyparticipantsbehaveastheydo.Thesignificantchangetothedatacollectionmethodincycletwoistheuseofindividualinterviewsattheendoftheprocess,whichwaspossibledue to theplacementof the cycle in the secondhalf of theacademicyear.

MethodsandtheiruseAswith cycle one, the data collected came from activity data recorded by thesystemasitwasused,alongwiththeartifacts,notes,logsandfeedbackfromtheparticipants made during the cycle (table 6.1). This is supplemented with theaddition of 11 semi-structured interviews, from the 17 participants available.

139

These were chosen to reflect the broad spread of participation levelsdemonstrated.Table6.1-Methods,meansandsample

Theanalysisprocessformuchofthisinformationisthesameasincycleone,asdetailedinthemethodssectionofchapterfive.Theprocessusedintheanalysisoftheinterviewresultsisdetailedinsection3.4oftheresearchdesignchapter.

Thedesignofthee-PortfoliolearningcommunityThereisagreaterfocusonthedevelopmentofthecommunityincycletwo,withWenger’s principals of community design (2002) and Schwier’s elements ofcommunity (2001) used in an attempt at encouraging both initial and regularparticipation.Some ofWenger’s principals are embedded in the existingmodel, for exampledesigningforevolution,whichhereisanacknowledgementofthechangingroleofthemoderator,astheteachermovesmoretowardsfacilitationoverthelifeofthe community. Different levels of participation are suggested in the non-privilegingofstrongconnectionsthroughnetworkedlearningprinciples.Wenger suggests combining familiaritywith excitement and a transition at theentrance.Thiscanbehardertoachieve,butaredesignofthelandingpageofthee-Portfolio should allow for more explicit changes and activities to be visible,throughrecentandrecommendedpostsofinterest.Hissuggestionforcreatingarhythmforthecommunityshouldbeaddressedbythedesignofthee-Portfoliomoderation framework, which is designed to have step levels of activityinvolvingincreasingcomplexity.

Thee-PortfoliomoderationframeworkThestructureusedhereisderivedfromSalmon’sfivestages,butacknowledgesthattherewillbeflexibilityinthespacingandlessrigidityintheprocessestaking

140

place (figure 6.2). Ownership is moved up several rungs from its originalplacement,whichiscommonine-Portfolioactivities(Ehiyazaryan-White,2012).Stageone–InitiatingactivityMcConnell (2006) suggests initiating activity with invitations, welcomingmessages and an explicit agenda setting with scaffolding. Similar to Salmon’saccessandmotivationrungone,thisstagecoverspracticalissuessuchassigningupandfamiliarisingtheparticipantswiththefunctionalityofthee-Portfolio.Assome cycleone studentsdelayeduploadinganywork, a specific induction taskwill ask students to create and upload an artifact, with appropriate tags andreflectivestatement.Participantsarethendirectedtolookateachothers’workandtotrythecommentingsystemParticipants are then encouraged to create artifacts from work performedoutside thedesignated lab time,whichwill be small, closedproblems from labworkandexercises.Foliothinking,whereartifactcreationisembeddedintothelearningprocessisintroducedatthisstageandthestudentsareaskedtotrythisoutwithanexerciseinthelab.Asincycleone,themoderatorwillissuewelcomestylemessages,reinforcingtheimportanceintheuseofthemetainformationattachedtotheartifacts.Stagetwo–OnlinesocialisationSchwier(2001)indicatestheimportanceofcreatingasharedhistoryandculturein the emergence of a learning community, but here the participants alreadyknoweachother frompreviousmodules, so theneed for thesocialisationrungwaschanged fromagetting toknoweachotherprocess toacknowledging thatothers’ work would be visible in the system and the possibilities inherent inartifactreuse(Jones&Peachey,2005).Thisisanattemptataddressingtheissuethatsomeparticipantsincycleonehadnotexploredthesystemandrealisedthenatureofthevisibilitypresentintheuploadedwork.

141

Tasksandactivitieswhichtheparticipantscanuseasthesourceofartifactsarestill well defined at this stage and participants are asked to comment or askquestions inside thee-Portfolio itself.This is to address someof thehesitationissuesassociatedwithweakersocialcuesthatcanoccur innetworked learningcommunityformation(DeLaat,Lally,Lipponen,&Simons,2006a).

1: Access and Motivation

ePortfolio moderator model

2: Online Socialisation

3: Information Exchange

4: Knowledge Construction

5: Development

Participant Activities / characteristics

Moderator Activities / characteristics

Logging in, creating an artifact, with tag and reflection, Exploring the ePortfolio

Set simple create artefact task, respond to at least one artifact for every participant with welcome messages - creating 'safe' climate

Creating multiple artifacts, interact with anothers artifact, respond to at least one

Setting tasks with multiple artifact responses, encourage posting of unfinished or incorrect work, answer 'left' questions after appropriate time

Setting closed and open tasks with range of possible responses, encouraging participants to look at / answer each others questions, moderating to equalise participation

Higher levels of unprompted artifact creation, posing answering questions, use recommendations, searching

Open tasks, distinct peer based activities, answering fewer questions in moderator role

Acting as peer rather than moderator

Using artifacts to answer questions, peer collaboration, artefacts from varied sources, acknowledging community, using analytics in peer context

Higher levels of reflective thinking, evidence of increased self-regulation and autonomy

Figure6.2-e-PortfoliomoderatorframeworkderivedfromSalmon(2003)

142

Stagethree–InformationExchangeThis stage is designed to encourage participants to realise the advantagesinherent in making visible draft work and to seek solutions from others byuploading work with errors. The e-moderation role places more emphasis ondirecting students to others’ artifacts and asking them to comment on others’workratherthanonsupplyinganswers.Thenatureofthetaskandactivitiesischangedherewithopen-endedproblemsthat broaden out the nature of the work that can be submitted. For example,ratherthansettingprescribedexercisesusingprogramminglanguagefunctions,studentsparticipateindemonstrationsofarangeoffunction,thelocationofthehelpsystemandhowtoreadonlinemanuals.Theyarethengiventhefreedomtochoose and write examples to demonstrate their use from the wide numberavailable.Stagefour–KnowledgeconstructionThisstageconsistsofpeeractivitieswithchoiceofartifactcreationdrivenbytheparticipants themselves. Commenting is free flowing and the levels ofparticipationshouldbebalancedanddistributed.At thisstage, therewillbeanemphasisonthecurationrole,whereparticipantswillbeencouragedtouseandbring in resources from outside the formal domain of class resources. ThePersonal Learning Environment and Personal Learning Network concepts willhavebeenintroducedintheinitialpartsofthecourse,buttheuseoftheseasasource of research and learning will bedemonstrated both in class and inexercisessuggestingtheiruse.Stagefive-DevelopmentThis stage suggests a level of maturity,wheretheconceptofleaderhasbeende-emphasised, purposes and activity aresettled with regular patterns ofbehaviour(Schwier,2001).Thereshouldbe regular peer-to-peer collaboration,and a broader set of artifacts produced,which suggest evidence of creation andcurationactivities.

AlgorithmfortherecommendersystemRecommend participant X’s artifact to Y if

• X’s artifacts have been regularly visited by Y

• A new artifact uses a tag or Blooms

partition that Y has recently searched on

• Y has uploaded an artifact with

similar tags to a new artifact For events over the last two weeks, sorted by most recent

Figure6.3-Recommendersystemalgorithm

143

TherecommendersystemandthelandingpageTagsassociatedwithartifacts,participant’sbrowsinghabitsandsearchactivitiescanbecombinedtocreatearecommendersystemwhichsuggestsitemsthatmaybeofinterest.Thisinformationisretrievedfromtheactivitydata,servingthedualpurposeofsource of information for the recommender system data and as data for theanalyticalpartofthiswork.Thereareanumberofwaysthatarecommendationcan be calculated – if a participant regularly visits another’s artifacts, it seemslikelythattheymaybeinterestedinnewartifactsfromthatperson.Ifasearchonaparticulartermhasoccurred,forexampleonaparticulartag,thenanyartifactswith that tag can be recommended. Finally, if a participant has uploaded anartifact which has tags that match another artifact, that artifact can berecommended (figure 6.3). Recommended artifacts are presented inchronologicalorderfrommostrecent,butonlyforresultsgeneratedoverthelasttwoweeks.Aprototypewrittenusing the cycle onedatademonstrated that atpeak moments, an overwhelming number of results could be returned, butlimitingresultsto14daysmadetheresultsaccessible.One of the participants from cycle one indicated concern over the nature oftracking taking place and this aligns with the literature which suggests thatmishandling trackingcanhaveseriousconsequences,withextremedisruptionspossibletothelearningprocess(Boyd,2008).Theguidingprincipalsusedhereemphasise transparency, so itwill bemade clear that a person’s activities arebeing tracked (Duval, 2011) andextended to include how theinformation will be used in therecommendation engine. It ishoped that any anxiety over thisissue would be dispelled byrevealing thenatureof the activityrecording that is taking place,during thebeginning of the courseand by showing the value for theparticipantsthemselvesinboththerecommender system and in thedataanalyticsthatarepossible.

Figure6.4–theredesignedfrontpanel

144

Folksonomy tag clouds can aid content indexing (Harvey, Baillie, Ruthven, &Elsweiler,2009),whichinthiscontextprovidesanotherwayforparticipantstoseethekeywordsattachedtorecentlycreatedartifacts.Thetagcloudplacedonthelandingpageisdesignedtoshowthetagsassociatedwithartifactsinthelast14days.Asistraditional,morefrequentlyusedtagsaredisplayedinlargerfontsizes,andclickingonatagrevealsaviewshowingartifactsassociatedwiththatkeyword in descending date order. Like the most recent uploads and thefolksonomytagcloud,artifactsthatarerecommendedarevisibleonthelandingpageforparticipants(figure6.4).Agauge stylebar indicates the totalnumberof artifacts and comments for thelogged in user (only). Here the analytics will act as an early warning system(Macfadyen & Dawson, 2010), and provide benefits by reflecting informationback for both the students and tutor (Bader-Natal & Lotze, 2011). Although itwasinitiallyplannedforbothgroupstousethesameindicators,itbecameeasierto code a summary analytic specifically for the tutor soon after the studentversionwasimplemented.Themethods,sampleandanalysisusedincycletwoareexplainedindetailinthenextsection.

6.4ThecycleThissectionbrieflyprovidesanoverviewoftheactivityinthecycleoverdistinctphases, dividing the cycle into three. This provides the context for the specificdatacollectedduringthecycle,whichispresentedhereandanalysedinthenextchapter.

WeekonetoweekthreeWeeks one to three cover database design and the use of the programminglanguageSQL,withdiscussionson the architectureof the internet. Followingageneral inductionand introductiontoe-Portfolios, thefirstweekfollowed levelone and two of the e-Portfolio moderator framework, with a much widerdemonstrationofthesystem.

145

The nature of artifacts, theway thatthe reflective statements can bewritten and the folksonomy andtaxonomysystemwereexploredwith(anonymous) examples from cycleone. Each participant was asked tocreate and upload artifacts in class,ensuring that every student hadexperienced the way that artifactcreation could be embedded in aworking practice. This initial dataallowed the recommender systemand ways in which others’ workcould be used to be demonstrated.Initial activities in the teachingmaterials contained closed exercisessuggesting distinct answers that were simplistic in nature and required littlediscussion. This promoted the idea of reificationwith short reflective text andopportunityfortagging(figure6.5).Studentsusedtagsfromthebeginning,withnudgestylecommentsusedtosuggesttheadvantagesofusingmoremeaningfulwordsorphrases.Therewasamorerapiduptakeintheuseofthefolksonomyfromthebeginning,comparedtocycleone.Activities from the secondweek onwardswere designed to facilitate level twoand three of themodel,with an increase in the number of open exercises andpromotionofcollaborationbetweentheparticipants.Thesecondsessionofweektwo introduced the notion of curation, with a mini research project wherestudentswereaskedtoseekandshareonlineresourcesonsubjectsrelevanttomaterialbeingcoveredinthemoreformaltaughtperiods.SmallsummationsandURLs linking towebpagesorvideos initiateddiscussionsaroundsubjectssuchasthe“control”oftheinternet.Week three sees all the participants using the system, with four sitting at theedgewithalowernumberofartifactuploadsandfewerconnectionstoaninnercore. Folksonomy tags become looser, less bound to the names of a series ofperceived exercises and more open, reflecting the nature of the artifactsencouragedinclass.Theintroductionofthefrontpageanalyticwasintroducedatthispoint,followingdiscussioninweektwoonitsnature.

Figure6.5–exampleartifactandcomments

146

WeekfourandfiveWeeks four and five cover the way that web pages are designed andimplemented as the front end and interface to the systems running over theinternetandthenmovesintotheuseofwebbasedprogramminglanguages.Thischangeof subjectmaterial sawadecrease inparticipation levels,withmanyoftheartifacts switching fromdatabasematerial toprogramming,with thewidervariety of open activities and solutions allowing more opportunities forparticipantstoofferadviceandcorrectionsoneachothers’artifacts.Discussionsattached to curation style artifacts continued around cutting edge webtechnologies,comparisonsofbrowsersandsoftwarerunningondifferenttypesofmobilephone.Theseweekssee theendof theapplicationof levels fourandfiveofthee-Portfoliomodel,withlessrequirementfortutordirection.

WeeksixtoeightThefinalthreeweeksofthecycleconcentrateonprogrammingmaterial,whereparticipants could write a wider range of artifacts encouraged with openexercises.Therewasanincreaseinthenumberofrequestsforhelp,specificallyconcernedwithdebuggingcode,whichwereansweredbothbythetutorandtheparticipants. In week seven the number of threads and discussions halvedcompared to thepreviousweek,whilst thenumberof artifacts and interactionwithothers’workincreasedafterthelowofweeksix.For the final week the number of discussions, the majority of which wereconcernedwithPHP issues, increased from7 to34,butwithadecrease in thevolume of artifacts being created and a lessening in the connectivity betweenparticipantswhowent directly to conversations theywere involvedwith fromthedashboard.

6.5ResultsThe following section details results derived from the cycle, including variousgraph measures, SNA diagrams and interview responses. The nature of thesemeasuresisfullyexplainedinchapterfour,andincycleonesection4.4.This is structured into three sections,with activity in the cycle described first,using summary tables, statistics, and social network activity diagrams. Theanalysis of the content and comments follow, using thematic analysis and theteachingpresence indicators fromthecommunityof inquirymodel.Finally, theinterviewresponsesarepresented,organisedbythemes.

147

ActivityinthecycleThe summary activity table (table 6.4) records the actions every participantperformsinsidethesystemingroupedcategories.Numbersinthetableindicatehowmanytimestheseeventsoccurredineachweekforallusersofthesystem,sodashboardnumbersareageneralindicatorofuse.There is a high level of activity in the first three weeks signalled by both thenumber of artifacts created (239 in week one, 169 in week two, 243 in weekthree), and by the level of interaction with others’ work (1090, 878, 1456 inweeksone,twoandthreerespectively).Thesenumbersstarttodecreaseinweekfour with lows in week five and six, before they climb in the last two weeks.Participantsuploadedatotalof1247artifactsovertheeight-weekperiod,givingthe following statistics - on average, a participant created four artifacts everythree days, reviewing their own work between three and four times a day,interactingwithothers’worksixtimesaday(table6.2).Table6.2–Generalstatistics

As incycleone,variousgeneralgraphmeasureshavebeencalculated fromtheactivity tables includingnumberof edges, numberof island, graphdensity andreciprocity,asexplainedinsection4.4.Numberof edges suggests connectionsbetweenparticipants and starts high inthe first threeweeks,beforedecreasing inweek fourdowntoasteady level inweeksfivetoseven.There isadropinthefinalweek.Numberof islands is thenumber of disconnected clusters, which when smaller is better as it in anindicatorofnon-interaction.Duringthecycle,thisnumberremainslow,althoughthe activity tables and sociograms reveal that it can be different participantsmovinginandoutofconnection.Thereisapeakvalueof five inthefinalweek(table6.3).

148

Table6.3-Generalgraphmeasures

Graph density indicates the proportion of connections present calculated fromtheratioofthenumberofedgestothenumberofpossibleedges.Highervaluessuggest more connections between nodes, zero indicates no connection, onesuggestsallnodesare connected toeachother.Thedensityhaspeakvalues inthefirstthreeweeksandthengraduallydecreases.Reciprocity defines the proportion ofmutual connections, in a directed graph,withzeroindicatingnomutualconnections,oneindicatingabalancebetweeninand out connections. Higher is better, indicating more balance between usingothers’workandaparticipant’sworkbeingused.Thisiscalculatedfromexistingtiesinthegraph,soactorswithnoconnectiondonotaffectthevalue.Therearestrongvaluesinweeksonetofour,butthereciprocitydecreasesinweeksfivetoeight.

Table6.4-Activitysummaryfortheeightweeksofcycletwo

150

Fordegree,betweennessandeigenvectorthevaluesinthetop20%(green)andbottom20%(red)havebeenhighlighted(table6.5,6.7and6.8).Table6.5-Degreevaluesforparticipants

Degree is the number of in/out links froma vertex (node).Higher numbers are better as they suggestmore connections areoccurring. Thismeasure does not count repeat links (i.e.multiple vertices to the same node count as one). Nicholas H. andHannahS.maintainahighdegreevalueovertheeightweeks;manyoftheothershavefluctuatingvalues.TristanC.has lowervaluesthantheothers(table6.5).Table6.6-Degreedistributionforparticipants

Degreedistributionsuggeststhenumberoflinksasaproportionofthetotal,henceforweekone6%oftheparticipantshadnoconnections,6%had1,3and5links.Highernumbersarebetter,indicatingmoreconnectionsbetweenmorepeople.Afterapeaknumberoflinksinweekthree,thenumberoflinksdecreases(table6.6).

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26Week.1 6% 6% 6% 6% 11% 6% 6% 22% 6% 6% 6% 6% 6% 6%Week.2 6% 6% 6% 6% 6% 6% 6% 6% 6% 11% 6% 11% 11% 6% 6%Week.3 6% 6% 6% 17% 11% 6% 11% 6% 6% 11% 6% 11%Week.4 11% 6% 11% 6% 22% 11% 6% 6% 6% 6% 11%Week.5 17% 6% 11% 17% 6% 6% 6% 11% 6% 17%Week.6 6% 11% 11% 6% 11% 11% 17% 6% 6% 11% 6%Week.7 6% 17% 6% 11% 6% 11% 17% 22% 6%Week.8 22% 6% 22% 22% 17% 6% 6%

Number.of.links

151

Table6.7-Betweennessfiguresforparticipants

Betweennesscentralitygivesahigherscoretoanodethatsitsontheshortestpathsofothernodepairs.Itsuggeststhosethatareoften foundat the intersectionsofmoredenselyconnectednetworkcommunities,where theyperformbrokeringrolesacrossclustersconnectingotherwisedisconnectedpeople.HannahandNicholashavestrongvaluesformanyoftheweeks,mirroringtheirdegreevalues,withmanyoftheotherparticipantshavingfluctuatingvalues(table6.7).Table6.8-Eigenvectorcentralityforparticipants

Eigenvectorcentralitygivesahigherscoretoanodeifitconnectstomanyhighscorenodesandsuggestshigherinfluencerswhodisseminateinformationquickly.Theydonotalwayshavethegreatestlocalinfluenceandmayhavelimitedbrokeringpotential,i.e.a lowerbetweennessvalue.Sixoftheparticipantshavehighvaluesforthreeweeksormore(CBa,GDa,HSu, JCo,NHa,andPDa),althoughthenumbersvaryfrequentlyoverthecycle(table6.8).

SNAdiagramsThe sociograms for the first four weeks show the high level of participation(figure6.6),withoneortwodisconnectedislandsineachweek. Weekone Weektwo

Weekthree Weekfour

Figure6.6-Sociogramsforweeksonetofour

BIS1112 Week 1

BAvBBr BHo

CBa

GDa

GJa

HSuJCo

JLa

KLl

MBr

NHa

PDa

RKa

SSc

SSi

SYo

TCa

BIS1112 Week 2

BAvBBr

BHoCBa

GDa

GJa

HSu

JCo

JLa

KLl

MBr

NHa

PDa

RKa

SScSSi

SYo

TCa

BIS1112 Week 3

BAv

BBr

BHo

CBa

GDa

GJa

HSu JCo

JLa

KLl

MBr

NHaPDa

RKa

SSc

SSi

SYo

TCa

BIS1112 Week 4

BAv BBr

BHo CBa

GDa

GJa

HSu JCo

JLa

KLl

MBr

NHa

PDa

RKa

SSc

SSiSYo

TCa

153

Weekfive Weeksix

Weekseven Weekeight

Figure6.7-Sociogramsforweeksfivetoeight

Thesociogramsforweeksfivetoeightrevealsthosehaveagreatercentralityandthosethatsitattheedgeofthenetwork,suchasBrianH.,SanjayS.andTristanC.(figure6.7).

BIS1112 Week 5

BAv

BBr

BHo

CBa

GDa

GJa

HSu

JCo

JLa

KLl

MBr

NHa

PDa

RKa

SSc

SSi

SYo

TCa

BIS1112 Week 6

BAv

BBr

BHo

CBa

GDaGJa

HSuJCo

JLa

KLl

MBr

NHaPDa

RKa

SSc

SSi SYo

TCa

BIS1112 Week 7

BAv

BBr

BHo

CBa

GDa

GJa

HSuJCo

JLa

KLl

MBr

NHa

PDaRKa

SSc

SSi

SYo

TCa

BIS1112 Week 8

BAv

BBr

BHo

CBa

GDa

GJa

HSu

JCo

JLaKLl

MBr

NHa

PDa

RKa

SSc

SSi

SYo

TCa

154

The next section presents the analysis of the artifact content and comments,using thematic analysis and the teaching presence indicators from thecommunityofinquirymodel.

AnalysisofartifactsandcommentsThereweremanymorecurationartifactscreatedincycletwo,includinglinkstoexternalweb pages, demonstrations of applications running onmobile phonesandlinkstovideosonYouTube.Thenumberofartifactscreatedwashighinthefirst threeweeks,withadecrease in themiddleof thecycleandan increase inactivityagaintowardstheend(table6.9).

Table6.9-Artifactcreationbyweek

From the beginning of the cycle, the majority of the artifacts created hadreflective statementsand tagsafterdemonstrations in the induction.Thesmallnumber of participants failing to do this started using them after nudge stylecommentstowardstheendofthefirstweek.Weeksone-threeThefirstweeksawasurgeinactivityfromfiveparticipants with 93 artifacts created in thefirsttwodays.The other participants started creatingartifactsafterthesecondsessionoftheweek,with fewer artifacts per day, but a greaterspread out over the next four days. Therewere 11 examples of supportive commentswhere participants offered suggestions or corrections in three threads,concerningprocess-howtoupload,makereflectivecommentsoroncorrectionsto theartifactsuploaded.Thetagsused in the firstweekarebalancedbetweenthosewithanappropriatesemanticmeaningalignedtotheartifactcontentandthose suggested by the ordering of the teaching materials, such as ex6, ex7(figure 6.8). Despite not formally introducing the curation process, fifteen

Figure6.8-Tagsforweekone

155

curation style artifacts were created by five students linking to the taughtmaterial invariousways, including links toexternalwebpagesandaYouTubevideo.The second week saw half the participants create artifacts in a steady flow,suggestingthattheyhadintegratedartifactcreationintotheirworkingpractices.Kevin, Sanjay and Susan are centrally based in the network, despite having alower number of published artifacts,indicatingthattheworktheycreatedearlyin the week was viewed and interactedwith more. A second set of five studentsuploadedworklaterintheweek,withthetime stamps suggesting that they werecollating work offline before uploading itallatonce.There are a strong set of links betweenmany of the participants, except forTristanwhoforthesecondweekhasalowparticipation level, creating ten artifacts.He failed to use tags or longer reflectivestatements,despitethetutor ‘nudges’andtheir use by the others. Both Brian andChris sit on the edge of the network,uploadingandinteractingless.There are a mixture of artifacts in week two, with taught material on webtechnologiesresultinginasurgeinworkpointingtovariouscomparisonsurveysand demonstrations. 47 curation style artifacts were created by participants,using external resources, YouTube videos,webpages and reviews (figure 6.9).Thecurationstyleartifactsweremorelikelytopromotecomments.

Figure6.9-Curationartifactsinweektwo

156

Themajorityof the remainingartifacts followed thepattern from thepreviousweekbydemonstratingsmall solutions toactivities,whichalignwith thesmallactivitiesandtaskssuggestedbytheteachingmaterial(figure6.10).

The thirdweek seesa change in subject, away fromdatabasework to creatingweb pages. All students interact, however four sit at the edge, with a lowernumberofartifactuploadsandfewerconnectionstoaprolificinnersetwhohavehigher participation levels. For Brian this represents a second week of lowparticipation, so noting this he was encouraged to bemore active in the nextweek.Fromthisweekthefolksonomytagsbecomelooser,lessboundtothenamesofaseriesofperceived exercises and more open,reflecting the more open nature of theartifactspostedandencouragedinclass.Thenumberof threadcommentspeaks inweektwo and three with participants offeringadviceandcorrectionsonothers’work.The breakdown of artifact creation by day(table 6.10), reveals a spread of activity

Figure6.10-Smalltasksolutionsasartifacts

Table6.10-Artifactcreationbyday

157

comparedtotheprevioustwoweeks,withthemajorityofparticipantscreatingwork on multiple days. Barbara B. breaks this pattern by having significantuploadsontheMondaymorning,“catchingup”beforeclass.Examples of artifacts fromweekthree includedtheuseof camera phones to createimages (figure 6.11),demonstrating standards inweb clients, prompting adiscussion whereparticipants compared testresults on various phonesandtablets.

Figure6.11-Curationartifactwithcomments

158

WeeksfourandfiveWeeksfourandfivecontinuethetransitionawayfromthedatabasematerialtomorecomplexwebdesignusingHTMLandCSS.Lectureandlabtimecoveredafullrangeofactivitiesandexercisesrangingfromsimplewebpagecreationtoacomplete web site redesign. As participants moved through the material, theartifacts reflect students catching up onwork from previousweeks (Chris B.),attempting simpler web design activities (Sanjay S., James L.), or showing theintermediate steps that were required tocompletea largerproject (PamelaD.,NicholasH.).The switch in subjects is reflected in the tagcloud,where thedescriptive labels startusingHTML and CSS terminology. These arecombined with tags that suggest sequencing(css4, css8), although these emerge from theparticipantsratherthanthetutor(figure6.12).Compared to the first threeweeks, the overall level of activity decreases overthese two weeks. The majority of participants create artifacts, but these arecreatedwith less distribution over theweek. Artifacts reflect longer andmorecomplexactivitiessuggestedbytheinclassmaterials.Thereisgreatervariabilityinparticipationlevelinthesetwoweeks.BrianH.wasencouragedtoincreasehisparticipationmoretowardstheendofweekthreeandrespondedby creating21 artifacts, although the analytics suggest that bothheand Chris had participated less, situating them on the edge of the community.Kevin L. continued his low level of interaction with others and created fewartifactscomparedtohisworkinweekthree.Thecreationofcurationstyleartifactscontinued,aligningwithmaterialcoveringthetheoryofwebdesignandbrowserarchitecture(figure6.13).Asbefore,theseweremorelikelytopromotecommentsanddiscussionontheircontent.

Figure6.12-Tagsforweekfour

159

Weekfiveintroduceswebprogramming,somanyoftheartifactsstarttoreflectthe smaller activities that are typical when starting a programming language.OtherparticipantscontinuepostingCSSartifactsfromthepreviousweek.Thereis a lower level of activity over theweek, after an initial surge of 57 artifactscreatedon thedayof the first session. Manyof the curationartifactswereonweb technologies attracting more comments than the programming artifacts,similar to week four. The timestamps and activities in class suggest morecompressionofartifactconstructionaroundthelabclassinthisperiod.MichaelB.andTristanC.weredisconnectedoutliersinweekfour,butincreasedtheiractivitylevelsslightlyinweekfivefollowinganudge.JimC.,BrianH.,JamesL.andKevinL.continuetheir lowlevelsofparticipationwithsmallnumbersofartifacts created. This separates them from a core group of seven participantswhofromthisweekonattend,postandinteractregularly.WeekssixtoeightWeeksixseesthemajorityoftheparticipantsfocus on problem solving activities inprogramming,withnocurationstyleartifactsbeing created. Chris B. and Brian H. havebursts of activity catching up on previousworks material, with Brian having minimalinteraction with others, whilst Chris B.interactsmore(table6.11).

Table6.11-Artifactcreationbyday

Figure6.13-Susancreatingartifactsusingbrowsertests

160

Tags used reflect semantics fromthe programming language e.g.uppercase, conditional, althoughPamela D. mechanically labels herartifacts using sequences (table6.14).Both Richard K. and Jim C. fail tocreate any artifacts this week,although they interactwith others’work in a similarpatternof activityto theirwork inweek five.Gillian J. and JamesL.haveminimal input,althoughtheir work is looked at by others. A core group of Hannah S., Chris B. andNicholasH.continueatightinteractionwitheachother.

Themultiplicity of the artifactsproduced decreases whendemonstrating smallerprogramming tasks comparedto the previous subject areas –typically a single artifact iscreated for a single activity,rather than many artifactsshowing a series of problemsolving stages. Some of theartifacts are framed with hintsand tips for the wider group(figure6.15).

Theconsequenceofaburstofactivityisareductioninthenumberofcommentsor views of thework – Chris B. creates 21 artifacts thisweek butmuch of hisworkisnot interactedwith.Thereisalsoa lackinvarietyinhistags,andtersereflectivestatements.Inweek seven (table6.12),manyof the artifacts are createdon thedayof thefirsttaughtsession,apatternwhichcontinuesonintothelastweekofthecycle.

Figure6.14-Tagcloudforweeksix

Figure6.15-Artifactframedwithhints

161

Jim C. creates 23 artifacts after little interaction or activity for nearly threeweeks.JamesL.createsartifactsovertwodaysseekinghelp,towhichtheonlinecommunityrespondswithsuggestions(figure6.16).The final week of the cycle seesmany of the participants continuingwith thecreationofartifactsafterthetaughtsessionwithlittleactivityfortherestoftheweek.Onlyfourparticipantscreateanyworkatlatertimes,withfiveofthemnotparticipating and Jim C. interacting with others but creating no new work.PamelaD., RichardK. and SusanY. createworkondifferent days, but of thesethree, only Richard K. interacts with others’ work. The remaining studentsreducetheironlineactivitytoaminimum,withmostoftheinteractionbeingatthestartoftheweek.ChrisB.hasalastsurgeinactivitycreating22artifactsattheendofthecycle.The next section presents the analysis of the teacher’s comments attached toeachartifact, analysedusing teachingpresencederived from the communityofinquirymodel.CodingcommentsforteachingpresenceForthefirstthirdofcycletwo,agreaterproportionofthetutorcommentsweredirected towards instructional design, organisation and facilitating discourse.This is tobeexpected in the initialdevelopingphase foranonline community,with the instructional design support evenly split between establishingnetiquette and utilising the medium effectively. Comments encouraging moremeaningfulreflectivestatementsandtagshavebeendesignatedasutilising themedium effectively, as the reflective statements would be useful for both theparticipantsandothers’reflectiononactionbylookingbackatownandothers’

Figure6.16-Artifactseekinghelp

Table6.12-Artifactcreationforweekseven

162

artifacts.Advisingabouttheuseofthesystemmoregenerally,suchasaskingforfieldstobefilled,hasbeenplacedunderthemoregeneralcategoryofnetiquette(table6.13).Table6.13-Codedteachingpresenceforweeksonetothree

Facilitatingdiscourseinthiscontextmostlyentailsdrawinginparticipantswithfurtherquestionsandthepromotionoflinks,encouragingstudentstolookatandcomment on others’ artifacts. Fewer comments fall in to the encouraging orreinforcingcategory.Instructional design in the middle third of the cycle drops away, with morecomments in other categories. The direct instruction codes show a variety ofcomments ranging from feedback, through to summarising and pointing toknowledgefromothersources.Thefacilitatingdiscoursecategory,whichiskeyto the networked learning philosophy, is used here to promote connectionsbetweentheparticipantsandtoreinforcetheworkdemonstrated(table6.14).

Table6.14-Codedteachingpresenceforweeksfourandfive

Thefinalthirdofthecycleindicatesadecreaseindiscoursecomments,althoughthis is a small drop. The comments from the second phase continue, with theincrease in participant activity offsetting the tutor’s role to some extent (table6.15).

163

Table6.15-Codedteachingpresenceforweeksfourandfive

Thedifferenceinfacilitatingdiscoursebetweenthethreepartsofthecycledoesnot seem tobe significantlydifferent, but theunderlying code reveal a greaternumber of comments designed to draw in the contributors in the first weeks,with a greater number of comments acknowledging and reinforcingcontributionsoverthelasttwothirdsofthecycle.Thenextsectionpresentstheinterviewresponses,fromelevenofthecycletwoparticipants.

InterviewresponsesResponsesinthissectionaregeneratedfromelevensemi-structuredinterviewsconducted after the end of the cycle. During the interviews, participants wereshowngraphicalrepresentationsoftheiractivityandaskedquestionsabouttheirwork, how they interacted with others and their perception of the group’sactivity as awhole. The process used to qualitatively analyse the responses isdetailedintheresearchdesign,section3.4.

Figure6.17-Activitybyweek,foranindividualparticipant

164

In theactivityrepresentationshowntoeachparticipant (figure6.17),activitiesaregroupedinto:

• interactionwiththedashboard/thelandingpage,whichgivesanindicationoftheamountofonlineactivity;

• interactionwithothers’work–viewing,searching,commenting;and• interactionwith‘own’work–creatingartifacts,viewing,searchingor

commentingonownwork.ArtifacttriggerParticipantswere free tochoosewhat theyuploadedasanartifact.The triggerfor uploadingwork varied, but for the programming parts of the curriculum aclearsuccessmomentinitiatedartifactcreation:

“solvetheproblemandthenuploadit”–Nicholas.“Ifsomethinggoesright,it'salwaysagoodone”–Steven.“ifitwassomethinghardthatIdidright!”–Susan.

Artifactscontainingcodewitherrorswerepostedwithrequestsforhelp:

“Whenitwasn'tworking,itwasatrigger.I'lluploadit.”–Susan.“ItwaseitherthingsIwasstuckonandcouldn'tfigureout"–Michael.

For the curationandbroaderbasedartifacts, itwas links tomaterials thathadbeen useful in their learning process, for their own and by extrapolationwhattheythoughtwouldbeusefulforothers:

“Forme,itwasjustuploadinganythingthatIlearnedoranything”–Tristan.“Iranthetestsonmyphoneandwhentheydidn'tworkthoughtitwouldbeinterestingtoseewhattheothersgot”–Chris.“IusedtheYouTubevideosonthebitsIgotstuckonandthoughttheymightbeusefulfortheothers”–Nicholas.

Fewcurationartifactswerecreatedtowardstheendofthecycle.FailinginpublicAll theparticipantsdeclared that theyhadno issueswithuploadingwork thatneeded help and could see the advantages of peer support when it came to

165

seeking assistance. Despite this, four of the participants did not take anyopportunity to upload work like this, with two saying that they preferred toresolve issues in class. When interviewed they suggested that upon reflectiontheywishedtheyhadacteddifferently:

“Ididn’tuploadthem,tobehonestIthinkIshouldhave”–TristanEnculturisationMost of the students looked at others’ artifacts to deduce what they shouldupload and how the reflective statement should be phrased. This alsoencouragedtheuseofthefolksonomytags,whichweresuccessfullyusedtotagartifacts“Irememberlookingatotherpeople’sstuffandsawthekey[words]”–Tristan.Theimmediateuseofthekeywordsforsearchingalsoencouragedtheiruse:

“Icouldseehowthekeywordscouldbeuseful”–Tristan.“otherpeopleputtingtextinencouragedmetodoit”–Susan.

However-thetaxonomylabellingfailed:

“Ididn’tseethepointofit”–Brian.“Itrieditbutcouldn’tseetherelevancetome”–Susan.“Igaveupafterawhile”–Pamela.

ActivitySeven of the participants suggested that the real-time presentation of artifactsencouraged more frequent regular working practices compared to othermodules,althoughthefrequencyofthisworkwasnotnecessarilyweekly:

“youhadtodoabitofworkeveryweektokeepup”–Hannah.“Ijustdividedit.Iwasaweekworking,aweeknot"–Chris.

As creating artifacts became part of their workflow, the peaks and troughsindicated by the activity graph do align with the work undertaken by theparticipants:

“Iwasreallyactiveoverthelasttwoweeks”[indicatesthepicture]-Hannah.

166

“IbecameregimentedinwhatIdo”–Pamela.The analytics make gaps in the activity pattern visible, although six of thestudentsdidnotidentifysmallergapsintheirownactivityuntilpresentedwiththeanalytics.Asidefromsuggesting itwasotherassignmentstakingtheirtime,theyfounditdifficulttoarticulatewhythedecreaseinactivitywasthere:

“Iwasathome,justeating”–Nicholas.Three of the participants with longer gaps were reluctant to be drawn into aconversationabouttheperiod:

“I can’t remember what I was doing during those weeks” [of inactivity] –Michael.

VisibilityAll thework in the e-Portfolio is visible to all. Participants had no issueswiththis,withtheadvantagesregardedasobvious:

“Icouldseeexamplesofhowit’sdoneproperly”–Tristan.“yes-Iknewwhotoask”[forhelp]–Kevin.“Itwasmoreopentohelpeveryone.Itwasveryhelpful”-Nicholas.

Oneparticipantclaimedthathewasn’tinterestedinothers’workandsaidthatitmadenodifferencetohisusageofthesystem,“evenifitdidbotheryou-youcanbrushitoffandjustseeyourownwork”–Chris.PerceptionofactivityStudentsthinkaboutworkloadandderivehowmuchworktheyshouldbedoingfromacomplexinterplayofsignalsfromthetutorandtheamountofworktheyperceive that others are doing. Making work in a trusted group is seen as amotivatingpositiveprocess:

“Ithinkitshowsmylearningprocess”–Pamela.“Ihadtoprovemyselfaswell–anobligation”–Nicholas.[Iworked]“becauseofthegeneralvolumeofwork”[ofothers]–Steven.“helpstellastory”–Tristan.

167

Despite interactingwithothers’work, theyhave littleperceptionofhowmuchtheirworkwasviewedorusedbytheothers.Participantssuggestthattheyhavean ideaofwhere theywouldsit in termsofactivitycompared toothersand inrelationtothewholegroup,althoughtheyarehesitanttorevealthisplacement.Nineparticipantspositionthemselvesroughlyinthecorrectthirdintermsoftop,middleorbottomlevelsofactivity.ParticipationandcommentingParticipantslikedthecollaboration,buttherewasanemphasisofquidproquo:

“Idon'treallymindsharingwhatIdo,aslongaseverybodyelseiswillingtodothesame,atleastoffercomments”–Sanjay.“Igotimmediatefeedbackwhichwasgood”–Chris.“I wasn’t sure whether other people would be interested in it or not” –Michael.

Interactingwithothersencouragedthefeelingofworkinginateam:

“Youjusthavethefeelingofteamworkabit”–Chris.“we'rethesameteam”–Gillian.

SearchingandlookingThe e-Portfolio provides a variety of ways to interact with others’ artifacts,includingusing links from thedashboard/landingpage, viewing recommendedartifacts or by participating in a comment chain attached to an artifact.Whenasked about navigating through the archive of artifacts, the participantssuggested that they had different ways of exploring the system, some bybrowsing backwards through time, others by keyword searching on thefolksonomysystem.Sevenoftheintervieweessuggestedthatacommonwaytofindartifactswasbysearchingthroughtheartifactsofanamedparticipant.Eightoftheparticipantscorrectlyidentifiedthemostprolificparticipantbyname,andlookedatherworkregularlysuggestingherdevelopingreputationandperceivedstrength:

“shewasalwaysfirsttopost”–GillianJ.“[shewas]prolific”–Steven.“it’s[their]reputationisn'tit”–Hannah.

168

Oneparticipantsaidthathavingthesamepeoplebeingthefirsttopostcanhaveanegativeeffect“itwasannoying”,althoughitmeant“herreports[artifacts]arequiteeasytofind”–Chris.TheideaofonlinereputationinfluencedGillian,“Ilookedforthestrongpeople”,whilstanotherclaimedthat“Iusedthemallfairlyequally”–Chris.Thenotionofstyle or usability was another factor, “I looked for people whose style I couldfollow”–Gillian.Afterthe initialactivityPamela, themostactiveparticipant,rarelysearchedforothers’workbecauseofherperceptionthatshewas“infront”oftheothers:

“alotofotherpeopledidn'thaveanythinguploadedonthosethingsyet,soIcouldn'tevenlookupandseewhatdidtheydo"

Despitethis,shereadilycommentedonothers’artifactsandrepliedtocommentsonherownwork.AnalyticsThree weeks into the cycle, the analytic was introduced onto the front pagerepresentingthenumberofartifactsandcommentsfortheloggedinuser,inthecontextofwholegroupactivitylevels.Mostsawthemeasureasapositiverepresentationoftheirownwork:

“Itkindofkeptmeontrack"–Tristan.“Itwasgoodbecauseitwasamotivationfactorforme.Inadifferentway”–Kevin.

Itdidnotintroduceacompetitiveaspectforthemajorityofdespitethemsharingthenumberwitheachother.Somestudentscomparednumbers:

“my friendaskedme, ‘Howmanyhaveyoudone?’ I said, ‘108’” [laughs], "Ihavedone40or45”–Hannah.“wetalkedabouttheline”–Steven.“wediscussedhowmanypostswereuploading”–Susan.

169

The analytic showing work in relation to the group as a whole was lesssatisfactory,asitresultedinrapidrecalculations:

“Ifounditabitannoying,tobehonest”–Chris.Pamelaexpressedlittleinterestinit:

“No,andIthinkwouldneverask,peopleneveraskmehowmanyartifacts,orhowmanyuploadsdoyouhave,andIwouldneveraskotherpeople,aboutanumber,becauseforme,itisnotonlyabouttheamountofstuffyouupload,butit'sthelearningprocess”

MonitoringandPrivacyDespite the explanation of the way in which activity was recorded in the e-Portfolioduring the induction, threeparticipantsweresurprisedat the levelofactivitydetailthatcouldbederivedfromthehistoryofinteractions:

“Mygoodness.You'vereallybeenkeepingtrackofalotofthings”–Steven.Although surprised, they did not seem to find this aspect objectionable. Fourparticipantsenquiredas tohowthe informationwasused inreal timetoshowothers’ work and would like to see more detail in the level of informationrecordedandthecalculationsusedintheanalytic.Thischapterhasdiscussedcycletwo,thechangesintroducedsincecycleone,theresultsofcycletwoandananalysisoftheinterviews.Thenextchapterdiscussestheseresultsandreflectsonthechangesthatcouldbetakenforwardintofurtherwork.

170

Chapter7 DiscussionandreflectiononcycletwoThis chapter discusses the second cycle in the context of the three researchquestionsandthenreflectsontheoutcomesfrombothcycles,withsuggestionsandimprovementsthatcanbemadeforthenextiteration.

7.1Discussion

Researchquestion1:Whatassessmentartifactsemergefromco-operatingparticipantsinalearningcommunity?As in cycle one, many of the artifacts created were in response to materialscreatedbythetutor,andconsistedofdemonstrations,solutionsandrequestsforhelp.Therewasasignificant increase in thenumberofartifactsandcommentscomparedtothefirstcycle,withagreaterlevelofengagementwiththeprocessindicated through both regularity of posting and content. Artifacts created incycletwofallintofourcategories:

• Artifactsrepresentingacompletedactivity,followingatutorsetexercise.• Artifactsdemonstratingtheapplicationoruseofatechniqueinitiatedby

participantsthemselves.• Artifactsaskingforhelporguidanceonanactivity.• Curationstyleartifactsrepresentingexternallearningresourcesthathave

beenofuse.Thereisanincreaseinthenumberofartifactsrequestinghelpordemonstratingcuration,althoughthemajorityofartifactsarestillcreatedinresponsetotutorset exercises and activities. In cycle one, little guidance was provided on thenatureofartifacts,whichinfluencedthetypesofartifactscreatedandresultedinaninitialhesitationtopublish.Thedirectguidanceprovidedduringtheinductionresultedinahighernumberofbothsolutionstyleartifactsandcurationartifacts,fromthebeginningofthecycle.Discussion threads were more likely to form around requests for help orcuration,althoughthereisadivideinthenatureoftheseasthecycleprogresses.Duringthefirstfiveweeks,threadsformedaroundcurationartifacts,withonlinediscussiononthewebsites,videosandnewsstoriesthathadbeenreferenced.Inthe final three weeks with the initiation of the programming material, thenumber of curation artifacts dropped to zero (table 7.1) and the discussionschangedtoaskingandansweringquestionsaboutprogramming.Manyof these

171

offeredformativecorrectivefeedbackindicatingtheusefulnessoferrors(Stefaniet al., 2007); two of the participants suggested in interview that the increasedcomplexity of the subject matter reduced their inclination to demonstratecurationalthoughotherscouldnotarticulatewhythisoccurred.The nature of curation was discussed during the induction and thendemonstratedingreaterdetailinthesecondweek.Thetimestampandcontentofthecurationartifactscreatedsuggestawider,moreingrainedengagementwithother sources of relevant educational material, which is also reflected in thediscussionsthatformedaroundtheseitems.Table7.1-Curationartifactproductionincycletwo

The usefulness of external resources to the rest of the community is a keydeterminingfactorwhenchoosingwhethertocreateacurationartifact:

“Ipostedthingsthatwereuseful”[toothers]–StevenS.Themajorityoftheseartifactscontainedclippingsandlinkstootherwebpagesthatwererelevanttothesubjectbeingdiscussedinclassatthetime.Sixof theartifactscontainedmultiplelinksandtextdescriptions,akintoaminiliteraturereview of useful links to a subject area. Other types of artifact demonstratedvarious web tests on tablets and mobile phones, which encouraged otherparticipantstorunthesametestontheirdevices.Thereisafartightercohesivenesstothegroupcomparedtocycleone,withthemajorityoftheparticipantsworkingthroughthesamesubjectstogether,visiblethroughthefolksonomytagcloudonthedashboard(figure7.1),andtheartifactsproduced.Therewerefewerorphanedcommentsincycletwoandwhentheydidoccurtheywereonartifactsoutofthegeneralsubjectflowofthecommunity,forexampleaquestionaboutwebdesigntwoweeksintotheprogrammingmaterial.The other instances of this occurred where the participant had created manyartifactsafteralullinactivity,forexample,Chrisinweekseven.

172

This module covers a wider range of materials compared to cycle one, whichhaveadifferentflowofartifactcreationdependingonthenatureofthesubjectand the teachingmaterial used. The introductory databasematerial hasmanysmallopportunitiesfordemonstratinglearning,whichresultsinlargernumbersofartifacts,postedmore frequently, coveringoneactivityperartifact.Thewebpagedesignand implementationmaterialhas longeractivities,whichresults inmanyartifactsshowingmorecomplexsnapshotsofasingleprojectinprogress.The three distinct phases of database, web and programming material wereintroduced in sequence and the flow ofartifactcreationchangeswhenthesubjectchanges, with a drop in activity, whichthenstartsto increaseagainasthenewersubjectprogresses.Themajorityoftheparticipantsintegratedthecreationofartifactsintotheirworkingpractices, revealed by the regularity oftheirpostingandtheartifactcreationtimeanddatestamps.Onlytwostudentsbulkuploadwork,andthiswasafterashortbreak in activity, typicallymeasured in days of “catch-up”work rather than inweeks.The analytics andartifacts in the e-Portfoliomake suchgaps in activityvisiblewhencircumstances,suchasotherassessments,reducethefocusonthismodule’swork.Participantscouldnot,orwouldnotarticulatewhythesegapsinactivityoccur.There is lessmechanical folksonomy labelling on artifacts in cycle two (figure7.1).Thetaxonomysystemwaspromotedandexplained in the inductionalongwith interface changes that embedded suggestions and help as artifacts werecreated.Despitethistheusagepatternswereverysimilartothoseincycleone,whereafterafewattempts,theuseofthetaxonomydroppedawayveryquickly.Similartocycleone,participantssuggestedthatthelackofanimmediateobviousvaluetotheirlearningprocesslimiteditsuse.Participantsrespondedpositivelytothechangesintroducedthroughthesteppedinductionprocess,byintegratingtheconstructionofartifactsintotheirworkingpractices,alongwith theuseof tagsand fuller reflectivestatements.There isarelationship between edge placement in the community and failing to attachinformationtotheartifacts,whichsuggeststhatalackofparticipationresultsin

Figure7.1-Richerfolksonomytags

173

feweropportunitiestoaccepttheritesandpracticesofthecommunity.Tristan,whowasslowtostartcreatingartifacts,isanedgeparticipantwhodidn’tfollowthe community in using tags and fuller reflective statements, despiteencouragementfromthetutor.

Researchquestion2:Howareartifactsshared,usedandreusedbythecommunity?Usingothers’workParticipantsincycletworeportednoissueswiththewayinwhichtheportfoliocommunitymadevisible their artifacts andparticipation levels.Thenumberofisolatednodes(islands),islessonaweekbyweekbasiscomparedtocycleone.In thiscase,however,participantsmove inandoutof isolationbetweenweeksfour toseven,so it is frequentlynot thesame inactivepersonsuggestedbytheislandcountmeasure,supportingtheimportanceoftriangulatingthisdatawiththesociograms.Thevaluesforreciprocityandgraphdensity(table7.2),highlighta change in the activity level that occurred at the switch in subject afterweekfour. The drop in reciprocity and density in week seven corresponds with anincreaseinthenumberofinteractionswithothers’work.Thisandthedatafromthe activity tables reveals that there were particularly popular artifacts beingviewedandinteractedwith.Searchrecordsandtheinterviewssuggestthatthiswasdrivenbyaperceptionofexpertiseoftheauthor.Table7.2–Density,reciprocityvalueswithartifactcreation

Even though the activity patterns are different between the cycleswith largernumber of artifacts and comments produced in cycle two, averaged individualactivity isaround thesame levelascycleone,with individuals lookingbackontheir ownwork four times adayand interactingwithothers’work six times aday (table7.3).The significant increase in the interactionwithothers’ artifacts(table7.4)resultedfrommanyoftheartifactbeingpresentedonthedashboard,suggestingachangeinthewaythatworkwasdiscovered.

174

Table7.3–Activityandartifactscreatedincycleoneandtwo

Participantswhositattheedgeofthenetworkbecauseofalowlevelofactivityor interactivity, typicallybulkupload,althoughthiswasnot to thesameextentshownincycleone.Theyoftenhadquestionsorqueriesleftdangling,suggestingthat the notion of quid pro quo and visibility of a participant’s activity was acontributingfactorastothelikelihoodthataquestionwouldgetaresponse.Insome cases this may also be because of their out of sequence uploading,compared to the flow of the rest of the community. The majority of theparticipants regarded the visibility of others’ work as a positive motivatingfactor, but two said that they feltobligated to creatework and that they couldthenseewhentheyfellbehind,despitethefactthatparticipantswerenotgivenguidance about frequency of posting. For many of the participants, theperceptionofalevelofrequiredworkinducedaregularitytotheirposting“youhadtodoabitofworkeveryweektokeepup”–Hannah.There were fewer islands and consistently higher degree values in cycle two,which indicates better connectedness, popularity and influence. Thoseparticipants identified in the interviews as beingmost influential in the grouphavehighereigenvectorcentralityvalues(Pamela,HannahandNicholas),whichis a measure of how well a person disseminates information, through thepopularityof theirartifacts.Participantsacknowledged themotivatingeffectofseeingothers’ artifacts, and in thevisibilityof thegroup’s levelof activity.Theshorter closed style activities at the beginning of the cycle provided solutionsthatworkcouldbecheckedagainst,whereas themorecomplexopenexercisesfrom subsequent weeks, provided hints, suggestions and opportunities fordiscussion.Thosethatwerefirsttopostineachweekusedothers’workless,astherewouldnotbeanyartifactsofrelevanceatthattime.Thereisthepossibilityofanegative

175

effectassociatedwithbecomingthefrequent‘firsttopost’participant,whichwasdescribedasannoyingbyonestudent.Table7.4–Interactionwithothers’work

Studentswithhigherlevelsofartifactcreationaremorelikelytobecentraltothenetworkastheyhavehighparticipationlevels.Analysingactorplacementineachsociogramrevealspatternsofparticipationwhereparticipantswithhigherlevelsof activity are centrally placed; those with lower levels are on the periphery.Combininginteractionlevelswithnumberofartifactsproducedbyweek(figure7.2), confirms a relationship between these two. In cycle one actors on theperiphery generally created one or two artifacts. This trend can be observedhere,althoughthehigherlevelsofoverallactivitymeanthatparticipantscanbeperipheralwithahighernumberofartifacts(forexample3,9,5and16).

Figure7.2–Sociogramforweekthreewithoverlaidartifactcreation

levels

There are occasional outliers,with larger volumes of artifact creation and lowlevelsof interaction,whichcansignalaparticipant returning toactivityaftera

BIS1112 Week 3

4

22

0

33

3

7

9 28

5

16

6

2220

9

20

6

24

9

176

gap,forexampleJimC.with28artifacts.Lullsinparticipationinthiscycletendtobeshorterthaninthefirstcycleandlastfivetotendays.Thesociogramscanbeusedtodeducethenumberoftimesthateachparticipantisontheperipheryofactivity(table7.5),whichhasbeenusedtocategorisethestudentsintothreebandsofactivity,afterperipheral,activeandintenselevelsofparticipation(Wenger,2002):

• Highperipheralactivity–morelikelytobeplacedattheedge,fivetoeighttimesovertheeightweeks.

• Mediumperipheralactivity–occasionallyattheedge,threetofourtimesovertheeightweeks.

• Lowperipheralactivity–infrequentlyattheedge,zero,oneortwotimes.This categorisation corresponds with the eigenvector centrality score; highperipheral activityplaces theparticipantsat the lowerendof the scaleas theyare less likely to connect to higher influencers. A red zero indicates noparticipation and no artifact creation, which is shown as an island on thesociogram. At the extreme ends sit Tristan (TCA) andNicholas (NHA); Tristanhas fourweeksof noparticipationor artifact creation,whilstNicholas activelyparticipateseveryweekandisneverontheperiphery.Pamela(PDa)iscorrectlyidentifiedasoneofthemostactiveparticipantsbytheothers,althoughshewason the periphery inweek fourwhere she created five artifacts and interactedwithothers’workless.

177

Table7.5–Artifactcreationlevelsforparticipantsontheperiphery

Low levels of artifact creation by peripheral participants can be seen from thenumbers of artifacts produced compared to the total; for example, the eightperipheral students in week three produced 54 artifacts, with the other 189producedbymoreactivecentralactors.Thisclearly indicatesthatthoseontheperipheryarecreatingfewerartifacts.Participants engage with feedback in a number of forms with comments onartifactsproviding formativeopportunities for reflection; the visible reificationprocessamongstthepeerssuggests levelsofappropriateactivity; thenatureoftheartifactsindicatestheoveralltypesofactivitiesthatarebeingfollowed.Theactivitiesforstudentsinsideeachbandofperipheralitycanbecalculated,shownas a raw score and as a ratio to the number of students in each group, so forexample,onaverage45artifactsperstudentwerecreatedinweeksonetoeightbythosewithhighperipheralparticipation(table7.6).

178

Table7.6–Activityoftheparticipantsbyband

The pattern of activity for participants interacting with their own work is aswouldbeexpected,withthoseontheperipheryuploadingandreflectingontheirownworkhalfas frequentlyas those in themiddle.Veryactivestudents in thelowcategoryhavethehighestvaluesforboththisandfordashboarduse.Inmostmodules the levelofworkothersaredoingcanbedifficult toperceiveandstudentshavelittleabilitytogaugetheirownactivitycomparedtothis.Here,all participants use the overviewmechanism equally indicated by the view allartifacts (other) value. This allows others’activitylevelstobededuced,“Youcouldgeta perception of who was actually doingloadsofwork.”–Kevin.Participants with medium levels ofparticipation search and view others’ workfarmore frequently that those on the edgeor with high activity levels. Highly activeparticipantswhocreatedworkearlyineachweek,were less likelyto interactwithothersastherewouldn’tbeanyartifactson the subjects they were working on, which aligns with Pamela’s interviewcomments.

Table7.7–Searchpopularityvsperipherality

179

Actorswithhighlevelsofartifactcreationtendtositmorecentrally,developanonlinereputationandhaveworkviewedmore frequently.Similar tostudiesbyCho (2002) and Beck (2003), participants are more influential if they arecentrallyplacedandaremorelikelytobesearchedforbyname. Thefivemostfrequentlysearchedforpeople,aremorecentrallylocatedactors,whiletheleastsearchedaremorelikelytobeontheperiphery(table7.7).Plottingperipheryplacementandnumbersofartifacts created reveals that the relationshipis complex as prolifically creating artifactsdoes not guarantee high levels of centrality(table7.8).Manyof those frequently citedasdemonstrating expertise had higher levels ofartifact creation, but were cited in theinterviews because of their regular early intheweekposting.Thegreaternumberofcomments,questionandparticipationincycletwoofferedmore opportunities for knowledge sharing and cascade, with the participantstendingtomorecloselyfollowpathsthroughthesamesubjectareasatthesametime. Analysisoftheartifactcontentbytime(figure7.3)revealsthealignmentbetween the participant’s work, withmany of them producing artifacts in thesamesubjectareasatthesametime.

Figure7.3–Artifactcreationforsubjectsovertime

Researchquestion3:Whatistheroleofthetutorandtheformofthecommunity?This sectionbreaks the researchquestiondown into threeparts, analysing theroleof the tutoras teacherprovidingdirect instruction,and tutoras facilitator

Table7.8Artifactcreationvsperipherality

180

shaping and forming the community. This is followed by a discussion on thenatureofthelearningcommunitypresentincycletwo.TutorasteacherDespite a greater amount of time demonstrating and explaining the nature ofartifacts during the stepped induction process, some of the participants stillneededtobeguided ‘incycle’during the first fewweeks inattachingreflectivestatements and tags, similar to behaviour in cycle one. Participants had a freechoice in thenature and frequency of artifact production, but theywere takenthroughtheprocessofartifactcreation inclass,alongwith theway inwhich itcould be embedded inside working practices, suggesting the advantages ofcreatingastheyprogressed.Theactivitytablesandartifactscreatedindicatethatthissucceeded,withthemajorityoftheparticipantsregularlypostingcomparedtocycleone.Encouraging folio-thinking was done here by explicitly demonstrating theapproach in class, with participants following along and publishing their ownartifacts, embedding the creation of artifacts in an active learning approach.Particular activities were also designed to show the advantages of postingquestionsandqueries in thee-Portfolio, byaskingparticipants to commentonartifactexamplesrequiringhelporwitherrors.Followingonfromthesuggestedpatternofactivityinthee-Portfoliomoderationmodel the first set of exercises were short, tended to be closed in style, haddefinitiveanswersandweredesignedtoencourageparticipation.Curationstyleartifactswere fostered through theuseofdiscussion topics,whichparticipantsrespondedtobycreating links toYouTubevideos, links tootherresourcesandreadinglists.Different subject areas create opportunities for different styles of activities,whichhavedifferentratesofartifactcreation.Exampleincluded:

Closedshortactivities

WriteaquerytofindthefirstnamesIncreaseeveryone’ssalaryby10%CreateadatabasetablewithparticularcolumnsPrint“helloworld”

181

Openactivities

DesignandimplementatabletorecordinformationonemployeesWritedemonstrationprogramstoexplaintheuseoffourfunctionsofyourwonchoosingCreateawebpagetodisplayyourresultsCreateawebformandresponsepagetoprinttheresultsCurationstyleactivities

WhatarethefeaturesofthenewHTMLstandard?IstheinfrastructureoftheInternetcontrolled?

Thetutorhastoblendthese,attemptingtobalancetheopportunitiesforcreatingdifferent artifacts across the teaching period. Initial closed, short activities aremorelikelytoresultinoneartifactperactivity,butenableanintroductiontotheprocessofreification.Graduallysegueingintoopenactivities,canresultinfewerartifactsbeingcreated,butthesetendtobemultipleartifactscoveringthesameprojectmaterial, reflecting the activities larger size and scope. In subject areasthat can be more exploratory in nature, such as design or programming, thedifferent solutions possible prompted discussions of best practice andopportunitiesforqueriestobesolvedbypeers.Curationstyleactivitiesaremorelikely to promote discussion and engagementwith external resources, but thishasaprerequisite that theparticipantshaveappropriateresearchingskillsandtools.The importance of attachingmeta-data to created artifactswas emphasised intheinductionandsuggestedasappropriatenetiquettebythetutoronlineinthefirst twoweeks.Manyof theparticipants responded to the tutor commentsbytaggingandaddingreflectivetexts,butdespitethis,twoofthestudentsdidnotaddreflectivestatementsregularly.Promoting theuseof tagsresulted inrapidadoption,butasincycleone,thetutoradvocatingtheuseofthetaxonomyfailedtoengagethestudents.Asinthepreviouscycle,theoriginalintentwasforthetutortocreateartifactsinapeerroleandtherewasmoresuccessinthiscycle.Mostoftheseartifactswerecuration artifacts, adding content by using artifacts that linked to externalmaterial, with some offering part solutions on the more complex parts of theprogrammingmaterial.

182

TutorasfacilitatorParticipants inonlinecommunitiesdevelopbehavioursover timeandthe tutorcan significantly impact on these by their presence and responses, directlyaffecting the density of networks (Martınez, Dimitriadis, Rubia, Gómez, & laFuente,2003).Participantswere informedthat thepaceandnatureofartifactsthey created was of their own choosing, but the facilitator role was used toencourageactivityandparticularbehaviours,suchasregularposting.Forthosestudentsworkingoutofthegeneral flowofthecommunity, thetutorhastoensurethatcommentsandrequestsforhelparespottedandaddressedbylinking to other artifacts or by transitioning the question onto another artifactthat would be relevant. The switch from web design to programming wasfragmented,which required extra vigilance in the tutor rolewhenparticipantstackledthesubjectoutofsyncwiththerestoftheclassorwhoweresatattheedgeof thenetwork(MichaelB.), resulting in fewerresponses to theirqueries.There were also parts of the cycle where participants sat at the edge of thenetworkwithminimalactivity;manyoftheparticipantsrespondedtotutorhintsor tips here but there were occasions when they stepped back from creatingworkandtutorcommentsonthishadlittleimmediateeffect(JimandGerry).The analytics system was used to provide relevant information on individualparticipation, alongwithgeneral levelsof communityactivity.Thereare issueswith the abstractions that are required to create this information, as activitygraphshavetimeboundarieswhichmaynotcorrelatewiththeworkingpracticesofthestudents.Astudentmayelecttocreateartifactseveryotherweek,ormayworkoutof thegeneral flowof theotherstudentsbychoice,whichmaycreateinitialconcerninthetutor.Participantsmayregardthisastheirnormalworkingpractice and may be reluctant on being drawn on gaps in this activity assuggestedintheinterviewresponses.The initialdesign for theanalyticsystemusedapeerbasedphilosophy, in thateveryparticipant includingthetutorsawthesameinterface.Participantscouldonly see information on their own performance in the context of the wholegroup, without any ranking information. Activity and discussions in theinterviewssuggestedthatthisapproachwasaccepted, “Itkeptmeontrack”–Tristan.

183

Although students expressed a desire to see how the analytics were beinggenerated. Using the same analytic for the tutor role revealed too littleinformation about the whole group activity, so a new ‘tutor’ perspective wascreatedtoenableedgecasestobemoreeasilyspotted.The importanceofdeterminingpossibleartifactconstructionopportunities isakey design role, allowing participants freedom but creating more directedsuggestions for those who find the process initially difficult falls into thefacilitationrole.TheanalyticsandinterviewresponsesforKevinL. indicatethehigherlevelsofsupportherequiredfromthetutor:

“IcomefromaveryminimalexperienceinIT.ThefirstdayitwasabitOK,whatamIgoingtodo,Ipanicked...Ididn'thaveanycluewhenIstarted.SoIlearnedalotfromothers”

Hewasalsotheonlystudentwhohadnotcollaboratedwiththeothersbeforeashewas taking themodule as an option from another course. He attributes hislower activity to both factors, followedby the switch in subject just as hewasstartingtocreateartifactsinthewebdesignmaterial.The combination of sitting at the edge of participation and being out of thegeneral flow of the community suggests high risk students where tutorfacilitationshouldbefocussed,astheseparticipantsarelesslikelytohavetheirwork looked at and integrated into the use of the community as a whole. Asparticipants with weak social ties are less likely to share resources (Dawson,2008),thetutorgoaloffacilitatingthetransitionfromweaktostrongsocialtiesisvital.ThelearningcommunityTheoptimalversionof the learningcommunity innetworked learningsuggeststhat participants have a greater autonomy in the subject areas that they coverand thateach individualdetermines theirownpath through the field.SituatingNLinthetraditionaluniversitysetting,wherecourseshaveprescribedlearningoutcomesisatoddswiththisidealwherethetraditionalroleoftutorasteacher,introducesmaterialseachweekusingavarietyof‘delivery’mechanisms.Here, participants were allowed tomake the decision about the nature of theartifactsthattheywishedtoproduceandtherateatwhichtheyproducedthem,

184

whichenablesthemtomoveoffanotionalprescribedpath.Boud(2006)arguesthatconstructivealignmentisastrategythatisunrepresentativeofthetypesofunstructured learning that occurs in lifelong post academy experiences,whereoutcomesarerarelyexplicitlyspecified.Here,participantswereallowedtomakethedecisionaboutthenatureoftheartifactsthattheywishedtoproduceandtherate at which they produced them, after the equifinality community model(Pedler, 1981). Despite this freedom, the nature of the artifacts produced areheavilyinfluencedbythetaughtsubject;thenatureofexercisesandactivitiesintheteachingmaterials;andbyotherartifactsproducedbythegroup.The regularity of posting is influenced by the subject and the activity of thegroup, which develops a rhythm of participation, as is typical of groups in acommunity context (Wenger, 2002). Prompting the initial creation of artifactsduring the extended induction resulted in the majority of the participantscreatingartifacts at the same time.As the cycleprogressed thegeneral flowofactivity followed on from the tutor introduced materials, with short delaysbetweenthematerialbeingintroducedandtheartifactcreationbeginning.Thismore regular posting continued through the cycle, with follow up individualactivitymeasuredindaysratherthaninweeks.Participatinginthecommunitywasseenasapositivewaytosharepractice,anditwasacknowledgedthatthiswasawaytogetvalidationoftheirownwork.Alevelofquidproquowaspresent:

“ifIcommentonothers’workmaybethey’llhelpme”–Susan.Inthesecondhalfofthecoursepeerfeedbackwasmorelikelytoprovidesimpleelaboration, particularly where artifacts were created indicating issues orproblems. The tutor has to provide more sophisticated formative feedback toalterforwarddirection,forexampleusingartifactsorcommentstosuggestareasforfurtherexploration.Despitethepeer-baseddesign,artifactsproducedbythetutor were viewed many more times than those from students. Sociogramssuggestthatdespiteattemptstoreducethe ‘presence’ofthetutor, itwasstillaroleregardedwithsignificanceandfrequentlyendedupcentrallyplaced.Networked learning emphasises the promotion of links, between users andbetween users to resources. The suggestion that connections have to bepromotedpromptsthequestionofwhowillbesuggestingthelinksandensuring

185

thatallparticipantshaveanequalityofopportunity.Asnotedbycriticalstances,thepossibilitiesofexclusionandbullyingarejustaspossibleinonlinescenarios.Despite emphasising a peer based community, the tutor and facilitator has tostand apart to some extent, promoting the connections and policing thecommunity. Although many of the connections were suggested through thedesignof the e-Portfolio system, therewere instanceswheredirect actionwasrequired, suchas toaddressorphanedcommentsor toencourage thoseon theperiphery.Another critique of community models exists in the possibility of conformitythroughcommunityconsensusandpeerpressure(Hodgson&Reynolds,2005).Althoughsomeofthepersonalandcollectivehistoryofacommunitycanbeseenintheartifactswhichperpetuatetherepertoiresofcommunitypractices,thereisthe possibility that conformity may repress a wider set of artifacts. It is alsopossiblethatamalformednotionmaybecomeregularpracticeifunregulated,forexample a poor style or an inefficient implementation in coding. Open sourceadvocates would suggest that “all bugs are shallow to many eyes” but thisrequires participants who are prepared to speak out, which may run againstcommunity pressure. It is possible that a recommender system built on aparticipant’swork and activity history could reinforce these perspectives, onlyshowing artifacts that mirror the artifacts that the participant has createdthemselves, analogous to the filter bubble possible in search engine results(Pariser,2011).Thesefactorstogetherhighlighttheimportanceoftheguideonthesideactingoutsidethepeerrole,bothtoredirectincorrectpathsthroughthelearningcommunityandtosuggestalternativesanddifferentiationpossibilities.

7.2ReflectionThe three significant changes introduced in cycle two were the use of themoderator framework, the recommender system and the more detaileddashboard.Themoderatorframeworkincreasedparticipationandtheregularityofposting,with although the highly structured induction may have delayed theopportunities for the evolution of customs and practices. Despite having welldefined layers of activity, different participants progressed at differing ratesmaking the formal application of the framework in the teaching materialsunnecessary for some.Using cycleoneexamples in the induction for cycle twoworkedwell, although some of the contextsweremissing due to the differentsyllabus. Allowing participants flexibility in artifact creation allows for an

186

individual’s ‘continuum of learning’ to be visible to the tutor in real time. Thelearning progression can then be influenced by informed action, throughformative feedbackbythetutor, thepeersandbytheactivityof thegroupasawhole.The analytics suggest that more artifacts were viewed after visiting theredesigneddashboard,withparticipantsclickingthroughfromthetagcloudandtherecommendedartifacts.Akeythemethatcameoutofanalysingparticipantinterviews was that they would like to know more details on the underlyingmechanicsusedhere,whichwouldrequire:

• awaytoexaminewhyanartifactwasrecommended,• howtheanalyticsonthedashboardwerecalculated,and• theabilitytoseewhowasusingtheirwork.

Bothinclassandduringinterview,participantsexpressedsurpriseoverthelevelofactivitydetail thatwasrecorded,despitetheeffort thatwasmadetoexplainthis incycletwo.Havingtheabilitytotrace intorecommendationsandanalyticcalculationsshouldmitigatethistosomeextent.Here the in-cycleactivitywasperceivedbyexporting rawactivity tables intoaspreadsheet, using ad-hoc calculations. Real time generation of the networkmeasures and the sociograms would help with monitoring the activity in thecycleandthemechanismsproducedwillenablethisprocesstobeautomated.Thegraphmeasuresshowninchaptersixwerecalculatedinasimilarfashiontothose incycleone.Thegeneralmeasuresareuseful indeducingbroadlevelsofactivity, butmaking comparisons to other cycles is difficult andwould requirelike-to-like comparisons where the subjects taught were the same and in thesamesequence.Asincycleone,betweennesscentralityseemslessmeaningfulinthiscontext,whereaseigenvectorcentrallyisusefulasitsuggestsdisseminationi.e.artifactsharingacrossthenetwork.Thismeasuresupportstheevidenceseenin the thematic analysisof theartifactsand interviews,where thehigh scoringparticipants are creating and sharing artifacts regularly (table 6.8). Degreedistribution is a usefulmeasure in networked learning, if success ismeasuredthrough the number of connections being promoted between the learners. Itrevealstheoverallpatternofactivity,butneedstobecombinedwithananalysisofthenatureoftheconnectionandtheartifactcontenttobemeaningful.

187

The next chapter summarises these findings against the research questions,alongwitha reflectionon theuseandsuccessofusingopensourceandactionresearch.

188

Chapter8 Discussion,conclusionsandfurtherworkThis chapter will summarise the findings against the research questions anddiscuss the implications for others wishing to use the portfolio framework intheir own contexts. This is followed by a reflection on the experience ofconducting the study to identify the contribution, limitations of the work andsuggestpotentialareasforfurtherresearch.

8.1AddressingtheresearchquestionsTheoverarchingquestionwas:Whatwouldacommunitybasedportfoliobasedonnetworkedlearningprincipleslooklikeifdevelopedinco-operationwiththelearners? The three research questions discuss the nature of the artifactsproduced;howtheyareusedbytherestofthegroup;andthenatureofthetutorroleandthecommunity.Research question 1:What assessment artifacts emerge from co-operatingparticipantsinalearningcommunity?Analysis of the artifacts and interview responses suggest that artifactrepresentationswith image, reflective text and folksonomy tags hasworked inboth cycles, with participants finding the mechanics of reificationstraightforward.Wherelittleinitialguidanceorstructureisprovided,asincycleone, artifacts tend to be created in response to a perception that tutor setactivitieshavetobecompleted,limitingtheirrange.Theyarealsolikelytoonlyshow correct work, emphasising preconceptions in the nature of assessmentpractices.Categorisingartifactssuggesttheyfallintothesetypes:

• Artifactsrepresentingasolutiontoacompletedactivity,followingatutorsetexercise.

• Artifacts demonstrating a technique thought useful to the group as awhole.

• Artifactsaskingforhelporguidance.• Curationstyleartifacts.

Networkanalysisandparticipantfeedbacksuggestthataninductioncontainingastepped introduction and examples to demonstrate the advantages of peerfeedback will increase initial participation. This needs to be combined withactivitiesthatbroadenthetypeofartifactsthatcanbedemonstrated,toensurethatparticipantsintegratefolio-thinkingintotheirworkingpractices.Thisisalso

189

required to promote the value in the idea that mistakes, broken code andcuration style artifacts can be shared with a broader community. Differingsubjectareascanproducedifferent flowsofartifactcreation,dependingon thenatureof theexercisesandactivities thatareusedduringthe learningprocess.Shorter, closedstyleactivitieswithdistinctanswersencourageoneartifactperexercise; longerprojectscreateopportunities formanyartifacts, showingworkin progress. Curation artifacts can provide a view of engagement and use ofexternalresources,or,asincycleone,thelackofit.Discussionthreadsaremorelikely to form around curation artifacts and where participants are seekingassistancefromthecommunity.With a degree of initial guidance, participants engage with tagging artifacts,resultinginanemergentsharedvocabularywithnamesandphrasesthatcloselyalignwith amovement through various subject areas. The analytics show thatfolksonomytagsareregardedasavaluablewaytosearchandsortartifactsandparticipantsdeemeditusefulwhenaskedaboutitininterviews.Participants who have irregular patterns of work aremore likely to have lessinnovativeartifactsinthetypestheyproduce,andtypicallyproducesolutionstoexamplesandexercisesinamechanicalfashion.Thereflectivestatementusedintheseartifactsarelikelytobeshorterandlessdetailed,suggestinga“catch-up”processandlackofengagementwiththeideaofcommunitysupport.Research question 2: How are artifacts, shared, used and reused by acommunity?Studentsarewilling to shareartifacts ina collaborative fashionand toprovidefeedbackandcommentonothers’work,withanunderstandingofquidproquo.Incycleonewhere littleguidancewasprovidedonthenatureofartifactstobeproduced,manyoftheparticipantsusedothers’artifactsasasuggestionastotheworkthattheyshouldbeproducingreinforcingthenotionthat tutorsetsworkthathastobecompleted.The analytics and patterns of activity show that generally, others’ work wasviewed more often than a participant’s own work, providing solutions tocommonproblems,opportunitiesfordiscussionandasuggestionofthelevelofwork to be produced. Most suggest that the visibility of artifacts can bemotivating, but it may produce an obligation to work, which is less positivelyviewed.

190

The way that knowledge can cascade through a community can be tracedthrough viewing patterns and successive artifacts. Tag clouds, recommendedartifactsandrecentworkdisplayedonthedashboardorgatewaytotheportfoliosuccessfully promote sharing and reuse, increasing the number of connectionsbetweenlearnersandthelearningresources.More popular artifacts are associated with participants who post early andregularly;theseparticipantstendtooccupyacentralpositioninthecommunityand have higher levels of activity. Those in the middle activity band tend tosearch and reflect muchmore than those on the edge or those that aremorecentrallyplaced.Participantswhopost firstandregularlycanfind itdifficult touse thecommunitymeaningfully,as therewon’tbeothers’worktoreviewandcollaborateon.Thereisarelationshipbetweenartifactcreationandactivity,withthosesittingontheedgeofthecommunitytypicallyproducingfewerartifacts.Ifcombinedwith irregular reification, thecommunitymay fail toprovidehelporguidancesothetutormayhavetointervene.Research question 3: What is the role of the tutor and the form of thecommunity?The lecturer has a multiplicity of roles, acting as learning designer; tutorfacilitatingonlinebehaviour;andteacherprovidingdirectinstructiononline.

As demonstrated in cycle two, activities and exercises in the learningmaterialformthebasis formanyof theartifacts.Thetutor in the learningdesignerrolehas to carefully construct these so as to allowaprogression fromsmall closedactivitiestowardsmoreopenactivitiesthatallowforgreaterdifferentiationandpeersharing.Amoretightlyspecifiedinductionsuggestingmoreregularactivityhashereresultedinmoreoftheparticipantscreatingartifactsaroundthesamesubjectsatthesametime,whichisimportantasoutofflowactivitymayresultinlesssupportfromthecommunity.

Using networked learning as an underlying philosophy requires the tutor toensurethereareopportunitiesforconnectionstoform,whichcanbeachievedbydirectingparticipants toothers’ artifacts, rather thanbydirect instruction.Theconstruction andnature of curation style artifacts canbedemonstratedby thetutor,butencouragingthecommunitytopostandrespondtorequests forhelpmayrequireextratutorvigilanceasparticipantswithregularoutlierplacementmay have difficulty getting responses to their questions or activities. Outliers

191

withlowerlevelofparticipationcaneasilybeidentifiedinthissystemasactivityisvisible,butthereisthepossibilitythatparticipantsareworking‘offline’iftheyhavenotintegratedartifactconstructionintotheirworkingpractices.

Both cycles show that participant’s who create artifacts that lack reflectivestatements ormeaningful tags can be nudged into attaching appropriatemetainformation,whichisalsohelpedwhenitdevelopsasacustomandpracticebytheotherparticipantsinthecommunity.

Analytics can provide valuable information about both community growth andindividualactivity,butthishastousedcarefullyasaperceptionofsurveillancedemotivatedaparticipant in cycleone, resulting in a change inbehaviour.Theevidence presented here suggests that a carefully constructed induction andvisibleadvantages intheuseofthedata inarecommendationsystemcanallaytheseworries.

Thereisapositiverelationshipbetweenartifactproductionandoverallactivity,with those producing work regularly having higher visibility, more centralplacementinthecommunityandasuggestionofexpertise inthefield.Creatingartifactsearlyandregularlyisadeterminingfactorinthegroup’sperceptionofproficiency.Ifastudentisregularlythefirsttopost,therecanbeafalloffinthatperson’sperceptionof theusefulnessofparticipationas thereare fewartifactsavailableonthesamematerial.Hereitcanfalltothetutortomaintainlevelsofengagement by direct instruction, setting extra work; or bymanually creatingconnectionsbyaskingparticipantstocheck,orhelpoutonsomeoneelse’swork.

The original intentionwas for activity and implementation inside the portfoliolearningcommunitytobefullypeerbased,withtheinteractionsandinterfaceforthe tutor to be the same as that of the student participants. In practice thefacilitation role required both actions and reporting tools to ensure the initialgrowthandmonitoringofthecommunity.

8.2ImplicationsforpracticeThe e-Portfolio and framework allow for the progression of participants in alearning community to be made visible, both for the tutor and the members.Whenreproducingthisactivity,caremustbetakentoensurethatthedesignofthelearningmaterialsreflectopportunitiesfordifferentkindsofactivity,whicharesuitableforsharingandencouragepeercollaboration.

Analyticsareausefulindicatorofactivity,butthereareanumberofissuestobe

192

awareofwhenusingtheminatutorrole:

• Participants have to integrate artifact production into their workingprocessesforthereporteddatatobemeaningful.

• Theweekly (or otherwise) cycle of activitywillmark out start and endpointsforanalyticcalculationsthathavetomatchwithparticipant’scycle.

• Anaturalrhythmofparticipationwilldevelop,butthiswillbeaffectedbythe nature of the activities in the teaching materials and the groupbehaviour.

• Overuse or lack of information about how analytics are calculatedmayraiseconcernsamongsttheparticipants.

Thewaythat teachingmaterialsareprovidedandthenatureof the interactiondirectlyeffects thenatureof theartifactsprovidedby thestudents. If the tutoractsaslecturer,theparticipantsaremorelikelytorespondinatraditionalway,supplyingartifactsthattheyperceiveassolutions.Ifanattemptismadetobreakthe learning process out of this pattern so the participants are encouraged totakemore control of their own learning, thedepth andvariety of artifactswillincrease.

8.3ReflectionsontheresearchandlimitationsoftheapproachThere are well defined guides to the nature of summative assessment innetworked learning (Goodyear, Jones, Asensio, Hodgson, & Steeples, 2001),typically emphasising a constructive approach where students have a greaterdetermination in the nature of the assessment and peer feedback is used tofeedbackontheinitialplan,thedraftandthemarkingscheme(McConnell,2006).Thenatureofthetutorandthefeedbackprocessesinthesecommunitieshasalsobeenthoroughlyresearched;thecommunityofpracticemodeldefinesthenatureof the expert and the relationship to novices; the community of inquirymodeldemonstratesbestpracticesandallowsananalysisof the interactionsbetweenthesegroups.What has been missing is an exploration of the practical mechanisms offormative feedback in the NL context. The different form of e-Portfolio shownhere provides further evidence that richer formative representations arepossiblecomparedtosummativemeasures(Yorke,2005)andhasprovedtobeaneffectivewayofencouragingtheseresponsesinnetworkedlearning.Thekeycharacteristic of formative feedback is that of a path, where students becomeowners of their own learning and forward directions are influenced by

193

interactions with peers and the tutor (Wiliam, 2011). Wenger’s conception ofartifacts enables a rich depiction of this path, where the negotiation -participationprocessfostersreflectionandprogression(Romer,2002).Allowingparticipantsflexibilityinartifactcreationaftertheequifinalitycommunitymodel(Pedler,1981)allowsforanindividual’s ‘continuumoflearning’tobevisibletothe tutor in real time, enabling the learning progression to be influenced byinformed action. The positioning of where the learner is and where they aregoingisakeyaspectinformativeassessment(Black&Wiliam,2009b),andthee-Portfoliomakesthis‘location’visibletoboththeparticipantsandthetutor.Formative feedback is typically categorised into verification and elaboration,whereverificationisasimplecommentonthevalidityofthework(Shute,2008).Thepopularityofanartifactprovidesimplicitverificationfromthecommunityasawhole;commentsinsidethee-Portfoliofall intotheelaborationcategorywithrapid and frequent replies providing multi-layered responses and guidance,suggestingvaliditythroughimprovedaction(Harlen&James,1997),discussingerrors,providingguidanceandpromotingconnections.The tutor feedbackrolecan be shared out amongst participants by using links and directions to otherartifactsiffacilitatedthroughastructuredinductionprocess.Participants engage with other forms of feedback inside the e-Portfoliocommunity with the visible reification process amongst the peers suggestinglevels of appropriate activity; the nature of the artifacts indicates the overalltypesofactivitiesthatarebeingfollowedbythegroupasawhole;participantstendtoaligntheirlearningbyseeingthenatureoftheworkbeingcreatedbythecommunity,evenifitisoutofsequencewiththetutordriveninteractions(figure7.3,p.179).Thisresearchfocussesonanarrowfieldofstudy,wherethereareopportunitiesforpracticalskillsandtheoreticaltopicstobeblendedinthelearningdesignandthrough that into the artifact construction process. The standard critiques ofactionresearchapplyhere;thereareinherentissuesinthelecturerresearchingtheirownpracticeasdiscussed in theresearchdesignchapter. It ishoped thatthere is an appropriate level of detail here to support Heikkinen’s qualityindicators(2007)suchasanevocativeaccountandworkablepractices.Aligningactionresearchwithanopensourcephilosophycanwork,butonly inparticular domains where there are particular technical skill sets in evidence.

194

Therecanbedifferencesbetweencodegeneratedinaneducationalenvironmentandproduction code; codewhich is readable in class and suitable for teachingand learningmaynotbescalableandsafe touse inaproductionenvironment.The rapid creation and implementation required to adjust the software as thecycleprogressescanalsobechallenging,particularlyaschangesaremadelivetothesoftwareinuse.

Itisalsopossiblethatprocessesoractivitiesthatseemdesirable,maytakealongtimetoimplementandthenfailinpractice,forexamplethegraphicaltaxonomicrepresentations. It is the nature of open source software for features to bedevelopedthatsubsequentlyfailduetolackofdemand,butthiscanbedifficultwheretherearelimitedresourcesandthetutorissimultaneouslydeveloper.

Therecommendationsystemisusedtocreateconnectionsbetweenthelearnersand the electronic resources (artifacts), supporting the networked learningphilosophy.Thereareissueswithsuchsystems;thereexiststhepossibilitythatonlyartifactsfromalimitedsetarereturned,reflectingartifactsfromanarrowselectionthatmatchandreinforcetheparticipant’sownviewsandworksimilartothe‘filterbubble’thathasbeenidentifiedasanissueinnewssitesandsearchengines. To counter this, it is important to add a degree of randomness torecommendationresults,acknowledgingtheadvantagesofdesignedserendipity(Acosta,2012;Saadatmand&Kumpulainen,2013).A more recent, potentially unethical experiment was conducted by Facebook,where the behaviours of participantswere influenced by returning positive ornegativenewsstoriestoseparateclassesofusers(Kramer,Guillory,&Hancock,2014).Facebookdoesnotallowuserstoseethealgorithmthatsuggestsstories,sothismanipulationwasonlydiscoveredwhenitwasannouncedinanacademicjournal. To counter possibilities of this, recommendation systems should beexaminableinplace,thatis,itshouldbepossibletoquerythemechanicsofwhyanitemwasrecommended.Despite the fact that the use of analytics was directly integrated and used toreflectworkback to the students, itwasonly in thepost reflectiveperiod thatstudents expressed surprise over the degree of tracking possible, reflecting awidertrendofgeneralpublicdisinterest.Itisonlywhendirectlyconfrontedwiththe evidence of what is recorded that participants acknowledged the level ofactivity detail possible. It is not widely understood that every interaction inonline educational systems are being stored and are available for analysis,

195

perhapsbecausetheresultsfromtheseinteractionsarenotcurrentlyanalysedorusedinsidemostacademicinstitutions.

Analytical information has to be used cautiously, as without a rich picture ofparticipant activity, incorrect conclusions could be drawn. De Laat’s (2006b)original mixed methods framework reflects the complex nature of praxis thatexists in NL. Similarly, the use of analytics to create sociograms for real-timecommunitymonitoringisvalid,butaricherpictureaboutthedepthofindividualparticipation requires it to be mixed with detailed content from actualinteractions.

8.4ContributionandsuggestionsoffurtherresearchThe implicit assumption at the heart of networked learning is that promotinginteraction between participants and learning resources will result in betteroutcomes. This work supports that claim and demonstrates that networkedlearning can be the community pedagogy used in the design of a learningcommunitycentredaroundane-Portfolio.

Thisworkhasalsoproducedane-Portfolio frameworkconsistingofaseriesofstagesthatcanbefollowedtobuildacommunitywhereartifactconstructionisintegratedintotheworkingpracticesofthemembers.

Campbell and Debloise (2007) raise a number of concerns about the use ofanalytics in teaching and learning projects; concerns over the notion of bigbrother tracking, the simplification of complex holistic situations tomathematicalrepresentations,privacyandtheuseofprofiling.Thisworkshowsthat theseconcernsarereal,but that theanxietiesassociatedwith these issuescanbealleviatedbydemonstrating thevalueof thesecalculationsbyreflectingthembacktotheparticipantsinrealtimeandbyhavinganopenapproachaboutwherethisinformationisused.

The source code for the project will be placed under a full open sourcecommunity licence (GPL3) so that others may use, refine and improve thesoftware.

Furtherwork

As indicated in the limitations section, the use of the portfolio framework hasbeendemonstratedinaparticularsubjectbaseddomain,wherethenatureofthelearning taking place is technical and practical, with lower numbers ofparticipants. Further work could use the portfolio in a different subject area

196

and/orwith largernumbersofparticipants.Thenextstudywill takethemodelintoamarketingsubjectarea,wherethenatureoftheself-producedartifactswilllikelybeadifferentform.

After an institutional initiative to refresh the curriculumand course structuresacross Kingston University, the portfolio use has been embedded in thedocumentation for theMScBusiness Internet Technology course. This is beingused as a major differentiator in the course marketing and should give theopportunity to investigate the possibilities for a larger number of participantsacrossawiderrangeofsubjects.

Currently theartifacts,commentsandanalyticsareautomaticallyexported intoelectronic documents at the end of the process to satisfy institutionalrequirements for portfolios of evidence to be visible for external examiners. Itwould be valuable for participants to have amore direct hand in this process,selecting their own work in some way. It should be possible for a group tocollaborativelyconstructtheirownlearningoutcomesretrospectively,andthenpin artifacts to them to demonstrate individual learning. This may be anopportunity torevisit theuseofa taxonomies thatallow for levelsorstagesoflearningtobedemonstrated.

There are examples of deep analysis of reflective statements to suggest thenature of the learning taking place, typically in e-Portfolio systems where thedelineationbetweenartifact entries is not as clear ashere (Jenson, 2011).Theaddition of such an analytical framemay prove useful in improving the tutorsunderstandingofparticipantprogressionduringthelifeofaportfoliopractice.

Thereissomeindicationthatprogrammingstyle,whichcanbedifficulttoteachthrough tutor led demonstrations may be better disseminated through goodpracticeamongstthepeersinsuchacollaborativesystem,suggestingthatstyleisimplicitknowledgebetterlearntthroughacommunitybasedpedagogy.Itwouldbestraightforwardtoinvestigatethisthroughachangetolearningmaterialsandartifactanalysisovertime.

Finally,thisworkwasdrivenbyadesiretoimprovepractice.Itishopedthatbyplacingthepractices,processesande-Portfolioprototypeinthepublicdomain,itwillbeusedtofurtherthatgoal.

197

ReferencesAcosta,A.(2012).Usingserendipitytoadvanceknowledgebuildingactivities.

Presentedatthe16thAnnualKnowledgeBuildingSummerInstitute.RetrievedAugust20,2015fromhttp://ikit.org/SummerInstitute2012/

Acosta,T.,&Lui,Y.(2006).ePortfolios-beyondassessment.InA.Jafari&C.Kaufman(Eds.),HandbookofResearchonePortfolios(pp.15–23).Hershey,PA:IdeaGroupPublishing.

Adam,S.(2002).TheuseofEuropeancredits(ECTS)torecordpriorlearning(APEL)-acontributiontotheBolognaprocessandthecreationoftheEuropeanhighereducationareaby2010.RetrievedAugust20,2014,fromhttp://webworld.unesco.org/gcexhibition2003/121/content/ebook/Chapter_10/2365.html0,5

Adomavicius,G.,&Tuzhilin,A.(2005).Towardthenextgenerationofrecommendersystems:asurveyofthestate-of-the-artandpossibleextensions.IEEETransactionsonKnowledgeandDataEngineering,17(6),734–749.

Altbach,P.G.,Reisberg,L.,&Rumbley,L.(2010).TrendsinGlobalHigherEducation.Paris:SensePublishers.

An,H.,Shin,S.,&Lim,K.(2009).Theeffectsofdifferentinstructorfacilitationapproachesonstudents’interactionsduringasynchronousonlinediscussions.Computers&Education,53(3),749–760.

Anderson,B.,&Simpson,M.(2007).Ethicalissuesinonlineeducation.OpenLearning:TheJournalofOpenandDistanceLearning,22(2),129–138.

Anderson,L.W.,Krathwohl,D.R.,Airasian,P.W.,Cruikshank,K.A.,Mayer,R.E.,&Pintrich,P.R.(2000).ATaxonomyforLearning,Teaching,andAssessing:ARevisionofBloom'sTaxonomyofEducationalObjectives.NewYork:Pearson.

Anderson,T.,Liam,R.,Garrison,D.R.,&Archer,W.(2001).Assessingteacherpresenceinacomputerconferencingcontext.JournalofAsynchronousLearningNetworks,5(2),1–17.

Andrade,H.G.(1997).Understandingrubrics.EducationalLeadership,54(4),14–17.

Annand,D.(2011).Socialpresencewithinthecommunityofinquiryframework.InternationalReviewofResearchinOpenandDistanceLearning,12(5),40–56.

Arbaugh,J.B.,&Hwang,A.(2006).Does“teachingpresence”existinonlineMBAcourses?TheInternetandHigherEducation,9(1),9–21.

Ashford-Rowe,K.A.S.C.(2013).AqualitativeanalysisofanLMSusagebystaff(pp.1–10).PresentedattheTHETA13conference.Retrievedfrom

198

http://eprints.utas.edu.au/16511/Ashworth,P.,Bannister,P.,&Thorne,P.(1997).Guiltyinwhoseeyes?University

students'perceptionsofcheatingandplagiarisminacademicworkandassessment.StudiesinHigherEducation,22(2),187–203.

Attwell,G.(2008,August21).Whatisthedifferencebetweenane-PortfolioandaPersonal

LearningEnvironment?www.pontydysgu.org.RetrievedSeptember16,2014,fromhttp://www.pontydysgu.org/2008/04/what-is-the-difference-between-an-e-

portfolio-and-a-personal-learning-environment/Bader-Natal,A.,&Lotze,T.(2011).Evolvingalearninganalyticsplatform(pp.

180–185).ProceedingsoftheLAK'11:InternationalConferenceonLearningAnalyticsandKnowledge,Alberta,Canada.

Baggs,R.,&Wu,C.W.(2010).Explorationofasynchronousonlineteachingwithanetworkanalysisofclassinteraction(pp.355–362).ProceedingsoftheIEEESecondInternationalConferenceonSocialComputing,Minneapolis,MN.

Balabanovic,M.,&Shoham,Y.(1997).Fab:content-based,collaborativerecommendation.CommunicationsoftheACM,40(3),66–72.

Barrett,H.C.(1998).Strategicquestions:whattoconsiderwhenplanningforelectronicportfolios.Learning&LeadingwithTechnology,26(2),6–13.

Barrett,H.C.(2005).Whitepaper:researchingelectronicportfoliosandlearnerengagement.RetrievedJune.RetrievedMarch15,2013,fromhttps://www.helenbarrett.com/reflect/whitepaper.pdf

Barrett,H.C.(2006).Usingelectronicportfoliosforformative/classroom-basedassessment.ClassroomConnectConnectedNewsletter,13(2),4–6.

Barrett,H.C.(2008).MultiplepurposesofdigitalstoriesinePortfolios.RetrievedAugust20,2014,fromelectronicportfolios.com/portfolios/SITE2008DSpaper.pdf

Barrett,H.C.(2010).BalancingthetwofacesofePortfolios.Educação,Formação&Tecnologias,3(1),6–14.

Barrett,H.C.,&Carney,J.(2005).Conflictingparadigmsandcompetingpurposesinelectronicportfoliodevelopment.TaskStream.RetrievedMay12,2013,fromhttp://helenbarrett.com/portfolios/LEAJournal-BarrettCarney.pdf

Barrett,H.C.,&Wilkerson,J.(2006,March18).Conflictingparadigmsinelectronicportfolioapproaches.electronicportfolios.org.RetrievedJanuary18,2014,fromhttp://electronicportfolios.org/systems/paradigms.html

Batson,T.(2002).Theelectronicportfolioboom:what'sitallabout.CampusTechnology.RetrievedAugust25,2013,from

199

http://www.msmc.la.edu/include/learning_resources/emerging_technologies/eportfolio/ePortfolio_boom.pdf

Batson,T.(2011).Situatedlearning:atheoreticalframetoguidetransformationalchangeusingelectronicportfoliotechnology.InternationalJournalofePortfolio,1(1),107–114.

Beagrie,N.(2006).Digitalcurationforscience,digitallibraries,andindividuals.InternationalJournalofDigitalCuration,1(1),3–16.

Bean,W.(2005).PartI:Whyrubrics?peta.edu.au.RetrievedAugust27,2008,fromhttp://www.peta.edu.au/mediaLibrary/documents/pens/PEN148.pdf

Beauvoir,P.(2010).Thepersonallearningenvironmentsblog.Cetisprojectwebsite.RetrievedOctober9,2014,fromhttp://zope.cetis.ac.uk/members/ple/

Beck,R.J.,Fitzgerald,W.J.,&Pauksztat,B.(2003).Individualbehaviorsandsocialstructureinthedevelopmentofcommunicationnetworksofself-organizingonlinediscussiongroups.InB.Wasson,S.Ludvigsen,&U.Hoppe(Eds.),DesigningforChangeinNetworkedLearningEnvironments(pp.313–322).Dordrecht:SpringerNetherlands.

Beetham,H.,&Sharpe,R.(2007).RethinkingPedagogyforaDigitalAge:DesigningandDeliveringE-learning.NewYork:Routledge.

BenKDaniel,McCalla,G.I.,&Schwier,R.A.(2008).SocialNetworkAnalysistechniques:implicationsforinformationandknowledgesharinginvirtuallearningcommunities.InternationalJournalofAdvancedMediaandCommunication,2(1),20–34.

Bereiter,C.(2002).Artifacts,canons,andtheprogressofpedagogy:aresponsetocontributors.InB.Smith(Ed.),Liberaleducationinaknowledgesociety(pp.223–244).Chicago,IL:OpenCourt.

Bereiter,C.(2005).EducationandMindintheKnowledgeAge.NewYork:Routledge.

Bereiter,C.,&Scardamalia,M.(2005).BeyondBloom’staxonomy:rethinkingknowledgefortheknowledgeage.InM.Fullan(Ed.),TheInternationalHandbookofEducationalChange(pp.5–22).Dordrecht:KluwerAcademicPublishers.

Bhattacharya,M.,&Hartnett,M.(2007).E-portfolioassessmentinhighereducation(pp.19–24).ProceedingsoftheIEEEFrontiersinEducationConference,Milwaukee,WI.

Biggs,J.(1996).Enhancingteachingthroughconstructivealignment.HigherEducation,32(3),347–364.

Biggs,J.(2014,April9).ConstructiveAlignment.johnbiggs.com.au.Retrieved

200

April9,2014,fromhttp://www.johnbiggs.com.au/academic/constructive-alignment/

Biggs,J.,&Tang,C.(2010).TeachingforQualityLearningatUniversity(3rded.).NewYork:SocietyforResearchintoHigherEducation&OpenUniversityPress.

Black,P.,&Wiliam,D.(1998).Assessmentandclassroomlearning.AssessmentinEducation:Principles,PolicyandPractice,5(1),7–74.

Black,P.,&Wiliam,D.(2009).Developingthetheoryofformativeassessment.EducationalAssessment,EvaluationandAccountability,21(1),5–31.

Blair,C.H.(2011).Panicandplagiarism:authorshipandacademicdishonestyinaremixculture.MediaTropes,2(1),159–192.

Blignaut,S.,Blignaut,S.,&Trollip,S.R.(2003).Developingataxonomyoffacultyparticipationinasynchronouslearningenvironments—anexploratoryinvestigation.Computers&Education,41(2),149–172.

Boog,B.W.M.(2003).Theemancipatorycharacterofactionresearch,itshistoryandthepresentstateoftheart.JournalofCommunity&AppliedSocialPsychology,13(6),426–438.

Bostock,S.(2000).Studentpeerassessment.LearningTechnology.RetrievedAugust27,2008,fromhttp://www.reading.ac.uk/web/FILES/engageinassessment/Student_peer_assessment_-_Stephen_Bostock.pdf

Boud,D.J.(1995).Assessmentandlearning:contradictoryorcomplementary.InP.Knight(Ed.),AssessmentforLearninginHigherEducation(pp.35–48).London:KoganPress.

Boud,D.J.(2000).Sustainableassessment:rethinkingassessmentforthelearningsociety.StudiesinContinuingEducation,22(2),151–167.

Boud,D.J.,&Falchikov,N.(2006).Aligningassessmentwithlong-termlearning.Assessment&EvaluationinHigherEducation,31(4),399–413.

Boud,D.J.,&Walker,D.(1998).Promotingreflectioninprofessionalcourses:thechallengeofcontext.StudiesinHigherEducation,23(2),191–206.

Boud,D.J.,Cohen,R.,&Sampson,J.(2001).Peerlearningandassessment.Assessment&EvaluationinHigherEducation,24(4),413–426.

Boud,D.J.,Keogh,R.,&Walker,D.(1985).Reflection:TurningExperienceIntoLearning.Philadelphia:KoganPage.

Boyatzis,R.E.(1998).TransformingQualitativeInformation.London:SAGE.Boyd,D.(2008).Facebook'sprivacytrainwreck:exposure,invasion,andsocial

Convergence.Convergence:TheInternationalJournalofResearchintoNewMediaTechnologies,14(1),13–20.

201

Boylan,M.(2010a).Ecologiesofparticipationinschoolclassrooms.TeachingandTeacherEducation,26(1),61–70.

Boylan,M.(2010b).Schoolclassrooms:communitiesofpracticeorecologiesofpractice.TeachingandTeacherEducation,26(1),61–70.

Brackley,A.(2013,August16).Whateverhappenedtohybridmanagers?Theshorthistoryofthehybrids.BCSArchive.RetrievedAugust16,2013,fromhttp://itnow.oxfordjournals.org/content/38/6/22.short

Braun,V.,&Clarke,V.(2006).Usingthematicanalysisinpsychology.QualitativeResearchinPsychology,3(2),77–101.

Brookfield,S.(1983).Adultlearners,adulteducationandthecommunity.MiltonKeynes:OpenUniversityPress.

Brown,A.L.(1992).Designexperiments:theoreticalandmethodologicalchallengesincreatingcomplexinterventionsinclassroomsettings.JournaloftheLearningSciences,2(2),141–178.

Brown,J.S.,&Duguid,P.(1991).Organizationallearningandcommunities-of-practice:towardaunifiedviewofworking,learning,andinnovation.OrganizationScience,2(1),40–57.

Bruns,A.,&Humphreys,S.(2005).Wikisinteachingandassessment:theM/Cyclopediaproject(pp.25–32).ProceedingsoftheInternationalWikiSymposium,SanDiego.

Bryant,L.H.,&Chittum,J.R.(2013).ePortfolioeffectiveness:A(nill-fated)searchforempiricalsupport.InternationalJournalofePortfolio,3(2),189–198.

Butler,P.(2006).Areviewoftheliteratureonportfoliosandelectronicportfolios.Techreport,1–23.MasseyUniversityCollegeofEducation.Retrievedfromhttp://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.175.4678

Cambridge,D.R.(2006).IntegralePortfoliointeroperabilitywiththeIMSePortfoliospecification.InA.Jafari&C.Kaufman(Eds.),HandbookofResearchonePortfolios(pp.234–246).Hershey,PA:IdeaGroupPublishing.

Cambridge,D.R.(2009).Twofacesofintegrativelearningonline.InD.R.Cambridge,B.L.Cambridge,&K.B.Yancey(Eds.),ElectronicPortfolios2.0(pp.41–49).Sterling:StylusPublishers.

Cambridge,D.R.,&Perez-Lopez,K.(2012).Firststepstowardsasociallearninganalyticsforonlinecommunitiesofpracticeforeducators(pp.69–72).ProceedingsoftheLAK'12:InternationalConferenceonLearningAnalyticsandKnowledge,Vancouver.

Campbell,J.P.,DeBlois,P.B.,&Oblinger,D.G.(2007).Academicanalytics:anew

202

toolforanewera.EducauseReview.RetrievedSeptember17,2013,fromhttp://www.educause.edu/library/erm0742

Carmean,C.,&Christie,A.(2006).ePortfolios:constructingmeaningacross,time,spaceandcurriculum.InA.Jafari&C.Kaufman(Eds.),HandbookofResearchonePortfolios(pp.33–43).Hershey,PA:IdeaGroupPublishing.

Carney,J.(2002).Theintimaciesofelectronicportfolios:confrontingpreserviceteachers'personalrevelationdilemma(pp.544–548).ProceedingsoftheSocietyforInformationTechnologyTeacherEducationInternationalConference,Nashville,Tennessee,USA.

Carr,W.,&Kemmis,S.(2003).BecomingCritical:EducationKnowledgeandActionResearch.London:Taylor&Francis.

Carter,J.(1999).Collaborationorplagiarism:whathappenswhenstudentsworktogether.ACMSIGCHIBulletin,31(3),52–55.

Chang,C.K.,Chen,G.D.,&Li,L.Y.(2008).Constructingacommunityofpracticetoimprovecourseworkactivity.Computers&Education,50(1),235–247.

Chang,C.-C.,&Tseng,K.-H.(2009).Useandperformancesofweb-basedportfolioassessment.BritishJournalofEducationalTechnology,40(2),358–370.

Chau,J.(2010).ePortfolio,technology,andlearning:arealitycheck.JournalofInteractiveLearningResearch,21(4),465–482.

Chen,H.L.,&Mazow,C.(2002,June16).ElectronicLearningPortfoliosandStudentAffairs.RetrievedApril18,2014,fromhttp://www.naspa.org/netresults/article.cfm?ID=825

Chen,H.L.,Cannon,D.,Gabrio,J.,&Leifer,L.(2005).Usingwikisandweblogstosupportreflectivelearninginanintroductoryengineeringdesigncourse(pp.95–105).ProceedingsoftheHumanBehaviourinDesignConference'05,Melbourne.

Cho,H.,Stefanone,M.,&Gay,G.(2002).SocialnetworkanalysisofinformationsharingnetworksinaCSCLcommunity(pp.43–50).ProceedingsoftheConferenceonComputerSupportforCollaborativeLearning,Hillsdate,NJ.

Choi,J.H.,&Strobel,J.(2012).Validationofamodifiedinstrument:usingsocialnetworkanalysistomeasuresocialpresence(pp.1–23).ProceedingsoftheAnnualMeetingoftheAmericanEducationalResearchAssociation,Vancouver.

Clark,J.E.,&Eynon,B.(2009).E-portfoliosat2.0-surveyingthefield.PeerReview,11(1),18–23.

Clow,D.(2012).Thelearninganalyticscycle:closingtheloopeffectively(pp.134–138).ProceedingsoftheLAK'12:InternationalConferenceonLearningAnalyticsandKnowledge,Vancouver,BC.

203

Clow,D.(2013).Anoverviewoflearninganalytics.TeachinginHigherEducation,18(6),683–695.

Coghlan,D.,&Brannick,T.(2009).DoingActionResearchinYourOwnOrganization.London:SAGE.

Cohen,P.L.,Manion,L.,&Morrison,K.(2008).ActionResearch.InResearchMethodsinEducation(6ed.,pp.297–313).London:Routledge.

Conole,G.(2010).Theoryandmethodologyinnetworkedlearning.RetrievedMay20,2010,fromhttp://www.slideshare.net/grainne/theory-and-methodology-in-networked-learning

Contu,A.,&Willmott,H.(2003).Re-embeddingsituatedness:theimportanceofpowerrelationsinlearningtheory.OrganizationScience,14(3),283–296.

Cox,A.(2005).Whatarecommunitiesofpractice?Acomparativereviewoffourseminalworks.JournalofInformationScience,31(6),527–540.

Creswell,J.W.(2013).ResearchDesign:Qualitative,Quantitative,andMixedMethodsApproaches(4ed.).London:SAGEPublications.

Cross,J.(2012).Fromthereflectivee-practitioner:apilotmodelofteacherpreparationemployingeportfolio.InternationalJournalofePortfolio,2,39–48.

Curant,N.(2009).E-portfoliosforstaffdevelopment–usinge-portfoliosforanITQqualification.RetrievedMay1,2013,fromhttp://www.eife-l.org/publications/eportfolio/proceedings2/learning-forum-london-2009/LFL2009.pdf

Çimer,S.O.(2011).Theeffectofportfoliosonstudents'learning:studentteachers'views.EuropeanJournalofOpen,DistanceandE-Learning,34(2),161–176.

Dalsgaard,C.(2006).Socialsoftware:e-learningbeyondlearningmanagementsystems.EuropeanJournalofOpen,DistanceandE-Learning,2006(2),1–7.

Dalziel,C.(2008).UsingePortfoliostocombatplagiarism(pp.206–210).ProceedingsoftheAsciliateConference,Melbourne.

Danielson,C.,&Abrutyn,L.(1997).AnIntroductiontoUsingPortfoliosinTheClassroom.Alexandria:AssociationforSupervisionandCurriculumDevelopment.

Davis,J.(2007).Rethinkingthearchitecture:anactionresearcher'sresolutiontowritingandpresentingtheirthesis.ActionResearch,5(2),181–198.

Dawson,S.(2008).Astudyoftherelationshipbetweenstudentsocialnetworksandsenseofcommunity.EducationalTechnology&Society,11(3).

DeLaat,M.F.(2005,June5).NetworkedLearning.Apeldoorn:PolitieAcademy.DeLaat,M.F.,&Lally,V.(2003).Complexity,theoryandpraxis:researching

204

collaborativelearningandtutoringprocessesinanetworkedlearningcommunity.InstructionalScience,31(1),7–39.

DeLaat,M.F.,&Lally,V.(2004).It'snotsoeasy:researchingthecomplexityofemergentparticipantrolesandawarenessinasynchronousnetworkedlearningdiscussions.JournalofComputerAssistedLearning,20(3),165–171.

DeLaat,M.F.,Lally,V.,Lipponen,L.,&Simons,R.-J.(2006a).Onlineteachinginnetworkedlearningcommunities:amulti-methodapproachtostudyingtheroleoftheteacher.InstructionalScience,35(3),257–286.

DeLaat,M.F.,Lally,V.,Lipponen,L.,&Simons,R.-J.(2006b).Analysingstudentengagementwithlearningandtutoringactivitiesinnetworkedlearningcommunities:Amulti-methodapproach.InternationalJournalofWebBasedCommunities,2(4),394–412.

DeLaat,M.F.,Lally,V.,Lipponen,L.,&Simons,R.-J.(2007).Investigatingpatternsofinteractioninnetworkedlearningandcomputer-supportedcollaborativelearning:Aroleforsocialnetworkanalysis.InternationalJournalofComputer-SupportedCollaborativeLearning,2(1),87–103.

DeWever,B.,Schellens,T.,Valcke,M.,&VanKeer,H.(2006).Contentanalysisschemestoanalyzetranscriptsofonlineasynchronousdiscussiongroups:areview.Computers&Education,46(1),6–28.

Delandshere,G.(2001).Implicittheories,unexaminedassumptionsandthestatusquoofeducationalassessment.AssessmentinEducation:Principles,8(2),113–133.

Delandshere,G.,&Arens,S.A.(2003).Examiningthequalityoftheevidenceinpreserviceteacherportfolios.JournalofTeacherEducation,54(1),57–73.

Dennen,V.P.(2005).Frommessagepostingtolearningdialogues:factorsaffectinglearnerparticipationinasynchronousdiscussion.DistanceEducation,26(1),127–148.

Denscombe,M.(2010).TheGoodResearchGuide(4ed.).NewYork:McGraw-HillEducation.

DeVoss,D.,&Rosati,A.C.(2002).“Itwasn’tme,wasit?”Plagiarismandtheweb.ComputersandComposition,19(2),191–203.

Dewey,J.(1910).Howwethink.Boston,Mass.:D.C.Heath&Co.Dick,B.(2011).Actionresearchliterature2008--2010:themesandtrends.Action

Research,9(2),122–143.Diez,M.E.(1994).Theportfolio:sonnet,mirrorandmap.(pp.1–14).

ProceedingsoftheConferenceonLinkingLiberalArtsandTeacherEducation,SanDiego,CA.

Dirckinck-Holmfeld,L.,&Jones,C.(2009).IssuesandconceptsinNetworked

205

Learning.InL.Dirckinck-Holmfeld,C.Jones,&B.Lindtröm(Eds.),AnalysingNetworkedLearningPracticesinHigherEducationandContinuing

ProfessionalDevelopment(pp.259–285).Rotterdam:SensePublishers.Dirckinck-Holmfeld,L.,Hodgson,V.E.,&McConnell,D.(2011).Exploringthe

Theory,PedagogyandPracticeofNetworkedLearning.NewYork:Springer.Dochy,F.,Segers,M.,&Sluijsmans,D.(1999).Theuseofself-,peerandco-

assessmentinhighereducation:areview.StudiesinHigherEducation,24(3),331–350.

Doig,B.,Illsley,B.,McLuckie,J.,&Parsons,R.(2006).UsingePortfoliostoenhancereflectivelearninganddevelopment.InA.Jafari&C.Kaufman(Eds.),HandbookofResearchonePortfolios(pp.158–167).Hershey,PA:IdeaGroupPublishing.

Doran,P.,Doran,C.,&Mazur,A.(2011).Socialnetworkanalysisasamethodforanalyzinginteractionincollaborativeonlinelearningenvironments.JournalofSystemics,Cybernetics&Informatics,9(7),10–16.

Draaijer,S.,&vanBoxel,P.(2006).Summativepeerassessmentusing“Turnitin”andalargecohortofstudents:acasestudy.RetrievedOctober4,2014,fromhttps://dspace.lboro.ac.uk/dspace/handle/2134/4559

Dringus,L.P.,&Ellis,T.(2005).Usingdataminingasastrategyforassessingasynchronousdiscussionforums.Computers&Education,45(1),141–160.

Dube,R.(2010,August30).Educationalabandonware-freeandusefulsoftwareorillegalpiracy?educational-freeware.com.RetrievedJune30,2012,fromhttp://www.educational-freeware.com/news/educational-abandonware.aspx

Duval,E.(2011).Attentionplease!:learninganalyticsforvisualizationandrecommendation(pp.9–17).ProceedingsoftheLAK'11:InternationalConferenceonLearningAnalyticsandKnowledge,Banff,AB,Canada:ACM.

Ehiyazaryan-White,E.(2012).ThedialogicpotentialofePortfolios:formativefeedbackandcommunitiesoflearningwithinapersonallearningenvironment.InternationalJournalofePortfolio,2(2),173–185.

Eisner,E.(2003).Reshapingassessmentineducation:somecriteriainsearchofpractice.JournalofCurriculumStudies,3,219–233.

Elliott,J.(1991).ActionResearchforEducationalChange.Buckingham:McGraw-HillInternational.

Entwistle,N.(2005).Learningoutcomesandwaysofthinkingacrosscontrastingdisciplinesandsettingsinhighereducation.Curriculumjournal,16(1),67–82.

Eraut,M.(1995).Schonshock:acaseforrefrainingreflection-in-action?TeachersandTeachingtheoryandpractice,1(1),9–22.

206

Eraut,M.(2002).Conceptualanalysisandresearchquestions:dotheconceptsof"LearningCommunity"and“CommunityofPractice”provideaddedvalue?(pp.1–12).ProceedingsoftheAnnualMeetingoftheAmericanEducationalResearchAssociation,NewOrleans.

Feng,F.(2006).Towardaframework/datamodel:fromePortfoliothinkingtofolioculture.InA.Jafari&C.Kaufman(Eds.),HandbookofResearchonePortfolios(pp.217–232).Hershey,PA:IdeaGroupPublishing.

Ferreday,D.,&Hodgson,V.(2008).Thetyrannyofparticipationandcollaborationinnetworkedlearning(pp.640–647).InV.Hodgson,C.Jones,T.Kargidis,D.McConnell(Eds.)Proceedingsofthe6thInternationalConferenceonNetworkedLearning,Halkidiki,Greece.

Ferreday,D.,&Hodgson,V.(2010,December17).Heterotopiainnetworkedlearning:beyondtheshadowsideofparticipationinlearningcommunities.LancasterUniversityManagementSchoolWorkingPapers.RetrievedApril10,2014,fromhttp://eprints.lancs.ac.uk/49033/

Fisher,R.(2003).ThegoldenageofBTEC:thebusinesseducationcurriculumin1980sfurthereducation.TheCurriculumJournal,14(2),253–277.

Fitch,D.,Peet,M.,Reed,B.G.,&Tolman,R.(2010).TheuseofePortfoliosinevaluatingthecurriculumandstudentlearning.Journalofsocialworkeducation,44(3),37–54.

Flanigan,E.,&Amirian,S.(2006).ePortfolios:pathwayfromclassroomtocareer.InA.Jafari&C.Kaufman(Eds.),HandbookofResearchonePortfolios(pp.102–111).Hershey,PA:IdeaGroupPublishing.

Fruchterman,T.,&Reingold,E.(1990).Graphdrawingbyforce-directedplacement.Software-PracticeandExperience,21(1),1129–1164.

Fuller,A.(2013).Critiquingtheoriesoflearningandcommunitiesofpractice.InJ.Hughes,N.Jewson,&L.Unwin(Eds.),Communitiesofpractice:Criticalperspectives(pp.17–29).London:Routledge.

Galbraith,P.,Renshaw,P.,&Goos,M.(1999).Technology,mathematics,andpeople:interactionsinacommunityofpractice(pp.4–7).Proceedingsofthe22ndAnnualConferenceoftheMathematicsEducationResearchGroupofAustralasia,Adelaide.

Garrett,N.,Thoms,B.,Alrushiedat,N.,&Ryan,T.(2009).Sociale-portfoliosasthenewcoursemanagementsystem.OntheHorizon,17(3),197–207.

Garrison,D.R.(2011).E-Learninginthe21stCentury(2nded.).London:Routledge.

Garrison,D.R.,&Anderson,T.(2003).E–Learninginthe21stcentury(1sted.).London:Routledge/Falmer.

207

Garrison,D.R.,&Arbaugh,J.B.(2007).Researchingthecommunityofinquiryframework:review,issues,andfuturedirections.TheInternetandHigherEducation,10(3),157–172.

Garrison,D.R.,&Cleveland-Innes,M.(2005).Facilitatingcognitivepresenceinonlinelearning:interactionisnotenough.AmericanJournalofDistanceEducation,19(3),133–148.

Garrison,D.R.,&Kanuka,H.(2004).Blendedlearning:uncoveringitstransformativepotentialinhighereducation.TheInternetandHigherEducation,7(2),95–105.

Garrison,D.R.,Anderson,T.,&Archer,W.(2001).Criticalthinking,cognitivepresence,andcomputerconferencingindistanceeducation.AmericanJournalofDistanceEducation,15(1),7–23.

Gibbs,G.,&Simpson,C.(2004).Conditionsunderwhichassessmentsupportsstudents’learning.LearningandTeachinginHigherEducation,1(1),3–31.

Gibson,D.(2006).ePortfoliodecisionsanddilemmas.InA.Jafari&C.Kaufman(Eds.),HandbookofResearchonePortfolios(pp.134–144).Hershey,PA:IdeaGroupPublishing.

Gibson,D.,&Barrett,H.C.(2002).Directionsinelectronicportfoliodevelopment.ContemporaryIssuesinTechnologyandTeacherEducation,2(4),556–573.

Gielen,S.,Dochy,F.,&Dierick,S.(2003).Evaluatingtheconsequentialvalidityofnewmodesofassessment:theinfluenceofassessmentonlearning,includingpre-,post-,andtrueassessmenteffects.InM.Segers,F.Dochy,&E.Cascaller(Eds.),OptimisingNewModesofAssessment:InSearchofQualitiesandStandards(Vol.1,pp.37–54).Netherlands:Springer.

Gielen,S.,Dochy,F.,&Onghena,P.(2011).Aninventoryofpeerassessmentdiversity.Assessment&EvaluationinHigherEducation,36(2),137–155.

Gillen,J.,&Barton,D.(2010).Digitalliteracies.TLRP.RetrievedAugust20,2013,fromhttp://www.tlrp.org/docs/DigitalLiteracies.pdf

Goldschmid,B.,&Goldschmid,M.(1973).Modularinstructioninhighereducation:areview.HigherEducation,2(1),15–32.

Goodyear,P.,&Steeples,C.(2008).Creatingshareablerepresentationsofpractice.ALT-J,ResearchinLearningTechnology,6(3),16–23.

Goodyear,P.,Jones,C.,Asensio,M.,&Hodgson,V.(2005).Networkedlearninginhighereducation:students’expectationsandexperiences.HigherEducation,50,473–508.

Goodyear,P.,Jones,C.,Asensio,M.,Hodgson,V.,&Steeples,C.(2001).Effectivenetworkedlearninginhighereducation:notesandguidelines.RetrievedMay,17,2006fromhttp://www.csalt.lancs.ac.uk/jisc/guidelines_final.doc.

208

Goodyear,P.,Jones,C.,Asensio,M.,Hodgson,V.,&Steeples,C.(2004).Undergraduatestudents’experiencesofnetworkedlearninginUKhighereducation:asurvey-basedstudy.InP.Goodyear,S.Banks,V.Hodgson,&D.McConnell(Eds.),AdvancesinResearchonNetworkedLearning(pp.91–121).NewYork:Springer.

Goos,M.,Galbraith,P.,&Renshaw,P.(2003).Establishingacommunityofpracticeinasecondarymathematicsclassroom.InL.Burton(Ed.),LearningMathematics(pp.36–61).NewYork:Taylor&Francis.

Gourlay,S.(1999).Communitiesofpractice:anewconceptforthemillenium,ortherediscoveryofthewheel?BusinessFacultyPapersRepository.RetrievedMarch1,2013,fromhttp://eprints.kingston.ac.uk/3719/1/Gourlay%201999%20CoP.pdf

Granovetter,M.S.(1973).Thestrengthofweakties.AmericanJournalofSociology,78(6),1360–1380.

Grant,S.(2009).e-Portfolioportability.leapspecs.org.RetrievedApril14,2014,fromhttp://www.leapspecs.org/2A/introduction

Graves,D.(1994).Helpchildrentorevisetheirwork.InD.Graves(Ed.),AFreshLookatWriting,Afreshlookatwriting(pp.23–36).Portsmouth,NH:Heinemann.

Gray,L.(2008).EffectivePracticewithe-Portfolios.Supporting21stcenturylearning.JISCInnovationGroup.JISC.

Greene,J.C.,Caracelli,V.J.,&Graham,W.F.(1989).Towardaconceptualframeworkformixed-methodevaluationdesigns.EducationalEvaluationandPolicyAnalysis,11(3),255–274.

Grundy,S.(1987).Curriculum:ProductOrPraxis?London:FalmerPress.Guldberg,K.,&Pilkington,R.(2006).Acommunityofpracticeapproachtothe

developmentofnon-traditionallearnersthroughnetworkedlearning.JournalofComputerAssistedLearning,22(3),159–171.

Gulikers,J.(2006).Authenticityisintheeyeofthebeholder:beliefsandperceptionsofauthenticassessmentandtheinfluenceonstudentlearning.Heerlen:OpenUniversityoftheNetherlands.RetrievedAugust20,2014,fromhttp://hdl.handle.net/1820/1043

Gulikers,J.,Bastiaens,T.J.,&Kirschner,P.A.(2004).Afive-dimensionalframeworkforauthenticassessment.EducationalTechnologyResearchandDevelopment,52(3),67–86.

Gulikers,J.,Bastiaens,T.,Kirschner,P.,&Kester,L.(2006).Relationsbetweenstudentperceptionsofassessment,authenticy,studyapproachesandlearningoutcome.StudiesinEducationalEvaluation,32(4),381–400.

209

Gullifer,J.M.,&Tyson,G.A.(2013).Whohasreadthepolicyonplagiarism?Unpackingstudents'understandingofplagiarism.StudiesinHigherEducation,39(7),1202–1218.

Hansen,B.,Stith,D.,&Tesdell,L.S.(2011).Plagiarism:what'sthebigdeal?BusinessCommunicationQuarterly,74(2),188–191.

Harasim,L.M.(1995).LearningNetworks:AFieldGuidetoTeachingandLearningOnline.Cambridge,MA:MITPress.

Harlen,W.,&James,M.(1997).Assessmentandlearning:differencesandrelationshipsbetweenformativeandsummativeassessment.AssessmentinEducation:Principles,PolicyandPractice,4(3),365–379.

Harper,M.G.(2006).Hightechcheating.NurseEducationToday,6(6),364–371.Hartnell-Young,E.(2007,August26).Impactstudyofe-portfoliosonlearning.

Becta.RetrievedMay26,2012,fromhttp://partners.becta.org.uk/upload-dir/downloads/page_documents/research/impact_study_eportfolios.pdf

Harvey,M.,Baillie,M.,Ruthven,I.,&Elsweiler,D.(2009).Folksonomictagcloudsasanaidtocontentindexing.arXiv.org.RetrievedNovember20,2013,fromhttp://core.ac.uk/download/pdf/11551454.pdf

Haythornthwaite,C.,&DeLaat,M.F.(2010).Socialnetworksandlearningnetworks:usingsocialnetworkperspectivestounderstandsociallearning.InV.Hodgson,C.Jones,M.F.DeLaat,&T.Ryberg(Eds.),(pp.183–190).Proceedingsofthe7thInternationalConferenceonNetworkedLearning,Aalborg,Denmark.

HeatonShrestha,C.,Edirisingha,P.,Burke,L.,&Linsey,T.(2005).IntroducingaVLEintocampus-basedundergraduateteaching:Staffperspectivesonitsimpactonteaching.InternationalJournalofEducationalResearch,43(6),370–386.

Heikkinen,H.L.T.,Huttunen,R.,&Syrjälä,L.(2007).Actionresearchasnarrative:fiveprinciplesforvalidation.EducationalActionResearch,15(1),5–19.

Henri,F.,&Pudelko,B.(2003).Understandingandanalysingactivityandlearninginvirtualcommunities.JournalofComputerAssistedLearning,19(4),474–487.

Herr,K.,&Anderson,G.L.(2005).TheActionResearchDissertation:AGuideforStudentsandFaculty.London:SAGEPublications.

Hodgson,V.,&Reynolds,M.(2005).Consensus,differenceand“multiplecommunities”innetworkedlearning.StudiesinHigherEducation,30(1),11–24.

Hodgson,V.,&Watland,P.(2004).Researchingnetworkedmanagementlearning.

210

ManagementLearning,35(2),99–116.Hopkins,D.,&Jackson,D.(2009,July28).Networkedlearningcommunities:

capacitybuilding,networkingandleadershipforlearning.NationalCollegeforSchoolLeadership,Nottingham.RetrievedJuly28,2009,fromhttp://networkedlearning.ncsl.org.uk/knowledge-base/research-papers/networked-learning-communities-capacity-building.doc

Hugo,J.M.(2002).Learningcommunityhistory.Newdirectionsforadultandcontinuingeducation,2002(95),5–26.

Huot,B.,&Williamson,M.(1997).Rethinkingportfoliosforevaluatingwriting:issuesofassessmentandpower.InK.B.Yancey,&I.Weiser(Eds.),SituatingPortfolios:FourPerspectives(pp.43–56).Logan,Utah:UtahStateUniversityPress.

Hussey,T.,&Smith,P.(2008).Learningoutcomes:aconceptualanalysis.TeachinginHigherEducation,13(1),107–115.

IELTSscoringindetail.(2016).Retrievedfromhttps://www.ielts.org/ielts-for-organisations/ielts-scoring-in-detail

Illeris,K.(2007).HowWeLearn:LearningandNon-LearninginSchoolandBeyond.London:Taylor&Francis.

Jafari,A.(2004).The“sticky”ePortfoliosystem:tacklingchallengesandidentifyingattributes.EducauseReview,39(4),38–48.

Jenson,J.D.(2011).Promotingself-regulationandcriticalreflectionthroughwritingstudents’useofelectronicportfolio.InternationalJournalofePortfolio,1(1),49–60.

Jézégou,A.(2010).Communityofinquiryine-learning:acriticalanalysisoftheGarrisonandAndersonmodel.JournalofDistanceEducation,24(3),1–18.

JISC.(2008).Learningprocesses.RetrievedJuly17,2010,fromhttps://www.jisc.ac.uk/guides/e-portfolios/learning-processes

JISC.(2010).Effectiveassessmentinadigitalage.EffectiveAssessmentinaDigitalAge.RetrievedJune1,2013,fromhttp://www.jisc.ac.uk/whatwedo/programmes/elearning/assessment/digiassess.aspx

Johnsen,H.L.(2012).Makinglearningvisiblewithe-Portfolios:couplingtherightpedagogywiththerighttechnology.InternationalJournalofePortfolio,2(2),139–148.

Johnson,L.,Adams,S.,Cummins,M.,Estrada,V.,Freeman,A.,&Ludgat,H.(2013).NMCHorizonProjectShortList(Vol.43).NewMediaConsortium.Retrievedfromhttp://www.nmc.org/pdf/2013-horizon-k12-shortlist.pdf

Johnson,R.B.,&Turner,L.A.(2003).Datacollectionstrategiesinmixedmethod

211

research.InA.Tashakkori,C.Teddlie,&C(Eds.),Handbookofmixedmethodsinsocialandbehavioralresearch(pp.297–319).ThousandOaks:Sage.

Johnson,R.B.,Onwuegbuzie,A.J.,&Turner,L.A.(2007).Towardadefinitionofmixedmethodsresearch.Journalofmixedmethodsresearch,1(2),112–133.

Johnson-Eilola,J.,&Selber,S.A.(2007).Plagiarism,originality,assemblage.ComputersandComposition,24(4),375–403.

Johnston,B.(2004).Summativeassessmentofportfolios:anexaminationofdifferentapproachestoagreementoveroutcomes.StudiesinHigherEducation,29(3),395–412.

Jones,C.,&Esnault,L.(2004).Themetaphorofnetworksinlearning:communities,collaborationandpractice.InS.Banks,P.Goodyear,V.E.Hodgson,C.Jones,V.Lally,D.McConnell,&C.Steeples(Eds.),ProceedingsoftheFourthInternationalConferenceonNetworkedLearning(pp.317–323),Lancaster.

Jones,C.,Asensio,M.,&Goodyear,P.(2000).Networkedlearninginhighereducation:practitioners’perspectives.ALT-J,ResearchinLearningTechnology,8(2),18–28.

Jones,C.,Ferreday,D.,&Hodgson,V.(2008).Networkedlearningarelationalapproach:weakandstrongties.JournalofComputerAssistedLearning,24(2),90–102.

Jones,N.,&Peachey,P.(2005).Thedevelopmentofsocializationinanon-linelearningenvironment.JournalofInteractiveOnlineLearning,3(3),1–20.

Jones,R.M.(1981).ExperimentatEvergreen.Cambridge,Mass:SchenkmanPublishingCompany.

Kemmis,S.(1993).Actionresearchandsocialmovement:achallengeforpolicyresearch.EducationPolicyAnalysisArchives,1(1),1–6.

Kemmis,S.,McTaggart,R.,&Nixon,R.(2013).TheActionResearchPlanner:DoingCriticalParticipatoryActionResearch.London:Springer.

Kennedy,G.,Ioannou,I.,Zhou,Y.,Bailey,J.,&O'Leary,S.(2013).Mininginteractionsinimmersivelearningenvironmentsforreal-timestudentfeedback.AustralasianJournalofEducationalTechnology,29(2),172–183.

Kenny,D.(2007).Studentplagiarismandprofessionalpractice.NurseEducationToday,27(1),14–18.

Kirkham,T.,Winfield,S.,Smallwood,A.,Coolin,K.,Wood,S.,&Searchwell,L.(2009).Introducinglivee-Portfoliostosupportselforganisedlearning.EducationalTechnology&Society,12(3),107–114.

Klamma,R.,Chatti,M.A.,Duval,E.,Hummel,H.,Hvannberg,E.T.,Kravcik,M.,Law,E.,etal.(2007).Socialsoftwareforlife-longlearning.Educational

212

Technology&Society,(3),72–83.Klein,L.F.(2013).ThesocialePortfolio:integratingsocialmediaandmodelsof

learninginacademicePortfolios.InK.Wills&R.Rice(Eds.),ePortfolioPerformanceSupportSystemsConstructing,Presenting,andAssessing

Portfolios(pp.57–74).FortCollins,Colorado:PerspectivesonWriting.Knapper,C.,&Cropley,A.J.(2000).LifelongLearninginHigherEducation.

London:KoganPage.Knight,P.(2002).Summativeassessmentinhighereducation:practicesin

disarray.StudiesinHigherEducation,27(3),275–286.Kolb,D.(2005).Experientiallearning:experienceasthesourceoflearningand

development.EnglewoodCliffs,NJ:Prentice-Hall.Koshy,V.(2009).ActionResearchforImprovingEducationalPractice:AStep-by-

StepGuide.London:SAGEPublications.Kotzee,B.(2012).Privatepractice:exploringthemissingsocialdimensionin

“reflectivepractice.”StudiesinContinuingEducation,34(1),5–16.Kramer,A.D.I.,Guillory,J.E.,&Hancock,J.T.(2014).Experimentalevidenceof

massivescaleemotionalcontagionthroughsocialnetworks.ProceedingsoftheNationalAcademyofSciences,111(24),8788–8790.

Kreijns,K.,VanAcker,F.,Vermeulen,M.,&vanBuren,H.,III.(2014).CommunityofInquiry:socialpresencerevisited.E-LearningandDigitalMedia,11(1),5–18.

Kuh,G.D.(1991).Involvingcolleges:successfulapproachestofosteringstudentlearninganddevelopmentoutsidetheclassroom.SanFrancisco:Jossey-BassPublishers.

Lankshear,C.,&Knobel,M.(2008).DigitalLiteracies:Concepts,PoliciesandPractices.NewYork:PeterLang.

Lave,J.,Wenger,E.,&Wenger,E.(1991).SituatedLearning:LegitimatePeripheralParticipation.NewYork:CambridgeUniversityPress.

Lawson,M.,Kiegaldie,D.,&Jolly,B.(2006).DevelopinganePortfolioforhealthprofessionaleducators:acasestudy.InA.Jafari&C.Kaufman(Eds.),HandbookofResearchonePortfolios(pp.273–282).Hershey,PA:IdeaGroupPublishing.

Ledwith,A.,&Rísquez,A.(2008).Usinganti-plagiarismsoftwaretopromoteacademichonestyinthecontextofpeerreviewedassignments.StudiesinHigherEducation,33(4),371–384.

Lewin,K.(1946).Actionresearchandminorityproblems.JournalofSocialIssues,2(4),34–46.

Li,L.C.,Grimshaw,J.M.,Nielsen,C.,Judd,M.,Coyte,P.C.,&Graham,I.D.(2009).

213

EvolutionofWenger'sconceptofcommunityofpractice.ImplementationScience,4(1),1–8.

Li,X.,&Prasad,C.(2005).Effectivelyteachingcodingstandardsinprogramming.InR.Friedman(Ed.),(pp.239–244).ProceedingsoftheSIGITEConferenceonInformationTechnologyEducation,NewYork,USA:ACM.

Liedtka,J.(1999).Linkingcompetitiveadvantagewithcommunitiesofpractice.JournalofManagementInquiry,8(1),5–16.

Lin,Q.(2008).Preserviceteachers'learningexperiencesofconstructinge-portfoliosonline.TheInternetandHigherEducation,11(3-4),194–200.

Lisewski,B.,&Joyce,P.(2003).Examiningthefive-stagee-moderatingmodel:designedandemergentpracticeinthelearningtechnologyprofession.ResearchinLearningTechnology,11(1),55–66.

Littlejohn,A.,&Pegler,C.(2004).PreparingforBlendedE-learning.Oxford:Taylor&Francis.

Liyanagunawardena,T.R.,&Adams,A.A.(2013).MOOCs:asystematicstudyofthepublishedliterature2008-2012.TheInternationalReviewofResearchinOpenandDistributedLearning,14(3),202–227.

Lopez-Fernandez,O.,&Rodriguez-Illera,J.L.(2009).Investigatinguniversitystudents’adaptationtoadigitallearnercourseportfolio.Computers&Education,52(3),608–616.

Lorenzo,G.,&Ittelson,J.(2005).Anoverviewofe-portfolios.EducauseLearningInitiative.RetrievedJuly15,2013,fromhttp://www.case.edu/artsci/cosi/cspl/documents/eportfolio-Educausedocument.pdf

Love,D.,McKean,G.,&Gathercoal,P.(2004).Portfoliostowebfoliosandbeyond:levelsofmaturation.EducauseQuarterly,27(2),24–38.

Lowenthal,P.,White,J.W.,&Cooley,K.(2011).Remake/remodel:usingeportfoliosandasystemofgatestoimprovestudentassessmentandprogramevaluation.InternationalJournalofePortfolio,1(1),61–70.

Lucas,C.(1992).Writingportfolios—changesandchallenges.InK.B.Yancey(Ed.),Portfoliosinthewritingclassroom:Anintroduction(pp.1–17).Urbana,IL:NationalCouncilofTeachersofEnglish.

Macdonald,J.,&Twining,P.(2002).Assessingactivity-basedlearningforanetworkedcourse.BritishJournalofEducationalTechnology,33(5),603–618.

Macfadyen,L.P.,&Dawson,S.(2010).MiningLMSdatatodevelopan‘‘earlywarningsystem”foreducators:Aproofofconcept.Computers&Education,54(2),588–599.

Macpherson,A.,Elliot,M.,Harris,I.,&Homan,G.(2004).E-learning:reflections

214

andevaluationofcorporateprogrammes.HumanResourceDevelopmentInternational,7(3),295–313.

Margaryan,A.,Littlejohn,A.,&Vojt,G.(2011).Aredigitalnativesamythorreality?Universitystudents’useofdigitaltechnologies.Computers&Education,56(2),429–440.

Martyn,M.(2003).Thehybridonlinemodel:goodpractice.EducauseQuarterly,26(1),18–23.

Martınez,A.,Dimitriadis,Y.,Rubia,B.,Gomez,E.,&laFuente,de,P.(2003).Combiningqualitativeevaluationandsocialnetworkanalysisforthestudyofclassroomsocialinteractions.Computers&Education,41(4),353–368.

Mason,R.,Pegler,C.,&Weller,M.(2004).E-portfolios:anassessmenttoolforonlinecourses.BritishJournalofEducationalTechnology,35(6),717–727.

Mathes,A.(2004).Folksonomies-cooperativeclassificationandcommunicationthroughsharedmetadata.RetrievedApril10,2010,fromhttp://adammathes.com/academic/computer-mediated-communication/folksonomies.pdf

Matthews,R.S.,Smith,B.L.,&MacGregor,J.(2012).Theevolutionoflearningcommunities:aretrospective.NewDirectionsforTeachingandLearning,2012(132),99–111.

Mazur,A.,Doran,C.,&Doran,P.(2010).Theuseofsocialnetworkanalysissoftwaretoanalyzecommunicationpatternsandinteractioninonlinecollaborativeenvironments.ProceedingsoftheInternationalConferenceonEducation,TrainingandInformatics,Orlando,FL.

Mazzolini,M.,&Maddison,S.(2007).Whentojumpin:theroleoftheinstructorinonlinediscussionforums.Computers&Education,49(2),193–213.

McConnell,D.(2000).ImplementingComputerSupportedCooperativeLearning(2nded.).London:KoganPress.

McConnell,D.(2002).Theexperienceofcollaborativeassessmentine-learning.StudiesinContinuingEducation,24(1),73–92.

McConnell,D.(2006).E-LearningGroupsAndCommunities.McGraw-HillEducation(UK).

McDuff,N.,&Marcelline,C.(2012).Equalityannualreport2013.KingstonUniversity.RetrievedAugust19,2013,fromhttp://www.kingston.ac.uk/aboutkingstonuniversity/equality-diversity-and-inclusion/equality-reports/

McMillan,D.W.,&Chavis,D.M.(1986).Senseofcommunity:adefinitionandtheory.JournalofCommunityPsychology,14(1),6–23.

McNiff,J.(2013).ActionResearch:PrinciplesandPractice.London:Taylor&

215

Francis.McNiff,J.,&Whitehead,J.(2002).ActionResearchinOrganisations.London:

Routledge.Meiklejohn,A.(1932).TheExperimentalCollege.NewYork:HarperandRow.Mihailidis,P.,&Cohen,J.N.(2013).Exploringcurationasacorecompetencyin

digitalandmedialiteracyeducation.JournalofInteractiveMediainEducation,2013(1),1–19.

Mika,P.(2007).SocialNetworksandtheSemanticWeb.Boston:Springer.Miller,J.P.(2007).TheHolisticCurriculum.Ontario:UniversityofTorontoPress.Moule,P.(2007).Challengingthefive-stagemodelfore-learning:anew

approach.ALT-J,ResearchinLearningTechnology,15(1),37–50.Nielson,J.(2010).“90-9-1”Ruleforparticipationinequality:lurkersvs.

contributorsininternetcommunities(JakobNielsen'sAlertbox).RetrievedOctober21,2011,fromhttp://www.useit.com/alertbox/participation_inequality.html

NING.(2014).Buildandcultivateyourowncommunity.www.ning.com.RetrievedOctober4,2014,fromhttp://www.ning.com

Nisbet,J.,&Watt,J.(1984).Casestudy.InJ.Bell,T.Bush,&A.Fox(Eds.),ConductingSmall-ScaleInvestigationsinEducationalManagement.London:HarperandRow.

Nolen,A.L.,&VanderPutten,J.(2007).Actionresearchineducation:addressinggapsinethicalprinciplesandpractices.EducationalResearcher,36(7),401–407.

Ooms,A.,Burke,L.,Linsey,T.,&HeatonShrestha,C.(2008).Introducinge-developerstosupportauniversity’sblendedlearningdevelopments.ALT-J,ResearchinLearningTechnology,16(2),111–122.

OpenSchool.(n.d.).OpenSchoolePortfolioTour.OpenSchoolePortfolio.RetrievedAugust1,2013,fromhttp://www.openschooleportfolio.com/tour/

Orsmond,P.,Merry,S.,&Reiling,K.(2000).Theuseofstudentderivedmarkingcriteriainpeerandself-assessment.Assessment&EvaluationinHigherEducation,25(1),23–38.

OSI.(2013,August19).OpenSourceInitiativeFAQ.OpenSourceInitiative.RetrievedJanuary31,2014,fromhttp://opensource.org/faq

Pachler,N.,&Daly,C.(2011).KeyIssuesinE-Learning.London:Continuum.Palloff,R.,&Pratt,K.(1999).Buildinglearningcommunitiesincyberspace.San

Francisco:Wiley.Palmer,C.,&Ottley,S.(1990).Frompotentialtoreality:“hybrids”-acriticalforce

intheapplicationofinformationtechnologyinthe1990s.London:British

216

ComputerSociety.Pariser,E.(2011).TheFilterBubble:HowtheNewPersonalizedWebIsChanging

WhatWeReadandHowWeThink.London:Penguin.Parkes,K.A.(2013).ePortfolioasameasureofreflectivepractice.International

JournalofePortfolio,3,99–115.Paul,H.,&Kimble,C.(2002).Thedualityofknowledge.InformationResearch,

8(1),1–18.Paulson,F.L.,&Paulson,P.R.(1994).Assessingportfoliosusingthe

constructivistparadigm.ProceedingsoftheAnnualMeetingoftheAmericanEducationalResearchAssociation,NewOrleans.

Paulson,F.L.,Paulson,P.R.,&Meyer,C.A.(1991).Whatmakesaportfolioaportfolio.EducationalLeadership,48(5),60–63.

Pawan,F.,Paulus,T.M.,Yalcin,S.,&Chang,C.(2003).Onlinelearning:patternsofengagementandinteractionamongin-serviceteachers.LanguageLearning&Technology,7(3),119–140.

Pedler,M.(1981).Developingthelearningcommunity.InT.Boydell&M.J.Pedler(Eds.),ManagementofSelf-Development:ConceptsandPractices(pp.68–94).Aldershot,England.

Pieterse,V.(2008).ReflectionsoncodingstandardsintertiaryComputerScienceeducation.SouthAfricanComputerJournal,49,29–37.

Pitri,E.(2004).Situatedlearninginaclassroomcommunity.ArtEducation,57(6),6–12.

Pitts,W.,&Ruggirello,R.(2012).Usingthee-Portfoliotodocumentandevaluategrowthinreflectivepractice:thedevelopmentandapplicationofaconceptualframework.InternationalJournalofePortfolio,2(1),49–74.

Popper,K.R.(1979).Threeworlds.TheTannerlectureonhumanvaluesattheUniversityofMichigan.MichiganQuarterly,2,140–167.

Price,M.,Carroll,J.,O'Donovan,B.,&Rust,C.(2011).IfIwasgoingthereIwouldn’tstartfromhere:acriticalcommentaryoncurrentassessmentpractice.Assessment&EvaluationinHigherEducation,36(4),479–492.

PurdueResearchFoundation.(2013).CoursesignalssystematPurdueUniversity.RetrievedAugust10,2014,fromhttp://www.itap.purdue.edu/studio/signals/

QAA.(2004).Guidelinesontheaccreditationofpriorlearning(p.25).QualityAssuranceAgencyforHigher

Education.Retrievedfromhttp://www.qaa.ac.uk/Publications/InformationAndGuidance/Documents/APL.pdf

217

Quinn,J.(2010).LearningCommunitiesandImaginedSocialCapital:LearningtoBelong.London:Bloomsbury.

Raymond,E.S.(2008).TheCathedralandtheBazaar.NewYork:O'ReillyMedia,Inc.

Reason,P.,&Torbert,W.R.(2001).Theactionturn:towardatransformationalsocialscience.InternationalJournalofActionResearchandOrganisationalRenewal,6(1),1–37.

Rhodes,T.,Chen,H.L.,Watson,C.E.,&Garrison,W.(2014).Editorial:acallformorerigorouse-Portfolioresearch.InternationalJournalofePortfolio,4(1),1–5.

Rickards,W.H.,Diez,M.E.,Ehley,L.,&Guilbault,L.F.(2008).Learning,reflection,andelectronicportfolios:steppingtowardanassessmentpractice.TheJournalofGeneralEducation,57(1),31–50.

Ring,G.,&Ramirez,B.(2012).ImplementingePortfoliosfortheassessmentofgeneraleducationcompetencies.InternationalJournalofePortfolio,2(1),87–97.

Robins,A.,Rountree,J.,&Rountree,N.(2003).Learningandteachingprogramming:areviewanddiscussion.ComputerScienceEducation,13(2),137–172.

Rodgers,C.(2002).Definingreflection:anotherlookatJohnDeweyandreflectivethinking.TeachersCollegeRecord,104(4),842–866.

Romer,T.A.(2002).Situatedlearningandassessment.Assessment&EvaluationinHigherEducation,27(3),233–241.

Roth,R.W.(1980).Theteachingofdocumentationandgoodprogrammingstyleinaliberalartscomputerscienceprogram(Vol.12,pp.139–153).Proceedingsofthe4thAnnualConferenceonInnovationandTechnologyinComputerScienceEducation,Krakow,Poland.

Roth,W.,&Lee,Y.(2007).“Vygotsky’sNeglectedLegacy”:Cultural-HistoricalActivityTheory.ReviewofEducationalResearch,77(2),186–231.

Roth,W.-M.,&Lee,Y.-J.(2006).Contradictionsintheorizingandimplementingcommunitiesineducation.EducationalResearchReview,1(1),27–40.

Rourke,L.(2009).Learningincommunitiesofinquiry:areviewoftheliterature.JournalofDistanceEducation,23(1),19–48.

Rust,C.,Price,M.,&OʼDonovan,B.(2003).Improvingstudentsʼlearningbydevelopingtheirunderstandingofassessmentcriteriaandprocesses.Assessment&EvaluationinHigherEducation,28,147–164.

Ryan,G.W.,&Bernard,H.R.(2003).Techniquestoidentifythemes.FieldMethods,15(1),85–109.

218

Ryberg,T.,&Larsen,M.C.(2008).Networkedidentities:understandingrelationshipsbetweenstrongandweaktiesinnetworkedenvironments.JournalofComputerAssistedLearning,24(2),103–115.

Saadatmand,M.,&Kumpulainen,K.(2013).Contentaggregationandknowledgesharinginapersonallearningenvironment:serendipityinopenonlinenetworks.InternationalJournalofEmergingTechnologiesinLearning(iJET),8(1),1–8.

Salmon,G.(2003).E-Moderating:TheKeytoTeachingandLearningOnline.London:Routledge,ChapmanandHall.

Salmon,G.(2007).Thetippingpoint.ALT-J,ResearchinLearningTechnology,15(2),171–172.

Salmon,G.(2011).E-moderating:Thekeytoonlineteachingandlearning(3rded.).NewYork:Routledge.

Scardamalia,M.,&Bereiter,C.(2003).KnowledgeBuilding.InEncyclopediaofEducation(2nded.,pp.1370–1373).NewYork:MacmillanReference.

Schon,D.A.(1983).TheReflectivePractitioner.NewYork:BasicBooks.Schon,D.A.(1990).EducatingtheReflectivePractitioner:TowardaNewDesign

forTeachingandLearningintheProfessions.SanFrancisco:Jossey-Bass.Schreurs,B.,Teplovs,C.,Ferguson,R.,DeLaat,M.F.,&Shum,S.B.(2013).

Visualizingsociallearningtiesbytypeandtopic:rationaleandconceptdemonstrator(pp.33–37).ProceedingsoftheLAK'13:InternationalConferenceonLearningAnalyticsandKnowledge,Leuven,Belgium.

Schwier,R.A.(2001).Catalysts,emphases,andelementsofvirtuallearningcommunities:implicationsforresearchandpractice.QuarterlyReviewofDistanceEducation,2(1),5–18.

Scouller,K.(1998).Theinfluenceofassessmentmethodonstudents'learningapproaches:multiplechoicequestionexaminationversusassignmentessay.HigherEducation,35(4),453–472.

Scriven,M.(1966).TheMethodologyofEvaluation.SocialScienceEducationConsortium.RetrievedApril9,2014,fromhttp://www.comp.dit.ie/dgordon/Courses/ILT/ILT0005/TheMethodologyOfEvaluation.pdf

Seitzinger,J.(2014).CurateMe!Exploringonlineidentitythroughsocialcurationinnetworkedlearning(pp.412–419).InS.Bayne,C.Jones,M.DeLaat,T.Ryberg&C.Sinclair(Eds.)Proceedingsofthe9thInternationalConferenceonNetworkedLearning,Edinburgh.

Selwyn,N.(2007).Theuseofcomputertechnologyinuniversityteachingandlearning:acriticalperspective.JournalofComputerAssistedLearning,23(2),

219

83–94.Selwyn,N.(2013).DistrustingEducationalTechnology:TheQuestionsWeShould

BeAsking,ButAreNot:CriticalQuestionsforChangingTimes.NewYork:Taylor&Francis.

Seo,K.K.(2007).Utilizingpeermoderatinginonlinediscussions:addressingthecontroversybetweenteachermoderationandnonmoderation.AmericanJournalofDistanceEducation,21(1),21–36.

Sharples,M.,McAndrew,P.,Weller,M.,Ferguson,R.,Fitzgerald,E.,Hirst,T.,&Gavid,M.(2013).InnovatingPedagogy2013:OpenUniversityInnovationReport2.MiltonKeynes:TheOpenUniversity.

Shea,P.J.,Pickett,A.M.,&Pelz,W.E.(2003).Afollow-upinvestigationof“teachingpresence”intheSUNYLearningNetwork.JournalofAsynchronousLearningNetworks,7(2),61–80.

Shea,P.,Hayes,S.,Vickers,J.,Gozza-Cohen,M.,Uzuner,S.,Mehta,R.,Valchova,A.,etal.(2010).Are-examinationofthecommunityofinquiryframework:Socialnetworkandcontentanalysis.TheInternetandHigherEducation,13(1),10–21.

Shea,P.,SauLi,C.,&Pickett,A.(2006).Astudyofteachingpresenceandstudentsenseoflearningcommunityinfullyonlineandweb-enhancedcollegecourses.TheInternetandHigherEducation,9(3),175–190.

Shea,V.A.H.(2010).Onlineinstructionaleffortmeasuredthroughthelensofteachingpresenceinthecommunityofinquiryframework:are-examinationofmeasuresandapproach.InternationalReviewofResearchinOpenandDistanceLearning,11(3),127–154.

Sherman,G.(2006).Instructionalrolesofelectronicportfolios.InA.Jafari&C.Kaufman(Eds.),HandbookofResearchonePortfolios(pp.1–14).Hershey,PA:IdeaGroupPublishing.

Sherry,A.,&Bartlett,A.(2005).Worthofelectronicportfoliostoeducationmajors:a“twobyfour”perspective.JournalofEducationalTechnologySystems,33(4),399–419.

Shulman,L.S.(1998).Teacherportfolios:atheoreticalactivity.InN.M.Lyons(Ed.),Withportfolioinhand.NewYork:TeachersCollegePress.

Shute,V.J.(2008).Focusonformativefeedback.ReviewofEducationalResearch,78(1),153–189.

Siemens,G.,&Long,P.(2011).Penetratingthefog:analyticsinlearningandeducation.EducauseReview.RetrievedSeptember17,2013,fromhttp://www.elmhurst.edu/~richs/EC/OnlineMaterials/SPS102/Teaching%20and%20Learning/Penetrating%20the%20Fog.pdf

220

Sikes,P.P.,&Potts,A.(2008).ResearchingEducationfromtheInside:InvestigationsfromWithin.Oxford:Routledge.

Singh,H.(2003).Buildingeffectiveblendedlearningprograms.EducationalTechnology,43(6),51–54.

Smith,B.L.(2001).Thechallengeoflearningcommunitiesasagrowingnationalmovement.PeerReview,4(1),4–8.

Smith,B.L.,MacGregor,J.,Matthews,R.,&Gabelnick,F.(2004).LearningCommunities:ReformingUndergraduateEducation.SanFrancisco:Wiley.

Smith,G.(2008).Informationarchitecture:tagging-emergingtrends.BulletinoftheAmericanSocietyforInformationScienceandTechnology,34(6),14–17.

Smith,K.,&Tillema,H.(1998).Evaluatingportfoliouseasalearningtoolforprofessionals.ScandinavianJournalofEducationalResearch,42(2),193–205.

Snyder,B.R.(1971).TheHiddenCurriculum.NewYork:Knopf.Somekh,B.(1995).Thecontributionofactionresearchtodevelopmentinsocial

endeavours:apositionpaperonactionresearchmethodology.BritishEducationalResearchJournal,21(3),339–355.

Stallman,R.M.,&Gay,J.(2009).Freesoftware,freesociety:SelectedessaysofRichardM.Stallman.CreateSpace.Retrievedfromhttps://www.gnu.org/philosophy/fsfs/rms-essays.pdf

Steeples,C.,&Goodyear,P.(1999).Enablingprofessionallearningindistributedcommunitiesofpractice:descriptorsformultimediaobjects.JournalofNetworkandComputerApplications,22(2),133–145.

Steeples,C.,Jones,C.,&Goodyear,P.(2002).BeyondE-learning.InC.Steeples&C.Jones(Eds.),NetworkedLearning:PerspectivesandIssues(pp.323–341).London:Springer.

Stefani,L.,Mason,R.,&Pegler,C.(2007).Theeducationalpotentialofe-portfolios:supportingpersonaldevelopmentandreflectivelearning.London:Routledge.

Stenhouse,L.(1975).AnIntroductiontoCurriculumResearchandDevelopment.London:Heinemann.

Stevenson,H.J.(2006).UsingePortfoliostofosterpeerassessment,criticalthinking,andcollaboration.InA.Jafari&C.Kaufman(Eds.),HandbookofResearchonePortfolios(pp.112–124).Hershey,PA:IdeaGroupPublishing.

Stödberg,U.(2012).Aresearchreviewofe-assessment.Assessment&EvaluationinHigherEducation,37(5),591–604.

Strivens,J.,Baume,D.,Owen,C.,Grant,S.,&Ward,R.(2009).Theroleofe-Portfoliosinformativeandsummativeassessment.CentreforRecordingAchievementforJISC(pp.1–66).JISC.RetrievedAugust10,2010fromhttp://www.jisc.ac.uk/media/documents/programmes/elearning/eportfinal

221

report.doc.Suthers,D.,&Rosen,D.(2011).Aunifiedframeworkformulti-levelanalysisof

distributedlearning(pp.64–74).ProceedingsoftheLAK11:InternationalConferenceonLearningAnalyticsandKnowledge,NewYork.

Szabo,A.,&Underwood,J.(2004).Cybercheats:isinformationandcommunicationtechnologyfuellingacademicdishonesty?JournalofResearchinInternationalEducation,5(2),180–199.

Taras,M.(2008).Summativeandformativeassessment:perceptionsandrealities.ActiveLearninginHigherEducation,9(2),172–192.

Tinto,V.(1995).Takingstudentretentionseriously.SyracuseUniversity.Tinto,V.(2000).Learningbettertogether:theimpactoflearningcommunitieson

studentsuccess.HigherEducationmonographseries,8(1),1–8.Toikkanen,T.,&Lipponen,L.(2011).Theapplicabilityofsocialnetworkanalysis

tothestudyofnetworkedlearning.InteractiveLearningEnvironments,19(4),365–379.

Torras,M.E.,&Mayordomo,R.(2011).Teachingpresenceandregulationinanelectronicportfolio.ComputersinHumanBehavior,27(6),2284–2291.

Tosh,D.,Werdmuller,B.,Chen,H.L.,Light,T.P.,&Haywood,J.(2006).Thelearninglandscape:aconceptualframeworkforePortfolios.InA.Jafari&C.Kaufman(Eds.),HandbookofResearchonePortfolios(pp.24–32).Hershey,PA:IdeaGroupPublishing.

UniversityofQueensland.(2013,August19).eLearningstatisticsreport.UniversityofQueenslandeLearningStatisticsreport.RetrievedAugust19,2013,fromhttp://www.elearning.uq.edu.au/content/elearning-statistics-report-april-2013.

Valentine,K.(2006).Plagiarismasliteracypractice:recognizingandrethinkingethicalbinaries.CollegeCompositionandCommunication,58(1),89–109.

vanHarmelen,M.(2006).PersonalLearningEnvironments(pp.815–816).ProceedingsoftheSixthInternationalConferenceonAdvancedLearningTechnologies,Kerkrade,TheNetherlands:IEEE.

Vonderwell,S.(2003).Anexaminationofasynchronouscommunicationexperiencesandperspectivesofstudentsinanonlinecourse:acasestudy.TheInternetandHigherEducation,6(1),77–90.

Vygotsky,L.S.(1980).MindinSociety:TheDevelopmentofHigherPsychologicalProcesses.Cambridge,Mass.:HarvardUniversityPress.

Walz,P.(2006).AnoverviewofstudentePortfoliofunctions.InA.Jafari&C.Kaufman(Eds.),HandbookofResearchonePortfolios(pp.193–204).Hershey,PA:IdeaGroupPublishing.

222

Wang,C.X.(2009).Comprehensiveassessmentofstudentcollaborationinelectronicportfolioconstruction:anevaluationresearch.TechTrends,53(1),58–66.

Wasserman,S.,&Faust,K.(1994).SocialNetworkAnalysis:MethodsAndApplications(19thed.).NewYork:CambridgeUniversityPress.

Watson,D.(2006).HowtothinkaboutwideningparticipationinUKhighereducation.Bristol:HEFCE.

Wenger,E.(1998).Communitiesofpractice:learningasasocialsystem.SystemsThinker,9(5),1–10.

Wenger,E.(1999).CommunitiesofPractice:Learning,Meaning,andIdentity.NewYork:CambridgeUniversityPress.

Wenger,E.(2001).Supportingcommunitiesofpractice:asurveyofcommunity-orientedtechnologies.RetrievedSeptember16,2015,fromhttps://guard.canberra.edu.au/opus/copyright_register/repository/53/153/01_03_CP_technology_survey_v3.pdf

Wenger,E.(2002).CultivatingCommunitiesofPractice:AGuidetoManagingKnowledge.Boston:HarvardBusinessSchoolPress.

Wenger,E.(2010).Communitiesofpracticeandsociallearningsystems:thecareerofaconcept.RetrievedJanuary31,2012,fromhttp://wenger-trayner.com/wp-content/uploads/2012/01/09-10-27-CoPs-and-systems-v2.0.pdf

Whitelock,D.(2009).Editorial:e-assessment:developingnewdialoguesforthedigitalage.BritishJournalofEducationalTechnology,40(2),199–202.

Wiggins,G.P.(1999).AssessingStudentPerformance:ExploringthePurposeandLimitsofTesting.SanFrancisco:Wiley.

Wiliam,D.(2006).Formativeassessment:gettingthefocusright.EducationalAssessment,11(3-4),283–289.

Wiliam,D.(2011).Whatisassessmentforlearning?StudiesinEducationalEvaluation,37(1),3–14.

Williams,J.B.,&Jacobs,J.(2004).Exploringtheuseofblogsaslearningspacesinthehighereducationsector.AustralasianJournalofEducationalTechnology,20(2),232–247.

Wong,A.(2008).Assessmentofexperientiallearning:supportingorsubvertingthepromiseofRPLattheuniversitylevel.ProceedingsoftheSaskatchewanRPLConference,Saskatchewan.

Wood,D.,Bruner,J.,&Ross,G.(1976).Theroleoftutoringinproblemsolving.JournalofChildPsychologyandPsychiatry,17(2),89–100.

Woodford,K.,&Bancroft,P.(2005).Multiplechoicequestionsnotconsidered

223

harmful.ProceedingsoftheAustralasianconferenceonComputingeducation,Darlinghurst.

Yancey,K.B.(2009a).Electronicportfoliotechnologyanddesignforlearning.InD.R.Cambridge,B.L.Cambridge,&K.B.Yancey(Eds.),ElectronicPortfolios2.0(pp.5–16).Sterling:StylusPublishers.

Yancey,K.B.(2009b).Reflectionandelectronicportfolios.InD.R.Cambridge,B.L.Cambridge,&K.B.Yancey(Eds.),ElectronicPortfolios2.0(pp.5–16).Sterling:StylusPublishers.

Yorke,M.(2005).Formativeassessmentinhighereducation:itssignificanceforemployability,andstepstowardsitsenhancement.TertiaryEducationandManagement,11(3),219–238.

Zeichner,K.,&Wray,S.(2001).TheteachingportfolioinUSteachereducationprograms:whatweknowandwhatweneedtoknow.TeachingandTeacherEducation,17(5),613–621.