Development of an electronic Portfolio system success model: An information systems approach

Download Development of an electronic Portfolio system success model: An information systems approach

Post on 25-Nov-2016




1 download

Embed Size (px)


<ul><li><p>ePortfolio review process</p><p>s, cons6; Fernat ePosuccening (Eer, exp</p><p>that constitute a successful ePortfolio implementation.The D&amp;M model is well-known. Originally developed in 1992, it has been used and cited in more than 100 papers (DeLone &amp; McLean,</p><p>1992; Petter, DeLone, &amp; McLean, 2008). An updated version was developed ten years later and has also been widely used (DeLone &amp;McLean, 2003; Petter et al., 2008). Given its acceptance in the IS research community, the D&amp;M model constitutes a suitable theoretical</p><p>* Corresponding author. Tel.: 385 42 390 858; fax: 385 42 213 413.E-mail addresses: (I. Balaban), (E. Mu), (B. Divjak).</p><p>1 Tel.: 1 412 578 8729.2</p><p>Contents lists available at SciVerse ScienceDirect</p><p>Computers &amp; Education</p><p>Computers &amp; Education 60 (2013) 396411Tel.: 385 42 390 842.deployment of an ePortfolio in an academic institution, it is necessary to take into account the different possible dimensions: ePortfolio asa source of quality educational information that needs to be incorporated into the curriculum, ePortfolio as an information system archi-tecture that needs to satisfy the users needs and also be reliable, usable and seamlessly incorporated into the institutional ICT architecture,and ePortfolio as a new phenomenon that enhances learning by bringing the learner closer to the educational institution and potentialemployer while becoming a source of personal growth and development.</p><p>The proposed study on ePortfolio success employs an information system approach that takes the above-mentioned aspects intoconsideration. The success of an ePortfolio can, therefore, be interpreted as equivalent to the success of a specic information system (IS). Forthis reason, the DeLone and McLean (2003) model (hereafter D&amp;Mmodel), which has been widely used to measure IS success, is used toassess the success of an ePortfolio system. According to this model, IS success consists of six interconnected constructs. The presence ofinner connections among the six dimensions (constructs) needs to be established to comprehend the exact structure and dependencies1. Introduction</p><p>Electronic Portfolios, or ePortfolioyears (Buzzetto-More &amp; Alade, 200ePortfolio literature review reveals thFurthermore, a model that describes</p><p>The European Institute for E-learillustrating a persons learning, care0360-1315/$ see front matter 2012 Elsevier Ltd. process are also discussed. 2012 Elsevier Ltd. All rights reserved.</p><p>titute an extension to e-learning. They have become an important research topic in the last fewndez, 2008; Mu, Wormer, Foizey, Barkon, &amp; Vehec, 2010; Stevenson, 2006). An extensivertfolio systems are widely used but still not thoroughly studied in all their different dimensions.ssful implementation of an ePortfolio system does not exist yet.IfEL) (2009) denes ePortfolio as a personal digital collection of information describing anderience and achievements. This denition suggests that in order to assess the success of theePortfolio successePortfolio system successmentation and use of ePortfolio systems through the analysis of the causal relationships of their differentdimensions. Finally, initial guidelines about how to use the instrument as part of an ePortfolio systemDevelopment of an electronic Portfolio system success model: An informationsystems approach</p><p>Igor Balaban a,*, Enrique Mu b,1, Blazenka Divjak a,2</p><p>aUniversity of Zagreb, Faculty of Organization and Informatics Varazdin, Pavlinska 2, 42 000 Varazdin, CroatiabCarlow University, 3333 Fifth Avenue, Pittsburgh, PA 15213, USA</p><p>a r t i c l e i n f o</p><p>Article history:Received 11 January 2012Received in revised form4 June 2012Accepted 10 June 2012</p><p>Keywords:Electronic portfolios</p><p>a b s t r a c t</p><p>This research has two main goals: to develop an instrument for assessing Electronic Portfolio (ePortfolio)success and to build a corresponding ePortfolio success model using DeLone and McLeans informationsystems success model as the theoretical framework. For this purpose, we developed an ePortfoliosuccess measurement instrument and structural model, at the individual level of analysis, usingresponses from 186 ePortfolio student users from higher education institutions worldwide. Academicinstitutions can use the results of this research to assess the success of their ePortfolio implementationsfrom their students perspective. The ePortfolio success model can also help to improve the imple-</p><p>journal homepage: www.elsevier .com/locate/compeduAll rights reserved.</p></li><li><p>framework to assess ePortfolio system success. However, because the D&amp;M model has not been previously used in the ePortfolio context,a new set of model elements needs to be considered that match ePortfolio requirements. Themain purpose and contribution of this researchis the development of an ePortfolio success model applicable at the individual level of analysis.</p><p>2. Background on ePortfolios</p><p>Since Portfolios, in general, were developed to support the learning and teaching process and have been used for a long time, there arenumerous denitions of student learning portfolios proposed by educators. The literature reveals a dozen possible denitions of the portfolioterm; however, there are three that in our opinion illustrate the most important different aspects of the term. A comprehensive denitionwasofferedbyPaulson, Paulson, andMeyer (1991),whodescribed Portfolios as ameaningful collectionof studentwork thatdemonstratesprogressand/or mastery guided by standards and includes evidence of student self-reection. Abrenica (1996) dened Portfolios as a collection ofstudentachievementartifacts createdduringaperiodof time that serveasauthentic assessment toolsused toevaluate student learning. Finally,Barrett (1998) dened Portfolios as a purposeful collection of students work that illustrates efforts, progress and achievement.</p><p>All these denitions describe Portfolio as a concept or a set of procedures and data that result in the demonstration of a studentscapabilities. However, the full educational potential of Portfolios is obtained when the procedures and data identied in the aforementioned</p><p>I. Balaban et al. / Computers &amp; Education 60 (2013) 396411 397denitions are implemented as an information system. The current trend is moving toward implementing Portfolio initiatives as infor-mation systems that are supported by current information and communication technology (ICT) and in particular web-based technology.For this reason, this research refers exclusively to Web-based Portfolios. In order to differentiate a paper Portfolio from its electroniccounterpart, the letter e will be added to the word Portfolio. Therefore, the term ePortfolio will be used hereafter to denote the mostpopular type of electronic Portfolio, i.e. the Web-based Portfolio being used currently.</p><p>Although the analysis of the aforementioned denitions may suggest that ePortfolios are used only by students, other entities such asadministration, potential employers, lifelong learners and education institutions can use ePortfolios as well. Drawing on previous denitionsand taking into consideration that none of them specically included the IT component, new denitions of ePortfolio have been coined.Barker (2003) considers ePortfolios as an electronic learning record which enables an individual to store, organize and present their workand accomplishments. The EIfEL, which leads the Europortfolio consortium and is a founding member of the European Foundation forQuality in E-Learning (EFQUEL), denes ePortfolio as a personal digital collection of information describing and illustrating a personslearning, career, experience and achievements. Furthermore, the denition proposed by EIfEL emphasizes that ePortfolios are privatelyowned and the owner has a complete control over who has access to what and when. The Inter/National Coalition for Electronic PortfolioResearch that mostly deals with ePortfolios across the USA, denes ePortfolio as a collection of diverse evidence created in authenticactivity that is brought together and re-contextualized to say something about what I know and can do (how I have grown or changed).and with an added interpretation intended for one or more specic audiences (Cambridge, Cambridge, &amp; Yancey, 2009, p. 145).</p><p>It should be noted that the above denitions emphasize the information system aspect of ePortfolios, but do not explicitly address thenature and range of ePortfolio ownership. In addition, the learning aspect of ePortfolios not only embraces the storage and presentation ofpast work and experience, but also encompasses reection and feedback. It is these two latter features that may represent the biggestpotential of ePortfolios as a Lifelong Learning (LLL) tool for growth and development, and should therefore not be overlooked.</p><p>The ePortfolio has evolved over the years and Table 1 presents a brief comparison between 2 major generations of ePortfolio denitionsthat can be identied during the ePortfolio development cycle.</p><p>Considering the various denitions referred to in this section, one more encompassing denition will be coined for this study toovercome the shortcomings of the previous ones (see Table 1). The proposed denitionwill take into account the ePortfolio purpose, type ofinformation, entities involved and IT component. Therefore for the purpose of this research ePortfolio will be dened as a personal digitalrecord that supports formal, informal and non-formal learning and contains evidence about ones accomplishments in the form of artifacts andreection on learning which can be provided to whomever the owner has chosen to grant permission.</p><p>3. ePortfolio as an information system</p><p>Several references that indicate the interrelationship between ePortfolio and IS can be found in the literature (Jafari, 2004; Mu et al.,2010; Richardson &amp; Ward, 2005). In his description of ePortfolio, Jafari (2004) approached its development using the IS framework. Mu</p><p>Table 1Analysis of existing Portfolio denitions.</p><p>1st generation of denitions (Portfolios) Shortcomings:</p><p>Meaningful collection of student work that demonstrates progress and/or masteryguided by standards and includes evidence of student self-reection (Paulson,Paulson, &amp; Meyer, 1991).</p><p> Student only actor (does not include other possibletypes of actors such as organization or teacher)</p><p> IT component missing Ownership issues such as copyright not consideredCollection of student achievement artefacts created during a period of time thatserve as authentic assessment tools used to evaluate student learning (Abrenica, 1996).</p><p>Purposeful collection of students work that illustrates efforts, progress andachievement (Barrett, 1998).</p><p>2nd generation of denitions (ePortfolio) Shortcomings:</p><p>An electronic learning record which enables an individual to store, organize and present their workand accomplishments (Barker, 2003).</p><p> Ownership issues not considered Does not include all possible types of user entities(such as potential employer)</p><p> Does not include the most important type of supportin learning ((self)reection, feedback, etc.) that makes</p><p>A personal digital collection of information describing and illustrating a persons learning, career,experience and achievements (European Institute for E-learning (EIfEL), 2009).the process far more advanced than before</p></li><li><p>et al. (2010) attempted to conceptualize the functional requirements for ePortfolio systems referring to ePortfolios as consisting of peopleand technology. Although a supercial use of the IT framework for ePortfolios may lead one to consider it merely as an IT tool, an ePortfoliosystem actually comprises much more. As with any other IS, where ePortfolios are concerned, it is not sufcient to merely embrace thetechnology; it has to be adopted and used by people supporting all the required business processes in a proper way. An ePortfolio is a set ofinterrelated or meshed components and functionalities, similar to IS. Therefore, ePortfolio applications should be put in a wider organi-zational context. This approach was taken in the research by Mu et al. (2010) in order to understand ePortfolio functionalities and theirprioritization criteria. In the same paper, the authors discussed the challenges associated with the adoption of ePortfolios drawing from theliterature about IS adoption and assimilation. Furthermore, in their survey conducted in the UK, Richardson and Ward (2005) argued thatePortfolios should support LifeLong Learning. They also reported a signicant discrepancy between ePortfolio applications and therequirements of a LifeLong Learning environment as an organizational system to be supported with ePortfolio.</p><p>In Table 2 common attributes extracted from IS denitions and ePortfolio denitions are grouped. It is assumed that similarities indenitions reect similarities between objects.</p><p>Considering the common attributes between IS denitions and ePortfolio denitions above (Table 2), the following conclusions can bedrawn:</p><p> An ePortfolio is a set of interrelated components at the technical level. It is comprised of a Web application, hardware and softwaresupport and a network infrastructure. These features qualify ePortfolio as IS from the technical perspective.</p><p> An electronic learning record established at the technical level supports processes from a business system. It enables users to collect,store, manage, process and disseminate information in the form of an artifact, which occurs at the data level. The purpose of anePortfolio is equivalent to IS.</p><p> Providing support to data and having ICT features is not sufcient for an entity to qualify as an IS. ePortfolio fullls its purpose only whenan individual interacts with others by giving and receiving feedback in different forms, that is, when it is used within a community.</p><p>Based on a review of the extant literature, ePortfolio implementations must take into account three different stakeholders: individuals</p><p>I. Balaban et al. / Computers &amp; Education 60 (2013) 396411398(students and teachers); institutions; and employers. A study of ePortfolios from all different perspectives is beyond the scope of a singleresearch study, and therefore this research will focus on the assessment of ePortfolio deployment from the perspective of individualstudents. The majority of research on ePortfolio (Batson, 2002; Gathercoal, Love, Bryde, &amp;McKean, 2002; Love, McKean, &amp; Gathercoal, 2004;Ring &amp; Foti, 2006; Stefani, Mason, &amp; Pegler, 2007; Stevenson, 2006) focuses on the deployment process within an institution, deningePortfolio requirements, and case studies of institutions that have implemented ePortfolios at the course level. However, . ePortfoliosystem implementation is in general a comprehensive educational innovation and therefore support has to be provided in both a peda-gogical and technical sense (sic) (Ring &amp; Foti, 2006, p.353). Several attempts have been made to investigate the perceived values of users</p><p>Table 2Comparison between denitions...</p></li></ul>