self-efficacy, learning method appropriation and software skills acquisition in learner-controlled...
TRANSCRIPT
Self-efficacy, learning method appropriationand software skills acquisition inlearner-controlled CSSTS environmentsisj_12016 3..27
Andrew M. Hardin,* Clayton A. Looney† & Mark A. Fuller‡
*Lee Business School, University of Nevada, Las Vegas, Nevada, USA, email:[email protected], †College of Business, University of Montana, Missoula,Montana, USA, email: [email protected], and ‡School of Management,University of Massachusetts, Amherst, Massachusetts, USA, email: [email protected]
Abstract. A computer-simulated software training system (CSSTS) delivers aspecific form of computer-based training in which participants are allowed to selectvarious training features within a simulated software environment. Given thegrowing use of these systems as end-user training (EUT) aids, there is a need forgreater understanding of how participants use these systems, as well as whetherparticipant-controlled learning environments are truly effective. The presentresearch examines how a particular learner characteristic, software self-efficacy,drives appropriation in a high learner control, CSSTS environment. Contrary tonotions in the literature, results from spreadsheet and database software trainingcourses reveal that pre-training specific software self-efficacy constitutes a signifi-cant, negative predictor of faithful appropriations of the CSSTS. This research alsoestablishes a positive relationship between faithful appropriation and increases insoftware self-efficacy (SSE). In essence, faithful appropriations lead to greaterincreases in SSE, which influences software skills performance. In addition, theresearch validates prior EUT research by extending prior findings to a databasetraining environment. A psychometrically sound measure is put forth to capturedatabase self-efficacy.
Keywords: computer-simulated software training system, end-user training,adaptive structuration theory, appropriation, software self-efficacy, databaseself-efficacy.
INTRODUCTION
Organisations continue to make substantive investments in employee training, highlighting apersistent desire to develop and maintain a highly skilled workforce in today’s competitivemarketplace. In 2009, organisations with over 100 employees spent in excess of $52.8 billion
doi:10.1111/isj.12016
Info Systems J (2014) 24, 3–27 3
© 2013 Wiley Publishing Ltd
on employee training, or $1041 per employee. Fourteen per cent of these expenditures wereused to fund computer skills training [Anonymous, 2010]. While these numbers were slightlyhigher than 2008, instructor-led training fell sharply from the previous year (from 47% to27.7%). To reduce costs associated with instructor-led training, organisations expended$562 934 on learning tools and technologies, up nearly $100 000 from the prior year. Corre-spondingly, organisations reduced training professionals salaries by an average of $2000[Anonymous, 2010].
These data reflect the pace at which organisations offset accelerating expenditures forsoftware skills instruction through the implementation of e-learning self-study software appli-cations, which accounted for over 30% of student training hours in 2009 [Anonymous, 2010].A computer-simulated software training system (CSSTS) represents a specific type ofe-learning self-study system that has become immensely popular for facilitating softwareinstruction. One CSSTS was used by over 86 000 trainees in 2005 alone [Speckler, 2006].CSSTSs facilitate the simultaneous instruction of large numbers of participants, reducinginstructor preparation time1 and allowing organisations to cost-effectively train participants withvarious degrees of prior software experience [Speckler, 2006]. When using these systems,participants are presented with instructional features, which they are free to use as frequentlyor infrequently as they wish, employing their own strategy for practicing the targeted softwareskills. Given the vast flexibility provided to users during the learning process, such systemsoffer a high degree of learner control.
Despite the widespread use of CSSTSs, relatively little is known about their impact onsoftware training success [Brown, 2001; Yi & Davis, 2001; DeRouin et al., 2004]. Although agreat deal of end-user training (EUT) research has been conducted [Olfman & Bostrom, 1991;Davis & Bostrom, 1993; 1994; Bostrom et al., 1994; Olfman & Mandviwalla, 1994; 1995], muchof this research involves human-led instruction followed by hands-on use of the target system.CSSTSs constitute qualitatively different settings, as these systems represent a specifictechnology-mediated learning (TML) application that facilitates a ‘learning from computers’environment [Gupta & Bostrom, 2009, p. 694]. To our knowledge, only one study has exam-ined learning outcomes in a CSSTS setting [Gupta, 2008]. Because Gupta’s [2008] studyfocused exclusively on spreadsheet training, it remains to be seen whether the findings can beextended to other software training contexts. Moreover, no study to date has investigated howand why users adopt particular instructional features in CSSTS training environments, orwhether software skills acquisition depends on such interactions. This research addressesthese gaps in the literature.
Laying the conceptual groundwork for this effort, Gupta & Bostrom [2009] put forth a modelof TML that invokes both social cognitive theory (SCT) and adaptive structuration theory (AST)to explain how users acquire software skills. The current study builds on this research byinvestigating software self-efficacy, an important component within SCT, as both a predictor ofCSSTS feature use and as a learning outcome in high learner control environments. Comple-
1Although not required to teach the material, in CSSTS environments, instructors must set up student access, implement
the respective training modules, track computer-generated test scores, etc.
4 A M Hardin et al.
© 2013 Wiley Publishing Ltd, Information Systems Journal 24, 3–27
menting SCT, AST provides an overarching perspective for understanding the relationshipbetween software self-efficacy and faithfulness of appropriation (FOA), defined in this contextas the combined use of the CSSTS features. Further, AST helps explain how this faithfulappropriation influences subsequent software self-efficacy perceptions and, ultimately, soft-ware skills acquisition. Examining the interplay among software self-efficacy, FOA and traineeperformance informs and extends the conceptual relationships proposed in the TML literature,while simultaneously addressing the need for research investigating the agents of appropria-tion in EUT settings [Gupta, Bostrom, & Huber, 2010].
The following questions are addressed by this research: (1) Does pre-training softwareself-efficacy influence appropriation in high learner control, CSSTS environments? (2) Doesfaithful appropriation in high learner control CSSTS environments lead to increases in softwareself-efficacy? (3) Can findings from prior research in high learner control CSSTS environmentsbe generalised beyond spreadsheet software training?
The remainder of this paper is organised as follows. First, brief overviews of SCT and ASTare provided. Hypotheses are then developed. Next, we describe our research design, includ-ing a methodology section and a discussion of our analyses and results associated with twoseparate studies. Finally, we review the theoretical implications of our findings, discuss thestudy’s limitations, and explore potential avenues for future research.
THEORY
SCT
SCT proposes a triadic reciprocal relationship between the person, their behaviour and theenvironment. Residing within the person element of SCT, self-efficacy is defined as ‘a belief inone’s capability to organize and execute the courses of action required to produce givenattainments’ [Bandura, 1997, p. 3]. The specificity of self-efficacy beliefs can range fromgeneral (e.g. computer self-efficacy) to specific (e.g. software self-efficacy) [Compeau &Higgins, 1995; Compeau, Gravill, Haggerty, & Kelly, 2006]. As a self-regulatory process,people’s self-efficacy beliefs influence the actions they pursue, the effort they put forth, theirpersistence against repeated failure and control of their cognitive functioning, such as sus-ceptibility to self-debilitating thoughts [Bandura, 1997].
The literature identifies four primary sources of self-efficacy [Gist & Mitchell, 1992; Bandura,1997]. Enactive mastery, considered the strongest source of self-efficacy information[Bandura, 1997; Pajares, 2004], is acquired through prior experience in a given domain[Compeau & Higgins, 1995] or through hands-on training in which the participant is performingthe target behaviour [Bandura et al., 1977]. Vicarious experience, the second source ofself-efficacy information, can be delivered in two ways: through behavioural modelling or verbalguidance. Behavioural modelling provides training information by observing another whodemonstrates the desired skills or behaviours [Yi & Davis, 2001]. Verbal guidance providestraining information by giving either verbal or written instructions about how to accomplish a
Learning method appropriation 5
© 2013 Wiley Publishing Ltd, Information Systems Journal 24, 3–27
behaviour. Thus, verbal guidance ‘describes rather than shows how to perform given activities’[Bandura, 1997, p. 371]. Because verbal guidance can also be presented in written form, it canbe used as a means of retention enhancement following behavioural modelling demonstrations[Yi & Davis, 2001]. Not to be confused with verbal guidance, verbal persuasion represents thethird source of efficacy information. Verbal persuasion provides evaluative feedback throughverbal cues. Depending on how the feedback is conveyed, self-efficacy beliefs can either beweakened or bolstered. In contrast to the instruction-oriented focus of verbal guidance, verbalpersuasion focuses on the provision of performance feedback. Finally, physiological states actas the fourth source of self-efficacy information by providing cues that influence a person’sbelief in his or her capabilities. For example, muscle pain during physical activities or stresscan reduce one’s sense of self-efficacy. These four sources of self-efficacy informationcombine through a complex cognitive integration process, where individuals weigh thesesources and use them in the development of self-efficacy perceptions [Bandura, 1997].
AST
AST was originally proposed to integrate the decision-making and institutional schools ofthought guiding information systems (IS) research [DeSanctis & Poole, 1994]. Drawing uponstructuration theory [Giddens, 1984], AST proposes that advanced IS are designed such thatthey reinforce certain social structures. A system’s ‘spirit’, defined as ‘the general intent withregard to the values and goals underlying a given set of structural features’ [DeSanctis &Poole, 1994, p. 126], represents one method for describing the social structures embedded inadvanced information technologies. Originally designed for understanding how teams appro-priate group decision support systems, AST has more recently been applied in individual-levelTML models. Depending on factors such as aptitude, motivation, or self-efficacy, learners maychoose to appropriate an advanced technology either faithfully or unfaithfully to the system’sspirit [Gupta & Bostrom, 2009].
Appropriation moves provide a useful way of organising how users adopt structural features.DeSanctis & Poole [1994] identify four general appropriation moves: directly adopting thestructures, relating structures to other structures, constraining or interpreting the structures ormaking judgments regarding the usefulness of the structures. Within these general categories,specific types of appropriation moves are further defined. These include directly adoptingstructures through direct appropriation, relating structures to other structures through combi-nation, substitution, enlargement, and contrast, constraining or interpreting the structuresthrough constraint or making judgments regarding the usefulness of the structures through theuse of affirmation, negation and neutrality.
Learning dimensions, features and efficacy sources
SIMNET™ represents one CSSTS frequently used for software training. Unlike traditional TML,SIMNET™ provides a simulated experience with the software package in which various trainingfeatures are available for use as the trainee deems appropriate. In this simulated environment,
6 A M Hardin et al.
© 2013 Wiley Publishing Ltd, Information Systems Journal 24, 3–27
functions normally included in the actual software package (e.g. the help feature) are renderedinactive. SIMNET™ does not limit the number of times a particular feature may be used, nordoes it restrict the order in which the various features can be accessed. Thus, the systemfacilitates a high learner control, learning from computers environment [Gupta & Bostrom,2009; Gupta, Bostrom, & Huber, 2010].
Technology structures can be described in terms of their features and dimensions[DeSanctis & Poole, 1994]. Learning dimensions provided by these technology structures arescalable [Gupta & Bostrom, 2009]. Gupta & Bostrom [2009] map the scalability of specificlearning dimensions to video- and web-based training. Specifically, the findings of Gupta[2008] are used to propose that scalability of the learning dimensions production pattern (thetiming of declarative knowledge), structuredness of practice (specificity of practice), restric-tiveness of practice (whether or not the practice was guided), feedback (immediacy of feed-back), and guidance (how closely the guidance was mapped to the leaning goals) is generallyhigher for web-based training compared with video training. We build on and extend thesepropositions by more precisely examining the respective learning dimensions, their scalabilityand the features used to deliver them within CSSTS environments. In particular, we breakdown the learning structures delivered by the CSSTS2 to show that the learning dimensionsdescribed by Gupta [2008] vary in scalability across the respective features.
SIMNET™ targets MS Office™ software skills acquisition. The CSSTS includes Teach Me,Show Me and Let Me Try features. The Teach Me feature provides written instructions usingtext-based sidebars and graphical callouts. Through text-based instruction, this feature pro-vides an experience similar to the self-efficacy source, verbal guidance. The Show Me featuredemonstrates skills using animation and voice, delivering an experience similar to behaviouralmodelling training. The Show Me feature also manipulates the cursor to visually demonstratethe training exercises for participants. Finally, the Let Me Try feature allows learners to attemptthe simulated software exercises unaided, and provides participants with system-generatedmessages, indicating whether the exercise has been completed successfully or unsuccess-fully. Because the exercises are similar to completing unassisted exercises using an actualsystem, the Let Me Try feature provides a learning experience that delivers the most potentform of self-efficacy building source, enactive mastery (i.e. unaided hands-on experience),followed by verbal persuasion (i.e. feedback acknowledging whether the task was completedcorrectly). Table 1 outlines the relationship between the SIMNET™ features, the respectivesources of self-efficacy that the features provide and the learning dimensions proposed byGupta & Bostrom [2009].
Learning dimensions, features, appropriation moves and frequency of use
AST suggests that designers’ intentions partly determine a system’s spirit. Additional factors,such as the design metaphor, system features, user interface and training materials, should
2The system described by Gupta [2008] appears to include features similar to those provided by the CSSTS used in the
current study. In particular Gupta [2008] describes features called ‘Show Me’ and ‘I do’. These features appear similar to
the SIMNET™ features Show Me and Let Me Try described in the subsequent paragraph.
Learning method appropriation 7
© 2013 Wiley Publishing Ltd, Information Systems Journal 24, 3–27
also be considered by the researcher when determining the system’s spirit. Based upondiscussions with the instructional designers, an extensive review of the system, and a carefulexamination of the associated user manuals and training materials, SIMNET™ was determinedby the authors to portray a spirit consistent with the theoretical underpinnings of SCT, providinga theoretical linkage between the training system and learning effectiveness. In addition, thereading of the system spirit suggested that the respective features should be used incombination to optimise the benefits of the learning dimensions described in Table 1. Thus,
Table 1. Learning dimensions, features and efficacy sources
Learning dimensions*
Teach Me (verbal
guidance)
Show Me (behavioural
modelling)
Let Me Try (enactive
mastery) Feature combination
ProductionPattern
Delayed Production Delayed Production Delayed Production Immediate Production
The lag between
demonstration of an
action and practice
by the learner.
Feature provides written
instructions using
text-based sidebars and
graphical callouts.
Students do not practice
with feature.
Demonstrates skills using
animation and voice.
Manipulates the cursor to
visually demonstrate the
training exercises.
Students do not practice
with feature.
Learners practice specific
tasks, but can complete
them on their own using
techniques not
demonstrated by the
system.
Students can use the Let
Me Try feature
immediately following
instruction by the Teach
Me or Show Me features.
Structuredness ofPractice
Low Low Low High
The extent to which
technology imposes
its procedures on
the learner.
Feature provides textual
guidance using specific
steps. Students do not
practice with feature.
Feature demonstrates the
skill using specific steps.
Students do not practice
with feature.
Learners practice specific
tasks, but can complete
them on their own using
techniques not
demonstrated by the
system.
Students can follow
system instructions when
completing tasks.
Restrictiveness ofPractice
Low Low Low High
The degree to
which a system
limits an action.
Feature provides written
instructions only. Students
do not practice with
feature.
Feature demonstrates the
skill only. Students do not
practice with feature.
Students can complete
the exercises on their own
using techniques not
demonstrated by the
system.
Students can follow
system instructions when
completing tasks.
Feedback Low Low Moderate High
The degree to
which a system
provides a
response, including
correction, addition
or approval and
speed of response.
Feature provides written
instructions. Students do
not practice with feature.
Feature demonstrates the
skill. Students do not
practice with feature.
Participants are provided
feedback that the exercise
has been completed
correctly or incorrectly.
Does not enforce
procedures taught using
Show Me and Teach Me.
The Let Me Try Feature
provides feedback that the
exercise has been
completed correctly or
incorrectly. Students can
use procedures taught
using Show Me and
Teach Me.
Guidance Moderate Moderate Moderate High
The degree to
which a system
provides direction
or advice towards a
course of action.
Feature provides written
instructions that are
directly linked to learning
goals.
Feature demonstrates
skills directly linked to
learning goals.
Students can complete
the exercises on their own
using techniques not
demonstrated by the
system.
Teach Me and Show Me
provide specific guidance
on how to complete a
task. Let Me Try allows
students to complete
tasks using those
procedures.
*Gupta & Bostrom [2009].
8 A M Hardin et al.
© 2013 Wiley Publishing Ltd, Information Systems Journal 24, 3–27
FOA in the current context can be captured by assessing how frequently participants rely onall of the features.
AST suggests that users may invoke one of several appropriation moves. Appropriationmoves provide a useful method for understanding whether features are appropriated faithfullyor unfaithfully according to the system’s spirit. However, AST has generally focused onunderstanding group level appropriation in decision support settings [DeSanctis & Poole,1994]. Given the inability to analyse group level discourse in individual-level training settings,assessing appropriation requires a different approach. One method that may be well suited forthese environments involves analysing participants’ behaviours during training sessions. Inparticular, how appropriation moves relate to the frequency of feature use. Table 2 details therelationships between learning dimensions, CSSTS features, appropriation moves and fre-quency of use. We refer to these respective relationships when developing our hypotheses inthe following.
HYPOTHESES AND THEORY DEVELOPMENT
Software self-efficacy, feature selection and use
Individuals with low general computer self-efficacy are less willing to attempt hands-on trainingthan those with high general computer self-efficacy [Gist et al., 1989]. Although less studied,the relationship can also be expected to hold for software self-efficacy. General computerself-efficacy and software self-efficacy operate on similar principles, merely at different levelsof abstraction [Marakas et al., 1998]. Because low pre-training software self-efficacy (PreSSE)participants doubt their abilities, they will likely gravitate towards features that guide themthrough the learning process. Rather than attempting to complete a task unaided, tentativelearners will utilise features that demonstrate tasks or deliver written explanations. Through theuse of these guided features, low PreSSE participants will become more confident in theirsoftware skills. As confidence builds, these users will eventually migrate towards the use of thehands-on, enactive mastery feature, which allows them to perform the task on their own. Anexample of an individuals with low presentation self-efficacy can help illustrate this behaviour.Because of low self-efficacy, such individuals will rely heavily on supporting aids such astext-laden slides or detailed speaker notes. Based on successful presentation experiencesover time, presentation self-efficacy beliefs will increase. As the individuals become moreconfident in their abilities, they will be more assured about delivering presentations withoutaids.
Trainees with robust PreSSE beliefs are unlikely to be fazed by the prospect of completingtasks unassisted. Given their highly confident ability beliefs, they will likely bypass features thatmerely explain or demonstrate software skills. Rather than reviewing skills they believe theyalready possess, they will seek out hands-on, enactive learning training methods. In essence,PreSSE will be inversely related to faithful appropriations of the CSSTS. Weaker PreSSEbeliefs will lead to a greater utilisation of the training features embedded in a CSSTS than willstronger PreSSE beliefs.
Learning method appropriation 9
© 2013 Wiley Publishing Ltd, Information Systems Journal 24, 3–27
H1: PreSSE will be a significant, negative predictor of faithfully appropriating the CSSTS.
CSSTS training effectiveness
Gupta & Bostrom [2009] distinguish between structural features and dimensions. Structuralfeatures in an IS context reflect functionalities, whereas structural dimensions reflect whatthe features do for the actor/learner. In the current context, the set of features are Teach
Table 2. Learning dimensions, features, appropriation moves and frequency of use
Learning dimensions* Feature combination Appropriation moves Frequency of use
Production Pattern Immediate Production Based on the reading of the
SIMNET™ spirit,
constraining the structures
by favouring one structure
over the other results in
unfaithful appropriation.
Relying primarily on the use
of one feature is consistent
with constraining the
structures by favouring one
structure over the other.
The lag between
demonstration of an action
and practice by the learner.
Students can use the Let
Me Try feature immediately
following instruction by the
Teach Me or Show Me
features.
Based on the reading of the
SIMNET™ spirit, faithful
appropriation requires
combining the learning
structures by relating the
structures to other
structures.
Relying on the use of all
three features allows
participants to combine the
learning structures by
relating structures to other
structures.Structuredness of Practice High
The extent to which
technology imposes its
procedures on the learner.
Students can follow system
instructions when
completing tasks.
Restrictiveness of Practice High
The degree to which a
system limits an action.
Students can follow system
instructions when
completing tasks.
Feedback High
The degree to which a
system provides a response,
including correction, addition
or approval and speed of
response.
Students receive feedback
on whether a task has been
completed correctly or
incorrectly when following
the instructions provided by
the Teach Me and Show Me
features.
Guidance High
The degree to which a
system provides direction or
advice towards a course of
action.
Students can follow the
instructions provided by the
Teach Me and Show Me
features when completing
the exercises using the Let
Me Try feature.
*Gupta & Bostrom [2009].
10 A M Hardin et al.
© 2013 Wiley Publishing Ltd, Information Systems Journal 24, 3–27
Me, Show Me and Let Me Try, and these features deliver structural dimensions affiliatedwith the self-efficacy building sources of verbal persuasion, vicarious experience and enac-tive learning. As described in Table 1, using a combination of instructional features producesmore profound results across each learning dimension (production pattern, structuredness ofpractice, restrictiveness of practice, feedback and guidance). Therefore, faithfully appropri-ating the CSSTS should produce a positive effect on learning outcomes [Gupta & Bostrom,2009; Gupta, Bostrom, & Huber, 2010; Thomas et al., 2008]. By relating the structural fea-tures to others structural features, combined feature use delivers a high level of the struc-tural dimensions positively related to leaning outcomes.
Prior research indicates that some learners initially overestimate their abilities [Zimmerman,1995]. Episodes of overestimation occur more frequently when learner control is increased[Piccoli et al., 2001]. Participants who initially overestimate their ability tend to exclude impor-tant facts in high learner control environments [Williams, 1996]. We embrace and extend thisline of reasoning by suggesting that because of an overestimation of ability, high PreSSEparticipants will make inaccurate judgments regarding the usefulness of the learning structuresprovided by the CSSTS. Given robust beliefs in their abilities, these individuals may mistakenlyshun the use of the Teach Me and Show Me vicarious experience features, instead choosingto complete the exercises relying primarily upon the Let Me Try, enactive mastery feature,which allows them to tackle the task on their own. By invoking appropriation moves that favourthe enactive mastery feature over the vicarious experience features, high PreSSE participantswill likely miss important pieces of self-efficacy building information delivered by the respectivelearning dimensions. Because high PreSSE participants overestimate their abilities, they willconstrain the CSSTS structures by favouring one structure over the other, resulting in anunfaithful appropriation. Because low PreSSE participants are more likely to faithfully appro-priate the CSSTS than high PreSSE participants, low PreSSE users will experience a greaterincrease in post-training software self-efficacy (PostSSE).
H2: Through faithful appropriation of the CSSTS, low PreSSE participants will realise agreater increase in software self-efficacy than will high PreSSE participants.
Over three decades of SCT research confirm that when properly isolated within the domain ofinterest, the self-regulatory process of self-efficacy predicts performance [Bandura, 1997].Although well established in the self-efficacy literature, we evaluate this relationship to assessthe meaningfulness of the changes in software self-efficacy following the computer simulatedsoftware training.
H3: PostSSE will be a significant, positive predictor of objective software skills performance.
METHODOLOGY
Two studies were undertaken to test the research hypotheses. Study 1 examines the hypoth-esised relationships during SIMNET™ Excel spreadsheet training, whereas Study 2 attempts to
Learning method appropriation 11
© 2013 Wiley Publishing Ltd, Information Systems Journal 24, 3–27
replicate these relationships during SIMNET™ Access database training. Such an approachenables us to assess whether the proposed relationships hold across disparate types ofsoftware, lending additional credence to our results.
Study 1
Participants included 207 undergraduates enrolled in a pre-certification IS course at a largeNorth American university. Participants utilised SIMNET™ as their exclusive means for learningspreadsheet skills. The spreadsheet training took place 1 hour and 15 minutes per day, 3 daysa week, for 2 weeks. Each day, participants entered a computer training facility in whichSIMNET™ had been installed. After entering their respective login credentials, participantswere provided with the opportunity to complete a series of spreadsheet exercises. Consistentwith high learner control environments, students could practice the exercises as many times asthey wished, using any of the available features (i.e. Teach Me, Show Me, Let Me Try) duringthe process. Upon completion of the training session, participants were required to take anobjective hands-on spreadsheet assessment, which was designed by the faculty and deliveredby the system. The assessment represented a significant portion of the students’ final grade inthe course. While graduate assistants were present during lab sessions, their role involvedtaking attendance, providing non-task-related technology support and dismissing the class forthe day. Other than through the CSSTS, no instructional advice was provided during thesoftware training.
Task
Participants were given access to a series of computer-simulated spreadsheet exercisesprovided by the CSSTS. Exercises varied in difficulty and were designed specifically to prepareparticipants for an objective, hands-on spreadsheet skills assessment. Practice exercisesranged from the generation of simple graphs to the creation of complex formulas. Collectively,the exercises cultivated the skills necessary for using spreadsheets in a business setting.Providing an objective measure of performance, the post-training spreadsheet skills assess-ment consisted of hands-on exercises, which evaluated participants’ cumulative understandingof the software. Arrived at through faculty consensus to provide a sufficient level of challengefor business users, the difficulty of the spreadsheet training was consistent with expectationsof learning for the spreadsheet software portion of the course.
Sample demographics
The sample consisted of participants enrolled in a business school pre-certification courserequired of all business majors. The average age of the participants was 20.1 years. Genderwas distributed as 46% female and 54% male. Self-reported majors were approximately equalacross the disciplines, with ‘undecided’ being reported slightly more often than the others.
12 A M Hardin et al.
© 2013 Wiley Publishing Ltd, Information Systems Journal 24, 3–27
Participants reported an average spreadsheet software experience level of 57.44, measuredon a 0–100 scale anchored by ‘no experience’ (0) and ‘very experienced’ (100).
Measures
Surveys were administered both pre- and post-training. PreSSE was measured before thespreadsheet training commenced, while the measures for FOA and PostSSE were adminis-tered after the 2-week training, immediately prior to the spreadsheet skills assessment.
To control for individual differences as proposed by Gupta & Bostrom [2009], computeranxiety (affective states), outcome expectancies (motivation), grade point average (GPA;aptitude) and gender were entered as covariates in the respective analyses. Computer anxietyand outcome expectancies were measured using validated measures [Compeau & Higgins,1995; Compeau, Higgins, & Huff, 1999].
PreSSE and PostSSE were measured using an existing spreadsheet self-efficacy measure[Johnson & Marakas, 2000]. The response format for the spreadsheet self-efficacy measureutilised a two-part question typical of efficacy measures [Bandura, 2005].
FOA was measured using a three-item, seven-point Likert-type scale developed by theauthors. Consistent with our conceptualisation of appropriation, the items captured the degreeto which the SIMNET™ features were collectively utilised.
Finally, user performance was measured objectively by the spreadsheet skills assessment,administered and scored by the CSSTS. This test entailed the hands-on completion of a set ofspreadsheet tasks designed to evaluate participants’ cumulative spreadsheet abilities. Theinstrument items are listed in Table 3.
Results
Measurement model results. Other than FOA, study 1 utilised previously validated existingmeasures. As evidenced by the factor loadings provided in Table 4, exploratory factor analysisrevealed that all items loaded higher on their target constructs than on any other construct.Cronbach’s alpha was 0.71, 0.96 and 0.93 for FOA, PreSSE and PostSSE, respectively.Respective reliabilities for computer anxiety and outcome expectancies were 0.78 and 0.91.Consequently, the measures demonstrated acceptable levels of validity and reliability. Tofacilitate the analyses of the relationships among variables, composite scores were created foreach multi-item measure.
Hypothesis 1. Hypotheses 1 proposed a significant, inverse relationship between PreSSE andFOA. Specifically, the hypothesis predicted that individuals with weaker PreSSE beliefs wouldmore faithfully appropriate the CSSTS than trainees with robust PreSSE beliefs. Given theanticipated linear relationships between PreSSE and FOA, the data were examined viaregression. PreSSE and the covariates identified by Gupta & Bostrom [2009] were entered asindependent variables, whereas FOA served as the dependent variable. Referring to Table 5,one covariate, computer anxiety (b = 0.16, t (201) = 2.17, p < 0.05) and PreSSE (b = -0.27, t
Learning method appropriation 13
© 2013 Wiley Publishing Ltd, Information Systems Journal 24, 3–27
(201) = 3.77, p < 0.001) proved to be significant predictors of FOA. As evidenced by thenegative beta weight, participants with more fragile PreSSE beliefs more faithfully appropriatedthe CSSTS features than trainees with stronger PreSSE beliefs. Thus, H1 was supported.
Hypothesis 2. Hypothesis 2 proposed that through faithful appropriation of the CSSTS, lowPreSSE participants would realise a significantly greater increase in spreadsheet self-efficacythan high PreSSE participants. To evaluate the influence of FOA on spreadsheet self-efficacyacross the low and high PreSSE groups, a dichotomisation of the sample was necessary.
Table 3. Measurement items
Construct Item
Spreadsheet
self-efficacy
I believe I have the ability to manipulate the way a number appears in a spreadsheet.
I believe I have the ability to use and understand the cell references in a spreadsheet.
I believe I have the ability to enter numbers in a spreadsheet.
I believe I have the ability to summarise numeric information using a spreadsheet.
I believe I have the ability to use a spreadsheet to communicate numeric information to others.
I believe I have the ability to use a spreadsheet to display numbers as graphs.
I believe I have the ability to use a spreadsheet to assist me in making decisions.
I believe I have the ability to write a simple formula in a spreadsheet to perform mathematical
calculations.
I believe I have the ability to use a spreadsheet to share numeric information with others.
Database
self-efficacy
I believe I have the ability to customise the display format for a field in a table.
I believe I have the ability to build a report.
I believe I have the ability to write a query using the query design wizard.
I believe I have the ability to customise the data type of a field in a table.
I believe I have the ability to import data into a database.
I believe I have the ability to password protect a database.
I believe I have the ability to update records in a table.
I believe I have the ability to build a form.
Faithfulness
of appropriation
How often did you use the SIMNET ‘Show Me’ feature when preparing for the Excel (Access)
skill assessment?
How often did you use the SIMNET ‘Teach Me’ feature when preparing for the Excel (Access)
skill assessment?
How often did you use the SIMNET ‘Let Me Try’ feature when preparing for the Excel (Access)
skill assessment?
Computer
anxiety
I feel apprehensive about using computers.
It scares me to think that I could cause the computer to destroy a large amount of information
by hitting the wrong key.
I hesitate to use a computer for fear of making mistakes I cannot correct.
Computers are somewhat intimidating to me.
Outcome
expectancies
I would be better organised.
I would increase my effectiveness on the job.
I would spend less time on my routine job tasks.
I would increase my sense of accomplishment.
I would increase my chance of obtaining a promotion.
I would increase my chance of getting a raise.
14 A M Hardin et al.
© 2013 Wiley Publishing Ltd, Information Systems Journal 24, 3–27
Table 4. Study 1 factor loadings
Component
1 2 3 4
Excel 1 0.817 0.127 -0.043 -0.136
Excel 2 0.812 0.178 -0.093 -0.127
Excel 3 0.807 0.109 -0.127 -0.100
Excel 4 0.799 0.055 -0.092 -0.031
Excel 5 0.789 0.075 -0.120 -0.027
Excel 6 0.783 0.149 -0.124 0.067
Excel 7 0.780 0.104 -0.118 -0.040
Excel 8 0.767 0.039 -0.152 0.033
Excel 9 0.691 -0.054 -0.054 -0.083
OE 1 0.123 0.849 -0.066 0.011
OE 2 0.045 0.833 0.033 -0.014
OE 3 0.160 0.807 -0.017 0.033
OE 4 0.119 0.789 -0.132 0.026
OE 5 0.102 0.781 -0.074 0.116
OE 6 0.006 0.751 0.059 -0.241
CA 1 -0.095 -0.045 0.812 0.134
CA 2 -0.192 -0.022 0.808 0.093
CA 3 -0.189 -0.080 0.789 0.025
CA 4 -0.097 -0.011 0.658 0.129
FOA1 -0.150 0.010 0.147 0.818
FOA2 -0.177 -0.080 0.102 0.793
FOA3 0.057 0.046 0.113 0.724
Bold represents relevant construct item loadings.
Table 5. Hypothesis 1 – regression results
b t Supported
Study 1
Hypothesis
H1 -0.270 3.77*** Yes
Covariates
Computer anxiety 0.160 2.17*
Study 2
Hypothesis
H1 -0.260 3.44*** Yes
Covariates
Outcome expectancies 0.137 2.34*
GPA -0.131 2.24*
Gender 0.200 3.22**
***p < 0.001.
**p < 0.01.
*p < 0.05.
Learning method appropriation 15
© 2013 Wiley Publishing Ltd, Information Systems Journal 24, 3–27
Similar to Gist et al. [1989], who investigated the influence of alternative training methods onthe development of computer self-efficacy by comparing the upper and lower sample thirds[Gist et al., 1989], we dichotomised the sample based upon PreSSE scores. A median splitwas used to retain the entire sample, as well as to provide a more conservative test of theresearch hypotheses than would have a split based upon quartiles or thirds.
An analysis of covariance (ANCOVA) compared the differences in spreadsheet self-efficacyacross the low and high PreSSE groups. Table 6 contains the results. No covariates weresignificant. Low PreSSE participants realised a greater increase in spreadsheet self-efficacy(PreSSE = 54.47, PostSSE = 69.53) than did high PreSSE participants (PreSSE = 89.63,
Table 6. Results for study 1 – Hypothesis 2
Independent variable n
Software self-efficacy Type III fixed effects results
M(pre) M(post) df F p h2
Participant type 201 64.08 0.000 0.24
Low PreSSE 103 54.47 69.53
High PreSSE 104 89.63 85.50
Covariate n Estimate SE df t p h2
Gender 207 -0.209 2.401 201 -0.087 0.931 0.000
GPA 207 1.230 1.088 201 1.131 0.259 0.006
Computer anxiety 207 -0.278 0.959 201 -0.290 0.772 0.000
Outcome expectancies 207 2.303 1.561 201 1.476 0.142 0.000
Independent variable n M(pre) M(post) df F p h2
Appropriation (low PreSSE) 94 2.900 0.092 0.030
Low 48 58.36 69.95
High 52 50.88 69.15
Covariate n Estimate SE df t p h2
Gender 100 1.397 4.282 94 -0.326 0.745 0.001
GPA 100 1.899 2.066 94 0.919 0.360 0.009
Computer anxiety 100 0.013 1.603 94 0.008 0.994 0.000
Outcome expectancies 100 4.282 2.758 94 1.553 0.124 0.025
Independent variable n M(pre) M(post) df F p h2
Appropriation (high PreSSE) 101 0.061 0.806 0.001
Low 57 91.35 87.17
High 50 87.67 83.60
Covariate n Estimate SE df t p h2
Gender 107 -0.231 2.517 101 -0.092 0.927 0.000
GPA 107 0.569 1.084 101 0.525 0.600 0.003
Computer anxiety 107 -0.847 1.087 101 -0.780 0.437 0.006
Outcome expectancies 107 0.326 1.607 101 0.203 0.840 0.001
16 A M Hardin et al.
© 2013 Wiley Publishing Ltd, Information Systems Journal 24, 3–27
PostSSE = 85.50). This difference was significant (F (1, 201) = 64.08, p < 0.001, h2 = 0.24),supporting Hypothesis 2.
Despite a significant difference, a simple comparison of PostSSE across the low and highPreSSE participants may not represent a complete picture of appropriation effects. Specifi-cally, examining the effects of the level of appropriation within the low and high PreSSE groupsmay lead to a deeper understanding of the results.3 As such, we further dichotomised the lowand high PreSSE groups based on FOA. Changes in spreadsheet self-efficacy were thenassessed between participants who more faithfully and less faithfully appropriated the CSSTS.Referring to Table 6, ANCOVA results reveal that within the low PreSSE group, high appro-priation participants experienced a greater increase in spreadsheet self-efficacy (PreSSE =50.88, PostSSE = 69.15) than low appropriation participants (PreSSE = 58.36, PostSSE =69.95). Although only marginally significant (F (1, 94) = 2.900, p < 0.10), this result reinforcesthe assertion that faithful appropriation leads to greater increases in PostSSE. For the highPreSSE participants, no such difference was observed (F (1, 101) = 0.061, p = 0.806).However, the non-significant difference for the high PreSSE participants can be readilyexplained, as the level of faithful appropriation among these users was low overall. Becausethe high PreSSE group failed to fully reap the benefits of the Teach Me and Show Me features,the analysis essentially compares two groups of individuals with relatively low FOA levels. Inessence, PostSSE increases when FOA levels are high. These results further bolster theassertion that increases in PostSSE surface only when users faithfully appropriate the CSSTS.
Hypothesis 3. Hypothesis 3 predicted that PostSSE would be a significant, positive predictorof objective software skills performance. The results are illustrated in Table 7. After controllingfor the significant covariate GPA (b = 0.266, t (201) = 4.304, p < 0.001), the independent
3We would like to thank an anonymous reviewer for bringing this to our attention.
Table 7. Hypothesis 3 – regression results
b t Supported
Study 1
Hypothesis
H1 0.344 5.165*** Yes
Covariates
GPA 0.266 4.304***
Study 2
Hypothesis
H1 -0.260 3.440*** Yes
Covariates
Computer anxiety -0.155 2.656**
GPA -0.180 3.152**
***p < 0.001.
**p < 0.01.
*p < 0.05.
Learning method appropriation 17
© 2013 Wiley Publishing Ltd, Information Systems Journal 24, 3–27
variable PostSSE (b = 0.344, t (201) = 5.165, p < 0.001) was a significant, positive predictor ofperformance. This result supports Hypothesis 3.
Study 2
Study 2 served two purposes. First, although one previous study has examined the effects ofspreadsheet training on CSSTS outcomes [Gupta, 2008], no study has assessed whetherfindings can be extended beyond spreadsheet software training. Second, study 1 showedrelatively high levels of PreSSE. As a result, ceiling effects represent an alternative explanationfor the results. To address these possibilities, a second study was conducted during adatabase software training course, where PreSSE levels were expected to be lower. Partici-pants included 280 undergraduates enrolled in an IS course at a large North Americanuniversity. The training session was conducted in exactly the same manner as study 1; the onlydifference was the focus on database rather than spreadsheet software.
Sample demographics
The sample consisted of participants enrolled in a business school pre-certification courserequired of all business majors. The average age of the participants was 21.4 years. Genderwas distributed as 42% female and 58% male. Self-reported majors were approximately equalacross the disciplines, with ‘undecided’ being reported slightly more often than the others.Participants reported an average database software experience level of 26.63, measured ona 0–100 scale anchored by ‘no experience’ (0) and ‘very experienced’ (100). As expected,participants reported much less database experience (M = 26.63) compared with spreadsheetexperience (M = 57.44).
Measures
Surveys were administered both pre- and post-training. Like study 1, PreSSE was measuredbefore the database training commenced, while the FOA and PostSSE measures wereadministered after the 2-week training, immediately prior to the database skills assessment. Asthe task involved acquiring skills with database software, PreSSE and PostSSE were mea-sured using a database self-efficacy measure developed by the authors. The response formatfor the database self-efficacy measure required a two-part question typical of efficacy mea-sures [Bandura, 2005], exactly like the spreadsheet self-efficacy scales utilised in study 1.Appendix 1 describes the instrument development process used to validate the databaseself-efficacy measure. All other items were the same as study 1, although some were minimallytailored to focus on database software rather than spreadsheets.
Results
Measurement model results. Referring to Table 8, all items loaded higher on their targetconstructs than on any other construct. Cronbach’s alphas for FOA, PreSSE, PostSSE,
18 A M Hardin et al.
© 2013 Wiley Publishing Ltd, Information Systems Journal 24, 3–27
computer anxiety and outcome expectancies were 0.71. 0.96, 0.93, 0.75 and 0.91, respec-tively. Given evidence for acceptable levels of validity and reliability, composite scores werecreated for the multi-item measures to facilitate the subsequent analyses.
Hypothesis 1. Hypotheses 1 proposed that PreSSE would be significantly, inversely related toFOA. A regression model was used to analyse the data. PreSSE and the covariates identifiedby Gupta & Bostrom [2009] were entered as independent variables, whereas FOA served asthe dependent variable. Table 5 contains the results. After controlling for the significantcovariates outcome expectancies (b = 0.137, t (274) = 2.34, p < 0.05), GPA (b = -0.131, t (274)= 2.24, p < 0.05) and gender (b = 0.200, t (274) = 3.22, p < 0.01), PreSSE proved to be asignificant predictor of FOA (b = -0.260, t (274) = 3.44, p < 0.001). As expected, the significantrelationship between PreSSE and FOA was negative, indicating that participants with weakerPreSSE beliefs more faithfully appropriated the CSSTS than participants with stronger PreSSEbeliefs. These results support H1 and replicate the findings of study 1.
Hypothesis 2. Hypothesis 2 proposed that through faithful appropriation of the CSSTS, lowPreSSE participants would realise a greater increase in PostSSE than would high PreSSEparticipants. To evaluate the influence of FOA on PostSSE across the low and high PreSSEgroups, the same dichotomisation procedure described in study 1 was utilised. An ANCOVA
Table 8. Study 2 factor loadings
Study 2
1 2 3 4
ACC 1 0.886 0.118 -0.024 0.056
ACC 2 0.873 -0.057 0.017 0.040
ACC 3 0.871 0.057 0.001 0.127
ACC 4 0.868 0.064 -0.062 0.003
ACC 5 0.865 0.070 -0.015 0.106
ACC 6 0.846 -0.040 0.002 0.103
ACC 7 0.827 0.128 -0.089 0.040
ACC 8 0.824 0.066 -0.034 0.105
OE 1 0.036 0.856 -0.040 0.043
OE 2 0.093 0.847 -0.132 0.043
OE 3 0.002 0.841 0.052 0.002
OE 4 0.037 0.768 -0.044 0.101
OE 5 0.107 0.729 -0.094 0.037
OE 6 0.106 0.612 0.138 -0.071
CA 1 -0.016 -0.080 0.858 0.040
CA 2 -0.023 -0.008 0.858 0.033
CA 3 -0.086 -0.066 0.784 0.042
CA 4 0.039 0.049 0.649 0.006
FOA1 0.038 0.064 0.090 0.836
FOA2 0.145 0.068 0.041 0.812
FOA3 0.203 0.020 -0.021 0.704
Learning method appropriation 19
© 2013 Wiley Publishing Ltd, Information Systems Journal 24, 3–27
compared the differences in database self-efficacy gains across the two participant groups. Nocovariates were significant. As shown in Table 9, a significant difference between the groupsemerged (F (1, 274) = 104.15, p < 0.001, h2 = 0.28). Specifically, low PreSSE participantsrealised a greater increase in database self-efficacy (PreSSE = 9.56, PostSSE = 53.29) thandid high PreSSE participants (PreSSE = 52.31, PostSSE = 64.69).
Consistent with study 1, additional analyses were performed to confirm FOA drives PostSSEincreases. Specifically, FOA levels within the low and high PreSSE groups detected whetherappropriation affected subsequent changes in database self-efficacy. Table 9 provides theANCOVA results. Within the low PreSSE group, high appropriation participants experienced a
Table 9. Results for study 2 – Hypothesis 2
Independent variable n
Software self-efficacy Type III fixed effects results
M(pre) M(post) df F p h2
Participant type 274 104.15 0.000 0.28
Low PreSSE 137 9.56 53.29
High PreSSE 143 52.31 64.69
Covariate n Estimate SE df t p h2
Gender 280 -2.306 3.184 274 -0.724 0.470 0.002
GPA 280 1.777 1.491 274 1.192 0.234 0.005
Computer anxiety 280 0.444 1.211 274 0.366 0.715 0.000
Outcome expectancies 280 1.906 0.980 274 0.980 0.328 0.003
Independent variable n M(pre) M(post) df F p h2
Appropriation (low PreSSE) 131 12.205 0.001 0.085
Low 64 11.64 47.18
High 73 7.74 56.81
Covariate n Estimate SE df t p h2
Gender 137 5.062 4.516 131 1.121 0.264 0.009
GPA 137 3.668 2.152 131 1.705 0.091 0.022
Computer anxiety 137 -1.087 1.579 131 -0.689 0.492 0.004
Outcome expectancies 137 -0.604 2.516 131 -0.240 0.811 0.000
Independent variable n M(pre) M(post) df F p h2
Appropriation (high PreSSE) 137 0.808 0.370 0.006
Low 66 51.29 59.68
High 77 55.00 68.98
Covariate n Estimate SE df t p h2
Gender 143 -5.705 4.486 137 -1.272 0.206 0.012
GPA 143 0.973 2.033 137 0.479 0.633 0.002
Computer anxiety 143 1.862 1.819 137 1.024 0.308 0.008
Outcome expectancies 143 1.289 3.170 137 0.407 0.685 0.001
20 A M Hardin et al.
© 2013 Wiley Publishing Ltd, Information Systems Journal 24, 3–27
greater increase in database self-efficacy (PreSSE = 7.74, PostSSE = 56.81) than low appro-priation participants (PreSSE = 11.64, PostSSE = 47.18). The difference was significant (F (1,131) = 12.205, p = 0.001), reinforcing the notion that faithful appropriation leads to greaterincreases in SSE. Regarding the high PreSSE participants, no significant difference emerged(F (1, 137) = 0.808, p = 0.370), arguably for the same reasons described in study 1. In sum,these results not only support Hypothesis 2, but also reproduce study 1 findings. Faithfulappropriation appears to lead to greater increases in subsequent self-efficacy beliefs.
Hypothesis 3. Hypothesis 3 predicted that PostSSE would serve as a significant, positivepredictor of objective database skills performance. Given the expected linear relationship, aregression model was executed. Table 7 illustrates the results. PostSSE, computer anxiety,outcome expectations, GPA and gender were entered as independent variables, whereas theobjective assessment score was entered as the dependent variable. After controlling for thesignificant covariates GPA (b = 0.180, t (274) = 3.152, p = 0.002) and computer anxiety (b =-0.155, t (274) = 2.656, p = 0.008), PostSSE was a significant, positive predictor of theobjective measure of database performance (b = 0.259, t (274) = 4.554, p < 0.001). Theseresults support Hypothesis 3 and confirm the findings of study 1.
DISCUSSION
The purpose of this research involved answering three important research questions. The firstsought to examine whether PreSSE influenced appropriation in high learner control CSSTSenvironments. Across spreadsheet and database training sessions, the results revealed aninverse relationship between PreSSE and FOA. Trainees with weaker PreSSE beliefs morefaithfully appropriated the CSSTS than participants with stronger PreSSE beliefs. In essence,those who doubted their capabilities leveraged more of the CSSTS features during therespective training sessions. More confident trainees, on the other hand, appropriated theCSSTS less faithfully, essentially constraining the learning dimensions by favouring onestructural feature over the others. These findings extend and inform Gupta & Bostrom [2009]by establishing that individual differences, such as software self-efficacy, play a criticallyimportant role in influencing appropriation in high learner control environments. This finding,however, runs contrary to conventional AST wisdom. Rather than making more faithful appro-priation moves, participants with stronger PreSSE beliefs tended to constrain the learningstructures rather than using them in combination. Strong PreSSE beliefs appear to underminea trainee’s motivation to utilise the entire set of structural features, despite their documentedbenefits.
The second research question asked whether faithful appropriations in high learner control,CSSTS environments leads to increased software self-efficacy beliefs. Our findings suggestthat participants who faithfully appropriated the CSSTS realised greater increase in softwareself-efficacy (PostSSE) than those who appropriated the CSSTS less faithfully. To furtherexplore this issue, appropriation levels within the low and high PreSSE groups were assessed.
Learning method appropriation 21
© 2013 Wiley Publishing Ltd, Information Systems Journal 24, 3–27
Within the low PreSSE group, more faithful appropriation led to greater increases in PostSSE.The high PreSSE group, on the other hand, often failed to utilise the CSSTS features set incombination, restricting the structural dimensions available to them. Given the low levels offaithful appropriation, no significant increases in PostSSE emerged. These patterns wereconsistent across spreadsheet and database training sessions, serving to emphasise theprominent role that appropriation plays in CSSTS environments.
These findings not only shed light on EUT research, but also inform the high learner controlliterature, which has typically found that individuals with stronger self-efficacy beliefs tend toperform better in such settings. A credible explanation exists for this contradiction. Most highlearner control research has relied on general learning self-efficacy, while the current studyfocused on task-specific software self-efficacy. To facilitate predictiveness, the generality ofself-efficacy beliefs should match the relevant outcome variables they intend to predict[Bandura, 1997]. When examining outcomes confined to a particular task, software self-efficacy should prove to be a stronger predictor compared with decontextualised generalbeliefs. Understanding the nature of these differences in self-efficacy generality provides anexciting avenue for future research in high learner control training environments.
The third research question sought to understand whether findings from prior EUTresearch can be generalised beyond spreadsheet training to other applications. Answeringa call for research targeting computer-based training in new software settings [Gupta,Bostrom, & Huber, 2010], the results obtained from database training were remarkablysimilar to spreadsheet training. In fact, all hypotheses were supported across the studies.Nonetheless, the effects of faithfully appropriating the structural features, and the dimen-sions they provide, appear to be more pronounced in settings where trainees have hadminimal exposure to the software. Consequently, examining database software skills acqui-sition moved the literature beyond the examination of training outcomes in spreadsheetenvironments, where many participants possess relatively extensive experience. Expandingon this contribution, a psychometrically sound measure of database self-efficacy was devel-oped. Researchers can now confidently apply this measure in future research. While theseresults hold promise, the extent to which they generalise to other software applications (e.g.application development, web design) provides an interesting avenue for research incomplex software training environments.
In addition to addressing three important research questions, this research contributes to theliterature in a number of other ways. Built upon AST, Gupta & Bostrom [2009] anticipated thepotential of self-efficacy as an influential factor in training method appropriation. To ourknowledge, no empirical evidence existed to support such claims. Our results clearly show thatthrough faithful appropriation, CSSTSs enhance learning outcomes. Combining the comple-mentary elements of AST and SCT provided a clearer picture of the mechanisms underlyingtrainee learning processes in CSSTS environments and ultimately software training success.
In addition, this research put forth a new conceptualisation of appropriation well suited forstudying individual-level phenomena in CSSTS contexts. Establishing a link between learningdimensions and FOA, defined in this context as frequency of use across all features, providesa novel conceptualisation of FOA. This theoretical connection was established by linking
22 A M Hardin et al.
© 2013 Wiley Publishing Ltd, Information Systems Journal 24, 3–27
learning dimensions to the respective CSSTS features. The establishment of these theory-based relationships provides researchers with a potential new method for assessing trainingsystem appropriation at the individual level.
Finally, to our knowledge this research is the first study to examine appropriation in a highlearner control CSSTS environment. As the results attest, FOA plays a crucial role in softwareskills acquisition. Understanding appropriation in these environments is vital, as it affordsacademicians and practitioners alike the opportunity to develop interventions that help usersutilise the structural dimensions, improving learning outcomes in organisational training con-texts. Our studies clearly show that high levels of appropriation are essential to developing thesoftware skills demanded in today’s highly competitive marketplace. These findings providevaluable insights into how theoretical propositions associated with EUT operate in real worldsettings. Not only were the studies conducted in a realistic setting, but university studentsconstituted the actual users for which the CSSTS was designed. For all intents and purposes,the current context represents an organisational setting, allowing us to examine whether priorexperimental findings could be generalised to a real training environment.
Limitations and future research
Like any research undertaking, our study is not without limitations. While this research used asample appropriate to our research question – examining a software artefact and its userswithin the context of a typical computer skills training course – using students enrolled in abusiness pre-certification course may be viewed as a limitation in terms of generalising ourfindings. However, our intent was not to generalise our findings to contexts outside of similartraining environments. Rather, our purpose was to generate a theoretical perspective thatcould be evaluated for its generalisability in other settings [Lee & Baskerville, 2003]. None-theless, future research should attempt to replicate our results using training participants withvarious backgrounds, as well as in different settings.
A field study methodology was used to investigate learning in a high learner control CSSTSenvironment. Although the implementation of a controlled experimental design would haveeliminated alternative explanations for our findings, it would have artificially restricted the veryhigh learner control environment that trainees in these environments typically confront.Lending credence to our findings, the results were replicated across two separate trainingsessions using two different software packages. Nonetheless, no methodology can be deemedperfect. While field studies often sacrifice rigorous controls, experimental designs have theirown disadvantages, such as the creation of artificial environments [Saks, 1995]. In addition,previous studies of computer self-efficacy have used similar designs [Compeau & Higgins,1995; Agarwal et al., 2000], where users interact with systems in a field setting. Moreover, theimportance of confirming theoretical relationships using non-experimental methods has beenrecognised elsewhere [Dennis et al., 2001]. Consequently, we chose a methodology thatmaximised a realistic setting rather than forcing participants to utilise an artificial CSSTS thatmay not reflect reality. Regardless, future research should use alternative methodologies tobuild upon our findings. For example, experimental studies could be used to compare partici-
Learning method appropriation 23
© 2013 Wiley Publishing Ltd, Information Systems Journal 24, 3–27
pants who were free to select any feature, with participants who were guided through featureselection as predicted by learning theories. Such a design could lead to a better understandingof how the ordered progression of feature selection and use may affect computer skillsacquisition in computer-based training environments. This could also help ascertain whethertrainees, who utilise all the features without guidance, select features in a manner consistentwith prior research, which has demonstrated the superiority of behavioural modelling followedby retention enhancement and hands-on enactive mastery. The limitation of this approach,however, is that the artificial setting would no longer constitute a high learner control environ-ment, as the use of CSSTS features would be dictated by the experimental environment.Future research should consider the balance of these approaches.
Measuring FOA using a three-item, seven-point Likert-type scale is subject to recall bias. Inessence, participants may have forgotten how often they used particular features during thetraining sessions. However, a self-reported measure was adopted to maintain consistency withour research goals, namely, observing learners using CSSTS in a natural setting as opposedto an experimental one. Overt monitoring possesses the very real potential of altering thebehaviours we wanted to observe [Shadish et al., 2002]. Supporting this decision, priorresearch has highlighted the benefits of self-reports for investigating intercorrelations amongperceptions in the early stages of research [Spector, 1994], criteria applicable to our effort. Inaddition, our FOA measure differs from those previously used in the AST literature. Forinstance, neither attitude nor consensus was assessed. However, we believe our measure ofFOA contributes to the field by offering a novel method for assessing appropriation in highlearner control EUT environments. Nonetheless, future research should both examine ourconceptualisation of FOA and investigate our findings using alternative measurementapproaches.
An additional limitation involves our assessment of training effectiveness for the highPreSSE participants. Because PreSSE scores approached the scale’s upper bound, ceilingeffects could have occurred. In essence, the measure may not have provided sufficient roomto detect increases in software self-efficacy. Study 1 showed a non-significant increase insoftware self-efficacy for the high PreSSE group. Several students who entered the spread-sheet training session reported high PreSSE, meaning that ceiling effects could haveaccounted for the result. Countering this notion, however, the same pattern of results wasobserved in the database training context, where participants reported much lower levels ofPreSSE. In this case, the self-efficacy measure contained plenty of room to detect increases.Therefore, ceiling effects can be confidently ruled out as a rival explanation. Nonetheless,future research should account for other types of software applications to identify the boundaryconditions of our findings.
CONCLUSION
This study investigated the viability of CSSTSs for improving software skills acquisition duringtwo separate software training sessions. Theoretical linkages between specific CSSTS fea-
24 A M Hardin et al.
© 2013 Wiley Publishing Ltd, Information Systems Journal 24, 3–27
tures and the sources of self-efficacy information were established. Using AST and SCT as atheoretical guide, PreSSE played an important role in influencing appropriation in high learnercontrol CSSTSs environments. Contrary to previous AST research, self-efficacy beliefs wereinversely related to faithful appropriation. Participants with weaker PreSSE more faithfullyappropriated the CSSTS than participants with stronger PreSSE beliefs. Consistent with thereading of the system’s spirit, low PreSSE participants combined the learning structures byrelating the structures to other structures, while high PreSSE participants constrained thestructures by favouring one structure over the other. In turn, those participants who morefaithfully appropriated the system enjoyed greater increases in software self-efficacy. Inessence, faithfully appropriating CSSTS features is crucial for users to acquire the advancedsoftware skills necessary to succeed in a highly competitive marketplace. Armed with thisknowledge, IS researchers and practitioners alike are now better positioned to improvelearning outcomes in organisational training contexts.
REFERENCES
Agarwal, R., Sambamurthy, V. & Stair, R. (2000) The
evolving relationship between general and specific com-
puter self-efficacy – an empirical assessment. Informa-
tion Systems Research, 11, 418–430.
Anonymous (2010) Industry report. Training, 44, 8–24.
Bandura, A. (1997) Self-Efficacy: The Exercise of Control.
W. H. Freeman/Times Books/Henry Holt & Co, New York.
Bandura, A. (2005) Guide for constructing self-efficacy
scales. In: Adolescence and Education, Self-Efficacy
Beliefs of Adolescents, Vol. 4, Pajares, F. & Urdan, T.
(eds), pp. 1–39. Information Age Publishing, Greenwich.
Bandura, A., Adams, N.E. & Beyer, J. (1977) Cognitive
processes mediating behavioral change. Journal of Per-
sonality and Social Psychology, 35, 125–139.
Bostrom, R.P., Olfman, L. & Sein, M.K. (1994) The impor-
tance of learning style in end user training. MIS Quar-
terly, 14, 101–119.
Brown, K.G. (2001) Using computers to deliver training:
which employees learn and why? Personnel Psychol-
ogy, 54, 271–296.
Compeau, D.R., Gravill, J., Haggerty, N. & Kelly, H. (2006).
Computer self-efficacy: A review. In D. Galletta & P.
Zhang (Eds.), Human-Computer Interaction in Manage-
ment Information Systems: Foundations, Advances in
Management Information Systems (Vol. 5, pp. 225–
261). Armonk, NY: M. E. Sharpe.
Compeau, D.R. & Higgins, C.A. (1995) Application of
social cognitive theory to training for computer skills.
Information Systems Research, 6, 118–143.
Compeau, D.R., Higgins, C.A. & Huff, S. (1999). Social
cognitive theory and individual reactions to computing
technology: A longitudinal study. MIS Quarterly, 23(2),
145–158.
Davis, S.A. & Bostrom, R.P. (1993) Training end users: an
experimental investigation of the roles of the computer
interface and training methods. MIS Quarterly, 17,
61–85.
Davis, S.A. & Bostrom, R.P. (1994) The effects of an
intrinsically motivating instructional environment on soft-
ware learning acceptance. Information Systems Journal,
4, 3–26.
Dennis, A., Wixom, B. & VandenBerg, R. (2001) Under-
standing fit and appropriation effects in group support
systems via meta-analysis. MIS Quarterly, 25, 167–193.
DeRouin, R.E., Fritzsche, B.A. & Salas, E. (2004) Optimiz-
ing learning: research-based guidelines for learner-
controlled training. Human Resource Management, 43,
147–162.
DeSanctis, G. & Poole, M.S. (1994) Capturing the com-
plexity in advanced technology use – adaptive structu-
ration theory. Organization Science, 5, 121–147.
Giddens, A. (1984) The Constitution of Society. University
of California Press, Los Angeles, CA.
Gist, M. & Mitchell, T.R. (1992) Self-efficacy: a theoretical
analysis of its determinants and malleabilty. Academy of
Management Review, 17, 183–211.
Gist, M., Schwoerer, C. & Rosen, B. (1989) Effects
of alternate training methods on self-efficacy and
Learning method appropriation 25
© 2013 Wiley Publishing Ltd, Information Systems Journal 24, 3–27
performance in computer software training. Journal of
Applied Psychology, 74, 884–891.
Gupta, S. (2008) New Approaches to End-User Training.
VDM Verlag, Saarbrucken, Germany.
Gupta, S., Bostrom, R.P. & Huber, M. (2010). End User
Training: What we Know, Need to Know, The Database
for Advances in Information Systems, 42, 9–39.
Gupta, S. & Bostrom, R.P. (2009) Technology-mediated
learning: a comprehensive theoretical model. Journal of
the Association for Information Systems, 10, 686–
714.
Johnson, R.D. & Marakas, G.M. (2000) Research report:
the role of behavioral modeling in computer skills acqui-
sition: toward refinement of the model. Information
Systems Research, 11, 402–417.
Lee, A.S. & Baskerville, R.L. (2003) Generalizing general-
izability in information systems research. Information
Systems Research, 14, 221–243.
Marakas, G.M., Yi, M.Y. & Johnson, R.D. (1998) The mul-
tilevel and multifaceted character of computer self-
efficacy: toward clarification of the construct and an
integrative framework for research. Information Systems
Research, 9, 126–163.
Olfman, L. & Bostrom, R.P. (1991) End-user software
training: an experimental comparison of methods to
enhance motivation. Information Systems Journal, 1,
249–266.
Olfman, L. & Mandviwalla, M. (1994) Conceptual versus
procedural software training for graphical user inter-
faces: a longitudinal field experiment. MIS Quarterly, 18,
405–426.
Olfman, L. & Mandviwalla, M. (1995) An experimental
analysis of end-user training manuals. Information
Systems Journal, 5, 19–36.
Pajares, F. (2004) Current directions in self-efficacy
research. In: Advances in Motivation and Achievement,
Vol. 10, Maehr, M. & Pintrich, P.R. (eds), pp. 1–49. JAI
Press, Greenwich.
Piccoli, G., Ahmad, R. & Ives, B. (2001) Web-based virtual
learning environments: a research framework and pre-
liminary assessment of effectiveness in basic IT skills
training. MIS Quarterly, 25, 401–426.
Saks, A.M. (1995) Longitudinal field investigation of the
moderating and mediating effects of self-efficacy on the
relationship between training and newcomer adjust-
ment. Journal of Applied Psychology, 80, 211–235.
Shadish, W.R., Cook, T.D. & Campbell, D.T. (2002)
Experimental and Quasi-Experimental Designs for
Gneralized Causal Inference. Houghton Mifflin
Company, New York.
Speckler, M.D. (2006) A Report on the Success of Train
and Assess ITTM in Higher Education Microsoft OfficeTM
Instruction. Prentice Hall, Boston, MA.
Spector, P.E. (1994) Using self-report questionnaires in
OB research: a comment on the use of a controversial
method. Journal of Organizational Behavior, 15, 385–
392.
Thomas, M.T., Gupta, S. & Bostrom, R.P. (2008) A meta-
theory for understanding IS in socio-technical systems.
Paper presented at the Hawaii International Conference
on System Sciences, Hawaii.
Williams, M.D. (1996) Learner control and instructional
technologies. In: Handbook of Research for Educational
Communications and Technology, Jonassen, D.H. (ed.),
pp. 170–198. Macmillan, New York.
Yi, M.Y. & Davis, F. (2001) Improving computer training
effectiveness for decision technologies: behavior mod-
eling and retention enhancement. Decision Sciences,
32, 521–544.
Zimmerman, B.J. (1995) Self-regulation involves more
than metacognition: a social cogntive perspective. Edu-
cational Psychologist, 30, 217–211.
Biographies
Andrew M. Hardin is the Director of the Center for Entre-
preneurship and an Associate Professor in the Lee Busi-
ness School at the University of Nevada, Las Vegas.
Professor Hardin’s research is focused on organizational
collaboration and virtual work, financial decision support
systems, and research methodologies. His work has
appeared in journals such as Management Science, MIS
Quarterly, Organizational Behavior and Human Decision
Processes, Journal of Management Information Systems,
European Journal of Information Systems, Information
Systems Journal, Journal of the Association for Informa-
tion Systems, The DATA BASE for Advances in Informa-
tion Systems, Group Decision and Negotiations, Small
Group Research, and Educational and Psychological Mea-
surement. Hardin currently serves as Senior Editor for the
Information Systems Journal and The DATA BASE for
Advances in Information Systems, Senior Associate Editor
for the European Journal of Information Systems, and
Guest Associate Editor for MIS Quarterly.
Clayton A. Looney is an Associate Professor and the
Ron and Judy Paige Faculty Fellow in the School of Busi-
ness Administration at The University of Montana. He
earned his Ph.D. in Information Systems from Washington
State University. Leveraging expertise in human-computer
interaction, cognitive psychology, and behavioral econom-
26 A M Hardin et al.
© 2013 Wiley Publishing Ltd, Information Systems Journal 24, 3–27
ics, his cross-disciplinary research focuses on designing
technologies to overcome decision-making biases. His
work has appeared in Management Science, Organiza-
tional Behavior and Human Decision Processes, Decision
Sciences, Information Systems Journal, Communications
of the ACM, Communications of the Association for Infor-
mation Systems, Journal of Computer Information
Systems, Journal of Information Technology Education,
and Group Dynamics:Theory, Research, and Practice as
well as various international conferences.
Mark A. Fuller is the Dean and the Thomas O’Brien
Endowed Chair of the Isenberg School of Management at
the University of Massachusetts Amherst. Professor Fuller
received his Ph.D. in Management Information Systems
from the University of Arizona. His research focuses on
virtual teamwork, technology supported learning, and trust
and efficacy in technology-mediated environments, and
has appeared in outlets such as Information Systems
Research, Management Information Systems Quarterly,
Journal of Management Information Systems, Decision
Sciences, Journal of the Association for Information
Systems, Journal of Organizational Behavior, IEEE Trans-
actions of Engineering Management, and Decision
Support Systems. Professor Fuller has won multiple teach-
ing awards, has published a textbook on Information
Systems Project Management, and has taught graduate
and undergraduate courses on a variety of topics, includ-
ing global information systems and strategy, information
systems project management, and collaborative
technology.
APPENDIX 1
Instrument development
Fourteen items were developed by the authors based upon a review of the computer self-efficacy literature. Consistent with the existing measure of spreadsheet self-efficacy used instudy 1, items were generated that assessed participants beliefs in their abilities to completespecific database exercises. For example, I believe I have the ability to build a report.Consistent with the self-efficacy literature, a two-part, 0–100 scale anchored by ‘cannot do’ (0)and ‘totally confident’ (100) was used [Bandura, 2005]. Content validity was established byhaving self-efficacy researchers review the items.
Measurement model results
To establish the factor structure of the database self-efficacy measure, the instrument wasadministered to 476 participants during various database training sessions. The factor struc-ture was examined by conducting a confirmatory factor analysis using AMOS 19. Initial fitwas unsatisfactory due to the presence of correlated error terms. The associated itemswere deleted and a model with eight items was re-evaluated. Fit for the new model was good,c2 (20, n = 476) = 55.98, p < 0.001), CMIN/DF = 2.79, CFI = 0.990, NFI = 0.985, GFI = 0.970,AGFI = 0.945, RMSEA = 0.065 (confidence interval = 0.045–0.087).
As a final step, the measurement model was revaluated using the study 2 data. Once again,model fit was good, c2 (20, n = 280) = 37.98, p = 0.009), CMIN/DF = 1.90, CFI = 0.991, NFI =0.982, GFI = 0.968, AGFI = 0.943, RMSEA = 0.057 (confidence interval = 0.028–0.084). Basedon these respective tests, the eight-item measure of database self-efficacy was used in thesubsequent analysis. As a final test, support for Hypotheses 3 in study 2 established thepredictive validity of the database self-efficacy measure. The database self-efficacy items arelisted in Table 3.
Learning method appropriation 27
© 2013 Wiley Publishing Ltd, Information Systems Journal 24, 3–27