FEATURE ARTICLE

Evaluating acceptance of an electronic data managementsystem at a tertiary care institutionWendy Young, PhD; George Klima, PhD; Winston Isaac, PhD, CHE

Abstract—This research reports on satisfaction with the introduction of an electronic document management system in a tertiaryhospital environment. A buffet of training and familiarization options were offered: one-on-one training, open house, drop-in,e-learning, classroom training, and self-study. It was found that professions differ in their pattern of satisfaction with training andthey also differ in their satisfaction with both the usefulness and the ease of use of the system. Satisfaction among administratorswas highest and that among nurses lowest. There was an association between attendance at the open house event and satisfactionwith the system.

New information technologies offer healthcare insti-tutions many benefits, provided acceptance by us-ers is high. A great deal of research has addressed

“acceptance” by studying diffusion of innovations, organi-zational change, human-computer interaction, and marketresearch. Several reviews1,2 and meta-analyses3 have ex-amined the literature, and two points of agreement haveemerged.First, among physicians, perceived usefulness of a new

technology is the one factor most closely correlated withhaving a positive attitude to the new technology andforming an intention to adopt it.4 Perceived usefulness is“the degree to which a person believes that using a par-ticular system will enhance his or her job performance.”5

Second, the ease of use is associated with perceived use-fulness but is not directly associated with intention toadopt4; that is, “. . . no amount of ease of use will compen-sate for low usefulness.”These points are important, but they are not enough to

understand what drives acceptance of Information Tech-nology (IT) and how to design and lead new initiatives. Thisis where models of IT acceptance have been useful. Oneprominent family of models originated with the Technol-ogy Acceptance Model (TAM).6 This family includes theupdated TAM2, the Universal Theory of Acceptance andUse of Technology (UTAUT), the Theory of Reasoned Ac-tion, and the Theory of Planned Behaviour. Within thefamily, the UTAUT model (Fig. 1) is still prominent for its

From the School of Nursing, Memorial University of Newfoundland, StJohn’s, NL, Canada (Young and Klima); School of Health Services Manage-ment, Ryerson University, Toronto, ON, Canada (Isaac).

Corresponding author: Wendy Young, PhD, School of Nursing and Divi-sion of Community Health and Humanities, Memorial University of New-foundland, 300 Prince Philip Drive, St John’s, NL A1B 3V6, Canada.

(e-mail: youngw@mun.ca).Healthcare Management Forum 2011 24:170–1730840-4704/$ - see front matter© 2011 Canadian College of Health Leaders. Published by Elsevier Inc. Allrights reserved.

doi:10.1016/j.hcmf.2011.04.002

ability to predict whether a new technology is likely to beaccepted.6

The UTAUT model says that employees are more likely toadopt a new technology if they (1) expect that new tech-nology will help them do their jobs (“performance expec-tancy”), (2) expect that learning will be manageable (“effortexpectancy”), (3) believe that important people expectthem to use it (“social influence”), and (4) receive fullsupport from the organization (“facilitating conditions”).UTAUT also highlights four mediating factors that candampen or stimulate acceptance.Of course, the UTAUT model is not complete. For exam-

ple, among the determinants, we see “facilitating condi-tions,” one of which is training, a key enabler of adoption.7

It would be useful to have more evidence about the impactof training on acceptance. It would also be useful to ex-plore the mediating factor “experience.” The purpose ofour study was to examine these issues. Our trainers offeredseveral training formats without compulsion or restrictionand tracked the professions of attendees.We witnessed the introduction of a new technology to

one 500�-bed Canadian urban tertiary care teaching hos-pital. Compared with other teaching hospitals, this hospitalis typical in its mix of services including cancer, cardiovas-cular, children’s health, complex care, critical care, diabe-tes, emergency, mental health, seniors’ health, trauma, andwomen’s health. However, the hospital’s clients comprise abroader mix of socioeconomic and ethnic populations thanmost. The introduction of the technology was mandatoryand, therefore, we measured acceptance as satisfactionand not adoption.The present study asks the following questions: (1) Do

health professionals differ in their choice of training op-tions and in their satisfaction with training? (2) Do healthprofessionals differ in their satisfaction with the usefulnessand ease-of-use of a new technology? and (3) Does expo-sure to a training option have an effect on subsequentsatisfaction with the usefulness and ease of use of the new

technology?

EVALUATING ACCEPTANCE OF AN ELECTRONIC DATA MANAGEMENT SYSTEM AT A TERTIARY CARE INSTITUTION

BACKGROUND

In the summer of 2007, the hospital IT staff made avail-able six training options to all staff in preparation for anew Electronic Document Management System (EDMS).The options were (1) classroom instruction, (2) one-on-one training sessions on request, (3) open house ses-sions at scheduled times, (4) drop-in sessions offeredonce a week at noon, (5) e-learning videos via intranet,and (6) self-training. The system went live in December2007.

METHODS

In March 2008, e-mail invitations were sent to hospital staffwho had logged in at least once, asking them to completea web-based survey. A follow-up reminder was also sent.The survey consisted of seven, five-point alternative forcedchoice questions and one open-ended question. The sur-vey had been tested and then reviewed by a committee,who supported its face validity. All survey participants whoprovided an e-mail address were eligible for a $100 giftcard.Data were collected from 193 respondents. Fifty respon-

dents were excluded; 38 individuals did not complete thesurvey and 12 more had never used the application. Validsurveys were received from 143 individuals: 31 physicians,25 residents, 28 nurses, 14 health disciplines clinicians, 8researchers, and 37 administrative staff. The total numberof users in each profession is unknown and, therefore, weare unable to determine the representativeness of our

Figure 1. Summary of UTAUT model.

convenience sample.

Healthcare Management Forum ● Forum Gestion des soins de s

Because of the small numbers in these groups, data fromphysicians and residents were combined into “medicalstaff,” and data from nurses and health disciplines clini-cians were combined into “nurses�.” Data from adminis-trative staff remained in its own group. Data from the 8researchers were excluded because of the sample beingmuch smaller than the others. Furthermore, because of thesmall number of responses, the response categories wereaggregated: “agree” and “strongly agree” were re-coded as“favourable”; “disagree,” “strongly disagree,” and “neitheragree nor disagree” were recoded as “unfavourable orneutral.” This was performed to ensure stability of theresults.

Data analysis

Descriptive statistics, the Pearson chi-square and the Mantel-Haenszel common odds ratio estimate were calculatedusing SPSS V15.0 (SPSS Inc, Chicago, IL). For each trainingor familiarization event we calculated the odds of a favour-able response about the effectiveness of EDMS amongthose who attended compared with those who did notattend. We also calculated the same odds about ease ofuse.

RESULTS

Results are presented as they relate to the training, to theEDMS, and to the association between training taken and

response to the EDMS.

ante – Winter/Hiver 2011 171

Young, Klima, and Isaac

Training

The usage and the satisfaction related to training areshown in Table 1. Although there are some differencesamong professions in the pattern of training utilization, theoverall pattern of utilization was independent of profession(�2

10 � 16.063, P � .0978). Nevertheless, we can see somevariations. For example, we see that medical staff madeleast use of the training options overall. We also see thatadministrators favoured classroom training more than oth-ers. Finally, we see that self-study was the most usedoption among all professions.The pattern of satisfaction with training is, however,

dependent on profession (�210 � 63.356, P � .0001). Only

the medical staff was satisfied with one-on-one training,and only administrative staff expressed a high degree ofsatisfaction with classroom training.

The EDMS

At the time of the survey, 3 months after the installation ofthe EDMS, usage among respondents was moderate; 41%reported using the EDMS at least once a day, 41% at leastonce a month, and 18% less than once per month. Bothusefulness and ease of use were significantly dependenton profession. It was agreed that “EDMS is easy to use” by76% of medical staff, 71% of nurses�, and 87% of admin-istration (�2

2 � 1.435, P � .0488). With regard to useful-ness, it was agreed that “EDMS has improved the overallavailability of patient information” by 57% of medical staff,48% of nurses�, and 78% of administration (�2

2 � 19.924,P � .0000472).

Training options and satisfaction with the EDMS

For each type of training, we calculated the odds ratio of afavourable response to the following: “EDMS is easy to use”and “EDMS has improved the overall availability of patientinformation.” (The question about availability addressesthe question of usefulness because the purpose of EDMS isto make patient information more available.) The result is

Table 1. Percentage of each profession indicating attendance at

Medical staff

Usage (%) Satisfied (%) Us

One-on-one 38 67Open house 29 31Drop-in 36 45e-learning 38 52Classroom 32 33Self-study 79 86Total 252

displayed in Figure 2.

172 Healthcare Management For

Only with the open house sessions did we find a signif-icant influence on perceived usefulness (Mantel-Haenszelcommon odds ratio � 2.045, asymptotic significance [two-sided] � 0.048). In other words, individuals who attendedthe open house were twice as likely to agree that EDMS isuseful than those who did not attend. This result is founddespite the relatively low degree of satisfaction with theopen house event. The odds ratio estimates for the influ-ence of each type of training session on perceived ease ofuse vary from 0.475 to 1.349; none are significant.

DISCUSSION

Our study found that professions differ in their pattern ofsatisfaction with training. They also differ in their satisfac-tion with the usefulness and the ease of use of the EDMS.Individuals who attended an open house event subse-quently expressed satisfaction with the usefulness of EDMSat twice the rate as those who did not attend the openhouse.

Implications and related literature

Venkatesh et al7 have tested UTAUT, an extended TAMmodel, in several organizations under both voluntary andmandatory conditions. Our study addresses two parts of

satisfaction of each training and familiarization option

Nurses� Administration

%) Satisfied (%) Usage (%) Satisfied (%)

33 60 2320 46 5346 49 5654 65 5069 81 8074 84 68

385

Figure 2. Odds ratios of favourable responses among attendees versus

and

age (

574862676981

384

non-attendees of each type of training about usefulness of EDMS.

um ● Forum Gestion des soins de sante – Winter/Hiver 2011

EVALUATING ACCEPTANCE OF AN ELECTRONIC DATA MANAGEMENT SYSTEM AT A TERTIARY CARE INSTITUTION

the UTAUT model in a hospital setting: “facilitating condi-tions” and “experience.” We view training and familiariza-tion as facilitators of acceptance, and we found significanteffects. We saw that a buffet style of training leads indi-viduals to use many options, some of which appear to havelittle effect on acceptance. Health services staff membershave powerful demands on their time and so managerswould be prudent to avoid offering needless training.The association between the satisfaction with training

and satisfaction with the EDMS suggests that at least onekind of familiarization event (ie, the open house) is consid-erably more effective than others in developing percep-tions and attitudes. Managers may want to consider usingan open house format to increase the acceptance of newtechnology. With regard to experience, we note that pro-fessions had differing views on both usefulness and ease ofuse. Managers should ensure that they understand whichfeatures of a new technology are important to each pro-fessional group so that they can communicate that infor-mation effectively.

Limitations

Our study is uncontrolled and, therefore, we cannot ruleout non-representative sampling, self-selection bias, andother confounds. Second, the content of the various train-ing and familiarization events is not based on standardizedmaterials. Variations in response to training and, indeed, tothe association between training and subsequent per-ceived usefulness, could be related to uneven effectivenessof the events. Third, there is no follow-up of training at-tendees to determine whether they were sufficiently ex-posed to the EDMS to be able to offer a valid opinion on itsusefulness and ease of use. Finally, this is a case study thattook place at a single hospital using a single piece of newtechnology.

Future work

The TAM family of models has shown their usefulnessinsofar as they guide both our research and the introduc-tion of new technology in the workplace. The success ofthe open house event suggests that such events could beeven more effective if they were highly targeted to pro-fessional workflow and if they were presented as part ofroutine meetings with high attendance. We were surprisedat the large numbers who declared that they engaged in

“self-study,” which we presume could imply only a minimal

Healthcare Management Forum ● Forum Gestion des soins de s

effort. Instead, we suggest that it may be useful to examine“job aids” techniques as a way to provide just-in-timefamiliarization and training and thereby improve satisfac-tion.

CONCLUSIONS

We have shown some of the complexity related to tech-nology acceptance in multi-professional tertiary hospitalenvironment. We found that satisfaction with types oftraining differs by profession and that the pattern of train-ing utilization mirrors the pattern of training satisfaction(lesson: for each profession, provide training only in theirmost familiar format). We also found that satisfaction withboth ease of use and usefulness differs by profession, withadministrators being most favourable and nurses least fa-vourable (lesson: for each profession, be clear about ben-efits and manage expectations). We have found that anopen house event is associated with a favourable attitudeto a new piece of technology; we were unable to find suchan association with any of the other training and familiar-ization events (lesson: an effective familiarization sessioncan be invaluable for getting people on board). Finally, wehave seen that offering a buffet of training options resultsin individuals attending several training options (lesson: donot give people too many choices).

REFERENCES

1. Yusof MM, Stergioulas L, Zugic J. Health information systemsadoption: findings from a systematic review. Stud Health Tech-nol Inform. 2007;129:262–266.

2. Ward R, Stevens C, Brentnall P, Briddon J. The attitudes ofhealth care staff to information technology: a comprehensivereview of the research literature. Health Info Libr J. 2008;25:81–97.

3. Schepers J, Wetzels M. A meta-analysis of the technologyacceptance model: investigating subjective norm and moder-ation effects. Inform Manage. 2007;44:90–103.

4. Chau PYK, Jen-Hwa Hu P. Investigating healthcare profession-als’ decisions to accept telemedicine technology: an empiricaltest of competing theories. Inform Manage. 2002;39:297–311.

5. Davis FD. Perceived usefulness, perceived ease of use, and useracceptance of information technology. MIS Q. 1989;13:319–39.

6. Holden RJ, Karsh B-T. The technology acceptance model: itspast and its future in health care. J Biomed Inform. 2010;43:159–172.

7. Venkatesh V, Morris MG, Davis GB, Davis FD. User acceptanceof information technology: toward a unified view. MIS Q. 2003:

27:425–478.

ante – Winter/Hiver 2011 173