An empirical test of the DeLone-McLean model of information system success
Post on 10-Aug-2016
An Empirical Test of the DeLone-McLean Model of Information System Success Juhani Iivari University of Oulu
Acknowledgment I wish to express my gratitude to Minna Perl, M.Sc., for the data collection, and especially to Prof. Wynne Chin for his comments and for helping me to use PLS. This paper was submitted in February of 2002. Wynne Chin served as the Senior Editor.
Abstract This paper tests the model of information system success proposed by DeLone and McLean using a field study of a mandatory information system. The re-sults show that perceived system quality and perceived information quality are significant predictors of user satisfaction with the system, but not of system use. Perceived system quality was also a significant predictor of system use. User satisfaction was found to be a strong predictor of individual impact, whereas the influence of system use on individual impact was insignificant.
ACM Categories: J.1, K.6.2
Keywords: Information System Success, Infor-mation System Quality, System Quality, Information Quality, User Satisfaction, Use, Individual Impact Introduction Seddon et al. (1999) estimate that the total annual worldwide expenditure on information technology (IT) probably exceeds one trillion US dollars per year and is growing at about 10% annually. At the same time, information systems are pervading almost all aspects of human life. In view of the high investments in IT and its ubiquity, the success of such investments and the quality of the systems developed is of the utmost importance both for research and in practice.
This paper focuses on the success of individual information system applications. Following Gustafsson et al. (1982), we interpret an information system (IS) as a computer-based system that provides its users with information on specified topics in a certain organizational context. DeLone and McLean (1992) proposed in their influential paper a framework for IS success measures that distinguishes system quality, information quality, user satisfaction, use, individual impact and organizational impact. They also suggested a causal model for the success measures.
Despite the considerable interest in the DeLone-McLean model1, there is a dearth of studies that test it empirically. DeLone and McLean (2002) identify only sixteen empirical studies that have explicitly tested some of the associations of the original DeLone-McLean model. Among them Seddon and Kiew (1994) revised it considerably, by deleting system use and substituting perceived usefulness. In our view perceived usefulness reflects more the individual impact (Rai et al., 2002), i.e. the impact of the system
1 The Science Citation Index, Social Science Citation Index and Arts & Humanities Index identify 235 references to the article (as of January 10, 2002).
8 The DATA BASE for Advances in Information Systems - Spring 2005 (Vol. 36, No. 2)
on a users performance of his/her job.2 The idea of this paper is to test the DeLone-McLean model while sticking more faithfully to its original form. Leidner (1998) reports a partial test of the model in the case of Executive Information Systems, and more recently, Rai et al. (2002) tested both the DeLone-McLean (1992) model and the Seddon (1997) model, reporting reasonable support for both.
The composition this paper is as follows: Section 2 discusses the theoretical background; Section 3 introduces the research method; Section 4 describes the results; Section 5 discusses the results; and Section 6 concludes the paper. Theoretical Background The DeLone-McLean Model for IS Success
The DeLone-McLean model for IS success, described in Figure 1, assumes that system quality and information quality, individually and jointly, affect user satisfaction and use. It also posits use and user satis-faction to be reciprocally interdependent, and presumes them to be direct antecedents of individual impact, which should also have some organizational impact.
DeLone and McLean (1992) characterize system quality as desired characteristics of the information system itself, and information quality as desired characteristics of the information product. More concretely, they incorporate four scales from the Bailey-Pearson (1983) instrument into system quality (convenience of access, flexibility of the system, inte-gration of the system and response time) and nine scales into information quality (accuracy, precision, currency, timeliness, reliability, completeness, conciseness, format and relevance).
Much of the research on User Information Satisfaction has concerned users satisfaction with specific features of a system (Doll & Torkzadeh, 1988; Iivari & Koskela, 1987) or IS function (Bailey & Pearson, 1983; Baroudi & Orlikowski, 1988), covering features of both system quality and information quality. Even though the inclusion of service quality in the updated DeLone and McLean (2002) model reflects IS functions or IS organizations rather than IS application, the following will focus on the success of IS applications only. User satisfaction in DeLone and McLean (1992) refers to the overall user satisfaction (Seddon & Kiew, 1994) measured independently of
2 Davis original measure for perceived usefulness was developed to assess ex ante expectations of individual impact. We focus here, however, on ex post individual impact measured after six months experience of use.
system quality and information quality. Otherwise the relationship between system/information quality and user satisfaction would be an artifact of measure-ment.
Seddon (1997) claims that the DeLone-McLean model is ambiguous in the sense that one component of it, use, has three potential meanings (Table 1). His conclusion is that only Meaning 1 is justified in the light of the objections listed in the second column of Table 1. To me all these points of criticism seem questionable. His criticisms of Meaning 2 and Meaning 3 refer to the distinction between a variance model and a process model (Mohr, 1982).3 Without going into the details of this distinction, it is obvious that even though IS use as a process is assumed to lead to individual impact and organizational impact, it is not necessary to regard it as a discrete event to be stated (use vs. non-use), as implied by process theories (Mohr, 1982).4 This paper interprets use as the amount of use, which may be considered one measure of IS success.
DeLone and McLean (1992) characterize individual impact as an indication that an information system has given a user a better understanding of the decision context, has improved his or her decision-making productivity, has produced a change in user activity, or has changed the decision makers perception of the importance or usefulness of the in-formation system (p. 69). Seddon (1997) reinterprets individual impact to mean benefits accruing to individuals from use. Even though both DeLone and McLean (1992) and Seddon (1997) implicitly presuppose that individual impact is of benefit to the user, this paper interprets individual impact as referring to a unit of analysis rather than the benefi-ciary of the impact.5
3 As Mohr (1982, p.44) points out, he uses the term process theory in a highly specific meaning. Process theory does not imply that variance theories cannot address processes (e.g. IS acceptance). 4 We interpret the DeLone-McLean model as based on the reasoning that a system that is not used at all does not have any individual or organizational impact. On the other hand, the DeLone-McLean model also allows the hypothesis that more use is associated with more individual impact, which follows the logic of variance theories. 5 The likely explanation for this assumption is that the above authors implicitly assume that use of the system is voluntary. In that case a user will hardly continue to use a system if he or she does not perceive its use as beneficial. However, according to my reading, DeLone and McLean (1992) do not explicitly restrict their model to voluntary systems, although they do note that actual use makes sense only when system use is voluntary.
The DATA BASE for Advances in Information Systems - Spring 2005 (Vol. 36, No. 2) 9
Figure 1. The DeLone-McLean Model for IS Success
Meaning Seddons objections Counter-objections Meaning 1: Benefits from use
(The only justified mean-ing)
What is the meaning of use in benefits from use?
Meaning 2: Use as the dependent vari-able in a variance model of future use
IS success must bring benefits to somebody.
Cant a system (e.g. a piece of free software such as Linux) be genuinely considered a suc-cess when it is widely used without any consi-deration of its benefits or disadvantages to different stakeholders?
Meaning 3: Use as an event in a pro-cess leading to indi-vidual or organiza-tional impact
IS use is a process con-struct that should not have any place in a variance model predicting IS success.
Even though IS use as a process is assumed to lead to individual impact and organizational impact, it is not necessary to regard it as a discrete event to be stated (use vs. non-use), as implied by process theories.
Table 1. The Three Meanings of IS Use in the DeLone-McLean Model, According to Seddon (1997)
Following DeLone and McLean (1992) and Rai et al. (2002), we will specifically focus in this paper on the effect of an information system on the work perform-ance of individual users as measured by perceived usefulness.6 Hypotheses DeLone and McLean (1992) introduce the model shown in Figure 1 primarily as a causal-explanatory model of how system quality and information quality affect use and user satisfaction, how use and user satisfaction, affecting each other reciprocally, are direct antecedents of individual impact, and how individual impact leads to organizational impact. As an alternative, one could emphasize more the predictive nature of the model, how the preceding variables help to predict the dependent variables, 6 We do claim that perceived usefulness covers all aspects of individual impact. DeLone and McLean (1992) specifically focus on decision-makers as users of an information system. Assuming that the work of decision-makers is to make decisions, perceived usefulness essentially covers the impact on decision-making productivity. Perceived usefulness nevertheless misses those aspects of individual impact which do not directly concern work performance, e.g. the impact on the quality of work (Iivari, 1997).
even though the causal explanation of the relationship is not totally clear. The criticism of Seddon (1997), even though we do not accept it in its entirety, shows that some of the assumed causal relationships in the DeLone-McLean model are arguable and the model is incomplete. In particular, the model misses the feedback loops from individual impact and organiza-tional impact to user satisfaction and use. We interpret the DeLone-McLean model primarily as a predictive one that is worth testing empirically. Based on the DeLone-McLean model, we propose to test the hypotheses depicted in Figure 2 in the present paper. It is hypothesized in Figure 2 that system quality and information quality are positively associated with user satisfaction. Hypothesis H1 assumes that ceteris paribus the higher the system quality is perceived to be by users, the more satisfied they are with the system. Similarly, Hypothesis H2 posits that ceteris paribus the higher the information quality is perceived to be by users, the more satisfied they are with the system. If user satisfaction is interpreted as an attitude (Baroudi et al., 1986), hypotheses 1-2 essentially argue that the attitude is dependent on perceptions of the attitude object (Fishbein & Ajzen,
10 The DATA BASE for Advances in Information Systems - Spring 2005 (Vol. 36, No. 2)
1975; McGuire, 1969). There is considerable empirical evidence for these hypotheses.
S ystem quality
I information quality
Figure 2. The Model to Be Tested
Many of the instruments developed to measure User Information Satisfaction (UIS) in terms of attributes such as system quality and information quality (e.g. Bailey & Pearson, 1983; Ives et al., 1983; Doll & Torkzadeh, 1988) have used an independent measure of overall user satisfaction to test the predictive validity of the measure. They have consistently reported significant correlations between UIS or its factors and the independent measure of overall satisfaction. Doll and Torkzadeh (1988), for example, found correlations varying between 0.51 and 0.65 between the twelve items of their end-user satisfaction measure and the criterion variable that can be interpreted as overall satisfaction. The correlations between the five factors (content, accuracy, format, ease of use and timeliness) and the criterion varied between 0.55 and 0.69, and that between the 12-item instrument and the criterion variable was 0.76. Further, Seddon and Kiew (1994) found in their path analysis that information quality and system quality are significant determinants of overall user satisfaction (both path coefficients significant at the level 0.001).7 Similarly Rai et al. (2002) report significant path coefficients between ease of use (used to measure system quality) and user satisfaction ( = 0.30, p 0.01) and between information quality and user satisfaction ( = 0.52, p 0.01) in their LISREL analysis of the DeLone-McLean model.
DeLone and McLean (1992) hypothesize that the higher the system quality, the more the system is used (Hypotheses H3) and the higher the information quality, the more the system is used (Hypothesis H4).
7 System quality was measured using two items from Doll and Torkzadeh (1988), four items of ease of use from Davis (1989) and three additional items. The measure of information quality consisted of ten items from the Doll and Torkzadeh (1988) instrument. (Overall) satisfaction with the system was measured using four items: how adequately the application meets the information processing needs, how efficient it is, how effective it is and overall satisfaction.
At a general level, there is a considerable body of em-pirical research on the relationship between UIS (measured in terms of attributes such as system quality and information quality) and IS use, which suggests that the relationship is positive but relatively weak (Amoroso & Cheney, 1991; Barki & Huff, 1985; Baroudi et al., 1986; Ginzberg, 1981; Igbaria, 1990; Igbaria & Zviran, 1991; Nelson & Cheney, 1987; Srinivasan, 1985). Baroudi et al. (1986), for example, fou...