diagnosing information systems success: importance–performance maps in the health club industry
TRANSCRIPT
Diagnosing information systems success:importance–performance maps in the health club industry
Walter Skok*, Andrew Kophamel, Ian RichardsonKingston Business School, Kingston University, Kingston Hill, Kingston upon Thames, Surrey KT2 7LB, UK
Received 4 September 1999; accepted 20 September 2000
Abstract
There is now a growing global leisure industry that includes health, fitness, and sports clubs. Faced with increasing
competition, organizations within this sector have turned to information systems (IS) as a means of obtaining competitive
advantage. However, complaining of time shortages, lack of expertise, and a shortfall of perceived benefits, management often
find conducting post-implementation evaluations of their IS investments problematic.
Importance-performance (I–P) maps have been successfully applied in the marketing field for many years where they have
been shown to be a simple and effective management tool. However, there is little reported use of I–P maps in the IS field, and
this paper puts forward a case for using them in the diagnosis of IS success. This is illustrated in conjunction with an extension
of DeLone and McLean’s IS success typology for the UK health club industry. # 2001 Elsevier Science B.V. All rights
reserved.
Keywords: IS success; I–P maps; UK health club industry
1. Introduction
American Airlines’ SABRE reservation system was
one of the earliest examples of information systems
(IS) being used to obtain strategic commercial advan-
tages. Since then, there have been numerous quoted
cases discussing its importance [24,40]. Faced with
increasing globalization and competition, firms have
continued to seek potential gains from IS, reflected in
a £44.4 billion investment during 1998 in the UK
alone [18]. Doubts, however, have been raised about
the efficacy of these investments. Indeed, some
authors believe that ‘there is little evidence. . .that
IT investments systematically create value for firms’
[13]. The high failure rates quoted for IS [16,29], the
prominence of some of these, such as the London
Ambulance Service [5], and the debate surrounding
the IS productivity paradox [7,8] have added to an
already contentious debate.
Management, therefore, face a dilemma, in which
the supposed contribution of IS to competitive advan-
tage is pitted against uncertainties as to its actual
value. This has led to a clear need for tracking the
success of IS investments, and one which is well
established within the IS literature [17,35,37,41].
Indeed, it remains a key issue, particularly for larger
organizations [4].
A glance at this body of work reveals that firms are
faced with a number of problems in evaluating their
investments. Not least of these is the overwhelming
breadth and depth of indicators being forwarded as
proxies for IS success and failure: indeed, ‘there are
Information & Management 38 (2001) 409–419
* Corresponding author. Tel.: þ44-20-8547-2000;
fax: þ44-20-8547-7026.
E-mail address: [email protected] (W. Skok).
0378-7206/01/$ – see front matter # 2001 Elsevier Science B.V. All rights reserved.
PII: S 0 3 7 8 - 7 2 0 6 ( 0 0 ) 0 0 0 7 6 - 8
nearly as many measures as there are studies’ [12]. For
example, overall business performance [13], user
satisfaction [2,23], structural changes [9], and quality
of working life [25] have all been used to assess the
impact of IS. More broadly, the literature is replete
with advocates both of standardized accounting tech-
niques, such as return on investment or internal rate
of return [19], and IS-specific measures, such as
Strassman’s return on management [36]. Coupled with
the sensitive and political nature of IS [21], it is hardly
surprising that obtaining agreement on indicators of
successful IS often proves difficult.
Other problems in evaluating IS success have
arisen: attributing causality, timing the evaluation,
and separating the direct from the derived effects of
implementing an IS have all been identified as trou-
blesome. Additionally, and an area that remains under-
explored, is the isolated nature of evaluation, in which
assessments are generally made independently of a
firm’s environment, most notably its competition.
Competitive advantage is, after all, a relative not an
absolute construct, and benchmarking IS success
against a rival’s investments would seem to be an
integral part of IS evaluation.
Given these problems, it is unsurprising that firms
are reticent about carrying out post implementation
evaluations (PIEs). For example, in a study of 92
senior IS executives Kumar [26] found that time
shortages, a lack of qualified personnel, and a shortfall
in perceived benefits all contributed to a general
reluctance to conduct PIEs. Yet accepting the diffi-
culties inherent in assessing the success or failure of an
IS investment, its importance, and the sums of money
being spent suggest that it cannot be left to chance.
Clearly there is a need for a tool, simple enough for
easy management adoption yet sufficiently powerful
to diagnose areas of weakness and to facilitate the
feedback necessary for continuous re-alignment and
improvement. The purpose of this paper is, therefore,
to put forward I–P maps as a tool and to illustrate their
use in a benchmarking role, in conjunction with
DeLone and McLean’s IS success typology, within
the UK health club industry.
2. Importance-performance maps
Importance-performance (I–P) maps originated in
marketing and were developed to facilitate easy
management diagnosis of new product success [14].
Originally conceived in the late 1970s, I–P maps have
continued to be a popular managerial tool, witnessed
by the wide array of recent applications to be found.
Clarke [10], for example, successfully used them to
develop marketing strategies for organizations operat-
ing within the health care industry. Similarly, Babakus
et al. [1] applied them in the airline sector, where a
catering firm was trying to assess the perceptions of
several of its major customers with a view to improv-
ing service quality. Finally, Hollenhorst and Gardner
[22] generated managerial recommendations based on
I–P maps in the US tourist industry. The significance
and reliability of I–P maps, therefore, has been widely
tested [20,30].
Briefly, I–P maps are a matrix-based technique that
seek to present customer perceptions of importance
and performance in an easy to interpret format. Pro-
ducing I–P maps begins with the generation of an
agreed list of elements on which evaluation is con-
ducted. This list is critical to the analysis and is
generally obtained through literature review and inter-
views. From this, survey instruments are developed,
often using Likert or numerical scales, and these
questionnaires are then administered to respondents.
Finally, the importance and performance of the iden-
tified elements are plotted against each other, allowing
comparisons to be made: this may also include com-
petitor information. Fig. 1 summarizes the process of
constructing I–P maps and Fig. 2 presents a standar-
dized I–P framework.
Interpreting I–P maps is relatively straightforward,
involving the mapping of elements to one of four
types. Concentrate here refers to elements that are
Fig. 1. Constructing I–P maps (based on [14]).
410 W. Skok et al. / Information & Management 38 (2001) 409–419
perceived as important but where performance is
lacking. Keep up the good work are those elements
that are both important and are being delivered to a
high level. Low priority elements are those that are
recognized as both being of low importance and only
small performance, making improvement unneces-
sary. Finally, possible overkill refers to elements that
are being over delivered, having high performance and
low importance.
Interpretation can be extended by drawing an iso-
rating line from the origin, along which performance
equals importance [34]. Above this line, importance
exceeds performance, providing opportunity gaps;
below it, performance exceeds importance, suggesting
satiated needs.
I–P maps bring with them many desirable charac-
teristics for the evaluation of attitudes. First, manage-
ment may experience difficulties in understanding and
interpreting the complex statistical analyses often
associated with evaluation, even where access to ran-
dom samples is not an issue [15]. The relative sim-
plicity of I–P maps is, therefore, highly apparent.
Moreover, the simplicity of the technique and its
resulting low cost both facilitate continuous tracking
of IS investments over time, allowing management to
track the progress of any interventions. Second, the
use of visual representations makes the relaying of key
findings to senior management easier, simplifying the
decision making process and allowing more timely
conclusions to be drawn and actions recommended.
Indeed, the use of I–P maps in diagnosing possible
areas for improvement and setting managerial prio-
rities has been widely discussed. Finally, I–P maps are
extremely flexible. The elements used in I–P analysis
can be changed according to the decision making
context, as can the scales on which they are assessed,
with satisfaction or expectations being two alterna-
tives. Thus, I–P maps may fit in with data already
being collected by a firm. I–P maps are also amenable
to the use of competitor data, thereby facilitating the
analysis of a firm’s relative position.
Clearly I–P maps are an example of a portfolio-
based matrix, many of which have already been
widely discussed within the IS discipline and found
to be extremely useful, if simplistic [38,39]. However,
no direct reference to I–P maps could be found. While
Miller has independently demonstrated the use of both
importance and performance dimensions within IS
evaluation [31], no reference was made to their pre-
sentation via I–P maps — the primary purpose of this
paper.
With Kumar suggesting that post-implementation
IS evaluation is neglected because management feel
that it is too complex and too time-consuming, the
advantages that I–P maps appear to offer make them
attractive in an area that is assuming increasing impor-
tance. It is with this in mind, that their suitability in
assessing a firm’s IS success was tested within the
health club industry.
3. Applying I–P maps in the health clubindustry: the case of FitCo
Traditional health and fitness clubs provide a wide
range of sporting and social activities for members
and guests, as well as catering for external groups
through the provision of conference and banqueting
facilities.
As a result of changing attitudes towards fitness,
brought on, in part, by messages put out by govern-
ment campaigns and the media in general [27], UK
spending on health and fitness jumped by 58%
between 1992–1997, from 615 to £972 million [11].
With nearly 2 million people estimated to be members
of 2200 clubs around the UK, Mintel, the marketing
intelligence organization has described the recent
expansion as occurring at ‘breakneck speed’ [6].
Customers are, however, becoming more discerning
and, with the spread of health clubs throughout the
country bringing increased competition, companies
Fig. 2. A standard I–P map.
W. Skok et al. / Information & Management 38 (2001) 409–419 411
are looking at ways of utilizing IS to add customer
value. Typical applications include electronic point of
sale systems and membership databases, implemented
to allow personalized welcomes and the collection of
valuable marketing information.
FitCo is typical of many of the health clubs operat-
ing in the UK, having responded to the recent growth
in the fitness industry by developing its social and
sporting facilities. Complementing this, FitCo has also
invested in IS to develop its customer services as well
as to generate pertinent management information.
However, as with many other organizations, FitCo
is concerned about its IS investments: are they deli-
vering value for money? Are users satisfied with the
systems? How have they affected individuals and the
organization? Thus, there is a managerial need to
assess the value of its IS investments, especially in
the light of competitor performance.
Following the methodology outlined in Fig. 1, I–P
maps highlighting the success or failure of FitCo’s IS
investments can be constructed and interpreted.
3.1. Identification of critical elements
A review of the IS evaluation literature led to the
adoption of DeLone and McLean’s framework of
system quality, information quality, use, user satisfac-
tion, individual impact and organizational impact.
Recognizing the multi-dimensional nature of IS suc-
cess, this model is well established within the litera-
ture and represents a solid starting point in defining
success criteria [3]. However, criticism of the lack of
attention to IS department/user relations [28,32] have
led to calls for the inclusion of indicators of the IS
department’s service quality to be added as a seventh
element. Thus, the framework outlined in Fig. 3 was
adopted, and one which provides the potential for the
tracking of IS investments from technical solutions to
their delivering of real commercial advantage.
Within this broad conceptual framework and con-
sistent with the I–P literature, specific attributes were
operationalized from previous survey instruments
(e.g. [2,28,33]) and through interviews conducted with
senior executives and users at FitCo. Table 1 presents
the completed list of elements used to develop the
instrument.
3.2. Importance and performance ratings
generated for each element
Nine point numerical scales were developed for
both the importance and performance of the elements.
These were anchored by ‘critical’ and ‘not that impor-
tant’ for the former, and ‘poor’ to ‘excellent’ for the
latter. Consistent with previous recommendations,
importance and performance for each item was sepa-
rated to help minimize compounding and order
effects.
The sampling frame for the study was the private
health club industry in the southeast of England, from
which 15 organizations, all direct competitors to
FitCo, were chosen. After initial contact, 11 organiza-
tions agreed to participate in the study allowing
FitCo to assess its IS strengths against those of its
Fig. 3. Model of IS success categories (based on Pitt [32]).
412 W. Skok et al. / Information & Management 38 (2001) 409–419
competitors. Questionnaires were circulated to key
informants with the request to distribute them among
all those members of the organization directly
involved with the firm’s IS. Stamped addressed envel-
opes were included, and all respondents were
guaranteed confidentiality.
In all, 83 responses were received from the 11
organizations; these being broken down by position
in Table 2.
There is a clear spread of responses by position,
with all major groups being represented. However, the
small number of responses in certain categories and
the non-random sampling strategy precludes the use of
statistical analysis in testing potential differences.
This would, moreover, go against the I–P ethos, in
that the aim here is to move away from complex
quantitative techniques and to provide a clear and
simple IS evaluation tool.
3.3. Obtaining mean values
Table 3 presents a breakdown by position for
each of the categories. Visual inspection suggests,
perhaps surprisingly, that there is little difference in
Table 1
Questionnaire elements
Number Description
System quality
1 The computer hardware are up-to-date
2 The computer software is up-to-date
3 The system does not crash regularly
4 The speed of the system’s response
5 Control of people entering the computer system
Information quality
6 The system’s ability to reduce the number of errors that might be otherwise made
7 The system’s ability to provide useful information for management
8 Ability of the system to produce professional reports
Service quality
9 A high degree of technical competence from systems support staff
10 Extent of practical user training
11 Fast response from systems support staff to remedy problems
12 Positive and helpful attitude of computer staff to those who use the system
Use
13 It should be easy for users to access the computer facilities
14 Involvement of users in planning the requirements of the computer system
User satisfaction
15 User confidence in the system
16 Users’ understanding of the overall computer system and what it can do
Individual impact
17 Ability of the system to improve staff productivity
18 The system’s ability to allow faster processing of people/information
19 The system’s ability to provide improved staff morale
Organizational impact
20 The system’s ability to provide competitive business advantage over rival firms
21 The system’s ability to provide improved client/seller relationships
22 Overall cost effectiveness of the information system
23 The system’s ability to provide opportunity for developing extra income
24 The system’s ability to provide improved corporate image
25 The system’s ability to provide improved customer service
26 The system’s ability to keep up with the organization’s business requirements
W. Skok et al. / Information & Management 38 (2001) 409–419 413
importance and performance between positions for
each of the categories. For this reason, organizations
were subsequently treated in their entirety. Table 4
presents the mean values, by element, for FitCo, and
Table 5 shows the means of each of the categories for
both FitCo and the sample of the health club industry
in general.
3.4. Importance-performance maps
Fig. 4, which is based on Table 4, provides the I–P
map for FitCo on its own.
Concentrating on the diagonal iso-rating line, it is
immediately apparent that no elements fall into
satiated needs, i.e. importance is greater than perfor-
mance in every case. Thus, opportunities for improve-
ments to FitCo’s IS along all seven dimensions are
possible. However, knowing which to prioritize is,
naturally, a concern for FitCo’s management, and
insights into this issue can be gleaned from an analysis
of the I–P quadrants.
The top-right quadrant represents elements of
high performance/high importance, the prescription
being to keep up the good work. 12 of the 26, or just
under half fall into this category. While each of the
categories are represented, particular clusters are
found among system quality (elements 1, 2 and 5),
information quality (elements 6, 7 and 8), and indi-
vidual impact (elements 17 and 18). Clearly FitCo is
perceived as successfully matching needs and perfor-
mance in the technical aspects of its IS policies,
delivering reliable and useful information that is being
translated into positive productivity gains and, perhaps
surprisingly, increased staff morale.
The gap between performance and importance can
be ascertained from the distance between the points
and the iso-rating. Clearly, elements 7 and 8, the
systems’ ability to provide useful information for
management and the ability of the systems to produce
professional reports, respectively, have the closest
match between performance and importance. Conver-
sely, the greatest differences between importance and
performance, are for elements 13, 15, and 6. More-
over, elements 15 and 6, together with 21 are border-
line, meriting management attention to avoid slipping
left in to the concentrate here segment. In particular,
element 21 is FitCo’s only representative of organiza-
tional impact in this quadrant and, with its importance
in delivering added-value to health clubs, should be
carefully monitored.
Table 2
Response rates by position for FitCo and the health club industrya
Duty/Assistant Manager Reception Administration General Manager Missing Total
FitCo 4 (19) 3 (14) 11 (52) 2 (10) 1 (5) 21
Industry 12 (19) 16 (26) 17 (27) 11 (18) 6 (10) 62
Total 16 (19) 19 (23) 28 (34) 13 (16) 7 (8) 83
a Percentage in parentheses.
Table 3
Mean I–P values by position for both FitCo and the health club industry
Category Duty/Asst. Manager Reception Administration General Manager
I P I P I P I P
System quality 7.05 4.80 8.13 4.27 7.31 5.84 7.00 4.30
Information quality 6.83 5.92 7.67 5.44 7.03 6.15 7.00 4.17
Service quality 7.06 5.00 7.42 3.67 7.80 4.80 6.38 3.00
Use 7.50 4.63 7.67 4.27 7.73 4.89 8.00 4.00
User satisfaction 6.50 5.13 7.50 5.44 7.18 4.23 6.25 4.00
Individual impact 6.75 4.50 7.11 4.89 6.97 5.85 5.83 3.00
Organizational impact 6.32 4.58 7.05 4.48 6.44 5.18 6.14 3.36
414 W. Skok et al. / Information & Management 38 (2001) 409–419
While opportunities for improvement to all of these
elements are available, diminishing returns suggests
that FitCo would, perhaps, be better placed to focus its
attention on the top-left quadrant of the I–P map,
concentrate here. Immediately noteworthy is the fact
that more than half (14) of the 26 elements fall in this
category, providing a multitude of issues to be tackled.
Again there is a spread between each of the classifica-
tions but two clusters are of particular interest. The
first is FitCo’s IS department service quality (elements
9, 10 and 11). Clearly, users are neither being ade-
quately trained in FitCo’s IS nor being offered
Table 4
Importance and performance means for FitCo
Attribute number Description FitCo
I P Difference
System quality
1 The computer hardware are up-to-date 6.95 5.52 1.43
2 The computer software is up-to-date 6.81 5.81 1.00
3 The system does not crash regularly 8.43 4.24 4.19
4 The speed of the system’s response 7.19 4.43 2.76
5 Control of people entering the computer system 7.52 6.40 1.12
Information quality
6 The system’s ability to reduce the number of errors that might be otherwise made 7.67 5.05 2.62
7 The system’s ability to provide useful information for management 6.95 6.29 0.66
8 Ability of the system to produce professional reports 6.76 6.14 0.62
Service quality
9 A high degree of technical competence from systems support staff 8.14 4.95 3.19
10 Extent of practical user training 6.95 3.19 3.76
11 Fast response from systems support staff to remedy problems 7.81 4.00 3.81
12 Positive and helpful attitude of computer staff to those who use the system 7.19 5.76 1.43
Use
13 It should be easy for users to access the computer facilities 8.29 6.10 2.19
14 Involvement of users in planning the requirements of the computer system 7.24 3.50 3.74
User satisfaction
15 User confidence in the system 7.43 5.10 2.33
16 Users’ understanding of the overall computer system and what it can do 6.71 4.14 2.57
Individual impact
17 Ability of the system to improve staff productivity 7.05 5.81 1.24
18 The system’s ability to allow faster processing of people/information 7.19 5.35 1.84
19 The system’s ability to provide improved staff morale 6.48 4.38 2.10
Organizational impact
20 The system’s ability to provide competitive business advantage over rival firms 6.50 4.74 1.76
21 The system’s ability to provide improved client/seller relationships 6.29 5.16 1.13
22 Overall cost effectiveness of the information system 6.33 4.72 1.61
23 The systems ability to provide opportunity for developing extra income 6.29 4.24 2.05
24 The system’s ability to provide improved corporate image 6.00 4.62 1.38
25 The system’s ability to provide improved customer service 7.10 4.86 2.24
26 The system’s ability to keep up with the organization’s business requirements 7.48 4.86 2.62
Table 5
Category means for FitCo and the health club industry
Fitco mean Industry mean
I P I P
System quality 7.38 5.28 7.51 5.65
Information quality 7.13 5.83 7.25 6.22
Service quality 7.52 4.48 7.20 5.70
Use 7.77 4.80 7.38 5.72
User satisfaction 7.07 4.62 7.29 5.80
Individual impact 6.91 5.18 6.89 5.58
Organizational impact 6.57 4.74 6.88 5.74
W. Skok et al. / Information & Management 38 (2001) 409–419 415
sufficient support. Moreover, the large distances
between these service quality elements and the iso-
rating suggest that improvements in this area must be a
management priority.
Second, and perhaps of most concern is the over-
whelming presence of organizational impact elements
(21, 22, 23, 24, 25 and 26). FitCo’s IS investments are
not perceived as being effectively translated into
commercial gains, despite the relative high system
quality, information quality and individual impacts. Of
equal note, however, is the lack of importance given to
these criteria, suggesting that there is no clear policy
of alignment between IS investments and the com-
pany’s overall business strategy. FitCo’s management
must re-visit their IS investment strategy and monitor
its impact in order to make sure that relevant
commercial benefits are being realized.
Moving away from an isolated evaluation, it is
useful to benchmark the company’s perceived success
against industry averages. Fig. 5 represents an I–P map
of the categories for both FitCo and the 11 respondents
representing the health club industry in general.
It is perhaps surprising that the industry averages for
the seven categories of elements are, as with FitCo,
above the iso-rating. This can again be interpreted as a
perceived under-performance of IS investments within
Fig. 4. I–P map for FitCo.
Fig. 5. I–P map for FitCo and the Health Club Industry.
416 W. Skok et al. / Information & Management 38 (2001) 409–419
the health club industry as a whole. For the purposes of
this paper, however, it is more interesting to see how
FitCo compares with these industry means.
Clearly, the elements are clustered on or around the
border between the top right and top left quadrants.
However, it is also clear that FitCo is under-perform-
ing relative to the industry average. FitCo’s manage-
ment should be extremely concerned by this finding as
it suggests that investments are not performing against
industry benchmarks, providing its competition with a
significant competitive advantage. FitCo must address
this issue or find alternative means of competing.
An analysis of the length of the lines suggests
managerial priorities for tackling this under-perfor-
mance. First, FitCo’s match with system quality,
information quality, and individual impact is relatively
close, with the implication that improvements can be
made, but that they should be of less concern than the
remaining dimensions. Indeed, the remaining four
dimensions are also points in which FitCo is present
in the concentrate here segment while the industry is
in keep up the good work quadrant. This corroborates
the need for management attention.
Finally, and focusing on the relative importance of
elements represented by the vertical distance between
FitCo and the industry, it is clear that those elements
deemed to be most important, i.e. service quality and
use, are also those elements that FitCo performs least
well. Clearly, IS policies and priorities within FitCo
are not being backed up with effective managerial
action to ensure that they are realized. Aside from
these, all other elements are perceived by FitCo’s
competitors as more important. Most notable, orga-
nizational impact, is also one of those elements where
FitCo is badly under-performing. Consequently, one
question of immediate significance to FitCo’s manage-
ment, one which requires immediate attention, is why
are they investing in IS if not to deliver commercial
advantage?
3.5. Managerial implications
The two I–P maps provide detailed insights into
FitCo’s IS performance, especially in comparison to
that of its competitors. To summarize, it is clear that
FitCo under-performs in all areas of its IS investment.
However, there is a clear need to prioritize areas
of concern. While FitCo is relatively strong in the
technical and informational areas, the support given to
users and its translation of IS investment into com-
mercial advantage over its competitors remain para-
mount. Indeed, the failure of FitCo’s management
even to recognize the commercial importance of its
IS investments remains a fundamental issue. Finally,
even where importance has been recognized, FitCo
have failed to back this up with effective action to
ensure realization.
Several recommendations follow from this analysis.
First, there is a clear need to re-visit the company’s IS
strategy. Management must think about how IS can
support the overall business objectives and how it can
gain an advantage of commercial significance over its
rivals. Clearly, attitudes towards the commercial ben-
efits of IS must first be improved, with management
then being given the resources necessary to translate
this into performance. Regular monitoring must also
be undertaken.
Second, and in support of its IS strategy, FitCo must
improve the assistance given to users, who are of
paramount importance in facilitating customer satis-
faction and added value. Finally, FitCo should not rest
on its laurels. Although it is relatively strong in
delivering technical systems, it is still below the
industry mean.
4. Conclusions
This paper has sought to highlight the managerial
need for tracking the success of IS investments. I–P
maps have been suggested as a mechanism for doing
this, especially when they are used in conjunction with
an extension of DeLone and McLean’s view of IS
success.
The experiences of using I–P maps in diagnosing
the success of FitCo’s IS investments largely corro-
borate previous conclusions on their usefulness. Their
strengths can be considered as being both their
simplicity and their power in suggesting areas and
priorities for improvement.
First, contrary to much of the extant literature,
I–P maps were both simple to administer and in-
terpret. Questionnaire development was based on
standardized scales grounded in marketing research.
Moreover, used in conjunction with Pitt et al.’s
extension of DeLone and McLean’s framework, a
W. Skok et al. / Information & Management 38 (2001) 409–419 417
broad, multi-dimensional view of success was taken.
This broad framework was flexible enough to allow
tailoring to the specific needs of FitCo, through oper-
ationalization of the categories by management and
users. As a result of this, development of I–P maps was
a relatively quick and inexpensive means of gathering
management information. With cost and time pres-
sures being an issue, these are two highly significant
advantages.
Interpretation of I–P maps was also simple. Their
visual nature provided an overview of a firm’s IS
investments, from them, areas of weakness were easily
identified. This was enhanced through the categoriza-
tion of elements into broadly similar types. Moreover,
FitCo highlighted the ability of I–P maps to prioritize
these areas for improvement and when coupled with
industry benchmarks, they provide truly powerful
diagnostic tools.
However, two weaknesses of I–P maps deserve to
be emphasized. First, I–P maps clearly sacrifice depth
for breadth and convenience; they are unlikely to
provide the detailed insights found from, say, in-depth
interviews. Yet, management may be prepared to
make this sacrifice depending on their priorities.
Indeed, I–P maps could be used as a preliminary
device to allow areas requiring more detailed analysis
to be identified.
Second, I–P maps are attitudinal in nature, relying
on people’s perceptions of IS success and not, for
example, ‘harder’ measures such as financial returns
or ‘actual use’ statistics. The inability to account for
such metrics should be a recognized weakness of the
I–P approach.
One final issue to be taken into consideration is the
use of industry benchmarks. The health club industry
operates in a trade association manner that facilitated
the collection of competitor information. However,
many industries do not, potentially limiting the oppor-
tunities for benchmark analysis.
In conclusion, therefore, I–P maps should not
be seen as an IS success panacea. They should
be seen as a complement to, rather than a replacement
for, more detailed approaches. However, with the
endemic nature of evaluation, and the complexities
associated with traditional approaches, I–P maps
represent a simple tool, easily adopted by senior
management, and yet capable of powerful diagnostic
information.
References
[1] E. Babakus, D. Pedrick, A. Richardson, Assessing perceived
quality in industrial service settings: measure development
and application, Journal of Business to Business Marketing 2
(3), 1995, pp. 47–68.
[2] J.E. Bailey, S.W. Pearson, Development of a tool for
measuring and analysing computer user satisfaction, Manage-
ment Science 29 (5), 1983, pp. 530–545.
[3] J.A. Ballantine, M. Bonner, M. Levy, A. Martin, I. Munro, P.
Powell, Information systems success: the quest for the
dependent variable goes on, in: Proceedings of the paper
presented at first UKAIS Conference on ‘The Future of
Information Systems’, Cranfield University, 10–12 April
1996.
[4] J.A. Ballantine, R.D. Galliers, S.J. Stray, Information
systems/technology evaluation practices: evidence from UK
organizations, Journal of Information Technology 11, 1996,
pp. 129–141.
[5] P. Beynon-Davies, Information systems ‘failure’: the case of
the London ambulance service’s aided despatch program,
European Journal of Information Systems 4 (3), 1995, pp.
171–184.
[6] D. Brindle, Old and fat to power fitness clubs, The Guardian,
12 September 1997, p. 19.
[7] E. Brynjolfsson, The productivity paradox of information
technology, Communications of the ACM 35 (12), 1993, pp.
66–77.
[8] E. Brynjolfsson, L. Hitt, Beyond the productivity paradox,
Communications of the ACM 41 (8), 1998, pp. 49–55.
[9] M.E. Burkhardt, D.J. Brass, Changing patterns or patterns of
change: the effects of a change in technology on social
network structure and power, Administrative Science Quar-
terly 35, 1990, pp. 104–127.
[10] R.N. Clarke, Health plan marketing and reproductive health
services, Western Journal of Medicine 163, 1995, pp. 64–70.
[11] S. Daneshkhu, Mintel report on health and fitness in the UK,
The Financial Times, 31 October 1998, p. 2.
[12] W.H. Delone, E.R. McLean, Information systems success: the
quest for the dependent variable, Information Systems
Research 3 (1), 1992, pp. 60–95.
[13] B.L. Dos Santos, K. Peffers, Rewards to investors in
innovative information technology applications: first movers
and early followers in ATMs, Organizational Science 6 (3),
1995, pp. 241–259.
[14] C.R. Duke, A.S. Mount, Rediscovering performance-impor-
tance analysis of products, Journal of Product and Brand
Management 5 (2), 1996, pp. 43–54.
[15] C.T. Ennew, G.V. Reed, M.R. Binks, Importance-performance
analysis and the measurement of service quality, European
Journal of Marketing 27 (2), 1993, pp. 59–70.
[16] B. Farbey, F. Land, D. Targett, How to Assess Your IT
Investment: A Study of Methods and Practice, Butterworth-
Heinemann, Oxford, 1993.
[17] G. Fitzgerald, Evaluating information systems projects: a
multidimensional approach, Journal of Information Technol-
ogy 13, 1998, pp. 15–27.
418 W. Skok et al. / Information & Management 38 (2001) 409–419
[18] V. Graeser, L. Willcocks, N. Pisanias, Developing the IT
scorecard: a study of evaluation practices and integrated per-
formance measurement, Business Intelligence, London, 1998.
[19] J. Hares, D. Royle, Measuring the value of information
Technology, John Wiley & Sons, Chichester, 1994.
[20] J.M. Hawes, C.P. Rao, Using importance–performance
analysis to develop health care marketing strategies, Journal
of Health Care Marketing 5 (4), 1985, pp. 19–25.
[21] R. Hirschheim, S. Smithson, Evaluation of information
systems: a critical assessment, in: L.P. Willcocks, S. Lester
(Eds.), Beyond the IT Productivity Paradox, Wiley, Chiche-
ster, 1999.
[22] S. Hollenhorst, L. Gardner, The indicator performance
estimate approach to determining acceptable wilderness con-
ditions, Environmental Management 18 (6), 1994, pp. 901–906.
[23] B. Ives, M.H. Olson, J.J. Baroudi, The measurement of user
information satisfaction, Communications of the ACM 26
(10), 1983, pp. 785–793.
[24] T. Jelassi, European Casebook on Competing through
Information Technology, Prentice-Hall, UK, 1994.
[25] R. Kraut, S. Dumais, S. Koch, Computerisation, productivity,
and quality of work life, Communications of the ACM 32 (2),
1989, pp. 220–238.
[26] K. Kumar, Post implementation evaluation of computer-based
information systems: current practices, Communications of
the ACM 33 (2), 1990, pp. 203–212.
[27] Leisure Week, UK Health and Fitness Sector, 18 April 1997,
p. 19.
[28] E.Y. Li, Perceived importance of information systems success
factors: a meta analysis of group differences, Information and
Management 32, 1997, pp. 15–28.
[29] K. Lyytinen, R. Hirschheim, Information systems failures: a
survey and classification of the empirical literature, in:
Oxford Surveys in Information Technology, Oxford
University Press, Oxford, Vol. 4, 1987, pp. 257–309.
[30] J.A. Martilla, J.C. James, Importance-performance analysis,
Journal of Marketing 41, 1977, pp. 70–77.
[31] J. Miller, Measuring and aligning information systems with
the organization, Information and Management 25, 1993, pp.
217–228.
[32] L.F. Pitt, R.T. Watson, C.B. Kavan, Service quality: a measure
of information systems effectiveness, MIS Quarterly 19 (2),
1995, pp. 173–185.
[33] T. Saarinen, An expanded instrument for evaluating informa-
tion systems success, Information and Management 31, 1996,
pp. 103–118.
[34] N. Slack, The importance–performance matrix as a determi-
nant of improvement priority, International Journal of
Operations and Production Management 14 (5), 1994, pp.
59–75.
[35] S. Smithson, R. Hirschheim, Analysing information systems
evaluation: another look at an old problem, European Journal
of Information Systems 7, 1998, pp. 158–174.
[36] P. Strassman, The Business Value of Computers, The
Information Economics Press, 1990.
[37] V.J. Symons, A review of information systems evaluation:
content, context, and process, European Journal of Informa-
tion Systems 1 (3), 1991, pp. 205–212.
[38] J. Ward, Information systems and technology application
portfolio management: an assessment of matrix-based
analyses, Journal of Information Technology 3 (3), 1988,
pp. 205–215.
[39] J. Ward, A portfolio approach to evaluating information
systems investments and setting priorities, Journal of
Information Technology 5, 1990, pp. 222–231.
[40] J. Ward, P. Griffiths, Strategic Planning for Information
Systems, 2nd Edition, Wiley, New York, 1996.
[41] L. Willcocks, Evaluating information technology invest-
ments: research findings and re-appraisal, Journal of In-
formation Systems 2, 1992, pp. 243–268.
Walter Skok has been Principal Lecturer in Business Computing at
Kingston University’s Business School since 1987, having
previously worked as a management consultant. With a PhD from
the London School of Economics, he is currently involved in IS
teaching on under-graduate and masters programmes. His research
interests include: Internet start-up projects, IS evaluation and the
role of individuals in applying IT to business. He has presented
papers at UK and international conferences, is a Fellow of the
British Computer Society and Member of the United Kingdom
Academy of Information Systems.
Andrew Kophamel is a doctoral research student at Kingston
University. He is interested in IS evaluation and innovation
diffusion, and his research aims to contrast the success of early
and later adopters of ERP systems. He has written for and
presented papers at UK and international conferences.
Ian Richardson is Leisure Manager at a leading London Health
and Sports club, where he is responsible for the club’s IT strategy
and its successful implementation. His previous experience
includes work within both public and private sectors of the leisure
industry, as well as providing a Health and Safety consultancy
service. He completed an MBA (awarded with distinction) at
Kingston University in 1998.
W. Skok et al. / Information & Management 38 (2001) 409–419 419