technological forecasting & social change · measured the interdependencies between the...

Contents lists available at ScienceDirect

Technological Forecasting & Social Change

journal homepage: www.elsevier.com/locate/techfore

Measuring the maturity of business intelligence in healthcare: Supportingthe development of a roadmap toward precision medicine within ISMETThospital

Luca Gastaldia, Astrid Pietrosib, Sina Lessanibahria,c,⁎, Marco Paparellaa, Antonio Scaccianoceb,Giuseppe Provenzaleb, Mariano Corsoa, Bruno Gridellib

a Politecnico di Milano(Polimi), Department of Management, Economics and Industrial Engineering, Via Lambruschini, 4/B, 20156 Milano, ItalybMediterranean Institute for Transplantation and Advanced Specialized Therapies (IRCCS-ISMETT), Discesa dei Giudici, 4, 90133 Palermo, ItalycUniversidad Politécnica de Madrid (UPM), Department of Industrial Engineering, Business Administration and Statistics, Calle de José Gutiérrez Abascal, 2, 28006Madrid

A R T I C L E I N F O

Keywords:HealthcareBusiness intelligenceMaturity model

A B S T R A C T

Business Intelligence (BI) has the potential to disrupt the processes through which healthcare services are of-fered. Despite this key role, most healthcare organizations fail in implementing or extending BI suites from thepilot niches in which these solutions are usually developed and tested to larger domains. In fact, healthcarepractitioners lack comprehensive models that suggest the priorities to be followed for progressively developing aBI solution. This paper aims to start filling these gaps by developing a model through which: (i) to measure andincrease the maturity of a BI solution within a healthcare organization; (ii) to enable extensive processes ofbenchmarking and continuous improvement.

1. Introduction

The introduction of new and sophisticated medical technologies, theglobal trend of increased longevity, and the growth in non-transmit-table chronic diseases have the potential to drive the cost of healthcareto unsustainable levels (Janssen and Moors, 2015; Qaseem et al., 2012).Public and private healthcare organizations are focusing their efforts onfinding new, more affordable levels of care, and increasing efficiency(Pine et al., 2012; Romanow et al., 2012). In order to do this, in-formation technologies have a fundamental role in transforming datainto intelligence that can be used to improve patient care, healthcarefacilities, and process management (Behkami and Daim, 2012; Li andMao, 2015; Pai and Huang, 2011).

Given the crucial role of data in supporting organizational en-hancement, Business Intelligence (BI) has become an important area ofstudy for both researchers and practitioners (Chen et al., 2012).Davenport and Harris (2007) define BI as a set of technologies andprocesses that use data, statistical and quantitative analysis, ex-planatory and predictive models, and fact-based management, whichdrive decisions and actions, enabling an accurate understanding ofbusiness performance.

The field of BI has improved significantly in the last decade(Gandomi and Haider, 2015), and has promising applications in thehealthcare domain (Chen et al., 2012; El-Gayar and Timsina, 2014;Fichman et al., 2011). In fact, BI can not only improve the outcomes ofhealthcare organizations (Tremblay et al., 2012; Pine et al., 2012), butalso help them to progress along the continuum from intuitive to pre-cision medicine (Christensen et al., 2009; Gastaldi et al., 2015).

Despite its disrupting potential, BI is not very widespread inhealthcare (Hanson, 2011). Thus, there is still limited research on howto successfully implement a BI solution into this domain (Foshay andKuziemsky, 2014). More specifically, we completely lack comprehen-sive models to help practitioners with the priorities that should befollowed to develop a proper BI solution (Chen et al., 2012).

This paper aims to start filling these gaps by developing a modelthat provides maturity levels for assessing and improving a BI solutionin healthcare. The model has four maturity levels—initial, managed,systematic and disrupted—that represent the desired evolutionary pathfor the dimensions of a BI solution in the healthcare domain. Throughthe model, we assessed both the actual and the expected maturity levelsof a healthcare organization to suggest it a harmonized path for im-proving its BI solution. Moreover, we systematically evaluated and

https://doi.org/10.1016/j.techfore.2017.10.023Received 14 November 2016; Received in revised form 31 August 2017; Accepted 31 October 2017

⁎ Corresponding author at: Politecnico di Milano(Polimi), Department of Management, Economics and Industrial Engineering, Via Lambruschini, 4/B, 20156 Milano, Italy.E-mail addresses: [email protected] (L. Gastaldi), [email protected] (A. Pietrosi), [email protected] (S. Lessanibahri), [email protected] (M. Paparella),

[email protected] (A. Scaccianoce), [email protected] (G. Provenzale), [email protected] (M. Corso), [email protected] (B. Gridelli).

Technological Forecasting & Social Change xxx (xxxx) xxx–xxx

0040-1625/ © 2017 Elsevier Inc. All rights reserved.

Please cite this article as: Gastaldi, L., Technological Forecasting & Social Change (2017), https://doi.org/10.1016/j.techfore.2017.10.023

https://freepaper.me/t/349845 خودت ترجمه کن :

http://www.sciencedirect.com/science/journal/00401625

https://www.elsevier.com/locate/techfore

https://doi.org/10.1016/j.techfore.2017.10.023


mailto:[email protected]









measured the interdependencies between the different dimensions ofthe BI solution in order to distinguish and prioritize the aspects thatrequire immediate attention.

The remainder of the paper is organized as follows. In Paragraph 2,we discuss the theoretical background of the work. In Paragraph 3, wepresent the research settings. In Paragraph 4 we present the maturitymodel and how we developed it. In Paragraph 5, we apply the model toa healthcare organization and design a roadmap for better exploiting itsBI solution. Finally, in Paragraph 6, we discuss the theoretical andempirical implications of our work, and fruitful directions for furtherresearch on the topic.

2. Theoretical background

We have organized the theoretical background of the paper in twosections. The first one describes the latest developments in the field ofBI in healthcare and reveals the lack of models and tools to support thedevelopment of BI solutions. The second focuses on BI maturity modelsand identifies the opportunity to improve them in consonance withhealthcare organization.

2.1. BI in healthcare

Healthcare organizations have been under constant pressure to notonly achieve more outcomes with fewer resources (Foshay andKuziemsky, 2014), but also to progressively become information-drivensystems (Gastaldi and Corso, 2012). Considering that the amount ofdata recorded by electronic health records and medical registries isgrowing rapidly (Kuiler, 2014; Wang et al., 2016), healthcare organi-zations are trying to draw upon tools such as BI in order to improvetheir efficiency and effectiveness (Brooks et al., 2013; Wang et al.,2016).

If BI was initially perceived as merely a collection of tools for dataanalysis (Anandarajan et al., 2012), over time it has been increasinglyconsidered a strategic weapon by several organizations (Wixom andWatson, 2012). Today, BI is usually referred to as a set of theories,methods, processes, and techniques that converts data into useful andvaluable information for business purposes (Rud, 2009). BI solutionshelp decision-makers by providing practical information in the rightform, at right time, and in right place (Negash, 2004).

As recently highlighted (Gartner, 2015), the market of BI has grownsignificantly and has become the first investment priority for Chief In-formation Officers (CIOs). Also, the awareness of its potential benefits isincreasing (Chuah and Wong, 2011; Lahrmann et al., 2011). However,the implementation of BI in healthcare proceeds relatively slowly and inan ad hoc way (Foshay and Kuziemsky, 2014).

Some studies have reported decreased costs (e.g., Borzekowski,2009), higher income (e.g., Ayal & Seidman, 2009), and improvedproductivity (e.g., Lucas et al., 2010). Other studies have reported anegative or non-existent impact of BI on the performance of healthcareorganizations (Agarwal et al., 2010; Foshay and Kuziemsky, 2014).

The successful implementation of BI in healthcare relies on under-standing and analyzing the peculiarities of this domain (Avison &Young, 2007; Mettler & Vimarlund, 2009). Thus, one of the main ob-jectives of the research is to offer healthcare practitioners a maturitymodel for developing and constantly improving their BI solutions.

2.2. BI maturity models

Maturity models were introduced for the first time in the 1970s(Gibson and Nolan, 1974) to guide users from an initial state of asystem to a desired or naturally existing end state (Fraser et al., 2002;Marx et al., 2012). The key objective of maturity models is to detect thegap between current and desired states, and to anticipate an evolu-tionary path through which increasing the maturity of the system, andeventually achieve the desired state (Blondiau et al., 2015). The

evolutionary path implies that the progress toward higher maturitylevels is incremental, and realized through a set of intermediate states(Sen et al., 2012).

Dimensions, levels, and assessment tools are the principal elementsof maturity models (Marx et al., 2012).

• Dimensions are areas of mutually-exclusive capabilities that clusterinterrelated activities (Lahrmann & Marx, 2010b). With each di-mension come measures, which are used to assess the maturity ofthe dimension (Mettler & Rohner, 2009).

• Levels are the maturity states that the dimensions assume. Eachlevel has a distinct descriptor of the detailed corresponding maturitystate (Lahrmann & Marx, 2010a).

• Assessment tools are either qualitative or quantitative, and ques-tionnaires and scoring models are among the most common (Fraseret al., 2002).

While there are several BI maturity models in the literature, theyfocus mainly on data and information, without considering the idio-syncrasies of the peculiar domain in which the models are applied(Brooks et al., 2015). Few studies deepen the idiosyncrasies of health-care (e.g., Blondiau et al., 2015; Brooks et al., 2015), and there are nomodels focusing on the interdependencies among the different dimen-sions that characterize the BI solutions in this domain.

Generally, most of the maturity models in the literature are fixed-level models, i.e., models in which a fixed number of maturity levels areassumed for every dimension (Lahrmann & Marx, 2010b). The maindrawback of these models is that they are not developed with the in-terdependencies between their dimensions in mind (de Bruin et al.,2005; Maier et al., 2009). Thus, they cannot provide comprehensiveguidelines for prioritizing the potential improvement paths of the BIsolution that they aim to improve (Popovič et al., 2012).

The importance of interdependencies grows with the complexity ofthe domain of interest. Thus, for complex systems, such as healthcare, itis crucial to explicitly measure and incorporate the interactions be-tween different aspects, and identify the possible interdependenciesamong them. We develop a maturity model that takes into account theinteractions between the various aspects of a BI solution in healthcarein order to effectively prioritize the interventions to be accomplishedfor improving its effectiveness.

3. Research settings

In order to develop and apply our maturity model, we built upon aClinical Inquiry Research (CIR) project we did from October 2012 toMay 2013 with:

• The Mediterranean Institute for Transplantation and AdvancedSpecialized Therapies (ISMETT), a 656-employees (86-beds) privatehospital in Italy with a recognized experience in using digitaltechnologies (and especially BI) for improving healthcare treat-ment1;

• Five hospitals representative of the variety of the Italian healthcareindustry and with experience in the development of BI solutions.

All six hospitals were selected because of their experience in

1 ISMETT is public-private partnership between the Region of Sicily (through theCivico Hospital in Palermo) and UPMC, an integrated global health enterprise head-quartered in Pittsburgh, and one of the leading not-for-profit health systems in the UnitedStates. ISMETT is the first hospital in Southern Italy to receive Joint CommissionInternational (JCI) accreditation, and has been recently authorized by the Italian Ministryof Health as an IRCCS. IRCCSs are research-oriented hospitals operating in Italy withcompetences in research and treatment of important diseases, which represent a nationalnetwork where basic and translational biomedical research is undertaken in synergy withthe delivery of high qualitative healthcare. For more information, see Ferré et al. (2014).

L. Gastaldi et al. Technological Forecasting & Social Change xxx (xxxx) xxx–xxx

2


BI—measured in terms of the percentage of budget delivered to BI from2008 to 2013.

CIR is a well-known collaborative form of research developed bySchein (2008). Unlike most other collaborative research approaches,with CIR the learning opportunity arises in a situation led by practi-tioners who needs help (Coghlan and Brannick, 2005) and, because ofthis, is more likely to reveal important data. In these settings, re-searcher have a role of process facilitator and helps practitioners indiscovering insights through self-diagnosis/self-intervention (Stebbinsand Shani, 2009).

The request for a joint collaborative research came directly from theexecutives of ISMETT, interested in developing a model to assess andimprove the maturity of its BI solution. A Research Task Force (RTF)was formed in November 2012. Modeled on the leading literature oncollaborative research (Mohrman and Mohrman, 2004), the RTF in-cluded both researchers and practitioners. All members are depicted inTable 1 together with their expertise and their role into the project.

The RTF subdivided its work into two stages that are respectivelyrelated to the development of the BI maturity model and its applicationto ISMETT. Fig. 1 summarizes the stages as well as the four phasescharacterizing them, which are derived by Mohrman and Mohrman(2004). The figure reports also the various roles played by the RTF, themethods that have been used and the output produced.

As seen in the Fig. 1, business process analysis and mapping(Womack and Jones, 2003), face-to-face interviews, and multi-

participant interactive dialogues (Mikaelsson and Shani, 2004) werethe principal mechanisms utilized in the CIR project. Periodic meetingswere organized as well in order to progressively share the achievedknowledge, slightly re-orient the research process, and discuss theempirical as well as theoretical implications of the findings.

In the next two paragraphs we will describe the two stages, theirphases and their outputs.

4. STAGE A. Development of the BI maturity model

As shown in Fig. 1, this stage of the CIR project consists of threephases: knowledge acquisition, diagnosis, criteria setting. Each of themis deepened in the following sections. It is important to note that, whilethe phases are presented in a linear logic, in reality they were highlyintertwined, and the final maturity model was the result of a continuousiteration among them. This iteration has been stopped when a newinstance of a phase failed to add further insights. A last section proposesthe overall outputs that has been achieved: the BI maturity model andthe questionnaire through which assessing the maturity of the BI so-lution in a healthcare organization.

4.1. Knowledge acquisition

Given the complexity of creating the model, its development re-quired the design of intermediate constructs (Sen et al., 2012). First, we

Table 1Composition of the research task force.

Member Expertise Years of expertise Main role into the project

Researcher 1 Management of complex projects into the healthcare domain 15 Coordination of the overall projectResearcher 2 Maturity models 4 Development/improvement of the maturity modelResearcher 3 BI in healthcare 4 Application of the maturity model to ISMETTPractitioner 1 Management control and decision support systems in healthcare 10 Engagement of ISMETT practitionersPractitioner 2 BI in healthcare 6 Developing/improvement of the maturity modelPractitioner 3 BI in healthcare 2 Reporting of project results

Fig. 1. Research stages, phases, methods and outputs.


3


extensively analyzed the literature, also taking into account all thesectors in which BI maturity is evaluated (not only healthcare). Wewanted to understand the criteria used to define and select the metricsby which to assess the maturity of the BI solution, and identify potentiallogics to group these metrics. This exercise allowed producing a pre-liminary version of the BI maturity model, which we progressively re-fined through phase 2 and 3. The preliminary model was significantlydifferent from the one that will be presented in Section 4.4: literatureanalysis would have missed key aspects such as data provisioning orsupplier coordination. Despite several limits, the initial model provideda useful base to discuss with practitioners and focus their contributions.

Information acquisition and knowledge systematization were ac-complished in multiple modes depending on the specific intent, on thespecific stage of model elaboration and on the experience developed.Three brainstorming sessions (McGraw and Harbison-Briggs, 1989)were used within the RTF to elicit ideas for model conceptualization.The objectives of this phase were to assess the potential value of thematurity model, and to identify the different areas and dimensionscharacterizing a BI solution in healthcare.

4.2. Diagnosis

Eight sessions of consensus decision-making have been used toprogressively converge on the various dimensions of the model.Consensus decision-making is very useful to find the best solution to aproblem by letting a group weigh in on the advantages and dis-advantages of each alternative (Sen et al., 2012). Each session lasted atleast two hours. In the first one, leveraging on the results of phase 1, wegenerated ideas by working alone and in pairs (as suggested byVerganti, 2017). Next, we collected all the ideas in a unique list, and wediscussed them with the aims of: (i) building on other intuitions andstrengthen other ideas; (ii) converge on the most promising dimensions.

When the RTF was unable to achieve a consensus, it converged on aset of (no more than three) alternatives and asked the experts of theother hospitals engaged into the CIR project to discuss and evaluatethese alternatives. Virtual sessions of multi-participant interactive dia-logues created consensus, and allowed tackling the most controversialtopics. In two cases, a blind evaluation by both the members of the RTFand the engaged experts were necessary.

4.3. Criteria setting

In this phase, the RTF used four sessions of concept-sorting(McGraw and Harbison-Briggs, 1989) to define the maturity levels ofthe various dimensions. Concept sorting is a knowledge generationmode that is useful once a maturity model is outlined and its key areasare identified (Sen et al., 2012). Each session was based on an area ofdimensions of the maturity model (e.g. the technological ones, seeparagraph 4.4 for further details). The members of the RTF worked inpairs (different from the ones of the previous phase) and producedpreliminary alternatives for the maturity levels. Next, these alternativeshave been sorted into meaningful groups, exploring and explicatingthrough peer discussions the reasons for fitting various levels into thesame categories. If preliminary sessions lasted up to four hours, thelatest ones allowed converging on the final maturity levels in shortertimeframes.

The RTF also sent e-mails to a panel of three power-users of the BIsolution in ISMETT to solicit their opinions on some technical issuesassociated to the maturity levels. The power-users have been chosentogether with ISMETT CIO from a list of five employees who partici-pated in the development of the BI solutions and/or attended technicalcourses on how using it. Overall, 12 streams of topics have been dis-cussed with these three power-users. Examples span from the definitionof the maturity levels imagined for the interface of the BI solution (seeTable 2) to the definition of the capabilities to be assessed in normalusers (see Table 5).

All maturity levels have been validated with the help of both thepower-users as well as the experts from the five healthcare organiza-tions engaged into the CIR project.

4.4. Outputs of stage A. Maturity model and assessment questionnaire

At the end of stage A of the CIR the RTF defined 23 dimensions toassess the maturity of a BI solution in healthcare settings. The dimen-sions have been clustered in four different areas:

• Functional area (6 dimensions), which represent the different func-tionalities of the BI solution (e.g., the possibility of actively sup-porting decision-making in the clinical, quality, production, oreconomic domain);

• Technological area (7 dimensions), which represent the technologicalfeatures of the BI solution (e.g., the number of interoperabilitystandards supported by the solution);

• Diffusional area (3 dimensions), which measure the pervasiveness ofthe BI solution (e.g., its use by clinicians and nurses);

• Organizational area (7 dimensions), which describe how the health-care organization manages the BI solution (e.g., strategic coherencebetween BI development and company needs).

Four maturity levels have been defined for each dimension:

• Initial: the dimension is not covered or the implementation is in theearly stages;

• Managed: the dimension is implemented in a limited manner;

• Systematic: the dimension is fully implemented;

• Disrupted: the dimension is fully implemented and exploited for allits potential.

According the members of the RTF, the experts and the power-usersengaged into the research endeavor, these four levels, combined to theaforementioned 23 dimensions, form a model that is enough detailedand simple to suggest effective improvement paths to be pursued byhealthcare organizations.

Tables 2-5 show the dimensions and maturity levels defined for eachof the four areas.

Each dimension has been further subdivided into more granularsub-dimensions (metrics and sub-metrics) to assure a comprehensiveand accurate measurement of the maturity of the BI solutions. Somemetrics (e.g., the frequency of goal definition) have sub-metrics thatreflect the different domains in which the metric can be measured (e.g.,definition of the goal in the economic domain, in the production do-main, and in the qualitative domain). In these cases, the maturity levelsreported in the table are valid for each sub-metric.2 At most granularlevel 119 metrics/sub-metrics are defined. See Tables A.1, A.2, A.3 andA.4, in the Appendix, for a detailed representation of the granular di-mensions as well as the different maturity levels.

For each metric and/or sub-metric, the RTF produced a question(except for few sub-metrics that are assessed with more than onequestion) with four possible answers that reflect the increasing levels ofmaturity. All questions have been tested and refined with the expertsfrom the five Italian healthcare organizations involved with the CIRproject. Specifically, we shared the questionnaire with the engagedinformants, asking them to fill it in advance and to take notes on theaspects that were not completely clear. Next, we scheduled multiple

2 The economic, production, and qualitative domains are used in several dimensions ofthe BI maturity model. Economic domain refers to costs and revenues of the healthcareorganization, e.g., the cost of a hospitalisation. Production domain refers to volumes andhospital length of stay in the health care organization, e.g., the number of admissions perdepartment. Qualitative domain refers to the quality of the output of the health careorganization, e.g., the number of admissions related to a Major Diagnostic Category(MDC).


4


Table2

Maturityleve

lsforthe6dimen

sion

sin

thefunc

tion

alarea.

Dim

ension

Leve

l1:

initial

Leve

l2:

man

aged

Leve

l3:

system

atic

Leve

l4:

disrup

ted

F 1Goa

lde

finition

Thesupp

ortof

theBI

solution

togo

alde

finition

islim

ited

to:

-Dataat

orga

nization

alleve

l-Ann

ualrepo

rts

BIsolution

supp

orts

goal

definition

inmorethan

50%

ofthecaseswith:

-Dataat

unitan

d/or

wardleve

l-Inpu

tsof

consolidated

targets

-Qua

rterly

repo

rts

BIsolution

supp

orts

goal

definition

inmore

than

50%

ofthecaseswith:

-Dataon

sing

leev

entan

d/or

cure

episod

e-Bu

dget

deve

lopm

ent

-Mon

thly

repo

rts

BIsolution

supp

orts

goal

definition

inmorethan

50%

ofthecaseswith:

-Atomic

data

attheleve

lof

sing

letreatm

ent

proc

edure

-Dyn

amic

budg

etman

agem

ent

-Weeklyor

daily

repo

rts

F 2Measuremen

tTh

esupp

ortof

theBI

solution

tope

rforman

cemeasuremen

tis

limited

to:

-Dataat

orga

nization

alleve

l-Ann

ualrepo

rtsan

dda

ta

BIsolution

supp

orts

performan

cemeasuremen

twith:

-Dataat

unitan

d/or

wardleve

l-Man

ualmeasureson

lyforsomeop

erativeun

its

-Qua

rterly

repo

rtsan

dda

ta

BIsolution

supp

ortspe

rforman

cemeasuremen

twith:

-Dataon

sing

leev

entan

d/or

cure

episod

e-Man

ualmeasuresforalltheop

erativeun

its

-Mon

thly

repo

rtsan

dda

ta

BIsolution

supp

orts

performan

cemeasuremen

twith:

-Atomic

data

attheleve

lof

sing

letreatm

ent

proc

edure

-Autom

atic

measuresforalltheop

erativeun

its

-Weeklyor

daily

repo

rtsan

dda

ta

F 3Gap

analysis

Thesupp

ortof

theBI

solution

toga

pan

alysis

islim

ited

to:

-Dataat

orga

nization

alleve

l-Ann

ually

gapan

alyses

andrepo

rts

BIsolution

supp

orts

gapan

alysis

with:

-Dataat

unitan

d/or

wardleve

l-Onlyda

taco

llection

-Qua

rterly

gapan

alyses

andrepo

rts

BIsolution

supp

orts

gapan

alysis

with:

-Dataon

sing

leev

entan

d/or

cure

episod

e-Dataco

llectionan

dga

pvisualisation

-Mon

thly

gapan

alyses

andrepo

rts

BIsolution

supp

orts

gapan

alysis

with:

-Atomic

data

attheleve

lof

sing

letreatm

ent

proc

edure

-Dataco

llectionan

dau

tomatic

gapan

alyses

-Realtimega

pan

alyses

andrepo

rts

F 4.D

ecisionmak

ing

Thesupp

ortof

theBI

solution

tode

cision

mak

ingis

limited

to:

-Dataat

orga

nization

alleve

l-Ann

ually

upda

tedda

taan

drepo

rts

BIsolution

supp

orts

decision

mak

ingwith:

-Dataat

unitan

d/or

wardleve

l-Ex

-postan

alyses

-Qua

rterly

upda

tedda

taan

drepo

rts

BIsolution

supp

orts

decision

mak

ingwith:

-Dataon

sing

leev

entan

d/or

cure

episod

e-OnlineAna

lyticalProc

essing

(OLA

P)-Mon

thly

upda

tedda

taan

drepo

rts

BIsolution

supp

orts

decision

mak

ingwith:

-Atomic

data

attheleve

lof

sing

letreatm

ent

proc

edure

-Activesupp

ortto

decision

mak

ing

-Realtimeup

datedda

taan

drepo

rts

F 5.D

ataqu

ality

Noau

tomatic

qualityco

ntrolov

erinbo

undan

dou

tbou

ndda

taTh

eBI

solution

automatically

performsqu

ality

controlforless

than

40%

oftheinbo

undan

dou

tbou

ndda

ta

TheBI

solution

automatically

performsqu

alityco

ntrolless

than

80%

ofthe

inbo

undan

dou

tbou

ndda

ta

TheBI

solution

automatically

andsystem

atically

controls

alltheinbo

undan

dou

tbou

ndda

ta

F 6.F

unctiona

lintegration

TheBI

solution

does

notallow

working

inan

integrated

man

ner(integ

ration

withinan

dacross

differen

tdo

mains/areas)

TheBI

solution

allowsworking

inan

integrated

man

ner:

-Withineach

oftheecon

omic,p

rodu

ctionan

dqu

alitativedo

mains

inless

than

30%

ofthecases

-Be

tweentw

odo

mains

-Be

tweentw

ofunc

tion

alareas(goa

lde

finition

,measuremen

t,ga

pan

alysis,d

ecisionmak

ing)

TheBI

solution

allowsworking

inan

integrated

man

ner:

-Withineach

oftheecon

omic,p

rodu

ctionan

dqu

alitativedo

mains

inless

than

70%

ofthe

cases

-Betweenallthree

domains

inless

than

50%

ofthecases

-Be

tweenthreefunc

tion

alareas

TheBI

solution

allowsworking

inan

integrated

man

ner:

-Withineach

oftheecon

omic,prod

uction

and

qualitativedo

mains

inmorethan

70%

ofthe

cases

-Be

tweenallthree

domains

inmorethe50

%of

thecases

-Be

tweenallfunc

tion

alareas


5


Table3

Maturityleve

lsforthe7dimen

sion

sin

thetech

nologicalarea.

Dim

ension

Leve

l1:

initial

Leve

l2:

man

aged

Leve

l3:

system

atic

Leve

l4:

disrup

ted

T 1.BI

arch

itecture

Thereis

noreal

BIarch

itecture

astheda

taco

llectionan

dan

alyticsaremainlypa

perba

sed

TheBI

solution

does

notsupp

ortde

coup

ling

betw

eentran

sactions

andan

alytics

TheBI

solution

supp

orts

tran

sactiona

lan

dan

alyticsde

coup

ling

TheBI

solution

hasmulti-le

velarch

itecture

for

analytics

T 2.Rep

orting

TheBI

solution

prod

uces

only

static

repo

rts(text,

tables,e

tc.)an

dtherepo

rtsaredistribu

tedin

pape

rform

at

TheBI

solution

prod

uces

static

repo

rtswith

grap

hicalda

ta;r

eports

aredistribu

teddigitally

butman

ually

TheBI

solution

supp

orts

dyna

mic

data

naviga

tion

andrepo

rtsaredistribu

tedau

tomatically

and

digitally

TheBI

solution

supp

orts

dyna

mic

statistical

analysisan

dtherepo

rtsaredirectly

andco

nstantly

available

T 3.Interface

Noreal

BIsolution

sin

theorga

nization

TheBI

solution

hasaclient-serve

rinterfaceon

lyaccessible

throug

hspecificde

vices

TheBI

solution

isweb

-based

andis

accessible

throug

hallthede

sktopde

vices

TheBI

solution

hasad

vanc

edRIA

interface,

accessible

throug

hallthemob

ilede

vices

T 4.Userprofi

ling

TheBI

solution

does

notsupp

ortuser

profi

ling

TheBI

solution

only

supp

orts

macro-area

profi

ling

TheBI

solution

supp

orts

sing

leuser

profi

ling

TheBI

solution

supp

orts

context-ba

sedprofi

ling

T 5.Te

chno

logical

integration

-Th

eBI

solution

isno

tau

tomatically

alim

entedby

othe

rinternal

system

s-Alim

entation

withexternal

system

isco

mpletely

man

ual.

-Th

eBI

solution

isau

tomatically

alim

entedby

less

than

50%

oftheothe

rinternal

system

s-Alim

entation

withexternal

system

sis

mon

o-directiona

lbu

tman

ualto

someextent.

-Th

eBI

solution

isau

tomatically

alim

entedby

less

than

85%

oftheothe

rinternal

system

s-Alim

entation

withexternal

system

sis

bi-

directiona

lbu

tman

ualto

someextent.

-Th

eBI

solution

isau

tomatically

alim

entedby

morethan

85%

oftheothe

rinternal

system

s-Alim

entation

withexternal

system

sis

bi-

directiona

lan

dau

tomatic

T 6.Stan

dards

TheBI

solution

does

notsupp

ortan

yinterope

rabilitystan

dards

-The

BIsolution

hasinterope

rabilitystan

dardsfor

less

than

50%

oftheap

plications

-Th

ereis

only

oneinterope

rabilitystan

dard

supp

ortedby

theBI

solution

-Th

eBI

solution

hasinterope

rabilitystan

dards

forless

than

85%

oftheap

plications

-Th

erearefew

interope

rabilitystan

dards

supp

ortedby

theBI

solution

-The

BIsolution

hasinterope

rabilitystan

dardsfor

morethan

85%

oftheap

plications

-Majorityof

theinterope

rabilitystan

dardsare

supp

ortedby

theBI

solution

T 7.Dataprov

isioning

-Inmorethan

50%

ofthecasesthealim

entedda

taareag

greg

ated

toorga

nization

alleve

l-Dataarealim

entedev

eryqu

arter

-In

morethan

50%

ofthecasesthealim

ented

data

areat

unitan

d/or

wardleve

l-Dataarealim

entedmon

thly

-In

morethan

50%

ofthecasesthealim

ented

data

arein

even

tor

cure

episod

eleve

l-Dataarealim

entedarealim

entedweeklyor

daily

-Inmorethan

50%

ofthecasesthealim

entedda

taareav

ailableforeach

treatm

entproc

edure

-Dataarealim

entedin

real

time


6


face-to-face interviews in order to analyze whether the model and thequestionnaire should be modified. Each meeting has been led by at leasttwo members of the RTF, and lasted approximately two hours. In thefirst one we checked the coherence of the answers provided by theexperts with their BI solutions. Next, we focused on the comprehen-siveness, the understandability and the accuracy of both the model andthe questionnaires.

We revised the metrics and the maturity levels in order to makethem mutually exclusive and collectively exhaustive. Moreover, weslightly changed the wording of the questions used to assess the ma-turity levels. Table A.5, in the Appendix, shows a sample of the de-veloped questionnaire. Following the suggestion of an expert engagedinto the CIR project. For each metric we ask not only to evaluate thecurrent maturity levels achieved by a hospital, but also the levels ex-pected to be achieved in the next three years—according to the strategicplans already programmed and/or what seemed feasible targets in theconsidered timeframe. This choice further increased the level of ac-tionability of the model, which is able not only to easily spot in-harmonious developments related to the BI solution, but also to con-sider the gaps that are reasonable to fill in the near future.

5. STAGE B. Implementation of the BI maturity model

As shown in Fig. 1 this stage of the CIR project consists of one un-ique phase, namely implementation design. The paragraph is divided intwo sections. The first one described the steps followed in the phase.The second one focus on its results.

5.1. Implementation design

After validation of the model, the RTF presented it to ISMETT'sexecutive board, which asked the researchers to assess the maturity ofthe BI solution present in hospital, and, based on the assessment, to co-define a roadmap for fully exploiting BI.

Through a snowball technique (Patton, 2002), the RTF engaged notonly ISMETT experts in BI areas, but also the managers coordinating thedevelopment of BI solutions, power users handling day-to-day operativeissues, and physicians and administrative directors using it. Overall,five informants were involved.

The questionnaire was sent to these informants via e-mail, andcontained an endorsement by the strategic board (together with itsrequest of a full commitment to the tasks to be accomplished). We gaveeach informant the time to scan and preliminarily fill out the ques-tionnaire. Next, we organized face-to-face meetings in which we clar-ified any anomalies or inconsistencies in the answering of the ques-tionnaire, and led a discussion of informants' perspectives to find ashared synthesis. Consensus decision-making was the principal tech-nique used to come to a comprehensive positioning on the maturitymodel. The maturity of each dimension is calculated by first averagingover the metrics forming the dimension, and then averaging over ma-turity levels given by the different informants within the hospital.

One of the main limitations of the maturity models in the literatureis that the relationships among the different metrics and dimensions areoften tacit (Marx et al., 2012). In this regard, after assessing the currentand future maturity of the BI solution, ISMETT initiated collectivethinking on how to achieve—starting from its current position in thematurity model—the different maturity levels expected for the variousmetrics in the next three years. Researchers supported this reflection bysystematizing in a unique and coherent framework:

• the different interventions planned by the hospital for each area ofthe model (derived from a joint reflection on ISMETT's position inthe BI maturity model);

• some critical issues that were central for achieving the expectedlevels of maturity (derived partially from analysis of the literatureand partially from an analysis of other healthcare organizations thatTa

ble4

Maturityleve

lsforthe3dimen

sion

sin

thediffusiona

larea.

Dim

ension

Leve

l1:

initial

Leve

l2:

man

aged

Leve

l3:

system

atic

Leve

l4:

disrup

ted

D1.Accessing

users

-Le

ssthan

50%

ofthead

ministrativean

dclinical

directorsha

veaccess

totheBI

solution

-Le

ssthan

25%

oftheothe

rusersha

veaccess

totheBI

solution

-Be

tween50

%an

d70

%of

thedirectorsha

veaccess

totheBI

solution

-Be

tween25

%to

50%

oftheothe

rusersha

veaccess

totheBI

solution

-Be

tween70

%an

d90

%of

thedirectorsha

veaccess

totheBI

solution

-Be

tween50

%to

75%

oftheothe

rusersha

veaccess

totheBI

solution

-Morethan

90%

ofthedirectorsha

veaccess

totheBI

solution

-Morethan

75%

oftheothe

rusersha

veaccess

totheBI

solution

D2.Sy

stem

users

TheBI

solution

does

nottraceuser

access

-Le

ssthan

50%

ofthead

ministrativean

dclinical

directorsthat

have

access

totheBI

solution

consistently

useit

-Lessthan

25%

oftheothe

rad

ministrativean

dclinical

staff

that

have

access

totheBI

solution

consistently

useit

-Be

tween50

%an

d75

%of

thead

ministrative

andclinical

directorsthat

have

access

totheBI

solution

consistently

useit

-Be

tween25

%an

d50

%of

theothe

rad

ministrativean

dclinical

staff

that

have

access

totheBI

solution

consistently

useit

-Morethan

75%

ofthead

ministrativean

dclinical

directorsthat

have

access

totheBI

solution

consistently

useit

-Lessthan

50%

oftheothe

rad

ministrativean

dclinical

staff

that

have

access

totheBI

solution

consistently

useit

D3.Proc

ess

cove

rage

-Th

eBI

solution

does

notsupp

ortan

yof

thefour

func

tion

alactivities

(goa

lde

finition

,measuremen

t,ga

pan

alysis,active

decision

supp

ort)

inthead

ministrativeproc

esses(gen

eral

acco

unting

,logistics,

human

resources,

etc.)

-Th

eBI

solution

does

notsupp

ortan

yof

thefour

func

tion

alactivities

intheclinical

realm

(emerge

ncyroom

,outpa

tien

t,inpa

tien

t,im

aging,

labo

ratory,o

perating

room

,etc.)

-Th

eBI

solution

supp

orts

only

oneor

twoof

thefour

func

tion

alactivities

inthe

administrativeproc

esses

-Th

eBI

solution

supp

orts

only

oneor

twoof

thefour

func

tion

alactivities

intheclinical

realm

-Th

eBI

solution

supp

orts

threeof

thefour

func

tion

alactivities

inthead

ministrative

proc

esses

-Th

eBI

solution

supp

orts

threeof

thefour

func

tion

alactivities

intheclinical

realm

-The

BIsolution

supp

ortsallthe

four

func

tion

alactivities

inthead

ministrativeproc

esses

-The

BIsolution

supp

ortsallthe

four

func

tion

alactivities

intheclinical

realm


7


Table5

Maturityleve

lsforthe7dimen

sion

sin

theorga

nization

alarea.

Dim

ension

Leve

l1:

initial

Leve

l2:

man

aged

Leve

l3:

system

atic

Leve

l4:

disrup

ted

O1.BI

strategy

Thereis

noBI

strategy

define

din

the

orga

nization

forecon

omic,prod

uction

orqu

alitativeaspe

cts

-Th

eBI

strategy

isde

fine

din

term

oflocal

strategies

(inun

itsor

depa

rtmen

tleve

l-T

helocalstrateg

iesareon

lypa

rtially

aligne

dwith

theco

rporatestrategy

TheBI

strategy

isde

fine

din

theco

rporateleve

lan

dis

aligne

dwiththebu

sine

ssstrategy

TheBI

solution

lead

thech

ange

man

agem

entwithin

theorga

nization

O2.BI

budg

et-Ave

rage

annu

alspen

ding

onBI

isless

than

1%of

theICTop

erationa

lexpe

nditure

-Ave

rage

annu

alspen

ding

onBI

isless

than

1%of

theICTcapitalexpe

nditure

-Ave

rage

annu

alspen

ding

onBI

isbe

tween1%

and

3%of

theICTop

erationa

lexpe

nditure

-Ave

rage

annu

alspen

ding

onBI

isbe

tween1%

and

3%of

theICTcapitalexpe

nditure

-Ave

rage

annu

alspen

ding

onBI

isbe

tween

3%an

d7%

oftheICTop

erationa

lexp

enditure

-Ave

rage

annu

alspen

ding

onBI

isbe

tween

3%an

d7%

oftheICTcapitalexpe

nditure.

-

-Ave

rage

annu

alspen

ding

onBI

ismorethan

7%of

theICTop

erationa

lexpe

nditure

-Ave

rage

annu

alspen

ding

onBI

morethan

7%of

the

ICTcapitalexpe

nditure

O3.Organ

izationa

lcov

erag

e-Th

ereareno

internal

resourcesin

administrative,

ICTan

dclinical

realmsde

aling

withBI

-Th

ereareno

internal

proc

edures

specified

for

theBI

solution

-The

reareinternal

resourcesin

administrative,

ICT

andclinical

realmsde

dicatedto

BI,b

utthe

resourcesareno

texclusive

-Onlyforfew

aspe

ctstheinternal

proc

edures

are

specified

fortheBI

solution

-Th

ereareinternal

resourcesin

administrative,

ICTan

dclinical

realms

exclusivelyde

dicatedto

BI-Fo

rmostof

thetech

nicalan

dop

erationa

laspe

cts,

theinternal

proc

edures

arespecified

fortheBI

solution

-Th

ereis

anad

hocun

itexclusivelyde

dicatedto

controlan

dman

agetheBI

-The

adho

cun

itde

fine

san

dco

ntrolstheproc

edures

fortheBI

solution

O4.Key

-usercapa

bilities

Thereareno

key-usersof

theBI

solution

-Key

-users

have

only

tech

nicalcapa

bilities

-Th

eke

y-userstraining

prog

ramsaresporad

ican

dfocu

sedon

tech

nicalissues

-Th

eke

y-userspo

ssessproc

esscapa

bilities

-The

key-userstraining

prog

ramsaresporad

ican

dfocu

sedon

allBI

issues

-Key

-users

canan

ticipa

teusers'ne

eds/prob

lems

-Th

eke

y-userstraining

prog

ramsareco

ntinua

tive

andfocu

sedon

allBI

issues

O5.Usercapa

bilities

-Th

ead

ministrativean

dclinical

directorsdo

not

possessBI

expe

rtise(e.g.a

nalysisan

dinterpretation

oftherepo

rts)

-Other

administrativean

dclinical

usersdo

not

possessBI

expe

rtise.

-Th

ead

ministrativean

dclinical

directorsareon

lyab

leto

interpretstatic

repo

rts

-Other

administrativean

dclinical

usersareon

lyab

leto

interpretstatic

repo

rts

-Th

ead

ministrativean

dclinical

directors

have

theco

mpe

tenc

iesto

man

agedy

namic

repo

rts

-Other

administrativean

dclinical

userson

lyha

vetheco

mpe

tenc

yto

interprets

taticrepo

rts

-Th

ead

ministrativean

dclinical

directorsha

vethe

compe

tenc

iesto

man

agesoph

isticatedrepo

rtsan

dpe

rform

“pull”

analysis

-Other

administrativean

dclinical

usersha

vethe

compe

tenc

iesto

man

agesoph

isticatedrepo

rtsan

dpe

rform

“pull”

analysis

O6.Com

petenc

eim

prov

emen

tTh

ereareno

training

prog

ramsto

improv

etheBI

compe

tenc

iesof

theclinical

andad

ministrative

users

TheBI

training

prog

ramsaremainlyfocu

sedon

high

lightingBI

impo

rtan

ceTh

erearead

hocBI

training

prog

ramsfocu

sing

onspecificissues

-Con

tinu

ousBI

training

prog

rams

-Th

eBI

training

prog

ramsarede

sign

edto

create

power

usersthat

furthe

rco

ntribu

teto

improv

eco

mpe

tenc

iesof

theothe

rusers

O7.Pa

rtne

r/supp

lier

coordina

tion

-Th

ereareno

coordina

tion

andregu

latory

mecha

nism

swithBI

supp

liers

-The

BIsupp

liers

have

norole

inBI

man

agem

ent

-Se

rviceleve

lag

reem

ents

limited

toICTtopics

-Reactiveinvo

lvem

entof

theBI

supp

liers

limited

totech

nicalissues

-Se

rviceleve

lag

reem

ents

forco

ntinuo

usup

date

andim

prov

emen

tof

theBI

-Reactiveinvo

lvem

entof

theBI

supp

liers

for

operatingissues

-Th

eBI

supp

liers

prov

idepe

rforman

ceman

agem

ent

andKPIsfortheBI

solution

-Th

eBI

supp

liers'inv

olve

men

tis

proa

ctivean

dinno

vation

oriented


8


were developing a BI solution3);

• further evolutions that could be interesting to accomplish formaking the BI solution as synergic as possible with other state-of-the-art digital solutions in the healthcare domain (these evolutionsare the results of three face-to-face interviews with the CEO ofISMETT).

The RTF considered all the stimuli that emerged in this phase, re-cognizing that these stimuli were not sufficient to develop a roadmapthat would allow a real prioritization of the different interventions andinvestments. As a result, the RTF organized specific meetings to reflecton the different relationships among the dimensions and metricscharacterizing the model. To ensure reliability (Bourgeois andEisenhardt, 1988) all meetings were facilitated by two researchers,recorded, transcribed, and coded.

A cross-analysis of all the meetings allowed the researchers topropose a preliminary version of a framework of prerequisites (need fora prioritization in the development) and synergies (need for a con-current development) among the different dimensions of the BI ma-turity model. Exploiting the knowledge of ISMETT experts, the RTFreviewed the framework through a multi-participant interactive dia-logue (Mikaelsson and Shani, 2004), and worked together on a finalversion that was reviewed and then validated by the BI experts, man-agers, and users in the other five hospitals involved in the CIR project.

5.2. Outputs of stage B. A roadmap toward precision medicine for ISMETT

ISMETT's assessment of the BI maturity model allowed it to support,with sound evidence, management's impression that most of the effortsmade by ISMETT regarding its BI solutions had been in the technolo-gical area, in which the hospital had achieved an extremely high overall

level of maturity. The other areas were expected to be improved overthe three years following the assessment. This is consonant with what issuggested by both the literature (e.g., Sen et al., 2012) and the practi-tioners involved in the research project: inhomogeneous developmentstend to be resource-consuming, risky, and ineffective.

As an example of the considerations that the assessment enabled,consider the dimensions in the functional area (Table 1). Fig. 2 high-lights high maturity levels for the dimensions, which were expected togrow in the three years after the assessment. Among the four canonicalphases characterizing a management control system (F1, F2, F3 and F4),the one relative to “measurement” was the most supported by the BIsolution. Looking at the expected maturities, respondents envisioned adevelopment profile once again driven by “measurement,” but in whichthe other phases surpassed or approached level 3 of maturity.

In the final report produced for ISMETT, the researchers ofPolitecnico di Milano explained and detailed the position of all thedimensions characterizing the BI maturity model. When present, theycarried out specific analyses for sub-metrics. For example, at the time ofevaluation, the BI solution in ISMETT was extremely mature in themanagement of the data in the production and economic domains,while it looked to improve its support in the qualitative domains.

As mentioned, one of the main outputs of this stage is the frame-work of interdependencies (in terms of synergies and prerequisites)between the dimensions. Fig. 3 depicts the final framework producedby RTF. Considering two dimensions of the model (X and Y), allowed toidentify four different relationships between them:

• Strong prerequisite (►): this relationship indicates that in order toincrease the maturity of X, it is necessary to have previously reachedmid-high (3 or 4) levels of maturity in Y4;

• Prerequisite (→): this relationship indicates that in order to increasethe maturity of X, it is suggested to have previously reached mid-high (3 or 4) levels of maturity in Y;

• Strong synergy (••): this relationship indicates that it is necessary tosimultaneously evolve the maturity of X and Y;

• Synergy (•): this relationship indicates that it is suggested to si-multaneously evolve the maturity of X and Y.

The figure provides a healthcare organization aiming to increase thematurity of its BI solution, with important information to lead eachintervention/investment. A vertical analysis of the table emphasizes theprerequisites and the synergies necessary and/or suggested to increasethe maturity of a dimension. For example, consider the dimension“active support to decision-making” (the fourth column in the func-tional area). As indicated in Fig. 2, ISMETT had a maturity level of 2.22,consonant with current needs, but expected to achieve a level of 3.11over the next three years. To realize this maturity growth, it is notsufficient to improve the level of data granularity, the functional sup-port, and the frequency through which the BI solution supports thisfunction in economic, production, and qualitative domains. Fig. 3suggests that many other dimensions, both within the same develop-ment area as well as outside it, are critical in achieving this improve-ment.

Fig. 3 is extremely useful, even if read horizontally. In this case, it ispossible to verify the impacts produced by a dimension on the others,emphasizing the dimensions that have the highest priority due to thefact that they are a strong prerequisite of many other dimensions.

We linked the framework of the interdependencies presented in fig.3 with the maturity levels of the various dimensions of the BI model inorder to determine different clusters of dimensions to be prioritized. Toaccomplish this task, we: (i) averaged the maturity levels of the

Fig. 2. ISMETT position on the functional dimensions of the BI maturity model.

3 From this viewpoint, the RTF leveraged the assets of an Observatory (see Gastaldi &Corso (2013) for a description of the Observatory) and the managerial knowledge andexpertise of UPMC International.

4 For example, it is necessary to reach at least level 3 of maturity for the metric relativeto BI strategy (presence of a corporate strategy at a corporate level, see O1 in Table A.4) inorder to increase the maturity of the percentage of users who can access the system (D1 inTable A.3).


9


different metrics characterizing each dimension; (ii) translated thevarious prerequisites and synergies of each dimension into a compre-hensive value, calculated according to a predefined set of scores: 4points for each strong prerequisite of the dimension; 3 points for eachprerequisite; 2 points for each strong synergy; 1 point for each synergy.The relevance (priority) comprehensive value for a given dimension iscalculated by summing the scores on the row corresponding to the di-mension in Fig. 3; (iii) checked the consistency of the results in the RTF;(iv) presented them to the BI experts in ISMETT to collect their feed-back; and (v) validated the different clusters of dimension prioritiza-tion.

Crossing the relevance scores with the maturity levels assessedthrough the questionnaire (e.g., the values in Fig. 2 for the functionalarea), it has been possible to determine the four clusters depicted inFig. 4.5

The four clusters are the following:

• Strategic dimensions: dimensions in which ISMETT should con-solidate its investments because they are already mature, but arealso highly relevant (and often strong prerequisites) for the evolu-tion of other dimensions;

• Critical dimensions: dimensions on which ISMETT has to focus as

soon as possible because they are not mature, and are highly re-levant (and often strong prerequisites) for the evolution of otherdimensions;

• Consolidated dimensions: dimensions in which ISMETT should investmarginal resources because they have received a number of in-vestments in the past (reflected in high levels of maturity), and theirdevelopment has less influence on the development of other di-mensions;

• Postponable dimensions: dimensions that should be considered afterhaving tackled the critical dimensions, since their development hasless influence on the development of other dimensions, even if, in alogic of homogeneous development of the BI solution, their maturitylevels have to be aligned with others eventually.

The four clusters of dimensions have to be approached with dif-ferent modalities, resources, and timings.

In order to develop an even more specific, systematic and effectiveaction plan, and to harmonize the improvement of the maturity of thevarious dimensions, the RTF calculated a score for each dimension.Starting with Fig. 3, it translated the various prerequisites and synergiesof each dimension and their current and expected maturity into acomprehensive value (see Fig. 5).

Consider two dimensions Yj and Xj, where Yj is a prerequisite of (orhas synergy with) Xj. To prioritize the development of dimensions, ascore is assigned to each dimension Yj based on Eq.1:

Fig. 3. Perquisite and synergies among the dimensions of the BI maturity.

5 Two dimensions from the figure — BI strategy and BI budget — are considered asoutliers. According to practitioners these dimensions are so relevant that they representtwo pre-requisites of any BI solution aiming to be effective within a health care organi-zation.


10


Y

EM

AMAM

PS AM PS AM

Score

1

2j 1 n

j

i

n

X

XX

Y X Y Y X Y

1i

ii

j i j j i j

∑= ⎡⎣⎢

−− + − ⎤

⎦⎥ ∀ = …

=(

)( ) | |

(1)

where:

• n = number of components

• Score Yj = overall score given to component Yj,

• EMXi = expected maturity of component Xi in future

• AMXi = current maturity of component Xi

• PSYjXi = number of points given based on the relationship betweenYj and Xi (predefined scores of 4, 3, 2, or 1 respectively for strongprerequisite, prerequisite, strong interdependency and inter-dependency)

The higher values of the assigned score indicate the need for im-mediate investment or focus on the dimension.6

Fig. 5 depicts the calculated scores of ISMETT's BI dimensions. Asshown, BI budget, BI strategy, and process coverage are the first three

areas that deserve more attention and/or investment. This is also con-sistent with the results presented in Fig. 4 (the three dimensions areamong the critical dimensions).

6. Conclusions

The main contribution of this study is the development of a BImaturity model and the relative assessment questionnaire, which arespecific for the healthcare industry and allow to effectively address thepressing issues associated with BI solutions within it. In addition, thedesigned maturity model was applied to the BI solution of ISMETT todirectly show the applicability of the research results.

The research showed that the development of a BI solution is es-sentially an evolutionary process, and that is possible to identify severaldiscrete stages in the roadmap toward a full exploitation of BI in therealization of precision medicine. We proposed that an inadequate levelof BI solution maturity could be a major reason behind the failure ofmany BI initiatives. We mapped the relationships among the dimen-sions of the BI solution (especially those among different developmentareas), depicting the different interactions in terms of prerequisite andsynergies to be leveraged to successfully extend BI solutions to largerdomains.

We also showed how the level of maturity should always be con-sonant with organizational structure, management decisions, and stra-tegic changes in terms of growth foreseen in the near future.Furthermore, the maturity should be periodically re-evaluated to adjustBI implementation and diffusion according to company and environ-mental changes. The clusters and priority scores of the dimensions areupdated and maturity improvement strategy is adjusted after eachperiod. This can be performed easily as the assessment questionnairecan be filled in matter of hours. However further research is needed todetermine the optimal frequency for updating the maturity assessmentand adjusting the improvement strategy.

Moreover, by activating organization-wide processes of involve-ment, the BI maturity evaluation model described in this paper allows

Fig. 4. Relevance and maturity for healthcare BI at ISMETT.

6 The first part of Eq.1 takes into account the difference between the expected and

current maturity of the dependent dimension Xi, and the second coefficient ⎜ ⎟⎛⎝

⎞⎠AMXi

1

considers the inverse of the current maturity level of Xi, thus, the higher the differencebetween expected and actual maturity and the lower the current maturity of the de-pendent dimension Xi; the higher the score given to Yj. Finally, the third coefficient cal-culates the positive part of the difference of the relation score and the current maturityof Yj. If the current maturity of Yj is less than the maturity needed to develop Xi, thiscoefficient is positive; otherwise, it is zero. For example, if the dimension Yj is a pre-requisite of Xi, but the current maturity of Yj is high enough (3 or 4), the third coefficientis zero and eliminates the effect of the relation on the score of Yj. For better illustrations,the detailed calculations of the F4. Standards score are provided in the Appendix.


11


healthcare practitioners to monitor and predict the quality of their BIsolutions and the processes producing them. The artifacts are based onseveral dimensions and metrics, which not only enable any type ofbenchmarking regarding the strategies through which differenthealthcare organizations develop a BI solution, but are also useful toolsfor understanding which dimensions to focus in order to progressivelymake the BI solution of a healthcare organization more efficient andeffective.

We envision three streams of research emerging out of our work.Following the software process maturity paradigm (Krishnan andKellner, 1999), the first stream could focus on organizational attemptsto characterize BI practices by empirically examining the consensualbenefits attributed to a mature BI solution. For example, it is importantto use the BI maturity model to systematically measure a hospital'sability, commitment, goals, and roadblocks in evaluating its perfor-mance on the different metrics, and to develop benchmarks to transi-tion to higher levels of maturity. In this research stream, the basicpremise is that consistent application of well-defined and measured BIprocesses, coupled with continual process improvement, will streamlineBI project management, and substantially improve the productivity anddata quality of BI solutions.

A second stream of research could focus on the metrics, the maturitylevels and the interdependencies that have been developed. Based ontheir application at ISMETT, they appear to be comprehensive. But it isunclear whether all metrics and maturity levels are of equal value withrespect to BI maturity assessment. And it is unclear how to effectivelyweigh and precisely measure the strength of both the synergies and the

prerequisites that have been considered. It would be interesting toconduct field studies (in the form of surveys) that include a number oforganizational (e.g., size, system architecture, structural attributes, re-sources, management attitude, and culture) and environmental (e.g.,institutional and competitive forces, technology support structures)determinants of efforts that companies exert in pursuing initiatives toupgrade their BI maturity levels. Only in this way we can confirm thatthe synergies and perquisites developed for ISMETT are also general-izable to other hospitals.

Finally, an important future direction would be to employ ourmodel/questionnaire to assess the BI maturity in different healthcareorganizational settings and, based on those assessments, test a set ofhypotheses relating to the consequences of BI maturity on their per-formance. Moreover, if the maturity model were applied to all (or themajority of) healthcare organizations in a regional healthcare system,the model could provide the regional healthcare directorate with usefulknowledge to address the design of homogenizing policies and con-tinual improvement strategies at a regional level.

Acknowledgment

The third author of this paper holds an Erasmus Mundus JointDoctorate (EMJD) fellowship, namely the European Doctorate inIndustrial Management (EDIM), which is funded by the EuropeanCommission 2014-0699/001-001 EMJD (EDIM IV), Erasmus MundusAction 1.

Appendix A

Table A.1

Fig. 5. Priority scores of the dimensions for healthcare BI atISMETT.


12


Dimensions, metrics, and maturity levels in the functional area.

Dimension Metric Sub-metric⁎

Maturity levels

Level 1:initial

Level 2: managed Level 3: systematic Level 4: disrupted

F1. Goaldefinition

Level of datagranularity⁎⁎

Economic None ororganization

Unit or ward Event or cure episode Treatment procedureProductionQualitative

Functionalsupport

Economic No support Input of consolidatedtargets

Budget development Budget dynamicmanagementProduction

QualitativeFrequency⁎⁎ Economic Annually Quarterly Monthly Weekly or daily

ProductionQualitative

F2.Measureme-nt




Functionalsupport

Economic No support Only some operativeunits, manually

All operating units,manually

All operating units andautomatically⁎⁎Production

QualitativeFrequency⁎⁎ Economic Quarterly Monthly Weekly or daily Real time


F3. Gap analysis Level of datagranularity⁎⁎



Functionalsupport

Economic No support Only data collection Data collection and gapvisualisation

Data collection andautomatic gap analysisProduction



F4. Decisionmaking




Functionalsupport

Economic No support Ex post analysis OLAP Active support to decisionmakingProduction



F5. Data quality Controls oninbound data

Economic No automaticcontrols

Controls on less than40% of data


Systematic controls on allmanaged dataProduction

QualitativeControls onoutbound data

Economic No automaticcontrols



Systematic controls on allmanaged dataProduction

QualitativeF6. Functional

integrationInternal dataintegration⁎⁎⁎

Economic None Integration of less than30% of data

Integration of less than70% of data

Integration of more than70% of dataProduction

QualitativeExternal data integration⁎⁎⁎⁎ None Two Three, on less than 50%

of processesThree, on more than 50%of processes

Integration among functionalareas⁎⁎⁎⁎⁎

None Two Three All

⁎ Some metrics (e.g. the frequency of goal definition) have sub-metrics reflecting the different domains in which the metric can be measured (e.g., economic, production or qualitativedata); in these cases, the maturity levels reported in the table are valid for each sub-metric.

⁎⁎ The maturity levels for this metric/sub-metric are considered achieved by the healthcare organization only if it is valid in more than 50% of cases; for instance, a healthcareorganization is at level 3 of maturity for the frequency of goal definition only if in more than 50% of cases its BI solution allows it to define goals every month; for more information, seethe “prevalence logics” in Section 4.1.

⁎⁎⁎ Internal data integration refers to the extent to which the BI solution allows to work in an integrated manner on the data in a specific domain (economic data, production data orqualitative data).

⁎⁎⁎⁎ External data integration refers to the extent to which the BI solution allows to work in an integrated manner on the data in different domains (e.g. economic and production data).⁎⁎⁎⁎⁎ Integration among functional areas refers to the extent at which the BI solution allows to work in an integrated manner on the four functional areas characterizing the BI process

(goal definition, measurement, gap analysis and decision making).


13


Table A.2Dimensions, metrics, and maturity levels in the technological area.

Dimension Metric Sub-metric⁎

Maturity levels

Level 1:initial


T1. BIarchitecture

Structure of the architecture No architecture No decoupling Transactional andanalyticsdecoupling

Multi-levelarchitecture foranalytics

T2. Reporting Quality⁎⁎ Economic Static Static with graphicdata visualisation

Dynamic datanavigation

Dynamic statisticalsimulationProduction

QualitativeDistribution⁎⁎ Economic Paper-based Digital but manual Digital and automatic Always and directly

accessibleProductionQualitative

T3. Interface Interface characteristics Client-server Web-based Advanced (RIA)Device through which it ispossible to access the system

There is no BIsolution

Only through specificdevices

All desktop devices All mobile devices

T4. Userprofiling

Profiling level of BI users No profiling Macro-area profiling Single user profiling Context-basedprofiling

T5.Technologic-alintegration

Integration with internal systems(automatic data alimentation)

None Integration with lessthan 50% of systems

Integration with lessthan 85% of systems

Integration with morethan 85% of systems

Integration with externalsystems⁎⁎

Manual Mono-directional Manual bi-directional Automatic bi-directional

T6. Standards Usage of interoperabilitystandards by the BI solution

No standards usedby the BIapplications

Standards for lessthan 50% of BIapplications

Standards for lessthan 85% of BIapplications

Standards for morethan 85% of BIapplications

Number of interoperabilitystandards supported

No standards Only one standard Few standards The majority ofstandards

T7. Dataprovisioning

Level of datagranularity ofinbound data⁎⁎



Frequency⁎⁎ Economic Quarterly Monthly Weekly or daily Real timeProductionQualitative

⁎ Some metrics (e.g., the quality of reporting) have sub-metrics reflecting the different domains in which the metric can be measured (e.g., economic, production or qualitative data); inthese cases, the maturity levels reported in the table are valid for each sub-metric.

⁎⁎ The maturity levels for this metric/sub-metric are considered achieved by the healthcare organization only if it is valid in more than 50% of cases; for example, a healthcareorganization is at level 2 of maturity for the quality of reporting only if in more than 50% of cases its BI solution has static reporting with tools for graphic data visualisation; if the BIsolution provides these tools only in 30% of cases the healthcare organization achieves a maturity level of 1 (static reporting); for more info., see the “prevalence logics” in Section 4.1.

Table A.3Dimensions, metrics, and maturity levels in the diffusional area.

Dimension Metric Sub-metric⁎ Maturity levels

Level 1:initial

Level 2: managed Level 3: systematic Level 4:disrupted

D1.Accessin-g users

Administrativerealm

Directors Less than 50% ofdirectors

Between 50% and70% of directors


More than 90%of directors

Other users Less than 25% ofusers

Between 25% and50% of users



Clinical realm Directors Less than 50% ofdirectors




Physicians Less than 25% ofusers



More than 75%of directorsNurses

Other usersD2. System

usersAdministrativerealm

Directors The system does nottrace its use

Less than 50% of users Between 50% and75% of users


Other users The system does nottrace its use



Clinical realm Directors The system does nottrace its use




14


Physicians The system does nottrace its use


More than 50%of directorsNurses

Other usersD3. Process

coverageAdministrativerealm

General accounting⁎⁎ None One or two Three AllManagementaccounting⁎⁎

None One or two Three All

Purchasing⁎⁎ None One or two Three AllLogistics andwarehouse⁎⁎


Human resources⁎⁎ None One or two Three AllInformationsystems⁎⁎


Other processes⁎⁎ None One or two Three AllClinical realm Emergency room⁎⁎ None One or two Three All

Admission,discharge, transfers⁎⁎


Outpatient,inpatient⁎⁎


Operating rooms⁎⁎ None One or two Three AllLaboratory⁎⁎ None One or two Three AllImaging⁎⁎ None One or two Three AllCommunity care⁎⁎ None One or two Three AllOther processes⁎⁎ None One or two Three All

⁎ Some metrics (e.g., the accessing users in the admin. Realm) have sub-metrics reflecting the different domains in which the metric can be measured (e.g., economic, production orqualitative); in these cases, the maturity levels reported in the table are valid for each sub-metric.

⁎⁎ Process coverage reflects how many of the four functional areas characterizing the BI process (goal definition, measurement, gap analysis and decision making) are covered by the BIsolution.

Table A.4Dimensions, metrics, and maturity levels in the organizational area.

Dimension Metric Sub-metric⁎ Maturity levels

Level 1:initial


O1. BI strategy Presence of a BIstrategy

Economic No strategy Local strategies Corporate strategy The BI lead the changemanagementProduction

QualitativeO2. BI budget Average annual percentage of ICT OPEX

delivered to BI in the last 3 years⁎⁎Less than 1% Between 1% and 3% Between 3% and

7%More than 7%

Average annual percentage of ICTCAPEX delivered to BI in the last 3years⁎⁎

Less than 1% Between 1% and 3% Between 3% and7%

More than 7%

O3.Organization-al coverage

Dedicated resources Administrativerealm

No internal BIresources

BI resources but notdedicated

Dedicated BIresources

Ad hoc unit

ICT direction No internal BIresources



Ad hoc unit

Clinical realm No internal BIresources



Ad hoc unit

Coverage of specific procedures for BI Null Only some aspects Most tech. Andoperational aspects

Ad hoc unit fordefinition and control

O4. Key-usercapabilities

Experience of key users There are nokey users

Key users withtech.-onlycapabilities

Key users withprocess capabilities

Key users able toanticipate users' needs/problems

Training programs No trainingactivities

Sporadic andfocused on tech.Issues

Sporadic andfocused on all BIissues

Continuative andfocused on all BI issues


15


O5. Usercapabilities

Administrative realm Directors Nocapabilities

Interpretation ofstatic reports

Management ofdynamic reports

Sophisticated “pull”analyses

Other users Nocapabilities




Clinical realm Directors Nocapabilities




Physicians Nocapabilities




Nurses Nocapabilities




Other users Nocapabilities




O6. Competenceimprovement

Training programs inthe administrativerealm

Directors No trainingactivities

Transferring theimportance of BI

Ad hoc to solvespecific issues

Continuous training

Other users No trainingactivities



Continuous training

Training programs inthe clinical realm

Directors No trainingactivities



Continuous training

Physicians No trainingactivities



Continuous training

Nurses No trainingactivities



Continuous training

Other users No trainingactivities



Continuous training

O7. Partner/suppliercoordination

Coordinating mechanisms with BIsuppliers

Nocoordinatingmechanisms

SLAs limited to ICTtopics

Continuousimprovement SLAs

BI performancemanagement systemand KPIs

Role of partners and suppliers No role inmanaging BI

Reactiveinvolvement fortech.-only issues

Reactiveinvolvement foroperating issues

Proactive andinnovation-oriented

⁎ Some metrics (e.g., the presence of a BI strategy) have sub-metrics reflecting the different domains in which the metric can be measured (e.g., economic, production or qualitative); inthese cases, the maturity levels reported in the table are valid for each sub-metric.

⁎⁎ We asked for an average percentage of expenditure to avoid any potential fluctuation in BI budget linked to contingent events.


16


Table A.5Sample questionnaire.

Area Dimension Metric Sub-metric

Functional Level of data granularity Economic

Current Future Question : What is the level of granularity of the data on which the BI solutions can work in order

to support the definition of a goal in the economic domain (cost and revenues)?

In more than 50% of cases the BI solution does not support the definition of a goal in the economic

domain; when the BI solution supports this functionality, the relative data can be considered only at

a corporate level (e.g., when it is necessary to define the goals to be achieved, the BI solution presents

and allows working on drug costs for the entire healthcare organization)

In more than 50% of cases the BI solution supports the definition of a goal in the economic domain

with data at the level of unit and/or ward (e.g., when it is necessary to define the goals to be achieved,

the BI solution presents and allows working on the costs of diagnostic procedures of each department

constituting the healthcare organization)


with data at the level of the single event or cure episode (e.g., when it is necessary to define the goals

to be achieved, the BI solution presents and allows working on the cost of a DRG)


with atomic data at the level of the single treatment procedure (e.g., when it is necessary to define

the goals to be achieved, the BI solution presents and allows working on the costs of the HRs for

each treatment)

Organizational

F1. Goal definition

O1. BI strategy Presence of BI strategy Economic

Current Future Question : Is there a BI strategy in the hospital on the economic aspects (costs, revenues)?

There is no BI business strategy for the economic aspect

There is a BI strategy on this aspect, but it is defined locally by units or departments. The local strategies are partially aligned with the company's strategy

There is a BI strategy on this aspect, aligned with the business strategy and with a strong commitment from the corporate level management

The BI Lead the change management on this aspect and BI indicators directly affect the business strategy and strategic decisions

(continued on next page)


17


Table A.5 (continued)

Technological T3. Interface Interface characteristics —

Current Future Question: Which are the most advanced characteristics of the interface of the BI solution?

There is no real BI solution available or it has a command line interface

The BI solution has a graphical-based (point & click) interface available locally and accessible by specific

clients

The BI solution has a web interface

The BI solution has a Rich Internet Application (RIA) interface

Technological T3. Interface Devices that can access BI solution—

Current Future Question: Which are the devices that can be used to access the BI solution?

No real BI solution has been implemented in the organization

Users can access the BI solution only through specific workstations

Users can access the BI solution through any desktop devices (PC, notebook, etc.) connected to the network

Users can access the BI solution even with mobile devices (tablets, smartphones, etc.)

Appendix B

Detailed calculation of Eq. 1

According to Fig. 3, F4 is prerequisite of or has synergy with F5, F6, T3, T5, D1, D2, D3, O3 and O5 and the influence on the other dimensions is 0:

Score of F4 3 21

2

2 2 22 2 2 22

23 22 2 89

1

2 89

2 2 22 2 2 22

23 5 3 5

1

3 5

1 2 22 1 2 22

2

3 3 2 651

2 65

2 2 22 2 2 22

22 1 25

1

1 25

1 2 22 1 2 22

24 3 88

1

3 88

1 2 22 1 2 22

2

2 45 1 691

1 69

2 2 22 2 2 22

22 5 1 75

1

1 75

3 2 22 3 2 22

22 63 2

1

2

2 2 22 2 2 22

20 33

F F T

T D D

D O O

5 6 3

5 1 2

3 3 5

= − ∗ ∗ − + − + − ∗ ∗ − + − + − ∗ ∗ − + −

+ − ∗ ∗ − + − + − ∗ ∗ − + − + − ∗ ∗ − + −

+ − ∗ ∗ − + − + − ∗ ∗ − + − + − ∗ ∗ − + − =

( ) ( . ) | . | ( . . ).

( . ) | . | ( . . ).

( . ) | . |

( . . ).

( . ) | . | ( . ).

. | . | ( . ).

( . ) | . |

( . . ).

( . ) | . | ( . . ).

( . ) | . | ( . ) ( . ) | . | .

The positive part of the Eq. 1 is 0 for all the dimension except for O3 as the current maturity of the F4 (2.22) in equal or greater than the requiredmaturity based on the type of the dependency (As mentioned earlier Strong perquisite, perquisite, Strong synergies, synergies require maturity levels4, 3,2 and 1 respectively). In other words, an improvement in the maturity of F4 only facilitates the improvement of the O3.

(continued on next page)


18


References

Agarwal, R., Gao, G.G., DesRoches, C., Jha, A.K., 2010. The digital transformation ofhealthcare: current status and the road ahead. Inf. Syst. Res. 21, 796–809.

Anandarajan, M., Anandarajan, A., Srinivasan, C., 2012. Business IntelligenceTechniques: A Perspective from Accounting and Finance. Springer Science & BusinessMedia, Berlin.

Avison, D., Young, T., 2007. Time to rethink health care and ICT? Commun. ACM 50,69–74.

Ayal, M., Seidman, A., 2009. An empirical investigation of the value of integrating en-terprise information systems: the case of medical imaging informatics. J. Manag. Inf.Syst. 26, 43–68.

Behkami, N.A., Daim, T.U., 2012. Research forecasting for health information technology(HIT), using technology intelligence. Technological Forecasting & Social Change 79,498–508. http://dx.doi.org/10.1016/j.techfore.2011.08.015.

Blondiau, A., Mettler, T., Winter, R., 2015. Designing and Implementing Maturity Modelsin Hospitals: An Experience Report from 5 Years of Research. (Health informaticsjournal).

Borzekowski, R., 2009. Measuring the cost impact of hospital information systems:1987–1994. J. Health Econ. 28, 938–949.

Bourgeois, L.J., Eisenhardt, K.M., 1988. Strategic decision processes in high velocityenvironments: four cases in the microcomputer industry. Manag. Sci. 34, 816–835.

Brooks, P., El-Gayar, O., Sarnikar, S., 2013. Towards a Business Intelligence MaturityModel for Healthcare, in: 46th Hawaii International Conference System Sciences(HICSS).

Brooks, P., El-Gayar, O., Sarnikar, S., 2015. A framework for developing a domain specificbusiness intelligence maturity model: application to healthcare. Int. J. Inf. Manag. 35,337–345. http://dx.doi.org/10.1016/j.ijinfomgt.2015.01.011.

de Bruin, T., Freeze, R., Kaulkarni, U., Rosemann, M., 2005. Understanding the mainphases of developing a maturity assessment model. In: 16th Australasian Conferenceon Information Systems,. Sydney, (p. paper 109).

Chen, H., Chiang, R., Storey, V., 2012. Business intelligence and analytics: from big datato big impact. MIS Q. 36, 1165–1188.

Christensen, C.M., Grossman, J., Hwang, J., 2009. The Innovator's Prescription, theInnovator's Prescription: A Disruptive Solution for Health Care. McGraw–Hill, NewYork (NY).

Chuah, M., Wong, K., 2011. A review of business intelligence and its maturity models.African journal of business management 5, 3424–3428.

Coghlan, D., Brannick, T., 2005. Doing Action Research in your Own Organization. SAGEPublications, London (UK).

Davenport, T., Harris, J., 2007. Competing on Analytics: The New Science of Winning.Harvard Business School Press, Boston (MA).

El-Gayar, O., Timsina, P., 2014. Opportunities for business intelligence and big dataanalytics in evidence based medicine. In: 2014 47th Hawaii International Conferenceon System Sciences. IEEE, pp. 749–757. http://dx.doi.org/10.1109/HICSS.2014.100.

Ferré, F., de Belvis, A.G., Valerio, L., Longhi, S., Lazzari, A., Fattore, G., Ricciardi, W.,Maresso, A., 2014. Health system review: Italy. Health systems in transition. Health16, 1–168.

Fichman, R.G., Kohli, R., Krishnan, R., 2011. The role of information systems in health-care: current research and future trends. Inf. Syst. Res. 22, 419–428. http://dx.doi.org/10.1287/isre.1110.0382.

Foshay, N., Kuziemsky, C., 2014. Towards an implementation framework for businessintelligence in healthcare. Int. J. Inf. Manag. 34, 20–27. http://dx.doi.org/10.1016/j.ijinfomgt.2013.09.003.

Fraser, P., Moultrie, J., Gregory, M., 2002. The use of maturity models/grids as a tool inassessing product development capability. In: International Engineering ManagementConference (IEMC), pp. 244–249 Cambridge.

Gandomi, A., Haider, M., 2015. Beyond the hype: big data concepts, methods, and ana-lytics. Int. J. Inf. Manag. 35, 137–144. http://dx.doi.org/10.1016/j.ijinfomgt.2014.10.007.

Gartner, 2015. Flipping to Digital Leadership: Insights from the 2015 Gartner CIO AgendaReport. (Stamford).

Gastaldi, L., Corso, M., 2012. Smart healthcare digitalization: using ICT to effectivelybalance exploration and exploitation within hospitals. International Journal ofEngineering Business Management 4, 1–13.

Gastaldi, L., Corso, M., 2013. Academics as effective orchestrators of interorganizationalchange and development: the experience of Italian observatories, in: Pasmore, W.,Woodman, R., Shani, A.B. (Rami), Noumair, D.A. (Eds.), Research in OrganizationalChange and Development (Vol. 21). Emerald, Bingley (United Kingdom), pp. 59–119.

Gastaldi, L., Mangiaracina, R., Miragliotta, G., Tumino, A., Perego, A., 2015. Measuringthe benefits of tracking medical treatment through RFId. Int. J. Product. Perform.Manag. 64, 175–193.

Gibson, C., Nolan, R., 1974. Managing the four stages of EDP growth. Harv. Bus. Rev. 52,76–88.

Hanson, R., 2011. Good health information–an asset not a burden!. Aust. Health Rev. 35,9–13.

Janssen, M., Moors, E.H.M., 2015. Caring for healthcare entrepreneurs — towards suc-cessful entrepreneurial strategies for sustainable innovations in Dutch healthcare.Technological Forecasting & Social Change 80, 1360–1374. http://dx.doi.org/10.1016/j.techfore.2012.12.003.

Krishnan, M., Kellner, M., 1999. Measuring process consistency: implications for reducingsoftware defects. IEEE Trans. Softw. Eng. 25, 800–815.

Kuiler, E.W., 2014. From big data to knowledge: an ontological approach to big dataanalytics. Review of Policy Research 31, 311–318. http://dx.doi.org/10.1111/ropr.12077.

Lahrmann, G., Marx, F., 2010a. Systematization of Maturity Model Extensions. In: Winter,R., Zhao, J.L., Aier, S. (Eds.), Global Perspectives on Design Science Research.Springer, Berlin, pp. 522–525.

Lahrmann, G., Marx, F., 2010b. Business intelligence maturity models: an overview. In:VII Conference of the Italian Chapter of AIS (itAIS). Italian chapter of AIS, Naples.

Lahrmann, G., Marx, F., Winter, R., Wortmann, F., 2011. Business intelligence maturity:development and evaluation of a theoretical model. In: Proceedings of the 44thHawaii International Conference on System Sciences, pp. 1–10. http://dx.doi.org/10.1109/HICSS.2011.90.

Li, M., Mao, J., 2015. Hedonic or utilitarian? Exploring the impact of communicationstyle alignment on user's perception of virtual health advisory services. Int. J. Inf.Manag. 35, 229–243. http://dx.doi.org/10.1016/j.ijinfomgt.2014.12.004.

Lucas, F.L., Sirovich, B.E., Gallagher, P.M., Siewers, A.E., Wennberg, D.E., 2010. Variationin cardiologists' propensity to test and treat: is it associated with regional variation inutilization? Circulation. Cardiovascular quality and outcomes 3, 253–260. http://dx.doi.org/10.1161/CIRCOUTCOMES.108.840009.

Maier, A., Moultrie, J., Clarkson, P., 2009. Developing maturity grids for assessing or-ganisational capabilities: Practitioner guidance. In: 4th International Conference onManagement Consulting. Academy of Management, Vienna.

Marx, F., Wortmann, F., Mayer, J., 2012. A maturity model for management controlsystems. Business and Information Systems Engineering 4, 193–207.

McGraw, K., Harbison-Briggs, K., 1989. Knowledge Acquisition: Principles andGuidelines. Prentice-Hall, New York (NY).

Mettler, T., Rohner, P., 2009. Situational maturity models as instrumental artifacts fororganizational design. In: Proceedings of DESRIST, (New York).

Mettler, T., Vimarlund, V., 2009. Understanding business intelligence in the context ofhealthcare. Health informatics journal 15, 254–264. http://dx.doi.org/10.1177/1460458209337446.

Mikaelsson, J., Shani, A., Albers, S., 2004. Rethinking and transforming product devel-opment. In: Adler, N., Shani, A.B. (Eds.), Collaborative Research in Organizations.Sage, Thousand Oaks, pp. 181–196.

Mohrman, A.M.J., Mohrman, S.A., 2004. Self-designing a performance managementsystem. In: Alder, N., Shani, A.B., Styhre, A. (Eds.), Collaborative Research inOrganizations: Foundations for Learning, Change, and Theoretical Development.Sage, Thousand Oaks.

Negash, S., 2004. Business intelligence. The Commun. Assoc. Inf. Syst. 13, 54.Pai, F., Huang, K., 2011. Applying the technology acceptance model to the introduction of

healthcare information systems. Technological Forecasting & Social Change 78,650–660. http://dx.doi.org/10.1016/j.techfore.2010.11.007.

Patton, M., 2002. Qualitative Evaluation and Research Methods, 3rd ed. Sage, ThousandOaks.

Pine, M., Sonneborn, M., Schindler, J., Stanek, M., Maeda, J.L., Hanlon, C., 2012.Harnessing the power of enhanced data for healthcare quality improvement: lessonsfrom a Minnesota Hospital Association Pilot Project. J. Healthc. Manag. 57, 406–418.

Popovič, A., Hackney, R., Coelho, P.S., Jaklič, J., 2012. Towards business intelligencesystems success: effects of maturity and culture on analytical decision making. Decis.Support. Syst. 54, 729–739.

Qaseem, A., Alguire, P., Dallas, P., Feinberg, L.E., Fitzgerald, F.T., Horwitch, C.,Humphrey, L., Leblond, R., Moyer, D., Wiese, J.G., Weinberger, S., Alue, H.I.G.H.,Are, C.O.S.T.O.C., 2012. Appropriate use of screening and diagnostic tests to fosterhigh-value, cost-conscious care. Ann. Intern. Med. 156, 147–150.

Romanow, D., Cho, S., Straub, D., 2012. Riding the wave: past trends and future direc-tions for health IT research. MIS Q. 36, 3–10.

Rud, O., 2009. Business Intelligence Success Factors: Tools for Aligning your Business inthe Global Economy, 18th ed. John Wiley & Sons.

Schein, E., 2008. Clinical inquiry/research. In: Reason, P., Bradbury, H. (Eds.), The SAGEHandbook of Action Research: Participative Inquiry and Practice. Sage, ThousandOaks, pp. 266–279.

Sen, A., Ramamurthy, K., Sinha, A., 2012. A model of data warehousing process maturity.IEEE Trans. Softw. Eng. 38, 336–353.

Stebbins, M., Shani, A., 2009. Clinical inquiry and reflective design in a secrecy-basedorganization. J. Appl. Behav. Sci. 45, 59–89.

Tremblay, M.C., Hevner, A.R., Berndt, D.J., 2012. Design of an information volatilitymeasure for health care decision making. Decis. Support. Syst. 52, 331–341. http://dx.doi.org/10.1016/j.dss.2011.08.009.

Verganti, R., 2017. Overcrowded: Designing Meaningful Products in a Word Awash withIdeas. MIT press, New York (NY).

Wang, Y., Kung, L., Anthony, T., 2016. Big data analytics: understanding its capabilitiesand potential benefits for healthcare organizations. Technological Forecasting SocialChange. http://dx.doi.org/10.1016/j.techfore.2015.12.019.

Wixom, B., Watson, H., 2012. The BI-based organization. In: Herschel, R. (Ed.),Organizational Applications of Business Intelligence Management: Emerging Trends.IGI Global, Hershey (PA), pp. 193–208.

Womack, J.P., Jones, D.T., 2003. Lean Thinking. Simon and Schuster, New York (NY).


19


http://refhub.elsevier.com/S0040-1625(16)30638-2/rf0005










http://dx.doi.org/10.1016/j.techfore.2011.08.015











http://dx.doi.org/10.1016/j.ijinfomgt.2015.01.011















http://dx.doi.org/10.1109/HICSS.2014.100




http://dx.doi.org/10.1287/isre.1110.0382

http://dx.doi.org/10.1287/isre.1110.0382
























http://dx.doi.org/10.1111/ropr.12077

http://dx.doi.org/10.1111/ropr.12077









http://dx.doi.org/10.1161/CIRCOUTCOMES.108.840009

http://dx.doi.org/10.1161/CIRCOUTCOMES.108.840009










http://dx.doi.org/10.1177/1460458209337446

http://dx.doi.org/10.1177/1460458209337446

































http://dx.doi.org/10.1016/j.dss.2011.08.009

http://dx.doi.org/10.1016/j.dss.2011.08.009








Luca Gastaldi is assistant professor in the Department of Management, Economics andIndustrial Engineering at Politecnico di Milano, where he teaches “Analysis and Design ofManagerial Processes”. He holds a master of science in management engineering and aPhD in management from Politecnico di Milano. His research is focused on digital in-novation, ambidexterity, smart working and collaborative research. He is the director ofthe Digital Agenda Observatory at Politecnico di Milano, and senior researcher in theDigital Innovation in Healthcare Observatory.

Astrid Pietrosi is director of Management Control and Decision Support atIRCCS–ISMETT. She holds a master of science in economics at University of Palermo andan MBA from SAA Business School. Her research is focused on the application of businessintelligence to the healthcare industry.

Sina Lessanibahri received the bachelor of science in industrial engineering form IranUniversity of Science and Technology. He holds a master science in industrial engineeringfrom University Technology Malaysia and a master of science (cum laude) in mathema-tical modelling from the University of L'Aquila. He is currently pursuing a double degreePhD in Politecnico di Milano and Universidad Politécnica de Madrid. His main researchinterests concern business intelligence and business analytics and their application inhealthcare.

Marco Paparella is applied researcher at the Department of Management, Economics andIndustrial Engineering at Politecnico di Milano, where he works in the Digital InnovationObservatories. He holds a master of science in management engineering at Politecnico diMilano and a master in business administration and organizational development at MIP,the Graduate School of Business of Politecnico di Milano. He works as senior researcherand consultant on digital innovation, with an emphasis on healthcare and public

administration sectors.

Antonio Scaccianoce is controller at IRCCS–ISMETT. He holds a master of science ineconomics from University of Palermo and a master in business administration at ALMABologna Business School. His major research is on business intelligence development inhealthcare.

Giuseppe Provenzale is a junior controller at IRCCS–ISMETT. He holds a master ofscience in management engineering from University of Palermo. His major research is onbusiness intelligence development in healthcare.

Mariano Corso is full professor in the Department of Management, Economics andIndustrial Engineer-ing of Politecnico di Milano, where he teaches “Leadership andInnovation” and “Business Management and Organization”. He is a co-founder andmember of the scientific board of the Digital Innovation Observatories of Politecnico diMilano. His major research interests and consulting expertise relate to organization,change management and ICT governance.

Bruno Gridelli is a world-renowned expert in transplantation and has successfullytransferred these clinical skills into leading administrative roles as director of UPMC Italyand of IRCCS–ISMETT. With Dr. Gridelli's unique clinical insight and expertise,IRCCS–ISMETT has significantly grown the capability of its liver transplant program andhas embarked on new clinical programs in heart, lung and pancreas transplantation. Asexecutive vice president of UPMC International Services, Dr. Bruno Gridelli supports theclinical and research aspects of UPMC's international operations, contributing to thedevelopment of a network of collaborations and partnerships across the globe.


20


technological forecasting & social change · measured the interdependencies between the...

Documents