understanding the implementation and integration of e-health … · 2012. 1. 6. · queen's...

Queen's Printer and Controller of HMSO 2007

Understanding the Implementation and Integration of e-Health Services.

Research Report

Produced for theNational Institute for Health Research

Service Delivery and Organisation programme

January 2009

prepared by

Professor Frances S Mair

� Section of General Practice and Primary Care, University of Glasgow

Professor Carl May

� Institute of Health and Society, Newcastle University

Dr Elizabeth Murray

� Department of Primary Care and Population Health, University College London

Dr Tracy Finch

� Institute of Health and Society, Newcastle University

SDO Project (08/1602/135)


Dr George Anderson


Dr Catherine O’Donnell


Professor Paul Wallace

� Department of Primary Care and Population Health, University College London

Professor Frank Sullivan

� The Health Informatics Centre, University of Dundee

Address for correspondence

Professor Frances S Mair

Section of General Practice and Primary Care

Division of Community Based Sciences

1 Horselethill Road

Glasgow

G12 9LX

E-mail: [email protected]



Contents

Research Report .........................................................................1 Produced for the National Institute for Health Research Service Delivery and Organisation programme ..........................................................1

January 2009.............................................................................1

Contents.....................................................................3

Acknowledgements ....................................................8

1 INTRODUCTION .................................................9 1.1 Background .........................................................................9 1.2 Aims 10 1.3 Plan of investigation............................................................ 10

2 Developing the Conceptual Model ....................14

3 Work Package 1 (WP1): Summary and synthesis of literature on the implementation of e-Health ......................................................................18

3.1 Background ....................................................................... 18 3.2 Stage 1: Scoping Review – Aims/Objectives........................... 19 3.3 Stage 1 – Methods.............................................................. 19 3.4 Stage 1 – Results ............................................................... 21

3.4.1 Limitations of scoping exercise ....................................27 3.4.2 Scoping exercise conclusions.......................................27

3.5 Stage 2 – Aims/Objectives ................................................... 28 3.6 Methods ............................................................................ 28

3.6.1 Criteria for considering studies for this review ...............28 3.6.2 Search strategy for identification of studies...................29 3.6.3 Identification of papers for inclusion.............................31

3.7 Stage 2 - Results ................................................................ 32 3.7.1 Papers identified........................................................32 3.7.3 Thematic Coding of Systematic Review..........................39

3.8 Discussion and conclusions .................................................. 43

4 Work-package Two (WP2) Barriers and Facilitators to the Implementation of E-Health Services: The Perspective of Health Professionals....46

4.1 Background ....................................................................... 46 4.2 Aims/Objectives ................................................................. 46 4.3 Methods ............................................................................ 47

4.3.1 Design .....................................................................47 4.3.2 Task Groups..............................................................47



4.3.3 Setting and sample....................................................47 4.3.4 Key Informant Interviews. ..........................................50

4.4 Data Analysis. .................................................................... 52 4.5 Results .............................................................................. 53

4.5.1 Patient benefits and clear purpose. ..............................53 4.5.2 Collaborative working.................................................56 4.5.3 Standards.................................................................58 4.5.4 Ease of use ...............................................................60 4.5.5 Professional attitudes .................................................62 4.5.6 Clear rationale...........................................................64 4.5.7 Cost.........................................................................65

4.6 Results Using the NPM as a Coding Frame.............................. 67 4.7 Discussion. ........................................................................ 72 4.8 Conclusion: Recommendations for Implementation. ................ 75

5 Work package three (WP3) Development and Validation of the Technology Adoption Readiness Scale (TARS) ............................................................77

5.1 Background ....................................................................... 77 5.2 Aims/Objectives ................................................................. 78

5.2.1 Structure of the Work Package Report ..........................78 5.3 Phase 1: Item development & expert survey .......................... 78

5.3.1 Aim of phase 1 ..........................................................78 5.3.2 Methods ...................................................................78 5.3.3 Item construction and piloting.....................................78 5.3.4 Online survey of experts .............................................79

5.4 Results .............................................................................. 81 5.4.1 Description of sample.................................................81 5.4.2 Data analysis/results..................................................82 5.4.3 Key messages from Phase 1........................................82

5.5 Phase 2: Development of TARS ............................................ 83 5.5.1 Aim of phase 2 ..........................................................83

5.6 Developing TARS Generic..................................................... 83 5.6.1 Methods ...................................................................83 5.6.2 Results.....................................................................83 5.6.3 Testing TARS Generic with health care professionals ......84 5.6.4 Results TARS Generic .................................................85 5.6.5 Key messages from Phase 2........................................86

5.7 Phase 3: Testing and Validation of TARS Specific in study sites. 86 5.7.1 Aim of Phase 3 ..........................................................86 5.7.2 Methods ...................................................................86 5.7.3 Results.....................................................................88

5.8 Discussion ......................................................................... 90 5.8.1 Main findings ............................................................90 5.8.2 Methodological issues.................................................92



5.8.3 Comparison of findings with existing literature ..............93 5.8.4 Recommendations for further development & application

of TARS....................................................................93 5.9 WP3 Outcomes................................................................... 94

6 Work-Package 4 (WP4) The implementation of e-Health systems: perspectives of e-Health implementers on the integration of new technologies into everyday work. .................................................95

6.1 Background ....................................................................... 95 6.2 Phase 1 Aims and Objectives................................................ 95 6.3 Phase 1 Methods ................................................................ 96 6.4 Results .............................................................................. 98

6.4.1 Identification of Case Studies and Interviewees. ............98 6.4.2 Findings from case studies........................................ 100 6.4.3 Case Study 2: Picture Archiving and Communication

System .................................................................. 103 6.4.4 Case Study 3: Clinical Nurse Information System........ 106 6.4.5 Summary of overall findings...................................... 110

6.5 Discussion and conclusions ................................................ 111 6.5.1 Discussion and Conclusions....................................... 112

6.6 Phase 2 Development and Formative Evaluation of the e-Health Implementation Toolkit...................................................... 113 6.6.1 Introduction and Background. ................................... 113 6.6.2 Aims and objectives ................................................. 113 6.6.3 Methods ................................................................. 114

6.7 Results ............................................................................ 116 6.8 Discussion and Conclusions................................................ 116

7 Taking Analysis To A Higher Level..................118 7.1 Background ..................................................................... 118 7.2 Aims/Objectives ............................................................... 119 7.3 Methods .......................................................................... 120 7.4 Results ............................................................................ 120

7.4.1 Coherence .............................................................. 122 7.5 WP2 Analysis Using NPT .................................................... 125 7.6 WP3 Analysis using NPT..................................................... 129 7.7 The “Added Value” Provided by the NPM and NPT and a multiple

work-package approach..................................................... 130 7.7.1 Discussion .............................................................. 133

8 Conclusions/Key Findings ..............................135 8.1 Overview ......................................................................... 135

8.1.1 Implications for Policymakers .................................... 135 8.1.2 Implications for e-Health implementers. ..................... 138 8.1.3 Implications for Research.......................................... 140

References .............................................................142



Appendix 1 NPM Definitions ...................................150

Appendix 2 List of papers included in WP1, Phase 1 scoping review .......................................................154

Appendix 3 WP1 Papers included and excluded at full paper stage ......................................................168

Appendix 4 WP1 Details of included reviews..........176

Appendix 5 DATA EXTRACTED FOR THEMATIC ANALYSIS - SYSTEMATIC REVIEW..........................186

Appendix 6 WP2 Information letter Task Group .....262

Appendix 7 WP2 interview Invitation letter ...........266

Appendix 8 WP2 Summary of Key Findings ............271

Appendix 9 WP2 Task Group Schedule ...................274

Appendix 10 WP2 Interview Schedule....................276

Appendix 11 WP3 Sample characteristics of expert survey participants ................................................278

Appendix 12 WP3 Expert sample: Factor rating scores: frequencies, means and standard deviation279

Appendix 13 WP3 Phase 2: QUESTION ANALYSIS BY NPM CONSTRUCT....................................................282

Appendix 14 WP3 Analysis of rating items for Phase 2.............................................................................285

Appendix 15 WP3 Final set of TARS items mapped against WP2 data themes.......................................289

Appendix 16 WP3 TARS GENERIC...........................293

Appendix 17 WP3 Tars Generic: Means, standard deviations and frequencies.....................................300

Appendix 18 WP3 TARS SPECIFIC (Site 1) .............304

Appendix 19 WP3 TARS SPECIFIC (Site 2) .............310

Appendix 20 Sample characteristics for TARS Specific (Site 1 and Site 2) .....................................317

Appendix 21 WP3 TARS Specific Site 1: TARS items, means, standard deviations and frequencies..........320

Appendix 22 WP3 TARS Specific Site 2: TARS items, means, standard deviations and frequencies..........323

Appendix 23 WP3 Comparison of correlations between TARS Specific samples .............................327



Appendix 24 WP3 TARS Specific Site 1: Cross tab analysis of agreement with TARS items by perception of level of routineisation.......................330

Appendix 25 WP3 TARS Specific Site 2: Significant differences in comparison of Nursing/health staff (NH) with call handlers (CH) on TARS items, specified as percentages of professional group......333

Appendix 26 WP3 TARS Specific Site 2: Comparison of perception of not/partly routine (NP) with completely routine (C) by TARS items (bracketed figures refer to % within perceived routinisation grouping) ...............................................................334

Appendix 27 WP4 Interview Schedule for Case Study 1 (Choose and Book)..............................................336

Appendix 28 Salient Features of Selected Case Studies ...................................................................338

Appendix 29 WP4 Print-out of the e-HIT. ...............342



Acknowledgements

"This presentation/report/article/press release presents independent research commissioned by the National Institute for Health Research (NIHR) Service Delivery and Organisation (SDO) Programme. The views expressed in this publication/presentation are those of the author(s) and not necessarily those of the NHS, the NIHR or the Department of Health. The NIHR SDO programme is funded by the Department of Health."

� We would like to acknowledge the contribution of Jo Burns, former researcher at Department of Primary Care and Population Health, University College London.



The Report

1 INTRODUCTION

1.1 Background

E-Health has become more prominent in health policy over the last decade as policymakers throughout the world have recognised the possibilities it offers to transform the way healthcare is delivered. It is also a way of addressing health care challenges relating to changing demographics and the increasing prevalence of long term conditions. The United Kingdom Department of Health (DoH) estimates that 17.5 million people may be living with a chronic condition (DoH, 2004) while the World health Organisation (WHO) has suggested that by 2020 long term conditions could be the leading cause of disability (WHO 2002). E-Health may be able to address such problems, as well as improving quality, by facilitating self care and error reduction. It may also increase access to care, for example, by reducing inequalities resulting from geographical location.

The NHS plan (DoH 2000) required the development and implementation of Information and Communication Technologies to provide patient-centred services. Since the publication 10 years ago of Information for Health (NHS Executive 1998), many policy documents have been published with the aim of promoting e-Health (NHS Executive 2001; US National Institute of Medicine, 2000; Wanless 2002;Wanless 2004; DoH 2001;DoH 2002a; DoH 2002b; EU Commission 2004; EU Commission 2005). Within the UK, Connecting for Health (NPfIT/CfH) has been prominent in this activity.

However, despite rapid technological development the professional, organizational and institutional terrain into which this technology has been introduced has not accepted change readily. A key problem has been that of integrating e-Health services into professionals’ patterns of work, so that e-Health has rarely become part of routine service delivery. Utilisation of e-Health by health care professionals is complex as such services can change the dynamics of professional roles and relationships and affect the organisation of clinical work. Professional resistance to new technologies is often cited as a major barrier to progress but our previous work in the field of tele-Health suggests that such a view is overly simplistic (May et al. 2001a). Understanding barriers to implementation of e-Health services is crucial, however, if we are to overcome the apparent inertia of the NHS when it comes to introducing new technologies.



1.2 Aims

In response to the commissioning brief for EH94 our study aims were to:

1. Explore a range of health professionals’ attitudes towards e-Health systems and to identify, describe and understand those factors that promote or inhibit the use of e-Health systems and other computerised tools across different health care professional groups and sectors

2. Identify promoters, inhibitors and factors which have little impact on e-Health implementation.

3. Develop a structured instrument to test the readiness of health professionals to use new e-Health systems and their potential for integration in particular clinical settings to be known as the Technology Adoption Readiness Scale (TARS)

4. Develop a model implementation toolkit which would draw on the results obtained from answering aim 2, to be known as the e-Health implementation toolkit (e-HIT).

5. Make practical recommendations regarding strategies that will facilitate and enhance the utilisation of e-Health systems by health care professionals across the UK.

1.3 Plan of investigation

To meet these aims we divided the project into four distinct but linked work-packages. Each was led by one of the Glasgow, Newcastle or London teams respectively, but all of those working in the project contributed to each work-package. Sharing the results of each work-package within the group helped us to better understand the processes of e-Health implementation.

Work-package 1 (WP1) had two components, an initial scoping exercise and a systematic review of the literature, both aimed at providing a “review a reviews”: essentially a summary of what was known about e-Health implementation. We also hoped to identify the gaps in this literature, so that we could make clear recommendations about the agenda for further research in this field.

Work-package 2 (WP2) examined health care professionals’ views about factors that affect implementation and integration of e-Health services, particularly their perspectives on barriers and facilitators to implementation. Participants were asked to comment on the findings from WP1 in order to see how professional’s perspectives resonated, or not, with the literature on barriers and facilitators. The objective of WP2 was to help us better understand the challenges involved in implementing and integrating e-Health services into routine health care delivery. That would enable us to make clear recommendations regarding the implementation and integration of e-Health services.

The purpose of work-package 3 (WP3) was to use information from the other work-packages to refine our robust normalisation process model (NPM). This would increase our understanding of professional and organisational capacity to implement and integrate e-Health services within routine health service delivery contexts. The main deliverable from WP3 was intended to be a structured instrument known as the Technology Adoption Readiness Scale (TARS).

Finally, the purpose of work-package 4 (WP4) was to identify, describe and understand promoters, inhibitors, and factors which had little effect, on the



implementation of e-Health systems. We planned to draw on these findings to develop an e-Health implementation toolkit (e-HIT) which could be a practical guide to those involved in implementing e-Health services.

Whereas much research has focused on barriers to implementing new technologies in specific clinical contexts, in this project we have taken a “whole systems approach”. We have used a theoretical model, the Normalisation Process Model, which attends closely to the processes through which innovations are played out and focuses on the “whole-system” in which they are located - consistent with the recommendations made by Greenhalgh (Greenhalgh et al. 2004) in their innovations literature review. Furthermore, while much research into the uptake of new technologies has tended to focus on the processes of technology diffusion and adoption, it is worth highlighting from the outset that this project has focused on workability and has explored the problem of normalisation, that is, the routine embedding of newly introduced technologies into everyday practice.

The NPM has been the conceptual model underpinning the work of this study (May 2006; May et al. 2007a; May et al 2007b) and has played a pivotal role within the study. The NPM consists of four constructs: interactional workability (IW); relational integration (RI); skill set workability (SW) and contextual integration (CI). Interactional workability refers to issues such as the ease of use of systems, while relational integration refers to issues of confidence and accountability. Skill set workability covers issues of training, workload and roles and responsibilities; while contextual integration refers to organisational issues such as resource allocation. A fuller description of each of the constructs within the model is included in Appendix 1. In this context, an objective of the study was to develop an evidence-based conceptual model of new technology adoption by NHS professionals that uses professional responses to e-Health technologies as a vehicle for analysis. Such a model will have added value if, like our other work, it applies across a range of technologies and contexts.

All work-packages in the study contributed to this objective, either by conceptual development (WP1); empirical investigation of specific operational contexts (WP2); development of a predictive instrument, the TARS (WP3); and pilot work for development of the e-HIT. The strong theoretical grounding of our research will maximise the transferability of our predictive instrument and implementation package and of our evaluation findings across e-Health systems and health care contexts. Importantly, while the NPM has been used to help us study the processes of implementation, the analysis and interpretation of findings from the individual work-packages has in itself contributed to the further refinement of the NPM (May & Finch 2009). The next section describes in further detail the theory development work undertaken during this project.

Defining e-Health

A challenge for those researching e-Health is to define the term, since the topic encompasses a broad array of electronic technologies related to giving and managing health care. We had no wish to repeat work previously commissioned by the SDO (Pagliari et al. 2005), and for the purposes of this project we adopted the definition of e-Health provided by Eng (Eng 2001):

‘e-Health is the use of emerging information and communications technology, especially the Internet, to improve or enable health and healthcare.’



However, to conduct our work-packages successfully we needed to have a clear way of defining and characterising the different types of e-Health. For example, within WP1 it was crucial that we have clear inclusion and exclusion criteria for our systematic review, and the first stage in this process was to develop clear definitions of the terms to be used. Furthermore, an understanding of how we were defining the different types of e-Health was essential to ensure consistency in terminology across the work-packages, thus ensuring that the research team approached participants in a uniform way and conceptualised the field of study in a clear and consistent manner.

The project team therefore, via an iterative process, devised a conceptual framework to guide data collection and analysis. It was not our intention to provide a new definition of e-Health, rather to be able to clearly define and categorise different types of e-Health services for the purpose of presenting a framework that was both broad in scope and tightly focussed.

We therefore divided e-Health into four domains, and the descriptions of these four domains were agreed via consultation with all co-applicants and researchers. The definitions of the domains were worked and re-worked periodically throughout the first quarter of the project in parallel with the initial scoping exercise as concrete examples of different types of service were identified and discussed by the research team and the precise way to categorise different types of e-Health services was refined. The intention was to develop categories that could be easily distinguished from one another, which were straightforward to describe to health professionals and which, encompassed those technologies most relevant in the contemporary UK National Health Service.

The four domains were:

1. Management systems. Including electronic medical record systems (for example the systems being implemented within NPfIT (Connecting for Health).

2. Communication Systems. Including synchronous and asynchronous information and communications systems used for clinical information sharing, ranging from e-mail through telemedicine and telecare systems.

3. Computerised decision support systems. Including computerised/automated systems used to support health professionals in working within clinical guidelines and care pathways or providing best evidence based care.

4. Information systems. Including web-based resources or e-Health portals for information retrieval

The definition of these domains is shown in Box 1.



Box 1

Management Systems

A computer based system for the acquisition, storage, transmission and display of patient administrative or health information (including images) from various sources that can assist with administrative or clinical activities. Management systems are therefore intermediaries between users and a body of stored data with specific applications (for professionals or patients).

Communication systems.

This refers to the use of telecommunications systems as intermediaries where users are separated in space and/or time. This communication can be between health professionals or between health professionals and a patient or carer(s). This type of communication can be synchronous or asynchronous and involves the targeted exchange of information between specific individuals or individuals with specific roles for diagnostic, management, advice, educational or support purposes.

Computerised decision support systems.

A computerised decision support system acts as an intermediary between users (and future actions). It refers to a computer system that is interactive and rule based and uses two or more items of patient data to generate case or encounter-specific advice that can aid clinical decision making. Such systems would be automated, generally operate in real time and involve decision support from an artificial intelligence (e.g. computer system or software programme) rather than an individual (either in person or remotely located).

Information resources.

This refers to the use of internet technology by the public, health workers and others to access health and lifestyle information, services and support, where information resources are intermediaries between users and a body of stored data for general information (i.e. information is generic and not person specific) that is available for use by individuals or groups.

It is acknowledged that some e-Health services may fall within one or more categories but this was not viewed as problematic as the key requirement of this categorisation was that it was clear, concise, easy to understand, and that the research team were able to use it to categorise e-Health services in a standard fashion.

Section 2 describes the theory development undertaken during the project, while the methods and findings from each work-package are presented in Sections 3-6.



2 Developing the Conceptual Model

The call for EH94 required applicants to locate their study designs and aims in an explicit theoretical framework. Although there were several theories to which our work could be related, at the outset, we took the view that this study was one of a number of concurrent opportunities to develop new theory that answered in a practical way some of the core questions that have been consistently raised by policy-makers and clinicians who seek to operationalize ways of thinking about, organizing, and enacting, new techniques and technologies for health care delivery. The development of the Normalization Process Model (NPM) has been a component in a series of sociological studies—mainly led by May— into the operationalization of professional knowledge and practice in healthcare. In particular these studies have been concerned with the implementation processes that lead to particular outcomes for complex interventions, rather than focusing on evaluating the outcomes of these processes. Of course, other theoretical perspectives explore elements of the same questions, and we deal with these elsewhere in this report. The purpose of this section is to describe the development of NPM and show how it both contributed to this study and was developed and refined within it.

Work leading to the development of the theory began in the late 1990s when May and Mair collaborated with others on two important ethnographic studies of the development, implementation and evaluation of telemedicine systems. The first of these, funded by the NHSE between 1998-2000 focused on design and implementation problems, and led to analyses of problems of workability and integration in practice (May and Ellis, 2001b; May et al., 2001a; Mort et al., 2003). The second, funded by the DoH Policy Research Programme, between 2000-2002, used the evaluation of telemedicine systems as a vehicle to explore the relationship between rigorous practices of knowledge production (randomised controlled trials) and the organizational contexts in which these are located. An important outcome of these studies was a paper offering a framework for the structured analysis of the relationship between evaluation processes and communities of practice (May et al., 2003a).

This analysis located ‘normalization’ as an endpoint of implementation processes, defining it in terms of a relatively stable set of practices that led to these health technologies becoming routinely operationalized in everyday work (embedding), and sustained in practice (integration). One result of this work was a further collaboration, this time with the Virtual Outreach Trial team led by Paul Wallace at UCL during 2002. In this work (May et al., 2003a, we sought to synthesise the results of a large corpus of qualitative data—using constant comparative analysis (Strauss, 1987)—to define the conditions that would lead to the normalization of telemedicine services in practice. We now had two conceptual working models of normalization processes, one focusing on how evaluation studies act as a vehicle that makes stable clinical practice possible, and the other focusing on the policy and organizational processes. These formed the theoretical basis of our application for EH94.



Although these papers have been regularly and supportively cited, they formed only a conceptual model. This mapped a set of processes and relations but said little about what these meant in practice, or how they worked. Meanwhile May was working with colleagues on a series of papers that synthesized other areas of research. These added conceptual models of interpersonal relations in the clinical encounter (May et al., 2004); the production and reproduction of new patterns of work organization and allocation (May, 2005); and the development of robust socially patterned knowledge in healthcare systems (May et al., 2006). Each of these papers shared a common method: synthesising prior qualitative studies, and reanalysing their data. They also shared a common product: a set of analytic propositions or theoretical assertions which all focused on the organization and practice of professional work, across a range of contexts.

The results of these synthetic analyses were drawn together in work leading to an ESRC Research Fellowship for May (2004-2008). An explicit objective of this fellowship was the development of a middle range theoretical model that would assist in the process evaluation of large Health Technology Assessment trials. The development of this explicitly theoretical model—the Normalization Process Model (May, 2006)—coincided with the early phases of EH94, but at this stage it was not clear that it could be applied to this or other studies. Theory development nevertheless continued, with a review and critique of theory in the area (May, 2007), and major collaborative papers (May et al., 2007a; May et al., 2007b) that set out the model as a coherent and robust set of tools that fulfilled the three primary requirements of a theory (May et al., 2007a):

1. Accurate description. A theory must provide a taxonomy or set of definitions that enable the identification, differentiation, and codification of the qualities and properties of cases and classes of phenomena. The analytic assertions and propositions developed in earlier synthetic analyses of qualitative data perform this function.

2. Systematic explanation. A theory must provide an explanation of the form and significance of the causal and relational mechanisms at work in cases or classes of the phenomena defined by the theory, and should propose their relation to other phenomena.

3. Knowledge claims. A theory must lead to knowledge claims. These may take the form of abstract explanations, analytic propositions, or experimental hypotheses. They may also map relations with other phenomena that are believed to possess similar qualities and properties.

Further, it fulfils a fourth, but not mandatory, component of a theory in that it proposes a means of testing its knowledge claims:

4. Investigation. A theory must be testable. Such tests may be abstract (i.e. formal logical representations, simulations, or thought experiments); or concrete (empirical investigations).

As described in Section 1.3 the NPM consists of 4 constructs: interactional workability, relational integration, skill set workability and contextual integration: Appendix 1 provides full details.



This model was empirically ‘tested’ by the co-applicants in a series of data clinics where we jointly interpreted and analysed qualitative data collected in this study. Parallel processes of collaborative data analysis using the NPM also took place in other studies. This made it clear that the NPM was robust and could provide an analytic advantage in interpreting data collected in this study and in developing a readiness assessment instrument and implementation toolkit. The NPM was therefore used to inform development of the work-packages and also to analyse data within these work-packages.

The NPM in its published form at the end of 2006 (May, 2006), was sufficient as a tool to analyse specific processes—the material practices through which the operationalization of e-Healthcare systems is accomplished—but it was insufficient to account for other important processes. This was demonstrated by applying the model empirically to experimental (Wilkes, 2007), qualitative (Gask et al., 2008; Mair et al., 2008), and review (Elwyn et al., 2008) data. This showed that it was useful in explaining factors that promoted and inhibited collective action in operationalizing practices. For example, the interactional workability construct addressed the ways in which e-Health services helped or hindered professionals in performing various aspects of their work; while relational integration addressed professionals’ confidence in the safety or security of new systems; skill set workability examined how e-Health services affected workload and training requirements; and contextual integration looked at issues of resource allocation, infrastructure and policy.

It did not, however, explain how those interventions were formed in ways that were sustained, how actors were enrolled into them, or how they were appraised. May and Finch therefore collaborated to extend the model to a middle range theory by exploring new domains of coherence (the work of making a complex intervention hold together and cohere to its context), cognitive participation (the work of engaging and legitimising a complex intervention) and reflexive monitoring (the work of understanding and evaluating a complex intervention in practice). We describe in detail elsewhere the methods by which the theory was built (May and Finch, 2009; May and Finch, forthcoming).

The development of a set of explanatory ideas around normalization has shifted from an initial set of empirical generalizations presented as synthetic propositions or assertions (May et al., 2003b), to a robust conceptual model that presents generalizable propositions (May, 2006), and finally to a middle range theory that offers a set of mechanism-based explanations for processes of implementation, embedding and integration (May and Finch, 2009). Our aim in doing this work has been to develop a robust explanatory model that can underpin structured, prospective, and policy relevant studies, and which are genuinely open to interdisciplinary inquiry. Unusually for a theory building process, this has been a collaborative one, in which co-investigators in this study and others have made important contributions to the development of explanation.

Due to delays encountered within WP3 it became possible to incorporate elements of the extended NPM in this work-package, while reanalysis of collected data permitted exploration of its value in work-packages 1 and 2. The added value of this is discussed in Section 7.



This section has shown that the project has benefited greatly from being part of a wider programme of work involving the co-applicants, which has both contributed to further theory development of the NPM and increased the breadth of analysis possible within this project.



3 Work Package 1 (WP1): Summary and synthesis of literature on the implementation of e-Health

3.1 Background

How best to implement new systems of practice – new technologies and their associated ways of working – is a core problem for health service managers and clinicians. The large literature on the topic reflects the challenges of implementing, integrating and embedding e-Health systems into routine health care delivery (see Section 1.1). This literature is often inaccessible – either because it ranges across disciplines in a way that makes appropriate papers hard to find without specific subject expertise, or because it focuses on systems in a highly specific way – so that people charged with implementing e-Health systems in the NHS find it difficult to locate the appropriate evidence base. If they do, it is often difficult to determine its relevance to current circumstances. In order to identify factors which promote and inhibit e-Health implementation processes, and therefore to address one of our main research objectives, we conducted a review of reviews. This allows us to synthesise and highlight the key messages from this literature in an accessible way.

A secondary review is preferable to further primary systematic reviews as it allows us to provide an overview of this field of research and to identify strengths and weaknesses in the literature. A theory-led review also demonstrates gaps in the published literature and produces clear guidance for the direction of future research. The aims of WP1 were therefore to:

� summarise the e-Health implementation literature;

� identify and understand barriers and facilitators to e-Health implementation, using both thematic and framework analyses of the review data;

� produce a taxonomy of problems that could serve as the focus for discussion by expert task groups and key stakeholders within WP2;

� inform the development of the predictive instrument and implementation enhancement packages within WPs 3 and 4.

To achieve our study aims rigorously and efficiently we summarised and synthesised the literature in two stages.

(a) Scoping Review: We used free text search terms to identify papers, rather than more formal searching techniques. The objective of this stage was to identify a broad range of papers relating to the implementation of e-Health, and so to understand how scholars have generally framed implementation problems. This would inform the development of a thematically derived taxonomy of barriers and facilitators. This taxonomy



then informed the interviews in WP2, and the development of the structured instrument, the Technology Adoption Readiness Scale (TARS) in WP 3. Many of these papers could not have met the rigorous methodological benchmarking of a true systematic review, but nevertheless offered important insights.

(b) Formal systematic review of reviews: We focused on structured systematic and narrative reviews of studies of implementation of e-Health interventions. We conducted a rigorous systematic review of this material, but in a novel move then analysed these papers using (i) the thematic framework developed in the scoping review, and (ii) the NPM. This provides a unique and extremely robust process of theoretical triangulation in a systematic review that allows us to make clear statements about the gaps in the literature and recommendations for future research.

For both reviews the definition of e-Health was that outlined in Section 1.3 and the literature was also categorised into the four e-Health domains (management, communication, computerised decision support and information systems) described in Section 1.3, Box 1.

3.2 Stage 1: Scoping Review – Aims/Objectives

The aim of the scoping exercise was to identify the main barriers and facilitators to e-Health implementation that could be used to inform other work-packages within the project.

Objectives

� To undertake a broad examination of the e-Health implementation literature, including papers that would not meet the strict scientific criteria required of a systematic review.

� To undertake a thematic analysis of the e-Health implementation literature.

� To produce a list of barriers and facilitators to e-Health implementation, based on the identified literature which could be used to inform the work to be undertaken within other project work-packages.

3.3 Stage 1 – Methods

Scoping Exercise Search Strategy

The study team developed a broad search strategy, designed to include all e-Health domains, and entered this into Medline for 1990-2006. This period covers key developments in e-Health, as well as pertinent developments in health and technology (e.g. the spread of the World Wide Web after 1995), and it is noteworthy that the term “e-Health” did not come into widespread use until after 2000 (Pagliari et al 2005). As we were undertaking this scoping exercise to provide, amongst other things, a framework of



knowledge to discuss with health professionals, it was most appropriate that we searched a database orientated to this group. Medline, more than other databases, is aimed at health professionals as much as it is at academics, so papers identified in such a search would be most likely to reflect an e-Health knowledge base with which professionals are already, to some extent, acquainted. Also, a MedLine review was deemed sufficient to permit a rapid overview of the main issues to inform other WPs, because of its broad scope within a medical focus, its ability to filter reviews from original research papers, and its frequent use in other systematic reviews (de Lusignan 2005, Hanson et al 2001, Jaatinen et al 2002).

The free text search terms used were:

Barriers OR facilitators OR implementation OR adoption OR translation OR

stabilization OR normalisation OR readiness AND:

e-Health OR "electronic medical records" OR "computerised decision support" OR "health informatics" OR telecare OR tele-Healthcare OR tele-Health OR telemedicine OR teleinformatics OR telecommunications health (OR + medicine

OR + nursing) OR tele* OR + computers + health (OR + medicine OR + nursing) OR software + health (OR medicine OR nursing) OR information + communication + technologies OR "the internet" + health OR "electronic mail" OR e-mail/email OR "chat room" OR "bulletin board" OR CD-ROM OR multimedia OR hypermedia OR "virtual reality" OR "audiovisual aids" OR "cellular phone" OR mobile tele*

We also restricted our search to English language articles involving human subjects.

Inclusion criteria and exclusion criteria

Papers were included if they contained a quantitative or qualitative secondary synthesis of evidence, and if their conclusions were empirically substantiated. Ideally, the reviews would be systematic, clearly explaining what they reviewed, why, and how the material reviewed was obtained. Reviews of particular systems and narrative reviews were also included if they contained a secondary synthesis of evidence. Our inclusion rules for secondary synthesis were much broader than those of a formal systematic review, since our aim was to gain wide insight into barriers and facilitators. We therefore included papers that did not employ systematic search strategies so long as the commentary drew upon syntheses of: (a) the literature or (b) case studies, pilots, services, or trials to substantiate the narrative.

Studies were excluded if they contained no secondary synthesis of evidence, little or no reference to empirical work, were driven by speculative argument or measured outcomes rather than reviewing real data. Also, as this study was focussed towards the experiences of professionals, (as mandated by the commissioning brief) papers that exclusively focussed on patient experience were excluded.

The project researcher (GA) screened all identified abstracts for potential inclusion. Full papers were obtained for those identified as potentially



relevant and assessed against the inclusion and exclusion criteria. The PI (FM) reviewed marginal papers.

A thematic analysis of reviews

Data was abstracted from eligible studies using a specifically designed data extraction tool previously piloted on 10 papers. Data extraction was in two stages: (1) details of the paper, setting, methods, aims and conclusions were recorded; and (2) each paper was assessed for content in relation to pre-defined themes based on the research brief and a preliminary review of the literature. The themes arose from the aims of the review - to identify barriers and facilitators to the implementation of e-Health systems - and also included the impact of e-Health technologies on the relationship between patients and professionals. and the efficiency and ease of use of the technology.

The development of the thematic review was guided by the methodological literature, which indicated that thematic analysis should involve identifying prominent or recurrent themes, summarised under each thematic heading (Dixon-Woods et al, 2004, 2005). The content of these headings was used to identify and define overarching thematic categories, leading to a greater understanding of the topic. Thus, the relevant literature was examined and identified under each thematic focus, identifying both unifying concepts within each theme and discrepant issues. These were used to construct a narrative analysis for each theme. These were then reviewed across each theme, again to identify possible unifying concepts, as well as discrepant issues.

The analysis was conducted primarily by GA. Verification of the thematic analysis was obtained by data sharing and discussion with three other members of the team (FM, TF, CM) and by sharing the findings with members of the project team (KO, EM, PW, JB, FS) and with an Expert Advisory Group of international authorities in the field.

3.4 Stage 1 – Results

Identification of the literature

We identified 719 potentially useful citations and 62 of these were included in the thematic review. Of the 657 excluded, the main reason was that they were not focused on e-Health systems. Others were not secondary syntheses of primary evidence (they did not "review" original studies). More than half (34/62) of the included papers originated in the US; 8 were from the UK; 6 were from Canada. The rest originated from Europe, Hong Kong, Australia or New Zealand. Appendix 2 provides the full list of papers included in the scoping exercise with accompanying basic details. Of the 62 papers, 9 were generic; that is, they covered more than one e-Health domain; 8 were concerned with management systems; 29 with communication systems; 7 with computerised decision support systems and 9 with information systems. The quality and detail of the literature was mixed. The methods and focus of the reviews also varied: 12 were classifiable as systematic, 23 were narrative literature reviews while the remaining 27 were essentially reviews of systems and national policies with reference to relevant literature. Some papers were chiefly concerned with software design and specific systems, with only a brief mention of barriers and facilitators to implementation (Hussein et al 2004, Carrino et al 1998),



while others focused on these exclusively (Bond 2006, Weinstein et al 2001). However, the literature at all levels of detail presented similar emergent themes.

Summary of the main factors

The scoping review identified seven main factors that affected the implementation of e-Health systems. These were:

1. Conditions prior to implementation.

2. Cost.

3. The need for and impact of validation and evaluation.

4. Professional attitudes.

5. Ease of use of the system.

6. Security, confidentiality and standards.

7. Education and training.

There was some over-lap between these factors, and they are not presented here in a hierarchy of importance or in frequency of occurrence. Rather, they may be regarded as the key distinguishable barriers and facilitators emerging from the thematic analysis of the literature, though the extent of this distinction varied, as is demonstrated below.

Conditions prior to implementation

The literature showed that conditions prior to implementation were regarded as integral to the selection of a site for an e-Health intervention, in all e-Health domains. These conditions had two dimensions: those within the organisation itself (be it a health trust or a GP surgery); and broad societal conditions (e.g. the level of political support for the implementation of an e-Health system). For example, May et al (2003c) asserted the importance of a positive link between an organisation and a policy sponsor in the successful implementation of telemedicine.

An organisation’s willingness and readiness to adopt and operate an e-Health system along, with that of the wider community (Jennett et al 2004) were important facilitators to implementation. Many reviewers recommended adequate preparation for an intervention. This may be in terms of strategic planning (Anderson et al 1997, Jerant 1999), establishing a vision as to what is to be achieved, and preparatory needs analysis of the site(s) in question (Cook and Dupras 2004). Yellowlees (1997) recommended that “telemedicine applications and sites should be selected pragmatically, rather than philosophically”. In short, good planning and prior knowledge of the site in question were identified as important facilitators to e-Health implementation.

Other sources suggested that the initial planning of system implementation may depend on a positive relationship between agencies. Established inter-agency co-operation was a precondition of successful system implementation; however, it was also integral to the process and sustainability of such success and is hence more than merely a prior condition. However, a system could not be implemented in isolation (Sheng et al 1997) and required input from providers, financers, and all agencies concerned (Cook and Dupras 2004). Recommendations from experts in the field may be important (Jerant 1999). There might also be a need for sponsorship (Anderson and Aydin 1997) and good marketing of the system, depending on the financial structure of the health service concerned.



However, successful implementation of any e-Health system seemed reliant on a broader acceptance of principle. The implementation of a system from one domain of e-Health may be more likely to succeed if those from other domains were already established, or were implemented simultaneously. As Jennett et al (2004) concluded in relation to tele-Health:

“Tele-Health applications are more likely to be successful, in terms of cost and sustainability, if they are considered to be part of the larger domain of e-Health. Programs implemented and evaluated as independent initiatives are at greater risk of failure.”

While prior conditions in an organisation clearly seem to affect the likelihood of e-Health success, such conditions could not be separated from the broader socio-political climate. There may be significant differences in professional cultures between societies (Uhlenhopp et al. 1998) and the infrastructure of health services and indeed telecommunications infrastructure varies significantly between places (Anonymous 1999). If there were shortcomings in the funding and infrastructure of a health service, an intervention was more likely to fail (Rudowski 2003), whereas if the health service is well funded and supported, new interventions were easier to implement (Elford 1997, Guler and Ubeyli 2002, Weinstein et al. 2001).

Cost

The cost of a system as a factor in implementation was mentioned frequently in the literature across all four domains (Mairinger 2000, Wallace et al.1998, Elford 1997, Falas et al. 2003, Whitten and Love 2005, Al –Qirim 2005, Goldberg and Dwyer 1995), though it was not described in any depth. It was suggested that the more effective the technology, the more costly it may be in financial terms (Angood 2001). In terms of telecommunications, there was also the cost of regular usage to consider as well as that of initial implementation, since network tariffs may apply (Mun et al 1995); in terms of other systems there would also be costs associated with procurement and maintenance.

The need for and impact of validation and evaluation

Validation and evaluation of e-Health systems were important factors in their widespread implementation. Again, this was the case with all four domains of e-Health, albeit less so with information systems, perhaps because the world wide web is a completely normalized tool.

Without strong data demonstrating that a system works, improves standards of care, can be used efficiently and easily, and is cost-effective to implement, it is unlikely to win the confidence of policy makers and users. Indeed, lack of validation and evaluation was frequently presented as a barrier to system implementation (Lehmann 2004, Anderson and Aydin 1997, Yellowlees 1997, Jadad 2002, Swinglehurst 2005), while continued monitoring and evaluation was a facilitator (Elford 1997, Cook and Dupras 2004, Styra 2004, Jerant 1999).

The need for validation and research into e-Health systems was inextricably bound with the effectiveness of the technology and its impact on practice. Vreeman et al (2006) claimed that:



‘Further research is needed to better characterize the effect EHR (Electronic Health Records) can have on the process and outcomes of care and to elucidate what “active ingredients” are necessary to achieve these benefits.’

Related to the above was the issue of whether validation and evaluation already available to implementers and users served as barriers or facilitators. This would depend heavily on the content of the evaluation – a positive evaluation would support the enthusiasm of those seeking to implement it elsewhere. Yellowlees (2005) explained that when telemedicine system fails in a site:

‘…the whole practice of telemedicine itself attracts an unenviable reputation, and it may be years before another attempt is made to provide telemedicine services in a region where telemedicine has failed once already.’

There was also an aspect to evaluation and validation that was cyclical: good results encouraged further implementation, implementation led to further research. Because e-Health systems, particularly management systems, could retain clinical data in an easily accessible way, with their implementation came the ability to validate them quickly and efficiently (Lehmann 2004, Anderson 2000).

Professional attitudes

Accounts of professional attitudes to the implementation and integration of e-Health systems suggested a broad array of barriers and facilitators to success. These attitudes related to perceptions of the changing relationship between professionals and patients (Wallace et al. 1998, Mandl et al 1998) and perceptions of the effectiveness of the technology. There seemed to be a general anxiety towards the use of technology (Childs et al 2005), or clinicians felt that their roles were undermined by it (Lehmann 2004), with the introduction of new systems implying shortcomings in their own performance (Whitten and Love 2005). The latter related to how the e-Health system affected the allocation of tasks and how an organisational structure changed once technology was adopted. It could also relate to unease regarding patient safety, particularly in terms of how professionals could manage a critical situation at a distance.

Anderson (2000) summarised the impact implementation may have on professional attitudes:

‘Computer-based information systems change traditional practice patterns. They structure how clinical data are recorded, organized and displayed. Their implementation frequently affects professional relations between individual caregivers and professional groups within the organization. The responses of physicians and their health care professionals are shaped by their perceptions of how the system affects their work relations and accomplishment of their professional goal.’

Aside from concerns about particular aspects of the systems, there was also the issue of “ownership” in terms of professional willingness to use, manage, and be associated with them (Jennett et al 1996, Elford 1997). A related facilitator here was leadership (Anderson and Aydin). If authority and leadership were clearly designated (someone is willing to take responsibility), it was more likely that professionals would broadly accept an e-Health system. The fundamental precondition to confidence in the system was acceptance in the first instance. Maglogiannis (2004) claimed that acceptance by doctors is the most important factor in the successful implementation of telemedicine systems.



Ease of use of the system

The ease of use of an e-Health system, the interaction between the technology and its users, was self-evidently a factor in successful implementation (Goldberg and Dwyer 1995). The system needed to be appropriate to the work, not imposed for its own sake (Lehmann 2004, Mairinger 2000, Cook et al 2004, Mandl et al 1998, Sable 2001), be compatible with existing systems (Hussein et al 2004), and reliable (Stanberry 2000).

A system’s ease of use was closely related to its original design. Angood (2001) explained how the design of a system must reflect clinical need:

‘Over the past two decades, medical informatics has provided valuable lessons by showing that attempts to use information technology will often fail when the underlying motivation is the application of technology for its own sake rather than satisfying the need for providing solutions to clinical problems or an improved efficiency of patient care.’

E-Health systems were implemented because they were deemed to be impressive technologically rather than beneficial to clinicians and patients. Hence, a well-designed system was not necessarily a clinically effective one. Good design in itself was not a sufficient facilitator of successful implementation; rather, that a system’s design was compatible with existing work patterns was the integral factor. Safran and Goldberg (2000) raised the following issue with some electronic medical records systems:

‘Some EPRs (electronic patient records) have limited remote access capabilities or assume that no two clinicians would need to look at the same patient information at the same time.’

This was an issue of the system’s workability. If only one clinician could

view a record at any given time, it became more difficult to share

information instantaneously. Related to this were factors involving time and

reimbursement. The implementation of an e-Health system would not

necessarily reduce clinicians’ workload (Jaatinen et al 2002). If workload

increased clinicians would not always be reimbursed for the extra time taken

(Mairinger 2000, Rudowski 2003). One study (Shiffman et al 1999) also

explained that using the system was regarded as a tedious and unrewarding

process by some professionals. However, if an e-Health system replaces

face-to-face consultation, clinicians travel time would reduce (Whitten and

Love 2005).

Related to the general ease of use of a system was its ability to order and manage information. If the technology disrupted established working practices, and increased the time and effort required to record, communicate, and find information (and indeed arrive at a clinical decision), successful implementation was unlikely (De Lusignan 2005). A management or information system could contain insufficient, misleading, or too much information (Maulden 2003, De Lusignan 2005, Tierney 2001). However, if a system was flexible and adaptable to existing working practices, and efficient in ordering existing information in a way that was easily accessible, it was more likely to succeed (Yellowlees 2005).

Security, confidentiality, and standards

Security, confidentiality, legality and standards were almost exclusively issues of accountability and safety. For successful implementation, a system needed to be not only workable, but also secure. It needed to be safe for



both patients and professionals; for patients, in terms of the security of their personal data, and for professionals in terms of not compromising their standards of practice. Falas et al (2003), in a review of decision support systems, stated:

‘Ethical issues regarding the privacy of the patient’s data when transmitted over a telecommunications network, the risks involved when the patient is treated remotely, and the patient’s readiness to accept telecare devices in the home are very important.’

The implications of e-Health systems on patient privacy and confidentiality were cited frequently (Link et al 2001, Mairinger 2000, Jennet et al 1996). The extent to which these issues affect the success of an e-Health system depended on the security of, and access to, the technology. This was the case in all four domains of e-Health where patient data was concerned. Communicating patient data in any way involves some risk. However, when the means of communication were electronic, the risks of compromising a patient’s privacy, while no more or less severe, could be different to those of verbal and written communication.

There were several aspects of e-Health systems that could induce anxiety in health professionals. One related to the expectations of the patient, and to patient and professional autonomy. Lehmann (2004) gave an example relating to decision support technologies used in diabetes care:

‘…standalone use of a program by patients- without reference to a health carer- is the sort of thing that gives health-care professionals cause for concern, because of issues over patient safety.’

The perceived risk was that should a professional recommend a patient to use a computerised decision support programme, and the programme made an error which ultimately compromised the safety and well-being of the patient, it was the professional who remained accountable. In short, Lehman’s implication was that programmes that appear to increase patient autonomy could potentially decrease that of the professional, leading to anxiety over liability.

Since technology in health could detract from professional control in the care of the patient and so increase anxieties over liability, clear standards for use were essential. Indeed, the need for clear standards of practice regarding the use of all types of e-Health system was mentioned frequently (Jennet at al 2004, Rudowski 2003, Hussein et al 2004, Grams and Moyer 1997, Anderson 2000, Tierney 2001). The implication was that the use of such systems should be engineered to be within existing practice standards. However, because these technologies were relatively new, the standards and guidelines required to manage them had the potential to evolve through trial and error. Standardisation was hence a double-edged sword: while it provided clarity and safety to the users, it could stifle the technology’s potential. Hence, in a climate of defensive practice, if a programme was compatible with existing practice standards, its implementation was more likely to be successful.

Education and training

It was perhaps not surprising that education and training featured frequently in the literature (Uhlenhopp et al 1998, Carrino et al 1998). The success or failure of a new system depends on the capabilities of those using it and new technologies had to be learnt in the first instance (Pellegrino and Kobb 2005). Lack of skills and knowledge was frequently cited as a barrier to success (Childs et al 2005). Education could be as



simple as the communication of information between professionals and organisations for the purpose of influencing system implementation and usage (Mairinger 2000) to more formalised practices, such as officially sanctioned system-specific training courses (Anderson and Aydin 1997). Weinstein et al (2001) explained how education could help practitioners not only learn the system, but use it to its optimal effect:

‘Educating pathologists about certain limitations of telepathology will increase their awareness of the small but manageable list of problem diagnoses and provide them with strategies to circumvent such problems.’

Education was hence not simply about what a system can do, but also what it cannot. This in turn emphasised to the professionals what specific skills were required of them; the limitations of a programme would highlight their own capabilities as professionals. Education and training were hence not restricted merely to the learning of IT skills; rather, they could instigate improvements in clinical practice itself related to the use of e-Health systems.

3.4.1 Limitations of scoping exercise

The scoping exercise was not intended to be a comprehensive systematic review, and is thus likely to have missed some relevant publications. Inclusion decisions, and data extractions were conducted by only one reviewer, to ensure completion of the review in time to inform WPs 2, 3 and 4: this is likely to have produced some inconsistencies and inaccuracies. However, we are confident that the scoping exercise was sufficiently robust to meet its aims and objectives and in any case the rigorous systematic review described in Sections 3.5 – 3.7 provided the opportunity to examine the robustness of these initial findings.

3.4.2 Scoping exercise conclusions

The literature reviewed here presented an array of barriers and facilitators to the implementation of electronic health systems. Though there were many different types of system and different scenarios recorded, certain commonalities remained. Whether the factors identified here mirrored the real-life experience of those working on the ground remained to be seen. However, in the thematic analyse of current literature identified within this scoping exercise, it was apparent that, in order to increase the likelihood of an implementation being successful, certain dynamics needed to be considered. Good conditions prior to implementation, manageable cost, awareness of the most appropriate systems, professional willingness to adapt, an easy to use and secure system and adequate education and training were all significant factors in both the initial implementation and subsequent sustainability of an effective e-Health intervention.

A common factor for all factors identified in the scoping review was the relationship between the e-Health systems themselves and the organizational contexts in which they were applied. Hence, the factors identified as either impediments or contributors to successful implementation hint towards a holistic approach. E-Health systems, if adopted by an array of collaborative agencies, facilitated both the successful implementation of the system and the efficacy of inter-agency co-operation. The practice of one agency must be receptive to that of another. Hence, the overall receptiveness of an organisation to the implementation of an e-Health system may be determined by an array of factors, including those beyond the context of the inner workings of that organisation itself.



3.5 Stage 2 – Aims/Objectives

Stage 2 consisted of a formal systematic review of reviews of implementation of e-Health interventions into health care systems conducted by the co-applicants.

The aim of this review of systematic reviews was to identify and synthesise available reviews of implementation of e-Health interventions into health care systems.

Objectives

� To identify published reviews of e-Health implementation

� To synthesise the findings of identified reviews: 1) via an analysis of the papers which using the key themes identified in stage 1 as a coding framework; and 2) using Normalisation Process Model (NPM) as a direct coding framework.

� To compare and contrast the results of these two methods of data analysis.

� To summarise the available evidence, highlighting the key findings, with a view to both informing current and future e-Health implementation programmes and identifying outstanding research questions.

3.6 Methods

Standard, Cochrane approved systematic review methodology was used.

3.6.1 Criteria for considering studies for this review

This review focused on reviews of e-Health implementation. For the purposes of this study we defined these three components as follows:

Definition of a review

‘A review paper provides an analytic account of the research literature related to a specific topic or closely related set of topics. It is intended to contribute to knowledge by answering a research question.’

Inclusions

1. Systematic review: where relevant literature has been identified by means of structured search of bibliographic and other databases; where transparent methodological criteria are used to exclude papers that do not meet an explicit methodological benchmark, and which presents rigorous conclusions about outcomes.

2. Narrative review: where relevant literature has been purposively sampled from a field of research; where theoretical or topical criteria



are used to include papers on the grounds of type, relevance, and perceived significance; with the aim of summarising, discussing, and critiquing conclusions.

3. Qualitative metasyntheses or meta-ethnographies, where relevant literature has been identified by means of a structured search of bibliographic and other databases, where transparent methods had been used to draw together theoretical products, with the aim of elaborating and extending theory.

Exclusions

1. Secondary analyses (including qualitative metasyntheses or meta-ethnographies) of existing data-sets for the purposes of presenting cumulative outcomes from personal research programmes.

2. Secondary analyses (including qualitative metasyntheses or meta-ethnographies) of existing data-sets for the purposes of presenting integrative outcomes from different research programmes.

3. Discussions of literature included in contributions to theory-building or critique.

4. Summaries of literature for the purposes of information or commentary.

5. Editorial discussions that argue the case for a field of research or a course of action.

Where the abstract stated it was a review, but there was no supporting evidence in the main paper, such as details of databases searched or criteria for selection of papers (either on methodological or theoretical grounds), the paper was excluded.

Definition of e-Health

We used the definition of e-Health outlined in the introduction (Section 1.3):

‘e-Health is the use of emerging information and communications technology, especially the Internet, to improve or enable health and healthcare’

We classified e-Health interventions into 4 domains: management systems; communication systems; computerised decision support systems; and information resources as described in Box 1 of Section 1.3 and as used within the initial scoping exercise.

Definition of implementation

We used the definition of implementation provided by Linton (2002):

‘Implementation involves all activities that occur between making an adoption commitment and the time that an innovation either becomes part of the organizational routine, ceases to be new, or is abandoned (...) [and the] behavior of organizational members over time evolves from avoidance or non-use, through unenthusiastic or compliant use, to skilled or consistent use. (p 65)’

3.6.2 Search strategy for identification of studies

We searched the following electronic bibliographic databases:

� MEDLINE

� EMBASE



� CINAHL

� PSYCINFO

� The Cochrane Library, which included Cochrane Database of Systematic reviews, Cochrane Central Register of Controlled Trials, DARE, NHSEED (NHS Economic Evaluation Database), Health Technology Assessment Database.

The search was supplemented with papers known to any of the co-authors, and with papers identified in the earlier scoping review, undertaken by our researcher (GA). Our definitive search used thesaurus terms, while the scoping review had used similar search terms, but as text words. By combining these two modes of searching we aimed to identify the largest possible number of papers.

The search strategy for MEDLINE used the three concepts: e-Health; implementation; and review.

Thesaurus terms which referred to e-Health interventions were:

� Medical-Informatics-Applications

� Management-Information-Systems

� Decision-Making-Computer-Assisted

� Diagnosis-Computer-Assisted

� Therapy-Computer-Assisted

� Medical-Records-Systems-Computerized

� Medical-Order-Entry-Systems

� Electronic-Mail

� Videoconferencing

� Telemedicine

� Computer-Communication-Networks

� Internet

Where appropriate, thesaurus terms were exploded to include all terms below the searched term in the thesaurus tree. The lowest term was always exploded.

There are no thesaurus terms for implementation, so this concept was searched for by looking for these text words in title, keywords or abstract:

� Routin*

� Normali?*

� Integrat*

� Facilitate*

� Barrier*

� Implement*

� Adopt*

These two concepts were combined, and then the search was limited by publication type = review or meta-analysis.

Initial searching of the MEDLINE database was done setting date limits from 1990 – 2007, and no limitation of language. None of the non-English language citations were relevant, and there were no relevant publications published prior to 1995. Hence the MEDLINE search was re-run with date limits from 1.1.1995 – 31.12.2007, limited to publications in English, and these limits were used for searching all other databases.



The search was translated for the other databases, using each database’s thesaurus terms.

3.6.3 Identification of papers for inclusion

All citations were downloaded into a database in Reference Manager 11, and screened by two independent reviewers. EM screened all citations; CM and TF double screened 50% each. If either reviewer could not confidently exclude the paper based on the abstract or citation, the full paper was obtained. Papers that could not be definitely excluded at this point were obtained for further scrutiny. All papers obtained were double screened. CO’D screened all papers; EM, CM, TF, and FM double screened 25% each. If there was a disagreement about whether a paper should be included or excluded, it was read by all reviewers, and agreement reached through discussion (see Appendix 3 for summary of exclusions and inclusions at this stage).

Data extraction, analysis and synthesis

The aim of the analysis and synthesis was to determine where there is adequate current knowledge of particular processes, and where there are gaps in the current research literature, with a view to informing future research commissioning.

Data extraction consisted of two stages. First we used a simple but standardised data extraction instrument to categorise papers on the basis of: country of origin; e-Health domain; publisher and date of publication; review aims and methods; databases searched within the review; inclusion and exclusion criteria of review; and number of papers included in the review (See Appendix 4). EM, CM, TF, CO and FM all contributed to this exercise.

Secondly, for every paper a judgement was made (FM, CO, TF, EM, CM) as to whether material relevant to the four constructs of the NPM was present or absent. As this was a qualitative analysis, no attempt was made to quantify the weight put on any one NPM construct in a given review; the analysis was limited to whether material relevant to the construct was present or absent.

Having determined which constructs were represented in each paper, we then extracted all the process oriented key words or phrases in each paper, and code them to the relevant construct of the NPM. Two independent reviewers coded according to the constructs of the NPM. If a key word or phrase could not be coded to the NPM this was stated, as a way of identifying gaps in the model. FM; EM, TF, and KO’D each extracted data from 25% of papers. FM then rechecked all data extraction to ensure every action-oriented statement was identified and coded accurately according to the constructs of the NPM (May 2006). Data, which were statements treated as attributions, were thus extracted from all included papers, and these attributions were then recoded, independently by May.

Dual coding in this way permitted economy of effort and enabled differences in coding and interpretation to be identified and discussed. Any areas of disagreement, which were minimal, were resolved through discussion. If any areas of disagreement remained then a final reviewer (TF or EM) served as arbiter.

At the same time FM conducted a separate thematic analysis of the identified literature, examining to what extent emergent themes matched those identified through the scoping exercise. This then presented an



opportunity to explore the effects of different approaches to coding, that is, what differences, if any, were noted through using the NPM as a coding framework versus using results from an emergent thematic analysis, on our final conclusions regarding the key findings from the review.

3.7 Stage 2 - Results

3.7.1 Papers identified

Between them the searches yielded 6,585 unique citations (Figure 1). Of these, 6,439 could definitely be excluded on the basis of the title or abstract, leaving 146 citations where the full paper was needed before a decision could be made. Of the 146 full papers, 19 met the criteria for inclusion (Appendix 4).



Figure 1. WP1 Quorum Flow Chart

Reasons for exclusion at full paper

stage:

Not a review: n = 105

Not on e-health: n = 1

Not on implementation: n = 20

Only published as an abstract: n = 1

(Where more than 1 reason applied,

only the first was counted).

Medline

3,123

Embase

3,373

CINAHL

40

Cochrane

9

PsycINFO

40

References in database

6585

Order

146

Exclude

6,439

Include

19

Exclude

127



Quality of included reviews

The methodological quality of many of the included reviews was very poor. Some reviews did not even mention which databases were searched and search strategies were often rudimentary, and many reviews had not specified their inclusion and exclusion criteria (Appendix 4). Furthermore, for eight (42%) of the nineteen papers reviewed it was impossible to ascertain how many papers had been included as this information was not explicitly provided.

Description of included reviews

Of the 19 included reviews, 13 originated from the USA, 4 from Canada, and 1 each from the Netherlands and Malaysia. Reviews tended to cover one or more e-Health domain, with 8 reporting on management systems, 7 on communication systems, 5 on decision support, 1 on information systems.

Relationship with the Normalisation Process Model

Each statement from a paper relating to findings regarding barriers or facilitators to e-Health implementation was treated as an “attributive statement.” By this method 411 attributive statements were coded directly against the NPM. All the included papers contained data which could be coded according to the NPM. They also all included data which could not be coded to the NPM, such as that about technical specifications, beliefs, attitudes or the need for evaluation.

Table 1 shows the constructs of the NPM that were addressed within each review. It aggregates the number of attributions included for each construct within each paper. If the exact same attribution was made more than once for any given paper it was counted as a single attribution. It can be seen that the majority of the reviews contained a great deal of information that related to contextual integration issues (142/411), namely the degree to which the e-Health innovation related to organisational structure and goals. Rather less attention was paid to the impact of the interventions on health professionals’ interactions with patients (interactional workability)(55/411) and inter-professional relations (relational integration)(72/411). Skill set workability was largely considered as a training issue, rather than as a legitimate concern, with 42/142 comments falling into this category. These findings are illustrated clearly in Figure 2. It should be noted that Table 1 and Figure 2 are not meant to have statistical properties but rather serve to illustrate the distribution of comments relating to barriers and facilitators across the NPM. We have found it a valuable means of presenting this content analysis in an effort to determine whether coding to the NPM helps identify any gaps or issues given particular emphasis, by the published literature.

The following section illustrates how findings related to the NPM.



35

Table 1. Number of Attributive Statements Within Each Category of NPM

Interactional Workability Relational Integration Skill Set Workability Contextual Integration Outside NPM

1. Broens 2007 3 4 5 6 7

2 Chaudrhy 2006 0 1 1 18 1

3 Hebert 2006. 0 1 1 3 5

4 Hilty 2002 4 4 5 6 4

5 Jennett 2004 0 3 1 8 13

6 Jennett 2005 1 2 0 2 2

7 Johnson 2001 0 4 3 5 5

8 Kawamoto 2005 7 1 0 1 2

9 Kukafka 2003 1 3 3 6 2

10 Leatt 1 4 4 6 5

11 Lu 2005 10 12 3 9 6

12 Ohinmaa 1 3 1 8 7

13 Papshev 2001 2 3 1 12 6

14 Peleg 11 7 2 13 16

15 Shekelle 2006 2 2 1 4 3

16 Studer 2005 1 3 1 12 1

17 Vreeman 2006 2 7 4 11 4

18 Yarborough 3 6 3 7 7



36

19 Yusuf 6 2 3 5 4



Figure 2. Radar Plot of Distribution of Attributions within the NPM

Interactional Workability

Interactional workability issues accounted for less than 14% of the coded data relating to barriers and facilitators of e-Health. Most of the focus in the literature in relation to this construct concerned issues regarding “ease of use”, which could be presented as either a barrier or facilitator of implementation as the following examples demonstrate.

‘a common theme among all four features is that they make it easier for clinicians to use a decision support system….as a general principle then our findings suggest that an effective clinical decision support system must minimise the effort required by clinicians to receive and act on system recommendations.’(Kawamoto 2005)

‘the literature suggests that utilisation of PDAs saves clinicians time in regard to accessing, retrieving, and recording data’ (Lu et al .2005)

‘physician and organizational resistance due to the perceived negative impact on the physician’s workflow.’ (Shekelle 2006)”

Although ease of use was the main aspect of interactional workability identified, there were a few comments on other aspects, such as perceived negative effects on health professional – patient interaction:

‘…the unique relationship between a physician and his or her patient is different from that found in other industries. Many physicians value this interaction and are hesitant to give it up or take time away from the experience due to new information technologies.’ (Yarborough 2007).

Distribution of Attributions

050

100150

IW

RI

SW

CI



Relational Integration

Security and safety issues were a key issue within this construct. Such concerns could act both as facilitators or barriers. On the one hand people may expect e-Health technologies to reduce errors, which would encourage uptake:

‘Greater efficiency and increased error reduction will be achieved if data capture or data entry are permitted wherever and whenever it is available’ (Lu et al. 2005)

On the other hand security and safety concerns could serve as a barrier to widespread utilisation:

‘EMR system downtime: exposed physicians and practices to risk of patient injury and liability’ (Studer 2005).

‘concerns regarding the privacy and confidentiality of patient information stored in an EMR’ (Studer 2005).

Skill Set Workability

Skill set workability was concerned mainly with the need for adequate training for personnel who were going to be involved in any e-Health implementation, although issues relating to division of labour, that is, who does what when were also mentioned:

‘potential affect on the division of work among care providers’ (Peleg 2006)

‘A need for training users how to use these novel types of systems. Such training is needed at all levels’ (Broens 2007)

Contextual Integration

Contextual integration refers to infrastructure and organisational issues. The literature emphasises the need for adequate resources, particularly finance. Issues such as administrative supportive, policy support, standards and interoperability also fell within this construct.

‘factors within the organisational infrastructure required to enable usage eg available resources, supportive policies and accessibility to the IT system.’ (Kukafka 2003)

‘Barriers to implementation include funding’ (Jennett 2005)

‘Many programmes fail because of inadequate financial and other administrative support from the leadership, or the fact that telemedicine is not a match for the overall mission of the organisation’ (Hilty 2002).

‘challenges for adoption of electronic health records…..included cost, technical issues, system interoperability.’ (Shekelle 2006).

OUTSIDE THE NPM

Many issues, 24% (100/411) fell outside the model. These mostly related to technical issues, attitudes, and issues of engagement and evaluation. Sometimes items were unable to be coded to the NPM because they were very generic and non specific, making it unclear as to the “work” that was being done. Examples of items which did not fit the model were:



‘the importance of evaluating CDS’

(Peleg 2006)

‘Ownership, attitudes of participants, professional connections remote location’ (Ohinmaa 2006)

‘recommendations: include end users, especially clinicians into in the system development activities.’ (Vreeman et al. 2006).

It is important to note that the majority of data did fall within the constructs of the NPM and using this model helped to improve understanding of the processes at play and to identify strengths and gaps in the literature.

3.7.3 Thematic Coding of Systematic Review.

As well as coding the systematic review using the NPM as a coding frame we also continued to develop our thematic coding, that began during the initial scoping exercise. Most of the data in the systematic review mapped onto these themes, and the table of all coded data is shown in Appendix 5. It is worth noting that some additional codes were added which had not been in the original coding framework - technological issues, communication issues, organisational issues and ‘other’.

Issues that fell into the themes that matched the initial scoping exercise codes were similar and are not repeated here (although shown in Appendix 5) but examples of items that fell within the new themes are illustrated in the following section. Table 2 shows which themes were present within each paper.



Table 2. Thematic analysis codes (full list of papers with data mapped onto each theme included in Appendix 5)

Conditions Prior To implementation

Cost

Need and importance of validation and

Attitudes

Ease of Use of System

Security,

Confideniality

Education and Training

Technological Issues

Communication Issues

Organisational Issues

Other

1. Broens 2007 � � � � � � � � �

2 Chaudrhy 2006 � � � �

3 Hebert 2006. � � � � � �

4 Hilty 2002 � � � � � � � � � �

5 Jennett 2004 � � � � � � � � �

6 Jennett 2005 � � � � � � � � �

7 Johnson 2001 � � � � � � �

8 Kawamoto 2005 � � � � � �

9 Kukafka 2003 � � � � � �

10 Leatt 2006 � � � � � � � � � �

11 Lu 2005 � � � � � � � � �

12 Ohinmaa 2006 � � � � � � � � � �

13 Papshev 2001 � � � � � � � � � �

14 Peleg 2006 � � � � � � � � �

15 Shekelle 2006 � � � � � � � �

16 Studer 2005 � � � � � � � � � �

17 Vreeman 2006 � � � � � � � � �

18 Yarborough 2007 � � � � � � � � � �



19 Yusuf 2007 �

�

�

�

�

�

�



Technological issues

These ranged from system design issues to generic comments on product quality:

‘Technical issues under-reported including bandwidth, audio quality, FPS, size of transmitted video image speed of computed and name and make of CODEC and other equipment’ (Hilty 2002)

‘inefficient technology pg 5 discussion’ (Jennett 2005)

‘systems that prompted clinicians to record a reason when not following the advised course of action were significantly more likely to succeed than those that allowed the system to be bypassed without recording a reason to’

(Johnson 2001)

‘Products and technology’ (Papshev 2001).

It is therefore clear, that while technology proponents may frequently suggest that “the technology is not the problem”, the literature suggests that technical issues continue to be cited as either an important facilitator or barrier to implementation.

3.7.3.2 Communication

Communication issues related to the e-Health implementation itself and to communication using the system – such as during professional - patient interactions. Sometimes these were quite generic statements whose meaning was unclear.

‘it is also important to maintain bidirectional communication throughout the process to enable staff to see current measures indicating cost savings and error reductions, as well as to ensure that staff are providing input about the system’s usefulness and potential modifications’

(Leatt 2006)

‘Physicians’ concerns that system would interfere with or negatively impact the physician-patient encounter’ (Leatt 2006)

‘Benefits in communication’(Lu et al. 2005)

‘Communication between clinicians has also improved’ (Ohinmaa 2006).



Organisational

Organisational issues included, for example, the extent to which e-Health services matched organisational priorities, their effects on workflow, and how they might benefit organisations.

‘Alignment of decision support objectives with organisational priorities and financial interests’(Kawamoto 2005)

‘a shift toward specialised and focused system, interacting systems that are integrated into the clinical environment and workflow’(Peleg 2006)

‘the importance of considering work flow design as part of the EMR implementation process’ (Studer 2005)

‘Improved reporting capabilities …cited the capability for more comprehensive reporting that integrated clinical and administrative data as a key benefit. …Helped clinicians and administrators to be aware of the current departmental workload which helped provide the rationale for how pts and therapists were scheduled.’ (Vreeman et al. 2006)

3.7.3.4 Other

There were only two statements that fell out with our thematic coding and these were:

‘Context: stability of client’s condition and nurses ability to co-ordinate more than one visit’ (Hebert et al. 2006)

‘phase 5: system use inducing strategies focuses on developing and implementing approaches that are proactive and specifically targeted to influencing favourably the predisposing, enabling and reinforcing factors identified in phase 4.’ (Kukafka 2003).

3.8 Discussion and conclusions

A thorough and systematic search of the literature identified 19 reviews which addressed the implementation of e-Health technologies into health care settings. The majority of these reviews originated from North America (the USA and Canada). With a few exceptions, they were of poor quality, and their findings must be treated with caution. It is worth noting that very few reviews focused on information systems, e.g. use of the internet, which is surprising in view of the growing importance and potential of this area of e-Health.

Recommendations

1 Findings from existing systematic reviews about e-Health implementation should be viewed with caution as they are misleading.



2 Any future e-Health implementation reviews must address each of the 4 domains of e-Health.

It appears that the fundamental assumptions underpinning research in this area are unsatisfactory in that there is undue focus on organisational issues and insufficient attention being paid to problems of workability, that is, the work that health professionals, as individuals or teams, must undertake to make these systems function satisfactorily in practice. However, despite caveats about the lack of robustness of the findings of the reviews and the undue focus on organisational issues included in our review of reviews, certain conclusions can still be drawn concerning the identified barriers and facilitators to e-Health implementation.

First, the Normalisation Process Model (NPM) was shown to be a valuable analytical tool, which provided a good theoretical framework for synthesising the available data and proved more informative than an atheoretical thematic analysis would have done in highlighting the gaps in the literature and illustrating areas where extra work is necessary. If one were to draw only upon the thematic analysis one might gain the impression that the literature was fairly broad ranging in its coverage with few gaps. However, the use of the NPM reveals that the literature emphasises issues of CI, with an emphasis on the execution and realization of systems at the expense of understanding the work of embedding them in practice. Far less data was present on the impact of e-Health initiatives on interactions with patients (interactional workability issues), inter-professional relationships (relational integration issues, and fit with existing staff skills and roles (skill set workability). The published literature therefore provides an unbalanced account of barriers and facilitators to e-Health implementation, neglecting the wider social framework that must be considered when introducing new technologies. Thus, if e-Health implementers are being guided simply by the existing literature it helps explain why e-Health is not becoming normalised. Use of the NPM helps to highlight where the gaps in the existing literature are located and subsequently informs the development of clear recommendations as to how to address these problems.

Coding to the NPM also shows gaps in the model itself. Material that could not be coded within the model included technological issues, issues relating to professional beliefs and attitudes (the focus of other theoretical frameworks), and importantly issues relating to enrolment and engagement of health professionals and also to the ways individuals and groups appraise e-Health services as being beneficial or not. This has contributed, along with other related studies, to further development of the model as will be addressed in Section 7 taking analysis to a higher level.

3. The research team believes there is little to be gained at present from commissioning a further systematic review in this field.

Although the literature has methodological limitations, the findings resonate with two other reviews recently conducted by the co-applicants. Based on this fact, and our knowledge of the primary literature, we believe it is unlikely that a further review, even if robust, would add to our knowledge in this area. We would therefore not recommend this as a good use of scare research funds. However, we do feel there is a requirement for further primary e-Health implementation research of the kind suggested below.



Recommendations

4. Primary researchers of e-Health implementation need to broaden the scope of their research so that they give greater consideration to the impact of e-Health technologies on:

� interactional workability, that is, the effects of e-Health services on health professional – patient encounters;

� relational integration issues such as confidence in the safety of a system and the effects on efficiency;

� skill set workability issues which include not simply training and support issues but also examination of how new technologies affect roles and responsibilities;

� enrolment and engagement with participants;

� how new practices are evaluated and judged by participants.

Thus, greater attention to a broad range of socio-technical issues is merited. Researchers need to explore all these issues in greater depth. It was clear from the thematic analysis that some issues such as cost and training were addressed in quite a superficial and simplistic way, whereas much deeper examination of these issues is merited.



4 Work-package Two (WP2) Barriers and Facilitators to the Implementation of E-Health Services: The Perspective of Health Professionals.

4.1 Background

A key challenge for those wishing to encourage the use of e-Health services is how to move to mainstream service delivery. Understanding barriers to the implementation of e-Health services is crucial if this is to be achieved. Professional resistance is frequently cited as one of the most significant of these barriers. However, earlier suggests that implementation remains a problem even when health care professionals support the use of new technologies (May et al 2003a, Lapointe and Rivard 2006). It was not our intention to replicate this work in this work-package; instead our purpose was to examine health care professionals’ views about:

1. Factors that affect implementation and integration of e-Health services;

2. How to facilitate integration of e-Health service in different health service contexts

3. The effects, if any, of e-Health services on health care professional/patient interactions and clinical activities; and

4. Training and educational requirements.

4.2 Aims/Objectives

The aim of the work-package was to identify informants’ views of implementation and integration of e-Health services across the four e-Health domains outlined in Section 1.3, Box 1.

Objectives

The objectives of work-package 2 were to:

1. Increase our understanding of barriers to and facilitators of e-Health implementation.

2. Highlight training/educational requirements.

3. Help refine our conceptual model of e-Health normalization and thereby inform other work-packages.

4. Inform the development of recommendations regarding implementation and integration of e-Health services.

5. Compare the findings from a thematic analysis and an NPM-based analysis in order to determine, whether there were particular advantages or disadvantages to either approach, and which method would prove most helpful in conceptualising the gaps or strengths of the data.



4.3 Methods

4.3.1 Design

Work-package 2 used two complementary methods of qualitative data collection - task groups and semi-structured telephone interviews.

4.3.2 Task Groups

Task groups are a hybrid focus group intended to generate both conventional qualitative data and recommendations for action grounded in the experience of group members. They were used not only to obtain the views of participants on the issues affecting e-Health initiatives in their area, but also to identify their views on the issues that will affect the implementation of e-Health projects.

To obtain a wide range of experience we sought members from health professions such as primary and secondary care physicians, nurses and other allied health professionals from three contrasting NHS contexts. We held five task groups, each preceded by informal discussion and the distribution of information to participants, in particular the results of the initial scoping exercise within WP1.

4.3.3 Setting and sample

Identifying case study sites and participants

We aimed to identify three case study contexts which differed in respect of:

� Geographical location/coverage

� Levels of utilisation and normalisation of e-Health services

� Domains of e-Health Used

� Different service and policy contexts

We believed these criteria would allow us to examine health professional perspectives on e-Health implementation from those located in deliberately different environments – and so contribute to the objectives described in Section 1.2.

Criteria for task group participants were that they should be:

� health professionals from a range of backgrounds, including physicians; nurses; professions allied to medicine and pharmacy.

� from different sectors, particularly primary and secondary care.

� people with real experience of using e-Health services.

� managers and informatics specialists involved in running and/or developing e-Health services.

We wanted to recruit participants from different disciplines to obtain a broader view of barriers and facilitators to e-Health implementation than a uni-disciplinary approach might permit. We also wanted the views of individuals with practical experience of e-Health as opposed to hypothetical beliefs. We thought that the task groups would benefit from the experience of members who were directly involved in overseeing service delivery.



Having determined our selection criteria the project team selected sites that they believed would be suitable based on the criteria set out above. In order to protect participants’ anonymity, as promised in our consent forms, we are not providing identifiers of the study sites within this report. Suffice to say, that:

• � One site was an exemplar of an English NHS organisation with well integrated e-Health services mapping to the management and computerised clinical decision support system domains of our classification, involving both primary and secondary care sectors.

• � One site was a health board in Scotland which did not have an integrated electronic medical records system. However, it had several e-Health initiatives and services, ranging from specialist paediatric telemedicine services, to the use of electronic medical record systems in primary care (85% of Scottish GPs use the GPASS software). As elsewhere in Scotland the Emergency Care Summary project extracts recent medication and allergy information from GP electronic medical record systems and makes this available to out-of-hours clinical staff. It therefore used e-Health services that match the management and communication domains of our e-Health classification system.

• � The third site was an NHS call centre with services that included all four e-Health domains, as currently it uses an electronic medical records system, asynchronous electronic communication systems to relay patient information and triage decisions to integrated partner organisations, an electronic clinical decision-support system and has a web presence.

Task Group Recruitment.

Task group recruitment was facilitated by contacts established within each site, generally individuals well known within the organisation with respect to e-Health initiatives. Nevertheless, recruiting the task groups was a complex task. As well as initial ethical approval (attained from an MREC early in the project), we also needed project approval by the research governance bodies of the respective sites. At one study site we achieved this very quickly but for the other two this proved more challenging. In both cases staff turnover was a factor and our initial points of contact for governance left before the process was complete. Although we had received verbal assurances of approval from both contacts and had completed the relevant paperwork, we required an official written record of this approval in order to proceed both safely and accountably. After delays of several months we received written approval and honorary contracts for GA to conduct the task groups.

Potential participants, identified by our internal contacts, were sent a standard information and invitation letter via initial e-mail contact (see Appendix 6). Those who responded and expressed an interest were asked to suggest suitable dates and times. Once two or more participants offered compatible dates and times, these were offered to those remaining. One week before the task group, participants were sent a summary of the findings from the initial scoping literature review (Appendix 8). Since much of the task group was to be centred on the extent to which the findings of



the literature reflected the experience of professionals, it was important that they were given this initial frame of reference.

A factor in recruitment was igniting enough interest and willingness amongst the professionals within each site to participate in our project. Although we were given considerable help and support by our contacts, the response rate from potential group participants was mixed. At one site the response rate was initially very good. This may have been, at least in part, due to the fact that the organisation had a long-established electronic health system with which all professionals had some level of involvement. Hence, a broad array of professionals working at this study site would feel qualified to comment on their experiences. Despite a positive numerical response from these professionals, arranging a suitable time in which all who wished to could attend remained a challenge, as is often the case in any busy health service environment. However, the first group ran with the maximum number of 8 attending.

Arranging the second group at this site proved more challenging. Though we had had a very positive response rate for the first task group, our contacts had already utilised their influence and were limited as to how many new recruits they could provide. We contacted two professionals who had been unable to attend the first group. One of these agreed to attend if a group could be conducted at a suitable time. After further enquiries and advice from our contacts, we were able to attract the interest of three more. Despite further efforts we were unable to recruit more participants, so ran the second group with the willing four. Had we further delayed the running of this group, we may have found more willing participants, but it was equally likely that those we already had enrolled would lose interest.

Recruitment at the second site, where research governance approvals had been obtained with relative ease, was surprisingly difficult. Fifteen professionals were contacted directly, and others indirectly through our contacts. Three who initially expressed willingness subsequently ceased replying to correspondence, two declined to take part, and four did not respond to our initial (e-mail) approach. The remaining six agreed to take part, but could not all attend on the same date. They were subsequently split into two groups of three, though one who was to be a member of the second group was unable to attend on the day due to unforeseen professional commitments. He was subsequently interviewed as a key informant.

The nature of our NHS call centre study site meant that it was necessary to conduct the group at their working base during official working hours. The group consisted of four members of the night-shift at that particular time. Though eight were initially approached on our behalf by our recently established internal contact, this was done with the understanding that it was unlikely all would be able to attend, unless it happened to be an unusually quiet shift. GA hence travelled to the call centre early in the evening shift and four nurse advisors attended the task group.

Task groups were conducted between June 2007 and May 2008. The composition of each is shown in Table 3.



Table 3. Composition of Task Groups

GPs Secondary Care Doctors

Nurses Pharmacy/Allied Health Professionals/Health professional advisors

IT or Clinical Managers/Technical Staff

Task Group 1

2 3 2

Task Group 2

2 1 1

Task Group 3

4

Task Group 4

1 1 1 (former nurse)

Task Group 5

1 1

4.3.4 Key Informant Interviews.

To develop and elaborate on conclusions from the task groups, additional telephone interviews (16) were held with key stakeholders involved in e-Health implementation throughout the UK. These were used to validate and elaborate on the key issues highlighted within the scoping exercise (WP1) and the recommendations and principles from the task groups.

Recruiting strategy.

As with the task group participants, interviewees were sent an initial invitation and information letter, (Appendix 7). The interview schedule was almost identical to that of the task groups (see below); however, there were key qualitative differences both in the running of the interviews and subsequent data analysis. As the interviewees were chosen because they were involved in e-Health implementation at a more senior level than the task group participants, and the content of the interviews reflected this, they were analysed separately. They were interviewed as implementers and experts rather than as frontline users. Potential interviewees were identified initially from the prior knowledge of members of the research team (mostly by FM and FS). They were also identified by the interviewees themselves. Hence, the list of those eligible grew with interview, though as with the task groups, many of those approached (via e-mail) did not respond. Of the thirty five people approached by us directly, sixteen were interviewed. Others either refused or did not respond, or expressed interest and willingness but did not reply to further correspondence.

Three of the interviews were conducted face-to-face, though most were conducted by telephone. As our research was concerned more with content than interaction, nothing significant was lost in telephonic interviewing. Interviews were conducted between August 2007 and May 2008 and covered representatives of a range of professional groups with different



backgrounds and experiences of e-Health, including senior representatives from nursing, primary and secondary care (including clinical directors/leads and national programme leads), academia, royal colleges, and informatics specialist groups.

Task groups and interview schedules.

As explained above, prior to the task groups and interviews being conducted, all potential participants were sent an invitation and information letter, explaining our project and why we thought their contributions would prove useful. A few days before a group was due to be held, those who had confirmed they would participate were sent a short summary of the key findings of the WP1 literature review (the seven key barriers and facilitators to e-Health implementation illustrated in Appendix 8). The rationale for this approach was to give the participants enough time to acquaint themselves with the ideas which would guide the group or interview, but not so far in advance that they would either forget the material or present the group with overly-prepared answers. The key barriers and facilitators were presented in text boxes placed at random levels on the page; this was so as to avoid giving the impression that we had imposed upon them a hierarchy of importance. Informed consent for participation and to tape-recording of the meeting was confirmed for all participants.

The task groups began by gathering information from each participant about their roles, their experience of using e-Health systems, and the systems they had used. The group then explored the main factors identified in the review, and how they may impede or facilitate the implementation of e-Health systems. The same pattern was employed in the key informant interviews; the interviewee would be asked to describe their background in e-Health and appraise the findings of the literature review.

A schedule was developed for the task groups and interviews to guide the group facilitator/interviewer (GA) and to ensure that all the important areas were covered (Appendices 9 & 10). Participants were asked to consider specific systems in which they had been involved, and which factors, if any, they believed the most relevant to their experience and the type of system in which they had been involved. They were then asked about each factor specifically; whether they had personal experience of it (and could they report real examples) and whether they agreed with the findings of the review that the factor was an important barrier/facilitator. Finally, they were asked if they felt any important factor was missing from our findings.

Task groups and interviews were conducted in a semi-structured manner, with the schedules acting as a guide for the researcher. This encouraged participants to talk at length. As well as considering whether the factors covered in the literature were an accurate reflection of their personal and professional experience, participants were also encouraged to describe their working practices. In short, we were concerned with ascertaining how, as well as whether the literature findings were relevant.

A further benefit of this semi-structured approach was to allow respondents to determine the direction of the interviews and task groups. They were able to make their own linkages between, and venture their own understandings



of, the findings rather than have these imposed by the facilitator/interviewer. We wanted them to order our findings, it was their decision as to which they wished to concentrate upon. It was also possible that the answers given by participants in response to initial prompting would cover areas that would have emerged later in the schedule; the semi-structures approach thus avoided repetition.

The task group and interview schedules and the initial task group and key informant interviews were made available for review by the whole research team at a joint data meeting. The approach to running the interviews and task groups were then amended slightly to ensure that we achieved the aims and objectives of this WP. We believe this approach has improved the quality of data collection and facilitated achievement of the WP aims/objectives.

4.4 Data Analysis.

We analysed the textual data from our task groups and interviews in an identical fashion. The method involved analysing both sets of data in two distinct ways, one an atheoretical thematic analysis and the second a framework analysis using the NPM as the coding frame. This dual approach to data analysis was undertaken because:

1) we wished to explore whether free coding would highlight issues that might be less obvious or missed entirely within the constraints of a predefined coding framework;

2) our experience with analysis of the data in WP1 had shown that atheoretical thematic coding could conceal gaps or strengths in data, whereas using the NPM as a coding frame might help us conceptualise strengths and weaknesses within the data more clearly. It would also help to clarify where issues resided within a broad implementation model and thereby provide valuable information that would inform the recommendations arising from our results.

1) An initial thematic analysis was undertaken, assigning general themes to each segment of text (Kvale 1996) until the point of saturation (no new themes were emerging). Here, it was important to acknowledge the previous thematic findings of the scoping exercise, but not to impose them unnecessarily. Once a code had been assigned, it could them be re-assigned upon encountering further data to which it was appropriate. It was thus possible to ascertain quickly the extent to which specific phenomena appeared in the data as a whole. The initial coding was fairly general: large segments coded with key themes. However, as the analysis progressed, it was possible to embark upon more detailed coding of segments of data and relate certain codes to one another. All the data was initially coded freely. “Free” codes had no immediate relationship with other codes, hence their name. They could then be attached to other free codes to which they could be related, often in a conceptual hierarchy. When they were attached, they would become part of a “tree”; having a key concept at the root (e.g. ease of use) with various branches growing from it (e.g. compatibility and efficiency). In short, “trees” function as a tool with which to hierarchically order concepts (themes). Hence, many “root” codes would be sub-coded with related emergent themes. This was invaluable when choosing the data upon which to concentrate in writing the results. Segments rich in recurrent



themes and encompassing examples of a variety of significant phenomena would hence serve as effective illustrations of the dynamics of e-Health implementation as a whole. This is not to say that atypical phenomena would be discarded. On the contrary, the atypical nature of such themes made them potent subjects for inclusion. To find a case “unusual” serves to emphasise the typicality of other cases; what Silverman (1993:44), adapted from Becker et al. (1961), calls “deviant case analysis”.

Once we had identified the emergent themes within the data, they were mapped onto the NPM to identify higher order themes. In doing this, it was possible to consider the place of these emergent barriers and facilitators within a broader process, relating each theme to the constructs within the model, and considering illustrative examples. However, the free coding drove this first approach to data analysis.

2) Once the thematic analysis was complete, a second, entirely distinct analysis was undertaken where the statements made within the task groups and interviews were treated as attributive statements and coded using the NPM directly as a coding frame. Attributions were therefore coded to one or more of the four constructs of the model, or if they were deemed not to fit within the model were classified accordingly. Two reviewers coded the statements independently, enabled differences in coding and interpretation to be identified and discussed. The distribution of attributive statements within the model, were then examined. This approach was expected to facilitate understanding of where, within the model, the bulk of data resided.

4.5 Results

The main themes emerging from our thematic analysis of the task groups and key informant interviews relating to barriers and facilitators to implementation were:

1. Patient benefits and clear purpose.

2. Collaborative working.

3. Standards; including a) universality, b) clinical risk, legality and safety.

4. Ease of use; including a) familiarity (with the system/interface), compatibility, and ease to learn, b) efficiency and c) technical support.

5. Professional attitudes; including a) unity of purpose and b) sense of user empowerment and relationship with suppliers and designers.

6. Clear rationale.

7. Cost; including a) time, convenience and physical space and b) financial cost.

4.5.1 Patient benefits and clear purpose.

Perceived patient benefit was a theme apparent throughout the data, and often cited by participants as a theme missing from the findings of the literature review. Clear purpose was inextricably related to patient benefit, and it related to what health care professionals perceived as the patient’s understanding of what new systems were attempting to achieve for them. An ambulance service redesign manager spoke of how the system used by



the paramedics enabled the quick and remote transfer of information between paramedics and hospital based staff, to the ultimate benefit and safety of the patient.

‘… rather than have the ….case of every time someone phones up for an ambulance, we turn up, we do something and then (take them)…to the department……. we don’t have to do that ……..we can keep them in the community………..’

While this form of communication and decision support could reduce unnecessary hospital admissions without compromising the safety of the patient concerned, the redesign manager added that there was reluctance to do this because of safety concerns. This demonstrated one of the dilemmas faced by health professionals in relation to new technologies: the confidence to use them without in any way compromising patient care. Thus, even if the technology enabled more efficient working, remote working, or both, it would not necessarily be immediately embraced. The need for professionals to understand the benefits to patients derived from new technologies was not exclusively about protecting their own accountability. In the same group, the former nurse, spoke of the benefits of electronic communication in the rehabilitation service.

‘I mean, it’s purely speculation, I think that the true value that they make to the patient systems are possibly for the people that don’t attend rehab, the people that we don’t catch now. It’s a different medium and it is maybe a bit more different from a hospital and that might be why it might catch them. ……..’

What was significant here was the availability of a new means of communication between the patients and professionals, and indeed the potential to assist previously hard to reach patients. It was suggested that while some patients may have initially been reluctant to engage in a face-to-face encounter with a doctor or a nurse, if they could communicate remotely (and with relative anonymity) they might receive assistance that would not otherwise be available. It was suggested that professionals welcomed this (rather than feeling undermined) and that the technology could increase the connection between professionals and patients.

What was immediately significant about this theme is that it did not emerge in the literature. Indeed, it seemed that researchers have underestimated how important patient benefit was to professionals. Aside from the two examples presented above, the need for a demonstrably positive impact on the patient was frequently cited by professionals as a facilitator to the implementation of new technologies.

It was not simply the practicalities of giving benefit to the patients through new technologies that was a concern of task group members, but also the patients’ understanding of its uses. A consultant spoke of patient satisfaction.

‘And yet you can’t quantify patient satisfaction. That’s the hardest thing. Patients may actually feel more secure knowing that ……, after what you said, I know now, if I’m going in an ambulance, I’ll feel much more confident because I know that these things are going on and I think you can’t assess that, you don’t know if your survival’s better because you know that!’

The consultant was referring back to the ambulance redesign manager’s explanation of the communication that could now take place between



paramedics and hospital staff. In putting herself in the role of a patient, she demonstrated how patient understanding of these new technology-enabled practices could improve patient confidence and ultimately their satisfaction with the services available. However, the other side of patient understanding was patient misconception, which was also an emergent theme within the task groups. The nurses within the NHS call centre spoke at some length of the public perception of their service:

‘N3 The way they advertised it has clearly gone way off the mark, and that’s why the public’s expectation is way beyond, it’s ridiculous, even in a perfect world I don’t think you could meet their requirements, but that’s partly the way it’s been put forward to them, because you will have 24 hour access to professionals.’

Clearly, the nurse’s assertion was that the advertising of the service had led to an imbalance between patient expectation and a deliverable service. While the call centre provided 24 hour telephone triage they were still, in many respects, advisors. While they could potentially diagnose problems and offer a course of action to the patients, they were in no position to offer hands-on medical care. Hence, the service’s abilities had, in her eyes, been exaggerated to the detriment of the service’s relationship with patients.

Patient benefit was also cited as important by the interviewees. A nurse described one of the main findings of her research:

‘……the focus of the system, in a sense, has to support … with the electronic health record, for example, it must be a patient-centred approach, as opposed to a professional-centred approach. Because otherwise, you won’t get the integration, and the benefit, and… in the same way. So, if it’s viewed from the patient’s point of view, and how the professional interacts with that patient, we’re more likely to get something that supports the patient’s journey, which is, after all, what we’re trying to do.’

The point made here was that although the patient-centred approach was, in her view, most important, the professional’s unity of purpose in providing patients with the best possible service was inextricably linked to it. These sentiments were echoed explicitly by other interviewees, and were implicit throughout.

While it was clear that the benefits to the patient were highly significant factors in successful implementation of new health technologies, the above quotations showed that there were several aspects to this: professional accountability and confidence; the ability to increase both the level of care and the number of patients available to care for; and the relationship between the patients and professionals derived through a clear and mutual understanding of a system’s purpose.

There were clear links to the NPM in the phenomena described above. The ambulance service redesign manager’s reference to confidence in the new systems was an issue of relational integration in that it suggested an impact on the pre-established interpersonal relationships both between



professionals and professionals and between professionals and patients. It demonstrated how the professionals’ sense of their own accountability to the patients was reflected in their willingness (or reluctance) to utilise and rely upon a newly established system. Hence, while the technology in itself was undoubtedly important, professional trust in such technology was the over-riding factor.

There was also an issue of the interactional workability of such technology. If a new system enabled better communication between professionals based in different locations (e.g. a hospital and an ambulance), there was clear benefit for both professionals and patients; the technology could be said to be interactionally workable. However, without the related sense of relational integration (confidence in the technology), such workability may not have been fully realised. Likewise, without the workability, integration was less likely. The Normalization Process, in terms of knowledge between agents, at least in the context of patient benefits and clear purpose, was hence cyclical.

The more general impact of the new technologies in terms of the overall practices of an organisation was also apparent when considering patient benefits. Certain changes in professional roles enabled by new technologies would certainly serve to benefit the patient, e.g. paramedics being able to communicate better with the hospital base and hence being able to offer a greater level of patient care in transit. This highlighted the role of skill set workability in patient benefit. A relatively subtle change in the division of labour may potentially have had significant benefits for both patient care and patient confidence, as perceived by health care professionals.

Finally, there was a broader factor related to a more general notion of patient expectation. This was illustrated most explicitly by the call centre nurse when she reported on the “bad press” they had received. This may have been a factor which transcended e-Health and was an issue for both health and technology as a whole; hence, within the NPM, it was clearly an issue of the contextual integration of the service or system (how the system fitted into the broader context of current healthcare). This served to demonstrate the “double-edged” nature of new technologies: these new systems may have enabled professionals to do more for patients, but the technology itself may not have met the fullest expectation of its users (be they patients or professionals). With technology came an expectation for significantly improved services, and should such expectations be proved unrealistic, it was the reputation of the related services which suffered.

4.5.2 Collaborative working.

The data suggested that the organisational changes necessary for successful system implementation had to take place on a holistic basis. One participant said:

‘We have a bit of difficulty though, because we have a separate database for collecting information on the paediatric units, which has no link at all with any of the other systems.’



This demonstrated one of the potential problems created by the variety of available systems. While one system may have been highly effective in the organisation of inpatient care, another may have proved more efficient to other clinical uses, in this case, paediatricians. Even if these two systems were geared towards achieving the same end (be they recording the relevant information in such a way that it could be retrieved quickly and efficiently, or suggesting an appropriate course of action on inputting a specific list of symptoms), if these systems were designed for different patient groups or different parts of an organisation, they may not have been compatible with one another. What the paediatrician implied is that the data recorded by the system that he used was difficult to integrate with that collected on the other systems within the organisation. While there were clearly benefits to having specific systems for specific disciplines, problems emerged if these systems were not in some way linked, leading to the problematic isolation of disciplines. This issue of compatibility between systems was one of interactional workability: how the compatibility between systems affected the exchange of information between professionals in real time.

Collaborative working seemed particularly important at the initial stages of implementation. A clinical manager from a task group explained:

‘…you’ve got to get the clinicians on board, you’ve got to get the staff on board, you’ve got to make the staff look at the system and make it such that it’s user-friendly, so that the content is such that it works for users, otherwise we’ll have the first point of resistance here and some of the systems should be such that it’s flexible and can be changed for user requirement.’

The clinical manager argued here that professional agreement was an integral factor in successful implementation, and that this agreement had to be reached early, preferably in the planning stages. He also implied that while agreement on the general use and purpose of the system should be as universal as possible for maximum impact, there should also be scope to tailor such a system to the needs of specific professional groups. This need for a balance between universality and user-specific requirements emerged frequently in the data, in many different contexts. Here, however, it was clear that collaborative working must exist in the first instance if such a balance was to be attained.

Interviewees frequently cited collaborative working as an important facilitator to technological integration. The former telemedicine manager asserted:

‘Technology is not an issue per se. I think cost is not an issue per se. I think the biggest hurdle and the biggest thing to overcome is get the people to talk to one another. You need two people. We made the mistake at the beginning. We had one person. The people at the other end were not too keen. So you cannot play by yourself. You have got to play with somebody else.’



The implication here was that without collaborative working in the first instance, other factors such as technology and cost were of less significance, since the system was less likely to get to a stage in its development and continued usage where these factors became the primary concern. Several interviewees expressed the same sentiments in similar contexts - a system could not be successfully implemented unless those whose work was directly affected by the implementation were collaborative in their will to use it. Collaborative working was essential to the successful relational integration of a system. If it was to be utilised fully by all, it was undoubtedly beneficial for the relevant professionals to have a sense of shared ownership of the system.

4.5.3 Standards

a) Universality.

The need to balance standardisation with flexibility of individual requirements was a recurrent theme. A GP in a task group described the delicate balance between universal standards and individual requirements:

“………. with NPfIT, the government wisely decides to produce a one single unified system, it’s not a problem so long as that unified system can be tailored and bespoke to whatever, however you use it, be it the hospital, be it clinicians, nurses, admin staff…………...’

Any system that was overly rigid was also restrictive to both users and patients. Once again, this tension between the need for similar working practices to ensure compatibility between systems and collaboration between individuals and organisations using them, and the ability to tailor that system to individual needs was evident.

As with the theme of collaborative working, there was an issue here of both the interactional workability and relational integration of a system. For a system to be successful it must be workable for the individual user, but relational in terms of professional confidence in its ability to work with the systems of other users. There was also an aspect of contextual integration, in the broader political will to standardise systems.

b) Clinical risk, legality and safety.

Risk, legality and safety were frequently posed as complicating factors when using e-Health systems. One consultant explained how many clinicians had trouble remembering the passwords that they were compelled to use. This had led to the unsafe practice of writing passwords down, and some consultants were simply not using the systems because they could not remember all their passwords. Hence, while these security measures were a standard means of reducing the risk of unauthorised access to data, they could create new security problems, and deter people from using the systems.

Two nurses illustrated another aspect of risk and professional standards:

‘N3 You get the regular caller who knows exactly what they say, but because they say I’ve got a pain that’s going down my arm, etc, they know they will get an ambulance, you have that side of it as well. We’ve had 30



calls in the last week that are rubbish, but they may not be, so you have to send the ambulance, because we can’t see them. We can’t say, you’re lying to me, you have to go with the words they’re saying, so that’s difficult.’

This demonstrated one of the problems faced by health professionals working remotely from the patients. While remote working could reduce waiting times, time spent in hospital, and travel time for both patients and professionals (as explained above by the ambulance service redesign manager), it could also compel the professional to take an unnecessary action due to lack of information. With no physical or visual cues upon which to make a decision, the nurse advisor was obliged to send an ambulance. The comments of a second nurse illustrated this problem further.

‘N2 You can ask to speak to the patient, listen to them, if they’re really gasping, you can hear that. You get patients who say they can’t breathe, or you get a mother that comes on and says their child’s got projectile vomiting, and our version is it hits the wall, their version is it’s a lot. They use words that are incorrect because they’ve heard them bandied around, but for what we mean versus what they mean, they are not correct. We can’t assess colour, temperature, we only have their version.’

This was an issue of different “languages”, as patient and professional understandings of medical terminology may differ. Again, this was an issue of interactional workability which transcended the use of new technologies, but may have been emphasised by them. Attaching different meanings to he same words can lead to an unnecessary course of action. Computer-based decision support systems cannot distinguish between these different understandings.

Informants also frequently talked of safety. GA discussed security at some length with one of the clinical directors for electronic medical records systems:

‘GA Is a computer any less secure than a filing cabinet?’

‘Obviously, in many respects, it’s more secure. The snag is that if there’s one corrupt person in a GP’s surgery then obviously that is a rare phenomenon and obviously relatively a limited number of people. Whereas if you’ve one corrupt individual and they’ve got access to five million records then obviously (there is) the potential for …… loss of data. So, that’s where obviously one has got to build in securities and access and regular checks about who has been accessing. But if there was a robust system that will only allow the patient access and obviously the technology is there to do that with smart cards and appropriate passwords, then obviously, to a large extent, that gets around the issue because the only person who can access it are the primary care team and the patient. And the patient then would be the guardian of the security and would only pass that onto those looking after him if he wished. What’s always, of course, the argument against that is what if the patient was unconscious? Now, that’s a very rare phenomenon for actually people to be admitted unconscious and in that situation there has to be some sort of break the glass solution but obviously,



they’re rare and therefore you could have actual individual monitoring to make sure that whenever that happened we would follow up to see that it was justified.’

This illustrated the two-sided nature of the security surrounding patient data. On the one hand, it could only be viewed by a small number of people, and those who viewed it were clearly accountable. However, the rigidity of security measures could also (albeit very rarely) potentially compromise patient safety if the required information was inaccessible. Hence, the rigidity of standardised and secure record systems may not always be to the patient’s benefit - hence the need for a “break-glass” solution. This was an issue of interactional workability: the system must be secure, but not to the point at which it would compromise access when needed.

Security was clearly an issue of relational integration, both in terms of confidence (that a system was secure, that the given professional had the right to access the data in the right circumstances) and accountability (that professional integrity would not be compromised by an insecure system). This linked to an issue of skill set workability in that the technicians whose role it was to keep the system working safely and securely, potentially shared in the accountability of the medical professional. If a system was breached, it may have been the case that both the health professionals and the technicians were accountable.

4.5.4 Ease of use

a) Familiarity, congruence and ease to learn.

This theme arose both in the literature review and in the original data, since all users of e-Health systems share the concern about ease of use: either a system was easy to use (a facilitator to successful implementation) or it was not (a barrier). Familiarity was seen as beneficial, as a GP in one of the task groups described one of the advantages of the system used in his practice:

‘(The) system is different to some of the others as it’s Windows-based and it’s very good, and with it being Windows-based it’s got potential to share a common language with potentially other systems………….”

The implication here was that if certain aspects of the system were already known by virtue of their general usage, such a system could be learned more quickly. Professionals who were new to the e-Health system would probably have used Windows, so a Windows-based system was likely to be compatible with their established knowledge and expertise.

Interviewees also cited congruence as an important factor. A clinical director:

‘The change to their working lives in terms of sitting down in much more structured manner than what they would have done previously they always structure time in their day to do review of films but it was not infrequently interrupted but not being able to get the films to them and what have you,



but that doesn’t happen any longer…… and for the NHS in Scotland I mean just the fact that we can store things electronically, transfer things electronically compare images etc it’s a major service benefit.’

The implication here was that as well as being congruent with existing working practices, the use of technology could enhance such practices, creating stronger working structures and reducing the time and effort taken to conduct certain tasks. It almost suggested that technology could potentially create a certain idealised structure, allowing professionals to work to their optimum level. As the evidence has shown thus far, the reality was far from as simple as this, albeit the possibility of this ideal being fulfilled was undoubtedly a facilitator to successful implementation.

In general, the ease of use of a system was in its ability to accomplish tasks more quickly, or by offering an extra benefit not possible without the system. All three respondents quoted here referred to the interactional workability of the respective systems. However, while ease of use fitted most obviously with interactional workability, the clinical director’s reference to the structural changes enabled by it showed skill set workability to be an outcome. If the implementation of a new system impacted positively on the structure of working, it was likely to facilitate successful integration. This linked to a further aspect of ease of use: efficiency.

b) Efficiency.

Related to congruence was efficiency. The call centre nurses were in general agreement about certain problems in the efficiency of the programme they used, and its compatibility with the requirement for real-time action:

‘N2 I think there are very good things in there, if you had the time to play with them. For us using it at the speed that we’re expected to use it at, it’s extremely tiresome.’

‘N3 Too much scrolling up and down, down here and over there.’

‘N2 We need a lot of information, but sometimes it’s not in the right place, it’s hard to find, it’s difficult. Somebody showed me how to pinpoint something on a map, and then actually to hone in on it, to give you the actual name, and I can’t remember how to do that now. I went to the person who was showing me, and nobody knows how to do that. …… I still don’t know how to give people directions to their health centres. There’s a lot of information in there, but it’s cumbersome to use.’

What emerged here was the problem of too much information. It was clear that this particular aspect of the system (which told the nurse advisors of the nearest health centres to the patient’s location) could save a significant amount of time between telephone triage and treatment. However, if a large amount of information was instantly available, the nurses and the patients had to filter it themselves. While this may have sounded like a minimal disruption in itself, in a telephone triage service it may lead to patient dissatisfaction. Again, this was an issue of the real-time interactional workability of a system.



The need for an efficient system which complemented existing working patterns was cited frequently by task group members. A pharmacist spoke of his system:

‘It’s also quick and responsive in the main, which again, is easy for people to sign on, they can move through things fairly fast. So the idea is that hopefully it complements our workload rather than heeds it and to me that is one of the biggest strengths of what we’ve got.’

This linked back to the GP’s comment about familiarity with the system. It also suggested that the actual perception of the system being easy to use would lead to a greater willingness to use it. However, whether or not a system was in fact easy to learn and efficient to use, depended in part on suitable support staff being available.

c) Technical support.

Attitudes towards technicians varied, as shown by these comments from a laboratory technician:

‘…the person who is in IT support wants ease of use when something goes wrong they could do something about it, because users think they’re trying to be obstructive, but they’re not, they’re not really. And the person using it who’s used it for a long time, wants it to work in their way, and the person who’s coming in new wants to walk in and they want you to say, oh that’s obvious, I’d do it like that.’

These comments demonstrated how, while technical support was obviously necessary for users of the system, it could also be seen as an intrusion. The use of technology necessitated a reliance on those familiar with such technology, so that technicians become involved in the work of health professionals. This related to the skill set workability of a system: if it could not be consistently used by a health professional without technical support, giving technicians a significant role in the delivery of healthcare. While the health professionals who made up the task groups were on the whole grateful for this input, there was also a certain sense that this reliance on technicians could undermine their sense of autonomy. This linked to another frequently recurring theme within both the literature and the data: professional attitudes.

4.5.5 Professional attitudes

a) Engagement.

The necessity for engagement with health professionals using new e-Health systems was mentioned frequently within the task groups. The following comment was made in a task group.

‘X Absolutely, and that… you kill off the enthusiasm. I’m still so angry that they’ve slowed down such a lot and your very point; is that within a hospital setting everybody listens. It might not be implemented but you are given a reason why it’s not. Whereas nationally; there’s no reason. There doesn’t seem to be any ownership of anything anymore.’



This suggested that a lack of engagement and communication led to an undermining of professional enthusiasm for the new systems. If there was no clear rationale and no recognisable “ownership” of the system, the will of the workers on the ground to use it may have been diminished. This further related to confidence and successful relational integration.

b) Sense of empowerment and relationships with suppliers and designers.

Task group members and key informants were concerned with their relationships with those involved in design and implementation. While task group members were most concerned with having constructive dialogue with managers and implementers, key informants were more concerned with suppliers and designers. However, this desire for a constructive relationship with those involved in the earliest processes of implementation related to a wish for a sense of empowerment on the part of both implementers and users and the ability to influence the development and use of new technologies.

A pharmacist spoke of the need to involve clinicians in the initial planning prior to system implementation.

‘One of the big changes for me, just looking at what succeeded here before, and the way we’re going about it now, is obviously the different cultures, as well. The approach seems to me, 20 years ago, was to get clinicians at the heart of the project. We pulled doctors, pharmacists, radiologists, therapists, but things have moved on and it’s very much more like a project management approach in the treatment of the project. And I’m not sure that the field…test that we’re, even in our own organisation and nationally, gives enough to pull the clinical community on board as well.’

This suggested a link between empowerment and cultural change. As the technologies involved became ever more complex the relationships between the designers, implementers, and users may have become more remote.

Following on from this, a health professional advisor explained how implementation could potentially lead to a sense of disempowerment.

“It was just the whole concept of moving into this unknown, unseen world; I think really was the attitude that was prevalent. I think that it’s the whole concept of finding themselves straight-jacketed. And I found that more through the development of the data standards, where we want to be able to get them to deliver information in a way that doesn’t contain a lot of free text information. It’s squeezing them into boxes, and giving them… they would have a pick list that they would wish to have under particular categories”

This linked back to the rigidity of working structures that a technology could impose. A system which relied on certain algorithms for entering data may not have presented the user with all the variables they required. This is another example of a tension between the need for a standardised approach



to data and the needs of the individual clinician. The implication was that systems may be designed by those whose agenda was to get the best technological solution, rather than the best enhancement to the clinicians’ duties.

Some key informants identified their relationship with the suppliers and designers of new technologies as an important factor in implementation – and one that was not covered in our literature review. One of the clinical directors interviewed explained:

‘To me, the biggest constraint I have is relying on suppliers, which I don't think is on here (in reference to the literature review). My programme, xxxxx, is dependent on the delivery from several suppliers, all of which are critical for success. And yet levers we have in order to move them are minimal. So access to the suppliers is actually one of the biggest problems I have.’

While professional attitudes were obviously related to professional confidence in the system and relational integration, there was also a more general cultural factor illustrated here. The link between empowerment and cultural change, and indeed the political dimensions of relying on suppliers and/or managers was an issue of contextual integration. Implementation had to be understood in the broader context of health policy and management.

4.5.6 Clear rationale.

Though task group members sometimes referred implicitly to the need for a clear rationale, this theme was more prevalent in the key informant interviews. The rationale also had to be locally, not just nationally, recognisable.

A clinical director gave the following example of a clear purpose for an e-Health management system:

‘There are a number of reasons,…..our existing paper systems …. had problems of legibility and these were long standing problems’

In this setting, the clinical director explained clearly why a computerised system was necessary for his service: legibility of patient information in the notes which apparently were maintained by several different professional groups. The initial problem was one of how to decipher handwritten data . This rationale could be appreciated by all concerned and was used to justify standardisation in practice. By implication, such a widely appreciated rationale would inspire at least an initial general confidence in the new system. Hence, a clear rationale for the use of a system was inextricably linked to its relational integration.

As well as justifications which reflected individual and collective experiences, there were also those related to evidence presented to the implementers. This evidence could take the form of published research and/or knowledge from other implementations. The proliferation of this evidence would shape the context in which implementation was to take



place. A health informatics group member, when asked about justifications and knowledge about how implementations had worked elsewhere, explained:

‘I suppose we’re coming back to what is the definition of an e-Health system. If you’re talking about distant systems, tele-medicine systems and so on, then there’s quite a lot of work in the Nordic countries about the effectiveness of those. But the whole principle that you’ve just described of something that works well, that’s particularly the American experience because the Americans will build a system in their university hospital and, because they’re building it in their local hospital, then they’re making sure that it fits their local hospital. This applies both to electronic medical record systems and the distance orientated systems. It would be the same if you built a system for your local health environment; it would work very well because you’d be sitting and talking to them and you’d build in a way which fits with their ways of working. But if you then tried to transfer it to xxxxx, it wouldn’t work half as well.’

This individual referred to evidence available about successful implementations in other contexts. He explained that certain systems had been purposefully developed for very specific contexts. The implication was that, while this would produce evidence of a successful implementation, it would not necessarily be universally applicable. Hence, while evidence of previous successes could act as an initial rationale for implementing a specific system, it was not necessarily the best rationale. Once again, this emphasised the tension between universal standards and individual needs.

4.5.7 Cost

a) Time, convenience and physical space.

There were two essential aspects to the theme of cost, the first of these being that relating to the cost in terms of working practices; time, convenience and physical space. A task group GP explained concisely the impact of a “paperless practice” on the physical work space.

‘Well, we’ve only come to computers over the last six years or so and the one thing that we, we need a bigger building already, so we can’t be as flexible… it means you can’t work anywhere unless there’s a workstation there, and so you can’t just sort of sit in the corner of an office and do some paperwork. You need the computer.’

This showed how the presence of a new technology altered the working practices of health professionals. The implication was that the necessity to use a computer led to professionals being confined to a particular work space for certain tasks, and that the need for these particular spaces led to the need for an increase in the overall space available (which in turn had financial implications).

While the change in the structure of working related to skill set workability, the related change to the actual “hands on” work of the professional was interactional. A nurse in the same task group complemented the GPs comments regarding physical space by stating that “you can’t just use a pen now.”



b) Financial cost.

The financial cost of a system was less important to the professionals than it was to the implementers. As a task group doctor explained:

‘Cost is only relevant if that cost if we as a trust save X amount of money by decreasing x-rays, which means we can spend it on something else. If you save money and nothing happens, then cost becomes irrelevant. And cost seems to be irrelevant to this, because they seem to be throwing money left, right and centre, and if you can do something cheaper, one they’re not interested, and two they won’t give you refund anyway. So cost has become less relevant whereas it shouldn’t have.’

Here, the doctor demonstrated a belief in the apparent indifference with which cost was regarded by the trust. This belief was reflected further in the key informant interviews. It was interesting to note that while financial cost was undoubtedly an issue that key informants were aware of, many explained at some length that it was not as important as might be supposed, at least not to the users of the systems. One of the clinical directors explained:

‘I think as far as constraints on the ground as far as the health service is concerned, costs, although it's very important for the people who are managing the NHS, for the user isn't an issue, because they're not paying for it. So I think that, where it used to be an issue in general practice, even that's gone, because we've taken over the funding. So actually cost, where it was a lever, is now not a lever and people take it for granted.’

The clinical director described how the ownership of financial cost had, in some respects, shifted, particularly in relation to general practitioners, whose funding had been taken over by the health trusts. Hence, while the financial cost of a system was once an issue that was a concern of many involved in new implementations, changes in the management of finance within the health service, combined with an implied proliferation of innovations, had led to an increasing distance between concern with cost and the actual system users.

The informant from the health informatics professional group brought together all these points:

‘Well, that’s one of the other points I was going to raise because you’ve got a box for cost without a lot of description about it later on and of course cost covers a multitude of sins. And what nobody ever seems to think about is the total cost of ownership. That’s actually the only real way to make an effective judgement about cost. And, again, it’s slightly changed in England now because of the national program, although I’m not so sure it’s changed quite so much in Scotland. The cost, because it’s being born by the health service, is less of an issue, but of course it is for the health service staff in PCTs or whatever who are implementing it. And they tend to look at just



the headline cost and of course that’s not an efficient or effective way of ensuring adequate implementation.’

This respondent here linked cost in terms of time and space with the financial cost to an organisation; the “total cost of ownership”. He described the different levels of concern regarding cost between implementers and users. His assertion as to the “headline cost” implied the availability of prior knowledge of the overall financial cost, contrasted with a lack of how best to distribute such, and the broader implications for organisational structures. Hence, the contextual aspect of the overall financial cost of an implementation was inextricably linked to the cost in terms of the workability of the organisation and individual users.

4.6 Results Using the NPM as a Coding Frame.

Data from the transcripts were divided into 712 discrete attributive statements relating to implementation issues. These were then coded using the four constructs of the NPM described below.

Interactional Workability (IW)

As in the previous section items coded to the interactional workability construct included generic ease of use issues but also issues relating to whether technologies being implemented were deemed to be “fit for purpose” for users, or not, in terms of whether they facilitated their work.

An example includes this comment from one of our task groups:

‘I’m a user and …you’re right I could get any system in the account, but I can’t use that account on my computer. I have to go to a separate computer. So then I have to carry two computers around to do that. It’s just a waste of resource. So in effect the technology is not sufficiently sophisticated, if you like, to allow us to make use of the solutions that have been deployed.’

Suppliers often say that “technology is not the problem” however, users on the ground, often disagreed and experienced all sorts of practical problems in relation to the use of new system as this interviewee explains:

‘The other thing that has come out for me from that particular application is that the technology was always assumed to be fine. The written spec (company X) had said that it’s all done and dusted and they (had) done some basic testing in the XXX but then when they went out on the ground they discovered that actually firstly it didn’t do as it said on the tin, when they got it to extract records from the real live GP system, the system fell over endlessly and corrupted the data and all sort of things went wrong with the upload, in fact they’ve uploaded very few records already because it was just ill prepared ……………..Interviewee 2’

Such difficulties, relating to the effectiveness of new technologies when put into practice, influence professional perceptions in a negative way. There were many examples showing how systems which failed to deliver as anticipated led to disappointment and disillusion amongst users.



Relational Integration (RI)

Statements relating to this theme included issues of confidentiality, security and confidence in the systems in use. An illustrative example:

‘There was something in it for everybody, really. You know, the nurses got the decision support; so, they felt more confident in their decisions. The training, I think, helped with that as well, the clinical training. The patients, obviously… it was the service they were more interested in, I think, rather than in terms of how the service affected them. I mean, obviously, they hoped having a decision taken by a professional’s a good one anyway. So, I think, the impact of the IT was less obvious to them, it was much more the impact of the change in the service that was obvious.

But, so, for the nurses it really was a major improvement. And, I think, interestingly enough, the doctors felt more confidence in the decisions taken by the nurses, as well.’

This shows that professionals did not necessarily display negative attitudes - a recurring theme from our work is that “professional attitudes” do not represent a “fixed state” or unfounded “belief”. Rather they are shaped by the way an e-Health initiatives affects professionals’ perceptions of their work.

Skill Set Workability (SW)

Skill set workability issues related to training, workload and roles and responsibilities. The comments from respondents illustrated for example that the issue of training and support was a more complex issue than those implementing new e-Health services appreciated:

‘But then, on the other hand, if you say education and training, I think that’s a huge issue as well. I think people don’t really understand how best to use it, to get the best out of it.’

“The basic training in terms of using x is good. But the problem is if you’re only an occasional user, and most of my colleagues use it even less than I do, then it’s very difficult because there isn’t an indoor help system in here. So in all, if you only ever order blood tests when your SHO is on leave, you end up having to phone the helpline, which is bad enough, when it was a x helpline. But now we have this x line who don’t answer the questions. They say someone will phone you. So you phone and just hope they’ll get back to you. And I think I’m probably now comfortable enough with x not needing very much, but for anyone new in our department, because we don’t use it everyday as the busy paediatric service would, the training is all very well, but you actually need to practice.”

‘It’s not a training issue, it’s educating people to understand what the context is, the whole new remit of IT, and the usefulness of it, if there is a usefulness, or the deterrents of it. When you want to come into training, as far as I’m concerned I’ve been on endless technical courses, and I never remember a thing. I go in, I do it absolutely perfectly, I walk out of that



room and I’ve forgotten every single thing. The only time you ever start remembering anything is when you start doing it, and having to do it. So to me, I tend to look at the word training as it’s not really training, but you need somebody to come in and work with you in your environment. Somebody you can go to, oh, how do you do this, and then you go back and try it, and then come back, it’s not working, and can you just show me? And you feel so bad because you’ve asked them to stop what they’re doing, to come and show you, that you learn, and that to me is the real training. I think you can spend fortunes in classroom training, and getting them in, and you don’t get anywhere. Interviewee 13’

This shows clearly how simply providing e-Health “training” courses is perceived as inadequate and not meeting the needs of e-Health users. “On the job” training and support as well as help with “problem solving” which is readily available seems a far greater priority for health professionals in practice.

Contextual Integration (CI)

Contextual integration issues included broader organisational and contextual issues such as:

‘Yes, there's a very interesting dimension there because, where the user groups have a financial driver over the suppliers, it's a very different relationship, if the chequebook is in the hand of the user. And when it's not, in other words a third party operates the contract and the users are at the mercy of that third party and they have no control over the system supplier, that whole dimension changes. So it's quite an interesting one.’ Interviewee 6.

‘So, in terms of, there are many factors within the sustainability in our health care system, such as the shortage of future manpower, clinicians, and so forth, that we just have no alternative but to look at how technology can support delivery of health care. So, while cost is a big issue, it’s the realisation that we have no choice really, but to look at how e-Health can actually help us.’Interviewee 5

This illustrates how contextual integration concerns not only financial resources but wider issues affecting how and why organisations deliver services – such as personnel issues or the challenges being presented by the populations they serve.

Outside the NPM

Issues that fell outside the model included the need for champions and user engagement, as well as strictly technological issues. An example:

‘I think there’s very much a need for evaluation of things particularly with regard to the cost effectiveness. One of the sad things at the moment is that this concept of a new central record like the Spine seems to be, not only attractive, and somehow, politically considered the way forward in England but sadly also in Scotland. And for some reason, I think it’s partly



because the concept of building on a GP record is not acceptable to secondary-care clinicians and I think that’s been very unfortunate. So, I think with regard to objective analysis and assessment there’s very much a need for the same degree of effort going in to evaluating policies and implementations…… as goes into the assessment of a new drug. We would never implement a new drug unless it had gone through very stringent, and yet we seem to be very willing to suddenly adopt health policies without any evaluation at all.’

The distribution of attributive statements across the NPM constructs can be more easily visualised by means of a radar plot as illustrated in Figure 3. The axes represent the numbers of attributive statements and each of the constructs interactional workability (IW), relational integration (RI), skill set workability (SW) and contextual integration (CI) are shown at each of the poles. Table 4 provides the numbers of attributive statements in each category that were used to develop the radar plot. As in Section 3.7.2 these tables and figures have no statistical properties but merely demonstrate the distribution of comments relating to barriers and facilitators across the NPM. It is just a graphical way of presenting this content analysis in an effort to determine whether specific aspects of the model are ignored by health professionals or given particular emphasis, by being frequently commented upon. The findings from this WP reveal that health professionals provide a more balanced view of implementation, touching on issues of interactional workability (ease of use issues), relational integration (confidence, security and safety issues), skill set workability (training, workload, roles) and contextual integration (organisation resources and effort).

Table 4 shows the distribution of codes within each construct from each of the transcripts.



Table 4. Distribution of Coding of Attributive Statements



Skill set workability

Contextual Integration OUTSIDE

Task Group 1 20 12 15 16 5 Task Group 2 13 20 6 20 13 Task Group 3 4 10 4 8 17 Task Group 4 11 16 6 6 22 Task Group 5 5 18 9 23 41 Interview 1 2 5 3 3 9

2 4 4 6 3 11 3 8 2 2 5 4 4 2 7 6 3 7 5 7 9 10 16 17 6 3 4 1 5 3 7 1 4 2 0 10 8 2 4 1 3 5 9 1 7 4 4 8

10 9 2 2 1 6 11 1 6 3 9 5 12 1 1 2 11 7 13 4 6 3 7 5 14 0 3 4 8 3 15 0 3 1 0 8 16 2 5 8 1 4

100 148 98 152 210

Figure 3. Distribution of WP2 Coding Against NPM

Distribution of WP2 attributive statements across the NPM

0

50

100

150

200Interactional Workability






This clearly shows that the health professionals gave a more rounded view of implementation barriers and facilitators than the literature review as their comments covered a broader range of issues relating to socio-technical as well as organisational concerns. Figure 3, above, illustrates how the comments address all four constructs of the model.

4.7 Discussion.

In this work package, we collected an array of professional views on the barriers and facilitators to the implementation of e-Health systems. These findings suggested that, though there were strong correlations with certain themes in the literature review (such as professional attitudes and ease of use), there were also significant differences. First, professional “attitudes” did not simply represent resistant behaviours but often reflected experience of using e-Health systems. Second, the professionals’ need for a clear sense that a system would benefit patients (and not just themselves) was a prominent theme. Third, cost was considered in a broader sense, explicitly including staff costs and the total cost of ownership idea, while the literature, for the most part, assumed cost to be primarily a question of (financial) resources.

The themes which emerged from the task groups and interviews covered different, and in many instances, broader ground than those which emerged from the literature, and were obviously more personalised.

Mapping the emergent themes onto the Normalization Process Model was particularly useful in demonstrating the relationship between themes and how the various themes and sub-themes could be recognised as part of a broader organisational process. Table 5 maps each identified theme onto the NPM:

Table 5. The Normalization Process Model applied to emergent themes in task group and interview data.



Interactional Workability.

Relational Integration.

Skill-set workability.

Contextual integration.

Patient Benefit.

Remote working enabling better professional/patient communication.

Confidence in the system shared by professionals and patients.

Impact on working practices and roles; e.g. paramedics being able to do more outside the hospital.

Linked to a general idea of patient expectation; e.g. NHS 24 receiving a “bad press”.

Collaborative Working.

Compatibility and incompatibility between systems.

Collaborative working between professionals and organisations generally.

Standards.

Problems in accessing secure patient information.

Confidence that the system is safe and secure.

Involvement of IT technicians in formulating clinical standards.

Political will to “standardise”, broader societal concerns with data protection.

Ease of use.

Ease of use of the systems, familiarity and compatibility in general.

Changes to working structures: a more “structured structure” enabled by technology.

Professional attitudes.

Confidence in the system and the implementers, suppliers and designers.

Link between empowerment and cultural change.

Clear rationale.

Clear information given to health professionals.

The availability of evidence.

Cost.

Time and convenience: “you just can’t use your pen now”.

Change to working spaces: “you can’t just sit in the corner of an office and do some paperwork. You need the computer.”

Financial cost to the organisation.



Table 5 re-iterates the relationships demonstrated in the results section of this report; how the themes identified were recognisable as part of a process of normalization, or indeed hindrances to such a process.

On considering these themes, we drew two conclusions which relating these seven main themes to one another. These two meta-themes emerged because they were subtly and at times blatantly constant factors within the initial themes. They were less identifiable in the initial analysis of data as they were expressed in many different ways. It was through the exploration of the seven general themes that their presence became apparent.

1. The tension between universal standardisation and individual need.

2. Broader cultural change.

The tension existing between the desire for universal standardisation (for the sake of efficiency, compatibility and communication) and individual specification was a major issue. It was at times blatant, as in the theme of standards and universality, and at other times more subtle, such as when it emerged in relation to ease of use, collaborative working, and security. It was an issue of interactional workability in terms of its impact on the system with which the individual professional worked, relational integration, in terms of the affect on the confidence (either positive or negative) of the professional users, of skill set workability, in terms of the role-blurring potentially created in a standardised system, and of contextual integration as it related to a broader political will to standardise practice.

Broader changes in society and culture which had led to the current policy of e-Health implementation were referred to implicitly by task group members, in the terms described above, such as the desire to standardise, the expectations of patients surrounding technology and the friction between expectation and reality. However, key informants spoke more explicitly of societal and cultural change as a factor in implementation. This was most apparent when referring to cost, rationale and professional attitudes. The theme of culture was most obviously a factor in the contextual integration of an implementation. Changes in cultural expectation, influenced the interactional workability and relational integration of a system, in that the use of technology was all but unavoidable, and accompanied by a managerial expectation that its presence would lead to a more efficient and better service. Furthermore, the use of standardised technology amongst professionals had led to a more general cultural acceptance (albeit in some cases a reluctant acceptance) of a blurring of their roles; hence, the skill set workability of a system was also related to cultural change.

The additional direct coding to the NPM followed by examination of how the data was distributed across the model led to a richer analysis as it immediately showed that the health professionals provided a more complete picture of implementation than reference to the published literature alone permitted. Professionals did not focus so much on the organisational issues such as resource allocation and policy, instead paying greater attention to socio-technical issues such as usability, accountability, confidence, workload and support issues.



Methodological Issues/Limitations

We did experience difficulties in recruitment as discussed in Section 4.3.3. However, our task groups and interviewees covered a broad range of professionals, at different locations across the UK, and with different experiences of e-Health services. By the end of the interviews no new themes appeared to be emerging. Furthermore, a major strength of this work package was our dual approach to analysis which was undertaken rigorously, our strong theoretical underpinning and the discussion of our analyses within a multidisciplinary team. Such an approach to analysis adds robustness and increases confidence in our analyses and findings. Our analyses have helped to identify the main barriers and facilitators to the implementation of e-Health, as well as factors which would facilitate the successful establishment of these implementations and the normalization of an e-Health system.

This work-package allowed us to identify a broader range of implementation issues than the literature alone had provided. Importantly, direct coding to the model also illustrated areas highlighted by professionals as important that did not fit within the NPM, in particular, this included issues such as clinical engagement, user involvement, technical issues, and evaluation and appraisal of the benefits, if any, of e-Health systems. This therefore highlighted aspects of implementation, which the NPM does not consider, but that are still clearly relevant. This was less clear through use of thematic analysis alone. Thus our dual approach to analysis added value.

Recommendations stemming from these analyses are offered below.

4.8 Conclusion: Recommendations for Implementation.

1. The chances of a successful implementation will be improved by the establishment of an ongoing three-way dialogue between designers, implementers and professional users.

Such engagement is essential to maximise the interactional workability of new e-Health systems, exemplified by “ease of use” issues - the ability of an e-Health system to help the user in accomplishing defined tasks. Furthermore such interactions are needed to ensure that professionals can have confidence in new systems and also so that new systems are designed to be flexible and meet the workflow or “relational integration” needs of professionals and their organisations. Only once a system starts to be operationalised, will some difficulties in use be identified, hence the need for ongoing interaction between designers, implementers and professional users.

2) It is essential to communicate a clear rationale for implementation of any e-Health service, in terms of both its professional and patient benefits, in order to promote uptake and utilisation.

New e-Health services are more likely to be successfully implemented if system users, in this case the health professionals, have confidence that the effort required to get the system into everyday usage is balanced by the likely benefits not only to themselves but to their patients. Potential



benefits need to be perceived to have relevance from both a national and local perspective.

3) It is necessary to establish a balance between individual need and standardisation for the purpose of compatibility.

If arrived at consensually, this would lead to a broader, more enthusiastic uptake of a system by professionals.

4) Education, training and ongoing support is crucial.

Importantly, such training is not solely about how to work the system, but also about increasing appreciation of anticipated benefits and limitations of the system as well as how to use any system to greatest advantage. Such support also needed to be long term rather than one off.

5) The safety/reliability of any new e-Health system must be clear.

For example, a back-up plan should exist if a system fails or “locks a user out” in a medical emergency. Health care is unique in that a system failure can literally have “life or death” consequences unlike so many other sectors where new technologies have been introduced. Health professionals are particularly cognisant of medico-legal issues and the “risk” element of their work and are therefore understandably intolerant of system failure. Professionals implementing any new technology must first and foremost be convinced that it is a “safe” thing to do. Furthermore, health data is considered particularly personal and sensitive and therefore data security is seen as an even higher priority than in many other sectors.



5 Work package three (WP3) Development and Validation of the Technology Adoption Readiness Scale (TARS)

5.1 Background

Approaches to understanding innovation implementation have begun to focus on the concept of ‘readiness.’ Although, much of this work has centred on telehealth, we believe it is still has important implications when considering implementation issues in the broader field of e-Health. In the context of tele-health, readiness has been defined as ‘the degree to which a community is prepared to participate and succeed in tele-Health, and is the cognitive precursor to behavioural resistance to, or support for, change’ (Jennett et al. 2003a). A number of models have been developed (Jennett et al. 2005a; Jennett et al., 2003a; Jennett et al. 2003b; Oliver & Demiris, 2004; Overhage et al. 2005; Snyder-Halpern, 2002) that identify both different stages of readiness and different levels of assessment of readiness. In terms of stages, Jennett et al. (Jennett et al., 2003a) specify four types: core; engagement; structural; and non-readiness. Campbell et al. (Campbell et al. 2001) classify readiness into ‘fertile soil’, ‘partly fertile soil’, and ‘barren soil’ and suggest strategies for implementation that are stage appropriate. Levels of assessment of readiness that have been studied include patient, public/community, practitioner, and organisational levels. The majority of these studies focus on practitioner (Hebert et al. 2002, Paquin, & Iversen, 2002; Jennett et al., 2003a; Oliver & Demiris, 2004; Snyder-Halpern, 2002) and/or organisational readiness (Jennett et al., 2003a; Oliver & Demiris, 2004; Overhage et al., 2005; Snyder-Halpern, 2002; Hebert et al., 2002; Lehman et al. 2002), while practitioner readiness has been assessed mostly in relation to tele-Health in rural contexts. Summarising this literature, Jennett et al. (Jennett et al., 2005) identified three themes common to these models: an appreciation of the practice context; strong leadership; and a perceived need to improve practice. All of this research was conducted either in Canada or the United States of America, and that which focused on tele-Health was conducted in rural contexts where the needs of rural providers have been emphasised. Jennett and colleagues emphasise a need for readiness tools to be developed and utilised in the context of tele-health (Jennett et al., 2005).

The models referred to above identify a range of factors that relate to readiness to use information technology in healthcare (including tele-Health). The studies which are potentially most relevant to the present study have all focused on tele-Health or telemedicine (rather than technology innovation in general), and have included the perspectives of health professionals (Jennett et al. 2005, Hebet et al. 2002 and Campbell et al. 2001). However, these models, whilst rich in content as they have been developed through thematic analysis of qualitative data, have not been extended to the development of quantitative tools for assessment of staff perspectives. The research on organisational readiness has led to the development of a small number of quantitative instruments for assessing readiness. This includes the ‘Organisational Readiness for Change Model’ (ORC) (Lehman et al., 2002); the ‘Organizational Information



Technology/systems Innovation Model’ (OITIM) (Snyder-Halpern, 2002); and a questionnaire (Oliver & Demiris, 2004) who drew on both the ORC and the OITIM. However, with their primary focus on organisational issues, these instruments were not suitable for assessing health professionals’ perceptions of the impact of new technology on collaborative working. The latter approach is the objective of this work-package, thus reflecting the underlying propositions of the NPM as outlined in Section 2.

5.2 Aims/Objectives

To develop a structured, predictive instrument to test the contextual readiness of a health care setting for uptake and routine use of a specific e-Health system. The notion of ‘contextual readiness’ refers to a state of readiness for practice that is based on the interaction between individual and organisational factors affecting use of the system, rather than on individuals’ intentions to use a system. It is intended that the instrument developed in this project could be used to identify staff perceptions of factors related to the collaborative work required for the normalisation of particular e-Health systems, as set out by the Normalisation Process Model (outlined in Section 2 and Appendix 1).

5.2.1 Structure of the Work Package Report

This work package undertook the development and testing of a quantitative instrument to assess the readiness of a health care setting for the uptake of e-Health technologies by health professionals in their work. There were three phases of this work package, which are detailed in separate sections:

1 Instrument development processes that drew on the NPM, existing literature and a survey of experts in the field;

2 Development of a generic instrument of factors for rating relative importance and the testing of this with a sample of health professionals; and

3 Development of a specific instrument to elicit ratings of specific e-Health systems and testing of this in two different NHS contexts of e-Health use.

5.3 Phase 1: Item development & expert survey

5.3.1 Aim of phase 1

Phase 1 aimed to develop an initial set of items for inclusion in TARS and to assess the perceived relative importance of these items using a sample of experts knowledgeable about e-Health implementation.

5.3.2 Methods

We used two methods to achieve the aim: (i) construction of items representing factors (barriers and facilitators) affecting the normalisation of e-Health; and (ii) validation of the items using an online survey of experts.

5.3.3 Item construction and piloting

Item construction began by generating potential questions reflecting the four constructs of the NPM . (TF, CM). This involved translating the



constructs into plain language statements, each of which having a single and comprehensible meaning. For example, a direct translation of part of the CI construct in the NPM might be the statement ‘…… the extent to which organizational effort is allocated to an e-Health system in proportion to the work that the system is intended to do.’ Apart from the complexity of the language, the construct is multi-dimensional, so it was necessary to express it in more simply worded statements, such as ‘sufficient organisational effort has gone into supporting the system’ and ‘the rewards of using the system outweigh the effort’. This process of translation was necessary to devise theory derived statements for rating as stand-alone questionnaire items.

The theory derived statements were mapped (TF, CM, GA) against the thematic findings of the scoping review in WP1. This confirmed that the NPM derived statements reflected adequate coverage of key factors affecting the implementation of e-Health as identified in the review and no additional items were required. This process resulted in 23 items for rating. The draft item set was then circulated amongst the research team for critical review and comment, and as a result, amendments were made to develop a final set of 27 rating items to be included in the online survey.

The item set was pilot tested as a live link by members of the project advisory group (n = 5 of 10 invited). This resulted in several refinements to the questionnaire, including technical features concerning its delivery, content of text descriptions accompanying questions, and the revision of two factor items that were unclear to testers. .

5.3.4 Online survey of experts

The purpose of this stage was to collect expert views about the relative importance of the factors being considered for inclusion in TARS

Sample.

This consisted of experts on e-Health implementation issues - defined as the authors of published reviews of e-Health identified and included in the scoping review in WP1.

Recruitment.

We used a database containing details of the authors of the WP1 scoping review papers, especially email addresses. This resulted in a database of 203 potential respondents. It soon became apparent that a larger pool of potential respondents was required to achieve an adequate response rate (see Figure 4 below). A second database of authors was compiled through a bibliographic database search (ISI Web of Science and Medline) for papers published in the last five years that contained the word ‘review’ in the title and specified at least one of four keywords: telemedicine; tele-Health; telecare or e-Health. Records were scanned manually to ensure inclusion was based on relevance to the objective of the survey (i.e. a review paper, in the topic area, and addressing implementation issues). This process resulted in an Endnote database of 126 references, which was reduced to 116 potential respondents after removing duplicates with the initial sample database. Email addresses could be located for 105 authors in this second database, resulting in a total pool of 308 potential participants.

The approach to recruitment differed between the two databases. Database 1 authors were emailed an invitation to take part in the study, which contained a personalised link for completing the survey (to enable response tracking for targeting reminders). This led to many unused survey links due



to invalid email addresses, so Database 2 authors were first sent a personalised email inviting them to ‘opt in’. To these we sent a personalised link to the survey, as we had done for Database 1 authors.

Reminders

For Database 1 authors, targeted reminders (i.e. to non-responders) were issued twice, at approximately 10 day intervals. For Database 2 authors, those who had not responded two weeks after receiving a link were sent a reminder, specifying a closing date for the survey that was one week later.

Data collection

Participants were asked to rate the importance of each item to the routine use of e-Health, using a scale in which 0=not at all important; 1=some importance; 2=moderate importance; 3=very important; 4=extremely important; with the option of choosing 'don't know'.

The survey also invited respondents to suggest further factors that they felt were not covered in the questionnaire, using a free-text box. Online data was captured automatically from the website, in excel format, and converted to formatting for analysis with SPSS.

Analysis

Analysis aimed to assess the relative importance of items, and possible redundancy between items within the set. Quantitative and qualitative data analysis was undertaken. Quantitative analysis was descriptive, examining frequency distributions, means and standard deviations (means were calculated to exclude ‘don’t know’ responses) and inspection of correlation matrices (based on Spearman Rank Order Correlation co-efficient analysis). Although the data collected are categorical and not normally distributed, means were used to give an indication of relative importance for decision-making purposes (explained in Phase 2).

Qualitative analysis was conducted on free-text comments about participants’ perceptions of ‘omissions’ in the coverage of the items in the survey. Using thematic analysis, these comments were analysed in relation to (i) the constructs of the NPM, and (ii) the questions that were already included in the survey.



5.4 Results

Figure 4 shows the pattern of response to the survey of experts conducted in Phase 1:

Figure 4. Response to survey of experts

SAMPLE 1

SAMPLE 2

203 emailed 105 emailed

26 Undelivered

177 presumed received

75 presumed received

30 Undelivered

32 agreed in principle & sent link (43% of

delivered links)

8 Explicitly Declined

Completed = 42

(24% of

delivered links)

Completed = 2

(Sampling

phase

unknown)1

Completed = 19 (59% of those who

agreed; 25% of

those receiving

initial invitation)

Total completed = 63 (24% of those receiving invitations)

5.4.1 Description of sample

The characteristics of the sample, in terms of location of residence, research background, sex, and distribution of own work across the four domains of e-Health are presented in Appendix 11. This data shows that location of residence is relatively consistent with country of origin of review papers in the field. It also reveals that the authors had diverse backgrounds and broad expertise in e-Health research.

1 This was due to a temporary technical glitch on the survey site whereby the participants’ email address was not captured and thus their sample origin couldn’t be determined.



5.4.2 Data analysis/results

Quantitative data.

Descriptive data for the set of 27 factor rating questions are presented in Appendix 12. The questions are presented in descending order from the highest mean rating values to the lowest. Across rating items, the frequencies reveal that, although skewed towards the high importance end of the scale, there was some spread of scores across the full range of response options. Although data is not normally distributed and thus not ideally treated as continuous data, the magnitude of mean ratings on individual items indicate the experts’ perceptions of the relative importance of these factors to the routine use of e-Health. For example, item 3 ‘impact of the system on existing ways of working’ received the highest rating, followed closely by ‘ease of using the system’. The contextual integration (CI) items (items 1-4) were all rated in the top 7 positions (rating from highest to lowest). The bottom positions (lowest ratings) appeared to be dominated by relational integration (RI) items, though this may in part be due to the relatively higher number of items representing the construct as well as the more detailed nature of the items. Interactional workability (IW) and skill-set workability (SW) items tended to fall in the middle to top end of the ratings table.

Correlations between items were low to moderate, indicating little redundancy. Correlations of r>0.5 were considered when deciding whether to exclude or combine items in TARS in Phase 2. Correlations exceeding r=0.05 tended to occur within (rather than outside) sets of items representing the NPM constructs of contextual integration (CI), skill-set workability (SW), relational integration (RI) and interactional workability (IW) with the exception of some of the RI items (eg ‘availability of users’ knowledge of clinical effectiveness of the system’ and ‘availability of technical expertise’) correlating moderately with items in the IW set2 . Although subject to sample size limitations and lack of statistical power, this pattern of relationships provided some preliminary support for the face validity of the items with reference to the constructs of the NPM, as items within construct sets tended to correlate more highly with each other than with items outside their own construct set.

Qualitative data.

Appendix 13 presents conceptual analysis of factors underlying the free-text comments made by (n=31) participants. In general, ‘additional’ factors stated by respondents were usually more specific factor statements than the questions included in the survey and as such were considered to be represented. The exception was for the construct of ‘contextual integration’, as the existing items in the survey did not appear to adequately capture the breadth and specification of factors suggested by participants that could be classified as relating to this construct. This information was used in Phase 2 to create additional items for inclusion in TARS.

5.4.3 Key messages from Phase 1

Phase 1 represented a key stage in the initial development and validation of the factors to be included in TARS. The results (i) confirmed the importance of items within the set that we had developed, as perceived by experts within the field of e-Health; (ii) suggested that these items were

2 Selected correlations are reported in Phase 2 where they were used for decision-making about items.



conceptually valid with respect to the NPM; and (iii) led to the identification of other factors to include in the TARS that were perceived, by experts in the field, to be missing from the existing set.

5.5 Phase 2: Development of TARS

5.5.1 Aim of phase 2

The second phase aimed to:

(i) refine the set of factor items relevant for inclusion in TARS based on the results from Phase 1 (TARS Generic); and

(ii) test the generic TARS items (and the mode of delivery) with health care professionals.

5.6 Developing TARS Generic

5.6.1 Methods

Data collected in Phase 1 was used to make decisions about excluding or combining existing factor items. This process began by examining each question in terms of (i) the mean rating of importance for that item, and (ii) any correlations between the item and other items in the set (Appendix14). Mean ratings along with matrices of inter-item correlations (correlations of r>0.5) were explored in making such decisions. Items that were highly correlated with other items were either discarded or re-written into a single item, particularly where importance ratings were relatively low. This process reduced the 27 items to 21, which were circulated amongst the research team for comment.

This peer review resulted in further revisions to the item set including: (i) three new items to reflect further development of the NPM since the study commenced (Q.29-31); (ii) refining the wording of items to improve clarity (e.g. Q. 16, ‘obtainability’ of new skills became ‘learnability’); and (iii) distinction of items into separate components (e.g. ‘time spent with patients versus quality of interaction’). Some of these changes reflected data emerging from the perspectives of professionals participating in WP2 at the time, such as emphasis on patient benefit and issues of liability (thus the liability item was retained rather than omitted for TARS Generic).

In addition, the qualitative analysis in Phase 1 indicated some omissions from the factor list that participants had rated, particularly with respect to CI items. Six new items (Q.5-10 in Appendix 15) were added to represent contextual integration factors which the expert survey panel considered to be missing from the existing set of items. TARS Generic is presented as Appendix 16.

5.6.2 Results

These processes resulted in a final set of 31 generic TARS items. As a final check on the appropriateness and coverage of the 31 items, the full set was mapped against the major themes emerging from preliminary analysis of the qualitative data collection undertaken as part of WP2 (see Appendix 15, which also displays TARS Specific items as developed in Phase 3). This shows a good level of representation of key factors across the two work



packages. It is particularly noteworthy that the 10 questions added to TARS in the final stage represented themes that were emerging from WP2, which could have otherwise been considered significant omissions from the Generic TARS.

5.6.3 Testing TARS Generic with health care professionals

The second stage of Phase 2 development work involved testing the generic TARS items with health care professionals and testing the proposed mode of delivering the questionnaire. Before developing the TARS further, it was important to test the importance ratings approach conducted with the expert sample in Phase 1, with a sample of NHS staff including health professionals and administrative staff. This was not about assessing “readiness” to adopt new e-Health technologies but to ensure that the items selected for the generic TARS were appropriate to users of e-Health systems.

Sample

A study site was chosen which met these criteria: (i) use of the full range of e-Health technologies that were of interest to this study; (ii) use of such technologies for a sufficient time to provide users with experience of implementation; and (iii) a potentially large pool of participants for the survey. The sample chosen for testing TARS Generic was thus a regional NHS Hospitals Trust which met these criteria.

Recruitment

Participants were recruited through the key contact at the study site. Invitation to participate in the survey was made by emailing an invitation to the study that included a link to the survey website. All staff on the distribution list for the Trust were emailed by the site contact.

Reminders

Issuing of reminders depended on the site contact. A request for a reminder was made 10 days after the initial email invitation. Although the site contact agreed and stated that staff would be encouraged to respond, despite several follow-up emails, this reminder was not issued.

Data Collection

The survey was conducted online using a commercial survey site (surveymonkey.com). The survey itself contained the TARS Generic items, along with background and demographic questions developed in conjunction with input from the site contact to ensure the relevance of the demographic questions. In addition, the site contact suggested an additional TARS item for inclusion, and advised on the framing of introductory text for the survey and for the invitation email.

As for the expert survey in Phase 1, participants were asked to rate each item on importance using a five-point response scale. Participants were asked to indicate which category of e-Health (from the four domains specified in this study) they were thinking of when answering the questions, or whether they intended to respond about e-Health more generally.

Online data was captured automatically from the website, in excel format, and converted to formatting for analysis with the SPSS statistical package.



Analysis

Descriptive analysis (of frequency distributions, means and standard deviations) was undertaken. Correlations (based on Spearman Rank Order Correlation co-efficient analysis) were explored for relationships amongst the TARS items, but not reported due to sampling limitations.

5.6.4 Results TARS Generic

Response rate and sample characteristics.

The response was disappointingly low. Overall, we achieved 51 responses. Sample characteristics are presented in Table 6.

Table 6 Characteristics of TARS Generic sample

Age groups: % (n)

<25 0 (0)

25-34 18 (9)

35-44 26 (13)

45-54 33 (17)

55+ 23 (12)

Working role:

Administrative 41 (21)

Allied health professional 28 (14)

Nursing 24 (12)

Hospital based consultant 8 (4)

Sex

Male 35 (18)

Female 65 (33)

TARS items

Means, standard deviations and response frequencies for the 32 TARS items used in the TARS Generic survey are presented in Appendix 17, ordered by highest to lowest mean rating. The distribution of the responses indicates ceiling effects, in that most items have been rated by most participants as highly important. Interpretation of the meaning of this data is limited by the low response rate. However, the pattern of ranking of importance appears to differ from that of the survey of experts, in that issues of workability (rather than integration) appear higher in the rankings made by professionals. This reflected a similar distinction apparent between the key findings of WP1 (Scoping review) that also reflected emphasis on Contextual Integration issues compared with what the health professionals participating in interviews and workshops were reporting from WP2.



5.6.5 Key messages from Phase 2

Phase 2 developed and tested (with a sample of health professionals) a generic version of TARS. This was a necessary step towards ensuring that TARS would have comprehensive coverage of items relevant to end users of e-Health. Questions could then be developed for the rating of particular e-Health systems by health professionals (in Phase 3), rather than by experts as examined in Phase 1. Although limited by the poor response, the data confirmed that the factor items included in TARS Generic were considered to be of sufficient importance as factors affecting the routine use of e-Health for inclusion at sites where a specific e-Health system was to be assessed. The TARS Generic survey also served as a pilot for online data collection. The poor response indicated that although the online survey is potentially a useful and efficient means of collecting quantitative data, a much more targeted approach with specific user groups and more effective means of encouraging response rates is necessary to use this method successfully.

5.7 Phase 3: Testing and Validation of TARS Specific in study sites

5.7.1 Aim of Phase 3

This section reports the third and final phase of the work package, which involved developing and testing site-specific versions of the TARS (TARS Specific) through data collection at two study sites. In contrast to Phase 2, the aim of this Phase was to test the utility of TARS as framed specifically in relation to a particular e-Health system, amongst specific samples of health professional users of the system.

5.7.2 Methods

Site selection

Phase 3 undertook data collection in two of the NHS contexts sampled in WP2. These sites were chosen as (i) specific e-Health systems were in use by health professionals, that could be the subject of assessment using TARS, (ii) the two sites differed in terms of levels of ‘normalisation’ of e-Health technology thus enabling comparisons of the utility of the TARS instrument being developed; and (iii) access to the study sites had been agreed as part of the project.

At Site 1, use of the e-Health system (community nurses using PDA technology) was relatively new, and provided an opportunity to use TARS in a context where e-Health was still in the experimental stages for some users. At Site 2, the entire organisation is based on e-Health systems – so staff could be expected to have a relatively high experience of e-Health systems.

Development of TARS Specific for study sites

The TARS Generic items from Phase 2 were changed into specific questions. To minimise possible ceiling effects as evident in Phase 2 data collection, TARS Specific used a 7 point response scale representing level of agreement. The two specific versions differed in terms of introductory and explanatory text, and categorisation of work roles for the demographic items, as appropriate for the two sites being sampled. In addition, Q.10



from the full set, about commissioning of services, was irrelevant to site 2, so was omitted. As for TARS Generic, the survey included an item assessing ‘comfort with using computer based technology’. For TARS Specific, two additional questions were included to assess: (i) participants’ perceptions about whether the system was not at all, partly, or completely in routine use; and (ii) their perceptions about the likelihood of it becoming routine (on a 5 point scale). The two TARS Specific surveys are included as Appendices 18 and 19.

Recruitment

As for Phase 2, recruitment of staff into the survey was facilitated by key site contacts who negotiated the necessary permissions and approvals within their organisations to email the participation invitation (containing ethics approved participant information sheet and link to survey) to staff email lists. On approval, site contacts emailed the invitation to staff members within their organisations.

Reminders

As before, it was not possible to send reminders to non-responders as we did not have their email addresses (as we did not have ethical approval). It was thus necessary for site contacts to issue reminders upon request. At both Phase 3 sites, requests for reminders to staff were requested at one week intervals, however in practice these intervals were longer (10 – 14 days). At Phase 3 sites, two reminders were issued following the original invitation, which increased response rates.

Data collection

As in Phase 2, the two surveys for Phase 3 were administered using a commercial online survey service provider (www.surveymonkey.com). Online data was captured automatically from the website in Excel format, and converted for analysis with SPSS.

Analysis

Descriptive analysis (of frequency distributions, means and standard deviations) was undertaken. Correlations (based on Spearman Rank Order Correlation co-efficient analysis) were explored for relationships amongst the TARS items in each sample. Sub-group analyses were conducted using Cross-tab analysis with Pearson’s Chi Square statistic for comparisons of TARS items ratings according to (i) level of perceived integration of the e-Health system into routine practice (both sites); and (ii) professional grouping (Site 2 only). For cross-tab analysis at Site 1, responses to the TARS items were dichotomised into groups indicating non-agreement (responding 0 strongly disagree -3 neutral midpoint) and those responding with various levels of agreement (rating 4-6). At Site 2 (with a larger sample size and different spread of responses), TARS item responses were trichotomised as follows: Disagreement (0-2); neutral or some agreement (3 or 4); and moderate to strong agreement (5 or 6).



5.7.3 Results

Response rates and demographics

At Site 1, 46/243 nurses completed the survey (19% response rate). At Site 2, 231/1351 (17% response rate) completed the survey sufficiently for inclusion in the analysis. Sample characteristics for both sites are presented in Appendix 20.

TARS items

Means, standard deviations and response frequencies are presented in Appendix 21 for Site 1, and Appendix 22 for Site 2. Correlation matrices (based on inter-item correlations using Spearman’s Rank Order (RHO) analysis) were inspected, and for both sites, key findings based on correlations exceeding r=.06 are summarised in Appendix 23.

In relation to the frequency tables, the high number of ‘don’t know’ responses selected by participants particularly when rating contextual integration items such as ‘the e-Health system is adequately resourced financially’ (111/231 participants at Site 2) and ‘government policy initiatives are supportive of this e-Health system’ (106/229 Site 2 participants) is noteworthy. The data indicates that, in undertaking assessment of perceptions using TARS, closer consideration must be made concerning who particular questions are relevant for.

In terms of correlations between items, Appendix 23 shows that the items included in the TARS to assess perceptions relating to Interactional Workability appeared to relate quite strongly to each other, and consistently across study samples. In both sites, similar relationships amongst Skill-set Workability items were evident. Comparing these two sites on inter-item relationships within the Relational Integration item set, Site 1 generated a higher number of high correlations compared with Site 2, but included the key correlations evident at Site 2. In terms of Contextual Integration, again there appeared to be a higher number of high correlations within the data collected at Site 1. All three of the NPM items (q.28 ‘coherence’, q.29 ‘cognitive participation’ and q.30 ‘reflexive monitoring’) related moderately highly with each other. In each sample, some of these items related highly with other items in the full item set, though patterns differed between samples. Appendix 23 also indicates that some items within TARS were either consistently highly correlated or poorly correlated with other items across NPM construct categories. In both sites, for example, the item about efficient use of time seemed to relate highly across constructs (rather than just within RI), and the item about co-operation with others did not relate well with any items across the full item set.

Sub-group analyses

Cross-tab analyses are presented in Appendix 24 for Site 1, Appendix 25 and Appendix 26 for Site 2. For Site 1, due to the relatively small sample, frequencies from within the group (rather than percentages) are reported.

For Site 1, Appendix 26 reveals significant differences between groups perceiving e-Health as ‘partly routine’ compared with ‘completely routine’ in response to 12 out of the 30 items. For the majority of these items, the pattern of relationship is such that those who perceived the e-Health system to be completely a routine part of their work were more likely to agree than not agree with the statements about the system, or to show a higher



proportion within the group responding with agreement (ie overall, they indicated more positive responses). Here, the strongest significant differences occurred on two of the CI items ‘this organization has a culture that is supportive of change’ and ‘this e-Health system fits in with the priorities and challenges of our organization’, along with the coherence item ‘the staff who work here have a shared understanding of what the system is for and how it is to be used’.

For Site 2, analysis of differences between professional groups (Appendix 25) shows that compared with call handlers, nursing and health related staff were more likely to perceive the e-Health system as making them feel autonomous in their work; and were more likely to agree that they understood their own liability for practice. Health professionals were less likely than call handlers to feel the skills required by the systems they used were easily learned; or to agree that using the e-Health system involved spending the right amount of time with patients. Such differences were in directions that would be expected, based on their working roles.

At Site 2, analysis of TARS responses between participants perceiving e-Health to have become ‘completely’ routine in their work (n=174), with those indicating it had either not at all or only partly become routine (n=37) revealed significant differences on nine items (significant differences only are reported). These results (Appendix 26) suggest that compared with those who feel that e-Health has already become ‘completely routine’, those for whom it hasn’t become routine were less likely to agree that sufficient organisational effort has gone into supporting the system; and less likely to show strong agreement (rather than being neutral or some agreement) that e-Health is a different way of working; that the organisational culture is supportive of change; that they understand their own accountability and liability; and that there are ongoing mechanisms for monitoring and appraising how e-Health is used. The group for whom e-Health was not yet a completely routine part of their practice were also more likely to disagree that there is good evidence of clinical effectiveness of the e-Health system, and that there is a shared understanding of what the system is for and how it is to be used. Here, the strongest differences between groups were evident on items relating to liability, accountability and appropriateness of skills.

Key messages from Phase 3

The development and testing of TARS Specific in two practice sites resulted in key findings that contribute to contextual validation of the TARS and its further development.

Representation of NPM constructs

In general, there was a reasonable level of face validity of the TARS items, and support for their representation of the four constructs of the NPM (as suggested by patterns of correlations between items). Further exploration of these items as representing the NPM (in its original and extended form) in further development and testing of the model in data samples of sufficient size for the application of factor analytic techniques, is thus supported.



Relating TARS items to perceptions of level of routinisation

The findings of Phase 3 suggest some evidence of discriminant validity for the items in the TARS survey instrument, in that groups representing different levels of perceived routinisation of the e-Health systems differed in their ratings of some of the items (in the direction that would be predicted by the NPM). This is particularly promising, given that the sites surveyed had already been using e-Health for some time (particularly at Site 2), and that this would have limited the potential for variability in responses concerning whether or not e-Health was considered ‘routine’. Although the data is suggestive only of association rather than causality, this supports the further development and use of TARS in prospective studies where the predictive value of TARS items can be assessed.

Multi-perspective assessment within sites

The findings of Phase 3 suggest that the use of TARS should be sensitive to differences between professional groups. At Site 2, differences between professional groups in the TARS ratings were few, but in directions that would be expected. More relevantly however, the data showed that several items (mainly relating to contextual integration) were unanswerable for significant proportions of participants. This raises questions about how to incorporate different staff groups’ perspectives about a referent e-Health system within an overall assessment of readiness for e-Health.

5.8 Discussion

The primary objective of this work package was to develop an instrument for assessing the readiness of NHS settings for the uptake of e-Health technology, based on the perspective of health professionals and related staff. In developing the TARS instrument, we have drawn on several sources of relevant information concerning factors that affect the take-up of e-Health. In Phase 1, we translated the constructs of the NPM and literature findings into factor items representing issues known to affect the normalisation of e-Health and obtained data on the relative importance of these factors from ‘experts’ in the field. In Phase 2, we developed TARS Generic and surveyed health professionals to elicit importance ratings in order to develop TARS Specific for use in specific e-Health use contacts. Finally, in Phase 3, we translated TARS Generic into TARS Specific and tested it in two NHS sites, which were using different e-Health systems. This section discusses the main findings, strengths and limitations, implications, and recommendations for future work.

5.8.1 Main findings

Turning the NPM into factors

This work package has contributed to the development of the NPM in a number of ways. Firstly, it has successfully achieved the development of a set of quantitative questions that can be used to assess staff perceptions relating to different underlying aspects of the constructs within the NPM, along with pre-testing of single items that may be used to represent quantitative assessment of the constructs of coherence, cognitive participation, and reflexive monitoring as proposed in the extended version of the NPM (NPT) (discussed more fully in Section 7). The data collected in Phase 3 of this work package has confirmed a reasonable level of face



validity of these items as representing the constructs within the model (discussed below).

Assessment of the relative importance of factors affecting normalisation of e-Health

This work package generated information about the perceived relative importance of a comprehensive set of factors known to affect the normalisation of e-Health. The survey of experts in Phase 1 was particularly instrumental in providing face validation of the factors to be represented in the TARS instrument, as well as identifying what may have been missing so that further items could be developed for inclusion in TARS. The results of Phase 1 however, had the added benefit of providing a form of cross-validation between work packages in this project. The findings here were consistent with those of the literature reviews conducted in WP1, in that the experts participating in the survey also emphasised issues of contextual integration over practice-based issues of workability, compared with WP2 which found that health professionals considered workability issues to be more important.

Relationships between TARS items and constructs

The key findings from Phase 3 (Development and Testing of TARS Specific) offered support for the TARS items as reflecting the constructs of the NPM. From the patterns of correlations between items that were evident in both study sites, the relationships between items within the constructs of Interactional Workability, Relational Integration, Skill-set Workability and Contextual Integration were generally of the magnitude and direction that would be expected on the basis of the NPM. The support for the model in this respect warrants further work on factor analysis of items in studies where sample sizes are sufficient for such analysis.

This work package also provided a preliminary test of the relationship between responses on TARS items and perceptions of whether or not the e-Health systems in the respective study sites had become part of routine practice. Despite the collection of data in study sites in which e-Health had been ongoing for some time, several TARS items did significantly differentiate between groups perceiving different degrees of normalisation of the e-Health system they were rating. Although the data can only be treated as representing associations (rather than causality), it lends support to the assessment of the predictive value of TARS in prospective, longitudinal studies that provide an opportunity to assess staff perceptions of a new technology at the initial stages of its introduction to and use within a workplace.

Production of generic and specific versions of TARS

This work package has produced two versions of TARS: (i) TARS Generic, which is presented in the form of factor statements for the elicitation of importance ratings; and (ii) TARS Specific, which presents questions on which professional users of e-Health can rate a specific system based on their experience and their perceptions of the context in which they work. TARS Generic was developed as part of a process towards designing a useable instrument for assessing staff perceptions about a particular e-Health system they are using, rather than as an instrument for use in its own right.

However, the process of developing and using both generic and specific versions of TARS has raised both challenges and opportunities. It has been



intended that TARS Specific could be used to assess staff perceptions of a particular e-Health system, based on some experience of or familiarity with the system. Even with this objective in mind, for use in other clinical contexts, it is likely that the specific framing of the questions within TARS Specific will need to be adapted to appropriately reflect the stage of implementation/use of the e-Health system. For example, if the questions are to target perceptions in relation to a technology not yet used, then they will need to be framed (and interpreted) as the assessment of expectations of the technology. Another possible use of TARS Specific may be to compare perceptions of a new technology with an existing technology in used – in which case questions would need to be framed as including a comparison. Even in contexts where TARS Specific is framed appropriately for use in its current form, the inclusion or omission of certain questions will need to be considered on the basis of relevance to specific staff groups that are the intended recipients of the survey. These issues present challenges for further validation of TARS Specific as ‘an instrument’, but offers a range of opportunities for practical use in assessing staff perceptions of factors that this study has shown to be important for the normalisation of e-Health.

TARS Generic has not been developed with the intention of applied use in the kinds of ways suggested above, however it too may have practice and research-based uses. For example, assessments of staff perceptions of the relative importance of the factors included in TARS Generic may be useful in certain situations, such as collecting information about issues of importance during planning stages for e-Health implementations, and for comparisons of these assessments between different professional groups within an organisation.

5.8.2 Methodological issues

This work package has undertaken considerable instrument development and testing activity with respect to the assessment of NHS staff members’ perceptions of factors related to the normalisation of e-Health technologies in practice. However, several methodological limitations must be considered.

Response rates

In terms of scale development, response rates and sample sizes at survey sites were insufficient to permit the use of statistical scaling methods that would be necessary for formally testing the statistical properties of TARS as a research scale. Indeed, it was anticipated that statistical testing of TARS at this level would not be achievable within the scope of this particular study, and that this would be required in subsequent research using TARS.

Survey approach

The results of this study show that, although online survey research has the potential to achieve the collection of survey data from significant numbers of participants efficiently, several conditions are necessary to achieve this. The role of key facilitators at survey sites is essential, particularly when they are responsible for identifying appropriate staff groups and actively encouraging staff participation. Although all site contacts who facilitated this work package were helpful and encouraging, in the context of their busy working practices, the frequency and timing of reminders to non-participants were compromised by lack of control over this by the research team.



Site selection

The survey sites included in this study already had at least some level of adoption of e-Health technology. Although not an objective of this study, the sites included did not allow for prospective analysis of the predictive utility of either the NPM constructs, or the TARS items in particular in relation to the likelihood of normalisation of e-Health. Further research using TARS and the NPM should therefore be undertaken in longitudinal and prospective research studies conducted within practice contexts where a new technology is about to be introduced.

5.8.3 Comparison of findings with existing literature

The key findings of this work package are generally consistent with the empirical literature (reviewed as part of WP1) in terms of providing support for the importance of a comprehensive range of factors and issues known to affect the uptake and routine use of e-Health. In terms of the existing literature on technology readiness that has been conducted in relation to tele-Health (as one particular kind of e-Health within the taxonomy developed for this study), the themes emergent from that literature are generally compatible with those represented in the items contained within TARS. However, as such studies have not yet tested the importance or relevance of such themes in the form of quantitative survey items, it is difficult (if at all possible) to compare the findings of this study against existing research in this field.

In terms of the theoretical literature, the concept of normalisation as framed by the NPM can be interpreted as referring to both a ‘state’ (in that something is ‘normalised’) and to a process that is dynamic and ongoing. This does raise the question of what ‘readiness’ actually means in the context of technology adoption, and in particular, if measuring ‘the degree to which a community is prepared to participate and succeed in tele-Health…’ (Jennett et al 2003a) is the objective, then timing of assessment becomes a crucial question. In this study, we explored the concept of readiness as proposed by Jennett (Jennett et al. 2003a) in settings in which e-Health had already been in use by staff. It would be necessary to further explore responses to the items included in TARS, in contexts where ‘readiness’ of this kind can be assessed by using it in studies where e-Health implementations are about to be conducted (rather than already underway). However, the finding that TARS items did differentiate between groups perceiving different levels of normalisation of a technology even in contexts where use of e-Health had been ongoing for sometime suggests that a broader view of ‘readiness’ should be taken. We would propose that such assessments of likelihood of success in achieving routine use of a new technology such as e-Health in a practice context are more appropriately focused on determining ‘receptiveness’ rather than ‘readiness’ as has been the focus on previous literature concerning tele-Health, and that the role of factors affecting staff perceptions and behaviours in relation to e-Health should be considered as dynamic and changing across time.

5.8.4 Recommendations for further development & application of TARS

1. Further development of TARS for practical use should involve:

� Specification of different ways of wording TARS items for different purposes, and development of guidance of its use for different purposes and in different contexts; and



� Further research and analysis concerning the appropriateness of (i) different items within TARS alongside (ii) alternative assessment approaches, for assessing and combining the perceptions of different professional groups whose work is affected by the introduction of an e-Health technology

2. Further research concerning TARS and the NPM should involve:

� Testing the statistical properties of TARS as a research scale, in contexts where the necessary requirements concerning sample sizes and response rates can be met; and

� Testing the predictive utility of TARS in relation to normalisation of e-Health technology, by using TARS in longitudinal prospective studies in contexts where assessment of perceptions can be undertaken prior to the introduction of an e-Health system.

5.9 WP3 Outcomes

This work package has achieved key outcomes in relation to understanding the implementation and integration of e-Health in the NHS:

� An instrument (TARS) for assessing staff perceptions relating to receptiveness of e-Health technology within their working contexts that can be further developed and tested in subsequent research;

� Data concerning the relative importance of a range of factors that contribute to staff perceptions about workability and integration potential, from the perspective of staff members themselves;

� Preliminary quantitative testing of aspects of the NPM that contribute to further development of the model (and the subsequent NPT);

� Recommendations for further development and testing of TARS for use in practice contexts; and

� General guidance for undertaking assessments of staff perceptions about the likely normalisation of e-Health



6 Work-Package 4 (WP4) The implementation of e-Health systems: perspectives of e-Health implementers on the integration of new technologies into everyday work.

6.1 Background

Work-package 4 (WP4) focused on implementers, aiming to determine their views on barriers and facilitators to implementation of e-Health technologies, and to develop a toolkit to aid future implementations.

Implementers, defined here as any person who has been charged with an e-Health system implementation, have important insights into factors that lead to implementations being successful or otherwise. At the outset of this project, it was our view that their perspective had been relatively less studied than the views of health professionals, and this view was confirmed by the literature review undertaken in WP1. Hence the data collected in WP4 form a unique and valuable addition to the total literature on implementation of e-Health systems. Some of our interviewees were health professionals, but they were selected for interview on the basis of their role as implementers.

This work-package had two components:

Phase 1 - an exploratory phase to collect primary qualitative data on implementers’ perspectives; and Phase 2 where the primary data collected in WP1 (the scoping exercise) and the qualitative data from this work-package were examined and integrated, using the NPM as the underpinning theoretical framework to develop an implementation toolkit (e-HIT)

6.2 Phase 1 Aims and Objectives

The aim of this first phase was to identify, describe and understand core mechanisms in the implementation of e-Health systems, from the perspective of implementers, using the NPM as an analytical framework.

Definitions

The four work packages used common definitions, as presented and discussed in Section 1.3. As in the other work-packages we have classified e-Health interventions into the 4 domains introduced in Section 1.3 Box 1.

Implementers

In this study, an implementer is any person who has been charged with assisting with an e-Health system implementation. Depending on the policy



level sponsor, implementers may be found at national, regional and/or local levels and may include health service “supremos”, chief executives, clinical directors, senior healthcare managers, ICT staff, health professionals, local NHS managers, staff involved in training, and staff working for private companies contracted to supply, facilitate or support technology implementations. Although our focus was not on health professionals, some health professionals with a lead role in an e-Health implementation were interviewed.

In common with the other work packages the Normalization Process Model (NPM) provided the theoretical framework for WP4 data collection and analysis.

6.3 Phase 1 Methods

Design

A qualitative study, using interviews with a range of implementers drawn from three case studies.

Setting and sample

Identification of case studies

We identified three criteria to guide selection of case studies, with the goal of maximising the transferability of the results by achieving a maximum variability sample. The three criteria were:

1 The technologies in the case studies should include a range of e-Health domains (i.e. management, communication, decision support and information);

2 The case studies should include a range of clinical contexts (i.e. primary, secondary and community care);

3 The case studies should exemplify a range of sponsors of the implementation, as the first version of the NPM identified sponsorship as an important variable.

Finally, we limited our case studies to sites where the implementation occurred after 2004 but before 2006 to ensure that the implementation was both recent enough to remain alive in respondents’ memories, and old enough for it to be apparent whether the system implemented was becoming normalized into routine care or not. A fully normalized intervention is one that is taken for granted and has become completely embedded into routine practices.

Having determined these criteria for selecting three case studies, we undertook a three stage data collection exercise to identify appropriate case studies.

a) We held a brainstorming session with the entire EH94 project group to generate a list of potential e-Health initiatives and a list of people to interview to learn more about each initiative;

b) We searched the Department of Health, Connecting for Health and other suitable websites for information about national and local e-Health initiatives;

c) We interviewed key national informants for their views on which e-Health initiatives should be sampled and why.



Data derived from this three-stage process were combined and discussed at a steering group meeting of all EH94 co-investigators. The decision was made in two stages: first we agreed on the technologies we would study, and secondly we agreed on a geographical location for each technology. Thus each of the final case studies was described in terms of a technology and a location.

Identification of interviewees within each case study

Within each case study, we identified key informants. Once again, we adopted a purposive sampling strategy to obtain maximum variety. We aimed to include implementers from three levels:

1. Senior Department of Health or Connecting for Health staff with responsibility for a number of projects in multiple organisations;

2. Senior staff from within the Trust or Health Board of the case study with lead responsibility for implementing either this particular system or a group of e-Health initiatives (e.g. Chief Executive Officers, Departmental Leads or senior staff from the Local Service Provider (LSP);

3. Middle management, including IT leads, training leads, and others with day-to-day responsibility for the implementation under study.

Data Collection

We collected data through face-to-face or telephone interviews with staff identified through our purposive sampling framework. Semi-structured interviews were used to determine not only “what happened”, but also the implementers’ perceptions of “why it happened”. The content of the interview was informed by the normalization process model, and included.

� A description of the e-Health implementation process the interviewee was involved in.

� The context of the implementation process e.g. in which healthcare sector/s was the technology located? Who was the policy level sponsor of the initiative? Which staff groups were expected to use the new technology?

� Interviewee perceptions of those factors which had promoted, inhibited, or had little impact on, the integration of new technologies into every day work.

� Interviewee perceptions of the degree of normalization achieved (the extent to which it has become embedded in routine clinical practice), and the extent to which this varied between different groups of users.

� The extent to which the various professional groups targeted by the e-Health implementation reported changes to the stability and order of the clinical encounter (interactional workability) and implementers’ awareness of the nature of these changes e.g. what mechanisms existed to report the impact of the intervention on doctor-patient interactions?, what changes were reported within the clinical encounter?, and implementers’ views on the reasons for these changes.

� The extent to which the professional groups targeted by the implementation process reported changes in trust in professional knowledge and practice within networks (relational



integration) e.g. what was the effect of the initiative on professional confidence in clinical practice? How did implementers gain an awareness of these impacts? Why did they think this had happened?

� Identification of the impact of the e-Health implementation on the division of labour within the healthcare organization e.g. how were tasks relating to the e-Health initiative assigned to and agreed by particular staff groupings (skill set workability)? What was the impact upon the performance (competency, quality, autonomy) of those staff in subsequently carrying out this work?

� The extent to which new procedures and systems were used to integrate the e-Health service into the work of the organisation and the impact of these on existing patterns of activity (contextual integration) e.g. What, if any resource reallocations or system adaptations took place? How did these affect service delivery?

The interview schedule for Case Study 1 is shown in Appendix 27. The interviewer also kept contemporaneous field notes which provided additional information.

Data analysis

All interviews were tape-recorded and transcribed verbatim. Interviews were coded according to the main constructs of the NPM. Additionally we searched the data for evidence of the degree to which each system studied had normalized, and for data which could not be coded using the NPM.

Initial data were coded by the interviewer (Jo Burns, JB) and the lead investigator on this Work Package (Elizabeth Murray, EM). The coding framework devised was then tested and refined at a 2 day multi-disciplinary Data Analysis Clinic (FM, CM, TF, GA, KOD, EM, JB). The revised coding framework was re-applied to the previously coded interviews and all subsequent interviews. After the change of research staff toward the end of Case Study 2 EM coded all interviews. At the end of this process, we undertook a final quality check as CM independently coded all data.

6.4 Results

6.4.1 Identification of Case Studies and Interviewees.

Identification of case studies

Ten national figures in e-Health and Connecting for Health (CfH) were contacted for their advice. Five agreed to be interviewed, and provided information on the e-Health initiatives they considered to be of national importance, together with information on the relative progress made by a range of Trusts and Clusters across England.

The three case studies finally selected were:

1 Choose and Book (C & B) in an English early adopter site made up of one hospital and the lead PCT providing referrals to that hospital - Case Study 1 (CS 1);



2 Picture Archiving and Communication System (PACS) in one acute hospital trust. The trust included several hospitals at different sites, and was located in England – Case Study 2 (CS 2);

3 Clinical Nursing Information System (CNIS) designed for District Nurses working across 10 Community Health Care Partnerships within the largest Health Board in Scotland – Case Study 3 (CS 3).

The salient features of the three Case Studies are presented in Appendix 28. We achieved good coverage across our three criteria, with CS 1 involving primary and secondary care as well as the primary-secondary care interface, CS 2 located in secondary care, and CS 3 located in the community. CS 1 and 2 both covered the e-Health domains of communication and management, while CS 3 covered information and decision support. Finally, in CS 1, Choose and Book had been implemented prior to the establishment of CfH and had strong local sponsorship, CS 2 was a CfH priority, and CS 3 was in Scotland and out with CfH responsibility.

The relationship between the case studies selected and the sampling criteria are shown in Table 7.

Table 7 Relationship of Case Studies to Selection Criteria.

Context e-Health Domain

Management Communication Decision Support

Information

Primary Care C & B C &B C & B

Secondary Care C & B

PACS

C & B

PACS

Community Care CNIS CNIS CNIS

Primary/secondary care interface

C & B

Our third criterion was the level of policy sponsor: C & B and PACS both had a national level policy sponsor (CfH), while the CNIS had a local level policy sponsor.

Identification of interviewees

Within each case study, interviewees were selected according to the criteria described above. Ten interviews were carried out in Case Study 1, 5 in Case Study 2 and 8 in Case Study 3. We had pre-set a target of ten interviews per case study, but in both CS 2 and CS 3 saturation was achieved relatively early and, as no new data were being generated in subsequent interviews, we decided to halt the interviewing process. Table 8 presents details of the roles of the interviewees which included regional leads for the Cluster (CS 2) or Local Service Provider (CS 1), Chief Executives for the Trust or Health Board for all three Case Studies, Clinical and or IT leads, and a range of middle management with “on the ground” responsibilities. It can be seen that the intended sampling framework was achieved.



Table 8 Roles of interviewees

Case Study Choose and Book

PACS CNIS

Regional Level Lead for Local Service Provider

Regional Implementation Director for Cluster

Chief Executive CEO of Trust CEO of Trust Managing Director of provider company; General Manager of Health Board

Senior Management

Clinical Lead for Hospital Trust

Clinical Lead for Hospital Trust

IT Manager Health Board; Clinical Services Manager

Middle Management or “on the ground”

GP and clinical lead in PCT; Consultant; Practice Manager; Project Manager for Hospital Trust; Outpatient Manager; Primary Care Director for Hospital Trust

Radiology Manager; IT Manager

Lead Project Nurse; IT training manager Health Board; Senior Nurses x 2

6.4.2 Findings from case studies

Case Study 1: Choose and Book

Choose and Book (C & B) is a national electronic service that provides patients with the opportunity to choose which hospital their GP refers them to for a particular problem, and to book the time and date of their first appointment. Full normalization of Choose and Book would require normalization both in primary care / general practice, and in the hospitals referred to. At the time of our study, Choose and Book had been variably implemented across England, and our study site was an early adopter.

Degree of normalization

It was clear that Choose and Book had normalized to quite different extents in primary and secondary care. It was very patchy in primary care, with the Clinical Lead of the hospital estimating that about 30% of total referrals were electronic with a minority of practices generating most of the electronic referrals, whereas in the hospital, almost all clinical specialties had implemented Choose and Book, and it was well integrated into everyday work in most.



‘I think that 30% of our referrals are electronic. You will find that they are very skewed towards certain practices.’ (CS1 Clinical Lead)

‘I think it [integration] varies a little bit for specialties, specialties such as my own – it is now the majority referral methodology and so it is completely integrated…overall we yes happy with it in the hospital – it is working for us.’ (CS1 Clinical Lead).

‘We have got it in as mainstream business processes. My consultants are used to it ... it is becoming a way of life here…’ (CSI Chief Executive)

‘completely embedded in standard operational workings’ (CS1 Project Manager)


The hospital studied was very committed to Choose and Book as a means of improving their market share of referrals, at a time when the number of referrals from primary to secondary care was falling. The hospital could only survive financially if it could maintain a steady flow of inward referrals, and was in the challenging position of having a number of competing providers within a mile or two, including at least two teaching hospitals. Choose and Book became a central part of this hospital’s business plan, and was seen within the hospital as a very important tool to maintain inward referrals. This view had permeated throughout the senior management of the hospital, who invested considerable effort in making Choose and Book work. This translated into considerable managerial and financial support aimed at overcoming any and all difficulties encountered. In NPM terms, Choose and Book had very high levels of contextual integration from a hospital perspective.

In contrast, Choose and Book had relatively low contextual integration in general practice. At the point this study was undertaken there were no financial incentives for GPs to use Choose and Book, although there were some general exhortations to “offer choice” to patients at the point of referral (low contextual integration). As there was little requirement to use Choose and Book, relatively little effort was invested in making it work, and problems with using it went largely unaddressed. GP practices that adopted it hoped it would streamline the referral process, and reduce the amount of time spent chasing appointments in secondary care on behalf of patients. However, this potential advantage was offset by the amount of administrative time taken sorting out problems caused by Choose and Book.

‘I wanted to make it so easy to book an appointment in this hospital that people would start to use this hospital for booking’ (CS1 Hospital Chief Executive).

‘The potential competitive advantage that direct booking would have over my competitors in encouraging that small percentage of GP practices who are promiscuous referrers to refer into my hospital … use the business leverage …’ (CS1 Hospital Chief Executive)

‘the reason we went for it initially because we felt there would be real advantages to it and it would hopefully streamline the process of referring patients to hospital’ (CS1 GP Practice Manager)

‘… overall our secretary would say she’s got fewer people phoning saying I saw the doctor three weeks ago and I thought he’d made a referral but I still haven’t heard anything, I haven’t had an appointment yet….’ (CS1 GP Practice Manager)



‘If they [the patient] then have a problem, it’s invariably the secretary they come back to. So she’s had to deal with all of those.’ (CS1 GP Practice Manager).


For GPs, the defining feature of Choose and Book was the slowness of the web-based programme. Even GPs who wanted to use it found that it took 3 – 4 minutes to make a referral with Choose and Book, which had a severe negative impact on consultations with patients. GPs perceived themselves as under constant time pressure, and resented this additional task being added to consultations, particularly as consultations which resulted in referral tended to be complex. In NPM terms, Choose and Book had very poor interactional workability, both in terms of congruence (GPs did not think that time spent on booking an appointment was a good use of consultation time), and in terms of disposal (GPs found that using Choose and Book delayed the outcome of the consultation). This negative impact on the consultation and poor interactional workability dominated GP reactions to Choose and Book.

‘Right you are saying within my 10 minute slot and you have said Choose and Book will take a couple of minutes – it doesn’t – what, even two and a half years on it takes at least four and is not even working properly today. So it took me 10 minutes to do one this morning. So you really think that I am going to stop to help people exercise that choice…?’ (CS1 GP Early Adopter and IT enthusiast)

‘And again for people to say that a Choose and Book takes two minutes – it just doesn’t – it is two minutes if you are really lucky, if the wind is in the right direction on the right day. Even if those 2 minutes but still 20% of the common consultation … 20% that wasn’t there before.’ (CS1 GP Early Adopter and IT enthusiast).

Choose and Book had relatively little impact on interactional workability for consultants, except when patients had been booked into inappropriate clinics, which tended to lead to poor consultations (negative impact on IW).


Choose and Book had some negative impact on relational integration (trust and confidence in different professional groups) for both hospital specialists and general practitioners. Consultants were used to receiving referral letters, reading them, and allocating the referred patient to an appropriate clinic, often also allocating a degree of urgency (e.g. urgent, soon or routine). As Choose and Book referred patients directly into specified clinics, Consultants lost the opportunity to direct referrals appropriately, and had to rely on GPs making the right referral decisions. In turn, GPs were dependent on the information provided by Choose and Book, which was often inadequate to guide appropriate choice of a specific clinic. Choose and Book also mediated against referral to a specific consultant, and encouraged GPs to refer to the clinic with the shortest waiting time. This had a small negative impact on GPs trust and confidence in the specialist that the patient was referred to – GPs felt they were referring to a discipline rather than to a known, named, individual specialist. Equally specialists felt that they were no longer developing relationships with individual GPs.

‘… so that we now refer to a generic gastroenterologist or a generic chest physician …’ (CS1 GP Early Adopter and IT enthusiast)



‘ Sometimes a GP hasn’t had so much personal knowledge. I know a GP who’d say Dr P will sort this, he’s a good fellow’ (CS1 Hospital Consultant)..

‘one of the frustrations was and continues to be that it almost pulls referrals into a speciality so actually instead of writing a referral letter to a consultant its more likely through Choose and Book that you end up in an amorphous .. neurology or something…’ (CS1 Primary Care Director for Hospital Trust).

Skill set Workability

Skill set workability was also unsatisfactory for Choose and Book in both environments. GPs felt that booking an appointment for a patient was a clerical task, which should not be taken on by highly trained clinicians. In the hospital setting, implementing C & B required considerable investment in training for outpatient clerks and medical secretaries.

Summary

To sum up, Choose and Book had normalized to very different levels in the hospital setting and in primary care. Our analysis suggests that the relatively high degree of normalization within the hospital was due to the very high contextual integration, with Choose and Book seen as an essential part of a financial survival strategy which had to be made to work. This very high contextual integration overcame any problems that might have arisen due to difficulties with interactional workability, relational integration or skill set workability. In primary care, there was no such overwhelming financial or other imperative. This meant that the GPs perceptions of a negative impact on interactional workability, and low skill set workability led to relatively low levels of normalization in this environment.

6.4.3 Case Study 2: Picture Archiving and Communication System

The Picture Archiving and Communication System (PACS) is a system for digitizing images, such as X-rays, scans or photographs. The digitized images can be stored on-line, and can be accessed simultaneously by several clinicians in different locations. We studied PACS in one hospital trust, in the Region where PACS implementation was most mature. The trust was located across several sites which were widely geographically separated. This trust had simultaneously implemented PACS and a Reporting Information System (RIS). The focus of this work package was PACS, but for many of our informants, implementation of PACS was inextricably linked with implementing the RIS. In order to maintain the focus on PACS, we only present data pertaining to PACS.

Degree of normalization

The PACS implementation was widely seen as successful, and PACS appeared to have almost completely normalized in this study site.

‘It’s fully implemented now. We have had it, we’ve had it just over two years now PACS so you won’t see any images now. I think at all, I think we’re all completely filmless, yes.’ (CS2 IT Training Manager).

‘it’s just taken for granted that you come in and you use PACS and that’s how your images are that’s it…. Just normal practice now.’ (CS2 IT Training Manager).



‘I think, um, it is so into the routine that we couldn’t take it out… It would be impossible to step back.’ (CS2 Radiology Manager).

‘It [PACS] is firmly embedded. If you walk around this hospital two years on, the clinical team wouldn’t even stop to think about what’s they’re doing. It just, there’s not other way for them to get to look at the x-ray image. … No, no it’s truly embedded...’ (CS2 Hospital Chief Executive).

‘Oh, superbly well. I think we could not function without it any longer. Literally its like that, no one, no one would go back.’ (CS2 Consultant Radiologist).

Less successful had been the simultaneous implementation of a Reporting Information System (RIS), which had been dogged by software problems, particularly with the voice recognition component.


As with Choose and Book, the degree of contextual integration of PACS appeared to have been a major factor leading to normalization. PACS was a national flagship initiative, with both policy makers and senior management seeing PACS as an excellent way of improving efficiency within radiology services and meeting demanding waiting list targets.

PACS also has a high contextual integration at a local level. It was seen as a way of obtaining clinician “buy-in” to IT adoption, and the Chief Executive Officer (CEO) of the Trust was determined to make this project successful. The CEO set up an implementation project board, which included a strong clinical lead who was very committed to the project, and the finance director. This project board was able to ensure adequate resources were allocated to the implementation - time, staff expertise, and funding.

‘the main driver was that hard copy film technology was beginning the cause more and more problems… [the films] were never in the right place at the right time. Well never is too strong a word, but I think there were times when we were running up to about 20% lost films. And what I mean by ‘lost films’ is just not being in the right place at the right time. So if you were in clinic … [and the films] were on the ward, but they weren’t in the right place to be used, so they were lost films.’ (CS2 Radiology Manager).

‘… it [the main driver to the implementation] was to improve efficiency, was probably the first and primary thing of information travelling between doctors.’ (CS2 IT Training Manager).

‘Well, nationally it [the main driver to the implementation] was to capture new technology which allowed images to be stored electronically rather than in brown envelopes and moved around at huge human costs. The secondary aim, which I don’t think has been fulfilled, was the easy sharing of information between clinicians on different sites. So that was the national one. But the motivation for me was I was looking for a way of capturing my doctors to get them to understand the power of IT.’ (CS2 Hospital Chief Executive).

‘And he [the CEO] was also, he was also very aware that diagnostic services were going to become very important in the business plan of the NHS….So



he was, I think he was aware of what PACS could do for the organization.’ (CS2 Radiology Manager).


Clinicians, both radiologists and non-radiologists, found that PACS improved encounters with patients. Consultations were no longer disrupted by lost or missing X-rays. Images were available on-line wherever the patient was seen – which was a substantial advantage in this trust which covered several sites. Images could be more easily shared with patients.

‘I would say the biggest advantage is in having images available all the time to everyone. So where, if you were to come into A & E and you’ve broken an arm and you have to be referred to the orthopaedic surgeons, there is no backwards and forwards of one piece of film following you around or not as the case may be. The fact that you have a picture that any doctor can see, the orthopaedic surgeon can see, it can be in the theatre if you get up there in 10 minutes time.’ (CS2 Radiology Manager).

‘One of the things that clinicians have been more willing to do, is to actually show the patient the images on the computer screen.’ (CS2 Radiology Manager).

‘…it [impact of PACS on the clinical encounter] is very positive. They can view images in the theatre more readily, they can view images in the outpatient clinic more readily, on the ward more readily, they can share the images with patients and relatives, they [clinicians] can have their queries answered by real-time telephone consultation. You know if you’ve got the patient there at the time and you want to tee them up with a particular programme of investigation or a particular treatment programme, and a decision around the image is critical to that …it’s a one stop shop and you deal with it all at the same time and off the patient goes down that particular pathway.’ (CS2 Consultant Radiologist).

Relational integration

PACS had a positive impact on relational integration. Non-radiologists could look at an image on-line, and if they had questions about the image or report, could contact the radiologist by telephone. Both clinicians could look at the same image simultaneously, despite being separated by distance, and this improved trust and confidence in reporting on the image and on subsequent clinical decisions.

‘I believe that, I can’t quantify it but I firmly believe it’s improved patient safety because if you’re not losing x-rays, you’ve always got the image and therefore the clinician then can look at he image and help make their diagnosis. If you haven’t got the image because we’ve misplaced it then there’s a patient safety issue there. You’re not exposing patients unnecessarily to extra x-rays.’ (CS2 Hospital Chief Executive).



‘you have now got clinicians, talking to each other via video conferencing, discussing cases, looking at the same image and I would argue that makes a better quality of conversation for the sake of the patient.’ (CS2 Hospital Chief Executive)

‘We have probably put into PACS a billion plus images, we have lost 3. Now I am disappointed we lost three! But within the clinical team they became a strong talking point – ‘Huh PACS lost an image!’. Whereas before – ‘oh yeah, we’ve lost an image – send the patient back to x-ray. So the actual confidence in the efficiency of the department to produce what is asked for and for it to be available, is just there now.’ (CS2 Radiology Manager).

‘And I think particularly with the interaction between say one of the clinicians and one of the radiologists, that’s improved because the consultant outside knows that the consultant radiologist inside has access to those images.’ (CS2 Radiology Manager).

‘ two doctors in different places to actually view those images at the same time from a clinical point of view …’ (CS2 IT Training Manager).


There were relatively few problems with PACS in terms of skill set workability; however the accompanying RIS engendered a number of problems. Radiologists had to take on tasks that had previously been done by administrative staff, in terms of registering the patient on the system. Also some radiologists had difficulties with the voice recognition software for reporting images, and preferred to stick with the previous system of dictating to Dictaphone for subsequent typing up by the medical secretaries.

Summary

PACS had fully normalized in this study site. This appears to be due to high levels of contextual integration, interactional workability and relational integration. PACS clearly enabled the organization to meet its goals more efficiently than the old system. There was very strong leadership within the organization (including both managerial and clinical leadership). PACS had a very positive impact on interactional workability, as it clearly benefited patients and patient care. Finally, relational integration was improved, as communication and trust between radiologists and clinicians was made easier and more transparent. What problems still remained focused on the RIS, and occurred in the area of skill set workability.

6.4.4 Case Study 3: Clinical Nurse Information System

The Clinical Nurse Information System (CNIS) consisted of hand-held Personal Digital Assistant devices (iPAQs) which were wireless enabled. District Nurses could use them to record clinical assessment information while out in the community, and download the information to the central server once back at base. The system also included some minimal decision support, in the form of standardized assessment tools for ten common nursing problems, with associated algorithms for care. We studied the CNIS in one Community Health Care Partnership in Scotland. The implementation of the CNIS had started some 3 – 5 years prior to the interviews, and was initially designed as a pilot. It had been intended to act as a way of sharing



information between community nursing and social services, but this potential had yet to be realized, with social services not able to access information gathered by community nurses.

Degree of Normalization

The CNIS was at best only partially normalized in this district nursing community, and provides a good example of the difference between adoption and normalisation. Although some 80% of the District Nurses were using it, some teams were still maintaining dual systems (the new electronic record system and the old paper-based one) and it was evident that not all nurses felt comfortable using it, with the hand-held devices still seen as new or strange.

‘It’s been slow in, in places.’ (CS3 Senior Nurse A).

‘In our area 80% of the nursing staff are using it. That’s district nurses, and that’s all grades of district nurses.’ (CS3 Senior Nurse A).

‘that little bit different with the iPAQs - it’s got that sexiness to it.’ (CS3 Health Board Services Manager).


There were both positive and negative contextual integration components to the CNIS. On the positive side, the Scottish Executive were keen to modernize, and saw electronic record keeping as essential to this, so they made funds available for this initiative. Many of the nurses were frustrated with the existing paper record system, which often required them to drive long distances simply to obtain a patient record. Senior managers were keen to encourage IT use amongst district nurses, and saw the CNIS as a good way of doing this. On the other hand, there appears to have been weak leadership, with the dedicated implementation group being disbanded after the first year. Training was under-funded and under-resourced, with no planning for on-going or “top-up” training, despite an acknowledgement that the target workforce had very low IT literacy.

‘I think there’s probably a few things that prompted it, one being that nurses felt they needed an updating on the paper documentation that they used, and also it was coming into the 21st century, and we really felt that we wanted to move ahead and have some modern electronic system and a method of gathering patient information. One other external driver was the need to be able to share Single Shared Assessment information with social work departments, and we wanted to be able to do that electronically.’ (CS3 Lead Project Nurse)

‘a frustration of the nurses when they had to see a patient who was not residing in the area that they were based in and they had to drive to another area in order to take a copy of the records. That was one of the big drivers for nurses at that time. But one of the main drivers for nurses was actually the impracticalities of if you needed information about someone whose condition had deteriorated, perhaps on a Friday afternoon, you then



had to write a different set of documentation and drive it to the place that the patient needed to be seen, otherwise there was no way of getting the information to them. So one of the big things was to actually have a system that talked to nurses across the area.’ (CS3 Clinical Services Manager).

‘There's always been, it’s always been a wish for the business to have; electronic data capture and the catalyst for it was a mandate from the Scottish Executive that our staff should be recording a single shared assessment electronically. That was the driver for it, that was the catalyst that gave us some initial funding to pilot things, so wanting to get a pilot thing going. So, it had always been a wish, and this was the, this was the excuse that gave us some money to do it… A longstanding wish to have, from the Government aspect, to have electronic health registers for community nursing as opposed to bits of paper and typewriters and all the rest of it. This [funding for the pilot] was through the Scottish Executive Modernising Government fund. One of their mandates was that health and social work should be able to record single shared assessments electronically.’ (CS3 Health Board Services Manager).

‘it would introduce them [the district nurses] to – as individuals, as professionals – to this world of electronic record-keeping and information sharing, which they just simply had no experience of.’ (CS3 Director Community Health and Care Partnership).

‘the steering group that was set up for community nursing IT system hasn’t met, therefore there’s not a defined lead in this. There is a project manager, but we don’t really see him very much.’ (CS3 Senior Nurse A).

‘I was on the original Implementation Group and then it kind of went into abeyance... So we have raised it with our Heads of Service to say that we really feel that there needs to be an Implementation Group to drive this forward. And we need to have some support for the nurses in order to get this one, otherwise it won’t work. And I think the feeling is that we’ve never got this far in [name of Health Board] with an IT system for community nurses, that we want it to work, you know, as best as it can. But it won’t unless we stop and take recognition of where we are, and see what we can do to support the staff.’ (CS3 Senior Nurse B)


One of the positive features of the CNIS was its use of small hand held devices (iPAQs). These were relatively robust, cheap, and portable. This meant nurses felt comfortable carrying them when visiting patients, and were reassured that the device could be replaced if necessary. This was in contrast to previous initiatives, which had involved laptops. Laptops were too heavy for nurses to carry around, and made them feel vulnerable to attack. As the nurses were able to carry the iPAQs around, they were more likely to use them during, or immediately after, patient visits, which improved their efficiency.

‘You’ve seen how streamlined they are quite you know petite. You can put them in your pocket. Well the devices that we used in the past were sort of



bulbous and very big and the nurses just thought well I can’t be bothered carrying those sort of things around and especially in the environment and the area that they are.’ (CS3 IT trainer).

‘Well, that was the key thing. They wanted to use and record data at the point of patient contact as a domiciliary workforce, that was the only way we could do it. We couldn’t afford to get them laptops. They didn’t want laptops, because it was like an extra, big thing. They were really about some of the areas they go to, being seen as the weekly District Nurse to people having drug habits and things - it’s an expensive piece of equipment for that type of thing.’ (CS3 Health Board Services Manager)


Our data contained no information that suggested that the CNIS had any impact on relational integration (either positive or negative).

Skill set Workability

Skill set workability was the major factor impeding normalization of the CNIS. The nurses started from a very low level of IT literacy, and many were alarmed that this would jeopardize their future employment. Trainers had to spend a great deal of time on one-to-one training, and reassurance that nurses were not going to be sacked if they failed to learn how to use the CNIS.

‘It’s basically nurses who don’t even have a computer in their own homes and they haven’t actually come across this sort of technology and they’re having to face it at work and sometimes you get that sort of nervous reaction that they maybe might feel a bit inadequate in the sense that that oh this is really daunting. I’ve never used a computer system before. Will this mean I’ll be out of a job?’ (CS3 IT Trainer).

‘We do hold their hand quite a lot … we’ll go out to their health centre where they’ve got their caseloads and they’ve got basically computers that they actually use. So we’ll go out and train them and we’ll actually sit through, maybe putting on a patient register and we’ll go through the whole process with them so they get the training in terms of the functionality and then we’ll sit there with them doing one or two new patients that they’re having to register onto the system.’ (CS3 IT Trainer).

‘probably a hindrance, not all nurses are IT literate; you know, there IT is somewhat limited. And that’s quite frightening for staff when we’re saying to them, you know, you’re not the right…. (laughing) for the future. So we’ve had to address one or two IT issues.’ (CS3 Senior Nurse B).

Summary

The CNIS had achieved relatively low levels of normalization, despite data collection occurring some 3 – 5 years after the start of implementation. Factors contributing to this low level of normalization included mixed contextual integration, with a strong political will to introduce electronic record keeping, but poor levels of implementation planning and execution within the organization. A major hurdle was poor skill set workability, with



many nurses needing more substantial training than was provided. The positive impact on interactional workability was fairly slight, and possibly not sufficient to overcome the negative skill set workability and poor local contextual integration.

6.4.5 Summary of overall findings

The Normalization Process Model was found to be a suitable analytical framework with good explanatory power in these contrasting case studies (Table 9).

Table 9 Summary of Main Findings

NPM Construct CS 1 = Choose and Book

CS 2 = PACS CS 3 = CNIS

Interactional Workability (IW)

Poor, especially in GP Good Medium to good

Relational Integration (RI)

Poor to neutral Good Medium to poor

Skill Set Workability (SSW)

Poor for GPs, Neutral for Hospital

Medium to Good

Medium to poor

Contextual Integration (CI)

Neutral for GPs, very good for Hospital

Good Medium to poor

Choose and Book had normalized to very different extents in secondary and primary care. The overwhelming importance of Choose and Book to the hospital’s financial survival, in terms of maximizing the proportion of new outpatient referrals, combined with strong managerial and clinical leadership and a well planned, well resourced implementation strategy had resulted in relatively high levels of normalization within the hospital. In contrast, the negative impact on interactional workability in primary care, caused by the slowness of the technology, and the negative impact on skill set workability, with GPs resenting being asked to take on work that was essentially clerical in nature had led to low levels of normalization in primary care.

PACS had become almost fully normalized in our study site. This appeared to be due the positive impact that PACS had on three of the four constructs of the NPM, with no negative impact on the fourth. PACS had high contextual integration, in that it helped the study site achieve national targets on waiting lists for diagnostic services. It increased the site’s efficiency, reducing the number of lost x-rays, and hence the need for repeat imaging. These efficiency gains had been predicted by senior management, who had invested heavily, both in the system itself, and in re-structuring workflow to fit in with the new system. There was very strong managerial and clinical leadership, with adequate resources allocated to the implementation. Interactional workability was also high, with all concerned able to see (and experience in consultations) the benefits to patients. Relational integration was good, as images could be shared between



radiologists and clinicians, allowing for good interdisciplinary discussions, and streamlining patient care. There were some problems with skill set workability, particularly with the associated RIS and voice recognition software which had limited some of the benefit realization.

The CNIS had, at best, only partially normalized. There appeared to be low contextual integration, with no clear vision as to the benefits of CNIS for the organization or workforce, and weak, unsustained leadership around the implementation. The most positive aspect of the CNIS was its interactional workability, with light portable hand held devices which the nurses found useable and time saving. Skill set workability was an issue, with insufficient attention paid to on-going training after an initial good start.

6.5 Discussion and conclusions

Main results

This work package successfully identified three case studies which between them spanned community, primary and secondary care, as well as covering the main e-Health domains of communication, management, decision support and information. Views of those responsible for implementing these e-Health initiatives were successfully obtained, and the Normalisation Process Model (NPM) provided a good explanatory model for the relative normalization of the three e-Health initiatives.

Relationship with existing literature

Our work is highly congruent with existing literature. The problems with implementation of Choose and Book have been well documented, with just 63 referrals made using Choose and Book in the first year (Nolan and Whitfield) and Primary Care Trusts (PCTs) only half way to the Choose and Book target in 2007 (Bell 2007), with considerable discontent about Choose and Book voiced by senior managers in hospital trusts (Hendy et al. 2007). A questionnaire study of 500 GPs published in 2006 found that nearly two-thirds of respondents were not in favour of Choose and Book, citing time constraints and the inflexibility of the system (Pothier et al. 2006), reflecting our finding that poor interactional workability had impeded normalisation of Choose and Book in primary care.

In contrast, the literature on the Picture Archiving and Communication System (PACS) suggests that this has been widely adopted internationally (van de Wr et al 2008). Where system reorganisation has accompanied PACS implementation, there have been marked improvements in workflow (van de Wr et al 2006), with improvements in reporting times and productivity, and fewer requests for repeat x-rays (Collin et al 2008). An early interview study of users of PACS in one hospital in 1999 reported user preference for PACS over traditional films, because of improved ability to share images between clinicians (relational integration), faster reporting times (contextual integration), and potential benefit for patients (interactional workability).

The literature on the use of hand-held computer devices supports our findings in Case Study 3 that their lightness and portability are significant positive features, as were their potential to improve patient safety by



ensuring patient records were available at the point of care (Lu et al 2005). Looking specifically at the use of Portable Digital Assistants (PDA) by nurses, Lu found that contextual integration factors, such as lack of integration with hospital information systems or lack of support at work were perceived as major barriers by nurse users (Lu et al 2006). Overall, nurses are lower users of PDAs than doctors (Garritty and Ek 2006).

Methodological issues

The case study approach worked well, and identifying “implementers” was straightforward. “Implementers” did provide, as hypothesized, useful data from a perspective that differed to clinicians. These implementers tended to have a whole systems approach. The method for identifying the case studies appears to have worked, in that the three studies covered a range of health care contexts and e-Health domains. The interview schedule had been well piloted.

We had initially intended to undertake ten interviews per case study. We did achieve this number in Case Study 1, but in both Case Studies 2 and 3 it became apparent that we had sampled to redundancy after relatively few interviews. In contrast, we do not think that we sampled to redundancy in Case Study 1, as there were other perspectives that could perhaps have been usefully gained. As this was our first case study, we were not aware that we would not need to undertake so many interviews for the subsequent studies, and stopped doing interviews after ten to ensure the overall work package was completed in time and within budget.

An additional potential weakness was the limitation of Case Study 2 to PACS. Given that PACS had been implemented with the new Reporting Information System, it might have been better to expand Case Study 2 to include both technologies, particularly as they were inextricably linked in the minds of many of our informants.

However, a major strength of this work package was the analysis which was undertaken rigorously, with repeated double coding, and discussion within a multidisciplinary team, a process known to aid reflexivity and rigour in qualitative research (Barry et al. 1999).

6.5.1 Discussion and Conclusions

Our findings have major implications for policy, practice and research. This work package, like the preceding ones, highlights the value of the NPM as a theoretical framework for assessing the likely normalization of a new e-Health initiative. E-Health initiatives that score highly on all four constructs (contextual integration, interactional workability, relational integration and skill set workability), are highly likely to normalize. Low scores on any one construct should alert policy makers and senior managers to potential difficulties in this area, which need careful consideration during and after implementation. Low scores across all four constructs suggest that the initiative has relatively low likelihood of normalizing successfully, and that some rethink may be needed.

Contextual integration also depends on the organisation. The importance of strong leadership combined with a carefully planned, adequately resourced and well executed implementation strategy is clear.

It is worth noting that normalisation does not necessarily imply standardisation or the uniform deployment of e-Health technologies, and that this is important when any evaluation of their use-in-practice is made



in the future. As in the CNIS case study 3 it might well be possible for elements rather than a complete system to be normalised.

6.5.1.1 Recommendations

1. We strongly advise senior staff planning an e-Health initiative to consider the following triad prior to implementation:

� The overall context, including national priorities and strategies, as well as local priorities, leadership and resources.

� The nature of the e-Health initiative, its likely impact on clinical interactions and inter-professional relationships, and its user-friendliness.

� The impact of the implementation on the workforce, including workload, training needs, and alteration in responsibilities.

2. Future research should be commissioned to:

� Examine the potential of the NPM as a predictive tool, in the context of e-Health initiatives.

We have used the data from this work package to generate an e-Health implementation toolkit (e-HIT). This was designed to assist senior managerial staff with planning and implementing future e-Health initiatives, and is discussed in detail below.

6.6 Phase 2 Development and Formative Evaluation of the e-Health Implementation Toolkit.

6.6.1 Introduction and Background.

One of the initial planned deliverables from this Work Package was the development and formative evaluation of a Model Implementation Process (MIP). During the development phase it became apparent that, as far as implementation of e-Health initiatives is concerned, one size definitely does not fit all. With this in mind, it seemed more sensible and valuable to concentrate on developing an e-Health Implementation Toolkit (e-HIT), rather than a model implementation process. In view of our emerging research findings it became clear that such an instrument would prove useful as a sensitizing tool for senior managers or other staff who are considering, planning or undergoing an e-Health implementation.

6.6.2 Aims and objectives

The aim of phase 2 of this work package was to apply the understanding gained from this work package and the other work packages in EH94 to produce an e-Health Implementation Toolkit (e-HIT).

The e-Hit was designed to be used at multiple points during the implementation process, including whether or not to embark on the implementation, planning the process, monitoring progress, and reflecting on solutions to unanticipated problems. It was therefore envisaged as a tool that could be used widely.



6.6.3 Methods

The e-HIT was developed by Elizabeth Murray in collaboration with a commercial IT consultancy company “Rocket Science” with support from Carl May and Frances Mair. EM wrote the content, while Rocket Science designed the presentation and navigation, and CM and FM provided feedback throughout the development process.

Development of the content of the e-HIT.

The content of the e-HIT was derived from the data obtained in all four of the work packages making up the EH 94 programme, but with particular emphasis on the results of the scoping and systematic reviews undertaken in WP1 and the results of interviews with implementers undertaken in WP4. The Normalisation Process Model provided the theoretical framework, and the concepts within the NPM are well represented in the e-HIT.

The content of the e-HIT was derived by considering the main themes identified in the scoping review and the themes identified in interviews with implementers. These themes were then synthesised with each other to create a database of items which had theoretical and empirical validity. The theoretical validity came from the NPM, while the literature review and WP4 provided empirical support.

To facilitate comprehension and use of these items, they were grouped into three major components: the context (organizational factors, national and local policies, other drivers); the intervention (user friendliness, fitness for purpose, flexibility, impact on clinical practice, effectiveness and cost-effectiveness); and the workforce (impact on workload and workflow, training, power relationships between different professional groups, responsibility and accountability). The items were reviewed and reduced in number to yield a manageable toolkit. Each item was operationalised by statements, which were anchored by extreme negative and extreme positive positions. These statements were reviewed by the co-investigators and the RocketScience team.

Once the main theoretical and empirical content had been determined, the rest of the content was written with a view to making the e-HIT comprehensible and accessible. An introduction included information on what the e-HIT was, who should use it, when it should be used, and how it should be used. There was a section on how it was derived, and detailed instructions for first time users. This section could be easily bypassed, with return users encouraged to go straight to the toolkit.

The main body of the toolkit contained the content described above. The final section consisted of case studies (the three case studies described within Phase 1 of WP4), to give users an example of how they might complete the toolkit, and how the information gained could predict normalisation of an initiative.

Development of the presentation of the e-HIT.

We were fortunate to work with RocketScience, an independent software and consultancy company who have done a lot of work with e-Health and the NHS. They already had a template for a toolkit, which they adapted for the e-HIT. In the main content section, users were asked to provide a score between 0 and 10 for each statement, with a box for text comments to justify the score allocated.



These scores then automatically generated a report for the user, summarising the data entered, and providing a visual display. We emphasised that the toolkit should be used as a sensitising tool, to help implementers consider, address and thereby forestall potential difficulties.

Formative evaluation of the e-HIT.

A two stage formative evaluation of the e-HIT was undertaken, using on-line task groups. For the first stage of the formative evaluation, the draft e-HIT was circulated to the EH94 Advisory Group (6 senior academics and managers with extensive experience of e-Health and the NHS) and the EH94 steering group. These experts were asked to complete the e-HIT for an e-Health initiative they had personal experience of and, on the basis of this experience, to comment critically on the e-HIT. Specifically, respondents were asked whether they thought the e-HIT would be useful to senior managers considering, planning or undertaking an e-Health implementation, what would make the e-HIT more useful, what were the positive features of the e-HIT, and what features needed modifying. Respondents were asked to suggest modifications which would improve the e-HIT, and for other general or specific comments not covered by the questions listed.

There was a good response from both the Advisory Group and Steering Group, with all members contributing comments and discussion. The overall response was overwhelmingly positive, with comments including “fantastic” and “excellent piece of work”. Specifically, respondents thought that the non-prescriptive approach and emphasis on the e-HIT as a sensitizing tool would appeal to senior managers. Respondents thought that the overall layout was clear, the language easily comprehensible, and that the main areas of importance were well covered. They liked the sliding scales, space for explanatory text, and instant feedback.

Areas of concern were the mechanisms for dissemination, with several respondents referring to “toolkit fatigue”, with subsequent need for active steps to bring the toolkit to potential users, rather than leaving them to find it. Respondents also felt that the toolkit would be improved if it was fully web-enabled, allowing users to store their responses on the web for others to view, as this would allow build up of shared knowledge, expertise and experience across the NHS.

There were also specific suggestions for improvement about the navigation, layout and wording of individual components of the e-HIT, with requests for more information to help with completing some of the questions, and clarification of individual questions.

Representatives from Rocket Science and Elizabeth Murray met to discuss these comments, and agreed on the following changes:

� A redesign of the Introductory section, allowing experienced users to bypass this completely;

� Improved explanation of the uses of the report section;

� Provision of explanatory mouseovers to assist with completion of individual questions;

� A more streamlined overall lay-out.

Resources were not available for making the toolkit entirely web-enabled, and we agreed to seek subsequent funding for this. Similarly, the issue of dissemination needs further discussion with the Service and Delivery Organisation.



Once these changes had been implemented, the second stage of the formative evaluation of the e-HIT was undertaken. This involved circulating the revised toolkit to all participants in Work Package 4, i.e. all twenty-two of the implementers who had been interviewed for one of the three case studies (Choose and Book, Picture Archiving and Communication System, and the Clinical Nurse Information System). At this stage participants were thanked for their initial contribution to the study, and asked to comment on the e-HIT as one of the products of the study. They were asked to comment on the likely usefulness of the e-HIT, suggestions for improving its usefulness, whether it adequately reflected their experience of a specific e-Health implementation, and any other comments or suggestions.

This second formative evaluation yielded only minor suggestions, which were easily incorporated into the final version.

6.7 Results

The e-HIT is freely available for downloading from the following url:

http://www.ucl.ac.uk/pcph/research/

A colour print-out is also attached as an appendix 29.

6.8 Discussion and Conclusions.

The e-Health Implementation Toolkit combines a strong theoretical underpinning (the NPM) with empirical data derived from a literature review and primary data collection. It has been favourably formatively evaluated. For senior managers planning to use the toolkit we would recommend that representatives from the various different professional groups affected by any given e-Health initiative complete the tool, and discuss their relative inputs, to help generate a whole systems viewpoint. We believe that the e-HIT would prove most valuable when used in this way. The effort that the research team, together with our external advisory group, invested into understanding the ways in which e-HIT would be made most useful, was in itself an indication of the research team’s understanding of how best to normalise e-HIT and has informed the development of our recommendations regarding its further development and dissemination.

6.8.1.1 Recommendations:

1 We suggest the SDO host the e-HIT on their website, and disseminate it to senior managers involved in planning e-Health implementations.

2 Future research should be commissioned to build on this work, with a view to:

� determining whether the e-HIT is a useful tool for senior managers;

� extending the tool, by making it fully web-enabled, to allow live data capture, and storage of new data on additional case studies.

� extending it to non e-Health technological initiatives.



7 Taking Analysis To A Higher Level

7.1 Background

Throughout this project we have used the NPM as our conceptual framework. Each work-package has used the NPM to inform data collection and our approach to analysis. However, as indicated in Chapter 2, in parallel with this project the NPM was undergoing further development, and this led to a middle range theory of implementation and integration – Normalization Process Theory, or NPT (May and Finch 2009). Qualitative analysis in the systematic review (WP1) and task groups (WP2) revealed that between one third and one quarter of coded items did not ‘fit’ the NPM. This data was relevant to the topic of study, and required further analysis. We took this as an opportunity to employ the extended and refined NPT. We recoded qualitative data in work-packages 1 and 2, and added NPT oriented questions to structured instruments developed for work-package 3, (as data collection design in WP4 was based on the NPM, reanalysis using the NPT was not felt to be useful). This section of the report outlines the use of the ‘next generation’ conceptual model, as we employed it to develop and extend our analysis.

The NPT retains the key assumptions and constructs as presented in the NPM, but extends these in a number of ways. The constructs of interactional workability (IW), relational integration (RI), skill set workability (SW) and contextual integration (CI) are retained as the four key domains of work that represent a construct of ‘collective action’, but three other higher level constructs have been added to extend the utility of the model as a theory that can be applied to achieve a better understanding of the processes of normalisation not only in relation to the kinds of technologies that have been of interest to this project, but to complex interventions more generally. The NPT thus describes how the work of implementation is operationalized through four generative social mechanisms (coherence; cognitive participation; collective action; reflexive monitoring). These four mechanisms are defined as follows:

� Coherence refers to the sense-making work that people do individually and collectively when they are faced with the problem of operationalising some new way of working.

NPT proposes that: embedding a complex intervention depends on work that defines and organizes it as a cognitive and behavioural ensemble. Embedding work is shaped by factors that promote or inhibit actors’ making sense of it. The production and reproduction of a complex intervention requires that actors collectively invest meaning in it. This is the work that people do in preparing to engage with a new technology or complex intervention. In relation to e-Health, coherence refers to a range of ‘sense-making’ activities, such as, for example, agreeing collectively the purpose of an e-Health system and ensuring that it fits with organisational and practice objectives; and identifying and valuing the perceived benefits of working with an e-Health system (both collectively, and as individuals implicated in the use of the system).

� Cognitive participation is the work that people do to engage with a new technology or complex intervention. The NPT proposes that: embedding of a complex intervention is



dependent on work that defines and organizes its participants. Embedding work is shaped by factors that promote or inhibit actors’ participation. The production and reproduction of a complex intervention requires that actors collectively invest commitment in it. In relation to e-Health, for example, cognitive participation relates to issues such as organisational strategies for engaging different categories of workers in the design and organisation of e-Health systems in order to enable them to use the system effectively in working collaboratively; and facilitating processes that encourage individuals to commit themselves to the work required for embedding the new system into practice.

� Collective Action describes the work that people do to enact a new technology or complex intervention. This encompasses the four constructs of the NPM, namely interactional workability, relational integration, skill set workability and contextual integration. In relation to e-Health, this refers to the ongoing work of embedding itself, and how it gets done in practice, and this report has already detailed the kinds of work that promote and facilitate collective action with respect to these four domains of work.

� Reflexive monitoring relates to the informal and formal work that people do to appraise the effects of the new technologies’ complex interventions that they have enacted. NPT proposes that: embedding of a complex intervention is dependent on work that defines and organizes the everyday understanding of it in practice. Embedding work is shaped by factors that promote or inhibit individual and collective appraisal of the complex intervention. The production and reproduction of a complex intervention requires that actors collectively invest in its understanding. In relation to e-Health, reflexive monitoring refers to the processes by which organisations and individuals are able to reflect on and understand the effectiveness and utility of an e-Health system as experience and knowledge of the system is gained over time. This includes not only being able to make ongoing appraisals about whether the benefits of using the e-Health system are being realised and are in proportion to the effort required, but also being able to use that understanding to make changes and improvements to the design and organisation of the e-Health system in order to achieve the objectives of using it.

7.2 Aims/Objectives

The aim of this part of our project was to:

1 Use NPT to inform analysis of qualitative data gathered from our systematic literature review (stage 2 of WP1) and our task groups and interviews from WP 2.

2 Assess the potential contribution of NPT to our understanding of implementation and integration issues.

It is important to note that we did not seek to formally test the theory in WP1 and 2, since these were re-analyses of already collected data. In WP3 we sought indicative information about value if NPT through a small number of additional items. This work should be regarded as theory refining research, not theory testing.



7.3 Methods

Qualitative analyses had already been conducted in which all included papers in WP1 (systematic review) and interview and task group transcripts (in WP2) had been subjected to attribution analysis using NPM to define a coding framework. These methods have been described in detail elsewhere in the report. In this additional work, these same attributions were recoded using a coding framework defined by NPT. FM and CRM recoded this data independently of each other. There was a high degree of congruence between the two coders (NPT is easier to use) and disagreements were rare. When these occurred they were resolved by discussion. If any areas of disagreement remained then a third reader (TF or EM) served as arbiter.

In work-package 3, following the original protocol, the TARS questions reflected mainly the four domains of work specified in the NPM (IW, RI, SW, CI). Prior to administering the Phase 3 surveys however, single question items were written and included to reflect the three new constructs of the NPT as follows:

Coherence: The staff who work here have a shared understanding of what the system is for and how it is to be used

Cognitive participation: The staff here are committed to making the system work

Reflexive monitoring: There are ongoing mechanisms for monitoring and appraising how this e-Health system is used

The survey results relating to these items were analysed descriptively as part of the full item set, as specified in Section 5 that reports WP3.

7.4 Results

WP 1 Analysis Using NPT

Recoding of the attributions from the systematic literature review undertaken in WP1 resulted in the following distribution of codes (Table 8).

Reassuringly, it is immediately clear that the bulk of attributions (67%) fall within the Collective Action category of the NPT, which encompasses all four constructs of the NPM. This analysis therefore demonstrates that the bulk of implementation issues relate to the NPM constructs of interactional workability, relational integration, skill set workability and contextual integration. Table 10 and Figure 5 show that the rest of the attributions are fairly evenly spread across the other three components of the NPT, namely, coherence, cognitive participation and reflexive monitoring. A small number of attributions remain outside of the scope of the NPT, (36/411 coded statements, less than 9% of the total).



Table 10 Number of Attributive Statements From WP1 Within Each Category of the NPT

Coherence Cognitive

Participation

Collective

Action

Reflexive Monitoring

Outside NPT

1. Broens 2007 3 2 17 3 1

2 Chaudrhy 2006 0 1 18 2 0

3 Hebert 2006. 3 0 5 2 1

4 Hilty 2002 2 5 14 2 1

5 Jennett 2004 7 3 8 7 1

6 Jennett 2005 2 2 5 2 1

7 Johnson 2001 3 5 17 0 2

8 Kawamoto 2005 1 1 9 1 0

9 Kukafka 2003 1 3 13 2 2

10 Leatt 2006 6 6 15 9 5

11 Lu 2005 2 1 29 4 4

12 Ohinmaa 2006 4 3 10 3 2

13 Papshev 2001 3 3 15 1 2

14 Peleg 2006 0 7 28 7 8

15 Shekelle 2006 1 1 10 0 3

16 Studer 2005 1 4 13 1 0

17 Vreeman 2006 0 1 19 6 1

18 Yarborough 2007 3 2 16 3 2

19 Yusuf 2007 4 3 14 0 0

Totals 46 53 275 55 36

The results in Table 10 are visually displayed in the radar plot seen in Figure 5. This radar plot is provided for illustrative purposes only and graphically displays the distribution of coded statements in relation to the constructs of the NPT, namely, coherence, cognitive participation, collective action and reflexive monitoring. The shaded zone indicates where the bulk of coding was located, within the collective action domain. This latter domain contains all of the constructs of the NPM and therefore it can be seen that the bulk of implementation activity is addressed by the original NPM constructs.



Figure 5. Distribution of Attributed Statements From the Literature on Implementation

CO = Coherence; CP = Cognitive Participation; RM = Reflexive Monitoring; CA= Collective Action

Illustrative examples of what types of attributive statements from the literature fell into each category of the NPT are now presented.

7.4.1 Coherence

Coherence

Coherence could be taken to describe participants’ understandings of the broad policy mechanisms or actions that contributed to a rationale for promoting the uptake and implementation of e-Health interventions as this extract from Ohinmaa 2006 demonstrates:

‘Directed, systematic government policy aiming to increase investment in technology can enhance networking and collaboration within the healthcare system’ (Ohinmaa 2006)

Or it could be used to relate to the work needing to be undertaken with staff intended as implementers and users of the new e-Health service, to identify and explore beliefs about the system to be implemented as the following extract shows:

Distribution of Atributions Across NPT

Series1 0

50 100 150 200 250 300

CO

CP

CA

RM



‘Prior to implementation it may be beneficial to assess expectations and concerns regarding EMR system implementation’ (Studer 2005)

Thus coherence was concerned with preparatory work undertaken either locally or to facilitate understanding of the purpose and benefit of the e-Health service in support of its implementation but also addressed the issue of engaging with stakeholders regarding their expectations, concerns or requirements.

Cognitive Participation

Cognitive participation addressed different means of encouraging engagement with and enrolment of health professionals in the implementation of e-Health services and how such new services could be seen to become legitimised or not, by health professionals.

Hilty 2002 proposes ways to encourage health professional participation which included:

‘Incentives for each of the parties involved’ (Hilty 2002)

Cognitive participation, therefore, included actions that could potentially serve as drivers to participation in the implementation of new e-Health services. This could refer to a whole range of incentives, although obviously it is financial incentives that are referred to in this instance.

However, there were other types of issues that fell into the cognitive participation category. These included a range of actions that helped legitimise participation in the e-Health implementation process and promote it as a worthwhile activity. One aspect mentioned on a number of occasions was that of local “champions” as this extract from Johnson 2001 shows:

‘IT advocacy. In concert with increasing awareness of IT, we need to establish a network of IT advocates who can facilitate the movement of more apprehensive colleagues with the help of other groups.’ (Johnson 2001)

Such champions were seen as having the ability to promote utilisation of new e-Health services by more reticent colleagues. However, rather like all types of “peer pressure” this could be a double edged sword as Peleg suggests:

‘lack of improved performance could be due to any number of factors, such as lack of support among colleagues’ (Peleg 2006)

Thus health professionals, by being enthusiastic advocates, could legitimise and promote enrolment and commitment of colleagues or alternatively by



negative actions could jeopardise the commitment of staff needed to make the system work and thus impede implementation.

Collective Action

Collective action refers to the work undertaken to operationalise a new e-Health service. This encompasses all the constructs of the NPM previously described, interactional workability, relational integration, skill set workability and contextual integration. Examples of collective action attributions have been provided in great detail within the workpackages and will therefore not be repeated here.


Reflexive monitoring concerns how individuals or groups decide whether an e-Health intervention is worthwhile or not. For the most part this has to do with issues of evaluation and monitoring and how this is used to influence utilisation and future e-Health implementations as the following comments suggests:

‘Comprehensive cost effectiveness studies are essential in developing future financing structures’ (Broens 2007)

‘the complexity of medical practices and the high cost of implementing CDSSs make evaluation of CDSSs both a challenge and a necessity.’ (Peleg 2006)

However, evaluation was also promoted as necessary to ensure safety concerns were addressed as the following comment highlights:

‘computer systems have the potential to introduce errors’ (Vreeman et al. 2006)

Addressing such concerns through evaluation could of course, either alleviate them, or verify them, necessitating amendments to the e-Health service being implemented.

Issues Outside the NPT

There remained some issues that did not fit within the NPT. Although this related to only a minority of attributions it is important to consider what still lies outside the theory. Issues of a strictly technical nature or general attitudinal issues remain outside the NPT. Examples of technical issues were:

‘Customisation: various functions or programs can be added to support different medical specialities.’ (Lu et al. 2005)

and

‘product quality’ (Papshev 2001 )



Indeed, some statements, such as “product quality” as provided above are so generic and vague, without accompanying contextual data, that it is uncertain whether the concept actually lies outside the model or whether it is merely its generic nature that renders it unable to be coded.

Similarly, if attitudes were simply general comments such as:

‘providers resistance to change’ (Papshev 2001)

Then these too fell outside the theory. The NPT, therefore, focuses on the work that people do in the same way as the NPM. Thus general comments about culture, technology quality and attitudes are not included. However, all the work which individuals undertake to implement and integrate e-Health services does fit well within the extended NPT.

7.5 WP2 Analysis Using NPT

Normally, qualitative analysis excludes even simple quantitative descriptions of the contents of interview data. There are good reasons for this, this data is heterogeneous and is produced in non-replicable encounters. However there is merit in describing this body of data quantitatively here, because it shows first that NPT provides a framework that appears to account for more of the data collected. This is important if attribution analysis is used. If the theory does not account for the attributions, then it is an insufficient resource for interpretation. Full details of the count are presented in Table 11: 712 attributions were coded. Of these, 69% referred to aspects of collective action. This is to be expected since respondents were discussing enacting e-Health implementation processes Also, as before, only a minority, 19/712 (3%) of comments did not fit within the NPT.



Table 11 Number of Attributive Statements From WP2 Within Each Category of the NPT

Coherence Cognitive Participation

Collective Action

Reflexive Monitoring OUTSIDE

Task group 1 11 0 57 2 0

Task group 2 5 3 59 5 0

Task group 3 8 3 26 6 1

Task group 4 5 3 39 11 3

Task group 5 14 9 55 15 3

Interview 1 3 4 13 2 0




Interview 5 11 0 42 4 2





Interview 10 4 1 14 0 1

Interview 11 1 2 19 2 1

Interview 12 2 1 15 0 4

Interview 13 2 1 20 2 0

Interview 14 0 1 15 2 0


Interview 16 2 1 16 1 0

Totals 88 47 492 66 19

Figure 6 illustrates this data via a radar plot. This figure provides the data from Table 10 in a graphical form and again is used for illustrative purposes so that it is clear to the reader that the bulk of codes (the shaded zone) falls within the collective action category, as did the WP1 data, thus showing once again that the NPM captures the vast majority of implementation activity and that the additional constructs of the NPT identify other implementation issues in quite a balanced way.



Figure 6. Distribution of WP2 Attributions across the NPT

Distribution of WP2 attributions across the NPT

0

200

400

600Coherence


Collective Action


Coherence, cognitive participation and reflexive monitoring receive between 6% - 12% of the share of the coding. Thus the NPT addressed 97% of the implementation or integration issues raised by health professionals.

Examples of attributive statements from task group or interview participants within each of the categories are provided below.

Coherence

Coherence was sometimes exemplified by this group by descriptions of how beliefs and knowledge about e-Health systems could affect their utilisation as the following comments highlights:

‘That’s really important, the NHS 24 was completely new, and it was available to the public and they didn’t have a clue how to use it, particularly at the start’

(Task group member)

Also the following quotation illustrates how “attitude” could now be classified as part of NPT when it related to issues of engagement and involvement of users:

‘I think… professional attitude to implementation: that might come in terms of the management of change, but also, their involvement in understanding the engagement within the procurement process, I think is quite key. That’s actually quite key ….., if you have that engagement, I would suggest, that, what you get, is, you get confidence of that group of staff and their colleagues, that the people who use them have been involved in choosing them, or designing them, or, you know, specifying them.’ Interviewee 5



Attitude, in this instance, was not simply a descriptive term but instead was referring to improving individuals’ understandings of an e-Health service (in this case through processes of engagement) and therefore fell within the coherence domain of the NPT


Within this data set cognitive participation included issues about champions as before but highlighted the importance of getting the cooperation and involvement of health professionals and appreciating the important role of patients. One task group member commented:

‘And I’m not sure that the field…test that we’re, even in our own organisation and nationally, gives enough to pull the clinical community on board as well. And I think people now almost see the project to be implemented and deadlines to be fixed, rather than always appreciating that there’s patients at the end of the clinical……..’(Task Group Member)

The importance of involving health professionals and obtaining “buy in” was a recurring theme as this participant notes:

‘So we reviewed the decisions, the symptoms that we were looking for in light of best evidence and we got consensual agreement on what the level of care was, that whole process, the Airth process got support from Quality Improvement Scotland it was then called Clinical Standards Board for Scotland and I and others promoted this with other clinical leaders and management leaders across the NHS, so we got support for the process the algorithms was XXX the clinicians were bought into this so they made them pretty good in terms of information and what was the knowledge if you like’ (Interviewee 1)

Collective Action

Once again collective action referred completely to constructs contained within the NPM. Examples were provided within the WP2 section and are not repeated here.


As in the preceding section reflexive monitoring refers to evaluation issues but professionals often commented on the importance of this in terms of contributing to the further development and reconfiguration of new services. The value of such an approach was emphasised by this respondent:

‘Well, that you don’t make steps by a sudden giant leap, that what you do is that you take what you have and you logically improve it bit by bit. If you look at the modern PC, that wasn’t invented overnight. That has taken about 20 years to develop by a series of logical steps from where it first started to where it is now. And of course, this contrasts entirely with the Connecting for Health strategy where what they assumed was you would construct a model of where you wanted to be and then you would spend a



huge amount of money in buying that. And you would actually get people to build something that didn’t exist. I think there’s good evidence that that approach has been disastrous because they’ve spent huge amounts of money and they haven’t actually achieved very much.’ (Interviewee 7)

In addition, it was clear that when benefits of new e-Health services were visible to health professionals then they were much more likely to embrace them enthusiastically as this comment illustrates:

‘…..but I think the benefits from it are so immediately obvious and large that actually the little bits of pain that go with it people have tried to work around.’ (Task Group Member)

Thus reflexive monitoring was not simply about how individuals or groups evaluated systems but how their perceptions of the utility of systems affected integration and also how iterative development was possible and could improve the quality of the e-Health services in use.

Outside the NPT

There was very little that fell outside the NPT, mostly it related either to specific statements relating to technology or more often to generic attitudinal statements such as the following:

‘I think there was a perception of, buy the technology and it will happen. We had either begged, borrowed or stolen. We had loan of various pieces of equipment and I think there was a misconception that if you buy the equipment, then the problem is solved. I always said beware of the man who is trying to sell you Telemedicine equipment because he probably does not understand what he is doing.’(Interviewee 12)

7.6 WP3 Analysis using NPT

The data collected in the WP3 surveys indicated some significant findings in relation to the NPT constructs that are worthy of mention. At both the Scottish site (regarding use of CNIS) and the call centre, responses on TARS items were compared using cross-tab analysis for differences between sub-groups of participants rating the e-Health system in question as either not/partly routine or as completely routine. At the Scottish site, responses to all three global NPT items (coherence, cognitive participation and reflexive monitoring) differed according to perceptions of routinisation. For all three items, those indicating that use of the system was only ‘partly routine’ were more likely to disagree with the statements (i.e. respond negatively) compared to the group who saw the system as a completely routine part of their work. This difference was particularly strong for the coherence item (c=10.124, p<.001), suggesting a potentially important association between understanding the system and what it is intended for and seeing it as a routine part of working practice.

In the call centre sample, the same cross-tab analysis indicated significant differences for the coherence and reflexive monitoring items, in that those disagreeing with the statements were more likely to perceive the e-Health



systems they used as only partly rather than completely routine. In the call centre sample, the cognitive participation item failed to differentiate between groups indicating different levels of perceived routinisation – this may not be unexpected given that staff are employed specifically to use the systems in question, and therefore a certain level of commitment may be presumed from the outset. It is interesting however, that despite mandatory use of e-Health systems in this context, there was still a proportion of staff who did not view the systems as a completely routine part of their working practice (n=37 vs n=174 responding ‘completely routine’). Taken together, these results provide support for inclusion of the constructs of coherence, cognitive participation and reflexive monitoring in the NPT.

This project has explored barriers and facilitators to e-Health implementation by means of:

� Systematic review of the literature (WP 1)

� Semi-structured interviews and task groups with health professionals (WP 2)

� Online surveys of the authors of published literature in this sphere and of health professionals at selected case study sites. (WP3)

� Case studies and semi-structured interviews with e-Health implementers (WP 4)

We have used mixed methods in order to produce recommendations and to develop instruments and tools that are ready for validation and testing by e-Health implementers. We have undertaken this work within 4 work-packages which have been described in detail in the preceding sections. Although the co-applicants have been involved in all of the work-packages, each work-package has been quite distinct, although it is worth noting that WP 2 and 3 shared two case study sites and WP 3 and WP 4 shared one, thus allowing different methods to be used at common sites. Additionally, in WP 1 and 2 we have analysed the same data using two different techniques: conventional thematic analysis and framework analysis.

The common thread to all the work-packages has been our use of the NPM as our theoretical underpinning. All the work-packages have used the NPM to guide and inform data collection and to aid analysis and interpretation. This has linked the work-packages from conception to analysis, and has enabled theoretical triangulation. The following section highlights our key findings and what has been gained by addressing this important issue with different methods, using the NPM as the theoretical model.

7.7 The “Added Value” Provided by the NPM and NPT and a multiple work-package approach.

WP 1 had an initial scoping exercise which was used to inform data collection in work-packages 2, 3 and 4. However, the key element of WP1 was our systematic review of the e-Health implementation literature and the particularly novel element was our use of the NPM as a coding framework for this work. Our systematic review highlighted a range of weaknesses in the literature, both in terms of content and methodology. We undertook a conventional thematic analysis of this literature. In this approach, thematic categories were identified in the course of data analysis. This identified a



list of barriers and facilitators to implementation that included: conditions prior to implementation; costs; the need for and impact of validation and evaluation; professional attitudes; ease of use of systems; security, confidentiality and standards; education and training; technological and communication issues.

Although our review had highlighted the general methodological flaws in the literature and the lack of depth to many of the issues described, the main gaps, if any, in the literature were not immediately apparent. However, coding using the NPM as a coding frame made it immediately clear that the literature over-emphasised organisational issues. This meant that the literature gives insufficient attention to problems of workability and integration for health professionals working with e-Health technologies ‘on the ground’. This matters because of the important role that systematic reviews play in informing policy debates. Figure 2. pg 43 of Section 1.7 illustrated via a radar plot that the distribution of codes across the model showed how the focus of the literature was upon contextual integration issues – how an organisation resources and delivers an e-Health technology.

This lack of attention to the effects of e-Health services on the interaction between users, the technology and their workplace is a major gap in the evidence base and provides a misleading picture of e-Health implementation barriers and facilitators as it clearly does not provide a holistic view of the processes at play. The NPM therefore served as a guide to conceptualising the literature. Our analysis also demonstrated that there were policy relevant issues such as the importance of evaluation that were not covered by the NPM. Further analysis using the extended model, the NPT, allowed the vast majority of these to be coded and their role in the implementation and integration process more clearly conceptualised and reflected in our recommendations which should be of value to policymakers and e-Health implementers.

Importantly, our analysis using the NPT across both WPs 1 and 2 served to demonstrate the robustness of the NPM as a model as it was clear that the NPM addressed the bulk of important implementation issues. The NPT then captured most other issues, and data which did fall outside the NPT may have done so merely because the issues were framed too generally, for example, having insufficient detail to permit categorisation.

WP2, which involved key informant interviews and task groups, generated a vast amount of qualitative data. This data was again analysed thematically, using NPM as a coding framework. Using the two methods had clear benefits. First, use of a thematic coding approach highlighted immediately the emphasis users placed on the issue of benefits in determining whether to utilise any new e-Health service. It was immediately apparent that evidence of benefits, particularly for patients, would be a major driver for e-Health implementation. This was an issue, notable by its absence from the literature. Coding to the NPM showed that health professionals took a much more holistic view of implementation issues (compared with representation of the problem as evident in the literature review) as was illustrated by the distribution of codes across the NPM in Figure 3, section 4.6, which were fairly evenly distributed.

Thus socio-technical issues received greater attention from health professionals with relational integration issues (which refer to confidence in the system) and accountability issues featuring prominently. This reflects



the attention given to perceived benefits as an issue of importance as highlighted by the thematic analysis. This also helped us conceptualise our key recommendations, leading us to emphasise the need to communicate a clear rationale for the implementation of any e-Health service, in terms of both professional and patient benefits, to health professionals in order to increase confidence that the effort required to get the system into everyday usage will be worthwhile to themselves and their patients.

There is therefore “added value” gained from undertaking an initial emergent analysis (free coding) in addition to the coding using the NPM and NPT as a framework. This is because our findings demonstrate that although the NPM (and to a greater extent the NPT in the higher order analysis) explained most of the key issues, some themes within the data may have been less clearly conveyed. For example, it was the thematic coding that very clearly highlighted the importance of benefits as an issue of importance for health professionals. Free coding therefore ensured that all important issues were identified from the data, which might have appeared less prominent within the constraint of applying a pre-determined framework for the whole analysis.

WP3 used the constructs of the NPM to inform development of the items for use within TARS, but this too was informed by the scoping review conducted in WP1. Questions for TARS were generated initially by drawing on the NPM (and later added to by the NPT), but were also mapped against the key emergent findings of the scoping review to ensure that key issues concerning the implementation of e-Health as represented in the literature were included in the instrument. This again illustrates how the different work-packages were used to inform one another.

Our expert survey of the instrument showed that the items developed were deemed to be important and conceptually valid, thereby, reinforcing our view of the value of the NPM. The surveys of academics and health professionals, undertaken with TARS, showed interesting differences in ratings of importance attributed to statements reflecting factors within the model that are perceived to influence the routinisation of e-Health. These differences resonated with the differences in perspective noted in WP1 and WP2. The preliminary survey of experts (review authors), showed that Contextual Integration issues were accorded primary importance, reflecting similar results as our systematic literature review (WP1), while this emphasis on contextual integration issues was far less apparent in our surveys of health professionals, again reflecting the findings of our interviews with health professionals in WP2, where they were noted to take a much more holistic view of implementation issues. This triangulation of findings gives us added confidence in the robustness of our findings and the key recommendations which stem from them.

WP4 involved qualitative interviews with key informant implementers across three case study sites, one of which was shared with WP3. The content of the interview guide was informed by the NPM and responses coded against its constructs. We also searched the data for evidence of the degree to which each system studied had normalized, and for data which could not be coded using the NPM. This approach allowed us to determine that the level of normalisation of different e-Health services could be explained by the degree to which they had a positive impact on different issues relating to the four constructs of the NPM.



Our findings have important implications for policy makers and e-Health implementers as we have demonstrated that the NPM is a valuable tool for assessing the likely normalization of a new e-Health initiative. E-Health initiatives that score highly on all four constructs (contextual integration, interactional workability, relational integration and skill set workability), are likely to normalize. While low scores on any one construct should alert policy makers and senior managers to potential difficulties in this area, which need careful consideration during and after implementation. Low scores across all four constructs suggest that the initiative has relatively low likelihood of normalizing successfully, and that some rethink may be needed. In our view this represents a fundamental advance in knowledge in this field.

In addition, the findings from all four work-packages, but particularly WP4, have been used to develop an e-Health implementation toolkit (e-HIT) which can serve as a sensitizing tool for senior managers or other staff who are considering, planning or undertaking an e-Health initiative implementation. The e-Hit was derived from data from our work-packages and as these were all informed by the NPM, the concepts of the model are well represented within it. The theoretical validity of this toolkit came from the NPM, while the literature review and WP4 provided empirical support. Thus confirming the added value of our approach and how triangulation of data has added strength to our findings and helped us to produce concrete deliverables that will be of practical value to those involved in e-Health implementation.

7.7.1 Discussion

This further analysis of WP1 and WP 2 was undertaken because: a) we had noted that there were a number of gaps when using the NPM alone which we believed related to issues of some importance and relevance to policymakers and implementers in particular; b) it provided an opportunity to examine how well the NPT addressed these gaps; and c) it helped us to further conceptualise the barriers and facilitators to the implementation and integration of e-Health services. We believed it would be useful to undertake this work with two quite distinct types of data sets, the first being that arising from a systematic literature review, and the second, qualitative task groups and interviews. Such an approach would allow us to examine whether coding to the NPT would produce similar results when used in this way in quite distinct contexts.

Two key findings from these further analyses were:

� It is clear that the NPM effectively covered most important implementation and integration issues. This is shown clearly by the comparison of results of the analyses from the two datasets.

� The extended NPT addresses implementation problems not covered by NPM and offers a greater level of explanatory sophistication.

Recommendations

Additional recommendations stemming from these analyses are:



1. Engagement with users of e-Health services prior to and throughout the implementation and integration of new services is essential in order to maximise the potential for normalisation – both in terms of facilitating their understanding of the purpose and benefits of the system (coherence), and preparing them for effective participation in using the system (cognitive participation).

Such engagement should include professional and lay users of such systems and is likely to facilitate more effective uptake and utilisation of e-Health services.

2. Ongoing evaluation and feedback of results of the implementation of e-Health systems (reflexive monitoring) is crucial and should be an integrated part of any e-Health implementation process.

Such an approach is necessary in order to allow iterative development of systems to meet user’s needs and to ensure that users can be confident in the benefits and safety of e-Health services.



8 Conclusions/Key Findings

8.1 Overview

This project has explored barriers and facilitators to e-Health implementation by examining the published literature and exploring the perspectives of health professionals and e-Health implementers. Importantly, we have also developed practical tools for use by those tasked with implementing e-Health services, which represent a fundamental advance. This multi-perspective study of e-Health has major policy implications as we have highlighted barriers to e-Health implementation and identified strategies to promote e-Health implementation. Consequently, our findings should be of value to policymakers, e-Health implementers (health professionals/managers/industry) and researchers. In section 8.1.1 onwards we present our key findings and highlight matters that need to be considered when planning the introduction of new e-Health services. There are numerous illustrations within the full body of this report of the issues raised below.

8.1.1 Implications for Policymakers

1. Development of a simple taxonomy for e-Health

E-Health is a broad term with numerous definitions. Pagliari’s review (Pagliari et al 2005), commissioned by SDO, listed countless definitions for e-Health. A major challenge for policymakers tasked with communicating e-Health strategies to users (both lay and professional),is how best to describe and classify e-Health services in an accessible and easily understood manner. A major task in this project, which dealt with the broad field of e-Health, was to develop a classification of e-Health systems that would be easily understood by study participants. We have therefore created and shown that it is a useful, simple four domain taxonomy of e-Health with clear definitions, which can be used by policymakers to facilitate communication with e-Health users. The four domains of e-Health are:

� Management systems

� Communication systems

� Computerised Decision Support systems

� Information Systems.

Full definitions appear in Section 1.3, Box 1

2. Findings from systematic reviews of e-Health implementation are misleading

Our systematic review of published reviews of e-Health implementation shows that this literature is problematic and misleading because:

� The published literature is not robust, as it is methodologically flawed, so conclusions must be viewed with caution.



� It appears that the fundamental assumptions underpinning research in this area are unsatisfactory in that there is undue focus on purely organisational issues and insufficient attention being paid to problems of workability, that is, the work that health professionals, as individuals or teams, must undertake to make these systems function satisfactorily in practice. There is, for example, little attention given to how e-Health systems affect roles and responsibilities and whether they help or hinder health professionals in completing key tasks.

3. At the present time there is little to be gained by commissioning a further systematic review in this field.

Although the literature in this field has methodological limitations, the findings resonate with two other reviews recently conducted by the co-applicants. Based on this fact, and our knowledge of the paucity of high quality primary literature, we believe it is unlikely that a further review, even if robust, would add anything new to our knowledge in this area at the present time. We would therefore not recommend this as a good use of scare research funds.

4. Monitoring and evaluation should be an integral component of new e-Health Services that are being commissioned.

A great deal of money is being invested in e-Health services, as they are seen as having the potential to address many of the challenges of health care delivery in the world today. However, we live in an era of evidence based medicine and professionals expect that there should be evidence of benefit when new services are introduced. Importantly, lessons should be learned from both successful and unsuccessful e-Health implementations and shared with users in order to increase the chances for successful implementation and prevent wasted investment in repetition of initiatives, which experience shows are unlikely to succeed.

Furthermore, technologies are constantly undergoing development and a key theme from our work is the need for flexibility of systems and the ability to further develop and refine e-Health systems over time based on users’ experiences of the advantages and disadvantages of the systems in practice. To permit such iterative development requires evaluation and feedback to be an integral part of e-Health service delivery. This does not necessarily mean full scale research proposals but rather ongoing evaluation that can feed iteratively into the development and improvement of systems. Thus, policymakers must ensure that when commissioning new e-Health services they also contract for ongoing evaluation and an obligation to undertake iterative development of systems, as necessary based on user feedback. This should increase the likelihood that systems will be responsive to users’ needs and ensure knowledge obtained from current e-Health initiatives can be mobilised and utilised to inform future implementations.

5. The e-HIT is a useful tool which could be used by policymakers to inform the planning of new e-Health initiatives.

The e-HIT (http://www.ucl.ac.uk/pcph/research/ehealth/tools.htm) has not been designed as a research tool but as a means to highlight where areas of potential difficulty may exist with regard to specific planned e-Health



services. It looks like this tool would act as a sensitising tool that could be of practical value. Policymakers could use this instrument to help them identify potential areas of difficulty for any planned implementation which would then enable them to consider strategies to overcome potential barriers in advance of roll out of services.

6. Enhanced engagement with users, both lay and professional is likely to promote uptake and utilisation.

� Formalised mechanisms need to be put in place to ensure the establishment of an ongoing three way dialogue between designers, implementers and professional users

Although engagement exercises are often undertaken, at present these appear to be less effective than anticipated. This is possibly because engagement is concentrated on the wrong individuals, in particular, managers rather than the users of the systems. Engagement needs to be begun prior to roll out of the new technology to ensure that technology being purchased meets users’ specifications and needs to be ongoing to facilitate iterative development of systems. Ideally, the need for such ongoing interaction should feature in contractual arrangements.

� Communicating a clear rationale for implementing e-Health services.

A strong, yet recurring theme from our research was the need to demonstrate likely benefit and communicate a clear rationale for any e-Health service to users, particularly in relation to potential benefits of these systems. It was slightly surprising to the research team that this remains a problem for professionals on the ground. Although efforts have already been made to communicate benefits of e-Health services we believe it is essential that policymakers invest more money and energy into engaging with users, be they lay or professional, in a very specific manner in advance of implementation of e-Health services in specific contexts to increase the likelihood of future uptake and utilisation. It appears that generic engagement efforts need to be supplemented by targeted, specific local engagement efforts. Unless the benefits of any particular planned e-Health service can be seen to balance the effort required locally to make the system work then integration of new services will remain slow and implementation problematic. Where a new technology is seen to have clear benefits, PACS representing an excellent example, then implementation and normalisation into routine service delivery is far more likely.

7. Safety and reliability e-Health systems must be made transparent to users.

The fact that this remains a problem for professionals on the ground is an unexpected finding. Health professionals implementing any new technology must first and foremost be convinced that it is a “safe” thing to do, and one important aspect of this is knowing that “back up” or “over-ride” systems are in place for immediate use if anything goes wrong and that they can have real confidence in the security of the systems that are coming into play. Many people point to the successful implementation of new technologies within other sectors such as banking, where issues such as confidentially are also of importance, and question why it seems to be so



much more difficult to get new technologies implemented and integrated into the health care arena. However, a key factor that sets health care apart from other services is the fact that whereas in other sectors system failures may be a nuisance and inconvenience, in health care, system failures can literally have “life or death” consequences. Health professionals have been trained to be acutely aware of the “risk” element of their work and are therefore understandably intolerant of system failure. This may be linked to growing awareness of medico-legal issues. However, it is not just the well being of patients that is a major concern but also the safety of patient data which also appears to be a greater problem than in other sectors and may be due to the peculiarly sensitive nature of medical information. If professionals are not confident in the safety and reliability of e-Health systems uptake and utilisation will be resisted. While policymakers are likely to be cognisant of this as an issue, it is clear that this concern is still not being addressed to the satisfaction of health professionals and continues to serve as a barrier to utilisation. It is also noteworthy that there is likely to be variation between types of e-Health systems (management, communication, computerised decision support, information systems), with regard to the extent to which this is viewed as a cause for concern.

8. System compatibility

� There is a Need to Establish a Balance Between Individual Requirements and Standardisation

It is well accepted that standards and interoperability are important when introducing new e-Health systems in order to facilitate communication between sectors and data sharing. However, it is important to note that within the NHS, users often have quite different experiences and backgrounds when it comes to e-Health systems. Some are already quite advanced and experienced users, with well established systems in place, while others are relatively inexperienced. This creates a tension when trying to implement “universal” systems which needs to be recognised and addressed by policymakers. This is most likely to be most effectively addressed by increased engagement with users as outlined in recommendation 6 above.

9. Tensions between national and local policy priorities

� In order to increase the chances for successful implementation of any new e-Health service it is important that local and national policies and priorities are congruent.

It is clear that if national and local priorities match then this will promote successful implementation whereas the converse is true. Policymakers need to acknowledge this problem and ensure efforts are made to minimise conflicting strategies when trying to implement e-Health systems otherwise it is unlikely that an e-Health service will have sufficient local resources and support made available to permit integration into routine service delivery.

8.1.2 Implications for e-Health implementers.

As mentioned in section 8.1.1, within this project we have developed an e-Health implementation toolkit, the e-HIT, which we believe could serve as a useful tool for those planning to implement e-Health services. In particular, we would like to highlight the following issues that merit consideration by e-



Health implementers and which can be addressed in greater detail through use of the e-HIT:

1. The overall context

This includes;

� National priorities and strategies

� Local priorities

� Leadership

� Resources.

In order to increase the chances for successful implementation of any new e-Health service it is important that local and national policies and priorities are congruent and that there is senior leadership, both managerial and health professional, supporting the initiative. In addition, adequate resources, both financial and personnel, are essential in order to facilitate implementation of e-Health services. For example, without adequate initial and ongoing IT support e-Health initiatives will struggle to become integrated as part of routine service delivery. It is also important that implementation and integration is seen as a long term and ongoing activity rather than a short term activity as implementation of new services will invariably require much more time and effort than initially envisaged.

2. The nature of the e-Health initiative

In particular it is important that consideration is given to:

� User-friendliness

� Effects, if any, on Clinical Interactions

� Effects, if any, on Inter-professional relationships

Our research has demonstrated that the ease of use of systems, and the extent to which systems help professionals to complete tasks efficiently and effectively impacts greatly on implementation. Systems which hinder the smooth and efficient delivery of care and which are deemed unreliable or insecure will not be welcomed and are less likely to be successfully implemented. It is therefore important that contracts with suppliers include a substantial element of ongoing support so that difficulties encountered by users can be addressed as part of the implementation process.

3. The impact of the implementation on the workforce

This includes attention to a broad range of issues that includes:

� Workload

� Training needs

� Alterations in roles and/or responsibilities.

Education, training and ongoing support is crucial. Such training should not be simply focused on how to use the system but also should address limitations of the technology and how to optimise potential benefits of the technology. E-Health implementers need to try to anticipate likely effects on workload and roles and responsibilities and sure these issues are being examined and addressed throughout the implementation process.



8.1.3 Implications for Research

Our work has demonstrated that the evidence base in this area is weak. We therefore highlight the following areas for research.

1. Primary e-Health research to examine implementation processes

In particular there is a pressing need to address the following research questions:

� What are the key steps in any implementation process?

� How can these key steps or processes be classified and what is their relative contribution to implementation?

� How can researchers help e-Health implementers to predict which services will or will not become normalised into everyday practice? .

Our work has demonstrated that the evidence base in the area of e-Health implementation research is unexpectedly weak. Barriers and facilitators to e-Health implementation have been identified but their relative importance remains uncertain. For example, it is unclear whether lack of engagement or champions is outweighed by the presence of adequate funding and congruent local and national policies or vice versa. There is therefore a need for studies that examine the implementation process in a much more detailed and thorough manner.

Crucially, this project has led to the development of robust tools and instruments which can be utilised by researchers, e-Health implementers and policy makers when considering and/or introducing e-Health services. We have developed a technology adoption readiness scale (TARS) as part of WP3 and an e-Health implementation toolkit (e-HIT) as part of WP4. These developments represent a potentially fundamental advance in this area of research as these tools and instruments have strong conceptual underpinnings and have been developed through the rigorous analysis of an extremely broad range of data. We would suggest that further research be commissioned to enable:

2. Testing of the general and predictive utility of e-HIT

It would be useful to conduct longitudinal studies of e-HIT in contexts where assessment of perceptions can be undertaken prior to the introduction of an e-Health system in order to determine the true potential value of e-HIT.

3. The development of different versions of the Technology Adoption Readiness Scale (TARS)

� It would be useful to explore and develop different ways of wording TARS items for different purposes

� To create guidance for its use for different purposes and in different contexts.



4. Testing of the statistical properties of TARS as a research scale

� It will be important to test TARS, in contexts where the necessary requirements concerning sample sizes and response rates can be met.

Although not the focus of this study, our work has clearly contributed significantly to the growing portfolio of studies suggesting the value of the NPM and of the extended NPT as conceptual models. The NPM and NPT have demonstrated in this study that they can be used to aid understanding of the implementation of e-Health services and have explanatory value in terms of clarifying why e-Health services have or have not been successfully implemented. The NPT obviously has implications as a theory beyond the context of e-Health and also further potential specifically in the area of e-Health. We would therefore recommend that further research is needed to:

5. Examine the predictive power of the NPM and NPT

It would be valuable to determine to what extent the NPM and NPT can be proved useful as a means of predicting whether any particular e-Health implementation will or will not be successful.

6. To examine the wider utility of the NPT.

It will be important to explore how useful the NPT is as a theoretical model beyond the field of e-Health. In particular, it will be important to explore what role this theory is able to play in addressing a full range of complex interventions and policies.



References

Al-Qirim NA. 2005. Critical success factors for strategic telemedicine planning in New Zealand. Telemed J E Health. 11(5): 600-7.

Anderson JG, Aydin CE. 1997. Evaluating the impact of health care information systems. Int J Technol Assess Health Care. 13(2): 380-93.

Anderson JG. 2000. Computer-based ambulatory information systems: recent developments. J Ambul Care Manage. 23(2): 53-63.

Anderson JG. 2007. Social, ethical and legal barriers to e-health. Int J Med Inform. 76(5-6): 480-3.

Angood PB. 2001. Telemedicine, the Internet, and World Wide Web: overview, current status, and relevance to surgeons. World J Surg. 25(11): 1449-57.

Anonymous [No authors listed]. 1999. Telemedicine: an overview. Health Devices. 28(3):88-103.

Barry CA, Britten N, Barber N, Bradley C, Stevenson F. 1999. Using reflexivity to optimize teamwork in qualitative research. Qual Health Res. 9(1):26-44.

Becker, H S, Hughes, E C, Geer, B, and Strauss, A.L 1961. Boys in White: Student Culture in Medical School. Chicago: University of Chicago Press.

Bell J. 2007. PCTs still only half way to choose and book target. Health Serv J. 117(6050):9.

Bond GE. 2006. Lessons learned from the implementation of a Web-based nursing intervention. Comput Inform Nurs. 24(2): 66-74.

Broens, T. H. 2007. Determinants of Successful Telemedicine Implementations: a Literature Study. Journal of telemedicine and telecare. 13, 303-309.

Campbell, J. D., Harris, K. D., & Hodge, R. 2001. Introducing telemedicine technology to rural physicians and settings. Journal of Family Practice. 50(5), 419-424.

Carrino JA, Unkel PJ, Miller ID, Bowser CL, Freckleton MW, Johnson TG. 1998. Large-scale PACS implementation. J Digit Imaging. 11(3 Suppl 1): 3-7.

Chaudhry, B. 2006. Systematic Review: Impact of Health Information Technology on Quality, Efficiency, and Costs of Medical Care. Annals of internal medicine. 144, 742-752.

Childs S, Blenkinsopp E, Hall A, Walton G. 2005. Effective e-learning for health professionals and students--barriers and their solutions. A systematic review of the literature--findings from the HeXL project. Health Info Libr J. 22 Suppl 2:20-32.

Collin S, Reeves BC, Hendy J, Fulop N, Hutchings A, Priedane E. 2008. Implementation of computerised physician order entry (CPOE) and picture archiving and communication systems (PACS) in the NHS: quantitative before and after study. BMJ. 337:a939.



Cook DA, Dupras DM. 2004. A practical guide to developing effective web-based learning. J Gen Intern Med. 19(6):698-707.

Copeland M. 2002. e-Community health nursing. J Holist Nurs. 20(2):152-65.

De Lusignan S. 2005. The barriers to clinical coding in general practice: a literature review. Med Inform Internet Med. 30(2):89-97.

Department of Health. 2000. The NHS Plan - A plan for investment a plan for reform. Department of Health.

Department of Health. 2001. Building a Safer NHS for Patients: Implementing an organisation with a memory. Department of Health.

Department of Health. 2002a. Delivering the NHS Plan: The next steps on investment, next steps on reform. Department of Health.

Department of Health. 2002b. Delivering 21st Century IT Support for the NHS: National Strategic Programme. http://www.dh.gov.uk/PublicationsAndStatistics/Publications/. Department of Health.

Department of health 2004, Improving Chronic Disease management. Department of Health, London.

Dinh M, Chu M. 2006. Evolution of health information management and information technology in emergency medicine. Emerg Med Australas. 18(3):289-94.

Dixon -Woods M, Agarwal S, Young B, Jones D, Sutton A. 2004. Integrative approaches to qualitative and quantitative evidence. NHS Health Development Agency, Leeds.

Dixon-Woods M, Agarwal S, Jones D, Young B, Sutton A. 2005. Synthesising qualitative and quantitative evidence: a review of possible methods. J Health Serv Res Policy. 10[1], 45-53.

Elford DR. 1997. Telemedicine in northern Norway. J Telemed Telecare. 3(1):1-22.

Elwyn, G., et al. 2008. Arduous implementation: does the normalisation process model explain why it is so difficult to embed decision support technologies in routine clinical practice. Implement Sci. 31;3:57.

Eng TR. 2001. The e-Health Landscape: A Terrain Map of Emerging Information and Communication Technologies in Health and Health Care. The Robert Wood Johnson Foundation.

EU Commission. 2004. Health – Making Healthcare Better for European Citizens: An Action Plan for a European e-Health Area, EU Communication, COM 356. EU Commission.

EU Commission. 2005. i2010 - A European Information Society for growth and employment. EU Commission.

Falas T, Papadopoulos G, Stafylopatis A. 2003. A review of decision support systems in telecare. J Med Syst. 27(4):347-56.

Garritty C, El EK. 2006. Who's using PDAs? Estimates of PDA use by health care providers: a systematic review of surveys. J Med Internet Res. 8(2):e7.

Gask, L., et al. 2008. Beyond the limits of clinical governance: the case of mental health in primary care. BMC Health Services Research. Vol. 8.



Geibert RC. 2006. Using diffusion of innovation concepts to enhance implementation of an electronic health record to support evidence-based practice. Nurs Adm Q. 30(3):203-10.

Goldberg MA, Dwyer SJ 3rd. 1995. Telemammography: implementation issues. Telemed J. 1(3):215-26.

Grams RR, Moyer EH. 1997. The search for the elusive electronic medical record system--medical liability, the missing factor. J Med Syst. 21(1):1-10.

Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O. 2004. Diffusion of innovations in service organizations: Systematic review and recommendations. Milbank Quarterly. 82:581-629.

Guler NF, Ubeyli ED. 2002. Theory and applications of telemedicine. J Med Syst. 26(3):199-220.

Hanson CW, Marshall BE. 2001. Artificial intelligence applications in the intensive care unit. Crit Care Med. 29(2):427-35.

Hebert, M. A. 2006. Moving Research into Practice: A Decision Framework for Integrating Home Telehealth into Chronic Illness Care. International journal of medical informatics. 75, 786-794.

Hebert, M. A., Paquin, M. J., & Iversen, S. 2002. Predicting success: stakeholder readiness for home telecare diabetic support. Journal of Telemedicine and telecare. 8, 33-36.

Hendy J, Fulop N, Reeves BC, Hutchings A, Collin S. 2007. Implementing the NHS information technology programme: qualitative study of progress in acute trusts. BMJ. 334(7608):1360.

Hilty, D. M. 2002. Telepsychiatry: an Overview for Psychiatrists. CNS-drugs. 16, 527-548.

Houtchens BA, Allen A, Clemmer TP, Lindberg DA, Pedersen S. 1995. Telemedicine protocols and standards: development and implementation. J Med Syst. 19(2):93-119.

Hunt RC. 2002. Emerging communication technologies in emergency medical services. Prehosp Emerg Care. 6(1):131-6.

Hussein R, Engelmann U, Schroeter A, Meinzer HP. 2004. DICOM structured reporting: Part 2. Problems and challenges in implementation for PACS workstations. Radiographics. 24(3):897-909.

Jaatinen PT, Forsstrom J, Loula P. 2002. Teleconsultations: who uses them and how? J Telemed Telecare. 8(6):319-24.

Jadad AR. 2002. Evidence-based decision making and asthma in the internet age: the tools of the trade. Allergy. 57 Suppl 74:15-22.

Jennett P, Watanabe M, Igras E, Premkumar K, Hall W. 1996. Telemedicine and security. Confidentiality, integrity, and availability: a Canadian perspective. Stud Health Technol Inform. 29:286-98.

Jennett PA, Andruchuk K. 2001. Telehealth: 'real life' implementation issues. Comput Methods Programs Biomed. 64(3):169-74

Jennett, P., Jackson, A., Healy, T., Ho, K., Kazanjian, A., Woollard, R., et al. 2003a. A study of a rural community's readiness for tele-Health. Journal of Telemedicine and telecare. 9(5), 259-263.



Jennett, P., Yeo, M., Pauls, M., & Graham, J. 2003b. Organizational readiness for telemedicine: implications for success and failure. Journal of Telemedicine and telecare. 9, S27-S30.

Jennett PA, Scott RE, Affleck Hall L, Hailey D, Ohinmaa A, Anderson C, et al. 2004. Policy implications associated with the socioeconomic and health system impact of telehealth: a case study from Canada. Telemed J E Health. 10(1):77-83.

Jennett, P. A. 2005. Preparing for Success: Readiness Models for Rural Telehealth. Journal of postgraduate medicine. 51, 279-285.

Jerant AF. 1999. Training residents in medical informatics. Fam Med. 31(7):465-72.

Johnson, K. B. 2001. Barriers That Impede the Adoption of Pediatric Information Technology. Archives of pediatrics and adolescent medicine. 155, 1374-1379.

Kaplan B, Shaw NT. 2004. Future directions in evaluation research: people, organizational, and social issues. Methods Inf Med. 43(3):215-31.

Karlsson D, Forsum U. 2004. Medical decision-support systems and the concept of context. Med Inform Internet Med. 29(2):109-18.

Katz AS, Tilkemeier PL. 1997. Multimedia image display: a view to the future. Curr Opin Cardiol. 12(6):566-70.

Kawamoto, K. 2005. Improving Clinical Practice Using Clinical Decision Support Systems: A Systematic Review of Trials to Identify Features Critical to Success. British Medical Journal. 330, 765-768.

Kukafka, R. 2003. Grounding a New Information Technology Implementation Framework in Behavioral Science: a Systematic Analysis of the Literature on IT Use. Journal of biomedical informatics. 36, 218-227.

Kvale, S. 1996. InterViews. London: Sage.

Lapointe L, Rivard S. 2006. Getting physicians to accept new information technology: insights from case studies. CMAJ. 23;174(11):1573-8.

Lawrenson R, Williams T, Farmer R. 1999. Clinical information for research; the use of general practice databases. J Public Health Med. 21(3):299-304.

Leatt, P. 2006. IT Solutions for Patient Safety--Best Practices for Successful Implementation in Healthcare. Healthcare quarterly Toronto Ont. 9, 94-104.

Lehman, W. E. K., Greener, J. M., & Simpson, D. D. 2002. Assessing organizational readiness for change. Journal of Substance Abuse Treatment. 22(4), 197-209.

Lehmann ED. 2004. Computerised decision-support tools in diabetes care: hurdles to implementation. Diabetes Technol Ther. 6(3):422-9.

Liaskos J, Mantas J. 2002. Nursing Information System. Stud Health Technol Inform. 65:258-65.

Link RE, Schulam PG, Kavoussi LR. 2001. Telesurgery. Remote monitoring and assistance during laparoscopy. Urol Clin North Am. 28(1):177-88.

Implementation research: state of the art and future directions

JD Linton. 2002. Technovation. Elsevier.



Lu YC, Xiao Y, Sears A, Jacko JA. 2005. A review and a framework of handheld computer adoption in healthcare. Int J Med Inform. 74(5):409-422.

Lu YC, Xiao Y, Mills ME, Soeken K, Vaidya V. 2006. Top barriers and facilitators to nurses' PDA adoption. AMIA Annu Symp Proc. 1016.

Mackinnon AD, Billington RA, Adam EJ, Dundas DD, Patel U. 2008. Picture archiving and communication systems lead to sustained improvements in reporting times and productivity: results of a 5-year audit. Clin Radiol. 63(7):796-804.

Maglogiannis I. 2004. Design and implementation of a calibrated store and forward imaging system for teledermatology. J Med Syst. 28(5):455-67.

Mair, F. S. 2008. Understanding factors that inhibit or promote the utilization of telecare in chronic lung disease. Chronic Illness. 4, 110-117.

Mairinger T. 2000. Acceptance of telepathology in daily practice. Anal Cell Pathol. 21(3-4):135-40. 40

Mandl KD, Kohane IS, Brandt AM. 1998. Electronic patient-physician communication: problems and promise. Ann Intern Med. 15;129(6):495-500.

Maulden SA. 2003. Information technology, the internet, and the future of neurology. Neurologist. 9(3):149-59.

May, C.; Gask, L.; Atkinson, T.; Ellis, N.; Mair, F.; Esmail, A. 2001a. Resisting and Promoting New Technologies in Clinical Practice: the Case of Telepsychiatry. Social Science & Medicine. 52, 1889-1901.

May, C., Ellis, N. T. 2001b. When protocols fail: technical evaluation, biomedical knowledge, and the social production of 'facts' about a telemedicine clinic. Social Science and Medicine. 53, 989-1002.

May, C.; Mort, M.; Williams, T.; Mair, F.; Gask, L. 2003a. Health Technology Assessment in Its Local Contexts: Studies of Telehealthcare. Social Science & Medicine. 57, 697-710.

May, C.; Harrison, R.; Finch, T.; MacFarlane, A.; Mair, F.; Wallace, P. 2003b. Understanding the Normalization of Telemedicine Services Through Qualitative Evaluation. Journal of the American Medical Informatics Association. 10, 596-604.

May, C.; Allison, G.; Chapple, A.; Chew-Graham, C.; Dixon, C.; Gask, L., et al. 2004. Framing the Doctor-Patient Relationship in Chronic Illness: a Comparative Study of General Practitioners' Accounts. Sociology of Health & Illness. 26, 135-158.

May, C. 2005. Chronic illness and intractability: professional-patient interactions in primary care. Chronic Illness. 1, 15-20.

May, C.; Rapley, T.; Moreira, T.; Finch, T.; Heaven, B. 2006. Technogovernance: Evidence, Subjectivity, and the Clinical Encounter in Primary Care Medicine. Social Science & Medicine. 62, 1022-1030.

May C. 2006 A rational model for assessing and evaluating complex interventions in health care. BMC Health Serv Res. 6:86:86.

May C, Finch T, Mair F, Ballini L, Dowrick C, Eccles M et al. 2007a. Understanding the implementation of complex interventions in health care: the normalization process model. BMC Health Serv Res. 19;7(1):148.



May, C., et al. 2007b. Process evaluation of complex interventions in primary care: understanding trials using the normalization process model. BMC Family Practice. 8.

May, C. 2007. The clinical encounter and the problem of context. Sociology. 41, 29-45.

May, C., Finch, T. 2009. Implementation, embedding, and integration: an outline of Normalization Process Theory. In Press.

May, C., Finch, T. L., Forthcoming. Normalizing Health Technologies. Palgrave, London.

McDonald CJ. 1997. The barriers to electronic medical record systems and how to overcome them. J Am Med Inform Assoc. 4(3):213-21.

Mort, M.; May, C. R.; Williams, T. 2003. Remote Doctors and Absent Patients: Acting at a Distance in Telemedicine? Science Technology & Human Values. 28, 274-295.

Mosley-Williams A, Williams C. 2005. Computer applications in clinical practice. Curr Opin Rheumatol. 17(2):124-8

Mun SK, Elsayed AM, Tohme WG, Wu YC. 1995. Teleradiology/telepathology requirements and implementation. J Med Syst. 19(2):153-64.

NHS Executive. 1998. Information for Health: An information strategy for the modern NHS 1998–2005. September 1998. NHS Executive. http://www.nhsia.nhs.uk/def/pages/info4health/contents.asp.

NHS Executive. 2001. Building the Information Core – Implementing the NHS Plan. NHS Executive

Nolan A, Whitfield L. 2005. Choose and book. By the book. Health Serv J. 115(5947):suppl-6.

Ohinmaa, A. 2006. What Lessons Can Be Learned From Telemedicine Programmes in Other Countries? Journal of telemedicine and telecare. 12, S40-S44.

Oliver, D. R. P., & Demiris, G. 2004. An assessment of the readiness of hospice organizations to accept technological innovation. Journal of Telemedicine and telecare. 10(3), 170-174.

Overhage, J. M., Evans, L., & Marchibroda, J. 2005. Communities' readiness for health information exchange: The national landscape in 2004. Journal of the American Medical Informatics Association. 12(2), 107-112.

Pagliari C, Sloan D, Gregor P, Sullivan F, Detmer D, Kahan JP, et al. 2005. What Is eHealth (4): A Scoping Exercise to Map the Field. J Med Internet Res. 7(1):e9.

Papshev, D. 2001. Electronic Prescribing in Ambulatory Practice: Promises, Pitfalls, and Potential Solutions. The American journal of managed care. 7, 725-736.

Peleg, M. 2006. Decision Support, Knowledge Representation and Management in Medicine. Yearb-Med-Inform.

Pellegrino L, Kobb R. 2005. Skill sets for the home telehealth practitioner: a recipe for success. Telemed J E Health. 11(2):151-6.

Pothier DD, Awad Z, Tierney P. 2006. 'Choose and Book' in ENT: the GP perspective. J Laryngol Otol. 120(3):222-225.



Rogers, E. M. 1995. The diffusion of innovation. New York: Free Press.

Rudowski R. 2003. Telemedicine in the context of different medical specialities. The Polish perspective. Pol J Pathol. 54(3):219-21.

Sable C. 2001. Telecardiology: potential impact on acute care. Crit Care Med. 29(8 Suppl):N159-65

Safran C, Goldberg H. 2000. Electronic patient records and the impact of the Internet. Int J Med Inform. 60(2):77-83.

Shekelle, P. G. 2006. Costs and Benefits of Health Information Technology. Evidence report technology assessment.

Sheng OR, Hu PJ, Au G, Higa K, Wei CP. 1997. Urban teleradiology in Hong Kong. J Telemed Telecare. 3(2):71-7.

Shiffman RN, Liaw Y, Brandt CA, Corb GJ. 1999. Computer-based guideline implementation systems: a systematic review of functionality and effectiveness. J Am Med Inform Assoc. 6(2):104-14.

Silverman, D. 1993. Interpreting Qualitative Data. London: Sage.

Snyder-Halpern, R. 2002. Development and pilot testing of an Organizational Information Technology/systems Innovation Readiness Scale (OITIRS). Proceedings of the AMIA 2002 Annual Symposium. 702-706.

Stanberry B. 2000. Telemedicine: barriers and opportunities in the 21st century. J Intern Med. 247(6):615-28.

Strauss, A. 1987. Qualitative analysis for social scientists. Cambridge University Press, Cambridge.

Studer, M. 2005. The Effect of Organizational Factors on the Effectiveness of EMR System Implementation--What Have We Learned? Healthcare-quarterly-Toronto,-Ont. 8, 92-98.

Styra R. 2004. The Internet's impact on the practice of psychiatry. Can J Psychiatry. 49(1):5-11.

Swinglehurst DA. 2005. Information needs of United Kingdom primary care clinicians. Health Info Libr J. 22(3):196-204.

Tierney WM. 2001. Improving clinical decisions and outcomes with information: a review. Int J Med Inform. 62(1):1-9.

Uhlenhopp MB, Fliedner MC, Morris P, Van Boxtel T. 1998. A global perspective on nurses' Internet access and information utilization. Oncol Nurs Forum. 25(10 Suppl):27-32.

US National Institute of Medicine. 2000. To Err to Human: Building a safer health system. Academy Press.

Van de WR, Batenburg R, Versendaal J, Lederman R, Firth L. 2006. A balanced evaluation perspective: picture archiving and communication system impacts on hospital workflow. J Digit Imaging. 19 Suppl 1:10-7.:10-17.

Van de WR, Batenburg R. 2008. A PACS maturity model: A systematic meta-analytic review on maturation and evolvability of PACS in the hospital enterprise. Int J Med Inform.

Vreeman, D. J.; Taggard, S. L.; Rhine, M. D.; Worrell, T. W. 2006. Evidence for Electronic Health Record Systems in Physical Therapy... Including



Commentary by Zimny NJ With Author Response. Physical Therapy. 86, 434-449.

Wallace S, Wyatt J, Taylor P. 1998. Telemedicine in the NHS for the millennium and beyond. Postgrad Med J. 74(878):721-8.

Wanless D. 2002. Securing Our Future Health: Taking a long-term view re-investment priorities. http://www.hm-treasury.gov.uk/ Consultations_and_Legislation/wanless/consult_wanless_final.cfm

Wanless, D. 2004. Securing Good Health for the Whole Population. February 2004. HM Treasury.

Weinstein RS, Descour MR, Liang C, Bhattacharyya AK, Graham AR, Davis JR, et al. 2001. Telepathology overview: from concept to implementation. Hum Pathol. 32(12):1283-99/

Whitten P, Love B. 2005. Patient and provider satisfaction with the use of telemedicine: overview and rationale for cautious enthusiasm. J Postgrad Med. 51(4):294-300.

WHO. 2003. Information sheet: facts related to chronic diseases and the

WHO global strategy on diet, physical activity and health [pdf 260kb] .

World Health Organisation.

Wilkes, S. 2007. Evaluation of open access hysterosalpingography in the initial management of infertility in primary care. Centre for Primary & Community Care, Vol. Ph.D. University of Sunderland.

Wootton R. 2001. Telemedicine and developing countries--successful implementation will require a shared approach. J Telemed Telecare. 7 Suppl 1:1-6.

Yarbrough, A. K. 2007. Technology Acceptance Among Physicians: A New Take on TAM. Medical-Care-Research-and-Review. 64, 650-672.

Yellowlees PM, Kennedy C. 1997. Telemedicine; here to stay. Med J Aust. 166(5):262-5.

Yellowlees PM. 2005. Successfully developing a telemedicine system. J

Telemed Telecare. 11(7):331-5

Yusof, M. M. 2007. Health Information Systems Adoption: Findings From a Systematic Review. Medinfo. 12, 262-266.



Appendix 1 NPM Definitions

Interactional workability

The operationalization of a complex intervention in the immediate conditions

in which professionals and patients encounter each other [1]

One way of thinking about interactional workability (IW) is thinking about

the work that goes on in an interaction – in this case the interaction

between a health professional and a patient. In other words, IW refers to

the impact of the new intervention or technology on the work done in a

consultation.

There are two parts (or dimensions) to the construct of interactional

workability. The first, “congruence”, is concerned with the interaction itself:

what should legitimately be dealt with in an interaction (e.g. a

consultation), what is the form of the work to be done, what is the role of

each participant in the interaction, how is the work to be completed in the

time and space available, and the formal and informal rules that govern the

verbal and non-verbal conduct of an interaction. The second, called

“disposal” concerns the effects of the interaction. Disposal considers the

goals of an interaction – e.g. forming a diagnosis, agreeing actions such as

investigations, treatment or follow-up, recording information – and how

disagreement about the outcome of the work is minimised, together with

shared beliefs about the meaning and consequence of the work.


The mediation of knowledge and practice about a complex intervention in

the network of relations in which clinical encounters between professionals

and patients are located [1]

For relational integration (RI), think about the organization of work and

knowledge around the clinical encounter. The key issue here is whether

those involved in the implementation of a new technology or intervention

trust each other and the work that they are doing individually or as a group.

Do they believe that there is valid knowledge and expertise in the network

of actors for the implementation of the new technology?



There are two parts (or dimensions) to RI. The first is accountability. Is

there agreement between those involved in the clinical encounter about the

forms and validity of knowledge associated with the new work? Does the

new working practice embody what clinicians personally regard as valid

(clinical) knowledge, as appropriate expertise and appropriate sources of

that expertise? [1]

The second is confidence. Is there agreement about sources of authoritative

knowledge and practice? What criteria are used to assess credibility? Does

the new technology or intervention disrupt beliefs about the practical utility

of knowledge that is mediated through the network?

Skill Set Workability

The mechanisms by which knowledge and practice about complex

interventions are distributed and performed in the formal and informal

divisions of labour in health care settings [1]

Skill set workability (SSW) relates to institutional framing of work and

divisions of labour. It is concerned with the skills that are involved in the

implementation of a new technology or intervention. The key questions are

who needs to do what to implement the new technology or intervention and

to what extent is that work compatible with existing professional roles and

identities? SSW is influenced by both formal and informal policies and

practices.

The first dimension of SSW is allocation. This is about the extent to which

the new technology or intervention requires a shift from existing skills and

the organization of those skills within and among professional groups. Are

the skills needed for a new technology compatible with the current skill base

and division of labour or does the new technology require change and

negotiation around who should do what to complete the work involved?

The second dimension of SSW is Performance. Does the work around a new

technology push boundaries between professional groups and their skill sets

and/or does it affect the levels of autonomy that professional are used to

and require for their work.


The capacity of an organisation to understand and agree the allocation of

control and infra-structure resources to implementing a complex



intervention, and to negotiating its integration into existing patterns of

activity[1].

Contextual integration (CI) refers to the work that needs to be done at the

level of the organisation in order for the complex intervention to become

normalised. There are at least three components that need considering: the

organisational commitment to deploying and sustaining a new intervention;

the capacity of the organisation to do this; and the value that accrues (or is

expected to accrue) to the organisation as a result of deployment.

An organisation may be committed to implementing a new technology or

way of working because this has been mandated from on high – for

example, many of the Connecting for Health targets have been imposed on

trusts, and whether they like it or not, they have to implement them, and

reach pre-determined targets in terms of implementation and use.

Alternatively, an organisation may have reached a local decision, and want

to implement a new way of working or new technology because they think it

will help them reach locally agreed targets or goals. Sometimes there will

be an overlap – a centrally mandated technology may help meet a locally

determined goal. But if there is no commitment on the part of the

organisation to implementation, it is highly unlikely that the innovation will

be implemented across the organisation, even if a few individual enthusiasts

adopt it.

Whatever the level of commitment to the innovation, implementation will

only be possible if the organisation has the capacity to implement. For

example, a hospital trust may decide to implement electronic prescribing,

but be unable to, as there are no computers on the wards, no budget to

purchase computers, and even if the budget were available, no space to put

the computers. Further examples of capacity would include the availability

of IT support to install and maintain the computers, and either a workforce

already trained in the use of the software, or the ability of the organisation

to ensure that the workforce receives training.

A third component of contextual integration is value that accrues to the

organisation as a result of deploying (and sustaining) an innovation. This

value may be financial, in terms of enhanced efficiency or increased

revenue; motivational – for example by increasing status; or around

delivery – for example by improving service delivery, performance or

adherence of staff to agreed operating procedures or protocols.

There are two domains to CI: execution and realisation. Execution refers to

the ownership of control over the resources and agents required to

implement a complex intervention, and includes three components. These



are: resourcing, power; and evaluation. Resourcing refers to the ability of

the organisation to direct adequate resources to an implementation – the

less an innovation disrupts existing flows of resources (which may be time

or money), the easier it will be to normalise. Power is about the ability of

organisations, or sub-groups within an organisation, to control resources.

Evaluation refers to the ability of the organisation (or sub-groups within an

organisation) to evaluate the work that has been done (or not done).

Realisation also has three components: risk, action, and value. Risk is

about the disruption to current working patterns and systems. Action is

about how decisions are implemented, while value is about perceptions of

the worth of the work that is required relative to the value of the innovation.

Reference List

1. May C, Finch T, Mair F, Ballini L, Dowrick C, Eccles M et al.:

Understanding the implementation of complex interventions in health care:

the normalization process model. BMC Health Serv Res 2007, 19;7: 148.



Appendix 2 List of papers included in WP1, Phase 1 scoping review

Authors Year Country of origin

Type of health care system

Type of review Details of methods and results (if applicable)

Key themes Domain(s)

Al-Qirim

NA

2005 New

Zealand

Public. Review of the

health strategy of

the New Zealand

government, with

reference to

appropriate

literature.

None stated,

but draws on

a very

specific and

small body

of literature.

Cost, co-

operation,

security,

confidentiality

and leadership.

Communication

systems.

Anderson

JG, Aydin

CE

1997 United

States.

American

(insurance

based).

Narrative review of

literature and

systems.

None

specified.

Barriers may

be reduced

through

strategic

planning,

evaluation,

designated

authority and

leadership,

making

changes

incrementally

(one step at a

time), allowing

time to adjust,

identifying

benefits for all

concerned,

finding

sponsorship,

training,

managed

expectations,

and a focus on

communicatio

n.

Information

systems.

Anderson

JG

2000 United

States.

American

(insurance

based).

Narrative review. Notes it is

not an

exhaustive

review;

Acceptance of

new systems,

data security,

need for

Information

systems.



review;

articles

selected to

convey the

breath of

computer-

based

ambulance

information

systems.

need for

standards,

cost.

Anderson

JG.

2006 United

States

(also

applies to

Europe,

Canada,

Australia

and New

Zealand).

Mixed. Narrative. review. Literature

analysis and

survey data

from

primary care

physicians

on adoption

of

information

technologies

is reviewed.

Barriers may

be removed

through better

funding,

government

incentives,

standardization

of systems,

improved

security and

legislation.

Generic.

Angood PB 2001 United

States.

Mixed, but

chiefly

focused on

American

(insurance

based).

Narrative review

on the use of

telemedicine,

illustrated with a

case-study of its

use on Mount

Everest.

None

specified.

Barriers:

licensing and

reimbursement

issues,

liability, mixed

quality of

services and

technical

issues.

Facilitator: It

can be used on

Mount

Everest!

Communication

systems.

Bond GE 2006 United

States.

American

(insurance

based).

Review the

development and

implementation of

a web-based

diabetes

information system.

None

specified.

Description

of process.

Centred on

training needs.

Information

systems.

Carrino JA, 1998 United American Review of PACS None Design of Communication



Unkel PJ,

Miller ID ,

Bowser CL,

Freckleton

MW,

Johnson TG

States. (insurance

based).

implementation in

the US Department

of Defense and

applicability to

public health

service

implementation.

specified. system (ease of

use),

promotion of

system and

education.

systems.

Childs S,

Blenkinsopp

E, Hall A,

Walton G

2005 United

Kingdom.

Public

(NHS).

Systematic

literature review.

Systematic

data-base

search and

articles

screened by

3

researchers

with strict

inclusion

criteria. 57

of an

original 161.

Costs, poor

design,

inadequate

technology,

lack of skills,

need for face-

to-face

interaction,

time, intensity,

computer

anxiety

(general

resistance).

Information

systems.

Cook DA,

Dupras DM

2004 United

States.

American

(insurance

based).

Narrative review. None

specified.

Preparatory

needs analysis,

securing

commitment

from all

concerned,

appropriate

design, ease of

use,

evaluation,

piloting,

ongoing

problem-

solving

(continuous

implementatio

n/

integration).

Information

systems.

Copeland M 2002 United

States.

Not specific

(focused on

the internet

Theoretical article

including a

narrative literature

Online

search of

university

Barriers:

perceived ease

of use and

Information

systems.



as a global

phenomenon

).

review. library using

ProQuest,

excluding

none-

gendered

articles.

gendered

patterns of

communicatio

n.

de Lusignan

S

2005 United

Kingdom.

Public

(NHS).

Systematic

literature review.

Medline and

departmental

website

searches.

Clinicians’

attitudes, skill

with

computers,

technical

problems and

organisational

factors.

Management

systems.

Dinh M,

Chu M.

2006 Australia

(but

covers

English

language

studies

from

across the

globe).

Mixed. Systematic

literature review.

Medline,

references,

organisation

al websites.

Facilitators:

good

leadership,

governance

and

collaboration.

Generic.

Elford DR 1997 Norway Norwegian

(public).

Review of

telemedicine in

northern Norway.

None

specified.

Facilitators

include good

state funding,

the support of

opinion

leaders,

enthusiasm for

new

technologies

and lack of

legal

restriction.

Communication

systems.

Falas T,

Papadopoul

os G,

Stafylopatis

A

2003 Cyprus. Mixed. Narrative review of

computerised

decision support

systems.

None

specified.

Privacy,

security, cost,

efficiency of

systems,

managing

information.

Computerised

decision support

systems.

Garritty C,

El Emam K.

2006 Canada

(but

covers

English

Mixed. Systematic review

of PDA (personal

digitalised

assistants) usuage

Systematic

database

search with

strict

Facilitators:

effective

research and

evaluation,

Generic.



language

studies

from

across the

globe)..

surveys. exclusion

criteria. 23

articles

selected out

of 1775

found.

professionals

growing

accustomed to

technology

quickly.

Geibert RC. 2006 United

States.

American

(insurance

based).

Narrative review of

diffusion of

innovation research

relating to HER

(electronic health

records).

None

specified.

Barriers: too

much

information.

Facilitators:

good

leadership.

Management

systems.

Goldberg

MA, Dwyer

SJ 3rd

1995 United

States.

American

(insurance

based).

Review of

telemammography

systems and

components.

None

specified.

Technological

challenges;

systems which

are simple

enough to use.

Communication

systems.

Grams RR,

Moyer EH

1997 United

States.

American

(insurance

based).

Review of legal

issues in the use of

electronic medical

records.

None

specified.

The changing

nature of

medical

liability.

Management

systems.

Guler NF,

Ubeyli ED

2002 Turkey. None-

specific.

Describes

telemedicine

in an array

of different

countries.

Narrative review of

systems.

None

specified.

Fundamentally

, lack of

standardization

as a global

impediment.

Communication

systems.

Hanson CW,

Marshall

BE.

2001. United

States.

None

specific.

Systematic review

of the use of

artificial

intelligence in

intensive care.

Medline and

bilbliograhp

y search.

Facilitators: AI

is well-suited

to the intensive

care

environment,

reducing time

and cost,

improving

patient

welfare.

Generic.

Houtchens

BA, Allen

A, Clemmer

TP,

Lindberg

DA,

Pedersen S

1995 United

States.

American

(insurance

based).

Narrative review of

policy and

telemedicine

systems.

None

specified.

The need for

both

standardization

and flexibility.

Communication

systems.



Hunt RC 2002 United

States.

American

(insurance-

based).

Review of

communication

technology in

emergency

services.

Analysis of

an

automated

crash-

notification

service and

an

“enhanced

9-1-1

service”.

Efficient

system-design,

funding,

technological

short-comings,

concerns

regarding

privacy, staff

training.

Communication

systems.

Hussein R,

Engelmann

U, Schroeter

A, Meinzer

HP

2004 Germany. Not

specified

(presented

as generally

applicable).

Review of system

implementation/spe

cific system

(PACS).

None

specified.

Software

compatibility

with existing

systems and

the need for

standardization

.

Communication

systems.

Jaatinen PT,

Forsstrom J,

Loula P

2002 Finland. Mixed. Systematic

literature review.

Medline

search for

studies from

between

1966 and

2002. 128

articles

selected.

Appropriatene

ss of systems

for different

disciplines,

time factors.

Communication

systems.

Jadad AR 2002 Canada. Canadian

(insurance

based).

Review of

evidence-based

decision making

tools for asthma.

None stated

for the

assembly of

the paper,

but reviews

research

methods.

Describes

perceived

good

research

practice, but

is not self-

reflexive.

Barriers:

compatibility

with existing

systems.

Facilitators:

effective

technology.

General: the

need for

further

research.

Computerised

decision support.

Jennett P,

Watanabe

M, Igras E,

Premkumar

K, Hall W

1996 Canada Canadian

(insurance

based).

Review of systems

(security measures

for telemedicine.)

None

specified.

Chiefly

privacy and

confidentiality;

relates to

legality and

ownership of

Communication

systems.



the systems.

Jennett PA,

Andruchuk

K

2001 Canada. Canadian

(insurance

based).

Reviews

observations made

by Canadian

authorities as they

have implemented

tele-Healthe-Health

systems.

Not

comprehensi

vely stated.

Environmental

readiness

(policy issues

of privacy,

confidentiality,

security,

reimbursement

, standards and

licensing).

Communication

systems.

Jennett PA,

Scott RE,

Affleck Hall

L, Hailey D,

Ohinmaa A,

Anderson C,

Thomas R,

Young B,

Lorenzetti D

2004 Canada. Canadian

(insurance

based).

Systematic

literature review.

Search of

electronic

databases

(not named)

and hand

searches. 57

articles

found.

Need for

readiness for

tele-Healthe-

Health to be

measured prior

to

implementatio

n.

Communication

systems.

Jerant AF 1999 United

States.

Mixed. Systematic

literature review.

Medline

search 1966-

1999. 154

articles.

Training,

needs

assessment

(planning),

expert

recommendati

ons,

organizational/

faculty

support, user-

centred

approach, easy

access to

computers,

assessment.

Generic.

Johnson KB 2001 United

States.

American

(insurance

based).

Narrative review. Medline,

Google and

Norternlight

search for

articles on

barriers and

facilitators

to clinical IT

adoption.

Barriers:

“current

national health

environment”,

financial and

legal risks, and

lack of

knowledge and

training.

Generic.

Kaplan B,

Shaw NT

2004 United

States and

Canada.

Mixed. Narrative review. None

specified.

Potent

research

crucial to the

Generic.



development

and sustaining

of e-health

systems.

Karlsson D,

Forsum U

2004 Sweden

(covers

research

from an

array of

countries).

Mixed. Narrative review of

empirical and

theoretical studies.

No

information

on how the

studies were

chosen.

Decision-

support

systems are

used

differently by

professionals

and patients.

This may

strain the

professional–

patient

relationship.

Computerised

decision support.

Katz AS,

Tilkemeier

PL

1997 United

States.

American

(insurance

based).

Review of systems. None

specified.

Broader socio-

cultural factors

(the advent of

the digital age)

make the use

of digital

technologies

an

inevitability.

Management

systems.

Lawrenson

R, Williams

T, Farmer R

1999 United

Kingdom.

Public

(NHS).

Review of general

practice databases

for research;

systems review.

Analysis of

two database

systems:

VAMP

Medical and

Meditel.

Barriers: Cost

of access to

databases, size

and structure

of the

databases

(difficulty

extracting

readable

information.)

Management

systems.

Lehmann

ED

2004 United

Kingdom.

None

specific,

refers to

both British

and

American

contexts.

Review of a

decision support

system for

Diabetes.

None

specified.

Need for

research and

evaluation.

Barrier of

potentially

undermining

the role of the

physician.

Computerised

decision support.

Liaskos J,

Mantas J

2002 United

States.

None

specific.

Review of systems

and strategies.

None

specified.

Prior planning

and subsequent

evaluation

Information

systems.



important for

successful and

sustainable

implementatio

n.

Link RE,

Schulam

PG,

Kavoussi LR

2001 United

States.

American

(insurance

based).


specified.

Technological

barriers: time

and resources

taken up by

them,

reliability and

security; legal

barriers:

liability,

licensing,

privacy and

confidentiality.

Communication

systems.

Maglogianni

s I

2004 Greece. Greek

(public).


specified.

Doctors

acceptance the

most important

factor in

implementatio

n.

Communication

systems.

Mairinger T 2000 Austria. Mixed. Systematic

literature review

compared with

findings of 3

questionnaires (full

results of

questionnaires

published

elsewhere).

Medline

search. 283

articles (but

a

bibliography

of 24).

Communicatio

n and influence

(essentially

education),

cost,

knowledge

barriers

(training),

feasibility

(appropriatene

ss of system),

legality,

difficulties in

telediagnosis,

time-

consuming

without

reimbursement

.

Communication

systems.

Mandl KD,

Kohane IS,

Brandt AM

1998 United

States.

Mixed, but

mainly

American

(insurance-

based).

Narrative Review

of current

arguments on

electronic patient-

physician

No

information

as to how

the cited

articles were

Covers how

inappropriate

use, security

and

confidentiality

Communication

systems.



communication. found or

chosen.

issues can

effect

electronic

patient-

physician

communicatio

n.

Maulden SA 2003 United

States.

American

(insurance

based).

Narrative review of

the impact of

information

technology on the

practice of

neurology.

Uses

hypothetical

scenarios to

illustrate the

use and

impact of

information

technology

on the

practice of

neurology.

Security,

legality,

management of

large data-

bases (internet

searching etc),

quality of

information on

the Internet.

Generic.

McDonald

CJ

1997 United

States.

American

(insurance-

based).

Review of

electronic medical

records systems.

Narrative

based

around

systems used

in US

hospitals.

No

systematic

method.

Covers issues

of

overabundance

of knowledge

and

appropriate

levels of

knowledge for

electronic

medical

records.

Management

systems.

Mosley-

Williams A,

Williams C

2005 United

States.

Mixed. Review of systems,

policy and trends.

Medline

search

9/2003-

9/2004.

Systems can

save time and

money.

Professional

acceptance and

training key

factor.

Generic.

Mun SK,

Elsayed

AM, Tohme

WG, Wu

YC

1995 United

States and

Korea.

Insurance

based.

Review of common

components of

telemedicine

systems.

Description

of

components

with case

studies of

projects

(though

these

projects are

summarized

rather than

Technology

and cost.

Communication

systems.



researched).

[No authors

listed]

1999 United

States.

American

(insurance

based).

Review of

telemedicine

systems.

None

specified.

Barriers: lack

of

infrastructure,

unclear needs.

Facilitator:

adequate

training.

Communication

systems.

Papshev D,

Peterson

AM.

2001 United

States.

Mixed. Systematic review

of electronic

prescribing.

MEDLINE

search.

Barriers: lack

investment

capital,

segmentation

of the

healthcare

market, lack of

technology

standardization

, providers’

resistance to

change, and

regulatory

indecisiveness.

Management

systems.

Pellegrino L,

Kobb R

2005 United

States.

American

(insurance

based).


specified.

Education and

training,

integration into

regular

routine.

Communication

systems.

Rudowski R 2003 Poland. Public. Narrative review of

telemedicine and

policy in Poland.

None

specified.

Need for

standardization

in systems use

and policy.

Communication

systems.

Sable C 2001 United

States.

American

(insurance

based).


specified.

Software

design

limitations (not

specific

enough

software), cost,

whose fault

(designers or

users) that the

system has

problems, it

may “decrease

Communication

systems.



bedside

presence”,

legal issues,

physicians

discouraged by

limited

reimbursement

.

Sheng OR,

Hu PJ, Au

G, Higa K,

Wei CP

1997 Hong

Kong.

Public. Review of four

major teleradiology

services in Hong

Kong; review of

systems.

None

specified.

System

compatibility

with existing

structures, or

imposed as

part of a

broader

structural

change. Need

for education

and training.

Communication

systems.

Shiffman

RN, Liaw Y,

Brandt CA,

Corb GJ

1999 United

States.

American

(insurance

based).

Systematic

literature review.

Medline and

Cinahl

search for

articles

published

between

1992 and

1998. 25

articles.

Tedious data-

entry

requirements

lead to

disaffection

with the

system.

Computerised

decision support

systems.

Stanberry B 2000 United

Kingdom.

European

(mixed).


specified.

Reliability of

software,

responsibility

and

accountability.

Communication

systems.

Styra R 2004 Canada. Mixed. Systematic

literature review.

Medline,

Altavista

and Google

searching.

Availability of

information for

psychiatrists

and patients,

clinical

guidelines,

clinical trial

information,

continuing

education and

training, e-mail

and related

factors of

delayed

Information

systems.



responses and

confidentiality

(changing

physician-

patient

relationship).

Swinglehurst

DA

2005 United

Kingdom.

Public

(NHS).

Narrative review

with a case study.

None

specified,

but implies

use of

Cochrane

database.

The need for

evidence, lack

of time,

attitudes of

clinicians and

organization,

patients’

expectations.

Computerised

decision support/

information

systems.

Tierney WM 2001 United

States.

American

(insurance-

based).

Narrative review of

computerised

decision support.

None

specified.

Facilitators:

Need for clear

guidelines in

the use

computerised

decision

support.

Computerised

decision support

systems.

Uhlenhopp

MB,

Fliedner

MC, Morris

P, Van

Boxtel T

1998 United

States,

United

Kingdom,

Netherlan

ds (Global

study).

Mixed. Narrative literature

review with

anecdotal material.

None

specified.

Internet as a

global

phenomenon;

cultural

differences in

nurses’

reception and

use.

Information

systems.

Vreeman

DJ, Taggard

SL, Rhine

MD,

Worrell TW

2006 United

Kingdom.

Public

(NHS).

Narrative review. Medline

search.

Studies

screened for

inclusion by

3 authors. 18

articles of an

original

2,010.

Facilitators:

end-user

participation,

good training,

workflow

analysis, data

standardization

. Barriers:

“challenges

with behaviour

modification”,

poor

equipment,

lack of

training.

Management

systems.

Wallace S,

Wyatt J,

1998 United

Kingdom.

Public

(NHS).


specified.

Professional

attitudes,

Communication

systems.



Taylor P. safety, cost,

impact on

professional/

patient

relations.

Weinstein

RS, Descour

MR, Liang

C,

Bhattachary

ya AK,

Graham AR,

Davis JR,

Scott KM,

Richter L,

Krupinski

EA, Szymus

J, Kayser K,

Dunn BE

2001 United

States,

Poland,

Germany.

Mixed. Review of systems. None

specified.

Influence of

external socio-

political

factors;

importance of

professional

education.

Communication

systems.

Whitten P,

Love B

2005 United

States.

American

(insurance

based).


specified.

User

satisfaction,

safety, cost.

Communication

systems.

Wootton, R 2001 Australia Mixed

(telemedicin

e in the

developing

world)..

Systematic review. Unspecified

literature

search.

Lack of

research,

evaluation,

long-term

funding and

quality control.

Communication

systems.

Yellowlees

PM,

Kennedy C

1997 Australia. Australian. Describes the

technology and

experience of

implementation in

Australia.

None

specified.

Confidentiality

and the need

for further

research.

Communication

systems.



Appendix 3 WP1 Papers included and excluded at full paper stage

Papers included and excluded at full paper stage

Reference Include?

Reason for exclusion.

Y/N Not a review*

Not on e-Health

Not on implementation

Anon 1999 (Telemedicine an overview)

N � (d)

Anon 2006

N �

Al-Qirim 2007

N �

Al-Qirim 2003

N � (c)

Al-Qirim 2005

N � (d)

Alberdi 2005

N � (a)

Anderson 1997

N � (e)

Anderson 2007

N � (d)

Ash 2003

N �

Ash 2005

N � (e)

Avrin 2003 N �

Bagayoko 2006

N �

Bates 2002

N � (d,

e)

Beltrame 2001

N � (d)

Bensink 2006

N �

Berner 2005

N � (d)

Bick 1999

N � (d)

Bodenheimer 2003

N � (d)

Bowles 1997

N � (d)

Brender 2006 N � (a)

Broens 2007

YES



Burton 2004

N � (d)

Carrino 1998

N �

Reference Include?


Y/N Not a review* Not on e-Health


Chamorro 2001

N � (d)

Chaudhry 2006

YES

Clemmer 2004

N � (d)

Cohen 2005 N �

Crowe 2001

N �

Davies 2007

N �

De Backer 2004

N �

De-Lusignan 2005

N �

Eadie 2003

N �

Eger 2001 N �

England 2000 N � (d)

Fenton 2006

N � (d)

Finch 2003

N � (a)

Fujimoto 2000

N � (a)

Furness 2001

N � (d)

Gater 2004

N � (d)

Geibert 2006

N � (d)

Georgiou 2007

N �

Gordon 2007

N � (c)

Guler 2002

N � (d)

Hailey 2001

N �

Hailey 2002

N �

Hakansson 2000



N �

Handler 2004

N

� (d)

Hartzema 2007

N

� (d)

Hebert 2006 YES

Reference Include?




Helm 2004

N � (d)

Hill 2002

N � (d) �

Hilty 2002

YES

Hilz 2000

N � (d)

Huis 2006

�

Hussein 2004

N �

Jaspers 2006

N � (d)

Jennet 2004

YES

Jennett 2005

YES

Johnson C 2006

N �

Johnson K 2001

YES

Jones 2006

N � (d)

Kanthraj 2007

N �

Kaplan 2001 (#2193)

N �

Kaplan 2001 (#2194)

N � about

evaluation methods not implementation per se.

Kaplan 2004

N � (c) �

Kaufman 2006

N � (c)

Kawamoto 2005

YES

Kluge 2007

N � (d) �

Koch 2006

N �

Koshy 2005

N � (d)



Kuhn 2001

N � (b)

Kuhn 2006

N � (b)

Kukafka 2003

Yes

Reference Include?




Kukafka 2007

N � (d)

Kuperman 2007

N � (d)

Latifi 2005

N � (d)

Law 2003

N � (d)

Leatt 2006

Yes

Lee 2005 N Abstract only

Lehmann 2004

N � (d)

Lehmann 2006

N � (d)

Liaskos 2002

N � (d)

Loane 2002

N �

Lorenzi 1997

N � (c)

Lorenzi 2000

N � (c)

Lu 2005

Yes

MacFarlane 2006

N �3

Mair 2000

N �

Major 2005

N �

Matusitz 2007

N � (d)

Maulden 2003

N � (d)

Middleton 2005

N � (d)

Miller 2005

N � (d)

Miller 2001

N �

3 Not a review, but doesn’t meet review exclusion criteria a-e: it is a review of services (not literature) plus primary qualitative study



Miller 2007

N � (e)

Mony 2007

N � (d)

Mosley-Williams 2005

N �

Reference Include?




Mun 1999

N � (d,e)

Nagykaldi 2007

N � (a)

Nazi 2003

N � (b)

Nies 2006

N �

Norum 2007

N � (b) �

Noss

N � (d)

O’Meara 2007

N � (d)

Ohinmaa 2006

YES

Ozdas 2007

N � (d)

Pagliari 2004

N � 4

Papshev 2001

YES

Payne 2000

N � (d)

Peleg 2006

YES

Pellegrino L 2005

N

� (d)

Poissant L 2005

N �

Previte J 2006

N � (d)

Puskar KR 2004

N � (d)

Puskin DS 1995 N �

Rahimi B 2007

N �

Ruckdaschel 2006

N � (d)

Rudowski R 2003 N � (d)

4 This paper reports a programme evaluation, not a literature review



Saathoff 2005

N �

Sable S 2001

N �

Reference Include?




Shekelle 2006

YES

Sheng 1997

N � (b)

Shiffman 1999

N �

Smith A 2002

N �

Sonnenberg 2006

N �

Souther E 2001

N � (d)

Staggers N 2003

N � (d)

Studer 2005

YES

Subramanian S 2007

N � (d)

Swinglehurst D 2005

N � (d) � �

Tang P 2006

N � (e)

Tidd 1999

N �

Tierney W 2001

N � (d)

Upperman J 2005

N � (d)

Van Ginneken A 2002

N � (d)

Vincent DJ 2005

N �

Vreeman D 2006

YES

Wallace S 1998

N � (d)

Walz M 2000

N � (d)

Weiner MG 2000

N � (d)

Whitten P 2005

N � (d) �

Whitten 2007

No �



Xue Y 2007

N �

Yarbrough A 2007

YES

Yellowlees 1997 N

� (d)

Reference Include?




Yusof 2007

YES

Table Legend/Definition

A review paper provides an analytic account of the research literature

related to a specific topic or closely related set of topics. It is intended to

contribute to knowledge by answering a research question.

Review papers include:

(i) systematic review: where relevant literature has been identified by

means of structured search of bibliographic and other databases; where

transparent methodological criteria are used to exclude papers that do not

meet an explicit methodological benchmark, and which presents rigorous

conclusions about outcomes;

(ii) narrative review: where relevant literature has been purposively

sampled from a field of research; where theoretical or topical criteria are

used to include papers on the grounds of type, relevance, and perceived

significance; with the aim of summarising, discussing, and critiquing

conclusions;

(iii) qualitative metasyntheses or meta-ethnographies (where relevant

literature has been identified by means of a structured search of

bibliographic and other databases, where transparent methods had been

used to draw together theoretical products, with the aim of elaborating and

extending theory .

Review papers exclude:



(a) Secondary analyses (including qualitative metasyntheses or meta-

ethnographies) of existing data-sets for the purposes of presenting

cumulative outcomes from personal research programmes,

(b) Secondary analyses (including qualitative metasyntheses or meta-

ethnographies) of existing data-sets for the purposes of presenting

integrative outcomes from different research programmes,

(c) Discussions of literature included in contributions to theory-building or

critique,

(d) Summaries of literature for the purposes of information or

commentary.

(e) Editorial discussions that argue the case for a field of research or a

course of action



Appendix 4 WP1 Details of included reviews

Details of included reviews

Reference ID Authors Country of

Origin

E-Health

domain

Publisher and date

of publication

Aim of review Methods of review Databases

searched

Inclusion and exclusion criteria Number of

included

papers

Broens

2007

Broens THF,

Huis in’t Veld

RMHA,

Vollenbroek-

Hutten MMR,

Hermens HJ,

van Halteren

AT,

Nieuwenhuis

LJM

Netherlands Communication Journal of

Telemedicine and

Telecare 2007

To identify determinants

of implementation of

telemedicine

interventions

Qualitative review Papers presented

at Telemed

Conference 2004

in London

Any paper presented at this

conference

45

Chaudrhy

2006

Chaudrhy

B,Wang J, Wu

S, Maglione

M, Mojica W,

Roth E,

Morton SC,

Shekelle PG.

USA All 4 Annals of Internal

Medicine

2006

To systematically review

evidence on the effect of

health information

technology on quality,

efficiency and costs of

health care

Systematic review MEDLINE (1995 –

2004)

Cochrane Central

Register of

Controlled Trials,

Cochrane

Database of

Descriptive and comparative

studies and systematic reviews of

health information technology

257



Abstracts of

Reviews of

Effects,

Periodical

Abstracts

Database

“studies identified

by experts”

Hebert

2006

Hebert MA,

Korabek B,

Scott RE.

Canada Communication

systems.

International Journal

of Medical

Informatics 2006C

To examine research

evidence supporting the

effectiveness of tele-

Healthe-Health and to

develop a decision

framework to

demonstrate an

approach for decision

makers and practitioners

to transfer home tele-

Healthe-Health research

into practice

Narrative review Not explicit Not explicit 3 (these 3

were

themselves

systematic

reviews)

Hilty

2002

Hilty DM, Luo

JS, Morache

C, Marcelo

DA, Nesbitt

TS

USA Communication

systems;

Management

systems; CDSS

CNS Drugs 2002 To discuss the

advantages and

disadvantages of

telepsychiatry for clinical

and educational

purposes and also to

address the practical

Systematic review Searched

following

databases

between 1965 and

june 2001 using

the MEDLINE<

Embase, Science

Inclusion described, exclusion

criteria not explicit

Not provided



considerations about

delivery of patient care,

pt and provider

satisfaction, effects on

communication and

interpersonal behaviour

and costs. Aims to

outline principles for

using telepsychiatry

Citation index,

social sciences

citation index, and

telemedicine

information

exchange

databases.

Jennett

2004

Jennett PA,

Scott,RE, Hall

LA, Hailey D,

Ohinmaa A,

Anderson C,

Thomas R,

Young B,

Lorenzetti D.


(tele-Healthe-

Health)

Telemed J and e-

Healthe-Health 2004

Identify policy strategies

for successful

telemedicine

implementation

Comprehensive search

of peer-reviewed and

grey literature 1980-

2002 (including

‘electronic databases,

hand searches of

journals and conference

proceedings, and

communication with

consultants in the field’

Not explicit Not explicit ’57 Policy

Sources’

Jennett

2005

Jennett PA,

Gagnon MP,

Brandstadt HK


systems (tele-

Healthe-Health)

Journal of

Postgraduate

Medicine

To review and critique

peer-reviewed studies

that have focused on

assessing tele-Healthe-

Health readiness for

rural and remote health

Systematic review Papers between

1996 & 2005:

Medline, TIE,

Pubmed, AMED,

Google, Web of

Science, CINAHL,

EMBASE, AARP

Ageline, Cochrane

Central,

Papers about tele-Healthe-Health

‘readiness’, judged for inclusion

on the basis of theoretical and

methodology quality

Did not

specify

actual

numbers of

papers

included, but

referred to

four models

of readiness



PyschInfo, ERIC &

ProQuest

that were

critiqued

(cites 8

papers in

relation to

these

models).

Johnson

2001

Johnson KB

USA Not specified,

probably all 4.

Archives of Pediatric

and Adolescent

Medicine 2001

Identify barriers to

adoption of IT by health

professionals

Search using limited set

of search terms, follow

up of ‘relevant

references’ search on

Google and

Northernlight.com

Medline only

References included if they

discussed barriers to adoption

Unclear.

Paper has

63 relevant

references.

Kawamot

o 2005

Kawamoto K,

Houlihan CA,

Balas EA,

Lobach DF

USA CDSS BMJ 2005 To identify features of

CDSS’s critical for

improving clinical

practice

Systematic review Searched Medline,

CINAHL and the

Cochrane

Controlled Trials

Register up to

2003 and

searched

reference lists of

included studies

and relevant

reviews.

Inclusion criteria were studies that

evaluated the ability of decision

support systems to improve

clinical practice. Incl any RCT.

Excluded those with less than 7

units of randomisation per study

arm, non English, mandatory

compliance with CDSS, lack

Kukafka

2003

Kukafka, R;

Johnson, SB;

USA ALL 4 Journal of

Biomedical

To systematically review the

literature on implementation

Systematic review Medline (1999-

present); Science

Papers had to report empirical

research study. Hit at least one

142 papers

reduced to



Linfante, A;

Allegrante, JP.

Informatics 2003 of IT in health care, to

develop an integrative

framework drawing together

diverse theoretical

perspectives

Direct (1998-

present);

HealthStar (1998-

present); and ABI

Inform (1999-

present)

search term from each key

category (Health; Information;

behave/org.)

57 after

abstract

review;

reduced to

24 after

reading full

paper

Leatt

2006

Leatt P; Shea

C; Studer M;

Wang V.

USA Management

systems (EHRs;

CPOE;

Electronic

Medication

Administration

Records)

Electronic

HealthCare

To review the literature on

the facilitators and barriers

to successful

implementation of EMR,

CPOE & MAR

Literature review

(unspecified)

Not specified Not specified Not

specified

Lu 2005 Lu, Y-C; Xiao,

Y; Sears, A;

Jacko, JA.

USA ALL 4 International Journal

of Medical

Informatics

To review the literature on

issues related to adoption of

PDAs in health care and

barriers to PDA adoption

Systematic review Year span: 1998-

2004. Medline;

National Library of

Medicine’s

searchable

database of peer

reviewed

publications;

published

proceedings of

HIMSS

(Healthcare

Information &

Management

PDAs & mobile computing

devices, and use by all health

professionals

95 reviewed,

out of 200

identified.



Systems Society;

plus internet

searches

Ohinmaa

2006

Ohinmaa A Canada Communication

systems

Journal of

Telemedicine and

Telecare 2006

To assess telemedicine

projects outside the USA

and provide examples of

promising results that could

be disseminated to other

countries.

Narrative Review Articles from the

last 3 yrs of the

“Successes and

Failures in Tele-

Healthe-Health

Conference and a

special issue of

the International

Journal of

Circumpolar

health in 2004

Included articles that showed a

scientific basis for successful

telemedicine conducted outside

the USA with exception of Alaska.

Excluded programmes from

developing countries that were

seen to difficult to implenment in

the US health system. Articles

discussing non medical

applications eg education were

excluded. Review focused on

applications benefiting significant

segments of the health care

population rather than those

restricted to a targeted population

or geographical area.

Not stated

Papshev

2001

Papshev D,

Peterson

A.M.

USA e-prescribing American Journal of

Managed Care

2001

To examine advantages of

and obstacles to electronic

prescribing in the

ambulatory care

environment

Narrative review MEDLINE,

International

Pharmaceutical

Abstracts

January 1980 –

September 2000

Articles, symposia proceedings

and organisational positions

statements published in the US

on electronic prescribing and

automation in health care

Not stated

(35

references

provided).

Peleg Peleg M, Tu

S.

USA Decision

support

International Medical

Informatics

To find trends in Clinical

Decision Support Systems

Narrative review PubMed “past 5

years”.

“Decision Support Systems” in

title

Not stated

(63



2006

Association

Yearbook

2006

(CDSS) that were developed

over the last few decades

and give some indication of

future directions in

developing successful,

usable CDSS.

references

provided).

Shekelle

2006

Shekelle P,

Morton SC,

Keeler EB,

Wang JK,

Chaudhry

BI, Wu S,

Mojica WA,

Maglione M,

Roth EA,

Rolon C,

Valentine D,

Shanman R,

Newberry

SJ.

USA Management,

Decision

support

Agency for

Healthcare

Research and

Quality

2006

To assess the evidence

base regarding benefits and

costs of health information

technology (HIT) systems,

particularly those providing

pediatric care.

Systematic review MEDLINE,

Cochrane

Controlled Clinical

Trials Register,

Database of

Reviews of

Effectiveness,

Health Affairs,

and “several

reports prepared

by private

industry”.

1995 - 2003

Systematic reviews, meta-

analyses, hypothesis-testing or

predictive analyses on costs,

benefits and barriers to

implementing HIT.

256 studies

(156 on

decision

support, 84

on the EMR,

and 30 on

CPOE).

124 in

ambulatory

care, 82 in

in-patient

settings.

Studer 2005 Studer M USA Management Electronic

Healthcare 2005

To systematically review

studies assessing the effect

of organizational factors on

the effectiveness of EMR

system implementation

Systematic review MEDLINE 1900 –

2005, and

reference sections

of included

studies.

English language; available at the

University Library;

On electronic medical records;

Assessing the effect of

23 papers.



management support, financial

resource availability,

implementation climate and / or

implementation policies and

practices on the effectiveness of

EMR implementation.

No quality or study design filter.

Vreeman

2006

Vreeman DJ, Taggard SL, Rhine MD, Worrell TW.

USA Management

systems

Physical Therapy

2006

To investigate the role of

EHRs in physical therapist

practice. Aim of review was

to identify, review and

summarize the benefits,

barriers, and key factors for

success in implementing

EHRs in physical therapist

practice settings.

Systematic review Medline (1966 to

week 4 of October

2004); the

cumulative index

of nursing and

allied health

literature (1982 to

week 4 of October

2004); Ovid’s All

evidence based

Medicine reviews

(Cochrane

database of

systematic

reviews, American

College of

Physicians Journal

Club, Database of

Abstracts of

Reviews and

Effects, and

Studies were included if they met all of the following criteria: 1) an EHR was the intervention of interest in the study; 2) the HER contained the IOM (institute of medicine) core functionality of health information and 2 or more of the other core functionalities; 3) the study described a primary use of the HER; 4)physical therapists were study participants, and 5) the article reported outcomes that indicated benefits or barriers to system implementation. Studies were excluded if they described only physiological monitoring systems, communication technology for telemedicine applications, or only secondary uses of EHR.

18 articles

included…H

owever

authors of 6

of these

articles

reported on

the initial

implementati

on and

ongoing

analysis of

the same

HER so

these

articles were

aggregated

and

analysed as

1 unit giving

an analysis



Cochrane Cnetral

Registrar of

Controlled Trials

(third quarter

2004). Also

conference

proceedings from

the American

Medical

Informatics

Association

Annual

Symposium

(1998-2004)

based on a

total of 13

studies.

Yarborough

2007

Yarborough AK, Smith TB.

USA Management,

communication

and information

systems.

Med Care Res rev

2007

To Review the literature on

physician technology

acceptance in order to

improve understanding of

barriers to physician

adoption of new

technologies.

Narrative review. PubMed and ABI

Inform/Complete

databases

between Jan 1996

and Nov 2006.

English only articles in peer

reviewed journals that were about

technology acceptance research

pertinent to health care contexts.

Excluding those not directly

pertaining to : 1) physician IT, 2)

physician barriers to new

technology; or 3) the TAM.Also

studies using specific types of

qualtitative methodologies were

excluded in an effort to focus on

studies using more rigourous

methodologies. Thus case

studies that were purely

18 articles.



descriptive and limited to less

than two sites were excluded as

were review articles that only

summarised findings. Qualitative

studies of multiple organisations

or multiple groups of technology

users that used an inductive

approach were included as were

meta-analyses that used

statistical methods to provide a

review of prior research findings.

Yusuf 2007 Yusof MM, Stergioulas L, Zugic J

Malaysia Management,

communication,

CDSS

MEDINFO 2007 To review the literature on

health information systems

adoption in clinical practice

Systematic review Using a number of

database from

medical,

informatics and

engineering field.

Between 1985 and

2005

Inclusions all computer based info

systems that involve human

interaction used in health care

settings. Those used for training /

education excluded. English only.

Looking at evaluation studies in

clinical studies selecting case

studies only and excluding

experiment and surveys.

55



Appendix 5 DATA EXTRACTED FOR THEMATIC ANALYSIS - SYSTEMATIC REVIEW

Conditions

Prior To

implementatio

n

Cost Need and

importance of

validation and

evaluation

Attitudes Ease of Use

of System

Security/Confidenialit

y and standards

Education

and Training

Technological

Issues

Communicatio

n Issues

Organisational

Issues

Other

1 Broen

s

2007

�

Involvement of

patients and

professionals

in the

requirements

analysis and

design process

crucial in order

to fit

telemedicine

into their daily

work. Pg 305

para 3

Also related to

legislation and

�

Costs

related to

investments,

maintenanc

e and

operational

costs. Pg

305 para 6

Lack of

financing

structures

for ongoing

work.pg 305

para 6

�

Evidence

based medicine

is regarded as

a requirement

pg 305 para 4

. “ alternative

designs

needed to

evaluate

efficacy and to

convince

professionals,

policymakers

and insurance

companies

� Technology acceptance influenced by prof attitudes. Pg 305 para 3

Stimulating

role of

leading

champions

305 para 5

Technology

acceptance

influenced

by patient

�

Usability of

system is a

major factor

for

success304

para 7

“Professional

s should be

able to

access the

right patient

information at

the right time”

305 para 3

�

Standardisation

Related to

interoperability pg 306

para 3 and to ensure

quality and uniform

practice.

Security 306 para 4

patient physical safety

and pt info security.

For acceptance of

telemedicine

implementations

adequate security

�

Need for

support on

how to

install and

sustain the

system and

how to deal

with errors.

Training at

all levels

Pg 304

para5+6

Creating

familiarity

�

Technology quality

technical problems

a major barrier to

implementation….e

g non-connecting or

malfunctioning

devices. Pg 305

para 2

Eg lack of

working

protocols for

use of new

technologies

and ability to

accommodate

chgs in

collaboration

and team

roles, rights

and

responsibilities

pg 306 para 1

Telemedicine

might require

changes in



policy pg 306

para 2

about

implementation

.

Page 305 para

6

Comprehensive

cost

effectiveness

studes are

essential in

developing

future financing

structures

pg 305 para 6

attitudes

Pg 305

para 3

305 para 3

mechanisms have to

be taken into account.

These security

mechanisms should

support the crucial

trust relation between

health-care providers

and patients. P306,

para 4.

A need for secure

information transfer

and authorization

mechanisms. P 306,

interoperability

between telemedicine

applications is

important to support

the current trend of

transmural work

practices and is not

guaranteed without

globally accepted

standards

with the

intervention

(p 305, para

2).

Availability

of support to

users (p304,

para 3,

support);

supporting

staff and

doctors

should be

able to

operate the

devices and

should have

flexible

access to

services

offered by

the

telemedicin

e system. P

304,para 5.

changes in

collaboration

and (team)

roles, rights

and

responsibilities

. P306, para 1

Novel working

practices

introduced by

telemedicine

do not always

fit with existing

traditional

working

protocols in

health care (p

306, para 1).



a need for

training

users how

to use these

novel types

of systems.

Such

training is

needed at

all levels pg

304 para 4



Conditions Prior

To

implementation

Cost Need and

importance of

validation and

evaluation

Attitude

s

Ease

of Use

of

Syste

m


y and standards

Education and

Training

Technologica

l Issues

Communicatio

n Issues

Organisationa

l Issues

Other

2 Chaudrhy

2006

However, the

method used by

the benchmark

leaders to get to

this point – the

incremental

development

over many years

of an internally

designed

system led by

academic

research

champions – is

unlikely to be an

option for most

institutions

contemplating

implementation

of health

information

technology….th

One of the

chief barriers

to adoption of

health

information

technology is

the

misalignment

of incentives

for its use.

Specifying

policies to

address this

barrier is

hindered by

the lack of cost

data. Pg 749

2nd col 1st

para.

Effects on

Two critical

questions that

remain are: 1)

what will be the

benefits of

these initiatives

(health

information

technology)

Third a high

priority must be

the

development of

uniform

standards for

the reporting of

research on

implementation

of health

information

Finally, additional

work is needed on

inter-operability. pg

749 col 2 para 2

the study by

Koppel and

colleagues

included

detailed

contextual

information

relating to

human factors.

One health

record study

reported

physician

classroom

training time of

16 hrs before

implementation

. Another order

entry study

reported that

nurses

received 16



technology….th

e effectiveness

of these

technologies in

the practice

settings where

most health care

is delivered

remains less

clear. ….how

these tools are

used and the

context in which

they are

implemented are

critical pg 748

col 2 para 2

costs –data on

costs were

more limited

than the

evidence on

quality and

efficiency.

Because these

systems were

built,

implemented,

and evaluated

incrementally

over time, and

in some cases

were

supported by

research

grants, it is

unlikely that

total

development

and

implementatio

n costs could

be calculated

accurately and

in full detail.

747 1st

technology,

similar to the

CONSORT and

QUORUM

statements. pg

749 col 2 para

2

Published

evidence of the

information

needed to

make informed

decisions about

acquiring and

implementing

health

information

technology in

community

settings is

nearly non

existent. For

example,

potentially

important

evidence

related to initial

received 16

hours of

training, clerical

staff received 8

hours, and

physicians

received 2 to 4

hours. Pg 748

col 1 para 3



column, para

3.

these 9 studies

infrequently

reported or

measured data

on costs and

contextual

factors.. Two

reported

information on

costs. Neither

described the

total initial

costs of

purchasing or

implementing

the system

being

evaluated.

Data on

contextual

factors such

as

reimbursement

mix, degree of

capitation, and

capital costs,

effect on

provider

productivity,

resources

required for

staff training

(such as time

and skills), and

workflow

redesign is

difficult to

locate in the

peer reviewed

literature. Also

lacking are key

data on

financial

context, such

as degree of

capitation,

which has been

suggested by a

model to be an

important factor

in defining the

business case

for electronic

health record



barriers

encountered

during

implementatio

n were scant;

only 2 studies

included such

information. Pg

748 col 1 para

3

with respect to

the business

case for health

information

technology, we

found little

information

that could

empower

stakeholders

to judge for

themselves the

financial

effects of

adoption. Pg

749 col 1 last

para

use. Pg 748 col

2 para 3

In many

important

domains we

found few

studies. This

was particularly

true of health

information

technology

applications

relevant to

consumers and

to

interoperability,

areas critical to

the capacity for

health

information

technology to

fundamentally

change health

care. 748/749

col 2 last para

and col 1 1st

para.



who will pay

and who will

benefit? Pg 749

col 1 para 2

Regarding the

former

(benefits), a

disproportionat

e amount of

literature on the

benefits that

have been

realised comes

from a small set

of early adopter

institutions that

implemented

internally

developed

health

information

technology

systems.

These

institutions had

considerable

expertise in



health

information

technology and

implemented

systems over

long periods in

a gradual,

iterative

fashion.

Missing from

this literature

are data on how

to implement

multifunctional

health

information

systems in

other health

care settings.

Pg 749 col 1

para 3.

The limited

quantitative and

qualitative

description of

the

implementation



context

significantly

hampers how

the literature on

health

information

technology can

inform decision

making by a

broad array of

stakeholders…

pg 749 col1

para 3

Time utilization

p 748 column 1,

para

1…..relative

decreases in

other outcomes

were as follows:

medication

turnaround

time, radiology

completion

time, results

reporting time,

nurse



documentation

time

the major effect

of health

information

technology on

quality of care

was its role in

increasing

adherence to

guideline- or

protocol based

care (p 744,

final para).

the second

theme showed

the capacity of

health

information

technology to

improve quality

of care through

clinical

monitoring

based on large



scale screening

and

aggregation of

data pg 746 col

1 para

2……examples

include: using

an electronic

health record to

identify adverse

drug events (pg

746 col 1 para

2); the role of

health

information

technology

surveillance

systems in

identifying

infectious

disease

outbreaks (pg

746 col 1 para

4); third health

information

technology

mediated effect

on quality was

a reduction in



medication

errors pg 746

2nd col para 2.

Effects on

efficiency;

utilization of

care; provider

time. (p 746,

2nd column,

final para.)

studies

examined the

effect of

systems on

Utilization of

care. P 747 2nd

column.

To date the health information technology literature has shown many important quality and efficiency related benefits



199

as well as limitations relating to generalisability and empirical data on costs.. Studies from 4 benchmark leaders demonstrate that implementing a multifunctional system can yield real benefits in terms of increased delivery of care based on guidelines (particularly in the domain of preventive health), enhanced monitoring and surveillance activities, reduction of medication errors, and decreased rates of utilisation for potentially redundant or inappropriate care. Pg 748 col 1 last para.

More



information is

needed

regarding the

organisational

change,

workflow

redesign,

human factors,

and project

management

issues involved

with realising

benefits from

health

information

technology pg

749 col 2 para

2



Conditions

Prior To

implementation

Cost Need and

importance

of validation

and

evaluation

Attitudes Ease

of Use

of

System

Security/Confideniality

and standards

Education

and

Training

Technological

Issues

Communicatio

n Issues

Organisational

Issues

Other

3 Hebert

2006

�

Decision

framework can

form the basis

of a business

plan for

delivering

home care that

addresses

service

objectives,

costs , and

national

outcome

indicators.

Pg 792 para 2

Inertia and

�

Project

establishment

costs;

equipment

costs,

maintenance

costs,

communication

and staffing c

costs. Pg 790

para 3

Underlying

service delivery

model also

affects

costs…so

study context

important pg

�

Home

telehealth

evidence:

what are the

most

effective

technologies

to support

provision of

services. Pg

791 para6

Does the

evidence

support

using the

technology to

achieve

desired pt

�

Are the

organisation,

health care

providers

and clients

ready to

adopt

technology?

Pg 791

para7

Appropriate

allocation of

staff time to

the project pg

790 para 4

Radical re-

engineering of

the service pg

790 para 9

Context:

stability

of

client’s

condition

and

nurses

ability to

co-

ordinate

more

than one

visit

Pg 791

para9



202

resistance to

change within

the healthcare

system pg792

last para

790 para 5

outcomes or

reduce

delivery

costs pg 791

para 7

Using theory

driven

program

development

helps

decision

makers

develop clear

expectations

for realistic

outcomes of

using the

technology

pg 791 col 2

section F5.



203

Conditions Prior

To

implementation

Cost Need and importance

of validation and

evaluation


of System


and standards

Education and

Training

Technological

Issues

Communication

Issues

Organisational

Issues

Other

4 Hilty

2002

�

Some obstacles

eg access to

broad bandwidth

lines will reced

ewith the

advancement of

technology.

�

Costs

should be

considered

for patients,

clinics,

providers

and society

at large pg

543 para 5.

�

Long term

sustainability requires

collection of data and

feedback for clinical,

technical and

administrative staff

pg 544 para2.

evaluate options,

implementation and

maintenance of

telepsychiatry with a

team of clinicians,

technicians and

administrators in both

the hub and the

spoke sites. Pg 544

table VI

�

Physicians

may have

anxiety from

the

psychologic

al

resistance

to a change

in the status

quo pg 543

para 6

Suitable

clinical

chamption

is vital pg

544para 2

satisfaction

�

Satisfaction

with

telepsychiatry

was lower

than in person

consultation in

terms of ease

of use with the

process, pg

537 para 5

�

Concerns about

security pg 543 para 6

Legal issues pg 542

para 6

�

Standards are needed

for electronic

information exchange

in addn to established

provisions for security

and privacy pg 543

para 6

�

Lack of

experience or

training;

Inadequate

typing skills

pg 543 para 6

Inadequate

technical

support in

initiation,

maintenance,

and trouble

shooting

emergencies

alienates pts

and clinicians

alike.

Pg 544 para 1

�

with regard to

technical

issues,

programmes

need to review

a variety of

vendors

products

before making

a purchase,

involve the

clinical staff

who will use

the product

from the

beginning. Pg

544 col 2 para

1

Technical

Telepsychiatry

appears to allow

for the building

of relationships

based on the

advantage of

creating a social

presence and

providing more

visual cues than

telephone and

the

disadvantages of

limiting

nonverbal

communication

compared with in

person care. Pg

543 para 3

Effects of

telepsychiatry on

interpersonal

Incentives for

each of the

parties

involved pg

544 para 2

Inadequate support from the specialists providing the service pg 543 para 7

Many

programmes

fail because of

inadequate

financial and

other

administrative

support from

the

leadership, or



204

with

telepyschiat

ry was

lower than

in person

consultation

in terms of

ease of use

with the

process pg

537 para 5

Training

practitioners

to practice

telepsychiatry

requires

comfort with

the

equipment,

adapting to its

clinical

practice and

being aware

of its

limitations. Pg

544 para 2

issues under-

reported

including

bandwidth,

audio quality,

FPS, size of

transmitted

video image

spped of

computed and

name and

make of

CODEC and

other

equipment pg

543 para 4

promptly solve

technical

problems pg

544 col 2 para

1

interactions pg

540 para3

Effect on

communication

and relationships

pg 539 para 4

one concern

about

telemedicine is

that the

technology may

adversely affect

the development

of a positive

therapeutic

alliance.

Telepsychiatry

seems to have

positive and

negative effects

on

communication

pg 539 col 2

para 3

the fact that

telemedicine

is not a match

for the overall

mission of the

organisation

Pg 543 para 7

diversifying

personnel is

helpful to

sustain the

programme

despite

departures pg

544 col 1

para 2



205

Conditions Prior

To

implementation


of validation and

evaluation


of System


and standards

Education and

Training

Technological

Issues

Communication

Issues

Organisational

Issues

Other

5 Jennett

2004

�

Important that all

people involved

in health care

particularly

policy and

decision-makers,

start to consider

telehealth as an

integral part of

… health

systems.

(p79, col2,

para4)

This can be

achieved thru

aligning

telehealth

initiatives with

suitable

frameworks

for

economic

analysis

need to be

developed

that capture

non

monetary

and

unintended

consequen

ces as well

as

monetary

measures.

Pg 80 table

2 C

Evaluation.

Evaluation and

research….driving

forces for telehealth

as …..and proof of

technological

feasibility.

Pg 82 para9

suitable outcome

indicators, measures

and reliable and valid

instruments for

socioeconomic

benefit of telehealth

must be identified,

defined, and

consistently applied

within a recognised

framework that asks

relevant questions.

Pg 80 table 2 C

evaluation

It is important that professional and regulatory groups become involved in e-policy decision making so that telehealth can be applied to its full extent pg 82 col 1 para 1

take note of

patient and

professional

s’ views

(p82, col 1

para1)

Standards are needed

to foster use of

technology in specific

clinical areas pg 82

para 1

Barriers identified

were… privacy and

security

It is imp to assess risk

prior to

implementation pg 82

para 7

Barriers identified

were… privacy and

security,

accreditation,

intellectual property

�

Various

strategies,

such as

training and

education,

may enhance

trusting

relationships

pg 82 para2

to facilitate

access to

many

telehealth

applications

increased

broadband

connectivity is

needed,

particularly to

rural and

remote

communities.

Pg 80 table

2A policy.

collaboration,

partnerships and

sharing:

interjurisdictional

sharing of skills,

information, and

services through

telehealth

programs

creastes

opportunities to

improve health

care.

Collaboration

between industry

and health care

organisations

holds several

advantages,

such as

opportunities to

test technologies

and the ability to

telehealth

applications

are more

likely to be

successful, in

terms of cost

and

sustainability,

if they are

considered to

be part of the

larger domain

of e-health.

Programs

implemented

and evaluated

as

independent

initiatives are

at greater risk

of failure.

Important that

all people



206

initiatives with

existing strategic

health plans. Pg

80 table 2A

policy.

establishment of

a policy forum

that focuses on

telehealth policy

would facilitate

these needs pg

80 table 2A

policy.

partnerships in

telehealth should

be established

where there are

opportunities to

improve

efficiency in

health care and

decrease

duplication. Pg

80 table 2A

policy

evaluation

telehealth programs

should implemented

in a culturally aware

and culturally

sensitive manner pg

80 table 2C

Evaluation

evaluations should

include examination

of the social,

organisational and

policy aspects of

telehealth pg 80 table

2 C evaluation.

and copyright (p82,

col 1 para1)

and the ability to

communicate

local needs pg

82 col 1 para 5

effective

communication

about the goals,

rationale,

benefits and

limitations of the

program, and

training and

education may

enhance trusting

relationships pg

82 col 1 para 2

As teleheatlh

continues to

evolve, input

from all

stakeholders…in

to policy

development is

required.

Consideration of

all people

involved in

health care

particularly

policy and

decision-

makers, start

to consider

telehealth as

an integral

part of

….health

systems. Pg

79 col 2 para

4

organisational

readiness,

which refers to

the fit between

the telehealth

application

and the

processes,

and standards

within the

organisation,

is also



207

establishing a

guiding

philosophy while

planning for

implementation

is the key to

success pg 80

col 2 para 1

needs as well as

practical

experience is

essential for a

meaningful

exchange of

information and

views. Pg 80

table 2A policy

important pg

82 col 1 para

3



Conditions Prior

To

implementation


of validation and

evaluation


of System


and standards

Education and

Training

Technological

Issues

Communication

Issues

Organisational

Issues

Other

6 Jennett

2005

�

Turf, practice

context,

ownership…pg 3

last para

organisational

readiness and

system

readiness pg 5

discussion para

1

�

Funding pg

5 para3

�

Projection of benefits

pg 4 para3

�

Innovators

and

champions

pg 5 para3

Strong

leadership

pg 5 disc

para 2

perceived

need for

improveme

nt/awarenes

s of risks

and benefits

pg 5

discussion

para 2

Inefficient

technology

Pg 5 para3

Assessment of risk

pg 4 para 2

sense of risk pg 5

discussion para 2

Awareness

and education

pg 4 para 2

inefficient

technology pg

5 discussion

para 2

organisational

readiness pg

5 discussion

para 1

demands on

time pg 5

discussion

para 2



resistance

to change

pg 5 para 3



Conditions Prior

To

implementation

Cost Need and

importanc

e of

validation

and

evaluation

Attitudes Ease of Use of

System


y and standards

Education

and Training

Technological

Issues

Communicatio

n Issues

Organisationa

l Issues

Other

7 Johnso

n 2001

�

Situational –

external

environmental

factors affecting

access to or use

of technology. Pg

1para 5

Access to IT is

influenced by the

financial situation,

the location of the

practice (rural vs

urban), the size of

the practice, and

the average age

of the PHCPs –

with older PHCPs

less likely to have

�

Situational –

economic

realities. Pg

1para 5

Time pressure

represents the

most

signigicant

barrier to the

adoption of

potentially

useful

technologies.

Today’s

PHCPs spend

an average of

17 mins per

patient

�

�

attitudinal

behaviours or

opinions

contrary to

those needed

to adopt a

technology pg

1 para 5.

Knowledge

and attitudinal

barriers pg

1375 col 2

para 4

Lack of insight

about the

benefits of IT

interventions

succeeded

when decision

support was

provided to

clinicians

automatically pg

3 col 2 para 6

similarly

systems that

were provided

as an integrated

component of

charting were

significantly

more likely to

succeed than

stand alone

Legal – regulated or

unregulated practices

that affect use of a

technology pg1 para

5

The diversity of our

health care

environment, in terms

of policies, practice

styles and revenue

generating activities

make it difficult to

achieve consensus

about what or how

technology should be

applied to the field.

Pg 1375 col 1 para 5

Cognitive

and/or

physical

insufficient

skills or ability

to use a

technology pg

1 para 5

Other

environmenta

l barriers that

are critical to

primary care

practices

include the

cost of

training

personnel

(1375, col 2,

interventions

succeeded

when decision

support was

provided to

clinicians

automatically pg

3 col 2 para 6

similarly

systems that

were provided

as an integrated

component of

charting were

significantly

more likely to

succeed than

stand alone



less likely to have

access to (or

comfort with)

technology. Pg

1375 col2 para1

Collaborators with

stakeholders –

professional

societys such as

the American

Academy of

Pediatrics….have

assumed a

leadership role in

taking steps to

promote the

adoption of IT.

These societies

have begun to

collaborate with

industry,

government, and

consumer

leaders.

Collaboration…..i

s critical both to

short term

patient

encounter and

are

reimbursed

based on the

time needed

by each

patient.

Studies have

shown that

time pressure

clearly affects

the tolerance

of providers

for

technologies

such

as….computer

based

documentatio

n tools pg

1375 para 4

IT is neither a

one time nor a

low risk

investment.

The sources

benefits of IT

pg 1376 col 1

para 2

IT advocacy.

In concert with

increasing

awareness of

IT, we need to

establish a

network of IT

advocates who

can facilitate

the movement

of more

apprehensive

colleagues

with the help of

other groups.

Pg 1378 col 1

para 4

as our new

systems affect

larger, more

heterogenous

groups of

stand alone

systemes pg 3

col 2 para 6

systems that

used a

computer to

generate the

decision support

were

significantly

more effective

than systems

that relied on

manual

processes pg 3

col 2 para 6

systems that

prompted

clinicians to

record a reason

when not

following the

advised course

of action were

significantly

The adoption of IT

often is impeded by

questions about

liability associated

with the dissemination

of information as well

as concerns about

what balance

between security and

and access is

acceptable to

consumers.

Concerns about

security, patient

confidentiality, and

liability have affected

the PHCPs

confidence in using

email to communicate

with patients. Pg 1375

col2 para3

para 1)

When

surveyed,

PHCPs

identify a lack

of IT training

as a major

barrier to

using

technologies

they consider

valuable

(1375, col 2,

para 2)

Continuing

medical

education

courses

should be

developed

that teach not

only

computer

literacy skills

but also cover

stand alone

systemes pg 3

col 2 para 6

systems that

used a

computer to

generate the

decision support

were

significantly

more effective

than systems

that relied on

manual

processes pg 3

col 2 para 6

systems that

prompted

clinicians to

record a reason

when not

following the

advised course

of action were

significantly



successes….and

longer term

goals….. pg 1378

col 2 para3

of this risk

include the

financial cost

of these

systems to a

practice or a

payer and the

uncertain cost

benefit. Pg

1375 para5

Other

environmental

barriers that

are critical to

primary care

practices

include the

expense of

systems9 and

the cost of

customizing

systems for a

PHCP-based

practice

(1375, col 2,

para 1)

people and

more

organisational

areas, the

major

challenges to

systems

success often

become more

behavioural

than technical..

a “technically

best” system

can be brought

to its knees by

people who

have low

psychological

ownership in

the system

and who

vigorously

resist its

implementatio

n (1375 col 2

para 5)

more likely to

succeed than

those that

allowed the

system to be

bypassed

without

recording a

reason to

pg 3 col 2 para

6

systems that

provided a

recommendatio

n were more

likely to succeed

than systems

that provided

only an

assessment pg

3 col 2 para 6

issues such

as:

confidentiality

and security

on the

Internet, as

well as

existing

policies

affecting the

use of IT in

medicine;

benefits of

adopting

specific IT

tools; how to

evaluate IT

solns; where

to learn more

about IT.pg

1376 col 2

para 4

more likely to

succeed than

those that

allowed the

system to be

bypassed

without

recording a

reason to

pg 3 col 2 para

6

systems that

provided a

recommendatio

n were more

likely to succeed

than systems

that provided

only an

assessment pg

3 col 2 para 6



These

problems

include

working in a

delivery

system that

rewards time

efficiency

more than the

quality of care

provided, and

incurring the

financial

burden of

costly IT with

a limited

potential for

return on

investment

(pg 1375, col

1 para 1)

access to IT is

influenced by

the financial

situation pg

137

their lack of

ownership may

be due to a

variety of

factors,

including a

lack of insight

about the

benefits of IT,

their concern

about the

magnitude of

change

caused by IT,

and their

ambivalence

about the

processes that

IT is designed

to improve (pg

1376, col 1

para 1)

Appreciating

the magnitude

of change .

Primary care

physicians



137

5 col 2 para 1

may be lesss

enthusiastic

about the pace

with which

benefits are

realised after

implementing

IT. For eg.

Physicians

have found

that newe

systems result

in increased

patient waiting

time and staff

workload for a

while after the

system is

implemented.

pg 1376 col 2

para 2.

access to IT is

influenced

by….the

average age of

the PCHPs –



with older

PHCPs less

likely to have

access to (or

comfort with)

technology pg

1375 col 2

para 1



Conditions Prior

To

implementation

Cost Need and

importance of

validation and

evaluation


System


y and standards

Education and

Training

Technological

Issues

Communication

Issues

Organisational

Issues

Other

8 Kawamoto

2005

�

Alignment of

decision support

objectives with

organisational

priorities and

financial

interests pg 3

table 2

System

developed

through iterative

refinement

process pg 3

table 2

Active

involvement

of local

opinion

leaders pg

3 table 2

a common

theme among all

four features is

that they make it

easier for

clinicians to use

a decision

support

system….as a

general principle

then our findings

suggest that an

effective clinical

decision support

system must

minimise the

effort required by

clinicians to

receive and act

on system

recommendation

s. Pg 7 col 1

para 5

assessments and

recommendations are

accurate pg 3 table 2

System

developed

through

iterative

refinement

process pg 3

table 2

interventions

succeeded

when decision

support was

provided to

clinicians

automatically

pg 3 col2 para

6

similarly,

systems that

were provided

Alignment of

decision

support

objectives with

organisational

priorities and

financial

interests pg 3

table 2



para 5

clinician –

system

interaction

features: 1.

saves clinicians

time or requires

minimal time to

use; clear and

intuitive user

interface with

prominent

display of advice

pg 3 table 2

interventions

succeeded when

decision support

was provided to

clinicians

automatically pg

3 col2 para 6

similarly,

systems that

as an

integrated

component of

charting were

significantly

more likely to

succeed than

stand alone

systems pg 3

col 2 para 6

….computer

to generate

the decision

support were

significantly

more effective

than systems

that relied on

manual

processes pg

3 col 2 para 6

systems that

provided a

recommendati

on were more



were provided as

an integrated

component of

charting were

significantly

more likely to

succeed than

stand alone

systems pg 3 col

2 para 6

system is fast –

pg 2 table 2

computer to

generate the

decision support

were significantly

more effective

than systems

that relied on

manual

processes pg 3

col 2 para 6

systems that

likely to

succeed than

systems that

provided only

an

assessment

pg 3 col 2

para 6

systems that

prompted

clinicians to

record a

reason when

not following

the advised

course of

action were

significantly

more likely to

succeed that

allowed the

system to be

bypassed

without

recording a

reason pg 3

col 2 para 6



prompted

clinicians to

record a reason

when not

following the

advised course

of action were

significantly

more likely to

succeed that

allowed the

system to be

bypassed

without recording

a reason pg 3

col 2 para 6

systems that

provided a

recommendation

were more likely

to succeed than

systems that

provided only an

assessment pg 3

col 2 para 6



Conditions

Prior To

implementatio

n

Cos

t

Need and

importanc

e of

validation

and

evaluation


of System


y and standards

Education

and

Training

Technologica

l Issues

Communicatio

n Issues

Organisational

Issues

Other

9 Kukafk

a 2003

�

factors within

the

organisational

infrastructure

required to

enable usage

eg available

resources,

supportive

policies and

accessibility to

the IT system.

Pg 224 col 1

para 1

in addition,

they must

ensure that the

users are

�

predisposing

factors….

Include

perceived

usefulness,

pg 223 col2

para2

in addition,

they must

ensure that

the users are

prepared in

terms of

knowledge of,

and belief, in

the the new

system pg

226 col 2 1st

predisposing

factors….

Include,

ease of

use…pg

224, col 1

para1

Construction

of the

system’s

decision-

support rules

should be

consistent,

salient, user-

friendly and

customizabl

e for a wide

range of

skills that

an

individual

needs to

use a

system pg

224 col 1

para 1

usage

inducing

strategies,

eg skills

training pg

223 col 2

para 2.

phase 4:

factors

issues may relate to

phase 1:

organisational

needs and goals, in

terms of engaging

stakeholders. Pg

222 para 3

phase 2:

organisational

needs amenable to

IT system

solutions…Detailing

the system

….functionality…an

d how they relate to

perceived needs of

end users pg 223

para 2

phase 5:

system use

inducing

strategies

focuses on

developing

and

implementin

g

approaches

that are

proactive

and

specifically

targeted to

influencing

favourably

the

predisposing

, enabling

and



users are

prepared in

terms of

knowledge of,

and belief, in

the the new

system pg 226

col 2 1st para

the healthcare

organisation

also must

prepare its

workforce for

major changes

that are

associated

with the

instalment of a

new

innovation.

They must

prepare the

environment

for change and

then assist in

the adoption of

that change by

providing the

226 col 2 1st

para

reinforcing

factors follow

a behaviour

and provide

the reward or

incentive for

the repetition

or

persistence of

the

behaviour. Pg

224 col 1

para 1

is more

clearly

reflected in

phase 3:

behaviours

linked with

system use,

where

reference is

made to

collaboration

users (p101

col 2 para 3)

associated

with

behaviours

under

“enabling

factors”

,examples

of which

include

skills

…..that

enable IT

usage. Pg

223 para 4

para 2

phase 3 :

behaviours linked

with system use, to

define the actual

steps through which

a system user will

have to move pg

223 3rd para

factors within the

organisational

infrastructure

required to enable

usage eg available

resources,

supportive policies,

and accessibility to

the IT system pg

224 col 1 para 1

phase 4: factors

associated with

behaviours under

and

reinforcing

factors

identified in

phase 4.



providing the

economic

resources,

incentives and

social support

to facilitate the

change pg 236

col 2 1st para

Phase 1 an

assessment of

the

organisational

needs and

goals pg 222

col 2 para 1

collaboration

patterns,

practical and

social roles

pg 223 para 3

phase 4:

“predisposing

factors are

mostly

psychological

, including

cognitive

dimensions of

knowing,

feelings,

believing, and

having a

sense of

efficacy. They

are the

antecedents

to behaviour

that provide

the basis for

motivation.

Pg 223 col 2

last para

“enabling factors”

,examples of which

include… other

organisational

structure that enable

IT usage. Pg 223

para 4

the results of our

analysis indicate the

necessity for

developers to focus

on the needs and

goals of the

organisation (phase

1) pg 236, col 1 last

para



Conditions

Prior To

implementation

Cost Need and

importance of

validation and

evaluation


of System


y and standards

Education and

Training

Technologica

l Issues

Communication

Issues

Organisational

Issues

Other

10 Leat

t

2006

�

Strong

management

commitment

(p96, under

mgt support-

EMR

Management

support should

involve clinical

as well as

administrative

leadership pg

96 col 2 para 5

Size of practice

and

organisation

(p98, para1&4)

�

financial

resource

availability –

commit

sufficient

financial

resources for

hardware,

software, and

reduced

productivity;

avoid

underestimatin

g necessary

financial

resources;

include costs

for training,

technical

support, and

productivity

losses,

integrate CQL

processes fig 2

pg 97

Employment of

continuous

quality

improvement

processes to

routinely

monitor and

assess

implementatio

n processes

and new

technology

(p100 col 1,

para 1 under

MAR)

��

Since

experience

has shown

that

resistance

to CPOE

comes

primarily

from

physicians

and other

staff who do

not want to

change their

work

processes

(96, under

mgt

support-

CPOE)

�

System

accessibility,

efficiency of

use and

ease of

navigation

(p99 col 2

para 2)

construction

of the

systems’

decision

support roles

should be

consistent,

salient, user

friendly, and

customisabl

e for a wide

range of

develop new practice

standards and

routines fig 2 pg 97

Scheduling

fewer patients

during the

learning period

and ensuring

protected and

adequate time

for training

(p98 para2;

p98 col2 –

p99). Training

initially but

also ongoing

(also p100 col

1 para 2)

Use of one-to-

one

implementatio

n training

sessions (p100

col1 para1)

involve key

individuals in

the EMR

system

design fig 2

pg 97

Ongoing

technical

support and

back-up (p99

col2 para3;

p100 col 1

para2)

Iterative

refinements

or revisions

to the system

(p101 col2

para 3)

communicate

about

expectations and

experiences fig 2

pg 97

capitalise on the

ability of

individuals holding

dual roles (ie

clinical and

administrative) to

bridge gaps

between groups of

stakeholders fig 2

pg 97

Role of leaders in

communicating

and connecting

with staff and

addressing

Establishment

of

interdisciplinary

committees

showing

support (p97,

last para; p100

top of col 2)

‘studies

identified the

importance of

organizational

preparation for

reductions in

productivity,

both during and

after

implementation

, as critical to

the effective



‘studies

identified the

importance of

organizational

preparation for

reductions in

productivity,

both during and

after

implementation

, as critical to

the effective

implementation

of EMR’ (p98

para2)

Culture of

change and

innovation (p98

col2 para2)

also fig 2. pg

97

Communicating

the reasons for

adopting and

losses,

emphasise

patient safety

gains to justify

financing;

secure

sufficient

resources for

implementation

processes FIG

1 pg 97

Cost and

funding (p98,

all of col 1)

Users’

expectations

of the

system (p99

col 1 para3)

Shared

belief in

system’s

potential

(p98 col2

para4)

Physician

champion

(p96, under

mgt

support-

EMR)

Previous

experience

of

computers –

mixed

findings

users pg 101

col 2 para 3

design the

system for

efficiency of

use and

avoid

interfering

with patient

care fig 2 pg

97

provide

adequate,

timely and

ongoing

training,

protect time for

training, fig 2

pg 97

ensure EMR

system back

up fig 2 pg 97

provide

ongoing on

site technical

support fig2

pg 97

involve key

individuals in

the EMR

system

design

design the

system for

efficiency of

use and

avoid

interfering

with patient

care fig 2 pg

97

addressing

workflow concerns

(p97, para1)

it is also important

to maintain

bidirectional

communication

throughout the

process to enable

staff to see current

measures

indicating cost

savings and error

reductions, as well

as to ensure that

staff are providing

input about the

system’s

usefulness and

potential

modifications(p10

0 col1 para3)

Physicians’

concerns that

system would

interfere with or

implementation

of EMR’ (p98

para2)

Shifting

organisational

priorities to

support system

(p98 col 1 last

para)

Substantial

reconfiguration

of roles,

responsibilities

and work tasks

among staff in

various

departmental

units (p100 col

2 last para

before

discussion)

develop

committees

and teams that



adopting and

implementing

CPOE .In

addition to

conveying

expected cost

savings and

reductions in

errors prior to

implementation

, however ,

(p100 col1

para3)

Partnerships

between

vendors and

purchasers

(p100 col 2 top)

Preparation of

staff for

changes in

responsibilities

as well as

patterns and

(p99 col 1

para2)

foster

positive

perceptions

of MARS

potential to

decrease

medication

errors and

turnaround

time for

medication

orders fig 2

pg 97

staff

resistant to

change pg

96 under

mgt support

CPOE

encourage

participation

in the design

of the system

by clinicians

pg 97 fig

2/inclusion of

appropriate

individuals in

the design of

the system

pg 99 col 2

para 2

inclusion of

appropriate

individuals in

the design of

the system

pg 99 col 1

last para.

construction

of the

systems

decision

interfere with or

negatively impact

the physician-

patient encounter

(p99 col 2 para 2)

and teams that

meet and work

over an

extended

period of time

fig 2 pg 97

develop formal

goals,

objectives and

key indicators

of success fig 2

pg 97

foster a climate

reflecting that

the new system

is being

promoted

supported and

rewarded fig 1

pg 97



priorities of

work (p101 col

2 just before

conclusion)

Inclusion of

appropriate

individuals in

the design of

the system

(RI?) (p99 col 1

last para)

involve key

individuals in

the EMR

system design

design the

system for

efficiency of

use and avoid

interfering with

patient care fig

2 pg 97

solicit clinician

input on system

decision

support rules

should be

consistent,

salient, user

friendly and

customizable

for a wide

range of

users pg 101

col 2 para 3



design to

facilitate buy in

fig 2 pg 97

encourage

participation in

the design of

the system by

clinicians pg 97

fig 2/inclusion

of appropriate

individuals in

the design of

the system pg

99 col 2 para 2

clear and

formal goals

and objectives

and

communication

strategies pg

100 col 1 under

MAR

anticipate



challenges to

implementation

fig 2 pg 97



Conditions

Prior To

implementatio

n

Cost Need and

importance of

validation and

evaluation


of System

Security/Confidentialit

y and standards

Education

and Training

Technological

Issues

Communicatio

n Issues

Organisational

Issues

Other

11 Lu

2005

Cost-saving

(sec 4.1.1)

Cost-

effectivenes

s (Table 2)

Maintenance

: repair costs

constitute

another

barrier

(Table 4 pg

415)

Clinical

impact: the

literature

suggests that

utilisation of

PDAs saves

clinicians time

in regard to

accessing,

retrieving, and

recording data

4.1.5 pg 414

Clinical impact

(quality of

care) It can

help health

care providers

to better take

care of their

patients by

improving their

Personal

factors (eg

comfort with

technology,

comfort with

device, age,

memory

problem

(forgot to

carry the

device):

Dependency

or over

reliance on

the device

table 4 pg

415. (Table

4) pg 415

lack of

needs or

motivation

Usability pg

409

summary.

Time-saving

for

physicians,

easily

incorporated

into workflow

(4.1.3)

Time saving

table 2 pg

413

Real-time

access to

patient

information

Security concerns.

Summary pg 409

Security and speed of

wireless transmission:

an integrated

input/output device

with data encryption at

both ends of

transmission should

be developed to meet

security needs table 4

pg 415.

Experts estimate that

wireless technology

will become an

essential component

of hospital operations

within the next few

years, after some

Educational

benefit (-

reported

improvemen

t in

participants

educational

experience

(4.1.2

Seamless

integration of

PDA

technology with

hospital

information

systems

summary pg

409

Barrier in

cases of

technology not

integrated with

EMRs (Table

4)

Inadequate

technology

support or

access barriers

Benefits in

communication

(Table 2) pg

413

alternative

methods used

or competing

technologies

eg some

physicians

prefer to write

notes on paper

than PDAs.

Table 4 pg 415

Lack of

technical and

organisational

support

summary pg

409 summary

Organisational

barriers, eg.



personal and

professional

information

mangement,

providing

medical

decision

support via

acces to

evidence

based

materials and

allowing

remote access

to patient

data.(4.1.5) pg

414

Evidence-

based practice

support (Table

2) pg 413

Ineffectivenes

s table 4 pg

415

Table 4 pg

415

Most health

professional

s perceive

PDAs as

useful tools

that can

enhance

their practice

by offering

mobility and

functionality

in a small

device that

fits into one’s

pocket. 4.2

pg 414

Negative

patient

perception

(Table 4)

(Table 2)

Mobility: In

the

healthcare

environment,

PDAs allow

the clinicians

to have

access to

patient

information

whenever

and

wherever

they prefer,

to record

patient data

in real time,

to look up

refs and to

communicat

e with

colleagues.

Table 2 pg

413

Data entry –

unintuitive

major issues,

bandwidth, availability

and security are

resolved. Pg 416 col 1

para 2.

Error reduction (4.1.4)

Increased safety or

reduce medical errors.

Table 2 pg 413.

Table 4 pg 415

Physical

Design : size,

weight and

small screen.

Table 4) pg

415

Technical

difficulties:

difficulties in

gaining internet

access at

times; and

synchronizatio

n with home

computers

table 4 pg 415.

Experts

estimate that

wireless

technology will

become an

essential

component of

barriers, eg.

Lack of

institutional

support and

concerns

about legal

issues. Table 4

pg 415

Some barriers

can be

eliminated by

organisational

changes such

as providing

the necessary

infrastructure

for the

handhelds,

technical

support, and

funding for the

devices and

software

applications.

Pg 416 col 2

para 1.



wider

organisational

or institutional

adoption is

relatively slow,

perhaps

because there

still relatively

little published

evidence of a

positive impact

of handheld

computers on

patient

outcome.

More PDA

evaluation

studies are

also

encouraged to

explore

expanding

roles and

clinical

impacts of the

technology in

health care.

and not easy

to use table

4 pg 415

Usability

issues, eg

difficult in

use, limited

memory,

screen size.

Table 4 pg

415

For example,

Larkin

reported that

if a PDA

could not fit

into a

physician’s

workflow

seamlessly

or if it

required

extra effort,

the physician

was less

likely to use

component of

hospital

operations

within the next

few years, after

some major

issues,

bandwidth,

availability and

security are

resolved. Pg

416 col 1 para

2.

Greater

efficiency and

increased error

reduction will

be achieved if

data capture or

data entry are

permitted

wherever and

whenever it is

available table

2 pg 413.

More and

more health

care

professionals,

clinics and

hospital

organisations

are

considering

the purchase

and

implementatio

n of handheld

technologic

solutions to

save time,

improve

medical data

access, assist

with medical

information

management,

improve

patient care

efficiency and

reduce

medical errors.

Pg 416 col 1

para 3



232

Pg 417 col 2

para 1.

likely to use

the device.

Pg 416 col 1

para 4

section 4.3

Advances in

technology

and usability

help

overcome

some of the

barriers to

adoption pg

416 col 1 last

para.

Customisation:

various

functions or

programs can

be added to

support

different

medical

specialities.

Table 2 pg 413

Delicate

devices: fears

of breaking the

device make

some users

limit their uses

to avoid

damaging it.

Table 4 pg

415.

Personal

factors:

physical factors

eg large fingers

(too big for the

Better

designed PDA

hardware and

software

applications

summary pg

409



buttons) and

poor eyesight

(cannot read

the small fonts)

table 4 pg 415



Conditions Prior

To

implementation

Cost Need and

importance of

validation and

evaluation


of System


y and standards

Education and

Training

Technological

Issues

Communication

Issues

Organisational

Issues

Other

12 Ohinmaa

2006

Planning

readiness (incl.

telemedicine

strategic plan,

needs

assessment

analysis and a

business plan)

Workplace

readiness (incl.

designating a

telemedicine co-

ordinator and

ensuring change

management

readiness) pg 42

para 2

Statewide

cooperative

Allowed more

patients to be

treated at

lowered costs

pg 42 para3

Reimburseme

nt pg 42 para

4

For costing it

is imp that the

non clinical

use be

considered in

the allocation

of investment

as well as line

and staff

assessment

of these

programmes

had shown

significant

cost savings,

a decrease in

……turnaroun

d time and

improved

productivity

pg 42 para 6

Lack of “buy

in” ,

resistance to

change pg 42

para2

Ownership,

attitudes of

participants,

professional

connections

remote

location pg 42

para 4

Significantly

reduced time

for the PACS

support staff

to transfer

imaged….red

uced the

clinical

decision-

making time

(esp in

neurosurgery)

pg 42 para1

Systems have been

shown to be safe pg

42 para3

Liability requirements

pg 43 para 5

Lack of staff

preparation pg

42 para 2

Limitations of

technology

associated

with pt

selection (eg

severely

demented pts

who could not

sit still) pg 42

para4

Communication

between

clinicians has

also improved pg

42 para 1

one of the

biggest

difficulties in

adapting

telemedicine

applications

from abroad is

the difference

in the health

care delivery

systems. Pg

43 para 4 col

1

Improved

productivity in

healthcare pg

42 para6



systems that

provide

scheduling

support pg 43

para 6

Changes in health

care legislation by

government

agencies pg 42

para 5

Directed,

systematic

government

policy aiming to

increase

investment in

technology can

enhance

networking and

collaboration

within the

healthcare

system pg 43

para 1

Government

charges for

each type of

service. Pg 43

6



money to build

infrastructures

such as

broadband

networks or

statewide

cooperative

systems that

provide

scheduling

support. Pg 43

para6

Changes in health

care legislation by

government

agencies pg 42

para 5

Workplace

readiness (incl.

designating a

telemedicine co-

ordinator and

ensuring change

management

readiness) pg 42



para 2

Distance is a key

factor in the use

of telemedicine

pg 43 para 2



Conditions

Prior To

implementation

Cost Need and

importance of

validation and

evaluation


of System


y and standards

Education and

Training

Technologica

l Issues

Communicatio

n Issues

Organisationa

l Issues

Other

13 Papshev

2001

Fragmentation

of the health

care market

(p731) and

table 3 pg 729

Regulatory

indecisiveness

(p 733)

regulatory

indecisiveness

table 3 pg 729

and pg 735

summary

Considerable

funding

requirements,

segmentation

of healthcare

markets, lack

funding

requirement

s table 3 pg

729

considerable

investment

capital pg

735

summary

Funding

requirement

s (p731)

Retail profits

may be

unfavourably

impacted

(p733,

column 2,

para 1).

Unification of

point of care

and point of

service

processes (p

728). “electronic prescribing would enable prospective drug utilisation review (DUR) to service its intended purpose in a most efficient manner. Advise pharmacists of potential therapeutic conflicts that should be resolved

providers

resistance to

change Pg 725

abstract data

synthesis and

table 3 pg 729

and pg 732

and pg 735

summary

potential

solutions

include gaining

stakeholder

support in

implementatio

n of the

technology. Pg

725 abstract

data synthesis

through

eliminating

these

redundant,

time

consuming

steps,

electronic

prescribing

should allow

practitioners

to focus on

providing

care rather

than on data

management

. Pg 729 col

1 para 1

Efficiency,

productivity

(p728 final

Lack of technology

standardization

(p732) and table 3 pg

729 and pg 735

summary

potential solutions

include establishing a

standardising

warehouse or a

router. Pg 725

abstract data

synthesis and pg 733

Familiarizatio

n with the

technology (p

733 line 3)

Products and

technology

pp 726 – 7

Pharmacists:

potential for

improved

communication

with the patient

(p733 para 3

column 1).

Improved

quality of care

pg 729. and

table 3 pg 729

and pg 735

summary



of technology

standardisation

, and regulatory

indecisiveness

create

boundaries to

the widespread

use of

automated

prescribing

Pg 725 abstract

data synthesis

resolved by physicians. If DUR initiated at the point medications are prescribed, a higher intervention rate could be achieved.

Reduction of

medication

errors (p 729)

and table 3 pg

729 and pg

735 summary.

computerised

prescribing

can provide

immense

benefits to

healthcare

providers,

patients and

managed

care. Pg 735

summary.

Stakeholder

buy in.

Stakeholders

include MCOs,

legislators,

providers and

patients pg

734

Increased

patient

satisfaction (p

731)

“facilitating

seamless

resolution of

formulary and

prior

authorization

issues …

healthcare

providers had

more time to

devote to their

patients, while

patients were

less frustrated

in efforts to

paragraph)

elimination of

time gap

between

point of care

and point of

service table

3 pg 729 and

pg 735

summary



“a means of monitoring and restricting physicians’ prescriptive authority” p 733 para 1.

receive

medications

prescribed by

their

physicians”.



Conditions

Prior To

implementatio

n

Cost Need and

importance of

validation and

evaluation


System


y and standards

Educatio

n and

Training

Technological

Issues

Communicatio

n Issues

Organisational

Issues

Other

14 Peleg

2006

Establish

public policies

that provide

incentives for

implementing

CDSS p 76

middle column

and

designin

g them to

be cost

effective

pg 73

middle

col para

1

lack of

improved

performance

could be due

to any number

of factors,

such as lack of

support among

colleagues pg

78 col 1 para 1

lack of

improved

performance

could be due

to any number

of factors,

such as

human-

computer

interface

“include

patients’

perspective

s of their

health

problems

and

preferences”

p 76 column

3

“physicians

ability to

change the

knowledge

base” p 76

3rd column

para 2

timely advice

should be

provided p 76

middle column

and last col

speed is

everything p 76

middle column

bottom

anticipate

needs and

deliver in real

time p 76

middle column

bottom

simple

interventions

sharing is enhanced

through standards pg

74 col 2 para 3

implementation of

CDSSs are greatly

aided by

standardization in

information system

infrastructure,

including standard

terminology, data

model, data exchange

format, and other

clinical information

systems services. Pg

77 col 1 para 2

in which

modellers

represent the

medical

knowledge that

enables the

CDSS to deliver

appropriate

decision support

services during

the care

process. Pg 73

col 3 .

provision of a

direct

recommendatio

n rather than

just an

assessment that

is presented to

the clinician for

in this context,

the goals of

modern cdss

go beyond the

original focus

of producing

expert level

advisories and

extend to

include….

facilitating

communication

among

providers pg 73

middle column

para 1

that

implementers

of CDSS can

use to work

through the

process of

identifying

stakeholders,

pg 73 col 3

bottom

workflow

integration

should be

considered. p

76 middle

column and

also bottom of

column and

also 3rd column



interface

problems or

lack of time pg

78 col 1 para 1

when

evaluators

were not also

the system

developers,

the proportion

of systems

reporting

improvement

dropped

significantly pg

78 col 1 para 2

it is difficult to

distinguish

effects caused

by the CDSS

from effects

caused by the

change in the

work practices

respond to

user needs

p 76 last

column 2nd

para

Need to

consider

patient

preferences

pg 73

middle col

para 1

work best p 76

middle column

p 76 middle

column bottom

automatic

decision

support as part

of clinician

workflow pg 76

col 3 para 3

provision of

decision

support at the

time and

location of

decision making

pg 76 col 3 para

3

The

implementation

s of these

rationalistic

technological

consideration pg

76 col 3 para 3

to be effective

and successful

CDSSs need to

be integrated

into health

information

systems that

supply the

patient data

CDSSs need.

Pg 77 col 1 para

2

Lots of technical

aspects

The knowledge

modelling tasks

involves

elicitation,

representation,

sharing,

evolution, and

delivery of

a shift toward

specialised

and focused

system,

interacting

systems that

are integrated

into the clinical

environment

and workflow

pg 73 col

middle para1

“elucidates …

the information

flow and the

work flow, the

roles and

responsibilities

, and the

communication

and co-

ordination

patterns of the

care process”

p73 3rd

column.



induced by the

implementatio

n of CDSS.

Pg78 col 1

para 3

Clinical effects

and costs of

the system

should be

evaluated p 76

middle column

the importance

of evaluating

CDS pg 73

middle column

para 1

that

implementers

of CDSS can

use to work

through the

process of

monitoring

interventions

required a

“disciplined

practice” where

clinicians

entered well

defined input

data at

appropriate

times and the

output of the

systems is

realisable in the

clinic. The

conflict between

these

requirements

and the

evolving,

contingent,

emergent

nature of

medical work

contributed

towards

difficulties in the

adoption of

CDSSs/ pg 73

col 1 bottom of

delivery of

knowledge to

users pg 74 col

1 para 2

The system

should be

developed with

an ability to be

maintained and

extended p 76

middle column

Capture

evidence in

machine

interpretable

knowledge

bases p 76

middle column

Develop

maintainable

foundations for

computer based

decision support

The

introduction of

the system

caused an

increase in the

number of

coordination

and verification

requirements,

sharing of login

session by

different users,

and the

disruption of

workflow due

to the

geographical

locations of the

clinical

workstations.

Pg 78 col 2 1st

para

“potential

affect on the

division of



specific CDS

interventions

pg 73 col 3

bottom

clinical effects

and costs of

the system

should be

evaluated p 76 middle column mentioned twice in this section

monitor

impact, get

feedback and

respond p 76

middle column

bottom

the complexity

of medical

practices and

the high cost

of

implementing

page.

p 76 middle

column

Flexibility p 76

last column 2nd

para

work among

care providers”

p 73, middle

column, top

“how CDS

would shape,

and in turn be

shaped by the

organizational

structure and

practices of

providers” p 73

middle column

top

Elucidates the

organization

goals, p73 3rd

column near

top

….that

implementers

of CDSS can

use to work



implementing

CDSSs make

evaluation of

CDSSs both a

challenge and

a necessity.

Pg 77 col 2

bottom para

through the

process of

determining

the goals and

objectives of

the CDSS,

cataloguing the

host

information

systems

capabilities

and selecting,

deploying

specific CDS

interventions

pg 73 col 3

bottom

the main

challenge in

implementing

CDS is not so

much trying to

fit CDS into

existing

workflow, as it

is managing

the ongoing



process of

organisational

development

that was

triggered by

the CDS

intervention.

Pg 74 col 1

para 1

CDSSs are

part of a

knowledge

management

toolkit that a

healthcare

organisation

can employ to

deliver the

“right

knowledge to

the right

people in the

right form at

the right time.

Pg 73 last col

near top



much of the

current

research

emphasises

the importance

of modelling

the integration

of cdss with

the

organisational

workflow and

information

systems. Pg

75 col 2

bottom

integration into

IT environment

workflow

integration

should be

considered. p

76 last column

2nd para



Conditions

Prior To

implementatio

n

Cost Need and

importanc

e of

validation

and

evaluation


System


y and standards

Education

and

Training

Technological

Issues

Communicatio

n Issues

Organisationa

l Issues

Other

15 Shekell

e 2006

Additional

barriers

included

difficulty with

financial

incentives Pg

57 para 3

Cognitive and

or physical

barriers

include

physical

disabilities .pg

57 para 2

Challenges for

adoption of

electronic health

records…..include

d cost, . Pg 57

bottom para

Misalignment of

costs and benefits

(p58 top line)

Situational barriers

included time and

financial

pressures,

unproven return on

investment,

insufficient access

to the internet or to

Additional

barriers

included

difficulty with

physician

attitudes. Pg 57

para 3

physician and

organizational

resistance due

to the perceived

negative impact

on the

physician’s

workflow” p 57,

para 3.

satisfaction

satisfaction

(physician) was

correlated most

strongly with the

ability of the HIT

system to

perform tasks in

a

“straightforward”

manner pg 57

para 1

Liability barriers

included ….Concerns

about privacy and

confidentiality (p 57,

bottom para) and pg

57 para 2

Challenges for

adoption of electronic

health

records…..included

cost, technical issues,

system

interoperability. Pg

57 bottom para

Cognitive

and or

physical

barriers

include

insufficient

computer

skills.pg

572nd

para).

Lack of a

well

trained

clinical

informatic

s

workforce

to lead the

process

Situational barriers

included software

not being

supportive of

pediatric practice

needs Pg 57 para

2

Challenges for

adoption of

electronic health

records…..include

d cost, technical

issues, system

interoperability. Pg

57 bottom para



computer

technology in the

office setting, the

prohibitive cost of

information

technology for

practices. Pg 57

par 2 high cost pg

57 para 3

(physician) was

correlated most

strongly with the

ability of the HIT

system to

perform tasks in

a

“straightforward”

manner pg 57

para 1

(p57

bottom

para)

Additional barriers

included

complementary

changes in

support, electronic

data exchange. Pg

57 para 3

Product / vendor

immaturity pg 57

para 3

Additional barriers

included difficulty

with technology,

Pg 57 para 3



Conditions

Prior To

implementation

Cost Need and

importance of

validation and

evaluation

Attitudes Ease of

Use of

System


y and standards

Education and

Training

Technologica

l Issues

Communication

Issues

Organisational

Issues

Other

16 Studer

2005

Prior to

implementation

it may be

beneficial to

assess

expectations

and concerns

regarding EMR

system

implementation

. Pg 95 col 2

para4

Appropriate

individuals

throughout the

organisation

should be

included in the

design of the

EMR sytem

and

Lack of

financial

incentives and

rewards is a

barrier (p 96

col 1 para 4)

Financial

resource

Availability.

Costs

Associated

with EMR

System Start

UP and

Ongoing

Maintenance

are a major

barrier (p 94

col 2 last

para– 95 col 1

1st para);

It is important

to determine

the effect of

EMR

implementatio

n of patient

safety, clinical

quality and

costs. Pg 97

col 1 para 3.

“A

physician

champio

n is

essential”

p 94

column 2

para 6

The EMR

system

must be

accessible

, efficient

to use and

not

interfere

with or

negatively

impact the

physician

– patient

encounter

(p 96,

column 1,

bottom

para).

EMR system

downtime: exposed

physicians and

practices to risk of

patient injury and

liability (p 96, column

2).

concerns regarding

the privacy and

confidentiality of

patient information

stored in an EMR pg

96 col 2 para4

lack of standards for

data coding and

exchange pg 96 col 2

para 4

Subsequent

communication

and training

should address

unrealistic

expectations

and concern

and convey a

clear statement

of the goals for

and anticipated

benefits of

EMR

implementation

. Pg 95 col 2

para4

Training for

EMR system

users should

be adequate,

timely, tailored

EMR system

downtime:

exposed

physicians

and practices

to risk of

patient injury

and liability (p

96, column

2).

Subsequent

communication

and training

should address

unrealistic

expectations

and concern

and convey a

clear statement

of the goals for

and anticipated

benefits of

EMR

implementation

. Pg 95 col 2

para4

the importance

of considering

work flow

design as part

of the EMR

implementatio

n process pg

96 col 2 para 4

Management

support “strong

management

commitment is

critical and

should be

broadly

communicated

through the

organization” p

94. col 2 para

4



implementation

planning

process and

physician buy

in must be

obtained. Pg 96

col 1 para 6

Implementation

climate theme:

organisations

with a culture of

change and

that value

innovation may

have a greater

likelihood of

effectively

implementing

and emr

system (p 95,

column 1, para

5).

1st para);

Financial

resource

Availability

Theme

Number 2 :

The

Commitment

of Adequate

Financial

Resources,

including both

hardware and

software costs

and costs

associated

with reductions

in productivity

during and

after EMR

system

implementatio

n are vital. Pg

95 col 1 para 2

… available on

an ongoing as

needed basis

and include

simulated

patient

encounters

(p95 column 1,

bottom para).

There must be

Sufficient

protected time

for training for

all EMR system

users. (p 95

column 2, para

2 )

On going, on

site technical

support is

critical pg 96

col 1 para 2

4

preparation for

reductions in

productivity

during and

after

implementatio

n critical to

effective

implementatio

n (p 95, para

3).



Conditions Prior

To

implementation

Cost Need and

importance of

validation and

evaluation


System

Security/

Confideniality and

standards

Education and

Training

Technological

Issues

Communication

Issues

Organisational

Issues

Other

17 Vreeman

2006

End user

participation in

the development

process

fundamental to

implementation

success…for eg

a committee with

stakeholder and

user

representatives

that oversaw

system

implementation

P441 PLUS

para7 end user

participation in

the development

process. Pg 441

col 2 para3/

table 2 pg

440/pg 442 col 2

para

3recommendatio

financial

implications of

implementing

EHRs…the

financial

incentives and

disincentives to

implementation

are a major, if

not the most

important

concern. Pg 443

col 1 para 3

�

Computer

systems have

the potential to

introduce errors

pg 443 col 1

para 2

Plan and test

carefully to

ensure adequate

software and

hardware system

performance pg

443 col 2 para 6

Incorporating

workflow

analysis into

system

design…noted

�

Improved reporting

capabilities

…..facilitated

clinical decision

making for

individual patients.

Pg 438 col 2 para

5 and

Workflow or

behaviour

modification pg

441 para 2 eg.

Using the system

challenged

therapists prior

practices with

regard to

documentation and

reqd them to

change their

traditional

practices

Improved data

accuracy pg 440

para 4/ table 2 pg

440/ pg 442 col 2

para 2

security, privacy,

and confidentiality

were not prominent

issues in the

studies we

reviewed. Pg 443

col 1 para 3

likewise there is

widespread

recognition of the

crucial role of data

standards in health

care pg 442 col 2

para 3.

Workflow or

behaviour

modification pg

441 para 2…The

automated

process shifted

much of the

responsibility

…creating

workflow

inefficiencies

that reqd special

training and

system

modifications.pg

441 col 1 para 2/

table 2 pg 440

Staff training pg

441 para 6 and

pg 442 col 1

para 1./ table 2

pg 440

Software or

hardware

inadequacy.

For example

using donated

equipment or

using systems

with shared

mainframes

causing

problems pg

441 col 1 para

4/ table 2 pg

440

recommendati

ons: pursue

the efficient

capture of

coded data

pg 443 col 2

para 7

Improved inter-

departmental

communication

pg 440 para 3/

table 2 pg 440

enhanced

communication

between

physicians and

nurses was

noted…pg 442

col 2 para 1

Improved

reporting

capabilities

…cited the

capability for

more

comprehensiv

e reporting

that integrated

clinical and

administrative

data as a key

benefit.

…Helped

clinicians and

administrators

to be aware of

the current

departmental

workload

which helped

provide the



3recommendatio

ns: include end

users, especially

clinicians into in

the system

development

activities. pg 443

col 2 para 3

that workflow

shifts caused by

the system could

decrease clerical

work while

simultaneously

increasing the

responsibilities

of higher paid

professional

staff. Pg 442

para 2/ table 2

pg 440

Incorporating

workflow

analysis into

system

design…of

identifying

needed data

elements,

identifying how

collected, and

determining

whether

additional

elements are

practices

recommendations:

commit to data

standards pg 444

col 1 para 2

pg 440

Staff training

requiring

overtime on

nights and

weekends pg

441, para6.

recommendation

s: devote

significant

resources to

training pg 443

col 2 para 5

provide the

rationale for

how pts and

therapists

were

scheduled. Pg

438 col 2 para

5 and 6

Workflow or

behaviour

modification

pg 441 para

2…The

automated

process

shifted much

of the

responsibility

…creating

workflow

inefficiencies

that reqd

special

training and

system

modifications.

pg 441 col 1



needed to

support the

desired

functionality.. Pg

442 para 2/ table

2 pg 440/pg 442

col 2 para 3

para 2/ table 2

pg 440

Incorporating workflow analysis into system design…noted that workflow shifts caused by the system could decrease clerical work while simultaneously increasing the responsibilities of higher paid professional staff. Pg 442 para 2/ table 2 pg 440

computer

systems have

the potential

to disrupt

workflow pg

443 col 1 para

2

computer systems have



the potential to disrupt workflow pg 443 col 1 para 2

recommendati

ons:

incorpororate

workflow

analysis into

the system

design and

implementatio

n pg 443 col 2

para 3

recommendati

ons: devote

significant

resources to

training pg

443 col 2 para

5



Conditions

Prior To

implementatio

n

Cost Need and

importance of

validation and

evaluation

Attitudes Ease

of Use

of

Syste

m


and standards

Education

and Training

Technologica

l Issues

Communicatio

n Issues

Organisational

Issues

Other

18 Yarborough

2007

Perceived

organisational

support are

positive

predictors of

perceived

usefulness. Pg

660 para 2

The most

common

reason cited

by physicians

for resisting

email

communicatio

n with patients

is the lack of

reimbursement

for providing

such a service.

Pg 651 para 3

Cost as a

barrier pg 659

para 2

Costs, lack of

standards

as…major

another major

impediment to

physician

technology

utilisation is

the lack of

documentatio

n suggesting

that available

technologies,

do, in fact,

increase

quality of care

pg651 para

3/lack of

empirical

evidence

linking

information

systems to

quality or

financial

improvements

Personal

characteristics

have also

been identified

as barriers to

physician

technology

acceptance pg

661 para

2….examples

anxiety

regarding

computers,

value a

physician

places on a

close patient

relationship,

lack of

familiarity and

knowledge of

available

resources pg

Reliability and

dependency as the main

concerns with adopting

such technology

(handheld devices) pg

662 para 1

Fears about litigation,

privacy/confidentiality….

. also serve as barriers

to physician technology

utilisation. Pg 651 para 3

Confidentiality and clear

policies regarding error

reporting pg 661 para 1

Lack of infrastructure or

regulations to deal with

such issues (litigation)

Lack of

adequate

organisationa

l support

….also

presence of

appropriate

training and

infrastructure

pg 660 para 2

The ability to

customise

and organise

the

knowledge

captured on a

local level is

critical for

physician

technology

acceptance

pg 664

summary

Poor system

fit with

practice

patterns pg

659 para 2

Misinformation

….also serve

as barriers to

physician

technology

utilisation. Pg

651 para 3

The unique

relationship

between a

physician and

his or her

patient is

different from

that found in

other

industries.

Many

physicians

value this

interaction and

Interruption of

traditional

practice

patterns pg 656

para2-5…in

particular

focusing on

systems taking

more time…eg

more time per

physician per pt

when using

information

systems

Time as a

barrier to usage

of handheld

computers pg

659 para 2



barriers to IT

adoption. Pg

660 para 3

The cost of

technology

adoption has

been largely

ignored in the

literature

specific to this

framework

(TAM)..the

cost of

physician time

and change in

practice

patterns have

been identified

as significant

barriers to

technology

acceptance.

pg 667 para

2/3/pg 667

para 3

. Pg 659 para

4, pg 664

summary

Although the

benefits of

technology

adoption are

widely

prophesized,

little to no

empirical

evidence to

support such

claims are

offered in the

literature. Pg

651 para 3.

Lack of

evidence of

benefits of IT.

Pg 656 para

2. and pg 659

para 4…lack

of empirical

evidence

linking info

661 para

2/pg664

summary

The most

common

reason cited

by physicians

for resisting

email

communicatio

n with patients

is the lack of

reimbursement

for providing

such a service.

Pg 651 para 3

The report

also suggests

that physician

satisfaction is

highly

correlated with

efficiency. Pg

656 para 4

also serve as barriers to

physician technology

utilisation. Pg 651 para 3

Costs, lack of standards

as…major barriers to IT

adoption. Pg 660 para 3

Computer experience

positive predictors of

perceived usefulness.

Pg 660 para 2

Social and personal

norms appear to predict

technology acceptance

while physician self

identity factors actually

suppress technology

acceptance. Pg 661

para 3

are hesitant to

give it up or

take time away

from the

experience due

to new

information

technologies.

Pg 667 para 3

Lack of time

as…major

barriers to IT

adoption. Pg

660 para 3 /pg

664 summary

Organisational

characteristics

significantly

influence

technology

acceptance pg

660 para 3.

For the

implementation

s of CPOE or

EMRs to be

successful, a

physician’s

practice

environment

must have a

collaborative

organisational

culture that



linking info

systems to

quality or

financial

improvements

.

Physician

perceptions of

the utility of

new

technology

must be

measured pg

668 para 2.

emphasises

teamwork. Pg

664 summary



Conditions Prior

To

implementation

Cost Need and

importance of

validation and

evaluation


of System


y and standards

Education and

Training

Technological

Issues

Communication

Issues

Organisational

Issues

Other

19 Yusuf Installation

strategy pg 264

table 1

Resources

pg 264

table 1

User

perception pg

264 table 1

User attitudes

pg 264 table 1

Leadership

and support

pg 264 table 1

/pg 265 col 1

para 3.

Ease of use

pg 264 table

1/pg264 para

1 col 1

Time

(response and

turnaround)

pg 264 table 1

/pg 264 col 1

para 3

System

Flexibility pg

262/pg 262

col2 para 2

System

Usefulness pg

264 table 1

User training,

roles, skills pg

262 col 2 para

4and5/ pg 264

table 1

Successful

implementations

attributed to

provision of

extensive,

continuous user

training and

support pg 264

para 6

Clarity of system

purpose pg 262

table 1/ pg 262

col 1 para 2

System purpose

pg 264 table 1

Organisational

factors

(leadership

and support,

internal

communicatio

n) pg 264

table 1

User

involvement

pg 264

table 1

Organisational

factors (inter

organisational

system) pg

264 table 1



264 table 1

/pg 264 para 2

col 1

Information

accessibility

and relevancy

pg 262 para 3

Information

quality/comple

teness/legibilit

y/format table

1 pg 264

Problems in

the EMR

implementatio

n indicated a

design that

interfered with

clinical

practices it

was supposed

to support pg

265 col2 para

Technical

support pg

264 table 1



1

Ability of

system to fit

with clinical

practice pg

265 para4/



Appendix 6 WP2 Information letter Task Group

Mair Page 1 06/01/2009 Version 2.

Understanding the Implementation and Integration of E-Health Services; Information Sheet for Task Group Participants Workpackage 2.

Thank you for taking the time to consider participating in this research project which has been funded by the NHS Service Delivery and Organisation R&D Programme. Before you decide it is important for you to understand why the research is being done and what it will involve. Please take time to read the following information carefully. Talk to others about the study if you wish.

This information sheet explains the purpose of the study, what will happen to you if you take part, and gives more detailed information about the conduct of the study.

Ask us if there is anything that is not clear or if you would like more information you can discuss this with our researcher. Take time to decide whether or not you wish to take part.

What is the purpose of the study?

The study examines those factors that act as facilitators and barriers to the use of e-Health systems by health care professionals. E-Health covers a range of types of service. In this project we will be looking at four specific aspects of e-Health that are particularly important to the NHS:

1. Management systems (this includes electronic medical record systems);

2. Communication systems (includes real time or store and forward communication systems such as e-mail, telecare and telemedicine systems)

3. Decision support systems.

4. Information systems (including use of the internet).

In this study we are looking particularly at: uptake of different types of e-Health services; roots of resistance (if any); views about the effects of e-Health services on the patient/provider relationship; and training and support needs.

Why have I been chosen?

We are interested in hearing about the experiences of health care professionals from a range of backgrounds and with different levels of experience with the range of e-Health systems listed above in order to get as clear an understanding as possible of the key issues facing health professionals when new e-Health services are introduced. You have been chosen because you are an active NHS health care professional.

What do I have to do?



If you are interested in taking part or would like to speak to someone about the study you should contact either Prof Mair or Dr Anderson, one of the researchers, working on the project.

If you decide to take part they will then arrange an appointment to discuss study participation, via telephone, at a time that is convenient to you. Study participation will involve attending a single focus group session with up to seven other health professionals.

The Focus Group will last approximately two hours and will take place at a local venue (travel expenses will be paid). Prior to attending the focus group you will be sent summary information relating to a literature review that has been recently undertaken about barrier/facilitators to e-Health service utilisation. During the focus group you will be asked to comment on the information provided and to provide your views regarding:

� ̈ Barriers to using e-Health systems in practice.

� ̈ Ways of helping health professionals to integrate use of e-Health systems within their routine work.

� ̈ Types of support necessary to help with successful introduction of e-Health services.

� ̈ Potential effects, if any, of the different types of e-Health systems (electronic records, telecare, decision support, and the internet) on the health professional/patient relationship.

When you arrive for the focus group, the researcher will ask you to fill in a consent form. This is to show that you understand what the study is about, that you are happy to participate, and that you are happy for us to record the focus group.

We will give you a copy of this information sheet and the consent form to keep.

What will happen during the focus group?

The focus group will consist of questions and discussion relating to the use of different types of e-Health services by health professionals as outlined above. The focus group will last about 2 hours and will be tape recorded.

Why is the focus group recorded?

We record the focus groups because it is hard to take notes of what people say, listen carefully and think all at the same time! After the focus group the recording will be listened to carefully and every word that participants and the researcher say will be typed down. We use this written record, to help us remember what people said. Once the study has concluded, all the original audio recordings will be destroyed.

Are there any benefits to helping with this work?

Although there may be no direct benefits to you personally, we hope that you find participation an interesting experience. The subject matter is relevant to your work as across the United Kingdom the NHS is committed to the adoption of new e-Health Services such as the use of electronic medical records and home telecare technologies. Your involvement will give us a better of understanding of the problems faced by health professionals when using new e-Health services and how difficulties, if any, can best be overcome. We believe this will be of use to health care professionals and



the wider NHS, providing a better understanding of the needs of health professionals whenever new e-Health services are being introduced and how best to support the introduction of such services.

Are there any disadvantages to helping with this research?

The main disadvantage is the time it will take; the focus group usually lasts about two hours and there will be some documentation to read prior to the meeting. We are however able to offer remuneration for the time of the interview.

Will my taking part in the study be kept confidential?

Yes. Everything you tell us during the focus group is completely confidential. All personal information – your name, names of colleagues or patients - or anything else which might identify you will be removed so that no-one can identify you personally. Only anonymised data from the focus groups will be used in future publications from the research team so that individual participants cannot be identified. All recordings and written records are treated as confidential material. They are stored securely within the University under the supervision of Prof Frances Mair and will not be played or shown to anyone outside the research team. .

Who is organising and funding the research?

The research if funded by the NHS Service Delivery and Organisation R&D Programme and is being organised by the University of Glasgow (Division of Community Based Sciences).

Who has reviewed the study?

This study was given a favourable review by the NHS Service Delivery and Organisation R&D Programme and has had full ethical approval.

How can I get more information about the study?

If you want to know about what the focus group will involve or about the study in general or you have any concerns about any aspect of the study, please get in touch with us. Our telephone number is: 0141 330 8317, please ask for Professor Frances Mair or Dr George Anderson. Our address is: Department of General Practice & Primary Care, University of Glasgow, 1 Horselethill Road, Glasgow G12 9LX

WP1 Invitation letter Task Group

General Practice and Primary Care Division of Community Based Sciences University of Glasgow 1 Horselethill Road Glasgow G12 9LX

Dear ,

Re: Research participation

We are writing to you to ask for your help with a research study that is being funded by the NHS Service Delivery and Organisation R&D Programme, and conducted by the Universities of Glasgow, Newcastle upon Tyne, Dundee and University College of London.



The study is about NHS staff members’ experiences of using e-Health services in their every day work. e-Health covers a range of types of service, but for example, includes electronically facilitated systems for management (eg. electronic medical record systems), communication (eg. email/telecare/telemedicine), decision support (i.e. computerised programmes to guide decision-making) and other information systems, such as use of the internet and database systems. We are interested in gaining a better understanding of how NHS staff use such systems in their every day work, including perceived barriers to, and facilitators of effective use of e-Health systems for health care provision. We hope that the results of our research will be very important for guiding NHS managers and policy makers in ways of introducing e-Health services into the NHS so that it is easier for staff to use them as part of their every day working practice.

We are therefore inviting you to take part in this study by agreeing to take part in a task group (with between 4 and 7 other health professionals) to discuss your views and experience, if any of using e-Health technologies in your work. A detailed information sheet is provided so that you can gain a better understanding of what your participation in this study would involve, and thus enable you to make an informed decision about whether or not you would like to take part. If, however, you need further information to make a decision, or simply wish to speak to someone about this research, contact details are provided and we are happy to answer your queries.

We sincerely hope that you will be able to help with this research.

Yours faithfully,

Dr George Anderson

On behalf of Professor Frances Mair.



Appendix 7 WP2 interview Invitation letter

General Practice and Primary Care

Division of Community Based Sciences

University of Glasgow

1 Horselethill Road

Glasgow

G12 9LX

Dear ,

Re: Research participation

We are writing to you to ask for your help with a research study that is

being funded by the NHS Service Delivery and Organisation R&D

Programme, and conducted by the Universities of Glasgow, Newcastle upon

Tyne, Dundee and University College of London.

The study is about NHS staff members’ experiences of using e-Health

services in their every day work. e-Health covers a range of types of service,

but for example, includes electronically facilitated systems for management

(eg. electronic medical record systems), communication (eg.

email/telecare/telemedicine), decision support (i.e. computerised

programmes to guide decision-making) and other information systems, such

as use of the internet and database systems. We are interested in gaining a

better understanding of how NHS staff use such systems in their every day

work, including perceived barriers to, and facilitators of effective use of e-

Health systems for health care provision. We hope that the results of our

research will be very important for guiding NHS managers and policy

makers in ways of introducing e-Health services into the NHS so that it is

easier for staff to use them as part of their every day working practice.

We are therefore inviting you to take part in this study by agreeing to take

part in an interview to discuss your views and experiences of using or

implementing e-Health services. A detailed information sheet (Version 1,

dated 30/07/07) is provided so that you can gain a better understanding of

what your participation in this study would involve, and thus enable you to

make an informed decision about whether or not you would like to take

part. If, however, you need further information to make a decision, or

simply wish to speak to someone about this research, contact details are

provided and we are happy to answer your queries.

We sincerely hope that you will be able to help with this research.

Yours faithfully,



Professor Frances Mair

interview gp info

Mair Page 180 21/01/2009

Understanding the Implementation and Integration of E-Health Services;

Information Sheet for Interview Participants Workpackage 2.

Thank you for taking the time to consider participating in this research

project which has been funded by the NHS Service Delivery and

Organisation R&D Programme. Before you decide it is important for you to

understand why the research is being done and what it will involve. Please

take time to read the following information carefully. Talk to others about

the study if you wish.

This information sheet explains the purpose of the study, what will happen

to you if you take part, and gives more detailed information about the

conduct of the study.

Ask us if there is anything that is not clear or if you would like more

information you can discuss this with our researcher. Take time to decide

whether or not you wish to take part.

What is the purpose of the study?

The study examines those factors that act as facilitators and barriers to the

use of e-Health systems by health care professionals. E-Health covers a

range of types of service. In this project we will be looking at four specific

aspects of e-Health that are particularly important to the NHS:

Management systems (this includes electronic medical record systems);

Communication systems (includes real time or store and forward

communication systems such as e-mail, telecare and telemedicine systems)

Decision support systems.

Information systems (including use of the internet).

In this study we are looking particularly at: uptake of different types of e-

Health services; roots of resistance (if any); views about the effects of e-

Health services on the patient/provider relationship; and training and

support needs.

Why have I been chosen?

We are interested in hearing about the views of health care personnel who

have expertise or particular experience of some aspect of e-Health services

as described above who are well placed to act as key informants. You have



been chosen because of your leadership role and/or experience/expertise in

the use of e-Health services.

What do I have to do?

If you are interested in taking part or would like to speak to someone about

the study you should complete the ‘consent to contact’ form included with

this information sheet and return it with your completed questionnaire.

Within a few weeks you will be contacted by Prof Mair or Dr Anderson, one

of the researchers, working on the project.

If you decide to take part they will then arrange an appointment to discuss

study participation, via telephone, at a time that is convenient to you and at

that point a consent form will be mailed to you. This is to show that you

understand what the study is about, that you are happy to participate, and

that you are happy for us to record the interview.

We will give you a copy of this information sheet and the consent form to

keep.. Study participation will involve participating in a single telephone

interview. The interview will last approximately 45 minutes and will take

place via telephone or the researcher can come to your place of work if that

is preferable. Prior to the interview you will be will be sent summary

information relating to a literature review and five task groups that have

been undertaken with health care professionals about barrier/facilitators to

e-Health service utilisation. During the interview you will be asked to

comment on the literature review and the task group findings and provide

your views regarding:

Barriers to using e-Health systems in practice.

Ways of helping health professionals to integrate use of e-Health systems

within their routine work.

Types of support necessary to help with successful introduction of e-Health

services.

Potential effects, if any, of the different types of e-Health systems

(electronic records, telecare, decision support, and the internet) on the

health professional/patient relationship.

When you arrive for the focus group, the researcher will ask you to fill in a

consent form.

What will happen during the interview?



The researcher will ask a number of fairly open questions regarding the

findings of the research to date. The aim is to get your views on the study

findings relating to the use of different types of e-Health services by health

professionals as outlined above. The interview will last about 45 minutes

and will be tape recorded.

Why is the interview recorded?

We record the interviews because it is hard to take notes of what people

say, listen carefully and think all at the same time! After the interview the

recording will be listened to carefully and every word that participants and

the researcher say will be typed down. We use this written record, to help

us remember what people said. Once the study has concluded, all the

original audio recordings will be destroyed.

Are there any benefits to helping with this work?

Although there may be no direct benefits to you personally, we hope that

you find participation an interesting experience. The subject matter is

relevant to your work as across the United Kingdom the NHS is committed

to the adoption of new e-Health Services such as the use of electronic

medical records and home telecare technologies. Your involvement will give

us a better of understanding of the problems faced by health professionals

when using new e-Health services and how difficulties, if any, can best be

overcome. We believe this will be of use to health care professionals and

the wider NHS, providing a better understanding of the needs of health

professionals whenever new e-Health services are being introduced and how

best to support the introduction of such services.

Are there any disadvantages to helping with this research?

The main disadvantage is the time it will take; the interview usually lasts

about 45 minutes and there will be some documentation to read prior to the

interview.

Will my taking part in the study be kept confidential?

Yes. Everything you tell us during the interview is completely confidential.

All personal information – your name, names of colleagues or patients - or

anything else which might identify you will be removed so that no-one can

identify you personally. Only anonymised data from the interviews will be

used in future publications from the research team so that individual

participants cannot be identified. All recordings and written records are

treated as confidential material. They are stored securely within the

University under the supervision of Prof Frances Mair and will not be played

or shown to anyone outside the research team. .

Who is organising and funding the research?



The research if funded by the NHS Service Delivery and Organisation R&D

Programme and is being organised by the University of Glasgow (Division of

Community Based Sciences).

Who has reviewed the study?

This study was given a favourable review by the NHS Service Delivery and

Organisation R&D Programme and has had full ethical approval.

How can I get more information about the study?

If you want to know about what the interview will involve or about the study

in general or you have any concerns about any aspect of the study, please

get in touch with us. Our telephone number is: 0141 330 8317, please ask

for Professor Frances Mair or Dr George Anderson. Our address is:

Department of General Practice & Primary Care, University of Glasgow, 1

Horselethill Road, Glasgow G12 9LX



Appendix 8 WP2 Summary of Key Findings

A Summary of Our Key Findings.

The main factors to consider, and how they may impede or facilitate the implementation of e-Health systems, as identified in our review are as follows:

Note that these are not shown in a hierarchy of importance.

Conditions prior to implementation.

The literature shows that conditions prior to implementation are regarded as integral to the selection of a site for an e-Health intervention. There are essentially two levels on which the conditions prior to the implementation of an e-Health system will influence its success or failure: these are the conditions within the organisation itself (be it a health trust or a GP surgery), and the broad societal conditions (how strong is the support of national government in the general implementation of e-Health systems

Conditions prior to implementation.

Cost. Education and training.

The need for and impact of validation and evaluation. Security,

confidentiality and standards.


Ease of use of the system.



etc). Thus, the impact of pre-existing conditions, be they internal or external to the setting, cannot be under-estimated.

Cost.

The cost of a system is an inevitable factor to consider in the implementation of an e-Health system. It can be assumed that the term “cost” includes both the financial cost of initial hardware, software and its implementation and cost in terms of time and human resources, though the literature itself does little to differentiate between these types. In terms of telecommunications, there is also the cost of regular usage to consider as well as that of initial implementation, since network tariffs may apply; in terms of other systems there will also be costs associated with procurement and maintenance.

The need for and impact of validation and evaluation.

Validation and evaluation of e-Health systems are important factors in their widespread implementation. Without strong data demonstrating that a system works, improves standards of care, can be used efficiently and easily, and is cost-effective to implement, it is unlikely to win the confidence of policy makers and users. Indeed, lack of validation and evaluation is frequently presented as a barrier to system implementation in the literature, while continued monitoring and evaluation is a facilitator.

. Professional attitudes.

Professional attitudes to the implementation and integration of e-Health systems contain within them a broad array of barriers and facilitators to success. These attitudes may relate to perceptions of the changing relationship between professionals and patients and perceptions as to the effectiveness of the technology. It may be that there exists a general anxiety towards the use of technology, or even that clinicians feel that their roles are undermined by it and that the introduction of new systems implies shortcomings in their own performance. The latter can relate how the e-Health system affects the allocation of tasks and how an organisational structure may change once technology is adopted. It can also relate to unease regarding patient safety, particularly in terms of how professionals can manage a critical situation at a distance.


The ease of use of an e-Health system, the interaction between the technology and its users, is self-evidently a factor in successful implementation. The system needs to be appropriate to the work and not imposed for its own sake, be compatible with existing systems, and reliable.

Related to the general ease of use of a system is its ability to order and manage information. If the technology is disruptive to established working practices and increases the amount of time and effort required to record, communicate, and find information and indeed arrive at a clinical decision,



its successful implementation is unlikely. A management or information system may contain insufficient, misleading, or too much information.

Security, confidentiality, and standards.

For successful implementation, a system needs to be not only workable, but also secure. It needs to be safe for both patients and professionals, for patients, in terms of the security of their personal data, and for professionals in terms of not compromising their standards of practice.

Education and training.

The success or failure of a new system will depend on the capabilities of those using it; and new technologies, though they may strive to ultimately simplify the work, must be learnt in the first instance. Lack of skills and knowledge is frequently cited as a barrier to success. Education can be as simple as the communication of information between professionals and organisations for the purpose of influencing system implementation and usage to more formalised practices, such as officially sanctioned system-specific training courses.



Appendix 9 WP2 Task Group Schedule

Initial information gathering:

(Each group member in turn) Who are you and what is your role (this is also for the transcriber)?

What e-Health systems have you used or had experience of?

-What type of system (link to 4 domains)? (2 minutes per participant.)

Introduction:

The main factors to consider, and how they may impede or facilitate the implementation of e-Health systems, as identified by our review are as follows:


Central Question(s):

To Start off:

Thinking of a specific system in which you have had involvement, which of these factors, or which factor, would you consider the most important, if any?

(Or which would you most like to talk to about? Which seems most relevant to your experience and the type of system you have been involved with?)

Conditions prior to

implementation.



Security, confidentiality and

standards.





Following on from this, respondents will be asked about each factor specifically; whether they have personal experience of each factor (can they report on real-life instances of such?) and whether they agree with the findings of the review that the factor is an important barrier/facilitator. (1 hour +)

Respondents will be asked for two things: firstly, if they deem anything important to be missing from our findings; secondly, if they deem any of our findings insignificant. (Last half hour.)



Appendix 10 WP2 Interview Schedule

Initial information gathering:

What is your role?

What e-Health systems have you implemented or had experience of?

-What type of system (link to 4 domains)? (5 minutes.)

Introduction:

The main factors to consider, and how they may impede or facilitate the implementation of e-Health systems, as identified by our review are as follows:


Central Question(s):

To Start off:

Thinking of a specific system in which you have had involvement, which of these factors, or which factor, would you consider the most important, if any?

(Or which would you most like to talk to about? Which seems most relevant to your experience and the type of system you have been involved with?)

Conditions prior to

implementation.



Security, confidentiality and

standards.





Following on from this, respondents will be asked about each factor specifically; whether they have knowledge or personal experience of each factor (can they report on real-life instances of such?) and whether they agree with the findings of the review that the factor is an important barrier/facilitator.

Respondents will be asked for two things: firstly, if they deem anything important to be missing from our findings; secondly, if they deem any of our findings insignificant. (25 minutes.)

Finally, respondents will be asked to describe how to integrate a system

successfully, and whether the same factors are of equal importance across



Appendix 11 WP3 Sample characteristics of expert survey participants

Sample characteristics of expert survey participants

Location of Residence %

USA 37

UK 27

Canada 13

Europe (excluding Scandinavia) 10

Australia/New Zealand 8

Scandinavia 6

Research background

Medical 32

Social science 24

Informatics 21

Nursing 11

Economics 2

Health Services Research 5

Non-specific 6

Sex

Male 59

Female 41

E-health domain Mostly (%) Partly (%) Not at all (%)

Management Systems 29 46 25

Communication Systems 44 32 24

Computerised decision support systems 14 38 48

Web based Information Resources 22 29 49



279

Appendix 12 WP3 Expert sample: Factor rating scores: frequencies, means and standard deviation

Expert sample: Factor rating scores: frequencies, means and standard deviation

Score* frequencies % Q.

N Mean score

(SD) 0 1 2 3 4 dk

Q3 Impact of the system on existing ways of working (CI) 63 3.49 (.759) 0 1.6 11.1 23.8 63.5 0

Q27 Ease of using the system (IW) 63 3.44 (.616) 0 0 6.3 42.9 50.8 0

Q2 Allocation of organizational effort to the system (CI) 62 3.32 (.845) 0 4.8 9.5 33.3 50.8 1.6

Q8 Additional workload created by the system (SW) 61 3.20 (.771) 0 1.6 15.9 41.3 38.1 3.2

Q4 Balance of effort against rewards of using the system (CI) 63 3.08 (.848) 0 4.8 17.5 42.9 34.9 0

Q23 Perceived impact of the system on ways of working with patients (IW) 62 2.90 (.970) 1.6 4.8 27.0 33.3 31.7 1.6

Q1 Allocation of financial resources to the system (CI) 61 2.90 (.87) 0 3.2 31.7 33.3 28.6 3.2

Q6 level of co-operation required from others within the organisation, in using the

system (SW)

63 2.89 (.918) 0 9.5 19 44.4 27.0 0

Q26 Perceived impact of the system on the amount of time spent with patients

(IW)

63 2.87 (1.008) 1.6 9.5 19 39.7 30.2 0

Q19 Availability of technical expertise in using the system (RI) 63 2.84 (1.003) 0 11.1 25.4 31.7 31.7 0



280

Q11 Obtainability of new skills required to use the system (SW) 63 2.79 (.986) 0 14.3 17.5 42.9 25.4 0

Q22 How flexibly the system can be used for conducting work (IW) 62 2.79 (.908) 1.6 7.9 19.0 50.8 19.0 1.6

Q24 Perceived impact of the system on outcomes for patients (IW) 60 2.72 (1.075) 3.2 7.9 28.6 28.6 27.0 4.8

Q15 Individuals’ perceptions of the efficiency of using the system (RI) 63 2.67 (.916) 0 12.7 25.4 44.4 17.5 0

Q10 Compatibility of the system with existing skills (SW) 63 2.65 (.786) 0 4.8 39.7 41.3 14.3 0

Q5 Impact of the system on individual’s perceptions of autonomy in their work

(SW)

63 2.62 (.991) 0 14.3 31.7 31.7 22.2 0

Q25 Perceived impact of the system on communication with patients (IW) 62 2.52 (1.083) 3.2 19.0 15.9 44.4 15.9 1.6

Q9 Impact of the system on allocation of work between individuals (SW) 60 2.47 (.965) 3.2 9.5 34.9 34.9 12.7 4.8

Q13 Individuals’ own confidence in the safety of using the system (RI) 62 2.37 (.945) 1.6 14.3 41.3 28.6 12.7 1.6

Q21 Availability of users’ knowledge of the clinical effectiveness of the system

(RI)

61 2.34 (1.109) 4.8 19.0 25.4 33.3 14.3 3.2

Q16 Impact of the system on the distribution of responsibilities between

individuals (RI)

59 2.34 (.958) 3.2 12.7 36.5 31.7 9.5 6.3

Q20 Availability of an evidence base about the clinical effectiveness of the system

(RI)

62 2.31 (1.249) 9.5 17.5 23.8 28.6 19.0 1.6

Q7 level of co-operation required from others outside the organisation, in using

the system (SW)

62 2.27 (.890) 0 20.6 38.1 31.7 7.9 1.6

Q17 Impact of the system on individuals’ beliefs about their accountability for their

work (RI)

62 2.26 (1.023) 6.3 12.7 38.1 31.7 9.5 1.6

Q12 Impact of the system on individuals’ perceptions of personal liability (RI) 59 2.25 (.993) 0 25.4 30.2 27 11.1 6.3



281

Q18 Impact of the system on individuals’ beliefs about others’ expectations of

their accountability for their work (RI)

60 2.08 (.962) 6.3 15.9 41.3 27 4.8 4.8

Q14 Individuals’ confidence in the safety of others’ use of the system (RI) 60 2.02 (1.033) 6.3 22.2 38.1 20.6 7.9 4.8

*Scale: 0=not at all important; 1=some importance; 2=moderate importance; 3=very important; 4=extremely important



282

Appendix 13 WP3 Phase 2: QUESTION ANALYSIS BY NPM CONSTRUCT

Phase 2: QUESTION ANALYSIS BY NPM CONSTRUCT

INTERACTIONAL WORKABILITY SKILL-SET WORKABILITY

Existing questions:

1. How flexibly the system can be used for conducting work

2. Perceived impact of the system on ways of working with

patients

3. Perceived impact of the system on outcomes for patients

4. Perceived impact of the system on the amount of time

spent with patients

5. Ease of using the system

‘Additional’ factors mentioned but covered:

possibility to modify technical system (1)

Flexibility of use of system/open access interface (x2) (q.1)

Preferences regarding patient interaction (2)

access to (fast) internet in work setting (5)

User friendliness (5)

Requirement to use multiple tools for one task (5)

Perceived convenience/inconvenience (5)

Existing questions:

1. Impact of the system on individual’s perceptions of autonomy in

their work

2. Level of co-operation required by others in using the system

3. Additional workload created by the system

4. Impact of the system on allocation of work between individuals

5. Compatibility of the system with existing skills

6. Obtainability of new skills required to use the system


Loss of autonomy of practice (1)

Integration of IT in clinical workflow (x2) (3)

user abilities (re educational patient materials; ability to rate info

quality) (5)



283

Mentioned and not covered:

technical performance

RELATIONAL INTEGRATION CONTEXTUAL INTEGRATION

Existing questions:

1. Individuals’ own confidence in the safety of using the

system

2. Individuals’ perceptions of the efficiency of using the

system

3. Impact of the system on the distribution of responsibilities

between individuals

4. Impact of the system on individuals’ beliefs about their

accountability for their work

5. Availability of technical expertise in using the system

6. Availability of evidence about the clinical effectiveness of

the system


Perceived effect on patients’ responsibilities/roles (3)

operation of a back-up system (q.1?)


Quality/evidence base of information within the system (3

mentions)

Use is expected as normal practice (maybe q.4?)

Existing questions:

1. Allocation of financial resources to the system

2. Allocation of organizational effort to the system

3. Impact of the system on existing ways of working

4. Balance of effort against rewards of using the system


Incentives for use (1)

Balance of costs/benefits across different stakeholders (4)

Immediate perception of benefit (4)


Characteristics of the organization itself (stability; commitment &

champions; leadership; complexity; readiness/fear of change)

Wider contexts outside the organization (policy – broad and e-

Health specific; legal context; compatibility with other

systems/agencies)

Organizational process (Adequate feasibility testing in practice;

commissioner autonomy; targets for use; ongoing technical

support; ‘best practice’ relating to patient-provider email

communications; multi-professional involvement in planning &

development (4 mentions; e-Health embedded in

CPD/education; responsibility for data management)



284

Alignment of system with philosophy of care /health system

priorities and challenges (2 comments)



285

Appendix 14 WP3 Analysis of rating items for Phase 2

Analysis of rating items for Phase 2

Considerations Decision Final Item

Q1

Allocation of financial resources to the system Ranked in top half of table. Correlates with q.2 (0.527) and

q. 18 (0.531)

retain Allocation of financial

resources to the system

Q2

Allocation of organizational effort to the system Third highest mean rating score. Correlates with q.1

(0.527).

retain Allocation of organizational

effort to the system

Q3

Impact of the system on existing ways of working

Ranked no 1 in importance. No r’s > 0.5.

retain Impact of the system on

existing ways of working

Q4

Balance of effort against rewards of using the

system Ranked 5th. Doesn’t correlate well with any other item

retain Balance of effort against

rewards of using the system

Q5

Impact of the system on individual’s perceptions of

autonomy in their work Mid-table in importance ratings. Correlates with q.9 (r

0.573).


individual’s perceptions of

autonomy in their work

Q6

level of co-operation required from others within the

organisation, in using the system Ranked 8th. Correlates with q.7 (0.560).

Q7

level of co-operation required from others outside

the organisation, in using the system

Correlates with q.6 (0.560), but most correlations near

zero. (ranked 5th from bottom)

Combine 6 and 7

Level of co-operation required

by others in using the system

Q8

Additional workload created by the system Ranked 4th in importance. No r’s above 0.5, but

approaching that on q. 26 and 27.

retain Additional workload created

by the system



286

Q9

Impact of the system on allocation of work between

individuals

Correlates with q.5 (r 0.573).


allocation of work between

individuals

Q10

Compatibility of the system with existing skills

Ranked mid-table. Correlates with q.11 (0.519)

retain

Compatibility of the system

with existing skills

Q11

Obtainability of new skills required to use the

system

Ranked 11th. Correlates with q.10 (0.519). Several

significant (but low) correlations with other items.

retain Obtainability of new skills

required to use the system

Q12

Impact of the system on individuals’ perceptions of

personal liability

Ranked 3rd from bottom. Correlates with q. 17 (r .564) &

18 (r .569). Correlations < but approaching 0.5 for q. 13 &

14.

exclude

Q13

Individuals’ own confidence in the safety of using

the system

Ranked mid-table. High r (0.725) with q. 14. Correlates

with q. 18 (0.565). Approaches 0.5 with q.12.

Q14

Individuals’ confidence in the safety of others’ use of

the system

Ranked least important. High r (0.725) with q. 13, and

correlates with q.18 (0.531). Approaches 0.5 with q.12.

Combine 13 and

14

Individuals’ own confidence in

the safety of using the system

Q15

Individuals’ perceptions of the efficiency of using the

system Ranked mid-table. No correlations > 0.5.

Retain Individuals’ perceptions of the

efficiency of using the system

Q16

Impact of the system on the distribution of

responsibilities between individuals

Ranked in bottom half. No correlations > 0.5.

Retain Impact of the system on the

distribution of responsibilities

between individuals

Q17

Impact of the system on individuals’ beliefs about

their accountability for their work Ranked near bottom. High r with Q.18 (0.806). Correlates

with q. 12 (r .564)


individuals’ beliefs about their




287

Q18

Impact of the system on individuals’ beliefs about

others’ expectations of their accountability for

their work

Ranked second bottom. High r with Q.17 (0.806).

Correlates with q. 12 (r .569), 13 (0.565) and q.14

(0.531).

Exclude question

Q19

Availability of technical expertise in using the

system

Ranked in top half. Correlates with q.21 (0.557) & 25

(0.581).

retain Availability of technical

expertise in using the system

Q20

Availability of an evidence base about the clinical

effectiveness of the system

Ranked in bottom half. High r with Q.21 (0.721). Also r

0.619 with Q.24.

Q21

Availability of users’ knowledge of the clinical

effectiveness of the system Ranked in bottom half. High r with Q.20 (0.721). Correlates

with q.19 (0.557), q. 24 (0.517) & q.25 (0.514).

Combine 20 and

21

Availability of evidence about

the clinical effectiveness of the

system

Q22

How flexibly the system can be used for conducting

work Ranked in top half. Correlates with q.23 (0.533).

retain How flexibly the system can

be used for conducting work

Q23

Perceived impact of the system on ways of

working with patients Ranked 6th in importance. Correlates with Q.22 (0.533). &

q.25 (0.586).

Retain Perceived impact of the

system on ways of working

with patients

Q24

Perceived impact of the system on outcomes for

patients Ranked mid-table. Correlates with Q.20 (0.619) & q.21

(0.517).

retain Perceived impact of the

system on outcomes for

patients

Q25

Perceived impact of the system on communication

with patients

Ranked mid-table. Correlates with q.19 (0.581), q. 21

(0.514) & q.23 (0.586)

Exclude (covered

in q 23)

Q26

Perceived impact of the system on the amount of

time spent with patients

Ranked in top half. Approaches 0.50 with q.8. & q.25.

retain Perceived impact of the

system on the amount of time

spent with patients

Q27 Ease of using the system Ranked second highest in importance. Doesn’t correlate retain Ease of using the system



288

>.05 with any item.



289

Appendix 15 WP3 Final set of TARS items mapped against WP2 data themes

WP3 Final set of TARS items mapped against WP2 data themes

Tars Generic Tars specific WP2 Themes

1. Allocation of financial resources to the system 1. The e-Health system is adequately resourced

financially

TG7: Cost (KI:11 Cost)

2. Allocation of organizational effort to the system 2. Sufficient organizational effort has gone into

supporting the e-Health system

TG9: Organisational drive and

leadership

3. Impact of the system on existing ways of

working

3 The e-Health system is a different way of

working

(KI10 Broader societal/cultural change?)

4. Balance of effort against rewards of using the

system

4. The rewards of using the e-Health system

outweighs the effort

5. The supportiveness or otherwise of the broader

policy context in relation to e-Health

5. Government policy initiatives are supportive of

this e-Health system

KI10 Broader societal/cultural change

6. The technical and organisational compatibility

of the e-Health system with other systems and

agencies that it must relate to for the conduct

of work

6. This e-Health system is technically and

organisationally compatible with other systems

and agencies that we are required to work with

TG3: Collaborative working (KI8:

collaborative/interagency working)

KI6: Relationship with suppliers &

designers

7. How well the e-Health system fits in with

priorities and challenges of the organisation

7. This e-Health system fits in with the priorities

and challenges of our organisation

TG2: Clear purpose

KI5: Clear rationale



290

8. An organisational culture that is supportive of

change

8. This organisation has a culture that is

supportive of change

TG9: Organisational drive and

leadership

9. An organisational culture of involving staff in

planning and development

9. There is a culture in this organisation of

involving staff in planning and development

TG5: Sense of user empowerment

10. The level of autonomy that the organisation has

in terms of commissioning services

[Item omitted due to irrelevance to study sites]

11. Impact of the system on individual’s

perceptions of autonomy in their work

10. Using the e-Health system makes me feel

autonomous in my work

12. Level of co-operation required by others in

using the system

11. Using the e-Health system requires co-

operation with other staff

TG3: Collaborative working (KI8)

13. Additional workload created by the system 12. The workload involved in using the e-Health

system is manageable

TG6: Ease of use (KI3)

14. Impact of the system on allocation of work

between individuals

13. In using the e-Health system, the allocation of

work between individuals is appropriate

TG3: Collaborative working (?)

15. Compatibility of the system with existing skills 14. The skills I have are appropriate for using the e-

Health system

TG6: Ease of use

KI3: Ease of use

16. Learnability of new skills required to use the

system

15. The skills needed to use the e-Health system

are easily learned

TG6: Ease of use (simplicity & ease to

learn) (K3)

17. Individuals’ own confidence that using the

system does not put patients at risk

16. I have confidence that using the e-Health

system does not put patients at risk

TG4: Standards (clinical risk)

TG1: Patient benefits

KI2: Standards



291

18. Individuals’ perceptions of the efficiency of

using the system

17. Using the e-Health system is an efficient use of

time

TG6: Ease of use (efficiency)

19. Impact of the system on the distribution of


18. In using the e-Health system, responsibilities

are divided between individuals appropriately

?

20. Impact of the system on individuals’ beliefs

about their accountability for their work

19. In using the e-Health system, I understand my

accountability for my work

TG4: Standards (?)

21. Impact of the system on individuals’

perceptions of liability

20. In using the e-Health system, I understand my

liability for my practice

TG4: Standards (legality)

22. Availability of technical back-up in using the

system

21. Technical back-up in using the e-Health system

is available if I need it

TG6: Ease of use (tech support)

K9: Technical support

KI6: Relationship with suppliers &

designers

23. Availability of evidence about the clinical

effectiveness of the system

22. I believe there is good evidence about the

clinical effectiveness of using the e-Health

system

TG4: Standards

KI7: Generating new knowledge

KI5: Clear rationale (evidence)

24. How flexibly the system can be used 23. There is some flexibility in how the e-Health

system can be used

TG6: Ease of use

KI3: Ease of use

25. Perceived impact of the system on outcomes

for patients

24. Using the e-Health system leads to positive

outcomes for patients


26. Perceived impact of the system on the amount

of time spent with patients

25. Using the e-Health system involves the right

amount of time spent with patients




292

27. Perceived impact of the system on the quality

of the interaction between professionals and

patients

26. In using the e-Health system, the quality of

professional and patient interaction is good


28. Ease of using the system 27. The e-Health system is easy to use TG6: Ease of use; KI3: Ease of use

29. The existence of a shared understanding of

what the system is for and how it is to be used

28. The staff who work here have a shared

understanding of what the system is for and

how it is to be used

TG2: Clear purpose

TG8 & KI1: Professional attitudes (unity

of purpose; perceived usefulness)

30. The commitment of individuals to making the

system work

29. The staff here are committed to making the

system work

TG8: Professional attitudes (willingness)

31. The existence of ongoing mechanisms for

monitoring and appraising how the system is

used

30. There are ongoing mechanisms for monitoring

and appraising how this e-Health system is used

TG4: Standards

TG6 Ease of Use

KI7: Generating new knowledge

32. The balance between the needs of individual

users, other users of the system, and the

organization itself [suggested addition by site

contact]



Appendix 16 WP3 TARS GENERIC



300

Appendix 17 WP3 Tars Generic: Means, standard deviations and frequencies

WP3 Tars Generic: Means, standard deviations and frequencies

Frequencies (% are approx double)

N* Means Sds

1

2

3

4

5

8 DK

Total

ans

3 Impact of the system on existing ways of working 49 4.57 .677 5 11 33 1 50

9 An organizational culture of involving staff in planning

and development

47 4.57 .677

2 14 31 2 49

15 Compatibility of the system with existing skills 46 4.57 .677 1 11 13 21 3 49

21 Impact of the system on individuals' perceptions of

liability

45 4.57 .677

1 3 4 14 23 3 48

27 Perceived impact of the system on the quality of the

interaction between professionals and patients

45 4.57 .677

6 15 24 3 48

6 The integration of the e-Health system with other

systems and agencies that it must relate to for the

conduct of work

47

4.53 .620

3 16 28 4 51

12 Level of co-operation required by others in using the

system

47 4.53 .620

1 5 15 26 3 50



301

18 Individuals' perceptions of the efficiency of using the

system

48 4.53 .620

1 5 19 23 2 50

24 How flexibly the system can be used 46 4.53 .620 3 13 30 2 48

30 The commitment of individuals to making the system

work

46 4.53 .620

5 17 24 2 48

2 Allocation of organizational effort to the system 48 4.50 .684 5 14 29 3 51

8 An organizational culture that is supportive of change 48 4.50 .684 3 12 33 2 50

14 Impact of the system on allocation of work between

individuals

44 4.50 .684

1 2 9 14 18 6 50

20 Impact of the system on individuals' beliefs about their


47 4.50 .684

2 5 16 24 2 49

26 Perceived impact of the system on the amount of time

spent with patients

43 4.50 .684

1 5 16 21 5 48

32 The balance between the needs of individual users,

other users of the system, and the organisation itself

47 4.50 .684

9 14 24 1 48

4 Balance of effort against rewards of using the system 49 4.37 .755 8 15 26 2 51

10 The level of autonomy that the organization has in terms

of commissioning services

45 4.37 .755

8 19 18 5 50

16 Learnabiliity of new skills required to use the system 48 4.37 .755 1 4 17 26 2 50

22 Availability of technical support in using the system 47 4.37 .755 3 11 33 1 48

28 Ease of using the system 47 4.37 .755 1 1 14 31 1 48



302

5 The supportiveness or otherwise of the broader context

in relation to e-Health

39 4.18 .885

1 9 11 18 11 50

11 Impact of the system on individual's perceptions of

autonomy in their work

44 4.18 .885

1 7 15 21 6 50

17 Individual's own confidence that using the system does

not put patients at risk

47 4.18 .885

1 3 7 36 2 49

23 Availability of evidence about the clinical effectiveness of

the system

47 4.18 .885

1 8 10 28 1 48

29 The existence of a shared understanding of what the

system is for and how it is to be used

45 4.18 .885

7 18 20 2 47

1 Allocation of financial resources to the system 48 4.10 .951 2 7 21 18 3 51

7 How well the e-Health system fits in with priorities and

challenges of the organization

48 4.10 .951

1 5 15 27 3 51

13 Additional workload created by the system 48 4.10 .951 2 2 10 10 24 2 50

19 Impact of the system on the distribution of


45 4.10 .951

2 15 16 12 5 50

25 Perceived impact of the system on outcomes for patients 45 4.10 .951 1 3 15 26 2 47

31 The existence of ongoing mechanisms for monitoring

and appraising how the system is used

45 4.10 .951

1 5 15 24 3 48



Appendix 18 WP3 TARS SPECIFIC (Site 1)



Appendix 19 WP3 TARS SPECIFIC (Site 2)



317

Appendix 20 Sample characteristics for TARS Specific (Site 1 and Site 2)

Site 1

% (n)

Site 2

% (n)

Age groups: Age groups:

<25 0 (0) <25 9 (20)

25-34 4 (2) 25-34 20 (47)

35-44 24 (11) 35-44 32 (73)

45-54 59 (27) 45-54 33 (75)

55+ 13 (6) 55+ 7 (15)

Sex Sex

Male 0 (0) Male 14 (32)

Female 100 (46) Female 86 (199)

Working role: Working role:

Community Enrolled Nurse 0 (0) Call handlers 47 (109)

Community Staff Nurse 28 (13) Nurse advisors 24 (56)

District Nursing Sister/Charge Nurse 61 (28) Team leaders 9 (21)

Practice Development Nurse 9 (4) Health information advisors 3 (7)



318

Senior Nurse 2 (1) Other 16 (38)

Time working in role: Time working in role:

<2 years 7 (3) < 1 year 15 (36)

2 to < 5 years 22 (10) 1 year to 23 months 10 (23)

5 to <10 years 28 (13) 2 years to 47 months 20 (45)

10 years plus 30 (14) 4 years to 71 months 16 (36)

Did not specify 13 (6) 6 years + 16 (37)

Did not specify 23 (54)

Time using e-Health system

(Time using not assessed for Site 2)

no months of use 9 (4)

some but <3 mths 20 (9)

4 or 5 mths 9 (4)

6 mths but <12 20 (9)

1 yr but <2 yrs 22 (10)

2 years + 22 (10)



319

Perceived level of routinisation of e-Health Perceived level of routinisation of e-Health

Not at all 0 (0) Not at all 1 (2)

Partly 68 (30) Partly 17 (35)

Completely 32 (14) Completely 83 (174)



320

Appendix 21 WP3 TARS Specific Site 1: TARS items, means, standard deviations and frequencies

WP3 TARS Specific Site 1: TARS items, means, standard deviations and frequencies

Frequencies (not percentages)

N*

Mea

ns Sds

0

1

2

3

4

5

6

DK

N

19 In using the e-Health system, I understand my

accountability for my work 43 4.70 1.15 0 0 2 5 9 15 12 2 45

20 In using the e-Health system, I understand my liability

for my practice 43 4.65 1.11 0 0 1 6 12 12 12 1 44

21 Technical back-up in using the e-Health system is

available if I need it 43 4.21 1.71 1 2 6 5 6 10 13 2 45

14 The skills I have are appropriate for using the e-Health

system 44 4.14 1.37 3 1 9 13 10 8 1 45

11 Using the e-Health system requires co-operation with

other staff 44 3.98 1.99 10 1 6 7 3 17 2 46

5 Government policy initiatives are supportive of this e-

Health system 36 3.97 1.28 1 1 11 12 6 5 8 44

30 There are ongoing mechanisms for monitoring and 29 3.97 1.64 2 2 6 8 5 6 16 45



321

appraising how this e-Health system is used

3 The e-Health system is a different way of working

compared with other parts of the NHS 38 3.87 1.21 1 1 14 10 9 3 8 46

16 I have confidence that using the e-Health system does

not put patients at risk 41 3.85 1.71 1 5 4 3 11 10 7 3 44

29 The staff here are committed to making the system

work 38 3.82 1.23 2 2 10 15 5 4 5 43

15 The skills needed to use the e-Health system are easily

learned 43 3.79 1.47 1 3 4 8 10 14 3 2 45

8 This organization has a culture that is supportive of

change 43 3.77 1.38 2 5 13 10 7 6

0 43

2 Sufficient organizational effort has gone into supporting

the e-Health system 42 3.64 1.69 2 4 2 13 5 10 6 4 46

13 In using the e-Health system, the allocation of work

between individuals is appropriate 35 3.51 1.27 2 4 13 9 4 3 9 44

28 The staff who work here have a shared understanding of

what the system is for and how it is to be used 41 3.51 1.33 1 1 6 13 11 6 3 3 44

18 In using the e-Health system, responsibilities are divided

between individuals appropriately 37 3.41 1.36 1 3 4 10 11 7 1 6 43

27 The e-Health system is easy to use 41 3.39 1.74 3 4 4 10 8 7 5 2 43

22 I believe there is good evidence about the clinical

effectiveness of using the e-Health system 43 3.37 1.22 1 2 3 21 7 8 1 0 43



322

23 There is some flexibility in how the e-Health system can

be used 36 3.36 1.42 1 3 5 9 11 5 2 9 45

4 The rewards of using the e-Health system outweighs the

effort 37 3.22 1.65 2 3 8 9 6 5 4 8 45

17 Using the e-Health system is an efficient use of time 42 3.17 1.77 5 3 3 15 5 7 4 1 43

12 The workload involved in using the e-Health system is

manageable 43 3.16 1.60 3 2 11 9 8 7 3 3 46

1 The e-Health system is adequately resourced financially 33 3.15 1.58 2 4 2 13 6 3 3 13 46

7 This e-Health system fits in with the priorities and

challenges of our organization 41 3.15 1.70 4 3 5 13 7 5 4 3 44

24 Using the e-Health system leads to positive outcomes

for patients 40 3.10 1.39 2 4 4 15 9 5 1 5 45

9 There is a culture in this organization of involving staff in

planning and development 44 2.93 1.73 4 6 7 11 9 2 5 0 44

26 In using the e-Health system, the quality of professional

and patient interaction is good 30 2.87 1.46 3 2 4 12 6 2 1 14 44

10 Using the e-Health system makes me feel autonomous

in my work 40 2.83 1.63 4 6 1 20 4 1 4 2 42

25 Using the e-Health system involves the right amount of

time spent with patients 31 2.61 1.36 3 3 8 7 9 1 0 12 43

6 This e-Health system is technically and organizationally

compatible with other systems and agencies that we are

required to work with 39 1.33 1.34 15 8 7 6 3 0 0 6 45



323

Appendix 22 WP3 TARS Specific Site 2: TARS items, means, standard deviations and frequencies

WP3 TARS Specific Site 2: TARS items, means, standard deviations and frequencies

Frequencies

N*

Mea

ns Sds

0

1

2

3

4

5

6

DK

N

1 The e-Health system is adequately resourced

financially 120 3.84 1.32 2 3 11 34 26 34 10 111 231

2 Sufficient organizational effort has gone into

supporting the e-Health system 182 3.9 1.36 3 6 16 42 49 46 20 49 231

3 The e-Health system is a different way of working

compared with other parts of the NHS 174 4.68 1.31 1 9 29 30 41 64 56 230

4 The rewards of using the e-Health system

outweighs the effort 181 4.15 1.23 2 12 46 49 41 31 44 225

5 Government policy initiatives are supportive of this

e-Health system 123 3.93 1.30 1 4 7 37 30 29 15 106 229

6 This e-Health system is technically and

organizationally compatible with other systems 176 3.35 1.53 10 10 26 50 36 31 13 53 229



324

and agencies that we are required to work with

7 This e-Health system fits in with the priorities and

challenges of our organization 207 4.07 1.27 7 11 52 60 45 32 21 228

8 This organization has a culture that is supportive

of change 212 4.01 1.49 7 7 11 47 59 41 40 10 222

9 There is a culture in this organization of involving

staff in planning and development 210 2.99 1.78 28 19 28 50 39 30 16 19 229

10 Using the e-Health system makes me feel

autonomous in my work 201 3.55 1.58 11 13 19 54 38 48 18 23 224

11 Using the e-Health system requires co-operation

with other staff 216 3.88 1.38 3 5 21 64 49 42 32 14 230

12 The workload involved in using the e-Health

system is manageable 215 4.08 1.29 2 4 14 52 57 54 32 15 230

13 In using the e-Health system, the allocation of

work between individuals is appropriate 188 3.87 1.36 3 4 19 52 44 43 23 42 230

14 The skills I have are appropriate for using the e-

Health system 219 4.73 1.14 1 6 30 43 74 65 11 230

15 The skills needed to use the e-Health system are

easily learned 221 4.03 1.29 1 6 17 52 60 55 30 9 230

16 I have confidence that using the e-Health system

does not put patients at risk 213 4.22 1.33 2 4 16 39 53 60 39 15 228

17 Using the e-Health system is an efficient use of

time 218 4.21 1.28 1 3 19 42 49 70 34 13 231



325

18 In using the e-Health system, responsibilities are

divided between individuals appropriately 192 3.77 1.45 8 6 12 56 44 46 20 35 227


accountability for my work 214 4.84 1.14 1 6 27 32 74 74 8 222


liability for my practice 204 4.76 1.28 2 2 5 27 34 61 73 19 223

21 Technical back-up in using the e-Health system is

available if I need it 203 3.59 1.50 5 14 27 50 45 41 21 26 229

22 I believe there is good evidence about the clinical

effectiveness of using the e-Health system 229 3.79 1.42 11 5 10 66 63 51 23 229

23 There is some flexibility in how the e-Health

system can be used 203 3.24 1.58 16 11 30 59 39 35 13 28 231

24 Using the e-Health system leads to positive

outcomes for patients 208 4.03 1.19 2 2 13 48 71 49 23 22 230

25 Using the e-Health system involves the right

amount of time spent with patients 201 3.52 1.44 9 8 26 51 53 42 12 28 229

26 In using the e-Health system, the quality of

professional and patient interaction is good 211 3.92 1.19 2 19 60 61 49 20 19 230

27 The e-Health system is easy to use 219 4.1 1.27 1 3 18 49 63 51 34 8 227

28 The staff who work here have a shared

understanding of what the system is for and how it

is to be used 215 4.12 1.27 2 3 17 41 66 54 32 11 226

29 The staff here are committed to making the 209 4.24 1.22 2 13 47 54 56 37 15 224



326

system work

30 There are ongoing mechanisms for monitoring and

appraising how this e-Health system is used 179 4.36 1.17 1 6 38 51 47 36 50 229



327

Appendix 23 WP3 Comparison of correlations between TARS Specific samples

WP3 Comparison of correlations between TARS Specific samples

Correlations: Site 1 Site 2

CI items ‘financial resource’ with ‘organizational effort’

‘culture supportive of change’ with both

‘supportive policy initiatives’ and ‘fit with

priorities of organization’

‘priorities’ with ‘reward outweighs effort’

financial resource with organizational effort

SW items Allocation of work with both manageable

workload and existing skills are appropriate

(but autonomy in work and cooperation with

others were low with other SW items)

‘workload is manageable’ with both

‘allocation of work’ & ‘skills I have are

appropriate’

RI items ‘Efficiency’ with ‘confidence in not risking

patients’

‘efficiency’ with both ‘confidence in not

risking patients’ & 'appropriate divisions of



328

‘appropriate divisions of responsibilities’ &

‘belief in evidence of effectiveness’

‘confidence in not risking patients’ with

‘availability of technical back-up’

‘accountability’ and ‘liability’ very high

responsibilities’

‘accountability’ and ‘liability’ very high

IW items ‘flexibility in use’ with both ‘positive outcomes’ &

‘time spent with patients’

‘time with patients’ with ‘outcomes’ & ‘quality of

interaction’

‘time spent with patients’ with ‘quality of

interaction’

‘outcomes for patients’ with both ‘time with

patients’ and ‘quality of interaction’

‘quality of interaction’ with ‘easy to use’

Extended NPM

items

All three with each other plus:

‘Coherence’ with ‘allocation’ ‘efficiency’, ‘division

of responsibilities’, & ‘easy to use’.

All three with each other plus:

reflexive monitoring with ‘accountability’ &

with ‘liability’



329

‘Cognitive Participation’ with: ‘skills are

appropriate’; ‘confidence in not risking patients’

& ‘easy to use’

‘Reflexive monitoring’ with: ‘allocation’;

‘confidence’; ‘efficiency’; ‘responsibilities’ &

‘easy to use’

High correlators

across constructs

efficiency

rewards outweigh effort

allocation of work

confidence in not risking patients

efficiency

the workload is manageable

Poor correlators

across constructs

Co-operation with others

Organisational compatibility

Co-operation with others

Different way of working



330

Appendix 24 WP3 TARS Specific Site 1: Cross tab analysis of agreement with TARS items by perception of level of routineisation

WP3 TARS Specific Site 1: Cross tab analysis of agreement with TARS items by perception of level of routinisation

Partly routine

N (non-agree, agree)

Completely routine

N (non-agree, agree)

χχχχ

1 The e-Health system is adequately resourced financially 19 (11, 8) 12 (8, 4) .239

2 Sufficient organizational effort has gone into supporting the e-Health

system 26 (15, 11) 14 (4, 10) 3.095

3 The e-Health system is a different way of working compared with other

parts of the NHS 26 (11, 15) 10 (5, 5) .031

4 The rewards of using the e-Health system outweighs the effort 23 (17, 6) 14 (5, 9) 5.268*

5 Government policy initiatives are supportive of this e-Health system 26 (13, 13) 11 (0, 11) 8.479**

6 This e-Health system is technically and organizationally compatible with

other systems and agencies that we are required to work with 25 (22, 3) 14 (13, 1) .230

7 This e-Health system fits in with the priorities and challenges of our

organization 27 (21, 6 ) 14 (3, 11) 12.061***

8 This organization has a culture that is supportive of change 30 (18, 12) 14 (1, 13) 10.870***

9 There is a culture in this organization of involving staff in planning and

development 30 (19, 11) 14 (8, 6) .154



331

10 Using the e-Health system makes me feel autonomous in my work 29 (23, 6) 14 (8, 6) 2.306

11 Using the e-Health system requires co-operation with other staff 28 (11, 17) 14 (6, 8) .049

12 The workload involved in using the e-Health system is manageable 27 (18, 9) 14 (5, 9) 3.586

13 In using the e-Health system, the allocation of work between individuals

is appropriate 21 (13, 8) 14 (5, 9) 2.307

14 The skills I have are appropriate for using the e-Health system 29 (10, 19) 14 (1, 13) 3.707

15 The skills needed to use the e-Health system are easily learned 29 (11, 18) 14 (4, 10) .364

16 I have confidence that using the e-Health system does not put patients

at risk 28 (12, 16) 14 (1, 13) 5.570*

17 Using the e-Health system is an efficient use of time 29 (20, 9) 14 (5, 9) 4.289*

18 In using the e-Health system, responsibilities are divided between

individuals appropriately 26 (15, 11) 13 (4, 9) 2.514

19 In using the e-Health system, I understand my accountability for my

work 29 (7, 22) 14 (0, 14) 4.036*

20 In using the e-Health system, I understand my liability for my practice 29 (6, 23) 14 (1, 13) 1.271

21 Technical back-up in using the e-Health system is available if I need it 28 (11, 17) 14 (1, 13) 4.725*

22 I believe there is good evidence about the clinical effectiveness of using

the e-Health system 30 (21, 9) 14 (7, 7) 1.650

23 There is some flexibility in how the e-Health system can be used 23 (12, 11) 13 (6, 7) .120

24 Using the e-Health system leads to positive outcomes for patients 26 (18, 8) 14 (7, 7) 1.436

25 Using the e-Health system involves the right amount of time spent with 29 (15, 4) 14 (7, 7) 3.039



332

patients

26 In using the e-Health system, the quality of professional and patient

interaction is good 28 (14, 4) 12 (7, 5) 1.296

27 The e-Health system is easy to use 28 (17, 11) 14 (3, 11) 5.775*

28 The staff who work here have a shared understanding of what the

system is for and how it is to be used 27 (18, 9) 14 (2, 12) 10.124***

29 The staff here are committed to making the system work 25 (12, 13) 14 (2, 12) 4.433*

30 There are ongoing mechanisms for monitoring and appraising how this

e-Health system is used 15 (8, 7) 14 (2, 12) 4.887*



333

Appendix 25 WP3 TARS Specific Site 2: Significant differences in comparison of Nursing/health staff (NH) with call handlers (CH) on TARS items, specified as percentages of professional group

WP3 TARS Specific Site 2: Significant differences in comparison of Nursing/health staff (NH) with call handlers (CH) on

TARS items, specified as percentages of professional group

Item:

Disagree

N (NH, CH)

Neutral or

some

agreement

N (NH, CH)

Moderate or

strong

agreement

N (NH, CH)

χχχχ

Using the e-Health system makes me feel autonomous in my work 49 (16, 30) 102 (40, 55) 68 (43, 15) 20.458***

The skills needed to use the e-Health system are easily learned 24 (15, 5) 119 (54, 49) 87 (31, 46) 9.509**

In using the e-Health system, I understand my liability for my practice 9 (2, 8) 62 (25, 34) 141 (73, 58) 7.490*

Using the e-Health system involves the right amount of time spent with

patients 43 (25, 17) 104 (56, 47) 54 (19, 36) 7.210*

* denotes significance level of p<0.05; **p<.01; ***p<.001



334

Appendix 26 WP3 TARS Specific Site 2: Comparison of perception of not/partly routine (NP) with completely routine (C) by TARS items (bracketed figures refer to % within perceived routinisation grouping)

WP3 TARS Specific Site 2: Comparison of perception of not/partly routine (NP) with completely routine (C) by TARS items

(bracketed figures refer to % within perceived routinisation grouping)

Item:

Disagree

N (NP, C)

Neutral or

some

agreement

N (NP, C)

Moderate or

strong

agreement

N (NP, C)

χχχχ

Sufficient organizational effort has gone into supporting the e-Health system 23 (30, 11) 83 (52, 50) 59 (17, 39) 7.757*

The e-Health system is a different way of working compared with other parts of

the NHS 9 (0, 7) 51 (63, 28) 98 (37, 66) 9.818**

This organization has a culture that is supportive of change 24 (9, 13) 98 (72, 47) 70 (19, 40) 6.868*

The skills I have are appropriate for using the e-Health system 7 (10, 2) 69 (55, 31) 123 (35, 67) 12.714**

In using the e-Health system, I understand my accountability for my work 6 (8, 2) 53 (50, 24) 135 (42, 74) 10.918**

In using the e-Health system, I understand my liability for my practice 9 (20, 3) 54 (36, 28) 124 (44, 70) 16.503***

I believe there is good evidence about the clinical effectiveness of using the e-

Health system 25 (24, 9) 119 (54, 58) 65 (22, 33) 7.109*



335

The staff who work here have a shared understanding of what the system is for

and how it is to be used 19 (22, 8) 98 (52, 50) 78 (26, 42) 6.576*

There are ongoing mechanisms for monitoring and appraising how this e-Health

system is used 6 (5, 4) 79 (74, 45) 78 (21, 51) 6.196*

* denotes significance level of p<0.05; **p<.01; ***p<.001



Appendix 27 WP4 Interview Schedule for Case Study 1 (Choose and Book)

This schedule was amended for use with Case Studies 2 and 3 by substituting references to Choose and Book with either Picture Archiving and Communication System (CS 2) or the Clinical Nursing Informatics System (CS 3).

Implementation:-

• What do you feel has prompted the implementation of C&B? (Drivers, needs) • Can you describe the implementation process followed? What stage is it now

at? • What has been your role in its implementation?

Within those elements of the implementation process that you have been involved with:-

• What factors do you think have assisted with implementation? Can you tell me a bit more about that? Have any factors that you felt would have helped with implementation been missing?

• What factors have not been so helpful with implementation? Can you tell me a bit more about that? What could have changed this? Are different factors important at different stages in the process?

• With hindsight, has anything that you thought would help (or hinder) the rollout, in reality not made a difference either way? Looking at C&B now that it has “gone live”:-

• What staff groups/units have been involved or affect by the implementation? • What do people who use it report back? • What particular aspects of C&B do you think have been positively received by

staff into their everyday practice? How do you know this? Why do you think this is?

• What has not been received so well into everyday work? How do you know this? Why do you think this is? Is there a difference been units or groups of staff?

• In your experience, which aspects of NHS work has C&B improved? • What dis-benefits have you come across? • Overall how well do you think C&B has been integrated into normal routines?

In this final section I would like to ask you a few more detailed questions on how C&B has impacted in practice on the work that healthcare organizations do. We may have touched on some of these issues earlier - however it would be helpful if you could think about them in more detail here, if you have any knowledge of them.

• What do you think has been the impact of C&B on the clinical encounter? Prompts:

o Changes within consultations and appointments (time, fluency, structure).

o How has it affected doctor-patient interactions? o How do you know this?



• What has been the impact of C&B on professional relationships between individuals and teams. (Clinicians, AHP and managers/admin staff). Prompt:

o Has C&B affected agreements about levels of authority, competency or performance/quality?

• How the division of labour with the organization been affected by the introduction of C&B? Prompts:

o What changes have occurred? o Was that expected? o What impact have any changes in work made to those groups

affected? o Has this led to any renegotiation of the way people work?

(workloads, wages, rewards or status) • How has introducing C&B impacted on the on the work of the

organization or its delivery of services. Prompts:

o What changes (positive or negative) have there been to policies, processes or budgets?

o Has performance / quality been affected? o What about existing arrangements about the procurement or

allocation of resources? o How successfully has this been managed? o How do you know this?



338

Appendix 28 Salient Features of Selected Case Studies

Salient Features of Selected Case Studies

Case Study 1 Choose and Book (C&B)

Background The Choose and Book service has been introduced across the NHS in England. CFH deployments began in the Summer of 2004. C&B combines electronic booking and a choice of place, date and time for outpatient appointments. C&B is located across the primary and secondary care interface and requires GPs to launch an electronic referral using the C&B software within the consultation. Via Choose and Book patients can choose their initial hospital appointment from available slots and book it either on the spot (within the consultation) or later on the phone or internet. Consultants log into the system via hospital IT networks to view work-lists and to accept, reject or change priority on referrals, as necessary.

Region London CFH Cluster: The London Cluster was one of the first to launch Choose and Book into Early Adopter Sites (EAS). Many EAS sites implemented Choose & Book onto non LSP*-delivered Patient Administration Systems. The experience of EASs was then used to produce an implementation toolkit for PCTs and acute Trusts across England. However despite such early efforts to facilitate implementation 2006 data4showed that within NPfIT Choose and Book remained the service attracting most criticism from NHS staff especially amongst doctors, practice managers and administrators. *Local Service Provider.

Contexts

• Policy level sponsor National policy level sponsor - part of NPfIT

• Service Sector C&B is located across the primary-secondary care interface via NHS network and IT systems. It also has a web based e-booking service application accessible by patients via the internet

• Proposed Users General Practitioners, Consultants, Patients

• Modality Software package implemented on to GP desktop clinical systems and hospital patient administration systems. Specific service application available on internet.

E-Health fields of activity

• Management systems Provision of an electronic booking system. Promotes an electronic method of sending referral letters and electronically tracking progress of approved appointments.

• Decision support systems

N/A



339

• Communication systems

The system allows clinicians to communicate information on referral need and priority.

• Information systems Allows both doctors and patients to access information on choice of hospital via NHS IT systems or the internet.

Timeline 2004 – 2007

Case Study 2 Picture Archiving & Communication System (PACS)

Background PACS enables digital images such as x-rays and scans to be stored and viewed electronically, so that doctors and other health care professionals can access the information and compare it with previous images. Files are stored centrally and images travel electronically to doctors and other healthcare professionals. NHS PACS is being delivered by CFH throughout England via five regions or clusters of strategic health authorities. PACS sites are live in all 5 clusters in England. 2006 data has shown that PACS continues to enjoy high favourability and low levels of critical opinion amongst NHS staff.4

Region Southern CFH Cluster: The southern cluster covers Cornwall to Kent and is where the implementation of PACS is most mature. CFH roll out of PACS began in the southern cluster in the spring of 2005 and at the time of the interviews 22 PACS had been implemented in the region.

Contexts

• Policy level sponsor National policy level sponsor - part of NPfIT.

• Service Sector PACS is currently being deployed within NHS secondary care locations, primarily within acute trusts. However in due course it will be implemented in any location where pictures of a medical nature need to be taken, stored or used for the purposes of NHS diagnosis or treatment. This will included some primary care and community settings.

• Proposed Users Staff involved in radiology, radiotherapy, angiography, cardiology, fluoroscopy, CT, MRI, nuclear medicine, ultrasound, dental and symptomatic mammography.

• Modality Digital images with accompanying hardware and software implemented onto hospital IT and network systems.


• Management systems Allows electronic storage and retrieval of digital images, integrates pictures with electronic patient records.

• Decision support systems N/A

• Communication systems Provides images that can be shared and multiply viewed.

• Information systems N/A

Timeline 2005 - 2008



340

Case Study 3 Community Nursing Information System

Background A mobile information system for use by community nurses which has been implemented in one Scottish Health Board. The Personal Digital Assistants (PDAs) support patient registration, case activation, clinic and visit scheduling, clinical document generation and information sharing between team members. The PDAs also have a facility for providing electronic access to clinical protocols although this not currently in use in the deployed system.

Region Scotland. The Health Board is the largest of it’s kind in Scotland employing over 800 District Nurses across 10 Community Health Care Partnerships (CHCP’s). The CNIS was first piloted in 2004 and subsequently rolled out across the CHCPs.

Contexts

• Policy level sponsor Local policy level sponsor. Mandate from the Scottish Executive to develop an electronic Single Shared Assessment document.

• Service Sector Primary Care. However the Health Board had integrated Social Services and Primary Care managed via 10 Community Health Care Partnerships with co-terminous boundaries.

• Proposed Users District nurses, staff nurses and nursing assistants

• Modality Web-based system Hardware: Personal Digital Assistants (handheld computers) that can be synchronized with desktop PC’s/wireless laptops.


• Management systems Helps maintain an electronic patient record. Potential to use the CNIS for caseload profiling, audit and resource management.

• Decision support systems Custom designed Shared Nursing Assessment document containing 10 assessment tools e.g. a nutritional scaling tool; a wound assessment tool; a continence assessment; a leg ulcer assessment tool and the Waterlow Scale. Offers some decision support functions.

• Communication systems Data from the Nursing Shared Assessment can be automatically imported into the Single Shared Assessment (SSA) document used by Social Services. Currently the SSA is sent by post however the intention is to send the document electronically via a multi-agency store. This should promote more efficient inter-agency communication and would enable either DN or SW to initiate an assessment.

• Information systems Some clinical information is available in “help” files.

Timeline 2007-8



Appendix 29 WP4 Print-out of the e-HIT.


Disclaimer This report presents independent research commissioned by the National Institute for Health Research (NIHR). The views and opinions expressed by authors in this publication are those of the authors and do not necessarily reflect those of the NHS, the NIHR, the NIHR SDO programme or the Department of Health. The views and opinions expressed by the interviewees in this publication are those of the interviewees and do not necessarily reflect those of the authors, those of the NHS, the NIHR, the NIHR SDO programme or the Department of Health Addendum This document was published by the National Coordinating Centre for the Service Delivery and Organisation (NCCSDO) research programme, managed by the London School of Hygiene & Tropical Medicine. The management of the Service Delivery and Organisation (SDO) programme has now transferred to the National Institute for Health Research Evaluations, Trials and Studies Coordinating Centre (NETSCC) based at the University of Southampton. Prior to April 2009, NETSCC had no involvement in the commissioning or production of this document and therefore we may not be able to comment on the background or technical detail of this document. Should you have any queries please contact [email protected].

understanding the implementation and integration of e-health … · 2012. 1. 6. · queen's...

Documents