Available online at www.sciencedirect.com

journal homepage: www.elsevier.com/locate/hlpt

Health Policy and Technology (2012) 1, 35–49

2211-8837/$ - see frodoi:10.1016/j.hlpt.2

E-mail addresses

TEMPEST: An integrative model for healthtechnology assessment

Wendy L. Currie

Editor-in-Chief, Health Policy and Technology, Fellowship of Postgraduate Medicine, London, UKAvailable online 8 February 2012

nt matter & 2012012.01.004

: wendy.currie@fp

AbstractThis paper responds to calls for a national forum to track enabling (current) and emerging(future) technologies in healthcare. An integrative model for health technology assessment isdeveloped from prior empirical research, secondary source material and peer review on a rangeof healthcare technologies: information and communications technology; medical devices,imaging/monitoring technology; personalised medicines; drug discovery and diagnostics. TheTEMPEST model is an acronym for technology, economic, market, political, evaluation, socialand transformation. These themes are sub-divided into focal areas, where quantitativeindicators/metrics are used for comparative analysis. The model provides a conceptual andanalytical tool for policy-makers, healthcare professionals IT vendors, citizens and otherstakeholders for understanding and evaluating the scale and scope of health technologyadoption and implementation at national, regional and local levels.& 2012 Fellowship of Postgraduate Medicine. Published by Elsevier Ltd. All rights reserved.

Introduction

The global healthcare industry is changing in three importantareas. Firstly, politicians are exploring alternative ap-proaches to fund state-run healthcare systems by introducingmarket mechanisms in healthcare service delivery [1,2].Public–private partnerships are developing to provide citi-zens with greater choice in healthcare products and services[3,4]. The impetus behind this is partly because of burgeon-ing healthcare budgets with OECD countries annuallyspending anywhere between 5% and 16% of Gross DomesticProduct on healthcare, with an average between 8% and 9%[5]. Secondly, while healthcare technology in the EuropeanUnion is considered a ‘low maturity’ sector compared withfinance and manufacturing, spending on information andcommunications technology (ICT) is increasing and is

Fellowship of Postgraduate Medic

m-uk.org

estimated to be between 5% and 8% of GDP [6]. The growingmarket for ICTs also extends to medical devices and imagingequipment [7] as these technologies are becoming increas-ingly converged [8]. Thirdly, citizens are encouraged to takegreater control over their healthcare choices, moving fromtraditional medical professionalism to a patient-centredapproach. Health tourism is increasing with new productsand services supported by a range of enabling and emergingtechnologies, i.e. telemedicine, mobile devices to alertpatients about appointments, prescriptions and test results [9].

Prior research shows that introducing healthcare technol-ogies is high risk where potential and realised benefits aredifficult to measure [10]. Unintended outcomes are notuncommon, particularly in large-scale publicly funded ICTprojects that fail to meet expectations. For example, theNational Programme for Information Technology (NPfIT) inEngland was created to deliver local Information Technologysolutions as part of the NHS Care Records Service. Englandwas initially divided into five geographic areas, each of whichwould work together to take forward the procurement and

ine. Published by Elsevier Ltd. All rights reserved.

1This paper introduces the TEMPEST methodology and discussesits rationale in relation to the wider health technology literature. Ithas currently been applied to 27 EU Member States, although spacelimitations preclude its further discussion and analysis in this paper.Future issues of Health Policy and Technology will include cross-country comparisons of TEMPEST data points to give examples ofcurrent and potential benefits and barriers.

W.L. Currie36

implementation of the NHS Care Records Service at a locallevel. These comprised of the: Eastern, North East, NorthWest and West Midlands, London, and Southern clusters. In2007, responsibility for local delivery of the NPfIT wasdevolved to Strategic Health Authorities (SHA’s). England wasthen divided into only three Programme for IT regions eachwith its own Local Service Provider. In 2011, a decision wasagain taken to restructure the NPfIT as SHAs were beingdisbanded. This would allow for more choice and decen-tralisation of IT services, rather than the original top-down,centralised approached of the previous decade. Following allthese changes, the UK coalition government in the autumn of2011 decided to completely rethink the NPfIT, although thiswill necessitate a renegotiation of contractual terms withleading IT and management consulting firms. This change ofgovernment policy has resulted in media reports that healthtechnology in the NHS is in total disarray, as the aim ofproviding citizens with full access to their electronic healthrecords remains a vision rather than a reality. By 31 March2011, total expenditure on the Programme totalled some£6.4 billion [11].

Other cases in Europe, i.e. the European Health Card(EHC) implementation in Germany and the 2012 HospitalsPlan in France aimed at reorganising regional healthcareservices, upgrading hospitals to meet safety standards andintroducing new IT systems in hospitals have also incurredproblems, not only of a technical nature but also politically,organisationally and culturally. Health technology assess-ment (HTA) is not only a technical issue, but also needs to beunderstood within a larger, socio-political and economiccontext, where stakeholders often have different prioritiesand interests.

In recent years, the challenge to transform healthcaredue to rising costs and increased demand continuesunabated. Health technologies are seen to play a majortransformational role [8]. Yet whether they are introducedcentrally by government planning, or through regional orlocal initiatives, i.e. at the hospital level, remains acontentious issue. While the former has advantages ofeconomies of scale (i.e. procurement contracts), standardi-sation, and interoperability, the latter is more likely toachieve the political buy-in of clinicians, hospital adminis-trators and patients, especially if these groups have beeninvolved in the decision-making process from an early stage.The scientific research agenda should therefore focus onidentifying and learning from best practice examples inhealth technology adoption and diffusion at national,regional and local levels. Scientific research should addressthe wide ranging interests of stakeholders so that policiesand plans for introducing transformational change usingtechnology meets the needs of health professionals,patients and the community at large.

The TEMPEST methodology responds to calls to develop aninternational forum to track emerging and enablingtechnologies and their potential for diffusion intohealthcare environments. This forum is more likely to beeffective if it is built on evidence-based research whichserves as a resource-base available to multiple stakeholdersengaged in the policy, design and implementation of healthtechnologies. Prior research indicates the outcomes ofintroducing new technologies into healthcare organisationsare mixed due to poor policy making, management practice

and technical factors [12]. TEMPEST thus adopts a multi-dimensional and integrative approach to understanding thecomplexities and challenges of introducing healthcaretechnologies. It aims to provide a useful HTA tool forpolicy-makers and communities of practice engaged inhealth service delivery.

TEMPEST: an integrative model

The TEMPEST1 methodology leverages its unique frameworkof 7 themes, 21 sub-themes and 84 (coded) quantitativeindicators to deliver evidence-based insights that addressscientific and policy issues (see Appendix A). This isreinforced by an inter-disciplinary, multi-dimensional modelof health technology assessment (HTA) that allows thoseapplying it to analyse specific health technologies, orrelated issues, from a market, political, commercial,stakeholder or individual perspective, through a national,regional, local or organisational lens. So far, TEMPEST hasbeen applied to the 27 European Union Member Stateswhere data can be compared at the EU and national levels.Incorporating key EU policy priorities, such as encouragingactive and healthy ageing, preventive medicine and healthylifestyles, improving citizens’ health security, reducinghealth inequalities, and promoting and disseminating healthinformation [13] TEMPEST offers an integrative approachwhere data points can be compared and contrasted to showbenefits and barriers in health technology adoption anddiffusion.

HTA is a multi-disciplinary process which aims to informthe development of effective health policies that are:(i) patient-focused; (ii) market- and consumer-friendly, and(iii) deliver optimal value and outcomes to all stakeholdersinvested in public health. As a research approach, it focuseson information about the medical, social, economic andethical issues related to the use of a health technology, aswell as the short- and long-term consequences of its use, ina systematic, transparent, unbiased and robust manner.However, the different academic perceptions and practicesof what constitutes HTA raise concerns about the adequacyof evaluation methods. The siloed approach to HTA wherestudies isolate the economic, organisational or culturalaspects of specific health technologies leads to observationsthat ‘ythe striking finding from our review (of healthtechnology) is that there has been so little solid evaluationof these (EHR) applications’ [10]. Others comment that,‘When technologies are disruptive, operating and financialimpacts are challenging to estimate, which makes itdifficult to construct a ‘business case’ for investmentydisruptive technologies make it difficult to conduct return-on-investment (ROI) analyses’ [8]. So gathering quantitativeand/or qualitative data on selected health technologieswithout considering the wider socio-political, organisationaland ethical issues is a futile exercise [14,15]. It is also

2OECD (2012) International Workshop: Benchmarking adoptionand use of Information and Communications Technologies in theHealth Sector, 30–31 January, Paris, France.

TEMPEST: An integrative model for health technology assessment 37

apparent that, while there are ‘serious structural barriersto the use of IT that have nothing to do with technolo-gyylegal and financial incentives provide little motivationto share information across institutions, which is critical toimproving patient outcomes as well as efficiency’ [16].

Scientific research has shown the emphasis in theapplication of HTA has primarily focused on clinical aspectsof health technologies, rather than economic, patient-related, and organisational aspects. There are some notableexceptions. Danish HTAs, for example, have a wider scopecompared with HTAs in other countries, as patient-relatedand organisational dimensions are included [17]. EvaluatingHTA institutions in nine countries (Australia, Canada, Den-mark, Netherlands, New Zealand, Norway, Sweden, UK andUSA) the authors found that Denmark and Sweden scoredhighest for including organisational issues, with the USAscoring lowest. In the case of the latter, organisational,economic and patient aspects were given less attention withthe inclusion of a limited number of questions. Economicissues were more frequently included than patient issuesand this order of priority was fairly consistent for all HTAinstitutions. Policy recommendations were also usuallyomitted from HTA reports. Conclusions suggest that reportswhich omit wider aspects have dubious value for decision-making, and there is a vital need for further research [17].

With regard to health technologies, monitoring data shouldtherefore be sufficiently robust for HTA to inform optimal useof technology [12]. An evidence-based approach to HTA canoffer decision makers data and information about the optimalor sub-optimal use of health technology. Evidence fromnational or regional studies where organisational and socialindicators, for example, are not integrated into HTA, are lesslikely to provide a robust evaluation of clinical and costeffectiveness. For certain complex technologies (such as ICTsand medical devices), providing evidence about the expectedimpact of a technology on health system structures,processes, and resources is valuable within the HTA exerciseas it can contribute to the development of an effectiveimplementation plan. However, the potential choice ofrelevant indicators, and the need to harmonise evidence fordecision-making, adds further complexity to HTA. A debate istherefore needed to identify key quantitative and qualitativemeasures, metrics and indicators for HTA, as this is importantnot only for benchmarking the adoption and use of healthtechnologies, but also for policy-making.

Integral to the TEMPEST methodology is a multi-disciplinary,cross-national, and diverse health technology landscape, whichutilises and extends prior work on HTA within a structured andthemed framework. TEMPESTaggregates health and technologyindicators from a wide range of reputable data sources, tocontribute to state-of-the-art research by building on priorresearch that addresses health policy priorities. The pragmaticrationale for TEMPEST is to create a cross-disciplinary,integrative model aimed to bring together disparateacademic communities in clinical and social science. Forexample, academic researchers in health policy, healthinformatics, health management and health sciences, tendto publish their findings in specific academic outlets (i.e.academic journals and conference proceedings) whichreflect disciplinary silos. This offers limited opportunitiesfor knowledge sharing, as dissemination of common issues inhealth policy and technology needs to be shared with all

relevant stakeholder communities, not least patients andtheir representative groups.

The scale and scope of health technology is a furthercomplexity. ICTs, medical devices, imaging/monitoringtechnology, personalised medicines, drug discovery anddiagnostics, all impact on clinical and non-clinical stake-holder groups. This suggests that a common language needsto be found to enable these diverse groups to communicatewith each other. As prior research has shown, the introduc-tion of technologies in healthcare settings has producedmixed results as technologies are in various stages of theirmaturity life cycle, with untried and untested emergingtechnologies posing the most risk. Even where technologiesare perceived to highly mature, they may not have deliveredexpected benefits, measured by an adequate return-on-investment. For example, electronic medical records (EMRs)were first introduced as early as the 1960s, yet their useacross the international healthcare landscape is patchy, withmany clinicians experiencing problems of inadequate pa-tient data (i.e. missing and/or incorrect). Paper-basedrecords continue to be widespread, as the move toelectronic records is fraught with difficulties. Recentattempts to introduce electronic patient records (EPRs)and electronic health records (EHRs) have proved equallychallenging [18] particularly where the patient is also a userof these systems. Although the reasons for slow adoption ofhealth technologies are often cited as ‘computer problems’,clinicians’ and patients’ concerns are more commonlyfocused around ease of use, data integrity, confidentialityand patient safety. Another critical issue is around incen-tives, particularly where perverse incentives highlight theasymmetries of risks and rewards (i.e. the groups whichexperience the highest risks are not those who experiencethe rewards from moving to electronic records, forexample).

Recognising that HTA is a complex task, the TEMPESTmethodology rejects ethnocentric perspectives which ad-vocate a one size fits all, towards transforming healthcare.Improvements from health technology in one healthcareenvironment, may not work in another. For example, arecent study by the OECD2 claimed that Finland has over100% adoption of EHRs in hospitals and almost the same inprimary care. Yet, electronic exchange of key documents(i.e. referrals and discharge letters) between these organi-sations has been slow. This compares with the UK experi-ence, where the drive to introduce EHRs (i.e. the summarycare record) has received significant investment, althoughmarred by complex problems, a notable one being poorclinical and user engagement [37].

Identifying the contextual reasons why health technolo-gies either succeed or fail in different health environmentsis a research priority. To assist this endeavour, the TEMPESTmethodology aggregates 84 (coded) quantitative indicatorsfrom a wide range of robust data sets (WHO, OECD, WorldBank, EU, and others), developed from prior empiricalresearch and secondary source material to provide anunbiased, transparent and robust approach to isolating keydata points (indicators) to drill-down and compare and

W.L. Currie38

contrast specific themes and sub-themes at the country andregional units of analysis. The benchmarking of selectedindicators contributes to a more generic profile andinformed understanding of the current health-technologynexus to provide stakeholders (i.e. policy-makers) with auseful template for evaluating the current and potentialhealth technology landscape. TEMPEST thus identifies thefront-runners, followers and laggards in health policy andtechnology. However, this does not suggest that the laggardsshould simply replicate the policies and practices of thefront-runners, as contextual factors may point to leapfrog-ging opportunities which require a tailored health technol-ogy policy agenda that may not work across all countries.The following sections introduce the 7 TEMPEST categoriesto demonstrate the multi-dimensional research agenda ofhealth policy and technology.

Technology

Health technology spans inter-disciplinary literature fromhealth informatics, information systems and technology,management and organisations, and clinical decision making[11,12]. The more technically focused literature, particu-larly from the health informatics or information systems andtechnology fields is complementary to the broader work onmanagement and organisations, since it considers thetechnical and engineering challenges of introducing newhardware and software in healthcare settings. The healthinformatics literature looks at technologies designed andapplied to healthcare. Similarly, there is a wealth ofliterature from information systems and technology thatalso considers these issues across different geographical,industrial, organisational and operational contexts. Obser-vations from reading this literature suggest that technicalchallenges faced in banking or manufacturing, for example,are similar to healthcare, albeit the solutions will bedifferent. Complex cultural and organisational factors inhealthcare will need to be considered as managementconsulting solutions, such as, lean thinking (i.e. used inmanufacturing) will need to take into account contextualfactors (i.e. incentive structures, clinical buy-in, userengagement and risk mitigation).

Within the field of health informatics, there are manystudies on how ICT can enhance clinical decision making,particularly as new technologies come on stream. Health-care IT has been described as an ‘alphabet soup’ [19] as newacronyms are increasingly introduced. Confusion existsabout the acronyms electronic health records (EHRs),electronic medical records (EMRs) and personal healthrecords (PHRs). They are defined respectively as, ‘‘EHRsprovide clinicians with access to patient information andprovide evidence-based decision support; EMRs are theelectronic version of a legal health record; PHRs are digitalhealth records that are owned, updated and controlled bythe consumer’’ [19]. PHRs are also likely to be referred to aselectronic patient records (EPRs). Despite the many defini-tions of electronic health technologies, a common under-standing is needed between politicians, healthcareproviders, academics and the public more generally,particularly as each type of record implies different access,usage, accountability and responsibility levels, each of

which relate to economic, social, professional, managerialand technical issues.

Other studies on electronic records consider the clinicaladvantages of introducing electronic records (medical,patient or health). In the United States, electronic healthrecords are seen as one solution for reducing the number ofmedical errors, which account for 44,000 to 98,000 personsdying in hospitals every year [20]. A national poll found that4 out of 10 persons believe that quality of care was decliningacross the health service despite rising healthcare budgets[21]. The US Congressional Budget Office predicts that totalspending on health care will increase from 16% of theeconomy in 2007 to 25% in 2025 [22]. Electronic records,with functionality for automated error checking, clinicaldecision support, and reliable information flow and integra-tion among different individuals and departments involved inpatient care, are viewed as an integral part of makinghealthcare less error prone and more efficient for cliniciansand patients. A recent publication shows that, in spite of thepotential of electronic records, less than 20% of US physiciansare using these systems [23]. Two challenges are identified.One, to develop appropriate clinical and financial expecta-tions, and two, to build or buy the skills needed for successfulimplementation [21]. Although these studies are US-centric,the challenge to develop electronic records is being con-fronted by politicians, professionals and public internationally.

Healthcare technology also includes medical devices andimaging systems. The most promising medical devicesegments within the region pertain to neurology, cardiovas-cular devices, imaging, in vitro diagnostics, general surgery,orthopaedics, urology and respiratory devices [24]. Theannual growth rate of the medical devices market in France,Germany, Italy, Spain and the UK was close to 9%, in anestimated market value of $51 billion. Imaging technologiesinclude picture archiving and communications (PACs) sys-tems and computer tomography (CT) and magnetic reso-nance imaging (MRI) scanners. Europe’s medical technologyindustry is growing steadily with around $5.05bn spent onR&D in 2006. The major European exporting countries ofmedical technology and their contributions are Germany($18.6bn), Ireland (£8.76bn), France ($8.1bn) and the UK($7.4bn). Germany, Ireland, the UK, Denmark, Sweden andFinland have medical technology trade surpluses. Themedical device market in Germany is expected to increasefrom $12,863 in 2007 to $16,973 in 2012 [24].

Health technology further includes personalised medicine(genetic diagnosis, testing and therapy), regenerativemedicine and remote patient monitoring [7]. The movetowards personalised medicine is gaining ground, althoughits vision and scope needs to be evaluated in terms of itspractical application. Part of the impetus for developingpersonalised medicine technologies is genetic profiling fortargeting drugs. The Human Genome project has fuelled thedrive in the pharmaceutical and biotechnology industries todevelop new therapeutic treatments. With almost twomillion people reported in the USA suffering from adversedrug reactions (ADRs), with tens of thousands leading tofatalities, the need to identify through genetic testing thosepatients who are likely to have an adverse reaction to asingle drug or cocktail of drugs is a key priority.

Healthcare technology will further be developed forregenerative medicine. In particular, stem cell research

TEMPEST: An integrative model for health technology assessment 39

where ‘restorative medicine aims to ‘culture a person’s owncells to create replacement tissues for repairing internalorgans damaged by trauma, disease, or aging’. Remotepatient monitoring will also revolutionise the healthcaretechnology field extending from an early forerunner in theform of electrocardiogram (EKG) devices for cardiacmonitoring, where readings are transmitted through dedi-cated telephone lines to cardiac intensivists from remotelocations including small rural hospitals [7]. BroadbandInternet connectivity has further enabled the remotemanagement of large numbers of intensive care unit (ICU)patients. Remote ICU systems digitally integrate voice,visual images, telemetry data and summary patient historyinformation and allow the monitoring of numerous patients.

The TEMPEST methodology identifies three importantthemes relating to technology: enabling/emerging technol-ogies; interoperability of eHealth; and eHealth servicedelivery model. These themes are not mutually exclusivebut are broadly concerned with access and usage oftechnologies. For example, broadband penetration acrossthe 27 EU Member States varies widely, with Denmark thehighest and Slovakia the lowest. Data shows that mobilecellular subscriptions are highest in the EU for Italy andlowest in Malta. Although this indicator provides just onedata point, it is extremely important for politicians andindustry leaders to bear this in mind when developingmobile eHealth applications. Another example is broadbandpenetration. Countries which score low on this indicator, willneed to develop policies to improve technology diffusion,particularly if citizens are to increase their use of eHealthapplications (i.e. government supported sites for seekinginformation about health). Countries with high mobilecellular subscriptions, and less developed health infrastruc-tures (i.e. Italy) are likely to offer greater opportunities forhealth application developers using mobile phones. This willprovide more efficient and cheaper means of communica-tion for health professionals and citizens (i.e. applications toinform patients about appointments, prescriptions and testresults).

Economic

The turbulent economic conditions suggest that govern-ments are seeking additional ways to reduce healthcarebudgets while achieving greater economies of scale andscope in healthcare service delivery. Economic environ-ments vary widely across the 27 EU Member States, andhealthcare continues to be a major priority for politicalleaders and citizens alike. Health policy is inextricablylinked to economic factors, and it is noteworthy thathealthcare spending as a percentage of gross domesticproduct (GDP) is very low in some countries, but high inothers. Across the EU, Member States vary widely in howthey pay for healthcare, with the UK having the highestgeneral government expenditure on health as a percentageof total expenditure on health, with Cyprus, the lowest.Conversely, Cyprus has the highest private expenditure onhealth as a percentage of total expenditure on health, withDenmark, the lowest. These figures are particularly im-portant in relation to health policy and technology, as anunder-developed health sector will pose problems for

introducing transformational change. The main drivers ofrising healthcare costs in Europe are: ageing populations andthe related rise in chronic disease, costly technologicaladvances, patient demand driven by increased knowledge ofoptions and by less healthy lifestyles, legacy prioritiesand financing structures that are less relevant for today’ssociety [25].

Along with government expenditure on health, the EUvaries enormously in terms of GDP growth and populationsize. For example, whereas Germany has a population ofover 81 millions, Luxembourg and Malta have less than half amillion. The leaders in eHealth are typically Denmark,Sweden and Finland, all with relatively low population sizes.In terms of introducing top-down, government-led health ITprogrammes, this is easier to achieve in smaller countries(with small economic regions), than large countries, wherean economic region (i.e. London) is larger than a country.

Other country variations include the urban/rural divideamong the population, and the labour force segmentationbetween agriculture, industry and services. These factorsare particularly critical for understanding the potentialadoption and diffusion of health technologies among diversepopulations. For example, countries with a high percentageof the population employed in agriculture (i.e. Romania,29%) provide very different opportunities for health tech-nology diffusion, than countries with a very low percentage(i.e. Netherlands, 2%, Sweden, 1.1%). This is partly becauseagricultural workforces are less likely to have access toinformation technologies than populations employed inindustry and services.

National, regional and local environments are thereforeimportant as healthcare technologies will need to bedeveloped and targeted to citizens across widely differingeconomic, population and labour force settings. Recognisingthese differences, Halamka [26] advocates ‘regional im-plementation organisations’ designed to offer an ‘enterpriseapproach’ to health technology implementation where theymay work with technology service organisations, that helpto orchestrate group purchasing plans and model contracts,on behalf of clients. Such an approach is necessary due, inpart, to the relatively low maturity of business and ITexpertise in healthcare organisations, especially in countrieswith less developed health and IT sectors.

The TEMPEST methodology integrates three economicthemes: health funding; performance and population; andlabour market segmentation. These areas reflect importanteconomic factors including expenditure, population size,labour markets and health infrastructure.

Market

The drive to develop a market-driven healthcare system is adeparture in some European countries from a state-ownedand controlled approach, particularly in the light ofburgeoning healthcare budgets. As we have seen above,the market for ICTs, devices and imaging technologies isgrowing considerably. However, other market considerationsinvolve the development of healthcare products andservices that run alongside new technologies. Market-drivenhealthcare includes the development of product andservices providers in the EU and beyond which may offer

W.L. Currie40

citizens greater choice (i.e. where they source theirhealthcare requirements). While patients have generallybeen offered few or even no choice of healthcare provider,particularly in state-run healthcare systems where theyhave no access to private healthcare, the trend is to widenthe market by allowing more competition. In the UK, forexample, patients are now able to use primary GP servicesacross the country and also to opt to use secondaryhealthcare providers (i.e. acute trusts) based on eithertheir own selection of a physician or for convenience oflocation. This is similar to Belgium, where citizens havegreater freedom to visit a GP of their choice, rather thanone located in a particular surgery.

Moves to develop private healthcare services, either as asupplement or an alternative to state-run healthcare, iscontroversial, yet inevitable given increasing annual publichealthcare budgets [1]. Past healthcare markets haveoperated on a push strategy where the customer (patient)has been a passive recipient of healthcare delivery. Futuremarkets are expected to balance this with a pull strategywith the public becoming more actively engaged in theirhealthcare. Evidence of a more customer-driven marketapproach is witnessed by the growing market for persona-lised healthcare services, including, the Microsoft Health-Vault and Google Health web portals, where citizens can setup their own electronic health/patient record and gainaccess to a databank of healthcare information resources.In addition, other websites include online pharmacies whichsell a range of medicines, with the inevitable problem thatsome of these sites are selling counterfeit goods withpotentially harmful consequences.

As the global healthcare market grows, healthcaretechnologies will gain traction as IT market capabilitiesand skills mature. As healthcare has been a laggard intechnology adoption compared with other industries, it willbe necessary for the various stakeholders to gain increasedexpertise in procuring and managing technology. Forexample, healthcare organisations, particularly, secondarycare providers (hospitals) will need to forge greater linksbetween the various stakeholder groups (i.e. clinicians,managers and users of technology). Different levels of ITmaturity exist among healthcare stakeholders. For example,in primary care, it is usual for GPs/physicians/doctors to runtheir own practices or surgeries. More generally, theseindividuals are likely to be relatively knowledgeable aboutthe latest ICT, having used patient administration systemsfor many years. But in secondary care, hospital consultants,nurses, junior doctors, managers and administrators, may beless knowledgeable about clinical and non-clinical uses oftechnology. Nurses, for example, may not have access to thelatest IT software applications, particularly in countries withlow IT investment. Notwithstanding the variations in ITcapabilities and skills across countries, secondary care isgenerally perceived as a late mover in health technologyadoption, compared with primary care and other industrysectors [27]. For example, Wanless [28] advised the UKgovernment that when comparing the annual spend on IT ina range of business sectors, financial services spend around£9000 per employee per year on computers, compared withonly £1000 per employee in healthcare. This led the UKgovernment to launch the National Programme for Informa-tion Technology with an initial pledge to spend over £6bn on

‘transforming’ healthcare using ICTs (see above). Policy-makers therefore need to address the issue of IT marketcapabilities and skills as this is essential for building newmarkets in healthcare.

The TEMPEST methodology covers three market themes:market-driven healthcare; consumer-driven healthcare; andIT market capabilities and skills. These areas relate to howhealth professionals engage with technology, how citizens orconsumers of health services use health information, andthe numbers of health professionals per population.

Political

The business and management literature includes studies onpolicy and economic issues relating to value creation inpublic services and redefining the healthcare industry. Thesecontributions consider how publicly funded healthcareorganisations may adopt methods and practices used in theprivate sector to create a ‘New Public Management’ [29]which better serves the public interest. More specifically,the global healthcare industry is high on the agenda ofinternational governments as healthcare budgets soar with-out a commensurate increase in service levels to patientsand public [1]. While these studies may not place healthtechnologies at centre stage in their recommendations tomodernise the healthcare industry, government fundedreports increasingly make a direct link between IT invest-ment and improvement in health service delivery. Most EUcountries now produce policy reports on health IT strategy,or eHealth policy. For example, as early as the 1990s, the UKNHS introduced the Resource Management Initiative (RMI) todeploy information systems to connect medical activity toresource usage and to costs [30]. This study showed how ITwas used to improve ‘greater communication, planning andcontrol, and even to bring about changes in organisationalcultures’. Since this period, governments have sought waysto use health technologies with outcomes that go beyondthe technical imperative [31]. This has led to observationsthat, ‘It is impossible to separate technology and policy innetworked systemsyStandards in health IT have beenlikened to construction specifications for building theinterstate highly system. Yet it turns out that an explicitpolicy framework is as important as any effort to createtechnical standards’ [16].

International governments have developed a health IT oreHealth policy with varying levels of detail. In the US, theHealth Information Technology Economic and Clinical Health(HITECH) Act is part of the American Recovery andReinvestment Act (ARRA) of 2009. ARRA authorises anestimated $20 billion in direct grants and financial incen-tives to promote the adoption and meaningful use ofelectronic health records among health care providers[32]. The entire US health information technology industryhad estimated revenues of $27 billion in 2008. Despitepolitical support and extensive investment in healthtechnology, some have expressed caution about the risksof ‘‘mis-spending this money and having little to show for itlater.’’ [26].

In the European Union, all 27 Member States havedeveloped policy statements on healthcare technologyinvestment, adoption and diffusion, which taken as a whole

TEMPEST: An integrative model for health technology assessment 41

is highly fragmented and patchy. This coincides with manychallenges, such as, the growth in health tourism [59] whichwill see increased cross-border traffic in patient data againsta backdrop of diverse legal and regulatory systems operatingat national and regional levels. This will increase the needfor legislation covering data security, confidentiality andownership rights and responsibilities.

eHealth policy will need to be integrated with e-Government policy, particularly as online availability ofpublic services will include health and eHealth. Countrieswith low access to online government services will thereforeneed to address these shortcomings, particularly if eHealthis to expand. An example is Austria, with 100% of basicpublic services for citizens fully available online, againstRomania, with at only 8%. Another example is thepercentage of the population using e-government services,with Finland at 53% and Bulgaria at 8% [64]. Clearly, unlesscitizens have both availability and access to governmentservices, progress towards eHealth will be slow. Otherimportant themes include education and training, particu-larly as a demographic profile with many elderly people andlow socio-economic groupings will serve as an inhibitor toeHealth. Many countries have introduced training pro-grammes to help citizens become IT literate by offeringpublicly available Internet access in libraries.

The institutional structure is a key factor in eHealth,particularly as the relationship between politicians andhealthcare professionals in policy-making and implementa-tion may either promote or discourage eHealth. Forexample, research shows that institutional priorities differacross the European Union on a variety of health-relatedareas. Data shows that establishments with a documentedpolicy, system or action plan on health and safety vary, withthe UK at 98% and Greece at 38% [33].

Empirical research has shown that where clinicians andother healthcare professionals (i.e. hospital managers andadministrators) reject government policies on health IT, it isunlikely that implementation plans for new systems will befulfilled [34,35]. While the literature has many excellentcase studies on health ITwhich focus on a single-site analysisof a particularly application (i.e. EHRs), there are fewerstudies on the health policy–technology intersection. Whilethese studies make a good contribution to the field, policyissues are an important element in health technology. Inparticular, where health IT policy for large-scale projects isdesigned at government level, but requires the support atregional and local levels. Health IT is therefore a highlypolitical phenomenon and becomes more so when there islittle consensus among politicians, healthcare professionalsand patient groups about the benefits and risks alongsideother important health priorities (i.e. staffing, patientservices, access to medicines, etc.).

The TEMPEST methodology covers three political themes:eHealth policy; education and training; and institutionalstructure. The health policy–technology nexus is critical forunderstanding political priorities for health IT at national,regional and local levels. The next steps for the politicalagenda should focus on identifying the drivers and barriersfor eHealth. While there are many studies that presenteHealth data and indicators at the country level, animportant research aim of TEMPEST is to identify andmeasure adoption and use of health IT across countries.

This will provide benchmarking data for policy-making at theregional, national and international level.

Evaluation

One of the issues relating to health technology evaluation is,‘What is being evaluated?’ This is a multi-faceted conceptwith researchers evaluating health technology from avariety of perspectives (i.e. eHealth policy, social andcultural conditions, value for money, benefits realisationand risk assessment). Many government sponsored evalua-tions on health technology, such as the summary care recordfor the National Programme for IT (NPfIT) in England areex-post evaluations, which seek to assess what has beenachieved at the implementation stage, rather than at thedesign and development stages [36]. Even at the imple-mentation stage, health technology may only be partiallyimplemented, with an expectation on the part of thesponsor that full implementation will take place at somedefined point in the future. The evaluation of the NPfITfocused largely on the social and cultural aspects of anelectronic health record, or NHS Summary Care Record,designed to store general health data on 50 million citizensin England. Despite several evaluations of the NPfIT over aten-year period, this large-scale programme ran into severalproblems, with the latest recommendation to replace thecentrally (top-down) approach with a more decentralised(bottom-up) one, where stakeholders at regional and locallevels can define their own health technology policies to fitwith their specific requirements [11].

Evaluating health technology is a complex task [38]. Priorstudies have identified the importance of understandingex ante evaluation comparing secondary source data (i.e.health IT strategy/policy documentation) about plannedimplementation of health technology. This data is oftenmerged with ex-post rationalisations where stakeholdersrecall what happened following implementation [39]. Acommon finding is that stakeholder perceptions about thedesired outcomes of health technology become confusedwith actual outcomes, where the initial shopping list ofpotential benefits changes over time [30,40]. Otherssuggest, ‘we need a new model of health IT that supportsvalue-added, patient-centred processes over individualtasks’ [41].

Evaluation studies from the health informatics communityfocus on benefits and risks from health technology to theuser community, which includes, clinicians, nurses, admin-istrators and patients [42–44]. The business and managementcommunity focuses more on issues of cost reduction andstreamlining operational processes in healthcare. Extendingthe field of systematic evaluation of health technology, Rossiet al. [45] identify a number of categories, including: needsassessment, programme theory, process analysis, impactanalysis, cost–benefit and cost–effectiveness analysis. A needsassessment examines the nature of the problem the pro-gramme is designed to address. This involves identifyingstakeholder groups affected by the programme and how/iftheir working practices are likely to change. The programmetheory is the formal description of the concepts and design ofthe programme. Programme theory delineates the componentsof the programme and shows anticipated short and long-term

W.L. Currie42

effects. It considers how an organisation will achieve desiredoutcomes as well as unforeseen consequences (both positiveand negative). Programme theory drives the hypotheses to betested in the evaluation. The development of a logic modelmay further build common understanding among stakeholders.Process analysis and modelling goes beyond the theory of theformal aims and objectives of the programme, and evaluateshow the programme is being implemented. Evaluation andimplementation are therefore separate focal areas. Theimpact analysis considers causal effects of the programme.Finally, cost–benefit or cost–effectiveness analysis assesses theefficiency, performance and risks of the programme. Evalua-tions occurring across sites may therefore identify bestpractice scenarios.

Since the late 1980s, new technologies have beenincreasingly viewed as a means to improve performancemanagement of health service delivery. Many studies havelooked at how private sector methods and practices can betransferred to public services, especially healthcare. Theseinitiatives include total quality management techniques [46]business process re-engineering [47,48] knowledge manage-ment [49] and more recently, new variants on qualitymanagement and lean manufacturing [50]. While thesepractices may produce variable benefits across healthcareservices, many writers caution that simply applying manage-ment tools and techniques from the private sector intohealthcare is not a guarantee for success [48] particularly aseach sector is governed by different management philoso-phies, policies, practices and procedures.

Performance measurement and risk assessment arecritical factors in conducting a pragmatic evaluation intohealth technology. Yet studies show that simply gatheringquantitative and qualitative data on isolated health tech-nology without considering the wider organisational struc-tural, cultural and managerial issues is seldom useful [14].Equally, measuring key performance indicators (KPIs) with-out acting on this information is an exercise in datacollection rather than technology evaluation. We thereforeconcur with the view that, ‘‘As with medicine, managementis and will likely always be a craft that can be learned onlythrough practice and experience’’ [51]. To evaluate healthtechnology, it is important to understand how businessprocesses may change as a result of introducing newtechnology. Common tools and techniques include valuechain analysis [1] process mapping and balanced scorecard[52]. Mixed data collection methods are useful for providinga rich picture of how clinical, non-clinical staff and patientsinteract with health technology across different settingssuch as the hospital or in the home.

Empirical research into the design and implementation ofhealth technology in the UK NHS has found that poor userengagement is a significant barrier to adoption [34]. Theconcept of a national health service is misleading, particu-larly as health technology policy and implementationreflects the federated and fragmented system of the healthservice, made even more complex by different politicalpriorities in England, Scotland, Northern Ireland and Wales.Health technology evaluation is further complicated by thealmost constant policy changes in health, culminating instructural and organisational changes. For example, in somehealthcare organisations (i.e. the large teaching hospitals)more money has been invested in developing health

technology in specific clinical and non-clinical settings.However, other less well resourced organisations (i.e. smalllocal hospitals) have not been able to embrace the fullbenefits from health technology [27]. Vastly differing levelsof health technology maturity therefore exist across thenational healthcare environment, and this is an importantfactor for understanding levels of skills and capabilities ofhealth professionals to procure, implement and use healthtechnologies. Introducing health technology is thereforeclosely related to the contextual situation and this needs tobe factored into any evaluation of health technology [53].

A report by the British Computer Society [54] stressedthat health technology projects should be evaluated as‘business change projects’. The report advised that healthinformatics should consume around 4% of turnover, withproject management emphasising a business led as opposedto technology led approach. While this advice is self-evident, others emphasise the disruptive and unpredictablenature of new technology which makes evaluation moredifficult. Thus, ‘When technologies are disruptive, operat-ing and financial impacts are challenging to estimate, whichmakes it difficult to construct a ‘business case’ forinvestmentydisruptive technologies make it difficult toconduct return-on-investment (ROI) analyses’ [8].

Different stakeholder perceptions and practices of whatconstitutes health technology evaluation have led tosuggestions that current evaluation methods and techniquesare inadequate. The piecemeal approach where somestudies emphasise organisational and cultural aspects, andothers focus on economic factors alone, is too narrow. Arecent paper on EHR evaluation found, ‘y.the strikingfinding from our review is that there has been so little solidevaluation of these applications’ [10]. Evaluation of healthtechnology should therefore integrate quantitative datawith more contextual qualitative approaches from bothclinical and social science [7]. For example, one study foundthe use of computers in European GP practices was 100 percent for Finland and Estonia, but as little as 57% forLithuania [55]. This data is interesting and can be furthersupported by comparative country analysis using otherindicators (i.e. GP computing training, etc.). Anotherexample is the Electronic recording and storage of individualadministrative patient data, with Hungary at 100% andLatvia at only 26% (www.euphix.org). Quantitative data cantherefore be supplemented with contextual data which mayinvolve case studies on specific organisations (i.e. hospitals)to gain a rich picture of the health IT landscape.

The TEMPEST methodology covers three evaluationthemes: eHealth policy; governance, regulation and com-pliance; eHealth adoption/user engagement; and perfor-mance measurement and benefits realisation. Healthtechnology evaluation needs to include both hard and softindicators where incentives for adoption and implementa-tion are considered alongside perverse incentives forstakeholders.

Social

Social aspects in the development and diffusion of healthtechnology relate to issues about inclusion and access forcitizens to health services and products. Politicians are

TEMPEST: An integrative model for health technology assessment 43

aware that the digital divide where inequalities of accessand usage of health technologies is a serious impediment tothe further growth in eHealth. Factors like population size,race, ethnicity, age, disability, life expectancy, nationalincome per capita, and access to healthcare, are allimportant in the drive to promote increased use of healthtechnology. The EU, however, faces serious challenges overthe coming years as the population across the 27 EU MemberStates is projected to increase from 501 million in 2010 to525 million in 2035, peaking at 526 million around 2040. Agradual decline to 517 million is forecast for 2060. Coupledwith population growth, the EU population is expected tocontinue to grow older, with the share of the populationaged 65 years and over rising from 17% in 2010 to 30% in2060, and those aged 80 and over rising from 5% to 12% overthe same period [56] (Eurostat, 2010). Ageing populationsand declining birth rates put pressure on labour marketswith reduced economic output as labour forces decreasewith more people retiring. This increases the burden on theexisting workforce. By 2020, Hungary will have the highestold-age dependency ratio in emerging market economies(EMEs) of 30.1%, up from 24.0% in 2010. Other countries withan old-age dependency ratio over 20.0% in 2020 includePoland and Romania. A decreasing workforce means therewill be a smaller working-age population paying taxesresulting in lower government revenues and lesser potentialfor public spending. Pension spending will have to rise inorder to accommodate for more elderly who will haveincreased health care needs. This will increase the cost ofhealthcare systems as they will need to be upgraded [57].

Efforts to move from established approaches to healthservice delivery, where the patient enters the healthcaresystem, either following a visit to a GP or a hospital, and isthen diagnosed and treated, to one which emphasisesprevention and wellbeing, will require a significant culturalchange [13]. To meet this challenge, EU27 politicians arekeen to promote social inclusion schemes to develop healthinitiatives which measure patient outcomes rather thansimply meeting fixed targets. Fostering health literacy,where citizens take an active role in their health andwellbeing is about social change, as the health care systemis currently built on the presumption of medical profession-alism (i.e. the clinical judgement of the professional) ratherthan on consumerism (i.e. the patient’s right to choose).Changing these embedded cultural traditions will not occurover-night as citizens (patients) have traditionally acted aspassive recipients of the health system, and not activeconsumers of health services. Increasing public awarenessabout prevention and wellbeing by governments will need tobe stepped up, but the growth in global health tourism willfuel this goal, particularly as health professionals andpatients seek cheaper and more efficient solutions to healthproblems. Recent figures show that approximately 4 millioninternational patients per year seek health solutions outsidetheir country, with a worldwide market worth between US$20 and US $40 billion [58]. Medical tourism is furtherestimated to account for around 5% of total tourismworldwide [59].

As health technology products and services continue toenter the marketplace, this will encourage the growth ofpatient-centred healthcare, where citizens increasingly useonline and mobile devices to manage their health needs.

Across the EU, large and small companies are developinghardware and software applications for clinical and patientuses. These include, mobile phone applications to book GP/physician and hospital appointments, receive prescriptionsand test results, access to an online electronic healthrecord, participation in health blogs/chat rooms and otheronline health applications. Studies show the availability anduse of health technology products and services is increasing.However, the differing demographic profiles across the EUwill mean that governments will need to target and tailorhealth technology offerings to meet the needs of the vastmajority. For example, the percentage of the populationaged over 65 in the EU in 2010 shows Germany has 20.6%compared with Ireland at only 11.3% and Slovakia at only12.3%. This suggests that governments and companies needto consider age demographics with health technologiestargeted at the elderly population in Germany, which mayinclude, online monitoring of chronic conditions using ICTand medical devices. For Ireland and Slovakia, thesetechnologies will be less relevant, as the demographicsshow a much larger proportion of the population under65 years. Health and wellbeing programmes facilitated byhealth technologies will be more usefully targeted atmiddle-aged people and younger. Market segmentation forhealth technology products and services will thus need toconsider variations in social factors.

The TEMPEST methodology covers three social themes:social inclusion/access to IT; patient-centred healthcare;and demographics. The adoption and diffusion of healthtechnologies will continue to be patchy as a result ofdiffering access/inclusion and demographic profiles bothacross and within EU Member States. In addition, the driveto make healthcare more ‘patient-centred’ will occur ifhealth literacy increases, encouraged not only by govern-ment campaigns on prevention and wellbeing, but alsothrough the emerging international market in healthtourism.

Transformation

The transformation of healthcare focuses on the macro- andmicro-levels. Across the EU, politicians have increasinglylooked at health technology as a means to transformhealthcare. For the past few decades, ICT, medical devicesand monitoring technologies have been perceived as a magicbullet to drive change in the healthcare industry. All EUMember States have developed eHealth policy, but therelationship between policy formation and implementationis uncertain. The technology sector is also looking to developnew products and services to improve health servicedelivery. Important questions include: How can technologybe used to transform healthcare? What are the ‘bestpractice’ examples?

The TEMPEST data on 27 EU Member States points to vastdifferences in the health systems of individual countries andtheir eHealth status. Politicians, industrialists and aca-demics, alike, continue to promote the potential of healthtechnology to transform the health sector for the benefit ofall citizens. A key goal is to create an integrated or seamlesshealthcare system where patients receive cradle to graveservices, which are meticulously documented and stored on

W.L. Currie44

electronic or personal health records. Yet healthcaresystems across EU Member States are fragmented, with amixture of public and private sector providers and payers.This detracts from an integrated approach since patientdata is recorded across multiple systems, owned andcontrolled by many different healthcare organisations. Evenwithin the same system, i.e. a national healthcare system,patient records are patchy as health events (i.e. visits todifferent healthcare professionals for general appointments(i.e. GP check-ups) and specific treatments (i.e. diagnosis,testing, treatment) will capture data from many differentsources. As patient data is located and stored in manydifferent physical environments, it is likely that thecontextual nature of the data and how it is used byprofessionals also varies [60]. While health technology hasthe potential to improve health service delivery, policy-makers and professionals working in this field need to reducethe barriers, which include: implementation problems fromimmature/untested health technologies (i.e. EHRs), per-verse financial incentives for providers/payers, resistancefrom health executives/clinicians/patients, lack of re-sources, fears about privacy/security/confidentiality (ofpatient data), inertia, lack of IT capabilities and skills,under-developed health infrastructures, resource con-straints, and many others.

To understand the potential of how technology willtransform healthcare, the TEMPEST methodology calls fora wider debate which does not simply advocate reinventingthe wheel (i.e. investing in more studies on health IT) toreproduce existing findings, but to support a researchagenda that encourages the cross fertilisation of academicand practitioner communities of practice involving multiplestakeholders. One study reported the findings from aliterature review of 4,683 titles on health IT between2004–2007. This research found that many studies focusedon patient-focused applications with little formal evalua-tion, descriptions of commercial EHRs and health IT systemsdesigned to run independently from EHRs, and fewer studiesfrom health IT leaders. The authors claim, ‘We found thatalthough predictive analysis suggest that health IT has thepotential to enable a dramatic transformation in healthcare delivery, the empirical research evidence base sup-porting its benefits is limited’ [10] The authors call for morepublic–private partnerships and policies which address the‘mis-alignment of incentives’ to accelerate the adoption ofhealth IT, and for a more robust evidence base for ITimplementation.

The TEMPEST methodology covers three transformationalthemes: education and training; the reform agenda; andeHealth strategy and implementation. Politicians are cur-rently preoccupied with transforming healthcare with muchemphasis on health technology as an enabler in this process.However, the TEMPEST methodology, which aggregates datasets and material from a wide variety of academic,government and professional sources, suggests that trans-forming current healthcare systems needs to take intoconsideration wider socio-economic and political factorswhich go far beyond the technical imperative. For example,the increasing calls to move from diagnosis and treatment toprevention and well-being cannot be achieved throughtechnical change alone but a cultural and behavioural shiftin attitudes and lifestyle. Government campaigns on health

and wellness can be targeted to citizens using the Internetand other mobile technologies, but this needs to beaccompanied by incentives, financial or otherwise. Educa-tion and training thus plays an essential role in transforminghealthcare, and goes far beyond simple instruction on howto use technology. The TEMPEST data identifies indicatorswhich measure government expenditure on prevention andwellbeing, but this will only make an impact if combinedwith large-scale investment to change behaviour and habits(i.e. healthy eating and exercise).

The reform agenda in healthcare further considerschanges to business models [1]. As governments looktowards reducing health costs by treating more patients athome rather than at hospital, R&D expenditure on newproducts and services will be crucial in changing the wayhealthcare is delivered. Citizens are now exposed to theexponential growth in social media sites for health productsand services, most of which is currently unregulated. Inconjunction with the large companies offering technology-enabled healthcare (i.e. online EHRs from Microsoft andGoogle), smaller technology firms will increasingly enter themarket with new systems and applications.

So far, the success of centrally-imposed governmentpolicy to introduce health technology has seen mixedresults. Prior research suggests the relationship betweeneHealth strategy and implementation will also need to bemore closely coupled [61,62] since top-down policy deci-sions often result in poor implementation and unexpectedoutcomes [63]. The TEMPEST methodology thus aims totrack health technologies from policy decisions through tostrategy and implementation to give stakeholders a widerunderstanding of the factors that both encourage or inhibitbest practice in healthcare settings. So an importantindicator in this process is to consider not only the extentof technology diffusion, but also its deployment and use byhealth professionals and patients.

Conclusion

Following an extensive review of the health policy andtechnology literature (which is not exhaustive in this paper)the TEMPEST methodology which combines 7 core cate-gories, 21 themes and 84 indicators/metrics, is developedon the premise that, a one-size-fits-all approach totransforming healthcare using technology is misguided. Thispaper has introduced the core thinking behind the TEMPESTmethodology, giving some examples of key differencesbetween EU Member States. To determine what will workin one country, as opposed to another, it is important to gaina deeper understanding of the contextual factors that mayenhance or inhibit the adoption and diffusion of healthtechnology. While the technical imperative is overstated inmuch of the health technology literature, other factors playa large part in determining success and failure scenarios.These include: differences in health expenditure; demo-graphic conditions; health infrastructure; ICT skill levels;health literacy; clinical and patient engagement; and manyother factors. Understanding key differences within andacross EU Member States is therefore critical to policy-makers and other health and health technology stakeholders[65]. For example, while Denmark, Finland and Sweden are

TEMPEST: An integrative model for health technology assessment 45

considered front-runners in eHealth, they all benefit from arelatively low population size and a mature social and healthinfrastructure. These countries are the same size as manyregions in larger countries. Germany, the UK and France,with well developed social and health infrastructures havemuch larger population sizes, where centrally-imposedhealth and eHealth policies are more difficult to implement.The TEMPEST methodology thus offers a useful tool forcomparative country analysis of EU Member States, usingcross-disciplinary indicators for benchmarking to assistpolicy-makers in their decision-making. The limitations ofthis approach, however, are not insignificant as the relativepaucity of reliable and robust data sets is a major barrier.This is recognised by bodies such as the OECD, who are keento identify key indicators for benchmarking eHealth. Wetherefore conclude with a call to increase and enhance thequantity and quality of data to support policy-making toimprove health service delivery and outcomes in the 21stcentury.

Acknowledgements

I would like to take this opportunity to thank all those whohave supported the study. They include in alphabeticalorder, Jos Aarts, Erasmus University Rotterdam, MargunnAanestad, University of Oslo, Nicola Bedlington, EuropeanPatients’ Forum, David Bevan, European Academy ofBusiness in Society, Nicole Denjoy, COCIR, Jens Dibbern,University of Bern, Gilbert Lenssen, EABIS, ThomasKnothe, Fraunhofer Institute, Tina Blegind-Jensen, Co-penhagen Business School, Patrick Oliver, Accenture,Cristina Vela Marimon, Telefonica, Simon Pickard, EABIS,Nancy Pouloudi, Athens University of Economics andBusiness, M Rajarajan, City University, Loic Sadoulet,INSEAD Centre for Social Innovation, Donald Singer,University of Warwick Medical School/Fellowship of Post-graduate Medicine, Karl Stroetmann, Empirica, SinanTumer, SAP, and Vishanth Weerakkody, Brunel University.I would also like to thank the sponsors, Microsoftcolleagues for spearheading the Information Technology(IT) to Enabling Technology (ET) research initiative. In

Table A1

TEMPEST Theme Indicator/metric

Technology Enabling/emergingtechnologies

T1a: Broadband pT1b: Speed—% oT1c: % of househT1d: % of househ

Interoperability ofeHealth

T2a: % of individuT2b: % of individuwork to access thT2c: Obtaining paconsentT2d: Obtaining pconsent

particular, John Vassallo, Elena Bonfiglioi, Thaima Sam-man and Ray Pinto. Thanks also go to Bruna Guimaraes andRochelle Eng. I would like to thank Professor Dan Hamiltonof the Johns Hopkins University for his excellent coordina-tion and management of the research programme. Thanksalso go to Dr. David Finnegan, for his work on the earlierphases of TEMPEST study. I recognise the help and supportof all the above individuals and look forward to continuingto work with these colleagues.

Appendix A. The TEMPEST methodology

The TEMPEST Health methodology (see Appendix A) isdeveloped by Professor Wendy Currie, Editor-in-Chief ofHealth Policy and Technology, and is a research-baseddecision-making tool to help health stakeholders identifythe opportunities and barriers to transforming healthcarewith enabling technologies. TEMPEST Health is sponsored bythe ‘‘Enabling Technology Coalition,’’ a collaborative group-ing of business, non-governmental organisations and otherstakeholders, initiated by Microsoft and facilitated by JohnsHopkins University. The Enabling Technology Coalition pro-motes research on the potential for enabling technologiesin facilitating European economic growth in four areas:health; low-carbon economy; education; and governance.The research is supported by a coalition of partners, including,Accenture, COCIR, The European Academy of Business inSociety and GE Healthcare. Microsoft leads the coalition on theIT to ET Enabling Technology programme, which supports otherscientific studies co-ordinated by The Johns Hopkins University.The TEMPEST concept was initially developed as a prototypewith quantitative and qualitative indicators (see: http://www.enablingtechnology.eu/ehealth/academic_study). Sincethis initial study, TEMPEST has been extensively revised andnow contains 84 quantitative indicators to provide a morerobust and reliable means for comparative country analysis onhealth technology assessment. Professor Currie is working withthe coalition of partners to apply the methodology to 27 EUMember States (Table A1).

enetration (as % of total population)f broadband subscription above 2 Mbpsolds with an internet connectionolds with a broadband connection

als using a mobile phone via UMTs (3G) to access the Internetals using a laptop via wireless connection away from home/e Internettient consent for data storage and transfer: Oral/Written

atient consent for data storage and transfer: No specific

Table A1 (continued )

TEMPEST Theme Indicator/metric

eHealth servicedelivery model

T3a: Mobile cellular subscriptions (per 100 people) 2008T3b: Fixed Internet Subscribers (per 100 people) 2008T3c: % of population who are regular Internet users (using the Internet at leastonce a week)T3d: % of population who are frequent Internet users (using the Internet everyday or almost every day)

Economic Healthcare funding E1a: Total health expenditure as a % of GDPE1b: General government expenditure on health as a % of total expenditure onhealthE1c: Private expenditure on health as a % of total expenditure on healthE1d: General government expenditure on health as a % of total governmentexpenditure

Performance andpopulation

E2a: GDP growthE2b: GDP per capita ($)E2c: Country populationE2d: Urban population (% of total)

Labour marketsegmentation

E3a: Labour force—Agriculture (%)E3b: Labour force—Industry (%)E3c: Labour force—Services (%)E3d: Telephone lines (per 100 people) 2008

Market Market-drivenhealthcare

M1a: % Practices having websitesM1b: GP practices receiving professional IT supportM1c: Population covered by mobile cellular network (%)M1d: Personal computer (per 100 people)

Consumer-drivenhealthcare

M2a: % of Population seeking health information on injury, disease or nutritionM2b: Electronic storage of individual medical patient dataM2c: Density of physicians (per 10,000 population)M2d: Practising physicians per 100,000 inhabitants

IT marketcapabilities andskills

M3a: Density of nursing and midwifery personnel (per 10,000 population)M3b: Density of densitry personnel (per 10,000 population)M3c: Density of pharmaceutical personnel (per 10,000 population)M3d: % GPs facing interoperability problems in data exchange

Policy eHealth policy P1a: % basic public services for citizens fully available onlineP1b: % basic public services for enterprises fully available onlineP1c: % population using e-government servicesP1d: % enterprises using e-government services

Education andtraining

P2a: Per capita total expenditure on health at average exchange rate (US$)P2b: Per capita total expenditure on health (PPP int. $)P2c: Per capita government expenditure on health at average exchangerate (US$)P2d: Per capita government expenditure on health (PPP int. $)

Institutionalstructure

P3a: Establishments with documented policy, system or action planP3b: Hospital beds (per 10,000 population)P3c: Radiotherapy units (per 1,000,000 population)P3d: Take-up of Internet services: looking for information about goods andservices (% of population)

Evaluation Governance,regulation andcompliance

Ev1a: E-Government web measure indexEv1b: Workers answering ‘very well informed’ to ‘regarding the health andsafety risks related to your job, how well informed would you say you are?’Ev1c: Euro Health Consumer Index (Patient Rights and information, eHealth,Waiting time for treatment, outcomes, Range and reach of services provided,Pharmaceuticals) Total Score and RankEv1d: Bang for the buck score in Euro Health Consumer Index 2009

W.L. Currie46

Table A1 (continued )

TEMPEST Theme Indicator/metric

eHealth adoption/user engagement

Ev2a: Use of computers in European GP practicesEv2b: Electronic recording and storage of individual adm. patient dataEv2c: Computer in consultation roomEv2d: Use of a computer during consultations

Performancemeasurement andbenefits realisation

Ev3a: Internet Users (per 100 people)Ev3b: % of population who have never used the InternetEv3c: Use of the Internet in European GP practicesEv3d: Use of broadband in European GP practices

Social Social inclusion/access to it

S1a: People/sq. mileS1b: Life expectancy at birth, all population, yearsS1c: Proportion of the population assessing their health as good or very goodS1d: Health life years at age 65

Patient-centredhealthcare

S2a: Connecting to different types of health actors (other GPs)S2b: Connecting to different types of health actors (hospitals)S2c: Connecting to different types of health actors (health authorities)S2d: Connecting to different types of health actors (pharmacies)

Demographics S3a: Adult mortality rate (probability of dying between 15 and 60 years per1000 population)S3b: Population aged over 60 (%)S3c: Annual population growth rateS3d: Gross national income per capita (PPP itn. $)

Transformation Education andtraining

Tr1a: Prevention and Public Health Services (% current health exp.)Tr1b: Provision and adm. of public health programs (% current health exp.)Tr1c: % of persons employed with ICT user skillsTr1d: % of persons employed with ICT specialist skills

Reform agenda Tr2a: R&D intensity (R&D expenditure as a % of GDP)Tr2b: R&D expenditure in EUR million by sector of performanceTr2c: Business enterprise R&D expenditure on manufacturing as a percentageof totalTr2d: Business enterprise R&D expenditure on services as a percentage of total

eHealth strategyand implementation

Tr3a: Business enterprise R&D expenditure in EUR millions as a percentage oftotalTr3b: Increase range of goods or services during 2006–2008 as a percentage ofinnovative enterprisesTr3c: Electronic exchange of patient data for at least one purposeTr3d: Electronic exchange of patient data (Lab results from laboratories)

TEMPEST: An integrative model for health technology assessment 47

References

[1] Teisberg EO, Porter ME, Brown GB. Making competition inhealthcare work. Harvard Business Review 1994:131–41.

[2] Kaplan RS, Porter M. How to solve the cost crisis in health care.Harvard Business Review 2011:47–64.

[3] Starr P. Health care reform and the new economy. HealthAffairs 2000;19(6):23–32.

[4] Callaghan GD, Wistow G. Publics, patients, citizens, consu-mers? Power and decision making in primary health care PublicAdministration 2006;84(3):583–601.

[5] OECD. Health at a glance. Europe 2010. [online] /http://www.oecd.org/document/19/0,3746,en_2649_37407_46460563_1_1_1_37407,00.htmlS; 2010.

[6] World Bank. /http://data.worldbank.org/topic/healthS; 2009.[7] Goldsmith J. Technology and the boundaries of the hospital:

three emerging technologies. Health Affairs 2004;23(6):149–57.[8] Coye MJ, Kell J. How hospitals confront new technology. Health

Affairs 2006;25(1):163–73.[9] Reisman D. Health tourism. UK: Edward Elgar; 2010.[10] Goldzweig CL, Towfigh A, Maglione M, Shekelle PG. Costs and

benefits of health information technology: new trends from theliterature. Health Affairs 2009:282–93.

[11] Wilkinson TM. Medical reasoning and the failure of electronicmedical record systems in the United States. InternationalJournal of Clinical Practice 2010;64(7):839–42.

[12] Currie WL. Integrating healthcare. In: Currie W, Finnegan D,editors. Integrating healthcare with ICT. Radcliffe; 2009. p. 1–35.

W.L. Currie48

[13] European Commission. European economy. Joint report onhealth systems. Occasional papers, vol. 74. Report prepared bythe European Commission and the Economic Policy Committee(AWG); December 2010.

[14] Greenhalgh T, Robert G, MacFarlane F, Bate P, Kyriakidou O,Peacock R. Storylines of research in diffusion of innovation: ameta-narrative approach to systematic review. Social Scienceand Medicine 2005;61:417–30.

[15] de Lusignan S, Aarts J. UK’s national programme for ITwelcomes recommendation for a more sociotechnical approachto evaluation: a commentary on the Greenhalgh evaluation ofthe summary care record. Information for Primary Care2008;16(2):75–7. [England].

[16] Diamond CC, Shirky C. Health information technology: a fewyears of magical thinking. Health Affairs 2008:W383–90.

[17] Draborg E, Gyrd-Hansen D, Poulsen PB, Horder M. Internationalcomparison of the definition and the practical application ofhealth technology assessment. International Journal of Tech-nology Assessment in Health Care 2005;21(1):89–95.

[18] Hammond WE, Bailey C, Boucher P, Spohr M, Whitaker P.Connecting information to improve health. Health Affairs2010;29(2):284–8.

[19] Thomas RL. Learning the alphabet of healthcare IT. [online]/http://findarticles.com/p/articles/mi_m3257/is_3_60/ai_n16119129/S; 2006.

[20] Kohn LT, Corrigan JM, Donaldson MS. To err is human: building asafer health system. Washington, DC: National AcademiesPress; 1999.

[21] Carter JH. Electronic health records.ACP Press; 2008.[22] Health Reform Summit Committee. HRSC on finance. United

States Senate (opportunities to increase efficiency in health care:statement by Peter R. Orszag). /http://www.cbo.gov/ftpdocs/93xx/doc9384/06-16-HealthSummit.pdfS; June 16 2008.

[23] DesRoches CM, Campbell EG, Rao SR, Donelan K, Ferris TG,Jha A, et al. Electronic health records in ambulatory care: anational survey. The New England Journal of Medicine 2008;359(1):50–60.

[24] DesRoches Catherine M, Campbell Eric G, Rao Sowmya R,Donelan Karen, Ferris Timothy G, Jha Ashish, Kaushal Rainu,Levy Douglas E, Rosenbaum Sara, Shields Alexandra E,Blumenthal David, /http://www.themedica.com/articles/2009/06/europe-medical-equipment-indus.htmlS.

[25] Economist Intelligence Unit. The future of healthcare inEurope; 2011.

[26] Halamka JD. Making smart investments in health informa-tion technology: core principles. Health Affairs 2009(March):W385–9.

[27] Brennan S. The N.H.S. IT project. Radcliffe Publishing Ltd.;2005.

[28] Wanless D. Securing our future health: taking a long-term view.Final report of an independent review of the long-termresource requirement for the NHS. London. /www.hm-treasury.gov.uk/Consultations_and_Legislation/wanless/consult/wanless_final.cfm. 2002S; April, 1995.

[29] Moore M. Creating public value. MA: Harvard Business SchoolPress; 2006.

[30] Bloomfield BP. The role of information systems in the UKnational health service: action at a distance and the fetish ofcalculation. Social Studies of Science 1991;21:701–34.

[31] Currie W, Finnegan D, editors. Integrating healthcare with ICT.UK: Radcliffe; 2009.

[32] DesRoches CM, Campbell EG, Vogeli C, Zheng J, Sowmya R,Rao SR, et al. Electronic health records’ limited successsuggest more targetted uses. Health Affairs 2010;29(4):639–46.

[33] Esener. /http://www.hse.gov.uk/statistics/european/tables.htm#table11S; 2009.

[34] Currie WL, Guah MW. A national program for IT in theorganizational field of healthcare: a example of conflicting

institutional logics. Journal of Information Technology2007(September):1–13.

[35] Currie WL, Guah MW. IT-enabled healthcare delivery: the UKNational Health Service. Information Systems Management2006(Spring):7–22.

[36] Greenhalgh T, Wood GW, Bratan T, Stramer K, Hinder S.Patients’ attitudes to the summary care record and health-space: qualitative study. British Medical Journal 2008(June):1–11.

[37] National Audit Office. The national programme for IT in theNHS: an update on the delivery of detailed care recordssystems. Report by the comptroller and auditor general. HC888. Session 2010–2012; 18 May 2011.

[38] Svorronos T, Mate KS. Evaluating large-scale health pro-grammes at district level in resource-limited countries.Bulletin of the World Health Organization (Policy and Practice)2011;89:831–7.

[39] Currie WL. The art of justifying new technology to topmanagement. Omega; 1989.

[40] Gaunt N. Initial evaluation of patient interaction with electronichealth record. South and West Devon Community ERDIP; 2001.

[41] Walker JM, Carayon P. From tasks to processes: the case forchanging health information technology to improve healthcare. Health Affairs 2009;28(2):467–77.

[42] Brown PJB, Warmington V. Data quality probes—exploiting andimproving the quality of electronic patient record data andpatient care. International Journal of Medical Informatics2002;68:91–8.

[43] Gagnon M-P, Scott RE. Striving for evidence in e-healthevaluation: lessons from health technology assessment. Journalof Telemedicine and Telecare 2005;S2(Suppl. 2):34–6.

[44] Heathfield H, Pitty D, Hanka R. Evaluating informationtechnology in healthcare: barriers and challenges. BritishMedical Journal 1998;316:1959–61.

[45] Rossi P, Lipsey MW, Freeman HE. Evaluation: a systematicapproach. 7th ed. Thousand Oaks: Sage; 2004.

[46] Currie W. Management strategy for information technology: aninternational approach. London: Pitman; 1995. 310 p.

[47] McNulty T, Ferlie E. Re-engineering healthcare: the complex-ities of organizational transformation. Oxford: OxfordUniversity Press; 2002.

[48] McNulty T, Ferlie E. Process transformation: limitations toradical organizational change within public service. Organiza-tion Studies 2004;25(8):1389–412.

[49] Willcocks L, Currie WL. Pursuing the re-engineering agenda inpublic administration. Public Administration 1997;75(4):617–50.

[50] Currie G, Suhomlinova O. The impact of institutional forcesupon knowledge sharing in the UK NHS: the triumph ofprofessional power and the inconsistency of policy. PublicAdministration 2006;84(1):1–30.

[51] Boaden R, Harvey G, Moxham C, Proudlove N. Qualityimprovement: theory and practice in healthcare. UK: NHSInstitute for Innovation and Improvement; 2008.

[52] Pfeffer J, Sutton RI. Evidence based management. PublicManagement 2005(September):14–25.

[53] Kaplan RS, Norton DP. The balanced scorecared: measures thatdrive performance. Harvard Business Review 2005(July/August).

[54] Doolin B. Power and resistance in the implementation of amedical management information system. Information SystemsJournal 2004(14):343–62.

[55] British Computer Society. The way forward for NHS healthinformatics. A report on behalf of the British computer society(BCS) by the BCS health informatics forum strategic panel;15 December 2006.

[56] Empirica. [online] /http://www.empirica.com/empirica/empirica_en.htmS; 2008.

[57] Eurostat. [online] /http://epp.eurostat.ec.europa.eu/portal/page/portal/health/introductionS; 2011.

TEMPEST: An integrative model for health technology assessment 49

[58] Euromonitor. [online] /http://www.euromonitor.com/health-and-wellnessS; 2011.

[59] Smith RD, Chanda R, Tangcharoensathien V. Trade in health-related services. The Lancet 2009;373:593–601.

[60] Reisman D. Health tourism. UK: Edward Elgar; 2010.[61] Berg M. The contextual nature of medical information.

International Journal of Medical Informatics 1999;56:51–60.[62] Boonstra A, Boddy D, Ball S. Stakeholder management in IOS

projects: analysis of an attempt to implement an electronicpatient file. European Journal of Information Systems 2008;17:100–11.

[63] De Lusignan S. UK’s national programme for IT welcomesrecommendations for a more socio-technical approach toevaluation: a commentary on the Greenhalgh evaluation of thesummary care record. Informatics in Primary Care 2008;16:75–7.

[64] DeVore SD, Figlioll K. Lessons premier hospitals learned aboutimplementing electronic health records. Health Affairs2010;29(4):664–7.

[65] European Commission. Europe’s digital competitiveness report,vols. 1 and 2. SEC; 2009. 1104 p.