health technology assessment - nivel.nl · 1danish centre for health technology assessment &...

1Danish Centre for Health Technology Assessment & 2EUnetHTA Secretariat, National Board of Health

6 April 2010

Prepared for the Working Conference ‘Health Services

Research in Europe’, Thursday 8 & Friday 9 April 2010

Health Technology Assessment 1Camilla Palmhøj Nielsen, 1Tina Maria Funch and 2Finn Børlum Kristensen

Draft report

Health Services Research and Health Technology Assessment – draft report

6 April 2010 2

Disclaimer: The research leading to these results has received funding from the European

Community's Seventh Framework Programme (FP7/2007-2013) under grant agreement

n°223248. Sole responsibility lies with the authors and the European Commission is not

responsible for any use that may be made of the information contained therein.


6 April 2010 3

Contents

Health Technology Assessment ................................................................................................. 5

Introduction.................................................................................................................................. 5

Health technology assessment (HTA)................................................................................ 5

HTA and Health Services Research................................................................................... 5 Objective ............................................................................................................................. 6

Methods .............................................................................................................................. 6

Overview over the literature ........................................................................................................ 9

Existing Research....................................................................................................................... 15

The content of analysis in HTA................................................................................................. 15

Economic evaluation......................................................................................................... 16 Ethics................................................................................................................................. 17

Legal aspects.................................................................................................................... 18 Organisational aspects ..................................................................................................... 19

Social aspects................................................................................................................... 19

Best practice in undertaking HTA..................................................................................... 20

HTA products ............................................................................................................................ 21

Horizon scanning/early warning ....................................................................................... 22

Rapid assessment ............................................................................................................ 23 Mini HTA ........................................................................................................................... 23

Core HTA .......................................................................................................................... 23 Adaptation toolkit .............................................................................................................. 24

Life cycle perspectives of health technologies ......................................................................... 24

HTA and specific groups of technologies ................................................................................. 25

Pharmaceuticals ............................................................................................................... 25 Screening .......................................................................................................................... 26

Development of HTA capacity and HTA programmes ............................................................. 27

HTA Capacity .................................................................................................................... 28

HTA Programmes ............................................................................................................. 28

Priority setting ................................................................................................................... 29 Policy-HTA links................................................................................................................ 29

Input for future research agenda .............................................................................................. 32

Closing remarks.......................................................................................................................... 33


6 April 2010 4

Literature...................................................................................................................................... 33


6 April 2010 5

Health Technology Assessment

Introduction

Health technology assessment (HTA)

The International Network of Agencies for Health Technology Assessment (INAHTA) defines

health-care technology as: “…prevention and rehabilitation, vaccines, pharmaceuticals and

devices, medical and surgical procedures, and the systems within which health is protected and

maintained.” Technology assessment in health care is defined as: “…a multidisciplinary field of

policy analysis. It studies the medical, social, ethical and economic implications of development,

diffusion, and use of health technology”. The EUnetHTA Project has added the following

explanatory clarification that emphasizes the process and aims of an assessment: Health

technology assessment (HTA) is a multidisciplinary process that summarises information about

the medical, social, economic and ethical issues related to the use of a health technology in a

systematic, transparent, unbiased, robust manner. Its aim is to inform the formulation of safe,

effective, health policies that are patient focused and seek to achieve best value. Despite its

policy goals, HTA must always be firmly rooted in research and the scientific method. The

practice of HTA within this definition varies considerably across national settings. It informs

policy- and decision-making in specific political, economic and institutional contexts. In order to

be useful HTA has to be designed with processes and outputs that fit the relevant context.

HTA and Health Services Research

HTA is a field that connects clinical research with health services research (HSR) in order to

provide a broad based input to policy-making and to the development of an evidence-based

health care practice. HTA is not in itself research, but is building on both clinical research and

on health services research. However, HTA is sometimes combined with clinical research

and/or HSR to answer policy questions satisfactorily.

The working definition of HSR used in this report is:

HSR is the multidisciplinary field of scientific investigation that studies how social factors, financial systems, organisational structures and processes, health technologies and personal behaviours affect access to health care, the quality and cost of health care and, ultimately, the health and wellbeing of citizens


6 April 2010 6

In the field of HSR HTA is obviously related to analysis of health technologies, but with strong

links to the full field of HSR in order to analyse different prerequisites for and consequences of

the use of health technologies.

Objective

This chapter provides an overview over health services research (HSR) of relevance for health

technology assessment (HTA) in Europe. Based on review of published research we identified

the trends in health services research in relation to HTA so far. In addition we provide input to a

future research agenda by describing the research called for in the existing literature. In order to

give input to a future research agenda we focus on development of: 1) the content of analysis in

HTA (e.g. analysis of economy, organisation, ethics, legal aspects and social aspects), 2) the

‘HTA products’ developed to adequately meet the needs of policy-makers (e.g. early

warning/horizon scanning, rapid assessment, mini-HTA, core HTA), 3) themes/problems where

HTA should be developed to be able to address them more comprehensibly in future (e.g. fast

evolving technologies, information and communication technologies, the life cycle of

technologies, and link between policy and HTA), 4) development of HTA in relation to specific

technologies, and 5) development of HTA capacity and of HTA programmes. To give input to a

future research agenda we discuss need for development in relation to theoretical approaches

and in research methodologies in relation to HTA. Basically these themes concerns two different

contributions:

HSR as part of HTA methods to analyse consequences of use of technologies

HSR as analyses and improvement of HTA as a tool to provide input to policy-making

Methods

This section describes the methods we applied in this literature review presenting existing

research and input for a future research agenda. At this stage the draft report builds on a

literature review. Later input from the conference will be used in the report


6 April 2010 7

Literature review

In this section, the methods used in the literature review concerning HSR of relevance for HTA

are summarised, including search strategies and inclusion/exclusion criteria used for the

selection of relevant literature.

Initially we piloted different search strategies to find the strategy with the largest sensitivity and

specificity. In general it was difficult to define a strategy which captured relevant articles without

losing too many. Based on these pilot literature search we decided to search exclusively in

PubMed. This decision was made since the search seemed to catch a large majority of the

relevant literature. However the consequence of this strategy is that we have not specifically

searched in specialised databases for articles concerning the ‘HTA disciplines’ e.g. health

economy, organisation, and ethics, and we therefore risk losing the specialised research within

each discipline. To compensate for this we decided that the only Mesh term used for the

literature search was ‘technology assessment’. This ensured a high sensitivity but a low

specificity of the search, and therefore allowed us to find the large majority of relevant articles

included in the rather broad search. The search was conducted on the 19th February 2009 and

was limited to include articles dating back ten years. The Literature search included 3360

references. These references were scanned at title/abstract level by two researchers and were

in this process reduced to 555 articles. The next step was to review each article and to decide

whether to include or exclude the article. This process reduced the number of relevant articles

to 195 articles. These articles were supplemented with articles from a recent volume of

International Journal of Technology Assessment in Health Care (vol.25, 2009, SUPPL. 2) which

contain 14 articles summarising the findings from the EUnetHTA project. Not all articles are

directly cited in this draft version of the report.

Criteria to select the relevant articles were the following:

Inclusion:

Articles containing HSR within the area of HTA or debating future needs for HSR within HTA. All

languages included.

Exclusion:

Articles which did not include HSR within HTA (clinical research, health impact assessment,

assessment of specific technologies, articles from outside Europe, priority-setting within health

care not specifically concerning HTA).


6 April 2010 8

Articles from outside Europe were excluded from this systematic review, but some were still

used to show the state-of-the art, to put the European HSR into perspective, and to give input to

the section on future research agenda. Within HTA the European and non-European research is

closely related and it is useful to take the non-European research into account when providing

input to a future research agenda.

The systematic literature search was supplemented with grey literature. This includes all

products from the EUnetHTA project (www.eunethta.net) and a number of recent reports

commenting on the development of HTA. These reports were selected because they are

examples of the most recent research within HTA including HSR and because they specifically

address the need for development within HTA including need for research. The included grey

literature is listed in appendix 1.

The articles and the grey literature were analysed and distributed into relevant categories

reflecting the following themes:

The content of analysis in HTA

o Economic evaluation

o Ethics

o Legal aspects

o Organisational aspects

o Social aspects

o Best practice in undertaking HTA

HTA products

o Horizon scanning/early warning

o Rapid assessment

o Mini-HTA

o Core HTA

o Adaptation toolkit


6 April 2010 9

Life cycle perspectives of technologies

HTA and specific technologies

o Pharmaceuticals

o Screening

Development of HTA capacity and HTA programmes

o HTA capacity

o HTA programmes

o Priority setting within HTA programmes

o Links between policy and HTA

Overview over the literature

This part of the report will give a descriptive, graphical overview of the literature included in the

literature review in relation to selected variables. The variables chosen for classifying the

articles included in this rerview were: 1) the nationality of the first author, 2) the number of

nationalities involved in the research 3) the institution(s) involved in the research, 4) the topics

of the research, 5) the journal where the research was published, and 6) the distribution of

articles in the decade 2000-2009.Overall this overview intend to provide a visual overview of

characteristics of the body of research undertaken in relation to HTA.


6 April 2010 10

Nationality of first author

0

20

40

60

80

100

120

Brittis

hIri

sh

Germ

an

Dutch

Swedish

Norwegia

n

Danish

Finnish

Frenc

h

Spanis

h

Italia

n

Greek

Israeli

Belgian

Swiss

Austri

an

Hungar

ian

Canadia

nUS

Austra

lian

This diagram provides an overview over which countries are active in publishing research in

relation to HTA, and it is a clear tendency that British research are very active within HSR/HTA.

Also it is shown that a large number of European countries are involved in this field. The

Canadian, US, and Australian articles are included, because there are European co-authors.

Number of nationalities involved in the research


6 April 2010 11

Number of nationalities involved

0

20

40

60

80

100

120

140

160

1 2 3 4 or more

This diagram shows to what extent the research is based on collaboration between researchers

from different countries. Even if most research is based in one country, it is still clear that

collaboration between researchers from different countries is established. This is partly due to a

long tradition of (EU-supported) projects among HTA institutions in Europe.

Types of institutions involved in the research


6 April 2010 12

Institution involved in the research

0

20

40

60

80

100

120

140

160

University HTA institution Consultancy Industry Other

This diagram provides an overview over the scientific environment in which the research was

undertaken. Mixed scientific environment is identified in 88 of 215 cases. Even if the universities

are most active it is remarkable that research in relation to HTA is characterised by involvement

of other actors – primarily HTA institutions. However, also private consultancies and industry

has published relevant research.

Topic


6 April 2010 13

Research topic

0

10

20

30

40

50

60

70

Economics Widerimpacts of

technologies

Best practice HTAproducts

Developmentof HTA

methodology

HTA andspecific

groups oftechnologies

HTA capacityand HTA

programmes

HTA policylinks

This diagram provides an overview over the research topics identified in the analysis of the

literature. The diagram shows the especially research in health economics and in the links to

policy have been conducted during the last 10 years. However, a broad field of topics have

been covered.

Publisher


6 April 2010 14

Journal

0

20

40

60

80

100

120

Intl J

Tec

hnol A

sses

s Hea

lth C

are

Health P

olicy

J Hea

lth S

erv R

es P

olicy

Health E

cono

mics

Eur J

Health E

con

BMJ

EUnetH

TA

NHS R&D H

TA P

rogr

amme

Other

Not surprisingly, the International Journal of Technology Assessment in Health Care is the most

used journal for publication of HSR in relation to HTA, but also other journals or programmes

are represented in the list of publishers.

Year of publication


6 April 2010 15

Year of Publication

0

5

10

15

20

25

30

35

2000 2001 2002 2003 2004 2005 2006 2007 2008 2009

This diagram provides an overview over the number of articles published and the year of

publication of research in relation to HTA. The number of identified articles varies between 12

and 30 pr. year.

Existing Research

This part of the report has the purpose of presenting the content of the identified research. As

earlier indicated the research is categorized into different topics of relevance to HSR in the area

of HTA.

The content of analysis in HTA

This part includes research within the ‘HTA disciplines’ and articles concerning best practice in

undertaking HTA.


6 April 2010 16

Economic evaluation

Economic evaluation is the comparison of two or more alternatives courses of action in terms of

both their costs and consequences. Economists usually distinguish several types of economic

evaluation, differing in how consequences are measured:

Cost minimisation analysis

Cost benefit analysis

Cost effectiveness analysis

Cost utility analysis

Economic evaluation is a well established research discipline with lots of different methods

available. However, in spite of the great amount of research, there is still disagreement

concerning the methods to apply when conducting an economic evaluation as part of an HTA.

Hence, even though the economic aspect might have well established methodologies available,

there is still lack of clarity about recommendations and best practice concerning economic

evaluation in relation to HTA.

This literature review revealed that a lot of research is being done in the area of health

economics in relation to health technology assessment. This was the area in which the greatest

body of literature was identified. In total 60 articles were identified to provide an overview of the

European research in this area. Among these articles 37 reported specific studies and were

relevant to provide an overview of the current research activity in this area, while the rest solely

point to or debate areas of future research.

14 studies concerned different aspects of economic modelling (1-14). 4 studies concerned

different aspects of guidelines of how to carry out economic evaluations and how to present

them (1-4). 5 studies concerned cost-effectiveness analysis or different aspects of such analysis

(10, 15-21). 3 studies concerned methods of how to estimate learning effects, that is the

consequences of increasing effectiveness over time (22-24). 4 studies concerned the

assessment of effects. One study concerned the assessment of utility (25-28), one study

concerned the assessment of quality of life (5), one study concerned assessments of

willingness to pay (6), and one study concerned discrete choice experiments (29). Finally, 4

studies concerned the use of economic evaluations and the influence on decision making (30-

33).


6 April 2010 17

In addition EUnetHTA has contributed to analysis of how provide core elements of an economic

analysis which can be used to adapt to different settings (7-11).

Thus, this literature review showed, that economic modelling, guidelines, cost-effectiveness

analysis, the effects of learning curves, estimation of effect and utility, the use of economic

evaluation, influence on decision making, and use of economic analysis in different settings

were the themes that make up the majority of the current research activity dealing with

economic evaluation in relation to HTA.

Other studies dealt with the use of systematic reviews and evidence and the assessment of

uncertainty.

The studies primarily applied quantitative methods or review quantitative methods. Those

studies, that concern the use of economic evaluations and the influence on decision making,

were the only studies that apply qualitative methods or a combination of qualitative and

quantitative methods.

The studies were predominantly conducted in university environments. Also, the research

activity was most prevalent in the United Kingdom.

Additionally, this review revealed that a lot of research is also undertaken in especially Canada

and US. This body of research overall reflected the same research themes as the current

European research.

Ethics

The ethical domain of HTA deals with prevalent morals, values and behavioral models of the

society relevant for the technology. It has been argued that ethics should be part of HTA since

its inception in the beginning of the 1970’ies. However, today there is still little agreement on the

methods for integrating ethics in HTA, and only few HTAs report on these issues.

A variety of methods to integrate ethics in HTA exits, e.g. causistry, coherence analysis, and

principlism. Both qualitative methods taken from existing research disciplines and methods

developed specifically for the integration of ethics in HTA were applied.

Working groups within both INAHTA and HTAi have tried to come to some agreement of which

methods that are most appropriate to approach the ethical issues, and most recently EUnetHTA


6 April 2010 18

developed methodology to address the ethical domain as part of the Core model. However, still

only few HTAs address these issues.

6 articles relevant for the ethical domain were identified in the literature search.

4 articles were methodological articles that provided and addressed specific methods of how to

include the ethical issues in HTA (12-15). Also relevant for the consideration of ethics, 2 articles

address criteria for how qualitative research should be evaluated (16, 17). 2 articles discuss why

ethics should be part of HTA and the reasons for the lack of its inclusion (18, 19). In EUnetHTA,

ethics were included in the core model (7-11).

Several articles addressed ethics and discussed the opportunities and challenges in relation to

specific technologies that are considered controversial or ethically complex (e.g. genetic

screening).

All of the articles use qualitative methods or discuss qualitative methodology.

The research is mainly undertaken at universities.

In addition, this literature search revealed that research and debate concerning the ethics in

HTA is undertaken in Canada and Australia.

Legal aspects

The legal aspects of HTA address legislation questions concerning the technology under

assessment. For an issue to be considered a legal one, one must be able to point out the legal

source (stipulation, convention or agreement) that makes the issue legally relevant. Also, this is

what separates the legal domain from domains concerning ethics, social issues and safety.

No articles explicitly focusing on the legal aspects were identified in the literature review. Only

one European article address the role of legal aspects in relation to genetic screening. The

article proposes an Internet citizens’ jury as a method to address ethical, societal and legal

implications (34). Additionally one Canadian study also focusing on the legal aspects in relation

to genetic screening was identified in the literature (35). EUnetHTA included legal aspects in the

core model (7-11).


6 April 2010 19

Organisational aspects

The organisational aspect of HTA focuses on the delivery modes of the assessed technologies.

This includes aspects of e.g. the management, financing and controlling issues, and can thus

contribute to assessments by clarifying challenges and barriers in implementing health

technologies.

One European article identified in the literature search addressed the organisational aspects of

HTA. The authors aimed to make a case for a greater emphasis in research on how health

services are managed, organised and delivered, and discuss the theoretical differences

between and within disciplines, and discuss their implications for research methods (20). In

addition organisational issues are incorporated into the EUnetHTA core model (7-11) and the

adaptation tool kit (21-25)

One article underlined the future need for inclusion of organisational analysis in HTA due to

demand from policymakers (36).

Social aspects

The literature revealed a great deal of mixing of different concepts such as social aspects and

patient perspective under this heading. According to EUnetHTA the social domain of HTA takes

the patient as a point of departure in the analysis of the manifold social implications of a health

technology. The analysis should reveal the resources needed when using a technology and the

consequences of its use in the life of the patients.

No European articles focusing explicitly on the assessment of social aspects were identified in

the literature review. One Danish study presented a research design including a citizens’ jury

that aims at exploring ethical, social and legal implications (34).

However, two Belgian studies concern the use of qualitative research in HTA, how knowledge of

the nature of this type of research can be improved, become more trustworthy and how the

contribution can be increased and criteria for judging this type of research (16). EUnetHTA has

included social aspects as part of the core model and introduces a broader range of literature on

this topic, which has not been included in this review (7-11).

EUnetHTA included social aspects in the core model (7-11).


6 April 2010 20

Additionally, three Canadian studies, two American studies, and one Australian study focused

on the social aspects, how to assess the social issues, how to incorporate the assessment in

HTA and how to evaluate methods (26-29).

Best practice in undertaking HTA

Seven articles analysed and debated best practice in undertaking HTA in general. One article

from the ECHTA/ECAHI project describes the current best practice in undertaking and reporting

HTA (37). One articles described the time-trends in health technology assessment from 1989 to

2002 and thereby gave an overview over the content of HTA reports (38) Another gave en

overview over cross-national HTA processes (39), while a further article gave an international

comparison of the definition and practical application of HTA (40). An statistical analysis

analysed what factors that influenced assessment methods in HTA (41). On article discussed

key principles for the improved conduct of HTA (42), while the last identified article debates

advantages and disadvantages of standardization of evidence requirements for HTA (43).

In addition the EUnetHTA products – mainly the core model - could be seen as a further

implementation of the conclusions from the ECHTA/ECAHI project, and contributed to

developing a more standardized form for reporting HTA (7-11).

Transparency

As part of the discussion on best practice, a lot of attention has been given to transparency

issues – both in relation to use of methods and to reporting of HTA. An example of this is the

INAHTA checklist which promotes different requirements for creating increased transparency in

the reporting of HTA (30).

As the implementation of HTA in EU member states increased, this led to a series of EU funded

projects from 1993-2002, including EUR-ASSESS and the ECHTA/ECAHI. The

recommendations of these projects are currently being implemented through the EUnetHTA

project, and include a core model of HTA reports and an adaption toolkit to expand proper

sharing and production of information – including promoting transparency.


6 April 2010 21

Five articles dealing with issues of transparency were identified. In one article two HTA reports

on the same topic are compared in order to address issues of transparency and variations in

HTA methods (31). The authors concluded, that efforts to guarantee transparency in the original

studies, and in the HTA reports themselves, needed to be taken, as such lack of transparency

may affect policy decisions. Another article presents the INAHTA checklist for HTA reports (30).

The three latter articles were debating articles. One article reviewed and discusses the issues

associated with standardisation taking into account the perspectives of multiple stakeholders

(32), while two articles debates issues of transparency in NICE technology assessments in the

case of drug evaluations (33, 34).

Three reports debated and critiqued the lack of transparency in HTA reports in general and

opted for more focus on transparency issues (44-46)

HTA products

A variety of different types of health technology assessments have emerged, typically from the

work of HTA agencies. A lot of literature addresses HTA production in general. However, a

more limited literature addresses specific ‘HTA products’ which have been developed to meet

the needs of users of HTA.

Mini HTA and rapid assessments are two types of assessments that have emerged as

responses to the existing conflict between traditional thorough HTAs and the need to produce

input to decision making in a timely fashion. Political and administrative decision makers require

a faster production of HTA.

Horizon scanning/early warning is a HTA product that was developed in response to the needs

of assessing technologies in the early phases of adaption before thorough research have been

undertaken.

Core HTA and an adaptation toolkit for ‘translation’ of foreign HTA reports has been develop as

part of the EUnetHTA project with the aim of reducing duplication of work and promoting already

existing foreign reports.


6 April 2010 22

Horizon scanning/early warning

Health technologies in an early stage of its life cycle sometimes spread rapidly in a health care

system despite lack of convincing evidence. Also, new technologies exist, that may be

underused resulting in lack of benefit for the patients.

Horizon scanning systems/early warning systems is a method evolved from the work of HTA

agencies. The International Information Network on New and Emerging Health Technologies

(EuroScan) is an international collaboration established in 1999. EuroScan defines the focus of

horizon scanning as those technologies that are not yet adopted by the healthcare system and

those that are in the phase of adoption. The purpose of horizon scanning is to provide input to

decision making that is timely and relevant.

The members of EuroScan have agreed on a common terminology, classification and

understanding of their activities. Their activities consist of five main components which are the

identification and filtering of technologies, prioritization, early assessment, dissemination, and

monitoring the assessed technologies.

Nine articles concerning different aspects of horizon scanning were identified. One of these

articles reported from a workshop and subsequent discussions, that reviewed the achievements

and progress of Euroscan (35), and one article provided an overview of processes and practices

of horizon scanning (36). Six articles evaluate different aspects of what constitutes a proper

horizon scanning. This includes the use of selection criteria in horizon scanning, the sources to

search in the process of identifying technologies, and the effectiveness and accuracy of HSS

(37-40). Two articles explore aspects of adaption of established horizon scanning to Danish

horizon scanning (41, 42). Finally, one article explores the intrascientific citation (bibliometric

impact) of the Swedish early warning reports (SBU Alerts) and the science base of these reports

(43).

EUnetHTA produced a newsletter on new and emerging health technologies for European

policy makers in collaboration with Euroscan (44). The process was reported in an article which

described the process of producing a newsletter (45).


6 April 2010 23

Rapid assessment

Rapid assessment is a tool to produce input to decision making in a timely fashion. The rapid

assessments are typically carried out by HTA agencies within a timeframe of 6 months or less.

A number of programs exist. However, they vary in scope and methods. At this point, no

common definition and methodology of rapid assessment exists and the quality of the

assessments varies.

The literature review identified only two European articles concerning rapid assessments.

One German article compares different rapid assessment programs with respect to scope,

methods and time to complete assessments, and introduces and discusses a model for

processing rapid assessment in the German context (46). Likewise, a British study describes a

method for the rapid appraisal of new interventional procedures and compares its conclusions

with those derived from a slower, more thorough method (47).

Additionally, one Canadian article and one Australian article were identified concerning the

impact and methods of rapid assessments (48, 49).

Mini HTA

Mini HTA is a tool mainly used in hospital settings when making decisions about the uptake of

new technologies. It typically consists of a form containing a number of questions corresponding

to the domains of a thorough HTA. This tool aims at providing input to decision making within

few weeks adjusted to the local settings.

Only two articles concerning mini HTA were identified in the literature search (50, 51). Both

studies were Danish studies undertaken by Danish University Hospitals. Both studies evaluated

the use of HTA and the attitudes of decision makers towards the tool. The studies revealed that

different versions of the tool, and with varying quality, are commonly used in hospital settings in

Denmark.

Core HTA

Core HTA built on recommendations from previous European HTA projects and developed a

specific structure for undertaking and reporting HTA. The main aim was to avoid duplication of


6 April 2010 24

HTA reports in Europe since there are many examples of reports on technologies produced

synchronically across Europe – and also across the rest of the world.

The core model was reported in five different publications. One model for medical and surgical

prcedures (7), and one for diagnostic technologies (8). A handbook introducing the model was

published (9), and two articles presented the idea behind the core model (10), and the testing of

the model during the development phase (11).

Adaptation toolkit

EUnetHTA also developed an adaptation toolkit. This product was a further development of an

already existing activity among the HTA agencies, which used foreign HTA reports and

‘translated’ them into the relevant setting. The aim to avoid as much works as possible by

reusing e.g. the systematic review from the foreign report. EUnetHTA worked to structure the

activity by publishing a glossary of HTA adaption terms (22, 23). Also the adaptation toolkit itself

(21) and two additional articles described the development of the toolkit (24, 25).

Life cycle perspectives of health technologies

Many existing healthcare interventions diffused before the establishment of the current

assessment and evaluation procedures. Assessment of ineffective or inappropriately applied

practices is growing as a priority for international health policy, both for improved quality of care

and sustainability of resource allocation. It might be difficult for assessments to have an impact

on an established practice (52). On the other hand, assessments of resource consuming

technologies that remain in use are needed in order to determine disinvestments and to improve

the overall quality of care and sustainable resource allocation (53-55).

The literature review identified four articles concerning aspects related to the assessment of

technologies already in use including disinvestments. One of these articles introduced a method

of constructive medical technology assessment that aim to change the development and

diffusion of a medical device to improve its later clinical effectiveness, illustrated by the case of

heart assist devices (52). The three other articles discussed disinvestment in relation NICE

evaluations. These articles pointed out, that NICE had a tendency to focus on the assessment

of new technologies while the assessments of existing technologies are given less priority (55).


6 April 2010 25

This concerned the aim of the establishment of NICE which partly was to ensure the overall best

use of resources in the NHS so that patients receive the greatest benefit (54). Hence, NICE

recommendations may not necessarily increase the overall efficiency of resource allocation

decisions in the NHS due to lack of focus on the full life cycle of health technologies (54). This

dilemma called for an overall increased focus on the assessment of technologies already in use

(55). One article added to this debate by exploring the policy options available to NICE as it

prepared to launch a programme to meet the NHS request for guidance on disinvestment. All of

the possible options present different challenges, as NICE will need to collaborate in new ways

with partners inside, and perhaps outside, the NHS (53).

EUnetHTA also focused on the life cycle perspective in relation to assessment of technologies,

but so far concentrated on the earlier stages of the life cycle by focusing on evidence generation

on promising health technologies (56-58). However the core model encouraged that new

technologies should always be compared to existing technologies, and the model can also be

used to assess possible obsolete technologies.

HTA and specific groups of technologies

In this section two groups of technologies are addressed. The first group concern primarily

pharmaceuticals that possess challenges to HTA in terms of the ability to provide input to

decision making in a timely matter and address the literature that concern Coverage with

Evidence Development (CED) as an option to overcome this obstacle. The other group concern

screening methods that represent a group of technologies that in many cases by practitioners

and to the general public is received as harmless.

Pharmaceuticals

To provide assessments of technologies as an input to a rational decision making process in a

timely matter is always a challenging task. A sufficient amount of studies for thorough

assessments are usually not available until the technologies already are in use. On the other

hand technologies that are in early phases of their lifecycle may lack sufficient studies for

thorough assessments which affect the quality of assessments. Hence, the balance between


6 April 2010 26

too early and too late assessment of technologies is a challenge for HTA (see also paragraph

on life cycles).

The cases of pharmaceuticals often make up a great body of this discussion especially in

relation to the evaluation of the economic benefits of new pharmaceuticals. Resulting from this

discussion, the possibility of providing pharmaceuticals while ongoing evaluation is still

undertaken has been put forward. This has been referred to as Coverage with Evidence

Development (CED) and refers to the reimbursement of pharmaceuticals given that an ongoing

evaluation of the benefits is undertaken.

Even though the possibilities of providing pharmaceuticals in terms of CED, the issue can be

generalized to other health technologies, as the challenge to provide assessments of

technologies as an input to a rational decision making process in a timely matter relates to all

new technologies. CED gives an opportunity to provide an input to the decision making process,

but with certain reservations.

4 articles concerning CED were identified in the literature search. One article address the

tension between pharmaceutical regulators and healthcare reimbursement authorities and CED

as an option to overcome this issue (59). One article discusses CED as a possibility for NICE

recommendations to the NHS (60). One study explores the conceptual and policy issues in

general relation to CED and discusses issues involved in operationalizing CED in practice and

presents criteria for which technologies may be most suitable for CED (61), and finally one

article comments on this study (62).

Screening

Screening is the application of a test to detect potential disease or condition in a person who

has no known signs or symptoms of that disease or condition. The goal of screening is the early

detection of disease or risk factors of disease so that intervention can reduce morbidity and

mortality from the involved disease (63). Methods of screening represent a group of

technologies essentially different from clinical practice. The main issues being that screening is

carried out in apparently healthy persons, so the moral duty is essentially different from that of

typical clinical practice (64). Also the technologies are typically received as harmless by

practitioners and the general public resulting in a growing demand for and uncontrolled use of


6 April 2010 27

these technologies. In some cases traditions of use of screening technologies is a challenge

(65).

In the literature review a series of articles concerning HTA and screening were identified carried

out as a specific theme in IJTAHC. The objectives of these articles were to explore the impact of

HTA on health policy and practice in the case of preventional technologies in nine different

countries illustrated by three cases of screening: Mammography screening for breast cancer,

screening for prostate cancer, and the use of ultrasound screening in pregnancy (63-71).

Mammography screening represents a technology with a surprising lack of active policies

concerning implementation despite the weigh of evidence that indicates benefits. Only Sweden,

UK and The Netherlands had at the time of the publishing of the articles implemented such

policies (64, 65, 67, 68).

Ultrasound screening in pregnancy represented a technology that had spread despite lack of

assessment showing benefit. The technology was perceived as an appealing, apparently safe

and relatively cheap technology resulting in a pressure on demand side from both patients and

physicians (65).

Screening for prostate cancer presented a technology that (despite no assessment program

supported such screening so far] was supported by many physicians resulting in a growing use

of the technology (65).

An overall concern was the case of opportunistic screening that is when screening tools is used

as diagnostic tools outside the context of a national screening program. This practice was

reported extremely difficult to control (65).

Development of HTA capacity and HTA programmes

An always important issue for the HTA community is to build capacity and develop existing

programmes to ensure sustainability of HTA in Europe. Related themes are 1) how to ensure

that an HTA programme prioritises assessment of the right technologies to ensure as much

benefit of HTA as possible, and 2) how to improve the links between policy and HTA to improve

the impact of HTA.


6 April 2010 28

HTA Capacity

Internationally there is a growth in HTA activities. However, despite this growth many European

countries have none or only limited HTA capacity, especially low and middle income countries.

There are several obstacles to the introduction of HTA. In many countries, there is a general

lack of awareness of HTA and the manpower to conduct assessments is insufficient. Hence, an

important aspect of capacity building is the establishment of educational programs to promote

training and knowledge in HTA. Moreover data availability in these countries is low and the

quality and validity of morbidity and mortality data is questionable. For these reasons,

developing countries are dependent on studies carried out in other countries. However, while

some data might be transferable, such as clinical and epidemiological data, resource utilization,

costs, cost-effectiveness data, prices, ability and willingness to pay vary from country to country.

These issues are particularly relevant due to the recent uptake of several new member

countries in the European Union. The literature review revealed, that some efforts, European

and international, are undertaken in areas of capacity building. Furthermore, one of the main

goals of the EUnetHTA Collaboration is to increase transparency and transferability of HTAs.

The literature review revealed that European initiatives of capacity building are undertaken in

the case of Hungary (72, 73) and Estonia (74).

As a part of the ECHTA project, Working Group 5 provided an overview of current HTA

education and training programs in Europe revealing that education programs are limited in the

EU12 countries. Additionally, the working group developed a curriculum of a European Master

of science in HTA (75). Likewise, the Ulysses Program, a training program developed in

corporation between Canadian, Spanish and Italian HTA agencies, aimed at evaluators who will

produce HTA and decision makers who will use HTA (76).

EUnetHTA published a handbook on capacity building to support the development of HTA in

Europe (77).

HTA Programmes

Also the description and analysis of different organizational aspects of HTA programs seem to

be of increasing interest. The particular organizational features of the body producing HTA


6 April 2010 29

reports may influence the diffusion of knowledge and the effect of those activities on policy

making.

The literature search revealed two European articles concerned with the performance and

description of HTA programs. One article focused on the differences and similarities of HTA

agencies {448}, the other article analyse what constitutes an “optimal” HTA program {3070}.

EUnetHTA published a survey on characteristics of existing HTA organisations to provide

information on how to develop vigorous HTA organisations in Europe (78).

Priority setting

The resources for HTA within existing HTA programmes fall short of that needed to evaluate all

health care technologies. Therefore, for resources to be used cost effectively, priorities are set

by the HTA agencies concerning which technologies to assess. Several methods exist to make

this prioritization, and some are more transparent and systematic than others. Different HTA

agencies apply different methods for priority setting.

In total four European articles concerning different aspects of priority setting were identified in

the literature search. Two articles evaluated the methodology of two priority setting approaches

in UK and the Netherlands respectively (79, 80), and one article described and evaluated the

relative importance of the different sources used for priority setting by the NHS HTA Programme

(81). Finally one study aimed at developing a method (PATHS) for economic evaluation and at

the stage of research prioritization (82).

PolicyHTA links

The development of links between policy-making and HTA has been a built-in theme in relation

to HTA since the first HTAs were produced in the 1970ties. This was a result of the fact that

HTA was ‘invented’ to give input to policy-making at different levels. Thus development of and

research in the policy loop - ‘policy-HTA-policy’ - is an important theme since the utilisation of

HTA is necessary to justify the use of resources on production of HTA. In short…. if policy-

makers do not see the benefit of HTA and find it useful when making decisions, HTA is not

viable. Therefore researchers and HTA institutions has shown a growing interest in developing


6 April 2010 30

HTA so that it is useful to policy-makers and in documenting the impact of HTA on policy-

making.

44 contributions (articles, reports, book chapters) were identified which deal with the links

between policy and HTA.

A total number of 11 publications addressed the policy-HTA links in a UK setting. In one

comprehensive report the impact of the NHS HTA Programme was assessed (47). The overall

conclusions were that the programme has had considerable impact on knowledge generation,

perceived impact on policy and to some extend on practice. One article evaluated the impact of

a technology appraisal process in the South and West Development and Evaluation Committee

by measuring awareness, influence, and quality of reports among clinical and managerial staff

and concluded that the process was perceived to have an impact on policy decisions among the

staff. However, impact on practice could not be identified with routine data (48). Another article

debated the use of evidence (mainly economic evidence) in the development of local health

policies, and based their conclusions on an in-depth study of Health Improvement Programmes

in England. The questionnaire based study concluded that the main ways of increasing the use

of evidence were to produce more evidence-based national guidance and to disseminate

summaries to local decision makers (49). In addition to these three publications a total of eight

articles analysed the policy-HTA links specifically in relation to NICE processes. Two articles

analysed the implementation and impact of NICE guidance. One looked at impact on GP

prescribing and found only little impact unless the guidance coincided with information from

other sources (50). The other studied the extent and pattern of implementation using multiple

methods and concluded that implementation of the guidance has been variable. However,

guidance was more likely to be implemented when there was strong professional support, a

strong, stable evidence base, no increased or unfunded costs in organisations with good

systems for tracking guidance implementation, and where the professionals involved was not

isolated (51). Two article discussed the general procedures of NICE (52, 53), while two other

article analysed the use of cost-effectiveness analysis/economic appraisal and discusses the

consequences for the use of NICE appraisals (54, 55). One examined concerns from patient

groups in relation to NICE decision-making processes (56), and finally did one article analyse

NICE’s use of cost effectiveness as an exemplar of a deliberative process which include both

scientific context-free evidence about the general clinical potential of a technology, scientific

context-sensitive evidence about particular evidence in realistic scenarios, and colloquial


6 April 2010 31

evidence to into a context and to supply the best evidence short of scientific evidence to fill in

any relevant gaps (57)

A series of articles analysed HTA and its influence on health-care priority setting based on case

studies in England/Wales, France, the Netherlands, and Sweden (58-62). The overall

conclusion on this 2004 overview was that translating HTA into policy is a highly complex

business, and that its influence on policy making remains marginal (58).

Two articles investigated the usefulness and impact of HTA in hospital settings in France and

Italy, and the French article concluded positively that HTA has had significant impact on the

implementation of technologies (63, 64).

Six different articles addressed the questions of impact (65, 66), priority setting for adoption of

health technologies (67), integration of HTA recommandations into organisational and clinical

practice (68), needs of decision makers (69),and lessens learnt from a user perspective in

relation an HTA programme. These studies took place in different European countries.

Furthermore seven articles investigated different themes in relation to the policy HTA link (70-

76).

A series of articles, letters and replies debated the links between policy and HTA by focusing on

why HTA reports building on the same methods and material in relation to PET in oncology led

to different conclusions and policy decisions (77-79). Further the authors discussed the

requirement for documentation in relation to future HTAs of diagnostic methods.

Finally EUnetHTA published a book containing eight separate chapters analysing the policy-JTA

links in Europe. This included reviews of e.g. the impact of HTA on policy-making, the needs

and demands of policy-makers, and an overview of the producers of HTA in Europe (80).

Stakeholder involvement in HTA

Five European articles concerned stakeholder involvement in HTA processes. Two articles

specifically focused on the involvement of consumers. One article gave an overview of

consumer involvement in processes of INAHTA member organisations (81), while the other

describes consumer involvement in the NHS HTA program (82). Both articles described current

status of involvement and particularly the second article emphasized the need for explicit,

inclusive and reproducible methods for supporting consumer involvement. The same theme is


6 April 2010 32

dealt with in a Canadian article which identified what consumer organisations considered

meaningful involvement, what the current practices in Candian HTA processes were, and

developed a model for involvement based on priorities and needs (83). Two articles analysed

stakeholder involvement more generally in NICE processes (84, 85). The articles analysed the

processes where NICE was created and discusses the unique way of involving stakeholders,

but also concludes that there is a need for more flexible approaches to stakeholder involvement

in order to ensure legitimate and transparent processes. Two Canadian articles also analysed

stakeholder involvement – one as a means to improve the impact of HTA (86), and one a

involving the values and judgements of stakeholders in policy coverage decisions (87). The final

article described the processes developed in the EUnetHTA project with the purpose of

involving stakeholders in the further development of European collaboration in relation to HTA.

The article focused on stakeholder involvement in common processes across a large number of

HTA institutions and included European umbrella organisations in the development of a

stakeholder policy (88).

Input for future research agenda

Several inputs for the future research agenda concerning HTA in relation to health services

research in Europe were identified in the literature review.

Some research themes relate to the methodology relevant to the specific elements of HTA,

some relate to ways of improving the influence of HTA on health policy making while others

relate to the assessment of technologies at different stages in their life cycle and assessment of

technologies that challenge the current methodological framework of HTA.

In addition to the outcome of the literature review that is presented in this report, the

achievement of the Health Services Research Europe Working Conference the 8th and 9th of

April will provide even more insight and result in further inputs to this research agenda.

A thorough presentation of the input for future research agenda concerning HTA in relation to

health services research in Europe will thus be provided after the Conference.

However, to provide a basis for the parallel session concerning HTA at the conference, three

agendas will be presented and discussed in more detail at the conference. These are:

Assessing the wider impact of health technologies (e.g. organisational aspects)


6 April 2010 33

Improving the links between HTA and policy

Assessing technologies which challenge the common HTA methodology

Closing remarks

This version of the report should be considered a draft version of a final report on Health

Technology Assessment in relation to Health Services Research in Europe that will be provided

after the Health Services Research Europe Working Conference in Hague.

The finial report will provide a thorough presentation of the future research agenda concerning

HTA in relation to health services research in Europe incorporating the result of the literature

review, the achievements of the Working Conference and finally the results of the stakeholder

survey undertaken as a part of the project.

Literature

1. Philips Z, Bojke L, Sculpher M, Claxton K, Golder S. Good practice guidelines for decision-analytic modelling in health technology assessment: A review and consolidation of quality

assessment. Pharmacoeconomics. 2006;24(4):355-71.

2. Cooper N, Coyle D, Abrams K, Mugford M, Sutton A. Use of evidence in decision models: An

appraisal of health technology assessments in the UK since 1997. J Health Serv Res Policy.

2005 Oct;10(4):245-50.

3. Philips Z, Ginnelly L, Sculpher M, Claxton K, Golder S, Riemsma R, et al. Review of

guidelines for good practice in decision-analytic modelling in health technology assessment.

Health Technol Assess. 2004 Sep;8(36):iii,iv, ix-xi, 1-158.

4. Spiegelhalter DJ, Myles JP, Jones DR, Abrams KR. Bayesian methods in health technology

assessment: A review. Health Technol Assess. 2000;4(38):1-130.


6 April 2010 34

5. Vallejo-Torres L, Steuten LM, Buxton MJ, Girling AJ, Lilford RJ, Young T. Integrating health

economics modeling in the product development cycle of medical devices: A bayesian

approach. Int J Technol Assess Health Care. 2008 Fall;24(4):459-64.

6. Brennan A, Chick SE, Davies R. A taxonomy of model structures for economic evaluation of

health technologies. Health Econ. 2006 Dec;15(12):1295-310.

7. Barton P, Bryan S, Robinson S. Modelling in the economic evaluation of health care:

Selecting the appropriate approach. J Health Serv Res Policy. 2004 Apr;9(2):110-8.

8. Griffin S, Bojke L, Main C, Palmer S. Incorporating direct and indirect evidence using

bayesian methods: An applied case study in ovarian cancer. Value Health. 2006 Mar-

Apr;9(2):123-31.

9. Griffin S, Claxton K, Hawkins N, Sculpher M. Probabilistic analysis and computationally

expensive models: Necessary and required? Value Health. 2006 Jul-Aug;9(4):244-52.

10. Karnon J. Alternative decision modelling techniques for the evaluation of health care

technologies: Markov processes versus discrete event simulation. Health Econ. 2003

Oct;12(10):837-48.

11. Grieve R, Hutton J, Green C. Selecting methods for the prediction of future events in cost-

effectiveness models: A decision-framework and example from the cardiovascular field. Health

Policy. 2003 Jun;64(3):311-24.

12. Bloom BS, de Pouvourville N, Libert S. Classic or bayesian research design and analysis.

does it make a difference? Int J Technol Assess Health Care. 2002 Winter;18(1):120-6.


6 April 2010 35

13. Boas G, van der Stel H, Peters H, Joore M, Anteunis L. Dynamic modeling in medical

technology assessment. fitting hearing aids in the netherlands. Int J Technol Assess Health

Care. 2001 Fall;17(4):618-25.

14. Claxton K, Neumann PJ, Araki S, Weinstein MC. Bayesian value-of-information analysis. an

application to a policy model of alzheimer's disease. Int J Technol Assess Health Care. 2001

Winter;17(1):38-55.

15. Sprague S, Quigley L, Adili A, Bhandari M. Understanding cost effectiveness: Money

matters? J Long Term Eff Med Implants. 2007;17(2):145-52.

16. Eckermann S, Briggs A, Willan AR. Health technology assessment in the cost-disutility

plane. Med Decis Making. 2008 Mar-Apr;28(2):172-81.

17. Adam T, Koopmanschap MA, Evans DB. Cost-effectiveness analysis: Can we reduce

variability in costing methods? Int J Technol Assess Health Care. 2003 Spring;19(2):407-20.

18. IJzerman MJ, Reuzel RP, Severens HL. Pre-assessment to assess the match between cost-

effectiveness results and decision makers' information needs: An illustration using two cases in

rehabilitation medicine in the netherlands. Int J Technol Assess Health Care. 2003

Winter;19(1):17-27.

19. Kristiansen IS, Stavem K, Linnestad K, Pedersen KM. Evaluation of medical methods--can

we rely on cost-benefit-analysis? Tidsskr Nor Laegeforen. 2003 Mar 6;123(5):657-60.

20. Fenwick E, Claxton K, Sculpher M. Representing uncertainty: The role of cost-effectiveness

acceptability curves. Health Econ. 2001 Dec;10(8):779-87.


6 April 2010 36

21. Zivin JG. Cost-effectiveness analysis with risk aversion. Health Econ. 2001 Sep;10(6):499-

508.

22. Ramsay CR, Grant AM, Wallace SA, Garthwaite PH, Monk AF, Russell IT. Statistical

assessment of the learning curves of health technologies. Health Technol Assess. 2001;5(12):1-

79.

23. Ramsay CR, Wallace SA, Garthwaite PH, Monk AF, Russell IT, Grant AM. Assessing the

learning curve effect in health technologies. lessons from the nonclinical literature. Int J Technol

Assess Health Care. 2002 Winter;18(1):1-10.

24. Ramsay CR, Grant AM, Wallace SA, Garthwaite PH, Monk AF, Russell IT. Assessment of

the learning curve in health technologies. A systematic review. Int J Technol Assess Health

Care. 2000 Autumn;16(4):1095-108.

25. Brauer CA, Rosen AB, Greenberg D, Neumann PJ. Trends in the measurement of health

utilities in published cost-utility analyses. Value Health. 2006 Jul-Aug;9(4):213-8.

26. O'Brien BJ, Spath M, Blackhouse G, Severens JL, Dorian P, Brazier J. A view from the

bridge: Agreement between the SF-6D utility algorithm and the health utilities index. Health

Econ. 2003 Nov;12(11):975-81.

27. Feeny D, Blanchard C, Mahon JL, Bourne R, Rorabeck C, Stitt L, et al. Comparing

community-preference-based and direct standard gamble utility scores: Evidence from elective

total hip arthroplasty. Int J Technol Assess Health Care. 2003 Spring;19(2):362-72.

28. Mullahy J. Live long, live well: Quantifying the health of heterogeneous populations. Health

Econ. 2001 Jul;10(5):429-40.


6 April 2010 37

29. Ratcliffe J, Longworth L. Investigating the structural reliability of a discrete choice

experiment within health technology assessment. Int J Technol Assess Health Care. 2002

Winter;18(1):139-44.

30. Williams I, McIver S, Moore D, Bryan S. The use of economic evaluations in NHS decision-

making: A review and empirical investigation. Health Technol Assess. 2008 Apr;12(7):iii, ix,x, 1-

175.

31. Bryan S, Williams I, McIver S. Seeing the NICE side of cost-effectiveness analysis: A

qualitative investigation of the use of CEA in NICE technology appraisals. Health Econ. 2007

Feb;16(2):179-93.

32. Hoffmann C, Graf von der Schulenburg JM. The influence of economic evaluation studies on

decision making. A european survey. the EUROMET group. Health Policy. 2000 Jul;52(3):179-

92.

33. Williams IP, Bryan S. Cost-effectiveness analysis and formulary decision making in england:

Findings from research. Soc Sci Med. 2007 Nov;65(10):2116-29.

34. Moldrup C. Medical technology assessment of the ethical, social, and legal implications of

pharmacogenomics. A research proposal for an internet citizen jury. Int J Technol Assess

Health Care. 2002 Summer;18(3):728-32.

35. Potter BK, Avard D, Graham ID, Entwistle VA, Caulfield TA, Chakraborty P, et al. Guidance

for considering ethical, legal, and social issues in health technology assessment: Application to

genetic screening. Int J Technol Assess Health Care. 2008 Fall;24(4):412-22.

36. Battista RN, Hodge MJ. The evolving paradigm of health technology assessment:

Reflections for the millennium. CMAJ. 1999 May 18;160(10):1464-7.


6 April 2010 38

37. Velasco M, Perleth M, Drummond M, Gurtner F, Jorgensen T, Jovell A, et al. Best practice

in undertaking and reporting health technology assessments. working group 4 report. Int J

Technol Assess Health Care. 2002 Spring;18(2):361-422.

38. Draborg E, Gyrd-Hansen D. Time-trends in health technology assessments: An analysis of

developments in composition of international health technology assessments from 1989 to

2002. Int J Technol Assess Health Care. 2005 Fall;21(4):492-8.

39. Garcia-Altes A, Ondategui-Parra S, Neumann PJ. Cross-national comparison of technology

assessment processes. Int J Technol Assess Health Care. 2004 Summer;20(3):300-10.

40. Draborg E, Gyrd-Hansen D, Poulsen PB, Horder M. International comparison of the

definition and the practical application of health technology assessment. Int J Technol Assess

Health Care. 2005 Winter;21(1):89-95.

41. Draborg E, Andersen CK. What influences the choice of assessment methods in health

technology assessments? statistical analysis of international health technology assessments

from 1989 to 2002. Int J Technol Assess Health Care. 2006 Winter;22(1):19-25.

42. Drummond MF, Schwartz JS, Jonsson B, Luce BR, Neumann PJ, Siebert U, et al. Key

principles for the improved conduct of health technology assessments for resource allocation

decisions. Int J Technol Assess Health Care. 2008 Summer;24(3):244,58; discussion 362-8.

43. Hutton J, Trueman P, Facey K. Harmonization of evidence requirements for health

technology assessment in reimbursement decision making. Int J Technol Assess Health Care.

2008 Fall;24(4):511-7.

44. Kanavos P, Persson U, Drummond M. The future of health technology assessment in

europe. The Initiative for Sustainable Healthcare Financing in Europe; 2009. Report No.: 2.


6 April 2010 39

45. Sorenson C, Drummond M, Kanavos P. Ensuring value for money in health care

the role of health technology assessment in the european union. United Kingdom: World Health

Organisation 2008, on behalf of the European Observatory on Health Systems and Policies;

2008.

46. Sorenson C, Kanavos P, Drummond M. Ensuring value for money in health care: The role of

HTA in the european union. Luxembourg: Financing Sustainable Healthcare in Europe; 2007.

47. Hanney S, Buxton M, Green C, Coulson D, Raftery J. An assessment of the impact of the

NHS health technology assessment programme. Health Technol Assess. 2007 Dec;11(53):iii,iv,

ix-xi, 1-180.

48. Dixon S, Coleman P, Nicholl J, Brennan A, Touch S. Evaluation of the impact of a

technology appraisal process in england: The south and west development and evaluation

committee. J Health Serv Res Policy. 2003 Jan;8(1):18-24.

49. Weatherly H, Drummond M, Smith D. Using evidence in the development of local health

policies. some evidence from the united kingdom. Int J Technol Assess Health Care. 2002

Fall;18(4):771-81.

50. Wathen B, Dean T. An evaluation of the impact of NICE guidance on GP prescribing. Br J

Gen Pract. 2004 Feb;54(499):103-7.

51. Sheldon TA, Cullum N, Dawson D, Lankshear A, Lowson K, Watt I, et al. What's the

evidence that NICE guidance has been implemented? results from a national evaluation using

time series analysis, audit of patients' notes, and interviews. BMJ. 2004 Oct 30;329(7473):999.

52. Paul JE, Trueman P. 'Fourth hurdle reviews', NICE, and database applications.

Pharmacoepidemiol Drug Saf. 2001 Aug-Sep;10(5):429-38.


6 April 2010 40

53. Buxton M. Implications of the appraisal function of the national institute for clinical

excellence (NICE). Value Health. 2001 May-Jun;4(3):212-6.

54. Towse A, Pritchard C. National institute for clinical excellence (NICE): Is economic appraisal

working? Pharmacoeconomics. 2002;20 Suppl 3:95-105.

55. Williams I, Bryan S, McIver S. How should cost-effectiveness analysis be used in health

technology coverage decisions? evidence from the national institute for health and clinical

excellence approach. J Health Serv Res Policy. 2007 Apr;12(2):73-9.

56. Devlin N, Appleby J, Parkin D. Patients' views of explicit rationing: What are the implications

for health service decision-making? J Health Serv Res Policy. 2003 Jul;8(3):183-6.

57. Culyer AJ. NICE's use of cost effectiveness as an exemplar of a deliberative process. Health

Econ Policy Law. 2006 Jul;1(Pt 3):299-318.

58. Oliver A, Mossialos E, Robinson R. Health technology assessment and its influence on

health-care priority setting. Int J Technol Assess Health Care. 2004 Winter;20(1):1-10.

59. Stevens A, Milne R. Health technology assessment in england and wales. Int J Technol


60. Orvain J, Xerri B, Matillon Y. Overview of health technology assessment in france. Int J

Technol Assess Health Care. 2004 Winter;20(1):25-34.

61. Berg M, van der Grinten T, Klazinga N. Technology assessment, priority setting, and

appropriate care in dutch health care. Int J Technol Assess Health Care. 2004 Winter;20(1):35-

43.


6 April 2010 41

62. Carlsson P. Health technology assessment and priority setting for health policy in sweden.

Int J Technol Assess Health Care. 2004 Winter;20(1):44-54.

63. Lettieri E, Masella C, Nocco U. Budgeting and health technology assessment: First evidence

obtained from proposal forms used to submit the adoption of new technology. Int J Technol

Assess Health Care. 2008 Fall;24(4):502-10.

64. Bodeau-Livinec F, Simon E, Montagnier-Petrissans C, Joel ME, Fery-Lemonnier E. Impact

of CEDIT recommendations: An example of health technology assessment in a hospital

network. Int J Technol Assess Health Care. 2006 Spring;22(2):161-8.

65. Britton M, Jonsson E. Impact of health technology assessments. some experiences of SBU.

Int J Technol Assess Health Care. 2002 Fall;18(4):824-31.

66. Oortwijn WJ, Hanney SR, Ligtvoet A, Hoorens S, Wooding S, Grant J, et al. Assessing the

impact of health technology assessment in the netherlands. Int J Technol Assess Health Care.

2008 Summer;24(3):259-69.

67. Shani S, Siebzehner MI, Luxenburg O, Shemer J. Setting priorities for the adoption of health

technologies on a national level -- the israeli experience. Health Policy. 2000 Dec;54(3):169-85.

68. Gagnon MP, Sanchez E, Pons JM. Integration of health technology assessment

recommendations into organizational and clinical practice: A case study in catalonia. Int J


69. Andradas E, Blasco JA, Valentin B, Lopez-Pedraza MJ, Gracia FJ. Defining products for a

new health technology assessment agency in madrid, spain: A survey of decision makers. Int J

Technol Assess Health Care. 2008 Winter;24(1):60-9.


6 April 2010 42

70. Moret-Hartman M, van der Wilt GJ, Grin J. Health technology assessment and ill-structured

problems: A case study concerning the drug mebeverine. Int J Technol Assess Health Care.

2007 Summer;23(3):316-23.

71. Packer C, Simpson S, Stevens A, EuroScan: the European Information Network on New

and Changing Health Technologies. International diffusion of new health technologies: A ten-

country analysis of six health technologies. Int J Technol Assess Health Care. 2006

Fall;22(4):419-28.

72. Hartz S, John J. Contribution of economic evaluation to decision making in early phases of

product development: A methodological and empirical review. Int J Technol Assess Health

Care. 2008 Fall;24(4):465-72.

73. van der Wilt GJ, Rovers M, Straatman H, van der Bij S, van den Broek P, Zielhuis G. Policy

relevance of bayesian statistics overestimated? Int J Technol Assess Health Care. 2004

Fall;20(4):488-92.

74. Hutton J, McGrath C, Frybourg JM, Tremblay M, Bramley-Harker E, Henshall C. Framework

for describing and classifying decision-making systems using technology assessment to

determine the reimbursement of health technologies (fourth hurdle systems). Int J Technol


75. Drummond M, Weatherly H. Implementing the findings of health technology assessments. if

the CAT got out of the bag, can the TAIL wag the dog? Int J Technol Assess Health Care. 2000

Winter;16(1):1-12.

76. May C. Mobilising modern facts: Health technology assessment and the politics of evidence.

Sociol Health Illn. 2006 Jul;28(5):513-32.


6 April 2010 43

77. Hojgaard L. Are health technology assessments a reliable tool in th analysis of the clinical

value of PET in oncology? who audits the auditors? Eur J Nucl Med Mol Imaging. 2003

May;30(5):637-41.

78. Van Tinteren H, Hoekstra OS, Boers M. The need for health technology assessments of

PET. Eur J Nucl Med Mol Imaging. 2003 Oct;30(10):1438,9; author reply 1439.

79. Kristensen FB, Adams E, Briones E, Coburn D, Facey K, Hansen NW, et al. Health

technology assessment of PET in oncology: Re eur J nucl med mol imaging 2003; 30:637-641.

Eur J Nucl Med Mol Imaging. 2004 Feb;31(2):295,7; author reply 297-8; discussion 298.

80. Garrido MV, Kristensen FB, Nielsen CP, Busse R. Health technology assessment and

health policy-making in europe: Current status, challenges and potential. United Kingdom:

World Health Organization on behalf of the European Observatory on Health Systems and

Policies; 2008.

81. Hailey D, Nordwall M. Survey on the involvement of consumers in health technology

assessment programs. Int J Technol Assess Health Care. 2006 Fall;22(4):497-9.

82. Royle J, Oliver S. Consumer involvement in the health technology assessment program. Int

J Technol Assess Health Care. 2004 Fall;20(4):493-7.

83. Pivik J, Rode E, Ward C. A consumer involvement model for health technology assessment

in canada. Health Policy. 2004 Aug;69(2):253-68.

84. Culyer AJ. Involving stakeholders in healthcare decisions--the experience of the national

institute for health and clinical excellence (NICE) in england and wales. Healthc Q.

2005;8(3):56-60.


6 April 2010 44

85. Milewa T. Representation and legitimacy in health policy formulation at a national level:

Perspectives from a study of health technology eligibility procedures in the united kingdom.

Health Policy. 2008 Mar;85(3):356-62.

86. McGregor M, Brophy JM. End-user involvement in health technology assessment (HTA)

development: A way to increase impact. Int J Technol Assess Health Care. 2005

Spring;21(2):263-7.

87. Abelson J, Giacomini M, Lehoux P, Gauvin FP. Bringing 'the public' into health technology

assessment and coverage policy decisions: From principles to practice. Health Policy. 2007

Jun;82(1):37-50.

88. Nielsen CP, Lauritsen SW, Kristensen FB, Bistrup ML, Cecchetti A, Turk E, et al. Involving

stakeholders and developing a policy for stakeholder involvement in the european network for

health technology assessment, EUnetHTA. Int J Technol Assess Health Care. 2009 Dec;25

Suppl 2:84-91.

References

1. Thurston SJ, Craig D, Wilson P, Drummond MF. Increasing decision-makers' access to

economic evaluations: Alternative methods of communicating the information. Int J Technol

Assess Health Care. 2008 Spring;24(2):151-7.

2. Drummond M, Manca A, Sculpher M. Increasing the generalizability of economic evaluations:

Recommendations for the design, analysis, and reporting of studies. Int J Technol Assess

Health Care. 2005 Spring;21(2):165-71.


6 April 2010 45

3. Soto J. Health economic evaluations using decision analytic modeling. principles and

practices--utilization of a checklist to their development and appraisal. Int J Technol Assess


4. Hjelmgren J, Berggren F, Andersson F. Health economic guidelines--similarities, differences

and some implications. Value Health. 2001 May-Jun;4(3):225-50.

5. Stein K, Fry A, Round A, Milne R, Brazier J. What value health?: A review of health state

values used in early technology assessments for NICE. Appl Health Econ Health Policy.

2005;4(4):219-28.

6. Frew EJ, Wolstenholme JL, Whynes DK. Comparing willingness-to-pay: Bidding game format

versus open-ended and payment scale formats. Health Policy. 2004 Jun;68(3):289-98.

7. FinOHTA, Finnish Office for HTA. EUnetHTA HTA CORE MODEL FOR MEDICAL AND

SURGICAL INTERVENTIONS 1.0R WORK PACKAGE 4. Finland: EUROPEAN NETWORK

FOR HEALTH TECHNOLOGY ASSESSMENT; December 2008.

8. FinOHTA, Finnish Office for HTA. EUnetHTA HTA CORE MODEL FOR DIAGNOSTIC

TECHNOLOGIES 1.0R WORK PACKAGE 4. Finland: EUROPEAN NETWORK FOR HEALTH

TECHNOLOGY ASSESSMENT; December 2008.

9. FinOHTA, Finnish Office for HTA. HTA core model handbook. Finland: EUnetHTA; 2008.

10. Lampe K, Makela M, Garrido MV, Anttila H, Autti-Ramo I, Hicks NJ, et al. The HTA core

model: A novel method for producing and reporting health technology assessments. Int J

Technol Assess Health Care. 2009 Dec;25 Suppl 2:9-20.


6 April 2010 46

11. Pasternack I, Anttila H, Makela M, Ikonen T, Rasanen P, Lampe K, et al. Testing the HTA

core model: Experiences from two pilot projects. Int J Technol Assess Health Care. 2009

Dec;25 Suppl 2:21-7.

12. Saarni SI, Hofmann B, Lampe K, Luhmann D, Makela M, Velasco-Garrido M, et al. Ethical

analysis to improve decision-making on health technologies. Bull World Health Organ. 2008

Aug;86(8):617-23.

13. Autti-Ramo I, Makela M. Ethical evaluation in health technology assessment reports: An

eclectic approach. Int J Technol Assess Health Care. 2007 Winter;23(1):1-8.

14. Hofmann B. Toward a procedure for integrating moral issues in health technology

assessment. Int J Technol Assess Health Care. 2005 Summer;21(3):312-8.

15. Moldrup C. Medical technology assessment of the ethical, social, and legal implications of

pharmacogenomics. A research proposal for an internet citizen jury. Int J Technol Assess


16. Leys M. Health care policy: Qualitative evidence and health technology assessment. Health

Policy. 2003 Sep;65(3):217-26.

17. Leys M. Health technology assessment: The contribution of qualitative research. Int J


18. Hofmann BM. Why ethics should be part of health technology assessment. Int J Technol


19. ten Have H. Ethical perspectives on health technology assessment. Int J Technol Assess



6 April 2010 47

20. Fulop N, Allen P, Clarke A, Black N. From health technology assessment to research on the

organisation and delivery of health services: Addressing the balance. Health Policy. 2003

Feb;63(2):155-65.

21. NCCHTA, NIHR Coordinating Centre for HTA. EUnetHTA HTA ADAPTATION TOOLKIT

WORK PACKAGE 5. UK: EUROPEAN NETWORK FOR HEALTH TECHNOLOGY

ASSESSMENT; October 2008.

22. NCCHTA, NIHR Coordinating Centre for HTA.

Glossary of HTA adaptation terms. . November 2007.

23. Rosten C, Chase DL, Hicks NJ, Milne R, European network for Health Technology

Assessment (EUnetHTA). Enhancing understanding: The development of a glossary of health

technology assessment adaptation terms. Int J Technol Assess Health Care. 2009 Dec;25

Suppl 2:42-7.

24. Turner S, Chase DL, Milne R, Cook A, Hicks NJ, Rosten C, et al. The health technology

assessment adaptation toolkit: Description and use. Int J Technol Assess Health Care. 2009

Dec;25 Suppl 2:37-41.

25. Turner S, Chase DL, Milne R, Cook A, Hicks NJ, Rosten C, et al. The adaptation of health

technology assessment reports: Identification of the need for, and development of, a toolkit to

aid the process. Int J Technol Assess Health Care. 2009 Dec;25 Suppl 2:28-36.

26. Street JM, Braunack-Mayer AJ, Facey K, Ashcroft RE, Hiller JE. Virtual community

consultation? using the literature and weblogs to link community perspectives and health

technology assessment. Health Expect. 2008 Jun;11(2):189-200.

27. Bridges JF, Jones C. Patient-based health technology assessment: A vision of the future. Int

J Technol Assess Health Care. 2007 Winter;23(1):30-5.


6 April 2010 48

28. Potter BK, Avard D, Graham ID, Entwistle VA, Caulfield TA, Chakraborty P, et al. Guidance

for considering ethical, legal, and social issues in health technology assessment: Application to

genetic screening. Int J Technol Assess Health Care. 2008 Fall;24(4):412-22.

29. Lehoux P, Williams-Jones B. Mapping the integration of social and ethical issues in health

technology assessment. Int J Technol Assess Health Care. 2007 Winter;23(1):9-16.

30. Hailey D. Toward transparency in health technology assessment: A checklist for HTA

reports. Int J Technol Assess Health Care. 2003 Winter;19(1):1-7.

31. Porzsolt F, Kajnar H, Awa A, Fassler M, Herzberger B. Validity of original studies in health

technology assessment reports: Significance of standardized assessment and reporting. Int J

Technol Assess Health Care. 2005 Summer;21(3):410-3.

32. Hutton J, Trueman P, Facey K. Harmonization of evidence requirements for health

technology assessment in reimbursement decision making. Int J Technol Assess Health Care.

2008 Fall;24(4):511-7.

33. Poole C, Agrawal S, Currie CJ. Let cost effectiveness models be open to scrutiny. BMJ.

2007 Oct 13;335(7623):735.

34. Maynard A. Transparency in health technology assessments. BMJ. 2007 Mar

24;334(7594):594-5.

35. Simpson S, Packer C, Carlsson P, Sanders JM, Ibarluzea IG, Fay AF, et al. Early

identification and assessment of new and emerging health technologies: Actions, progress, and

the future direction of an international collaboration--EuroScan. Int J Technol Assess Health

Care. 2008 Fall;24(4):518-25.


6 April 2010 49

36. Wild C, Langer T. Emerging health technologies: Informing and supporting health policy

early. Health Policy. 2008 Aug;87(2):160-71.

37. Murphy K, Packer C, Stevens A, Simpson S. Effective early warning systems for new and

emerging health technologies: Developing an evaluation framework and an assessment of

current systems. Int J Technol Assess Health Care. 2007 Summer;23(3):324-30.

38. Douw K, Vondeling H. Selection of new health technologies for assessment aimed at

informing decision making: A survey among horizon scanning systems. Int J Technol Assess


39. Simpson S, Hyde C, Cook A, Packer C, Stevens A. Assessing the accuracy of forecasting:

Applying standard diagnostic assessment tools to a health technology early warning system. Int

J Technol Assess Health Care. 2004 Summer;20(3):381-4.

40. Douw K, Vondeling H, Eskildsen D, Simpson S. Use of the internet in scanning the horizon

for new and emerging health technologies: A survey of agencies involved in horizon scanning. J

Med Internet Res. 2003 Jan-Mar;5(1):e6.

41. Douw K, Vondeling H, Oortwijn W. Priority setting for horizon scanning of new health

technologies in denmark: Views of health care stakeholders and health economists. Health

Policy. 2006 May;76(3):334-45.

42. Douw K, Vondeling H, Sorensen J, Jorgensen T, Sigmund H. "The future should not take us

by surprise": Preparation of an early warning system in denmark. Int J Technol Assess Health

Care. 2004 Summer;20(3):342-50.

43. Lundberg J, Brommels M, Skar J, Tomson G. Measuring the validity of early health

technology assessment: Bibliometrics as a tool to indicate its scientific basis. Int J Technol



6 April 2010 50

44. Simpson S, Wild C. On the horizon european newsletter on new and emerging health

technologies. EUnetHTA; 2008.

45. Wild C, Simpson S, Douw K, Geiger-Gritsch S, Mathis S, Langer T. Information service on

new and emerging health technologies: Identification and prioritization processes for a european

union-wide newsletter. Int J Technol Assess Health Care. 2009 Dec;25 Suppl 2:48-55.

46. Perleth M, Luhmann D, Gibis B, Droste S. Rapid assessments"--quick evaluation of medical

technology. Gesundheitswesen. 2001 Mar;63 Suppl 1:S79-84.

47. Warren V. Health technology appraisal of interventional procedures: Comparison of rapid

and slow methods. J Health Serv Res Policy. 2007 Jul;12(3):142-6.

48. Hailey D, Corabian P, Harstall C, Schneider W. The use and impact of rapid health

technology assessments. Int J Technol Assess Health Care. 2000 Spring;16(2):651-6.

49. Watt A, Cameron A, Sturm L, Lathlean T, Babidge W, Blamey S, et al. Rapid reviews versus

full systematic reviews: An inventory of current methods and practice in health technology

assessment. Int J Technol Assess Health Care. 2008 Spring;24(2):133-9.

50. Ehlers L, Vestergaard M, Kidholm K, Bonnevie B, Pedersen PH, Jorgensen T, et al. Doing

mini-health technology assessments in hospitals: A new concept of decision support in health

care? Int J Technol Assess Health Care. 2006 Summer;22(3):295-301.

51. Folkersen J, Pedersen PH. Attitudes to the use of a decision support method when

introducing new medical technology at the university hospital of copenhagen. Ugeskr Laeger.

2006 May 22;168(21):2069-74.


6 April 2010 51

52. Hummel JM, van Rossum W, Verkerke GJ, Rakhorst G. Medical technology assessment:

The use of the analytic hierarchy process as a tool for multidisciplinary evaluation of medical

devices. Int J Artif Organs. 2000 Nov;23(11):782-7.

53. Pearson S, Littlejohns P. Reallocating resources: How should the national institute for health

and clinical excellence guide disinvestment efforts in the national health service? J Health Serv

Res Policy. 2007 Jul;12(3):160-5.

54. Walker S, Palmer S, Sculpher M. The role of NICE technology appraisal in NHS rationing.

Br Med Bull. 2007;81-82:51-64.

55. Maynard A, Bloor K, Freemantle N. Challenges for the national institute for clinical

excellence. BMJ. 2004 Jul 24;329(7459):227-9.

56. HAS, French National Authority for Health. EUnetHTA web-based toolkit to facilitate

european collaboration on evidence generation on promising health

technologies WORK PACKAGE 7-STRAND A. . DECEMBER 2008.

57. Quentin F, Carbonneil C, Moty-Monnereau C, Berti E, Goettsch W, Lee-Robin SH, et al.

Web-based toolkit to facilitate european collaboration on evidence generation on promising

health technologies. Int J Technol Assess Health Care. 2009 Dec;25 Suppl 2:68-74.

58. Carbonneil C, Quentin F, Lee-Robin SH, European network for Health Technology

Assessment (EUnetHTA). A common policy framework for evidence generation on promising

health technologies. Int J Technol Assess Health Care. 2009 Dec;25 Suppl 2:56-67.

59. McCabe C, Claxton K, O'Hagan A. Why licensing authorities need to consider the net value

of new drugs in assigning review priorities: Addressing the tension between licensing and

reimbursement. Int J Technol Assess Health Care. 2008 Spring;24(2):140-5.


6 April 2010 52

60. Chalkidou K, Hoy A, Littlejohns P. Making a decision to wait for more evidence: When the

national institute for health and clinical excellence recommends a technology only in the context

of research. J R Soc Med. 2007 Oct;100(10):453-60.

61. Hutton J, Trueman P, Henshall C. Coverage with evidence development: An examination of

conceptual and policy issues. Int J Technol Assess Health Care. 2007 Fall;23(4):425-32.

62. Tunis SR, Chalkidou K. Coverage with evidence development: A very good beginning, but

much to be done. commentary to hutton et al. Int J Technol Assess Health Care. 2007

Fall;23(4):432-5.

63. Oortwijn W, Banta HD, Cranovsky R. Introduction: Mass screening, health technology

assessment, and health policy in some european countries. Int J Technol Assess Health Care.

2001 Summer;17(3):269-74.

64. Gray JA. Evidence-based screening in the united kingdom. Int J Technol Assess Health

Care. 2001 Summer;17(3):400-8.

65. Banta HD. Health policy, health technology assessment, and screening in europe. Int J

Technol Assess Health Care. 2001 Summer;17(3):409-17.

66. Faisst K, Schilling J, Koch P. Health technology assessment of three screening methods in

switzerland. Int J Technol Assess Health Care. 2001 Summer;17(3):389-99.

67. Jonsson E, Banta HD, Schersten T. Health technology assessment and screening in

sweden. Int J Technol Assess Health Care. 2001 Summer;17(3):380-8.

68. Banta HD, Oortwiin W. Health technology assessment and screening in the netherlands:

Case studies of mammography in breast cancer, PSA screening in prostate cancer, and

ultrasound in normal pregnancy. Int J Technol Assess Health Care. 2001 Summer;17(3):369-79.


6 April 2010 53

69. Mousiama T, Loakimidou S, Largatzi E, Kaitelidou D, Liaropoulus L. Health technology

assessment in the area of prevention: Selected screening cases in greece. Int J Technol Assess


70. Perleth M, Busse R, Gibis B, Brand A. Evaluation of preventive technologies in germany:

Case studies of mammography, prostate cancer screening, and fetal ultrasound. Int J Technol

Assess Health Care. 2001 Summer;17(3):329-37.

71. Vermeulen V, Coppens K, Kesteloot K. Impact of health technology assessment on

preventive screening in belgium: Case studies of mammography in breast cancer, PSA

screening in prostate cancer, and ultrasound in normal pregnancy. Int J Technol Assess Health

Care. 2001 Summer;17(3):316-28.

72. Gulacsi L. The time for cost-effectiveness in the new european union member states: The

development and role of health economics and technology assessment in the mirror of the

hungarian experience. Eur J Health Econ. 2007 Jun;8(2):83-8.

73. Gulacsi L, Boncz I, Drummond M. Issues for countries considering introducing the "fourth

hurdle": The case of hungary. Int J Technol Assess Health Care. 2004 Summer;20(3):337-41.

74. Gibis B, Artiles J, Corabian P, Meiesaar K, Koppel A, Jacobs P, et al. Application of

strengths, weaknesses, opportunities and threats analysis in the development of a health

technology assessment program. Health Policy. 2001 Oct;58(1):27-35.

75. Antes G, Briones E, Britton M, Burnand B, Engel G, Gallo P, et al. Education and support

networks for assessment of health interventions. working group 5 report. Int J Technol Assess



6 April 2010 54

76. Lehoux P, Battista RN, Granados A, Gallo P, Tailliez S, Coyle D, et al. International master's

program in health technology assessment and management: Assessment of the first edition

(2001--2003). Int J Technol Assess Health Care. 2005 Winter;21(1):104-12.

77. Catalan Agency for Health Technology Assessment and Research. Catalan Health Service.

Department of Health Autonomous Government of Catalonia.

EUnetHTA work package 8. EUnetHTA handbook on health technology assessment capacity

building. Barcelona (Spain): EUROPEAN NETWORK FOR HEALTH TECHNOLOGY

ASSESSMENT; 2008.

78. Moharra M, Kubesch N, Estrada MD, Parada T, Cortés M, Espallargues M, et al. EUnetHTA

survey report on HTA organisations. . 2008.

79. Shepherd J, Briggs J, Payne L, Packer C, Kerridge L, Ashton-Key M. Setting the future

policy agenda for health technology assessment: A specialty mapping approach. Int J Technol


80. Oortwijn WJ, Vondeling H, van Barneveld T, van Vugt C, Bouter LM. Priority setting for

health technology assessment in the netherlands: Principles and practice. Health Policy. 2002

Dec;62(3):227-42.

81. Chase D, Milne R, Stein K, Stevens A. What are the relative merits of the sources used to

identify potential research priorities for the NHS HTA programme? Int J Technol Assess Health

Care. 2000 Summer;16(3):743-50.

82. Townsend J, Buxton M, Harper G. Prioritisation of health technology assessment. the

PATHS model: Methods and case studies. Health Technol Assess. 2003;7(20):iii, 1-82.

health technology assessment - nivel.nl · 1danish centre for health technology assessment &...

Documents