review of information retrieval – first experiences in germany with manufacturer dossiers on new...

9th HTAi Annual Meeting BILBAO26 June 2012Dr. Siw Waffenschmidt

Information Management Unit, IQWiG

Review of information retrieval – first experiences in Germany with manufacturer dossiers on new drugs

Content

1) Introduction to the early benefit assessment of new drugs

2) Requirements for information retrieval in the dossier template

3) Review of information retrieval by IQWiG

4) Examples

Informationsbeschaffung in Dossiers 2

Submission of a dossier by the manufacturer

Assessment (by IQWiG)

Decision on added benefit by the Federal

Joint Committee

Added benefit?

Permissible for reference price?

no

Reference price

Price at no added cost versus comparator treatment

no

Price negotiations SHI and

manufacturer

Agreement?Discount on sales price

Decision by arbitration body

On request of SHI or manufacturer: health economic evaluation

yes

yes

Month 6

Month 12

Market entry

Month 3

Month 15

no

yes

Price set by manufacturer

Drug assessment according to AMNOG

3

Content




4) Examples


Requirements for information retrieval in dossier templates – Module 4 4.2.3 Methods of information retrieval

Bibliographic literature search Search in trial registries Selection of relevant studies

4.3.1 Results of information retrieval RCTs of the drug to be assessed Studies for indirect comparisons

Indirect comparisons on the basis of RCTs Non-randomized comparative studies

Further studies Appendix

4-A: Search strategies – bibliographic literature search 4-B: Search strategies – search in trial registries 4-C: List of studies screened as full texts and excluded (with reason for

exclusion) Attachments

RIS files


Content




4) Examples



Check the search in bibliographic databases

Check the search in trial registries

Evaluate information retrieval

Depending on results of evaluation: (potentially) conduct own search in bibliographic databases and/or trial

registries

Depending on results of evaluation:No further action

Documentation

Process in Information Management Unit:

Checking the search in bibliographic databases (formal check)

Methods of information retrievalCheck whether requirements for the search procedure and the presentation of results have been fulfilled as specified in the dossier template

ScreeningWas the procedure for study selection described?Are deviations from the above requirements justified?

Result of the information retrievalCheck consistency of retrieved hits Do the individual sections of the dossier contain consistent information on inclusion and exclusion criteria?

Documentation of the above steps in checklists developed for this purpose.


Checking the search in bibliographic databases (content check)

Is the number of hits plausible?

Checking the search strategy presented in Appendix 4-A Were search filters used and correctly transferred? Syntax acc. to PRESS Checklist

Was the study pool identified via the search strategy? Is the search sensitive enough?


Sampson M, McGowan J, Lefebvre C, Moher D, Grimshaw J. PRESS: Peer Review of Electronic Search Strategies. Ottawa: Canadian Agency for Drugs and Technologies in Health; 2008. URL: http://www.msmandhiv.org/documents/globaltopics/PRESS.pdf.






registries


Documentation


Potential literature searches by IQWiG

1. Search in public trial registries - ClinicalTrials.gov- ICTRP Search Portal and - potentially in industry registries

2. Search in bibliographic databases- Simple Structured Boolean Search (SSBS) - combined with the first 20 Related Citations in PubMed


Sampson M. Updating searches for systematic reviews. Aberystwyth: University; 2009. URL: http://cadair.aber.ac.uk/dspace/bitstream/2160/3760/1/Sampson%20-%20Updating%20Searches%20for%20Systematic%20Reviews%20-%20PhD.pdf.

Waffenschmidt S, Janzen T, Hausner E, Kaiser T. Assessing the completeness of systematic reviews via the “related articles” function and/or a simple structured Boolean search in PubMed – a pilot study.2011. (Scientific evidence for healthcare quality and patient safety: 19th Cochrane Colloquium; 19.-22.10.2011; Madrid, Spanien).






registries


Documentation


Documentation


Content




4) Examples


Examples of shortcomings potentially resulting in a check of the completeness of the study pool by IQWIG

Search strategy errors Lack of sensitivity Lack of plausibility of the number of hits Inconsistencies in search documentation


Syntax errors → Search not sensitive enough


EMBASEInterface: Elsevier


Ovid Interface results in 64.508

Lack of plausibility of the number of hits


Search checked on 08.07.2011: n= 61New citations since 01.05.2011: n=3


Experiences so far

additional relevant studies have only been identified from the list of studies provided by the manufacturer (e.g. study listed but excluded by manufacturer), not by own searches

Challenges in the checking and evaluation of information retrieval: Assessment of the sensitivity of the searches Indirect comparisons and further studies


Conclusions

Our approach to the review of information retrieval in manufacturer dossiers seems to be workable.

Besides checking the search strategies, peer review of information retrieval should cover all aspects of a search.

Ultimately, the in(completeness) can only be verified by doing a focused search yourself .


22

Institute for Quality and Efficiency in Health Care (IQWiG)

Dr. Siw Waffenschmidt

Potential Conflicts of Interest: NONE

Im Mediapark 8 50670 Cologne, Germany Tel: +49-221/3 56 85-0 Fax: +49-221/3 56 85-1 [email protected] www.iqwig.de

22Informationsbeschaffung in Dossiers

Search in bibliographic databases

Choice of search techniques is based on the work by Sampson 2009 (“Updating searches for systematic reviews” [PhD])

On the basis of Sampson’s approach, we conducted our own analysis based on systematic reviews of drugs:

Þ Main result:

The combination of the first 20 Related Citations and the SSBS seems to be the most appropriate technique, with high completeness (sensitivity: 97.1%), good reliability (sensitivity range: 80-100%) and an efficient use of resources (precision: 5.0%, NNR: 24). (manuscript submitted)


Example – IQWiG’s search in PubMed


Search QueryItems found

#1 Search Sativex*[tiab] or Nabiximol*[tiab] or Cannab*[tiab] or GW*1000*[tiab] or *THC*[tiab] or Marinol*[tiab] or Tetrahydrocannabinol*[tiab] or Nabilon*[tiab]

20478

#2 Search multiple sclerosis 51363

#3 Search #1 and #2 355

#4 Search (Therapy/Narrow[filter]) AND (#3) 33

#5 Search 14615106[pmid] 1

#6 Related Citations for PubMed (Select 14615106) 106

#7 Select 19 document(s) 19

#8 …… ….

Simple Structered Boolean Search

The first 20 Related Citations of the 5 relevant citations included in the dossier

Pubmed: 13.01.2012Total number of hits without duplicates: 70Screening result: no relevant additional studies found

review of information retrieval – first experiences in germany with manufacturer dossiers on new...

Health & Medicine