a comparison of criteria for clinical validity and utility ...€¦ · capability subcommittee 1...
TRANSCRIPT
CAPABILITY Subcommittee 1 Draft ReportMay 2008
A comparison of criteria for clinical validityand utility in various national and
international frameworks
Prepared byJörg Schmidtke on behalf of Subcommittee 1Hannover Medical School, Institute of Human Genetics, Carl-Neuberg-Str. 1, 30625 Hannover, [email protected]
Subcommittee 1 Tasks
• systematic review of existing approaches of genetic test evaluation with an emphasis on clinical utility of tests with a potential to be implemented in general pract ice care and prevention
• included in the review will be genetic test evaluat ion approaches developed in the USA such as ACCE and EGAPP and the approaches of the United Kingdom Genetic Testing Network and the reviews done so far by Unit 3 of EuroGentest
• Draft of an analytic framework (=questions that nee d to be addressed) for evaluating genetic test applicati on in different settings
Chairperson will prepare the draft for the CAPABILI TY Working Group
Subcommittee 1 Tasks
CAPABILITY Subcommittee 1 MeetingFriday, 21st September 2007Sils Maria, Switzerland
Chair: Jörg Schmidtke
1. Report from the coordination office (Nippert)
2. Tasks of Subcommittee 1 (Nippert)
3. Frameworks developed so far and underlying definitions of clinical utility (joint discussion)• ACCE/problems applying ACCE criteria in practice • EGAPP Approach and topics under current review• CanGèneTest• Criteria developed by the UK GTN (Zimmern)• Criteria developed by EuroGentest Unit 3 (Schmidtke)• Disease-specific guidelines developed by the German Society of
Human Genetics (Schmidtke)• Paper for the United Kingdom Genetic Testing Network (Zimmern)
4. Frameworks developed so far and underlying definitions of clinical utility (joint discussion)
US Funded Projects:The ACCE Evaluation Process for Genetic Testing
AA nalytic validity
CC linical validity
CC linical utility
EE thical, Legal &Social implications
Analytic framework40 + targeted questions
Funded by CDC (2000-2004), Office of Genomics and Disease Prevention
What facilities/personnel are available or easily put in place?Facilities
What are the financial costs associated with testing?Health Risks
Is there general access to that remedy or action?Intervention
What health risks can be identified for follow-up testing and/orintervention?
Health Risks
What are the results of pilot trials?Pilot Trials
What quality assurance measures are in place?Quality Assurance
Is the test being offered to a socially vulnerable population?Intervention
Is there an effective remedy, acceptable action, or other measurablebenefit?
Intervention
If applicable, are diagnostic tests available?Intervention
What is the impact of a positive (or negative) test on patient care?Intervention
What is the natural history of the disorder?Intervention
Specific QuestionComponent
http://www.cdc.gov/genomics/gtesting/file/print/FBR/BCCliUti.pdf
ACCE - Clinical Utility
Specific QuestionComponent
What are the economic benefits associated with actions resulting from testing?
Economic
What guidelines have been developed for evaluating program performance?
Monitoring
Are there informed consent requirements?Education
What methods exist for long term monitoring?Monitoring
What educational materials have been developed and validated andwhich of these are available?
Education
ELSIWhat is known about stigmatization, discrimination, privacy/confidentiality and personal/family social issues?
Impediments
What safeguards have been described and are these safeguards in place and effective?
Safeguards
Are there legal issues regarding consent, ownership of data and/orsamples, patents, licensing, proprietary testing, obligation to disclose, or reporting requirements?
Impediments
ACCE - Clinical Utility
http://www.cdc.gov/genomics/gtesting/file/print/FBR/BCCliUti.pdf
US Funded Projects:Evaluation of Genomic Applications in Practice and
Prevention (EGAPP)
http://www.egappreviews.org/workingrp/topics_diagram.htm
Funded by CDC (2004-2008)
Topics Chart
*variants or mutations in the identified gene or genes
Completed Reports on: http://www.egappreviews.org/workingrp/reports.htm
EGAPP – Completed Topics (last update February 12, 2008)
Although research remains promising, adaptation of genomic tests into clinical practice must await
appropriately designed and powered studies in relevant clinical settings.
1) and 2) Detection and management
1) General population of
women and; 2) women at increased
risk for ovarian cancer
Genomic TestsOvarian Cancer
The sensitivity, specificity, and reliability of the tests used to evaluate individuals for suspected HNPCC is
not known confidently. Data regarding the net benefits and harms associated with predictive genetic testing in patients with HNPCC-related cancers and their families
members is incomplete
Management of individuals and early detection/prevention for
family members
Individuals diagnosed with CRC and their
family members
Mismatch repair gene mutations
Hereditary Nonpolyposis
Colorectal Cancer (HNPCC)
There is a paucity of good-quality data addressing the questions of whether testing for CYP450
polymorphisms in adults entering SSRI treatment for non-psychotic depression leads to improvement in outcomes, or whether testing results are useful in
medical, personal, or public health decision making.
Treatment with SSRI drugs
Individuals diagnosed with
depressionCYP450Depression
For all tests, the relationship of predicted to observed risk in different populations still needs further study, as
does their incremental contribution, optimal implementation, and relevance to patients on current
therapies.
Improve prognostic accuracy,
treatment choice and health outcomes
Women diagnosed with
early stage breast cancer
3 marketed gene expression based
assaysBreast Cancer
Intended UseTarget Population
Outcome Evaluation
Clinical Scenario
Test Assessed*Disorder/
Effect
Treatment with irinotecan
Individuals diagnosed �with CRC
UGT1A1Colorectal Cancer
(CRC)
Risk prediction or nutritional/lifestyle
management
General populationMultigene panel
Cardiovascular Disease
Intended UseTarget Population
Clinical ScenarioTest Being Assessed*Disorder/
Effect
*variants or mutations in the identified gene or genes
EGAPP – Topics under Review(last update February 12, 2008)
Completed Reports on: http://www.egappreviews.org/workingrp/reports.htm
The UKGTN has developed a “Gene Dossier“ process to evaluategenetic tests and recommend which
tests will be provided by theNational Health Service
UK Genetic Testing Network (UKGTN) Experience
Areas covered in the gene dossier include:
• The laboratory details of the test
• The test characteristics
• The clinical details of the condition
• The prevalence of the condition
• The purpose of the test
• The healthcare context in which the test is to be used
• The clinical utility of the test
UK Genetic Testing Network (UKGTN) Experience
Source: M. Kroese et al: How can genetic tests be evaluated for clinical use?, European Journal of Human Genetics (2007) 15, 917-921
The purposepurpose of a test must be specified in advance
because the effectivenesseffectiveness of the test is defined to
be the extent to which it fulfils the purpose for
which it was undertaken.
It is not possible to measure effectiveness without
defining purpose
Test Purpose
The purposes for which genetic testing are carried
out all fall within three categories. They are:
1. Reduce morbidity or mortality
2. Provide information salient to the care of the
patient or family members and/or
3. Assist the patient or family members with
reproductive decision-making
Test Purpose
• The time and effort to complete a genedossier was greater than originally predictedbecause of the gaps in genetic and laboratoryinformation
and
• the need to calculate test performancemeasures.
UK Genetic Testing Network (UKGTN) Experience
Source: M. Kroese et al: How can genetic tests be evaluated for clinical use?, European Journal of Human Genetics (2007) 15, 917-921
• This raised the question of the requisitebalance between the degree of detail requiredfor the evaluation of test performance
and
• the negative impact of not providing the test.
UK Genetic Testing Network (UKGTN) Experience
Source: M. Kroese et al: How can genetic tests be evaluated for clinical use?, European Journal of Human Genetics (2007) 15, 917-921
• The UKGTN experience of evaluatinggenetic tests for rare single gene disordershas shown clinical utility to be the mostimportant, and the most complex, of all theACCE domains.
UK Genetic Testing Network (UKGTN) Experience
Source: M. Kroese et al: How can genetic tests be evaluated for clinical use?, European Journal of Human Genetics (2007) 15, 917-921
• The UKGTN experience suggests that thiswas less of an issue for tests for highlypenetrant inherited diseases (where a modified evaluation would usually suffice) than for tests used in complex disorders(where complete and thorough evaluationwould most likely be required)
UK Genetic Testing Network (UKGTN) Experience
Source: M. Kroese et al: How can genetic tests be evaluated for clinical use?, European Journal of Human Genetics (2007) 15, 917-921
This discussion
framework was
produced for, and has
now been endorsed by
the UK Genetic Testing
Network
The PHG Foundation Framework for Genetic Test Evaluation
The PHG Foundation Framework for Genetic Test Evaluation (1)
Source: Burke, W., Zimmern, R.; Moving beyond ACCE: An Expanded Framework for Genetic Test Evaluation; PHG Foundation, 2007
The PHG Foundation Framework for Genetic Test Evaluation (2)
Source: Burke, W., Zimmern, R.; Moving beyond ACCE: An Expanded Framework for Genetic Test Evaluation; PHG Foundation, 2007
By the end of 2007, the EuroGenteststeering committee decided that the German
“indication criteria” are to be regarded as prototypic examples for a future set of
similar guidelines which will be developed by EuroGentest, endorsed by the European Society of Human Genetics, and published
in the European Journal of Human Genetics.
The EuroGentest Approach towards Genetic Test Evaluation
Aims at:
Developing criteria to determine when potential tests are ready to move from research to clinical practice
Involving leading experts from across Europe and elsewhere to develop evaluation tools
The EuroGentest Approach towards Genetic Test Evaluation
EuroGentest process for developingrecommendations for genetic test evaluation
EuroGentest Process, 1st Step
Data Collection
and analysis
Data Collection
and analysis
Expert led
Discussion
Expert led
Discussion
Draft
Recommendations
Draft
Recommendations
ConsultationConsultation
PublicationPublication
Survey 2006
Recommendations2008
http://content.karger.com/ProdukteDB/produkte.asp?A ktion=Ausgabe&Ausgabe=234201&ProduktNr=224224
� There is developing consenus on the four major domains of genetic test evaluation, namely analyticalvalidity, clinical validity, clinical utility, and et hical, legal and social implications (ACCE).
� While the ACCE definitions for analytical and clinicalvalidity are straightforward and globally applicable, criteria for clinical utility are highly context-dependent. Such contexts include the health caresystem within which a test is to be provided as well as locally available resources and set priorities. An evaluation of clinical utility is also stronglyinfluenced by surrounding ethical, legal, and socialissues.
Conclusions & Recommendationsbased upon the EuroGentest Survey
� The natural history of the disease, if known, should be con sideredso that testing and intervention can be properly timed.
� Interventions that might follow a positive test result sh ould beeffective and available.
� Qualified pre-test, test, and post-test measures, incl udingappropriate consent processes and genetic counseling, shou ld bein place when needed.
� Health risks associated with testing and interventions foll owingpositive and negative test results as well as with not testingshould be considered.
� Financial costs and benefits of testing should be evalua ted.� Testing services should provide educational materials, acce ss to
genetic counselling, and maintain surveillance over thei r activities.
Ideally, the following components should be considered whe n theclinical utility of a genetic test needs to be assessed :
Conclusions & Recommendationsbased upon the EuroGentest Survey
� Currently, these components can beassessed with sufficient confidence and reliability for a small number of conditionsonly, i. e. conditions with relatively high prevalence.
� For these, any genetic service system orprogramme should design disease-specific recommendations to guide test implementation in routine practice.
Conclusions & Recommendationsbased upon the EuroGentest Survey
� The great majority of conditions due to heritable mutations with high penetrance isso rare that sufficient quantitative data fordisease-specific guidelines are currentlynot available and may indeed never begathered.
� For assessing the clinical utility of genetictesting for such conditions, a muchabbreviated decision tree is proposed:
Conclusions & Recommendationsbased upon the EuroGentest Survey
1 assumed that the patient is clinically examined (including non-invasive intervention, e.g. imaging, elctrophysiology)2 Including predictive prenatal diagnosis 3 Predominantly processes with a high risk for patients – or other processes like biochemical tests instead of genetic tests
Decision Tree
� There is great heterogeneity in genetictesting services among the countriessurveyed.
� It is premature to mandate that genetictesting provided by clinical services meetsprofessional standards regarding clinicalvalidity and utility, because there is to date no consensus within the scientificcommunity and among health careproviders to what extent clinical validityand utility can and need to be assessed.
Conclusions & Recommendationsbased upon the EuroGentest Survey
Whenever limitations are necessary, healthcare systems may prioritize within genetictesting services according to prior risk, severity (if measurable in a meaningful way), disease prevalence, or availablitiy and cost of test methodology.
Conclusions & Recommendationsbased upon the EuroGentest Survey
• In Germany, the decision for or against prescribing/offe ringany particular genetic test is up to the individual docto r, in a case-by-case fashion
• No genetic screening programmes (except newbornscreening)
• One single specific guideline: management of ICSI –exclusion of CFTR mutation
• Prerequisites for a predictive test are fulfilled whenev er a fully informed patient considers it beneficial (BÄK guide lineof 2003)
• Medical measures are indicated provided they are"necessary, adequate and economic" (SGB V)
Disease-specific guidelines developed by the German Society of Human Genetics
The German Society of Human Genetics aims to move towards disease specific
guidelines rather than developing a framework for test evaluation. The
“indication criteria” are developed for some disorders based on ACCE model´s questions and are meant as a guide for
clinical practice.
Disease-specific guidelines developed by the German Society of Human Genetics
• Other relevant diagnosis methods• Economical aspects • Intervention • Prognosis • Management • Predictive setting• Genetic risk evaluation of the family members• Prenatal diagnosis • Other consequences
Clinical Utility
• Analytical Sensitivity• Analytical Specificity • Clinical Sensitivity • Clinical Specificity• PPV• NPV
Test characteristics
• Name and OMIM number of disorder and gene/chromosom e/segment• Spectrum of the mutations • Analysis method• Validation • Prevalence in Germany• Test application in which setting:
– (Differential) diagnosis– predictive diagnosis– risk evaluation of family members– prenatal diagnosis
Disorder/approach
Specific QuestionElement/Component
Disease-specific guidelines developed by the German Society of Human Genetics
List of diseases for which “Indication Criteria“ have be en completed(more than 25 diseases, as of May 2008 available in E nglish (www.eurogentest.org))
Poupak Javaher, MD, MPHFrauke Becker, Dipl. Oec.Markus Preylowski, Dipl. Oec.
Hannover Medical School
Ulf KristofferssonHelena KäriäiinenIrma NippertJorge Sequeiros
EUROGENTEST Unit 3 Partners
Acknowledgement