An AHRQ Perspective on CER and An AHRQ Perspective on CER and Methodology Considerations Methodology Considerations

Jean SlutskyJean SlutskyDirector, Center for Outcomes and EvidenceDirector, Center for Outcomes and Evidence

New York CityNew York CityAugust 12, 2010August 12, 2010

How Can We Make Better How Can We Make Better Healthcare Decisions?Healthcare Decisions?

Improve research baseImprove research base– RelevanceRelevance– QualityQuality

Improve use of data in decision makingImprove use of data in decision making

The Role of Clinical Decision The Role of Clinical Decision Makers in CERMakers in CER

Researcher looking through lens of decision maker to do research

Decision maker looking through lens of research to make decisions

CER

EBM

+

+

=

=

Comparing Evidence: Medical Comparing Evidence: Medical vs. Semiconductor Researchvs. Semiconductor Research

““When I was doing semiconductor device research, it was When I was doing semiconductor device research, it was expected that I would compare my results with other expected that I would compare my results with other people's previously published results and that I would people's previously published results and that I would comment on any differences. But it seemed to be different comment on any differences. But it seemed to be different in medicine. in medicine.

“ “Medical practitioners primarily tended to publish their own Medical practitioners primarily tended to publish their own data; data; they often didn't compare their data with the data of they often didn't compare their data with the data of other practitioners, even in their own fieldother practitioners, even in their own field, let alone with the , let alone with the results of other types of treatments for the same condition.”results of other types of treatments for the same condition.”

Intel co-founder and prostate cancer patient Andy Grove Intel co-founder and prostate cancer patient Andy Grove Forbes 5/13/96Forbes 5/13/96

Questions for Setting Clinical Policy:Questions for Setting Clinical Policy: A Systematic Process A Systematic Process

1.1. What is the outcome I care most about?What is the outcome I care most about?2.2. How good is the evidence that the interventions How good is the evidence that the interventions

can improve those outcomes?can improve those outcomes?3.3. How sure am I that it will work in “real world”?How sure am I that it will work in “real world”?4.4. How do the potential benefits compare to How do the potential benefits compare to

possible harms and costs?possible harms and costs?5.5. What constitutes “good enough” evidence?What constitutes “good enough” evidence?6.6. What other considerations are relevant?What other considerations are relevant?

Some History on Comparative Some History on Comparative Effectiveness ResearchEffectiveness Research

Comparative Effectiveness: AHRQComparative Effectiveness: AHRQEffective Health Care ProgramEffective Health Care Program

Created in 2005, authorized by Created in 2005, authorized by Section 1013 of the Medicare Section 1013 of the Medicare Prescription Drug, Improvement, Prescription Drug, Improvement, and Modernization Act (MMA) of and Modernization Act (MMA) of 2003 2003

To improve the quality, To improve the quality, effectiveness, and efficiency of effectiveness, and efficiency of health care delivered through health care delivered through Medicare, Medicaid, and S-CHIP Medicare, Medicaid, and S-CHIP programs by focusing onprograms by focusing on– What is known What is known now now – What research What research gapsgaps are critical to fill are critical to fill– Clinical effectivenessClinical effectiveness

Definition: HHSDefinition: HHS

Comparative effectiveness research is the Comparative effectiveness research is the conduct and synthesis of research comparing the conduct and synthesis of research comparing the benefits and harms of different interventions and benefits and harms of different interventions and strategies to prevent, diagnose, treat and monitor strategies to prevent, diagnose, treat and monitor health conditions in “health conditions in “real worldreal world” settings. The ” settings. The purpose of this research is to improve health purpose of this research is to improve health outcomes by developing and disseminating outcomes by developing and disseminating evidence-based information to patients, evidence-based information to patients, clinicians, and other decision-makers, clinicians, and other decision-makers, responding responding to their expressed needs, about which to their expressed needs, about which interventions are most effective for which patients interventions are most effective for which patients under specific circumstancesunder specific circumstances. .

Recovery Act Spend Plan, Office of the Secretary, 11/09Recovery Act Spend Plan, Office of the Secretary, 11/09

Horizon Horizon ScanningScanning

EvidenceEvidence NeedNeed

IdentificationIdentification

Evidence Evidence SynthesisSynthesis

EvidenceEvidence GenerationGeneration

StrategiesStrategiesInterventionsInterventionsConditionsConditionsPopulationsPopulations

DisseminationDisseminationTranslationTranslation

ImprovementsImprovements inin

Health CareHealth Care

Research PlatformResearch PlatformInfrastructure – Methods Development – Training Infrastructure – Methods Development – Training

A Framework for CERA Framework for CER

An Unprecedented InvestmentAn Unprecedented Investment

$681M (62%)$681M (62%) $268M (24%)$268M (24%) $132M (12%)$132M (12%)

$19M (2%)$19M (2%)

Allocations for the $1.1 billion in Allocations for the $1.1 billion in comparative effectiveness research comparative effectiveness research funding in the American Recovery and funding in the American Recovery and Reinvestment Act of 2009:Reinvestment Act of 2009: ResearchResearch Data InfrastructureData Infrastructure Dissemination and Dissemination and

AdoptionAdoption Administrative Administrative

support, inventory, support, inventory, evaluationevaluation

AHRQ FY 2008 – 2010 AHRQ FY 2008 – 2010 (including ARRA) Investments(including ARRA) Investments

3%

57%6%

12%6%

3%

13%

Public EngagementPublic Engagement

Topic nominations Topic nominations for researchfor research

Comment period for Comment period for key research key research questionsquestions

Comment period for Comment period for draft reviewsdraft reviews

Focus testing Focus testing translation productstranslation products

www.effectivehealthcare.ahrq.govwww.effectivehealthcare.ahrq.gov

Categorical Investments in Categorical Investments in Evidence GenerationEvidence Generation

Distributed data networks and related infrastructure Distributed data networks and related infrastructure development development

Methods development Methods development Non-experimental/observational studies, both Non-experimental/observational studies, both

retrospective and prospectiveretrospective and prospective Sponsorship of pragmatic clinical trials addressing Sponsorship of pragmatic clinical trials addressing

key foci of comparative effectivenesskey foci of comparative effectiveness

Evidence Generation ActivitiesEvidence Generation ActivitiesUnder ARRAUnder ARRA

Request for Registries: Up to five awards for the creation or Request for Registries: Up to five awards for the creation or enhancement of national patient registries, with a primary enhancement of national patient registries, with a primary focus on 14 priority conditions defined under MMA ($48M)focus on 14 priority conditions defined under MMA ($48M)

DEcIDE Consortium Support: Expansion of multi-center DEcIDE Consortium Support: Expansion of multi-center research system and funding for distributed data network research system and funding for distributed data network models that use clinically rich data from electronic health models that use clinically rich data from electronic health records ($24M)records ($24M)

Clinical and Health Outcomes Initiative in Comparative Clinical and Health Outcomes Initiative in Comparative Effectiveness (CHOICE): Coordinated national effort to Effectiveness (CHOICE): Coordinated national effort to establish a series of ~10 pragmatic clinical comparative establish a series of ~10 pragmatic clinical comparative effectiveness studies ($100M)effectiveness studies ($100M)

iADAPT: Innovative dissemination and implementation iADAPT: Innovative dissemination and implementation grants for CER ($29.5M)grants for CER ($29.5M)

Underlying PrinciplesUnderlying Principles

Attentive to subgroupsAttentive to subgroups Under-represented in Under-represented in

clinical trials (elderly, clinical trials (elderly, children, racial and ethnic children, racial and ethnic minorities)minorities)

Inclusive of co-morbiditiesInclusive of co-morbidities Explore heterogeneity of Explore heterogeneity of

treatment effects treatment effects

Citizen’s Forum for Comparative Citizen’s Forum for Comparative Effectiveness ResearchEffectiveness Research

ARRA funding will support the Citizens’ Forum ARRA funding will support the Citizens’ Forum Initiative to develop new mechanisms and refine Initiative to develop new mechanisms and refine existing approaches to eliciting public input existing approaches to eliciting public input

The Forum will increase our use of public input to The Forum will increase our use of public input to inform health care policy, especially involving inform health care policy, especially involving comparative effectiveness research for AHRQ’s comparative effectiveness research for AHRQ’s Effective Health Care Program Effective Health Care Program

It will expand AHRQ’s efforts to obtain professional It will expand AHRQ’s efforts to obtain professional and consumer input, build methods and capacity and consumer input, build methods and capacity for obtaining public input and allow the program for obtaining public input and allow the program to obtain guidance and insight from a to obtain guidance and insight from a broader publicbroader public

Horizon Scanning for CERHorizon Scanning for CER

Horizon Scanning Horizon Scanning methodology and methodology and reporting for CERreporting for CER

Registry of Patient RegistriesRegistry of Patient Registries

The Recovery Act calls for AHRQ to fund The Recovery Act calls for AHRQ to fund development of a registry that would serve development of a registry that would serve broader research purposes. This type of broader research purposes. This type of registry would:registry would:– Benefit multiple stakeholdersBenefit multiple stakeholders– Reduce unnecessary duplication of effortReduce unnecessary duplication of effort– Encourage communication between registry Encourage communication between registry

developersdevelopers

““For example, researchers undertaking a patient registry of cardiovascular disease For example, researchers undertaking a patient registry of cardiovascular disease among women under the age of 55 in a certain region may find several similar among women under the age of 55 in a certain region may find several similar registries underway with which they might collaborate or choose to differentiate to registries underway with which they might collaborate or choose to differentiate to better answer certain questions.”better answer certain questions.” ARRA Spend Plan, Office of the Secretary ARRA Spend Plan, Office of the Secretary

Educating CliniciansEducating Clinicians

ConciseConcise ActionableActionable Paired with Paired with

consumer guidesconsumer guides Convey level of Convey level of

uncertainty/certainty uncertainty/certainty of findingsof findings

Effective Health Care Web 2.0 Effective Health Care Web 2.0 and Social Media Toolsand Social Media Tools

Product Buttons

Product Collections

WidgetsSocial Media

RSSFeeds

CME/CE ActivitiesCME/CE Activities

CME/CE activities are available for the followinghealth conditions and methods areas: Diabetes Heart and Blood Vessel Conditions Muscle Bone and Joint Conditions AHRQ Training Modules for the Systematic Reviews

Methods Guide Other Topics

Emerging Methods in Comparative Emerging Methods in Comparative Effectiveness, EBM & SafetyEffectiveness, EBM & Safety

Variation in methods among Variation in methods among systematic reviews undercuts systematic reviews undercuts transparencytransparency

Poorly done new research can Poorly done new research can be misleadingbe misleading

Methods must continue to Methods must continue to evolve and not remain stagnant evolve and not remain stagnant

AHRQ has and will continue to AHRQ has and will continue to make investments in improving make investments in improving methods, esp. in understanding methods, esp. in understanding clinical heterogeneity. clinical heterogeneity.

New Resource on Comparative New Resource on Comparative Effectiveness Research MethodsEffectiveness Research Methods

June 2010 supplement to June 2010 supplement to Medical CareMedical Care on on CER MethodsCER Methods

22 original articles, special focus on:22 original articles, special focus on:– Ways to enhance the inclusion of clinically heterogeneous Ways to enhance the inclusion of clinically heterogeneous

populations in clinical and comparative effectiveness studiespopulations in clinical and comparative effectiveness studies– Methods for implementing longitudinal investigations that Methods for implementing longitudinal investigations that

capture longer term health outcomes, including patient-capture longer term health outcomes, including patient-reported outcomesreported outcomes

Printed copies available free of charge Printed copies available free of charge through the AHRQ Publications through the AHRQ Publications ClearinghouseClearinghouse

www.effectivehealthcare.ahrq.gov www.effectivehealthcare.ahrq.gov AHRQ publication number: OM10-0067AHRQ publication number: OM10-0067

Questions Related to Questions Related to Surrogate EndpointsSurrogate Endpoints

Easy to measure but..Easy to measure but..– How well do they link to true How well do they link to true meaningfulmeaningful

health outcomeshealth outcomes– What options exist to capture distal What options exist to capture distal

outcomes of interestoutcomes of interest– How well do we understand the natural How well do we understand the natural

course of some conditionscourse of some conditions

http://effectivehealthcare.ahrq.govhttp://effectivehealthcare.ahrq.gov