evaluating dissemination of ahrq cer products darren mays, phd, mph department of oncology...
TRANSCRIPT
Evaluating Dissemination of AHRQ CER Products
Darren Mays, PhD, MPH
Department of OncologyGeorgetown University Medical Center
Lombardi Comprehensive Cancer Center Washington, DC
How will iADAPT help?
• Poised to make progress– What approaches work? For whom? In what
settings/conditions?• Presents an evaluation challenge– Creative methods/approaches– Diverse populations– Different clinical areas
• Need for a flexible evaluation framework
What is RE-AIM?
• Evaluate public health impact• Focus on dissemination• Barriers include design, setting, approach• Impact assessed on multiple domains• RE-AIM domains:– Reach, Efficacy/Effectiveness, Adoption,
Implementation, Maintenance
Glasgow, Vogt, & Boles, 1999; Glasgow, Lichtenstein, & Marcus, 2003
RE-AIM Domains
• Reach– Did the CER products reach the intended
population(s)?• Participation rate(s), characteristics, baseline “risk”
• Efficacy/effectiveness– What is the impact on intended outcomes?• Clinical outcomes, CER product utilization, occurrence
of harms/unintended consequences
Refer to RE-AIM domains handout; Glasgow et al., 2006
RE-AIM Domains
• Adoption– Did the intended units use the CER product(s)?
• Participation and characteristics of setting(s), delivery agents, barriers to adoption
• Implementation– Were the CER products implemented as intended?
• Adherence, fidelity, technical success
• Maintenance– What is the long-term impact of CER products?
• Long-term efficacy/effectiveness, sustained implementation, barriers to long-term use
Refer to RE-AIM domains handout; Glasgow et al., 2006
Determining Impact
• Quantitatively determining impact• Original application– Reach x Efficacy = Impact
• RE-AIM overall impact– Product of all 5 domains– Requires quantifiable measures
Glasgow, Vogt, Boles, 1999; Glasgow et al. 2006
Application to iADAPT?R E A I M Audience(s) Clinical Area(s)
In PersonCHW OutreachAcad. Detailing
MedMed
MedMed
LowLow
LowLow
LowMed
PatientProvider
DiabetesDiabetes
GroupCER TrainingSchoolGroup Therapy
MedHighLow
LowLowMed
MedMedMed
MedLowLow
MedLowMed
PolicyPatientPatient
MultipleHeart DiseaseDiabetes
eHealthClinic KioskWeb Patient Portal
MedMed
MedMed
LowMed
HighHigh
MedHigh
PatientPatient &Provider
DiabetesDiabetes
Print/MediaTargeted Video Med Med Med Med Med Patient Heart Disease
Adapted from Glasgow et al., 2001
A Closer Look
Clinic Kiosk• R: n = 200 patients
Well-controlled diabetes • E: Small effect size
200 CERSGs (~1/pt.)• A: 75% of clinics• I: 50% completion rate
Technical problems• M: Few support resources
Limited patient interest
Web Portal• R: n = 1,000 patients
Poor diabetes control• E: Medium effect size
2,500 CERSGs (~2.5/pt.)• A: 100% of clinics• I: 75% completion rate
Few technical problems • M: Minimal maintenance
Low-cost to direct patients
Conclusions
• A flexible evaluation framework• Multi-domain evaluation approach• Identify facilitators, barriers, and future
directions• Creative approaches may be needed!
RE-AIM Resources
• NCI DCCPS web site for RE-AIM– http://cancercontrol.cancer.gov/IS/REAIM
• Resources include:– Figures/graphics illustrating key concepts– Checklists and planning tools– Example measures– Publications, presentation
ReferencesBernhardt, JM, Mays, D, & Kreuter, MW. (2011). Dissemination 2.0: Closing
the gap between knowledge and practice with new media. J Health Comm, 16(S1), 32-44
Glasgow, RE, Vogt, TM, & Boles, SM. (1999). Evaluating the public health impact of health promotion interventions: The RE-AIM framework. AJPH, 89(9), 1322-1327
Glasgow, RE, et al. (2001). The RE-AIM framework for evaluating interventions: What can it tell us about approaches to chronic illness management. Patient Ed. & Counsel., 44, 119-127.
Glasgow, RE, Lichtenstein, E, & Marcus, AC. (2003). Why don’t we see more translation of health promotion research to practice? Rethinking the efficacy-to-effectiveness transition. AJPH, 93(8), 1261-1267
Glasgow, RE, et al. (2006). Using RE-AIM metrics to evaluate diabetes self-management support interventions. AJPM, 30(1), 67-73