strategies and outcomes: measurement in d & i research · strategies & measures. your...
TRANSCRIPT
![Page 1: Strategies and Outcomes: Measurement in D & I Research · Strategies & measures. Your university published an article bragging about your selection to MT-DIRC training at Washington](https://reader036.vdocuments.mx/reader036/viewer/2022071116/5fff5c49f1410a165b2880fb/html5/thumbnails/1.jpg)
David Chambers and Enola ProctorJune 7, 2017
1
Strategies and Outcomes:Measurement in D & I Research
![Page 2: Strategies and Outcomes: Measurement in D & I Research · Strategies & measures. Your university published an article bragging about your selection to MT-DIRC training at Washington](https://reader036.vdocuments.mx/reader036/viewer/2022071116/5fff5c49f1410a165b2880fb/html5/thumbnails/2.jpg)
Strategies & measuresYour university published an article bragging about your selection to MT-DIRC training at Washington University. The next day, you received a phone call from a clinic director in a nearby city. She informs you that the city’s consortium of three Federally Qualified Health Centers wants to improve their implementation of colon cancer screening guidelines, and they believe that you can offer good advice. She asks you to come to a meeting with the agency directors next week to share your suggestions.
June 8, 2016 2
![Page 3: Strategies and Outcomes: Measurement in D & I Research · Strategies & measures. Your university published an article bragging about your selection to MT-DIRC training at Washington](https://reader036.vdocuments.mx/reader036/viewer/2022071116/5fff5c49f1410a165b2880fb/html5/thumbnails/3.jpg)
She tells you:
• Clinic A is already doing very well with the guideline implementation and their director has volunteered their agency as a model for the other two.
• Clinic B is receiving staff pushback because they feel overloaded with other programs and don’t know what approach could possibly help.
• Clinic C has a mixed track record with implementing breast cancer screening, and the director doubts that these kinds of guidelines are very effective in the first place.
3June 8, 2016
![Page 4: Strategies and Outcomes: Measurement in D & I Research · Strategies & measures. Your university published an article bragging about your selection to MT-DIRC training at Washington](https://reader036.vdocuments.mx/reader036/viewer/2022071116/5fff5c49f1410a165b2880fb/html5/thumbnails/4.jpg)
DiscussionGroup 1 Group 2 Group 3How might you learn about the strategies Clinic A has used thus far in implementing evidence-based programs and interventions in cancer prevention and control?
What questions would you ask Clinic A about their approach?
Which implementation outcomes might be important, and which would you want to explore for potential measurement?
Would you assess the same implementation outcomes in each clinic?
If not, which implementation outcomes would you assess for which clinic?
What additional information would you want to learn about to inform your discussion with Clinic C, and why?
4June 8, 2016
![Page 5: Strategies and Outcomes: Measurement in D & I Research · Strategies & measures. Your university published an article bragging about your selection to MT-DIRC training at Washington](https://reader036.vdocuments.mx/reader036/viewer/2022071116/5fff5c49f1410a165b2880fb/html5/thumbnails/5.jpg)
Implementation OutcomesFeasibility
FidelityPenetration
AcceptabilitySustainability
UptakeCosts
*IOM Standards of Care
Conceptual Model for Implementation Research
What?
QIsESTs
How?
ImplementationStrategies
Implementation Research Methods
ServiceOutcomes*Efficiency
SafetyEffectiveness
EquityPatient-
centerednessTimeliness
Patient Outcomes
Clinical/health status
SymptomsFunction
Satisfaction
Proctor et al 2009 Admin. & Pol. in Mental Health Services
CONTEXT
CONTEXT
CON
TEXT
CON
TEXT
The UsualThe Core of
Implementation Science
5June 8, 2016
![Page 6: Strategies and Outcomes: Measurement in D & I Research · Strategies & measures. Your university published an article bragging about your selection to MT-DIRC training at Washington](https://reader036.vdocuments.mx/reader036/viewer/2022071116/5fff5c49f1410a165b2880fb/html5/thumbnails/6.jpg)
IMPLEMENTATION STRATEGIES
Key constructs
6June 8, 2016
![Page 7: Strategies and Outcomes: Measurement in D & I Research · Strategies & measures. Your university published an article bragging about your selection to MT-DIRC training at Washington](https://reader036.vdocuments.mx/reader036/viewer/2022071116/5fff5c49f1410a165b2880fb/html5/thumbnails/7.jpg)
Implementation Strategies:Definition
Systematic intervention process to adopt and integrate evidence-based healthcare innovations into usual care *
Active ingredient in processes for moving EST’s and QI’s into usual care
*Powell, McMillen, Proctor et al., Medical Care Research and Review, 2012
7June 8, 2016
![Page 8: Strategies and Outcomes: Measurement in D & I Research · Strategies & measures. Your university published an article bragging about your selection to MT-DIRC training at Washington](https://reader036.vdocuments.mx/reader036/viewer/2022071116/5fff5c49f1410a165b2880fb/html5/thumbnails/8.jpg)
Implementation Strategies:Complexity*
Discrete• involve one process or action, such as “meetings,”
“reminders”Multifaceted**• uses two or more discrete strategies, such as “training +
technical assistance”Blended• several discrete strategies are interwoven & packaged as
protocolized or branded strategies, such as “ARC,” IHI Framework for Spread”
*Powell, McMillen, Proctor et al., 2012 ** Grimshaw et al., 2001, Grol & Grimshaw, 2003
8June 8, 2016
![Page 9: Strategies and Outcomes: Measurement in D & I Research · Strategies & measures. Your university published an article bragging about your selection to MT-DIRC training at Washington](https://reader036.vdocuments.mx/reader036/viewer/2022071116/5fff5c49f1410a165b2880fb/html5/thumbnails/9.jpg)
68 “discrete” strategies in 6 categories:• Plan• Educate• Finance• Restructure• Manage quality• Attend to policy context
Strategies Compilation
![Page 10: Strategies and Outcomes: Measurement in D & I Research · Strategies & measures. Your university published an article bragging about your selection to MT-DIRC training at Washington](https://reader036.vdocuments.mx/reader036/viewer/2022071116/5fff5c49f1410a165b2880fb/html5/thumbnails/10.jpg)
Train and educate
stakeholders
Change infrastructure
1
23
283442 49
66&709
10
111213
22
44
4
5
14
18
23
61
26
27
46
56
37
39
41
50
6269
6
7
1740
47
5264
48
5163
67
68
24
25
36 38
45
65
72
3557
8
33
53
54
58
21
30
32
59
15
1629
60
1920
3143
55
7173
Develop stakeholder interrelationships
Provide interactive assistance
Utilize financial strategies
Engage consumers
Use evaluative and iterative strategies
Adapt & tailor to context
Support clinicians
Identification of Implementation Strategies: ERIC Cluster Solution
![Page 11: Strategies and Outcomes: Measurement in D & I Research · Strategies & measures. Your university published an article bragging about your selection to MT-DIRC training at Washington](https://reader036.vdocuments.mx/reader036/viewer/2022071116/5fff5c49f1410a165b2880fb/html5/thumbnails/11.jpg)
Current NIH funded research:Evidence-based practices
vs. Implementation StrategiesInstituteMechanismPI
Evidence-basedintervention
ImplementationStrategy
NIMHR01Dr. Yuhua Bao
Collaborative Care Model in community health clinics
Value-Based Purchasing (apolicy-level strategy)
NIMHR01Dr. Martha Bruce
Depression treatment in home health agencies
Web-based training, supervision and consultation with audit and feedback
NIDAR01Dr. Todd David Molfenter
Use of buprenorphine in addiction treatment organizations in Ohio
Clinician training and payment policy changes
![Page 12: Strategies and Outcomes: Measurement in D & I Research · Strategies & measures. Your university published an article bragging about your selection to MT-DIRC training at Washington](https://reader036.vdocuments.mx/reader036/viewer/2022071116/5fff5c49f1410a165b2880fb/html5/thumbnails/12.jpg)
Current NIH funded research:Evidence-based practices vs.Implementation Strategies
InstituteMechanismPI
Evidence-basedPractices
ImplementationStrategy
NIAIDR01Dr. John Lucian Davis
Contact Investigation for TB detection in Uganda
Mobile-phone health text messages and incentives
NIDDKR01Dr. Chester Fox
Evidence-based care of chronic kidney disease in primary care
Computer Decision Support tool, facilitation including physician champions and audit and feedback
NHLBIR01Dr. James Herbert
Evidence-based diet and physical activity intervention in churches
CBPR with train-the-trainer
![Page 13: Strategies and Outcomes: Measurement in D & I Research · Strategies & measures. Your university published an article bragging about your selection to MT-DIRC training at Washington](https://reader036.vdocuments.mx/reader036/viewer/2022071116/5fff5c49f1410a165b2880fb/html5/thumbnails/13.jpg)
Specifying Implementation strategies
Conceptually defined consistent with literatureOperationally defined
13
Actor Who?
Action Does what?
Action target(s) At what part of the context? (intervention? Organization? Providers?)
Dose How much?
Temporality When? In what order or sequence?
Implementation Outcome addressed
To achieve what implementationoutcome?
Justification Theoretical or empirical rationale?June 8, 2016
![Page 14: Strategies and Outcomes: Measurement in D & I Research · Strategies & measures. Your university published an article bragging about your selection to MT-DIRC training at Washington](https://reader036.vdocuments.mx/reader036/viewer/2022071116/5fff5c49f1410a165b2880fb/html5/thumbnails/14.jpg)
Example:strategies & their specification
June 8, 2016 14
![Page 15: Strategies and Outcomes: Measurement in D & I Research · Strategies & measures. Your university published an article bragging about your selection to MT-DIRC training at Washington](https://reader036.vdocuments.mx/reader036/viewer/2022071116/5fff5c49f1410a165b2880fb/html5/thumbnails/15.jpg)
Implementation strategy:Provision of intensive implementation support
15
Strategy specificationActor Practice facilitators
Action Provide formal clinic-wide stafftraining on intervention components andunderlying evidence (often in conjunction with clinic champion)
Action target(s) Build knowledge and acceptance of theintervention among clinic staff
Dose 1-h meeting at each clinic
Temporality Annually at start of implementationyears 1 and 2
Implementation Outcome addressed
Improved staff trust, understanding,uptake of intervention
Justification Evidence strength and quality, relativeadvantage, staff engagement,knowledge and beliefs, self-efficacyJune 8, 2016
![Page 16: Strategies and Outcomes: Measurement in D & I Research · Strategies & measures. Your university published an article bragging about your selection to MT-DIRC training at Washington](https://reader036.vdocuments.mx/reader036/viewer/2022071116/5fff5c49f1410a165b2880fb/html5/thumbnails/16.jpg)
Measuring implementation strategies
• Exposure:– Observational – Controlled
• Data types and sources– Key informants
• Interviews, focus groups, checklists– Direct observation
• Participant observation– Archival
• Tx disseminator records• Implementing site records (meeting notes, memos, policies)
16June 8, 2016
![Page 17: Strategies and Outcomes: Measurement in D & I Research · Strategies & measures. Your university published an article bragging about your selection to MT-DIRC training at Washington](https://reader036.vdocuments.mx/reader036/viewer/2022071116/5fff5c49f1410a165b2880fb/html5/thumbnails/17.jpg)
IMPLEMENTATION OUTCOMES
Key constructs
17June 8, 2016
![Page 18: Strategies and Outcomes: Measurement in D & I Research · Strategies & measures. Your university published an article bragging about your selection to MT-DIRC training at Washington](https://reader036.vdocuments.mx/reader036/viewer/2022071116/5fff5c49f1410a165b2880fb/html5/thumbnails/18.jpg)
Implementation Outcomes:Measure distinct from clinical
outcomesCould have an effective intervention:Not adoptedPoor penetration in health systemImplemented but
with poor fidelityNot sustained
18June 8, 2016
![Page 19: Strategies and Outcomes: Measurement in D & I Research · Strategies & measures. Your university published an article bragging about your selection to MT-DIRC training at Washington](https://reader036.vdocuments.mx/reader036/viewer/2022071116/5fff5c49f1410a165b2880fb/html5/thumbnails/19.jpg)
Implementation Outcomes: Key Concepts
• Acceptability• Adoption• Appropriateness• Feasibility• Fidelity• Implementation cost• Penetration• Sustainability
19June 8, 2016
![Page 20: Strategies and Outcomes: Measurement in D & I Research · Strategies & measures. Your university published an article bragging about your selection to MT-DIRC training at Washington](https://reader036.vdocuments.mx/reader036/viewer/2022071116/5fff5c49f1410a165b2880fb/html5/thumbnails/20.jpg)
Dissemination and Implementation Research in Health
PAR # 16-236; 237, 238Primary purpose: identify, understand, & develop • Strategies
for the • Adoption, adaptation, integration, scale-up,&
sustainability----these are implementation outcomes
• Of EB interventions, tools, policies, & guidelines
![Page 21: Strategies and Outcomes: Measurement in D & I Research · Strategies & measures. Your university published an article bragging about your selection to MT-DIRC training at Washington](https://reader036.vdocuments.mx/reader036/viewer/2022071116/5fff5c49f1410a165b2880fb/html5/thumbnails/21.jpg)
From PAR 16-238
• Dissemination research outcomes:– spread and sustain knowledge– reach many different stakeholders
• Implementation research outcomes:– adopt and integrate evidence-based health
interventions into clinical and community settings– adoption, implementation and sustainability
21
![Page 22: Strategies and Outcomes: Measurement in D & I Research · Strategies & measures. Your university published an article bragging about your selection to MT-DIRC training at Washington](https://reader036.vdocuments.mx/reader036/viewer/2022071116/5fff5c49f1410a165b2880fb/html5/thumbnails/22.jpg)
Outcome Issues
• What are the right outcomes?• Which outcomes are relevant to particular settings,
stakeholders, situation? (context, coming up)• Realistic timing• How to intervene to achieve the outcome
– What are the barriers to achieving the outcome?– What conceptual models/ frameworks are applicable to
which outcomes?– What implementation strategies are effective in achieving
which outcome?• Validity and reliability
June 8, 2016 22
![Page 23: Strategies and Outcomes: Measurement in D & I Research · Strategies & measures. Your university published an article bragging about your selection to MT-DIRC training at Washington](https://reader036.vdocuments.mx/reader036/viewer/2022071116/5fff5c49f1410a165b2880fb/html5/thumbnails/23.jpg)
Evidence-based intervention/ program/guideline
Implementation outcomes
Service system outcomes
Clinical outcomes NIH study(PI, University)
Rapid ART initiation for pregnant mothers for PMTCT of HIV
Acceptability, Feasibility, Cost-effectiveness
Rate of retention in care through 6mo
Viral suppression at time of delivery; MTC transmission at 6mo post-partum;
R01 HD074558(Abrams, E.; Columbia University)
Mobile computer-based video intervention to increase HIV testing rates in ED
Acceptability, Feasibility
None specified
Post-intervention HIV testing rate (RCT; 4 groups)
R34 DA037129 (Aronson, I.; National Development and Research Institutes)
Tobacco use treatment guidelines
Fidelity, Cost, Organizational readiness to implement change, Provider adherence to guidelines
None specified
Smoking abstinence at 6mo
R01 CA175329 (Shelley, D.; NYU School of Medicine
![Page 24: Strategies and Outcomes: Measurement in D & I Research · Strategies & measures. Your university published an article bragging about your selection to MT-DIRC training at Washington](https://reader036.vdocuments.mx/reader036/viewer/2022071116/5fff5c49f1410a165b2880fb/html5/thumbnails/24.jpg)
Evidence-based intervention/ program/guideline
Implementation outcomes
Service system outcomes
Clinical outcomes NIH study(PI, University)
Cognitive Processing Therapy for PTSD
Fidelity, adaptation, penetration
Capacity to deliver CPT
PTSD symptom change
R01 MH106506 (Wiltsey Stirman, S.; Palo Alto Veterans Institute for Research
SafeCare w/Technological Enhancement (compared with SafeCare as usual)
Feasibility, Acceptability, Fidelity
Perceived job demands & resources
Parent and child behavior change, Client satisfaction
R21 MH098244 (Self-Brown, S.; Georgia State University)
FAST TB Transmission Control in Hospitals
Acceptability, Barriers to use, Cost, Cost-effectiveness, Reach, Adoption
None specified Reduction in hospital worker TB infections; Reduced time to treatment
R01 AI112748 (Nardell, E.; Brigham & Women’s Hospital)
![Page 25: Strategies and Outcomes: Measurement in D & I Research · Strategies & measures. Your university published an article bragging about your selection to MT-DIRC training at Washington](https://reader036.vdocuments.mx/reader036/viewer/2022071116/5fff5c49f1410a165b2880fb/html5/thumbnails/25.jpg)
June 8, 2016 25
Bryan Weiner (PI): Increasing Colorectal Cancer Screening Rates in Community Health Centers, R21 CA161657
![Page 26: Strategies and Outcomes: Measurement in D & I Research · Strategies & measures. Your university published an article bragging about your selection to MT-DIRC training at Washington](https://reader036.vdocuments.mx/reader036/viewer/2022071116/5fff5c49f1410a165b2880fb/html5/thumbnails/26.jpg)
June 8, 2016 26
Donna Shelley(PI): Implementing Tobacco Use Treatment Guidelines in Community Health Centers in Vietnam, R01 CA175329
![Page 27: Strategies and Outcomes: Measurement in D & I Research · Strategies & measures. Your university published an article bragging about your selection to MT-DIRC training at Washington](https://reader036.vdocuments.mx/reader036/viewer/2022071116/5fff5c49f1410a165b2880fb/html5/thumbnails/27.jpg)
June 8, 2016 27
Ross Brownson(PI): Disseminating Evidence-Based Interventions to Control Cancer, R01 CA 160327
![Page 28: Strategies and Outcomes: Measurement in D & I Research · Strategies & measures. Your university published an article bragging about your selection to MT-DIRC training at Washington](https://reader036.vdocuments.mx/reader036/viewer/2022071116/5fff5c49f1410a165b2880fb/html5/thumbnails/28.jpg)
June 8, 2016 28
Ross Brownson(PI): Disseminating Evidence-Based Interventions to Control Cancer, R01 CA 160327
![Page 29: Strategies and Outcomes: Measurement in D & I Research · Strategies & measures. Your university published an article bragging about your selection to MT-DIRC training at Washington](https://reader036.vdocuments.mx/reader036/viewer/2022071116/5fff5c49f1410a165b2880fb/html5/thumbnails/29.jpg)
Measurement: Toward Standardization & Harmonization
• Society for Implementation Research Collaboration Instrument Review Project– http://www.societyforimplementationresearchcoll
aboration.org/sirc-projects/sirc-instrument-project/
• Grid-Enabled Measures developed by the National Cancer Institute– http://cancercontrol.cancer.gov/brp/gem.html
29June 8, 2016
![Page 30: Strategies and Outcomes: Measurement in D & I Research · Strategies & measures. Your university published an article bragging about your selection to MT-DIRC training at Washington](https://reader036.vdocuments.mx/reader036/viewer/2022071116/5fff5c49f1410a165b2880fb/html5/thumbnails/30.jpg)
Three primary outcomes for this project series include:(1) a comprehensive library of D&I instruments measuring the implementation outcomes identified by Proctor and colleagues (2010) and organized by the Consolidated Framework for Implementation Research (CFIR; Damschroder et al., 2009) for SIRC members;(2) a rating system reflecting the degree of empirical validation of instruments, adapted from the evidence-based assessment (EBA) work of Hunsley and Mash (2008) and Terwee et al (2012);(3) a consensus battery of instruments decided upon by D&I expert task force members using the EBA criteria to guide future D&I research efforts.
An update on SIRC Instrument Review Project Progress (10/15/14)We have now identified nearly 450 instruments to be included in our repository. The instruments assessing Implementation Outcomes (Proctor et al., 2009) have all been twice rated using our evidence-based assessment rating criteria. Results are available throughthe IRP portion of the website. The repository is available to all SIRC memberswww.societyforimplementationresearchcollaboration.org
THE SIRC INSTRUMENT REVIEW PROJECT
![Page 31: Strategies and Outcomes: Measurement in D & I Research · Strategies & measures. Your university published an article bragging about your selection to MT-DIRC training at Washington](https://reader036.vdocuments.mx/reader036/viewer/2022071116/5fff5c49f1410a165b2880fb/html5/thumbnails/31.jpg)
http://www.societyforimplementationresearchcollaboration.org/sirc-projects/sirc-instrument-project/
![Page 32: Strategies and Outcomes: Measurement in D & I Research · Strategies & measures. Your university published an article bragging about your selection to MT-DIRC training at Washington](https://reader036.vdocuments.mx/reader036/viewer/2022071116/5fff5c49f1410a165b2880fb/html5/thumbnails/32.jpg)
June 8, 2016 32https://implementationscience.biomedcentral.com/articles/10.1186/s13012-016-0401-y
![Page 33: Strategies and Outcomes: Measurement in D & I Research · Strategies & measures. Your university published an article bragging about your selection to MT-DIRC training at Washington](https://reader036.vdocuments.mx/reader036/viewer/2022071116/5fff5c49f1410a165b2880fb/html5/thumbnails/33.jpg)
Context informs strategy selectionBarriers assessment
Flottorp, S.A (2013) A checklist for identifying determinants of practice: A systematic review and synthesis of frameworks and taxonomies of factors that prevent or enable improvements in healthcare professional practice. Implementation Science 8:35
Antecedents for the interventionEmmons, K. M., Weiner, B., Fernandez , M.E., & Tu, S.
(2012), Systems Antecedents for Dissemination and Implementation : A Review and Analysis of Measures, Health Educ Behav 39: 87
33June 8, 2016
![Page 34: Strategies and Outcomes: Measurement in D & I Research · Strategies & measures. Your university published an article bragging about your selection to MT-DIRC training at Washington](https://reader036.vdocuments.mx/reader036/viewer/2022071116/5fff5c49f1410a165b2880fb/html5/thumbnails/34.jpg)
Take home points
• Implementation involves distinct– Strategies– Outcomes
• Measurement is evolving• Harmonization and standardization enhances
quality and rigor for implementation science
34June 8, 2016
![Page 35: Strategies and Outcomes: Measurement in D & I Research · Strategies & measures. Your university published an article bragging about your selection to MT-DIRC training at Washington](https://reader036.vdocuments.mx/reader036/viewer/2022071116/5fff5c49f1410a165b2880fb/html5/thumbnails/35.jpg)
Support: National Institute of Mental Health
P30 MH068579R25 MH080916P30 DK092950U54 CA155496UL1 RR024992 (Clinical and Translational Science Award, CTSA)
Washington University Institute for Public HealthBrown School of Social Work
Conflicts: none35