health care provider performance review - world bank · 2019-08-20 ·...
TRANSCRIPT
![Page 1: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/1.jpg)
Contact for this presentation:
Alexander K. Rowe, MD, MPH
Malaria Branch, Division of Parasitic Diseases and Malaria, Center for Global Health
Centers for Disease Control and Prevention
Mailstop A06
1600 Clifton Road
Atlanta, GA 30329
United States
Telephone: 1-404-718-4754
Fax: 1-404-718-4815
Email: [email protected]
Saved as:
C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\
HCPP Sept 2014 v3 (30 min - World Bank workshop)(few extra slides).ppt
last updated: Sept 3, 2014
Health Care Provider Performance Review
Presentation for World Bank workshop, September 5, 2014
![Page 2: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/2.jpg)
Center for Global Health
Malaria Branch
Systematic review of strategies
to improve health care provider
performance in low- and middle-
income countries
Alexander K. Rowe (Malaria Branch, Centers for Disease
Control and Prevention [CDC])
Samantha Y. Rowe (CDC)
David H. Peters (Johns Hopkins University)
Kathleen A. Holloway (World Health Organization)
John Chalker (Management Sciences for Health)
Dennis Ross-Degnan (Harvard Medical School)
Health Care Provider Performance Review:
Near-final results
![Page 3: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/3.jpg)
Acknowledgments
• Sushama Acharya
• Charity Akpala
• Dinorah Calles
• Tashana Carty
• Nirali Chakraborty
• Helen Chin
• Adrijana Corluka
• Didi Cross
• Bhavya Doshi
• Onnalee Gomez
• Meg Griffith
• Karen Herman
• Atsumi Hirose
• Simon Lewin
• Banafsheh Siadat
• Sanja Stanojevic
• Laura Steinhardt
• Savitha Subramanian
• Megan Thompson
• Anil Thota
• Ryan Wiegand
• Jeff Willis
• Kindra Willis
• Shannon Wood
• Karen Wosje
• Abera Wouhib
• Alicia Wright
• Chunying Xie
• Special thanks to investigators who responded to queries
• Funding: Bill and Melinda Gates Foundation, CDC, World Bank
• Qing Li
• Connie Liu
• Earl Long
• Jason McKnight
• Eliza McLeod
• Huseyin Naci
• Jan Odgaard-Jensen
• Dawn Osterholt
• Andy Oxman
• Magdalena Paczkowski
• Gabriel Ponce-de-Léon
• Nancy Pulsipher
• Atiq Rahman
• Monica Shah
![Page 4: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/4.jpg)
Background
• HCPs play essential roles in delivering health
care
• In LMICs, however, HCP performance often
inadequate (i.e., adherence to clinical guidelines)
![Page 5: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/5.jpg)
Background
• HCPs play essential roles in delivering health
care
• In LMICs, however, HCP performance often
inadequate (i.e., adherence to clinical guidelines)
• Ex: review by Holloway et al.* of 900 studies
from 1990–2009 found proportion of patients
treated according to standard guidelines was:
• 40% in public facilities
• <30% in private, for-profit facilities
* Trop Med Int Health 2013; 18 (6): 656–664.
![Page 6: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/6.jpg)
Background
• Many strategies exist to improve performance,
and summary of evidence would be useful
• Existing reviews have limitations, with key
limitation being that they typically focus on one
strategy (e.g., job aids, or audit & feedback)
• Decision-makers, however, often ask broader
question: What are most effective ways to
improve performance?
• To answer this broader question, one needs to
compare multiple strategies
![Page 7: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/7.jpg)
HCPP review: Objectives
Conduct systematic review of all strategies to improve
HCP performance and related health outcomes in
LMICs, and produce:
1. Database of studies on improving HCP performance for
decision-makers, donors, researchers, and others
![Page 8: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/8.jpg)
HCPP review: Objectives
Conduct systematic review of all strategies to improve
HCP performance and related health outcomes in
LMICs, and produce:
1. Database of studies on improving HCP performance for
decision-makers, donors, researchers, and others
2. Estimates of effectiveness of a wide variety of strategies
to improve HCP performance, and comparisons to
identify the most and least effective strategies
![Page 9: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/9.jpg)
HCPP review: Objectives
Conduct systematic review of all strategies to improve
HCP performance and related health outcomes in
LMICs, and produce:
1. Database of studies on improving HCP performance for
decision-makers, donors, researchers, and others
2. Estimates of effectiveness of a wide variety of strategies
to improve HCP performance, and comparisons to
identify the most and least effective strategies
3. Analyses of training and supervision to identify factors
associated with greater effectiveness
![Page 10: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/10.jpg)
HCPP review: Objectives
Conduct systematic review of all strategies to improve
HCP performance and related health outcomes in
LMICs, and produce:
1. Database of studies on improving HCP performance for
decision-makers, donors, researchers, and others
2. Estimates of effectiveness of a wide variety of strategies
to improve HCP performance, and comparisons to
identify the most and least effective strategies
3. Analyses of training and supervision to identify factors
associated with greater effectiveness
4. The real goal: Evidence-based guidance on how to
improve HCP performance in LMICs
![Page 11: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/11.jpg)
HCPP review: Objectives
Conduct systematic review of all strategies to improve
HCP performance and related health outcomes in
LMICs, and produce:
1. Database of studies on improving HCP performance for
decision-makers, donors, researchers, and others
2. Estimates of effectiveness of a wide variety of strategies
to improve HCP performance, and comparisons to
identify the most and least effective strategies
3. Analyses of training and supervision to identify factors
associated with greater effectiveness
4. The real goal: Evidence-based guidance on how to
improve HCP performance in LMICs
5. Research agenda to fill critical knowledge gaps on how
to improve HCP performance
![Page 12: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/12.jpg)
HCPP review: Objectives
Conduct systematic review of all strategies to improve
HCP performance and related health outcomes in
LMICs, and produce:
1. Database of studies on improving HCP performance for
decision-makers, donors, researchers, and others
2. Estimates of effectiveness of a wide variety of strategies
to improve HCP performance, and comparisons to
identify the most and least effective strategies
3. Analyses of training and supervision to identify factors
associated with greater effectiveness
4. The real goal: Evidence-based guidance on how to
improve HCP performance in LMICs
5. Research agenda to fill critical knowledge gaps on how
to improve HCP performance
![Page 13: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/13.jpg)
Methods: inclusion criteria
• Any study of effectiveness of any strategy to improve
HCP performance in LMIC, on any health topic, in any
language, published or not
• HCP. Any facility- or community-based health worker,
pharmacists, shopkeepers who sell drugs, private sector
![Page 14: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/14.jpg)
Methods: inclusion criteria
• Any study of effectiveness of any strategy to improve
HCP performance in LMIC, on any health topic, in any
language, published or not
• HCP. Any facility- or community-based health worker,
pharmacists, shopkeepers who sell drugs, private sector
• Literature search
− 15 electronic databases of published studies
(e.g., MEDLINE), completed in 2006
− Document inventories of 30 organizations for
unpublished studies, completed in 2008
![Page 15: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/15.jpg)
Methods: eligible study designs
• Pre-post with comparison (+/- randomization)
• Post-only with randomized controls
• Interrupted time series (>3 data points before
and after intervention)
![Page 16: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/16.jpg)
Methods: screening and data abstraction
• Titles and abstracts from search screened
• Performed double, independent data
abstraction
• Queries often sent to study authors for
clarifications and additional details
![Page 17: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/17.jpg)
Effect size = (FU – BL)intervention – (FU – BL)control
Methods: analysis of effect sizes
• Effect size in terms of %-point change
• Example formula for outcomes expressed as %:
• Calculated effect size such that positive values
mean improvement
![Page 18: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/18.jpg)
• Goal = groups neither too specific, nor too broad
• Approach
– Determined which individual strategy components were
used (e.g., training + supervision = 2 components)
– Created 10 component categories (e.g., training,
supervision, incentives, etc.). Not based on effect size
– Defined strategy groups as unique combinations of 10
component categories, for example:
Training only
Training + supervision
Training + supervision + incentives
Etc.
Defining strategy groups
![Page 19: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/19.jpg)
• Studies used wide variety of outcomes; probably
inappropriate to include all in same analysis
(e.g., outcomes on mortality and history-taking)
• For more of an “apples–apples” comparison,
data were divided into 24 subgroups, based on:
1. Outcome category (6 general categories)
2. Outcome scale (dichotomous/% vs. continuous)
3. HCP type (all HCPs vs. CHW-only studies)
• Only compare results within subgroups
• This presentation: process outcomes (%)
(e.g., % of patients correctly treated)
Defining which results can be compared
![Page 20: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/20.jpg)
• Only include strategy vs. control comparisons
(no head-to-head studies)
• Results from studies with >1 primary outcome
summarized by median effect size (MES)
Making comparisons that minimize bias:
Primary analysis
![Page 21: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/21.jpg)
• Only include strategy vs. control comparisons
(no head-to-head studies)
• Results from studies with >1 primary outcome
summarized by median effect size (MES)
• Effect sizes adjusted for differences in effect modifiers
(i.e., factors that influence effect size, regardless of
strategy): baseline performance & public HF setting
• Compare MES distributions of various strategies:
weighted medians, IQRs
• Weight = 1 + ln(no. of HCPs or service provision sites)
Making comparisons that minimize bias:
Primary analysis
![Page 22: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/22.jpg)
• For strategies with greatest effectiveness, check for
“confounding by limited variability” (i.e., are high
effect sizes due to studies from settings unusually
well suited for strategy?)
• Broaden strategy definition to include more studies
with same basic strategy, and see if results change;
large decreases indicate bias
Making comparisons that minimize bias:
Primary analysis
![Page 23: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/23.jpg)
• Meta-analysis
• Meta-regression
• Network meta-analysis, which includes
head-to-head studies
Making comparisons that minimize bias:
Secondary analysis
![Page 24: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/24.jpg)
Results
![Page 25: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/25.jpg)
• >105,000 citations screened
• 829 reports included in review
• 66% of reports published in scientific journals
• 491 studies included 674 comparisons
– 448 strategy vs. control group comparisons
– 226 head-to-head comparisons
Literature search
![Page 26: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/26.jpg)
• 79 countries
• 54% from low-income countries
• Wide range of contexts
– Urban and rural
– Public and private health facilities, and
community settings
– Numerous health conditions
General descriptive results
![Page 27: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/27.jpg)
Overall risk of bias (N = 490 studies)
Low: 14%
Moderate: 19%
High: 31%
Very high: 36%
![Page 28: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/28.jpg)
HCPP review: Objectives
Conduct systematic review of all strategies to improve
HCP performance and related health outcomes in
LMICs, and produce:
1. Database of studies on improving HCP performance for
decision-makers, donors, researchers, and others
2. Estimates of effectiveness of a wide variety of strategies
to improve HCP performance, and comparisons to
identify the most and least effective strategies
3. Analyses of training and supervision to identify factors
associated with greater effectiveness
4. The real goal: Evidence-based guidance on how to
improve HCP performance in LMICs
5. Research agenda to fill critical knowledge gaps on how
to improve HCP performance
![Page 29: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/29.jpg)
Results from the “all HCP”
group of studies
(i.e., not CHW-only studies)
![Page 30: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/30.jpg)
• 171 comparisons from 141 studies
• 56 unique strategy groups, most tested
by only 1 or 2 comparisons each
(generalizability uncertain)
• Focus on results of strategies with at
least 3 comparisons (studies) each
Numbers of studies and strategy groups
![Page 31: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/31.jpg)
Effectiveness of strategies with most evidence (3+ comps)
•
–10 0 10 20 30 40 50
•
•
•
•
Strategy group (no. of comp)
Effect size (%-points)
median adj. MES & IQR
•
•
•
•
•
Wt.
![Page 32: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/32.jpg)
Effectiveness of strategies with most evidence (3+ comps)
Print/electronic info. for HCPs only (6)
–10 0 10 20 30 40 50
•
•
•
•
Strategy group (no. of comp)
Effect size (%-points)
median adj. MES & IQR
–4
•
•
•
•
•
No. of comparisons that
evaluated the strategy
Wt.
![Page 33: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/33.jpg)
Effectiveness of strategies with most evidence (3+ comps)
Print/electronic info. for HCPs only (6)
–10 0 10 20 30 40 50
•
•
•
•
Strategy group (no. of comp)
Effect size (%-points)
median adj. MES & IQR
–4
•
•
•
•
•
No. of comparisons that
evaluated the strategy
Weighted
median of
adjusted MES
Wt.
![Page 34: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/34.jpg)
Effectiveness of strategies with most evidence (3+ comps)
Print/electronic info. for HCPs only (6)
–10 0 10 20 30 40 50
•
•
•
•
Strategy group (no. of comp)
Effect size (%-points)
median adj. MES & IQR
–4
•
•
•
•
•
No. of comparisons that
evaluated the strategy
Weighted
median of
adjusted MESInterquartile
range
Wt.
![Page 35: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/35.jpg)
Effectiveness of strategies with most evidence (3+ comps)
Print/electronic info. for HCPs only (6)
–10 0 10 20 30 40 50
Training only (46)
•
Supervision only (14)
•
Strategy group (no. of comp)
Effect size (%-points)
median adj. MES & IQR
•
•
Consumer support + supervision + train (5)
•
•
Ex: Patient education
–4
7
7
6
Wt.
![Page 36: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/36.jpg)
Effectiveness of strategies with most evidence (3+ comps)
Print/electronic info. for HCPs only (6)
–10 0 10 20 30 40 50
Training only (46)
•
Supervision only (14)
•
Strategy group (no. of comp)
Effect size (%-points)
median adj. MES & IQR
•
Consumer support + training (6)
Consumer support + supervision + train (5)
Institutional approach + other management
techniques + supervision + training (3)
•
Ex: licensing & accreditation
–4
7
7
7
6
8
Wt.
Ex: HCP self-assessment
![Page 37: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/37.jpg)
Effectiveness of strategies with most evidence (3+ comps)
Print/electronic info. for HCPs only (6)
–10 0 10 20 30 40 50
Training only (46)
Consumer support + other mgmnt tech (3)
Supervision only (14)
Group problem solving only (14; BD)
Strategy group (no. of comp)
Effect size (%-points)
median adj. MES & IQR
–4
7
11
Group problem solving + training (8; BD) 15
Consumer support + training (6)
Consumer support + supervision + train (5)
7
7
6
Institutional approach + other management
techniques + supervision + training (3) 8
Supervision + training (20; BD) 16
13
Wt.
![Page 38: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/38.jpg)
Effectiveness of strategies with most evidence (3+ comps)
Print/electronic info. for HCPs only (6)
–10 0 10 20 30 40 50
Training only (46)
Consumer support + other mgmnt tech (3)
Supervision only (14)
Group problem solving only (14; BD)
Strategy group (no. of comp)
Effect size (%-points)
median adj. MES & IQR
–4
7
11
Group problem solving + training (8; BD) 15
Consumer support + training (6)
Consumer support + supervision + train (5)
7
7
6
Institutional approach + other management
techniques + supervision + training (3) 8
Supervision + training (20; BD) 16
13
Wt.
![Page 39: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/39.jpg)
Average baseline performance = 40%, so even strategies
with highest median effect sizes are typically increasing
performance from 40% to only 56% (still large gap)
Print/electronic info. for HCPs only (6)
–10 0 10 20 30 40 50
Training only (46)
Consumer support + other mgmnt tech (3)
Supervision only (14)
Group problem solving only (14; BD)
Effect size (%-points)
–4
7
11
Group problem solving + training (8; BD) 15
Consumer support + training (6)
Consumer support + supervision + train (5)
7
7
6
Institutional approach + other management
techniques + supervision + training (3) 8
Supervision + training (20; BD) 16
13
![Page 40: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/40.jpg)
Secondary analyses
![Page 41: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/41.jpg)
• Methods: meta-analysis, meta-regression,
and network meta-analysis
• Results generally support primary analysis,
which is reassuring
Secondary analyses
![Page 42: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/42.jpg)
• Includes head-to-head studies
Network meta-analysis: preliminary resultsPreliminary
results!
.
![Page 43: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/43.jpg)
First, display all unique strategies, plus control group (all “treatments”)
49
1 comparison each (N = 24 strategies):
11, 15, 18, 20, 21, 22, 23, 24, 27, 28, 41, 44,
45, 47, 53, 56, 58, 59, 61, 67, 73, 75, 76, 78
55
2 comparisons
each: 13, 37, 51
3 comparisons
each: 9, 26, 54
3
29
70
72
233
32 30
8
10
36
19
1
5014
165
4
6
7
31
38
63
4852
57
17 25
12
62
6866
Control group
![Page 44: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/44.jpg)
Strategy vs. controls comparisons only (lines = comparisons)
49
1 comparison each (N = 24 strategies):
11, 15, 18, 20, 21, 22, 23, 24, 27, 28, 41, 44,
45, 47, 53, 56, 58, 59, 61, 67, 73, 75, 76, 78
55
2 comparisons
each: 13, 37, 51
3 comparisons
each: 9, 26, 54
3
29
70
72
233
32 30
8
10
36
19
1
5014
165
4
6
7
31
38
63
4852
57
17 25
12
62
6866
![Page 45: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/45.jpg)
49
1 comparison each (N = 24 strategies):
11, 15, 18, 20, 21, 22, 23, 24, 27, 28, 41, 44,
45, 47, 53, 56, 58, 59, 61, 67, 73, 75, 76, 78
55
2 comparisons
each: 13, 37, 51
3 comparisons
each: 9, 26, 54
3
29
70
72
233
32 30
8
10
36
19
1
5014
165
4
6
7
31
38
63
4852
57
17 25
12
62
6866
Head-to-head comparisons only (lines = comparisons)
Control group
![Page 46: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/46.jpg)
All comparisons shown, includes 225 comparisons.
49
1 comparison each (N = 24 strategies):
11, 15, 18, 20, 21, 22, 23, 24, 27, 28, 41, 44,
45, 47, 53, 56, 58, 59, 61, 67, 73, 75, 76, 78
55
2 comparisons
each: 13, 37, 51
3 comparisons
each: 9, 26, 54
3
29
70
72
233
32 30
8
10
36
19
1
5014
165
4
6
7
31
38
63
4852
57
17 25
12
62
6866
![Page 47: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/47.jpg)
All comparisons shown, includes 225 comparisons.
49
1 comparison each (N = 24 strategies):
11, 15, 18, 20, 21, 22, 23, 24, 27, 28, 41, 44,
45, 47, 53, 56, 58, 59, 61, 67, 73, 75, 76, 78
55
2 comparisons
each: 13, 37, 51
3 comparisons
each: 9, 26, 54
3
29
70
72
233
32 30
8
10
36
19
1
5014
165
4
6
7
31
38
63
4852
57
17 25
12
62
6866
![Page 48: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/48.jpg)
• Includes head-to-head studies
• Additional strategies now deserve attention. In 1° analysis,
we mostly ignored them because <3 comparisons each. By
adding head-to-head studies, several “new” strategies have
at least 3 comparisons. All have 95% CrI that exclude 0.
Network meta-analysis: preliminary resultsPreliminary
results!
.
![Page 49: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/49.jpg)
• Includes head-to-head studies
• Additional strategies now deserve attention. In 1° analysis,
we mostly ignored them because <3 comparisons each. By
adding head-to-head studies, several “new” strategies have
at least 3 comparisons. All have 95% CrI that exclude 0.1. Consumer support + strengthen infrastructure + institutional approaches +
other mgmnt techniques + supervision + low-intensity training (45 %-pts)
2. Financing/incentives + supervision + high-intensity training (35 %-points)
3. Consumer support + low-intensity training (29 %-points)
4. Financing/incentives + institutional approaches + other mgmnt techniques
+ supervision (25 %-points)
5. Strengthen infrastructure + supervision + low-intensity training (24 %-pts)
6. Other mgmnt techniques + low-intensity training (20 %-points)
7. Other mgmnt techniques + supervision + low-intensity training (21 %-pts)
8. Financing/incentives + institutional approaches + supervision (20 %-points)
• NB: More work needed to fully understand results
Network meta-analysis: preliminary resultsPreliminary
results!
.
![Page 50: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/50.jpg)
Results from studies
that only included
community health workers
![Page 51: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/51.jpg)
• 7 comparisons from 7 studies
• 6 unique strategy groups, most tested by
only 1 or 2 comparisons each
• For training only (N = 3 studies), weighted
median of adjusted MES = 10 %-pts
• For strategies that included “consumer
supports + training”, weighted median of
adjusted MES = 29 %-pts
Caution! Just 2 studies
Results
![Page 52: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/52.jpg)
Factors associated with
training effectiveness
![Page 53: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/53.jpg)
HCPP review: Objectives
Conduct systematic review of all strategies to improve
HCP performance and related health outcomes in
LMICs, and produce:
1. Database of studies on improving HCP performance for
decision-makers, donors, researchers, and others
2. Estimates of effectiveness of a wide variety of strategies
to improve HCP performance, and comparisons to
identify the most and least effective strategies
3. Analyses of training and supervision to identify factors
associated with greater effectiveness
4. The real goal: Evidence-based guidance on how to
improve HCP performance in LMICs
5. Research agenda to fill critical knowledge gaps on how
to improve HCP performance
![Page 54: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/54.jpg)
Methods
• Goal: Identify attributes of more effective training
strategies
![Page 55: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/55.jpg)
Methods
• Goal: Identify attributes of more effective training
strategies
• Analysis: Unweighted mixed linear regression
model with random intercept, which accounts for
clustering of effect sizes within studies
• Data sources: “All HCP” studies on strategies with
training
• Dependent variable: Process of care effect sizes
• Independent variables:
- Attributes of training
- Other strategy components
![Page 56: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/56.jpg)
FactorHypothesized effect on
HCP performance
Increasing training duration
Interactive educational method
Small group size
Professional trainer
Trainer is content expert
On-site training
Some of training was ongoing
Multiple topics
Effectiveness of training: Main hypotheses
![Page 57: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/57.jpg)
• Analyzed 661 effect sizes from 112 comparisons
in 99 studies
• In 5 comparisons, training duration >20 days
– These were excluded, as they were uncommon
and probably represented interventions different
from typical in-service training in LMICs
• Substantial numbers of missing values for
several factors of interest
– Validity of multivariable modeling questionable
– Focus on “univariable” results (still adjusted for
strategy components)
Effectiveness of training: Results
![Page 58: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/58.jpg)
Factors associated with training effectiveness
Training attribute Effect sizeP-
value
Clinical practice 10 %-points 0.01
![Page 59: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/59.jpg)
Factors associated with training effectiveness
Training attribute Effect sizeP-
value
Clinical practice 10 %-points 0.01
Four “key” training methods (lectures,
interactive sessions, role play, clinical
practice in combination)
19 %-points 0.03
![Page 60: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/60.jpg)
Factors associated with training effectiveness
Training attribute Effect sizeP-
value
Clinical practice 10 %-points 0.01
Four “key” training methods (lectures,
interactive sessions, role play, clinical
practice in combination)
19 %-points 0.03
Some or all training on-site 7 %-points 0.04
![Page 61: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/61.jpg)
Factors associated with training effectiveness
Training attribute Effect sizeP-
value
Clinical practice 10 %-points 0.01
Four “key” training methods (lectures,
interactive sessions, role play, clinical
practice in combination)
19 %-points 0.03
Some or all training on-site 7 %-points 0.04
Training duration
No single effect size because the effect of
training duration depended on whether the
training was on a single topic or multiple topics
![Page 62: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/62.jpg)
Factors associated with training effectiveness
Training attribute Effect sizeP-
value
Clinical practice 10 %-points 0.01
Four “key” training methods (lectures,
interactive sessions, role play, clinical
practice in combination)
19 %-points 0.03
Some or all training on-site 7 %-points 0.04
Training duration
Single-topic trainings 0 %-points NS
Multiple-topic trainings
![Page 63: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/63.jpg)
Factors associated with training effectiveness
Training attribute Effect sizeP-
value
Clinical practice 10 %-points 0.01
Four “key” training methods (lectures,
interactive sessions, role play, clinical
practice in combination)
19 %-points 0.03
Some or all training on-site 7 %-points 0.04
Training duration
Single-topic trainings 0 %-points NS
Multiple-topic trainings2 %-points per
added day 0.008
![Page 64: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/64.jpg)
Evidence-based guidance
on improving HCP
performance in LMICs
![Page 65: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/65.jpg)
HCPP review: Objectives
Conduct systematic review of all strategies to improve
HCP performance and related health outcomes in
LMICs, and produce:
1. Database of studies on improving HCP performance for
decision-makers, donors, researchers, and others
2. Estimates of effectiveness of a wide variety of strategies
to improve HCP performance, and comparisons to
identify the most and least effective strategies
3. Analyses of training and supervision to identify factors
associated with greater effectiveness
4. The real goal: Evidence-based guidance on how to
improve HCP performance in LMICs
5. Research agenda to fill critical knowledge gaps on how
to improve HCP performance
![Page 66: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/66.jpg)
1. Two strategies might be good choices: “supervision +
training” or “group problem solving + training”
2. Provision of printed or electronic materials to HCPs as
a sole strategy is unlikely to change HCP performance
Guidance on improving processes of care:
settings that do not only include CHWs
![Page 67: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/67.jpg)
3. To increase effectiveness of strategies that include
training, the following might be beneficial:
a. Using all 4 key training methods (lectures, interactive
sessions, role play, and clinical practice)
b. If use of all 4 key training methods is not feasible,
at least consider including clinical practice
c. Increasing training duration for courses on multiple
topics
d. Conducting at least some training on-site
Guidance on improving processes of care:
settings that do not only include CHWs
![Page 68: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/68.jpg)
4. In settings with CHWs only, a good strategy
choice might be consumer supports + training
Guidance on improving processes of care:
settings with CHWs only
![Page 69: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/69.jpg)
• Wide ranges of effect sizes for most strategies
demonstrate difficulty in predicting how effective a
strategy will be in a particular setting
• Therefore, it seems prudent to monitor any strategy
so program managers can know how well a strategy
is working
Guidance on monitoring and adaptation
![Page 70: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/70.jpg)
• Wide ranges of effect sizes for most strategies
demonstrate difficulty in predicting how effective a
strategy will be in a particular setting
• Therefore, it seems prudent to monitor any strategy
so program managers can know how well a strategy
is working
• Monitoring data could be used to better adapt
strategies to local conditions, with the aim of
increasing effectiveness
• Note that this guidance is partly a hypothesis that
should be tested, but is partly supported by relative
effectiveness of group problem solving strategies,
which often include monitoring and adaptation
Guidance on monitoring and adaptation
![Page 71: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/71.jpg)
1. Limitations of studies: lack of detail on strategy
and context, lack of standardization, and
difficulty in assessing study precision
2. Overview analysis—i.e., intentionally designed
to identify broad patterns across all studies.
Thus, results do not reflect nuances, e.g., all
financial strategies considered equivalent.
Future analyses will be more specific.
Limitations
![Page 72: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/72.jpg)
1. Limitations of studies: lack of detail on strategy
and context, lack of standardization, and
difficulty in assessing study precision
2. Overview analysis—i.e., intentionally designed
to identify broad patterns across all studies.
Thus, results do not reflect nuances, e.g., all
financial strategies considered equivalent.
Future analyses will be more specific.
3. With numerous statistical tests, results
represent hypothesis screening, not true
hypothesis testing
4. Review already out of date
Limitations
![Page 73: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/73.jpg)
1) Large number of studies on many strategies,
although most tested by only 1 or 2 studies
2) Major challenges: heterogeneity, lack of detail
and standardization
Conclusions
![Page 74: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/74.jpg)
1) Large number of studies on many strategies,
although most tested by only 1 or 2 studies
2) Major challenges: heterogeneity, lack of detail
and standardization
3) Although results should be interpreted with
some caution because of review’s limitations,
data do suggest greater effectiveness for
several strategies and attributes of training
4) Results also suggest it would be prudent to
monitor effectiveness for all strategies
Conclusions
![Page 75: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/75.jpg)
5) Health Care Provider Performance Review
is largest review of strategies to improve
HCP performance in LMICs
− Programs, donors, and other development
partners consider results when making
decisions
− To help disseminate results and encourage
more specific, local analyses, the entire
database will be made publicly available at
end of this year
Conclusions
![Page 76: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/76.jpg)
Center for Global Health
Malaria Branch
For more information please contact Centers for Disease Control and Prevention
1600 Clifton Road NE, Atlanta, GA 30333
Telephone: 1-800-CDC-INFO (232-4636)/TTY: 1-888-232-6348
E-mail: [email protected] Web: http://www.cdc.gov
The findings and conclusions in this report are those of the authors and do not necessarily represent the official position of the Centers for Disease Control and Prevention.
Thanks for
your attention!
Any questions or
comments?
![Page 77: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/77.jpg)
Extra slides:
10 strategy
components
![Page 78: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/78.jpg)
• Consumer support: E.g., community health education
• Strengthening infrastructure: E.g., new info. system, provision of drugs
• Financing and incentives: E.g., changing user fees, financial or non-
financial incentives
• Institutional approaches: E.g., licensing and accreditation schemes, and
resource control given to local government or civil society organizations
• Group problem solving: E.g., continuous quality improvement with or
without formal teams
• Supervision: E.g., improving routine supervision, audit with feedback
• Management techniques, but not group problem solving & supervision:
E.g., HCP self-assessment, change processes to improve use of services
• High-intensity training: Duration >5 days (or ongoing training) + at least
one interactive education method (e.g., role play or interactive sessions)
• Low-intensity training: Any training that is not high-intensity training
• Printed or electronic information or job aids for HCPs that is not integral
part of another component
Ten component categories
![Page 79: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/79.jpg)
• Consumer support: E.g., community health education
• Strengthening infrastructure: E.g., new info. system, provision of drugs
• Financing and incentives: E.g., changing user fees, financial or non-
financial incentives
• Institutional approaches: E.g., licensing and accreditation schemes, and
resource control given to local government or civil society organizations
• Group problem solving: E.g., continuous quality improvement with or
without formal teams
• Supervision: E.g., improving routine supervision, audit with feedback
• Management techniques, but not group problem solving & supervision:
E.g., HCP self-assessment, change processes to improve use of services
• High-intensity training: Duration >5 days (or ongoing training) + at least
one interactive education method (e.g., role play or interactive sessions)
• Low-intensity training: Any training that is not high-intensity training
• Printed or electronic information for HCPs only
Ten component categories
Little difference;
results often combined
![Page 80: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/80.jpg)
Extra slides:
Confounding by
limited variability
![Page 81: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/81.jpg)
Effectiveness of strategies with most evidence (3+ comps)
Print/electronic info. for HCPs only (6)
–10 0 10 20 30 40 50
Training only (46)
Consumer support + other mgmnt tech (3)
Supervision only (14)
Group problem solving only (14; BD)
Strategy group (no. of comp)
Effect size (%-points)
median adj. MES & IQR
–4
7
11
Group problem solving + training (8; BD) 15
Consumer support + training (6)
Consumer support + supervision + train (5)
7
7
6
Institutional approach + other management
techniques + supervision + training (3) 8
Supervision + training (20; BD) 16
13
Wt.
![Page 82: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/82.jpg)
Effectiveness of strategies with most evidence (3+ comps)
Print/electronic info. for HCPs only (6)
–10 0 10 20 30 40 50
Training only (46)
Consumer support + other mgmnt tech (3)
Supervision only (14)
Group problem solving only (14; BD)
Strategy group (no. of comp)
Effect size (%-points)
median adj. MES & IQR
–4
7
11
Group problem solving + training (8; BD) 15
Consumer support + training (6)
Consumer support + supervision + train (5)
7
7
6
Institutional approach + other management
techniques + supervision + training (3) 8
Supervision + training (20; BD) 16
13
Wt.
![Page 83: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/83.jpg)
Strategy groupNo. of
comps
Wt. median adj.
MES (IQR)
Original definition of row 2
Group problem solving + low-intensity
training
3 60 (30, 77)
Broadened definition
Example of “confounding by limited variability”
![Page 84: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/84.jpg)
Strategy groupNo. of
comps
Wt. median adj.
MES (IQR)
Original definition of row 2
Group problem solving + low-intensity
training
3 60 (30, 77)
Broadened definition
Group problem solving + any training
+/- other components
Example of “confounding by limited variability”
![Page 85: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/85.jpg)
Strategy groupNo. of
comps
Wt. median adj.
MES (IQR)
Original definition of row 2
Group problem solving + low-intensity
training
3 60 (30, 77)
Broadened definition
Group problem solving + any training
+/- other components
Example of “confounding by limited variability”
• Broadening definition adds studies (to improve generalizability)
and those studies have extra components
• Assume “extra” components unlikely to decrease effect size
• If effect size decreases, then likely cause is confounding
![Page 86: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/86.jpg)
Strategy groupNo. of
comps
Wt. median adj.
MES (IQR)
Original definition of row 2
Group problem solving + low-intensity
training
3 60 (30, 77)
Broadened definition
Group problem solving + any training
+/- other components
8
Example of “confounding by limited variability”
• Broadening definition adds studies (to improve generalizability)
and those studies have extra components
• Assume “extra” components unlikely to decrease effect size
• If effect size decreases, then likely cause is confounding
![Page 87: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/87.jpg)
Strategy groupNo. of
comps
Wt. median adj.
MES (IQR)
Original definition of row 2
Group problem solving + low-intensity
training
3 60 (30, 77)
Broadened definition
Group problem solving + any training
+/- other components
8 15 (6, 36)
Example of “confounding by limited variability”
Bottom line. Adjusted results from broadened definition
are probably best: more generalizable, greater validity,
more conservative.
![Page 88: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/88.jpg)
Extra slides:
Adjustment for
effect modifiers
![Page 89: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/89.jpg)
• Adjusted effect sizes
– Baseline performance. For every 10 %-point
increase in baseline, effect size decreased by
2 %-points, on average (less room to improve…a
“disadvantage”, regardless of strategy).
Making comparisons that minimize bias
![Page 90: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/90.jpg)
![Page 91: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/91.jpg)
• Adjusted effect sizes
– Baseline performance. For every 10 %-point
increase in baseline, effect size decreased by
2 %-points, on average (less room to improve…a
“disadvantage”, regardless of strategy).
Example adjustment: Mean baseline = 40%.
If baseline for effect size is 50% (i.e., 10 %-points
above mean), then adjustment increases effect
size by 2 %-points (removes “disadvantage”).
Making comparisons that minimize bias
![Page 92: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/92.jpg)
• Adjusted effect sizes
– Baseline performance. For every 10 %-point
increase in baseline, effect size decreased by
2 %-points, on average (less room to improve…a
“disadvantage”, regardless of strategy).
Example adjustment: Mean baseline = 40%.
If baseline for effect size is 50% (i.e., 10 %-points
above mean), then adjustment increases effect
size by 2 %-points (removes “disadvantage”).
– Public HF setting. Mean effect size is 8 %-points
higher than other settings, regardless of strategy
Making comparisons that minimize bias
![Page 93: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/93.jpg)
• Adjusted effect sizes
– Baseline performance. For every 10 %-point
increase in baseline, effect size decreased by
2 %-points, on average (less room to improve…a
“disadvantage”, regardless of strategy).
Example adjustment: Mean baseline = 40%.
If baseline for effect size is 50% (i.e., 10 %-points
above mean), then adjustment increases effect
size by 2 %-points (removes “disadvantage”).
– Public HF setting. Mean effect size is 8 %-points
higher than other settings, regardless of strategy
– Goal: adjust results to partly “standard” context
(what would results be if all had same baseline?)
Making comparisons that minimize bias
![Page 94: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/94.jpg)
Extra slides:
24 outcome groups
![Page 95: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/95.jpg)
Which results can be compared: 24 subgroups
General outcome category
1. Facilitators (e.g.,
availability of supplies)
2. Processes of care
(e.g., correct treatment)
3. Health impact
(e.g., mortality)
4. Utilization/care-seeking
5. Other patient behaviors
(e.g., adherence)
6. Cost
![Page 96: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/96.jpg)
Which results can be compared: 24 subgroups
General outcome category
Outcome expressed as a
percentage
Continuous outcome, not
% (e.g., mortality rate)
1. Facilitators (e.g.,
availability of supplies)
2. Processes of care
(e.g., correct treatment)
3. Health impact
(e.g., mortality)
4. Utilization/care-seeking
5. Other patient behaviors
(e.g., adherence)
6. Cost
![Page 97: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/97.jpg)
Which results can be compared: 24 subgroups
General outcome category
Outcome expressed as a
percentage
Continuous outcome, not
% (e.g., mortality rate)
All HCPs CHW only All HCPs CHW only
1. Facilitators (e.g.,
availability of supplies)
2. Processes of care
(e.g., correct treatment)
3. Health impact
(e.g., mortality)
4. Utilization/care-seeking
5. Other patient behaviors
(e.g., adherence)
6. Cost
![Page 98: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/98.jpg)
Which results can be compared: 24 subgroups
General outcome category
Outcome expressed as a
percentage
Continuous outcome, not
% (e.g., mortality rate)
All HCPs CHW only All HCPs CHW only
1. Facilitators (e.g.,
availability of supplies)
2. Processes of care
(e.g., correct treatment)
3. Health impact
(e.g., mortality)
4. Utilization/care-seeking
5. Other patient behaviors
(e.g., adherence)
6. Cost
![Page 99: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/99.jpg)
Which results can be compared: 24 subgroups
No. of studies in each subgroup
General outcome category
Outcome expressed as a
percentage
Continuous outcome, not
% (e.g., mortality rate)
All HCPs CHW only All HCPs CHW only
1. Facilitators (e.g.,
availability of supplies)52 7 12 4
2. Processes of care
(e.g., correct treatment)178 14 79 4
3. Health impact
(e.g., mortality)55 15 64 3
4. Utilization/care-seeking 94 38 33 13
5. Other patient behaviors
(e.g., adherence)53 19 56 23
6. Cost 9 1 38 2
![Page 100: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/100.jpg)
Extra slides:
Risk of bias,
results on cost &
cost-effectiveness
![Page 101: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/101.jpg)
• Wide variety of cost and cost-effectiveness data
reported; thus, no quantitative synthesis
Results on cost and cost-effectiveness
(N = 491 studies)
![Page 102: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/102.jpg)
• Wide variety of cost and cost-effectiveness data
reported; thus, no quantitative synthesis
• Descriptive results
– 172 (35%) reported any information on strategy
costs or other economic evaluations
– 110 (22%) reported cost on at least 1 strategy
component
– 146 (30%) compared strategy costs of 2 or more
study groups
– 111 (23%) compared strategy costs of 2 or more
study groups in terms of cost ratio (e.g., cost per
service provided)
Results on cost and cost-effectiveness
(N = 491 studies)
![Page 103: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/103.jpg)
Extra slides:
Guidance vs.
recommendations
![Page 104: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/104.jpg)
• Decisions about which strategy to use in a given
setting depend on many factors, such as effectiveness,
cost, feasibility, and cultural acceptability
• This review only has information on effectiveness,
and limitations of evidence base require that
generalizations must be made with caution
General considerations
![Page 105: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/105.jpg)
• Decisions about which strategy to use in a given
setting depend on many factors, such as effectiveness,
cost, feasibility, and cultural acceptability
• This review only has information on effectiveness,
and limitations of evidence base require that
generalizations must be made with caution
• When evidence base has considerable limitations but a
public health problem is important, the decision about
whether or not to make recommendation must consider
cost of making erroneous recommendation vs. cost of
no recommendation and letting the status quo persist
• A middle way is to make statements on “guidance”
rather than “recommendations” (which suggest a
higher level of certainty)
General considerations
![Page 106: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/106.jpg)
Extra slides:
Future research
![Page 107: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/107.jpg)
Future research
![Page 108: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/108.jpg)
HCPP review: Objectives
Conduct systematic review of all strategies to improve
HCP performance and related health outcomes in
LMICs, and produce:
1. Database of studies on improving HCP performance for
decision-makers, donors, researchers, and others
2. Estimates of effectiveness of a wide variety of strategies
to improve HCP performance, and comparisons to
identify the most and least effective strategies
3. Analyses of training and supervision to identify factors
associated with greater effectiveness
4. The real goal: Evidence-based guidance on how to
improve HCP performance in LMICs
5. Research agenda to fill critical knowledge gaps on how
to improve HCP performance
![Page 109: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/109.jpg)
Ultimate goals of research in this field
1. Identify which strategies more reliably lead to
larger improvements in HCP performance
![Page 110: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/110.jpg)
Ultimate goals of research in this field
1. Identify which strategies more reliably lead to
larger improvements in HCP performance
2. Determine which strategies are best for a given
problem in a given setting on a given budget
− Ex 1: Getting physicians in a prosperous hospital to
adhere to hand-washing recommendations
− Ex 2: Getting CHWs to follow case-management
guidelines for multiple illnesses in a poor district with
few external resources
![Page 111: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/111.jpg)
Ultimate goals of research in this field
1. Identify which strategies more reliably lead to
larger improvements in HCP performance
2. Determine which strategies are best for a given
problem in a given setting on a given budget
− Ex 1: Getting physicians in a prosperous hospital to
adhere to hand-washing recommendations
− Ex 2: Getting CHWs to follow case-management
guidelines for multiple illnesses in a poor district with
few external resources
3. Develop processes that help program managers
select optimal strategies for a given performance
problem, setting, and budget
![Page 112: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/112.jpg)
Theme: Coordinate and standardize
• Why emphasize coordination?
− Numerous studies in past 50 years, but relatively few
evidence-based recommendations. Why?
− Many factors, but one key reason is most studies don’t
fit together or build on each other (some exceptions)
− No “vaccine” for improving HCP performance. Success
will probably be incremental and based on future
reviews and meta-analyses
− Thus, studies need to fit together better
− Coordination could channel researchers’ efforts and
donor’s funds to achieve research goals more quickly
![Page 113: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/113.jpg)
Theme: Coordinate and standardize
• Why emphasize coordination?
− Numerous studies in past 50 years, but relatively few
evidence-based recommendations. Why?
− Many factors, but one key reason is most studies don’t
fit together or build on each other (some exceptions)
− No “vaccine” for improving HCP performance. Success
will probably be incremental and based on future
reviews and meta-analyses
− Thus, studies need to fit together better
− Coordination could channel researchers’ efforts and
donor’s funds to achieve research goals more quickly
• Why emphasize standardization?
− Major challenges with synthesizing the evidence are a
lack of standardization in study methods and how
context, methods, and results are reported
![Page 114: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/114.jpg)
Recommendation 1
• Coordinate research so studies of agreed-upon,
high-priority topics can be completed more quickly
• Create an evidence-based research agenda
− Input by range of stakeholders (e.g., Ministries
of Health, multilateral and non-governmental
organizations, researchers, and donors)
− Donors should link funding to agenda
− Researchers should refer to agenda
− Results should be linked to agenda
![Page 115: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/115.jpg)
Recommendation 2
• Work towards standardization of:
− Strategy definition and description
− Outcome definition
− Measurement and analysis methods
− Description of setting
Which contextual elements?
How should they be reported?
• Improve reporting of details
− E.g., Standards for Quality Reporting Excellence
(SQUIRE)*
− Other guidance
* Davidoff F et al. Ann Intern Med. 2008;149:670-676.
![Page 116: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/116.jpg)
Recommendation 2, cont.
• Encourage studies that:
1. Use standard methods and reporting of results
2. Are larger than a few health facilities (although small
pilot studies are important before large-scale trials)
3. Have follow-up periods of at least 12 months
4. Use pre- versus post-intervention RCT designs that
include time series data to better characterize trends
5. Include process of care outcomes
‒ Useful programmatically
‒ Help explain causal link to other outcomes
(e.g., utilization of services or health outcomes)
6. Include cost estimates & cost-effectiveness analyses
7. Actively search for unintended negative consequences
![Page 117: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/117.jpg)
Recommendation 3
• Proposed research priorities
1. Large-scale RCTs of improvement collaboratives
‒ Studies have compelling results but important limitations
2. RCTs that compare different training and supervision models
for community health workers
3. Additional trials of strategies with large effect sizes, according
to network meta-analysis
− Ex 1: Consumer support + strengthen infrastructure +
institutional approaches + other management techniques
+ supervision + training (45 %-points)
− Ex 2: Financing/incentives + supervision + training
(35 %-points)
![Page 118: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/118.jpg)
Recommendation 4
• There should be support for ongoing efforts to
synthesize research findings via systematic
reviews and meta-analysis:
1. Guide programs, donors, and development
partners
2. Inform future research
• HCPP Review is a step towards the goal this
recommendation aims to achieve
![Page 119: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/119.jpg)
Extra slides:
network meta-
analysis
![Page 120: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/120.jpg)
Control group
Consumer
support +
low-intensity
training
[79.050]
Consumer
support +
supervision +
low-intensity
training
[79.052]
Consumer support +
strengthen
infrastructure +
financing/incentives +
institutional approaches
+ other management
techniques +
supervision +
low-intensity training
[79.077]
Supervision
only
[79.004]
Low-intensity
training only
[79.002]
Supervision + low-intensity training [79.006]
Consumer support +
other management
techniques + low-intensity
training [79.056]
Commodity supply +
low-intensity training
[79.035]
High-intensity training
only [79.003]
d e
c
a b
2 comparisons
A simple network of
studies: circles are
strategies (controls
considered a
“strategy”), and lines
are comparisons
![Page 121: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/121.jpg)
Extra slides:
Descriptive
results
![Page 122: Health Care Provider Performance Review - World Bank · 2019-08-20 · C:\alex\Improving_HW_performance\Analysis\Analysis for Europe 2014\ppt\ HCPP Sept 2014 v3 (30 min - World Bank](https://reader035.vdocuments.mx/reader035/viewer/2022062917/5ed7121362136e72fb7bc7be/html5/thumbnails/122.jpg)
A
I118
51
C35
49
J 43
17
G 18 1
H
17
17
17
D20
7
16
15
15
118
43
1
E
F
16
12
11
4
51
Combinations of HCP types
(80% [539/672] of comparisons from
all 491 studies)
A Physician
B Clinical officer
C Nurse/Midwife
D Pharmacist/laboratorian
E Paramedic/unspec. non-MD HCP
F Health educator/information officer
G Student
H Aide
I Community health worker
J Unspecified health professional
4
3
B