impact evaluation of malaria prevention and...
TRANSCRIPT
Impact Evaluation Workshop for Health Sector Reform
Cape Town, South Africa, December 7-11, 2009
Impact Evaluation of Malaria
Prevention and Treatment
Jed Friedman (World Bank)
Evaluating Health
Programs is Different
• Evaluation methods often used in medicine to
determine efficacy of treatment
• What we know less about:
– How to get people to utilize prevention / treatment
services?
– What is the most cost effective mode of
prevention / treatment given behavioral
response?
– What are the socioeconomic effects of health
interventions?
PLASMODIUM
“THE AGENT”
MOSQUITO
“THE VECTOR”
MAN
“THE HOST”
Some Differences
• HIV/AIDS: Largely behaviorally driven
• Malaria: Vector-born disease
…But the behavioral aspects are important
Treat Infected
- Early Diagnosis &
Drug Type (AMT, ACT)
Problems
- Access
- Compliance
- Cost-Effectiveness
- Long-Term Effect
Vector Control
- Prevent Breeding (DDT)
- Prevent Entry (Proofing)
- Prevent Bite (ITN, Spray)
Problems
- Resistance to Insecticides
- Compliance
Protect Everyone
- ITN, LLIN
- Mosquito Proofing
Problems
- Valuation/Usage
- Compliance
Malaria Control
Is Not Principally a Question of
Technical Innovation
“Widespread use of ITNs and state-of-the-art drugs
succeeded in cutting malaria deaths half in 2 countries most
heavily affected by the disease, Rwanda and Kenya.”
Washington Post (01/31/08)
“This is a genuinely historic achievement. This is not
theoretical. We do not have to wait for a vaccine or new
drugs. If we implement today’s technologies aggressively on
a national scale we will have a big impact.”
Richard Feachem, former Director of the Global Fund
“With the resources available, we should be able to eradicate
malaria before I hang up my lab coat.”
Peter Agre, Malaria Research Institute, JHU
Impact of Policy Change on Malaria
Prevalence: South AfricaNational Geographic 07/07
Policy Regime Switches
1. Pesticide Resistance
- DDT Stops: 1996
- Spraying Resumes: 2000
Drug Resistance
- Multidrug Therapy: 2000
Aggravating Exogenous Factors
- Refugee Flow
- Heavy Rains
- Worsening Resistance
In Kwazulu-Natal the combined effect of switching
from SP to the fixed combination of AL and IRS
with DDT was associated with a decrease in cases
of 78% and an increase in cure rate of 87%
(Barnes K, unpublished data; in Yeung et al. 2004).
Case 1: DDT - IndiaQuasi Experiment (Cutler et al. 2007)
• Long-term Effects of Malaria Eradication
• Outcome Measure: Educational Gains 1. Literacy Rate (LR); 2.School Completion Rate (SCR)
• Method: Quasi Experiment using Diff-in-Diff
Sample: 300,000 Households, 1 million Individual Observations (NSS)
- Control: Pre-Eradication Cohorts (C0): 1912-1952
- Treatment: Post-Eradication Cohorts (C1): 1962-1972
- Omitted: Eradication Cohorts: 1953-1961
1947 Pre-Eradication- Population: 334 million
- Cases: 75 million (annual)
- Prevalence: 22% (annual)
- Mortality: 800,000 (annual)
- Mortality: 10% of total deaths
1953 Intervention1st: 1953 NMCP Launched
DDT Spraying
- 2 Rounds per Year
- 125 Malaria Control Units
2nd: 1958 NMEP Launched
Gains12% ↑ in LR, SCR
- Malaria explains
half of these gains
- Income ↓ through
malaria 7-10%
Case 2: ITN – Kenya
Field Experiment (Cohen and Dupas 2008)
• Explore tradeoffs between cost-sharing (CS) & free distribution for ITNs
• Randomize price of ITNs (0 ≤ p <pPrevailingCS) in prenatal clinics in Kenya
• Evaluate impact on pregnant women
1. Demand / Uptake
- Cost-sharing (C/S) does considerably dampen demand.
- Uptake Drops: i) by 75% from 0 price to prevailing CS price; ii) by 20% for ↑10Ksh.
2. Usage
- No evidence that C/S reduces wastage on those who do not use the net.
- Free ITN owner is not less likely to use net than those who paid higher prices.
- Coverage (Uptake + Usage): 63% (Free Net) v. 14% (40Ksh)
3. Need (Health)
- No evidence that C/S induces selection of those who need net more.
- Those paying higher prices appear no sicker (anemia) than control group.
4. Compare Cost Effectiveness (Externality Assumptions)
- Number of child lives saved highest under free distribution.
- Free distribution is more CE when externality threshold is medium level.Source: Cohen and Dupas 2008
1) Demand for ITNs: Monthly
Net Sales by ITN Price
0
20
40
60
80
100
120
140
160
180
200
0 10 20 40
ITN Price (in Ksh)
Avera
ge N
um
ber
of
ITN
s
So
ld/D
istr
ibu
ted
per
Mo
nth
Source: Cohen and Dupas 2008
2) ITN Usage Rates by Price: Share of
“Takers” who Report Using ITN at Home
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0 10 20 40
ITN Price (Ksh)
All First Prenatal Visits Only First Pregnancy Only
Source: Cohen and Dupas 2008
Case 3: ITN – UgandaExperimental Evidence on ITNs from
Rubagano and Kimuli Villages (Hoffman1)
Do free goods stick to poor households?
Design
NHH=193 T1: Free Net: 71, T2: Cash Transfer: 72, C: Uncompensated: 50
Findings
- Wealth and endowment effects result in very few HHs selling free net (FN).
- Only 6% of FN would be sold in frictionless market. Accounting for
transaction cost would further reduce this number.
- No significant gender gradient in average compensated valuation of ITNs.
- Man have higher income elasticity of supply for ITNs. Men have higher ATP.
Implications
- Distributing FN to women ►less leakage.
- Marketing among men ► more effective.
Source: Hoffmann, Barrett, and Just (2007)
Case 3: ITN – UgandaExperimental Evidence on ITNs from
Rubagano and Kimuli Villages (Hoffman2)
Psychology, gender, intra-HH allocation of ITNs
Design:
NHH=143 T1: Free Net: 71, T2: Cash Transfer: 72
Findings
- Free nets lead to greater number of children covered, even for HHs with ATP.
- Net retention is higher for free nets (Endowment Effect): NFN > NCT
- Women tend to cover larger proportion of HH with nets.
- Intra-HH allocation of purchased nets (CT) depends on cost-benefit
calculations, with income-earners, net purchasers, and people often suffering
from malaria receiving it.
- Accompanied with a message, in-kind nets (FN) induce allocation to children.
Implications
Beyond price, mode of allocation and communication are important.
Source: Hoffmann (2007)
Overview of the World Bank’s Malaria
Impact Evaluation Program (MIEP)
• Program launched to conduct malaria policy relevant operational impact
evaluation under the Booster Program for Malaria Control
Goals:
• to generate evidence on effective approaches to increase utilization of
malaria prevention and treatment services
• to increase familiarity with IE approaches and build evaluative capacity in
national malaria control programs
•currently conducting evaluation studies in 6 countries assessing
effectiveness of a variety of control strategies
Access to effective treatment
Nigeria MIEP is assessing the involvement of trained community- and
private sector-based agents (CDDs and PMVs) in malaria prevention, and
case management to increase access to prompt diagnosis with RDTs and
treatment with ACTs.
Zambia estimating the gains in access through the introduction of RDTs
and ACTs through village Community Health Workers (CHW)
Access to effective treatment (cont)
Nigeria assessing the introduction of RDTs and ACTs to the workforce of
a large sugar cane plantation in order to estimate productivity costs of
adult malaria infection
India estimating the gains in quality of care through the involvement of
local organizations in supportive supervision and training of CHWs
Integrated vector control
Eritrea determining the cost-effectiveness of continued Indoor Residual
Spraying (IRS) in a low-endemic setting
School based programs
Kenya and Senegal asking whether the introduction of preventive
therapy through schools results in higher attendance and learning as well
as better student health
Overview of the World Bank’s Malaria
Impact Evaluation Program (MIEP) (cont.)
MIEP Portfolio SummaryCountry Project PIs Impact Evaluation Project Value
Eritrea IDF Grant – Capacity building for evidence-based policy making in the health sector
P. Carneiro
J. Keating
Experimental Design (RCT) of impact of indoor residual spraying, and incentives for improving larval habitat management
$485,000
India Leveraging local capacity to assist malaria control efforts
J. Friedman Experimental design of CBO assistance to government malaria prevention and fever case management initiatives
$200 million
Kenya School-based Malaria Prevention
S. Brooker
M. Jukes
Experimental Design (RCT) of teacher training and school-based intermittent preventive treatment
$4 million approved + $10.4 million Pipeline
Nigeria Malaria Control Booster Program
P. Carneiro
E. Velenyi
Experimental Design (RCT) of Community- and Private Sector-based Malaria Control
$180 million + $100 million Additional Financing
Senegal School-based Malaria Prevention
S. Brooker
M. Jukes
Experimental Design (RCT) of teacher training and school-based intermittent preventive treatment
$5 million
Zambia Zambia Access to ACTs Initiative
J. Friedman
E. Velenyi
Quasi Experimental Design (Matching and RCT) of Public Sector Supply Chain Management and Community- and Private Sector-based Malaria Control
$26.85 million
Conclusions
• Impact evaluation studies have already contributed much information for understanding the efficacy and cost-effectiveness of various malaria prevention options
• Recent impact evaluation studies will help policy makers work through certain malaria control decisions such as free ITN distribution vs. cost-recovery
• However many critical questions remain. For example, how do we affect people's behavior to ensure adoption and proper usage of nets? How do we increase the proportion of fever cases seeking treatment at facilities with adequate diagnostic and curative care?
• Impact Evaluation of Malaria Programs (See Handout)– Friedman, Legovini, and Velenyi; Development Dialogue Notes Vol. 1 (2009)
• This week we will discuss methods through which we can answer these questions.
References 1
1. Malaria Control http://www.malariasite.com/malaria/ControlOfMalaria.htm
2. Brown, D. “Malaria Deaths Drop in Rwanda, Kenya.” Washington Post January 31, 2008
3. Finkel, M. “Raging Malaria – Stopping a Global Killer.” National Geographic July 2007
4. Yeung et al. 2004. “Antimalarial Drug Resistance, Artemisinin-Based Combination Therapy, and the
Contribution of Modeling to Elucidating Policy Choices.” Am. J. Trop. Med. Hyg., 71(Suppl 2), 2004, pp.
179–186.
5. Cutler et al. 2007. “Mosquitoes: The Long-term Effects of Malaria Eradication in India.” NBER.
6. Cohen and Dupas. 2008. “Free Distribution or Cost-Sharing? Evidence from a Randomized Malaria
Prevention Experiment.” Draft. January. Presented at the Brookings Institution. January 24, 2008.
7. Hoffmann, Barrett, and Just. 2007. “Do free goods stick to poor households? Experimental evidence on
insecticide treated bednets.” Draft. November. Department of Applied Economics, Cornell University.
8. Hoffmann. 2007. “Psychology, gender, and the intra-household allocation of free and purchased
mosquito nets.” Draft. November. Department of Applied Economics, Cornell University.
9. Clarke et al. 2008. “Health and educational impact of intermittent preventive treatment of malaria in
schoolchildren: a cluster-randomized controlled trial.” Draft. LSHTM.
10. Arrow, K.J. et al., Eds. 2004. “Saving lives, buying time: economics of malaria drugs in an age of
resistance.” Board on Global Health. Washington, D.C.: Institute of Medicine.
11. Laxminarayan, Over, and Smith. 2005. “Will a Global Subsidy of Artemisinin-Based Combination
Treatment (ACT) for Malaria Delay the Emergence of Resistance and Save Lives?” World Bank Policy
Research Working Paper 3670, July.
References 2
12. Gollin and Zimmermann. 2007. “Malaria: Disease Impacts and Long-Run Income Differences.”
Discussion Paper No. 2997. IZA.
13. McCarthy, Wolf, and Wu. 1999. “Malaria and Growth.” World Bank Working Paper. WPS 2303.
14. Gallup and Sachs. 2001. “The economic burden of malaria.” Am. J. Trop. Med. Hyg. 64(1,2)S: 1-11.
15. Acemoglu and Johnson. 2006. “Disease and development: The effect of life expectancy on economic
growth.” NBER Working Paper 12269.
16. Weil. 2007. “Accounting for the effects of health on economic growth.” The Quarterly Journal of
Economics. August. Vol. 122, No. 3, Pages 1265-1306.
17. Bleakley. 2007. “Malaria in the Americas: A retrospective analysis of childhood exposure.” Department
of Economics, University of Chicago.
18. Lucas. 2005. Economic Effects of Malaria Eradication: Evidence from the Malarial Periphery.
Manuscript. Department of Economics, Brown University.
19. Hong. 2007. "A Longitudinal Analysis of the Burden of Malaria on Health and Economic Productivity:
The American Case." University of Chicago.
20. Barreca. 2007. "The Long-Term Economic Impact of In Utero and Postnatal Exposure to Malaria." UC
Davis.
21. Coleman et al. 2004. “A Threshold Analysis of the Cost-Effectiveness of Artemisinin-Based
Combination Therapies in Sub-Saharan Africa.” Am. J. Trop. Med. Hyg., 71(Suppl 2), 2004, pp. 196–
204.
22. Breman et al. 2006. “Conquering Malaria.” Eds. Jamison et al. Disease Control Priorities. Chapter 21.
Impact Evaluation Workshop for Health Sector Reform
Cape Town, South Africa, December 7011, 2009