Innovations for Poverty Action

Evaluating the Impact of Agricultural Development Programs

Africa Rising23 October 2012

Agenda

Introduction: IPA and our research methods

IPA Agricultural Research Projects

M&E Tools

The Way Forward

Introduction

Introduction to IPA and IPA’s research methods

What does IPA do?

Established in 2002 to measure the effects of poverty alleviation programs Uses the randomized controlled trial (RCT) method

of rigorous evaluation Works in the areas of agriculture, health, education,

microfinance, water and sanitation, and governance Openly shares findings about what does and doesn’t

work with policymakers, donors, practitioners, researchers and other critical stakeholders

Established in Ghana in 2007 Ongoing national and regional projects in

agriculture, education, health and microfinance

What is randomized control trial evaluation?

Control Treatment

Impact

Our Approach

We generate insights on what works and what does not through randomized evaluations, and ensure that those findings will be useful to, and used by practitioners and policy makers

Innovate Evaluate Replicate Communi-cate Scale

- Understand market failures-Develop innovative solutions to poverty- Use frontier knowledge from economics, and psychology

Randomized Controlled Trials-Impact evaluations- Comparing variations of an intervention- Experiment with product designs

Replicate evaluations in various settings to :- Generalize research findings - Tell practitioners what works (and not), when

Effectively communicate to practitioners: -Conferences-Workshops with policy makers and practitioners-Policy memos and focus notes

Facilitate scale-up of effective solutions :- Active policy outreach -Practitioners’ toolkits - Hands-on technical assistance

• Surprisingly little hard evidence on what works

• Can do more with given budget with better evidence

• If people knew money was going to programs that worked, could help increase pot for anti-poverty programs

• We should be asking:• Which programs work best, why and when?• How can we scale up what works?

Why Impact Evaluation?

IPA Agricultural Research Projects

Examining Underinvestment in Agriculture

Background: EUI (2008-2012)

Research design EUI: Examining Underinvestment in Agriculture Investigators: economists at Yale and Ghana Legon Question: why do smallholders underinvest in farms? Hypotheses: capital constraints and risk aversion Treatments: unconditional cash grants and rainfall

index insurance Findings

Investment effects: insurance significantly increased farm investment, while capital alone did not

Profitability effects: higher investment did not lead to higher farm profitability

Insurance demand: high, when priced reasonably

IPA Agricultural Research Projects

Disseminating Innovative Resources and Technologies to

Smallholders

EUI and DIRTS

Questions that arise from EUI results: How can we take advantage of increased investment by

insured farmers? How can farmers move away from risk-averse farming

techniques, and towards profit maximization? What influences investment decisions? Information?

Access to fertilizer and seeds?

Background: DIRTS (2013-2014)

Research design DIRTS: Disseminating Innovative Resources and

Technologies to Smallholders Investigators: economists at Yale, UDS, SARI, IFPRI Question: building on EUI findings, why are smallholder

farm productivity and profitability levels so low, even when investment increases?

Hypotheses: risk aversion, limited access to quality inputs, limited access to good information

Treatments: varying combinations of: (1) drought index insurance (2) access to improved-yield technologies and (3) Community Extension Agent (CEA) program

IPA Agricultural Research Projects

Community Extension Agent Pilot

CEA: Community Extension Agent program Partners: IPA, MoFA, NRGP, Grameen Foundation Objective: complement existing extension services by training

community members to use Android phones to provide better information resources to farmers. CEAs will be connected to AEAs, providing a link between

AEAs and the community. Improved information developed in content workshops with

key stakeholders and experts made available through Grameen application

Messaging allows for improved communication between supervisors, CEAs, AEAs, and community farmers

Training provided through modules sent via phones Collecting quality data through survey app

Background: CEA pilot (2012)

The Way Forward

Partnership Opportunities

Partnership Opportunities: What we bring

Program evaluation study design

Survey data collection

M&E tools Paper Electronic Data quality protocols

Partnership Opportunities: What we are looking for

On current projects: Integrating agricultural research findings into best practices

resources Identifying context-relevant technology input packages to test Designing mobile extension programs Designing community extension agent programs

Future project interests: Identifying promising solutions for increasing farmer investment and

profitability for evaluation Childhood nutrition Other issues of importance to development partners?

Contact Information

Annie Smith ◦ rsmith@poverty-action.orgSenior Project Associate

Elizabeth Schultz◦ eschultz@poverty-action.orgGhana Research Cluster Manager

M&E Tools

M&E Tools: Quality Data for Quality Evaluation

M&E tools: Why?

Is the program being implemented? Record keeping by implementing agents Spot checks Interviews with program recipients Spending tracking

Is the program influencing outcomes as measured by key indicators? Administrative data (eg. Fertilizer sales) Survey data (eg. Farmer-reported use of fertilizer) Observational data (eg. Fertilizer use witnessed by

enumerators)

M&E tools: How?

Paper: low investment, easy to teach Field monitoring records Paper surveys

Electronic: fast, additional data collection capabilities Netbooks: electronic surveys Mobile: automatic recording of database use patterns by

CEAs Mobile: collection of GPS coordinates for monitoring Mobile: frequent labor use data collection by CEAs

M&E tools: Ensuring Quality

Collect the right data

Is the data source biased?

Does the data source have the capacity to do quality data collection?TimeTraining ToolsFunds