![Page 1: Intervention on Vegetable Seed Minipacks€¦ · Intervention on Vegetable Seed Minipacks - ZAfter survey of users and non-users - Going back to the ~500 samples from the before surveys](https://reader034.vdocuments.mx/reader034/viewer/2022042313/5edcbe31ad6a402d66678a7d/html5/thumbnails/1.jpg)
Measuring Impact of Intervention on
Vegetable Seed Minipacks
Shovan Chakraborty Monitoring and Results Measurement Manager
Katalyst, Swisscontact
![Page 2: Intervention on Vegetable Seed Minipacks€¦ · Intervention on Vegetable Seed Minipacks - ZAfter survey of users and non-users - Going back to the ~500 samples from the before surveys](https://reader034.vdocuments.mx/reader034/viewer/2022042313/5edcbe31ad6a402d66678a7d/html5/thumbnails/2.jpg)
Table of Content
• Project History
• Project Approach
• Evolution of Monitoring and Results Measurement
• Positioning of MRM
• Intervention on Vegetable Seed Minipacks
• Result Chain on Minipacks
• Early efforts for Impact Assessment
• Process of Iterative Measurement
• Achievement of Minipacks
• Major Learning from Evaluation
![Page 3: Intervention on Vegetable Seed Minipacks€¦ · Intervention on Vegetable Seed Minipacks - ZAfter survey of users and non-users - Going back to the ~500 samples from the before surveys](https://reader034.vdocuments.mx/reader034/viewer/2022042313/5edcbe31ad6a402d66678a7d/html5/thumbnails/3.jpg)
Project History
Phase 1 Phase 2 Phase 3
Donors
DFID
SDC
SIDA CIDA Danida
The Netherlands
Duration Oct’02 – Mar’08 Mar’08 – Mar’13 Mar’13 –Mar’17
Budget CHF 26 million CHF 51 million CHF 32 million
Implementers Swisscontact, GIZ
![Page 4: Intervention on Vegetable Seed Minipacks€¦ · Intervention on Vegetable Seed Minipacks - ZAfter survey of users and non-users - Going back to the ~500 samples from the before surveys](https://reader034.vdocuments.mx/reader034/viewer/2022042313/5edcbe31ad6a402d66678a7d/html5/thumbnails/4.jpg)
Phase 1 Phase 2 Phase 3 • Developing markets for
transacted and
embedded, public BDS
(2002)
• Service, industrial and
rural sectors (2003)
• Pro-poor market
development (2004-5)
• MRM develops
• Market development
approach
• Large scale and
sustainable systemic
changes in targeted
industrial and rural
sectors
• MRM improves and
proves impact
• PPM for 1 year • Focus on high-value
agriculture sub-sectors • Deepen and widen pro-
poor impact • Capitalisation • MRM measures systemic
changes
Innovating, testing and
proving the methodology Reaching greater scale in
sectors Consolidating and anchoring in the
Bangladeshi context
General Approach of Project
Project History
![Page 5: Intervention on Vegetable Seed Minipacks€¦ · Intervention on Vegetable Seed Minipacks - ZAfter survey of users and non-users - Going back to the ~500 samples from the before surveys](https://reader034.vdocuments.mx/reader034/viewer/2022042313/5edcbe31ad6a402d66678a7d/html5/thumbnails/5.jpg)
Achievements in Phase 2
2.3 Million Beneficiaries
Income impact of ~USD 290 million
Project History
![Page 6: Intervention on Vegetable Seed Minipacks€¦ · Intervention on Vegetable Seed Minipacks - ZAfter survey of users and non-users - Going back to the ~500 samples from the before surveys](https://reader034.vdocuments.mx/reader034/viewer/2022042313/5edcbe31ad6a402d66678a7d/html5/thumbnails/6.jpg)
Targets in Phase 3
1.43 Million Beneficiaries
Income impact of ~USD 260 million
![Page 7: Intervention on Vegetable Seed Minipacks€¦ · Intervention on Vegetable Seed Minipacks - ZAfter survey of users and non-users - Going back to the ~500 samples from the before surveys](https://reader034.vdocuments.mx/reader034/viewer/2022042313/5edcbe31ad6a402d66678a7d/html5/thumbnails/7.jpg)
Project Approach
Poor Farmers
Market for Poor (M4P)
Value Chain Intermediaries
Value Chain Intermediaries
Value Chain Intermediaries
Companies In
te
rv
en
tio
ns
![Page 8: Intervention on Vegetable Seed Minipacks€¦ · Intervention on Vegetable Seed Minipacks - ZAfter survey of users and non-users - Going back to the ~500 samples from the before surveys](https://reader034.vdocuments.mx/reader034/viewer/2022042313/5edcbe31ad6a402d66678a7d/html5/thumbnails/8.jpg)
Evolution of Monitoring and Results Measurement (MRM)
2003
2004
2005
2007
2006
2008
2009
2010
2011
No clearly defined SYSTEM
Focused on ACTIVITIES & OUTPUTs… Thoughts on RESULTs CHAINs
Trial & error with RESULTs CHAINs
Draft of 1st MANUAL
Discussion on STANDARD starts; 1st MANUAL
Centralized TEAM; MOCK AUDIT
Intensive discussions on THEORY & PRACTICALITY
Refined LOGFRAME; New MANUAL; 2nd MOCK AUDIT
DCED audit
Strong ATTRIBUTION
DCED audit; REVIEW of processes and tools
2012
2013
![Page 9: Intervention on Vegetable Seed Minipacks€¦ · Intervention on Vegetable Seed Minipacks - ZAfter survey of users and non-users - Going back to the ~500 samples from the before surveys](https://reader034.vdocuments.mx/reader034/viewer/2022042313/5edcbe31ad6a402d66678a7d/html5/thumbnails/9.jpg)
Positioning of MRM
Steering
Proving and Improving by ensuring the quality of information
![Page 10: Intervention on Vegetable Seed Minipacks€¦ · Intervention on Vegetable Seed Minipacks - ZAfter survey of users and non-users - Going back to the ~500 samples from the before surveys](https://reader034.vdocuments.mx/reader034/viewer/2022042313/5edcbe31ad6a402d66678a7d/html5/thumbnails/10.jpg)
Positioning of MRM
Strongly mainstreamed among project staff
Reporting
Results measurement and analysis
Intervention management
Sector selection and
strategy development
![Page 11: Intervention on Vegetable Seed Minipacks€¦ · Intervention on Vegetable Seed Minipacks - ZAfter survey of users and non-users - Going back to the ~500 samples from the before surveys](https://reader034.vdocuments.mx/reader034/viewer/2022042313/5edcbe31ad6a402d66678a7d/html5/thumbnails/11.jpg)
Intervention on Vegetable Seed Minipacks
Context of the Vegetable Seed Market in Bangladesh
S E E D M A R K E T
Quality Seeds
Farmers Farmers
![Page 12: Intervention on Vegetable Seed Minipacks€¦ · Intervention on Vegetable Seed Minipacks - ZAfter survey of users and non-users - Going back to the ~500 samples from the before surveys](https://reader034.vdocuments.mx/reader034/viewer/2022042313/5edcbe31ad6a402d66678a7d/html5/thumbnails/12.jpg)
Intervention on Vegetable Seed Minipacks
Identified Major Constraint
Seeds
Land size
![Page 13: Intervention on Vegetable Seed Minipacks€¦ · Intervention on Vegetable Seed Minipacks - ZAfter survey of users and non-users - Going back to the ~500 samples from the before surveys](https://reader034.vdocuments.mx/reader034/viewer/2022042313/5edcbe31ad6a402d66678a7d/html5/thumbnails/13.jpg)
Intervention on Vegetable Seed Minipacks
• Target: to introduce Minipacks priced USD 0.12-0.50
• Partner Seed Companies
– Lalteer Seed
– AR Malik
![Page 14: Intervention on Vegetable Seed Minipacks€¦ · Intervention on Vegetable Seed Minipacks - ZAfter survey of users and non-users - Going back to the ~500 samples from the before surveys](https://reader034.vdocuments.mx/reader034/viewer/2022042313/5edcbe31ad6a402d66678a7d/html5/thumbnails/14.jpg)
Result Chain on Minipacks Additional income
Small, marginal, and homestead farmers have increased yields due
to use of good quality seed
Farmers increase use of quality seeds
Farmers have increased access to quality vegetable seed
Channel members are making quality seed available to
last-mile farmers through mini packets
Companies promote and distribute quality vegetable seeds to
last-mile farmers through networks of knowledgeable MSVs
Facilitated 2 seed companies to assess market, develop strategic
plan & packaging for promoting vegetable seeds in mini packs
GOAL
PURPOSE
OUTCOME
OUTPUT
ACTIVITY RESULTS
ACTIVITY
Intervention on Vegetable Seed Minipacks
![Page 15: Intervention on Vegetable Seed Minipacks€¦ · Intervention on Vegetable Seed Minipacks - ZAfter survey of users and non-users - Going back to the ~500 samples from the before surveys](https://reader034.vdocuments.mx/reader034/viewer/2022042313/5edcbe31ad6a402d66678a7d/html5/thumbnails/15.jpg)
Intervention on Vegetable Seed Minipacks
![Page 16: Intervention on Vegetable Seed Minipacks€¦ · Intervention on Vegetable Seed Minipacks - ZAfter survey of users and non-users - Going back to the ~500 samples from the before surveys](https://reader034.vdocuments.mx/reader034/viewer/2022042313/5edcbe31ad6a402d66678a7d/html5/thumbnails/16.jpg)
Early Efforts for Impact Assessment
Intervention on Vegetable Seed Minipacks
Sales Target for First Season: 100,000 minipacks
Sales Before End of First Season: > 200,000 minipacks
Realization about Scale Post DCED Audit drive to improve attribution
![Page 17: Intervention on Vegetable Seed Minipacks€¦ · Intervention on Vegetable Seed Minipacks - ZAfter survey of users and non-users - Going back to the ~500 samples from the before surveys](https://reader034.vdocuments.mx/reader034/viewer/2022042313/5edcbe31ad6a402d66678a7d/html5/thumbnails/17.jpg)
Major Challenges During Early efforts for Impact Assessment:
- Farmer tracing
- Sample Bias
- Understanding the distribution channel
- Assessing the number of users
- Getting time from respondents
- Covering the geographic spread
- Varying seasonality across the country
- Recall bias
- Phrasing style in questionnaire
- Defining the attribution strategy
Intervention on Vegetable Seed Minipacks
![Page 18: Intervention on Vegetable Seed Minipacks€¦ · Intervention on Vegetable Seed Minipacks - ZAfter survey of users and non-users - Going back to the ~500 samples from the before surveys](https://reader034.vdocuments.mx/reader034/viewer/2022042313/5edcbe31ad6a402d66678a7d/html5/thumbnails/18.jpg)
Process of Iterative Measurement - User Tracking Survey in Informal Markets
Intervention on Vegetable Seed Minipacks
Formal Market
Informal Market Informal Market
Informal Market Informal Market
![Page 19: Intervention on Vegetable Seed Minipacks€¦ · Intervention on Vegetable Seed Minipacks - ZAfter survey of users and non-users - Going back to the ~500 samples from the before surveys](https://reader034.vdocuments.mx/reader034/viewer/2022042313/5edcbe31ad6a402d66678a7d/html5/thumbnails/19.jpg)
Process of Iterative Measurement - User Tracking Survey in Informal Markets
Intervention on Vegetable Seed Minipacks
- Use of company sales data in more than 50 districts (out of 64) - ~1,100 samples from 6 divisions (out of 7)
- Covered good geographic spread - Enumerators could get time from respondents - Take details of purchase and address of buyer/user farmer - Average no. of minipacks bought per user = ~1.8 - Total no. of users = (no. packs sold) / (packs bought per user)
- ~285,000 users by end of a year - Interviews with mobile seed vendors
- Gained understanding of the distribution channel - Assessed no. of repeat buyers: 25% - Better understanding of seasonality across the country
![Page 20: Intervention on Vegetable Seed Minipacks€¦ · Intervention on Vegetable Seed Minipacks - ZAfter survey of users and non-users - Going back to the ~500 samples from the before surveys](https://reader034.vdocuments.mx/reader034/viewer/2022042313/5edcbe31ad6a402d66678a7d/html5/thumbnails/20.jpg)
Process of Iterative Measurement - User Tracking Survey in Informal Markets
Intervention on Vegetable Seed Minipacks
![Page 21: Intervention on Vegetable Seed Minipacks€¦ · Intervention on Vegetable Seed Minipacks - ZAfter survey of users and non-users - Going back to the ~500 samples from the before surveys](https://reader034.vdocuments.mx/reader034/viewer/2022042313/5edcbe31ad6a402d66678a7d/html5/thumbnails/21.jpg)
Process of Iterative Measurement - ‘Before’ survey of users
- Representative from major categories
- Crop, crop value, geographical spread, etc.
- >300 samples (x 1.5 crops per sample farmer)
- Covering 13 sub-districts of 5 divisions
- Defining attribution methods
Intervention on Vegetable Seed Minipacks
Fallow 35%
35%
30%
![Page 22: Intervention on Vegetable Seed Minipacks€¦ · Intervention on Vegetable Seed Minipacks - ZAfter survey of users and non-users - Going back to the ~500 samples from the before surveys](https://reader034.vdocuments.mx/reader034/viewer/2022042313/5edcbe31ad6a402d66678a7d/html5/thumbnails/22.jpg)
Process of Iterative Measurement
Intervention on Vegetable Seed Minipacks
- ‘Before’ survey of non-user comparison group
- Representative of needed comparison categories
- Sample selection from same localities within villages
- Reducing sample bias
- >200 samples
- Covering 13 sub-districts of 5 divisions
![Page 23: Intervention on Vegetable Seed Minipacks€¦ · Intervention on Vegetable Seed Minipacks - ZAfter survey of users and non-users - Going back to the ~500 samples from the before surveys](https://reader034.vdocuments.mx/reader034/viewer/2022042313/5edcbe31ad6a402d66678a7d/html5/thumbnails/23.jpg)
Process of Iterative Measurement
Intervention on Vegetable Seed Minipacks
- Reduction of Recall Bias
SUMMER WINTER WINTER
Sow
ing
Sow
ing
Har
vest
ing
Har
vest
ing
Before Surveys
After Survey
![Page 24: Intervention on Vegetable Seed Minipacks€¦ · Intervention on Vegetable Seed Minipacks - ZAfter survey of users and non-users - Going back to the ~500 samples from the before surveys](https://reader034.vdocuments.mx/reader034/viewer/2022042313/5edcbe31ad6a402d66678a7d/html5/thumbnails/24.jpg)
Process of Iterative Measurement
Intervention on Vegetable Seed Minipacks
- ‘After’ survey of users and non-users
- Going back to the ~500 samples from the before surveys
- Covering 13 sub-districts of 5 divisions
- Space to put in additional questions
- Quantitative towards more qualitative
- Proving towards more improving
![Page 25: Intervention on Vegetable Seed Minipacks€¦ · Intervention on Vegetable Seed Minipacks - ZAfter survey of users and non-users - Going back to the ~500 samples from the before surveys](https://reader034.vdocuments.mx/reader034/viewer/2022042313/5edcbe31ad6a402d66678a7d/html5/thumbnails/25.jpg)
Process of Iterative Measurement
Intervention on Vegetable Seed Minipacks
- Challenges in Data Analysis
- Multiple crops
- Multiple type of conversions
- Data cleaning
![Page 26: Intervention on Vegetable Seed Minipacks€¦ · Intervention on Vegetable Seed Minipacks - ZAfter survey of users and non-users - Going back to the ~500 samples from the before surveys](https://reader034.vdocuments.mx/reader034/viewer/2022042313/5edcbe31ad6a402d66678a7d/html5/thumbnails/26.jpg)
Process of Iterative Measurement
Intervention on Vegetable Seed Minipacks
- Results
- ~70% users were getting benefit (based on DiD)
- Income impact per individual per season: USD 17
- Repeat use: 25%
- Second user tracking survey
- Retest ‘no. of minipacks bought per farmer’
- Repeat use: 50%
- Other qualitative aspects in the distribution channel
![Page 27: Intervention on Vegetable Seed Minipacks€¦ · Intervention on Vegetable Seed Minipacks - ZAfter survey of users and non-users - Going back to the ~500 samples from the before surveys](https://reader034.vdocuments.mx/reader034/viewer/2022042313/5edcbe31ad6a402d66678a7d/html5/thumbnails/27.jpg)
Achievements
Intervention on Vegetable Seed Minipacks
By 18 months
Available in more than 50 districts (out of 64)
~ 2million Minipacks sold
Used by 660,000
Benefitting 460,000
Beneficiaries below USD 2.50 a day: 88% (based on PPI)
Beneficiaries below USD 1.25 a day: 23% (based on PPI)
![Page 28: Intervention on Vegetable Seed Minipacks€¦ · Intervention on Vegetable Seed Minipacks - ZAfter survey of users and non-users - Going back to the ~500 samples from the before surveys](https://reader034.vdocuments.mx/reader034/viewer/2022042313/5edcbe31ad6a402d66678a7d/html5/thumbnails/28.jpg)
Intervention on Vegetable Seed Minipacks
![Page 29: Intervention on Vegetable Seed Minipacks€¦ · Intervention on Vegetable Seed Minipacks - ZAfter survey of users and non-users - Going back to the ~500 samples from the before surveys](https://reader034.vdocuments.mx/reader034/viewer/2022042313/5edcbe31ad6a402d66678a7d/html5/thumbnails/29.jpg)
Major Learning from Evaluation
Intervention on Vegetable Seed Minipacks
- Iterative studies are effective in designing context-specific
methodology
- Triangulation can help
- A combination of techniques and tools can give leverage
- Mainstreaming of MRM among project staff is crucial
- DCED alignment should be continued
![Page 30: Intervention on Vegetable Seed Minipacks€¦ · Intervention on Vegetable Seed Minipacks - ZAfter survey of users and non-users - Going back to the ~500 samples from the before surveys](https://reader034.vdocuments.mx/reader034/viewer/2022042313/5edcbe31ad6a402d66678a7d/html5/thumbnails/30.jpg)
Over to You