rapid mobile phone-based (ramp) survey
DESCRIPTION
Rapid Mobile Phone-based (RAMP) survey. Changing the way we plan and manage health and development surveys Version September 10th 2012. Presentation overview. Introducing RAMP: an overview How does RAMP work? Stakeholder benefits RAMP survey sampling approach - PowerPoint PPT PresentationTRANSCRIPT
www.ifrc.orgSaving lives, changing minds.
RAMPRAMP
Rapid Mobile Phone-based (RAMP) survey
Changing the way we plan and manage health and development surveys
Version September 10th 2012
www.ifrc.orgSaving lives, changing minds.
RAMP
Presentation overview
1) Introducing RAMP: an overview
2) How does RAMP work?
3) Stakeholder benefits
4) RAMP survey sampling approach
5) How much does a RAMP survey cost?
6) Costs of the RAMP pilot surveys
7) RAMP malaria survey
8) Reporting and dissemination of results
9) RAMP survey toolkit
www.ifrc.orgSaving lives, changing minds.
RAMP
Introducing RAMP: an overview
www.ifrc.orgSaving lives, changing minds.
RAMP
Purpose of RAMP
To provide a survey methodology with which Red Cross Red Crescent National Societies, governments, NGOs and other partners can conduct health and development surveys: at reduced cost with limited technical assistance to achieve high standards of survey design and quality
To decrease dramatically the time and effort needed to complete data collection and have results available for decision-making
www.ifrc.orgSaving lives, changing minds.
RAMP
Background
IFRC and partners (WHO, CDC) developed and refined the RAMP methodology and tools over several years
Based on: improvements to the EPI survey method (WHO EPI) improvements based on scientific article published by
Donna Brogan, Emory professor of biostatistics, Emory U, Atlanta, USA, specialist in survey methods and analysis, and Emory and CDC colleagues in 1994
public-domain mobile phone software Pilot tested in four surveys in Africa 2011-2012
(Kenya, Namibia and Nigeria)
www.ifrc.orgSaving lives, changing minds.
RAMP
Improvement over old EPI methodSurvey activity Old EPI method Improved
RAMP method
Choosing start point (first household) in the cluster
Biased: “Spin the bottle method” random direction
Simple random sampling
Choosing subsequent households (2nd, 3rd, etc.)
Often biased in practice: next nearest door
Simple random sampling
Fixed number of households visited per cluster
Biased: quota sampling without weight adjustment
Yes
Attempt to re-interview missing households
No Yes
Adjusts for non-response, household and individual
No Yes
Adjusts for inaccurate population estimate of clusters
No Yes (as option)
Analysis based on weight adjustments
No Yes
www.ifrc.orgSaving lives, changing minds.
RAMP
Key features of RAMP
Simplicity of sampling design to facilitate survey implementation and reduce field costs
Use of standard survey sampling methods Use of mobile phones and internet for data collection
that enables: web-based questionnaire design downloading of questionnaires onto standard mobile
phones data collection using low-cost mobile phones
Real-time, web-based dataset that can be easily accessed and exported for rapid analysis and reporting
Availability of many tools to guide survey planning and implementation
www.ifrc.orgSaving lives, changing minds.
RAMP
Changing the way we work
Paper questionnaires filled out in the field
Data entered into a computer at a central location
Data analysis and reporting often takes months to complete
Local capacity is often under-utilized and there is a dependence on external experts
Mobile and internet-based technologies are used to reduce the time taken from data collection to reporting
RAMP changes the way that data are collected and enables rapid reporting of results
RAMP empowers local ownership of evaluation and research
The “old” The “new”
www.ifrc.orgSaving lives, changing minds.
RAMP
When might a RAMP survey be suitable?
Surveys where rapid results are key Surveys where cost is a significant issue Sub-district surveys involving multiple villages that are
typically carried out by non-governmental organizations (NGOs)
National, provincial or district-level surveys carried out by Ministry of Health (MoH) or government departments (immunization, maternal and child health, malaria, HIV/AIDS, nutrition, water and sanitation, etc.) to collect intervention coverage data quickly and at low cost
www.ifrc.orgSaving lives, changing minds.
RAMP When might a RAMP survey be suitable? (cont’d)
District-level surveys conducted by the district health management team
Baseline and endline surveys for donor-funded projects/programmes
Repeated surveys to track time trends for key indicators
www.ifrc.orgSaving lives, changing minds.
RAMP When might a RAMP survey be suitable? Flexibility
Items that can be adjusted Comments
Adjust precision ±10%, 5%, 3%, etc.
Adjust indicator type (denominator of indicator) including mixtures of indicator types
- Person all ages- Children <5 years old- Pregnant women- Households- Schools
Adjust number of domains - 1 domain with 30 clusters- 2 domains with 30 clusters to
compare statistically - 10 domains with 30 clusters each to
compareAdjust overall sample size - 200 to 5000 households
www.ifrc.orgSaving lives, changing minds.
RAMP When might a RAMP survey not be suitable: limitations?
Very long questionnaires with a large quantity of skip patterns Paper method or commercial mobile phone software
might work better
www.ifrc.orgSaving lives, changing minds.
RAMP
How does RAMP work?
(Short version)
www.ifrc.orgSaving lives, changing minds.
RAMP
With RAMP you can transform a standard mobile phone into a cutting-edge evaluation and research tool
Conduct surveys and capture data
froma standard mobile
phone
Manage surveys, people and data from your web-based server
www.ifrc.orgSaving lives, changing minds.
RAMP
How does it work?
www.ifrc.orgSaving lives, changing minds.
RAMP
Web-based server
Create a free account using Datadyne’s Episurveyor software
Access EpiSurveyor’s web-based server from a web browser anywhere in the world
Design your questionnaire with embedded logic and in multiple languages
Monitor, manage and communicate with your team
Export data and analyse results in real-time
www.episurveyor.org/user/index
www.ifrc.orgSaving lives, changing minds.
RAMP
Why use mobile phones to collect data?
Real-time data entry on mobile phones Daily upload of data from mobile phone over 2G mobile
network to internet database Real-time data monitoring and data quality checks Real-time data cleaning Real-time data analysis Rapid production of survey results within hours or days
of last interview
www.ifrc.orgSaving lives, changing minds.
RAMPMobile phone applicationTransforms complex logic into an effortless step-by-step process
• Install Java application onto a standard mobile phone
• Log in to EpiSurveyor using user name and password, and download form
• Capture data in the field, even without network coverage
• Enter data using the interface of the mobile phone
• Automated submission of data when 2G reception is available
www.ifrc.orgSaving lives, changing minds.
RAMP
How does RAMP work?
(Longer version)
www.ifrc.orgSaving lives, changing minds.
RAMP RAMP takes advantage of two technologies
Low-cost, standard mobile phones
Web-based, easily accessible software application (EpiSurveyor) that enables mobile phones to become a data collection platform
www.ifrc.orgSaving lives, changing minds.
RAMP
How does it work?
www.ifrc.orgSaving lives, changing minds.
RAMP Web-based server
• Get started and set up your team to work with the EpiSurveyor software
• Create a free account using EpiSurveyor
• Assign privileges• Access the web-
based server from anywhere in the world
www.ifrc.orgSaving lives, changing minds.
RAMP Design your questionnaire
•
Collaborate with others to design the web-based questionnaire• Easy to master• Supports multiple languages • Embedded logic and skip patterns
www.ifrc.orgSaving lives, changing minds.
RAMP
www.ifrc.orgSaving lives, changing minds.
RAMP
www.ifrc.orgSaving lives, changing minds.
RAMP
www.ifrc.orgSaving lives, changing minds.
RAMP Store questionnaire on phone
• Survey questionnaire is downloaded onto the mobile phone
• Install Java application onto a standard mobile phone
• Log in to EpiSurveyor using a user name and password and download the forms
www.ifrc.orgSaving lives, changing minds.
RAMP Collect data in the field
Fieldworker retrieves the questionnaires on the mobile phone Enters data directly into the mobile phone during interviews
www.ifrc.orgSaving lives, changing minds.
RAMP Data sent to web-based server
…
Fieldworker submits data when 2G reception is available Data are available immediately for processing and
analysis Can transfer data from phone to computer if 2G cell
network is not available
www.ifrc.orgSaving lives, changing minds.
RAMP Data monitoring and analysis
Real-time data monitoring Real-time data cleaning Real-time analysis Allows for preliminary
analysis even before the data collection is complete
www.ifrc.orgSaving lives, changing minds.
RAMP Produce survey results, rapidly
RAMP deceases the time between data collection and the production of the survey results
The results can be available within days of the last interviews
www.ifrc.orgSaving lives, changing minds.
RAMP So, why use mobile phones to collect data?
Real-time data entry on mobile phones Daily upload of data from mobile phone over
2G cell network to internet database (eliminates paper-to- electronic database transfer)
Real-time data monitoring and data quality checks
Real-time data cleaning Real-time data analysis Rapid production of survey results within hours
or days of last interview
www.ifrc.orgSaving lives, changing minds.
RAMP
Stakeholder benefits
www.ifrc.orgSaving lives, changing minds.
RAMP
Stakeholders
<Insert a photo collage illustrating the various stakeholders (decision-makers; programme managers; evaluators/researchers; fieldworkers)>
www.ifrc.orgSaving lives, changing minds.
RAMP
Stakeholder benefits: decision-makers
• Scalable solution for studies of varying sizes
• Optimizes resource usage and reduces environmental impact
• No software licensing or subscriptions
• No consultants needed• Maintain control of data; data
not analysed by third party• Data rapidly available for
dissemination and decision-making
www.ifrc.orgSaving lives, changing minds.
RAMP Stakeholder benefits: programme managers
• Do not have to reinvent the wheel. The RAMP toolkit offers a variety of knowledge and tools to use and adapt for needs and context of programme managers
• Enables monitoring of survey team work rate, productivity and quality
www.ifrc.orgSaving lives, changing minds.
RAMP Stakeholder benefits: evaluators and researchers
• Incorporate a multitude of question types with custom logic and validation
• Manage and upload surveys in multiple languages
• Export data for custom analysis using any statistical analysis package
www.ifrc.orgSaving lives, changing minds.
RAMP
Stakeholder benefits: fieldworkers
• Builds local capacity for monitoring and evaluation
• Use standard and familiar mobile phones
• No more paper to collect, transport or return
• Automated submission of data when network reception is available
www.ifrc.orgSaving lives, changing minds.
RAMP
RAMP survey sampling approach
www.ifrc.orgSaving lives, changing minds.
RAMP
RAMP survey sampling approach
Simplicity of sampling design to facilitate survey implementation and reduce field costs
Uses standard survey sampling methodologies Uses segmentation to break clusters into small
pieces makes fieldwork more manageable lowers the cost of field surveys
Households are selected by simple random sampling at the last stage
Described in detail in the RAMP toolkit (Volumes 1 and 2)
www.ifrc.orgSaving lives, changing minds.
RAMP
RAMP: a two-stage cluster survey
First stage: selection of clusters using probability proportionate to estimated size (PPES)
Second stage: selection of households segmentation of clusters into smaller segments using
PPES (if there are too many households for simple random sampling [SRS])
SRS of a fixed number of households in a selected segment
www.ifrc.orgSaving lives, changing minds.
RAMP How is the RAMP method different from MIS/DHS?MIS/DHS RAMP
Complex design, uses external consultants to design survey
Simple design, external consultants not needed
Listing of all households is done in all clusters; expensive, often taking several days in each cluster
Divides clusters into manageable-sized segments using standard survey methods; takes <1/2 day
Simple random sampling of households (from the cluster list)
Simple random sampling of households (from the final segment list)
Real-time data cleaning not possible Real-time data cleaning during the survey
Real-time data analysis not possible; results take several months
Real-time data analysis and results/draft report finished within 3 days of last interview
Data analysis done by third-party consultants
Organization performs analysis, building capacity and maintaining control of data
www.ifrc.orgSaving lives, changing minds.
RAMP
Accuracy of the estimates
Sampling bias Very, very small theoretical bias
related to potential errors in the estimated sizes of clusters (there is an option showing how to eliminate this potential bias)
related to potential errors in the estimated sizes of segments (bias likely to be nearly non-existent)
Potential non-sampling bias generally more important than sampling issues Examples including non-response, information bias
including social desirability, faulty questionnaires, interviewer recording errors, data processing errors and others
www.ifrc.orgSaving lives, changing minds.
RAMP
The RAMP survey team
www.ifrc.orgSaving lives, changing minds.
RAMP
RAMP survey team
Programme manager Survey coordinator Local data manager/data analyst Field survey team
field supervisor, interviewers, drivers, local guide Supervisory support and monitoring team
www.ifrc.orgSaving lives, changing minds.
RAMP
How much does a RAMP survey cost?
www.ifrc.orgSaving lives, changing minds.
RAMP
Cost of a RAMP survey (average)Description Cost (US $)
Training (4 or 5 days) including two facilitators 10,623
Field survey, including transportation, daily allowances and accommodation 12,415
Mobile phones, accessories and air time 3,806
Survey administration 2,243
Total in-country expenditure (US $) US $ 29,087
www.ifrc.orgSaving lives, changing minds.
RAMP
Costs of the RAMP pilot surveys
www.ifrc.orgSaving lives, changing minds.
RAMP
Cost of the Kenya RAMP surveyDescription Cost (US $)
Training (4 days) including two facilitators 8,792
Number of trainees 20
Field survey, including transportation, daily allowances and accommodation (vehicles provided free of charge by KRC)
6,767
Number of survey teams 6
Mobile phones, accessories and air time 3,261
Survey administration 3,355
Total in-country expenditure (US $) US $ 22,175
www.ifrc.orgSaving lives, changing minds.
RAMP
Cost of the Namibia RAMP survey
Description Cost (US $)
Training (5 days) including two facilitators 12,425Number of trainees 18Field survey, including transportation, daily allowances and accommodation (vehicles rented) 17,134
Number of survey teams 6Mobile phones, accessories and air time 3,433Survey administration 1,782Total in-country expenditure (US $) US $ 34,774
www.ifrc.orgSaving lives, changing minds.
RAMP
Cost of the Namibia II RAMP survey
Description Cost (US $)
Training (4 days) including two facilitators 10,912Number of trainees 18Field survey, including transportation, daily allowances and accommodation (vehicles rented) 14,531
Number of survey teams 6Mobile phones, accessories and air time 6,203Survey administration 3,247Total in-country expenditure (US $) US $ 34,893
www.ifrc.orgSaving lives, changing minds.
RAMP
Cost of the Nigeria RAMP survey
Description Cost (US $)
Training (5 days) including two facilitators 10,362
Number of trainees 20
Field survey, including transportation, daily allowances and accommodation (vehicles rented) 11,229
Number of survey teams 6
Mobile phones, accessories and air time 2,326
Survey administration (includes locally hired data manager) 3,943
Total in-country expenditure (US $) US $ 27,860
www.ifrc.orgSaving lives, changing minds.
RAMP
RAMP malaria survey
www.ifrc.orgSaving lives, changing minds.
RAMP Initial focus of the RAMP survey tool: malaria
First RAMP surveys were done to evaluate malaria bed net mass distribution programmes
Sample size was calculated to provide results to guide management decision-making
Questionnaires created based on alignment with MIS/RBM; core RBM/WHO indicators related to nets are measured
Training curriculum and fieldwork procedures have been developed and can be adapted to the local needs and situation
Rapid reporting of results (a survey bulletin and a survey report are produced)
www.ifrc.orgSaving lives, changing minds.
RAMP
Survey methods
Cross-sectional, population-based survey Sampling frame:
First stage: selection of 30 clusters/PSUs using probability proportional to estimated size (PPES) sampling
Second stage: segmentation of the cluster/PSU; one segment chosen using PPES
Simple random sampling (SRS) to choose 10 households in the segment
Sample size: 300 households Precision:
+/- 7-8% for each key net indicator
www.ifrc.orgSaving lives, changing minds.
RAMP
Questionnaires
Survey questionnaires (modelled and adapted from MIS/RBM) Household questionnaire
number of sleeping spaces IRS household characteristics (wealth asset questions, etc.)
Person roster number and age of persons slept in household last
night diagnosis of malaria/fever in children under five and
its treatment Net roster
type of nets, source of nets who slept under each net last night
www.ifrc.orgSaving lives, changing minds.
RAMP
Recruitment and training
Interviewers and field supervisors recruited and trained Training: 4 to 5 days RAMP toolkit: Volume 3, Guide for trainers, provides
sample agenda, curriculum, tools and forms
www.ifrc.orgSaving lives, changing minds.
RAMP
Training content and methods
Content survey background and purpose questionnaires informed consent mobile phone basics interview techniques field procedures field logistics/reporting supervisor training
Methods presentations, role play, group discussion, demonstrations,
field practice (2 outings), review of data collected
www.ifrc.orgSaving lives, changing minds.
RAMP Household questionnaire
v
www.ifrc.orgSaving lives, changing minds.
RAMP
Persons roster
www.ifrc.orgSaving lives, changing minds.
RAMP
Bed net roster
www.ifrc.orgSaving lives, changing minds.
RAMP
RAMP malaria questionnaires: job aid
www.ifrc.orgSaving lives, changing minds.
RAMP RAMP malaria questionnaires: job aid (cont’d)
www.ifrc.orgSaving lives, changing minds.
RAMP
Fieldwork
Most surveys can be finished in one week (5 days)!
6 teams, 1 cluster per day per team, 30 clusters in 5 days
www.ifrc.orgSaving lives, changing minds.
RAMP
Fieldwork
Survey teams: 6 teams of 1 team supervisor and 2
interviewers per team
Survey supervisory support and monitoring team (eg., RCRC, IFRC, MoH): Planning, logistic and financial
responsibilities, field support, daily “quality” rounds, and remote monitoring of data quality
www.ifrc.orgSaving lives, changing minds.
RAMP
A typical day’s schedule
Morning briefing and travel to cluster
Meeting with community leader and guide, preparation
of sketch maps, segmentation, selection of households
to be interviewed
Conduct household interviews
Fieldworker sends data to server
Debriefing at day’s end with supervisory support and
monitoring team
Data monitoring, cleaning and preliminary analysis
www.ifrc.orgSaving lives, changing minds.
RAMP
Survey results bulletin and report
www.ifrc.orgSaving lives, changing minds.
RAMP
Precision <slide to be updated>
Key indicators Point estimate Width of confidence interval, ±%
ITN use, all ages 60 6
ITN use, children <5 years old
71 6
Percentage of ITNs hanging
79 4
% households with > 1 ITN 82 6
www.ifrc.orgSaving lives, changing minds.
RAMP
Reporting and dissemination of results
www.ifrc.orgSaving lives, changing minds.
RAMP
Survey bulletin
A brief, spreadsheet-based results bulletin can be produced 24 hours from last interview
Visually displays the data from analysis tables contained in the analysis plan
Shows main indicators and the results
www.ifrc.orgSaving lives, changing minds.
RAMP
Survey report
Full survey report provides a detailed description of background, methods, results, and discussion of results
Disseminate report to stakeholders and others to prompt action on the survey findings
www.ifrc.orgSaving lives, changing minds.
RAMP
RAMP survey toolkit
www.ifrc.orgSaving lives, changing minds.
RAMP
The RAMP survey toolkit
Three linked publications plus files on the website (www.ifrc.org/ramp) make up the RAMP toolkit Volume 1: Designing a RAMP survey: technical considerations Volume 2: Implementing a RAMP survey: practical field guide Volume 3: Training a RAMP survey team: guide for trainers Files on the website: dynamic and kept up to date
Example database and STATA files for data cleaning and analysis of a sample malaria survey
Latest up-to-date malaria questionnaires and STATA files for data cleaning and analysis
Country reports and results bulletins, information, useful links
Together these provide a methodology, operations protocol and numerous tools to carry out a RAMP survey
www.ifrc.orgSaving lives, changing minds.
RAMP
RAMP website: a dynamic resource
<Insert images of the RAMP website, with “www. ifrc.org/ramp superimposed in large font over
the images”>• See key documents from RAMP surveys being
done around the world. Survey questionnaires, training curricula, fieldwork tools and forms, survey reports—and more!
• Become part of a network of those using RAMP for development work
FOR FURTHER INFORMATION ON RAMP, PLEASE CONTACT:
IFRC HEALTH DEPARTMENT
JASON PEAT, SENIOR OFFICER, MALARIA
TEL. : +41 022 730 4419
EMAIL: [email protected]
THIS PRESENTATION IS PUBLISHED BY
INTERNATIONAL FEDERATION OF
RED CROSS AND RED CRESCENT SOCIETIES
P.O. BOX 372
CH-1211 GENEVA 19
SWITZERLAND
TEL.: +41 22 730 42 22
FAX.: +41 22 733 03 95