1 surveys in humanitarian emergencies. 2 methods of data collection assessmentsurveysurveillance...
TRANSCRIPT
1
Surveys in Humanitarian Emergencies
2
Methods of Data Collection
Assessment Survey Surveillance
Objective Rapid appraisalMedium-term appraisal
Continuous appraisal
Data TypeQualitative/
Cross sectional snapshot
Quantitative/
Cross sectional
snapshot
Quantitative/ Longitudinal trends
MethodObservational /
Secondary source
Sample with survey instrument
Periodic, standardized data collection
3
Overview
When to do a survey
The need for a standardized tool and methodology
Key activities in designing and implementing health and nutrition surveys
Survey interpretation
4
Surveys in Emergencies
Why would you consider doing a survey?
When would you do a survey?
5
Why Do a Survey in an Emergency
Estimate prevalence (example acute malnutrition)
Determine mortality
Prioritize interventions
Collect baseline data for program planning
Evaluate program success
Advocate for intervention
6
When To Do a Survey in an Emergency?
After the initial emergency phase has passed
After basic needs for survival been met
When major population movement has stopped
When you are in a position to take action
After preparatory information has been collected
When security allows
7
Difference Between Emergency and Development Surveys
Emergency surveys• Must be done with action in mind• Must be implemented and analyzed quickly• Hard to find trained/trainable personnel• Difficult to come up with sampling frame• Multi sectoral, multi partner (UNICEF, UNFP,
etc.)• Security
8
The Problem: Non-standardization of Methods
Ethiopia 1999-2000
Among 67 nutrition and mortality surveys, only 6 (9%) met all 5 eligibility criteria to be valid and precise*
Probabilistic sampling
Sample size > 500 children aged 6-59 months
25 clusters
20 or more children per cluster
Weight for height used as anthropometric index
Source: Spiegel P et al, JAMA 2004
9
Survey design and implementation: key tasks
Determine objectives of the survey
Determine broad questions to be answered outcomes to be measured
Estimate Budget needed and Identify key partners
Define the sampling frame & sampling design
Design the questionnaire, translate, and test
Conduct training , plan the logistics, equipment, and survey team needed
Data entry, analysis, and Interpretation of results
Preparation and dissemination of results
Take action
10
What Might Be Included in an Emergency Survey?
Demographics Household
Characteristics Child Health Child Nutrition Reproductive health Mortality
Food Sources and Consumption
Food Aid deliveries & usage
Health interventions Water sources Sanitation
11
Defining The Sampling Frame
What population do you want to extrapolate your results to?
What population is or will be the target of the program?
Is the population homogeneous? Are there geography and access issues? Security conditions?
12
Sampling Frame for the Darfur Survey
13
Sampling Frame for the Ethiopia Survey
Population not homogeneous
• Highlands versus lowlands However, intervention targeted by Woreda
14
Decisions Concerning Sampling Design
Probability sampling: Simple random sampling Systematic random
sampling Cluster sampling
Non-probability sampling: Key informants Convenience sampling Purposive sampling
Sampling methodology: types
15
Each individual or sampling unit in the population has the same chance or probability of being selected
The selection of one individual should be independent of the selection of another
Representative Sampling
16
Simple Random Sampling
Most basic type of sampling Selection of units independent and random Steps:
• Number each sampling unit
• Choose new random number for each selection
17
Systematic Random Sampling
Similar to simple random sampling First sampling unit chosen randomly Systematic selection of subsequent sampling units Steps:
• Compute sampling interval (SI) (Number in population / Sample size)
• Select random start between 1 and SI
18
Simple and systematic random sampling
What is required for both simple and systematic random sampling?
Both require a complete list of all basic sampling units.
19
Calculating the sample size For A Random Sample or Systematic Sample
The sample size is calculated using the following formula:
n = { t2 x p x q }
d2
Where: n = sample size
t = the risk of error (1.96 or 5% error)
p = expected prevalence (fraction of 1)
q = 1- p (expected non-prevalence)
d = level of precision (fraction of 1)
Sample size should by multiplied by design effect in cluster surveys
20
An example of calculating the sample size
The sample size is calculated using the following formula:
81 = { 1.962 x 0.3 x 0.7 }
0.12
Where: n = sample size
t = the risk of error (1.96 or 5% error)
p =expected prevalence (fraction of 1)
q = 1- p (expected non-prevalence)
d = level of precision (fraction of 1)
21
Cluster sampling
Objective: To choose smaller geographic areas in which simple or systematic random sampling can be done
22
Simple Random Sampling
Simple randomSampling: (300 households required)
Samplinguniverse
Selected households
Non-selected households
23
Cluster samplingCluster sampling : 30 Clusters of 10 households each
1
2
3
But, because we are using cluster sampling, we must increase the sample size because we are not choosing randomly, but households are clustered together. May end up choosing 30 clusters of 15 households.
24
The sample size is calculated using the following formula:
n = { t2 x p x q } x design effect
d2
Calculating the sample size in cluster surveys
25
Which sampling method here?
26
Which sampling method here?
27
Common Problems With Sampling
• For surveys:- Was a random or probability sample taken using a recognized
method?– Did everyone have an known probability of selection?
Who was left out? (selection bias)– What was the geographical coverage?
• Does it relate to catchment area of program?– Were case definitions clear and standardized and piloted or
validated?– How was age determined?– What was the sample size?– Are confidence limits reported?– Watch out for denominator of rates
28
Sampling - Goal
Regardless of the sampling method, if the sample is not representative, you can’t generalize the findings to the whole population.
Remember this point!
29
Sampling – Sample size
So how do we decide on the needed precision? One-time results for advocacy alone does not
need much precision (0.10 good enough) Results that you will need to compare against in
the future need greater precision (0.05 if program will have large impact)
Results you will monitor frequently (e.g. year by year) need even greater precision (0.02)
30
Adjustments to Sample Size
Source of information about number of analysis units (e.g. adult males) per samping unit (household)• Calculate from census statistics• Use previous surveys to calculate
Finally, need to add margin for non-response• Look at previous surveys• May differ by region of country• May be higher for some measures (e.g. blood)
31
Training & data analysis
Training needs to include: what data to collect, how to select households, data collection method, consent form, interview method
Data entry: advise to begin data entry simultaneously as data collection
Data analysis: select the analysis tool (Epi-info, SPSS), identify key personnel
32
Interpreting ResultsMeasles coverage: Measles immunization coverage 59.4% [CI:54.6-
64.1]
Major morbidity in last two weeks: Watery diarrhea 17.2% [CI:13.8-21.2] Mortality rates: (over 7 month period) Crude mortality rate: 3.2/10,000/day <5 mortality rate: 9.8/10,000/day
33
Interpretation: Other Factors
Trends and change Confidence Intervals (CI) Seasonality Aggravating factors or risks Baseline or ‘normal’ prevalence Prevalence of other types of malnutrition e.g.
chronic malnutrition Mortality levels
34
Mortality Trends in Selected Refugee Camps
0
2
4
6
8
10
1 2 3 4 5 6 7 8 9 10
Months
CM
R: D
eath
s/10
,000
/day
Thailand: 1979-1980 Somalia: 1980-1981
35
Assessing the Quality Of Survey Implementation
Did the survey answer the question it was designed to? Were objectives clear measurable and measured? Was the sampling frame adequate? Questionnaire
• Clear?
• In local language?
• Translated, back translated? Sampling design
• Representative?
• Sample size adequate?
36
Assessing the Quality Of Survey Implementation
Logistics, equipment, and survey team
• Equipment standardized?
• Training adequate?
• Personnel? What checks for data quality have been done? Do you agree with the interpretation of results? Have results been disseminated to all partners What action is planned?
37
One Way to Evaluate MethodsCharacteristic Assessment Points
Sampling frame Unadjusted
Adjusted
1
2
Sampling method Convenience
Random
1
2
Sample size Small 1
Good (justified) 2
Geographical coverage Small
Appropriate
1
2
Case definitions Not defined Standardized
1
2
Confidence limits Not reported 1
Appropriate 2
38
Distribution of ht/age z-scores, Country X, 2000
02
468
10
121416
1820
-6.00to -5.0
-4.99to -4.5
-4.49to -4.0
-3.99to -3.5
-3.49to -3.0
-2.99to -2.5
-2.49to -2.0
-1.99to -1.5
-1.49to -1.0
-0.99to -0.5
-0.49to 0.0
0.01to 0.5
0.51to 1.0
1.01to 1.5
1.51to 2.0
2.01to 2.5
2.51to 3.0
3.01to 3.5
3.51to 6.0
Ht/Age z-score
Per
cent
of
child
ren Survey
NCHS reference
Standard deviationof sample = 1.75
Assessing The Quality of the Survey
39
Take Action
Interpret and understand findings Review and revise program objectives Advocate for resources e.g. food pipeline or access to
clean water Address underlying causes of poor health or nutrition Increase coverage of programs
• Vaccination Close emergency nutrition programs Use findings as part of wider country information system Use findings as baseline data