sampling : error and bias. sampling definitions sampling universe sampling frame sampling unit ...
Post on 16-Jan-2016
289 Views
Preview:
TRANSCRIPT
Sampling : Error and bias
Sampling definitions
Sampling universe Sampling frame Sampling unit Basic sampling unit or elementary unit Sampling fraction Respondent Survey subject Unit of analysis
Sampling types
Two basic categories of sampling Probability sampling
• Also called formal sampling or random sampling Non-probability sampling
• Also called informal sampling
Probability sampling
What is probability sampling?
A selection of elements in a population, such that every element has a known, non-zero probability of being selected.
Types of probability sampling
Simple random sampling (SRS) Systematic random sampling Stratified sampling Cluster sampling Multi-stage sampling
Questions for sampling design
Presampling choices
• What is the nature of the study: exploratory, descriptive, analytical?
• What are the outcomes of interest?
• What are the target populations?
• Do you want estimates for subpopulations or just for the entire population?
• How will the data be collected?
• Is sampling necessary and appropriate?
Questions for sampling design
Sampling choices
• What listing will be used as the sampling frame?
• What is the desired precision?
• What type of samping will be done?
• Will the probability of selection be equal or unequal?
• What is the sample size?
Questions for sampling design
Postsampling choices
• How can the effect of nonresponse be assessed?
• Is weighted analysis necessary?
• What are the confidence limits for the major estimates?
Result from survey is never
exactly the same as
the actual value in the population
WHY?
But…
Components of total error
0% 100%
True population
value50%
Pointestimate
from survey40%
Total error
Nonsamplingbias
Sampling bias
Samplingerror
Prevalence
Nonsampling bias
Is present even if sampling and analysis done correctly
Would still be present if survey measured outcome in ENTIRE sampling frame
In sum, you have either sampled the wrong people or screwed up your measurements!
Nonsampling bias
Types:
• Sampling frame is not equal to population to which you want to generalize (sampling universe)
• Sampling frame out of date• Non-response among sampling units in sampling frame
• Measurement error• Tape incorrectly fixed to height board• Scale consistently reads low by 0.5 kg• Failure to remove heavy clothing before weighing• Misleading questions• Recall bias
Nonsampling bias
Source of biasSampling frame out of date
Non-response
Measurement error
Use current sampling frameLimit generalizations
Minimize non-responseUse various statistical
methods to weight data
Standardize instrumentsWrite clear & simple
questionsTrain survey workersSupervise survey workers
Prevention or cure
Sampling bias
Selection of nonrepresentative sample, i.e., the likelihood of selection not equal for each sampling unit
Failure to weight analysis of unequal probability sample
In sum, you have not sampled people with equal probability and you have not accounted for this
in your analysis!
Sampling bias
Examples
• Nonrepresentative sample• Selecting youngest child in household• Choosing households close to the road• Using a different sampling fraction in different
provinces
• Failure to do statistical weighting
Sampling bias
Source of biasNonrepresentative sampling
Failure to do weighting
ALWAYS ask yourself "Will this choice enhance representativeness or reduce it"?
Calculate the probabilities of selection
Apply appropriate statistical weights if selection probabilities unequal
Prevention or cure
Sampling error
Difference between survey result and population value due to random selection of sample
Influenced by:
• Sample size
• Sampling scheme
Unlike nonsampling bias and sampling bias, it can be predicted, calculated, and accounted for.
Sampling error
Measures of sampling error:• Confidence limits• Standard error• Coefficient of variance• P values• Others
Use these measures to:• Calculate sample size prior to sampling• Determine how sure we are of result after
analysis
Bias and sampling error
Nonsampling bias
Sampling bias
Sampling error
Bias
Sampling error
In sum…
Bias Includes nonsampling bias
and sampling bias Is due to mistakes which
can be avoided Cannot be precisely
measured Control and prevention
requires careful attention
Sampling error Is unavoidable if sampling
< 100% of population Can be controlled by
selecting appropriate sample size and sampling method
Can be precisely calculated after-the-fact
Essential concepts
Bias & Accuracy
Sampling error & Precision
Accuracy
What is accuracy?
The degree to which a measurement, or anestimate based on measurements, representsthe true value of the attribute that is beingmeasured.
Last. A Dictionary of Epidemiology. 1988
In short, obtaining results close to the TRUTH.
Accuracy
Associated terms: Validity
Precision
What is precision?
Precision in epidemiologic measurementscorresponds to the reduction of random error.
Rothman. Modern Epidemiology. 1986.
In short, obtaining similar results withrepeated measurement
Precision
Associated terms: Reliability Reproducability
Accuracy vs. precision
Accuracy: obtaining results close to truth
Survey 1
Survey 2
Survey 3
Real population value
Accuracy vs. precision
Precision: obtaining similar results with repeated measurement (may or may not be accurate)
Accuracy vs. precision
Poor precision (from small sample size) with reasonable accuracy (without bias):
Accuracy vs. precision
Good precision (from small sample size) with reasonable accuracy (without bias):
Accuracy vs. precision
Good precision (from large sample size), but with poor accuracy (with bias):
In sum…
Sampling error• Difference between survey result and population value due to
random selection of sample
• Greater with smaller sample sizes
• Induces lack of precision Bias
• Difference between survey result and population value due to error in measurement, selection of non-representative sample or other factors
• Due to factors other than sample size
• Therefore, a large sample size cannot guarantee absence of bias
• Induces lack of accuracy, even with good precision
Usual situation after a survey
95% confidence limitsResult of single survey
Usual situation after a survey
95% confidence limits
Result of single survey
Usual situation after a survey
Result of single survey
95% confidence limits
Usual situation after a survey
How can you tell which situation you have?
95% confidence limits
Result of single survey Result of single survey
95% confidence limits
Precision, bias, and sample size
Precision vs. bias Larger sample size increases precision
• It does NOT guarantee absence of bias• Bias may result in very incorrect estimate• If little sampling error, may have confidence in
this wrong estimate Quality control is more difficult the larger the
sample size Therefore, you may be better off with smaller
sample size, less precision, but much less bias.
top related