sampling error and bias

8/12/2019 Sampling Error and Bias

1/37

Sampling : Error and bias


2/37

Sampling definitions

Sampling universe

Sampling frame

Sampling unit Basic sampling unit or elementary unit

Sampling fraction

Respondent

Survey subject

Unit of analysis


3/37

Sampling types

Two basic categories of sampling

Probability sampling Also called formal sampling or random sampling

Non-probability sampling

Also called informal sampling


4/37


5/37

Types of probability sampling

Simple random sampling (SRS)

Systematic random sampling

Stratified sampling Cluster sampling

Multi-stage sampling


6/37

Questions for sampling design

Presampling choices

What is the nature of the study: exploratory,

descriptive, analytical?

What are the outcomes of interest?

What are the target populations?

Do you want estimates for subpopulations or just

for the entire population? How will the data be collected?

Is sampling necessary and appropriate?


7/37


Sampling choices

What listing will be used as the sampling frame?

What is the desired precision?

What type of samping will be done?

Will the probability of selection be equal or

unequal?

What is the sample size?


8/37


Postsampling choices

How can the effect of nonresponse be assessed?

Is weighted analysis necessary?

What are the confidence limits for the major

estimates?


9/37

Result from survey is never

exactly the same as

the actual value in the population

WHY?

But


10/37

Components of total error

0% 100%

True

population

value

50%

Point

estimate

from survey

40%

Total error

Nonsamplingbias

Sampling bias

Samplingerror

Prevalence


11/37

Nonsampling bias

Is present even if sampling and analysis done

correctly

Would still be present if survey measured outcome

in ENTIRE sampling frame

In sum, you have either sampled the wrong

people or screwed up your measurements!


12/37

Nonsampling bias

Types:

Sampling frame is not equal to population to which

you want to generalize (sampling universe)

Sampling frame out of date Non-response among sampling units in sampling frame

Measurement error

Tape incorrectly fixed to height board

Scale consistently reads low by 0.5 kg

Failure to remove heavy clothing before weighing

Misleading questions

Recall bias


13/37

Nonsampling bias

Source of bias

Sampling frame out of date

Non-response

Measurement error

Use current sampling frame

Limit generalizations

Minimize non-response

Use various statistical

methods to weight data

Standardize instrumentsWrite clear & simple

questions

Train survey workers

Supervise survey workers

Prevention or cure


14/37

Sampling bias

Selection of nonrepresentative sample, i.e., the

likelihood of selection not equal for each sampling

unit

Failure to weight analysis of unequal probability

sample

In sum, you have not sampled people with equalprobability and you have not accounted for this

in your analysis!


15/37

Sampling bias

Examples

Nonrepresentative sample

Selecting youngest child in household

Choosing households close to the road

Using a different sampling fraction in different

provinces

Failure to do statistical weighting


16/37

Sampling bias

Source of bias

Nonrepresentative sampling

Failure to do weighting

ALWAYS ask yourself "Will

this choice enhance

representativeness orreduce it"?

Calculate the probabilities of

selection

Apply appropriate statisticalweights if selection

probabilities unequal

Prevention or cure


17/37

Sampling error

Difference between survey result and population

value due to random selection of sample

Influenced by:

Sample size

Sampling scheme

Unlike nonsampling bias and sampling bias, itcan be predicted, calculated, and accounted for.


18/37

Sampling error

Measures of sampling error:

Confidence limits

Standard error

Coefficient of variance

P values

Others

Use these measures to: Calculate sample size prior to sampling

Determine how sure we are of result afteranalysis


19/37


20/37

Bias and sampling error

Nonsampling bias

Sampling bias

Sampling error

Bias

Sampling error


21/37

In sum

Bias

Includes nonsampling bias

and sampling bias

Is due to mistakes whichcan be avoided

Cannot be precisely

measured

Control and preventionrequires careful attention

Sampling error

Is unavoidable if sampling

< 100% of population

Can be controlled byselecting appropriate

sample size and sampling

method

Can be preciselycalculated after-the-fact


22/37

Essential concepts

Bias & Accuracy

Sampling error & Precision


23/37

Accuracy

What is accuracy?

The degree to which a measurement, or an

estimate based on measurements, representsthe true value of the attribute that is being

measured.

Last. A Dictionary of Epidemiology. 1988

In short, obtaining results close to the TRUTH.


24/37

Accuracy

Associated terms:

Validity


25/37

Precision

What is precision?

Precision in epidemiologic measurements

corresponds to the reduction of random error.

Rothman. Modern Epidemiology. 1986.

In short, obtaining similar results with

repeated measurement


26/37

Precision

Associated terms:

Reliability

Reproducability


27/37


28/37

Accuracy vs. precision

Precision: obtaining similar results with repeated

measurement (may or may not be accurate)


29/37


Poor precision (from small sample size) with

reasonable accuracy (without bias):


30/37


Good precision (from small sample size) with

reasonable accuracy (without bias):


31/37


Good precision (from large sample size), but with

poor accuracy (with bias):


32/37

In sum

Sampling error

Difference between survey result and population value due to

random selection of sample

Greater with smaller sample sizes

Induces lack of precision

Bias

Difference between survey result and population value due to

error in measurement, selection of non-representative sample or

other factors

Due to factors other than sample size

Therefore, a large sample size cannot guarantee absence of

bias

Induces lack of accuracy, even with good precision


33/37

Usual situation after a survey

95% confidence limits

Result of single survey


34/37





35/37





36/37


How can you tell which situation you have?


Result of single survey Result of single survey



37/37

Precision, bias, and sample size

Precision vs. bias

Larger sample size increases precision

It does NOT guarantee absence of bias

Bias may result in very incorrect estimate

If little sampling error, may have confidence in thiswrong estimate

Quality control is more difficult the larger thesample size

Therefore, you may be better off with smallersample size, less precision, but much lessbias.

sampling error and bias

Documents