Dr. Michael R. Hyman, NMSU

Survey Error:

Focus on Systematic Error

2

3

Totalerror

Systematicerror (bias)

Random samplingerror

Total Survey Error: Components

4

Random Sampling Error

A statistical fluctuation that occurs because of chance variation in the elements selected for the sample.

5

Systematic Error

Systematic error results from some imperfect aspect of the research design or from a mistake in the execution of the research

6

7

Respondent Error

A classification of sample bias resulting from some respondent action or inaction

• Nonresponse bias

• Response bias

8

Respondenterror

Nonresponseerror

Responsebias

Respondent Error: Components

9

Non-response Error

• Non-respondents

– Refuse to cooperate

– Not-at-homes

• Self-selection bias

– Over-represents extreme positions

– Under-represents indifference

• Varies by type of interview

10

Why People Cooperate

• To be supportive/helpful

• Have a social interaction

• Curiosity

• Remuneration

11

Factors Contributing to Refusal

• Fear/anxiety

• Invasion of privacy

• Hostility toward

– Survey sponsor

– Topic of interview

– Interviewer

12

Response Bias

A bias that occurs when respondents tend to answer questions with a certain slant that consciously or unconsciously misrepresents the truth

13

Responsebias

Unconsciousmisrepresentation

Deliberatefalsification

Response Bias: Components

14

Unwillingness to Respond Accurately

• Invasion of privacy

• Time pressure and fatigue

• Physical/social environment

• Questionnaire-specific issue

– Question/response wording

– Layout

• Mischievous respondents

15

16

Inability to Respond Accurately

• Inarticulate

• Ignorant

– Asked about other people

– Asked to predict own behavior

– Asked question meant for other people

17

18

19

Inability to Respond Accurately

• Faulty memory

– Only ask about recent events

– Problem with questions about behavior over time (e.g., frequency)

– Telescoping

• Everything happened yesterday

20

Acquiescence bias

Extremity bias

Interviewer bias

Auspices bias

Social desirability bias

Unconscious Misrepresentation: Components

21

Acquiescence Bias

A type of response bias due to some people tending to agree with all questions or to concur with a particular position.

22

Extremity Bias

A type of response bias due to response styles that vary from person to person; some people tend to use extremes when responding to questions.

23

Interviewer Bias

A response bias that occurs because the presence of the interviewer influences answers.

24

25

Auspices Bias

Bias in respondents’ answers due to being influenced by the organization conducting the study.

26

Social Desirability Bias

Bias in responses caused by respondents’ desire, either conscious or unconscious, to gain prestige or appear in a different social role.

27

28

29

Systematicerror (bias)

Administrativeerror

Respondenterror

Systematic Error: Components

30

Administrative Error

• Improper administration of the research task

• Blunders

• Confusion

• Neglect

• Omission

31

Administrative Error

• Interviewer cheating - filling in fake answers or falsifying interviewers

• Data processing error - incorrect data entry, computer programming, or other procedural errors during the analysis stage.

• Sample selection error -improper sample design or sampling procedure execution.

• Interviewer error - field mistakes

32

Interviewer Error

• Subtle source of cues

– Can inadvertently reinforce some responses

• Appearance of incompetence

33

34

35

Interviewer Error

Insufficient or poor probing

36

37

38

39

Interviewer Error

Fraud

• Why commercial houses validate roughly 10-15% of completed interviews

Selecting unqualified respondents

40

Checklist for Unacceptable Questionnaires

• Major portions of questionnaire or key questions left unanswered

• Evidence that respondent did not understand instructions or did not take task seriously

• Missing pages• Respondent not qualified for target

population• Questionnaire returned after cutoff date