quantitative research methods survey research ebrahim mohseni - phd candidate at u of md - director...

Quantitative Research Methods

Survey ResearchEbrahim Mohseni- PhD Candidate at U of MD

- Director of Pars Advanced Research Scholars

Background (1)

In 1916 “Literary Digest” accurately predicted the election of US President Woodrow Wilson Using Postal Survey.

Literary Digest was able to accurately predict the next 4 presidential winners as well.

In 1936, however, failed to predict the winner despite their large sample (2.3 million). Sample bias (sample was composed of affluent

Americans who tended to be Republicans)

Background (2)

During the same presidential elections, George Gallup surveyed a much smaller sample, using scientific methods, and was able to predict the victory of FDR.

This was the start of using scientific methods to evaluate the situation on the ground and predict future events.

Background (3)

Sociopolitical survey research has its roots in the “scientific approach”, seeking to understand the truth (reality) trough objective techniques.

The Scientific Approach

Ibn al-Haytham (965–1039 A.D.) is recongnized as the first person who emphasized on the scientific approach:

“Truth is sought for its own sake. And those who are engaged upon the quest for anything for its own sake are not interested in other things. Finding the truth is difficult,

and the road to it is rough."

Elements of the Scientific Approach

Define the question Conduct preliminary research Form a hypothesis Perform experiment and collect data Analyze data Interpret the data

Elements of Survey Research Define the question Conduct preliminary research Form a hypothesis (Focus Groups)

Dependent Vs. Independent variables Questionnaire design

Pre-test Identify your target population Identify the most suitable (Practical and

effective) sampling technique Collect data Analyze data (Objective) Interpret the data (Subjective)

Focus Groups

Formulating a Hypothesis (1)

ObserveStudy the problemThink about your dependent and

independent variablesFormulate your own understanding of

the relationship between your dependent and independent variables


Read past literature (Lit. review) Evaluate their objectiveness Understand their mode of analysis

Philosophical vs. Mathematical Uncover their independent and dependent

variables. Develop your own interpretation of their

conclusions.


Focus Groups (FG) Conduct intensive direct group interviews

with individuals who are in your target population.

Uncover group consensus Uncover group disagreements Uncover the relationship between different

opinions and mind frames Note: The FG results are not representative.

Questionnaire Design(Questions -1) Is more of an Art than a precise science: Focus

Every Question should only and only focus on a single, very specific issue or topic and be as precise as possible.

When and how do you usually go to work? What time do you usually leave home for

work? What means of transportation do you usually

use to go to work from your home?

Questionnaire Design(Questions -2)

Brevity The questions should be as short and to

the point as possible. When was the last time that you went to the

doctor for a physical examination on your own or because you had to?

When was the last time you had a physical examination?



Clarity The meaning of the questions must be

completely clear to all respondents. ALL respondents must have an identical understanding of the questions.

Do you drink? Do you drink alcoholic beverages?


State the criteria of evaluation Never assume the criteria to be common

knowledge or that it is obvious. How important is it for stores to carry a large

variety of different snacks? How important is it to you that the store you

generally shop at carry a large variety of different snacks?


Avoid giving examples Examples skew the mind toward the given

example and not what seeks to elucidate. Do you watch international news channels

such as CNN or BBC? Do you watch news channels that are

internationally broadcasted?


Avoid Overgeneralization Make your scale of evaluation as precise

as possible. Do you participate in national elections? Did you participate in the previous

presidential elections?


Avoid Overemphasis Avoid adjectives or labels that are not

conventionally accepted. Would you favor increasing taxes to cope

with the current fiscal crises? Would you favor increasing taxes to cope

with the current fiscal problems?


Avoid Ambiguity Nothing should be ambiguous in a

question. Do your parents work fulltime? Does your father work more than 30 hours a

week? How about your mother?


Avoid double barrel questions Do you regularly take vitamins to avoid

getting sick? Do you regularly take vitamins? Why or

why not?


Avoid leading question. Questions should not have any

assumptions. Don’t you see some danger in the new

policy? Do you see any danger in the new policy?


Avoid loaded questions Would you support attacking Iran to

prevent it from destroying israel?

Questionnaire Design(Questions -12) Relevant (one size never fits all)

The same question cannot usually be used to measure attitudes toward an identical topic across different cultural or sociopolitical landscapes. Hence, the questions must be relevant to the target population.

To measure religiosity: How often do you go to the mosque during a

regular month? How often do you perform your prayers during a

regular day?


Vocabulary Limit your vocabulary to words that could

easily be understood. For a national survey:

In US: 8th grade vocabulary In Iran: 5th grade vocabulary


Avoid non-questions Do you still beat your wife?

Questionnaire Design(Response options - 1)

There are four general types of response options: Pure Open-ended Pre-coded open-ended Close ended Hybrid


Pure Open-ended: No response options are read. Responses

are recorded verbatim. Best when response options are not known Difficult to analyze

Requires coding


Pre-coded Open-ended: No response options are read. Responses

are placed under pre-determined categories.

Best when response options are inherently leading.

Best when looking for “top of the mind” responses.

Difficult to execute Requires coding


Closed ended questions: Responses are read and the respondents

must select from the list of provided options. Must be Mutually exclusive Must be mutually exhaustive (comprehensive)

Questionnaire Design(Pre-test)

Once a questionnaire is designed, it must be pre-tested (piloted) to make sure it is ready for the field: Nuclear weapons vs. Atomic Bomb

Sampling

Identify your target population Filter questions

Probability sampling Vs. Non-probability sampling SRS – Uniform Probability

Face to Face (FTF)

More intimate respondent – interviewer conversation (source of comfort AND interviewer related bias)

Lengthier interview Longer time before data is collected Very costly

Computer Assisted Telephone Interviewing (CATI)

Best for short and non-intrusive interviews

Better supervision Sampling problems

Landline penetration Mobile phone vs. Landline

Postal and Internet

Best for long and intrusive questions Sampling problems

Margin of Error

quantitative research methods survey research ebrahim mohseni - phd candidate at u of md - director...

Documents

scientific methods

scientific approachibn

data objectiveinterpret

sample bias sample

questionnaire designquestions

independent variablesformulate

smaller sample

large sample