Qualitative Research- Part 2Tools and Techniques

DIE 4564 Research Methods

Chapter Outline• Topics Appropriate for Survey Research• Guidelines for Asking Questions• Questionnaire Construction• Self-Administered Questionnaires• Interview Surveys• Telephone Surveys• Online Surveys• Comparison of the Different Survey Methods• Strengths and Weaknesses of Survey Research• Secondary Analysis• Ethics and Survey Research

Topics Appropriate for Survey Research

• Descriptive, exploratory, and explanatory

• Units of analysis = respondents Respondents – a person who provides data for

analysis by responding to a survey questionnaire.

• Large samples, original data, measuring attitudes and orientations

Guidelines for Asking Questions• Questionnaire - a document containing questions

and other types of items designed to solicit information appropriate for analysis.

Open-Ended Questions – questions for which the respondent is asked to provide his/her own answers.

Closed-Ended Questions – survey questions in which the respondent is asked to select an answer from a list provided by the researcher.

Guidelines for Asking Questions• Select appropriate question forms.• Make items clear.• Avoid double-barreled questions.• Respondents must be competent to answer.• Respondents must be willing to answer.• Questions should be relevant.• Short items are best• Avoid negative items.• Avoid biased items and terms.

Guidelines for Asking Questions• Bias – that quality of measurement device

that tends to result in a misrepresentation of what is being measured in a particular direction.

Guidelines for Asking Questions• Review Question

Whenever we ask people for information, they answer through a filter of what will make them look good. This filter is called _________.

Guidelines for Asking Questions• Review Question

Whenever we ask people for information, they answer through a filter of what will make them look good. This filter is called social desirability.

Questionnaire Construction• General Questionnaire Format

Uncluttered One question per line Consistent format

Questionnaire Construction• Formats for Respondents

Questionnaire Construction• Contingency Question – a survey question intended for only

some respondents, determined by their responses to some other question.

Questionnaire Construction• Contingency Question – a survey question

intended for only some respondents, determined by their responses to some other question.

Questionnaire Construction• Matrix Questions

Questionnaire Construction• Ordering Items in a Questionnaire

Appearance Open-Ended or Closed-Ended First?

Questionnaire Construction• Questionnaire Instructions

Introductory comments and clear instructions

• Pre-testing the Questionnaire

Questionnaire Construction• A Composite Illustration

Questionnaire Construction• Review Question

Which of the below is not a characteristic of matrix questions?A. They use space efficiently when asking

multiple questions.B. They must all have the same response

categories.C. They are less valid measures of concepts.D. They will speed the response time for

respondents.

Questionnaire Construction• Review Question

Which of the below is not a characteristic of matrix questions?The answer is C. Matrix questions use

space efficiently when asking multiple questions, have the same response categories, and will speed the response time for respondents. They are not less valid measures of concepts.

Self-Administered Questionnaires

Questionnaires in which respondents are asked to complete the questionnaire by themselves• Mail Distribution and Return

Why do people not return questionnaires?

• Monitoring Returns• Follow-Up Mailings

Self-Administered Questionnaires• Response Rate – the number of people

participating in a survey divided by the number selected in the sample. Ideal = higher than 70%

• Other kinds of rates: Cooperation rates Refusal rates Contact rates

Self-Administered Questionnaires• Compensation for Respondents

Effect on response rates

• A Case Study

Self-Administered Questionnaires• Review Question

The proportion of all potentially eligible cases in which the respondent refuses to be interviewed or breaks off an interview is called the ________ rate.

Self-Administered Questionnaires• Review Question

The proportion of all potentially eligible cases in which the respondent refuses to be interviewed or breaks off an interview is called the refusal rate.

Interview Surveys• Interview – a data-collection encounter in which

one person (interviewer) asks questions of another (respondent).

• The Role of the Survey Interviewer

Interview Surveys• Guidelines for Survey Interviewing

Appearance and demeanor Familiarity with the questionnaire Following question wording exactly Recording responses exactly Probing for responses

Probe – a technique employed in interviewing to solicit a more complete answer to a question.

Interview Surveys• Coordinate and Control

TrainingGeneral guidelinesHow to handle difficult situationsPractice interviews

Interview Surveys• Review Question

Which of the below is not one of the advantages of using a survey interviewer?A. Increased response ratesB. Decreased number of “don’t knows”

and “no answers”C. Better understanding of questionsD. Increased refusal rate

Interview Surveys• Review Question

Which of the below is not one of the advantages of using a survey interviewer?The correct answer is D. The use of survey

interviewers increases response rates, decreases “don’t knows” and “no answers,” and assures the respondents better understand the questions. Using a survey interviewer does not increase the refusal rate.

Telephone Surveys• Advantages

95.5% of households have a telephone Time and money

• Disadvantages Unlisted phone numbers Cell phones

Telephone Surveys• Random-Digit Dialing (RDD) – a sampling

technique in which random numbers are selected from within the range of numbers assigned to active telephones.

Telephone Surveys• Computer-Assisted Telephone Interviewing

(CATI) – a data-collection technique in which a telephone-survey questionnaire is stored in a computer, permitting the interviewer to read the questions from the monitor and enter the answers on the computer keyboard

Telephone Surveys• Response Rates in Interview Surveys

Telephone Surveys• Review Question

This type of telephone survey is more cost-effective because it cuts out the labor costs of hiring interviewers and it automatically prepares data for analysis: _______.

Telephone Surveys• Review Question

This type of telephone survey is more cost-effective because it cuts out the labor costs of hiring interviewers and it automatically prepares data for analysis: CATI.

Online Surveys• DO use consistent wording.• DO use simple language.• DON’T force excessive scrolling.• DO offer to share select result with respondents.• DO plan time and day of initial mailing.• DO be aware of technical limitations.• DO test incentives, rewards, and prizes.• DO limit studies to less than 15 minutes.

Online Surveys• Review Question

True or False: The relative youth of online surveys makes them fertile ground for innovation and experimentation.

Online Surveys• Review Question

True: The relative youth of online surveys makes them fertile ground for innovation and experimentation.

Comparison of the Different Survey Methods

Self-Administered Questionnaires Cheaper and faster than face-to-face interviews National is the same cost as local mailings Requires small staff More willingness to answer controversial items

Interview Surveys Fewer incomplete questionnaires More effective for complicated questionnaires Face-to-face is more intimate

Telephone Surveys Cheaper and more time efficient

Online Surveys Available software and websites

Examples of Methods for Collecting Data

Comparison of the Different Survey Methods

One advantage of _______ surveys is that they work better for questionnaires with complicated contingency questions.A. Self-administeredB. InterviewC. TelephoneD. Online

Comparison of the Different Survey Methods

The correct answer is B. Interview surveys are better for complicated questionnaires.

Recruiting MethodsOnce a data collection method has been selected, the next step is to decide on an effective method for recruiting members of the sample population to be participants in the study.

The best method for initiating contact with potential participants is often related to the intended data collection method.

Recruiting MethodsParticipation rates will likely be higher if:• Recruits understand the importance and value of

the research project• Researchers provide multiple invitations and

opportunities to participate, and make participation as easy as possible

• Incentives (such as small gifts) are offered

Data Recording MethodsA decision must also be made about how responses will be recorded and when they will be entered into a computer database. There are two basic options:• Record the responses on paper and to enter

them into a computer database later• Have interviewers or participants enter

responses directly into a database

Methods for Collecting and Recording Survey Data

Training InterviewersThe interview process should be the same for all participants in a study, whether they are being interviewed in-person or by telephone interview.

Uniformity is easiest to accomplish when all interviewers attend training sessions where they have an opportunity to practice their interview skills.

Strengths and Weaknesses of Survey Research

Strengths Useful in describing large populations Surveys are flexible Standardized questions

Strengths and Weaknesses of Survey Research

Weaknesses Round pegs in square holes Seldom deal with context of social life Inflexible Artificial Weak on validity

Strengths and Weaknesses of Survey Research

By presenting all subjects with a standardized stimulus, survey research goes a long way toward eliminating _______ in observations.

A. InvalidityB. UnreliabilityC. ValidityD. Reliability

Strengths and Weaknesses of Survey Research

The correct answer is B. Because all respondents react to the same, standardized stimulus, survey research reduces unreliability.

Secondary Analysis• Secondary Analysis – a form of research in which

the data collected and processed by one researcher are reanalyzed by another. General Social Survey

• Secondary analysis provides social researchers with an important option for “collecting” data cheaply and easily but at a potential cost in validity.

Secondary AnalysisWhich of the below is not an advantage of secondary analysis?

A. It’s cheaper than original surveys.B. It’s faster than original surveys.C. It has greater validity.D. It has the enhanced possibility of meta-

analysis.

Secondary Analysis• Review Question

The correct answer is C. Secondary analysis often has problems with validity.

Ethics and Survey Research• Surveys often ask for private information,

and researchers must keep such information confidential.

• Because asking questions can cause psychological discomfort or harm to respondents, the researcher should minimize this risk.

Chapter Summary• Bias• Closed-ended questions• Computer-assisted telephone interviewing

(CATI)• Contingency question• Interview• Open-ended questions

Quick Quiz(Babbie text)

Chapter 9 Quiz

When is survey research the best method available?

A. when collecting original dataB. when describing a population too large to

observe directlyC. when measuring attitudesD. all of the above

Chapter 9 Quiz

Answer: D.Survey research the best method available when collecting original data, when describing a population too large to observe directly, and when measuring attitudes.

Chapter 9 Quiz

_____ questions have a respondent select an answer from a list provided.

A. Open-endedB. PretestC. ExperimentalD. Closed-ended

Chapter 9 Quiz

Answer: D.Closed-ended questions have a respondent select an answer from among a list provided.

Chapter 9 Quiz

As a general rule, a questionnaire should be

A. spread out.B. uncluttered.C. relevant.D. all of the above

Chapter 9 Quiz

ANSWER: D.As a general rule, a questionnaire should be spread out, uncluttered, and relevant.

Chapter 9 QuizWhich of these are among the many advantages that underlie the growing popularity of telephone surveys?

A. moneyB. timeC. convenienceD. all of the above choicesE. none of the above choices

Chapter 9 Quiz

ANSWER: D.Money, time, and convenience are among the many advantages that underlie the growing popularity of telephone surveys.

Chapter 9 Quiz

Which is not an advantage of survey research?

A. increased validityB. increased reliabilityC. increased generalizabilityD. increased flexibility in analysis

Chapter 9 Quiz

ANSWER: A.Increased validity is not an advantage of survey research.

Chapter 9 Quiz

The major problem with secondary analysis pertains to

A. theory.B. hypotheses.C. validity.D. sampling.E. empirical generalization.

Chapter 9 Quiz

ANSWER: C.The major problem with secondary analysis pertains to validity.