structured interviews and instrument design part i lecture 7
TRANSCRIPT
STRUCTURED INTERVIEWS AND INSTRUMENT DESIGN PART I
Lecture 7
“Continuum” of Interview Situations
2
Unstructured Structured
Open-ended interviews
Interview Guides, still open-ended
Identical Instrument (questions, etc) given to all participants
3
What is a Survey?
To survey: “act of looking or seeing, observing”
Research Surveys Qualitative interviews, focus groups Specific, systematic, quantitative data-collection
instruments
4
5
Example Surveys
US Census
General Social Survey
Online Surveys
Structured Interview Formats6
Face-to-face, self-administered (email, mail, etc), phone surveys
Typically either cross-sectional or longitudinal
Often a balance between (1) standardization (2) ability to reach sample (3) available resources (4) type of questions you are asking
Structured Interviewing7
Standardization of Wording and Format in an Interview Instrument
8
Must have a logic for each question Must have a logic for the question responses (if provided) Must have a logic for the sequencing of questions
Must have clear wording
Must have clear formatting
Must take into account the sample population that will actually take/use the survey instrument Culture, language, interpretive ambiguity
Question Logic9
Avoid asking unnecessary questions Example: Survey on computer usage
Socio-demographic questions (age, gender)? Risk behavior questions (drug use, etc)?
Closed or Open-ended?
Closed versus Open Questions10
(Q1a) What do you spend most of your time doing on the Internet?
(Q1b) Which of the following things do you do most of the time on the Internet?
(a) Email(b) Browse the web(c) Play games(d) Other
Question Logic11
Avoid redundant questions unless you have a clear reason Exception: Sometimes it is a good idea to ask
two questions that tap same concept (first as a scale, then as a categorical decision)
Example (ranks): First ask respondent to rate 1-10 how much they agree
with several statements. Next, ask respondents to rank the statements by how
much they agree with them (i.e., 5 statements, rank them 1-5).
Question Response Logic: Scales versus Categories
12
As a basic rule, metric scales with more range are better than binary or categorical responses (all other things being equal).
Example: Age Are you 18 or older (yes/no)? How old are you (in years)? _________
With general scales and Likert-scales, consider having no “middle” category (neutral, no opinion).
13
Question Response Logic: Categorical Responses
Responses must be mutually exclusive Example (bad): Where do you live?
Berkeley, San Fran, Bay Area, Other
Responses must be exhaustive Example (bad): What kind of computer do you
have? PC, Mac
Use ‘Don’t Know’, ‘Other’, ‘Not Applicable’ when absolutely necessary
14
Question Sequence
Static order for questions needs to have some rationale/logic Grouping similar items together Scattering similar items throughout survey
Personal demographic questions work best at end of survey (response rate and completion)
Randomization for all respondents
15
Clear Wording / Leading Questions
Clear Wording Example:
“What ISP do you use?” “If you have Internet service at home, what company or
service provider do you use for Internet access?”
Leading Questions Example:
“Don’t you think we should support our troops in Iraq?” “How strongly do you agree or disagree with the
following question: ‘We should support our troops in Iraq’”
16
Clear Formatting, Logic
Not all questions apply to everyone Example:
“How much do you spend on gas heat each month?”
Branching is a possible solution Example:
“Do you have gas heat? If yes, go to next question. If not, skip to question #3.
Condense when possible to avoid unnecessary branching. Example:
“How much do you spend on gas heat each month? (write 0 if you do not have gas heat)
17
Know your sample population
Regional language and terminology
Cultural differences
How you conduct survey can influence your valid sample Door to door? Registered telephone directory? Internet-based survey?
18
Replication and Using Existing Survey Instruments
ALWAYS a good idea to find other surveys that are used in your area of interest.
Especially with large, funded surveys the questions may have been tested for reliability.
Allows for comparisons between different samples if the question wording is the same.
If a question or set of questions is accepted as a good operationalization of the concept you are interested in, you don’t want to reinvent it unless you really intend to argue that your measure is more appropriate.