qualitative research methods mary h. hayden, phd ncar summer was*is boulder, co july 14, 2007 july...

Qualitative Research Qualitative Research MethodsMethods

Mary H. Hayden, PhDMary H. Hayden, PhDNCAR Summer WAS*ISNCAR Summer WAS*IS

Boulder, CO Boulder, CO July 14, 2007July 14, 2007

Presentation OutlinePresentation Outline

Distinguishing qualitative and quantitative Distinguishing qualitative and quantitative approachesapproaches

Qualitative methods – Qualitative methods – Types of qualitative methodsTypes of qualitative methodsAdvantages vs. DisadvantagesAdvantages vs. Disadvantages

Real World ExamplesReal World Examples Survey Questionnaire CritiqueSurvey Questionnaire Critique

Qualitative and Quantitative ApproachesQualitative and Quantitative Approaches

Qualitative Quantitative

(Usually) Non-probability based sample

Typically a probability-based sample

Non-generalizable Generalizable

Answers Why? How? Answers How many? When? Where?

Formative, earlier phases Tests hypotheses, latter phases

Data are “rich” and time-consuming to analyze

Data are more efficient, but may miss contextual detail

Design may emerge as study unfolds

Design decided in advance

Researcher IS the instrument Various tools, instruments employed

Qualitative “vs.” Quantitative?Qualitative “vs.” Quantitative?

Research question guides the choice Research question guides the choice Choice of methods is situationalChoice of methods is situational ComplementaryComplementary

– Research more complete with bothResearch more complete with both

Each requires resources and timeEach requires resources and time– Can be amended for rapid analysisCan be amended for rapid analysis

Types of Qualitative MethodsTypes of Qualitative Methods

InterviewsInterviews

InformalInformal – researcher is required to recollect – researcher is required to recollect discussiondiscussion

UnstructuredUnstructured – e.g. ethnographic interviewing – – e.g. ethnographic interviewing – researcher allows interview to proceed at researcher allows interview to proceed at respondent’s pace and subjects to vary by respondent’s pace and subjects to vary by interviewee (to an extent)interviewee (to an extent)

Semi-structuredSemi-structured – researcher uses an interview – researcher uses an interview guide guide

StructuredStructured – researcher uses identical stimuli and – researcher uses identical stimuli and adheres to interview scheduleadheres to interview schedule

Structured InterviewsStructured Interviews

QuestionnairesQuestionnaires

Three methods:Three methods:

1.1. Face-to-face interviewsFace-to-face interviews

2.2. Self-administered questionnairesSelf-administered questionnaires

3.3. Telephone interviewsTelephone interviews

Face-to-Face Interviews (and Face-to-Face Interviews (and intercept interviews)intercept interviews)

Advantages:Advantages: Can be used with respondents who wouldn’t be Can be used with respondents who wouldn’t be

able to provide information in another format – able to provide information in another format – bedridden, illiterate, etc.bedridden, illiterate, etc.

Researcher can elicit more in-depth response or Researcher can elicit more in-depth response or fill in information if participant doesn’t understand fill in information if participant doesn’t understand the questionthe question

Different data collection techniques – open-ended Different data collection techniques – open-ended questions, visual aids, etc.questions, visual aids, etc.

Certainty about who answered the questionsCertainty about who answered the questions

Face-to-Face Interviews, cont.Face-to-Face Interviews, cont.

Disadvantages:Disadvantages:

Intrusive and reactiveIntrusive and reactive Cost time and moneyCost time and money Difficult to locate respondents for callbacksDifficult to locate respondents for callbacks

Self Administered QuestionnairesSelf Administered Questionnaires

Advantages:Advantages: Post Office locates participantsPost Office locates participants Everybody gets the same questionsEverybody gets the same questions Researcher can ask more complex Researcher can ask more complex

questions questions No response effect (willing to divulge more No response effect (willing to divulge more

info w/o face-to-face contact; less likely to info w/o face-to-face contact; less likely to try to impress interviewertry to impress interviewer

Can be computer-basedCan be computer-based

Self Administered Questionnaires, Self Administered Questionnaires, cont.cont.

Disadvantages:Disadvantages: No control over participant interpretationNo control over participant interpretation Low response ratesLow response rates Uncertainty about who actually filled out the Uncertainty about who actually filled out the

questionnairequestionnaire Useless with non-literate, illiterate populations Useless with non-literate, illiterate populations

(same problem with English language (same problem with English language sampling) or hard-to-reach populationssampling) or hard-to-reach populations

Telephone InterviewsTelephone Interviews

Advantages:Advantages: Combo of face-to-face personal quality with Combo of face-to-face personal quality with

impersonal self-administered questionnairesimpersonal self-administered questionnaires Inexpensive and convenient (maybe)Inexpensive and convenient (maybe) Safe for interviewersSafe for interviewers

Telephone Interviews, cont.Telephone Interviews, cont.

Disadvantages:Disadvantages:

Changing demographics – more cell phones? Changing demographics – more cell phones? – May miss certain population segmentsMay miss certain population segments

Survey must be short or people will hang upSurvey must be short or people will hang up ““No Call Lists” presenting increasing challengeNo Call Lists” presenting increasing challenge

Semi-structured InterviewsSemi-structured Interviews

Focus GroupsFocus Groups

Recruited to discuss particular topicRecruited to discuss particular topic One focus group is ONE unit of analysisOne focus group is ONE unit of analysis Complement surveys – often the 1st step Complement surveys – often the 1st step

in tapping critical questions to be used in a in tapping critical questions to be used in a surveysurvey

Identify why people feel certain way and Identify why people feel certain way and elucidate steps in their decision-making elucidate steps in their decision-making processprocess

Focus Group MethodsFocus Group Methods

Ideal size: 6 – 12 people and a Ideal size: 6 – 12 people and a moderator/note takermoderator/note taker

Series of groups is necessary for validitySeries of groups is necessary for validity Homogeneity and anonymity in selection of Homogeneity and anonymity in selection of

groupsgroups– people may open up with others who are people may open up with others who are

perceived to think along similar lines perceived to think along similar lines ANDAND whom they may never see againwhom they may never see again

Focus Group Methods, cont.Focus Group Methods, cont.

Often segment according to expected Often segment according to expected meaningful differences (e.g. disease status, meaningful differences (e.g. disease status, gender…)gender…)

Running a focus group – fine line between Running a focus group – fine line between leading too much and not getting people to leading too much and not getting people to contributecontribute

Important to keep discussion on topic w/o Important to keep discussion on topic w/o shutting people downshutting people down

No right or wrong answersNo right or wrong answers


AnalysisAnalysis Tapes are usually transcribed verbatimTapes are usually transcribed verbatim Text is sorted into emergent themes by at Text is sorted into emergent themes by at

least 2 researchers to ensure validity least 2 researchers to ensure validity either through pile-sort method or either through pile-sort method or computerized version such as CDC’s computerized version such as CDC’s EasyText (free!)EasyText (free!)

Themes are compared with field notes Themes are compared with field notes taken by second researchertaken by second researcher


Thematic coding factors:Thematic coding factors:

1. Frequency – number of times something is 1. Frequency – number of times something is mentionedmentioned

2. Specificity – details2. Specificity – details

3. Emotion – enthusiasm, passion, etc. in 3. Emotion – enthusiasm, passion, etc. in responsesresponses

4. Extensiveness – how many different 4. Extensiveness – how many different people said something similarpeople said something similar

Focus Groups vs. SurveysFocus Groups vs. Surveys

Surveys offer quantitative measurements Surveys offer quantitative measurements based on a representative samplebased on a representative sample

Focus groups offer content insight – the Focus groups offer content insight – the whywhy of what people think of what people think

Real World ExamplesReal World Examples

West Nile Virus (WNV) focus group research in West Nile Virus (WNV) focus group research in CO in 2003 CO in 2003

11 semi-structured focus group interviews were 11 semi-structured focus group interviews were conductedconducted– Groups were recruited through community gatekeepers Groups were recruited through community gatekeepers

and composed of participants from senior centers, and composed of participants from senior centers, health departments, neighborhood and volunteer health departments, neighborhood and volunteer organizations and local businessesorganizations and local businesses

Purpose was to explore and contextualize Purpose was to explore and contextualize use/non-use of repellent and KAP around WNV use/non-use of repellent and KAP around WNV and mosquitoes in COand mosquitoes in CO

Real World ExamplesReal World Examples

WNV in Colorado – Selected resultsWNV in Colorado – Selected results * * NO ONE thought they were elderly, so message NO ONE thought they were elderly, so message needed to be changed to reflect “real” riskneeded to be changed to reflect “real” risk– CO residents didn’t perceive mosquitoes to be a CO residents didn’t perceive mosquitoes to be a

problem in the “high plains”problem in the “high plains”– The high proportion of WNV Fever cases diagnosed in The high proportion of WNV Fever cases diagnosed in

some areas led to perception that “everyone was sick” some areas led to perception that “everyone was sick” and higher levels of concernand higher levels of concern

– Younger people wanted to get infected before they got Younger people wanted to get infected before they got ‘old’ and the risk of neuroinvasive disease increased‘old’ and the risk of neuroinvasive disease increased

– Residents perceived that media overplayed the issueResidents perceived that media overplayed the issue

Real World ExamplesReal World Examples Border ResearchBorder Research

– WNV focus group research in Imperial Valley WNV focus group research in Imperial Valley along MX/CA border in 2004along MX/CA border in 2004

– WNV focus group research in Tijuana/San WNV focus group research in Tijuana/San Diego in 2005Diego in 2005

– WNV intercept interviews in Tucson, AZ and WNV intercept interviews in Tucson, AZ and Nogales, Sonora in 2006Nogales, Sonora in 2006

– Traditional surveys would be especially Traditional surveys would be especially difficult due to international agreements, lack difficult due to international agreements, lack of phones, lack of sampling framework, of phones, lack of sampling framework, inability to contextualize responses with inability to contextualize responses with neighborhoodsneighborhoods

MethodsMethods Topics CoveredTopics Covered

– Knowledge about WNVKnowledge about WNV– Sources of informationSources of information– Efficacy of existing educational materialsEfficacy of existing educational materials– Actions to avoid/kill mosquitoes/prevent bitesActions to avoid/kill mosquitoes/prevent bites

Repellent, chemicals, household controlRepellent, chemicals, household control

– Existence of health committees or health Existence of health committees or health information in local areainformation in local area

ResultsResults

Household control of breeding sites - Most Household control of breeding sites - Most people familiar with “control of standing people familiar with “control of standing water”water”– Varied understandings of “standing water”Varied understandings of “standing water”

Many Many CulexCulex breeding sites beyond means of breeding sites beyond means of household control—related to irrigation household control—related to irrigation and/or sewage canalsand/or sewage canals

Results—Information Results—Information SourcesSources

Media most frequently cited as source by Media most frequently cited as source by those who knew of WNVthose who knew of WNV

Spanish speakers in US may not be Spanish speakers in US may not be accessing local newsaccessing local news– Many report primarily watching satellite Many report primarily watching satellite

channels from Mexicochannels from Mexico Imperial County (for example) challenged by Imperial County (for example) challenged by

lack of local stationslack of local stations

Results—Special PopulationsResults—Special Populations

Needs and risks among agricultural workers Needs and risks among agricultural workers (US) could be significant during an outbreak, (US) could be significant during an outbreak, especially among undocumented workers if especially among undocumented workers if unlikely to seek medical careunlikely to seek medical care

Need approaches for non-English and non-Need approaches for non-English and non-Spanish speakersSpanish speakers

Migrant laborers unlikely to have access to Migrant laborers unlikely to have access to television, mail, etc.television, mail, etc.

FotonovelasFotonovelas

Focus groups and interviews were again Focus groups and interviews were again employed in 2005 and 2006 in migrant employed in 2005 and 2006 in migrant worker campsworker camps

Assess efficacy of existing educational Assess efficacy of existing educational materials materials

Develop attractive, pictorial-based material Develop attractive, pictorial-based material for farm workers and others, focused on for farm workers and others, focused on repellent userepellent use

Sample Survey Questionnaire Sample Survey Questionnaire CritiqueCritique

While surveys are a strong and popular While surveys are a strong and popular method, the following are some examples of method, the following are some examples of

questions where a closed response may questions where a closed response may present significant limitations present significant limitations

Katrina Questionnaire CritiqueKatrina Questionnaire Critique

Tell me all the places you saw or heard Tell me all the places you saw or heard information about the hurricane before Katrina.information about the hurricane before Katrina.

Environmental cues (look outside)Environmental cues (look outside)

Family or friendsFamily or friends The Weather ChannelThe Weather Channel InternetInternet Local radio stationsLocal radio stations Local television stationsLocal television stations NOAA Weather RadioNOAA Weather Radio

Cell PhoneCell Phone Other (please describe)Other (please describe)

____________________________________________________________________________________

Katrina Critique, cont.Katrina Critique, cont.

Did you feel like the urgency of the Did you feel like the urgency of the situation was conveyed strongly enough situation was conveyed strongly enough to help you understand the severity of to help you understand the severity of the situation?the situation?

□ □ YES □ NOYES □ NO


Before Katrina, how much personal risk Before Katrina, how much personal risk did you think there would be for you for did you think there would be for you for major flooding, loss, displacement, and major flooding, loss, displacement, and disease? disease?

□ □ Great risk □ Some risk □ Little risk □ No riskGreat risk □ Some risk □ Little risk □ No risk


Before Katrina, how serious did you think Before Katrina, how serious did you think the hurricane and its consequences the hurricane and its consequences would be?would be?

□ □ Very serious □ Moderately serious Very serious □ Moderately serious

□ □ A little serious □ Not serious at allA little serious □ Not serious at all


When you left your home before Katrina, When you left your home before Katrina, where did you go? (Choose One)where did you go? (Choose One)

□ □ Family/friend house in my city Family/friend house in my city □ □ Family/Friend house in other city Family/Friend house in other city □ □ Hotel in my city Hotel in my city □ □ Hotel in other city Hotel in other city □ □ Evacuation center in my city Evacuation center in my city □ □ Evacuation center in other city Evacuation center in other city □ □ OtherOther


Before Katrina how confident were you in Before Katrina how confident were you in your ability to take the necessary actions your ability to take the necessary actions to protect you and your family during a to protect you and your family during a disaster?disaster?

□ □ Very confidant □ Moderately confidant Very confidant □ Moderately confidant

□ □ A little confidant □ Not confidantA little confidant □ Not confidant


Do you think that you will stay in this area Do you think that you will stay in this area (shelter community)?(shelter community)?



Should this shelter need to close would Should this shelter need to close would you be willing to move another shelter?you be willing to move another shelter?


Dengue Questionnaire CritiqueDengue Questionnaire Critique

Do you travel back and forth between the Do you travel back and forth between the US and Mexico?US and Mexico?


If so, number of days per year?If so, number of days per year?

Dengue Critique, cont.Dengue Critique, cont.

Do you have piped water 24 hours/day?Do you have piped water 24 hours/day?


If no, do you store water in an open container?If no, do you store water in an open container?


Dengue Critique, cont.Dengue Critique, cont.

Do you have screens on all windows Do you have screens on all windows and doors?and doors?


SummarySummary

Qualitative data gathering is a rich and Qualitative data gathering is a rich and important tool in some settingsimportant tool in some settings

Variety of methods availableVariety of methods available May overcome limitation of closed survey May overcome limitation of closed survey

questions and assist in development of questions and assist in development of better instrumentsbetter instruments

ReferencesReferences

Bernard, HR. (2002) Bernard, HR. (2002) Research Methods in Research Methods in Anthropology: Qualitative and Quantitative Anthropology: Qualitative and Quantitative Approaches. Approaches. Walnut Creek: AltaMira PressWalnut Creek: AltaMira Press

Krueger, RA and Casey, MA. (2000). Krueger, RA and Casey, MA. (2000). Focus Focus Groups: A Practical Guide for Applied Research.Groups: A Practical Guide for Applied Research. Thousand Oaks: Sage PublicationsThousand Oaks: Sage Publications

Puchta, C. and Potter, J. (2004) Puchta, C. and Potter, J. (2004) Focus Group Focus Group Practice. Practice. London: Sage PublicationsLondon: Sage Publications

AcknowledgmentsAcknowledgments

Emily Zielinski-Gutierrez, CDCEmily Zielinski-Gutierrez, CDC Maureen Fonseca-Ford, BIDSMaureen Fonseca-Ford, BIDS Stephen Waterman, BIDSStephen Waterman, BIDS Elisabeth Gleckler, Eye+CiteElisabeth Gleckler, Eye+Cite US-Mexico Border Health CommissionUS-Mexico Border Health Commission

qualitative research methods mary h. hayden, phd ncar summer was*is boulder, co july 14, 2007 july...

Documents