survey research - department of public healthpublichealth.lacounty.gov › ha › present ›...
TRANSCRIPT
![Page 1: Survey Research - Department of Public Healthpublichealth.lacounty.gov › ha › present › Staff_presentations › SurveyResearch.pdfSurvey Research Amy S. Lightstone, MPH, MA Health](https://reader033.vdocuments.mx/reader033/viewer/2022060211/5f04d6897e708231d40ff5d7/html5/thumbnails/1.jpg)
Survey Research
Amy S. Lightstone, MPH, MA
Health Assessment Unit
Office of Health Assessment and Epidemiology
![Page 2: Survey Research - Department of Public Healthpublichealth.lacounty.gov › ha › present › Staff_presentations › SurveyResearch.pdfSurvey Research Amy S. Lightstone, MPH, MA Health](https://reader033.vdocuments.mx/reader033/viewer/2022060211/5f04d6897e708231d40ff5d7/html5/thumbnails/2.jpg)
Objectives
Define survey research and why survey
research is conducted
List at least 3 methods of conducting survey
research
Describe at least 3 limitations of survey research
![Page 3: Survey Research - Department of Public Healthpublichealth.lacounty.gov › ha › present › Staff_presentations › SurveyResearch.pdfSurvey Research Amy S. Lightstone, MPH, MA Health](https://reader033.vdocuments.mx/reader033/viewer/2022060211/5f04d6897e708231d40ff5d7/html5/thumbnails/3.jpg)
What Is A Survey?
Commonly used method of collecting
information about a population of interest
Gathering information from a sample of
individuals in order to learn something about the
larger population from which is was drawn
![Page 4: Survey Research - Department of Public Healthpublichealth.lacounty.gov › ha › present › Staff_presentations › SurveyResearch.pdfSurvey Research Amy S. Lightstone, MPH, MA Health](https://reader033.vdocuments.mx/reader033/viewer/2022060211/5f04d6897e708231d40ff5d7/html5/thumbnails/4.jpg)
Why Conduct a Survey?
Considerations
What data is already available?
Can the required information even be collected
in a survey?
What are the data collection goals?
What do you need and from whom?
Are the results to be descriptive, causal, or
relational?
![Page 5: Survey Research - Department of Public Healthpublichealth.lacounty.gov › ha › present › Staff_presentations › SurveyResearch.pdfSurvey Research Amy S. Lightstone, MPH, MA Health](https://reader033.vdocuments.mx/reader033/viewer/2022060211/5f04d6897e708231d40ff5d7/html5/thumbnails/5.jpg)
Why Conduct a Survey?
Additional Considerations
Want timely & accurate data
Provide benchmarks or make comparisons?
Prioritize actions based on data?
![Page 6: Survey Research - Department of Public Healthpublichealth.lacounty.gov › ha › present › Staff_presentations › SurveyResearch.pdfSurvey Research Amy S. Lightstone, MPH, MA Health](https://reader033.vdocuments.mx/reader033/viewer/2022060211/5f04d6897e708231d40ff5d7/html5/thumbnails/6.jpg)
What Is Involved in Conducting a Survey?
Set objectives for the information collected
Determine the research design
Prepare a reliable & valid data collection
instrument
Administer & score the instrument
Analyze the data
Report the results
![Page 7: Survey Research - Department of Public Healthpublichealth.lacounty.gov › ha › present › Staff_presentations › SurveyResearch.pdfSurvey Research Amy S. Lightstone, MPH, MA Health](https://reader033.vdocuments.mx/reader033/viewer/2022060211/5f04d6897e708231d40ff5d7/html5/thumbnails/7.jpg)
How To Conduct a Survey:
Components
Survey Design
Sample & Sampling
Modes/Types of Surveys
Questionnaire Design
Limitations
Logistics
Ethical Considerations
![Page 8: Survey Research - Department of Public Healthpublichealth.lacounty.gov › ha › present › Staff_presentations › SurveyResearch.pdfSurvey Research Amy S. Lightstone, MPH, MA Health](https://reader033.vdocuments.mx/reader033/viewer/2022060211/5f04d6897e708231d40ff5d7/html5/thumbnails/8.jpg)
Survey Design
Cross sectional
Longitudinal
Successive independent samples
Test-retest
Case-control
![Page 9: Survey Research - Department of Public Healthpublichealth.lacounty.gov › ha › present › Staff_presentations › SurveyResearch.pdfSurvey Research Amy S. Lightstone, MPH, MA Health](https://reader033.vdocuments.mx/reader033/viewer/2022060211/5f04d6897e708231d40ff5d7/html5/thumbnails/9.jpg)
Sample & Sampling
Target Population – who want to generalize to
Study Population – what population can you get
access to
Sampling Frame – how gain access to them (what
source used)
Sample – who is in your study
![Page 10: Survey Research - Department of Public Healthpublichealth.lacounty.gov › ha › present › Staff_presentations › SurveyResearch.pdfSurvey Research Amy S. Lightstone, MPH, MA Health](https://reader033.vdocuments.mx/reader033/viewer/2022060211/5f04d6897e708231d40ff5d7/html5/thumbnails/10.jpg)
Sampling Frame
Probability Sampling
Examples: Simple Random Sample, Stratified Random
Sample, Cluster (Area) Sample
Non-Probability Sampling
Convenience Sample (Haphazard), Purposive Sample
(Snowball, Quota)
![Page 11: Survey Research - Department of Public Healthpublichealth.lacounty.gov › ha › present › Staff_presentations › SurveyResearch.pdfSurvey Research Amy S. Lightstone, MPH, MA Health](https://reader033.vdocuments.mx/reader033/viewer/2022060211/5f04d6897e708231d40ff5d7/html5/thumbnails/11.jpg)
Sample & Sampling: Goal
In a properly designed sample, all units in the target
population have a known measurable, non-zero
chance of selection or being included in the sample,
so results can reliably be projected to the population
![Page 12: Survey Research - Department of Public Healthpublichealth.lacounty.gov › ha › present › Staff_presentations › SurveyResearch.pdfSurvey Research Amy S. Lightstone, MPH, MA Health](https://reader033.vdocuments.mx/reader033/viewer/2022060211/5f04d6897e708231d40ff5d7/html5/thumbnails/12.jpg)
Modes/Types of Surveys:
Mail Pros
Relatively inexpensive
Respondent completes when convenient
Can include pictures
Cons
Survey must be short & simple
Literacy & education levels
Languages
![Page 13: Survey Research - Department of Public Healthpublichealth.lacounty.gov › ha › present › Staff_presentations › SurveyResearch.pdfSurvey Research Amy S. Lightstone, MPH, MA Health](https://reader033.vdocuments.mx/reader033/viewer/2022060211/5f04d6897e708231d40ff5d7/html5/thumbnails/13.jpg)
Modes/Types of Surveys:
Phone Pros
Can dial random numbers
Complex survey & skip patterns
Data entered as survey completed
Cons
Oversaturation of calls, caller ID
Response rates historically have declined
More cell-only households, cost & logistics
![Page 14: Survey Research - Department of Public Healthpublichealth.lacounty.gov › ha › present › Staff_presentations › SurveyResearch.pdfSurvey Research Amy S. Lightstone, MPH, MA Health](https://reader033.vdocuments.mx/reader033/viewer/2022060211/5f04d6897e708231d40ff5d7/html5/thumbnails/14.jpg)
Source: Blumberg SJ, Luke JV. Wireless substitution: Early release of estimates from the National Health Interview
Survey, January–June 2012. National Center for Health Statistics. December 2012. Available from:
http://www.cdc.gov/nchs/nhis.htm.
:
![Page 15: Survey Research - Department of Public Healthpublichealth.lacounty.gov › ha › present › Staff_presentations › SurveyResearch.pdfSurvey Research Amy S. Lightstone, MPH, MA Health](https://reader033.vdocuments.mx/reader033/viewer/2022060211/5f04d6897e708231d40ff5d7/html5/thumbnails/15.jpg)
Modes/Types of Surveys:
In-Person Pros
Longer interviews
More detailed information, can probe
Flexibility
Cons
Very costly
Very experienced interviewers
Logistics of finding/obtaining sample
![Page 16: Survey Research - Department of Public Healthpublichealth.lacounty.gov › ha › present › Staff_presentations › SurveyResearch.pdfSurvey Research Amy S. Lightstone, MPH, MA Health](https://reader033.vdocuments.mx/reader033/viewer/2022060211/5f04d6897e708231d40ff5d7/html5/thumbnails/16.jpg)
Modes/Types of Surveys:
Internet Pros
Complex survey & skip patterns
Data entered as survey completed
Cost
Show pictures
Cons
Internet not universal
Not representative of population, may not know who
Quit in process of questionnaire, not as likely if talking
Logistics of finding/obtaining sample
![Page 17: Survey Research - Department of Public Healthpublichealth.lacounty.gov › ha › present › Staff_presentations › SurveyResearch.pdfSurvey Research Amy S. Lightstone, MPH, MA Health](https://reader033.vdocuments.mx/reader033/viewer/2022060211/5f04d6897e708231d40ff5d7/html5/thumbnails/17.jpg)
56.0% 60.5%
66.5% 70.6%
0%
20%
40%
60%
80%
100%
2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
Percent of Adults (18+ yrs) Who Access the
Internet in a Typical Week, LACHS 2002-2011
![Page 18: Survey Research - Department of Public Healthpublichealth.lacounty.gov › ha › present › Staff_presentations › SurveyResearch.pdfSurvey Research Amy S. Lightstone, MPH, MA Health](https://reader033.vdocuments.mx/reader033/viewer/2022060211/5f04d6897e708231d40ff5d7/html5/thumbnails/18.jpg)
Questionnaire Design
Keep in mind:
Analysis plan
How are you going to use data
What are the data collection goals
What do you need and from whom
![Page 19: Survey Research - Department of Public Healthpublichealth.lacounty.gov › ha › present › Staff_presentations › SurveyResearch.pdfSurvey Research Amy S. Lightstone, MPH, MA Health](https://reader033.vdocuments.mx/reader033/viewer/2022060211/5f04d6897e708231d40ff5d7/html5/thumbnails/19.jpg)
Questionnaire Design
Draft Questionnaire
Content
Wording
Response format
Order/sequence
Programming & Training
Pre-Test
Edit & Revise
![Page 20: Survey Research - Department of Public Healthpublichealth.lacounty.gov › ha › present › Staff_presentations › SurveyResearch.pdfSurvey Research Amy S. Lightstone, MPH, MA Health](https://reader033.vdocuments.mx/reader033/viewer/2022060211/5f04d6897e708231d40ff5d7/html5/thumbnails/20.jpg)
Questionnaire Design: Goals
Clear & concise
Unambiguous
Well-defined concepts
Distinct phraseology
Length
Order
![Page 21: Survey Research - Department of Public Healthpublichealth.lacounty.gov › ha › present › Staff_presentations › SurveyResearch.pdfSurvey Research Amy S. Lightstone, MPH, MA Health](https://reader033.vdocuments.mx/reader033/viewer/2022060211/5f04d6897e708231d40ff5d7/html5/thumbnails/21.jpg)
Drafting Questionnaire: Considerations
What types of questions (open/closed-ended)
How complex will questions be
Will lengthy questions be asked
Will screening questions be needed
Will long response scales be used
Can sequence be controlled
![Page 22: Survey Research - Department of Public Healthpublichealth.lacounty.gov › ha › present › Staff_presentations › SurveyResearch.pdfSurvey Research Amy S. Lightstone, MPH, MA Health](https://reader033.vdocuments.mx/reader033/viewer/2022060211/5f04d6897e708231d40ff5d7/html5/thumbnails/22.jpg)
Drafting Questionnaire: Considerations
Can respondent be expected to know about the
issue
Will respondent need to consult records
What is respondents literacy/reading level
Visuals needed
Standardized/validated question
Can compare over time or across groups
![Page 23: Survey Research - Department of Public Healthpublichealth.lacounty.gov › ha › present › Staff_presentations › SurveyResearch.pdfSurvey Research Amy S. Lightstone, MPH, MA Health](https://reader033.vdocuments.mx/reader033/viewer/2022060211/5f04d6897e708231d40ff5d7/html5/thumbnails/23.jpg)
Questionnaire Design: Examples
Double-barreled
Do you buy frozen meat and frozen fish?
![Page 24: Survey Research - Department of Public Healthpublichealth.lacounty.gov › ha › present › Staff_presentations › SurveyResearch.pdfSurvey Research Amy S. Lightstone, MPH, MA Health](https://reader033.vdocuments.mx/reader033/viewer/2022060211/5f04d6897e708231d40ff5d7/html5/thumbnails/24.jpg)
Questionnaire Design: Examples
Biased or Loaded
What do you see as the advantages of eliminating welfare?
![Page 25: Survey Research - Department of Public Healthpublichealth.lacounty.gov › ha › present › Staff_presentations › SurveyResearch.pdfSurvey Research Amy S. Lightstone, MPH, MA Health](https://reader033.vdocuments.mx/reader033/viewer/2022060211/5f04d6897e708231d40ff5d7/html5/thumbnails/25.jpg)
Questionnaire Design: Examples
Screener or filter question needed
Do you think the TV show Grey’s Anatomy shows too
much blood for being on prime time?
![Page 26: Survey Research - Department of Public Healthpublichealth.lacounty.gov › ha › present › Staff_presentations › SurveyResearch.pdfSurvey Research Amy S. Lightstone, MPH, MA Health](https://reader033.vdocuments.mx/reader033/viewer/2022060211/5f04d6897e708231d40ff5d7/html5/thumbnails/26.jpg)
Questionnaire Design: Examples
Double negative
Parents should not be allowed to send their children to
school if they are not immunized.
Change to:
Parents should be allowed to send their children to school
even if they are not immunized.
![Page 27: Survey Research - Department of Public Healthpublichealth.lacounty.gov › ha › present › Staff_presentations › SurveyResearch.pdfSurvey Research Amy S. Lightstone, MPH, MA Health](https://reader033.vdocuments.mx/reader033/viewer/2022060211/5f04d6897e708231d40ff5d7/html5/thumbnails/27.jpg)
Questionnaire Design: Examples
Does respondent have needed info & terminology
Have you ever heard of a vaccine to prevent HPV and
cervical cancer?
Change to:
Human papillomavirus also called HPV, is a common sexually
transmitted infection known to cause cervical cancer in women.
A vaccine to prevent HPV infection is available and is called the
cervical cancer vaccine or HPV shot. Before today, have you
ever heard of a vaccine to prevent HPV and cervical cancer?
![Page 28: Survey Research - Department of Public Healthpublichealth.lacounty.gov › ha › present › Staff_presentations › SurveyResearch.pdfSurvey Research Amy S. Lightstone, MPH, MA Health](https://reader033.vdocuments.mx/reader033/viewer/2022060211/5f04d6897e708231d40ff5d7/html5/thumbnails/28.jpg)
Questionnaire Design: Examples
Specific enough
Where were you born?
![Page 29: Survey Research - Department of Public Healthpublichealth.lacounty.gov › ha › present › Staff_presentations › SurveyResearch.pdfSurvey Research Amy S. Lightstone, MPH, MA Health](https://reader033.vdocuments.mx/reader033/viewer/2022060211/5f04d6897e708231d40ff5d7/html5/thumbnails/29.jpg)
Questionnaire Design: Examples
All relevant responses included
Do you use regular or premium gas in your car?
All responses are mutually exclusive
In which of the following do you live?
House
Apartment
Suburbs
![Page 30: Survey Research - Department of Public Healthpublichealth.lacounty.gov › ha › present › Staff_presentations › SurveyResearch.pdfSurvey Research Amy S. Lightstone, MPH, MA Health](https://reader033.vdocuments.mx/reader033/viewer/2022060211/5f04d6897e708231d40ff5d7/html5/thumbnails/30.jpg)
Questionnaire Design:
Response Categories Examples
Discrete enough
How often do you have pain from your arthritis?
All the time
Most of the time
Some of the time
A little of the time
Rarely
Never
![Page 31: Survey Research - Department of Public Healthpublichealth.lacounty.gov › ha › present › Staff_presentations › SurveyResearch.pdfSurvey Research Amy S. Lightstone, MPH, MA Health](https://reader033.vdocuments.mx/reader033/viewer/2022060211/5f04d6897e708231d40ff5d7/html5/thumbnails/31.jpg)
Exercise
Did you buy the new brand of laundry detergent
Clean and did you like it?
What do you see as the benefits of a tax cut?
Did you like the movie Life of Pi?
Do you smoke?
How much soda or sugar-sweetened beverages do
you drink?
![Page 32: Survey Research - Department of Public Healthpublichealth.lacounty.gov › ha › present › Staff_presentations › SurveyResearch.pdfSurvey Research Amy S. Lightstone, MPH, MA Health](https://reader033.vdocuments.mx/reader033/viewer/2022060211/5f04d6897e708231d40ff5d7/html5/thumbnails/32.jpg)
Drafting Questionnaire
Can the question be misunderstood
What assumptions does the question make
Does the question contain difficult or unclear
terminology
Is time frame specified
![Page 33: Survey Research - Department of Public Healthpublichealth.lacounty.gov › ha › present › Staff_presentations › SurveyResearch.pdfSurvey Research Amy S. Lightstone, MPH, MA Health](https://reader033.vdocuments.mx/reader033/viewer/2022060211/5f04d6897e708231d40ff5d7/html5/thumbnails/33.jpg)
Order
Strong intro conveying what, who, why
Start with easy, short, non-threatening, answerable
question
Is placement of question influenced by prior question
Ask about one topic at a time
When switching topics, use a transition
More sensitive questions towards the end
Conclusion – friendly & grateful for time & effort
![Page 34: Survey Research - Department of Public Healthpublichealth.lacounty.gov › ha › present › Staff_presentations › SurveyResearch.pdfSurvey Research Amy S. Lightstone, MPH, MA Health](https://reader033.vdocuments.mx/reader033/viewer/2022060211/5f04d6897e708231d40ff5d7/html5/thumbnails/34.jpg)
Programming & Training
Programming/Writing Questionnaire
Coding
Logical checks
Skip patterns
Training
Interviewers trained and monitored
![Page 35: Survey Research - Department of Public Healthpublichealth.lacounty.gov › ha › present › Staff_presentations › SurveyResearch.pdfSurvey Research Amy S. Lightstone, MPH, MA Health](https://reader033.vdocuments.mx/reader033/viewer/2022060211/5f04d6897e708231d40ff5d7/html5/thumbnails/35.jpg)
Pre-Test
Critical for identifying questionnaire problems
“Do not know”/”Refused” responses can be a sign of
misunderstanding or sensitivity of question
Assist in checking skip patterns & formatting
Monitoring interviewers
![Page 36: Survey Research - Department of Public Healthpublichealth.lacounty.gov › ha › present › Staff_presentations › SurveyResearch.pdfSurvey Research Amy S. Lightstone, MPH, MA Health](https://reader033.vdocuments.mx/reader033/viewer/2022060211/5f04d6897e708231d40ff5d7/html5/thumbnails/36.jpg)
Limitations & Biases
Coverage error – when your study population is not
inclusive
Sampling error – random differences that exists
between the sample and the study population
Non-response error – differences between those
who did not respond compared to those who did
Measurement error – difference between the
observed/reported value & true value
![Page 37: Survey Research - Department of Public Healthpublichealth.lacounty.gov › ha › present › Staff_presentations › SurveyResearch.pdfSurvey Research Amy S. Lightstone, MPH, MA Health](https://reader033.vdocuments.mx/reader033/viewer/2022060211/5f04d6897e708231d40ff5d7/html5/thumbnails/37.jpg)
Administrative
Costs
Facilities
Time
Personnel
Potential other resources
![Page 38: Survey Research - Department of Public Healthpublichealth.lacounty.gov › ha › present › Staff_presentations › SurveyResearch.pdfSurvey Research Amy S. Lightstone, MPH, MA Health](https://reader033.vdocuments.mx/reader033/viewer/2022060211/5f04d6897e708231d40ff5d7/html5/thumbnails/38.jpg)
Simplicity
Simplicity, simplicity, simplicity
Greatest challenge is to take important topics and
translate them into simple concepts, behaviors, and
simple words….
![Page 39: Survey Research - Department of Public Healthpublichealth.lacounty.gov › ha › present › Staff_presentations › SurveyResearch.pdfSurvey Research Amy S. Lightstone, MPH, MA Health](https://reader033.vdocuments.mx/reader033/viewer/2022060211/5f04d6897e708231d40ff5d7/html5/thumbnails/39.jpg)
Simplicity
…Otherwise, the result could be
incomplete or misleading data
item refusals
respondent fatigue effects
even the respondent’s complete refusal
![Page 40: Survey Research - Department of Public Healthpublichealth.lacounty.gov › ha › present › Staff_presentations › SurveyResearch.pdfSurvey Research Amy S. Lightstone, MPH, MA Health](https://reader033.vdocuments.mx/reader033/viewer/2022060211/5f04d6897e708231d40ff5d7/html5/thumbnails/40.jpg)
Thank You