• What kind of surveys do you typically receive during the year?
• What is your usual response to them?
• What makes them more intriguing so that you take them?
• Think of an example for your survey– customer satisfaction
Questions
Targeting your survey
• who is your intended market?
• how diverse are they?
• what do you most need to know?
• what level of information is needed?
• how are they best accessed?
• what are their characteristics that may affect response?
• what are their expected objections?
Designing the Questionnaire
Step 1: Specify what information will be sought
Step 2: Determine type of questionnaire and administration
method
Step 3: Determine content of individual questions
Step 4: Determine form of response to each question
Step 5: Determine wording of each question
Step 6: Determine sequence of questions
Step 7: Determine physical characteristics of questionnaire
Step 8: Reexamine steps 1 - 7 and revise if necessary
Step 9: Pretest questionnaire and revise if necessary
Wording of Questions
(a) Use simple words & short sentences
(b) Avoid ambiguous words and questions
(occasionally, sometimes, often, etc. Try to anchor each point with a specific description)
(c) Avoid leading questions
(d) Avoid implicit alternatives
(e) Avoid implicit assumptions
(f) Avoid generalizations and estimates
(g) Avoid double-barreled questions
(h) Avoid response order (i.e., position) bias
The attribute "customer service," for example, can be phrased as any one of the following:• Direct question
How satisfied are you with customer service? [Very satisfied, satisfied, Neither
satisfied nor disatisfied, Disatisfied, Very Dissatisfied
• Direct request
Please rate customer service. [Excellent, Very Good, Good, Fair, Poor]
• Statement with which to agree or disagree (Likert scale)
ABC Company provides good customer service. [Strongly agree, Agree,
Neither agree nor disagree, Disagree, Strongly Disagree]
• Question of frequency
How often does customer service exceed expectations? [Very Frequently,
Frequently, Neither frequently nor infrequently, Infrequently, Very infrequently]
• Question of extent
To what extent does customer service exceed expectations? [To very great
extent, To great extent, To some extent, To little extent, To very little extent]
Flame-broiled or Fried?
Original Question: “Do you prefer your hamburgers flame-broiled or fried?”Result: flame-broiling (Burger King) beat frying (McDonald’s) by a 3-1 margin
Revised Question #1: “Do you prefer a hamburger that is grilled on a hotstainless-steel grill or cooked by passing the raw meat through an open gas flame?”
Result: 53% preferred the grill (McDonald’s)
Revised Question #2: “The chain that grills on a hot stainless-steel griddle serves its cooked hamburgers at the proper temperature without having to use a microwave oven. And the chain that uses the gas flame puts the hamburgersafter they are cooked into a microwave oven before serving them. Just knowingthis, from which of these two chains would you prefer to buy a hamburger?”
Result: the grill (McDonald’s) beat the flame (Burger King) by a 5.5 to 1 margin
Handling Sensitive Questions
• Don’t ask unless absolutely necessary!• Give broad response categories (e.g., income, age)
• Place near end of questionnaire• “Hide” them in less sensitive questions• Use counterbiasing statement• Use randomized response technique• Transition sections & inform of rights
Use of Counterbiasing Statement
“Recent studies have shown that a high percentage of males use their wives’ cosmetics to hide blemishes. Have you used your wife’s cosmetics in the past week?”
Providing an acceptable context for answering what might have been sensitized before
Rank-Order Scales
Rank the following soft-drinks from 1 (best) to 5 (worst) according to your taste preference:
Coca-Cola _____
7-Up _____
Dr. Pepper _____
Pepsi-Cola _____
Tab _____
Paired Comparison Scale
For each of the following pairs, which soft-drink do you think is better (please check one soft-drink for each pair).
____Coca-Cola or ______7-Up
____7-Up or ______Pepsi-cola
____Tab or ______Coca-Cola
____Dr. Pepper or ______Pepsi-cola
____ Pepsi-cola or ______Tab
____ Coca-Cola or ______ DR. Pepper
____ Pepsi-cola or ______Coca-Cola
____Tab or ______ Dr. Pepper
____7-Up or ______Tab
____ Dr. Pepper or ______ 7-Up
Constant Sum Scales
Allocate a total of 100 points among the following soft-drinks depending on how favorable you feel toward each; the more highly you think of each soft-drink, the more points you should allocate to it. (Please check that the allocated points add to 100.)
Coca-Cola _____ points
7-Up _____ points
Dr. Pepper _____ points
Tab _____ points
Pepsi-Cola _____ points
100 points
Customized Rating Scales
Odd vs. Even Scale Points
Odd
Strongly Agree _____
Agree _____
Neutral _____
Disagree _____
Strongly disagree _____
Even
Strongly Agree_____
Agree _____
Disagree _____
Strongly disagree___
Balanced vs. Unbalanced Scales
Balanced
Very good ______
Good ______
Fair ______
Poor ______
Very Poor ______
Unbalanced
Excellent ______
Very Good ______
Good ______
Fair ______
Poor ______
Forced vs. Unforced Scales
Forced
Extremely Reliable ___
Very Reliable ___
Somewhat Reliable ___
Somewhat Unreliable ___
Very Unreliable ___
Extremely Unreliable ___
Unforced
Extremely Reliable ___
Very Reliable ___
Somewhat Reliable ___
Somewhat Unreliable ___
Very Unreliable ___
Extremely Unreliable ___
Don’t know ___
Labeled vs. End Anchored Scales
Labeled
Excellent _____
Very Good _____
Fair _____
Poor _____
Very Poor _____
End Anchored
Excellent __________
_____
_____
Poor _____
Number of Scale Points
5 Point
Excellent _____
_____
_____
_____
Poor _____
10 Point
Excellent __________________
_____________
_____________
_____________
_____________
_____________
_____________
_____________
_____________
Poor
Recommended Number of Scale Points
Amount of
Information
Number of Scale Points1 2 3 4 5 10 .................100
More than 5 options will provide little additional
discrimination
Begins to flatten out in return of information
Agreement:
• Strongly Agree
• Agree
• Undecided
• Disagree
• Strongly Disagree
Frequency:
• Very Frequently
• Frequently
• Occasionally
• Rarely
• Very Rarely
• Never
Importance:
• Very Important
• Important
• Moderately Important
• Of Little Importance
• Unimportant
Quality:
• Extremely Poor
• Below Average
• Average
• Above Average
• Excellent
Likelihood:
• Almost Always True
• Usually True
• Often True
• Occasionally True
• Sometimes But Infrequently True
• Usually Not True
• Almost Never True
Formats for Likert Items
1…2…3…4…5
Semantic Differential Scale
X= ABC Bank0= NBC Bank
old-fashioned ____ ____ ____ ____ ____ ____ ____ modern
not friendly ____ ____ ____ ____ ____ ____ ____ friendly
expensive ____ ____ ____ ____ ____ ____ ____ inexpensive
not trustworthy ____ ____ ____ ____ ____ ____ ____ trustworthy
X O
XO
XO
XO
Student’s fruit preferences by eating occasion(importance of time in consumption decisions)
grape
kiwi
apple
plum
orange
stra
wberry
banan
a
peach
grape
kiwi
apple
plum
orange
stra
wberry
banan
a
peach
grape
kiwi
apple
plum
orange
stra
wberry
banan
a
peach
Breakfast
Snack during day
Supper dessert
Preferences
Physical Characteristics of Questionnaire: Recommendations
• questionnaire should appear simple to complete (white space is your friend!)
• minimize number of pages (smaller fonts are OK, provided form appears simple) & items
• mix-up response formats occasionally (avoids response set bias and breaks monotony)
• use directions as necessary for each group of items, but keep them short and simple
• number items within each section• have respondents check boxes rather than lines• Use consistent response formats (don’t change too often)• use shading, boxes, lines, etc., to keep it interesting
Question considerations
• start with easy, non-threatening questions (warm up)
• place more personal, threatening questions at end
• never start mail survey with open-ended questions
• for historical demographics, follow chronological order
• ask about one topic at a time
• when switching topics use a transition
• reduce response set
• for filter or contingency questions, make a flow chart
• thank the respondent at beginning and end
• keep the survey short
• keep language simple
• include “don’t know” and “NA” for appropriate options
• try not to make questions dependent on each other
Foot in the door…
First make a small request, then when granted, make a larger (more desirable) request
• works best with prosocial/altruistic requests
• works best if no extra incentives are offered
• 10% improvement over simple request, 20% if no incentive
• works by cognitive dissonance & self esteem
Door in the face…
Make a large request, then when it is refused, make a smaller (more desired) request
• more effective if prosocial/altruistic (do it for everybody) than selfish
• should be no delay between requests
• slight delay produces 10% improvement, no delay 20%
• works by reciprocal concessions
Scale Development
• define target population
• define specific domains to be measured
• generate items reflect each domain
• construct item format
• construct survey
• test items for feedback
• revise
• administer to representative group
• frequency distribution (representative?)
• Cronbach’s alpha (above .70?)
• validation & reliability checks
• complete final version
Items Items
Items
• Frequency distribution• Cross tab & chi-square• Correlation & regression• t-test & ANOVA• Cluster analysis• CHAID• MDS
Analyzing Data
Each cluster represents a similarity grouping
Cluster Analysis: hierarchical grouping on the basis of similarity
Unity
People with older computers more likely
to upgrade
Sole user more likely to personalize
by upgrading
Influence of system
characteristics
Online realize need for faster modem
CHAID
Focus on less recent PC buyers, target for upgrade packages with faster modems
Comparison of Three Primary Methods of Administration
mail telephone personal(A) Sampling Controlability to secure sampling frame + +ability to secure correct respondent - + +response rate - + +
(B) Information Controlability to probe for detailed answers - +ability to handle complex information - +amount of information obtained - +flexibility of question sequencing - + +protection from interviewer bias + - -ability to obtain personal information + - -ability to show visual displays - +ability to establish rapport with respondent - +ability to offer anonymity + - -
(C) Administrative Controltime requirements - +cost requirements + -quality control / supervisory requirements + -
Return Rate
• sample size calculator
• notify beforehand
• free postage (SASE)
• personalize (name & address)
• relevant to needs & interests
• incentives
• closing deadline in 2-2.5 weeks
• reminder
• security, confidentiality, anonymity
• if <50% check for bias (what have in common)
• mail-ins have 3-5% response rate (10% is good)
Considerations for Web Based Surveys
• Draft the questions
• Provide a password (if needed)
• E-mail reminders (know return cutoffs)
• Personalize them (mailing list)
• Design for people with minimal skills
• Use a larger font
• Response icons
• Textured background
• Open questions use a text box
• E-mail to researcher for comments
• Thank you page
Personal and Telephone Interviews:
“foot-in-the-door” strategy (Reingen & Kernan 1977)
Mail Surveys
(1) pre-notification, especially by telephone (Schelegelmilch & Diamantopoulos 1991)
(2) cover letter / type of appeal (Houston and Nevin 1977)
(3) use SASE, not business reply envelope
(4) personalized address
(5) no bulk mail
(6) sponsorship (if corporate, refer to “research department” and never include sales pitch
(7) monetary incentives (Brennan, Hoek & Astridge 1991)
(8) offer survey results
(9) follow-up postcard, duplicate questionnaire, or telephone call