Scales and Measurement

Chapters 11, 12

Constructs and Measurement

• Construct Development• Identifying and defining what is to be measured• A construct is a hypothetical variable composed of

different elements that are thought to be related (e.g., 5 questions tapping brand loyalty)

• Measurement• Figuring out how to measure what you want to measure• Measure needs to be reliable and valid

Different Types of Reliability• Internal Reliability

• Extent to which items on a scale “hang together” or are correlated with one another

• Cronbach’s alpha (covered in last class)• Split-half reliability (split measure into halves, correlate)

• Test-Retest Reliability• Extent to which scores are stable over time• Have people complete questionnaire twice and correlate scores

Validity: Overview of Key Definitions• Validity (in general)

• The extent to which conclusions drawn from a study are true

• Internal Validity• When a researcher can clearly identify cause and effect relationships (i.e.,

there are no confounds)

• External Validity• The extent to which what you find in your study can be generalized to your

target population

• Construct Validity• Extent to which your constructs of interest (e.g., sensation seeking) are

accurately and completely identified (measured)• In other words, the extent to which you are actually measuring what you say

you are measuring (your sensation seeking scale really does measure the true construct of sensation seeking)

Other Forms of Validity• Content Validity (Face Validity)

• Extent to which a measure is appropriate according to experts in the domain of interest

• Concurrent Validity (Convergent Validity)• Extent to which one measure of a construct overlaps with other similar

measures of that construct• Discriminant Validity

• Extent to which a measure of one construct does not overlap with measures of different constructs

• Predictive Validity• Extent to which a measure of a construct can predict theoretically-

relevant outcomes• Nomological Validity

• How a construct fits within a broader set of related constructs

Key Idea

• To develop reliable and valid measures which we can subsequently (and appropriately) use in statistical analyses, we must understand:

• Properties of scales

• How to design good questions that do not lead to biased or inconsistent responses

Measurement/Scaling Properties

• Assignment• You can assign objects to categories

• Order (Magnitude)• You can order objects in terms of having more or less of some quality

• Distance (Equal Intervals)• The distance between adjacent points on the scale is identical

• Origin (Absolute Zero Point)• Zero “means something” (absence of a given quality)

Types of Scales• Nominal Scale

• Has Assignment Only (Political Party)

• Ordinal• Has Assignment, Order (Rank Order of Finish in a Race)

• Interval• Has Assignment, Order, Equal Intervals (Temperature)

• Hybrid Ordinally-Interval Scale• Like an ordinal scale, but researcher “pretends” it is an interval scale (e.g.,

assumes 1 to 7 scale is an interval scale); commonly used in questionnaires

• Ratio• Has Assignment, Order, Equal Intervals, Absolute Zero (Number of Cars)

What Type of Scale?• Number of Sweaters Purchased This Year? _______

• What is Your Ethnicity?

• To what extent do you agree or disagree that Congress should have approved the $700 bailout? (1 = strongly disagree to 7 = strongly agree)

• Please rank the following issues from most to least important (Iraq, Health Care, Economy, Environment)

• What is your income? (5-10k; 11-15k; 16-20k; 20-25k; 25-30k)

On the Importance of Attitudes

I think McCain has a good grasp

of national security issues I feel a strong

connection with Obama

I am inclined to

vote for McCain

I’m planning on voting for Obama

I think Obama’s Plan

for Health Care Reform

is a good one

I believe both candidatesbring strengths to the table

I feel McCain would be

a good leader

Three Components of Attitudes• Cognitive Component

• How a person thinks about an attitude object (product, issue, candidate, idea)

• Affective Component• How a person feels about an attitude object

• Behavioral– A person’s behavioral predisposition to respond to an attitude

object in a certain way

Three Components of Attitudes?

I think McCain has a good grasp

of national security issues I feel a strong

connection with Obama

I am inclined to

vote for McCain

I’m planning on voting for Obama

I think Obama’s Plan

for Health Care Reform

is a good one

I believe both candidatesbring strengths to the table

I feel McCain would be

a good leader

Measuring Attitudes• While attitudes not perfect predictors of behavior, still very important

• We need to understand how to measure attitudes accurately

• Today, we’ll look at some creative ways to measure attitudes

• And, some standard approaches taken in marketing research

• As we do this, think about the types of questions we might want to ask in

our restaurant concepts study

Creative Measures of Attitudes

• Projective techniques (partially structured)

• Physiological (GSR; Heart Rate; Blood Pressure)

• Functional magnetic resonance imaging (fMRI)

• Flushing toilets

• Bogus pipeline

• Lost letters and emails*

• Implicit association test (IAT)*

You’ve Got Mail!

Dear Peter Price,

Thank you for applying for a Glassner Foundation Scholarship. As you know, these scholarships are highly competitive and are given only to a few select students. They cover tuition for four years at a state-funded university. There is also an additional $500 per year for academic supplies.

Because of the large number of applicants this year we are late in sending out these notices. Because of the time sensitive nature of this material, we wanted to immediately inform you of the committee’s decision regarding your application.

We are happy to inform you that you have been selected to receive a Glassner Scholarship.

We ask that you respond within 48 hours. Due to the high number of applicants, we would like to extend the scholarship to others applicants if you do not accept the scholarship.

You’ve Got Mail!

Dear Mohammed Hameed,

Thank you for applying for a Glassner Foundation Scholarship. As you know, these scholarships are highly competitive and are given only to a few select students. They cover tuition for four years at a state-funded university. There is also an additional $500 per year for academic supplies.

Because of the large number of applicants this year we are late in sending out these notices. Because of the time sensitive nature of this material, we wanted to immediately inform you of the committee’s decision regarding your application.

We are happy to inform you that you have been selected to receive a Glassner Scholarship.

We ask that you respond within 48 hours. Due to the high number of applicants, we would like to extend the scholarship to others applicants if you do not accept the scholarship.

You’ve Got Mail!

Dear Mohammed Hameed,

Thank you for applying for a Glassner Foundation Scholarship. As you know, these scholarships are highly competitive and are given only to a few select students. They cover tuition four four years at a state-funded university. There is also an additional $500 per year for academic supplies.

Because of the large number of applicants this year we are late in sending out these notices. Because of the time sensitive nature of this material, we wanted to immediately inform you of the committee’s decision regarding your application.

We regret to inform you that you have not been selected to receive a Glasser Scholarship.

We ask that you respond within 48 hours. Due to the high number of applicants, we would like to extend the scholarship to others applicants if you do not accept the scholarship.

You’ve Got Mail!(Method)

• Bushman & Bonacci (2004, JESP)

• Pretested on Arab-American Prejudice

• 2 weeks later, receive email intended for a different person

• 2 IVs

• Intended recipient had European-American (Peter Price) vs. Arab-American name (Mohammed Hameed)

• Intended recipient won or didn’t win a scholarship (4 years support)

• DV = willingness to return the email to the sender to indicate it was incorrectly delivered (must be done in 2 days)

You’ve Got Mail!(Results)

00.10.20.30.40.50.60.70.8

Low Med High Low Med High

Arab-American Prejudice

Od

ds

of

Re

turn

ing

Em

ail

European Name Arab Name

Good News! Bad News

You’ve Got Mail!(Results)

00.10.20.30.40.50.60.70.8

Low Med High Low Med High

Arab-American Prejudice

Od

ds

of

Re

turn

ing

Em

ail

European Name Arab Name

Good News! Bad News

You’ve Got Mail!(Results)

00.10.20.30.40.50.60.70.8

Low Med High Low Med High

Arab-American Prejudice

Od

ds

of

Re

turn

ing

Em

ail

European Name Arab Name

Good News! Bad News

You’ve Got Mail!(Results)

GOOD NEWS! YOU WON!!

• P (returning email) went down as prejudice went up, especially when the intended recipient had an Arab-American name

BAD NEWS…YOU DIDN’T WIN.

• European-American: P (returning email) went down as prejudice went up

• Arab-American: P (returning email) went up as prejudice went up.

• People with Arab-American prejudice were actually more likely to return the bad news than the good news email when recipient was Arab American

WHAT MIGHT EXPLAIN SUCH ARAB-AMERICAN PREJUDICE?

Terror Management TheoryTerror Management Theory

Basic AssumptionBasic Assumption Humans have developed “anxiety buffering mechanisms” against

fear of death (cultural worldview, self-esteem, relationships)

Typical MethodTypical Method Write about own death (mortality salience) or dental pain Filler task Judge others from ingroup vs. outgroup

Typical ResultTypical Result We judge worldview threatening others more harshly under

conditions of mortality salience Support for Bush (vs. Kerry) goes up after MS prime

Implicit Association Task (IAT)• An implicit attitude is an attitude which people are not conscious of (or

would not want to admit to) but which can be assessed via the associations people hold in their minds between evaluative words (good, bad) and attitude objects (Caucasian, Arab)

• The IAT is a reaction time (RT) task in which subjects categorize words (positive, negative) and attitude objects (pictures of Caucasians, Arabs) on the left or right side of screen

• To simplify, some trials involve categorizations that are consistent with an implicit attitude, others are inconsistent with the attitude

• If there is an implicitly negative attitude toward Arabs, then reaction times should be slower to the inconsistent trials

The following set of trials is consistent

with an implicitly negativeattitude toward Arabs

Reaction times should berelatively fast if there

is an implicitly negative attitudetoward Arabs

Good Word orCaucasian

Bad Word orArab

If you see a good word or a Caucasian, hit “e” (left)

If you see a bad word or an Arab, hit “i” (right)

Good Word orCaucasian

Bad Word orArab

Good Word orCaucasian

Bad Word orArab

Dirt

Good Word orCaucasian

Bad Word orArab

Good Word orCaucasian

Bad Word orArab

Happy

Now Change CategorizationNow Change Categorization

(The following trials areinconsistent with implicit negative

attitude toward Arabs)

Reaction times should be slower on these trials if there is an implicitly

negative attitude toward Arabs

Bad Word orCaucasian

Good Word orArab

Categorization Switched

Bad Word orCaucasian

Good Word orArab

Bad Word orCaucasian

Good Word orArab

Joy

Bad Word orCaucasian

Good Word orArab

Death

Bad Word orCaucasian

Good Word orArab

What are we saying?!

• Sensitive attitudes (e.g., racial prejudice) are sometimes hard to assess with a self-report scale (due to social desirability concerns)

• The IAT measures associations (automatic attitudes)

• Implicitly negative attitudes are not uncommon, but…

• People can also override these automatic responses with controlled processes (so behavior is not discriminatory)

• For more information, visit https://implicit.harvard.edu/implicit/ where you can take an IAT for the 2008 election (or other issues)

Verbal Rating Scales

How satisfied were you with today’s meal?

1 2 3 4 5 6 7

Extremely Dissatisfied Somewhat Neither Somewhat Satisfied ExtremelyDissatisfied Dissatisfied Satisfied Satisfied

Issues in Designing Verbal Rating Scales

• Many measures taken by marketing researchers are verbal ratings

• What do we need to consider when we develop verbal rating scales?– Number of categories– Forced vs. unforced scale– Balanced or unbalanced scale– Extent of verbal description– Should response categories be numbered or not– Comparative vs. noncomparative scale– Scale direction

Number of Response Categories?• To what extent are you satisfied with your current MP3 player?

• Most researchers suggest between 5 and 7 categories; for example:

1 2 3 4 5 6 7

Extremely Dissatisfied Somewhat Neither Somewhat Satisfied ExtremelyDissatisfied Dissatisfied Satisfied Satisfied

• Too few does not give you enough information

• Too many and it will be hard for people to discriminate between the options (e.g., a 100-point scale)

Forced vs. Unforced Scale?• How likely would you be to buy a car manufactured in Brazil?

• Forced Scale (even number of options forces the respondent to lean one way or the other):

1 2 3 4 5 6

Very Unlikely Somewhat Somewhat Likely VeryUnlikely Unlikely Likely Likely

• Unforced scale gives people a neutral option:

1 2 3 4 5 6 7Very Unlikely Somewhat Neither Somewhat Likely Very

Unlikely Unlikely Likely Likely

Balanced vs. Unbalanced Scale?• How satisfied are you with your current hair stylist?

• Balanced scale (same number of positive and negative options):

1 2 3 4 5 6 7

Extremely Dissatisfied Somewhat Neither Somewhat Satisfied ExtremelyDissatisfied Dissatisfied Satisfied Satisfied

• Unbalanced scale (here all options are positive):

1 2 3 4 5 6 7Somewhat Very

Satisfied Satisfied

• Unbalanced scale can give biased results; unless distribution is naturally skewed to one side of the scale, should use balanced scale

Extent of Verbal Description?• The U.S. should invest in wind powered energy

• Label endpoints or label all options?

1 2 3 4 5 6 7Strongly Strongly

Disagree Agree

1 2 3 4 5 6 7Strongly Moderately Slightly Neither Agree Slightly Moderately Strongly

Disagree Disagree Disagree or Disagree Agree Agree Agree

• Labeling all options can aid in interpretation.

Should Categories be Numbered?• Toyota is an Environmentally Friendly Company

Strongly Moderately Slightly Neither Agree Slightly Moderately StronglyDisagree Disagree Disagree or Disagree Agree Agree Agree

1 2 3 4 5 6 7-3 -2 -1 0 1 2 3

• Numbers can help respondents understand scale

• 1 to 7 scale quite common

• But -3 to +3 can help interpretation of scale (disagree is negative, agree is positive); it may, however, overemphasize negativity

• Judgment call; pretesting both scales could help identify problems

Should we havenumbers here?

Comparative vs. Noncomparative?• Noncomparative question

• How would you evaluate Mintifresh toothpaste?

• Comparative question• Compared to your current brand, how would you evaluate

Mintifresh toothpaste?

• Comparative questions establish the referent and can be useful if you need to know how your product compares to a specific competitor or the customer’s current brand

• Noncomparative have the advantage of allowing the respondent to create their own referent, which can potentially improve accuracy

Direction of Scale?• Typical direction (lower values, negative connotation on left):

Strongly Moderately Slightly Neither Agree Slightly Moderately StronglyDisagree Disagree Disagree or Disagree Agree Agree Agree

1 2 3 4 5 6 7

• Some scales are not valenced, so must be careful about positioning. Here we see a semantic differential scale, with amusing positioning:

Unpleasant -2 -1 0 1 2 Pleasant

Flimsy -2 -1 0 1 2 SturdyMale -2 -1 0 1 2 Female

• Hmmm…this arrangement suggests that males are to be evaluated negatively; must be careful in designing scales so as not to bias results

Other Types of Scales

Rank order the restaurants above where 1 = most preferred, 3 = least preferred

Rank-Order andPaired Comparison Techniques

• Rank-order technique• Advantages: easy to understand, typically what we do in real-

life (vs. ratings)• Disadvantage: it may be that a person dislikes all of the

options, so ranking not that informative• Paired comparisons

• Take n products, compare each one to every other (in each pair, pick the one you prefer)

• Gives you direct comparisons, but # paired comparisons can be very large as number of products to be compared increases

pairings 62

)14(4

2

)1(pairings ofNumber products, 4With

nn

Semantic Differential Scale• Please provide your impression of Politician X:

Dumb _____ _____ _____ _____ __X__ Smart

Cold _____ _____ _____ __X__ _____ Warm Boring _____ _____ _____ _____ __X__ Funny Mean _____ _____ _____ __X__ _____ Nice

• Please provide your impression of Politician Y:Dumb _____ __X__ _____ _____ _____ Smart

Cold __X__ _____ _____ _____ _____ Warm Boring _____ _____ __X__ _____ ____ Funny Mean _____ __X__ _____ _____ _____ Nice

• Profile Analysis of Politician X vs. Y:Dumb _____ _____ _____ _____ _____ Smart

Cold _____ _____ _____ _____ _____ Warm Boring _____ _____ _____ _____ _____ Funny Mean _____ _____ _____ _____ _____ Nice

X

Y

Stapel Scale• Please provide your impression of Shoe Store X. (Use negative numbers

if you feel it is inaccurate; positive numbers if you feel it is accurate):

-5 -4 -3 -2 -1 Fast Service 1 23 4 5

-5 -4 -3 -2 -1 Friendly 12 3 4 5-5 -4 -3 -2 -1 Knowledgeable 12 3 4 5

• Can draw comparative profile analysis (e.g., of various shoe stores) as we did with semantic differential scale (previous slide)

Questionnaire Design

Chapter 13

Some Key Issues

Questions: Common Pitfalls• Incomprehensible

• When you consider a new stadium, is it possible that part of your determining factors might rest on the fact that you sometimes choose to forgo entertainment in favor of more pedestrian activities like walking your dog?

• Unanswerable• Will building a new stadium in Boise cost too much?

• Leading• Wouldn’t you agree that it is a great idea to build a new multiuse

facility in Boise?• Double-barreled

• Do you think the old stadium needs to be replaced and a new stadium should be built downtown?

Purpose of Questions• Descriptive

• Describe characteristics of your sample

• Predictive (Hypothesis Testing)• What demographic factors are associated with support for the stadium?

• Every question should be designed to provide useful information• What is our primary goal in the stadium study?• What questions are we trying to address?

• How do our questions stack up? • Good, bad questions? • Do they each answer a question? Descriptive? Predictive?

Arrangement of Questions: Flowerpot Approach

Exhibit 13.5