Battle Of The Scales

49%

42%

Scales are a cornerstone of market research

Measurement Scales & Response Styles

A person’s tendency to systematically respond to questionnaire items in a certain way regardless of content

Paulhus1991

ARSMRS

ERS

Two studies to examine scale usage across two modes of data collection

Scope Of This Paper

Mobile surveys

Web surveys

Overall Research PlanWeb and Mobile

Include behavioral statements that

should correlate to attitudinal questions

Globallyrelevant topics

7-minuteattitudinal survey

Mix of positive & negative wording

Scales with published measures of reliability

Fielded in multiple countries to analyze

cross-cultural differences

Battle of the ScalesAnalysis Plan for Studies 1 & 2

Extreme Response

Style Index (ERSI)

Acquiescence Response

Style Index (ARSI)

Medium Response

Style Index (MRSI)

1 1 1Indices were averaged for respondents

Value of each index ranges

from a minimum of 0 to a maximum

of 1.0.

Overall Research PlanWeb & Mobile Surveys

Used only 5-point Likert scales for all constructs

Used only 5-point Likert scales for all constructs

Slider scales on native app

Slider scales on mobile web browser

Standard grid questions on mobile web browser

Fielded in 10 countries

4-point scales

5-point unlabeled scales

5-point labeled scales

7-point scales

10-point scales

Research Questions & Hypotheses: Study 2Differences in Response Styles Across Countries, Genders & Ages

Acquiescent Midpoint Extreme

H1: There will be nodifferences between

male & femalerespondents with

respect toERS & MRS

H2: ERS & MRS willdiffer across age

categories, but therelationship between

age & ERS or MRSwill not be monotonic

H3: There will be significant differencesacross countries withrespect to ERS, MRS

& ARS

Results 1: Study 2Impact of Gender on Response Styles

Hypothesis 1: There will be no meaningful differences in ERS, MRS or ARS indices between male & female respondents

.43 .32 .28

.43 .33 .28

ERSI ARSI MRSI

Results 2: Study 2Impact of Age on Response Styles

15-17 18-24 25-34 35-44

50

45

30

25

20

045-54 55-64 65-74

35

Hypothesis 2: ERS & MRS will differ across age categories,but the relationship between age & ERS or MRS will not be monotonic

40

75-84 85+

MRS (Medium) ARS (Acquiescent) ERS (Extreme)

Results 3: Study 2Differences in Response Styles Across Countries

60

30

20

10

0

40

H3: There will be significant differences across countries with respect to ERS, MRS & ARS

50

ERS (Extreme) ARS (Acquiescent)

Results 4: Study 2Impact of Response Scales on Response Styles

4 Point 5 Point Labelled

50

45

30

25

20

05 Point Unlabelled 7 Point

35

RQ3: Does the use of 4-point, 5-point, 7-point & 10-point scales make any difference to the level of extreme, acquiescent or midpoint responding?

40

10 Point

MRS (Medium) ARS (Acquiescent) ERS (Extreme)

Scope Of This Paper

What is the effect of using 3 alternative data collection optionson response styles in mobile surveys?

Compare differences in responses to 5-point Likert scalesacross 3 data collection options:

Standard grid questions on mobile web browser

Slider scales on mobile web browser

Slider scales onnative app

Results 1: Study 1Differences in Extreme Response Style Across Modes

Health environment

sensitivity

Personalhealth

responsibility

Attitudetoward

helping others

Materialvaluesscale

70

60

40

30

20

0

Extreme response style(ERS) index

Attitude towards advertising in general (AAG)

Onlinepiracy

concern

Lietolerance

scale

50

Online web labeled Online web unlabeled Mobile native app

Mobile web grid Mobile web slider

Results 2: Study 1

Health environment

sensitivity

Personalhealth

responsibility

Attitudetoward

helping others

Materialvaluesscale

50

30

20

10

0

Midpoint Response Style (MRS) Index

Differences in Midpoint Response Style Across Modes

Attitude towards advertising in general (AAG)

Onlinepiracy

concern

Lietolerance

scale

40

Online web labeled Online web unlabeled Mobile native app

Mobile web grid Mobile web slider

Results 3: Study 1

Health environment

sensitivity

Personalhealth

responsibility

Attitudetoward

helping others

Materialvaluesscale

80

60

40

20

0

Acquiescence Response Style (ARS) Index

Differences in Acquiescence Response Style Across Modes

Attitude towards advertising in general (AAG)

Onlinepiracy

concern

Lietolerance

scale

Online web labeled Online web unlabeled Mobile native app

Mobile web grid Mobile web slider

ConclusionsWeb

SurveysMobile

Surveys