bad boy matrix question - whatcha gonna do when they come for you?

Post on 18-Jul-2015

952 views

Category:

Embed Size (px)

TRANSCRIPT

• BAD BOY MATRIX QUESTION Whatcha gonna do when they come for you?

Florian Tress

ODC Services GmbH, Germany

GENERAL ONLINE RESEARCH 2012

5 7 March 2012 at the DHBW Mannheim

• 2 THE VENN DIAGRAM OF FIELDWORK

PEOPLE RESEARCHERS

Fun & entertainment

Incentives

Disport

Curiosity

Express an opinion

Solve problems

Collect (a lot of) valid data

Gain a lot of knowledge

Implement special

(and possibly boring)

methods

• 3 BAD BOY MATRIX QUESTION

PEOPLE

Monotonous, boring

Overwhelming variety of options: decision paralysis

Nondifferentiation, satisficing behavior

Another question format would be more appropriate: Inferior data

RESEARCHERS

Standardized data format, comparability of responses (with common orientation)

Maximize amount of data, minimize length of interview

Facilitate statistical procedures, e.g. factorial analysis, indices, etc.

Availability of validated test instruments , e.g. Big Five

• 4 STANDARD ALTERNATIVES

MULTIPLE CHOICE

Matrix with a two point scale

Additional option None of these (mandatory question)

RANKING

Bring statements in an order

Perfectly differentiated data

• 5 SPECIAL ALTERNATIVE: THE CAROUSEL

Only one statement presented at once

New statements slide in from the left

Response options remain in the same place

• 6 SPECIAL ALTERNATIVE: DRAG ME

All statements presented at once

Center stands for respondent

Measures distance from center

Arrangement of statements unimportant

• 7 COMPARISON OF THESE ALTERNATIVES

Monadic Questionnaire : Random assignment to one of these alternatives

n = 1080; 216 interviews per alternative; good spread over age and education

Indicators: Comparability, Trustworthiness, Data Quality, Satisfaction, Technical Requirements

• 8 COMPARABILITY: BRAND LIKEABILITY

39

39

43

47

52

53

54

62

66

73

McDonalds

Siemens

BMW

Coca Cola

Nutella

Volkswagen

IKEA

Nivea

Amazon

Multiple Choice

3,2

3,7

3,8

3,9

4,2

4,2

4,6

5,0

5,8

6,6

McDonalds

Siemens

IKEA

Nutella

Coca Cola

BMW

Volkswagen

Nivea

Amazon

Ranking

3,1

3,5

3,5

3,5

3,5

3,6

3,7

3,9

3,9

4,0

McDonalds

IKEA

BMW

Siemens

Coca Cola

Nutella

Volkswagen

Nivea

Amazon

Matrix

44

48

49

51

52

52

57

61

66

68

McDonalds

BMW

Nutella

Volkswagen

Siemens

Ikea

Coca Cola

Nivea

Amazon

Drag Me

3,2

3,4

3,4

3,5

3,5

3,5

3,7

3,9

4,1

4,1

McDonalds

Nutella

Coca Cola

BMW

Siemens

Ikea

Volkswagen

Nivea

Amazon

Carousel

3 F

acto

rs (

Cum

.%:

49)

| C

ronbachs

:

n.a

. 3 F

acto

rs (

Cum

.%:

56)

| C

ronbachs

:

0,7

7.

3 F

acto

rs (

Cum

.%:

58)

| C

ronbachs

:

0,7

8.

3 F

acto

rs (

Cum

.%:

57)

| C

ronbachs

:

0,7

3.

3 Factors (Cum.%: 60) | Cronbachs : 0,80.

• 9 COMPARABILITY: ATTITUDES

11

14

18

24

28

28

30

38

44

62

I

H

J

G

B

D

F

E

C

A

Multiple Choice

3,0

3,4

3,6

3,8

4,0

4,6

4,9

4,9

6,4

6,4

G

H

I

J

B

F

E

D

A

C

Ranking

2,8

2,8

3,0

3,1

3,2

3,2

3,5

3,5

3,6

3,8

J

I

H

G

F

E

D

C

B

A

Matrix

32

39

41

44

49

50

52

55

63

65

H

I

J

B

D

G

F

E

C

A

Drag Me

2,9

3,0

3,1

3,1

3,4

3,4

3,5

3,5

3,7

3,9

I

J

G

H

F

E

D

B

C

A

Carousel

(A) Wenn ich gute Erfahrungen mit einer Marke mache, empfehle ich sie aktiv weiter. (B) Werbung sollte mich strker ber Marken informieren, die ich noch nicht kenne. (C) Ich

habe feste Marken, die ich bevorzugt einkaufe. (D) Neuartige und innovative Produkte passen gut zu meinem Lebensstil. (E) Ich probiere hufig neue Marken aus, die ich noch

nicht kenne. (F) Ich bevorzuge Marken, die auf eine lange Tradition zurckblicken. (G) Ich bin bereit, fr Markenprodukte mehr Geld auszugeben. (H) Werbung sollte mich strker

ber Marken informieren, die ich bereits gut kenne. (I) Je bekannter eine Marke ist, desto leichter kann man ihr vertrauen. (J) Die Bekanntheit einer Marke sagt etwas ihre Qualitt aus.

3 F

acto

rs (

Cum

.%:

51)

| C

ronbachs

:

n.a

. 3 F

acto

rs (

Cum

.%:

50)

| C

ronbachs

:

0,5

6.

3 F

acto

rs (

Cum

.%:

61)

| C

ronbachs

:

0,8

0.

3 F

acto

rs (

Cum

.%:

63)

| C

ronbachs

:

0,8

2.

3 Factors (Cum.%: 63) | Cronbachs : 0,82.

• 10 RESULT: COMPARABILITY

WINNER

Results correspond (roughly) for the most / least likeable brands (agreeable statements)

But: Drag Me seems to measure something different / to be biased by third variables

LOSER

• 11 TRUSTWORTHINESS: BRAND LIKEABILITY

MULTIPLE CHOICE

Words pos.: 5,5

Words neg.: 5,4

Non-Response: 3%

MATRIX

Words pos.: 6,8

Words neg.: 7,3

Non-Response: 2%

RANKING

Words pos.: 6,3

Words neg.: 6,7

Non-Response: 3%

CAROUSEL

Words pos.: 7,7

Words neg.: 7,0

Non-Response: 3%

DRAG ME

Words pos.: 7,2

Words neg.: 7,2

Non-Response: 2%

Follow-Up-Exploration: Why do you think, this is the most / least likeable brand?

• 12 TRUSTWORTHINESS: ATTITUDES

MULTIPLE CHOICE

Words pos.: 7,4

Words neg.: 8,6

Non-Response: 11%

MATRIX

Words pos.: 7,9

Words neg.: 9,6

Non-Response: 9%

RANKING

Words pos.: 8,3

Words neg.: 8,1

Non-Response: 7%

CAROUSEL

Words pos.: 9,3

Words neg.: 10,0

Non-Response: 8%

DRAG ME

Words pos.: 8,4

Words neg.: 8,6

Non-Response: 7%

Follow-Up-Exploration: Why did you agree / disagree with this statement?

• 13

WINNER

RESULT: TRUSTWORTHINESS

LOSER

nonspecific / general

because I like it specific / rich in detail

good quality, fair prices

• 14 DATA QUALITY: NONDIFFERENTIATION

13%

0%

6%

4%

0%

8%

0%

8%

5%

0%

Multiple Choice Ranking Matrix Carousel Drag Me

Brand Likeability Attitudes

Low sample size!

• 15 DATA QUALITY: NONDIFFERENTIATION

2% 5%

3%

3%

Brand Likeability Attitudes

2% 2%

2% 2%

Brand Likeability Attitudes

6% 3%

7%

4%

Brand Likeability Attitudes

6%

2%

7%

5%

Brand Likeability Attitudes

3% 5%

2%

3%

Brand Likeability Attitudes

2% 3%

2% 1%

Brand Likeability Attitudes

MULTIPLE CHOICE MATRIX CAROUSEL

Educati

on

Age

younger

40

old

er

low

er

A-L

. h

igher

Low sample size!

• 16

WINNER

RESULT: DATA QUALITY

LOSER

The data is perfectly differentiated with the Ranking and Drag Me Question.

Among the other three alternatives, the Carousel performs best.

• 17 SATISFACTION

66

70

71

73

75

28

23

18

20

19

3

6

8

6

3

Multiple Choice

Matrix

DragMe

Ranking

Carousel

Usability

63

65

66

67

71

31

25

23

22

20

4

9

10

11

5

Multiple Choice

Matrix

DragMe

Ranking

Carousel

Layout

60

61

64

66

69

22

21

19

18

15

15

15

11

11

13

Matrix

Multiple Choice

DragMe

Ranking

Carousel

Length

54

54

55

57

64

12

15

16

14

15

14