bad boy matrix question - whatcha gonna do when they come for you?

Download Bad Boy Matrix Question - Whatcha gonna do when they come for you?

Post on 18-Jul-2015

952 views

Category:

Business

0 download

Embed Size (px)

TRANSCRIPT

  • BAD BOY MATRIX QUESTION Whatcha gonna do when they come for you?

    Florian Tress

    ODC Services GmbH, Germany

    GENERAL ONLINE RESEARCH 2012

    5 7 March 2012 at the DHBW Mannheim

  • 2 THE VENN DIAGRAM OF FIELDWORK

    PEOPLE RESEARCHERS

    Fun & entertainment

    Incentives

    Disport

    Curiosity

    Express an opinion

    Solve problems

    Collect (a lot of) valid data

    Gain a lot of knowledge

    Implement special

    (and possibly boring)

    methods

  • 3 BAD BOY MATRIX QUESTION

    PEOPLE

    Monotonous, boring

    Overwhelming variety of options: decision paralysis

    Nondifferentiation, satisficing behavior

    Another question format would be more appropriate: Inferior data

    RESEARCHERS

    Standardized data format, comparability of responses (with common orientation)

    Maximize amount of data, minimize length of interview

    Facilitate statistical procedures, e.g. factorial analysis, indices, etc.

    Availability of validated test instruments , e.g. Big Five

  • 4 STANDARD ALTERNATIVES

    MULTIPLE CHOICE

    Matrix with a two point scale

    Additional option None of these (mandatory question)

    RANKING

    Bring statements in an order

    Perfectly differentiated data

  • 5 SPECIAL ALTERNATIVE: THE CAROUSEL

    Only one statement presented at once

    New statements slide in from the left

    Response options remain in the same place

  • 6 SPECIAL ALTERNATIVE: DRAG ME

    All statements presented at once

    Center stands for respondent

    Measures distance from center

    Arrangement of statements unimportant

  • 7 COMPARISON OF THESE ALTERNATIVES

    Monadic Questionnaire : Random assignment to one of these alternatives

    n = 1080; 216 interviews per alternative; good spread over age and education

    Indicators: Comparability, Trustworthiness, Data Quality, Satisfaction, Technical Requirements

  • 8 COMPARABILITY: BRAND LIKEABILITY

    39

    39

    43

    47

    52

    53

    54

    62

    66

    73

    McDonalds

    Siemens

    BMW

    Coca Cola

    Nutella

    Volkswagen

    IKEA

    Google

    Nivea

    Amazon

    Multiple Choice

    3,2

    3,7

    3,8

    3,9

    4,2

    4,2

    4,6

    5,0

    5,8

    6,6

    McDonalds

    Siemens

    IKEA

    Nutella

    Coca Cola

    BMW

    Volkswagen

    Google

    Nivea

    Amazon

    Ranking

    3,1

    3,5

    3,5

    3,5

    3,5

    3,6

    3,7

    3,9

    3,9

    4,0

    McDonalds

    IKEA

    BMW

    Siemens

    Coca Cola

    Nutella

    Volkswagen

    Google

    Nivea

    Amazon

    Matrix

    44

    48

    49

    51

    52

    52

    57

    61

    66

    68

    McDonalds

    BMW

    Nutella

    Volkswagen

    Siemens

    Ikea

    Coca Cola

    Nivea

    Google

    Amazon

    Drag Me

    3,2

    3,4

    3,4

    3,5

    3,5

    3,5

    3,7

    3,9

    4,1

    4,1

    McDonalds

    Nutella

    Coca Cola

    BMW

    Siemens

    Ikea

    Volkswagen

    Google

    Nivea

    Amazon

    Carousel

    3 F

    acto

    rs (

    Cum

    .%:

    49)

    | C

    ronbachs

    :

    n.a

    . 3 F

    acto

    rs (

    Cum

    .%:

    56)

    | C

    ronbachs

    :

    0,7

    7.

    3 F

    acto

    rs (

    Cum

    .%:

    58)

    | C

    ronbachs

    :

    0,7

    8.

    3 F

    acto

    rs (

    Cum

    .%:

    57)

    | C

    ronbachs

    :

    0,7

    3.

    3 Factors (Cum.%: 60) | Cronbachs : 0,80.

  • 9 COMPARABILITY: ATTITUDES

    11

    14

    18

    24

    28

    28

    30

    38

    44

    62

    I

    H

    J

    G

    B

    D

    F

    E

    C

    A

    Multiple Choice

    3,0

    3,4

    3,6

    3,8

    4,0

    4,6

    4,9

    4,9

    6,4

    6,4

    G

    H

    I

    J

    B

    F

    E

    D

    A

    C

    Ranking

    2,8

    2,8

    3,0

    3,1

    3,2

    3,2

    3,5

    3,5

    3,6

    3,8

    J

    I

    H

    G

    F

    E

    D

    C

    B

    A

    Matrix

    32

    39

    41

    44

    49

    50

    52

    55

    63

    65

    H

    I

    J

    B

    D

    G

    F

    E

    C

    A

    Drag Me

    2,9

    3,0

    3,1

    3,1

    3,4

    3,4

    3,5

    3,5

    3,7

    3,9

    I

    J

    G

    H

    F

    E

    D

    B

    C

    A

    Carousel

    (A) Wenn ich gute Erfahrungen mit einer Marke mache, empfehle ich sie aktiv weiter. (B) Werbung sollte mich strker ber Marken informieren, die ich noch nicht kenne. (C) Ich

    habe feste Marken, die ich bevorzugt einkaufe. (D) Neuartige und innovative Produkte passen gut zu meinem Lebensstil. (E) Ich probiere hufig neue Marken aus, die ich noch

    nicht kenne. (F) Ich bevorzuge Marken, die auf eine lange Tradition zurckblicken. (G) Ich bin bereit, fr Markenprodukte mehr Geld auszugeben. (H) Werbung sollte mich strker

    ber Marken informieren, die ich bereits gut kenne. (I) Je bekannter eine Marke ist, desto leichter kann man ihr vertrauen. (J) Die Bekanntheit einer Marke sagt etwas ihre Qualitt aus.

    3 F

    acto

    rs (

    Cum

    .%:

    51)

    | C

    ronbachs

    :

    n.a

    . 3 F

    acto

    rs (

    Cum

    .%:

    50)

    | C

    ronbachs

    :

    0,5

    6.

    3 F

    acto

    rs (

    Cum

    .%:

    61)

    | C

    ronbachs

    :

    0,8

    0.

    3 F

    acto

    rs (

    Cum

    .%:

    63)

    | C

    ronbachs

    :

    0,8

    2.

    3 Factors (Cum.%: 63) | Cronbachs : 0,82.

  • 10 RESULT: COMPARABILITY

    WINNER

    Results correspond (roughly) for the most / least likeable brands (agreeable statements)

    But: Drag Me seems to measure something different / to be biased by third variables

    LOSER

  • 11 TRUSTWORTHINESS: BRAND LIKEABILITY

    MULTIPLE CHOICE

    Words pos.: 5,5

    Words neg.: 5,4

    Non-Response: 3%

    MATRIX

    Words pos.: 6,8

    Words neg.: 7,3

    Non-Response: 2%

    RANKING

    Words pos.: 6,3

    Words neg.: 6,7

    Non-Response: 3%

    CAROUSEL

    Words pos.: 7,7

    Words neg.: 7,0

    Non-Response: 3%

    DRAG ME

    Words pos.: 7,2

    Words neg.: 7,2

    Non-Response: 2%

    Follow-Up-Exploration: Why do you think, this is the most / least likeable brand?

  • 12 TRUSTWORTHINESS: ATTITUDES

    MULTIPLE CHOICE

    Words pos.: 7,4

    Words neg.: 8,6

    Non-Response: 11%

    MATRIX

    Words pos.: 7,9

    Words neg.: 9,6

    Non-Response: 9%

    RANKING

    Words pos.: 8,3

    Words neg.: 8,1

    Non-Response: 7%

    CAROUSEL

    Words pos.: 9,3

    Words neg.: 10,0

    Non-Response: 8%

    DRAG ME

    Words pos.: 8,4

    Words neg.: 8,6

    Non-Response: 7%

    Follow-Up-Exploration: Why did you agree / disagree with this statement?

  • 13

    WINNER

    RESULT: TRUSTWORTHINESS

    LOSER

    nonspecific / general

    because I like it specific / rich in detail

    good quality, fair prices

  • 14 DATA QUALITY: NONDIFFERENTIATION

    13%

    0%

    6%

    4%

    0%

    8%

    0%

    8%

    5%

    0%

    Multiple Choice Ranking Matrix Carousel Drag Me

    Brand Likeability Attitudes

    Low sample size!

  • 15 DATA QUALITY: NONDIFFERENTIATION

    2% 5%

    3%

    3%

    Brand Likeability Attitudes

    2% 2%

    2% 2%

    Brand Likeability Attitudes

    6% 3%

    7%

    4%

    Brand Likeability Attitudes

    6%

    2%

    7%

    5%

    Brand Likeability Attitudes

    3% 5%

    2%

    3%

    Brand Likeability Attitudes

    2% 3%

    2% 1%

    Brand Likeability Attitudes

    MULTIPLE CHOICE MATRIX CAROUSEL

    Educati

    on

    Age

    younger

    40

    old

    er

    low

    er

    A-L

    . h

    igher

    Low sample size!

  • 16

    WINNER

    RESULT: DATA QUALITY

    LOSER

    The data is perfectly differentiated with the Ranking and Drag Me Question.

    Among the other three alternatives, the Carousel performs best.

  • 17 SATISFACTION

    66

    70

    71

    73

    75

    28

    23

    18

    20

    19

    3

    6

    8

    6

    3

    Multiple Choice

    Matrix

    DragMe

    Ranking

    Carousel

    Usability

    63

    65

    66

    67

    71

    31

    25

    23

    22

    20

    4

    9

    10

    11

    5

    Multiple Choice

    Matrix

    DragMe

    Ranking

    Carousel

    Layout

    60

    61

    64

    66

    69

    22

    21

    19

    18

    15

    15

    15

    11

    11

    13

    Matrix

    Multiple Choice

    DragMe

    Ranking

    Carousel

    Length

    54

    54

    55

    57

    64

    12

    15

    16

    14

    15

    14