1

A Methodological Review of PCAC in the Free-Choice Paradigm

Amir Mohammad TahamtanGraduate School of Management and Economics

Sharif University of TechologyMay 7, 2016

5/6/2016, Friday

2

A historical overview of the methodology of PCAC in the FCP1. Chen (unpublished) , 2008, Rationalization and cognitive dissonance: do choices affect or reflect preferences2. Sagarin & Skowronski , 2008, The implications of imperfect measurement for free-choice carry-over effects3. Chen and Risen, 2008, Is choice a reliable predictor of choice?4. Sagarin & Skowronski , 2009, In pursuit of the proper null5. Chen and Risen, 2010, How Choice Affects and Reflects Preferences: Revisiting the Free-Choice Paradigm6. Risen and Chen, 2010, How to Study Choice-Induced Attitude Change: Strategies for Fixing the Free-Choice

Paradigm7. Alós-Ferrer et al., 2012, Choices and preferences: Evidence from implicit choices and response times8. Izuma & Murayama, 2013, Choice-induced preference change in the free-choice paradigm: a critical

methodological review 9. Alós-Ferrer & Shi, 2015, Choice-induced preference change and the free-choice paradigm: A clarification

5/6/2016, Friday

3

Scope of the problem in FCP type 1 (Chen 2008)

• Subjects are never completely indifferent

between two options.

• subjects should be expected to choose good C 66% of the time.

• CD would suggest numbers significantly higher than 66%.

• Three-door problem (or Montey Hall effect) in economics.

FCP type 1: Shifts in choices; Egan et al (2007): Children choose C 63% and capuchins choose C 60% of the time;

Use a continuous scroll for rating

?

5/6/2016, Friday

4

Scope of the problem in FCP type 2 (Chen 2008)

• FCP type 2: Shifts in rankings or ratings; most of the literature.

• If the initial ranking is an imperfect measure of preferences, then a subject’s choices teach us something new about their preferences;

• The selection bias is weaker in easy choices because fewer choose against their initial rankings, and absent in no choice.

• If a subject’s choices always conformed to their initial ratings, their choices would contain no new information.

• If 25% of subjects who initially rank A over B then choose B, then we could easily expect an increase in the spread of preferences of at least one ranking point, completely absent CD.

P(A rises and B falls | choosing A) > P(B rises and A falls | choosing A)P(A rises and B falls) = P(B rises and A falls)

P(choosing A | A rises and B falls) > P(choosing A | B rises and A falls)

5/6/2016, Friday

5

Sagarin & Skowronsky (2009)• Choice behavior tends to be probabilistic:

options that are slightly more valuable are chosen only slightly more often than options perceived as slightly less valuable.

• a statistical ‘‘expected value” for choice-making behavior: • The probability of choosing the preferred

item over the less preferred item • The probability that the chosen item

correctly identifies the under-lying preference.

5/6/2016, Friday

6

Chen & Risen’s reply(2009)• It is essential that researchers experimentally control for revealed preferences rather than

speculate how much of a role they may play.• a misunderstanding of the function of the null-hypothesis :

• the correct analysis would be to calculate what would render the observed results statistically indistinguishable from, rather than identical to experimental results.

• a misunderstanding of preference-measurement psychometrics.• The fact that external measures may contain very little predictive power for an individual’s

choice does not imply that choice itself must be only very loosely tied to the preferences the individual has at that moment.

• This is very trivial that when you force the subject to only and only choose one option, finally he chooses one of them but you cannot be sure that this was due to his definite preference. How do you know that why he has chosen that item??!!

5/6/2016, Friday

7

Sagarin & Skowronsky’s reply (2009)• Subjects would select a preferred option over

a less preferred option only a proportion of the time, and that this proportion is likely related to the magnitude of the preference difference between the two options.

• The more often the assumptions made by Chen (2008) overstate subjects’ real behavioral tendencies in choice-making, the more the expected value for the second choice in a typical two-choice study falls below 66.7% and approaches 50%.

• sufficiently accurate pretest (or suite of pretests) are needed.

How common are intransitive permutations?

Establishing a relation between the magnitude of any pretest difference and choice probabilities

Transitive permutations

Intransitive permutations

5/6/2016, Friday

8

The Problem for Studying Moderators and Mediators in the FCP(Chen 2008, Chen & Risen 2010)

• The criticism also applies to forms of the FCP that measure spreading of neural activation after a choice or examine moderators or mediators of spreading.

• Example: Japanese subjects show less cognitive dissonance than Canadians. One possible confound is that even if never asked to make choices, Japanese subjects may rate and re-rate goods more consistently that do Canadians. If this is true, then their subsequent choices contain less information and induce less selection.

5/6/2016, Friday

9

The Problem of Self-selection for spread (Chen 2008, Chen & Risen 2010)

• Because participants are not randomly assigned to make their choice, this procedure results in participants ‘self-selecting’ how chosen spread is calculated.

• If participants choose different items because they have different underlying preferences for the two items, then it is unclear whether the documented effects in the FCP are the result of attitude change following choice or, a reflection of existing preferences that are revealed by choice.

5/6/2016, Friday

10

Chen & Risen (2010)• With a formal mathematical proof and based on three assumptions demonstrate that the FCP

will measure positive chosen spread even if people have perfectly stable preferences.1. People’s ratings ⁄ rankings are at least partially (meaningfully) guided by their preferences.2. People’s choices are at least partially guided by their preferences. 3. People’s ratings ⁄ rankings are often not a perfect measure of their preferences.

 What is the difference between first and last assumption?

5/6/2016, Friday

11

Proposed solutions to fix FCP, Chen and Risen, 2010• Researchers can isolate the effect of the choice process on subsequent preferences by

i. ensuring that all participants make the same choiceii. controlling for the information revealed by choiceiii. removing the information revealed by choiceiv. manipulating the choices that people make

5/6/2016, Friday

12

1. Ensure that everyone makes the same choice• First, researchers must effectively get all participants to make the same choice. • Second, in doing so, researchers must avoid directly manipulating preferences. • It is not a preference-based choice, it is not a real choice, it is an induced one, a random one.

Something similar to ESTEKHKARE or flipping coins. This process does not make people to believe a free-choice. If there is an attitude change, it is not Choice-induced attitude change but acceptance-induced attitude change. Do the writers mean this to be used as control condition?

5/6/2016, Friday

13

2. Controlling for the information revealed by choice• Rate-Rate-Choose (RRC)• The spreading found for RRC participants demonstrates that spreading can occur in the absence

of dissonance reduction. • using a within-subject RCRC design, researchers can calculate the RRC and RCR spread for each

participant, allowing for a paired comparison. • I think it is wrong since the first RCR would affect the subsequent C.

5/6/2016, Friday

14

3. Removing the information from choice• so that participants’ choices do not reflect their preference.• 2nd assumption no longer holds.• for example, by making the choice ‘blind’.

5/6/2016, Friday

15

4. Manipulation the choice• Participants rank 15 art prints.• Before choosing between #7 and #9, they are asked to flip a coin. • If heads, they will get $1 extra if they choose #7 • If tails, they will get $1 extra if they choose #9. • If the $1 shifts people’s choices, but does not directly influence preferences, then researchers can

test for attitude change following choice by calculating how much #7 improves and #9 declines if the coin lands on heads and how much #9 improves and #7 declines if the coin land on tails.

• If spreading is positive, it suggests that the choice process influenced subsequent preferences.

5/6/2016, Friday

16

One more recent solution (Alós-Ferrer et al., 2012)

• Implicit choice paradigm• measure preference changes for two items that are not compared in a direct choice between

them, but rather through other comparisons.• participants made choices between a and h, and b and l.• h > a >b > l in the first preference rating. participants were likely to choose h over a, and b over l. • Then, preference change between the first and second rating were compared between the

rejected item a and the chosen item b.• If one of choices in the a-h and b-l pairings was not as expected, the four items were excluded

from the analysis (i.e., selection bias).

5/6/2016, Friday

17

Alós-Ferrer & Shi (2015)• We show that the result in Chen & Risen (2010) is mathematically incorrect. • Specifically, we present a formal model of decision making which satisfies all assumptions in that

article but such that spreading needs not be positive in the absence of choice-induced preference change.

• Hence, although the free-choice paradigm is flawed, the present research shows that reasonable models of human behavior need not predict positive spreading. As a consequence, experimental results remain informative.

5/6/2016, Friday