testing transitivity with individual data michael h. birnbaum and jeffrey p. bahra california state...
Post on 22-Dec-2015
218 views
TRANSCRIPT
![Page 1: Testing Transitivity with Individual Data Michael H. Birnbaum and Jeffrey P. Bahra California State University, Fullerton](https://reader035.vdocuments.mx/reader035/viewer/2022081513/56649d7e5503460f94a61622/html5/thumbnails/1.jpg)
Testing Transitivity with Individual Data
Michael H. Birnbaum and Jeffrey P. Bahra
California State University, Fullerton
![Page 2: Testing Transitivity with Individual Data Michael H. Birnbaum and Jeffrey P. Bahra California State University, Fullerton](https://reader035.vdocuments.mx/reader035/viewer/2022081513/56649d7e5503460f94a61622/html5/thumbnails/2.jpg)
Transitivity of Preference
• If A > B and B > C then A > C.• Weak Stochastic Transitivity: • If P(A, B) > 1/2 and P(B, C) > 1/2
then P(A, C) > 1/2.
![Page 3: Testing Transitivity with Individual Data Michael H. Birnbaum and Jeffrey P. Bahra California State University, Fullerton](https://reader035.vdocuments.mx/reader035/viewer/2022081513/56649d7e5503460f94a61622/html5/thumbnails/3.jpg)
Tversky (1969)
• Tversky (1969) reported that selected subjects showed a pattern of intransitive data consistent with a lexicographic semi-order.
• Tversky tested Weak Stochastic Transitivity.
![Page 4: Testing Transitivity with Individual Data Michael H. Birnbaum and Jeffrey P. Bahra California State University, Fullerton](https://reader035.vdocuments.mx/reader035/viewer/2022081513/56649d7e5503460f94a61622/html5/thumbnails/4.jpg)
Issues
• Iverson & Falmagne (1985) argued that Tversky’s statistical analysis was incorrect of WST.
• Tversky went on to publish transitive theories of preference (e.g., CPT).
![Page 5: Testing Transitivity with Individual Data Michael H. Birnbaum and Jeffrey P. Bahra California State University, Fullerton](https://reader035.vdocuments.mx/reader035/viewer/2022081513/56649d7e5503460f94a61622/html5/thumbnails/5.jpg)
Renewed Interest in Intransitive Preference
• New analytical methods for analysis of transitivity (Iverson, Myung, & Karabatsos; Regenwetter & Stober, et al); Error models (Sopher & Gigliotti, ‘93; Birnbaum, ‘04; others).
• Priority Heuristic (Brandstaetter, et al., 2006); stochastic difference model (González-Vallejo,
2002; similarity judgments, Leland, 1994; majority rule, Zhang, Hsee, Xiao, 2006). Renewed interest in Fishburn, as well as in Regret Theory.
![Page 6: Testing Transitivity with Individual Data Michael H. Birnbaum and Jeffrey P. Bahra California State University, Fullerton](https://reader035.vdocuments.mx/reader035/viewer/2022081513/56649d7e5503460f94a61622/html5/thumbnails/6.jpg)
Lexicographic Semi-order• G = (x, p; y, 1 - p). F = (x’, q; y’, 1 -
q).
• If y - y’ ≥ L choose G
• If y’ - y ≥ L choose F
• If p - q ≥ P choose G
• If q - p ≥ P choose F
• If x > x’ choose G; if x’ > x choose F;• Otherwise, choose randomly.
![Page 7: Testing Transitivity with Individual Data Michael H. Birnbaum and Jeffrey P. Bahra California State University, Fullerton](https://reader035.vdocuments.mx/reader035/viewer/2022081513/56649d7e5503460f94a61622/html5/thumbnails/7.jpg)
Priority Heuristic• 10% of largest prize, rounded to nearest
prominent number(In this study, L = $10).
• Compare gambles by lowest consequences. If difference exceeds the aspiration level, choose by lowest consequence.
• If not, compare probabilities; choose by probability if difference ≥ 0.1 (P = 0.1).
• Compare largest consequences; choose by largest consequences.
![Page 8: Testing Transitivity with Individual Data Michael H. Birnbaum and Jeffrey P. Bahra California State University, Fullerton](https://reader035.vdocuments.mx/reader035/viewer/2022081513/56649d7e5503460f94a61622/html5/thumbnails/8.jpg)
Birnbaum & Gutierez (OBHDP, in press)
• Four studies used Tversky’s 5 gambles, formatted with tickets or with pie charts.
• Failed to find evidence that more than a very small percentage of participants (~ 6%) were intransitive.
• Other tests refuted lexicographic semiorder and priority heuristic.
![Page 9: Testing Transitivity with Individual Data Michael H. Birnbaum and Jeffrey P. Bahra California State University, Fullerton](https://reader035.vdocuments.mx/reader035/viewer/2022081513/56649d7e5503460f94a61622/html5/thumbnails/9.jpg)
Pie Chart Format
![Page 10: Testing Transitivity with Individual Data Michael H. Birnbaum and Jeffrey P. Bahra California State University, Fullerton](https://reader035.vdocuments.mx/reader035/viewer/2022081513/56649d7e5503460f94a61622/html5/thumbnails/10.jpg)
Response to Birnbaum-Gutierrez
• Perhaps the intransitivity only develops in longer studies. Tversky used 20 replications of each choice.
• Perhaps consequences of Tversky’s gambles diminished since 1969 due to inflation. Perhaps those prizes are now too small.
![Page 11: Testing Transitivity with Individual Data Michael H. Birnbaum and Jeffrey P. Bahra California State University, Fullerton](https://reader035.vdocuments.mx/reader035/viewer/2022081513/56649d7e5503460f94a61622/html5/thumbnails/11.jpg)
Birnbaum & Bahra• Collected up to 40 choices/pair per
person. (20 reps). 2 Sessions, 1.5 hrs, 1 week apart.
• Cash prizes up to $100. • 51 participants, of whom 10 to win
the prize of one of their chosen gambles.
• 3 5 x 5 Designs to test transitivity vs. Priority heuristic predictions
![Page 12: Testing Transitivity with Individual Data Michael H. Birnbaum and Jeffrey P. Bahra California State University, Fullerton](https://reader035.vdocuments.mx/reader035/viewer/2022081513/56649d7e5503460f94a61622/html5/thumbnails/12.jpg)
Choice Format
![Page 13: Testing Transitivity with Individual Data Michael H. Birnbaum and Jeffrey P. Bahra California State University, Fullerton](https://reader035.vdocuments.mx/reader035/viewer/2022081513/56649d7e5503460f94a61622/html5/thumbnails/13.jpg)
Notation-Two-branch Gambles
• G = (x, p; y, 1 - p); x > y ≥ 0• L = Lower Consequence• P = Probability to win higher prize• H = Higher consequence
![Page 14: Testing Transitivity with Individual Data Michael H. Birnbaum and Jeffrey P. Bahra California State University, Fullerton](https://reader035.vdocuments.mx/reader035/viewer/2022081513/56649d7e5503460f94a61622/html5/thumbnails/14.jpg)
LH (Lower & Higher Consequences) Design
• A = ($84, .50; $24)• B = ($88, .50; $20)• C = ($92, .50; $16)• D = ($96, .50; $12)• E = ($100, .50; $8)
![Page 15: Testing Transitivity with Individual Data Michael H. Birnbaum and Jeffrey P. Bahra California State University, Fullerton](https://reader035.vdocuments.mx/reader035/viewer/2022081513/56649d7e5503460f94a61622/html5/thumbnails/15.jpg)
LP (Lower & Probability) Design
• F = ($100, .50; $24)• G = ($100, .54; $20)• H = ($100, .58; $16)• I = ($100, .62; $12)• J = ($100, .66; $8)
![Page 16: Testing Transitivity with Individual Data Michael H. Birnbaum and Jeffrey P. Bahra California State University, Fullerton](https://reader035.vdocuments.mx/reader035/viewer/2022081513/56649d7e5503460f94a61622/html5/thumbnails/16.jpg)
PH (Probability & Higher) Design
• K = ($100, .50; $0)• L = ($96, .54; $0)• M = ($92, .58; $0)• N = ($88, .62; $0)• O = ($84, .66; $0)
![Page 17: Testing Transitivity with Individual Data Michael H. Birnbaum and Jeffrey P. Bahra California State University, Fullerton](https://reader035.vdocuments.mx/reader035/viewer/2022081513/56649d7e5503460f94a61622/html5/thumbnails/17.jpg)
Priority Heuristic implies: Intransitive and Consistent
LH design Second GambleFirst A B C D E
A=(84, 24) 2 2 1 1B= (88, 20) 1 2 2 1C= (92, 16) 1 1 2 2D= (96, 12) 2 1 1 2E= (100, 8) 2 2 1 1
![Page 18: Testing Transitivity with Individual Data Michael H. Birnbaum and Jeffrey P. Bahra California State University, Fullerton](https://reader035.vdocuments.mx/reader035/viewer/2022081513/56649d7e5503460f94a61622/html5/thumbnails/18.jpg)
Transitive & Consistent1 = Chose First; 2 = Chose
SecondSecond Gamble
First A B C D EA 1 1 1 1B 2 1 1 1C 2 2 1 1D 2 2 2 1E 2 2 2 2
![Page 19: Testing Transitivity with Individual Data Michael H. Birnbaum and Jeffrey P. Bahra California State University, Fullerton](https://reader035.vdocuments.mx/reader035/viewer/2022081513/56649d7e5503460f94a61622/html5/thumbnails/19.jpg)
Within-Rep Consistency
• Count the number of consistent choices in a replicate of 20 choices (10 x 2).
• If a person always chose the same button, consistency = 0.
• If a person were perfectly consistent within replicate, consistency = 10.
• Randomly choosing between responses produces expected consistency of 5.
![Page 20: Testing Transitivity with Individual Data Michael H. Birnbaum and Jeffrey P. Bahra California State University, Fullerton](https://reader035.vdocuments.mx/reader035/viewer/2022081513/56649d7e5503460f94a61622/html5/thumbnails/20.jpg)
Within-Replicate Consistency
• The average rate of agreement was 8.63 (86% self-agreement).
• 46.4% of all replicates were scored 10.
• An additional 19.9% were scored 9.
![Page 21: Testing Transitivity with Individual Data Michael H. Birnbaum and Jeffrey P. Bahra California State University, Fullerton](https://reader035.vdocuments.mx/reader035/viewer/2022081513/56649d7e5503460f94a61622/html5/thumbnails/21.jpg)
LH Design: Overall Proportions Choosing Second Gamble
Second GambleFirst A B C D E
A= (84, 24) 0.41 0.38 0.34 0.27
B= (88, 20) 0.58 0.40 0.36 0.30
C= (92, 16) 0.61 0.59 0.44 0.32
D= (96, 12) 0.64 0.61 0.55 0.33
E= (100, 8) 0.70 0.69 0.66 0.66
![Page 22: Testing Transitivity with Individual Data Michael H. Birnbaum and Jeffrey P. Bahra California State University, Fullerton](https://reader035.vdocuments.mx/reader035/viewer/2022081513/56649d7e5503460f94a61622/html5/thumbnails/22.jpg)
LP Design: Overall Proportions Choosing Second Gamble
LP Second GambleFirst F G H I J
F 0.44 0.43 0.42 0.36G 0.54 0.42 0.42 0.38H 0.54 0.55 0.45 0.40I 0.56 0.56 0.53 0.41J 0.60 0.59 0.57 0.56
![Page 23: Testing Transitivity with Individual Data Michael H. Birnbaum and Jeffrey P. Bahra California State University, Fullerton](https://reader035.vdocuments.mx/reader035/viewer/2022081513/56649d7e5503460f94a61622/html5/thumbnails/23.jpg)
PH Design: Overall Proportions Choosing Second Gamble
PH Second GambleFirst K L M N O
K 0.61 0.64 0.64 0.64L 0.37 0.61 0.63 0.65M 0.34 0.37 0.64 0.64N 0.34 0.35 0.33 0.63O 0.34 0.33 0.35 0.34
![Page 24: Testing Transitivity with Individual Data Michael H. Birnbaum and Jeffrey P. Bahra California State University, Fullerton](https://reader035.vdocuments.mx/reader035/viewer/2022081513/56649d7e5503460f94a61622/html5/thumbnails/24.jpg)
Averaged Data Fit WST
• LH Design A > B > C > D > E• LP Design F > G > H > I > J• PH Design O > N > M > L > K• Consistent with special TAX with its
“prior” parameters.• This analysis obscures individual
diffs
![Page 25: Testing Transitivity with Individual Data Michael H. Birnbaum and Jeffrey P. Bahra California State University, Fullerton](https://reader035.vdocuments.mx/reader035/viewer/2022081513/56649d7e5503460f94a61622/html5/thumbnails/25.jpg)
Individual Data
• Choice proportions calculated for each individual in each design.
• These were further examined within each person replication by replication.
![Page 26: Testing Transitivity with Individual Data Michael H. Birnbaum and Jeffrey P. Bahra California State University, Fullerton](https://reader035.vdocuments.mx/reader035/viewer/2022081513/56649d7e5503460f94a61622/html5/thumbnails/26.jpg)
S# 8328 C = 9.6 Reps = 20
![Page 27: Testing Transitivity with Individual Data Michael H. Birnbaum and Jeffrey P. Bahra California State University, Fullerton](https://reader035.vdocuments.mx/reader035/viewer/2022081513/56649d7e5503460f94a61622/html5/thumbnails/27.jpg)
S# 8328 C = 9.8 Reps = 20
LP Second GambleF G H I J
F 0.05 0.02 0.00 0.00G 0.00 0.00 0.00H 0.05 0.02I 0.00J
![Page 28: Testing Transitivity with Individual Data Michael H. Birnbaum and Jeffrey P. Bahra California State University, Fullerton](https://reader035.vdocuments.mx/reader035/viewer/2022081513/56649d7e5503460f94a61622/html5/thumbnails/28.jpg)
S# 8328 C = 9.9 Reps = 20
PH Second GambleK L M N O
K 1.00 1.00 1.00 0.98L 1.00 1.00 1.00M 0.95 1.00N 0.95O
![Page 29: Testing Transitivity with Individual Data Michael H. Birnbaum and Jeffrey P. Bahra California State University, Fullerton](https://reader035.vdocuments.mx/reader035/viewer/2022081513/56649d7e5503460f94a61622/html5/thumbnails/29.jpg)
S# 6176 C = 9.8 Rep = 20; started with this pattern, then switched to perfectly consistent
with the opposite pattern for 4 replicates at the end of the first day; back to this pattern for 10
reps on day 2.
PH Second GambleK L M N O
K 0.28 0.20 0.23 0.20L 0.25 0.20 0.20M 0.20 0.20N 0.20O
![Page 30: Testing Transitivity with Individual Data Michael H. Birnbaum and Jeffrey P. Bahra California State University, Fullerton](https://reader035.vdocuments.mx/reader035/viewer/2022081513/56649d7e5503460f94a61622/html5/thumbnails/30.jpg)
Data Summary
• With n = 51, there are 153 matrices. Of these, 87% were perfectly consistent with WST: P(A,B) ≥ 1/2 & P(B,C) ≥ 1/2 then P(A,C) ≥ 1/2.
• 32 people (63%) had all three arrays fitting WST; no one fit priority heuristic nor did anyone have all three intransitive arrays.
• Those arrays that were not perfect fits to WST were either close to perfect, from “noisy” participants, or from people who changed orders.
![Page 31: Testing Transitivity with Individual Data Michael H. Birnbaum and Jeffrey P. Bahra California State University, Fullerton](https://reader035.vdocuments.mx/reader035/viewer/2022081513/56649d7e5503460f94a61622/html5/thumbnails/31.jpg)
Within-Person Changes in Preference Pattern
• Criterion: Person show perfect consistency (10 out of 10) to one pattern in one replication, and perfect consistency to different pattern on another replication.
• 16 Such arrays were found (~10% of 153) involving 12 participants.
• This result is troubling to the assumption that errors arise independently, but consistent with idea that people have changing parameters that drift rather than random changes.
![Page 32: Testing Transitivity with Individual Data Michael H. Birnbaum and Jeffrey P. Bahra California State University, Fullerton](https://reader035.vdocuments.mx/reader035/viewer/2022081513/56649d7e5503460f94a61622/html5/thumbnails/32.jpg)
Summary
• Recent studies fail to confirm systematic violations of transitivity predicted by lexicographic semiorders, including priority heuristic.
• No individual’s data agreed with predictions of priority heuristic. These data add to growing case against this model as a description.
• Individual data are mostly transitive.