lessons from an oops at consumer reports consumer follow experts; ignore invalid information
Post on 22-Feb-2016
18 Views
Preview:
DESCRIPTION
TRANSCRIPT
Uri SimonsohnThe Wharton School 1
The paper in one slide: Jan 4th 2007: Consumer Reports on carseats Jan 18th: Retraction Unique opportunity: Do consumers continue
using Jan 4th info? Test on 6,000+ eBay auctions for carseats Main finding:
Full return to baseline My interpretation: voluntarily ignored info. Alt explanations
Information ‘depreciates’Post-retraction buyers didn’t knowKind-of alternative: Sellers’ behavior
2
Outline Background New information: release and retraction Auction data Main results Alternative specifications Conclusions
3
Can people voluntarily ignore information they possess?
Existing evidence:Debriefing paradigmHindsight biasAnchoringMock juries and inadmissible evidence
4
Debriefing ParadigmRoss, Lepper & Colleageus (JPSP 1975;1980) Critique of false feedback in PsychParadigm: Give false feedback on personality test Debrief: “feedback was false” Ask their beliefs …still influenced by retracted feedback
5
Anchoring Subjects asked to make numerical
estimateLength of Mississippi riverWTP for keyboard.
Asked first: is the amount larger or smaller than anchor.
Final estimate is correlated with anchor. Even when anchor is roulette or SS#
6
OPIM 690 Write down the last 2 digits of your SS#:__ Would you be willing to pay that amount for yearly access to NYTimes.com?
What is the most you would pay? _____
7
$29.84
$45.07
$0.00
$5.00
$10.00
$15.00
$20.00
$25.00
$30.00
$35.00
$40.00
$45.00
$50.00
Social Security # <50 Social Security # >50
Hindsight Bias People told some outcome Asked to estimate what those without
information would predict. Finding: estimates are biased towards
the to-be-ignored outcome.
Next: results from Fischhoff (1975)
8
9
Predicted ex-ante probabilities of subjects who "knew" outcome & actual probabilities of control
group
57%
38%
48%
27%
34%
21%
32%
12%
0%
10%
20%
30%
40%
50%
60%
70%
1 2 3 4
Outcome
Prob
abili
ty
Knew
Control
Mock Juries & inadmissible evidence Dozens of studies Random assignment across “jurors” Control: baseline evidence T1: control + extra evidence T2: T1 + extra evidence is inadmissible. Decisions by T2 fall between control and
T1.
10
11
Outline Background New information: release and retraction Auction data Main results Alternative specifications Conclusions
12
January 4th, 2007
13
•Corr( rank 2007,rank 2005) = -.08
Retraction and empirical strategy Jan 18th : Oops! Outsourced, 30 vs 38 vs 70 MPH Unique opportunity to study:
1) Causal effects of expert adviceContributions:○ Individual level measures of WTP○ Simple identification strategy (wrong info)
Compared to- Discontinuities around discrete scores- Differences across sites- Timing
2) Ability of consumers to ignore retracted information.14
How would people learn of a new Consumer Rerports carseat rating?
Important because: 1) Face validity of quick market
reactions 2) Post-retraction awareness.
15
From CR to consumers. CR in print
Subscribers: slow○ Library got it 01/11○ They claim: letter for retraction○ Otherwise, not till May
Newstands: slower○ No retraction till May
cr.org Comscore 100k users15% of carseat buyers visit within 305% same dayNot a direct source of info
16
How about the media?
17
0
10
20
30
40
50
60
1/1/
2007
1/2/
2007
1/3/
2007
1/4/
2007
1/5/
2007
1/6/
2007
1/7/
2007
1/8/
2007
1/9/
2007
1/10
/200
71/
11/2
007
1/12
/200
71/
13/2
007
1/14
/200
71/
15/2
007
1/16
/200
71/
17/2
007
1/18
/200
71/
19/2
007
1/20
/200
71/
21/2
007
1/22
/200
71/
23/2
007
1/24
/200
71/
25/2
007
1/26
/200
71/
27/2
007
1/28
/200
71/
29/2
007
1/30
/200
7
# of
new
spap
er a
rticl
es
0
100
200
300
400
500
600
700
800
# of
TV/
Radi
o st
orie
s
Newspaper coverage
"Television & Radio Coverage"
Number of stories about “Consumer Reports” and “Carseats”sources: newsbank+lexisnexis
18
50+ Newspapers 600+ Stories
Internet coverage Can’t do same search for web-coverage Can use web.archive.org to check specific
sites. All major sites covered it
19
In Sum
CR info indirectly received via mediaFast Retracted information remained available
following retraction
I’d argue: Post-retraction buyers probably read stories before being retracted.
20
Outline Background New information: release and retraction Auction data Main results Alternative specifications Conclusions
21
Why auctions Retailers don’t change prices often Few decision makers behind them Auctions:
1000s of DMs interactingPrices change continuously
Aside: Unexploited side to eBay data: pulse on
demand shocks.
22
Auctions Data 6 months: 3 before & 3 after
Many analyses focus on:○ Before: 3 weeks ○ During: 2 weeks ○ After: 3 weeks
Auctions: 6k Bids: 35k
23
24
Descriptive statistics
25
Table 1 - Descriptive Statistics by Carseat Model
Britax Companion
Safety 1st Designer
Peg Perego Viaggio
Graco Snug Ride
Baby Trend Adjustable
Back
Evenflo Discovery
Rankings (rating ) in Consumer Reports® In 2005 1st (90) 2nd (88) 3rd (70) 4th (69) 5th (55) 6th (45)
In 2007 [retracted] 7th (20) 9th (16) 4th (27) 2nd (61) 1st (64) 11th (0)
Means (standard deviation ) for key variables
Number of observations 606 243 1327 2682 312 301
Final Price (sold items) $99.5 $33.5 $87.1 $40.5 $56.4 $15.1(28.72) (17.85) (50.06) (27.25) (26.84) (10.03)
Shipping (sold items) $25.3 $21.3 $25.1 $20.3 $21.1 $19.8(8.41) (6.06) (18.11) (10.18) (10.81) (9.77)
Starting Price $64.22 $17.49 $48.14 $26.61 $45.48 $10.59(48.43) (20.76) (44.63) (27.62) (34.22) (9.77)
Percentage Sold 72.1% 62.1% 71.8% 70.9% 62.1% 60.5%
Number of bids (sold items) 13.19 10.48 13.19 9.98 12.11 7.98(8.57) (7.56) (10.21) (7.19) (8.70) (5.68)
Number of (paid) extra features included with listing 1.17 1.06 1.14 1.28 1.10 1.12(0.52) (0.23) (0.84) (0.75) (0.52) (0.74)
Percentage of items known to be new 70.1% 75.3% 33.8% 22.9% 51.6% 25.3%
BrandModel
Annoyance:
Shipping is only observed for sold items. Estimate OLS for sold items
(w/shipping) Estimate Tobit for all (w.o./shipping)
26
Outline or regression specifications Y: (tot.pricei/Avg.Pricei,k)
i:auction, k:carseat model
Time variables (dummies): Primarily: before, during, after. Also: biweekly dummies (next slide) Also: 3-day-dummies
Key predictor Primarily: ΔRanking Also: carseat-model-dummies
e.g. Y=OLS(during*ΔRanking , after* ΔRanking, controls) 27
-4%
-3%
-2%
-1%
0%
1%
2%
-6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6 7
Fortnights (biweeks) since/from new information
Pric
e ch
ange
per
pos
ition
lost
in ra
nkin
g
First: bird’s eye view Estimate Y=OLS(biweekly*ΔRanking)
1 observation every 14 days. Plot point estimates
28
3.98 SD
Next: more fine grained look
Time: before, during, after
29
30
Table 2. Regressions predicting final price of carseat auctionsRegression type OLS OLS OLS Tobit Tobit Tobit Tobit Tobit
Specification Base specification Adds controls forauction-design
Adds controls for competition
Includes unsold items Adds more weeks Full sample Full sample
New itemsFull sampleUsed items
Sample period (in weeks)3 Before 2 During 3 After
3 Before 2 During 3 After
3 Before 2 During 3 After
3 Before 2 During 3 After
5 Before 2 During 5 After
14 Before 2 During 13 After
14 Before 2 During 13 After
14 Before 2 During 13 After
0.159*** 0.145*** 0.150*** 0.180*** 0.159*** 0.169*** 0.238*** 0.128***
(0.034) (0.030) (0.032) (0.033) (0.033) (0.040) (0.083) (0.034)
0.063** 0.081*** 0.092*** 0.096** 0.081* 0.042 0.056 0.042
(0.029) (0.027) (0.031) (0.043) (0.044) (0.048) (0.077) (0.036)
-0.033*** -0.022* -0.020 -0.016* -0.009 -0.001 0.005 -0.003
(0.010) (0.011) (0.012) (0.009) (0.009) (0.009) (0.012) (0.008)
"During" * ΔRanking -0.028** -0.024** -0.026** -0.031*** -0.035*** -0.034*** -0.047*** -0.011(0.011) (0.011) (0.011) (0.010) (0.010) (0.012) (0.017) (0.014)
"After" * ΔRanking -0.001 0.003 0.001 -0.007 -0.008 -0.006 -0.010 -0.009(0.011) (0.011) (0.011) (0.010) (0.010) (0.011) (0.014) (0.012)
0.515*** 0.373*** 0.390*** 0.376*** 0.325*** 0.320***
(0.083) (0.062) (0.063) (0.086) (0.069) (0.045)
-0.048 -0.033 -0.028 -0.067 -0.098** -0.042
(0.048) (0.027) (0.025) (0.054) (0.041) (0.038)
Includes additional base? (1-yes, 0-no) 0.131** 0.142*** 0.139*** 0.061 0.028 0.022 -0.243** 0.031
(0.056) (0.045) (0.043) (0.047) (0.035) (0.026) (0.111) (0.030)
0.491*** 0.491*** 0.366*** 0.388*** 0.391*** 0.450*** 0.572***
(0.076) (0.076) (0.076) (0.075) (0.073) (0.069) (0.031)
0.036*** 0.036*** 0.033*** 0.032** 0.024* 0.005 0.023
(0.010) (0.009) (0.012) (0.013) (0.015) (0.035) (0.015)
Reserve price present (1-yes, 0-no) 0.243*** 0.243*** 0.104*** 0.147*** 0.154*** 0.282*** 0.122***
(0.025) (0.026) (0.037) (0.031) (0.020) (0.075) (0.016)
Offered buy-it-now (1-yes, 0-no) 0.007 0.006 0.126*** 0.101*** 0.099*** -0.010 0.106***
(0.032) (0.032) (0.031) (0.029) (0.021) (0.032) (0.024)
Numer of same-model auctions that day -0.002 -0.003 -0.001 0.003 0.007 -0.000
(0.002) (0.003) (0.004) (0.003) (0.006) (0.002)
-0.058 0.023 0.020 -0.033 -0.163* 0.130**
(0.080) (0.065) (0.069) (0.079) (0.097) (0.057)
Number of observations 1052 1052 1052 1438 2227 5471 2480 3346R2 / Pseudo-R2 .32 .47 .47 .30 .29 .26 .16 .16
Percentage of same-model auctions that day that are new
Auction's starting price (as % of model's average price)
Is item known to be new? (1-yes, 0-no)
Is item known to be used? (1-yes, 0-no)Excluded category: unknown status
"During" (1-yes, 0-no) Does auction end during the two weeks in which new ranking was valid?"After" (1-yes, 0-no) Does auction end after the retraction?
ΔRanking (2007-2005) Difference in Consumer Reports' safety of auctioned carseat model.
Number of (paid) extra features included with listing
Plotting time*dranking betas
31
-6%
-4%
-2%
0%
2%
4%
6%
baseline xb xb+ tobit tobit 12weeks
tobit 6months
new used
During After
``
So far: just before-after How quick are the reactions?
Y=OLS(3-day-dummies* Δranking)
32
price=OLS(3-day dummies * Δrank)omitted cat.: two previous weeks
-5%
-4%
-3%
-2%
-1%
0%
1%
2%(-6
,-4)
(-3,-1
)
(0,2
)
(3,5
)
(6,8
)
(9,1
1)
(12,
14)
(15,
17)
(18,
20)
(21,
23)
(24,
26)
(27,
29)
(30,
32)
(33,
35)
Days after new ranking (in 3-day intervals)
Poin
t est
imat
e fo
r 3-d
ay ti
me-
dum
my
inte
ract
ed
with
rank
ing-
chan
ge
New ranking released
Retracted
33
How about non-winning bids? Camerer et al (1989) “Curse of
Knowledge”Market forces reduce itRational agents trade more
Same here?Are non winning bidders ‘cursed’?
Unit of observation: auction bid Quantile Regression
34
Specification Bids are unit of observation. If more than one bid by same bidder,
take highest only. Estimate quantile regressions of:
bid $ = f(Time*ΔRanking) With quantiles at 20% ,40% ,60% ,80%.
35
36
Table 3. Quantile regressions with bid-amount as dependent variable
20% 40% 60% 80%
0.052*** 0.064*** 0.086*** 0.089**
(0.012) (0.016) (0.016) (0.035)
0.013 0.031* 0.063*** 0.042
(0.012) (0.016) (0.018) (0.033)
-0.001 -0.002 -0.001 -0.008
(0.003) (0.003) (0.006) (0.005)
"During" * ΔRanking -0.007** -0.013*** -0.021*** -0.022***(0.003) (0.004) (0.006) (0.007)
"After" * ΔRanking 0.002 0.004 0.003 0.006(0.004) (0.003) (0.006) (0.007)
Is item known to be new? (1-yes, 0-no) 0.122*** 0.238*** 0.255*** 0.221***
(0.022) (0.022) (0.036) (0.054)
-0.021 -0.018 -0.041 -0.162***
(0.025) (0.020) (0.038) (0.058)
Does auction include additional carseat-base? (1-yes, 0-no) 0.026 0.068* 0.071** 0.125**
(0.022) (0.036) (0.031) (0.053)
Auction's starting price (as % of model's average price) 0.869*** 0.776*** 0.687*** 0.558***
(0.019) (0.023) (0.022) (0.033)
Auction's duration in days -0.003 -0.002 -0.001 0.002
(0.002) (0.003) (0.004) (0.004)
Number of (paid) extra features included with listing 0.011** 0.019*** 0.021** 0.033*
(0.006) (0.007) (0.010) (0.017)
Reserve price present (1-yes, 0-no) 0.089*** 0.098*** 0.124*** 0.160***
(0.021) (0.032) (0.033) (0.043)
Offered buy-it-now (1-yes, 0-no) -0.024* -0.007 0.015 0.067***
(0.014) (0.017) (0.018) (0.023)
log (seller reputation + 2) -0.008*** -0.008*** -0.013*** -0.020***
(0.003) (0.002) (0.004) (0.005)
# of same-model auctions that day 0.000 0.001** 0.001*** 0.002**
(0.000) (0.000) (0.000) (0.001)
% of same-model new that day -0.006 -0.064*** -0.037 -0.034
(0.022) (0.022) (0.048) (0.047)
Number of observations 3529 3529 3529 3529Pseudo R2 .212 .219 .224 .211
Is item known to be used? (1-yes, 0-no)Note. Excluded category: unknown new/used status.
Quantile
"During" (1-yes, 0-no) Does auction end during the two weeks in which new ranking was valid?
"After" (1-yes, 0-no) Does auction end after the retraction?
ΔRanking (2007-2005) Difference in Consumer Reports' safety ranking of auctioned carseat model.
Plotting the betas
37
Point estimates for time*drank
-0.025
-0.02
-0.015
-0.01
-0.005
0
0.005
0.01
20% 40% 60% 80%
Quantil Regression for all Bids at X%
poin
t est
imat
e
During
After
38
Point estimates for time*drank
-0.03
-0.025-0.02
-0.015
-0.01
-0.0050
0.005
0.01
20% 40% 60% 80%
Quantile Regression for all Bids
poin
t est
imat
e
During
After
Dividing point estimates by average bid % at quantile
39
From ΔRanking to model-dummies Previous analyses: Impose
Δ%price=b* ΔRanking Don’t allow for heterogeneity in effect Next: estimates by model. Plot avg(OLS,Tobit)
40
41
-8%-10% -10%
13%
35%
-30%
0% 2%6%
0%
-25%
0%
-40%
-30%
-20%
-10%
0%
10%
20%
30%
40%
% C
hang
e in
pric
es w
ith re
spec
t to
Befo
re p
erio
d
DuringAfter
Britax (1st → 7th)
Safety 1st(2nd → 9th)
Evenflo(6th → 11th)
Baby Trend(5th → 1st)
Graco(4th → 2nd)
Peg Perego(3rd → 4th)
42
Price=f(demand AND supply)
43
44
Starting Price Number of paid features
45
-4%
-2%
0%
2%
4%
-6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6
Biweeks before/after new information
% c
hang
e in
st
artin
g-pr
ice
-0.10
-0.05
0.00
0.05
0.10
Chan
ge in
# o
f pai
d fe
atur
es
Starting Price Paid features
# of items for sale% New
46
-10%
-5%
0%
5%
10%
-6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6
Biweeks before/after new information
% c
hang
e
Number of carseats % new
47
Summary of evidence Biweekly: biggest price drop in 6 months During vs. After:
Market responded to information Ceased to once retracted
3-day: Market respond virtually immediately
Quantile regressions Bidders across the full spectrum do so.
Carseat dummies Every carseat (6/6) exhibits the pattern
Supply: No evidence of changes in supply
48
Interpretation Consumers successfully ignored
information they possessed once it was retracted.
49
Alternative Explanations1) Knowledge depreciates
…& coincides w/retractionBut: 3-day graphs
2) Buyers never knewRetracted information still available online- Evenflo
50
Why cursed in the lab but not here? Field, but not lab: credible instruction to
ignore.Mock juries & substantive instructionsDebriefing paradigm & credible instructionShould you really ignore info in
○ Hindsight Bias○ Knowledge curse
Field, but not lab: DM control informationDilution effect goes away when you can scratch
irrelevant infoHindsight and anchoring attenuate when explicitly
consider alternatives.51
Future research
Run lab experiments explicitly manipulating variables that differ in vs. outside the lab.
52
top related