harnessing the wisdom of crowds - … harnessing the wisdom of crowds zhi da, university of notre...
TRANSCRIPT
![Page 1: HARNESSING THE WISDOM OF CROWDS - … HARNESSING THE WISDOM OF CROWDS Zhi Da, University of Notre Dame Xing Huang, Michigan State University Second Annual News …](https://reader033.vdocuments.mx/reader033/viewer/2022052711/5adfbbdc7f8b9a8f298d4c58/html5/thumbnails/1.jpg)
1
HARNESSING THE WISDOM OF
CROWDS
Z h i D a , U n i v e r s i t y o f N o t r e D a m e
X i n g H u a n g , M i c h i g a n S t a t e U n i v e r s i t y
S e c o n d A n n u a l N e w s & F i n a n c e C o n f e r e n c e
M a r c h 8 , 2 0 1 7
![Page 2: HARNESSING THE WISDOM OF CROWDS - … HARNESSING THE WISDOM OF CROWDS Zhi Da, University of Notre Dame Xing Huang, Michigan State University Second Annual News …](https://reader033.vdocuments.mx/reader033/viewer/2022052711/5adfbbdc7f8b9a8f298d4c58/html5/thumbnails/2.jpg)
2
Many important decisions in l i fe are
made in a group
F O M C m e e t i n gB o a r d m e e t i n g
J u r y
“Who wants to be a millionaire”
A s k t h e a u d i e n c e
![Page 3: HARNESSING THE WISDOM OF CROWDS - … HARNESSING THE WISDOM OF CROWDS Zhi Da, University of Notre Dame Xing Huang, Michigan State University Second Annual News …](https://reader033.vdocuments.mx/reader033/viewer/2022052711/5adfbbdc7f8b9a8f298d4c58/html5/thumbnails/3.jpg)
3
Wisdom of crowdsJ e l l y b e a n s i n t h e j a r e x p e r i m e n t
Ask a group of people to guess:
How many jelly beans in the jar?
A large group’s average answer to a question involving
quantity estimation is generally as good as, and often
better than, the answer provided by any individual in
that group
Law of large number requires independence
94
13599
77
67
97
112
102
59161
68
78
47
126
80
106
95 jelly beans
![Page 4: HARNESSING THE WISDOM OF CROWDS - … HARNESSING THE WISDOM OF CROWDS Zhi Da, University of Notre Dame Xing Huang, Michigan State University Second Annual News …](https://reader033.vdocuments.mx/reader033/viewer/2022052711/5adfbbdc7f8b9a8f298d4c58/html5/thumbnails/4.jpg)
4
CONS: “MEN THINK IN HERDS”
Herding can reduce the accuracy of a group’s average answer
“The more influence we exert on each other, the more likely it is that we will believe the same things and make the same mistakes. That means it’s possible that we could become individually smarter but collectively dumber.”
---James Surowiecki, The Wisdom of Crowds.
Sequential sett ing: Pros vs. Cons
95
![Page 5: HARNESSING THE WISDOM OF CROWDS - … HARNESSING THE WISDOM OF CROWDS Zhi Da, University of Notre Dame Xing Huang, Michigan State University Second Annual News …](https://reader033.vdocuments.mx/reader033/viewer/2022052711/5adfbbdc7f8b9a8f298d4c58/html5/thumbnails/5.jpg)
5“None of us is as dumb as al l of us.”
![Page 6: HARNESSING THE WISDOM OF CROWDS - … HARNESSING THE WISDOM OF CROWDS Zhi Da, University of Notre Dame Xing Huang, Michigan State University Second Annual News …](https://reader033.vdocuments.mx/reader033/viewer/2022052711/5adfbbdc7f8b9a8f298d4c58/html5/thumbnails/6.jpg)
6
CONS: “MEN THINK IN HERDS”
Herding can reduce the accuracy of a group’s average answer
“The more influence we exert on each other, the more likely it is that we will believe the same things and make the same mistakes. That means it’s possible that we could become individually smarter but collectively dumber.”
---James Surowiecki, The Wisdom of Crowds.
Additional information production may improve the accuracy of private signals.
We become both individually smarter and collectively smarter.
PROS:GENERATE ADDITIONAL INFORMATION PRODUCTION
Sequential sett ing: Pros vs. Cons
95
![Page 7: HARNESSING THE WISDOM OF CROWDS - … HARNESSING THE WISDOM OF CROWDS Zhi Da, University of Notre Dame Xing Huang, Michigan State University Second Annual News …](https://reader033.vdocuments.mx/reader033/viewer/2022052711/5adfbbdc7f8b9a8f298d4c58/html5/thumbnails/7.jpg)
7
WE STUDY THE QUESTION:
WE QUANTIFY THE IMPACT OF HERDING ON ECONOMIC OUTCOMES
Do individuals herd in a sequential setting and reduce the usefulness of information aggregated across individuals (i.e., the wisdom of crowds)?
The empirical challenge is that individual’s information set is usually unobservable
We overcome the empirical challenge by directly measuring and randomizing on individual’s information set
In this paper
![Page 8: HARNESSING THE WISDOM OF CROWDS - … HARNESSING THE WISDOM OF CROWDS Zhi Da, University of Notre Dame Xing Huang, Michigan State University Second Annual News …](https://reader033.vdocuments.mx/reader033/viewer/2022052711/5adfbbdc7f8b9a8f298d4c58/html5/thumbnails/8.jpg)
8
WHY IT’S A GOOD SETTING?
• Forecasts and realizations are clearly measured and easily observable• The forecasters do not have direct influence on realizations• Corporate earnings are of crucial importance
Our sett ingA c r o w d - b a s e d e a r n i n g s f o r e c a s t p l a t f o r m
( E s t i m i z e . c o m )
![Page 9: HARNESSING THE WISDOM OF CROWDS - … HARNESSING THE WISDOM OF CROWDS Zhi Da, University of Notre Dame Xing Huang, Michigan State University Second Annual News …](https://reader033.vdocuments.mx/reader033/viewer/2022052711/5adfbbdc7f8b9a8f298d4c58/html5/thumbnails/9.jpg)
9
M A I N A N A LY S I S
• Estimize.com related• Sample statistics
1D AT A A N D S T AT I S T I C S
2
• Herding behavior• The influence of herding on
forecast accuracyo Individual forecasto Consensus forecast
I N F L U E N T I A L U S E R A N D H E R D I N G
• Herding behavior• The influence of herding on
forecast accuracyo Individual forecasto Consensus forecast
3R A N D O M I Z E DE X P E R I M E N T
4
• Are users herding more with “influential” users?
• Does herding lead to return predictability
Roadmap
![Page 10: HARNESSING THE WISDOM OF CROWDS - … HARNESSING THE WISDOM OF CROWDS Zhi Da, University of Notre Dame Xing Huang, Michigan State University Second Annual News …](https://reader033.vdocuments.mx/reader033/viewer/2022052711/5adfbbdc7f8b9a8f298d4c58/html5/thumbnails/10.jpg)
10
![Page 11: HARNESSING THE WISDOM OF CROWDS - … HARNESSING THE WISDOM OF CROWDS Zhi Da, University of Notre Dame Xing Huang, Michigan State University Second Annual News …](https://reader033.vdocuments.mx/reader033/viewer/2022052711/5adfbbdc7f8b9a8f298d4c58/html5/thumbnails/11.jpg)
11
Open web-based platform founded in 2011, where users can make earnings forecasts
Diverse user group: buy-side | sell-side | independent analysts | other working professionals | students
Estimize consensus is more accurate than WS | IBES consensus, also complementary
• Jame et. al. (2016) | Adebambo and Bliss (2015)• Now available on Bloomberg
Various incentives for making forecasts
• Competition, monetary and professional prizes• Reputation, useful track record• Altruism, social preference
Estimize.com
Scoring system [-25, 25]:- Positive if more accurate than WS
consensus- Awarded on an exponential scale,
which encourages independent opinion
![Page 12: HARNESSING THE WISDOM OF CROWDS - … HARNESSING THE WISDOM OF CROWDS Zhi Da, University of Notre Dame Xing Huang, Michigan State University Second Annual News …](https://reader033.vdocuments.mx/reader033/viewer/2022052711/5adfbbdc7f8b9a8f298d4c58/html5/thumbnails/12.jpg)
12
Sample period for :
• Main analysis: 2012/03 - 2015/03• Randomized experiment: Q2 and Q3 in 2015
Ticker-quarter-forecast panel:
• 2147 quarterly earnings • 2516 users covering • 730 stocks: mostly large-growth
An average release (a sequence of forecasts for a ticker-quarter):
• 20 forecasts from 16 users
User activity on Estimize.com is tracked by Mixpanel
• Events: release page, estimate page, submit estimates, etc.• Timestamp, location, device, etc.
Sample statistics
![Page 13: HARNESSING THE WISDOM OF CROWDS - … HARNESSING THE WISDOM OF CROWDS Zhi Da, University of Notre Dame Xing Huang, Michigan State University Second Annual News …](https://reader033.vdocuments.mx/reader033/viewer/2022052711/5adfbbdc7f8b9a8f298d4c58/html5/thumbnails/13.jpg)
13Release page view
Viewing activity = 1: If a user spent more than 5 seconds on the release page before making her own forecast
![Page 14: HARNESSING THE WISDOM OF CROWDS - … HARNESSING THE WISDOM OF CROWDS Zhi Da, University of Notre Dame Xing Huang, Michigan State University Second Annual News …](https://reader033.vdocuments.mx/reader033/viewer/2022052711/5adfbbdc7f8b9a8f298d4c58/html5/thumbnails/14.jpg)
14
M A I N A N A LY S I S
• Estimize.com related• Sample statistics
1D AT A A N D S T AT I S T I C S
2
• Herding behavior• The influence of herding on
forecast accuracyo Individual forecasto Consensus forecast
I N F L U E N T I A L U S E R A N D H E R D I N G
• Herding behavior• The influence of herding on
forecast accuracyo Individual forecasto Consensus forecast
3R A N D O M I Z E DE X P E R I M E N T
4
• Are users herding more with “influential” users?
• Does herding lead to return predictability
Roadmap
![Page 15: HARNESSING THE WISDOM OF CROWDS - … HARNESSING THE WISDOM OF CROWDS Zhi Da, University of Notre Dame Xing Huang, Michigan State University Second Annual News …](https://reader033.vdocuments.mx/reader033/viewer/2022052711/5adfbbdc7f8b9a8f298d4c58/html5/thumbnails/15.jpg)
15
Note: Fixed effects subsume the need to control for stock or user characteristicsClustered standard errors account for autocorrelations in forecast errors
Herding behavior
Information weighting regression, Chen and Jiang (2006, RFS)
FE 0Dev 0 0 : overweight on public information0 0 : overweight on private information
I n d i v i d u a l s h e r d m o r e w h e n t h e y v i e w o t h e r s ’ f o r e c a s t s
More weight on the consensus forecast after viewing the release page
![Page 16: HARNESSING THE WISDOM OF CROWDS - … HARNESSING THE WISDOM OF CROWDS Zhi Da, University of Notre Dame Xing Huang, Michigan State University Second Annual News …](https://reader033.vdocuments.mx/reader033/viewer/2022052711/5adfbbdc7f8b9a8f298d4c58/html5/thumbnails/16.jpg)
16
Influence of herding on forecast accuracy• Herding makes individual forecast more accurate • but reduces the accuracy of the consensus forecast
![Page 17: HARNESSING THE WISDOM OF CROWDS - … HARNESSING THE WISDOM OF CROWDS Zhi Da, University of Notre Dame Xing Huang, Michigan State University Second Annual News …](https://reader033.vdocuments.mx/reader033/viewer/2022052711/5adfbbdc7f8b9a8f298d4c58/html5/thumbnails/17.jpg)
17
Note: Nonzero Views = 1: if a user spent more than 5 seconds on the release page before making her own forecast
CTA dummy = 1: if the forecast is submitted during the last three days before announcements
Individual’s absolute forecast error decreases after viewing release page
Influence of herding on forecast accuracyH e r d i n g m a k e s i n d i v i d u a l f o r e c a s t m o r e a c c u r a t e
![Page 18: HARNESSING THE WISDOM OF CROWDS - … HARNESSING THE WISDOM OF CROWDS Zhi Da, University of Notre Dame Xing Huang, Michigan State University Second Annual News …](https://reader033.vdocuments.mx/reader033/viewer/2022052711/5adfbbdc7f8b9a8f298d4c58/html5/thumbnails/18.jpg)
18
Note: LnNumView = ln (1+ the percentage of forecasts with release views)Std Dev(FE)/Abs(Median(FE)) controls for uncertainty
Consensus’s absolute forecast error increases as there are more viewing activities within a release
Magnitude: 0.0551 X ln(1+1) = 3.82 centsThis is more than the distance between the perfect forecast and the forecast with median Abs(FE) (3 cents)
The consensus of group without viewing activity wins more than 50% with statistical significance
Influence of herding on forecast accuracyH e r d i n g m a k e s c o n s e n s u s f o r e c a s t l e s s a c c u r a t e
![Page 19: HARNESSING THE WISDOM OF CROWDS - … HARNESSING THE WISDOM OF CROWDS Zhi Da, University of Notre Dame Xing Huang, Michigan State University Second Annual News …](https://reader033.vdocuments.mx/reader033/viewer/2022052711/5adfbbdc7f8b9a8f298d4c58/html5/thumbnails/19.jpg)
19
Note: Consistent bias indicator = 1: if the bias in both early period and CTA period are in the same direction.
LnNumView = ln (1+ the percentage of forecasts with release views)
More viewing activity in the close-to-announcement period, the biases in these two periods are more likely to be consistent.
Influence of herding on bias persistence
Early period Close-to-announcement period
![Page 20: HARNESSING THE WISDOM OF CROWDS - … HARNESSING THE WISDOM OF CROWDS Zhi Da, University of Notre Dame Xing Huang, Michigan State University Second Annual News …](https://reader033.vdocuments.mx/reader033/viewer/2022052711/5adfbbdc7f8b9a8f298d4c58/html5/thumbnails/20.jpg)
20
VIEWING ACTIVITY MAY BE ENDOGENOUS
• Less informed users are more likely to view others’ forecasts.• Consistent with less accurate consensus.• But inconsistent with more accurate individual forecast.
ADDRESS THIS CONCERN:
• randomizing users’ information sets
Endogeneity concern
![Page 21: HARNESSING THE WISDOM OF CROWDS - … HARNESSING THE WISDOM OF CROWDS Zhi Da, University of Notre Dame Xing Huang, Michigan State University Second Annual News …](https://reader033.vdocuments.mx/reader033/viewer/2022052711/5adfbbdc7f8b9a8f298d4c58/html5/thumbnails/21.jpg)
21
M A I N A N A LY S I S
• Estimize.com related• Sample statistics
1D AT A A N D S T AT I S T I C S
2
• Herding behavior• The influence of herding on
forecast accuracyo Individual forecasto Consensus forecast
I N F L U E N T I A L U S E R A N D H E R D I N G
• Herding behavior• The influence of herding on
forecast accuracyo Individual forecasto Consensus forecast
3R A N D O M I Z E DE X P E R I M E N T
4
• Are users herding more with “influential” users?
• Does herding lead to return predictability
Roadmap
![Page 22: HARNESSING THE WISDOM OF CROWDS - … HARNESSING THE WISDOM OF CROWDS Zhi Da, University of Notre Dame Xing Huang, Michigan State University Second Annual News …](https://reader033.vdocuments.mx/reader033/viewer/2022052711/5adfbbdc7f8b9a8f298d4c58/html5/thumbnails/22.jpg)
22
RANDOMIZED EXPERIMENTS
• Pilot round (Q2 in 2015): 13 stocks are randomly selected• Second round (Q3 in 2015): 90 stocks are randomly selected
WHAT WE DO
• Randomly select users and disable the release page • Ask them to make earnings forecasts (blind forecasts)• Afterwards, the release page is restored • They can immediately revise their forecasts (revised forecasts)• Others not selected still view the original release page and make forecasts (default forecasts)
Bl ind experiments
![Page 23: HARNESSING THE WISDOM OF CROWDS - … HARNESSING THE WISDOM OF CROWDS Zhi Da, University of Notre Dame Xing Huang, Michigan State University Second Annual News …](https://reader033.vdocuments.mx/reader033/viewer/2022052711/5adfbbdc7f8b9a8f298d4c58/html5/thumbnails/23.jpg)
23Blind view
![Page 24: HARNESSING THE WISDOM OF CROWDS - … HARNESSING THE WISDOM OF CROWDS Zhi Da, University of Notre Dame Xing Huang, Michigan State University Second Annual News …](https://reader033.vdocuments.mx/reader033/viewer/2022052711/5adfbbdc7f8b9a8f298d4c58/html5/thumbnails/24.jpg)
24
Note: Estimate view, Avg #releases, #tickers, and abs(FE) are based on users’ forecasts before the experiment
Users in blind and default group are similar in observable user characteristics.
Blind vs. DefaultU s e r c h a r a c t e r i s t i c s
![Page 25: HARNESSING THE WISDOM OF CROWDS - … HARNESSING THE WISDOM OF CROWDS Zhi Da, University of Notre Dame Xing Huang, Michigan State University Second Annual News …](https://reader033.vdocuments.mx/reader033/viewer/2022052711/5adfbbdc7f8b9a8f298d4c58/html5/thumbnails/25.jpg)
25
HERDING BEHAVIOR
Default forecasts put more weight on Estimizeconsensus relative to blind forecasts
Blind vs. DefaultH e r d i n g a n d c o n s e n s u s a c c u r a c y
CONSENSUS ACCURACY (HORSE RACE)
• Blind consensus significantly beats default consensus 56.3% of the time• Default consensus significantly beats blind consensus 36.9% of the time
Placebo test with concurrent WS consensus
![Page 26: HARNESSING THE WISDOM OF CROWDS - … HARNESSING THE WISDOM OF CROWDS Zhi Da, University of Notre Dame Xing Huang, Michigan State University Second Annual News …](https://reader033.vdocuments.mx/reader033/viewer/2022052711/5adfbbdc7f8b9a8f298d4c58/html5/thumbnails/26.jpg)
26
Note: This test only includes releases in the pilot round, because only in the pilot round, most users revise their forecasts immediately after the release page is restored.
The blind consensus is much more accurate than the revised consensus
Blind vs. RevisedC o n s e n s u s f o r e c a s t a c c u r a c y
Information weighting regression
![Page 27: HARNESSING THE WISDOM OF CROWDS - … HARNESSING THE WISDOM OF CROWDS Zhi Da, University of Notre Dame Xing Huang, Michigan State University Second Annual News …](https://reader033.vdocuments.mx/reader033/viewer/2022052711/5adfbbdc7f8b9a8f298d4c58/html5/thumbnails/27.jpg)
27
M A I N A N A LY S I S
• Estimize.com related• Sample statistics
1D AT A A N D S T AT I S T I C S
2
• Herding behavior• The influence of herding on
forecast accuracyo Individual forecasto Consensus forecast
I N F L U E N T I A L U S E R A N D H E R D I N G
• Herding behavior• The influence of herding on
forecast accuracyo Individual forecasto Consensus forecast
3R A N D O M I Z E DE X P E R I M E N T
4
• Are users herding more with “influential” users?
• Does herding lead to return predictability
Roadmap
![Page 28: HARNESSING THE WISDOM OF CROWDS - … HARNESSING THE WISDOM OF CROWDS Zhi Da, University of Notre Dame Xing Huang, Michigan State University Second Annual News …](https://reader033.vdocuments.mx/reader033/viewer/2022052711/5adfbbdc7f8b9a8f298d4c58/html5/thumbnails/28.jpg)
28
Note: page rank is also how Google ranks websites.
Measuring user influenceP a g e r a n k a l g o r i t h m b a s e d o n v i e w i n g a c t i v i t i e s
D views A’s estimate
B and C are influential users in this network
• Viewed by many users in general• Viewed by another influential user
Factors behind user influence:
![Page 29: HARNESSING THE WISDOM OF CROWDS - … HARNESSING THE WISDOM OF CROWDS Zhi Da, University of Notre Dame Xing Huang, Michigan State University Second Annual News …](https://reader033.vdocuments.mx/reader033/viewer/2022052711/5adfbbdc7f8b9a8f298d4c58/html5/thumbnails/29.jpg)
29
Note: Influenced = 1 if the number of influential users ahead of the observed user is above the 80th percentile across all observations
Individuals put more weight on the consensus if it contains the forecasts of influential users
Influential user and herdingI n f o r m a t i o n w e i g h t i n g r e g r e s s i o n
Robustness check using data from the experiment
![Page 30: HARNESSING THE WISDOM OF CROWDS - … HARNESSING THE WISDOM OF CROWDS Zhi Da, University of Notre Dame Xing Huang, Michigan State University Second Annual News …](https://reader033.vdocuments.mx/reader033/viewer/2022052711/5adfbbdc7f8b9a8f298d4c58/html5/thumbnails/30.jpg)
30
Predict ing forecast errors of inf luent ial users
SENTIMENT-BASED MEASURE
• Influential users’ revision: upward | downward • Contemporaneous firm returns: positive | negative
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30
![Page 31: HARNESSING THE WISDOM OF CROWDS - … HARNESSING THE WISDOM OF CROWDS Zhi Da, University of Notre Dame Xing Huang, Michigan State University Second Annual News …](https://reader033.vdocuments.mx/reader033/viewer/2022052711/5adfbbdc7f8b9a8f298d4c58/html5/thumbnails/31.jpg)
31
Market does not seem to undo the optimism bias completely and is negatively surprised on average at the earnings announcement
Predicting earnings announcement returns
![Page 32: HARNESSING THE WISDOM OF CROWDS - … HARNESSING THE WISDOM OF CROWDS Zhi Da, University of Notre Dame Xing Huang, Michigan State University Second Annual News …](https://reader033.vdocuments.mx/reader033/viewer/2022052711/5adfbbdc7f8b9a8f298d4c58/html5/thumbnails/32.jpg)
32
Herding improves the accuracy of individual forecast but reduces the accuracy of consensus forecast
Wisdom of crowds can be better harnessed by encouraging independent voice
Conclusions
Aftermath:
Our experiment results convinced Estimize.com to
switch to a “blind” platform in November 2015