university of fribourg, switzerlandmedo/physics/talks/info_filtering.pdf · matúš medo university...
TRANSCRIPT
![Page 1: University of Fribourg, Switzerlandmedo/physics/talks/info_filtering.pdf · Matúš Medo University of Fribourg, Switzerland ICT Applications to Non-Equilibrium Social Sciences 11-12](https://reader034.vdocuments.mx/reader034/viewer/2022050411/5f880cc49a246a4f306d09e1/html5/thumbnails/1.jpg)
Complexity insights into information filtering
Matúš Medo
University of Fribourg, Switzerland
ICT Applications to Non-Equilibrium Social Sciences11-12 June, 2013, Lisbon
Matúš Medo (Uni Fribourg) Complexity insights into information filtering 1 / 25
![Page 2: University of Fribourg, Switzerlandmedo/physics/talks/info_filtering.pdf · Matúš Medo University of Fribourg, Switzerland ICT Applications to Non-Equilibrium Social Sciences 11-12](https://reader034.vdocuments.mx/reader034/viewer/2022050411/5f880cc49a246a4f306d09e1/html5/thumbnails/2.jpg)
Outline
1 Growth of information networks
2 Physics-motivated approach to recommendation
3 Crowd-avoidance in recommendation
Matúš Medo (Uni Fribourg) Complexity insights into information filtering 2 / 25
![Page 3: University of Fribourg, Switzerlandmedo/physics/talks/info_filtering.pdf · Matúš Medo University of Fribourg, Switzerland ICT Applications to Non-Equilibrium Social Sciences 11-12](https://reader034.vdocuments.mx/reader034/viewer/2022050411/5f880cc49a246a4f306d09e1/html5/thumbnails/3.jpg)
Part 1
Growth of information networks
Matúš Medo (Uni Fribourg) Complexity insights into information filtering 3 / 25
![Page 4: University of Fribourg, Switzerlandmedo/physics/talks/info_filtering.pdf · Matúš Medo University of Fribourg, Switzerland ICT Applications to Non-Equilibrium Social Sciences 11-12](https://reader034.vdocuments.mx/reader034/viewer/2022050411/5f880cc49a246a4f306d09e1/html5/thumbnails/4.jpg)
Preferential attachment
A classical network model
Yule (1925), Simon (1955), Price (1976), Barabási & Albert (1999)
Growth of cities, citations of scientific papers, WWW,. . .
Nodes and links are added with time
Probability that a node acquires a new linkproportional to its current degree
P(i , t) ∼ ki(t)
1
2
3
4
Pros: simple, produces a power-law degree distribution
Matúš Medo (Uni Fribourg) Complexity insights into information filtering 4 / 25
![Page 5: University of Fribourg, Switzerlandmedo/physics/talks/info_filtering.pdf · Matúš Medo University of Fribourg, Switzerland ICT Applications to Non-Equilibrium Social Sciences 11-12](https://reader034.vdocuments.mx/reader034/viewer/2022050411/5f880cc49a246a4f306d09e1/html5/thumbnails/5.jpg)
Preferential attachment
A classical network model
Yule (1925), Simon (1955), Price (1976), Barabási & Albert (1999)
Growth of cities, citations of scientific papers, WWW,. . .
Nodes and links are added with time
Probability that a node acquires a new linkproportional to its current degree
P(i , t) ∼ ki(t)
1
2
3
4
Pros: simple, produces a power-law degree distribution
Matúš Medo (Uni Fribourg) Complexity insights into information filtering 4 / 25
![Page 6: University of Fribourg, Switzerlandmedo/physics/talks/info_filtering.pdf · Matúš Medo University of Fribourg, Switzerland ICT Applications to Non-Equilibrium Social Sciences 11-12](https://reader034.vdocuments.mx/reader034/viewer/2022050411/5f880cc49a246a4f306d09e1/html5/thumbnails/6.jpg)
Preferential attachment
A classical network model
Yule (1925), Simon (1955), Price (1976), Barabási & Albert (1999)
Growth of cities, citations of scientific papers, WWW,. . .
Nodes and links are added with time
Probability that a node acquires a new linkproportional to its current degree
P(i , t) ∼ ki(t)
1
2
3
4
Pros: simple, produces a power-law degree distribution
Matúš Medo (Uni Fribourg) Complexity insights into information filtering 4 / 25
![Page 7: University of Fribourg, Switzerlandmedo/physics/talks/info_filtering.pdf · Matúš Medo University of Fribourg, Switzerland ICT Applications to Non-Equilibrium Social Sciences 11-12](https://reader034.vdocuments.mx/reader034/viewer/2022050411/5f880cc49a246a4f306d09e1/html5/thumbnails/7.jpg)
PA in scientific citation data
Journals of the American Physical Society from 1893 to 2009:
0 50 100 150 200 250 300degree k
0
1
2
3
4
5d
egre
e in
crem
ent
∆k
empirical datapreferential attachment
See also Adamic & Huberman (2000), Redner (2005), Newman (2009),. . .
0 50 100 150 200 250 300degree k
0
1
2
3
4
5
deg
ree
incr
emen
t ∆
k
1 year
10 years
30 years
Matúš Medo (Uni Fribourg) Complexity insights into information filtering 5 / 25
![Page 8: University of Fribourg, Switzerlandmedo/physics/talks/info_filtering.pdf · Matúš Medo University of Fribourg, Switzerland ICT Applications to Non-Equilibrium Social Sciences 11-12](https://reader034.vdocuments.mx/reader034/viewer/2022050411/5f880cc49a246a4f306d09e1/html5/thumbnails/8.jpg)
PA in scientific citation data
Journals of the American Physical Society from 1893 to 2009:
0 50 100 150 200 250 300degree k
0
1
2
3
4
5
deg
ree
incr
emen
t ∆
k
empirical datapreferential attachment
See also Adamic & Huberman (2000), Redner (2005), Newman (2009),. . .
0 50 100 150 200 250 300degree k
0
1
2
3
4
5d
egre
e in
crem
ent
∆k
1 year
10 years
30 years
Matúš Medo (Uni Fribourg) Complexity insights into information filtering 5 / 25
![Page 9: University of Fribourg, Switzerlandmedo/physics/talks/info_filtering.pdf · Matúš Medo University of Fribourg, Switzerland ICT Applications to Non-Equilibrium Social Sciences 11-12](https://reader034.vdocuments.mx/reader034/viewer/2022050411/5f880cc49a246a4f306d09e1/html5/thumbnails/9.jpg)
Time decay is fundamental
Matúš Medo (Uni Fribourg) Complexity insights into information filtering 6 / 25
![Page 10: University of Fribourg, Switzerlandmedo/physics/talks/info_filtering.pdf · Matúš Medo University of Fribourg, Switzerland ICT Applications to Non-Equilibrium Social Sciences 11-12](https://reader034.vdocuments.mx/reader034/viewer/2022050411/5f880cc49a246a4f306d09e1/html5/thumbnails/10.jpg)
The model (PRL 107, 238701, 2011)
Probability that node i attracts a new link
P(i , t) ∼ ki(t)︸︷︷︸degree
× Ri(t)︸ ︷︷ ︸relevance
Relevance of every node decays with timeWhen Ri(t)→ 0, the popularity of nodes eventually saturates
Good news:Produces various realistic degree distributions (power-law, etc.)
Matúš Medo (Uni Fribourg) Complexity insights into information filtering 7 / 25
![Page 11: University of Fribourg, Switzerlandmedo/physics/talks/info_filtering.pdf · Matúš Medo University of Fribourg, Switzerland ICT Applications to Non-Equilibrium Social Sciences 11-12](https://reader034.vdocuments.mx/reader034/viewer/2022050411/5f880cc49a246a4f306d09e1/html5/thumbnails/11.jpg)
The model (PRL 107, 238701, 2011)
Probability that node i attracts a new link
P(i , t) ∼ ki(t)︸︷︷︸degree
× Ri(t)︸ ︷︷ ︸relevance
Relevance of every node decays with timeWhen Ri(t)→ 0, the popularity of nodes eventually saturates
Good news:Produces various realistic degree distributions (power-law, etc.)
Matúš Medo (Uni Fribourg) Complexity insights into information filtering 7 / 25
![Page 12: University of Fribourg, Switzerlandmedo/physics/talks/info_filtering.pdf · Matúš Medo University of Fribourg, Switzerland ICT Applications to Non-Equilibrium Social Sciences 11-12](https://reader034.vdocuments.mx/reader034/viewer/2022050411/5f880cc49a246a4f306d09e1/html5/thumbnails/12.jpg)
Datasets
1 Citations among papers published by the APS
2 Citations among the US patents
3 User collections of web bookmarks
4 Paper downloads from the Econophysics Forum
data description label nodes links span/resolution ∆t
APS citations APS 450k 4.7M 117 years/daily 91 daysU.S. patents PAT 3.2M 24M 31 years/yearly 1 year
web bookmarks WEB 2.3M 4.2M 4 years/daily 10 dayspaper downloads EF 600 16k 23 months/daily 10 days
Matúš Medo (Uni Fribourg) Complexity insights into information filtering 8 / 25
![Page 13: University of Fribourg, Switzerlandmedo/physics/talks/info_filtering.pdf · Matúš Medo University of Fribourg, Switzerland ICT Applications to Non-Equilibrium Social Sciences 11-12](https://reader034.vdocuments.mx/reader034/viewer/2022050411/5f880cc49a246a4f306d09e1/html5/thumbnails/13.jpg)
Decay of relevance
0 10 20 30 40 50 6010-2
10-1
100
101
102
0 5 10 15 20 25 3010-1
100
101
0 100 200 300 400 500 60010-2
10-1
100
101
102
103
0 300 600 900 120010-1
100
101
102
APS
EF
PAT
DEL
aver
age
empir
ical
rel
evan
ce X
(t)
age t in years (top) or days (bottom)
Matúš Medo (Uni Fribourg) Complexity insights into information filtering 9 / 25
![Page 14: University of Fribourg, Switzerlandmedo/physics/talks/info_filtering.pdf · Matúš Medo University of Fribourg, Switzerland ICT Applications to Non-Equilibrium Social Sciences 11-12](https://reader034.vdocuments.mx/reader034/viewer/2022050411/5f880cc49a246a4f306d09e1/html5/thumbnails/14.jpg)
Future challenges
Other properties of the model networks:clustering, community structure,. . .
Improved statistical validationTroubles with high-dimensional statisticsEconophysics Forum data: new model is much more likely(10410-times) than the second-best model (Eom-Fortunato, 2011)
Knowledge of the dynamics can help select nodesthat are most relevant now =⇒ useful tools?
Theory says: Total relevance matters =⇒ useful tools?
Matúš Medo (Uni Fribourg) Complexity insights into information filtering 10 / 25
![Page 15: University of Fribourg, Switzerlandmedo/physics/talks/info_filtering.pdf · Matúš Medo University of Fribourg, Switzerland ICT Applications to Non-Equilibrium Social Sciences 11-12](https://reader034.vdocuments.mx/reader034/viewer/2022050411/5f880cc49a246a4f306d09e1/html5/thumbnails/15.jpg)
Part 2Physics-motivated approach to recommendation
Matúš Medo (Uni Fribourg) Complexity insights into information filtering 11 / 25
![Page 16: University of Fribourg, Switzerlandmedo/physics/talks/info_filtering.pdf · Matúš Medo University of Fribourg, Switzerland ICT Applications to Non-Equilibrium Social Sciences 11-12](https://reader034.vdocuments.mx/reader034/viewer/2022050411/5f880cc49a246a4f306d09e1/html5/thumbnails/16.jpg)
Recommendation challenge
There is often too much information availableToo many books to read, movies to watch, . . .How to choose?
Recommender systems analyze data on past user preferences topredict possible future likes and interests
Many approaches exist:Collaborative filteringContent-based analysisLatent semantic modelsSpectral methods. . .
Matúš Medo (Uni Fribourg) Complexity insights into information filtering 12 / 25
![Page 17: University of Fribourg, Switzerlandmedo/physics/talks/info_filtering.pdf · Matúš Medo University of Fribourg, Switzerland ICT Applications to Non-Equilibrium Social Sciences 11-12](https://reader034.vdocuments.mx/reader034/viewer/2022050411/5f880cc49a246a4f306d09e1/html5/thumbnails/17.jpg)
Recommendation challenge
There is often too much information availableToo many books to read, movies to watch, . . .How to choose?
Recommender systems analyze data on past user preferences topredict possible future likes and interests
Many approaches exist:Collaborative filteringContent-based analysisLatent semantic modelsSpectral methods. . .
Matúš Medo (Uni Fribourg) Complexity insights into information filtering 12 / 25
![Page 18: University of Fribourg, Switzerlandmedo/physics/talks/info_filtering.pdf · Matúš Medo University of Fribourg, Switzerland ICT Applications to Non-Equilibrium Social Sciences 11-12](https://reader034.vdocuments.mx/reader034/viewer/2022050411/5f880cc49a246a4f306d09e1/html5/thumbnails/18.jpg)
Recommendation by random walk
Two-step random walk:
0
1
0
0
1user 1
user 2
user 3
user 4
START
−→
0
1/3
5/6
5/6
STEP 1
−→
0
5/8
3/8
5/24
19/24STEP 2
Matúš Medo (Uni Fribourg) Complexity insights into information filtering 13 / 25
![Page 19: University of Fribourg, Switzerlandmedo/physics/talks/info_filtering.pdf · Matúš Medo University of Fribourg, Switzerland ICT Applications to Non-Equilibrium Social Sciences 11-12](https://reader034.vdocuments.mx/reader034/viewer/2022050411/5f880cc49a246a4f306d09e1/html5/thumbnails/19.jpg)
Recommendation by random walk
Two-step random walk:
0
1
0
0
1user 1
user 2
user 3
user 4
START
−→
0
1/3
5/6
5/6
STEP 1
−→
0
5/8
3/8
5/24
19/24STEP 2
Matúš Medo (Uni Fribourg) Complexity insights into information filtering 13 / 25
![Page 20: University of Fribourg, Switzerlandmedo/physics/talks/info_filtering.pdf · Matúš Medo University of Fribourg, Switzerland ICT Applications to Non-Equilibrium Social Sciences 11-12](https://reader034.vdocuments.mx/reader034/viewer/2022050411/5f880cc49a246a4f306d09e1/html5/thumbnails/20.jpg)
Recommendation by random walk
Two-step random walk:
0
1
0
0
1user 1
user 2
user 3
user 4
START
−→
0
1/3
5/6
5/6
STEP 1
−→
0
5/8
3/8
5/24
19/24STEP 2
Matúš Medo (Uni Fribourg) Complexity insights into information filtering 13 / 25
![Page 21: University of Fribourg, Switzerlandmedo/physics/talks/info_filtering.pdf · Matúš Medo University of Fribourg, Switzerland ICT Applications to Non-Equilibrium Social Sciences 11-12](https://reader034.vdocuments.mx/reader034/viewer/2022050411/5f880cc49a246a4f306d09e1/html5/thumbnails/21.jpg)
Key insights
Random walk favors high-degree nodesThey have more ways to receive resources
By contrast, heat diffusion favors low-degree nodesIf you touch many places, your temperature is likely to be averageIf you touch a few places, you may get “lucky” and end hot
Interestingly, the two processes are matematically closely relatedTheir matrices are transpose of each other: M and MT
Matúš Medo (Uni Fribourg) Complexity insights into information filtering 14 / 25
![Page 22: University of Fribourg, Switzerlandmedo/physics/talks/info_filtering.pdf · Matúš Medo University of Fribourg, Switzerland ICT Applications to Non-Equilibrium Social Sciences 11-12](https://reader034.vdocuments.mx/reader034/viewer/2022050411/5f880cc49a246a4f306d09e1/html5/thumbnails/22.jpg)
Key insights
Random walk favors high-degree nodesThey have more ways to receive resources
By contrast, heat diffusion favors low-degree nodesIf you touch many places, your temperature is likely to be averageIf you touch a few places, you may get “lucky” and end hot
Interestingly, the two processes are matematically closely relatedTheir matrices are transpose of each other: M and MT
Matúš Medo (Uni Fribourg) Complexity insights into information filtering 14 / 25
![Page 23: University of Fribourg, Switzerlandmedo/physics/talks/info_filtering.pdf · Matúš Medo University of Fribourg, Switzerland ICT Applications to Non-Equilibrium Social Sciences 11-12](https://reader034.vdocuments.mx/reader034/viewer/2022050411/5f880cc49a246a4f306d09e1/html5/thumbnails/23.jpg)
Hybridization (PNAS 107, 4511, 2010)
HEATDIFF
MT
RANDWALK
M
H(λ)
0
25
50
75
accu
racy
λ
0.6
0.7
0.8
dive
rsity
RANDWALK
HEATDIFF
Result: simultaneous improvement of accuracy and diversity
Matúš Medo (Uni Fribourg) Complexity insights into information filtering 15 / 25
![Page 24: University of Fribourg, Switzerlandmedo/physics/talks/info_filtering.pdf · Matúš Medo University of Fribourg, Switzerland ICT Applications to Non-Equilibrium Social Sciences 11-12](https://reader034.vdocuments.mx/reader034/viewer/2022050411/5f880cc49a246a4f306d09e1/html5/thumbnails/24.jpg)
Hybridization (PNAS 107, 4511, 2010)
HEATDIFF
MT
RANDWALK
M
H(λ)
0
25
50
75
accu
racy
λ
0.6
0.7
0.8
dive
rsity
RANDWALK
HEATDIFF
Result: simultaneous improvement of accuracy and diversity
Matúš Medo (Uni Fribourg) Complexity insights into information filtering 15 / 25
![Page 25: University of Fribourg, Switzerlandmedo/physics/talks/info_filtering.pdf · Matúš Medo University of Fribourg, Switzerland ICT Applications to Non-Equilibrium Social Sciences 11-12](https://reader034.vdocuments.mx/reader034/viewer/2022050411/5f880cc49a246a4f306d09e1/html5/thumbnails/25.jpg)
Part 3
Crowd-avoidance in recommendation
Matúš Medo (Uni Fribourg) Complexity insights into information filtering 16 / 25
![Page 26: University of Fribourg, Switzerlandmedo/physics/talks/info_filtering.pdf · Matúš Medo University of Fribourg, Switzerland ICT Applications to Non-Equilibrium Social Sciences 11-12](https://reader034.vdocuments.mx/reader034/viewer/2022050411/5f880cc49a246a4f306d09e1/html5/thumbnails/26.jpg)
The challenge
Traditional recommender systems do not care to how many usersan item gets recommended
Since they often have bias toward popularity, a small number ofwinners often emerges
This may be harmful:Bar recommended to many people becomes overcrowdedIn the long term, our information horizons shrink
bosons fermions
Matúš Medo (Uni Fribourg) Complexity insights into information filtering 17 / 25
![Page 27: University of Fribourg, Switzerlandmedo/physics/talks/info_filtering.pdf · Matúš Medo University of Fribourg, Switzerland ICT Applications to Non-Equilibrium Social Sciences 11-12](https://reader034.vdocuments.mx/reader034/viewer/2022050411/5f880cc49a246a4f306d09e1/html5/thumbnails/27.jpg)
The challenge
Traditional recommender systems do not care to how many usersan item gets recommended
Since they often have bias toward popularity, a small number ofwinners often emerges
This may be harmful:Bar recommended to many people becomes overcrowdedIn the long term, our information horizons shrink
bosons fermions
Matúš Medo (Uni Fribourg) Complexity insights into information filtering 17 / 25
![Page 28: University of Fribourg, Switzerlandmedo/physics/talks/info_filtering.pdf · Matúš Medo University of Fribourg, Switzerland ICT Applications to Non-Equilibrium Social Sciences 11-12](https://reader034.vdocuments.mx/reader034/viewer/2022050411/5f880cc49a246a4f306d09e1/html5/thumbnails/28.jpg)
Crowd-avoidance in recommendation(EPL 101, 20008, 2013)
Easy to achieve ex post:1 A recommendation algorithm produces a ranked list of items for
each user
2 We impose maximal occupancy m for every item
An example with m = 1
user 1
user 2 user 3
item 7X
item 3X item 7
item 3
item 4 item 2X
Two ways to enforce occupation:User-by-user (local optimization)
Minimize the total rank of chosen items (global optimization)
Matúš Medo (Uni Fribourg) Complexity insights into information filtering 18 / 25
![Page 29: University of Fribourg, Switzerlandmedo/physics/talks/info_filtering.pdf · Matúš Medo University of Fribourg, Switzerland ICT Applications to Non-Equilibrium Social Sciences 11-12](https://reader034.vdocuments.mx/reader034/viewer/2022050411/5f880cc49a246a4f306d09e1/html5/thumbnails/29.jpg)
Crowd-avoidance in recommendation(EPL 101, 20008, 2013)
Easy to achieve ex post:1 A recommendation algorithm produces a ranked list of items for
each user
2 We impose maximal occupancy m for every item
An example with m = 1
user 1 user 2
user 3
item 7X item 3X
item 7
item 3 item 4
item 2X
Two ways to enforce occupation:User-by-user (local optimization)
Minimize the total rank of chosen items (global optimization)
Matúš Medo (Uni Fribourg) Complexity insights into information filtering 18 / 25
![Page 30: University of Fribourg, Switzerlandmedo/physics/talks/info_filtering.pdf · Matúš Medo University of Fribourg, Switzerland ICT Applications to Non-Equilibrium Social Sciences 11-12](https://reader034.vdocuments.mx/reader034/viewer/2022050411/5f880cc49a246a4f306d09e1/html5/thumbnails/30.jpg)
Crowd-avoidance in recommendation(EPL 101, 20008, 2013)
Easy to achieve ex post:1 A recommendation algorithm produces a ranked list of items for
each user
2 We impose maximal occupancy m for every item
An example with m = 1
user 1 user 2 user 3item 7X item 3X item 7item 3 item 4 item 2X
Two ways to enforce occupation:User-by-user (local optimization)
Minimize the total rank of chosen items (global optimization)
Matúš Medo (Uni Fribourg) Complexity insights into information filtering 18 / 25
![Page 31: University of Fribourg, Switzerlandmedo/physics/talks/info_filtering.pdf · Matúš Medo University of Fribourg, Switzerland ICT Applications to Non-Equilibrium Social Sciences 11-12](https://reader034.vdocuments.mx/reader034/viewer/2022050411/5f880cc49a246a4f306d09e1/html5/thumbnails/31.jpg)
Crowd-avoidance in recommendation(EPL 101, 20008, 2013)
Easy to achieve ex post:1 A recommendation algorithm produces a ranked list of items for
each user
2 We impose maximal occupancy m for every item
An example with m = 1
user 1 user 2 user 3item 7X item 3X item 7item 3 item 4 item 2X
Two ways to enforce occupation:User-by-user (local optimization)
Minimize the total rank of chosen items (global optimization)
Matúš Medo (Uni Fribourg) Complexity insights into information filtering 18 / 25
![Page 32: University of Fribourg, Switzerlandmedo/physics/talks/info_filtering.pdf · Matúš Medo University of Fribourg, Switzerland ICT Applications to Non-Equilibrium Social Sciences 11-12](https://reader034.vdocuments.mx/reader034/viewer/2022050411/5f880cc49a246a4f306d09e1/html5/thumbnails/32.jpg)
Test case
Netflix subset with 2000 users
One object recommended to each user (m = 1)
There is no reason for real crowd avoidance in DVD rentals
Country-production data would be a better test candidate but. . .
Matúš Medo (Uni Fribourg) Complexity insights into information filtering 19 / 25
![Page 33: University of Fribourg, Switzerlandmedo/physics/talks/info_filtering.pdf · Matúš Medo University of Fribourg, Switzerland ICT Applications to Non-Equilibrium Social Sciences 11-12](https://reader034.vdocuments.mx/reader034/viewer/2022050411/5f880cc49a246a4f306d09e1/html5/thumbnails/33.jpg)
Test case
Netflix subset with 2000 users
One object recommended to each user (m = 1)
There is no reason for real crowd avoidance in DVD rentals
Country-production data would be a better test candidate but. . .
Matúš Medo (Uni Fribourg) Complexity insights into information filtering 19 / 25
![Page 34: University of Fribourg, Switzerlandmedo/physics/talks/info_filtering.pdf · Matúš Medo University of Fribourg, Switzerland ICT Applications to Non-Equilibrium Social Sciences 11-12](https://reader034.vdocuments.mx/reader034/viewer/2022050411/5f880cc49a246a4f306d09e1/html5/thumbnails/34.jpg)
Crowd-avoidance enhances diversity
neff =(∑
α
kα)2/∑
α
k2α
1 10 100 1000maximal occupancy m
101
102
103
effe
ctiv
e num
ber
of
item
s n
eff
Matúš Medo (Uni Fribourg) Complexity insights into information filtering 20 / 25
![Page 35: University of Fribourg, Switzerlandmedo/physics/talks/info_filtering.pdf · Matúš Medo University of Fribourg, Switzerland ICT Applications to Non-Equilibrium Social Sciences 11-12](https://reader034.vdocuments.mx/reader034/viewer/2022050411/5f880cc49a246a4f306d09e1/html5/thumbnails/35.jpg)
Crowd-avoidance enhances accuracy
100
101
102
103
maximal occupancy m
0.00
0.10
0.20
0.30pre
cisi
on P
(m)
local optimizationglobal optimization
+34%
+11%
Matúš Medo (Uni Fribourg) Complexity insights into information filtering 21 / 25
![Page 36: University of Fribourg, Switzerlandmedo/physics/talks/info_filtering.pdf · Matúš Medo University of Fribourg, Switzerland ICT Applications to Non-Equilibrium Social Sciences 11-12](https://reader034.vdocuments.mx/reader034/viewer/2022050411/5f880cc49a246a4f306d09e1/html5/thumbnails/36.jpg)
Is it because our recommendations are wrong?
1 10 100 1000rank r
0.00
0.05
0.10
0.15p
reci
sio
n P
(r)
1 10 100 1000rank r
0.00
0.05
0.10
0.15
pre
cisi
on
P(r
)
No
Matúš Medo (Uni Fribourg) Complexity insights into information filtering 22 / 25
![Page 37: University of Fribourg, Switzerlandmedo/physics/talks/info_filtering.pdf · Matúš Medo University of Fribourg, Switzerland ICT Applications to Non-Equilibrium Social Sciences 11-12](https://reader034.vdocuments.mx/reader034/viewer/2022050411/5f880cc49a246a4f306d09e1/html5/thumbnails/37.jpg)
Is it because our recommendations are wrong?
1 10 100 1000rank r
0.00
0.05
0.10
0.15
pre
cisi
on
P(r
)
1 10 100 1000rank r
0.00
0.05
0.10
0.15p
reci
sio
n P
(r)
No
Matúš Medo (Uni Fribourg) Complexity insights into information filtering 22 / 25
![Page 38: University of Fribourg, Switzerlandmedo/physics/talks/info_filtering.pdf · Matúš Medo University of Fribourg, Switzerland ICT Applications to Non-Equilibrium Social Sciences 11-12](https://reader034.vdocuments.mx/reader034/viewer/2022050411/5f880cc49a246a4f306d09e1/html5/thumbnails/38.jpg)
The (probably) correct explanation
Recommendation algorithms are often biasedTypically towards popular items but less tangible reasons exist too
Hypothesis: crowd-avoidance suppresses these biases
0 2 4 6 8 10maximal occupancy m
0.22
0.24
0.26
0.28
pre
cisi
on
P(m
)
without bias
ARTIFICIAL DATA
0 2 4 6 8 10maximal occupancy m
0.22
0.24
0.26
0.28
pre
cisi
on
P(m
)
without biaswith bias
ARTIFICIAL DATA
Matúš Medo (Uni Fribourg) Complexity insights into information filtering 23 / 25
![Page 39: University of Fribourg, Switzerlandmedo/physics/talks/info_filtering.pdf · Matúš Medo University of Fribourg, Switzerland ICT Applications to Non-Equilibrium Social Sciences 11-12](https://reader034.vdocuments.mx/reader034/viewer/2022050411/5f880cc49a246a4f306d09e1/html5/thumbnails/39.jpg)
The (probably) correct explanation
Recommendation algorithms are often biasedTypically towards popular items but less tangible reasons exist too
Hypothesis: crowd-avoidance suppresses these biases
0 2 4 6 8 10maximal occupancy m
0.22
0.24
0.26
0.28
pre
cisi
on
P(m
)
without bias
ARTIFICIAL DATA
0 2 4 6 8 10maximal occupancy m
0.22
0.24
0.26
0.28
pre
cisi
on
P(m
)
without biaswith bias
ARTIFICIAL DATA
Matúš Medo (Uni Fribourg) Complexity insights into information filtering 23 / 25
![Page 40: University of Fribourg, Switzerlandmedo/physics/talks/info_filtering.pdf · Matúš Medo University of Fribourg, Switzerland ICT Applications to Non-Equilibrium Social Sciences 11-12](https://reader034.vdocuments.mx/reader034/viewer/2022050411/5f880cc49a246a4f306d09e1/html5/thumbnails/40.jpg)
The (probably) correct explanation
Recommendation algorithms are often biasedTypically towards popular items but less tangible reasons exist too
Hypothesis: crowd-avoidance suppresses these biases
0 2 4 6 8 10maximal occupancy m
0.22
0.24
0.26
0.28
pre
cisi
on
P(m
)
without bias
ARTIFICIAL DATA
0 2 4 6 8 10maximal occupancy m
0.22
0.24
0.26
0.28
pre
cisi
on
P(m
)
without biaswith bias
ARTIFICIAL DATA
Matúš Medo (Uni Fribourg) Complexity insights into information filtering 23 / 25
![Page 41: University of Fribourg, Switzerlandmedo/physics/talks/info_filtering.pdf · Matúš Medo University of Fribourg, Switzerland ICT Applications to Non-Equilibrium Social Sciences 11-12](https://reader034.vdocuments.mx/reader034/viewer/2022050411/5f880cc49a246a4f306d09e1/html5/thumbnails/41.jpg)
The (probably) correct explanation
Recommendation algorithms are often biasedTypically towards popular items but less tangible reasons exist too
Hypothesis: crowd-avoidance suppresses these biases
0 2 4 6 8 10maximal occupancy m
0.22
0.24
0.26
0.28
pre
cisi
on
P(m
)
without bias
ARTIFICIAL DATA
0 2 4 6 8 10maximal occupancy m
0.22
0.24
0.26
0.28
pre
cisi
on
P(m
)
without biaswith bias
ARTIFICIAL DATA
Matúš Medo (Uni Fribourg) Complexity insights into information filtering 23 / 25
![Page 42: University of Fribourg, Switzerlandmedo/physics/talks/info_filtering.pdf · Matúš Medo University of Fribourg, Switzerland ICT Applications to Non-Equilibrium Social Sciences 11-12](https://reader034.vdocuments.mx/reader034/viewer/2022050411/5f880cc49a246a4f306d09e1/html5/thumbnails/42.jpg)
Crowd-avoidance: Summary
Crowd-avoidance can improve both accuracy and diversity ofrecommendation
A rare case where constraints improve the outcome
To do
How best to introduce occupancy constraints?Constraints heterogeneous over objectsApproaches to global optimization
Study real data with crowd avoidanceCountry-production data
Matúš Medo (Uni Fribourg) Complexity insights into information filtering 24 / 25
![Page 43: University of Fribourg, Switzerlandmedo/physics/talks/info_filtering.pdf · Matúš Medo University of Fribourg, Switzerland ICT Applications to Non-Equilibrium Social Sciences 11-12](https://reader034.vdocuments.mx/reader034/viewer/2022050411/5f880cc49a246a4f306d09e1/html5/thumbnails/43.jpg)
Crowd-avoidance: Summary
Crowd-avoidance can improve both accuracy and diversity ofrecommendation
A rare case where constraints improve the outcome
To do
How best to introduce occupancy constraints?Constraints heterogeneous over objectsApproaches to global optimization
Study real data with crowd avoidanceCountry-production data
Matúš Medo (Uni Fribourg) Complexity insights into information filtering 24 / 25
![Page 44: University of Fribourg, Switzerlandmedo/physics/talks/info_filtering.pdf · Matúš Medo University of Fribourg, Switzerland ICT Applications to Non-Equilibrium Social Sciences 11-12](https://reader034.vdocuments.mx/reader034/viewer/2022050411/5f880cc49a246a4f306d09e1/html5/thumbnails/44.jpg)
Thank you for your attention!
Questions?
Matúš Medo (Uni Fribourg) Complexity insights into information filtering 25 / 25