networked user engagement
DESCRIPTION
Online providers frequently offer a variety of services, ranging from news to mail. These providers endeavour to keep users accessing and interacting with the sites offering these services, that is to engage users by spending time on these sites and returning regularly to them. The standard approach to evaluate engagement with a site is by measuring engagement metrics of each site separately. However, when assessing engagement within a network of sites, it is crucial to take into account the traffic between sites. This paper proposes a methodology for studying networked used engagement. We represent sites (nodes) and user traffic (edges) between them as a network, and apply complex network metrics to study networked user engagement. We demonstrate the value of these metrics when applied to 728 services offered by Yahoo! with a sample of 2M users and a total of 25M online sessions.TRANSCRIPT
![Page 1: Networked User Engagement](https://reader033.vdocuments.mx/reader033/viewer/2022052618/554efed6b4c90577698b4d5a/html5/thumbnails/1.jpg)
Networked user engagement
user engagement optimization workshop - CIKM 2013
In collabora*on with: Mounia Lalmas,
Ricardo Baeza-‐Yates, Elad Yom-‐Tov
Jane%e Lehmann
![Page 2: Networked User Engagement](https://reader033.vdocuments.mx/reader033/viewer/2022052618/554efed6b4c90577698b4d5a/html5/thumbnails/2.jpg)
outline
1. Mo%va%on How to measure user engagement?
2. Network metrics From site to network engagement.
3. Measuring networked user engagement Case study based on the network of Yahoo sites.
Lights on by JC*+A!
![Page 3: Networked User Engagement](https://reader033.vdocuments.mx/reader033/viewer/2022052618/554efed6b4c90577698b4d5a/html5/thumbnails/3.jpg)
182-‐365+1 by meaganm
akes
How to measure User Engagement?
![Page 4: Networked User Engagement](https://reader033.vdocuments.mx/reader033/viewer/2022052618/554efed6b4c90577698b4d5a/html5/thumbnails/4.jpg)
Measuring user engagement
4 JaneHe Lehmann MoJvaJon
User engagement on Yahoo! Web traffic reports: Google Analy%cs, Alexa.com, …
Popularity (#Users) Ac%vity (DwellTime) Loyalty (AcJveDays)
![Page 5: Networked User Engagement](https://reader033.vdocuments.mx/reader033/viewer/2022052618/554efed6b4c90577698b4d5a/html5/thumbnails/5.jpg)
Measuring user engagement
5 JaneHe Lehmann MoJvaJon
However, Yahoo! has many sites: maps, flickr, weather, news, shine, etc. How to measure engagement within a network of sites?
![Page 6: Networked User Engagement](https://reader033.vdocuments.mx/reader033/viewer/2022052618/554efed6b4c90577698b4d5a/html5/thumbnails/6.jpg)
NETWORK OF SITES
6 JaneHe Lehmann MoJvaJon
frontpage
tv sports
shopping
autos
search
daJng
jobs
news
shine
groups
maps local
health answer
weather
games
omg
homes travel
flickr
finance
Large online providers (AOL, Google, Yahoo!, etc.) offer not one site, but a network of sites
![Page 7: Networked User Engagement](https://reader033.vdocuments.mx/reader033/viewer/2022052618/554efed6b4c90577698b4d5a/html5/thumbnails/7.jpg)
NETWORK OF SITES
7 JaneHe Lehmann MoJvaJon
frontpage
tv sports
shopping
autos
search
daJng
jobs
news
shine
groups
maps local
health answer
weather
games
omg
homes travel
flickr
finance
Each site is usually op%mized individually, with some effort to direct users between them
![Page 8: Networked User Engagement](https://reader033.vdocuments.mx/reader033/viewer/2022052618/554efed6b4c90577698b4d5a/html5/thumbnails/8.jpg)
NETWORK OF SITES
8 JaneHe Lehmann MoJvaJon
frontpage
tv sports
shopping
autos
search
daJng
jobs
news
shine
groups
maps local
health answer
weather
games
omg
homes travel
flickr
finance
Online mul%tasking Users switch between sites within one online session (e.g. emailing, reading news, social networking, search)
![Page 9: Networked User Engagement](https://reader033.vdocuments.mx/reader033/viewer/2022052618/554efed6b4c90577698b4d5a/html5/thumbnails/9.jpg)
NETWORK OF SITES
9 JaneHe Lehmann MoJvaJon
frontpage
tv sports
shopping
autos
search
daJng
jobs
news
shine
groups
maps local
health answer
weather
games
omg
homes travel
flickr
finance
Measuring user engagement should account for user traffic between sites...
![Page 10: Networked User Engagement](https://reader033.vdocuments.mx/reader033/viewer/2022052618/554efed6b4c90577698b4d5a/html5/thumbnails/10.jpg)
NETWORK OF SITES
10 JaneHe Lehmann MoJvaJon
frontpage
tv sports
shopping
autos
search
daJng
jobs
news
shine
groups
maps local
health answer
weather
games
omg
homes travel
flickr
finance
… to detect missing connec9ons
![Page 11: Networked User Engagement](https://reader033.vdocuments.mx/reader033/viewer/2022052618/554efed6b4c90577698b4d5a/html5/thumbnails/11.jpg)
NETWORK OF SITES
11 JaneHe Lehmann MoJvaJon
frontpage
tv sports
shopping
autos
search
daJng
jobs
news
shine
groups
maps local
health answer
weather
games
omg
homes travel
flickr
finance
… to detect sites that are used together
![Page 12: Networked User Engagement](https://reader033.vdocuments.mx/reader033/viewer/2022052618/554efed6b4c90577698b4d5a/html5/thumbnails/12.jpg)
NETWORK OF SITES
12 JaneHe Lehmann MoJvaJon
frontpage
tv sports
shopping
autos
search
daJng
jobs
news
shine
groups
maps local
health answer
weather
games
omg
homes travel
flickr
finance
… to detect isolated sites
![Page 13: Networked User Engagement](https://reader033.vdocuments.mx/reader033/viewer/2022052618/554efed6b4c90577698b4d5a/html5/thumbnails/13.jpg)
NETWORK OF SITES
13 JaneHe Lehmann MoJvaJon
frontpage
tv sports
shopping
autos
search
daJng
jobs
news
shine
groups
maps local
health answer
weather
games
omg
homes travel
flickr
finance
… to detect important sites
![Page 14: Networked User Engagement](https://reader033.vdocuments.mx/reader033/viewer/2022052618/554efed6b4c90577698b4d5a/html5/thumbnails/14.jpg)
Danboard by sⓘndy°
From site to network engagement
Looking beyond one own nose!
![Page 15: Networked User Engagement](https://reader033.vdocuments.mx/reader033/viewer/2022052618/554efed6b4c90577698b4d5a/html5/thumbnails/15.jpg)
ENGAGEMENT NETWORKS
15 Networked User Engagement JaneHe Lehmann
State of the art Our approach
Non-providerservice
ProviderserviceProvidernetwork
Tra!c
Provider service under study
G=(V, E, λ)
V: n<i> -‐ Sites, services, funcJonaliJes, etc. n<e> -‐ External sites
E: e<i> -‐ Traffic between internal nodes
λ(e): Traffic volume (#Users)
![Page 16: Networked User Engagement](https://reader033.vdocuments.mx/reader033/viewer/2022052618/554efed6b4c90577698b4d5a/html5/thumbnails/16.jpg)
Network metrics
TrafficVolume Def.: Sum of edge weights A high value is a sign of a high networked user engagement; users navigate oHen between the sites of the network.
Density Def.: ProporJon of edges to maximum possible A high value is a sign of a high networked user engagement; users navigate between many different sites.
Reciprocity Def.: RaJo of number of edges in both direcJons A high value is a sign of a high networked user engagement; users do not only navigate from one site to another, they also tend to return to previously visited sites.
Network-LEVEL METRICS
16 Networked User Engagement JaneHe Lehmann
30 78
22 15 45
9
90
Conn
ecJvity
Size
![Page 17: Networked User Engagement](https://reader033.vdocuments.mx/reader033/viewer/2022052618/554efed6b4c90577698b4d5a/html5/thumbnails/17.jpg)
Network metrics
Modularity [subNWmod] Def.: Captures the existence of modules based on a random walk approach A high modularity indicates that users visit many sites of one subnetwork, but hardly navigate to other subnetworks.
Global Clustering Coefficient [GCC] Def.: Captures the existence of Jghtly connected groups of nodes A high value is a sign of a high networked user engagement; users access sites directly, instead of using front pages.
Network-LEVEL METRICS
17 Networked User Engagement JaneHe Lehmann
Low GCC High GCC
Mod
ularity
![Page 18: Networked User Engagement](https://reader033.vdocuments.mx/reader033/viewer/2022052618/554efed6b4c90577698b4d5a/html5/thumbnails/18.jpg)
ENGAGEMENT NETWORKS
Modularity [subNWmod] Yahoo! Network with 3 countries
18 Networked User Engagement JaneHe Lehmann
low modularity è low engagement high modularity è high engagement
![Page 19: Networked User Engagement](https://reader033.vdocuments.mx/reader033/viewer/2022052618/554efed6b4c90577698b4d5a/html5/thumbnails/19.jpg)
ENGAGEMENT NETWORKS
Modularity [subNWmod] Yahoo! Network with 3 countries
19 Networked User Engagement JaneHe Lehmann
low modularity è low engagement high modularity è high engagement
low modularity è high engagement
Country
![Page 20: Networked User Engagement](https://reader033.vdocuments.mx/reader033/viewer/2022052618/554efed6b4c90577698b4d5a/html5/thumbnails/20.jpg)
lem by [ embr ] Case Study
Measuring Networked user engagement
![Page 21: Networked User Engagement](https://reader033.vdocuments.mx/reader033/viewer/2022052618/554efed6b4c90577698b4d5a/html5/thumbnails/21.jpg)
ENGAGEMENT NETWORKS
Interac%on data • July 2011 • 2M user, 25M sessions • 728 Yahoo! sites (news, mails, search, etc.)
Networks • 12 weighted directed networks, filtered by
» Edge types (internal, internal + returning) » Time (Wednesday, Sunday) » User loyalty (causal, acJve, VIP) » Country (5 EU countries)
• Applying metrics on networks, results are
normalized using the z-‐score
21 Networked User Engagement JaneHe Lehmann
external
returning edge
z-‐score
trafficVolume
Country1
Country3
Country4
Country2
Country5
average
aboveaverage
belowaverage
![Page 22: Networked User Engagement](https://reader033.vdocuments.mx/reader033/viewer/2022052618/554efed6b4c90577698b4d5a/html5/thumbnails/22.jpg)
Network-LEVEL METRICS
Edge-‐based networks
Leaving the network does not necessarily entail less engagement • Traffic increases significant when accounJng for returning traffic [18.26%] • ConnecJvity increases à User use new navigaJon paths • Modularity increases à Users stay in the same part of the network
22 Networked User Engagement JaneHe Lehmann
trafficVolume density reciprocity subNWmod
GCC
Size Connectivity Modularity
Edgeint
Edgeint+ext
Timewed
Timesun
Usercasual
Usernormal
UserVIP
![Page 23: Networked User Engagement](https://reader033.vdocuments.mx/reader033/viewer/2022052618/554efed6b4c90577698b4d5a/html5/thumbnails/23.jpg)
Network-LEVEL METRICS
Time-‐based networks
Naviga%on is more goal-‐oriented during the week • TrafficVolume and density are lower on Sunday à Less acJvity • Reciprocity, subNW, and GCC are higher on Sunday à User return more omen to already
visited sites and visit more sites than usual (browsing with less specific goals)
23 Networked User Engagement JaneHe Lehmann
trafficVolume density reciprocity subNWmod
GCC
Size Connectivity Modularity
Edgeint
Edgeint+ext
Timewed
Timesun
Usercasual
Usernormal
UserVIP
![Page 24: Networked User Engagement](https://reader033.vdocuments.mx/reader033/viewer/2022052618/554efed6b4c90577698b4d5a/html5/thumbnails/24.jpg)
Network-LEVEL METRICS
User-‐based networks
Loyal users have also a higher networked user engagement • TrafficVolume is higher for VIP user à More acJvity • ConnecJvity and modularity is higher for VIP user à Users are interested in more sites • GCC is lower for causal users à Users access sites using the front pages
24 Networked User Engagement JaneHe Lehmann
trafficVolume density reciprocity subNWmod
GCC
Size Connectivity Modularity
Edgeint
Edgeint+ext
Timewed
Timesun
Usercasual
Usernormal
UserVIP
![Page 25: Networked User Engagement](https://reader033.vdocuments.mx/reader033/viewer/2022052618/554efed6b4c90577698b4d5a/html5/thumbnails/25.jpg)
trafficVolume density reciprocity subNWmod
GCC
Size Connectivity Modularity
Country1
Country3
Country4
Country2
Country5
Network-LEVEL METRICS
Country-‐based networks
Networked user engagement differs between countries • Country1: Highest trafficVolume + GCC, high density + reciprocity, lowest subNWmod à high
engagement • Country5: Low trafficVolume, but high density + GCC, low subNWmod à less popular, but high
engagement
25 Networked User Engagement JaneHe Lehmann
![Page 26: Networked User Engagement](https://reader033.vdocuments.mx/reader033/viewer/2022052618/554efed6b4c90577698b4d5a/html5/thumbnails/26.jpg)
CONCLUSION AND FUTURE WORK • Network analysis enhances the understanding of user engagement.
It allows: • To analyze users browsing behavior on a global scale • To compare networks of sites (e.g. of different countries) • To observe differences over Jme
• Leaving the network does not entail less engagement • Some network metrics are more useful than others
Future work: • DefiniJon of metrics that combine network traffic and engagement • Comparing traffic-‐ and hyperlink-‐networks • Studying the effect of changes in the network
26 Networked User Engagement JaneHe Lehmann
![Page 27: Networked User Engagement](https://reader033.vdocuments.mx/reader033/viewer/2022052618/554efed6b4c90577698b4d5a/html5/thumbnails/27.jpg)
Janebe Lehmann Universitat Pompeu Fabra, Spain [email protected] Mounia Lalmas Yahoo Labs, London, UK [email protected] Ricardo Baeza-‐Yates Yahoo Labs, Barcelona, Spain [email protected] Elad Yom-‐Tov Microsog Research, Herzliya, Israel [email protected]