Embed Size (px)
DESCRIPTION
Online providers frequently offer a variety of services, ranging from news to mail. These providers endeavour to keep users accessing and interacting with the sites offering these services, that is to engage users by spending time on these sites and returning regularly to them. The standard approach to evaluate engagement with a site is by measuring engagement metrics of each site separately. However, when assessing engagement within a network of sites, it is crucial to take into account the traffic between sites. This paper proposes a methodology for studying networked used engagement. We represent sites (nodes) and user traffic (edges) between them as a network, and apply complex network metrics to study networked user engagement. We demonstrate the value of these metrics when applied to 728 services offered by Yahoo! with a sample of 2M users and a total of 25M online sessions.
Citation preview
Networked user engagement
user engagement optimization workshop - CIKM 2013
In collabora*on with: Mounia Lalmas,
Ricardo Baeza-‐Yates, Elad Yom-‐Tov
Jane%e Lehmann
outline
1. Mo%va%on How to measure user engagement?
2. Network metrics From site to network engagement.
3. Measuring networked user engagement Case study based on the network of Yahoo sites.
Lights on by JC*+A!
182-‐365+1 by meaganm
akes
How to measure User Engagement?
Measuring user engagement
4 JaneHe Lehmann MoJvaJon
User engagement on Yahoo! Web traffic reports: Google Analy%cs, Alexa.com, …
Popularity (#Users) Ac%vity (DwellTime) Loyalty (AcJveDays)
Measuring user engagement
5 JaneHe Lehmann MoJvaJon
However, Yahoo! has many sites: maps, flickr, weather, news, shine, etc. How to measure engagement within a network of sites?
NETWORK OF SITES
6 JaneHe Lehmann MoJvaJon
frontpage
tv sports
shopping
autos
search
daJng
jobs
news
shine
groups
maps local
health answer
weather
games
omg
homes travel
flickr
finance
Large online providers (AOL, Google, Yahoo!, etc.) offer not one site, but a network of sites
NETWORK OF SITES
7 JaneHe Lehmann MoJvaJon
frontpage
tv sports
shopping
autos
search
daJng
jobs
news
shine
groups
maps local
health answer
weather
games
omg
homes travel
flickr
finance
Each site is usually op%mized individually, with some effort to direct users between them
NETWORK OF SITES
8 JaneHe Lehmann MoJvaJon
frontpage
tv sports
shopping
autos
search
daJng
jobs
news
shine
groups
maps local
health answer
weather
games
omg
homes travel
flickr
finance
Online mul%tasking Users switch between sites within one online session (e.g. emailing, reading news, social networking, search)
NETWORK OF SITES
9 JaneHe Lehmann MoJvaJon
frontpage
tv sports
shopping
autos
search
daJng
jobs
news
shine
groups
maps local
health answer
weather
games
omg
homes travel
flickr
finance
Measuring user engagement should account for user traffic between sites...
NETWORK OF SITES
10 JaneHe Lehmann MoJvaJon
frontpage
tv sports
shopping
autos
search
daJng
jobs
news
shine
groups
maps local
health answer
weather
games
omg
homes travel
flickr
finance
… to detect missing connec9ons
NETWORK OF SITES
11 JaneHe Lehmann MoJvaJon
frontpage
tv sports
shopping
autos
search
daJng
jobs
news
shine
groups
maps local
health answer
weather
games
omg
homes travel
flickr
finance
… to detect sites that are used together
NETWORK OF SITES
12 JaneHe Lehmann MoJvaJon
frontpage
tv sports
shopping
autos
search
daJng
jobs
news
shine
groups
maps local
health answer
weather
games
omg
homes travel
flickr
finance
… to detect isolated sites
NETWORK OF SITES
13 JaneHe Lehmann MoJvaJon
frontpage
tv sports
shopping
autos
search
daJng
jobs
news
shine
groups
maps local
health answer
weather
games
omg
homes travel
flickr
finance
… to detect important sites
Danboard by sⓘndy°
From site to network engagement
Looking beyond one own nose!
ENGAGEMENT NETWORKS
15 Networked User Engagement JaneHe Lehmann
State of the art Our approach
Non-providerservice
ProviderserviceProvidernetwork
Tra!c
Provider service under study
G=(V, E, λ)
V: n<i> -‐ Sites, services, funcJonaliJes, etc. n<e> -‐ External sites
E: e<i> -‐ Traffic between internal nodes
λ(e): Traffic volume (#Users)
Network metrics
TrafficVolume Def.: Sum of edge weights A high value is a sign of a high networked user engagement; users navigate oHen between the sites of the network.
Density Def.: ProporJon of edges to maximum possible A high value is a sign of a high networked user engagement; users navigate between many different sites.
Reciprocity Def.: RaJo of number of edges in both direcJons A high value is a sign of a high networked user engagement; users do not only navigate from one site to another, they also tend to return to previously visited sites.
Network-LEVEL METRICS
16 Networked User Engagement JaneHe Lehmann
30 78
22 15 45
9
90
Conn
ecJvity
Size
Network metrics
Modularity [subNWmod] Def.: Captures the existence of modules based on a random walk approach A high modularity indicates that users visit many sites of one subnetwork, but hardly navigate to other subnetworks.
Global Clustering Coefficient [GCC] Def.: Captures the existence of Jghtly connected groups of nodes A high value is a sign of a high networked user engagement; users access sites directly, instead of using front pages.
Network-LEVEL METRICS
17 Networked User Engagement JaneHe Lehmann
Low GCC High GCC
Mod
ularity
ENGAGEMENT NETWORKS
Modularity [subNWmod] Yahoo! Network with 3 countries
18 Networked User Engagement JaneHe Lehmann
low modularity è low engagement high modularity è high engagement
ENGAGEMENT NETWORKS
Modularity [subNWmod] Yahoo! Network with 3 countries
19 Networked User Engagement JaneHe Lehmann
low modularity è low engagement high modularity è high engagement
low modularity è high engagement
Country
lem by [ embr ] Case Study
Measuring Networked user engagement
ENGAGEMENT NETWORKS
Interac%on data • July 2011 • 2M user, 25M sessions • 728 Yahoo! sites (news, mails, search, etc.)
Networks • 12 weighted directed networks, filtered by
» Edge types (internal, internal + returning) » Time (Wednesday, Sunday) » User loyalty (causal, acJve, VIP) » Country (5 EU countries)
• Applying metrics on networks, results are
normalized using the z-‐score
21 Networked User Engagement JaneHe Lehmann
external
returning edge
z-‐score
trafficVolume
Country1
Country3
Country4
Country2
Country5
average
aboveaverage
belowaverage
Network-LEVEL METRICS
Edge-‐based networks
Leaving the network does not necessarily entail less engagement • Traffic increases significant when accounJng for returning traffic [18.26%] • ConnecJvity increases à User use new navigaJon paths • Modularity increases à Users stay in the same part of the network
22 Networked User Engagement JaneHe Lehmann
trafficVolume density reciprocity subNWmod
GCC
Size Connectivity Modularity
Edgeint
Edgeint+ext
Timewed
Timesun
Usercasual
Usernormal
UserVIP
Network-LEVEL METRICS
Time-‐based networks
Naviga%on is more goal-‐oriented during the week • TrafficVolume and density are lower on Sunday à Less acJvity • Reciprocity, subNW, and GCC are higher on Sunday à User return more omen to already
visited sites and visit more sites than usual (browsing with less specific goals)
23 Networked User Engagement JaneHe Lehmann
trafficVolume density reciprocity subNWmod
GCC
Size Connectivity Modularity
Edgeint
Edgeint+ext
Timewed
Timesun
Usercasual
Usernormal
UserVIP
Network-LEVEL METRICS
User-‐based networks
Loyal users have also a higher networked user engagement • TrafficVolume is higher for VIP user à More acJvity • ConnecJvity and modularity is higher for VIP user à Users are interested in more sites • GCC is lower for causal users à Users access sites using the front pages
24 Networked User Engagement JaneHe Lehmann
trafficVolume density reciprocity subNWmod
GCC
Size Connectivity Modularity
Edgeint
Edgeint+ext
Timewed
Timesun
Usercasual
Usernormal
UserVIP
trafficVolume density reciprocity subNWmod
GCC
Size Connectivity Modularity
Country1
Country3
Country4
Country2
Country5
Network-LEVEL METRICS
Country-‐based networks
Networked user engagement differs between countries • Country1: Highest trafficVolume + GCC, high density + reciprocity, lowest subNWmod à high
engagement • Country5: Low trafficVolume, but high density + GCC, low subNWmod à less popular, but high
engagement
25 Networked User Engagement JaneHe Lehmann
CONCLUSION AND FUTURE WORK • Network analysis enhances the understanding of user engagement.
It allows: • To analyze users browsing behavior on a global scale • To compare networks of sites (e.g. of different countries) • To observe differences over Jme
• Leaving the network does not entail less engagement • Some network metrics are more useful than others
Future work: • DefiniJon of metrics that combine network traffic and engagement • Comparing traffic-‐ and hyperlink-‐networks • Studying the effect of changes in the network
26 Networked User Engagement JaneHe Lehmann
Janebe Lehmann Universitat Pompeu Fabra, Spain [email protected] Mounia Lalmas Yahoo Labs, London, UK [email protected] Ricardo Baeza-‐Yates Yahoo Labs, Barcelona, Spain [email protected] Elad Yom-‐Tov Microsog Research, Herzliya, Israel [email protected]
