large-scale social network data mining with multi-view

Large-Scale Social Network Data Mining withMulti-View Information

Hao Wang

Dept. of Computer Science and EngineeringShanghai Jiao Tong University

Supervisor: Wu-Jun Li

2013.6.19

Hao Wang Multi-View Social Mining 1 / 28

Our work

1 Hao Wang£�§¤, Binyi Chen, Wu-Jun Li. Collaborative Topic Regressionwith Social Regularization for Tag Recommendation. Proceedings of theTwenty-Third International Joint Conference on Artificial Intelligence(IJCAI), 2013.

2 Hao Wang£�§¤, Wu-Jun Li. Relational Collaborative Topic Regressionfor Recommendation Systems. IEEE Transactions on Knowledge and DataEngineering (TKDE), 2013. (submitted)

3 Hao Wang£�§¤, Wu-Jun Li. Online Egocentric Models for CitationNetworks. Proceedings of the Twenty-Third International Joint Conferenceon Artificial Intelligence (IJCAI), 2013.


Outline

1 Motivation

2 Relational CTR

3 CTR with Social Regularization

4 Online Egocentric Models

5 Conclusion


Outline

1 Motivation

2 Relational CTR



5 Conclusion


Motivation


Motivation

Content:1 Lee et al., 20102 Chen et al., 20103 Lipczak et al., 2009

Ratings:1 Salakhutdinov et al., 20072 Herlocker et al., 1999

Hybrid:1 Purushotham et al., 20122 Wang and Blei, 20113 Agarwal and Chen, 20104 A. Said, 2010


Contribution

1 Integrate ratings, contents and item networks2 Significantly improve the accuracy3 Even less training time4 Extend to dynamic networks


Outline

1 Motivation

2 Relational CTR



5 Conclusion


Comparison of article representation


Combination of both


Graphical model of CTR

v

k

K

u

J

I

v r

z w

u


Graphical model of RCTR

l jj ', v

sj vj

j

ui

ui

r ji,

w nj,z nj,

sj ' vj '

'j z nj ,' w nj ,'

r ji ',

u

kI

I

J

J

K

N

N

e

r


Objective function

The log-likelihood:

L = ρ∑(j,j′)

log σ(ηT (sj ◦ sj′) + ν)

−λr2

∑j

(sj − vj)T (sj − vj)−λe2η+

Tη+

− λu2

∑i

uTi ui −λv2

∑j

(vj − θj)T (vj − θj)

+∑j

∑n

log(∑k

θjkβk,wjn)−

∑i,j

cij2(rij − uTi vj)2.


Updating rules

For U and V :

ui ← (V CiVT + λuIK)−1V CiRi,

vj ← (UCiUT + λvIK + λrIK)−1(UCjRj + λvθj + λrsj),

For η+:

∇η+L = ρ∑lj,j′=1

(1− σ(η+Tπ+j,j′))π+j,j′ − λeη

+,

For φjnk:

φjnk ∝ θjkβk,wjn.

For βkw:

βkw ∝∑j

∑n

φjnk1[wjn = w].


Datasets

Description of datasets

citeulike-a citeulike-t

#users 5551 7947

#items 16980 25975

#tags 19107 52946

#citations 44709 32565

#user-item pairs 204987 134860

sparsity 99.78% 99.93%

#relations 549447 438722


Experimental results

50 100 150 200 250 300

0.05

0.1

0.15

0.2

0.25

0.3

M

Rec

all

CFCTRRCTR

The user-oriented recall of RCTR, CTR, and CF when M ranges from 50 to 300on dataset citeulike-t. P is set to 1. Similar phenomenon can be observed forother values of P .


Case study

Interpretability of the learning latent structures


Outline

1 Motivation

2 Relational CTR



5 Conclusion


Graphical model of CTR

v

k

K

u

J

I

v r

z w

u


Graphical model of CTR-SR

v

k

K

u

J

I

v

r

z w

u

r

l

s

A


Objective function

The log-likelihood:

L = −λl2tr(SLaS

T )− λr2

∑j

(sj − vj)T (sj − vj)

− λu2

∑i

uTi ui −λv2

∑j

(vj − θj)T (vj − θj)

+∑j

∑n

log(∑k

θjkβk,wjn)−

∑i,j

cij2(rij − uTi vj)2.

where

tr(SLaST ) =

1

2

J∑j=1

J∑j′=1

Ajj′ ||S∗j − S∗j′ ||2

=1

2

J∑j=1

J∑j′=1

[Ajj′K∑k=1

(Skj − Skj′)2]


Updating rules

For U and V :

ui ← (V CiVT + λuIK)−1V CiRi,

vj ← (UCiUT + λvIK + λrIK)−1(UCjRj + λvθj + λrsj),

For S:

Sk∗(t+ 1)← Sk∗(t) + δ(t)r(t)

r(t)← λrVk∗ − (λlLa + λrIJ)Sk∗(t)

δ(t)← r(t)T r(t)

r(t)T (λlLa + λrIJ)r(t)

For φjnk:

φjnk ∝ θjkβk,wjn.

For βkw:

βkw ∝∑j

∑n

φjnk1[wjn = w].



1 2 3 4 5 6 7 8 9 10

0.05

0.1

0.15

0.2

0.25

0.3

P

Rec

all

SCFSCF+LDACFTAGCOCTRCTR−SR

Recall@50 for all methods in citeulike-a.


Case study

Example articles with recommmended tags


Outline

1 Motivation

2 Relational CTR



5 Conclusion


From DEM to OEM

Three parts of variables:

1 Model parameters β2 Link features3 Topic features

Dynamic Egocentric Models (DEM, adapting only Model parameters):

L(β) =

m∏e=1

exp(βT sie(te))n∑i=1

Yi(te) exp(βT si(te))

Online Egocentric Models (OEM, adapting all three parts):

minimize − logL(β,ω) + λn∑k=1

‖ωk − θk‖22

subject to : ωk � 0, 1Tωk = 1,


Updating rules

Using coordinate descent:

1 Online β Step:

Lw(β) =

x+q−1∏e=x+q−Wt

exp(βT sie(te))n∑i=1

Yi(te) exp(βT si(te))

.

2 Online Topic Step:

∂f

∂ωk=−

p∑i=1

ai +

p∑i=1

aiαi exp(aTi ωk)

Ai + αi exp(aTi ωk)

+

q∑u=p+1

buγu exp(bTuωk)

Bu + γu exp(bTuωk)

+ 2λ(ωk − θk)


Datasets

Information of data sets

Data Set #Papers #Citations #Unique TimesarXiv-TH 14226 100025 10500arXiv-PH 16526 125311 1591

Data set partition for building, training and testing.

Data Sets Building Training TestingarXiv-TH 62239 1465 36328arXiv-PH 82343 1739 41229



0 5 10 15 20 25

−12

−11

−10

−9

−8

Paper batches

Ave

rage

log−

likel

ihoo

d

DEMOEM−betaOEM−apprOEM−full

(a) arXiv-TH

0 5 10 15 20 25

−11

−10

−9

−8

Paper batches

Ave

rage

log−

likel

ihoo

d


(b) arXiv-PH

0 5 10 15 20 250.25

0.3

0.35

0.4

0.45

Paper batches

Rec

all


(c) arXiv-TH(K=250)

0 5 10 15 20 250.2

0.25

0.3

0.35

0.4

Paper batches

Rec

all


(d) arXiv-PH(K=250)


Outline

1 Motivation

2 Relational CTR



5 Conclusion


Conclusion

Data Mining with Multi-View Information:

1 Relational CTR:Social information as prior

2 CTR with Social regularization:Social information as observation

3 Online Egocentric Models:Dynamic social information


Publication

1 Hao Wang£�§¤, Binyi Chen, Wu-Jun Li. CollaborativeTopic Regression with Social Regularization for TagRecommendation. Proceedings of the Twenty-Third InternationalJoint Conference on Artificial Intelligence (IJCAI), 2013.

2 Hao Wang£�§¤, Wu-Jun Li. Online Egocentric Models forCitation Networks. Proceedings of the Twenty-ThirdInternational Joint Conference on Artificial Intelligence (IJCAI),2013.

3 Hao Wang£�§¤, Wu-Jun Li. Relational Collaborative TopicRegression for Recommendation Systems. IEEE Transactions onKnowledge and Data Engineering (TKDE), 2013. (submitted)


Q & A


large-scale social network data mining with multi-view

Documents