node similarity and classification - hasso-plattner-institut · node similarity and classification...

Node similarity and classification

Graph Mining course Winter Semester 2017

DavideMottin,AntonTsitsulinHasso Plattner Institute

Acknowledgements

§ Somepartofthislectureistakenfrom:http://web.eecs.umich.edu/~dkoutra/tut/icdm14.html

§ Otheradaptedcontentisfrom SocialNetworkDataAnalytics(Springer)Ed.Charu Aggarwal,March2011

GRAPH MINING WS 2017 2

3GRAPH MINING WS 2017

FriendsnetworkinanAmericanhighschool:1. Nodesarepeople2. Edgesarefriendship3. Colors=races

"Race, school integration, and friendship segregation in America," American Journal of Sociology 107, 679-716 (2001).

?

Whichcolorwillthishave?

What about politics?

Aretheyrepublicanordemocrats?

?

Source: http://adequatebird.com/2010/05/03/the-political-blogosphere-and-the-2004-u-s-election-divided-they-blog/

5

Give me your own example!

GRAPH MINING WS 2017

Classification with Network Data

§ Givenagraphandfewnodesforwhichweknowthe”label”ora”class”howcanwepredict userattributesorinterests?

Predictthelabelsfornonmarkednodes?


Why node classification?

§ Isthisafriendoranaquitance?§ Recommendationsystemstosuggestobjects(music,movies,

activities)§ Automaticallyunderstandrolesinanetwork(hubs,activators,

influencingnodes,…)§ Identifyexpertsforquestionansweringsystems§ Targetedadvertising§ Studyofcommunities(keyindividuals,groupstarters...)§ Studyofdiseasesandcures§ Identifyunusualbehaviorsorbehavioralchanges§ Findingsimilarnodesandoutliers


Why node classification is useful?

§ Notallthenodeshavelabels(usersarenotwillingtoprovideexplanations)

§ Rolesarenotexplicitlydeclared(whoismoreimportantinacompany?Thinkabouttheexchangedemails;))

§ Labelsprovidedbytheuserscanbemisleading§ Labelsaresparse(somecategoriesmightbemissingor

incomplete)


Node classification problem

§ Given:• Graph𝐺: 𝑉, 𝐸,𝑊 withverticesV,edgesEandweightmatrixW• Labelednodes𝑉'() ⊂ 𝑉,unlabelednodes𝑉+, = 𝑉 ∖ 𝑉'()• 𝒴 thesetofm possiblelabels (e.g.,𝒴={republican,democrat})• 𝑌'() = 𝑦D, … , y' thelabelsonlabelednodesin𝑉'()

§ Problem:Inferlabels𝑌+, forallnodesin𝑉+,


G2

1

1

𝑉'()

𝑉+,

? ?

?

?

𝒴={1,2}

9

Node classification problem (2)

§ Canbegeneralizedtomultilabel andmulticlassclassification:• Withmulticlassclassificationassumethateachlabelednodehasaprobabilitydistributiononthelabels.

§ Canworkongeneralizedgraphstructures• hypergraphs,graphswithweighted,labeled,timestampededges,multigraphs,probabilisticgraphsandsoon.


Influential factors on Networks

§ Individualbehaviorsarecorrelatedinanetworkenvironment

Homophily Influence Confounding


Beingrepublican→ Friendship

Friendship→ Sameshoes

BorninBerlin→ (individual)Likeelectronicmusic(connection)participatetomarathon

Externalfactor

The importance of the graph structure

§ Thegraphstructureencodesimportantinformationfornodeclassification

§ Soitisreasonabletothinkthatlabelspropagateinthenetworkfollowingthelinks

§ Methodsthatworkwithpointsinthespaceperformpoorlyinagraph


Assumption:Thelabelpropagatesonthenetwork

12

Node features

§ Nodefeatures:measurablecharacteristicsofthenodesthathelpdiscriminatinganodefromanotherorstatingthethesimilaritywithothernodes.

§ Examplesoffeatures:• In/outdegreeofthenode• Numberofl-labelededgesfromthatnode• Numberofpathsinthatgoesthroughthenode• Numberoftriangles• Degreeandnumberwithinego-netedges• …


Node classification approaches

§ Similaritybased• Findnodesthatsharethesamecharacteristicswithothernodes

§ Iterativelearning• Learnasetoflabelsandpropagatetheinformationtosimilarnodes

§ Labelpropagation• Labelednodespropagatetheinformationtotheneighborswithsomeprobability


Lecture road

Similaritybased

Iterativeclassification


Labelpropagation

15

Real-world Applications


Movies recommendations


Search Engines (IR)Topical Sessions

“popularmusicvideos”

QueriesURLs

“music”

“yahoo”

similar


Similarity based approaches

§ Equivalencesintermsofstructure• Structural,Automorphic,andRegular

§ Roleextractionmethods:• RolX

§ Recursivesimilarities• Paths,Max-flow,SimRank


History: Equivalences


Twonodesuandvarestructurally equivalentiftheyhavethesamerelationshipstoallothernodes

(Lorrainandwhite,F.,1971)

Twonodesuandvareautomorphically equivalentifallthenodescanberelabeled toformanisomorphicgraphwiththe

labelsofuandvinterchanged(justchangethenodeid)(Borgatti andEverett,1992)

Twonodesuandvareregularly equivalentiftheyareequallyrelatedtoequivalentothers

(Borgatti andEverett,1992)

Regular equivalence

Borgatti,S.P.andEverett,M.G.,1992.Regularblockmodels ofmultiway,multimodematrices.SocialNetworks.


§ Assumesasimilaritybetweensetsofnodes

Billy John

Prof.Einstein Prof.Hilbert

Professors

Students

BillyandJohnaresimilarbecausetheyarebothconnectedtoaprofessor.Sameforprof.EinsteinandHilbert

Regularequivalencedoesn’tcareaboutwhichconnectionsbuttowhichset/groupanodeisconnected

21

Twonodesuandvareregularly equivalentiftheyareequallyrelatedtoequivalentothers

(Borgatti andEverett,1992)

Relation among equivalences


Whatistherelationamongthethreeequivalences?

Regular

Automorphic

Structural

22

RolX: Role eXtraction algorithm

Henderson,K.,Gallagher,B.,Eliassi-Rad,T.,Tong,H.,Basu,S.,Akoglu,L.,Koutra,D.,Faloutsos,C.andLi,L.,2012.Rolx:structuralroleextraction&mininginlargegraphs.SIGKDD


Adjacencymatrix(𝑛×𝑛)

RecursiveFeatureExtraction

NodexFeaturematrix

RoleExtraction

NodexRolematrix

RolexFeatureMatrix

Input

Output

ndimensionalspace

r dimensionalspace

ddimspace

Non-negativematrixfactorization(NMF)

23

Recursive Feature extraction (ReFex)

Henderson,K.,Gallagher,B.,Li,L.,Akoglu,L.,Eliassi-Rad,T.,Tong,H.andFaloutsos,C.,2011.It'swhoyouknow:graphminingusingrecursivestructuralfeatures.SIGKDD.


§ Transformthenetworkconnectivityintorecursivestructuralfeatures.

§ Technically,embedsthegraphintoan|ℱ| dimensionalspace,whereℱisasetoffeatures(degree,self-loops,avg edgeweight,#ofedgesinegonet)

24

ReFeX: mining features

§ Local:• Measuresofthenodedegree

§ Egonet:• Theegonet (orego-network)ofanodeisthenodeitself,theadjecent nodes,andthegraphinducedbythosenodes

• Computedbasedoneachnode’segonetwork:#ofwithin-egonet edges,#ofedgesentering&leavingtheegonet

§ Recursive• Someaggregate(mean,sum,max,min,…)ofanotherfeatureoveranode’sneighbors

• Theaggregationcanbecomputedoveranyreal-valuedfeature,includingotherrecursivefeatures(thisprocessmightnotstopifuncontrolled!!!)


Neighborhood

Regional

25

ReFex (2)

§ Numberofpossiblerecursivefeaturesisinfinite§ ReFeX pruning• Featurevaluesaremappedtosmallintegersviaverticallogarithmicbinning• Logbinning:discretizethefeaturestakingnonuniform(butlogarithmic)bins=thefirstp|V|nodeswiththelowestfeaturevalueareassignedtobin0,thandividetheremaining|V|- p|V|takingthefirstp(|V|- p|V|)nodesandsoon.

• Logarithmicbinningincreasethechancestwofeatures

§ Lookpairsoffeatureswhosevaluesneverdisagreeatanynodebymorethanathresholds,andconnectinagraph.Foreachcomponenttakeonefeature.

§ Agraphbasedapproach(motivatedbypowerlawdistribution)§ Thresholdautomaticallyset


RolX: Role eXtraction algorithm

Henderson,K.,Gallagher,B.,Eliassi-Rad,T.,Tong,H.,Basu,S.,Akoglu,L.,Koutra,D.,Faloutsos,C.andLi,L.,2012.Rolx:structuralroleextraction&mininginlargegraphs.SIGKDD


Adjacencymatrix(𝑛×𝑛)

RecursiveFeatureExtraction

NodexFeaturematrix

RoleExtraction

NodexRolematrix

RolexFeatureMatrix

Input

Output

ndimensionalspace

r dimensionalspace

ddimspace

Non-negativematrixfactorization(NMF)

27

Role extraction: Feature grouping

§ Findr overlappingclustersinthefeaturespace• Eachnodecanhavemultiplerolesatthesametime

§ GeneratearankrapproximationofthenodexfeaturematrixV§ Usenon-negativematrixfactorization:𝑉 ≈ 𝐺𝐹

§ TheGmatrixassignsnodestoroles§ TheFmatrixrepresentshowthefeaturesexplaintheroles


A (very brief) glimpse to matrix factorization

§ IfVisamatrix,itispossibletofindanapproximationofthismatrixmultiplyingtwo(lowerrank)matrices

§ Inparticular,wewanttofindtwomatricesG,F,suchthatV ≈ 𝐺𝐹

Example: 3 46 8 = 1

2 34

§ However, theexactfactorizationisnotalwayspossible!!!§ Idea:letusfindG, 𝐹, withG ≥ 0, 𝐹 ≥ 0 suchthat

argminZ,[

𝑉 − 𝐺𝐹 [

where ⋅ [ istheFrobenius norm§ Intuitively:youwanttominimizethedifferencebetweenthe

singleelementsofthematrixV andtheproductGF


?

§ Rolessummarizethebehaviororalternatively,theycompressthefeaturematrixV(lowerdimensiondescription)

§ Whatisthebestmodel?

§ Idea:usetheMinimumDescriptionLength(MDL)paradigmtoselectthenumberofrolesthatresultsinthebestcompression• L:descriptionlength• M:#ofbitstodescribethemodel• E:costofdescribingtheerrorsin𝑉 − 𝐺𝐹• Findrsuchthatitminimizes𝐿 = 𝑀 + 𝐸

§ Minimize𝑀 + 𝐸• Assuminganyvaluerequiresbbits(therolevalues),thanthenumberofbitsforMis𝑀 = 𝑏𝑟(𝑛 + 𝑓),why?(thinkaboutthedimensionofthematrices)

• WhataboutE?Eistheamountoferrors.However,since𝑉 − 𝐺𝐹 isnotnormallydistributed,RolX usestheKLdivergence𝐸 = ∑ 𝑉e,f log

gh,i(Z[)h,i

− 𝑉e,f + (𝐺𝐹)e,f�e,f

Thebestmodelistheonethathasfewererrors andrequireslessspace

Selecting the right number of roles


node similarity and classification - hasso-plattner-institut · node similarity and classification...

Documents