Semi-Supervised Feature Selection for Graph Classification

Xiangnan Kong Xiaoxiao Shi Philip S. YuMulti-Label Collective ClassificationUniversity of Illinois at Chicago

1Collective Classification Conventional classification approaches assume that instances are independent identically distributed (i.i.d.) x2y2x1y1labelinstanceindependentx2y2x1y1x3y3related In relational data and information networks, instances are correlated with each other. #2Example: Collective Classification

Training DataGiven a set of web pages linked with each other, we need to classify categories

Task: Predict the labels of webpages collectively, while considering dependencies among linked webpages

Test Datay1y2y5y6y4y3???????#3Examples

Coauthor Networks

Business Network#Collective ClassificationCollective ClassificationGiven a set of instances which are related to each other, how to predict their labels simultaneously

Existing Methods Exploit the dependencies among related instancesFocused on single-label settingsAssume one instance can only have one label

#

DMIRIRDB

DMAIDB

AIcollaborationsResearch AreaMulti-label Collective Classification

How to effectively predict the label sets of a group of related instances?#6The ProblemMultiple labels: # possible label sets is very large (the power set of all labels) 20 labels 1 million label setsRelational Data: the label sets of related instances are correlated with each other.The key is to exploit the correlations among the multiple labels#7Intra-instance Cross-label DependencyY1YmYkx1Y1YmYkx2Y1YmYkx3e.g. X1 is more likely to be DM, if labeled with DB or MLe.g. X1 is unlikely to be Bio, if labeled with OS

#8Inter-instance Single-label DependencyY1YmYkx1Y1YmYkx2Y1YmYkx3e.g. X1 is more likely to be DM, if collaborators (X2 X3) are labeled with DM

#9Inter-instance Cross-label DependencyY1YmYkx1Y1YmYkx2Y1YmYkx3e.g. X1 is more likely to be DM, if collaborators (X2 X3) are labeled with DB or ML

#10All dependencies: our approachY1YmYkx1Y1YmYkx2Y1YmYkx3

#11Relational Feature Aggregation1011relational features1111contentfeatures0111011101labels0001x1x2Y1Y3Y2Y1Y3Y22Inter-Instance Single-Label3Inter-Instance Cross-LabelIntra-Instance Cross-Label112310111010111001#12Iterative Classification ofMultiple Labels Inferenceonly use Content FeaturesUpdate label setsUpdate Relational FeaturesInitialize label setsusing predicted label setsusing content feature + relational feature#The ICML ApproachSimple & Efficient: train multiple local models to perform collective classification on multiple labelsProperties:Effective: By considering the dependencies among related instances and multiple labels, the classification performance can be greatly improved over independent models.#14DependenciesExploited Experiments:Compared MethodsBinary classificationBinary SVM binary decomposition + SVM [Boutell et.al., PR04] none 12121231Intra-Instance Cross-Label2Inter-Instance Single-Label3Inter-Instance Cross-LabelMulti-label collective classificationML-ICA a proposed baseline [this paper]ICMLthe proposed approach [this paper] Multi-label classification ECC & CC ensemble + classifier chains [Read et.al., ECML09]Collective classificationICA iterative classification algorithm [Lu&Getoor, ICML03] #Research Collaboration Networks (DBLP)Node: ResearcherFeatures: bag-of-words for paper titlesLink: CollaborationLabel: Research Area (DB, AI, IR, OS, etc)Movie Database (IMDB)Node: movieFeatures: bag-of-words for movie plotLink: share directorLabel: movie type (comedy, horror, etc)Experiments:Data Sets

#EvaluationMulti-Label MetricsHamming Loss [Elisseef&Weston NIPS02] average #labels being misclassifiedSubset 0/1 Loss [Ghamrawi&McCallum CIKM05] average #label sets being misclassifiedMicro-F1 [Ghamrawi&McCallum CIKM05] micro average of F1 scoreMacro-F1 [Ghamrawi&McCallum CIKM05] macro average of F1 score

the smaller the better the larger the better

5-fold cross-validation

#Experiment ResultsDBLP-A DatasetY1YmYkx1Y1YmYkx2#Experiment ResultsDBLP-A Dataset1Intra-Instance Cross-LabelY1YmYkx1Y1YmYkx2#Experiment ResultsDBLP-A DatasetY1YmYkx1Y1YmYkx22Inter-Instance Single-Label#Experiment ResultsDBLP-A Dataset1Intra-Instance Cross-Label2Inter-Instance Single-LabelY1YmYkx1Y1YmYkx2#Experiment ResultsDBLP-A Dataset1Intra-Instance Cross-Label2Inter-Instance Single-Label3Inter-Instance Cross-LabelY1YmYkx1Y1YmYkx2#Experiment ResultsDBLP-B Dataset#Experiment ResultsIMDB DatasetOur approach performed best at DBLP and IMDB datasets#Experiment Results#IterationICML approachDBLP-A dataset#ConclusionsMulti-label Collective ClassificationPropose an algorithm to exploit the dependencies among label sets of related instancesIntra-instance Cross-label DependencyInter-instance Single-label DependencyIntra-instance Cross-label DependencyClassification performances can be improved by considering the dependencies among instances and different labels.

Thank you!#