Complex Network Classification with ConvolutionalNeural NetworkRuyue Xin1, Jiang Zhang1,*, and Yitong Shao2

1School of Systems Science, Beijing Normal University, Beijing, 100875, P. R. China2School of Mathematical Sciences, Beijing Normal University, Beijing, 100875, P. R. China*zhangjiang@bnu.edu.cn

ABSTRACT

Classifying large scale networks into several categories and distinguishing them according to their fine structures is of greatimportance with several applications in real life. However, most studies of complex networks focus on properties of a singlenetwork but seldom on classification, clustering, and comparison between different networks, in which the network is treated asa whole. Due to the non-Euclidean properties of the data, conventional methods can hardly be applied on networks directly.In this paper, we propose a novel framework of complex network classifier (CNC) by integrating network embedding andconvolutional neural network to tackle the problem of network classification. By training the classifiers on synthetic complexnetwork data and real international trade network data, we show CNC can not only classify networks in a high accuracy androbustness, it can also extract the features of the networks automatically.

IntroductionComplex network is a highly simplified model of a complex system, and it has been widely applied in many fields, suchas sociology, economics, biology, and so on. However, most of current studies focus on the properties of a single complexnetwork1, but seldom pay attention to the comparisons, classifications, and clustering problems of different networks, eventhough these problems are also important.

For example, classifying different online groups and communities into healthy and non-healthy classes according to theirsocial network structures is of great importance for diagnosing the development of the community. As another example, acomplex network classifier can help us to classify a new product into an appropriate category according to the internationaltrade structure of the new product, and it may alleviate the arbitrariness and subjectivity of the conventional classificationmethods by hand. In addition, the classification problem of complex networks can be easily extended to prediction problems.Thus, by the similar method, we can predict the development of a country, a company, and so forth complex systems accordingto their backbone network structures.

However, seldom approaches can be applied on complex network classification problem in so far although the deeplearning technology has achieved its state-of-art results in traditional data such as convolutional neural network2 (CNN)can process images data, and recurrent neural network3 (RNN) can be used in natural language processing. Although theclassification technique for graph data has been developed4, such as random walk kernels5, shortest-path kernels6, graphletkernels7, and Weisfeiler-Lehman graph Kernels8, they can be hardly applied on large-scale and complex networks due to thelarge computational complexity of these graph classification methods.

In recent years, network representation learning, such as DeepWalk9, node2Vec10 and LINE11, emerged as an alternativeeffective way to solve the problem of standardizing network data. However, these methods can only be applied on the tasks(classification, community detection, and link prediction) on nodes but not the whole networks. Another way to deal withgraph data is based on the geometric deep learning12 (GDL) methods, e.g., GNN13, GGSNN14, GCN15 etc. Nevertheless, thesemethods can also focus on the tasks on node level but not the graph level. Another shortage of GDL is the requirement of thefixed network structure background.

In this paper, we proposed a new method on complex network classification problem, which is called as a complexnetwork classifier(CNC), by combining network embedding and CNN techniques together. We first embed a network into ahigh-dimensional space through the Deepwalk algorithm, which preserves the local structures of the network and convert it intoa 2-dimensional image. Then, we input the image into a CNN for classifying. Our model framework has the merits of the smallsize, small computational complexity, the scalability to different network sizes, and the automaticity of feature extraction.

arX

iv:1

802.

0053

9v1

[cs

.CV

] 2

Feb

201

8

MethodsOur strategy to classify complex networks is to convert networks into images, and use the standard CNN model to perform thenetwork classification task. Due to the development of network representation techniques, there are a bunch of algorithms toembed the network into a high dimensional Euclidean space. We select DeepWalk algorithm9, which is proposed by BryanPerozzi et al. to obtain the network representation. The algorithm will generate numeric node sequences by performinglarge-scale random walks on the network. After that, the sequences are fed into the SkipGram + Negative Sampling algorithmto obtain the Euclidean coordinate representation of each node.

Obviously high-dimensional space representation is hard to be processed, thus we use the PCA algorithm to reduce thedimension of node representations in two-dimensional space. However, the set of nodes is a point cloud which is still irregularand cannot be processed by CNN, thus we rasterize the 2-dimensional representation into an image. We divide all the areascovered by the two-dimensional scatter plot into a square area with 48∗48 grids and then count the number of nodes in eachgrid as the pixel grayscale. After that, a standard gray scale image is obtained. The reason why we do not embed the networkinto 2-dimensional space directly is because we believe that doing so may lose less information, particularly for the localstructures. This method can also be applied on directed and weighted networks like international trade flow networks. Byadjusting the probabilities according to the weight and direction of each edge for a random walker on a network, we can obtainan embedded image.

a b c

d

Figure 1. The pipeline of CNC algorithm. (a) The original input network. (b) The embedding of the network with DeepWalkalgorithm. In DeepWalk algorithm, to obtain enough ”corpus”, we set the number of walks to 10000 times and the sequencelength to 10, and embed the network into the 20-dimensional space and reduce it to 2-dimentional space. (c) The rasterizedimage from the 2D-embedding representation of the netowrk. (d) The CNN architecture of CNC algorithm, which includes oneinput image, two convolutional-pooling layers, one fully-connected layer and one output layer. The sizes of the convolutionalfilters are 5∗5, of the pooling operation is 2∗2. The first layer has 3 convolutional filters, and the second layer has 5convolutional filters, and the fully connected layer has 50 units. In all complex network classification experiments, we set thelearning rate = 0.01 and mini-batch = 100. The CNN architecture is selected as mentioned to minimize the computationalcomplexity as well as keeping the classification accuracy.

2/8

The final step is to feed the representative images into a CNN classifier to complete the classification task. Our convolutionalneural network architecture includes two convolutional layers (one convolutional operation and one max-pooling operation)and one fully-connected layer and one output layer. The whole architecture of our model can be seen in Fig.1(bottom).

Experimental data

A large number of experimental data is needed to train and test the classifier, thus we use both synthetic networks generated bynetwork models and empirical networks to test our model.

Synthetic data

The synthetic networks are generated by well known BA and WS models. According to the evolutionary mechanism of BAmodel, which iteratively adds m = 4 nodes and edges at each time, and the added nodes will preferentially link to the existingnodes with higher degrees until n=1000 nodes are generated, and the average degree of the generated network is about 8 whichis close to the degree of real networks16. We then use WS model (n = 1000, the number of neighbors of each node k = 8, andthe probability of reconnecting edges p = 0.1) to generate a large amount of small-world networks with the same averagedegrees as in BA model.

We then mix the generated 5600 BA networks and WS networks, respectively. And we separate the set of networks intotraining set (with 8000 networks), validation set (with 2000 networks), and test set (with 1000 networks).

Empirical data

Product specific international trade networks are adopted as the empirical data to test our classifier, the dataset is provided bythe National Bureau of Economic Research (http://cid.econ.ucdavis.edu/nberus.html) and covers the tradevolume between countries of more than 800 different kinds of products which are all encoded by SITC4 digits from 1962to 2000. Notice that the international trade network is a weighted directed network, in which the weighted directed edgesrepresent the volumes of trading flows between two countries. Thus, the random walk in DeepWalk algorithm should be basedon the weights and directions of edges. We train the CNC to distinguish the agricultural products and industrial products. Eachproduct class contains about 10000 networks obtained by the products and the products combinations within the category.

Results

We conduct a large number network classification experiments, and the results are present in this section. On the syntheticnetworks, we not only show the classification results, but also present how the CNC can extract the features of networks, andthe robustness of the classifier on network sizes. On the empirical networks, we show the results that our CNC apply on thetrade flow networks which are directed weighted networks.

Classification experiments on synthetic networks

BA and WS classification experiments

The first task is to apply CNC to distinguish BA network and WS network. Although we know the BA network is a scale-freenetwork, and WS network is a small-would network with high clustering coefficient, machine does not know. Thus this seriesexperiments show the possibility that the CNC network can extract the key features to distinguish the two kinds of networks.We generate 5600 BA networks with N = 1000, m = 8 and 5600 WS networks with the same size and p = 0.1, respectively.And we mix these networks to form the data set which is further randomly separated into training set (with 8000 networks),validation set (with 2000 networks), and test set (with 1000 networks). Figure 6(a) shows the decay of the loss on training setand error rate of validation set. Finally, we obtain the average error rate 0.1% on the test set. So we can say the model candistinguish the BA network and the WS network accurately.

3/8

a

b c

Figure 2. (a) The 2D representations and rasterized images of a BA network (upper) and a WS network(bottom). (b)Visualization of the three filters of the first convolutional layer. (c) Visualization of the five filters (size of 5∗5∗3) of thesecond convolutional layer.

Figure 3. We show the active nodes corresponding to the highlighted areas in the feature maps of the 3 filters of the firstconvolutional layer when inputting a typical BA network and WS network respectively. We draw the activated nodes (the greenpoints) and their links with other nodes as the background for the two networks. (Upper: scale-free network. Bottom:small-world network).

To understand what has been learnt by our CNC model, we can visualize the feature maps extracted from the network

4/8

representations by the filters of the CNN, which are visualized in Fig.2. However, it is hard to read meaningful informationbecause the network structure cannot be corresponded to the images.

To understand what the filters do, we need combine the network structure and the feature map. Therefore, we try to mapthe highlighted areas in feature maps of each filter on the nodes sets of the network. That is, we wonder which parts of thenetworks and what kind of local structures are activated by the first layer filters. We compare the activation nodes for thetwo model networks as input, and the results are shown in Fig.3. By observing and comparing these figures, we find that theconvolutional filters of the first layer has learnt to extract the features of the network in different parts. As shown in Fig.3, Filter0 is extracting the local clusters with medium density of nodes and connections; and Filter 1 tries to extract the local clusterswith sparse connections; while Filter 2 tries to extract the local clusters with dense nodes and connections.

By comparing BA and WS model networks, we can observe that the locations and the patterns of the highlighted areas aredifferent. The local areas with dense nodes and connections (Filter 0) locate the central area of the network representation forboth BA network and WS network. The local structures with sparse nodes and connections locate the peripheral area which isclose to the edges of the image for the WS network, but it is in the central area for the BA network. This combination of theactivation modes on feature maps can help the higher level filters and fully connected layer to distinguish the two kinds ofnetworks.

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0p

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1.0

p

0.0

0.1

0.2

0.3

0.4

0.5

(Erro

r rat

e)

Figure 4. The confusion matrix of the actual p class (horizontal) and the predicted p class(vertical). The colors represent theerror rates.

Small world networks classificationOne may think to distinguish the BA and WS networks is trivial because they are two different models at all. Our secondexperiment will consider whether the classifier can distinguish networks generated by different parameters of the same model,which is harder than the previous task.

In order to verify the discriminant ability of the model on this task, we use the WS model to generate a large number ofexperimental networks by changing the value of edge reconnection probability p from 0 to 1 in a step of 0.1, and then wemix the networks with two discriminant p values, eg. p = 0.1 and p = 0.6, and we train the CNC for networks, and test theirdiscriminant ability on the test sets.

We systematically do this experiment for any combination of the networks with each two probabilities, and the results areshown in Fig.4. We can see that the networks generated by p values less than 0.3 and p values greater than or equal to 0.4are easier to be distinguished. Interestingly, there is a sudden change for the error rate at p = 0.4. For the two networks withp > 0.4, the classifier cannot distinguish them. The reason behind this phenomenon may be due to the phase transition of thelink percolation in random networks because the WS networks with p > 0.5 may be treated as random networks.

5/8

Classification on trade flow networksWe want to verify the effectiveness of the model on empirical networks. We conduct a classification on international trade flownetworks with the dataset obtained from the National Bureau of Economic Research (http://cid.econ.ucdavis.edu/nberus.html). This data covers the trade volume and direction information between countries of more than 800 differentkinds of products which are all encoded by SITC4 digits from 1962 to 2000. We select food and chemical products as twolabels for this experiment, and their SITC4 encoding starts with 0 and 5 respectively. For example, 0371 is for Fish, prepared orpreserved and 5146 is for oxygen-function amino-compounds. Fig.5 shows the 2D representation of the 10 products for twocategories. After pre-processing, the number of the food trade networks is 10705 (including products and product combinationswith SITC4 digits starting with 0) and the chemicals trade network is 10016 (including products and product combinations withSITC4 digits starting with 5). Then, we divide them into training set, validation set and test set according to the ratio of 9: 1: 1.During the training, we adjust the network parameters to 15 convolutional filters in the first layer and 30 convolutional filters inthe second layer, 300 units of the full-connect layer. Fig.6(b) shows that the classification error rate can be cut down to 5%.

Figure 5. Network representations of 10 selected products in two classes: food(upper) and chemicals(bottom).

0 10 20 30 40 50 60 70 80Iteration

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

Loss

Loss and Error

losserror

0.0

0.1

0.2

0.3

0.4

0.5

0.6

Error r

ate

a

0 5 10 15 20 25 30Epoch

0.0

0.1

0.2

0.3

0.4

0.5

0.6

Loss

Loss and Error

losserror

0.05

0.10

0.15

0.20

0.25

0.30

0.35

Error r

ate

b

Figure 6. Plots of loss and validation error rate of the classification task on BA v.s. WS models (a) and the classification taskfor food v.s. chemicals products (b).

Robustness on sizes of the networkOur model has good classification performances on both synthetic and empirical data. Next, we want to test the robustness ofthe classification on different sizes (numbers of nodes and edges). Note that all the experiments performed in classificationexperiments contain the model networks with identical numbers of nodes and edges. Nonetheless, a good classifier shouldextract the features which are independent on size. Therefore, we examine the robustness of the classifier on various networksizes which are different from the training sets. In these experiments, we first apply the trained classifier for BA and WSnetworks with N = 1000 nodes and < E >= 8 , on new networks different numbers of nodes and edges. We generate 600mixed networks by BA and WS models with parameters M from [1,2,3, · · · ,16] for the BA model and K from [2,4,6, · · · ,32]for the WS model as test set such that their average degrees are similar.

6/8

We systematically compare how the number of nodes (left) and edges(right) on the test sets influence the error rates asshown in Fig.7. At first, we observe that the error rates are almost independent on small fluctuations of the number of nodes.However, the error rates increase as larger size differences are in the test data. This manifest our classifiers are robust on thesize of the networks.

600 800 1000 1200 1400N (Number of nodes)

0.1

0.2

0.3

0.4

0.5

Error rate

Test errorTrain N=800Train N=1000Train N=1200

a

2 4 6 8 10 12 14 16M (Average number of edges)

0.05

0.10

0.15

0.20

0.25

0.30

0.35

0.40

Error rate

Test errorTrain M=4Train M=6Train M=8

b

Figure 7. The dependence of the error rates on the number of nodes(left) and the number of edges(right) in the robustnessexperiments. (a) In test set, we set N (number of nodes) = [500,600,700, · · · ,1500], and we also retrain N = 800 andN = 1200 and test them with the different N test set. (b) In test set, we set M (average number of edges) = [1,2,3, · · · ,16], andwe also retrain M = 4 and M = 6 and test them with the different M test set.

Nevertheless, there are sudden changes for the variants on the number of edges, which indicates that the number of edgeshas larger impacts on the network structure. We observe that there is a sudden drop on error rates with increase of M for thetest set when M=8 for the training set. Through observing the network embedding grow we know that he reason behind thissudden change is the emergence of the multi-center on the representation space for the BA model. Therefore, the numberof links can change the overall structure in the scale free network, and this makes our classifier working worse. Anotherinteresting phenomenon is the error rates can keep small when the number of edges increase when M in the training set is set to8. Therefore, the classifiers training on the dense networks are more robust on the variance on edge densities.

DiscussionIn this paper, we propose a model, which mainly incorporates Deepwalk and CNN, to solve the network classification problem.With Deepwalk, we obtain an image for each network, and then we use CNN to complete the classification task. Our method isindependent on the number of network samples, which is a big limitation for the kernel methods on graph classification. Wevalidate our model by experiments with the synthetic data and the empirical data, which show that our model performs well inclassification tasks. In order to further understand the network features extracted by our model, we visualize the filters in CNNand we can see that CNN can capture the differences between WS and BA networks. Furthermore, we test the robustness ofour model by setting different sizes for training and testing. The biggest advantage of our model is that our model can dealwith networks with different structures and sizes. In addition, the architecture of our model is small and the computationalcomplexity is low.

There are several potential improvements and extensions to our model that could be addressed as future works. For example,we can develop more methods to deal with the network features in high-dimensional space. Besides, we think that our modelcan be applied to more classification and forecasting tasks in various fields. Finally, we believe that extending our model tomore graph-structure data would allow us to tackle a larger variety of problems.

References1. Pastor-Satorras, R., Rubi, M. & Diaz-Guilera, A. Statistical Mechanics of Complex Networks (Springer Berlin Heidelberg,

2003).

2. Krizhevsky, A., Sutskever, I. & Hinton, G. E. Imagenet classification with deep convolutional neural networks. Commun.Acm 60, 2012 (2012).

7/8

3. Graves, A., Mohamed, A. R. & Hinton, G. Speech recognition with deep recurrent neural networks. In IEEE InternationalConference on Acoustics, Speech and Signal Processing, 6645–6649 (2013).

4. Yanardag, P. & Vishwanathan, S. V. N. Deep graph kernels. 1365–1374 (2015).

5. Kashima, H. Marginalized kernels between labeled graphs. In Proceedings of the Twentieth International Conference onMachine Learning, 321–328 (2003).

6. Borgwardt, K. M. & Kriegel, H. P. Shortest-path kernels on graphs. In IEEE International Conference on Data Mining,74–81 (2005).

7. Shervashidze, N., Vishwanathan, S. V. N., Petri, T. H., Mehlhorn, K. & Borgwardt, K. M. Efficient graphlet kernels forlarge graph comparison. Aistats 5, 488–495 (2009).

8. Shervashidze, N., Schweitzer, P., Leeuwen, E. J. V., Mehlhorn, K. & Borgwardt, K. M. Weisfeiler-lehman graph kernels. J.Mach. Learn. Res. 12, 2539–2561 (2011).

9. Perozzi, B., Alrfou, R. & Skiena, S. Deepwalk: online learning of social representations. 701–710 (2014).

10. Grover, A. & Leskovec, J. node2vec: Scalable feature learning for networks. 855 (2016).

11. Tang, J. et al. Line: Large-scale information network embedding. 2, 1067–1077 (2015).

12. Bronstein, M. M., Bruna, J., Lecun, Y., Szlam, A. & Vandergheynst, P. Geometric deep learning: Going beyond euclideandata. IEEE Signal Process. Mag. 34, 18–42 (2017).

13. Scarselli, F., Gori, M., Tsoi, A. C., Hagenbuchner, M. & Monfardini, G. The graph neural network model. IEEETransactions on Neural Networks 20, 61 (2009).

14. Li, Y., Tarlow, D., Brockschmidt, M. & Zemel, R. Gated graph sequence neural networks. Comput. Sci. (2015).

15. Defferrard, M., Bresson, X. & Vandergheynst, P. Convolutional neural networks on graphs with fast localized spectralfiltering. (2016).

16. Wang, X. F. & Chen, G. Complex networks: small-world, scale-free and beyond. IEEE Circuits & Syst. Mag. 3, 6–20(2003).

AcknowledgementsWe acknowledge the financial support for this work from the National Science Foundation of China with grant number61673070, “the Fundamental Research Funds for the Central Universities”, grant number 310421103, and Beijing NormalUniversity Interdisciplinary Project.

Author contributions statementJ.Z. and R.X. conceived the experiments and write the manuscript. R.X. collected and analysed the empirical data. R.X. andY.S. plotted the graphs. All authors reviewed the manuscript.

Additional informationTo include, in this order: Accession codes (where applicable); Competing financial interests (mandatory statement).

The corresponding author is responsible for submitting a competing financial interests statement on behalf of all authors ofthe paper. This statement must be included in the submitted article file.

8/8