community detection in complex...

24
Community detection in complex networks Vinh Loc DAO

Upload: others

Post on 23-Aug-2020

2 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Community detection in complex networksdepartements.imt-atlantique.fr/data/lussi/matinale/slides/slide-16_05... · Summary 1 Introduction What is a network or a graph? Some notions

Community detection incomplex networks

Vinh Loc DAO

Page 2: Community detection in complex networksdepartements.imt-atlantique.fr/data/lussi/matinale/slides/slide-16_05... · Summary 1 Introduction What is a network or a graph? Some notions

Summary

1 Introduction

2 Datasets and Benchmarks

3 Community detection in network

4 Evaluating partition quality

5 Objectives and Perspectives

2/24 04/05/2016 Vinh Loc DAO Community detection in Complex Networks

Page 3: Community detection in complex networksdepartements.imt-atlantique.fr/data/lussi/matinale/slides/slide-16_05... · Summary 1 Introduction What is a network or a graph? Some notions

Summary

1 Introduction

What is a network or a graph ?

Some notions

Structure properties of real networks

2 Datasets and Benchmarks

3 Community detection in network

4 Evaluating partition quality

5 Objectives and Perspectives

3/24 04/05/2016 Vinh Loc DAO Community detection in Complex Networks

Page 4: Community detection in complex networksdepartements.imt-atlantique.fr/data/lussi/matinale/slides/slide-16_05... · Summary 1 Introduction What is a network or a graph? Some notions

What is a network or a graph ?

Example : Internet, transport network, power grid, food web, social network

Figure – A very simple network G(V ,E) with |V | Vertices and |E | Edges

Node : Entity in real life

Edge : Relation between two entities to which it connects

A natural language to describe complex systems.

4/24 04/05/2016 Vinh Loc DAO Community detection in Complex Networks

Page 5: Community detection in complex networksdepartements.imt-atlantique.fr/data/lussi/matinale/slides/slide-16_05... · Summary 1 Introduction What is a network or a graph? Some notions

A local mesh network - very small network

Node : Computer

Edge : Connection between two computer

5/24 04/05/2016 Vinh Loc DAO Community detection in Complex Networks

Page 6: Community detection in complex networksdepartements.imt-atlantique.fr/data/lussi/matinale/slides/slide-16_05... · Summary 1 Introduction What is a network or a graph? Some notions

Student network in a university - relatively small network

Node : Student

Edge : Relation (Could be any kind of relation)

6/24 04/05/2016 Vinh Loc DAO Community detection in Complex Networks

Page 7: Community detection in complex networksdepartements.imt-atlantique.fr/data/lussi/matinale/slides/slide-16_05... · Summary 1 Introduction What is a network or a graph? Some notions

The French highway network - large network

Node : City

Edge : Highway

7/24 04/05/2016 Vinh Loc DAO Community detection in Complex Networks

Page 8: Community detection in complex networksdepartements.imt-atlantique.fr/data/lussi/matinale/slides/slide-16_05... · Summary 1 Introduction What is a network or a graph? Some notions

The Internet - international network

8/24 04/05/2016 Vinh Loc DAO Community detection in Complex Networks

Page 9: Community detection in complex networksdepartements.imt-atlantique.fr/data/lussi/matinale/slides/slide-16_05... · Summary 1 Introduction What is a network or a graph? Some notions

Some notions

Complex network : Graph with non-trivial topology features

Network analysis : Studies of graph to extract non-trivial features

Community detection algorithm : Divide nodes into groups calledcommunities whose members are connected densely.

Figure – Uncover graph modules without specifying clusters’ size

9/24 04/05/2016 Vinh Loc DAO Community detection in Complex Networks

Page 10: Community detection in complex networksdepartements.imt-atlantique.fr/data/lussi/matinale/slides/slide-16_05... · Summary 1 Introduction What is a network or a graph? Some notions

Structure properties of real networks

- Graph representing real systems are normaly neither regular nor random.- Degree (nb of connections of a node) distribution often follows a power law,as connections often follow preferential patterns.- Nodes are often found to cluster into high density groups.

Figure – Regular lattice graph Figure – Graph with modular structure

10/24 04/05/2016 Vinh Loc DAO Community detection in Complex Networks

Page 11: Community detection in complex networksdepartements.imt-atlantique.fr/data/lussi/matinale/slides/slide-16_05... · Summary 1 Introduction What is a network or a graph? Some notions

Why community detection ?

- Comprehend network global organization- Reveal modular structures- Reveal hidden properties between nodes- Understand information diffusion process throughout network

APPLICATIONS :- Detect web clients with similar interests- Prevent epidemic transmission- Managing collaboration network- etc,- You name it

11/24 04/05/2016 Vinh Loc DAO Community detection in Complex Networks

Page 12: Community detection in complex networksdepartements.imt-atlantique.fr/data/lussi/matinale/slides/slide-16_05... · Summary 1 Introduction What is a network or a graph? Some notions

Summary

1 Introduction

2 Datasets and Benchmarks

Datasets - Real networks

Datasets - Artificial networks

3 Community detection in network

4 Evaluating partition quality

5 Objectives and Perspectives

12/24 04/05/2016 Vinh Loc DAO Community detection in Complex Networks

Page 13: Community detection in complex networksdepartements.imt-atlantique.fr/data/lussi/matinale/slides/slide-16_05... · Summary 1 Introduction What is a network or a graph? Some notions

Datasets - Real networks

- Social network of friendships between 34 members of a karate club at a USuniversity in the 1970s. (left)- An identity graph with 25 vertices and 31 edges. An identity graph has asingle graph auto-morphism, the trivial one. (right)

Figure – Zachary karate network Figure – Walther network

13/24 04/05/2016 Vinh Loc DAO Community detection in Complex Networks

Page 14: Community detection in complex networksdepartements.imt-atlantique.fr/data/lussi/matinale/slides/slide-16_05... · Summary 1 Introduction What is a network or a graph? Some notions

Datasets - Artificial networks

Parameters : Graph size, node distribution, link distribution, densitydistribution, etc.

Figure – GN benchmark network Figure – LFR benchmark network

14/24 04/05/2016 Vinh Loc DAO Community detection in Complex Networks

Page 15: Community detection in complex networksdepartements.imt-atlantique.fr/data/lussi/matinale/slides/slide-16_05... · Summary 1 Introduction What is a network or a graph? Some notions

Summary

1 Introduction

2 Datasets and Benchmarks

3 Community detection in network

Dense structure in modular network

From dense structure to community

4 Evaluating partition quality

5 Objectives and Perspectives

15/24 04/05/2016 Vinh Loc DAO Community detection in Complex Networks

Page 16: Community detection in complex networksdepartements.imt-atlantique.fr/data/lussi/matinale/slides/slide-16_05... · Summary 1 Introduction What is a network or a graph? Some notions

An example of graph partitioning by K-means

Apply K-means graph partitioning on karate club network (K = 2)

Figure – The Zachary karate networkFigure – K-means partition on theZachary karate club

- Good solution ?- How do we know the club has 2 communities ?- Wait, tell me again what is a community ?

16/24 04/05/2016 Vinh Loc DAO Community detection in Complex Networks

Page 17: Community detection in complex networksdepartements.imt-atlantique.fr/data/lussi/matinale/slides/slide-16_05... · Summary 1 Introduction What is a network or a graph? Some notions

Did we say a ”community” ? Isn’t it a ”cluster” ?

- What is a community ? - Answer : ”I know it when I see it”- No universal accepted definition.- More edges inside the community than edges linking its vertices with the restof the network.- Many detection methods : overlapping/non-overlapping, fast/slow,single-scale/multi-scale.

Figure – A graph division

Figure – The karate club is separateddue to a conflict coach/president

17/24 04/05/2016 Vinh Loc DAO Community detection in Complex Networks

Page 18: Community detection in complex networksdepartements.imt-atlantique.fr/data/lussi/matinale/slides/slide-16_05... · Summary 1 Introduction What is a network or a graph? Some notions

Summary

1 Introduction

2 Datasets and Benchmarks

3 Community detection in network

4 Evaluating partition quality

5 Objectives and Perspectives

18/24 04/05/2016 Vinh Loc DAO Community detection in Complex Networks

Page 19: Community detection in complex networksdepartements.imt-atlantique.fr/data/lussi/matinale/slides/slide-16_05... · Summary 1 Introduction What is a network or a graph? Some notions

Partition quality

- What is a good partition of a network into modules ?

- Quality function : assigns score to each partition of a graph- The most popular quality function is modularity

Q = 12|V |Σij(Aij − Pij)δ(Ci ,Cj)

Aij : Adjacency matrix, Pij : Expected connection adjacency matrixQ = Fraction of edges within communities - expected fraction of such edgesin a randomModularity favors inter community links.

19/24 04/05/2016 Vinh Loc DAO Community detection in Complex Networks

Page 20: Community detection in complex networksdepartements.imt-atlantique.fr/data/lussi/matinale/slides/slide-16_05... · Summary 1 Introduction What is a network or a graph? Some notions

Community detection on the karate club

Edge betweenness O(|V ||E |2)

Fast greedy O(|V ||E |log(|V |))

Label propagation O(|V |+ |E |)Louvain method O(|V |log(|V |)

Leading eigenvector O(|V |2 + |E |)Modular optimization (NP-complete)

Infomap O (|V |(|V |+ |E |))

Walktrap O(|E ||V |2)

20/24 04/05/2016 Vinh Loc DAO Community detection in Complex Networks

Page 21: Community detection in complex networksdepartements.imt-atlantique.fr/data/lussi/matinale/slides/slide-16_05... · Summary 1 Introduction What is a network or a graph? Some notions

Community detection on the Walther network

Many possible meaning divisions on a less modular network.

21/24 04/05/2016 Vinh Loc DAO Community detection in Complex Networks

Page 22: Community detection in complex networksdepartements.imt-atlantique.fr/data/lussi/matinale/slides/slide-16_05... · Summary 1 Introduction What is a network or a graph? Some notions

Quality is relative ? Goodness is subjective

- Community characteristics :

Community density

Community connectiveness

Robustness to perturbation, etc

Question : How to choose appropriate method to satisfy certaincharacteristics and utilizing as much as possible available information ?

Ex : Which method to chose to :

- Divide students into the most cohesive groups.

- Establish geographic sites to minimize remote works of collaborators.

- Compromise between community density and calculation time.

- Maximize range of ages in a dancing community.

22/24 04/05/2016 Vinh Loc DAO Community detection in Complex Networks

Page 23: Community detection in complex networksdepartements.imt-atlantique.fr/data/lussi/matinale/slides/slide-16_05... · Summary 1 Introduction What is a network or a graph? Some notions

Summary

1 Introduction

2 Datasets and Benchmarks

3 Community detection in network

4 Evaluating partition quality

5 Objectives and Perspectives

23/24 04/05/2016 Vinh Loc DAO Community detection in Complex Networks

Page 24: Community detection in complex networksdepartements.imt-atlantique.fr/data/lussi/matinale/slides/slide-16_05... · Summary 1 Introduction What is a network or a graph? Some notions

Objectives and Perspectives

Guide to choose best methods base on expected quality indicators, graphcharacteristics and available resources

Create generative model to summarize community characteristics

Create a benchmark base on generative model

Construct criteria for evaluating partition quality base on end user pointof view

Propose methods to improve detection quality

Tools :- R for network analyzing, data analyzing, visualization- Gephi : Visualization

24/24 04/05/2016 Vinh Loc DAO Community detection in Complex Networks