from local network motifs to global invariantsngns/docs/review_2010... · onr muri: nexgenetsci...
TRANSCRIPT
ONR MURI: NexGeNetSci
From Local Network Motifs From Local Network Motifs to Global Invariantsto Global Invariants
Third Year Review, October 29, 2010
Victor M. Preciado and Ali JadbabaieDepartment of Electrical and Systems Engineering
University of Pennsylvania
Theory DataAnalysis
Numerical Experiments
LabExperiments
FieldExercises
Real-WorldOperations
• First principles• Rigorous math• Algorithms• Proofs
• Correct statistics• Only as good
as underlying data
• Simulation• Synthetic,
clean data
• Stylized• Controlled• Clean,
realworld data
• SemiControlled•Messy,
realworld data
• Unpredictable• After action
reports in lieu of data
Preciado
Local Motifs and Global Invariants
ONR MURI: NexGeNetSci
• Motivation and context
• The role of local structural information
• Spectral analysis from local structural information
• Bounds on spectral properties via optimization
• Implications in dynamical processes
Outline
ONR MURI: NexGeNetSci
Complex Network: Properties
• Generic features:– Large number of nodes
– Sparse connectivity
– Lack of regularity
• Examples:– Comm networks (e.g. Internet)
– Social networks (e.g. Facebook)
– Biological networks
• We assume limited structural information:– Privacy and/or security concerns
– Storage/computing limitations??
??
• Challenges when only local structural information is available:– Estimation: How could we aggregate local measurements to
infer global properties of the network?
– Inference: What could we say about the behavior of a dynamical process in the network from local measurements?
– Actuation: How could we modify the structure of a network to induce a desired global behavior?
Complex Networks: Some Challenges
Estimation
Inference
Actuation
ONR MURI: NexGeNetSci
• Overly focused on random graph models and degree distributions, but we can have very different networks with the same degree distribution [Li et al., 2005]:
• Main drawbacks:
1. Degree distributions are a zero-th order approximation of the network structure, by far not enough
2. Random models are difficult, if not impossible, to justify from an engineering perspective
Usual Approach in “Network Science”
ONR MURI: NexGeNetSci
• We also find random graph models capturing increasingly richer structural properties [Mahadevan et al., 2006]
• Main drawbacks:
– Visual inspection is clearly not enough to measure similarity
– What structural measurements are relevant in the behavior of dynamical processes in networks?
More Structured Random Models
Average degree Degree distribution Joint Degree Distribution
Distribution of Triangles Original HOT model
ONR MURI: NexGeNetSci
• Our framework: We consider the dynamical behavior of networks
• Since the eigenvalues and eigenvectors are closely related with thenetwork dynamical behavior, spectral graph theory is a convenientframework to study network dynamics
• Some relationships between spectra and dynamics are:– Spreading processes � adjacency spectral radius
– Synchronization � (combinatorial) Laplacian eigenratio
– Diffusion/Consensus � (normalized) Laplacian eigenvalues
Networks = Graphs + Dynamics
0 500 1000 1500 2000
0
500
1000
1500
2000
nz = 45572
-40 -20 0 20 40 60 800
200
400
600
800
���)
Aggregation of local structural measurements
Dynamical Implications
ONR MURI: NexGeNetSci
Inference from Local Measurements� Our problems:
1. - What measurements are most relevant in the behavior of dynamical processes?
2. - How can we aggregate local measurements to say somethingabout the global dynamical behavior?
• We study those problems in the framework of spectral graph theory and convex optimization, without making any assumption on the global network structure (i.e., no random models)
ONR MURI: NexGeNetSci
I. Use algebraic graph theory to relate the frequency of certain small subgraphs, or motifs, with the so-called spectral moments of the network
II. Propose a distributed technique to compute the frequency of subgraphs from the distribution of local network measurements
III. Use convex optimization to extract relevant spectral information from a sequence of spectral moments
IV. Study implications on dynamical processes
Structure of our Approach
ONR MURI: NexGeNetSci
• Algebraic graph theory allows us to compute spectral moments from local structural information. We use the following result:
• Low-order moments: For k≤3 we have the following expressions
I. From Subgraphs to Spectral Moments
� �klengthofwalksclosednn
Amn
i
kik #
11)(
1
�� ��
�
k=2
k=3
n
ed
nm
i i
212 �� �
nt
nm
i i
��� � 6
21
3
i
i
ONR MURI: NexGeNetSci
• Higher-order moments: As we increase the order of the moments, a variety of more and more complicated subgraphs come into the picture. For k=4, we have the following types of closed walks:
• In the first expression, we observe that local measurements can be aggregated via distributed consensus to compute spectral moments
From Subgraphs to Moments
i i i
j
i
iiiii
ii dddddqn
m ������ � )1()1(21
4
� �eQn
m 8481
4 ����
Moments from local structural measurements
Moments from local structural measurements
Moments from subgraphsfrequencies
ONR MURI: NexGeNetSci
• Key observation: The spectral moments are linear combinations of subgraphs embedding frequencies [Preciado, Jadbabaie, 2010]. The coefficients for all nonisomorphic connected subgraphs with 4 or less nodes are
• For example, the fifth moment can be computed as:
mk
k=4 2 4 - - - - 8 - -
k=5 - - 30 - - 10 - - -
k=6 2 12 24 12 6 - 48 36 -
k=7 - - 126 - - 84 - 112 -
k=8 2 28 168 72 32 64 264 464 528
Moments from Subgraphs Frequencies
� �en
m 3010301
5 �����
ONR MURI: NexGeNetSci
• We propose a distributed technique to compute subgraphfrequencies. Note that each subgraph can be ’discovered’ by a number of its nodes. For example, for 1-hop neighborhoods:
• In general, if each node have access to its r-hops neighborhood, wecan discover all subgraphs involved in moments of order up to 2r+1(and part of the subgraphs involved in moments of higher order)
II. Distributed Computation of Moments
� ���
ii
nn 31 �� �����
�i iii i d
nnn)2(
11 ��inn
Q???
1
ONR MURI: NexGeNetSci
• Subgraph with 2,404 nodes and 22,786 edges obtained from crawling the Facebook graph in a breadth-first search around a particular node
• We can compute the relevant quantities
which allow us to compute moments
100
101
102
103
10-4
10-3
10-2
10-1
k,Degree
P(k
)
Node Degree Distribution
100
101
102
103
104
10-4
10-3
10-2
10-1
t,Triangles
P(t
)
Distribution of Triangles
100
101
102
103
100
101
102
103
104
k,Degree
t,T
riang
les
Degree-Triangle Scatter Plot
Empirical Example
ONR MURI: NexGeNetSci
• So far, we have
• We now present an SDP-based approach to extract information fromthe spectral moments that
1. It is agnostic, in the sense that it does not make any assumptionon the global network structure (no random model)
2. It allows to study the effect of arbitrarily complicated structuralmeasurements in the network spectral properties
III. Extracting Spectral Info from Moments
Counting subgraph
frequencies
Computing spectral
moments?
ONR MURI: NexGeNetSci
• How can we extract information from spectral moments? The following problem, called the classical moment problem, is closely related to ours:
Given a sequence of moments (m1,…,mk), and Borelmeasurable sets T � ��� R, we are interested in computing
where m in M(�), M(�) being the set of positive Borelmeasures supported by �.
• Generalization of Markov and Chebyshev´s inequalities from probability theory, when a sequence of moments is available
The Classical Moment Problem
ONR MURI: NexGeNetSci
• Using duality theory, we obtain the following formulation [Bertsimas, 2005]:
This dual problem is a sum-of-squares program (SOSP) and can be formulated as a semidefinite program [Parrilo, 2006].
• We define the spectral distribution of a graph as
and define the r.v. X��G. Hence, using SOS, we can compute optimal bounds on Pr(X � T)=#{�i � T}/n when we haveaccess to a sequence of spectral moments
Moment Problems, SOS and SDP
ONR MURI: NexGeNetSci
• From the set of spectral moments, we compute optimal bounds on #{�i � [a,b]}/n, and #{�i � [-c,c]}/n
• Notice that only those intervals [a,b] in region B and [-c,c] in Care able to support the whole set of eigenvalues. Hence,– We have a lower bound on the spectral radius �(A)>�*
– We can also compute a bound on the Laplacian eigenratio fromthe Laplacian spectral moments
Numerical Results
a
bB
1
�* cC
ONR MURI: NexGeNetSci
IV Dynamical Implications: Spreading Processes
� We study a stochastic dynamical model of viral dissemination:
� - Each node has two possible states:� 0. Susceptible (blue)� 1. Infected (red)
� - Spreading parameters:� � probability of contagion� � probability of recovering
��
�
� Spectral results [Draief et al., 2008]:
� - �(A)>�/� is a necessary condition for a small infection to infect a significant part of the network
� - The larger �(A), the better a network disseminate a virus/rumor
ONR MURI: NexGeNetSci
Spreading Processes: Simulations
0 20 40 60 80 100 120 140 1600
50
100
150
200
250
300
0.05% initial infection�/�=35 < 45 < �(A)=60
0 10 20 30 40 500
50
100
150
200
250
300
0.2% initial infection�/�=65 > �(A)=60
Counting subgraph
frequencies
Computing spectral
moments
Bound on �(A)>45.0
Implications on Spreading
# of
infe
ctio
ns
# of
infe
ctio
ns
tt
ONR MURI: NexGeNetSci
• Ongoing work: Design incentives for each individual to modify their local neighborhood in order to achieve a particular global spectral property. Some preliminaries results [Preciado et al., 2010]:
Decentralized Network Design
ONR MURI: NexGeNetSci
• Adapt our framework to:1. Evolving networks: Tracking the evolution of
subgraphs frequencies and model their interactions
2. Links with weights and directions: Since the eigenvalues become complex, we have to work with 2D support
3. Nodes with attributes
• Incentive design: How can we drive nodes to take local actions that improve the global dynamical behavior of the network?
Future Work
ONR MURI: NexGeNetSci
• Our work is devoted to study local structural properties and dynamical processes in large-scale complex networks
• There is a direct relationship between many dynamical processes in networks and the eigenvalues of the underlying graph
• There is plenty of information about the eigenvalue spectra from the distribution of local network measurements
• Our approach is agnostic, in which we do not assume any global structure (no random graphs)
• Our results can be of interest to analyze and design large-scale networks from a spectral point of view
Conclusions
ONR MURI: NexGeNetSci
• F. Chung, L. Lu, and V. Vu, "The Spectra of Random Graphs with Given Expected Degrees," Internet Mathematics, vol. 1, pp. 257-275, 2003.
• M. Draief, A. Ganesh, and L. Massoulié, "Thresholds for Virus Spread on Networks," Annals of Applied Probability, vol. 18, pp. 359-378, 2008.
• L. Li, D. Alderson, J.C. Doyle, and W. Willinger, " Towards a Theory of Scale-Free Graphs,“ InternerMath, vol. 2, pp. 431-523, 2005.
• P. Parrilo, Algebraic Techniques and Semidefinite Optimization, Massachusetts Institute of Technology: MIT OpenCourseWare, Spring 2006.
• L.M. Pecora, and T.L. Carroll, "Master Stability Functions for Synchronized Coupled Systems," Physical Review Letters, vol. 80(10), pp. 2109-2112, 1998.
• I. Popescu and D. Bertsimas, "An SDP Approach to Optimal Moment Bounds for Convex Classes of Distributions," Mathematics of Operation Research, vol. 50, pp. 632-657, 2005.
• V.M. Preciado, and G.C. Verghese, "Synchronization in Generalized Erdös-Rényi Networks of Nonlinear Oscillators," IEEE Conference on Decision and Control, pp. 4628-4633, 2005.
• V.M. Preciado, and A. Jadbabaie, "Spectral Analysis of Virus Spreading in Random Geometric Networks," IEEE Conference on Decision and Control, pp. 4802-4807, 2009.
• V.M. Preciado, M.M. Zavlanos, A. Jadbabaie, and G.J. Pappas, “Distributed Control of the LaplacianSpectral Moments of a Network,” American Control Conference, 2010.
• V.M. Preciado and A. Jadbabaie, " From Local Measurements to Network Spectral Properties: Beyond Degree Distributions, " IEEE Conference on Decision and Control, 2010.
Some References
ONR MURI: NexGeNetSci
• QUESTIONS?
ONR MURI: NexGeNetSci
• We study a collection of resistively coupled nonlinear oscillators
IV.b Dynamical Implications: Synchronization
� �
� �
� Spectral results [Pecora and Carrol]: L=D-A, and �i are its eigenvalues
� For stability of the synchronous state we need the Laplacianeigenration �n/�2 < � , where � depends on the individual oscillatordynamics
� Network dynamics:
� Question: What values of � do make the network synchronize?
�
ONR MURI: NexGeNetSci
• Using our SDP approach, we can also bound the Laplacian eigenratiofrom local structural properties via the Laplacian moments
• In the Laplacian moments, not only the frequencies of subgraphs are important, but also the degrees of the nodes involved. For example, for the4th moment the following substructures are involved
• The Laplacian moments are functions of the frequencies of these structures and the degrees of the nodes involved:
Laplacian Moments
� � � ����� ��������� ��3213213214321 ,,
212
22
1,,,,
321,,,
44 144681
)(kkk
Ekkkkkkkkkk
c kkkkPPkkkPPn
Lm
ONR MURI: NexGeNetSci
Synchronization: Simulations
Counting frequencies of substructures
Computing Laplacianmoments
Bound on �n/�2
Implications on Synchronization
• We simulate a network of 200 resistively coupled Rossler oscillators
�����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������