recovering temporally rewiring networks: a model-based approach

Fan Guo, Steve Hanneke, Wenjie Fu, Eric P. Xing

School of Computer Science, Carnegie Mellon University

04/20/23 1ICML 2007 Presentation

High School Dating

The Internet

Physicist Collaborations

All the images are from http://www-personal.umich.edu/~mejn/networks/. That page includes original citations.04/20/23

Protein-Protein Interaction Network in S. cerevisiaeFig. 1 from (H. Jeong et al., Nature 411, 41-42, 3 May 2001)

Model for the Yeast cell cycle transcriptional regulatory networkFig. 4 from (T.I. Lee et al., Science 298, 799-804, 25 Oct 2002)

04/20/23 3The small image is from http://www.raiks.de/img/dyna_title_zoom.jpg

Infer the hidden network topology from node attribute observations.

Methods: Optimizing a score function; Information-theoretic

approaches; Model-based approach …

Most of them pool the data together to infer a static network topology.

04/20/23 ICML 2007 Presentation 4


Network topologies and functions are not static: Social networks can grow as we know more friends Biological networks rewire under different conditions

Fig. 1b from Genomic analysis of regulatory network dynamics reveals large topological changesN. M. Luscombe, et al. Nature 431, 308-312, 16 September 2004


Network topologies and functions are not always static.

We propose probabilistic models and algorithms for recovering latent network topologies that are changing over time from node attribute observations.

Networks rewire over discrete timesteps

Part of the image is modified from Fig. 3b (E. Segal et al., Nature Genetics 34, 166-176, June 2003).04/20/23

Transition Model

Emission Model


Latent network structures are of higher dimensions than observed node attributes How to place constraints on the latent space?

Limited evidence per timestep How to share the information across time?


Energy-based conditional probability model (recall Markov random fields…)

Energy-based model is easier to analysis, but even the design of approximate inference algorithm can be hard.


( , )

( , )1

( | ) exp ( , )( ) kk C

k

x

E x y

E x ye

p x y x yZ ye



Based on our previous work on discrete temporal network models in the ICML’06 SNA-Workshop. Model network rewiring as a Markov process. An expressive framework using energy-based local

probabilities (based on ERGM):

Features of choice:

1 1

1

1exp ,

,t t t t

i iti

p A A A AZ A

1 1

1 11 2 3 1 1

, 1 1 ,

t t tij ik kj

ijkt t t t tij ij ij ij ij t t

ij ij ik kjijk

A A A

A A A A AA A

(Density) (Edge Stability) (Transitivity)


Given the network topology, how to generate the binary node attributes?

Another energy-based conditional model:

All features are pairwise which induces an undirected graph corresponding to the time-specific network topology;

Additional information shared over time is represented by a matrix of parameters Λ;

The design of feature function Φ is application-specific.

04/20/23 ICML 2007 Presentation 1204/20/23 12ICML 2007 Presentation

1, exp , , ,

, ,t t t t t

ij i j ij ijtij

p x A x x AZ A

The feature function

If no edge between i and j, Φ equals 0;

Otherwise the sign of Φ depends on Λij and the empirical correlation of xi, xj at time t.


2 1 2 1t t tij ij ij i jA x x


Time-invariant parameters dictating the direction of pairwise correlation in the example

Initial network to define the prior on A1

Hidden rewiring networks


A natural approach to infer the hidden networks A1:T is Gibbs sampling: To evaluate the log-odds

Conditional probabilities in a Markov blanket

1 1

1 1

1 , , ,log

0 , , ,

t t t t tij ijt

ij t t t t tij ij

P A A A A x

P A A A A x

Tractable transition model; the partition function is the product of per edge terms

Computation is straightforward

Given the graphical structure, run variable elimination algorithms, works well for small graphs

1, exp , , ,

, ,t t t t t

ij i j ij ijtij

p x A x x AZ A

Grid search is very helpful, although Monte Carlo EM can be implemented.

Trade-off between the transition model and emission model: Larger θ : better fit of the rewiring processes; Larger η : better fit of the observations.


Data generated from the proposed model. Starting from a network (A0) of 10 nodes and 14 edges. The length of the time series T = 50.

Compare three approaches using F1 score: avg: averaged network from “ground truth”

(approx. upper bounds the performance of any static network inference algorithm) htERG: infer timestep-specific networks sERG: the static counterpart of the proposed algorithm

Study the “edge-switching events”


F1 scores on different parameter settings (varying )


1 30.5, 4, 5, 100 iterations of Gibbs sampling, 10 repetitionsD k 2 ,


F1 scores on different number of examples1 2 30.5, 4, 4, 1,100 iterations of Gibbs sampling, 10 repetitionsk


Summary on capturing edge switching in networks Three cases studied: offset, false positive, missing (false negative) mean and rms of offset timesteps

1 2 30.5, 4, 4, 1, 5, 100 iterations of Gibbs sampling, 10 repetitionsD k


The proposed model was applied to infer the muscle development sub-network (Zhao et al., 2006) on Drosophila lifecycle gene expression data (Arbeitman et al., 2002).

11 genes, 66 timesteps over 4 development stages Further biological experiments are necessary for verification.

Network in (Zhao et al. 2006)

Embryonic Larval Pupal & Adult


A new class of probabilistic models to address the problem of recoving hidden, time-dependent network topologies and an example in a biological context.

An example of employing energy-based model to define meaningful features and simplify parameterization.

Future work Larger-scale network analysis (100+?) Developing emission models for richer context

Yanxin Shi CMU

Wentao Zhao Texas A&M University

Hetunandan Kamisetty CMU


recovering temporally rewiring networks: a model-based approach

Documents

presentation networks

model network rewiring

static network topology

latent network topologies

hidden network topology

conditional model

node attribute observations

binary node attributes