finding patterns in large, real networks

CMU SCS

Finding patterns in large, real networks

Christos Faloutsos

CMU

LLNL 05 C. Faloutsos 2

CMU SCS

Thanks to

• Deepayan Chakrabarti (CMU/Yahoo)

• Michalis Faloutsos (UCR)

• Spiros Papadimitriou (CMU/IBM)

• George Siganos (UCR)


CMU SCS

Introduction

Internet Map [lumeta.com]

Food Web [Martinez ’91]

Protein Interactions [genomebiology.com]

Friendship Network [Moody ’01]

Graphs are everywhere!


CMU SCS

Outline

• Laws– static

– time-evolution laws

– why so many power laws?

• tools/results

• future steps


CMU SCS

Motivating questions

• Q1: What do real graphs look like?• Q2: How to generate realistic graphs?


CMU SCS

Virus propagation

• Q3: How do viruses/rumors propagate?

• Q4: Who is the best person/computer to immunize against a virus?


CMU SCS

Graph clustering & mining

• Q5: which edges/nodes are ‘abnormal’?

• Q6: split a graph in k ‘natural’ communities - but how to determine k?


CMU SCS

Outline

• Laws– static



• tools/results

• future steps


CMU SCS

Topology

Q1: How does the Internet look like? Any rules?

(Looks random – right?)


CMU SCS

Laws – degree distributions

• Q: avg degree is ~2 - what is the most probable degree?

degree

count ??

2


CMU SCS

Laws – degree distributions

• Q: avg degree is ~2 - what is the most probable degree?

degreedegree

count ??

2

count

2


CMU SCS

I.Power-law: outdegree O

The plot is linear in log-log scale [FFF’99]

freq = degree (-2.15)

O = -2.15

Exponent = slope

Outdegree

Frequency

Nov’97

-2.15


CMU SCS

II.Power-law: rank R

• The plot is a line in log-log scale

R

Rank: nodes in decreasing degree order

April’98

log(rank)

log(degree)

-0.82

att.com

ibm.com


CMU SCS

III.Power-law: eigen E

• Eigenvalues in decreasing order (first 20)• [Mihail+, 02]: R = 2 * E

E = -0.48

Exponent = slope

Eigenvalue

Rank of decreasing eigenvalue

Dec’98


CMU SCS

IV. The Node Neighborhood

• N(h) = # of pairs of nodes within h hops


CMU SCS


• Q: average degree = 3 - how many neighbors should I expect within 1,2,… h hops?

• Potential answer:1 hop -> 3 neighbors

2 hops -> 3 * 3

…

h hops -> 3h


CMU SCS




2 hops -> 3 * 3

…

h hops -> 3h

WE HAVE DUPLICATES!


CMU SCS




2 hops -> 3 * 3

…

h hops -> 3h

‘avg’ degree: meaningless!


CMU SCS

IV. Power-law: hopplot H

Pairs of nodes as a function of hops N(h)= hH

H = 4.86

Dec 98Hops

# of Pairs # of Pairs

Hops Router level ’95

H = 2.83


CMU SCS

Observation• Q: Intuition behind ‘hop exponent’?• A: ‘intrinsic=fractal dimensionality’ of the

network

N(h) ~ h1 N(h) ~ h2

...


CMU SCS

Hop plots

• More on fractal/intrinsic dimensionalities: very soon


CMU SCS

Any other ‘laws’?

Yes!


CMU SCS

Any other ‘laws’?

Yes!

• Small diameter– six degrees of separation / ‘Kevin Bacon’

– small worlds [Watts and Strogatz]


CMU SCS

Any other ‘laws’?• Bow-tie, for the web [Kumar+ ‘99]

• IN, SCC, OUT, ‘tendrils’

• disconnected components


CMU SCS

Any other ‘laws’?• power-laws in communities (bi-partite cores)

[Kumar+, ‘99]

2:3 core(m:n core)

Log(m)

Log(count)

n:1

n:2n:3


CMU SCS

More Power laws

• Also hold for other web graphs [Barabasi+, ‘99], [Kumar+, ‘99]

• citation graphs (see later)

• and many more


CMU SCS

Outline

• Laws– static



• tools/results

• future steps


CMU SCS

How do graphs evolve?


CMU SCS

Time Evolution: rank R

The rank exponent has not changed! [Siganos+, ‘03]

Domainlevel

#days since Nov. ‘97


CMU SCS

How do graphs evolve?

• degree-exponent seems constant - anything else?


CMU SCS

Evolution of diameter?

• Prior analysis, on power-law-like graphs, hints that

diameter ~ O(log(N)) or

diameter ~ O( log(log(N)))

• i.e.., slowly increasing with network size

• Q: What is happening, in reality?


CMU SCS

Evolution of diameter?

• Prior analysis, on power-law-like graphs, hints that

diameter ~ O(log(N)) or

diameter ~ O( log(log(N)))

• i.e.., slowly increasing with network size

• Q: What is happening, in reality?

• A: It shrinks(!!), towards a constant value


CMU SCS

Shrinking diameter

[Leskovec+05a]• Citations among physics

papers • 11yrs; @ 2003:

– 29,555 papers– 352,807 citations

• For each month M, create a graph of all citations up to month M


CMU SCS

Shrinking diameter• Authors &

publications

• 1992– 318 nodes

– 272 edges

• 2002– 60,000 nodes

• 20,000 authors

• 38,000 papers

– 133,000 edges


CMU SCS

Shrinking diameter• Patents & citations

• 1975– 334,000 nodes

– 676,000 edges

• 1999– 2.9 million nodes

– 16.5 million edges

• Each year is a datapoint


CMU SCS

Shrinking diameter• Autonomous

systems

• 1997– 3,000 nodes

– 10,000 edges

• 2000– 6,000 nodes

– 26,000 edges

• One graph per day


CMU SCS

Temporal evolution of graphs

• N(t) nodes; E(t) edges at time t

• suppose that

N(t+1) = 2 * N(t)

• Q: what is your guess for

E(t+1) =? 2 * E(t)


CMU SCS


• N(t) nodes; E(t) edges at time t

• suppose that

N(t+1) = 2 * N(t)

• Q: what is your guess for

E(t+1) =? 2 * E(t)

• A: over-doubled!


CMU SCS


• A: over-doubled - but obeying:

E(t) ~ N(t)a for all t

where 1<a<2


CMU SCS


• A: over-doubled - but obeying:

E(t) ~ N(t)a for all t

• Identically:

log(E(t)) / log(N(t)) = constant for all t


CMU SCS

Densification Power Law

ArXiv: Physics papers

and their citations

1.69

N(t)

E(t)


CMU SCS



and their citations

1.69

N(t)

E(t)

‘tree’

1


CMU SCS



and their citations

1.69

N(t)

E(t)‘clique’

2


CMU SCS


U.S. Patents, citing each other

1.66

N(t)

E(t)


CMU SCS


Autonomous Systems

1.18

N(t)

E(t)


CMU SCS


ArXiv: authors & papers

1.15

N(t)

E(t)


CMU SCS

Summary of ‘laws’

• Power laws for degree distributions

• …………... for eigenvalues, bi-partite cores

• Small & shrinking diameter (‘6 degrees’)

• ‘Bow-tie’ for web; ‘jelly-fish’ for internet

• ``Densification Power Law’’, over time


CMU SCS

Outline

• Laws– static


– why so many power laws?• other power laws

• fractals & self-similarity

• case study: Kronecker graphs

• tools/results

• future steps


CMU SCS

Power laws

• Many more power laws, in diverse settings:


CMU SCS

A famous power law: Zipf’s law

• Bible - rank vs frequency (log-log)

log(rank)

log(freq)

“a”

“the”


CMU SCS

Power laws, cont’ed

• length of file transfers [Bestavros+]

• web hit counts [Huberman]

• magnitude of earthquakes (Guttenberg-Richter law)

• sizes of lakes/islands (Korcak’s law)

• Income distribution (Pareto’s law)


CMU SCS

The Peer-to-Peer Topology

• Frequency versus degree

• Number of adjacent peers follows a power-law

[Jovanovic+]


CMU SCS

Web Site Traffic

log(freq)

log(count)Zipf

Click-stream data

u-id’s url’s

log(freq)

log(count)

‘yahoo’

‘super-surfer’


CMU SCS

Lotka’s law

(Lotka’s law of publication count); and citation counts: (citeseer.nj.nec.com 6/2001)

log(#citations)

log(count)

J. Ullman


CMU SCS Swedish sex-web

Nodes: people (Females; Males)Links: sexual relationships

Liljeros et al. Nature 2001

4781 Swedes; 18-74; 59% response rate.

Albert Laszlo Barabasihttp://www.nd.edu/~networks/Publication%20Categories/04%20Talks/2005-norway-3hours.ppt


CMU SCS

Power laws

• Q: Why so many power laws?

• A1: self-similarity

• A2: ‘rich-get-richer’


CMU SCS

Digression: intro to fractals

• Fractals: sets of points that are self similar


CMU SCS

A famous fractal

e.g., Sierpinski triangle:

...zero area;

infinite length!

dimensionality = ??


CMU SCS

A famous fractal

e.g., Sierpinski triangle:

...zero area;

infinite length!

dimensionality = log(3)/log(2) = 1.58


CMU SCS

A famous fractal

equivalent graph:


CMU SCS

Intrinsic (‘fractal’) dimension

How to estimate it?


CMU SCS

Intrinsic (‘fractal’) dimension

• Q: fractal dimension of a line?

• A: nn ( <= r ) ~ r^1(‘power law’: y=x^a)

• Q: fd of a plane?

• A: nn ( <= r ) ~ r^2

fd== slope of (log(nn) vs log(r) )


CMU SCS

Sierpinsky triangle

log( r )

log(#pairs within <=r )

1.58

== ‘correlation integral’

= CDF of pairwise distances


CMU SCS

Sierpinsky triangle

log( r )


1.58




CMU SCS

Line

log( r )


1.58



1...


CMU SCS

2-d (Plane)

log( r )


1.58



2


CMU SCS

Recall: Hop Plot

• Internet routers: how many neighbors within h hops? (= correlation integral!)

Reachability function: number of neighbors within r hops, vs r (log-log).

Mbone routers, 1995log(hops)

log(#pairs)

2.8


CMU SCS

Fractals and power laws

They are related concepts:

• fractals <=>

• self-similarity <=>

• scale-free <=>

• power laws ( y= xa )– F = C r(-2)


CMU SCS

Fractals and power laws

• Power laws and fractals are closely related• And fractals appear in MANY cases

– coast-lines: 1.1-1.5– brain-surface: 2.6– rain-patches: 1.3– tree-bark: ~2.1– stock prices / random walks: 1.5– ... [see Mandelbrot; or Schroeder]


CMU SCS

Outline

• Laws– static


– why so many power laws?• other power laws

• fractals & self-similarity

• case study: Kronecker graph generator

• tools/results

• future steps


CMU SCS

Self-similarity at work

• Q2: How to generate realistic graphs?

• (Q2’) and how to grow it over time?


CMU SCS

Wish list for a generator:

• Power-law-tail in- and out-degrees

• Power-law-tail scree plots

• shrinking/constant diameter

• Densification Power Law

• communities-within-communities

Q: how to achieve all of them?


CMU SCS

Wish list for a generator:



• shrinking/constant diameter

• Densification Power Law


Q: how to achieve all of them?

A: Kronecker matrix product [Leskovec+05b]


CMU SCS

Kronecker product


CMU SCS

Kronecker product

N N*N N**4


CMU SCS

Properties of Kronecker graphs:



• constant diameter

• perfect Densification Power Law



CMU SCS

Properties of Kronecker graphs:



• constant diameter

• perfect Densification Power Law


and we can prove all of the above

(first and only generator that does that)


CMU SCS

Kronecker - ArXiv

real

(stochastic)Kronecker

Degree Scree Diameter D.P.L.

(det. Kronecker)


CMU SCS

Kronecker - patents

Degree Scree Diameter D.P.L.


CMU SCS

Kronecker - A.S.


CMU SCS

Outline

• Laws– static



• tools/results– virus propagation

– connection subgraphs

– cross-associations

• future steps


CMU SCS

Virus propagation




CMU SCS

The model: SIS

• ‘Flu’ like: Susceptible-Infected-Susceptible

• Virus ‘strength’ s= /

Infected

Healthy

NN1

N3

N2

Prob.

Prob. β

Prob.


CMU SCS

Epidemic threshold

of a graph: the value of , such that

if strength s = / < an epidemic can not happen

Thus,

• given a graph

• compute its epidemic threshold


CMU SCS

Epidemic threshold

What should depend on?

• avg. degree? and/or highest degree?

• and/or variance of degree?

• and/or third moment of degree?

• and/or diameter?


CMU SCS

Epidemic threshold

• [Theorem] We have no epidemic, if

β/δ <τ = 1/ λ1,A


CMU SCS

Epidemic threshold

• [Theorem] We have no epidemic, if

β/δ <τ = 1/ λ1,A

largest eigenvalueof adj. matrix A

attack prob.

recovery prob.epidemic threshold

Proof: [Wang+03]


CMU SCS

Experiments (Oregon)

/ > τ (above threshold)

/ = τ (at the threshold)

/ < τ (below threshold)


CMU SCS

Our result:

• Holds for any graph

• includes older results as special cases


CMU SCS

Virus propagation



• A4: the one that causes the highest decrease in 1


CMU SCS

Outline

• Laws– static






• future steps


CMU SCS

(A-A) Kidman-Diaz

• What are the best paths between ‘Kidman’ and ‘Diaz’?


CMU SCS

(A-A) Kidman-Diaz

Strong, direct link

• What are the best paths between ‘Kidman’ and ‘Diaz’? [KDD’04, w/ McCurley+Tomkins]


CMU SCS

Outline

• Laws– static






• future steps


CMU SCS





CMU SCS

Graph partitioning

• Documents x terms

• Customers x products

• Users x web-sites


CMU SCS

Graph partitioning




• Q: HOW MANY PIECES?


CMU SCS

Graph partitioning




• Q: HOW MANY PIECES?

• A: MDL/ compression


CMU SCS

Cross-associations

1x2 2x2


CMU SCS

Cross-associations

2x3 3x3 3x4


CMU SCS

Cross-associations


CMU SCS




• A6: choose the k that leads to best overall compression (= MDL = Minimum Description Language)


CMU SCS

Cross-associations

outlier edgemissing edge


CMU SCS

Outline

• Laws– static



• tools/results

• other related projects

• future steps


CMU SCS

Motivation

water distribution network

normal operation

May have hundreds of measurements, butit is unlikely they are completely unrelated!

Phase 1 Phase 2 Phase 3

: : : : : :

: : : : : :

chlo

rin

e c

on

cen

trati

on

s

sensorsnear leak

sensorsawayfrom leak


CMU SCS

Phase 1 Phase 2 Phase 3

: : : : : :

: : : : : :

Motivation

water distribution network

normal operation major leak

May have hundreds of measurements, butit is unlikely they are completely unrelated!

chlo

rin

e c

on

cen

trati

on

s

sensorsnear leak

sensorsawayfrom leak


CMU SCS

Motivation

actual measurements(n streams)

k hidden variable(s)

We would like to discover a few “hidden(latent) variables” that summarize the key trends

Phase 1

: : : : : :

: : : : : :

chlo

rin

e c

on

cen

trati

on

s

Phase 1

k = 1


CMU SCS

Motivation


chlo

rin

e c

on

cen

trati

on

s

Phase 1 Phase 1Phase 2 Phase 2



k = 2

: : : : : :

: : : : : :


CMU SCS

Motivation


chlo

rin

e c

on

cen

trati

on

s

Phase 1 Phase 1Phase 2 Phase 2Phase 3 Phase 3



k = 1

: : : : : :

: : : : : :


CMU SCS

Stream mining

• Solution: SPIRIT [VLDB’05]– incremental, on-line PCA


CMU SCS

Outline

• Laws

• tools/results

• other related projects

• future steps


CMU SCS

Future steps

• connection sub-graphs on multi-graphs [Tong]

• traffic matrix, evolving over time [Sun+]

• influence propagation on blogs [Leskovec+]

• automatic web site classification [Leskovec]


CMU SCS

Conclusions

• Power laws & self similarity– excellent tools, for real datasets,

– where Gaussian, uniform etc assumptions fail

• Additional tools– Kronecker, for graph generators

– eigenvalues for virus propagation

– MDL/compression for graph partitioning


CMU SCS

References

• [Aiello+, '00] William Aiello, Fan R. K. Chung, Linyuan Lu: A random graph model for massive graphs. STOC 2000: 171-180

• [Albert+] Reka Albert, Hawoong Jeong, and Albert-Laszlo Barabasi: Diameter of the World Wide Web, Nature 401 130-131 (1999)

• [Barabasi, '03] Albert-Laszlo Barabasi Linked: How Everything Is Connected to Everything Else and What It Means (Plume, 2003)


CMU SCS

References, cont’d

• [Barabasi+, '99] Albert-Laszlo Barabasi and Reka Albert. Emergence of scaling in random networks. Science, 286:509--512, 1999

• [Broder+, '00] Andrei Broder, Ravi Kumar, Farzin Maghoul, Prabhakar Raghavan, Sridhar Rajagopalan, Raymie Stata, Andrew Tomkins, and Janet Wiener. Graph structure in the web, WWW, 2000


CMU SCS


• [Chakrabarti+, ‘04] RMAT: A recursive graph generator, D. Chakrabarti, Y. Zhan, C. Faloutsos, SIAM-DM 2004

• [Dill+, '01] Stephen Dill, Ravi Kumar, Kevin S. McCurley, Sridhar Rajagopalan, D. Sivakumar, Andrew Tomkins: Self-similarity in the Web. VLDB 2001: 69-78


CMU SCS


• [Fabrikant+, '02] A. Fabrikant, E. Koutsoupias, and C.H. Papadimitriou. Heuristically Optimized Trade-offs: A New Paradigm for Power Laws in the Internet. ICALP, Malaga, Spain, July 2002

• [FFF, 99] M. Faloutsos, P. Faloutsos, and C. Faloutsos, "On power-law relationships of the Internet topology," in SIGCOMM, 1999.


CMU SCS


• [Leskovec+05a] Jure Leskovec, Jon Kleinberg and Christos Faloutsos Graphs over Time: Densification Laws, Shrinking Diameters and Possible Explanations KDD 2005, Chicago, IL. (Best research paper award)

• [Leskovec+05b] Jure Leskovec, Deepayan Chakrabarti, Jon Kleinberg and Christos Faloutsos, Realistic, Mathematically Tractable Graph Generation and Evolution, Using Kronecker Multiplication, ECML/PKDD 2005, Porto, Portugal.


CMU SCS


• [Jovanovic+, '01] M. Jovanovic, F.S. Annexstein, and K.A. Berman. Modeling Peer-to-Peer Network Topologies through "Small-World" Models and Power Laws. In TELFOR, Belgrade, Yugoslavia, November, 2001

• [Kumar+ '99] Ravi Kumar, Prabhakar Raghavan, Sridhar Rajagopalan, Andrew Tomkins: Extracting Large-Scale Knowledge Bases from the Web. VLDB 1999: 639-650


CMU SCS


• [Leland+, '94] W. E. Leland, M.S. Taqqu, W. Willinger, D.V. Wilson, On the Self-Similar Nature of Ethernet Traffic, IEEE Transactions on Networking, 2, 1, pp 1-15, Feb. 1994.

• [Mihail+, '02] Milena Mihail, Christos H. Papadimitriou: On the Eigenvalue Power Law. RANDOM 2002: 254-262


CMU SCS


• [Milgram '67] Stanley Milgram: The Small World Problem, Psychology Today 1(1), 60-67 (1967)

• [Montgomery+, ‘01] Alan L. Montgomery, Christos Faloutsos: Identifying Web Browsing Trends and Patterns. IEEE Computer 34(7): 94-95 (2001)


CMU SCS


• [Palmer+, ‘01] Chris Palmer, Georgos Siganos, Michalis Faloutsos, Christos Faloutsos and Phil Gibbons The connectivity and fault-tolerance of the Internet topology (NRDM 2001), Santa Barbara, CA, May 25, 2001

• [Pennock+, '02] David M. Pennock, Gary William Flake, Steve Lawrence, Eric J. Glover, C. Lee Giles: Winners don't take all: Characterizing the competition for links on the

web Proc. Natl. Acad. Sci. USA 99(8): 5207-5211 (2002)


CMU SCS


• [Schroeder, ‘91] Manfred Schroeder Fractals, Chaos, Power Laws: Minutes from an Infinite Paradise W H Freeman & Co., 1991 (excellent book on fractals)


CMU SCS


• [Siganos+, '03] G. Siganos, M. Faloutsos, P. Faloutsos, C. Faloutsos Power-Laws and the AS-level Internet Topology, Transactions on Networking, August 2003.

• [Wang+03] Yang Wang, Deepayan Chakrabarti, Chenxi Wang and Christos Faloutsos: Epidemic Spreading in Real Networks: an Eigenvalue Viewpoint, SRDS 2003, Florence, Italy.


CMU SCS

Reference

• [Watts+ Strogatz, '98] D. J. Watts and S. H. Strogatz Collective dynamics of 'small-world' networks, Nature, 393:440-442 (1998)

• [Watts, '03] Duncan J. Watts Six Degrees: The Science of a Connected Age W.W. Norton & Company; (February 2003)


CMU SCS

Thank you!

www.cs.cmu.edu/~christos

www.db.cs.cmu.edu