power law and its generative models
Post on 26-Feb-2016
96 Views
Preview:
DESCRIPTION
TRANSCRIPT
Power Law and Its Generative Models
Bo Young Kim2010-03-16
Applied Algorithm Lab. 2
Contents1. Recall The Definition of Power Law2. Recall Some Properties of Power Law3. Generative Models for Power Law
- Power Laws via Preferential Attachment- Power Laws via Multiplicative Processes
Applied Algorithm Lab. 3
1. Recall The Definition of Power Law2. Recall Some Properties of Power Law3. Generative Models for Power Law
- Power Laws via Preferential Attachment- Power Laws via Multiplicative Processes
Applied Algorithm Lab. 4
1. Recall The Definition of Power Law
• X: a nonnegative random variable
• Def Power LawX is said to have a power law distribution if Pr[X≥x]~cx-α for constants c>0, α>0
• Def f(x)~g(x) ⇔ limx f(x)/g(x) = 1
• What does this mean?In a power law distribution, asymptotically the tails fall according to the power α. (heavier tail than exponential distribution)
Applied Algorithm Lab. 5
1. Recall The Definition of Power Law2. Recall Some Properties of Power Law3. Generative Models for Power Law
- Power Laws via Preferential Attachment- Power Laws via Multiplicative Processes
Applied Algorithm Lab. 6
2. Recall Some Properties of Power Law
• E.g. The Pareto distributionPr[X≥x]=(x/k)-α
ln(Pr[X≥x])=-α(ln(x)-ln(k))
* Linear Log-log plot (complementary cumulative distribution function)
- X has a power law distribution- Then a log-log plot behavior is a straight line. (asymptotic sense)
Applied Algorithm Lab. 7
2. Recall Some Properties of Power Law
“Scale Invariance”- Let f(x) := P[X≥x]- f(x) ~ cx-α
- f(kx) ~ c(kx) -α = k-α(cx-α ) = k’f(x) ∝ f(x) (k’=k-α)
- Scaling by a constant simply multiplies the original power law relation by the constant k’.- If we change the measurement unit(=scale), it re-tains the same power law form w/ the same expo-nent. We cannot decide what scale we’re observing. (like Fractals)
Applied Algorithm Lab. 8
2. Recall Some Properties of Power Law
• Web follows power law. [4]• Recall (Rank exponent)
- dv: outdegree of a node v - rv: the rank of a node v
dv =k*rvR (R,k: constant)
• Designing random graph models that yield Web-like graphs?
• i.e. that yields power law distributions for the in-degree and outdegree?
Applied Algorithm Lab. 9
1. Recall The Definition of Power Law2. Recall Some Properties of Power Law3. Generative Models for Power Law
- Power Laws via Preferential Attachment- Power Laws via Multiplicative Processes
Applied Algorithm Lab. 10
Generative Models for Power Law- Power Laws via Preferential Attachment
• Def Preferential Attachment Process (=Yule Process)Any process s.t. some quantity (some form of wealth) is distributed among a number of individ-uals according to how much they already have, so that those who are already wealthy receive more than those who are not.
• ”The rich get richer”
Applied Algorithm Lab. 11
• The Chinese Restaurant Process- A Chinese restaurant has infinitely many tables- Each table can seat infinitely many customers- At each time step, customer Xt comes into the restau-rant.When Xt+1 comes into here…
(CRP1) Sits at an already occupied table k w/ prob. Nk/(t+α)(Nk : # of customers at table k ΣkNk=t)(CRP2)or, sits at the next unoccupied table w/ prob. α/(t+α)
Generative Models for Power Law- Power Laws via Preferential Attachment
Applied Algorithm Lab. 12
When Xt+1 comes into here…
(CRP1) Sits at an already occupied table k w/ prob. Nk/(t+α)(Nk : # of customers at table k ΣkNk=t)(CRP2)or, sits at the next unoccupied table w/ prob. α/(t+α)
Generative Models for Power Law- Power Laws via Preferential Attachment
Applied Algorithm Lab. 13
• CPR rule: Next customer sits at a table w/ prob. Proportional to # of customers already sitting at it(and sits at new table w/ prob. Proportional to α) Customers tend to sit at most popular tables Most popular tables attract the most new cus-tomers, and become even more popular
• The concentration parameter α: how likely cus-tomer is to sit at a fresh table
Generative Models for Power Law- Power Laws via Preferential Attachment
Applied Algorithm Lab. 14
• Generating Power law distribution via Preference At-tachment (Most models are variations of this form)
• Let’s say “Web Page Process”• Start w/ a single page• This single page has a link to itself• At each time step, a new page appears, w/ outdegree 1
Generative Models for Power Law- Power Laws via Preferential Attachment
(WPP1) The link of new page points to a page chosen u.a.r. w/ prob. α<1 (WPP2) The link of new page points to page chosen proportionally to the indegree of the page w/ prob. 1- α
Applied Algorithm Lab. 15
• Xj(t): # of pages w/ indegree j when ∃ t pages in the system
• Pr[Xj increase] = αXj-1/t+(1-α)(j-1)Xj-1/t• Pr[Xj decrease] = αXj/t+(1-α)jXj/t
Generative Models for Power Law- Power Laws via Preferential Attachment
(WPP1) The link of new page points to a page chosen u.a.r. w/ prob. α<1 (WPP2) The link of new page points to page chosen proportionally to the indegree of the page w/ prob. 1- α
Applied Algorithm Lab. 16
• Pr[Xj increase] = αXj-1/t+(1-α)(j-1)Xj-1/t• Pr[Xj decrease] = αXj/t+(1-α)jXj/t
dXj/dt = {α(Xj-1-Xj)+(1-α)((j-1)Xj-1-jXj-1)}/t
• Intuitively appealing, BUT how continuous DE de-scribes a discrete process?
This can be justified formally using martingales [Kumar et al 00] & theoretical frameworks of Kurtz, Wormald [Drinea et al. 00, Kurtz 81, Wormald 95].
Generative Models for Power Law- Power Laws via Preferential Attachment
Applied Algorithm Lab. 17
• dX0/dt=1-αX0/t
• Suppose in the steady state limit: Xj(t)=cj*t (portion cj) c0 =dX0/dt=1-αX0/t=1-αc0 ⇔ c0 = 1/(α+1)
• Substitute this assumption for dXj/dt = {α(Xj-1-Xj)+(1-α)((j-1)Xj-1-jXj-1)}/t cj(1+α+j(1-α))=cj-1(α+(j-1)(1-α)) We can determine cj exactly.
• Focusing on the asymptotic, for large jcj/cj-1=1-(2-α)/(1+α+j(1-α))~1-{(2-α)/(1-α)}*(1/j)
Generative Models for Power Law- Power Laws via Preferential Attachment
Applied Algorithm Lab. 18
• We have cj~cj^(- ) for some constant c, giving a power law.
• Note cj~cj^(- ) implies
WTS: Σj≥kcj behave the tail of power law distribution(Proof)
For some constant c’. So, we’re done.
Generative Models for Power Law- Power Laws via Preferential Attachment
12
12 )1)(
12(1~)1(~ 1
2
1 jjj
cc
j
j
1
1
'~~~ 12
12
kcdjcjcjckjkjkj
j
Applied Algorithm Lab. 19
1. Recall The Definition of Power Law2. Recall Some Properties of Power Law3. Generative Models for Power Law
- Power Laws via Preferential Attachment- Power Laws via Multiplicative Processes
Applied Algorithm Lab. 20
• Pareto: income distribution obeys power law• [Champernowne 53] offered an explanation for this be-
havior.• Partition income in the following manner:• 1st range: between m and γm for some γ>1• 2nd range: between γm and γ2m …• persons in class j: their income is between γj-1m and γjm • Pij: prob. of a person moving from class i to class j• At each time step, Pij depends only on the value (j-i).
Under this assumption, Pareto distribution can be ob-tained.
Generative Models for Power Law- Power Laws via Multiplicative Processes
Applied Algorithm Lab. 21
• E.g. γ=2, Pij=2/3 if j-i=-1 Pij=1/3 if j-i=1
• Special case: i=1 P11=2/3
• The equilibrium property of being in class k: 1/2k
X: a person’s income Pr[X≥2k-1m]=1/2k-1
Pr[X ≥ x]=m/x for x= 2k-1m
This is a power law distribution.
Generative Models for Power Law- Power Laws via Multiplicative Processes
Applied Algorithm Lab. 22
References[1] M. Mitzenmacher, A Brief History of Generative Models for Power Law and
Lognormal Distributions, Internet Mathematics, vol 1, No. 2, pp. 226-251, 2004. [2] Mark Johnson, Chinese Restaurant Processes(CG168 notes), cog.brown.edu/
~mj/classes/cg168/.../ChineseRestaurants.pdf[3] The lecture notes of C. Faloutsos, Carnegie Mellon University, 15-826 Multimedia
Databases and Data Mining, Spring 2008 http://www.cs.cmu.edu/~christos/courses/826.S08/FOILS-pdf/195_powerLaws.pdf
[4] Bruno Bassetti, Mina Zarei, Marco Cosentino Lagomarsino, and Ginestra Bianconi., Statistical mechanics of the “Chinese restaurant process”: Lack of self-av-eraging, anomalous finite-size effects, and condensation, Phys. Rev. E 80, 066118 (2009) [4 pages]
[5] http://en.wikipedia.org/wiki/Power_law, http://en.wikipedia.org/wiki/Chinese_restaurant_process, http://en.wikipedia.org/wiki/Preferential_attachment
top related