Probability and Random ProcessesLecture 6

• Differentiation

• Absolutely continuous functions

• Continuous vs. discrete random variables

• Absolutely continuous measures

• Radon–Nikodym

Mikael Skoglund, Probability and random processes 1/11

Bounded Variation

• Let f be a real-valued function on [a, b]

• Total variation of f over [a, b],

V ba f = sup

n∑k=1

|f(xk)− f(xk−1)|

over all a = x0 < x1 < · · · < xn = b and n

• f is of bounded variation on [a, b] if V ba f <∞

• f of bounded variation ⇒ f differentiable Lebesgue-a.e.

Mikael Skoglund, Probability and random processes 2/11

The indefinite Lebesgue integral

• Assume that f is Lebesgue measurable and integrable on [a, b]and set

F (x) =

∫ x

af(t)dt

for a ≤ x ≤ b, then F is continuous and of bounded variation,and

V baF =

∫ b

a|f(x)|dx

(the integrals are Lebesgue integrals). Furthermore F isdifferentiable a.e. and F ′(x) = f(x) a.e. on [a, b]

Mikael Skoglund, Probability and random processes 3/11

Absolutely Continuous on [a, b]

Definite Lebesgue integration

• For f : [a, b]→ R, assume that f ′ exists a.e. on [a, b] and isLebesgue integrable. If

f(x) = f(a) +

∫ x

af ′(t)dt

for x ∈ [a, b] then f is absolutely continuous on [a, b]

⇐⇒ for each ε > 0 there is a δ > 0 such that

n∑k=1

|f(bk)− f(ak)| < ε

for any sequence (ak, bk) of pairwise disjoint (ak, bk) in[a, b] with

∑nk=1(bk − ak) < δ

Mikael Skoglund, Probability and random processes 4/11

Absolutely Continuous on R

• f is absolutely continuous on R if it’s absolutely continuouson [−∞, b]

⇐⇒ f is absolutely continuous on every [a, b], −∞ < a < b <∞,V∞−∞f <∞ and limx→−∞ f(x) = 0

Mikael Skoglund, Probability and random processes 5/11

Discrete and Continuous Random Variables

• A probability space (Ω,A, P ) and a random variable X

• X is discrete if there is a countable set K ∈ B such thatP (X ∈ K) = 1

• X is continuous if P (X = x) = 0 for all x ∈ R

Mikael Skoglund, Probability and random processes 6/11

Distribution Functions and pdf’s

• A random variable X is absolutely continuous if there is anonnegative B-measurable function fX such that

µX(B) =

∫BfX(x)dx

for all B ∈ B⇐⇒ The probability distribution function FX is absolutely

continuous on R• The function fX is called the probability density function

(pdf) of X, and it holds that fX = F ′X a.e.

Mikael Skoglund, Probability and random processes 7/11

Absolutely Continuous Measures

• Question: Given measures µ and ν on (Ω,A), under whatconditions does there exist a density f for ν w.r.t. µ, such that

ν(A) =

∫Afdµ

for any A ∈ A?

• Necessary condition: ν(A) = 0 if µ(A) = 0 (why?)• e.g. the Dirac measure cannot have a density w.r.t. Lebesgue

measure

• ν is said to be absolutely continuous w.r.t. µ, notation ν µ,if ν(A) = 0 whenever µ(A) = 0

• Absolute continuity and σ-finiteness are necessary andsufficient conditions. . .

Mikael Skoglund, Probability and random processes 8/11

Radon–Nikodym

• The Radon–Nikodym theorem: If µ and ν are σ-finite on(Ω,A) and ν µ, then there is a nonnegative extendedreal-valued A-measurable function f on Ω such that

ν(A) =

∫Afdµ

for any A ∈ A. Furthermore, f is unique µ-a.e.

• The µ-a.e. unique function f in the theorem is called theRadon–Nikodym derivative of ν w.r.t. µ, notation f = dν

dµ

Mikael Skoglund, Probability and random processes 9/11

Absolutely Continuous RV’s and pdf’s, again

• (Obviously), the pdf of an absolutely continuous randomvariable X is the Radon–Nikodym derivative of µXw.r.t. Lebesgue measure (restricted to B),

µX(B) =

∫BfX(x)dx =

∫B

dµXdx

dx

Mikael Skoglund, Probability and random processes 10/11

Absolutely Continuous Functions vs. Measures

• If µ is a finite Borel measure with distribution function Fµ,then µ is absolutely continuous w.r.t. Lebesgue measure, λ, iffFµ is absolutely continuous on R, and in this case

dµ

dλ= F ′µ λ-a.e.

Mikael Skoglund, Probability and random processes 11/11