Math 710 Homework

Austin Mohr

June 16, 2012

1. For the following random “experiments”, describe the sample space Ω.For each experiment, describe also two subsets (events) that might be ofinterest, and describe how you might assign probabilities to these events.

(a) The USC football team will play 12 games this season. The experi-ment is to observe the Win-Tie-Loss record.

Solution: Define the sample space Ω to be the set

{(a1, . . . , a12) | ai ∈ {“Win”, “Tie”, “Loss”}},

where each ai reflects the result of the ith game.

One interesting event might be the event in which ai = “Win” for alli, corresponding to an undefeated season. Another interesting eventis the set

{(a1, . . . , a12) | ∃j ∈ [12] such that ai = “Loss” ∀i ≤ j and ai = “Win” ∀i > j}.

This event corresponds to all possible seasons in which the Game-cocks lose their first j games (here, j is nonzero), but rally to winthe remaining games.

To assign probabilities to each element of the sample space, we definea probability function pi for each ai. This can be accomplished byconsidering the history of the Gamecocks versus the opposing teamin game i and setting

pi(“Win”) =Games won against team i

Total games against team i

pi(“Tie”) =Games tied against team i

Total games against team i

pi(“Loss”) =Games lost against team i

Total games against team i.

Now, for each elementary event ω = (a1, . . . , a12), set

P (ω) =∏i∈[12]

pi(ai).

1

Finally, for any subset A of Ω, define

P (A) =∑ω∈A

P (ω).

(b) Observe the change (in percent) during a trading day of the DowJones Industrial Average. Letting X denote the random variablecorresponding to this change, we are observing

X = 100Value at Closing−Value at Opening

Value at Opening.

Solution: Strictly speaking, X may take on any real value. In theinterest of cutting down the sample space somewhat, we may roundX to the nearest integer. Thus, Ω = Z.One interesting event is X = 0, corresponding to no net change forthe day. Another interesting event is X = 100, corresponding to adoubling in value for the day.

An elementary event corresponds to specifying a single value m for X.A very rough way to define this probability to examine the (rounded)percent change data for all days that the DJIA has been monitoredand set

P (m) =Occurrences of m

Number of days in data set.

For an arbitrary subset of Z, we extend linearly, as before.(c) The DJIA is actually monitored continuously over a trading day. The

experiment is to observe the trajectory of values of the DJIA duringa trading day.

Solution: Suppose we sample the data every second and compile itinto a piecewise linear function f . The trajectory at time t (in secondsafter the opening bell) is given by g(t) = f(t) − f(t − 1), where wetake g(0) = 0. As before, g may take on any real value. We maycombat this by partitioning the real line into intervals of the form[x, x+ �) for some fixed � > 0. Our elementary events, therefore, areordered pairs (t, [x, x+ �)), corresponding to the trajectory at time tfalling into the interval [x, x+�). The sample space Ω is the collectionof all such elementary events.

One interesting (and highly suspicious) event might be {(t, [0, �)) |any t}, corresponding to a day in which the DJIA saw nearly no

change throughout the day. Anoter interesting event might be

{(t, I) | I =⋃x>0

[x, x+ �), any t},

corresponding to the event where the DJIA saw positive trajectorythroughout the day.

2

The probabilities might be assigned as in part b, where we now fur-ther divide the data to reflect the value of t. That is, we do not wantthe probability of seeing a given trajectory, but the probability ofseeing a given trajectory at a given time.

(d) Let G be the grid of points {(x, y) | x, y ∈ {−1, 0, 1}}. Considerthe experiment where a particle starts at the point (0, 0) and ateach timestep the particle either moves (with equal probability) to apoint (in G) that is “available” to its right, left, up, or down. Theexperiment ceases the moment the particle reaches any of the fourpoints (−1,−1), (−1, 1), (1,−1), (1, 1).Solution: One natural probability to assess is the probability thatthe experiment ceases after n steps. We note, however, that it is pos-sible (though infinitely-unlikely) that the experiement never ceases.Thus, we take the sample space to be Ω = Z+ ∪∞.One interesting event is that the experiment ceases after exactly 2steps (the minimum steps required to reach a termination state).Another intesting event is that the experiment takes at least 100 (orany constant number) steps before ceasing.

This problem suggests that an exact solution may be found usingMarkov chains. Barring that, we might run a computer simulationto gather data. From this data, we can set

P (m) =Number of occurrences of m

Total number of trials,

and then extend linearly to more general events.

(e) The experiment is to randomly generate a point on the surface of theunit sphere.

Solution: Given the abstract nature of the problem, we decline toimpose any artificial discretization as was done in previous problems.Now, any point in R3 may be specified by a spherical coordinate(r, θ, φ), where r denotes radial distance, θ inclination, and φ az-imuth. Since we are restricted to the unit sphere, we may discard rand consider ordered pairs (θ, φ). Thus,

Ω = {(0, 0)} ∪ {(π, 0)} ∪ {(θ, φ) | θ ∈ (0, π), φ ∈ [0, 2π)}

(the restrictions on θ and φ are to ensure a unique representation ofeach point).

One interesting event might be {(0, 0)} ∪ {(π, 0)}, corresponding tothe random point lying at either the north or south pole of the sphere.Another interesting event might be {(π2 , φ) | φ ∈ [0, 2π)}, correspond-ing to the random point lying somewhere along the equator.

As a point in the plane has measure zero, we cannot assign probabil-ities to elementary events and extend. Instead, given a subset A ofΩ, we must set P (A) to be the measure of A as a subset of R2.

3

2. (Secretary Problem) You have in your possession N balls, each labelledwith a distinct symbol. In front of you are N urns that are also labelledwith the same symbols as the balls. Your experiment is to place the ballsat random into these boxes with each box getting a single ball.

(a) Write down the sample space Ω of this experiment. How many ele-ments are there in Ω?

Solution: For simplicity, let the symbols be the first N integers.Thus,

Ω = {(a1, . . . , aN ) | ai ∈ [N ]∀i},

where ai = j ∈ [N ] means that the bucket labelled i received the balllabelled j. Observe that Ω is simply the collection of all permutationsof the N distinct objects, so |Ω| = N !.

(b) What probabilities will you assign to the elementary events in |Ω|?Solution: Each elementary event is equally-likely, so P (ω) = 1N ! forall ω ∈ Ω.

(c) Define a match to have occurred in a given box if the ball placed inthis box has the same label as the box. Let AN be the event thatthere is at least one match. What is the probability of AN?

Solution: For each i ∈ [N ], let Bi denote the set of arrangmentshaving a match in bucket i. Thus,

⋃i∈[N ]Bi is the collection of all

arrangements having at least one match. By the inclusion-exclusionprinciple, we have⋃i∈[N ]

Bi =∑i∈[N ]

|Bi| −∑

i,j∈[N ]i 6=j

|Bi ∩Bj |+ · · ·

=

(N

1

)|B1| −

(N

2

)|B1 ∩B2|+ · · · (since |Bi| = |Bj | for all i, j)

=

(N

1

)(N − 1)!−

(N

2

)(N − 2)! + · · ·

=∑i∈[N ]

(−1)i−1N !i!.

Therefore,

P (AN ) =1

N !

∑i∈[N ]

(−1)i−1N !i!

=∑i∈[N ]

(−1)i−1 1i!.

(d) When you let N → ∞, does the sequence of probabilities P (AN )converge?

4

Solution: It is well known that∑i∈N

(−1)i−1 1i!

=1

e.

(e) Is the answer in (d) surprising to you in the sense that it did notcoincide with your initial expectation of what the probability of atleast one match is when N is large? Provide some discussion.

Solution: I recall that, when first encountering this problem, I wasunable to form a conjecture either way. On the one hand, as N grows,the chance of placing a given ball in the right bucket is approaching0. On the other hand, the number of chances to get a match (i.e.the number of balls and buckets involved) is growing without bound.Whenever an infinite number of terms are involved, strange thingsmay happen. Regardless, I suspected to find the probability to be 0or 1. That is converges to something in between is quite astonishing.That it involves e is a nice feature, though not terribly surprisingconsidering the importance of factorials in the problem.

3. A box contains N identically-sized balls with K of them colored red andN −K colored blue. Consider the following two random experiments.Experiment 1: Draw n balls in succession without replacement, takinginto account the order in which the balls are drawn.

Experiment 2: Draw n balls in succession without replacement, disre-garding the order in which the balls are drawn.

If you let Ak be the event that there are exactly k red balls in the sample,do you get the same probability with Experiment 1 and Experiment 2?Justify your answer.

Solution: The probability is the same in both experiments. To see this,suppose there are ` distinct ways to draw a total of k red balls in whichorder matters (as in Experiment 1). Associated with each such event isa probability pi of witnessing the i

th ordering. Since these events areelementary (and so disjoint), we have P (Ak) =

∑i∈[`] pi in Experiment

1. In Experiment 2, an elementary event is drawing exactly k red balls inany order. This event may be viewed, however, as the collection of the `equivalent orderings, and so we still compute P (Ak) =

∑i∈[`] pi.

4. Prove the following basic results from set theory. Here, A,B,C, . . . aresubsets of some sample space Ω.

(a) A ∪ (B ∪ C) = (A ∪B) ∪ C

5

Solution:

x ∈ A ∪ (B ∪ C)⇔ x ∈ A or x ∈ B ∪ C⇔ x ∈ A or x ∈ B or x ∈ C⇔ x ∈ A ∪B or x ∈ C⇔ x ∈ (A ∪B) ∪ C

(b) A ∪ (B ∩ C) = (A ∪B) ∩ (A ∪ C)Solution: Observe that A∪(B∩C) ⊂ A∪B and A∪(B∩C) ⊂ A∪C.Hence, A ∪ (B ∩ C) ⊂ (A ∪B) ∩ (A ∪ C).Next, let x ∈ (A ∪ B) ∩ (A ∪ C). Thus, x ∈ A ∪ B and x ∈ A ∪ C.If x /∈ A, then x ∈ B and x ∈ C. That is, x ∈ B ∩ C. Hence,x ∈ A ∪ (B ∩ C).

(c) (DeMorgan’s Laws) Let {Aα | α ∈ A} for some index set A whereeach Aα is a subset of Ω. Prove that( ⋃

α∈AAα

)c=⋂α∈A

Acα.

Solution:

x ∈

( ⋃α∈A

Aα

)c⇔ x /∈

⋃α∈A

Aα

⇔ x /∈ Aα for all α ∈ A⇔ x ∈ Acα for all α ∈ A

⇔ x ∈⋂α∈A

Acα

(d) Let A1, A2, . . . be a sequence of subsets of Ω. Define the sequenceB1, B2, . . . according to

B1 = A1

B2 = Ac1 ∩A2

...

Bn = Ac1 ∩Ac2 ∩ · · · ∩Acn−1 ∩An

...

Prove that B1, B2, . . . is a pairwise disjoint sequence and that, foreach n, ⋃

j∈[n]

Aj =⋃j∈[n]

Bj

6

so that, in particular, ⋃j∈N

Aj =⋃j∈N

Bj .

Solution:

To see that the sequence is pairwise disjoint, let i, j ∈ N with i 6= j.Without loss of generality, let i < j. It follows immediately thatBi ∩Bj ⊂ Ai ∩Aci = ∅.For the second claim, observe first that Bj ⊂ Aj for all j ∈ [n], so⋃j∈[n]Aj ⊃

⋃j∈[n]Bj . For the reverse inclusion, let x ∈

⋃j∈[n]Aj .

This implies that, for some subset S of [n], x ∈ Ai for all i ∈ S.Let i0 be the least element of S. Thus, x ∈ Ai0 and x /∈ Aj for all1 ≤ j < i0. In other words,

x ∈

⋂j 0, define the basic open neighborhood U = (x−�, x+�). Choose N so that 1n < � for all n ≥ N . Thus, U∩An 6= ∅ for all n ≥ N .As AN ⊂

⋃n≥k An, for any k, it follows that B ∩

⋂n∈N

⋃n≥k Ak 6= ∅.

Hence,⋂n∈N

⋃n≥k Ak = [−1, 1]. Similarly, as AN ⊂

⋂n≥N An, B ∩⋃

n∈N⋂n≥k Ak 6= ∅. Hence,

⋃n∈N

⋂n≥k Ak = [−1, 1].

7

1. Let Ω = (0, 1] and consider the class of subsets of Ω given by

F0 ={A = ∪nj=1Ij : Ij = (aj , bj ] ⊂ Ω, n ∈ {0, 1, 2, . . .}

}.

That Ω ∈ F0 and if A ∈ F0 then Ac ∈ F0 are immediate. Show formallythat F0 is closed under finite unions, that is, if A1, A2, . . . , An are in F0,then ∪ni=1Ai ∈ F0. [These set of properties establishes that F0 is a field.]Solution: LetA1, . . . , An ∈ F0. Now, eachAi can be written as

⋃nij=1(aij , bij ].

Thus, we have

n⋃i=1

Ai =

n⋃i=1

ni⋃j=1

(aij , bij ]

=⋃k∈I

(ak, bk],

where I is the collection of all indices ij appearing in⋃nij=1(aij , bij ] as i

ranges from 1 to n. As n is finite and ni is finite for all i, it follows that|I| is finite, and so

⋃ni=1Ai ∈ F0.

2. For the Ω and F0 in Problem 1, define the set function P : F0 → < via:for A = ∪ni=1(aj , bj ] where (aj , bj ] ∩ (ai, bi] = ∅ for i 6= j, we have

P (A) =

n∑i=1

(bi − ai).

Establish that P is indeed a function by showing that for two differentrepresentations of A, you obtain the same value for P (A) according to thepreceding definition.

Solution: Let A ∈ F0 be given with representations⋃ni=1(ai, bi] and⋃m

i=1(ci, di]. Choose any maximally connected interval (x, y] of A. Thus,after appropriate reordering of the indices, (x, y] is of the form

(x, y] = (a1, a2] ∪ (a2, a3] ∪ · · · ∪ (an−1, an]= (x, a2] ∪ (a2, a3] ∪ · · · ∪ (an−1, y].

Using the other representation of A, we have also that

(x, y] = (c1, c2] ∪ (c2, c3] ∪ · · · ∪ (cn−1, cm]= (x, c2] ∪ (c2, c3] ∪ · · · ∪ (cn−1, y].

Now, using the first representation of A,

P (A) =

n∑i=1

(bi − ai)

= (a2 − a1) + (a3 − a2) + · · ·+ (an − an−1)= an − a1= y − x.

8

Using the second representation of A,

P (A) =

n∑i=1

(di − ci)

= (c2 − c1) + (c3 − c2) + · · ·+ (cm − cm−1)= cm − c1= y − x.

Now, since the maximally connected intervals in A are disjoint, P (A) isjust the sum of its value on these intervals. As we have demonstrated thatP agrees on all maximally connected intervals of A under both represen-tations, it follows that P agrees on all of A.

3. Let Ω be an uncountable sample space. A subset A ⊂ Ω is said to beco-countable if Ac is countable. Show that the class of subsets

C = {A ⊂ Ω : A is countable or co-countable}

is a σ-field of subsets in Ω.

Solution: As Ωc = ∅, we see that Ω ∈ C.If A ∈ C, then A is either countable or co-countable. Hence, Ac is eitherco-countable or countable, respectively. Thus, Ac ∈ C.Let {Ai} be a countable collection of elements of C. If each Ai is countable,then

⋃{Ai} is countable. If at least one of the elements, say A1, is co-

countable, then we have (⋃{Ai}

)c=⋂{Aci}

⊆ Ac1,

which is countable. That is,⋃{Ai} is co-countable.

4. Let Ω be an uncountable set, and let C = {{ω} : ω ∈ Ω} be the class ofsingleton subsets of Ω. Show that the σ-field generated by C is the σ-fieldconsisting of countable and co-countable sets.

Solution: Let C denote the σ-field generated by C, and let D denote theσ-field consisting of all countable and co-countable subsets of Ω.

Since any singleton set is countable, it is clear that C ⊆ D. As C is theintersection of all σ-fields containing C, it follows that C ⊆ D.Let now C0 be any σ-field containing C. As C0 contains every singletonsubset of Ω and is closed under countable union, it follows that C0 containsevery countable subset of Ω. Now, since C0 contains every countable subsetof Ω and is closed under complementation, it follows that C0 contains everyco-countable subset of Ω. Hence, D ⊂ C0. As C0 was chosen arbitrarily,we see that D is contained in any σ-field containing C, and so D ⊆ C.

9

5. Suppose C is a non-empty class of subsets of Ω. Let A(C) be the minimalfield over C (i.e., the field generated by C). Show that A(C) consists of setsof the form

∪mi=1 ∩nij=1 Aij

where Aij ∈ C or Acij ∈ C, and where the m sets ∩nij=1Aij , i = 1, 2, . . . ,m

are disjoint.

Solution: In what follows, let

F =

m⋂j=1

Aj | Aj ∈ C or Acj ∈ C

and

D =

{n⊔i=1

Fi | Fi ∈ F , {Fi}i∈[n] pairwise disjoint

}.

With this notation, we must show that A(C) = D.To see that D ⊆ A(C), notice that A(C) is a field containing C, and sois closed under complementation, finite unions, and finite intersections ofelements of C. In particular, A(C) must contain any element of the formspecified by D.We show next that A(C) ⊆ D. To accomplish this, note first that C ⊆ D.Thus, if we can show that D is itself a field, we will have the desiredinclusion, as A(C) is the smallest field containing C.Before proceeding, we establish a useful lemma.

Lemma 0.1. If F ∈ F , then F c ∈ D.

Proof. Let F ∈ F be of the form F =⋂j∈[m]Aj , where Aj or A

cj belongs

to C for each j and the collection of all Aj is pairwise disjoint. It followsthat

F c =⋃j∈[m]

Acj

= Ac1 ∪ (Ac2 ∩A1) ∪ · · · ∪ (Acm ∩A1 ∩ · · · ∩Am−1).

Consider a typical term Ack ∩ A1 ∩ · · · ∩ Ak−1 in the union. This set isevidently an element of F , as Ai or Aci belongs to C for all i (similarly forAck). Moreover, the collection of all terms in the union is pairwise disjointvia disjointification. Thus, F c ∈ D.

We now return to the task of showing that D is a field.Let A ∈ C. It follows that F contains A ∩ Ac = ∅. Thus, by the lemma,∅c = Ω belongs to D.

10

Next, we show that D is closed under finite intersections. To that end, letD1, D2 ∈ D with D1 =

⊔i∈[n1] Fi and D2 =

⊔j∈[n2] F

′j . Now,

D1 ∩D2 =

⊔i∈[n1]

Fi

∩ ⊔j∈[n2]

F ′j

=

⋃j∈[n2]

(F1 ∩ F ′j) ∪ · · · ∪⋃

j∈[n2]

(Fn1 ∩ F ′j).

Observe that this last line is a union of elements of the form Fi∩F ′j . As Fis closed under finite intersection, each term of the union is a member ofF . Moreover, the collection of all these terms is pairwise disjoint. To seethis, consider some distinct Fi1 ∩F ′j1 and Fi2 ∩F

′j2

in the union. Withoutloss of generality, let Fi1 6= Fi2 . By definition of D1, it must be that Fi1and Fi2 are disjoint, and thus Fi1 ∩F ′j1 and Fi2 ∩F

′j2

are disjoint. All told,we have that D1 ∩D2 ∈ D. Proceeding by induction, we have that D isclosed under finite intersections.

Finally, we show that D is closed under complementation. To that end,pick any D ∈ D with D =

⊔i∈[n] Fi. It follows that D

c =⋂i∈[n] F

ci .

By the lemma, each F ci is an element of D. As D is closed under finiteintersections, Dc ∈ D.Taken together, we have verified that D is indeed a field containing C, andso A(C) ⊆ D. Therefore, A(C) = D, as desired.

6. Let C be a class of subsets of Ω. It is said to be a monotone class if forA1 ⊂ A2 ⊂ . . . in C, then ∪∞n=1An = limAn ∈ C and for A1 ⊃ A2 ⊃ . . . inC, then ∩∞n=1An = limAn ∈ C. Prove that if C is a field and a monotoneclass, then it is a σ-field.

Solution: Since C is a field, we have Ω ∈ C and closure under comple-mentation.

Let now {Ai} be a countable collection of elements of C. Define, for allk, Bk =

⋃ki=1Ai. Thus, {Bi} is a monotone sequence of subsets of C,

and so⋃{Bi} ∈ C. It is clear, however, that

⋃{Ai} =

⋃{Bi}, and so we

conclude that⋃{Ai} ∈ C, thus verifying that C is a σ-field.

7. Let Ω = < and consider the two classes of subsets given by

C1 = {(a, b) : a, b are rationals in R};

C2 = {[a, b] : a, b are in R}.Establish that these two classes of sets generate the same σ-field. (Theircommon generated σ-field is the Borel σ-field in

andC2 =

⋂{D | D is a σ-field and C2 ⊂ D}.

We proceed by showing that any σ-field containing C1 contains C2 and viceversa, thus establishing that C1 = C2.

Let D1 be any σ-field containing C1. We need to show that, for anya, b ∈ R, [a, b] ∈ D1. To that end, choose two sequences {an} and {bn} ofrational numbers with an ↗ a and bn ↘ b. Now, (an, bn) ∈ D1 for all n,and so D1 contains

⋂n∈N(an, bn) = [a, b].

Let D2 be any σ-field containing C2. We need to show that, for anya, b ∈ Q, (a, b) ∈ D2. To that end, choose two sequences {an} and {bn} ofreal numbers with an ↘ a and bn ↗ b. Now, [an, bn] ∈ D2 for all n, andso D2 contains

⋃n∈N[an, bn] = (a, b).

1. Let S be a semi-algebra of subsets of Ω. Denote by A(S) the algebra orfield generated by S and by σ(S) the σ-algebra generated by S. Provethat

σ(A(S)) = σ(S).

Solution: Since S ⊂ A(S), we have immediately that σ(S) ⊂ σ(A(S)).To demonstrate the reverse inclusion, it suffices to show that σ(S) is itselfa σ-algebra containing A(S). To that end, recall that every element ofA(S) is of the form

m⊔i=1

Si

where S ∈ S. Now, as σ(S) is countable union, such elements belong toσ(S). Thus, σ(S) is a σ-algebra containing A(S), and so σ(A(S)) ⊂ σ(S).

2. Let Ω = C[0, 1], the space of continuous functions on [0, 1]. For t ∈ [0, 1]and a, b ∈

(b) Describe the structure of the typical element of S0 ≡ S(C0), thesemi-algebra generated by C0.Solution: We claim that a typical element of S0 has the form⋂ni=1Ai where, for each i, Ai = A(ti; ai, bi) or Ai = A(ti; ai, bi)

c =A(ti;−∞, ai) ∪ A(ti; bi,∞) with ti ∈ [0, 1], ai, bi ∈ R, where we un-derstand that intervals of the form (a,∞] should be interpreted as(a,∞).By the observations in part a, it suffices to show that S0 as it isdefined above is a σ-algebra (the new elements we’ve included arerequired at a minimum to patch up the deficiencies of C0). Now,it is clear that ∅ ∈ S0 (take the intersection of any disjoint balls)and Ω ∈ S0 (Ω can be represented, for example, by A(0; 0, 0)c). Byconstruction, S0 is closed under complementation. It is also closedunder finite intersection, since intervals of the form (a, b] with a, b ∈R ∪ {−∞,∞} are closed under finite intersection. Therefore, S0 isa semi-algebra, and so must be the smallest semi-algebra containingC0.

(c) Describe the structure of the typical element of A(C0), the algebragenerated by C0.Solution: By a previous homework, we know that the elements ofA(C0) are of the form

m⊔i=1

nj⋂j=1

Aij ,

where, for all i and j, Aij ∈ C0 or Acij ∈ C0 and the m sets⋂njj=1Aij

are pairwise disjoint. In this particular example, we know that Aij =A(t; a, b) and that Acij = A(t; a, b)

c = {f ∈ Ω | f(t) ∈ (−∞, a] ∪(b,∞)} for some t ∈ [0, 1] and a, b ∈ R. Adopting the conventions asin part b, we can represent any element of A(C0) as

m⊔i=1

nj⋂j=1

Ai.

(d) Denoting by B0 = σ(C0), the σ-field generated by C0, determine ifthe subset of Ω given by

E = {f ∈ Ω : supt∈[0,1]

|f(t)| ≤ B}

is an element of B0. [By the way, B0 is called the σ-field generatedby the cylinder sets.]

Solution: Let the sequence (rn) be an enumeration of Q∩ [0, 1] anddefine

F =

∞⋂n=1

A(rn,−B,B).

13

Since B0 is closed under countable intersection, we have that F ∈ B0.We claim that E = F .

Evidently, E ⊆ F , as any function f satisfying supt∈[0,1] |f(t)| ≤ Bsatisfies |f(rn)| ≤ B for all n.Suppose now, for the purpose of contradiction, that F 6⊆ E. That is,there exists continuous f and t0 ∈ R \Q such that |f(t0)| > B. Pickany 0 < � < B − |f(t0)|. By the continuity of f , there is δ > 0 suchthat, whenever |x− y| < δ, |f(x)− f(y)| < �. Now, as Q is dense inR, we can find a rational number r such that |t0 − r| < δ, yet

|f(t0)− f(r)| > ||f(t0)| − |f(r)||≥ ||f(t0)| −B|> �,

which is contrary to the continuity of f . Hence, F ⊆ E, as desired.

3. Same Ω as in Problem 2. Define the metric (distance) function d : Ω×Ω→< via

d(f, g) = supt∈[0,1]

|f(t)− g(t)|.

For � > 0 and f ∈ Ω, define the subset B(f ; �) of Ω according to

B(f ; �) = {g ∈ Ω : d(f, g) < �}.

These are the open balls in Ω. Gather these open balls in the collectionS1, that is,

S1 = {B(f ; �) : f ∈ Ω, � > 0}.

(a) Determine if S1 is a semi-algebra in Ω; and if it is not, find thesemi-algebra generated by S1.Solution: S1 does not contain Ω, and so is not a semi-algebra. Tosee this, observe that, for any f ∈ Ω and finite � > 0, there is acontinuous function on [0, 1] not contained in B(f ; �) (for example,the constant function M + 2� with M = supt∈[0,1] |f(t)|).For the same reason,

B(f ; �)c = {g ∈ Ω | d(f, g) ≥ �}

cannot be represented as a finite union of B(fi, �i) (for example, theconstant function M + 2� with M = maxi∈[n]{supt∈[0,1] |fi(t)|} willnot be contained in this union).

As before, we represent elements of S(S1) by⋂ni=1 Si where, for all

i, Si ∈ S1 or Sci ∈ S1.(b) Determine the algebra generated by S1, that is, A(S1).

14

Solution: In a previous homework, we have established this resultin more generality. In this particular case, we have

A(S1) =m⊔i=1

nj⋂j=1

Sij

with Sij ∈ S1 or Sij ∈ S2 for all i, j and the m sets⋂njj=1 Sij are

pairwise disjoint.

(c) Denote by B1 = σ(S1), the σ-field generated by S1, so that by defini-tion this is the Borel σ-field associated with the metric d. Determineif the subset of Ω defined in item (d) in Problem 2 is an element ofthis Borel σ-field.

Solution: The set

E =

{f ∈ Ω : sup

t∈[0,1]|f(t)| ≤ B

}

can be represented in B1 as∞⋂n=1

B

(0;B +

1

n

).

Evidently, E ⊂ B(0;B + 1n

)for each n, and so E ⊂

⋂∞n=1B

(0;B + 1n

).

Let now g ∈⋂∞n=1B

(0;B + 1n

). This implies that supt∈[0,1] |g(t)| <

B+ 1n for all n. Hence, supt∈[0,1] |g(t)| ≤ B, and so g ∈ E. Therefore,E =

⋂∞n=1B

(0;B + 1n

).

4. Investigate the relationship between B0 in Problem 2 and B1 in Problem3. Are these σ-fields identical; or is one strictly containing the other, andif so, which one is the larger σ-field?

Solution: We claim that B0 = B1. To prove this, we demonstrate thatany basic open ball of B0 can be represented as a countable union of basicopen balls of B1, and vice versa.Pick some A(t; a, b) ∈ B0. It is (relatively) well-known that the collectionof continuous piecewise linear functions on [0, 1] is countable and dense inthe collection of all continuous functions on [0, 1]. Denote by F the subsetof continuous piecewise linear functions on [0, 1] contained in A(t; a, b).Now, for each f ∈ F , define �f = min{f(t)− a, b− f(t)}. We claim that

A(t; a, b) =⋃{B(f, �f ) | f ∈ F} .

By our choice of �f , each B(f, �f ) ⊆ A(t; a, b). Conversely, if g ∈ A(t; a, b),then the density of F in A(t; a, b) guarantees that there is f ∈ F such thatd(f, g) < �f , and so g ∈ B(f, �f ). Hence, B0 ⊆ B1.

15

For the reverse inclusion, pick some B(f, �) ∈ B1 and let (rn) be an enu-meration of Q ∩ [0, 1]. We claim that

B(f, �) =

∞⋂n=1

A(rn; f(rn)−

�

2, f(rn) +

�

2

).

By construction, A(rn; f(rn)− �2 , f(rn) +

�2

)⊂ B(f, �) for each n. Con-

versely, if f ∈ B(f, �), then the density of the rationals in [0, 1] guaranteesthat f ∈ A

(rn; f(rn)− �2 , f(rn) +

�2

)for each n. Hence, B1 ⊆ B0.

5. As you might have already noticed, the typical element of S0 in Problem2 is of form ∩ni=1A∗ti where the tis are distinct and A

∗t is either of form

A(t; a, b) or A(t; a, b)c. Define the (set)-function P : S0 → < according to

P[∩ni=1A∗ti

]=

n∏i=1

[∫A∗ti

φ(v)dv

]

where φ(z) = (2π)−1/2 exp{−v2/2} is the standard normal density func-tion. Assuming that P is σ-additive on S0, provide a reason [do not proveanything, just a reason!] why you could conclude that there is a uniqueprobability measure W on B0 which extends P , that is, W|S0 = P. [Thisprobability measure W is infact the Wiener measure on (C[0, 1],B0)].Solution: The desired result is a direct application of the first and secondextension theorems. Since P is σ-additive on the semi-algebra S0, we canextend it uniquely to a probability measure P′ on A(S0). Applying thesecond extension theorem to P′, we can obtain the desired probabilitymeasure W on σ(S0) = B0.

Problem 1

Let Ω be some non-empty set, and let A and B be two subsets of Ω which arenot disjoint and with Ω 6= A ∪B. Define the semi-algebra

S = {∅,Ω, A,B,AB,AcB,ABc, AcBc}.

Define the function P : S→ < according to the following specification:

P (∅) = 0;P (Ω) = 1;P (A) = .2;P (B) = .6;P (AB) = .12;P (AcB) = .48;P (ABc) = .08;P (AcBc) = .32

Find the σ-field, A, generated by S, i.e., enumerate all the elements of thisσ-field. [Note: This should coincide with the algebra or field generated by S.]

Proof. Since S is finite, the σ-field generated by S coincides with the fieldgenerated by S. Thus, it suffices to consider A(S). Now, we know that A(S) isthe collection of all sums of finite families of mutually disjoint subsets of Ω in S(Resnick, Lemma 2.4.1). A first calculation gives the following as the elementsof A.

16

∅ A ∪AcB A ∪AcB ∪AcBc AB ∪AcB ∪ABc ∪AcBcΩ A ∪AcBc B ∪ABc ∪AcBcA B ∪AcB AB ∪AcB ∪ABcB B ∪AcBc AB ∪ABc ∪AcBcAB AB ∪AcB AcB ∪ABc ∪AcBcAcB AB ∪ABcABc AB ∪AcBcAcBc AcB ∪ABc

AcB ∪AcBcABc ∪AcBc

Many of these elements, however, are redundant. For example,

A ∪AcB ∪AcBc = A ∪Ac(B ∪Bc)= A ∪AcΩ= A ∪Ac

= Ω.

Removing redundant elements gives the following representation of A.

∅ A ∪AcBΩ A ∪AcBcA B ∪AcBcB AB ∪AcBcAB AcB ∪ABcAcB AcB ∪AcBcABc ABc ∪AcBcAcBc

Find the (unique) extension of P to A. You must enumerate the values ofthis extension for each possible element of A.

Proof. Since S is a semi-algebra, the unique extension P ′ of P to A is definedby

P ′

(∑i∈I

Si

)=∑i∈I

P (Si),

(Resnick, Theorem 2.4.1). By direct calculation, we define P ′ on each elementof A.

P (∅) = 0 P (A ∪AcB) = .68P (Ω) = 1 P (A ∪AcBc) = .52P (A) = .2 P (B ∪AcBc) = .92P (B) = .6 P (AB ∪AcBc) = .44P (AB) = .12 P (AcB ∪ABc) = .56P (AcB) = .48 P (AcB ∪AcBc) = .8P (ABc) = .08 P (ABc ∪AcBc) = .4P (AcBc) = .32

17

Problem 2

Show that a σ-field cannot be countably infinite, i.e., either it has a finite car-dinality or it is at least as large as R.

Proof. Let σ be an infinite σ-field of subsets of Ω and suppose we can find asequence {An | n ∈ N} of pairwise disjoint subsets of σ. Consider the functionf from the collection of infinite binary strings into σ defined by

f(s) =⋃i∈Is

Ai,

where Is is the set of indices on which s is 1. Now, given two binary strings s0and s1,

f(s0) = f(s1)⇒⋃i∈Is0

Ai =⋃i∈Is1

Ai

⇒ Is0 = Is1 (since the Ai are pairwise disjoint)⇒ s0 = s1.

Hence, f is injective, and so the cardinality of σ is at least ℵ1.It remains to show that we can indeed produce a sequence of pairwise disjoint

subsets of σ. To that end, let A ∈ σ. Since σ is closed under complementation,Ac ∈ σ. Now, A∪Ac = Ω, which is infinite (since σ is infinite). Hence, one of Aor Ac is infinite. Without loss of generality, let A be infinite and set A1 = A

c.Consider next the collection of subsets

{A ∩B | B ∈ σ}.

Observe that each A ∩ B is an element of σ, since σ is closed under countableintersection. Now, ⋃

{A ∩B | B ∈ σ} = A,

which is infinite, so some element C of {A∩B | B ∈ σ} is infinite. Set A2 = Ccand consider next

{C ∩B | B ∈ σ}.

Proceeding in this way, we generate a sequence {An} of elements of σ. The Aiare disjoint since, by construction, Ak ⊂ Acj for all k > j. Using this sequencein the above argument yields the desired result.

18

Problem 3

Let B be a σ-field of subsets of Ω and let A ⊂ Ω which is not in B. Show thatthe smallest σ-field generated by {B, A} consists of sets of form

AB1 ∪AcB2, B1, B2 ∈ B.

Proof. Denote by σ the σ-field generated by {B, A} and by C the collection{AB1 ∪ AcB2 | B1, B2 ∈ B}. We have immediately that C ⊆ σ, since σ isclosed under complementation, countable intersection, and countable union, bydefinition. It remains to show that C ⊆ σ. To accomplish this, we need onlyshow that C is itself a σ-field and appeal to the minimality of σ.

Evidently, Ω ∈ C, since Ω ∈ B.To establish closure under complementation, choose any C ∈ C, so that

C = AB1 ∪AcB2 for B1, B2 ∈ B. It follows that

Cc = (AB1 ∪AcB2)c

= (AB1)c ∩ (AcB2)c

= (Ac ∪Bc1) ∩ (A ∪Bc2)= AcA ∪AcBc2 ∪Bc1A ∪Bc1Bc2= AcBc2 ∪Bc1A ∪Bc1Bc2= AcBc2 ∪Bc1A ∪ (ABc1Bc2 ∪AcBc1Bc2)= A(Bc1 ∪Bc1Bc2) ∪Ac(Bc2 ∪Bc1Bc2)= ABc1 ∪AcBc2.

Since B is closed under complementation, Bc1 and Bc2 belong to B, and so C

c

belongs to C.To establish closure under countable unions, choose Ci ∈ C for every natural

number, so that each Ci is of the form ABi ∪AcB′i for Bi, B′i ∈ B. Now,⋃i∈N

Ci =⋃i∈N

(ABi ∪AcB′i)

=⋃i∈N

ABi ∪⋃i∈N

AcB′i)

= A

(⋃i∈N

Bi

)∪Ac

(⋃i∈N

B′i

).

Since B is closed under countable unions, we have that both⋃i∈NBi and⋃

i∈NB′i belong to B, and so

⋃i∈N Ci belongs to C.

Hence, C is a σ-field, and so contains σ. Therefore, σ and C coincide, asdesired.

19

Problem 6

If S1 and S2 are two semialgebras of subsets of Ω, show that

S1S2 = {A1A2 : A1 ∈ S1, A2 ∈ S2}

is again a semialgebra of subsets of Ω.

Proof. For ease of notation, let S denote S1S2. We show directly that S is asemialgebra.

Since both S1 and S2 are semialgebras, Ω ∈ S1 and Ω ∈ S2. Thus,Ω = ΩΩ ∈ S.

To see that S is closed under finite intersection, pick S, S′ ∈ S, so that S =S1S2 and S

′ = S′1S′2 with S1, S

′1 ∈ S1 and S2, S′2 ∈ S2. It follows immediately

that

SS′ = (S1S2)(S′1S′2)

= (S1S′1)(S2S

′2).

Since S1 and S2 are both semialgebras, S1S′1 ∈ S1 and S2S′2 ∈ S2. Thus,

SS′ ∈ S. By a simple induction argument, we have that S is closed underfinite intersection.

It remains to show that, given S ∈ S, Sc can be written as the union of afinite collection of pairwise disjoint elements of S. To that end, let S = S1S2with S1 ∈ S1 and S2 ∈ S2. It follows that,

Sc = (S1S2)c

= Sc1 ∪ Sc2

=

m∑i=1

Ai ∪n∑i=1

Bi,

where the Ai are pairwise disjoint elements of S1 and the Bi are pairwise disjointelements of S2. Denote

∑mi=1Ai by A. Observe that,

m∑i=1

Ai ∪n∑i=1

Bi =

m∑i=1

Ai ∪n∑i=1

BiAc.

Now, the Ai are pairwise disjoint because S1 is a semialgebra. Similarly, the Biare pairwise disjoint because S2 is a semialgebra, and so the BiA

c are pairwisedisjoint. Finally, any Ai is disjoint from any BjA

c, since Ai is disjoint fromAc. Thus, {Ai | i ∈ [m]} ∪ {BiAc | i ∈ [n]} is a pairwise disjoint collection ofelements from S1 and S2. We proceed by showing that each Ai and BiA

c canbe represented by a union of disjoint elements of S.

20

Evidently, every Ai belongs to S1, since each can be represented as AiΩ.Now, for any k ∈ [n], we have

BkAc = Bk

(m∑i=1

Ai

)c

= Bk

m∏i=1

Aci

= Bk

m∏i=1

mi∑j=1

Cij ,

where, for each i ∈ [m], {Cij | j ∈ [mi]} is a pairwise disjoint collection ofelements of S1. Let now M denote the collection of m-tuples

∏mi=1[mi]. We

may rewrite the above as

Bk

m∏i=1

mi∑j=1

Cij = Bk∑x∈M

m∏i=1

Cix(i).

Now, since S1 is a semialgebra, we have that∏mi=1 Cix(i) ∈ S1 for each x ∈M .

Moreover, the collection of all such elements is pairwise disjoint. That is, givenx, y ∈M with x 6= y,

m∏i=1

Cix(i) ∩m∏i=1

Ciy(i) = ∅,

since, for some j ∈ [m], x(j) 6= y(j) and Cj` ∩ Cj`′ = ∅ for any j and ` 6= `′.All told, we have shown that, for each k ∈ [n],

BkAc = Bk

∑x∈M

m∏i=1

Cix(i)

=∑x∈M

Bk

m∏i=1

Cix(i),

which is a disjoint union of elements of S. Therefore, Sc can be representedas a disjoint union of elements of S, thus completing the proof that S is asemialgebra.

Problem 7

Let B be a σ-field of subsets of Ω and let Q : B → < satisfying the followingconditions:

(i) Q is finitely additive on B.

(ii) Q(Ω) = 1 and 0 ≤ Q(A) ≤ 1 for all A ∈ B.

21

(iii) If Ai ∈ B are pairwise disjoint and∑∞i=1Ai = Ω, then

∑∞i=1Q(Ai) = 1.

Show that Q is σ-additive, so that it is, in fact, a probability measure on B.

Proof. Let {An} be a countable sequence of pairwise disjoint elements of B andlet A denote

∑n∈NAn. Since B is a σ-field, A ∈ B, and so Ac ∈ B. Now,

{An | n ∈ N} ∪ {Ac} is a pairwise disjoint collection of elements of B whoseunion is Ω, so it follows that∑

n∈NQ(An) +Q(A

c) = 1 (by property iii)

= Q(Ω) (by property ii)

= Q (A ∪Ac)= Q(A) +Q(Ac) (by property i)

= Q

(∑n∈N

An

)+Q(Ac).

Thus, ∑n∈N

Q(An) = Q

(∑n∈N

An

),

as desired.

Problem 1

Proposition 0.2. Let X : (Ω,F) → (

Proof. Let a ∈ R and consider Y −1(−∞, a]. It follows from the definition of Ythat

Y −1(−∞, a] = {ω ∈ Ω | Y (ω) ∈ (−∞, a]}= {ω ∈ Ω | XN (ω) ∈ (−∞, a]}= {ω ∈ Ω | N(ω) ∈ (−∞, a] and XN(ω)(ω) ∈ (−∞, a]}= {ω ∈ Ω | N(ω) ∈ (−∞, a]} ∩ {ω ∈ Ω | Xn(ω) ∈ (−∞, a], n ∈ (−∞, a] ∩ Z+}

= {ω ∈ Ω | N(ω) ∈ (−∞, a]} ∩

⋃n∈(−∞,a]∩Z+

{ω ∈ Ω | Xn(ω) ∈ (−∞, a]}

.Now, N is a random variable, so {ω ∈ Ω | N(ω) ∈ (−∞, a]} ∈ F. Similarly, Xnis a random variable for each n ∈ N, so each {ω ∈ Ω | Xn(ω) ∈ (−∞, a]} ∈ F.Finally, as F is closed under countable union and countable intersection, we seethat Y −1(−∞, a] ∈ F, and so Y is a random variable.

Problem 3

Proposition 0.4. If X is a random variable, then |X| is also a random variable.

Proof. Let a ∈ R and consider |X|−1(−∞, a]. We know that

|X|−1(−∞, a] = {ω ∈ Ω : |X(ω)| ∈ (−∞, a]}= {ω ∈ Ω : |X(ω)| ∈ [0, a]}= {ω ∈ Ω : X(ω) ∈ [−a, a]}.

Now, since X is measurable and [−a, a] ∈ B, |X|−1(−∞, a] ∈ F, and so |X| isa random variable.

Proposition 0.5. If |X| is a random variable, X need not be a random variable.

Proof. Let N denote some nonmeasurable subset of R and define X : (R,B)→(R,B) via

X(y) =

{−1 if y ∈ N1 if y /∈ N.

We see that |X| ≡ 1, and so

|X|−1(−∞, a] =

{∅ if a < 1R if a ≥ 1.

Hence, |X| is a random variable. At the same time, we have X−1(−∞,−1] = N ,and so X is not a random variable.

23

Problem 4

Proposition 0.6. Let (Ω,B, P ) be ([0, 1],B(0, 1], λ) where λ is the Lebesguemeasure on [0, 1]. Define the process {Xt : 0 ≤ t ≤ 1} according to

Xt(ω) = I{t = ω}.

Each Xt is a random variable.

Proof. Observe first that the range of each Xt is {0, 1}, so it suffices to considerpreimages of the generators of the σ-algebra {∅, {0}, {1}, {0, 1}}, namely {0}and {1}. Let now s ∈ [0, 1] be arbitrary but fixed. We see that

X−1s {0} = (0, 1] \ {s}X−1s {1} = {s}.

As both (0, 1] \ {s} and {s} belong to B(0, 1], we conclude that Xs is a randomvariable.

By the above, we see the σ-field generated by {Xt : 0 ≤ t ≤ 1} consists ofall sets A such that A itself or Ac is a countable union of singletons.

Problem 5

Proposition 0.7. If X and Y are random variables on (Ω,F, P ), then

supA∈B

|P{X ∈ A} − P{Y ∈ A}| ≤ P{X 6= Y }.

Proof. For any A ∈ B,

{X 6= Y } ⊇ X−1(A) ∪ Y −1(A) \ (X−1(A) ∩ Y −1(A)),

and so

|P (X 6= Y )| ≥ |P (X ∈ A) + P (Y ∈ A)− P ((X ∈ A) ∩ (Y ∈ A))|≥ |P (X ∈ A)− P (Y ∈ A)|.

As A was arbitrary, we have that P (X 6= Y ) ≥ supA∈B |P (X ∈ A)−P (Y ∈ A)|,as desired.

Problem 6

Proposition 0.8. If {An : n = 1, 2, 3, . . .} is an independent sequence of events,then

P

{ ∞⋂n=1

An

}=

∞∏n=1

P{An}.

24

Proof. Define for each n ∈ N the set Bn =⋂nk=1Ak. We see that {Bn} is a

nonincreasing sequence of sets, and thus limn→∞Bn =⋂n∈NBn. By definition

of the Bn, however, we have also that⋂n∈NBn =

⋂n∈NAn. It now follows that

P

{ ∞⋂n=1

An

}= P

{limn→∞

Bn

}= limn→∞

P (Bn)

= limn→∞

P

(n⋂k=1

Ak

)

= limn→∞

n∏k=1

P (Ak)

=

∞∏n=1

P (An).

Problem 7

Proposition 0.9. If X and Y are independent random variables and f, g aremeasurable and real-valued, then f(X) and g(Y ) are independent.

Proof. Let A,B ∈ B. Since f and g are measurable and real-valued, thereexist A′, B′ ∈ B such that f−1(A) = A′ and g−1(B) = B′. Since X andY are independent random variables, we have that X−1(A′) and Y −1(B′) areindependent. Thus, we have shown for any A,B ∈ B that X−1(f−1(A)) andY −1(g−1(B)) are independent. In other words, f(X) and g(Y ) are independent.

Problem 8

Proposition 0.10. A random variable X is independent of itself if and only ifthere is some constant c such that P{X = c} = 1.

Proof. (⇒) Choose some event ω ∈ Ω with nonzero probability and set c =X(ω). Since X is independent of itself, we have

0 = P ({ω} ∩ (Ω \ {ω}))= P ({ω})P (Ω \ {ω}) .

Since P ({ω}) > 0, it must be that P (Ω \ {ω}) = 0. Thus, we conclude thatP ({ω}) = 1, and so P (X = c) = 1.

(⇐) Let A,B ∈ B. We consider three cases.

25

If c ∈ A and c ∈ B, then P (X ∈ A) = P (X ∈ B) = 1. Moreover, c ∈ A∩B,so P ([X ∈ A] ∩ [X ∈ B]) = 1.

If c ∈ A and c /∈ B, then P (X ∈ A) = 1 and P (X ∈ B) = 0. Moreover,c /∈ A ∩B, so P ([X ∈ A] ∩ [X ∈ B]) = 0.

If c /∈ A and c /∈ B, then P (X ∈ A) = 0 and P (X ∈ B) = 0. Moreover,c /∈ A ∩B, so P ([X ∈ A] ∩ [X ∈ B]) = 0.

In any case, we have P ([X ∈ A] ∩ [X ∈ B]) = P (X ∈ A)P (X ∈ B), and soA and B are independent. Therefore, X is independent of itself.

Problem 9

Consider the experiment of tossing a fair coin some number of times, so thateach of the possible outcomes in the sample space are equally likely.

Proposition 0.11. There are three events A, B, and C such that every pairare independent, but P (ABC) 6= P (A)P (B)P (C).

Proof. The desired events can be constructed using two flips. Set

A = {TT, TH}B = {TH,HT}C = {HT,HH}.

We have P (A) = P (B) = P (C) = 12 and P (AB) = P (AC) = P (BC) =14 , so

P (AB) = P (A)P (B), P (AC) = P (A)P (C), and P (BC) = P (B)P (C). That is,the events are pairwise independent. At the same time, we have P (ABC) = 0(since ABC = ∅), which is not equal to P (A)P (B)P (C) = 18 .

Proposition 0.12. There are three events A, B, and C such that P (ABC) =P (A)P (B)P (C), but with at least one possible pair not independent.

Proof. The desired events can be constructed using three flips. Set

A = {TTT, TTH, THT, THH}B = {TTT, TTH, THT,HTT}C = {TTT, THH,HTT,HTH}.

We have P (A) = P (B) = P (C) = 12 and P (ABC) =18 , so P (ABC) =

P (A)P (B)P (C). At the same time, we have P (AB) = 38 , which is not equal toP (A)P (B) = 14 .

Proposition 0.13. There are four events A, B, C, and D such that theseevents are independent.

26

Proof. The desired events can be constructed using four flips. Set

A = {TTTT, TTTH, TTHT, THTT, THTH, THHT, THHH,HTTT}B = {TTTT, TTTH, TTHT, TTHH, THTT,HTTH,HTHT,HHTH}C = {TTTT, TTTH, TTHH, THTH, THHT,HTTH,HTHH,HHTT}D = {TTTT, TTHT, TTHH, THHH,HTTT,HTHT,HTHH,HTTT}.

For clarity, we list the possible intersections explicitly.

AB = {TTTT, TTTH, TTHT, THTT}AC = {TTTT, TTTH, THTH, THHT}AD = {TTTT, TTHT, THHH,HTTT}BC = {TTTT, TTTH, TTHH,HTTH}BD = {TTTT, TTHT, TTHH,HTHT}

ABC = {TTTT, TTTH}ABD = {TTTT, TTHT}BCD = {TTTT, TTHH}

ABCD = {TTTT}

From the above, it is routine to check (but lengthy to write out) that the eventsA,B,C,D are indepedent.

Problem 1

Consider then the experiment where a computer generates successive lettersindependently from the Roman alphabet randomly.

Proposition 0.14. The string “MOHR” will appear infinitely often with prob-ability one.

Proof. Break the infinite string generated by the computer into disjoint blocksof four characters each (i.e. the first four characters comprise the first block,the second four characters comprise the second block, and so on). Let Ai be theevent that the ith block is the string “MOHR”. Note that the Ai are independentsince the blocks are disjoint. Now, for all i, P (Ai) =

1264 . Hence,

∞∑n=1

P (An) =∞.

By Borel-Cantelli, An occurs infinitely often with probability one. That is, thestring “MOHR” appears infinitely often with probability one.

27

Problem 3

Proposition 0.15. If {An} are independent events satisfying P (An) < 1 forall n, then

P

{ ∞⋃n=1

An

}= 1 if and only if P (An i.o.) = 1.

Proof. (⇒) Suppose that

P

{ ∞⋃n=1

An

}= 1.

It follows that

0 = P

{( ∞⋃n=1

An

)c}

= P

{ ∞⋂n=1

Acn

}

=

∞∏n=1

P (Acn) (by independence)

=

∞∏n=1

1− P (An).

Now, since P (An) < 1 for all n, the fact that∏∞n=1 1− P (An) = 0 implies that

P (An) > 0 infinitely often. Thus,

0 =

∞∏n=1

e−P (An)

= e−∑∞

n=1 P (An).

Therefore,∞∑n=1

P (An) =∞,

and so P (An i.o.) = 1 by Borel-Cantelli.(⇐) Define Bn to be the event

⋃k≥nAk. Observe that {Bn} is a non-

increasing sequence of events. Now,

1 = P (An i.o.)

= P

∞⋂n=1

⋃k≥n

Ak

= P

( ∞⋂n=1

Bn

)= limn→∞

P (Bn).

28

As the Bn are non-increasing, the sequence {P (Bn)} is non-increasing, and soP (Bn) = 1 for all n. In particular,

1 = P (B1)

= P

( ∞⋃n=1

An

).

In the above proposition, the condition P (An) < 1 for all n cannot bedropped. To see this, consider the sequence {An} where A1 = Ω and An aresets of measure zero for all n ≥ 2. We have that

P

{ ∞⋃n=1

An

}≥ P (Ω)

= 1,

yet

P (An i.o.) = P

∞⋂n=1

⋃k≥n

Ak

≤ P

∞⋂n=2

⋃k≥n

Ak

= 0 (since all Ak are sets of measure zero).

Problem 5

In a sequence of independent Bernoulli random variables {Xn | n = 1, 2, · · · }with

P (Xn = 1) = p = 1− P (Xn = 0),

let An be the event that a run of n consecutive 1’s occurs between trials 2n and

2n+1.

Proposition 0.16. If p ≥ 12 , then P (An i.o.) = 1.

Proof. Break the trials into 2n

n disjoint blocks of length n. The probability ofnot getting all 1’s in a given block is 1−pn, and so the probability of not gettingall 1’s among any of the blocks is

(1− pn) 2n

n .

29

Now,

(1− pn) 2n

n ≤(e−p

n) 2n

n

= e−(2p)n

n .

Hence,

P (An) = 1− (1− pn)2n

n

≥ 1− e−(2p)n

n .

We consider two cases. If p > 12 ,

(2p)n

n=

(1 + �)n

n(some � > 0)

→∞

Thus,

∞∑n=1

P (An) ≥∞∑n=1

1− e−(2p)n

n

=

∞∑n=1

1−∞∑n=1

e−(2p)n

n

=∞,

and so P (An i.o.) = 1 by Borel-Cantelli.If p = 12 , then we use

e−1n =

∞∑k=0

(− 1n)k

k!.

Now,

P (An) = 1−∞∑k=0

(− 1n)k

k!

= 1− 1 + 1n−∞∑k=2

(− 1n)k

k!

=1

n−∞∑k=2

(− 1n)k

k!.

30

Thus,

∞∑n=1

P (An) =

∞∑n=1

(1

n−∞∑k=2

(− 1n)k

k!

)

=

∞∑n=1

1

n−∞∑n=1

∞∑k=2

(− 1n)k

k!

=∞,

and so P (An i.o.) = 1 by Borel-Cantelli.

Problem 7

Proposition 0.17. If the event A is independent of the π-system P and A ∈σ(P), then P (A) is either 0 or 1.

Proof. Since A is independent of P, σ(A) is independent of σ(P) by the BasicCriterion. As A ∈ σ(P), A is independent of itself. Thus,

P (A) = P (A ∩A)= P (A)P (A)

= P (A)2,

and so P (A) is either 0 or 1.

Problem 9

Proposition 0.18. If σ(X1, . . . , Xn−1) and σ(Xn) are independent for all n ≥2, then {Xn | n = 1, 2, . . . } is an independent collection of random variables.

Proof. Let a finite collection {Aij | Aij ∈ X−1ij (B), j ∈ [k]} be given. Withoutloss of generality, suppose i` < im for ` < m. Consider the event Ai1 ∩ · · ·∩Aik .Since

Ai1 ∩ · · · ∩Aik−1 ∈ σ(Xi1 , . . . , Xik−1)⊂ σ(X1, X2, . . . , Xik−1),

which is independent from σ(Xik), it follows that

P (Ai1 ∩ · · · ∩Aik−1 ∩Aik) = P (Ai1 ∩ · · · ∩Aik−1)P (Aik).

Proceeding inductively, we conclude that

P (Ai1 ∩ · · · ∩Aik−1 ∩Aik) =k∏j=1

P (Aij ),

as desired.

31

Problem 10

Proposition 0.19. Given a sequence of events {An | n = 1, 2, . . . } withP (An)→ 1, there exists a subsequence {nk} tending to infinity such that

P

(⋂k

Ank

)> 0.

Proof. Since P (An) → 1, we have that, for all � > 0 there exists N such thatP (An) > 1 − � for all n ≥ N . Choose a sequence of positive �k for k ≥ 1satisfying ∑

k

�k < 1.

(For example, �k =14k

suffices.) From the above, we can find corresponding nkfor each k such that P (Ank) > 1− �k. Now,

P

(⋂k

Ank

)= 1− P

((⋂k

Ank

)c)

= 1− P

(⋃k

Acnk

)≥ 1−

∑k

P (Acnk)

> 1−∑k

�k

> 0.

Problem 1

Let (Ω,F, P ) be a probability space, and let {An} and A belong to F. Let X bea random variable defined on this probability space with X ∈ L1.

Proposition 0.20.

limn→∞

∫|X|>n

XdP = 0

Proof. Define the sequence of random variables Xn for n ∈ N via

Xn(ω) =

{X(ω) if |X(ω)| > n0 otherwise.

Observe that, for all n, ∫|X|>n

XdP =

∫Ω

XndP.

32

Since ∣∣∣∣∫Ω

XndP

∣∣∣∣ ≤ ∫Ω

|Xn|dP,

it suffices to show that∫

Ω|Xn|dP → 0.

Now, |Xn| → 0 almost everywhere, since |X| is finite almost everywhere.Moreover, |Xn| ≤ |X| for all n. Thus, by the Dominated Convergence Theorem,

limn→∞

∫Ω

|Xn|dP =∫

Ω

limn→∞

|Xn|dP

=

∫Ω

0dP

= 0.

Proposition 0.21. Iflimn→∞

P (An) = 0,

then

limn→∞

∫An

XdP = 0.

Proof. Define the sequence of random variables Xn for n ∈ N via Xn = X ·1An .Observe that, for all n, ∫

An

XdP =

∫Ω

XndP.

Since ∣∣∣∣∫Ω

XndP

∣∣∣∣ ≤ ∫Ω

|Xn|dP,

it suffices to show that∫

Ω|Xn|dP → 0.

Now, |Xn| → 0, since P (An)→ 0. Moreover, |Xn| ≤ |X| for all n. Thus, bythe Dominated Convergence Theorem,

limn→∞

∫Ω

|Xn|dP =∫

Ω

limn→∞

|Xn|dP

=

∫Ω

0dP

= 0.

Proposition 0.22. ∫A

|X|dP = 0

if and only ifP{A ∩ [X > 0]} = 0.

33

Proof. Observe first that∫A

|X|dP =∫A∩[|X|>0]

|X|dP +∫A∩[|X|=0]

|X|dP

=

∫A∩[|X|>0]

|X|dP + 0.

Thus,∫A|X|dP = 0 if and only if

∫A∩[|X|>0] |X|dP = 0 if and only if P{A ∩

[|X| > 0]} = 0 (since we integrate over those ω ∈ Ω for which |X(ω)| > 0).

Proposition 0.23. Let X ∈ L2. If V (X) = 0, then P [X = E(X)] = 1.

Proof. Let � > 0. By Chebychev’s Inequality,

P [|X − E(X)| ≥ �] ≤ V (X)�2

= 0.

Equivalently,P [|X − E(X)| < �] = 1

for all � > 0. Therefore,P [X = E(X)] = 1.

Problem 2

If X and Y are independent random variables and E(X) exists, then, for allB ∈ B(R), ∫

[Y ∈B]XdP = E(X)P{Y ∈ B}.

Proof. For ease of notation, let A = [Y ∈ B]. Thus,∫[Y ∈B]

XdP =

∫A

XdP

=

∫Ω

X · 1AdP

= E(X · 1A).

Now, write X = X+−X−. Since each of X+ and X− is measurable, we can findsequences of nonneagtive, simple random variables {X+n } and {X−n } with X+n ↑X+ and X−n ↑ X−. By the Monotone Convergence Theorem, E(X+n ) ↑ E(X+)and E(X−n ) ↑ E(X−). Thus, by linearity of expectation, E(X+n − X−n ) →E(X). Moreover, E(X+n · 1A − X−n · 1A) → E(X · 1A). We show next thatE(X+n ·1A−X−n ·1A)→ E(X)P (A) and so conclude that E(X ·1A) = E(X)P (A).

34

For each n, we have

X+n =

n∑i=1

ai1Ai ,

where the ai are constants and the Ai partition Ω. It follows that

X+n · 1A =n∑i=1

ai1Ai1A

=

n∑i=1

ai1Ai∩A,

and so

E(X+n · 1A) =n∑i=1

aiP (Ai ∩A)

=

n∑i=1

aiP (Ai)P (A) (by independence of X and Y )

= P (A)

n∑i=1

aiP (Ai)

= P (A)E(X+n )

→ P (A)E(X+).

Similarly, E(X−n · 1A)→ P (A)E(X−), and so

E(X+n · 1A −X−n · 1A) = E(X+n · 1A)− E(X−n · 1A)→ P (A)E(X+)− P (A)E(X−)= P (A)E(X+ −X−)= P (A)E(X),

as desired.

Problem 3

Proposition 0.24. For all n ≥ 1, let Xn and X be uniformly bounded randomvariables. If

limn→∞

Xn = X,

thenlimn→∞

E|Xn −X| = 0.

35

Proof. The random variable that is identically K belongs to L1, since∫Ω

KdP = K · P (Ω)

= K.

Thus, the identically K random variable is a dominating random variable forthe Xn, and so by the Dominated Convergence Theorem, E|Xn −X| → 0.

Problem 4

On the Lebesgue interval (Ω = [0, 1],B([0, 1]), P = λ) define the random vari-ables

Xn =n

log nI0, 1n .

Proposition 0.25. For Xn defined as above,

limn→∞

Xn = 0

andlimn→∞

E(Xn) = 0,

yet the Xn are unbounded.

Proof. For any x ∈ [0, 1], we can choose N such that 1N < x. Thus, XN (x) = 0,since x /∈ [0, 1N ]. As x was arbitrary, we conclude that Xn → 0.

Next, observe that, for all n,

E(Xn) =n

log nλ

[0,

1

n

]=

n

log n· 1n

=1

log n,

and so E(Xn)→ 0.Finally, nlogn → ∞, and so the Xn are unbounded. Hence, Xn → 0 and

E(Xn)→ 0, yet the condition in the Dominated Convergence Theorem fails.

Problem 5

Proposition 0.26. Let Xn ∈ L1 for all n ≥ 1 satisfying

supnE(Xn)

Proof. Since Xn ↑ X, we see that X+n ↑ X+ and X−n ↓ X−. Since each of X+nand X−n belongs to L1 for all n, we have by the Monotone Convergence Theoremthat E(X+n ) → E(X+) and E(X−n ) → E(X−). By linearity of expectation,this implies that E(Xn) → E(X). Since supnE(Xn) is finite, so is E(X) byuniqueness of limits.

To show that X ∈ L1, it remains to rule out the case that E(X+) =E(X−) =∞ (if only one of them is infinite, then E(X) = ±∞, but supnE(Xn) <∞). Observe, however, X−n ↓ X−. Thus, if E(X−) = ∞, E(X−n ) = ∞ for alln, contradicting the fact that X−n ∈ L1 for all n.

Problem 6

Proposition 0.27. For any positive random variable X,

E(X) =

∫[0,∞)

P (X > t)dt.

Proof. We may view the area of integration as a subset A of the product spaceΩ× [0,∞) where

A = {(ω, t) | X(ω) > t}.

with product measureP ′ = P × µ.

Now, by Fubini’s Theorem,∫Ω×[0,∞)

1AdP′ =

∫Ω

∫[0,∞)

1A(ω, t)dtdP

=

∫Ω

X(ω)dP

= E(X).

On the other hand,∫Ω×[0,∞)

1AdP′ =

∫[0,∞)

∫Ω

1A(ω, t)dPdt

=

∫[0,∞)

P{ω | X(ω) > t}dt

=

∫[0,∞)

P (X > t)dt.

Proposition 0.28. For any positive random variable X and any constant α >0,

E(Xα) = α

∫[0,∞)

tα−1P (X > t)dt.

37

Proof. By direct computation, we have Xα(ω) =∫X(ω)

0αtα−1dt. It follows that

E(Xα) =

∫Ω

X(ω)P (dω)

=

∫Ω

∫ X(ω)0

αtα−1dtP (dω)

=

∫ ∞0

∫{P (X>t)}

αtα−1P (dω)dt (by Fubini’s Theorem)

=

∫ ∞0

αtα−1P (X > t)dt.

Problem 7

Proposition 0.29. Let X be a nonnegative random variable and let δ > 0,0 < β < 1, and C be constants. If

P{X > nδ} ≤ Cβn

for all n ≥ 1, then E(Xα) 0.

Proof. By the previous problem, it is equivalent to show that α∫

[0,∞) tα−1P (X >

t)dt is finite.To begin, pick N such that tα−1P (X > t) is strictly decreasing in t for all

t ≥ N . Such an N exists, as tα−1 is a polynomial in t and P (X > t) decaysexponentially. It follows that

α

∫[N,∞)

tα−1P (X > t)dt ≤ α∞∑n=N

δ · (nδ)α−1P (X > nδ) (since tα−1P (X > t) is strictly decreasing)

≤ αδα∞∑n=N

nα−1Cβn

= Cαδα∞∑n=N

nα−1βn

t)dt is also finite, we conclude that

∫[0,∞] t

α−1P (X >

t)dt is finite.

Problem 2

Let {Xn | n = 1, 2, . . . } be a sequence of random variables with

P{Xn = ±n3} =1

2n2and P{Xn = 0} = 1−

1

n2.

38

Proposition 0.30. For the sequence described above,

P{

limn→∞

Xn = 0}

= 1.

Proof. Let An be the event that Xn is nonzero. Formally,

An = {ω ∈ Ω | Xn(ω) 6= 0}.

We see that

P (An) = 1− P{X = 0}

= 1−(

1− 1n2

)=

1

n2,

and so

∞∑n=1

P (An) =

∞∑n=1

1

n2

(possibly both). Suppose the former is true. It follows that

E(X+n ) ≥ n3P{X+n = n3}

≥ n3 14n2

=n

4,

and so

limn→∞

E(X+n ) ≥ limn→∞

n

4

=∞.

Similarly, if P{X−n = n3} ≥ 14n2 , then limn→∞ E(X−n ) =∞. Therefore,

limn→∞

E(Xn) =

∞ if limn→∞ E(X+n ) =∞ and limn→∞ E(X−n )

(⇐) (Idea) Let a and b be real numbers and take f = 1(0,a] and g = 1(0,b].The support of f(X) is {ω | X(ω) ≤ a} and the support of g(Y ) is {ω | Y (ω)}.Thus, the measures of the supports are P (X ≤ a) and P (Y ≤ b), respectively.Using the fact that

E(f(X)g(Y )) = E(f(X))E(g(Y )),

I would like to derive that

P (X ≤ a, Y ≤ b) = P (X ≤ a) · P (Y ≤ b).

Since a and b were arbitrary, we could conclude that X and Y are independentby the Factorization Criterion (Resnick, 4.2.1). Perhaps this may be accom-plished by looking at the appropriate approximations of f(X) and g(Y ) bysimple functions (where the probabilty of the support becomes more evident inthe computation).

For each n, let Xn and Yn be a pair of independent random variables anddefine

limn→∞

Xn = X and limn→∞

Yn = Y.

Proposition 0.34. The functions X and Y are independent random variables.

Proof. (Idea) We have, for each n and for all continuous, non-negative f and g,

E(f(Xn)g(Yn)) = E(f(Xn))E(g(Yn)).

If we could switch limits with integrals, we would have

limn→∞

E(f(Xn)g(Yn)) = limn→∞

E(f(Xn))E(g(Yn))

E(

limn→∞

f(Xn)g(Yn))

= E(

limn→∞

f(Xn))E(

limn→∞

g(Yn))

E(f(X)g(Y )) = E(f(X))E(g(Y )),

where the last step makes use of the continuity of f and g. Thus, appealingagain to part b, we could conclude that X and Y are independent. I fail to seehow to accomplish the interchange, however, as the Xn need not be monotonenor does there appear to be any bounding function.

Problem 5

Suppose {pk | k ≥ 0} is a probability mass function on (Ω = {0, 1, 2, . . . },P =P(Ω)), where P(·) denotes the power set, so that pk ≥ 0 and

∑k pk = 1. Define

for all A ⊂ Ω,P (A) =

∑k∈A

pk.

41

Proposition 0.35. The function P defined above is a probability measure on(Ω,P).

Proof. Since pk ≥ 0 for all k, P (A) ≥ 0 for all A ⊂ Ω.We have, by definition of the probability mass function,

P (Ω) =∑k∈Ω

pk

= 1.

Let {An} be a countable sequence of disjoint events and let A =⋃{An}. It

follows that

P

( ∞⋃n=1

An

)= P (A)

=∑k∈A

pk

=

∞∑n=1

∑k∈An

pk (since the An are disjoint)

=

∞∑n=1

P (An).

Define the generating function Ψ : ([0, 1],B[0, 1])→ (R,B) via

Ψ(s) =

∞∑k=0

pksk.

Proposition 0.36. The function Ψ defined above satisfies

Ψ′(s) ≡ dds

Ψ(s) =

∞∑k=1

kpksk−1

for 0 ≤ s ≤ 1.Proof. Define the function Xn =

∑nk=1 kpks

k−1. Observe that 0 ≤ Xn ↑ X,and so by the Monotone Convergence Theorem

limn→∞

E(Xn) = E(

limn→∞

Xn

)limn→∞

E

(n∑k=1

kpksk−1

)= E

(limn→∞

n∑k=1

kpksk−1

)

limn→∞

n∑k=0

pksk = E

(limn→∞

n∑k=1

kpksk−1

)Ψ(s) = E (Ψ′(s)) .

42

Proposition 0.37. If X has probability measure P , then E(X) = lims↑1 Ψ′(s).

Proof. We have,

E(X) =∫

Ω

X(ω)dP

=

∞∑k=0

kP (X = k)

=

∞∑k=1

kpk

= lims↑1

Ψ′(s).

Problem 6

Let X1, X2, . . . , Xn ∈ L2(P ) be random variables defined on a probability space(Ω,F, P ). For each i, j ∈ {1, 2, . . . , n}, define the covariances

σij = C(Xi, Xj) = E{[Xi − µi][Xj − µj ]},

whereµi = E(Xi) and σ2i = σii = V(Xi) = E[(Xi − µi)2].

Lemma 0.38. For any random variable X and real numbers a and b,

V(aX + b) = a2V(X).

Proof. It follows from the linearity of expectation that,

V(aX + b) = E[(aX + b− E(aX + b))2]= E[(aX + b− aE(X)− b)2]= E[(a(X − E(X)))2]= a2E[(X − E(X))2]= a2V(X).

Lemma 0.39. For any random variables X and Y ,

V(X + Y ) = V(X) + 2C(X,Y ) + V(Y ).

43

Proof. It follows from the linearity of expectation that,

V(X + Y ) = E[(X + Y − E(X)− E(Y ))2]= E[((X − E(X)) + (Y − E(Y )))2]= E[(X − E(X))2 + 2(X − E(X))(Y − E(Y )) + (Y − E(Y ))2]= E[(X − E(X))2] + E[2(X − E(X))(Y − E(Y ))] + E[(Y − E(Y ))2]= V(X) + 2C(X,Y ) + V(Y ).

Proposition 0.40. For all i and j,

σij ≤ |σij | ≤ σiσj .

Moreover, |σij | = σiσj if and only if, for some α and β, we have P{Xj =α+ βXi} = 1.

Proof. For all real numbers x, we have x ≤ |x|, so certainly σij ≤ |σij |.For the second inequality, let t be a real variable. It follows from the lemmas

that

0 ≤ V[tXi +Xj ]= V(tXi) + 2C(Xi, Xj) + V(Xj)= σ2i t

2 + 2σijt+ σ2j .

Viewing this as a non-negative quadratic in t, we have that

0 ≥ 4σ2ij − 4σ2i σ2j ,

and so|σij | ≤ σiσj .

For the remaining claim, observe that

|σij | = σiσj ⇔ σ2ij = σ2i σ2j⇔ 0 = 4σ2ij − 4σ2i σ2j .

Hence, V[tXi + Xj ] has a unique real root t0. Now, the variance of a randomvariable is equal to 0 if and only if it is constant with probability one. That is,

P{t0Xi +Xj = α} = 1,

or equivalentlyP{Xj = α− t0Xi} = 1.

44

Proposition 0.41. For real constants αi and βi, i = 1, 2, . . . , n,

C

n∑i=1

αiXi,

n∑j=1

βjXj

=n∑i=1

n∑j=1

αiβjσij .

Proof. Applying linearity of expectation, we have

C

n∑i=1

αiXi,

n∑j=1

βjXj

= E(

n∑i=1

αiXi

) n∑j=1

βjXj

− E{

n∑i=1

αiXi

}E

n∑j=1

βjXj

= E

n∑i=1

n∑j=1

αiβjXiXj

− E{

n∑i=1

αiXi

}E

n∑j=1

βjXj

=

n∑i=1

n∑j=1

αiβjE(XiXj)−

{n∑i=1

αiE(Xi)

}n∑j=1

βjE(Xj)

=

n∑i=1

n∑j=1

αiβjE(XiXj)−n∑i=1

n∑j=1

αiβjE(Xi)E(Xj)

=

n∑i=1

n∑j=1

αiβjC(Xi, Xj)

=

n∑i=1

n∑j=1

αiβjσij .

Proposition 0.42. For real constants αi, i = 1, 2, . . . , n,

V

{n∑i=1

αiXi

}=

n∑i=1

α2iσ2i + 2

∑1≤i

the previous proposition,

V

{n∑i=1

αiXi

}= C

{n∑i=1

αiXi,

n∑i=1

αiXi

}

=

n∑i=1

n∑j=1

αiαjC(Xi, Xj)

=

n∑i=1

α2iC(Xi, Xi) + 2∑

1≤i

Thus, we take instead αi = (σis)−1 for all i. Using these values, we have

V

{n∑i=1

αiXi

}=

n∑i=1

(αiσi)2

=

n∑i=1

((σis)−1σi)

2

=

n∑i=1

s−2

= ns−2.

Problem 8

For i = 1, 2, let (Ωi,Bi, Pi) be probability spaces. Define Ω = Ω1 × Ω2 andB = B1 ⊗B2 = σ(RECTS), where

RECTS = {B1 ×B2 | B1 ∈ B1, B2 ∈ B2}.

Let P = P1 × P2 be the product probability measure so that, for B1 × B2 ∈RECTS, we have P (B1 ×B2) = P1(B1)P2(B2). Define the class of subsets

C =

{B ⊂ Ω |

∫Ω

1B(ω1, ω2)dP (ω1, ω2) =

∫Ω1

Y (ω1)dP1(ω1)

},

where Y (ω1) =∫

Ω21B(ω1, ω2)dP2(ω2).

Proposition 0.44. The class RECTS is a subset of the class C.

Proof. Let B = B1 ×B2 belong to RECTS. We have∫Ω

1B(ω1, ω2)dP (ω1, ω2) =

∫B

dP (ω1, ω2)

= P (B).

At the same time, we have∫Ω1

∫Ω2

1B(ω1, ω2)dP2(ω2)dP1(ω1) =

∫Ω1

∫Ω2

1B1(ω1)1B2(ω2)dP2(ω2)dP1(ω1)

=

∫Ω1

∫B2

1B1(ω1)dP2(ω2)dP1(ω1)

=

∫Ω1

1B1(ω1)P2(B2)dP1(ω1)

=

∫B1

P2(B2)dP1(ω1)

= P1(B1)P2(B2)

= P (B).

47

Hence, for all B ∈ RECTS,∫Ω

1B(ω1, ω2)dP (ω1, ω2) =

∫Ω1

Y (ω1)dP1(ω1),

and so RECTS ⊆ C.

Proposition 0.45. The class C is a λ-system.

Proof. We have immediately that Ω = Ω1 × Ω2 ∈ RECTS ⊆ C, so Ω ∈ C.Next, let B ∈ C. We have∫

Ω

1Bc(ω1, ω2)dP (ω1, ω2) =

∫BcdP (ω1, ω2)

= P (Bc)

= 1− P (B)

= 1−∫

Ω1

∫Ω2

1Bω1 (ω2)dP1(ω1) (since B ∈ C)

=

∫Ω1

∫Ω2

1− 1Bω1 (ω2)dP1(ω1)

=

∫Ω1

∫Ω2

1(Bω1 )c(ω2)dP1(ω1)

=

∫Ω1

∫Ω2

1(Bc)ω1 (ω2)dP1(ω1)

=

∫Ω1

∫Ω2

1Bc(ω1, ω2)dP2(ω2)dP1(ω1).

Finally, let {Bn | n = 1, 2, . . . } be a collection of disjoint elements of C. Wehave∫

Ω

1∑∞n=1 An

dP (ω1, ω2) =

∫∑∞

n=1 An

dP (ω1, ω2)

= P

( ∞∑n=1

An

)

=

∞∑n=1

P (An)

=

∞∑n=1

∫Ω1

∫Ω2

1(An)ω1 (ω2)dP1(ω1) (since An ∈ C for all n)

=

∫Ω1

∫Ω2

∞∑n=1

1(An)ω1 (ω2)dP1(ω1) (by MCT)

=

∫Ω1

∫Ω2

1(∑∞

n=1 An)ω1(ω2)dP1(ω1)

=

∫Ω1

∫Ω2

1∑∞n=1 An

(ω1, ω2)dP2(ω2)dP1(ω1).

48

Therefore, C is a λ-system.

Proposition 0.46. For every B ∈ B,∫Ω

1B(ω1, ω2)dP (ω1, ω2) =

∫Ω1

{∫Ω2

1B(ω1, ω2)dP2(ω2)

}dP1(ω1).

Proof. We have shown RECTS ⊆ C and that C is a λ-system. If we canshow also that RECTS is a π-system, then Dynkin’s Theorem gives B =σ(RECTS) ⊂ C, from which the conclusion follows.

To finish the proof, let B1 ×B2 and B′1 ×B′2 belong to RECTS. It followsimmediately that

(B1 ×B2) ∩ (B′1 ×B′2) = (B1 ∩B′1)× (B2 ∩B′2).

Since B1 and B2 are closed under intersections, B1∩B′1 ∈ B1 and B2∩B′2 ∈ B2,and so (B1 ∩B′1)× (B2 ∩B′2) ∈ RECTS.

To establish the more general result where 1B in part c is replaced withany B-measurable positive random variable X, we first establish the result forsimple functions of the form Xn =

∑ni=1 ai1Bi , where Bi ∈ B for all i. The

result for simple functions follows readily from the linearity of the integral. Sinceeach Xn is positive, we can take a sequence Xn ↑ X. By hypothesis,∫

Ω

Xn(ω1, ω2)dP (ω1, ω2) =

∫Ω1

{∫Ω2

Xn(ω1, ω2)dP2(ω2)

}dP1(ω1)

for all n. Applying the Monotone Convergence Theorem, we can conclude that∫Ω

Xn(ω1, ω2)dP (ω1, ω2) ↑∫

Ω

X(ω1, ω2)dP (ω1, ω2)

and∫Ω1

{∫Ω2

Xn(ω1, ω2)dP2(ω2)

}dP1(ω1) ↑

∫Ω1

{∫Ω2

X(ω1, ω2)dP2(ω2)

}dP1(ω1),

from which it follows that∫Ω

X(ω1, ω2)dP (ω1, ω2) =

∫Ω1

{∫Ω2

X(ω1, ω2)dP2(ω2)

}dP1(ω1).

Problem 10

Suppose that X and Y are independent random variables and let h : R2 →[0,∞) be a measurable function such that E{h2(X,Y )}

Proof. Define ĥ(x, ω) = h(x, Y (ω)). Since h and Y are measurable, ĥ is mea-surable, as it is defined by the composition of two measurable functions. Hence,we can take a collection {ĥn} of simple functions with ĥn ↑ ĥ. Define nowgn(x) =

∫Ωĥn(x, ω)dP (ω) for n = 1, 2, . . . . By the Monotone Convergence

Theorem, gn ↑ g. To conclue that g is measurable, it remains to show that eachgn is simple. To that end, observe that

gn(x) =

∫Ω

ĥn(x, ω)dP (ω)

=

∫Ω

k∑j=1

aj1Aj (x, ω)dP (ω) (constants aj and {Aj} a partition of R)

=

k∑j=1

∫Ω

aj1Aj (x, ω)dP (ω) (by MCT)

=

k∑j=1

∫Ω

aj1Aj (x)1Aj (ω)dP (ω)

=

k∑j=1

ajP (Aj)1Aj (x),

and so gn is simple.For k(x), we have

k(x) = V(ĥ)

= E(ĥ2)− E(ĥ)2

= E(ĥ2)− g2.

Now, since ĥ is measureable, ĥ2 is measurable. Following the same argumentas above, we find that E(ĥ2) is measurable, and so k is measurable.

Proposition 0.48. For g and h as defined above,

E{g(X)} = E{h(X,Y )}.

Proof. Suppose X is a random variable on Ω1 with probability measure P1 andY is a random variable on Ω2 with probability measure P2. Finally, let P bethe probability measure on Ω = Ω1 × Ω2 induced by P1 and P2. In order tomake use of Fubini’s Theorem later in the proof, we must establish first thatP = P1 × P2. To that end, observe that for any measurable sets A ⊂ Ω1 and

50

B ⊂ Ω2,

P (A×B) =∫

Ω

1A×BdP

=

∫Ω

1A · 1BdP

=

∫Ω

1AdP ·∫

Ω

1BdP (since X and Y are independent)

=

∫Ω1

1A(ω1)dP1(ω1) ·∫

Ω2

1B(ω2)dP2(ω2)

= P1(A) · P2(B).

Now,

E(g(X)) =∫

Ω1

g(X(ω1))dP1(ω1)

=

∫Ω1

∫Ω2

h(X(ω1), Y (ω2))dP2(ω2)dP1(ω1)

=

∫Ω

h(X(ω1), Y (ω2))dP (ω1, ω2) (by Fubini’s Theorem)

= E(h(X,Y )).

Proposition 0.49. For g, h, and k as defined above,

V{g(X)}+ E{k(X)} = V{h(X,Y )}.

Proof. We have

V(g(X)) + E(k(X))

=

∫Ω1

g(X(ω1))2dP1(ω1)−

[∫Ω1

g(X(ω1))dP1(ω1)

]2+

∫Ω1

k(X(ω1))dP1(ω1)

=

∫Ω1

g(X(ω1))2 + k(X(ω1))dP1(ω1)−

[∫Ω1

g(X(ω1))dP1(ω1)

]2=

∫Ω1

(∫Ω2

h(X(ω1), Y (ω2))dP2(ω2)

)2+

∫Ω2

h(X(ω1), Y (ω2))2dP2(ω2)

−(∫

Ω2

h(X(ω1), Y (ω2))dP2(ω2)

)2dP1(ω1)−

[∫Ω1

∫Ω2

h(X(ω1), Y (ω2))dP2(ω2)dP1(ω1)

]2=

∫Ω1

∫Ω2

h(X(ω1), Y (ω2))2dP2(ω2)dP1(ω1)−

[∫Ω1

∫Ω2

h(X(ω1), Y (ω2))dP2(ω2)dP1(ω1)

]2=E(h(X,Y )2)− E(h(X,Y ))2

=V(h(X,Y )).

51