ece 302: chapter 01 background - purdue university · c stanley chan 2019. all rights reserved....

c©Stanley Chan 2019. All Rights Reserved.

ECE 302: Chapter 01 Background

Fall 2019

Prof Stanley Chan

School of Electrical and Computer EngineeringPurdue University

1 / 26


Outline

1. Infinite Series

2. Integration

3. Linear Algebra

2 / 26


Sum of Finite Geometric Series

Theorem

The sum of a finite geometric series of power n is

n∑k=0

rk = 1 + r + r2 + . . .+ rn =1− rn+1

1− r. (1)

3 / 26


Sum of Infinite Geometric Series 1

Corollary

Let 0 < r < 1. The sum of an infinite geometric series is

∞∑k=0

rk = 1 + r + r2 + . . . =1

1− r. (2)

4 / 26


Sum of Infinite Geometric Series 2

Corollary

Let 0 < r < 1. It holds that

∞∑k=1

krk−1 = 1 + 2r + 3r2 + . . . =1

(1− r)2. (3)

5 / 26


Combination: n choose k

Definition

The symbol(nk

)denotes n choose k , and is defined as(

n

k

)def=

n!

k!(n − k)!. (4)

6 / 26


Binomial Series

Theorem

For any real numbers a and b, the binomial series of power n is

(a + b)n =n∑

k=0

(n

k

)an−kbk , (5)

where(nk

)= n!

k!(n−k)! .

Example. (1 + x)3 =

7 / 26


Taylor Approximation

Definition

Let f : R→ R be a continuous function with infinite derivatives. Leta ∈ R be a fixed constant. The Taylor approximation of f at x = a is

f (x) = f (a) + f ′(a)(x − a) +f ′′(a)

2!(x − a)2 + . . .

=∞∑n=0

f (n)(a)

n!(x − a)n,

where f (n) denotes the n-th order derivative of f .

8 / 26


Exponential Series

Theorem

Let x be any real number. Then,

ex = 1 + x +x2

2+

x3

3!+ . . . =

∞∑k=0

xk

k!. (6)

Example.∞∑k=0

λke−λ

k! =

9 / 26


Sine and Cosine

Theorem

The Taylor approximations of sine and cosines are:

sin(x) = x − x3

3!+

x5

5!− x7

7!+ . . .

cos(x) = 1− x2

2!− x4

4!+

x6

6!+ . . . .

Proof. Euler Formula:e jx = cos x + j sin x .

10 / 26


Interpreting the Results

https://betterexplained.com/articles/taylor-series/

11 / 26


Logarithmic Approximation

Theorem

Let 0 < x < 1 be a constant. Then,

log(1 + x) = x − x2 +O(x3). (7)

Proof. Let f (x) = log(1 + x). Then, the derivatives of f are

f ′(x) =1

(1 + x), and f ′′(x) = − 1

(1 + x)2.

Taylor approximation at x = 0 gives

f (x) = f (0) + f ′(0)(x − 0) +f ′′(0)

2(x − 0)2 +O(x3)

= x − x2 +O(x3).

12 / 26


2. Integration

13 / 26


Even and Odd Functions

Definition

A function f : R→ R is even if for any x ∈ R,

f (x) = f (−x), (8)

and f is odd iff (x) = −f (−x), (9)

14 / 26


Integration of Odd and Even Function

Example. Evaluate the integral∫ ∞−∞

x1√2π

e−x2

2 dx


1

2e−|x |dx

15 / 26


Three Ways of Doing Integration

There are literally only two three ways of doing integration:

Substitution (freshman calculus)By-part (freshman calculus)Unit probability (new!)


x1√2π

e−(x−µ)2

2 dx

16 / 26


Fundamental Theorem of Calculus

Theorem (Fundamental Theorem of Calculus)

Let f : [a, b]→ R be a continuous function defined on a closed interval[a, b]. Then,

f (x) =d

dx

∫ x

af (t)dt, (10)

for any x ∈ (a, b).

17 / 26


Example. Evaluate the integral

d

dx

∫ x−µ

0

1√2πσ2

exp

− t2

2σ2

dt.

Solution. Let y = x − µ. Then by using the fundamental theorem ofcalculus, we can show that

d

dx

∫ x−µ

0

1√2πσ2

exp

− t2

2σ2

dt

=dy

dx· ddy

∫ y

0

1√2πσ2

exp

− t2

2σ2

dt

=d(x − µ)

dx· 1√

2πσ2exp

− y2

2σ2

=

1√2πσ2

exp

−(x − µ)2

2σ2

.

18 / 26


3. Linear Algebra

19 / 26


Basic Notation

Vector: x ∈ Rn

Matrix: A ∈ Rm×n; Entries are aij or [A]ij .

Transpose:

A =

| | |a1 a2 . . . an| | |

, and AT =

— aT1 —— aT2 —

...— aTn —

.Column: ai is the i-th column of A

Identity matrix I

All-one vector 1 and all-zero vector 0

Standard basis e i .

20 / 26


Inner Product

Definition

Let x = [x1, x2, . . . , xN ]T and y = [y1, y2, . . . , yN ]T be two vectors. Theinner product xTy is

Example. Let x = [x1, x2]T . The inner productxTx =

21 / 26


Weighted Inner Product

Example.

Let x = [x1, x2]T , µ = [µ1, µ2] and C =

[a 00 b

]. The product

(x − µ)TC (x − µ) is

22 / 26


The `2-norm

Also called the Euclidean norm:

Definition

‖x‖2 =

√√√√ n∑i=1

x2i . (11)

The set Ω = x | ‖x‖2 ≤ r defines a circle:

Ω = x | ‖x‖2 ≤ r = (x1, x2) | x21 + x22 ≤ r2.

f (x) = ‖x‖2 is not the same as f (x) = ‖x‖22.

Triangle inequality holds:

‖x + y‖2 ≤ ‖x‖2 + ‖y‖2.

23 / 26


Cauchy-Schwarz Inequality

Theorem

Let x ∈ Rn and y ∈ Rn. Then,

|xTy | ≤ ‖x‖2‖y‖2, (12)

where the equality holds if and only if x = αy for some scalar α.

xTy/(‖x‖2‖y‖2) defines the cosine angle between the two vectors xand y .

Cosine is always less than 1. So is xTy/(‖x‖2‖y‖2).

The equality holds if and only if the two vectors are parallel.

24 / 26


Determinant and Inverse

Definition

Determinant Let Σ =

[a bc d

], the determinant of Σ is

Definition (Inverse)

Let Σ =

[a bc d

], the inverse of Σ is

25 / 26


Visualizing a 2D Gaussian

Definition

A d-dimensional Gaussian has a distribution

fX (x) =1√

(2π)d |Σ|exp

−1

2(x − µ)TΣ−1(x − µ)

.

26 / 26

ece 302: chapter 01 background - purdue university · c stanley chan 2019. all rights reserved....

Documents