ece 302: chapter 01 background - purdue university · c stanley chan 2019. all rights reserved....
Post on 29-Jun-2020
2 Views
Preview:
TRANSCRIPT
c©Stanley Chan 2019. All Rights Reserved.
ECE 302: Chapter 01 Background
Fall 2019
Prof Stanley Chan
School of Electrical and Computer EngineeringPurdue University
1 / 26
c©Stanley Chan 2019. All Rights Reserved.
Outline
1. Infinite Series
2. Integration
3. Linear Algebra
2 / 26
c©Stanley Chan 2019. All Rights Reserved.
Sum of Finite Geometric Series
Theorem
The sum of a finite geometric series of power n is
n∑k=0
rk = 1 + r + r2 + . . .+ rn =1− rn+1
1− r. (1)
3 / 26
c©Stanley Chan 2019. All Rights Reserved.
Sum of Infinite Geometric Series 1
Corollary
Let 0 < r < 1. The sum of an infinite geometric series is
∞∑k=0
rk = 1 + r + r2 + . . . =1
1− r. (2)
4 / 26
c©Stanley Chan 2019. All Rights Reserved.
Sum of Infinite Geometric Series 2
Corollary
Let 0 < r < 1. It holds that
∞∑k=1
krk−1 = 1 + 2r + 3r2 + . . . =1
(1− r)2. (3)
5 / 26
c©Stanley Chan 2019. All Rights Reserved.
Combination: n choose k
Definition
The symbol(nk
)denotes n choose k , and is defined as(
n
k
)def=
n!
k!(n − k)!. (4)
6 / 26
c©Stanley Chan 2019. All Rights Reserved.
Binomial Series
Theorem
For any real numbers a and b, the binomial series of power n is
(a + b)n =n∑
k=0
(n
k
)an−kbk , (5)
where(nk
)= n!
k!(n−k)! .
Example. (1 + x)3 =
7 / 26
c©Stanley Chan 2019. All Rights Reserved.
Taylor Approximation
Definition
Let f : R→ R be a continuous function with infinite derivatives. Leta ∈ R be a fixed constant. The Taylor approximation of f at x = a is
f (x) = f (a) + f ′(a)(x − a) +f ′′(a)
2!(x − a)2 + . . .
=∞∑n=0
f (n)(a)
n!(x − a)n,
where f (n) denotes the n-th order derivative of f .
8 / 26
c©Stanley Chan 2019. All Rights Reserved.
Exponential Series
Theorem
Let x be any real number. Then,
ex = 1 + x +x2
2+
x3
3!+ . . . =
∞∑k=0
xk
k!. (6)
Example.∞∑k=0
λke−λ
k! =
9 / 26
c©Stanley Chan 2019. All Rights Reserved.
Sine and Cosine
Theorem
The Taylor approximations of sine and cosines are:
sin(x) = x − x3
3!+
x5
5!− x7
7!+ . . .
cos(x) = 1− x2
2!− x4
4!+
x6
6!+ . . . .
Proof. Euler Formula:e jx = cos x + j sin x .
10 / 26
c©Stanley Chan 2019. All Rights Reserved.
Interpreting the Results
https://betterexplained.com/articles/taylor-series/
11 / 26
c©Stanley Chan 2019. All Rights Reserved.
Logarithmic Approximation
Theorem
Let 0 < x < 1 be a constant. Then,
log(1 + x) = x − x2 +O(x3). (7)
Proof. Let f (x) = log(1 + x). Then, the derivatives of f are
f ′(x) =1
(1 + x), and f ′′(x) = − 1
(1 + x)2.
Taylor approximation at x = 0 gives
f (x) = f (0) + f ′(0)(x − 0) +f ′′(0)
2(x − 0)2 +O(x3)
= x − x2 +O(x3).
12 / 26
c©Stanley Chan 2019. All Rights Reserved.
2. Integration
13 / 26
c©Stanley Chan 2019. All Rights Reserved.
Even and Odd Functions
Definition
A function f : R→ R is even if for any x ∈ R,
f (x) = f (−x), (8)
and f is odd iff (x) = −f (−x), (9)
14 / 26
c©Stanley Chan 2019. All Rights Reserved.
Integration of Odd and Even Function
Example. Evaluate the integral∫ ∞−∞
x1√2π
e−x2
2 dx
Example. Evaluate the integral∫ ∞−∞
1
2e−|x |dx
15 / 26
c©Stanley Chan 2019. All Rights Reserved.
Three Ways of Doing Integration
There are literally only two three ways of doing integration:
Substitution (freshman calculus)By-part (freshman calculus)Unit probability (new!)
Example. Evaluate the integral∫ ∞−∞
x1√2π
e−(x−µ)2
2 dx
16 / 26
c©Stanley Chan 2019. All Rights Reserved.
Fundamental Theorem of Calculus
Theorem (Fundamental Theorem of Calculus)
Let f : [a, b]→ R be a continuous function defined on a closed interval[a, b]. Then,
f (x) =d
dx
∫ x
af (t)dt, (10)
for any x ∈ (a, b).
17 / 26
c©Stanley Chan 2019. All Rights Reserved.
Example. Evaluate the integral
d
dx
∫ x−µ
0
1√2πσ2
exp
− t2
2σ2
dt.
Solution. Let y = x − µ. Then by using the fundamental theorem ofcalculus, we can show that
d
dx
∫ x−µ
0
1√2πσ2
exp
− t2
2σ2
dt
=dy
dx· ddy
∫ y
0
1√2πσ2
exp
− t2
2σ2
dt
=d(x − µ)
dx· 1√
2πσ2exp
− y2
2σ2
=
1√2πσ2
exp
−(x − µ)2
2σ2
.
18 / 26
c©Stanley Chan 2019. All Rights Reserved.
3. Linear Algebra
19 / 26
c©Stanley Chan 2019. All Rights Reserved.
Basic Notation
Vector: x ∈ Rn
Matrix: A ∈ Rm×n; Entries are aij or [A]ij .
Transpose:
A =
| | |a1 a2 . . . an| | |
, and AT =
— aT1 —— aT2 —
...— aTn —
.Column: ai is the i-th column of A
Identity matrix I
All-one vector 1 and all-zero vector 0
Standard basis e i .
20 / 26
c©Stanley Chan 2019. All Rights Reserved.
Inner Product
Definition
Let x = [x1, x2, . . . , xN ]T and y = [y1, y2, . . . , yN ]T be two vectors. Theinner product xTy is
Example. Let x = [x1, x2]T . The inner productxTx =
21 / 26
c©Stanley Chan 2019. All Rights Reserved.
Weighted Inner Product
Example.
Let x = [x1, x2]T , µ = [µ1, µ2] and C =
[a 00 b
]. The product
(x − µ)TC (x − µ) is
22 / 26
c©Stanley Chan 2019. All Rights Reserved.
The `2-norm
Also called the Euclidean norm:
Definition
‖x‖2 =
√√√√ n∑i=1
x2i . (11)
The set Ω = x | ‖x‖2 ≤ r defines a circle:
Ω = x | ‖x‖2 ≤ r = (x1, x2) | x21 + x22 ≤ r2.
f (x) = ‖x‖2 is not the same as f (x) = ‖x‖22.
Triangle inequality holds:
‖x + y‖2 ≤ ‖x‖2 + ‖y‖2.
23 / 26
c©Stanley Chan 2019. All Rights Reserved.
Cauchy-Schwarz Inequality
Theorem
Let x ∈ Rn and y ∈ Rn. Then,
|xTy | ≤ ‖x‖2‖y‖2, (12)
where the equality holds if and only if x = αy for some scalar α.
xTy/(‖x‖2‖y‖2) defines the cosine angle between the two vectors xand y .
Cosine is always less than 1. So is xTy/(‖x‖2‖y‖2).
The equality holds if and only if the two vectors are parallel.
24 / 26
c©Stanley Chan 2019. All Rights Reserved.
Determinant and Inverse
Definition
Determinant Let Σ =
[a bc d
], the determinant of Σ is
Definition (Inverse)
Let Σ =
[a bc d
], the inverse of Σ is
25 / 26
c©Stanley Chan 2019. All Rights Reserved.
Visualizing a 2D Gaussian
Definition
A d-dimensional Gaussian has a distribution
fX (x) =1√
(2π)d |Σ|exp
−1
2(x − µ)TΣ−1(x − µ)
.
26 / 26
top related