l14: orthogonal matching pursuit & lasso and compressed...
Post on 30-Aug-2020
11 Views
Preview:
TRANSCRIPT
L14: Orthogonal Matching Pursuit & Lasso
and Compressed Sensing
Je↵ M. Phillips
February 26, 2020
Midterm• No computers ,
calculators, planes
• I ' ' cheat she - t" ( both sides )
• Understand core definitions3 Qeesttxon C sub parts)
• Clustering-
Assessment - bas .ee
- HACa
'
k - grams• Similarityo Distance a Streaming
.Mitha shrug
NG
n Jace card
Ridge & Lasso Regression-
laps IcyX EIR
'd
get" X - ft ;D
God : RR : go=arsmi "
II. ( yi- 24 ;D )2tsll*R
a end " '
if rest?
Lasso ids = argentinian II. Cgi - Gord 't shake
-
I = Inseminated:( gi - 2x ;D } sit .
Hallie
#kill !
* = LIFE , II. Csi - LAW set . Hallett
equivalent tf chooses s → If
so Go =L : I RR )Let t= 1108115
or 29=27 C Lasso )
It = I n II Csi - Hird set .
lkh÷÷÷i÷÷÷÷:* .
Lasso Illustration
Find ↵⇤= argmin↵2Rd kX↵ � yk2 + sk↵k1
k↵k2 = t
k↵k1 = t
t↵�
t with ridge regression
↵� with OLS
↵�t with lasso
Hall lie
←
:
Matching Pursuit (MP)
Find ↵⇤= argmin↵2Rd kX↵ � yk2 + sk↵k1
Forward Subset Selection:
Matching Pursuit
Set r = y ; ↵ = 0.for i = 1 to t doSet Xj = argmaxXj02X |hr ,Xj 0i|.Set ↵j = argmin↵ kr � Xj↵k + s|↵|.Set r = r � Xj↵j .
Return ↵.
%. .
es
°
Orthogonal Matching Pursuit (OMP)
Find ↵⇤= argmin↵2Rd kX↵ � yk2 + sk↵k22
Forward Subset Selection:
Orthogonal Matching Pursuit
Set r = y ; ↵ = 0.for i = 1 to t doSet Xj = argmaxXj02X |hr ,Xj 0i|.Set ↵ = argmin↵ kr � [X1;X2; . . . ;Xj ]↵k + sk↵k22.Set r = r � Xj↵j . (Update using other ↵j 0 for j 0 < j)
Return ↵.
Ridge
full !! efficient'
a
Lasso Illustration
Find ↵⇤= argmin↵2Rd kX↵ � yk2 + sk↵k1
k↵k2 = t
k↵k1 = t
t↵�
t with ridge regression
↵� with OLS
↵�t with lasso
( east Angle Regression
µMP
i nst
¥D\÷*a. . . .¥i.on ,
Sparse Sensing
ST = [0 1 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 1 1 0 1 0 0 0 1 0 0 1 0 0]
xTi = [-1 0 1 0 1 1 -1 1 0 -1 0 0 1 -1 -1 1 0 1 0 1 -1 -1 -1 0 1 0 0 -1 0 1 0 0]
yi = hS , xi i= 0+0+0+0+1+0+0+0+0+0+0+0+0+0+0+1+0+0+0+1-1+0-1+0+0+0+0+0+0+1+0+0
= 2
what if d > n
but k ? n
keenLos I ( unknown )
ksnpasse C
-44279galbiddies ,
ord
"" " " "
I:*.
.
. " "
Matching Pursuit (OMP)
Matching Pursuit
Set r = y .for i = 1 to t doSet Xj = argmaxXj02X |hr ,Xj 0i|.Set �j = argmin� kr � Xj�k.Set r = r � Xj�j .
Return S where sj = �j (or 0).
OMP Example
signal: S = [0, 0, 1, 0, 0, 1, 0, 0, 1, 0]
measurement: X =
2
666664
0 1 1 �1 �1 0 �1 0 �1 0�1 �1 0 1 �1 0 0 �1 0 11 �1 1 �1 0 �1 1 1 0 01 0 �1 0 0 1 �1 �1 1 1�1 0 0 0 1 0 1 0 1 �10 0 �1 �1 �1 0 �1 1 �1 0
3
777775
observation: y = XST = [0, 0, 0, 1, 1,�2]T000
⑧
r,
= £ I,
o,
0, 0,0 ,
- I ]
top related