![Page 1: DSP-CIS Chapter-12: Least Mean Squares (LMS) Algorithm](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56812bd6550346895d903ee3/html5/thumbnails/1.jpg)
DSP-CIS
Chapter-12: Least Mean Squares (LMS) Algorithm
Marc MoonenDept. E.E./ESAT, KU Leuven
www.esat.kuleuven.be/scd/
![Page 2: DSP-CIS Chapter-12: Least Mean Squares (LMS) Algorithm](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56812bd6550346895d903ee3/html5/thumbnails/2.jpg)
DSP-CIS / Chapter-13 : Least Mean Squares (LMS) Algorithm / Version 2012-2013 p. 2
Part-III : Optimal & Adaptive Filters
: Optimal & Adaptive Filters - Intro• General Set-Up
• Applications
• Optimal (Wiener) Filters
– : Least Squares & Recursive Least Squares Estimation• Least Squares Estimation
• Recursive Least Squares (RLS) Estimation
• Square-Root Algorithms
– : Least Means Squares (LMS) Algorithm• LMS/NLMS : Stochastic Gradient Algorithms
• LMS analysis
• LMS Family
– : Fast Recursive Least Squares Algorithms
: Kalman Filtering
Chapter-11
Chapter-12
Chapter-13
Chapter-14
Chapter-15
![Page 3: DSP-CIS Chapter-12: Least Mean Squares (LMS) Algorithm](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56812bd6550346895d903ee3/html5/thumbnails/3.jpg)
DSP-CIS / Chapter-13 : Least Mean Squares (LMS) Algorithm / Version 2012-2013 p. 3
Least Mean Squares (LMS) Algorithm
![Page 4: DSP-CIS Chapter-12: Least Mean Squares (LMS) Algorithm](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56812bd6550346895d903ee3/html5/thumbnails/4.jpg)
DSP-CIS / Chapter-13 : Least Mean Squares (LMS) Algorithm / Version 2012-2013 p. 4
Least Mean Squares (LMS) Algorithm
![Page 5: DSP-CIS Chapter-12: Least Mean Squares (LMS) Algorithm](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56812bd6550346895d903ee3/html5/thumbnails/5.jpg)
DSP-CIS / Chapter-13 : Least Mean Squares (LMS) Algorithm / Version 2012-2013 p. 5
Least Mean Squares (LMS) Algorithm
(Widrow 1965 !!)
![Page 6: DSP-CIS Chapter-12: Least Mean Squares (LMS) Algorithm](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56812bd6550346895d903ee3/html5/thumbnails/6.jpg)
DSP-CIS / Chapter-13 : Least Mean Squares (LMS) Algorithm / Version 2012-2013 p. 6
Least Mean Squares (LMS) Algorithm
Bernard Widrow
![Page 7: DSP-CIS Chapter-12: Least Mean Squares (LMS) Algorithm](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56812bd6550346895d903ee3/html5/thumbnails/7.jpg)
DSP-CIS / Chapter-13 : Least Mean Squares (LMS) Algorithm / Version 2012-2013 p. 7
Least Mean Squares (LMS) Algorithm
![Page 8: DSP-CIS Chapter-12: Least Mean Squares (LMS) Algorithm](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56812bd6550346895d903ee3/html5/thumbnails/8.jpg)
DSP-CIS / Chapter-13 : Least Mean Squares (LMS) Algorithm / Version 2012-2013 p. 8
Least Mean Squares (LMS) Algorithm
![Page 9: DSP-CIS Chapter-12: Least Mean Squares (LMS) Algorithm](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56812bd6550346895d903ee3/html5/thumbnails/9.jpg)
DSP-CIS / Chapter-13 : Least Mean Squares (LMS) Algorithm / Version 2012-2013 p. 9
Least Mean Squares (LMS) Algorithm
large λ_max implies
a small stepsize
![Page 10: DSP-CIS Chapter-12: Least Mean Squares (LMS) Algorithm](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56812bd6550346895d903ee3/html5/thumbnails/10.jpg)
DSP-CIS / Chapter-13 : Least Mean Squares (LMS) Algorithm / Version 2012-2013 p. 10
Least Mean Squares (LMS) Algorithm
![Page 11: DSP-CIS Chapter-12: Least Mean Squares (LMS) Algorithm](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56812bd6550346895d903ee3/html5/thumbnails/11.jpg)
DSP-CIS / Chapter-13 : Least Mean Squares (LMS) Algorithm / Version 2012-2013 p. 11
Least Mean Squares (LMS) Algorithm
error vector projected onto eigenvectors initial error vector projected onto eigenvectors
(=projection on i-th eigenvector)
small λ_i implies slow convergence λ_min <<λ_max (hence small μ) implies *very* slow convergence
![Page 12: DSP-CIS Chapter-12: Least Mean Squares (LMS) Algorithm](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56812bd6550346895d903ee3/html5/thumbnails/12.jpg)
DSP-CIS / Chapter-13 : Least Mean Squares (LMS) Algorithm / Version 2012-2013 p. 12
Least Mean Squares (LMS) Algorithm
![Page 13: DSP-CIS Chapter-12: Least Mean Squares (LMS) Algorithm](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56812bd6550346895d903ee3/html5/thumbnails/13.jpg)
DSP-CIS / Chapter-13 : Least Mean Squares (LMS) Algorithm / Version 2012-2013 p. 13
Least Mean Squares (LMS) Algorithm
![Page 14: DSP-CIS Chapter-12: Least Mean Squares (LMS) Algorithm](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56812bd6550346895d903ee3/html5/thumbnails/14.jpg)
DSP-CIS / Chapter-13 : Least Mean Squares (LMS) Algorithm / Version 2012-2013 p. 14
Least Mean Squares (LMS) Algorithm
![Page 15: DSP-CIS Chapter-12: Least Mean Squares (LMS) Algorithm](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56812bd6550346895d903ee3/html5/thumbnails/15.jpg)
DSP-CIS / Chapter-13 : Least Mean Squares (LMS) Algorithm / Version 2012-2013 p. 15
Least Mean Squares (LMS) Algorithm
![Page 16: DSP-CIS Chapter-12: Least Mean Squares (LMS) Algorithm](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56812bd6550346895d903ee3/html5/thumbnails/16.jpg)
DSP-CIS / Chapter-13 : Least Mean Squares (LMS) Algorithm / Version 2012-2013 p. 16
Least Mean Squares (LMS) Algorithm
![Page 17: DSP-CIS Chapter-12: Least Mean Squares (LMS) Algorithm](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56812bd6550346895d903ee3/html5/thumbnails/17.jpg)
DSP-CIS / Chapter-13 : Least Mean Squares (LMS) Algorithm / Version 2012-2013 p. 17
Least Mean Squares (LMS) Algorithm
![Page 18: DSP-CIS Chapter-12: Least Mean Squares (LMS) Algorithm](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56812bd6550346895d903ee3/html5/thumbnails/18.jpg)
DSP-CIS / Chapter-13 : Least Mean Squares (LMS) Algorithm / Version 2012-2013 p. 18
Least Mean Squares (LMS) Algorithm
![Page 19: DSP-CIS Chapter-12: Least Mean Squares (LMS) Algorithm](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56812bd6550346895d903ee3/html5/thumbnails/19.jpg)
DSP-CIS / Chapter-13 : Least Mean Squares (LMS) Algorithm / Version 2012-2013 p. 19
Least Mean Squares (LMS) Algorithm
![Page 20: DSP-CIS Chapter-12: Least Mean Squares (LMS) Algorithm](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56812bd6550346895d903ee3/html5/thumbnails/20.jpg)
DSP-CIS / Chapter-13 : Least Mean Squares (LMS) Algorithm / Version 2012-2013 p. 20
Least Mean Squares (LMS) Algorithm
![Page 21: DSP-CIS Chapter-12: Least Mean Squares (LMS) Algorithm](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56812bd6550346895d903ee3/html5/thumbnails/21.jpg)
DSP-CIS / Chapter-13 : Least Mean Squares (LMS) Algorithm / Version 2012-2013 p. 21
Least Mean Squares (LMS) Algorithm
![Page 22: DSP-CIS Chapter-12: Least Mean Squares (LMS) Algorithm](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56812bd6550346895d903ee3/html5/thumbnails/22.jpg)
DSP-CIS / Chapter-13 : Least Mean Squares (LMS) Algorithm / Version 2012-2013 p. 22
Least Mean Squares (LMS) Algorithm
![Page 23: DSP-CIS Chapter-12: Least Mean Squares (LMS) Algorithm](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56812bd6550346895d903ee3/html5/thumbnails/23.jpg)
DSP-CIS / Chapter-13 : Least Mean Squares (LMS) Algorithm / Version 2012-2013 p. 23
Least Mean Squares (LMS) Algorithm
![Page 24: DSP-CIS Chapter-12: Least Mean Squares (LMS) Algorithm](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56812bd6550346895d903ee3/html5/thumbnails/24.jpg)
DSP-CIS / Chapter-13 : Least Mean Squares (LMS) Algorithm / Version 2012-2013 p. 24
Least Mean Squares (LMS) Algorithm
![Page 25: DSP-CIS Chapter-12: Least Mean Squares (LMS) Algorithm](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56812bd6550346895d903ee3/html5/thumbnails/25.jpg)
DSP-CIS / Chapter-13 : Least Mean Squares (LMS) Algorithm / Version 2012-2013 p. 25
Least Mean Squares (LMS) Algorithm
![Page 26: DSP-CIS Chapter-12: Least Mean Squares (LMS) Algorithm](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56812bd6550346895d903ee3/html5/thumbnails/26.jpg)
DSP-CIS / Chapter-13 : Least Mean Squares (LMS) Algorithm / Version 2012-2013 p. 26
Least Mean Squares (LMS) Algorithm
![Page 27: DSP-CIS Chapter-12: Least Mean Squares (LMS) Algorithm](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56812bd6550346895d903ee3/html5/thumbnails/27.jpg)
DSP-CIS / Chapter-13 : Least Mean Squares (LMS) Algorithm / Version 2012-2013 p. 27
Least Mean Squares (LMS) Algorithm
![Page 28: DSP-CIS Chapter-12: Least Mean Squares (LMS) Algorithm](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56812bd6550346895d903ee3/html5/thumbnails/28.jpg)
DSP-CIS / Chapter-13 : Least Mean Squares (LMS) Algorithm / Version 2012-2013 p. 28
Least Mean Squares (LMS) Algorithm
![Page 29: DSP-CIS Chapter-12: Least Mean Squares (LMS) Algorithm](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56812bd6550346895d903ee3/html5/thumbnails/29.jpg)
DSP-CIS / Chapter-13 : Least Mean Squares (LMS) Algorithm / Version 2012-2013 p. 29
Least Mean Squares (LMS) Algorithm
![Page 30: DSP-CIS Chapter-12: Least Mean Squares (LMS) Algorithm](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56812bd6550346895d903ee3/html5/thumbnails/30.jpg)
DSP-CIS / Chapter-13 : Least Mean Squares (LMS) Algorithm / Version 2012-2013 p. 30
Least Mean Squares (LMS) Algorithm
![Page 31: DSP-CIS Chapter-12: Least Mean Squares (LMS) Algorithm](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56812bd6550346895d903ee3/html5/thumbnails/31.jpg)
DSP-CIS / Chapter-13 : Least Mean Squares (LMS) Algorithm / Version 2012-2013 p. 31
Least Mean Squares (LMS) Algorithm
![Page 32: DSP-CIS Chapter-12: Least Mean Squares (LMS) Algorithm](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56812bd6550346895d903ee3/html5/thumbnails/32.jpg)
DSP-CIS / Chapter-13 : Least Mean Squares (LMS) Algorithm / Version 2012-2013 p. 32
Least Mean Squares (LMS) Algorithm
![Page 33: DSP-CIS Chapter-12: Least Mean Squares (LMS) Algorithm](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56812bd6550346895d903ee3/html5/thumbnails/33.jpg)
DSP-CIS / Chapter-13 : Least Mean Squares (LMS) Algorithm / Version 2012-2013 p. 33
Least Mean Squares (LMS) Algorithm
![Page 34: DSP-CIS Chapter-12: Least Mean Squares (LMS) Algorithm](https://reader035.vdocuments.mx/reader035/viewer/2022062314/56812bd6550346895d903ee3/html5/thumbnails/34.jpg)
DSP-CIS / Chapter-13 : Least Mean Squares (LMS) Algorithm / Version 2012-2013 p. 34
Least Mean Squares (LMS) Algorithm