dsp for matlab™ and labview™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · spectral...

127
DSP for MATLAB and LabVIEW Volume IV: LMS Adaptive Filtering

Upload: others

Post on 05-Jul-2020

13 views

Category:

Documents


1 download

TRANSCRIPT

Page 1: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

DSP forMATLAB™ and LabVIEW™Volume IV: LMS Adaptive Filtering

Page 2: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is
Page 3: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

SYNTHESIS LECTURESON SIGNAL

PROCESSINGEditorJosé Moura, Carnegie Mellon University

DSP for MATLAB™ and LabVIEW™ Volume IV: LMS Adaptive FilteringForester W. Isen2009

DSP for MATLAB™ and LabVIEW™

Volume III: Digital Filter DesignForester W. Isen2008

DSP for MATLAB™ and LabVIEW™

Volume II: Discrete Frequency TransformsForester W. Isen2008

DSP for MATLAB™ and LabVIEW™

Volume I: Fundamentals of Discrete Signal ProcessingForester W. Isen2008

The Theory of Linear PredictionP. P. Vaidyanathan2007

Nonlinear Source SeparationLuis B. Almeida2006

Spectral Analysis of SignalsYanwei Wang, Jian Li, and Petre Stoica2006

Page 4: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

Copyright © 2009 by Morgan & Claypool

All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted inany form or by any means—electronic, mechanical, photocopy, recording, or any other except for brief quotations inprinted reviews, without the prior permission of the publisher.

DSP for MATLAB™ and LabVIEW™ Volume IV: LMS Adaptive Filtering

Forester W. Isen

www.morganclaypool.com

ISBN: 9781598298994 paperbackISBN: 9781598299007 ebook

DOI 10.2200/S00164ED1V01Y200811SPR007

A Publication in the Morgan & Claypool Publishers seriesSYNTHESIS LECTURES ON SIGNAL PROCESSING

Lecture #7Series Editor: José Moura, Carnegie Mellon University

Series ISSNSynthesis Lectures on Signal ProcessingPrint 1932-1236 Electronic 1932-1694

Page 5: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

DSP forMATLAB™ and LabVIEW™

Volume IV: LMS Adaptive Filtering

Forester W. Isen

SYNTHESIS LECTURES ON SIGNAL PROCESSING #7

CM& cLaypoolMorgan publishers&

Page 6: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

ABSTRACT

This book is Volume IV of the series DSP for MATLAB™ and LabVIEW™. Volume IV is anintroductory treatment of LMS Adaptive Filtering and applications, and covers cost functions, per-formance surfaces, coefficient perturbation to estimate the gradient, the LMS algorithm, responseof the LMS algorithm to narrow-band signals, and various topologies such as ANC (Active NoiseCancelling) or system modeling, Noise Cancellation, Interference Cancellation, Echo Cancella-tion (with single- and dual-H topologies), and Inverse Filtering/Deconvolution. The entire seriesconsists of four volumes that collectively cover basic digital signal processing in a practical and ac-cessible manner, but which nonetheless include all essential foundation mathematics. As the seriestitle implies, the scripts (of which there are more than 200) described in the text and supplied incode form (available via the internet at http://www.morganclaypool.com/page/isen) will runon both MATLAB and LabVIEW. The text for all volumes contains many examples, and manyuseful computational scripts, augmented by demonstration scripts and LabVIEW Virtual Instru-ments (VIs) that can be run to illustrate various signal processing concepts graphically on the user’scomputer screen. Volume I consists of four chapters that collectively set forth a brief overview ofthe field of digital signal processing, useful signals and concepts (including convolution, recursion,difference equations, LTI systems, etc), conversion from the continuous to discrete domain andback (i.e., analog-to-digital and digital-to-analog conversion), aliasing, the Nyquist rate, normal-ized frequency, sample rate conversion, and Mu-law compression, and signal processing principlesincluding correlation, the correlation sequence, the Real DFT, correlation by convolution, matchedfiltering, simple FIR filters, and simple IIR filters. Chapter 4 of Volume I, in particular, provides anintuitive or “first principle” understanding of how digital filtering and frequency transforms work.Volume II provides detailed coverage of discrete frequency transforms, including a brief overview ofcommon frequency transforms, both discrete and continuous, followed by detailed treatments of theDiscrete Time Fourier Transform (DTFT), the z-Transform (including definition and properties,the inverse z-transform, frequency response via z-transform, and alternate filter realization topolo-gies including Direct Form, Direct Form Transposed, Cascade Form, Parallel Form, and LatticeForm), and the Discrete FourierTransform (DFT) (including Discrete Fourier Series, the DFT-IDFT pair, DFT of common signals, bin width, sampling duration, and sample rate, the FFT, theGoertzel Algorithm, Linear, Periodic, and Circular convolution, DFT Leakage, and computationof the Inverse DFT). Volume III covers digital filter design, including the specific topics of FIRdesign via windowed-ideal-lowpass filter, FIR highpass, bandpass, and bandstop filter design fromwindowed-ideal lowpass filters, FIR design using the transition-band-optimized Frequency Sam-pling technique (implemented by Inverse-DFT or Cosine/Sine Summation Formulas), design ofequiripple FIRs of all standard types including Hilbert Transformers and Differentiators via the Re-mez Exchange Algorithm, design of Butterworth, Chebyshev (Types I and II), and Elliptic analogprototype lowpass filters, conversion of analog lowpass prototype filters to highpass, bandpass, andbandstop filters, and conversion of analog filters to digital filters using the Impulse Invariance and

Page 7: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

vii

Bilinear Transform techniques. Certain filter topologies specific to FIRs are also discussed, as aretwo simple FIR types, the Comb and Moving Average filters.

KEYWORDSHigher-Level Terms:

LMS Adaptive Filter, Least Mean Square, Active Noise Cancellation (ANC),Deconvolution, Equalization, Inverse Filtering, Interference Cancellation, EchoCancellation, Dereverberation, Adaptive Line Enhancer (ALE)

Lower-Level Terms:

Single-H, Dual-H, Gradient, Cost Function, Performance Surface, Coefficient Per-turbation

Page 8: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is
Page 9: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

This volume is dedicated to

The following humane organizations, and all who work for them andsupport them:

Rikki’s Refuge (www.rikkisrefuge.org), Orange, VirginiaThe Humane Society of Fairfax County (www.hsfc.org), Fairfax, Virginia

The Washington Animal Rescue League (WARL) (www.warl.org),Washington, DC

Angels for Animals (www.angelsforanimals.org), Youngstown, Ohioand all similar organizations and persons

and to the feline companions who so carefully watched over the progress of thethe entire series during its evolution over the past eight years:

Richard (1985-2001)Blackwythe (1989-2001)

Tiger (1995? - )Mystique (2000 - )Scampy (2002 - )Nudgy (2003 - )Percy (2004 - )

Page 10: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is
Page 11: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

xi

ContentsSYNTHESIS LECTURES ON SIGNAL PROCESSING . . . . . . . . . . . . . . . . . . . . . . . . . iii

Contents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi

Preface to Volume IV . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xv

0.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .xv

0.2 The Four Volumes of the Series . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xv

0.3 Origin and Evolution of the Series . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xvi

1 Introduction To LMS Adaptive Filtering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1

1.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

1.1.1 In Previous Volumes 1

1.1.2 In this Volume 1

1.1.3 In this Chapter 1

1.2 SOFTWARE FOR USE WITH THIS BOOK . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

1.3 Cost Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

1.4 Performance Surface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

1.5 Coefficient Perturbation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

1.6 Method of Steepest Descent . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

1.7 Two Variable Performance Surface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

1.8 An Improved Gradient Search Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

1.9 LMS Used in an FIR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

1.9.1 Typical Arrangements 20

1.9.2 Derivation 22

1.9.3 Limitation on Mu 24

1.9.4 NLMS Algorithm 25

1.10 Contrast-True MSE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

Page 12: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

xii CONTENTS

1.11 LMS Adaptive FIR Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

1.12 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

1.13 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

2 Applied Adaptive Filtering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

2.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

2.2 Active Noise Cancellation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50

2.3 System Modeling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .51

2.4 Echo Cancellation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54

2.4.1 Single-H 57

2.4.2 Dual-H 58

2.4.3 Sparse Computation 60

2.5 Periodic Component Elimination or Enhancement . . . . . . . . . . . . . . . . . . . . . . . . . . . 61

2.6 Interference Cancellation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .65

2.7 Equalization/Deconvolution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69

2.8 Deconvolution Of a Reverberative Signal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74

2.8.1 Simulation 74

2.8.2 Spectral Effect of Reverberation 76

2.8.3 Estimating Delay 78

2.8.4 Estimating DecayRate 79

2.8.5 Deconvolution 80

2.9 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81

2.10 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82

A Software for Use with this Book . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .99

A.1 File Types and Naming Conventions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99

A.2 Downloading the Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100

A.3 Using the Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100

A.4 Single-line function calls . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100

A.5 Multi-line m-code examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101

A.6 How to Successfully Copy-and-Paste M-code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .102

Page 13: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

CONTENTS xiii

A.7 Learning To Use M-Code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .103

A.8 What You Need with MATLAB and LabVIEW . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103

B Vector/Matrix Operations in M-Code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105

B.1 Row and Column Vectors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105

B.2 Vector Products . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105

B.2.1 Inner Product 105

B.2.2 Outer Product 106

B.2.3 Product of Corresponding Values 106

B.3 Matrix Multiplied by a Vector or Matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106

B.4 Matrix Inverse and Pseudo-Inverse . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107

Biography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109

Page 14: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is
Page 15: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

Preface to Volume IV

0.1 INTRODUCTION

The present volume is Volume IV of the series DSP for MATLAB™and LabVIEW ™. The entireseries consists of four volumes which collectively form a work of twelve chapters that cover basicdigital signal processing in a practical and accessible manner, but which nonetheless include essentialfoundation mathematics. The text is well-illustrated with examples involving practical computationusing m-code or MathScript (as m-code is usually referred to in LabVIEW-based literature), andLabVIEW VIs. There is also an ample supply of exercises, which consist of a mixture of paper-and-pencil exercises for simple computations, and script-writing projects having various levels ofdifficulty, from simple, requiring perhaps ten minutes to accomplish, to challenging, requiring severalhours to accomplish. As the series title implies, the scripts given in the text and supplied in code form(available via the internet at http://www.morganclaypool.com/page/isen) are suitable for usewith both MATLAB (a product of The Mathworks, Inc.), and LabVIEW (a product of NationalInstruments, Inc.). Appendix A in each volume of the series describes the naming convention for thesoftware written for the book as well as basics for using the software with MATLAB and LabVIEW.

0.2 THE FOUR VOLUMES OF THE SERIES

The present volume, Volume IV of the series, LMS Adaptive Filtering, begins by explainingcost functions and performance surfaces, followed by the use of gradient search techniques us-ing coefficient perturbation, finally reaching the elegant and computationally efficient Least MeanSquare (LMS) coefficient update algorithm. The issues of stability, convergence speed, and narrow-bandwidth signals are covered in a practical manner, with many illustrative scripts. In the secondchapter of the volume, use of LMS adaptive filtering in various filtering applications and topologiesis explored, including Active Noise Cancellation (ANC),system or plant modeling, periodic compo-nent elimination, Adaptive Line Enhancement (ADE), interference cancellation, echo cancellation,and equalization/deconvolution.

Volume I of the series, Fundamentals of Discrete Signal Processing, consists of fourchapters. The first chapter gives a brief overview of the field of digital signal processing. This isfollowed by a chapter detailing many useful signals and concepts, including convolution, recursion,difference equations, etc. The third chapter covers conversion from the continuous to discrete do-main and back (i.e., analog-to-digital and digital-to-analog conversion), aliasing, the Nyquist rate,normalized frequency, conversion from one sample rate to another, waveform generation at varioussample rates from stored wave data, and Mu-law compression. The fourth and final chapter of Vol-

Page 16: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

xvi PREFACE

ume I introduces the reader to many important principles of signal processing, including correlation,the correlation sequence, the Real DFT, correlation by convolution, matched filtering, simple FIRfilters, and simple IIR filters.

Volume II of the series is devoted to discrete frequency transforms. It begins with anoverview of a number of well-known continuous domain and discrete domain transforms, and coversthe DTFT (Discrete Time Fourier Transform), the DFT (Discrete Fourier Transform), Fast FourierTransform (FFT), and the z-Transform in detail. Filter realizations (or topologies) are also covered,including Direct, Cascade, Parallel, and Lattice forms.

Volume III of the series is devoted to Digital Filter Design. It covers FIR and IIR design,including general principles of FIR design, the effects of windowing and filter length, characteristicsof four types of linear phase FIR, Comb and MA filters, Windowed Ideal Lowpass filter design,Frequency Sampling design with optimized transition band coefficients, Equiripple FIR design, andClassical IIR design.

0.3 ORIGIN AND EVOLUTION OF THE SERIES

The manuscript from which the present series of four books has been made began with an idea toprovide a basic course for intellectual property specialists and engineers that would provide moreexplanation and illustration of the subject matter than that found in conventional academic books.The idea to provide an accessible basic course in digital signal processing began in the mid-to-late1990’s when I was introduced to MATLAB by Dan Hunter, whose graduate school days occurredafter the advent of both MATLAB and LabVIEW (mine did not). About the time I was seriouslyexploring the use of MATLAB to update my own knowledge of signal processing, Dr. Jeffrey Gluckbegan giving an in-house course at the agency on the topics of convolutional coding, trellis coding,etc., thus inspiring me to do likewise in the basics of DSP, a topic more in-tune to the needs of theunit I was supervising at the time. Two short courses were taught at the agency in 1999 and 2000 bymyself and several others, including Dr. Hal Zintel, David Knepper, and Dr.Pinchus Laufer. In thesecourses we stressed audio and speech topics in addition to basic signal processing concepts. Thanksto The Mathworks, Inc., we were able to teach the in-house course with MATLAB on individualcomputers, and thanks to Jim Dwyer at the agency, we were able to acquire several server-basedconcurrent-usage MATLAB licenses, permitting anyone at the agency to have access to MATLAB.Some time after this, I decided to develop a complete course in book form, the previous courses havingconsisted of an ad hoc pastiche of topics presented in summary form on slides, augmented with visualpresentations generated by custom-written scripts for MATLAB. An early draft of the book waskindly reviewed by Motorola Patent Attorney Sylvia Y. Chen, which encouraged me to contact TomRobbins at Prentice-Hall concerning possible publication. By 2005, Tom was involved in starting apublishing operation at National Instruments, Inc., and introduced me to LabVIEW with the ideaof possibly crafting a book on DSP to be compatible with LabVIEW. After review of a manuscriptdraft by a panel of three in early 2006, it was suggested that all essential foundation mathematics beincluded so the book would have both academic and professional appeal. Fortunately, I had long since

Page 17: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

0.3. ORIGIN AND EVOLUTION OF THE SERIES xvii

retired from the agency and was able to devote the considerable amount of time needed for such aproject.The result is a book suitable for use in both academic and professional settings, as it includesessential mathematical formulas and concepts as well as simple or “first principle” explanations thathelp give the reader a gentler entry into the more conventional mathematical treatment.

This double-pronged approach to the subject matter has, of course, resulted in a book ofconsiderable length. Accordingly, it has been broken into four modules or volumes (described above)that together form a comprehensive course, but which may be used individually by readers who arenot in need of a complete course.

Many thanks go not only to all those mentioned above, but to Joel Claypool of Mor-gan&Claypool, Dr. C.L.Tondo and his troops, and, no doubt, many others behind the scenes whosenames I have never heard, for making possible the publication of this series of books.

Forester W. IsenJanuary 2009

Page 18: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is
Page 19: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

1

C H A P T E R 1

Introduction To LMS AdaptiveFiltering

1.1 OVERVIEW1.1.1 IN PREVIOUS VOLUMESThe previous volumes of the series are Volume I, Fundamentals of Discrete Signal Processing, VolumeII, Discrete Frequency Transforms, and Volume III, Digital Filter Design. Volume I covers DSP fun-damentals such as basic signals and LTI systems, difference equations, sampling, the Nyquist rate,normalized frequency, correlation, convolution, the real DFT, matched filtering, and basic IIR andFIR filters. Volume II covers the important discrete frequency transforms, namely, the Discrete TimeFourier Transform (DTFT), the Discrete Fourier Transform (DFT), and the z-transform. VolumeIII considers the design of both FIR and IIR filters that are intended to have fixed characteristicsfor use in a static environment.

1.1.2 IN THIS VOLUMEThere are many situations in which needed filter characteristics change rapidly, or in which theneeded characteristics are not precisely known in advance. Adaptive filtering fills these needs byautomatically changing filter coefficients to achieve some particular goal. In this volume, we takeup the study of filters that can adapt their characteristics according to some criteria. In particular,our study will center on LMS Adaptive filtering, which is popular because of its low computationaloverhead. It has found applications in Active Noise Cancellation (ANC), echo cancellation in tele-phone systems, beamforming, narrowband signal attenuation or enhancement, equalizers, etc. Inthis volume, consisting of two chapters, we first examine several fundamental ideas which culminatein derivation of the LMS Algorithm. In the second chapter, we examine a number of common filtertopologies or applications of LMS Adaptive filtering.

1.1.3 IN THIS CHAPTERWe begin our study of adaptive filtering by first investigating cost functions, performance surfaces,and the gradient, using a simple technique called Coefficient Perturbation. We’ll consider severalexamples, including one- and two- independent-variable quadratic cost functions, including theproblem of fitting a line to a set of points in a plane. The simplicity of coefficient perturbation willenable the reader to readily understand fundamental concepts and issues relevant to gradient-search-based adaptive filtering. We then derive a more efficient way of estimating the gradient, the simple,elegant, and efficient algorithm known as the LMS (Least Mean Squared) algorithm, which, with

Page 20: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

2 CHAPTER 1. INTRODUCTION TO LMS ADAPTIVE FILTERING

its various derivative algorithms, accounts for the majority of adaptive algorithms in commercial use.We’ll derive the algorithm for a length-2 FIR, which readily generalizes to a length-N FIR. Wethen explore one of the weaknesses of the LMS algorithm, its need for a wide-bandwidth signal forproper algorithm function in certain topologies, such as ANC.

By the end of the chapter, the reader will have gained fundamental knowledge of the LMSalgorithm and should be able (and, in the exercises at the end of the chapter, is expected) to writescripts to implement simple LMS adaptive FIR filters. The reader will then be prepared for the nextchapter in which we explore the LMS algorithm in various useful applications, such as active noisecancellation, signal enhancement, echo cancellation, etc.

1.2 SOFTWARE FOR USE WITH THIS BOOK

The software files needed for use with this book (consisting of m-code (.m) files, VI files (.vi), andrelated support files) are available for download from the following website:

http://www.morganclaypool.com/page/isen

The entire software package should be stored in a single folder on the user’s computer, and thefull file name of the folder must be placed on the MATLAB or LabVIEW search path in accordancewith the instructions provided by the respective software vendor (in case you have encountered thisnotice before, which is repeated for convenience in each chapter of the book, the software downloadonly needs to be done once, as files for the entire series of four volumes are all contained in the onedownloadable folder). See Appendix A for more information.

1.3 COST FUNCTION

A common problem is the need to fit a line to a set of test points.The test points, for example,might bedata taken from an experiment or the like.We’ll start by investigating how a “best fit” curve (or, in theexample at hand, straight line) can be determined mathematically, i.e., what mathematical criterioncan be used for an algorithm to determine when a candidate curve or line is better than another?

In Fig. 1.1, five test points (circles) are to have a horizontal line fitted in the “best” mannerpossible using purely mathematical criteria. For purposes of simplicity, the line is allowed to varyonly in the vertical direction. In a real problem, the points would likely be irregularly placed and theline would have to assume different slopes as well as different vertical positions (we will examineline-modeling in just this way later in the chapter).

In Fig. 1.1, if we define the vertical location of the line as y; the distance from each upperpoint to the line is (2 − y), and the distance from each of the two lower points to the line is y. We’llconstruct three alternative functions of y and compare their performance in determining the bestvalue of y to place the line in an intuitively, satisfying location.

Page 21: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

1.3. COST FUNCTION 3

0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 5.5−2

−1

0

1

2

3

4

Y = 1.2

Y

X

Figure 1.1: Five points in a plane to be modeled by a line y = a, where a is a constant.

For the first function, we’ll use a first order statistic, that is to say, we’ll sum the variousdistances and normalize by the number of distances (i.e., divide the sum by the number of distances),in accordance with Eq. (1.1):

F1(y) = 1

5(3(2 − y) + 2y) (1.1)

For the second function, we’ll also use a first order statistic, but we’ll sum the magnitudes ofthe various distances and normalize. This results in Eq. (1.2):

F2(y) = 1

5(3(|2 − y|) + 2|y|) (1.2)

For the third function, we’ll use second-order statistics, i.e., the square of distances from theline to the various points to be modeled. To do this, we square each distance mentioned above, sumall squared distances, and normalize. We’ll call this the MSE (mean squared error) method.

Using the MSE method, the function of distance looks like this:

F3(y) = 1

5(3(2 − y)2 + 2y2) (1.3)

We can evaluate Eqs. (1.1) through (1.3) with a test range for y to see how well each functiongenerates an intuitively satisfying location for the line between the two rows of test points. Letting

Page 22: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

4 CHAPTER 1. INTRODUCTION TO LMS ADAPTIVE FILTERING

the three functions given above be represented respectively by F1, F2, and F3, the results fromrunning the following m-code are shown in Fig. 1.2:

y = -1:0.05:3;F1 = (1/5)*(3*(2-y) + 2*(y));F2 = (1/5)*(3*abs(2-y) + 2*abs(y));F3 = (1/5)*(3*(2-y).ˆ2 + 2*(y.ˆ2));

−1 −0.5 0 0.5 1 1.5 2 2.5 30.5

1

1.5

(a) y

F1(y

)

−1 −0.5 0 0.5 1 1.5 2 2.5 30

2

4

F2(y

)

(b) y

−1 −0.5 0 0.5 1 1.5 2 2.5 30

5

10

(c) y

F3(y

)

Figure 1.2: (a) The function F1(y) (see text) evaluated over the range y = -1:0.05:3; (b) Same, forfunction F2(y); (c) Same, for function F3(y).

We can refer to F1, F2, and F3 as Cost Functions since there is a “cost” or value associatedwith placing the line at certain positions. As in purchases using money, the goal is to minimizethe cost.

Intuitively, the best location for the line is somewhere between the three upper points and thetwo lower points, and probably a little closer to the three upper points.

Neither first order cost function (plots (a) and (b) of Fig. 1.2) does well. The first functiondoes not provide any useful guidance; the second one does give guidance, but its minimum occurswhen the line passes through the three upper points (when y = 2). This does not suit our intuitivenotion of best placement.

The second order, or MSE cost function (plot (c) of Fig. 1.2) shows its minimum cost at y

= 1.2, which seems a reasonable placement of the line. Notice also that there is a single, distinct

Page 23: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

1.4. PERFORMANCE SURFACE 5

minimum for the second order cost function. It is clear that the best cost function for this simpleexample is MSE or second order. This is true in general for many problems, and it is the costfunction that we will be using in this and the following chapter in the development and use ofvarious adaptive algorithms.

• Second order cost functions (or MSE functions), have single (“global”) minima that producean intuitively satisfying “best” result for many line and curve fitting problems.

1.4 PERFORMANCE SURFACEAny of plots (a)-(c) of Fig. 1.2 may be described as Performance Curves. Each is a graph of anindependent variable versus the corresponding value of a cost function. In situations where thereare two or more independent variables, the cost function, graphed as a function of its independentvariables, takes on three or more dimensions and hence is a surface rather than a simple curve. Hence,it is called a Performance Surface. We will see examples of the performance surface a little later,but first, we’ll take a look at a couple of general methods for “reaching the bottom” of a performancecurve algorithmically, that it to say, using a computer program.

For this kind of problem, in general, we will have one or more independent variables whichwhen varied will cause the cost function to vary, and our goal is for a computer program or algorithmto change the values of the one or more independent variables in such a manner that eventually thecost function is minimized.

1.5 COEFFICIENT PERTURBATIONAn intuitive way of determining how to adjust each independent variable to minimize the costfunction is to let all of the independent variables assume certain fixed values (i.e., assign values tothe variables) and then evaluate the cost function. Next, change one of the variables by a very smallamount (this is called perturbation) while holding all the other variable values constant, and thenevaluate the cost function. If the change to the particular variable in question was in the positivedirection, for example, and the cost function increased in value, then logically, changing the value ofthat variable in the negative direction would tend to decrease the value of the cost function.

A mathematical way of describing this, for a cost function that depends on the single inde-pendent variable x, would be

CF = �(CostFcn)

�x= CostFcn(x + �x) − CostFcn(x)

�x(1.4)

The ratio in Eq. (1.4) tells how much and in what direction the cost function varies for a unitchange in the independent variable x. The limit of the ratio in Eq. (1.4) as �x goes to zero is thederivative of the cost function CF with respect to x:

CF ′(x) = d(CF)

dx= lim

�x→0

(CF(x + �x) − CF(x)

�x

)

Page 24: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

6 CHAPTER 1. INTRODUCTION TO LMS ADAPTIVE FILTERING

This concept is easily extended to a cost function that is dependent upon two or more inde-pendent variables. Then if

y = f (x1, x2, ...xN)

the partial derivative of y with respect to a particular independent variable xi is

∂(y)

∂(xi)= lim

�xi→0

f (x1, x2, ...(xi + �xi), ...xN) − f (x1, x2, ...xN)

�xi

In computer work, there is a limit to precision, so making �xi smaller and smaller does not,beyond a certain point, improve the estimate of the partial derivative.Thus it is not possible (referringto the single-independent-variable example at hand) to obtain

lim�x→0

CF(x + �x) − CF(x)

�x(1.5)

since, due to roundoff error, the true limit will in general not be monotonically approached orachieved. For most normal purposes, however, this is not a serious problem; a useful estimate of thepartial derivative can be obtained. One method that can increase the accuracy from the very firstestimate is to straddle the value of x. This is accomplished by dividing �x by two and distributingthe two parts about x as follows:

CF ′(x) � CF(x + �x/2) − CF(x − �x/2)

�x(1.6)

In Eq. (1.6), we have adopted the symbol � to represent the estimate or approximation ofthe partial derivative of CF(x) with respect to x. Note that the two values of x at which the costfunction is computed surround the actual value of x, and provide a better estimate of the slope ofthe cost function at x than would be provided by evaluating the cost function at x and x + �x. Thismethod is referred to as taking Central Differences, as described in Chapter 5 of Reference [1].

The Gradient of a function (such as our cost function CF ) for N independent variables xi isdenoted by the del (∇) operator and is defined as

∇(CF) =N∑

i=1

(∂(CF)/∂(xi))ui (1.7)

where ui is a unit vector along a dimension corresponding to the independent variable xi .The purposeof the unit vectors is to maintain each partial derivative as a distinct quantity. In the programs andexamples below involving cost functions dependent on two or more independent variables, eachpartial derivative is simply maintained as a distinct variable.

Once all the partial derivatives have been estimated using perturbation, it is possible to updatethe current best estimate for each of the independent variables xi using the following update equation:

Page 25: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

1.6. METHOD OF STEEPEST DESCENT 7

xi[k + 1] = xi[k] − μCF ′(xi, k) (1.8)

where CF ′(xi, k) means the estimate of the partial derivative of CF with respect to independentvariable xi at sample time k. Equation (1.8) may be described verbally thus: the estimate of thevariable xi at iteration (k + 1) is its estimate at iteration k minus the product of the partial derivativeof the cost function with respect to xi at iteration k and a small value μ, which helps regulate theoverall size of the update term, μCF ′(xi, k).

Too large a value of μ will cause the algorithm to become unstable and diverge (i.e., moveaway from the minimum of the cost function) while too small a value will result in protracted timesfor convergence. New estimates of the coefficients are computed at each sample time and hopefullythe coefficient estimates will converge to the ideal values, which are those that minimize the costfunction.Convergence to the true minimum of the performance surface,however, can only be reliablyachieved if the performance surface is unimodal, i.e., has one global minimum. Some performancesurfaces may possess local minima to which the gradient may lead, and the algorithm can fail toconverge to the true minimum of the cost function. Fortunately, the problems we’ll investigate dohave unimodal performance surfaces. As we work through various examples, we’ll see how variousparameters and situations affect algorithm convergence.

1.6 METHOD OF STEEPEST DESCENTUsing the negative of the gradient to move down the performance surface to the minimum value ofthe cost function is called the Method of Steepest Descent; this is because the gradient itself pointsin the direction of steepest ascent, while the negative of the gradient, which we use in coefficientupdate Eq. (1.8) moves in precisely the opposite direction–the direction with the steepest slope downthe performance surface.

The following discussion uses a simple quadratic cost function to explore a number of impor-tant concepts involved in adaptive processes, such as gradient (slope), step size, convergence speed,accuracy of gradient estimate, etc. In the several plots of Fig. 1.3, the cost function CF(x) is

CF(x) = x2 + 1

and the ultimate goal is to determine the value of x at which the CF(x) is minimized, i.e., reducedto 1.0.

Plot (a) of Fig. 1.3 shows the first step in navigating a Performance Surface to its minimumusing the method of steepest descent, in which an initial value of x was chosen arbitrarily to be 4.0.From this, the value of the gradient must be estimated and the estimate used to compute a new valueof x which is (hopefully) closer to the value of x which minimizes the cost function.

In any of the plots of Fig. 1.3, values of the cost function versus x are plotted as small circles;the method of steepest descent may be understood by imagining the circles as being marbles, andthe performance curve as a surface down which the marbles can roll under the pull of gravity. In

Page 26: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

8 CHAPTER 1. INTRODUCTION TO LMS ADAPTIVE FILTERING

−5 0 50

10

20y

(a) x

−5 0 50

10

20

y

(c) x

−5 0 50

10

20

y

(b) x

−5 0 50

10

20

y(d) x

Figure 1.3: (a) Graph of cost function and first value of parameter x and the corresponding value ofthe cost function; (b) Successive estimated values of x and respective values of the cost function for 75iterations; (c) Seventy-five iterations, attempting to reach the minimum value of the cost function usingtoo small a value of StepSize (μ); (d) Three iterations using a weighting function for μ.

a more complex problem with the performance surface being a (three-dimensional) paraboloid-of-revolution rather than a simple two-dimensional curve, such a “marble” would roll straight downalong the inside of the paraboloid (a bowl-shaped surface) toward the bottom.

The script

LVxGradientViaCP(FirstValX, StepSize,...

NoiseAmp,UseWgtingFcn, deltaX, NoIts)

(see exercises below) allows experimentation with the method of steepest descent using a second-order cost function having a single independent variable. A typical call would be:

LVxGradientViaCP(4,0.05,0.01,0,0.0001,50)

which results in Fig. 1.3, plot (a), for Iteration 1; the script is designed so that successive iterationsof the algorithm are performed by pressing any key.

Page 27: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

1.6. METHOD OF STEEPEST DESCENT 9

The first argument, FirstValX, is the starting value (the initial guess) for the only independentvariable, x. StepSize is equivalent to μ, the parameter that regulates the size of the update term.NoiseAmp is the standard deviation of white noise that is added to the current estimate of x to morerealistically simulate a real (i.e., noisy) situation (set it to zero if desired). The fifth argument, deltaX(�x in the pseudocode below), is the amount by which to perturb the coefficients when estimatingthe gradient. The sixth argument, NoIts, is the number of iterations to do in seeking the value of x

that minimizes the cost function. The variable UseWgtingFcn (UWF in the pseudocode below) willbe discussed extensively below.

The heart of the algorithm, in pseudocode terms, is

f or n = 1 : 1 : NoI ts

CF [n] = (x[n] − �x/2)2 + 1T stCF = (x[n] + �x/2)2 + 1∂(CF)/∂(x[n]) = (T stCF − CF [n])/�x

if UWF = 1x[n + 1] = x[n] − μCF [n](∂(CF)/∂(x[n])else

x[n + 1] = x[n] − μ(∂(CF)/∂(x[n])end

end

The pseudocode above parallels the preceding discussion exactly, i.e., implementing an algo-rithm that performs gradient estimation via central-differences-based coefficient perturbation. Theproduct of ∂(CF)/∂(x[n]) and μ (StepSize) is the amount by which the value of x is to be changedfor each iteration. This quantity may be further modified (as shown above) by multiplying it by thecurrent value of CostFcn (CF [n]) in order to cause the entire term to be large when the error (andhence CF ) is large, and small when CF becomes small. This helps speed convergence by allowingthe algorithm to take larger steps when it is far from convergence and smaller steps when it is closeto convergence. The fourth input argument to the function call, UseWeightingFcn (UWF in thepseudocode above), determines whether or not the update term uses CF [n] as a weighting functionor not.

The coefficient update term, the product of ∂(CF)/∂(x[n]),μ, and CF [n] if applied, has greatsignificance for convergence and stability. If the update term is too large, the algorithm diverges; ifit is too small, convergence speed is too slow. In the following examples, we’ll observe convergencerate and stability with the update term computed with and without the weighting function.

Example 1.1. Use the script LVxGradientViaCP to demonstrate stable convergence.

We call the script with no weighting function and StepSize = 0.03:

LVxGradientViaCP(4,0.03,0.01,0,0.0001,75)

Figure 1.3, plot (b), shows the result. Convergence proceeds in a slow, stable manner.

Page 28: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

10 CHAPTER 1. INTRODUCTION TO LMS ADAPTIVE FILTERING

• A useful line of experimentation is to hold all input arguments except StepSize (μ) constant,and to gradually increase μ while noticing how the algorithm behaves. For this set of inputparameters (i.e.,with no weighting function), the reader should note that the value ofμ at whichthe algorithm diverges is about 1.0. If using a weighting function, the algorithm will behavemuch differently for a given value of μ. The following examples point out different significantaspects of algorithm performance based (chiefly) on the parameters of μ and UseWgtingFcn.

Example 1.2. Demonstrate inadequate convergence rate due to too small a value of StepSize.

The result of the use of the same call as for the preceding example, but with μ = 0.005, isshown in Fig. 1.3, plot (c), in which 75 iterations are insufficient to reach the minimum value of thecost function.

Example 1.3. Demonstrate stable overshoot by use of too large a value of StepSize, such as 0.9,with no weighting function.

An appropriate call is

LVxGradientViaCP(4,0.9,0.01,0,0.0001,50)

which results in overshooting the minimum, but eventually descending to it. Figure 1.4 shows whathappens–the successive estimates of x oscillate around the ideal value of x, but the amplitude ofoscillation gradually decays, leading to convergence to the minimum of the cost function.

Example 1.4. Experimentally determine the threshold value of StepSize above which instabilityresults, without the use of the weighting function.

By letting μ vary upward in successive calls, it will be found that the following call, with μ = 1,shows a general borderline stability; values of μ above 1.0 (with no weighting function) will clearlydiverge. Figure 1.5 shows this condition, in which successive estimates of x lie at approximately ±4.

LVxGradientViaCP(4,1,0.01,0,0.0001,50)

Example 1.5. Demonstrate instability using the script LVxGradientViaCP with a value of StepSizegreater than about 0.0588 and a weighting function.

The call

LVxGradientViaCP(4,0.0589,0.01,1,0.0001,20)

will show that the algorithm rapidly diverges when μ is greater than about 0.058, and the fourthargument calls for CostFcn to be applied as a weighting function. The result is shown in Fig. 1.6.

Page 29: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

1.6. METHOD OF STEEPEST DESCENT 11

−6 −4 −2 0 2 4 60

5

10

15

20

25

Y

X

Figure 1.4: A stable overshoot (or underdamped) condition, in which, despite the fact that step size (μ)is over-large, convergence nonetheless occurs since μ is not quite large enough to lead to divergence.

−6 −4 −2 0 2 4 60

5

10

15

20

25

Y

X

Figure 1.5: A condition in which μ is such that x neither converges nor diverges, but oscillates betweentwo values.

Page 30: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

12 CHAPTER 1. INTRODUCTION TO LMS ADAPTIVE FILTERING

−6 −4 −2 0 2 4 60

5

10

15

20

25

Y

X

Figure 1.6: A condition in which a value of μ of 0.0589 is sufficient to make the algorithm rapidlydiverge. The successive values are connected by arrows; the fifth and successive estimates of x no longerfit on the graph.

Example 1.6. Demonstrate rapid, stable convergence by use of the weighting function and a mod-erate value of StepSize, 0.03.

A call meeting the stated requirement is

LVxGradientViaCP(4,0.03,0.01,1,0.0001,20)

and the results are shown in Fig. 1.3, plot (d).The difference from Fig. 1.3, plot (b), in which μ was 0.03, but with no weighting function, is

remarkable. In Fig. 1.3, plot (d), the conditions are just right for rapid convergence; in fact, after thesecond iteration, the movement of x is miniscule since it has very nearly reached the minimum, andthe weighting function has a very small value, 1.003, as opposed to its value of about 17 when x = 4.Thus the magnitude of the update term was, considering only the value of the weighting function,about 17 times greater at Iteration 1 than it was for Iteration 2, when x had jumped to a value nearto zero, the minimum of the cost function. The net value of the update term is the product not onlyof the weighting function and μ, but also the value of the partial derivative of the cost function withrespect to the (single) variable x. It should be intuitively obvious that for an equal change in x (�x),the cost function changes much more at, say, x = 4 than it does when x equals (say) 1. In fact, the

Page 31: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

1.6. METHOD OF STEEPEST DESCENT 13

partial derivative of our particular cost function is really just the slope of the cost function curve,which of course changes with x. The slope of our CostFcn (which equals x2 + 1) is readily obtainedfrom calculus as simply its derivative:

d(CF)

dx= d(x2 + 1)

dx= 2x (1.9)

Thus we can compute the value of the coefficient update term at different values of x to seehow it varies with x. The basic coefficient update formula is

�C = μ (∂(CF)/∂(x[n]))CF [n] (1.10)

with

∂(CF)

∂(x[n]) = d(CF)

dx= 2x

For x = 4, μ = 0.03, and CF [4] = 17 (42 +1 = 17), we get

�C = (0.03)(2)(4)(17) = 4.08x[2] = x[1] - �C

x[2] = 4 - 4.08 = -0.08

This places x(2) near the minimum of the cost function. If you run the script with theseconditions, the actual numbers will vary according to the amount of noise you specify in the functioncall. It should be clear that if the initial value of x is too large, the initial update term will take thenext estimate of x to a value having a magnitude greater than the initial value of x, but with oppositesign. The third value of x will have an even larger magnitude, with opposite sign from the secondvalue of x, and so forth.

• Since we have an explicit formula for the cost function, it is possible to compute values for thevariables, such as the initial value of x, μ, etc., which will result in borderline stability, or animmediate jump to the minimum value of the cost function. In most real problems, of course,no such explicit formula for the cost function is available, but this does not present a problemsince a number of methods, including coefficient perturbation, are available for estimating thegradient and thus enabling the algorithm to navigate to the bottom of the performance surface.

• The optimization and control of μ (StepSize) is an important consideration in adaptive filtering.If μ is too small, convergence is very slow, and if it is just right, a single iteration may place theindependent variable very near the optimum value, i.e., where the cost function is minimized.A weighting function can be used to effectively boost the value of μ when the distance fromthe cost function minimum is great, and reduce it as the cost function is reduced. If μ is largerthan optimum, overshoot occurs: x overshoots the target. In fact, if μ is too large, the algorithmwill diverge.

Page 32: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

14 CHAPTER 1. INTRODUCTION TO LMS ADAPTIVE FILTERING

• Another reason to reduce μ near convergence is that the steady-state error or misadjustmentis larger if μ is larger.

• μ is usually controlled by a weighting function, which must at least maintain stability, andhopefully allow for good convergence speed.

• The weighting function chosen for the script LVxGradientViaCP is unregulated, meaning thataccount has not been taken for all input conditions. For example, the call

LVxGradientViaCP(400,0.03,0.01,1,0.0001,75)

causes divergence because the weighting function is used with a very large starting value of x =400, which causes the initial update term (considering the value of the weighting function, whichis 16,001) to be too large. It should be obvious that the size of the update term must be carefullycontrolled to maintain adequate convergence speed and stability. We will discuss methods for ac-complishing this in practical algorithms, such as the LMS algorithm, and a regulated version calledthe NLMS algorithm, later in the chapter.

1.7 TWO VARIABLE PERFORMANCE SURFACESo far, we have examined adaptive problems with just one independent variable. A problem whichhas two variables is that of fitting a straight line to a set of points that may be distributed randomlyin a plane, but which may be modeled by the point-slope form of the equation for a line in thex-y plane:

y = Mx + b

where M is the slope of the line, and b is its y-axis intercept.The coefficient perturbation method of gradient estimation is employed in the script

LVxModelPtswLine(TestM,TestYInt,xTest,...

Mu,yMu2MuRatio,MStart,yIntStart,NoIts)

a typical call for which would be:

LVxModelPtswLine(2,0,[-10:1:10],0.005,1,-10,8,40) (1.11)

the results of which, plotted on the performance surface, are shown in Fig. 1.7.This script allows you to specify the slope and y-intercept values (TestM and TestYInt) which

are used to compute the test points to be modeled. Mu is the parameter μ, which scales the updateterm. To enable speedier convergence, the value of μ to be used for the y-intercept variable is equal

Page 33: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

1.7. TWO VARIABLE PERFORMANCE SURFACE 15

−15

−10

−5

0

5

10

15−20 −15 −10 −5 0 5 10 15 20

0

10000

Slope

y−Intercept

MS

E

First Coeff Est.N−th Coeff Est.

Figure 1.7: Performance Surface with plot of successive coefficient estimates using equal step size weightsfor both variables in coefficient update, leading to very slow movement down the Performance Surfaceonce the bottom of the gently-sloped trough portion has been reached. This illustrates the Method ofSteepest Descent.

to Mu multiplied by yMu2MuRatio. The fact that different variables have, in general, differentoptimum step sizes, is an important point that we’ll discuss and experiment with below.

MStart and yIntStart specify the starting values of the variables. You can experiment withdifferent initial values of the variables to see what happens. In some cases, when an algorithm isbeing used in specific environments, which may change suddenly but in a historically known way,the values of the variables may be preset to values near what they are likely to eventually convergeto, based on prior experience. This technique can greatly reduce convergence time.

Figure 1.8 shows a line to be modeled and the first, second, and fiftieth iterations of themodeling process. The process is very straightforward. The line to be modeled is created using givenslope (M) and y-Intercept (B) values. Since discrete mathematics are used, a set of test values aredetermined, starting with a range of values of x, such as

x = [0 : 0.5 : 3]for example. A set of corresponding values of y is generated using the vector x and the given valuesof M and B. The first estimates for M and B, which we’ll call eM and eB, are used to create, using

Page 34: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

16 CHAPTER 1. INTRODUCTION TO LMS ADAPTIVE FILTERING

the x vector, a corresponding set of y values, which we can call eY . Then the mean squared error isgiven as

MSE = 1

N

N−1∑n=0

(y[n] − eY [n])2

From this, estimates of the partial derivative of MSE with respect to eM and eB (the currentestimated values of M and B) are made using coefficient perturbation using central differences, andthen new values for eM and eB are computed using the relationship specified by Eq. (1.8). Theprocess is repeated until some criterion is reached, usually that the magnitude of the error becomesquite small, or that the amount of improvement per iteration becomes less than some thresholdvalue, or perhaps some combination of the two.

0 0.5 1 1.5 2 2.5 3

−1

0

1

2

3

4

5

x

y

Final coefficient estimates: m = 1.9498; b = −0.89285

Initial coefficient estimate

Second coefficient estimate

50th coefficient estimate

Line to be modeled

Figure 1.8: Determining the slope and y-Intercept of a line using an iterative method. The line to bemodeled is shown as a solid line which passes through the sample or data points which are plotted ascircles. Also shown are (estimated) dashed lines based on the first, second, and 50th coefficient estimates.

Figure 1.8 was created using the script

LV_ModelLineLMS(xVec, yVec, Mu, Its)

and in particular, the call

LV_ModelLineLMS([0,1,2,3], [-1,1,3,5], 0.05, 50)

Page 35: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

1.7. TWO VARIABLE PERFORMANCE SURFACE 17

On the other hand, the script LVxModelPtswLine provides a separate value of μ for the y-intercept variable by weighting μ for the y-intercept variable relative to μ for the slope variableaccording to the user-input variable yMu2MuRatio. It is easy to see why the coefficient for slopeconverges more quickly than the coefficient for y-intercept by inspecting Fig. 1.8 and imagininghow much a small amount of misadjustment of slope affects the value of MSE compared to an equalamount of misadjustment of y-intercept. Clearly, a slight misadjustment of slope leads to a muchlarger change in MSE than does the same misadjustment in y-intercept. By artificially boosting thecoefficient update term weight for y-intercept, we can make the algorithm converge more quickly.

Let’s return to Fig. 1.7, which shows the coefficient estimate track resulting from making thecall to the script given at (1.11), plotted with respect to the performance surface, which was itselfgenerated by systematically varying the two coefficients, computing the resulting MSE, and thenplotting the resultant data as a surface. Notice in Fig. 1.7 that the performance surface has differentslopes in the two dimensions representing slope and y-intercept. Since there are two independentvariables, each must converge to its ideal value to minimize the cost function. Since the partialderivative of one of them is very much smaller than the other (at least starting from the initialcoefficient estimate, neither coefficient of which is equal to its ideal or test value), it will actuallyconverge much more slowly. In this case, yIntNow (the variable representing the estimate for y-intercept in the script) converges very slowly compared to SlopeNow (the variable representing theestimate for slope in the script).

The method of steepest descent, which uses the negative of the gradient to estimate the nextpoint to move to on the performance surface, moves along the direction which reduces the MSE bythe largest amount for a given very small amount of movement. In Fig. 1.7, the performance surfaceis much like a river that has relatively steep banks which slope not only down to the river, but whichslope slightly downward along the direction of river flow. If a ball were released from the top ofsuch a river bank, its direction of travel would be determined by gravity. It would appear to mostlymove directly toward the river, i.e., straight down the river bank to the water. There would, however,under the given assumptions, be a slight bias toward the direction the river is flowing since in factthe banks also have a slight downward slope in the same direction as the river’s flow. In fact, theball would be following gravity along a direction which reduces gravitational potential the greatestamount for a given amount of travel. The path a ball would follow down such a river bank is similarto the path followed in Fig. 1.7 by the coefficient estimate pair yIntNow and SlopeNow as they movetoward the correct values of TestYInt and TestM. Carrying the analogy further, if the point of lowestgravitational potential (analogous to the minimum value of the cost function) is at the river’s mouth,the fastest way to get there (along the shortest path) is one which heads directly for the mouth, notone which first rolls down the banks to the river and then suddenly turns downstream to head for themouth at a very leisurely pace (this particular situation is analogous to the one shown in Fig. 1.7).

• The method of steepest descent is not necessarily one which takes the shortest and/orfastest path to the minimum of the cost function. This is a function of the shape of theperformance surface.

Page 36: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

18 CHAPTER 1. INTRODUCTION TO LMS ADAPTIVE FILTERING

Method of Steepest Descent ModifiedIn the script LVxModelPtswLine, the rate at which the slower coefficient converges is modified bythe multiplier yMu2MuRatio, leading to a more optimal path from starting point to minimum valueof cost function.

Example 1.7. Experimentally determine a value of yMu2MuRatio which will facilitate rapid con-vergence of the algorithm in the script LVxModelPtswLine.

A good way to start is with a basic call that results in stable convergence, and to graduallyincrease the value of yMu2MuRatio. A little such experimentation results in a useful set of parametersfor convergence with Mu = 0.009 and yMu2MuRatio = 37. The result of the call

LVxModelPtswLine(2,0,[-10:1:10],0.014,37,-10,8,12)

is shown in Fig. 1.9. You should repeat the above call a number of times, varying the parametersTestM, TestYInt, MStart, and yIntStart, to observe the result. Does the algorithm still convergeimmediately?

−15

−10

−5

0

5

10

15−20

−15−10

−50

510

1520

0

10000

Slope

y−Intercept

MS

E

First Coeff Est.

2nd through N−th Coeff Est.

Figure 1.9: Performance Surface with plot of successive coefficient estimates utilizing a weighting func-tion to accelerate the slower-converging of the two coefficients. Note that convergence is extremely rapidsince the value of μ for the y-intercept variable has been adjusted to greatly speed the convergence ofthat variable toward its value of best adjustment.

Page 37: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

1.8. AN IMPROVED GRADIENT SEARCH METHOD 19

Using the river analogy, we can see that instead of moving the way gravity would dictate,which would be along a nearly direct path toward the river (with only a slight bias downstream), andthen suddenly turning to head directly downstream, the algorithm immediately headed directly forthe point of minimized cost function, (the river mouth in our analogy).

In Fig. 1.9, near-convergence occurs in a few iterations, with additional iterations makingonly very minute improvements.

1.8 AN IMPROVED GRADIENT SEARCH METHOD

Thus far in the chapter we have used coefficient perturbation to estimate the gradient in severaladaptive processes; we will now develop an analytic expression to estimate the gradient in the two-variable line-modeling process just discussed. This will be accomplished by writing an analyticexpression for the mean squared error, and then obtaining the partial derivative with respect to m

and b. This will permit us to write coefficient update equations consisting of analytic expressions.If we represent the points to be modeled as two vectors xV ec and yV ec of length M , we can

write the mean squared error as

MSE[n] = 1

M

M−1∑i=0

(yV ec[i] − (m[n])(xV ec[i]) − b[n])2 (1.12)

which is the sum of the mean squared error at each point be modeled. Since yV ec and xV ec areconstants, and b is considered a constant when computing the partial derivative of MSE with respectto m, we get the partial derivative of MSE with respect to m as

∂(MSE[n])∂(m)

= 2

M

M−1∑i=0

(yV ec[i] − (m[n])(xV ec[i]) − b[n])(−xV ec[i]) (1.13)

and similarly the partial derivative of MSE with respect to b as

∂(MSE[n])∂(b)

= 2

M

M−1∑i=0

(yV ec[i] − (m[n])(xV ec[i]) − b[n])(−1) (1.14)

Note that the scalar error E[i, n] at data point (xi, yi) at sample index n is

E[i, n] = yV ec[i] − (m[n])(xV ec[i]) − b[n]so we can rewrite Eqs. (1.13) and (1.14) as

∂(MSE[n])∂(m)

= − 2

M

M−1∑i=0

E[i, n](xV ec[i]) (1.15)

Page 38: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

20 CHAPTER 1. INTRODUCTION TO LMS ADAPTIVE FILTERING

∂(MSE[n])∂(b)

= − 2

M

M−1∑i=0

E[i, n] (1.16)

The corresponding update equations are

m[n + 1] = m[n] − μ∂(MSE[n])

∂(m)(1.17)

and

b[n + 1] = b[n] − μ∂(MSE[n])

∂(b)(1.18)

The above formulas are embodied in m-code in the script (see exercises below)

LV xModelLineLMS_MBX(M, B, xV ec, Mu, bMu2mMuRat, NoI ts)

where M and B are values of slope and y-intercept, respectively, to be used to construct the y

values to be modeled from xV ec; Mu (μ) and NoI ts have the usual meaning. The value of Mu iscontrolled for stability (see exercises below). The rate of convergence of b is much slower than thatof m, and as a result the script receives the input variable bMu2mMuRat, which is used to generatea boosted-value of Mu for use in the b-variable update equation (1.18). The following call leads toslow, stable convergence:

LVxModelLineLMS_MBX(3,-1,[-10:1:10],0.5,1,150) (1.19)

On the other hand, after determining a suitable value by which to boost μ for the b-coefficientupdate term, the same thing can be accomplished in ten iterations rather than 150:

LVxModelLineLMS_MBX(3,-1,[-10:1:10],0.5,29,10)

1.9 LMS USED IN AN FIR1.9.1 TYPICAL ARRANGEMENTSIn order to visualize various aspects of the LMS algorithm as applied to an FIR, we’ll use a 2-tapFIR, which allows the graphing of both coefficient values versus MSE.

Figure 1.10 shows a basic 2-Tap adaptive FIR filter which is to model an LTI system repre-sented by the box labeled “Plant.” For continuous domain systems, the interfacing to and from theLMS adaptive filter system, is, of course, managed with ADCs and DACs, operated at a suitablesample rate with the required anti-aliasing filter. In most books, such ADCs and DACs are generallynot shown, but are understood to be a necessary part of an actual working embodiment.

Page 39: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

1.9. LMS USED IN AN FIR 21

+

+

+ X

x

E[n]

E[n]

z−1

z−1 z−1

Plant

c[1,n+1]

c[2,n]c[1,n]

c[1,n] 2μE[n]x[n+1]

c[2,n] = c[2,n−1]+2μE[n]x[n]

x[n]x[n+1]

x[n+2]

c[1] Update Flow Diagram

Figure 1.10: A flow diagram of a 2-Tap FIR configured to model a Physical Plant, which presumablyhas an impulse response of one or two samples length, i.e., a total delay equal or less than that of theadaptive FIR, which must be able to match each delay in the Plant’s impulse response to model it. ThePlant’s impulse response is thus being modeled as a 2-Tap FIR having coefficients PC1 and PC2.

In the example shown in Fig. 1.10, the Plant is itself a 2-tap FIR, with unknown tap weights.The LMS adaptive filter’s task will be to model the Plant, i.e., determine its impulse response, whichconsists of the two tap weights.

In general, modeling a particular Plant requires that the FIR have a length at least equalto the longest time delay experienced by the signal travelling through the Physical Plant. In areal-world system, of course, the number of taps involved might be dozens, hundreds, or eventhousands, depending on the particular application. A 2-Tap example, however, allows graphingof the coefficients versus MSE or Scalar Error Squared (a cost function which we will investigateshortly), which allows the reader to attain a visual impression of the manner in which the adaptivefilter works.

Referring to Fig. 1.10, the test signal enters at the left and splits, one part heading into thePlant, and the other part entering the filter’s two series-connected delay stages (labeled z−1). Thesignal at the output of each delay stage goes to a coefficient multiplier.

Page 40: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

22 CHAPTER 1. INTRODUCTION TO LMS ADAPTIVE FILTERING

1.9.2 DERIVATIONThe system of Fig. 1.10 may be represented as follows:

Err[n] = (PC1 · x[n + 1] + PC2 · x[n])...−(c1 · x[n + 1] + c2 · x[n])

where PC means Plant Coefficient and c1 and c2 are the filter tap coefficients. In the earlier examples,we used Mean Squared Error (MSE) as a cost function. This works well when MSE can be directlyobtained or computed. For many problems, however, such is not the case. For the type of systemshown in Fig. 1.10, the Plant’s impulse response is unknown, and the consequence of having onlythe Plant’s output to work with is that we do not have a true measure of MSE by which to estimatethe gradient.

A true MSE measure of coefficient misadjustment would be

T rueMSE[n] = ((PC1 − c1[n]) · x[n + 1])2 + ...

((PC2 − c2[n]) · x[n])2

This cost function has a true global or unimodal minimum since it can only be made equal tozero when both coefficients are perfectly converged to the plant values.

In accordance with our previously gained knowledge, we can write a general formula for thepartial derivative of TrueMSE with respect to the ith tap coefficient as

∂(T rueMSE[n])∂(ci[n]) = −2(PCi − ci[n]) · (x[n + N − i]) (1.20)

where PCi means the ith Plant Coefficient, ci is the ith tap coefficient, N is the total number oftaps, and n is the current sample being computed (and remember that the partial derivative of MSEfor any Plant coefficient is zero), and the partial derivative of MSE for any given coefficient cj [n]with respect to coefficient ci[n] is always zero.

The gradient might be notated as

∇(T rueMSE[n]) = −2N∑

i=1

(PCi − ci[n]) · x[n + N − i]) · ci (1.21)

Equation (1.21) requires explicit knowledge of the scalar error contributed by each pair of plantand filter coefficients. Returning to Eq. (1.20), what can readily be seen is that the partial derivativefor each coefficient is dependent on its individual misadjustment relative to the equivalent plantcoefficient, as well as the value of the signal x at the tap, which changes with n. The performancesurface for a true-MSE cost function is quadratic; for a two-tap filter, it graphs as a paraboloidof revolution, as shown in Fig. 1.11, in which MSE in decibels is graphed vertically against thetwo coefficients.

Page 41: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

1.9. LMS USED IN AN FIR 23

Figure 1.11: Performance surface for a 2-Tap Adaptive filter using a True-MSE algorithm.

The scalar error, the difference between the plant output and the filter output, is all we haveaccess to in most problems, and, for the two-tap problem at hand, we must approximate the trueMSE by squaring the scalar error:

ApproxMSE[n] = ScalarError2 =((PC1 − c1[n])x[n + 1] + (PC2 − c2[n])x[n])2 (1.22)

We can derive an estimate for the partial derivative of ApproxMSE with respect to, say, c1, (asin previous examples) as

∂(ApproxMSE[n])∂(c1[n]) = 2((PC1 − c1[n])x[n + 1]...

+ (PC2 − c2[n])x[n])(−x[n + 1]) (1.23)

This formula may be generalized for an N-Tap FIR as

∂(Err[n]2)

∂(ck[n]) = 2

[N∑

i=1

(PCi − ci[n])x[n + N − i]]

(−x[n + N − k]) (1.24)

Page 42: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

24 CHAPTER 1. INTRODUCTION TO LMS ADAPTIVE FILTERING

which may be rewritten as

∂(Err[n]2)

∂(ck[n]) = −2E[n](x[n + N − k]) (1.25)

where E[n], the scalar error at sample time n, is

E[n] =N∑

i=1

(PCi − ci[n])x[n + N − i]

Equation (1.25) may be interpreted as saying that the partial derivative of the scalar errorsquared at sample time n with respect to the kth tap coefficient at sample time n is negative twotimes the scalar error at sample time n, E[n], multiplied by the signal value at the kth tap at sampletime n.

• The coefficient update algorithm for an LMS adaptive FIR is, after inserting the update termweighting factor μ

ci[n + 1] = ci[n] + 2μ · E[n] · xi[n] (1.26)

where ci[n] represents the tap coefficient of tap index i and iteration n, μ is a scalar constant whichscales the overall magnitude of the update term, E[n] represents the scalar error at iteration n, andxi[n] is the signal value at tap i at iteration n.

• While the LMS algorithm is elegant, and moderately robust, it is not as robust as a true-MSEalgorithm in which the gradient is estimated from knowledge of the actual MSE. Equatingsquared scalar error to MSE allows the simplicity and usefulness of the LMS algorithm, butthere are limitations on the frequency content of the input signal to ensure that the LMSalgorithm properly converges. This will be explored below.

1.9.3 LIMITATION ON Mu

One idea of the maximum value that the stepsize � (2μ in Eq. (1.26), for example) may assume andremain stable is

0 < � <1

10NP(1.27)

where N is the filter length and P is the average signal power over M samples of the signal:

P = 1

M

M−1∑i=0

(x[i])2

Page 43: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

1.9. LMS USED IN AN FIR 25

Thus the quantity NP is a measure of signal power in the filter. The script (see exercisesbelow)

LV xLMSAdaptF iltMuCheck(k, NoT aps, C, SigT ype, F req, Delay)

filters either random noise of standard deviation k, or a cosine of amplitude k and frequency zero(DC), Halfband, Nyquist, or Freq, with an LMS adaptive FIR having a length equal to NoT aps,and plots the test signal in one subplot, and the filter output in a second subplot. The stepsize forcoefficient update is computed as the high limit given by Eq. (1.27), but is also multiplied by theconstant C for testing purposes. Variation of k and N has little effect on the convergence rate andstability of the algorithm when stepsize is computed using Eq. (1.27). In practice, a value of stepsizeconsiderably larger may often be used to achieve convergence in a number of iterations approximatelyequal to ten times the filter length. The parameter C can be experimentally used to observe this.Figure 1.12 shows the error signal for various values of C from 1 to 10 with k = 1 and NoT aps = 100,white noise as the test signal, and Delay = 10 (see the exercises below for a complete description ofthe input arguments). Figure 1.13 shows the same thing as Fig. 1.12 except that k = 3 and NoT aps

= 40. For the examples shown, the best value of C appears to lie between 5.0 and 7.0-this valuerange allows for both speedy convergence and good stability.

1.9.4 NLMS ALGORITHMA standard method to assist in controlling the stepsize in Eq. (1.26) is to divide by a factor relatedto the signal power in the filter, to which is added a small number ε in case the signal power in thefilter should be zero. We thus get

ci[n + 1] = ci[n] + �∑xi[n]2 + ε

· Err[n] · xi[n]

(where � = 2μ) and refer to this as the Normalized Least Mean Square, or NLMS, algorithm. Notethat in Eq. (1.27), the power is averaged over the entire input signal. In the NLMS method, onlysignal values in the filter are taken into account. The NLMS is thus more adaptive to current signalconditions. The various scripts we will develop using the LMS algorithm in an FIR will generallybe NLMS algorithms.

The script (see exercises below)

LV xLMSvNLMS(kV ec, NoT aps, Delta, Mu, SigType, F req, Delay)

passes a test signal determined by the value of SigType through two FIRs in parallel, one FIR usingthe LMS algorithm with a stepsize fixed as Mu, and the other FIR using the NLMS algorithmhaving a stepsize equal to Delta divided by the signal power in the filter (plus a very small constantto avoid potential division by zero). The test signal is given an amplitude profile defined by kV ec,which specifies a variable number of different equally spaced amplitudes imposed on the test signal.

Page 44: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

26 CHAPTER 1. INTRODUCTION TO LMS ADAPTIVE FILTERING

0 500 1000 1500

−2

0

2

Am

plitu

de

(a) Sample (Output/Error)

0 500 1000 1500

−2

0

2

Am

plitu

de

(b) Sample (Output/Error)

0 500 1000 1500−2

0

2

Am

plitu

de

(c) Sample (Output/Error)

0 500 1000 1500

−2

0

2

Am

plitu

de

(d) Sample (Output/Error)

0 500 1000 1500

−2

0

2

Am

plitu

de

(e) Sample (Output/Error)

0 500 1000 1500

−10

0

10

Am

plitu

de(f) Sample (Output/Error)

Figure 1.12: (a) Error signal for Stepsize = C/(10NP), with C = 1, k = 1 (see text); (b) Same with C =3; (c) Same with C = 5; (d) Same with C = 7; (e) Same with C = 9.5; (f ) Same with C = 10.

0 200 400 600

−5

0

5

Am

plitu

de

(a) Sample (Output/Error)

0 200 400 600

−5

0

5

Am

plitu

de

(b) Sample (Output/Error)

0 200 400 600−5

0

5

Am

plitu

de

(c) Sample (Output/Error)

0 200 400 600

−2

0

2

Am

plitu

de

(d) Sample (Output/Error)

0 200 400 600−20

0

20

Am

plitu

de

(e) Sample (Output/Error)

0 200 400 600−100

0

100

Am

plitu

de

(f) Sample (Output/Error)

Figure 1.13: (a) Error signal for Stepsize = C/(10NP), with C = 1, k = 3 (see text); (b) Same with C =3; (c) Same with C = 5; (d) Same with C = 7; (e) Same with C = 9.5; (f ) Same with C = 10.

Page 45: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

1.9. LMS USED IN AN FIR 27

In this manner, the test signal, be it noise or a cosine, for example, can undergo step changes inamplitude that test an algorithm’s ability to remain stable.

The call, in which Mu has been manually determined to allow the LMS algorithm to converge,

LVxLMSvNLMS([1,5,15,1],100,0.5,0.00005,1,[],50)

results in Fig. 1.14, while the call

LVxLMSvNLMS([1,5,25,1],100,0.5,0.00005,1,[],50)

in which the test signal amplitude has been boosted in its middle section, results in Fig. 1.15, inwhich it can be seen that the LMS algorithm has become unstable and divergent, while the NLMSalgorithm has remained stable and convergent.

Limitations of the LMS/NLMS AlgorithmsWe now proceed to study the LMS algorithm in detail for the 2-tap FIR example and observe itsbehavior with different types of input signals. Referring to Eq. (1.22), the cost function representedby this equation does not have a single minimum; rather, the performance surface (which resemblesthe lower half of a pipe split along its longitudinal axis) is linear in one direction, and quadratic ina direction perpendicular to the first direction; it has a perfectly flat bottom and defines (along thatbottom) an infinite number of pairs of coefficients (c1, c2) which will make ScalarErrorSquared = 0.

We can derive an expression for this type of performance surface by determining what valuesof c1 and c2 will drive the cost function (ApproxMSE) to any arbitrary value ESq. Then, fromEq. (1.22) we get

(PC1 − c1[n]) · x[n + 1] + (PC2 − c2[n]) · x[n] = √ESq

We proceed by solving for c2[n] in terms of c1[n]:

c2[n] = −(x[n + 1]

x[n] ) · c1[n] + ...

(PC2 + PC1 · x[n + 1]x[n] −

√ESq

x[n] ) (1.28)

Equation (1.28) tells, for any c1[n], what value of c2[n] is necessary to force the scalar errorto be

√ESq (or scalar error squared to be ESq). Equation (1.28) is a linear equation of the familiar

form y = Mx + b (or, in the iterative or adaptive sense, y[n] = M[n]x[n] + b[n] in which y[n]equates to c2[n],

M[n] = −x[n + 1]x[n] (1.29)

and

Page 46: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

28 CHAPTER 1. INTRODUCTION TO LMS ADAPTIVE FILTERING

0 500 1000 1500−40−20

02040

Am

plitu

de

(a) Sample, Test Signal

0 500 1000 1500

−10

0

10

Am

plitu

de

(b) Sample (Output/Error, LMS)0 500 1000 1500

−2

0

2

Am

plitu

de

(c) Sample (Output/Error, NLMS)

0 50 1000

0.5

1

Am

plitu

de

(d) Tap No. (LMS)0 50 100

0

0.5

1

Am

plitu

de

(e) Tap No. (NLMS)

Figure 1.14: (a) Test signal, with amplitude segments scaled by kV ec; (b) Error signal from the LMSalgorithm; (c) Error signal from the NLMS algorithm; (d) Final Tap Weights for the LMS algorithm;(e) Final Tap Weights for the NLMS algorithm.

0 500 1000 1500

−50

0

50

Am

plitu

de

(a) Sample, Test Signal

0 500 1000 1500−5

0

5x 10

4

Am

plitu

de

(b) Sample (Output/Error, LMS)0 500 1000 1500

−2

0

2

Am

plitu

de

(c) Sample (Output/Error, NLMS)

0 50 100−200

0

200

Am

plitu

de

(d) Tap No. (LMS)0 50 100

0

0.5

1

Am

plitu

de

(e) Tap No. (NLMS)

Figure 1.15: (a) Test signal, with amplitude segments scaled by kV ec; (b) Error signal from the LMSalgorithm; (c) Error signal from the NLMS algorithm; (d) Final Tap Weights for the LMS algorithm;(e) Final Tap Weights for the NLMS algorithm.

Page 47: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

1.9. LMS USED IN AN FIR 29

b[n] = (PC1 + PC0 · x[n + 1]x[n] −

√ESq

x[n] ) (1.30)

It can be seen that the slope M[n] in Eq. (1.29) is just the ratio of the two samples in thefilter. If the input signal is changing, then the slope will also change at each sample time, as will thevalue of b[n] in Eq. (1.30).

However, no matter what the slope M and intercept value b are, the line defined by theequation always passes through the point of perfect coefficient adjustment, that is, c1 = PC1 and c2

= PC2.when ESq = 0. This can be readily verified by setting c1 = PC1 and ESq = 0 in Eq. (1.28).The performance surface generated by Eq. (1.28) is linear in a first direction (lines of equal

ESq are parallel to this first direction), and quadratic in the direction perpendicular to the firstdirection. We’ll refer to this performance surface as a Linear-Quadratic Half Pipe (the surface ismore pipe-like in appearance when magnitude is graphed linearly), and to lines of equal squarederror as IsoSquErrs. Figure 1.16 shows an example of an LQHP performance surface, Scalar Errorsquared in dB is plotted against the coefficients.

Figure 1.16: Performance surface for a 2-Tap LMS Adaptive filter for a particular instantaneous pair ofsignal values in the filter; orientation of the “trough” in the performance surface changes as the ratio ofthe two signal values changes.

There are several important things to note about this:

Page 48: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

30 CHAPTER 1. INTRODUCTION TO LMS ADAPTIVE FILTERING

• The (negative of the) gradient will not, in general, be headed toward the point of best coeffi-cient adjustment, but rather it will be headed along the shortest path (a normal line, possiblycorrupted with noise) toward the bottom of a LQHP which varies orientation in the (c1, c2)

plane (slope- or angle-wise) according to the current values of the input signal in the filter.

• The very bottom of each of the these LQHPs actually passes through the point of idealcoefficient adjustment when ESQ = 0, no matter what the slope or angle is of the LQHP inthe (c1, c2) plane.

• Accordingly, the task of getting to the ideal adjustment point (i.e., when the coefficient esti-mates properly model the Plant) depends on the orientation of LQHPs changing from oneiteration to the next.

• This process may be viewed graphically by calling the script

LV _FIRLMSPSurf (PC1, PC2, Mu, c1St, c2St, ...

NoI ts, tstDType, SineF rq, NoiseAmp)

a typical call for which would be

LV_FIRLMSPSurf(1,-0.5,0.35,5,6,12,0,[],0) (1.31)

The input argument tstDType is an integer that selects the type of test sequence, which allowstesting of the algorithm with several possible data types. Use tstDType = 0, for white noise; tstDType =1, for a unit step, or DC signal; use tstDType = 2 for a sequence of alternating 1’s and -1’s, representinga sine wave at the Nyquist limit; use tstDType = 3 for a test signal at the half-band frequency ([1,0,-1,0,1,0,-1,0 ...]), and finally, a value of 4 yields a pure (noiseless) sine wave, the frequency of whichmay optionally be specified by entering another number after tstDType–if in this case SineFrq ispassed as the empty matrix, the sine’s frequency defaults to 25 cycles/1024 samples. If you aren’tcalling for a sine wave as the test signal, you may pass SineFreq as the empty matrix, [ ], as shown inthe sample call above.

A coefficient track which resulted from the call at (1.31), which specifies the test signal asrandom noise, is shown in Fig. 1.17.

Page 49: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

1.9. LMS USED IN AN FIR 31

−4 −2 0 2 4 6−8

−6

−4

−2

0

2

4

6

c1; Iteration = 12

c2

Figure 1.17: Coefficient track for a 2-Tap LMS Adaptive filter using random noise as the test signal.Minimum of cost function is a line (the center-most dotted line in the plot) rather than a point. Arandom noise test signal causes the orientation of the performance surface to change randomly, resultingin convergence to the correct coefficient values, which are 1, -0.5.

The script

LV _FIRLMS6Panel(PC1, PC2, Mu, c1St, c2St, ...

NoI ts, tstDType, SineF rq, NoiseAmp)

is similar to the script LV_FIRLMSPSurf, except that it plots the most recent six iterations in onefigure.

Example 1.8. Compute and plot the performance surface (several IsoSqErrs will do) and coefficienttrack for a number of iterations of an adaptive LMS 2-tap FIR, with white noise as the signal.

Figure 1.18, which was produced by the script call

LV_FIRLMS6Panel(-2,3,0.25,0,0,6,0,[ ],0) (1.32)

Page 50: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

32 CHAPTER 1. INTRODUCTION TO LMS ADAPTIVE FILTERING

allows one to see the sequence of events in which the performance surface constantly shifts with eachsample, but the coefficient values are inexorably drawn toward proper adjustment–provided certainconditions are met. We’ll see what those conditions are shortly.

In Fig. 1.18, noise was used as the test signal. At Iteration 1, the coefficient pair (c1, c2) movedfrom its initial location at (0,0) to a new location, and the direction of movement was generally towardIsoSquErr(0), i.e., the center-most solid line in each of the six graphs, where ScalarError2 = 0. Ateach successive iteration, the orientation of IsoSquErr(0) shifts, drawing the coefficients toward it.This process is very efficient at drawing the coefficients toward the point of ideal adjustment orconvergence when the input signal is noisy.

Unfortunately, any input signal that does not change enough can lead to stagnation andconvergence to an incorrect solution.

Example 1.9. Compute and plot the performance surface and successive coefficient estimates fora 2-tap FIR using a DC signal.

Figure 1.19 shows what happens if a DC signal (unit step) is applied to a 2-Tap LMSadaptive FIR filter. You should note that the coefficients do converge to a certain point (i.e., a certaincoefficient pair (c1, c2)), and the Scalar Error squared is in fact close to zero. Unfortunately, theconverged coefficient values are incorrect. A similar thing may happen with other types of highlyregular signals, such as a pure (noise-free) sine wave.

Example 1.10. Compute and plot the performance surface and successive coefficient estimates fora 2-tap FIR filtering a sinusoidal signal of frequency 25.

The script call

LV_FIRLMS6Panel(-1,3,0.25,5,3,6,4,25,0)

in which the seventh and eighth input arguments (4, 25) specify a test signal consisting of a puresine wave of frequency 25 cycles (over 1024 samples), results in the plots shown in Fig. 1.20. Theresult shown in Fig. 1.20 again shows convergence to an incorrect solution, with, nonetheless, thesquare of Scalar Error minimized. Logically, a sine wave does change over time, so it is necessary toperform enough iterations so that several cycles of the sine wave are traversed. In general, though,convergence is protracted and uncertain.

Example 1.11. Compute and plot the error signal and coefficient estimates for a 2-tap FIR thatmodels a Plant defined by Plant Coefficients PC1 and PC2, as shown in Fig. 1.10, using a test signalconsisting of a sine wave containing, respectively, small, medium, and large amounts of white noise.

The computations may be performed using the script (see exercises below)

Page 51: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

1.9. LMS USED IN AN FIR 33

−4 −3 −2 −1 00

2

4

6

c1; Iteration = 1

c2

−4 −3 −2 −1 00

2

4

6

c1; Iteration = 3

c2

−4 −3 −2 −1 00

2

4

6

c1; Iteration = 5

c2

−4 −3 −2 −1 00

2

4

6

c1; Iteration = 2

c2

−4 −3 −2 −1 00

2

4

6

c1; Iteration = 4

c2

−4 −3 −2 −1 00

2

4

6

c1; Iteration = 6

c2

Figure 1.18: Coefficient track in six successive panels/iterations for the two coefficients of an LMSAdaptive filter using random noise as a test signal.

−4 −3 −2 −1 00

2

4

6

c1; Iteration = 1

c2

−4 −3 −2 −1 00

2

4

6

c1; Iteration = 3

c2

−4 −3 −2 −1 00

2

4

6

c1; Iteration = 5

c2

−4 −3 −2 −1 00

2

4

6

c1; Iteration = 2

c2

−4 −3 −2 −1 00

2

4

6

c1; Iteration = 4

c2

−4 −3 −2 −1 00

2

4

6

c1; Iteration = 6

c2

Figure 1.19: Coefficient track in six successive panels/iterations for the two coefficients of an LMSAdaptive filter using a Unit Step (DC) as a test signal.

Page 52: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

34 CHAPTER 1. INTRODUCTION TO LMS ADAPTIVE FILTERING

−5 0 50

2

4

6

c1; Iteration = 1

c2

−5 0 50

2

4

6

c1; Iteration = 3

c2

−5 0 50

2

4

6

c1; Iteration = 5

c2

−5 0 50

2

4

6

c1; Iteration = 2

c2

−5 0 50

2

4

6

c1; Iteration = 4

c2

−5 0 50

2

4

6

c1; Iteration = 6

c2

Figure 1.20: Coefficient track in six successive panels/iterations for the two coefficients of an LMSAdaptive filter using a 25 Hz sine wave as a test signal.

LV xLMS2T apFIR(PC1, PC2, NoI ts, tstDType, CosFrq, T woMu, NoiseAmp)

with the specific calls

LVxLMS2TapFIR(-3.5,1.5,32,5,0.5,0.5,0)

LVxLMS2TapFIR(-3.5,1.5,32,5,0.5,0.5,0.2)

LVxLMS2TapFIR(-3.5,1.5,32,5,0.5,0.5,4)

which yield, respectively, Fig. 1.21, Fig. 1.22, and Fig. 1.23. Note that only the third call, (whichuses as a test signal a sine wave to which has been added a large amount of noise) results in rapidconvergence to the correct coefficients.

1.10 CONTRAST-TRUE MSEWe have seen that the LMS algorithm, as applied to a typical system, uses the square of scalarerror as an estimate of the true MSE in order to estimate the gradient. As a result, the actual

Page 53: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

1.10. CONTRAST-TRUE MSE 35

0 10 20 30

−1

−0.5

0

0.5

1

Am

plitu

de

(a) Input Sample0 10 20 30

−2

−1

0

1

2

(b) Sample/Iteration

Err

or

0 10 20 30

−2

0

2

(c) Iteration (PC1 = −3.5)

Am

plitu

de, c

1

0 10 20 30

−1

−0.5

0

0.5

1

(d) Iteration (PC2 = 1.5)

Am

plitu

de, c

2

Figure 1.21: (a) Input signal, a sine wave containing a small amount of random noise; (b) Output or errorsignal; (c) Estimate of 1st coefficient versus iteration; (d) Estimate of 2nd coefficient versus iteration.

0 10 20 30

−1

0

1

Am

plitu

de

(a) Input Sample0 10 20 30

−2

0

2

(b) Sample/Iteration

Err

or

0 10 20 30

−2

0

2

(c) Iteration (PC1 = −3.5)

Am

plitu

de, c

1

0 10 20 30

−1

−0.5

0

0.5

1

(d) Iteration (PC2 = 1.5)

Am

plitu

de, c

2

Figure 1.22: (a) Input signal, a sine wave containing a medium amount of random noise; (b) Output orerror signal; (c) Estimate of 1st coefficient versus iteration; (d) Estimate of 2nd coefficient versus iteration.

Page 54: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

36 CHAPTER 1. INTRODUCTION TO LMS ADAPTIVE FILTERING

0 10 20 30−10

−5

0

5

10

Am

plitu

de

(a) Input Sample0 10 20 30

−20

−10

0

10

20

(b) Sample/Iteration

Err

or

0 10 20 30−4

−2

0

2

4

(c) Iteration (PC1 = −3.5)

Am

plitu

de, c

1

0 10 20 30

−1

0

1

(d) Iteration (PC2 = 1.5)

Am

plitu

de, c

2

Figure 1.23: (a) Input signal, a sine wave containing a large amount of random noise; (b) Output or errorsignal; (c) Estimate of 1st coefficient versus iteration; (d) Estimate of 2nd coefficient versus iteration.

performance surface is not unimodal (i.e., does not possess a single minimum at the point of optimumcoefficient adjustment). However, under proper conditions, the LMS algorithm behaves as thougha true unimodal performance surface were being traversed; it generally performs poorly with inputsignals having narrow bandwidth spectra, such as DC, low frequency sine waves, etc.

When we actually know the plant coefficients to which the filter is trying to converge, we canconstruct an LMS-like algorithm that uses this information to give a much better gradient estimate.The result is that the algorithm works well with all signals, including sinusoids, DC, Nyquist, etc.The purpose of using an adaptive filter, of course, is usually to determine the plant coefficients, sothe situation hypothesized here is mainly academic. However, it serves to illustrate the differencebetween a true-MSE cost function and one based on the square of scalar error.

The script

LV _FIRPSurf T rueMSE(PC1, PC2, Mu, c1Strt, ...

c2Strt, NoI ts, tstDType, CosFrq)

embodies such an algorithm, where PC1 and PC2 are the Plant coefficients to be modeled, c1Strt

and c2Strt are the initial guesses or starting values for the coefficient estimates,NoI ts is the numberof iterations to perform, Mu has the usual meaning, and tstDType selects the type of test signal. Pass

Page 55: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

1.11. LMS ADAPTIVE FIR SUMMARY 37

tstDType as 0 for white noise, 1 for a unit step (DC), 2 for the Nyquist Limit frequency, 3 for thehalf-band frequency, and 4 for a cosine of frequency CosFrq (may be passed as [] if a cosine is notbeing used as a test signal).

Example 1.12. Compute and plot the performance surface for a 2-tap FIR using a true-MSE-basedcoefficient update algorithm; use a sinusoidal input.

Figure 1.24 shows the results after making the call

LV_FIRPSurfTrueMSE(-1,3,1,5,6,7,4,125)

Even though the test signal employed to generate the coefficient track in Fig. 1.24 was a sinewave, convergence is proper.

Example 1.13. Compute and plot the performance surface for a 2-tap FIR using a true-MSE-basedcoefficient update algorithm; use a DC input.

The script call

LV_FIRPSurfTrueMSE(-1,3,1,5,6,5,1,[])

uses a DC (unit step) signal as the test signal, and results in a coefficient track which forms a straightline from the entry point to the point of ideal adjustment, as shown in Fig. 1.25.

1.11 LMS ADAPTIVE FIR SUMMARYIn general, a pure frequency (a sinusoid) poses difficulties for an LMS algorithm, which uses thesquare of the scalar error as the cost function rather than the true MSE. However, depending on exactconditions, it is possible to achieve proper convergence. This is more likely with a higher frequencythan with a lower frequency–the key word is change. The more rapid the change in test signalvalues, especially change in sign, the better. Mixtures of sinusoids, to the extent more rapid sign andamplitude change result, improve performance. Recall from some of the examples, however, that anLMS adaptive filter can converge with a narrow bandwidth input (and in fact minimize the scalarerror), just as with other, wider-bandwidth signals. The difference is that with narrow bandwidthsignals, the filter coefficients do not necessarily model the Physical Plant properly, since there isno unique set of coefficients that will minimize the error (rather, there are many possible sets ofcoefficients that can minimize the scalar error).

Hence it is, in general, good to ensure that LMS adaptive filters receive signals for processingwhich are at least somewhat noisy in nature. In many applications, such as noise cancellation, theinput signal actually is more or less pure noise. In some applications, such as, for example, feedbackcancellation in a hearing aid, it may be necessary to inject noise into the signal to cause the LMSadaptive algorithm to converge. We’ll see an application in the next chapter, using a special filter

Page 56: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

38 CHAPTER 1. INTRODUCTION TO LMS ADAPTIVE FILTERING

c1; Test Sig = Cosine, Freq = 125; Iter = 7

c2

−10 −8 −6 −4 −2 0 2 4 6 8

−5

0

5

10

15

Figure 1.24: Plot of c1 and c2 for seven iterations of a True-MSE-based gradient algorithm, using as atest signal a cosine of 125 cycles over 1024 samples, i.e., a normalized frequency of a 125/512 = 0.24π .Plant coefficients (-1,3) lie at the center of the plot.

c2

−10 −8 −6 −4 −2 0 2 4 6 8

−5

0

5

10

15

c1; Test Sig = Unit Step (DC); Iter = 5

Figure 1.25: Plot of c1 and c2 for 5 iterations of a True-MSE-based gradient algorithm, using as a testsignal a 1024 sample unit step. Plant coefficients (-1,3) lie at the center of the plot.

Page 57: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

1.12. REFERENCES 39

topology, however, in which sinusoidal signals, but not random signals, can be cancelled using anLMS adaptive filter.

We have looked extensively at the inner workings and hidden mechanisms of the LMS al-gorithm as applied to a simple 2-point FIR. Limiting the number of coefficients to two enabledgraphing of the performance surfaces in a three-dimensional space. Adaptive FIRs are employedusing large numbers of taps, sometimes in the thousands. It is not possible to visualize the perfor-mance surfaces for numbers of taps beyond two, but nonetheless, the LMS algorithm behaves verysimilarly–that is to say, as a stochastic gradient search algorithm, in which the gradient estimate isnoisy. With optimum conditions, typical filters converge in a number of iterations equal to about 10times the filter length.

The weaknesses uncovered in the 2-Tap FIR are also weaknesses of the general N-Tap LMSadaptive FIR; that is, unreliability with input signals that lack randomness or are of narrow band-width. Pure sinusoids, and small combinations of the same can be problematic, and can lead toconvergence to incorrect coefficient values.

Reference [1] is a very accessible basic adaptive filtering text; while Reference [2] gives anexhaustive rundown of the LMS adaptive filter and numerous species thereof, along with extensivetheoretical discussions, illustrations, and computer exercises.

1.12 REFERENCES[1] Bernard Widrow and Samuel D. Stearns, Adaptive Signal Processing, Prentice-Hall, EnglewoodCliffs, New Jersey, 1985.

[2] Ali H. Sayed, Fundamentals of Adaptive Filtering, John Wiley & Sons, Hoboken, New Jersey,2003.

[3] Simon Haykin, Adaptive Filter Theory, Third Edition, Prentice-Hall, Upper Saddle River, NewJersey, 1996.

1.13 EXERCISES1. Write the m-code for the script

LVxGradientViaCP(FirstValX,StepSize,...NoiseAmp,UseWgtingFcn, deltaX, NoIts)

which is extensively described and illustrated in the text, and test it with the calls below. For eachinstance, characterize the performance of the algorithm with respect to speed of convergence andstability. Conditions of stability might be, for example: stable, convergent; stable, but nonconvergent,unstable or divergent. For convergent situations, note where there is overshoot (an “underdamped”)condition, very slow convergence (an “overdamped” condition) or immediate convergence (a “criti-cally damped” condition).

Page 58: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

40 CHAPTER 1. INTRODUCTION TO LMS ADAPTIVE FILTERING

(a) LVxGradientViaCP(4,0.05,0.01,0,0.0001,50)(b) LVxGradientViaCP(4,0.03,0.01,0,0.0001,75)(c) LVxGradientViaCP(4,0.9,0.01,0,0.0001,50)(d) LVxGradientViaCP(4,1,0.01,0,0.0001,50)(e) LVxGradientViaCP(4,0.07,0.01,1,0.0001,50)(f ) LVxGradientViaCP(4,0.03,0.01,1,0.0001,20)(g) LVxGradientViaCP(400,0.03,0.01,1,0.0001,75)

2. Using the relationships

CF [n] = x2 + 1

x[n + 1] = x[n] − �C

�C = μ (∂(CF)/∂(x[n]))CF [n]with μ = 0.03, write an expression that is a polynomial in x that can be used to solve for an initialvalue of x (denoted as x[1]) that will result in x[2] being equal or very nearly so to the value of x

corresponding to the minimum of the performance curve. There should be three values of x, one ofwhich is 0.0. Test your nonzero values of x with the call

LVxGradientViaCP(x,0.03,0,1,delX,6)

where delX is given the following values:(a) 0.001;(b) 0.00001;(c) 0.00000001(d) 0.0000000001

3. Write the m-code for the script

LVxModelPtswLine(TestM,TestYInt,xTest,...

Mu,yMu2MuRatio,MStart,yIntStart,NoIts)

as described and illustrated in the text, and test it with the given test calls:

function LVxModelPtswLine(TestM,TestYInt,xTest,Mu,...yMu2MuRatio,MStart,yIntStart,NoIts)% The line to be modeled is represented in the point-slope form,% and thus the test line creation parameters are TestM (the slope),% TestYInt (the test Y-intercept) and the vector xTest.% Mu is the usual weight for the gradient estimate update

Page 59: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

1.13. EXERCISES 41

% term; yMu2MuRatio is a ratio by which to relatively% weight the partial derivative of y-intercept relative to% slope; MStart is the initial estimate of slope to use;% yIntStart is the initial estimate of y-intercept to use,% and NoIts is the number of iterations to perform.% The script creates separate 3-D subplots on one figure% of the performance surface and the coefficient track,% both plots having the same axis limits enabling% easy comparison.% Test calls:% (a) LVxModelPtswLine(2,0,[-10:1:10],0.014,37,-10,8,12)% (b) LVxModelPtswLine(2,0,[-10:1:10],0.005,1,-10,8,40)% (c) LVxModelPtswLine(2,0,[-10:1:10],0.025,1,-10,8,60)% (d) LVxModelPtswLine(2,0,[-10:1:10],0.025,40,-10,8,30)% (e) LVxModelPtswLine(20,-90,[-10:1:10],0.005,10,-10,8,40)% (f ) LVxModelPtswLine(2,0,[0:1:10],0.01,50,-10,8,80);% (g) LVxModelPtswLine(1,-2,[0:1:10],0.01,50,-10,8,80);

The results from using LabVIEW for call (c) above, with separate plots for the performancesurface and the coefficient track, are shown in Fig. 1.26.

The result from using MATLAB, with the same arguments as in call (c) above, but with thescript written to use a single plot having the coefficient track plotted on top of the performancesurface, is shown in Fig. 1.27.

4. Write the m-code for a script

LV xModelLineLMS_MBX(M, B, xV ec, Mu, bMu2mMuRat, NoI ts)

in which the algorithm update equations are Eqs. (1.17)–(1.18) in the text. The input arguments M

and B are to be used to generate a set of y-values with corresponding to the x-values xV ec. Theinput argument bMu2mMuRat is a number to boost the value of Mu for updating the estimate ofb.

Include code within the iterative loop to control Mu to maintain stability, including code toprovide separate values of Mu for updating the estimates of m and b. The following code will returnthe algorithm to a stable condition if Mu is chosen too high, rolling back the coefficients to theprevious values and halving Mu if MSE increases.

if Ctr>1if MSE(Ctr)>1.02*MSE(Ctr-1)m(Ctr+1) = m(Ctr); b(Ctr+1) = b(Ctr);Mu = Mu/2

end; end

Page 60: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

42 CHAPTER 1. INTRODUCTION TO LMS ADAPTIVE FILTERING

−10

0

10

−20

0

20

0

5000

10000

15000

Slopey−Intercept

MS

E

−10

0

10

−20

0

20

0

5000

10000

15000

Slopey−Intercept

MS

E

Figure 1.26: (a) Performance Surface; (b) Separate plot of coefficient track along the PerformanceSurface in (a).

−10

0

10

−20−10

010

20

0

5000

Slope

y−Intercept

MS

E

Figure 1.27: Same as the previous figure, except that the coefficient track is plotted onto a plot of theperformance surface.

Page 61: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

1.13. EXERCISES 43

The script should, at the end of each call, create a figure with two subplots, the first of whichplots xV ec and the corresponding y-values, and a line created using the final estimates of m and b.The second subplot should show MSE v. Iteration.

Test your code with the following calls:

(a) LVxModelLineLMS_MBX(3,-1,[0 1 2 3 ],1.2, 3,30)(b) LVxModelLineLMS_MBX(3,-1,[-10:1:10],0.5, 30,5)(c) LVxModelLineLMS_MBX(3,-1,[0:0.1:2],1.15,3,45)(d) LVxModelLineLMS_MBX(3,-1,[0:1:2],1.1,2,30)

5. Write the m-code to implement the following script, which creates a two-sample Plant charac-terized by Plant Coefficients PC1 and PC2 (as shown in Fig. 1.10) and attempts to model it usinga 2-tap LMS adaptive FIR, in accordance with the function specification below. It should create afigure having four subplots which show the test signal, the error signal, and the successive estimatesof the plant coefficients, which are given the variable names c1 and c2. A sample plot, which wascreated using the call

LVxLMS2TapFIR(4,-3,64,3,[],0.5,0)

is shown in Fig. 1.28.function LVxLMS2TapFIR(PC1,PC2, NoIts,tstDType,CosFrq,...TwoMu,NoiseAmp)% PC1 and PC2 are Plant Coefficients, to be modeled by the 2-tap% FIR; NoIts is the test sequence length in samples% tstDType selects test signals to pass through the Plant and% the filter, as follows: 1 = Random Noise; 2 = Unit Step;% 3 = Nyquist Limit (Fs/2); 4 = Half-Band (Fs/4)% 5 = cosine of freq CosFrq; 6 = Cos(1Hz)+Cos(Fs/4)+...% Cos(Fs/2); TwoMu is the update term weight% Random noise having amplitude NoiseAmp is added to% the test signal selected by tstDType.% Test calls:% LVxLMS2TapFIR(4,-3,64,1,[],0.5,0)% LVxLMS2TapFIR(4,-3,64,2,[],0.5,0)% LVxLMS2TapFIR(4,-3,64,3,[],0.5,0)% LVxLMS2TapFIR(4,-3,64,4,[],0.5,0)% LVxLMS2TapFIR(4,-3,64,5,[25],0.5,0)% LVxLMS2TapFIR(4,-3,64,5,[1],0.5,0)% LVxLMS2TapFIR(4,-3,64,6,[],0.5,0)% LVxLMS2TapFIR(4,-3,64,0,[],1.15,0)% LVxLMS2TapFIR(4,-3,7,4,[],1.15,0)

6. Write the m-code for the script LVxLMSAdaptFiltMuCheck as specified below:

Page 62: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

44 CHAPTER 1. INTRODUCTION TO LMS ADAPTIVE FILTERING

0 20 40 60

−1

−0.5

0

0.5

1

Am

plitu

de

(a) Sample, Test Signal0 20 40 60

−5

0

5

(b) Sample/Iteration

Err

or0 20 40 60

−4

−2

0

2

4

(c) Iteration (PC1 = 4)

Am

plitu

de, c

1

0 20 40 60−4

−2

0

2

4

(d) Iteration (PC2 = −3)A

mpl

itude

, c2

Figure 1.28: (a) Test signal supplied to a two-sample Plant and a 2-Tap LMS Adaptive FIR; (b) ErrorSignal; (c) Successive estimate plot of first tap coefficient, c1; (d) Successive estimate plot of second tapcoefficient, c2. Note that the test signal is narrow band, and in fact, though the error signal converges tozero, the converged values of c1 and c2 are incorrect, as expected.

function LVxLMSAdaptFiltMuCheck(k,NoTaps,C,SigType,...Freq,Delay)% k is the standard deviation of white noise or amplitude of% a cosine to use as the test signal% NoTaps is the number of taps to use in the LMS adaptive filter;% The value for Stepsize is computed as C/(10*P*NoTaps) where% P = sum(testSig.ˆ2)/length(testSig);% SigType 0 gives a DC test signal (k*cos(0))% SigType 1 = random noise, 2 = cosine of frequency Freq% SigType 3 = Nyquist rate, and 4 = Half-Band Frequency% The current filter output sample is subtracted from the current% test signal sample, delayed by Delay samples. Delay must be% between 1 and NoTaps for the error signal to be able to go% to zero. Two plots are created, one of the test signal, and% one of the LMS adaptive FIR error/output signal% A typical call:

Page 63: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

1.13. EXERCISES 45

% LVxLMSAdaptFiltMuCheck(1,100,1,1,[15],50)% LVxLMSAdaptFiltMuCheck(1,100,4,1,[15],50)% LVxLMSAdaptFiltMuCheck(1,100,6,1,[15],50)% LVxLMSAdaptFiltMuCheck(1,100,8,1,[15],50)% LVxLMSAdaptFiltMuCheck(1,100,9.5,1,[15],50)% LVxLMSAdaptFiltMuCheck(1,100,10,1,[15],50)

7. Write the m-code for the script

LV xLMSvNLMS(kV ec, NoT aps, Delta, Mu, SigType, F req, Delay)

according to the following function specification, and test it with the given calls.

function LVxLMSvNLMS(kVec,NoTaps,Delta,Mu,...SigType,Freq,Delay)% kVec is a vector of amplitudes to impose on the test signal as% a succession of equally spaced amplitude segments over a% test signal of 15 times the filter length.% NoTaps is the number of taps to use in the LMS adaptive filter;% The value for Stepsize is computed as Delta/(P + sm) where% P = sum(testSig.ˆ2) for the samples of testSig in the filter,% and this is used in the NLMS algorithm (sm is a small% number such as 10ˆ(-6). Simultaneously, the test signal is% processed by an LMS filter of the same length using a% constant stepsize equal to Mu.% SigType 0 gives a DC test signal,% SigType 1 = random noise, 2 = cosine of frequency Freq% SigType 3 = Nyquist rate, and 4 = Half-Band Frequency% The current filter output sample is subtracted from the current% test signal sample, delayed by Delay samples. Delay must% be between 1 and NoTaps for the error signal to be able to% go to zero. Five subplots are created, one of the test signal,% one of the LMS adaptive FIR error/output signal, one of the% LMS final tap weights, one of the NLMS error signal, and% one of the final NLMS tap weights.% Test calls:% LVxLMSvNLMS([1,5,15,1],100,0.5,0.00005,1,[15],50)% LVxLMSvNLMS([1,5,25,1],100,0.5,0.00005,1,[15],50)

8. Consider the problem of modeling a set of test points P ti = (xi, yi) with the polynomial

y = a0x0 + a1x

1 + a2x2 + ... + amxm (1.33)

Page 64: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

46 CHAPTER 1. INTRODUCTION TO LMS ADAPTIVE FILTERING

Since Eq. (1.33) will undergo an iterative process, we designate y and the various coefficientsas functions of sample or iteration index n.

y[n] = a0[n]x0 + a1[n]x1 + a2[n]x2 + ... + am[n]xm (1.34)

Equation (1.34) states that the value of y at sample time n, for a certain input x, is equal tothe sum of various powers of x, each weighted with a coefficient which is in general a function ofiteration n (we seek, of course, to cause all the coefficients, over a number of iterations, to convergeto their ideal values, ones which yield the curve which best fits the data points (xi, yi)).

We write an expression for total MSE at iteration n, MSE[n], which is the sum of the MSEat each data point:

MSE[n] = 1

N

N∑i=1

(yi − (

m∑j=0

aj [n]xji ))2 (1.35)

Note that aj [n] and MSE[n] are continuous functions (evaluated at different discrete timesor iterations n), and thus may be differentiated. Equation (1.35) may be interpreted as saying thatMSE at sample time n is computed by summing, for each test point (xi , yi), the square of thedifference between the actual test point value yi and the predicted value of yi , which is obtained byevaluating the polynomial expression

m∑j=0

aj [n]xji

in which xi is raised in turn to various powers j (0, 1, 2...m), each of which is weighted with thecurrent estimate of the coefficient for that particular power at sample time n, aj [n].

Using calculus, derive an analytic expression for the partial derivative of MSE with respectto any coefficient ak and then write an analytic expression for the coefficient update equation toiteratively generate the best estimates of ak using a true-MSE-based gradient search algorithm, andthen test your equations by embodying them in m-code as specified by the function below. Note theadditional guidelines given following the function description.

function LVxFitCurveViaLMS(cTest,xTest,NoiseAmp,Mu0,...UseWts,NoIts)% cTest is a row vector of coefficients for a polynomial in% xTest, with the first element in cTest being the coefficient% for xTest to the zeroth power, the second element of cTest% is the coefficient for xTest to the first power, and so forth;% NoiseAmp is the amplitude of noise to add to the% points((xTest[n],yTest[n]) which are generated by the% polynomial expression using the coefficients of cTest;% Mu0 is the initial value of Mu to use in the LMS algorithm;

Page 65: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

1.13. EXERCISES 47

% UseWts if passed as 1 applies weights to the coefficient% update equations to speed convergence of the lower power% coefficients; pass as 0 to not apply any such weights% NoIts is the maximum number of iterations to perform.% The script computes the MSE at each iteration, and if it% increases, the coefficient estimates are rolled back and the% value of Mu is halved. The brings Mu to an acceptable% value if it is chosen too large. In order to standardize the% values of MuO, the initial value of Mu0 is divided by the% effective power of xTest “in the filter.” Additionally, the% update equation for each coefficient is effectively weighted% by a factor to cause all coefficients to converge at roughly% the same rate. The factor is computed as the xTest power% for the highest power being modeled divided by the% xTest power for the power whose coefficient is being updated.%% Typical calls might be:%% LVxFitCurveViaLMS([1, 1, 2],[-5:1:5],0,0.5,1,33)% LVxFitCurveViaLMS([1, 1, 2],[-5:1:5],0,0.5,0,950)% LVxFitCurveViaLMS([0, 1, 0, 2],[-5:1:5],0,0.5,1,85)% LVxFitCurveViaLMS([1, 1, 2],[-1:0.025:1],0,2.75,1,18)% LVxFitCurveViaLMS([1, 1, -2],[-1:0.025:1],0,4,1,26)% LVxFitCurveViaLMS([1, 1, -2],[-1:0.025:1],0,1,0,50)

The following code can be used to normalize the stepsize to the signal power of xT est (asdescribed in the function specification above) and to generate the relative xT est signal powers toweight the coefficient update equation to accelerate the convergence of the lower powers, whichgenerally converge much more slowly than the highest power (why?). The computations performedby the code are analogous to computing the signal power in the filter as used in the NLMS algorithm.Here, of course, the signal is the constant vector xT est .

PwrxVec = 0;for PwrCtr = 0:1:HighestPowerPartDerPwr(1,PwrCtr+1) = sum(xTest.ˆ(2*PwrCtr));PwrxVec = PwrxVec + PartDerPwr(1,PwrCtr+1);

endPwrxVec = sum(PwrxVec);Mu(1) = Mu0/PwrxVec; % normalize Mu[1]% Compute relative signal power for each polynomial coeffn = 1:1:HighestPower + 1;

Page 66: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

48 CHAPTER 1. INTRODUCTION TO LMS ADAPTIVE FILTERING

if UseWts==1MuPwrWts(1,n) = ...PartDerPwr(1,length(PartDerPwr))./PartDerPwr;else;MuPwrWts(1,n) = 1;end

A useful thing in the algorithm is to update Mu each iteration, thus making it a function ofdiscrete time, Mu[n]. This can quickly return the algorithm to stability if too large a value of Mu0 ischosen. To update Mu[n], compute MSE at each iteration, and if MSE has increased at the currentsample, reset (rollback) the coefficient estimates to the previous values, and divide Mu[n] by 2. IfMSE is the same or less than for the previous iteration, maintain Mu[n] at the existing value, i.e.,let Mu[N + 1] = Mu[n]. Note that the stepsize for the coefficient update equation, not includingthe term MuPwrWts, is 2*Mu[n] to work with the values of Mu0 in the test calls given above inthe function specification.

Page 67: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

49

C H A P T E R 2

Applied Adaptive Filtering2.1 OVERVIEW

In the previous chapter, we explored a number of basic principles of adaptive processes and devel-oped the LMS algorithm for an FIR arranged in a simple system-modeling problem. In this, we’llinvestigate several applications of the LMS-based adaptive FIR, including:

1. Active Noise Cancellation2. System Modeling3. Echo Cancellation4. Periodic Signal Removal/Prediction/Adaptive Line Enhancement5. Interference Cancellation6. Equalization/Inverse Filtering/Deconvolution7. Dereverberation

We have briefly seen, in the previous chapter, the filter topology (or arrangement) for systemmodeling,which is the same as that for Active Noise Cancellation (ANC).The converged coefficientsof the adaptive filter that model the Plant in a system-modeling problem necessarily produce a filteroutput the same as that of the Plant. By inverting this signal and feeding it to a loudspeaker in a duct,noise in a duct can be cancelled. Another common use of active noise cancellation is in headphonesbeing used in a noisy environment. Such headphones can be configured to cancel environmentalnoise entering the ear canal while permitting desired signals from the headphone to be heard.

With Echo Cancellation, useful in duplex communications systems (such as telephony), forexample, we introduce a new topology, which involves two intermixed signals, one of which must beremoved from the other. With this topology, we introduce the very useful Dual-H method, whichgreatly aids convergence in such systems.

Periodic Signal Removal and Adaptive Line Enhancement (both accomplished with the samefilter topology) find applications in communications systems–a sinusoidal interference signal thatpersists for a significant period of time can be eliminated from a signal that consists of shorterduration components. This is accomplished by inserting a bulk delay (called a decorrelating delay)between the input signal and the adaptive filter so that the filter output and the signal can onlycorrelate for persistent periodic signals. The LMS filter output itself enhances the periodic signal,thus serving as an Adaptive Line Enhancer (ALE). By subtracting the enhanced filter output fromthe signal (thus creating the error signal), the periodic component of the signal is eliminated.

With Interference Cancellation, we introduce an important concept that can be used inmany different environments. An Interference Cancellation arrangement uses two inputs, a signalreference input and a noise reference input. This arrangement, while discussed in this chapter in

Page 68: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

50 CHAPTER 2. APPLIED ADAPTIVE FILTERING

the context of audio, also applies to any other kind of signal, such as a received RF signal sufferinginterference. In that case, a signal antenna and a noise antenna serve as the inputs. As the speed andeconomic availability of higher speed digital sampling and processing hardware increase, applicationsof digital signal processing at ever-higher RF frequencies will develop. There are nonetheless manyapplications in the audio spectrum, such as cancelling vehicular noise entering a communicationsmicrophone along with the operator’s speech, helping to increase the effectiveness and reliability ofaudio communication.

Another topic finding applicability in communications systems is inverse filtering, which canbe used to perform several equivalent functions, such as equalization or deconvolution. Equalizationin communications systems, for example, becomes more necessary as systems become more digital,and maintenance of magnitude and phase responses in communications channels becomes criticalto accurate data transmission. A common problem in television reception or FM radio reception isghosting (in the TV environment) or multipath interference (in the FM environment). These aremanifestations of the same thing, multiple copies of the signal arriving at the receiving antenna andcausing an echo-like response. Since the echos are essentially attenuated duplicates of the signal, itis possible to determine the echo delay times and magnitudes and remove them. To illustrate thegeneral idea, we conclude the chapter with a problem in which we generate a reverberative audiosignal (an audio signal mixed with delayed, attenuated copies of itself ) using an IIR, and estimatethe parameters of the reverberative signal using adaptive filtering and autocorrelation. From theseparameters we are able to deconvolve the audio signal, returning it to its original state prior toentering becoming reverberative.

2.2 ACTIVE NOISE CANCELLATIONThe script

LV xLMSANCNorm(P lantCoeff V ec, k, Mu, f req, DV Mult)

(see exercises below) demonstrates the principle of cancelling noise in a duct or similar Plant, suchas that depicted in Fig. 2.1. This kind of simple arrangement can work when the sound travellingdown the duct propagates in the simplest mode, with pressure constant across the cross-section ofthe duct, only varying along the length of the duct. This is accomplished by making sure the duct’scross-sectional dimensions are small enough compared to the longest sound wavelength present toprevent higher propagation modes.

For our simulation of noise cancellation in a duct, we’ll use an adaptive LMS FIR filter having10 taps, with the Plant (duct) Coefficients specified in a vector of 10 values as PlantCoeffVector. Theinput argument k specifies an amount of noise to mix with two sinusoids having respective frequenciesdetermined as harmonics of freq; Mu is the usual scaling factor, and DVMult is a vector of amplitudesby which to scale the test signal, the purpose of which is discussed below.

Example 2.1. Simulate active noise cancellation in a duct using the script LVxLMSANCNorm.

Page 69: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

2.3. SYSTEM MODELING 51

ADC

DuctMIC1

MIC2

Output NoiseInput Noise

ADC FIR DAC +AMP

LMS CoefficientUpdate Algorithm

SPKR

Figure 2.1: In this schematic diagram of a duct noise canceller, MIC1 obtains the noise reference orinput for the LMS Adaptive filter (FIR), which is digitized by an analog-to-digital converter (ADC);SPKR (a loudspeaker) emits the negative of the LMS adaptive filter output as a counter-wave to cancelthe noise, and MIC2 picks up the residual or error signal, which is also the net duct acoustic output,which is then fed into the LMS coefficient update algorithm block to update the coefficients of FIR.

A typical call might be

LVxLMSANCNorm([0, 0, 1, 0, -0.5, 0.6, 0, 0, -1.2, 0],2,2,3,[1,2,4,8])

which results in Fig. 2.2. It can been seen at plot (a) that all tap coefficients start at a value of zero,and quickly head toward their final values. Tap numbers 3, 5, 6, and 9 can be seen to converge to thesame weights found in the Plant’s impulse response, PlantCoeffVec. Note the random meandering ofthe coefficient estimates as they slowly, but unerringly, head toward their proper converged values.

The algorithm is an NLMS algorithm (described in the previous chapter), that is to say, itcomputes the signal power in the filter and uses its reciprocal to scale the coefficient update term;this is what makes the filter stable and immune to the sudden changes in signal amplitude definedby the input argument DVMult.

By varying the value of k downward to 0, the deleterious effects on convergence of excessively-low levels of noise in an otherwise-sinusoidal signal can be seen. A typical call that illustrates this,and which results in Fig. 2.3 would be

LVxLMSANCNorm([0,0,1,0,-0.5,0.6,0,0,-1.2,0],0,2,3,[1,2,4,8])

2.3 SYSTEM MODELINGThe filter and Plant arrangement in System Modeling is the same as in noise cancellation. Note thatin noise cancellation, the noise signal travelled through a system, the Plant, and also through theLMS adaptive filter, which (when properly converged) modeled the Plant’s impulse response.

Page 70: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

52 CHAPTER 2. APPLIED ADAPTIVE FILTERING

0 50 100 150

−1

0

1

(a) Iteration

Am

plitu

de

0 50 100 150

−50

0

50

(b) Sample, Input Data

Am

plitu

de

0 50 100 150

−202

(c) Sample, Output/ErrorA

mpl

itude

Figure 2.2: (a) All filter Tap Wts versus iteration; (b) Test signal, consisting of noise which has anamplitude profile that increases in steps; (c) The system output (error).

0 50 100 150

−0.5

0

0.5

(a) Iteration

Am

plitu

de

0 50 100 150−10

0

10

(b) Sample, Input Data

Am

plitu

de

0 50 100 150

−202

(c) Sample, Output/Error

Am

plitu

de

Figure 2.3: (a) Tap Weights versus sample or iteration; (b) Test signal, consisting of sinusoids withoutnoise; (c) The system output, which is the same as the error signal. Note the poor convergence when theinput signal is narrowband.

Page 71: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

2.3. SYSTEM MODELING 53

An easy way to obtain any system’s impulse response is to send an impulse or other test signal,such as white or pink noise, into the system and record the output.This method is sometimes used inactive noise cancellation systems and echo cancellation systems. Often, however, the system cannotbe taken offline to determine its impulse response; rather, the system must be modeled (i.e., theimpulse response determined) while the system is online, i.e., operating with standard signals ratherthan test signals.

Example 2.2. A certain system can be accessed only through its input and output terminals. Thatis to say, we can inject a test signal into its input terminal, and record the resultant output signal,but we cannot otherwise determine what components the system contains. Assume that the systemcan be modeled as an IIR filter having a specified number of poles and zeros. Devise a method toobtain its poles and zeros.

We first specify a transfer function consisting of one or more poles and zeros to simulate asystem (Plant) to be modeled. We then process a test signal, such as noise or speech, with this Plantto produce a corresponding output sequence. Having these two sequences, we can then model a finite(i.e., truncated) version of the impulse response that produced the output sequence from the inputsequence. To obtain the poles and zeros, we use Prony’s Method. The script (see exercises below)

LV xModelP lant (A, B, LenLMS,

NoP rZs, NoP rP s, Mu, tSig, NAmp, NoI ts)

allows you to enter a set of filter coefficients A and B as the numerator and denominator coefficientsof the IIR filter that simulates the Plant, the number of LMS FIR filter coefficients LenLMS to useto model the Plant, the number of Prony zeros NoPrZs and poles NoPrPs to use to model the derivedLMS coefficients with a pole/zero transfer function, the value of Mu, the type of test signal to passthrough the Plant and the LMS filter (0 = White Noise, 1 = DC, 2 = Nyquist, 3 = Half-band, 4 =mixed DC, Half-band, and Nyquist), NAmp, the amplitude of white noise to mix with the test signaltSig, and the number of iterations to do, NoIts. The Prony pole and zero estimates are returned inLabVIEW’s Output window.

A typical call might be

LVxModelPlant([1,0,0.81],1,100,3,3,0.5,0,0,1000)

which results in Fig. 2.4 and Prony estimates of b = [0.9986, -0.0008, -0.0013] and a = [1, -0.0003,0.8099] which round off to b = [1] and a = [1, 0, 0.81].

The same call, modified to call for any test signal other than white noise, generally causes theLMS modeling to fail. Injecting white noise into such an input signal will assist convergence.

Using an LMS algorithm to directly model an IIR-based system is possible, although con-vergence can be protracted and uncertain; performance surfaces can have localized minima leading

Page 72: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

54 CHAPTER 2. APPLIED ADAPTIVE FILTERING

0 10 20 30 40 50 60 70 80 90 100−1

0

1

(a) Sample, Test Signal (Partial)

Am

plitu

de

0 10 20 30 40 50 60 70 80 90 100

−1

0

1

(b) Sample, Plant Output (Partial)

Am

plitu

de

0 10 20 30 40 50 60 70 80 90 100

−0.50

0.51

(c) Sample, Plant ImpResp (Stars); Est Plant ImpResp (Circles)

Am

plitu

de

Figure 2.4: (a) Test Signal; (b) Response of Plant to signal at (a); (c) Truncated (actual) impulse responseof Plant (stars) and LMS-estimated Plant impulse response (circles).

to incorrect solutions. The method used in this example, although unsophisticated, has the stabil-ity and convergence speed of an FIR-based LMS algorithm. Reference [2] supplies several LMSalgorithms, including one using an IIR.

2.4 ECHO CANCELLATION

The most common venue for unwanted echo generation is telephone systems, especially loudspeakertelephones.Feedback or howlaround (like the feedback in an auditorium PA system) occurs due to thecomplete acoustic circuit formed by the loudspeaker and microphone at each end of the conversation.Figure 2.5 shows the basic arrangement.

Referring to Fig. 2.5, we see that a person at the Far End (left side of the Fig.) speaks intomicrophone M1, the sound is emitted at the Near End by loudspeaker S2, is picked up by microphoneM2, and is sent (in the absence of effective echo cancellation at the Near End) down the wire towardthe Far End, where it is emitted by loudspeaker S1, and then picked up again by microphone M1,thus forming a complete feedback or echo loop. The adaptive filter ADF2 at the Near End modelsthe acoustic feedback path at the Near End, and trains according to the error signal (the output ofthe summing junction at the Near End), which is the same as the signal being sent down the wireto the Far End. Since the input to adaptive filter ADF2 is the Far End signal, and only a delayed

Page 73: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

2.4. ECHO CANCELLATION 55

Echo Cancellation Using LMS Adaptive Filters

+

+

M1

M2

S2

S1

2FDA1FDA

Err

Err

"Far End" "Near End"

Network

ConnectionAcoustic

Acoustic

Feedback

Feedback

Path

Path

Xmitto Far End

toNear EndXmit Receive

Receivefrom Near End

from Far End

SpeechFar End

Near EndSpeech

+

+

Figure 2.5: A basic echo cancelling system employed in a telephone system in which both subscriberterminal units are loudspeaker phones, thus leading to the likelihood of acoustic echo, necessitating anecho canceller.

version of it (we are assuming here that the person at the Near End is not speaking) is entering theerror junction, the filter converges well.

Problems arise when the person at the Near End speaks–then we have a signal entering theerror junction that is not also entering the adaptive filter. The signal from the error junction outputthus does not convey an accurate representation of how well the filter is doing in cancelling the FarEnd signal. As a result, a perfectly converged set of coefficients, which cancel the Far End signalvery well, can be caused to severely diverge due to Near End speech.

The patent literature is replete with patents which attempt to solve or regulate the problemcaused by Near End speech. The sheer number of such patents suggest that the problem has notbeen completely solved. Prior to the advent of echo cancellers, a method was used known as EchoSuppression. In this method, the levels of Near and Far end signals are compared, and the dominant(usually louder) direction is designated to be the active circuit, and the gain of the nondominantdirection is greatly reduced to the point that howlaround no longer exists.

Returning to the problem of simulating an echo cancellation system, the scripts

Page 74: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

56 CHAPTER 2. APPLIED ADAPTIVE FILTERING

LV xLMSAdaptF iltEcho(k, Mu, DHorSH, MuteNrEnd, ...

DblT kUpD, SampsV OX, V OXT hresh)

and

LV xLMSAdptF iltEchoShort (k, Mu, DHorSH, MuteNrEnd, P rPC)

allow you to experiment with echo cancellation. For the first mentioned script, two audio files,drwatsonSR8K.wav, and whoknowsSR8K.wav are used as the Near End and Far End signals, re-spectively. The second script is suitable when script execution speed needs to be improved; it usestwo bandwidth-reduced audio files, drwatsonSR4K.wav, and whoknowsSR4K.wav, and does notcompute or archive certain functions. The description of parameters below applies to the first men-tioned script, although the first four input parameters are identical for both scripts. A more completedescription of the second mentioned script is given in the exercises below.

Playing the unprocessed and processed signals for comparison can be programmed in thescript to take place automatically, but for repeat playing, the directions in the exercises below suggestthat the relevant variables representing the unprocessed and processed sounds be made global in thescript.Then, by declaring them global on the Command line, it is a simple matter to play them usingthe function sound. To convey to the user the appropriate variable names and the sample rate, theseare programmed in the script to be printed in the Command Window in MATLAB or OutputWindow in LabVIEW. For example, after making the call

LVxLMSAdptFiltEchoShort(0.01,0.2,1,1,50)

in LabVIEW, the following is printed in the Output Window:

global sound output variable is EchoErr & sample rate is 8000

To listen to the processed audio at will, enter the following lines, pressing Return after eachline:

global EchoErrsound(EchoErr,8000)

The method for playing audio signals just described also holds true for the other scripts in thischapter that produce audio outputs. When using MATLAB, it is possible to create pushbuttons onthe GUI with callback routines to play the audio files on demand (this method is also implementedusing global variables).

Both scripts simulate the acoustic path at the Near End as a single delay with unity gain. Inthe figures below, the plots of coefficients show, under ideal conditions, one coefficient convergingto a value of 1, and the nine other coefficients converging to the value 0. As we’ll see, however,

Page 75: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

2.4. ECHO CANCELLATION 57

conditions are not always ideal and often the coefficients remain misadjusted. Prior to conducting asystematic exploration of why this is and what can be done about it, we’ll define the input argumentsof the script LVxLMSAdaptFiltEcho:

• The argument k allows you to choose an amount of white noise to mix with the Far Endsignal to assist the LMS adaptive filter in converging to a set of coefficients which models theacoustic path between S2 and M2 as shown in Fig. 2.5.

• The argument Mu is the usual update term weighting constant for the LMS algorithm.

• The argument DHorSH tells the script whether or not to use a single LMS adaptive filter(Single-H ), or to use the Dual -H mode, in which an on-line filter and an off-line filter areused, and a figure-of-merit called ERLE is computed to determine how effectively the echosignal is being removed from the error signal, which is the signal being returned to the FarEnd from the Near End, along with Near End speech. A detailed discussion of the differencebetween the Single-H and Dual-H modes, along with a mathematical definition of ERLE, isfound below.

• MuteNrEnd can take on the values 0, 1, or 2; if specified as 0, the Near End signal is emittedcontinuously into microphone M2. If MuteNrEnd is specified as 1, the Near End signal ismuted for a brief period at the beginning of echo cancellation, thus giving the Far End signal abrief period without Near End speech in which to try to converge. If MuteNrEnd is specifiedas 2, the Near End signal is muted once about one-third of the way through. This mutingis meant to simulate near silence from the Near End speaker (person); such silence assiststhe LMS adaptive filter in converging to a set of coefficients which model the acoustic pathbetween the loudspeaker S2 and microphone M2 at the Near End.

• The argument DblTkUpD, if passed as 1, prevents coefficient update in Single-H mode when-ever Near End speech is detected. If passed as 0, coefficient update is permitted even whenNear End speech is present.

• The remaining two arguments, SampsVOX, and VOXThresh, are two parameters you can ma-nipulate to specify how Near End Speech is detected. The first argument, SampsVOX, tellshow many samples of the Near End speech signal to use in computing the RMS value, whichis then compared to the second argument, VOXThresh, which is a threshold below which it isjudged that Near End speech is not present.

2.4.1 SINGLE-HThe Single-H mode employs a single LMS adaptive filter to cancel a Far End acoustic echo thatis “contaminated” with Near End speech. Such an arrangement performs poorly unless coefficientupdate is stopped whenever Near End speech is detected. The script LVxLMSAdaptFiltEcho allows

Page 76: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

58 CHAPTER 2. APPLIED ADAPTIVE FILTERING

you to specify Single-H and whether or not to allow coefficient update during Near End speech(sometimes called “doubletalk,” since both Near End and Far End are making sound).

Example 2.3. Demonstrate adaptive echo-cancelling using the Single-H mode.

Suitable calls to use the Single-H mode with coefficient update during doubletalk would be

LVxLMSAdaptFiltEcho(0.02,0.2,0,0,0,50,0.02)

LVxLMSAdptFiltEchoShort(0.02,0.2,0,0,50)

Figure 2.6 shows the result of making the first call (the second script call plots only theoutput/error signal) for Single-H echo cancellation with update during doubletalk. Note that inplot (a) the coefficient values in the adaptive filter, which should converge to essentially constantvalues, are overladen with noise. Note that the Single-H mode does not use the parameter ERLE,but ERLE is computed and displayed for comparison with the Dual-H mode, in which ERLEincreases monotonically as better coefficient estimates occur.

Example 2.4. Demonstrate adaptive echo-cancellation using the Single-H mode, but freeze coef-ficient update during doubletalk.

Freezing coefficient update during doubletalk results in considerable improvement, as can beseen in Fig. 2.7, which resulted from the script call

LVxLMSAdaptFiltEcho(0.02,0.2,0,0,1,50,0.02)

Here we see that the filter coefficients, shown in plot (a), do assume reasonably constantvalues, but with a much decreased level of noise, since coefficient update during Near End speechwas inhibited, as shown in plot (d), where a value of zero at a given sample or iteration means thatcoefficient update was inhibited for that sample/iteration.

2.4.2 DUAL-HThe Dual-H mode is a simple way to overcome the difficulties caused by Near End speech. Inthe Dual-H mode, a main filter or on-line filter, which actually generates the counter-echo signalwhich is subtracted from the Near End’s microphone output (M2 in Fig. 2.5), has its coefficientsupdated only when a second, off-line filter, which adapts continuously (i.e., computes a new setof coefficients for each signal sample), generates a set of filter coefficients which result in a bettermeasure of merit, such as ERLE (Echo Return Loss Enhancement), which is defined here and inthe script LVxLMSAdaptFiltEcho (and others that follow in this chapter) as the ratio of Far Endpower (or energy) to Near End (output of LMS error junction) power (or energy). These valuescan, for better effect, be averaged over a number of samples (in the scripts mentioned above, the

Page 77: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

2.4. ECHO CANCELLATION 59

0 0.5 1 1.5 2 2.5 3 3.5 4

x 104

024

(a) IterationA

mp

0 0.5 1 1.5 2 2.5 3 3.5 4

x 104

−202

Am

p

(b) Sample

0 0.5 1 1.5 2 2.5 3 3.5 4

x 104

050

100

Mag

, dB

(c) Iteration

0 0.5 1 1.5 2 2.5 3 3.5 4

x 104

00.5

1

Bin

ary

(d) Iteration

Figure 2.6: (a) All of the adaptive filter coefficients versus sample or iteration; (b) Error signal; (c) ERLE,a figure of merit representing how much of the Far End signal has been removed from the signal;(d) Function “Coefficient Update Permitted?” A value of 0 means that coefficient update was inhibiteddue to the presence of Near End speech, while a value of 1 means coefficient update was allowed toproceed at the particular iteration or sample number.

0 0.5 1 1.5 2 2.5 3 3.5 4

x 104

−101

(a) Iteration

Am

p

0 0.5 1 1.5 2 2.5 3 3.5 4

x 104

−1

0

1

Am

p

(b) Sample

0 0.5 1 1.5 2 2.5 3 3.5 4

x 104

050

100

Mag

, dB

(c) Iteration

0 0.5 1 1.5 2 2.5 3 3.5 4

x 104

00.5

1

Bin

ary

(d) Iteration

Figure 2.7: (a) All of the adaptive filter coefficients versus sample or iteration; (b) Error signal; (c) ERLE.Note that, due to doubletalk, there was relatively little coefficient update during the course of the con-versation, but the performance was improved considerably over the constant coefficient update scenario;(d) Function “Coefficient Update Permitted?”

Page 78: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

60 CHAPTER 2. APPLIED ADAPTIVE FILTERING

number chosen is equal to the filter length, but this need not be the number). Thus, ERLE mightbe defined as the ratio of the average signal power input to the filter to the average error power forthe corresponding samples, i.e.,

ERLE[n] = (

n∑i=n−N+1

S[i]2)/((

n∑i=n−N+1

E[i]2) + ε) (2.1)

where n is the sample index, N is the chosen number of samples to use in the computation, S

represents the signal (the Far End signal in this case), E represents the error signal, and ε is a smallnumber to prevent division by zero if the error signal is identically zero over the subject interval ofsamples.

The current best set of filter coefficients and a corresponding best ERLE are stored andfor each new sample, the off-line filter is updated according to the LMS algorithm, and an echoprediction signal (i.e., LMS adaptive filter output) is generated and subtracted from the current NearEnd microphone output to generate a Test Error signal. A value for ERLE is generated based onthe Test Error, and if this Test ERLE is improved over the current best ERLE, the off-line or testcoefficients are stored into the on-line filter and also as the current best coefficients. The best ERLEis replaced by the newly computed Test ERLE.

Example 2.5. Demonstrate adaptive echo-cancelling using the Dual-H mode.

Figure 2.8 shows the results from the script call

LVxLMSAdaptFiltEcho(0.02,0.2,1,1,[],[],[])

in which the last three arguments are immaterial since the Dual-H mode has been specified by thethird argument. In the Dual-H system, it is unnecessary to restrict coefficient update since ERLE isused to judge whether or not coefficients in the on-line filter should be updated, and hence the inputparameters DblTkUpD, SampsVOX, and VOXThresh are ignored when in Dual-H mode, and thefunction “Coefficient Update Permitted” is assigned the value 1 for all iterations of the algorithm.

2.4.3 SPARSE COMPUTATIONThe plant delay in echo cancellers often runs into hundreds of milliseconds and hence, at, say, asample rate of 8 kHz, an FIR that covered every possible tap from a time delay of one sample upto and somewhat beyond the length of the impulse response could potentially be several thousandtaps long. Often in echo cancellers, out of a large number of taps, however, there are only a fewtaps having significant amplitude. These taps are, of course, located at corresponding echo timesrepresented by the impulse response of the echo-causing system (a room, for example, in the case ofa speakerphone). A common technique to greatly reduce the needed computation is to estimate thelocations of the taps having significant amplitudes, and to restrict tap computation to relatively shortruns of contiguous taps around each significant tap location. This technique also helps to ensure

Page 79: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

2.5. PERIODIC COMPONENT ELIMINATION OR ENHANCEMENT 61

0 0.5 1 1.5 2 2.5 3 3.5 4

x 104

−1

0

1

(a) IterationA

mp

0 0.5 1 1.5 2 2.5 3 3.5 4

x 104

−1

0

1

Am

p

(b) Sample

0 0.5 1 1.5 2 2.5 3 3.5 4

x 104

50

100

Mag

, dB

(c) Iteration

0 0.5 1 1.5 2 2.5 3 3.5 4

x 104

00.5

1

Bin

ary

(d) Iteration

Figure 2.8: (a) All of the adaptive filter coefficients versus iteration; (b) Error signal; (c) Current BestValue of ERLE. Note that the brief introductory portion of the signal was muted, allowing the systemto quickly converge to an excellent set of coefficients, which were not improved upon as ERLE remainedconstant after the initial convergence; (d) Function “Coefficient Update Permitted?”

proper convergence since all noncomputed taps are inherently valued at zero. The selection of tapsto be computed, and thus also those not to be computed, is itself a filtering process. One method toestimate the relevant taps is to cross correlate the Far End signal with the signal coming from theNear End microphone, which contains an echoed version of the Far End signal plus the Near Endsignal.

As sparse computation largely arises from practical and empirical knowledge and practice, anexcellent source for information is the website www.uspto.gov, which allows you to do word searchesfor relevant U.S. Patents, which contain much information not found in other sources. A number ofpatents relating to sparse computation, which may be viewed or downloaded from www.uspto.gov,are cited below. You may also conduct your own searches. The best (most focused and pertinent)results are usually obtained when a very specific search term is used, such as “pnlms,” “nlms,” “lms,”etc. Advanced searches can also be performed using multiple terms, such as “sparse” and “echo,”for example.

2.5 PERIODIC COMPONENT ELIMINATION ORENHANCEMENT

Figure 2.9 shows the basic arrangement for predictive filtering. Since the input to the filter is delayed,it is not possible for it to adapt to the input signal in time to cancel the input signal at the error

Page 80: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

62 CHAPTER 2. APPLIED ADAPTIVE FILTERING

summing junction.Of course, this is only true for random signals which change constantly.A periodicsignal, such as a sine wave, looks alike cycle after cycle, so the filter can still adjust and cancel it inthe error junction’s output. The implication is that it is possible to select or enhance a periodiccomponent with the adaptive filter. The filter’s output yields the enhanced periodic component(s),and the error junction output yields a version of the input signal with periodic components removed.This is useful for eliminating periodic interference/noise, such as heterodynes (whistles or tones) ina communications receiver or the like. The filter output per se (not the error junction output) is anenhanced version of any sinusoidal input signal. Used in this manner, the arrangement is called anAdaptive Line Enhancer.

Periodic Component Cancellation

AdaptiveFilter

Coefficient UpdateLMS

Signal

+

DecorrelatingDelay Periodic Signal

Output

Error(Input Sig less Periodic Components)

Figure 2.9: A basic arrangement which uses an LMS Adaptive filter to eliminate only periodic compo-nents from a signal.

The scripts

LV xLMSAdaptF iltDeCorr(k, Mu, f req, NoT aps, DeCorrDelay)

and

MLx_LMSAdaptF iltDeCorr(k, Mu, f req, NoT aps, DeCorrDelay)

(see exercises below) demonstrate elimination of periodic signals mixed with white noise, leavingmainly the white noise. The first input argument specifies the amplitude of white noise, the sec-ond argument is Mu, and the third argument, f req, is used to generate two sine waves having,

Page 81: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

2.5. PERIODIC COMPONENT ELIMINATION OR ENHANCEMENT 63

respectively, frequencies of freq and 3*freq.The sampling rate is 1024 for this script, so the maximumnumber you can use for the third argument is about 170 (without aliasing). The fourth argumentallows you to specify the number of taps used in the filter, up to 100, and the fifth argument al-lows you to specify the number of samples of decorrelating delay. In the MATLAB-suitable scriptMLx_LMSAdaptFiltDeCorr, the various signals may be heard by pressing buttons on the GUI cre-ated by the script. In the LabVIEW-suitable script, the sounds play automatically, and the soundfiles are global variables that can be played again by declaring the variables global on the Commandline and then giving suitable Commands. An example of this is given immediately below.

Example 2.6. Using the script LVxLMSAdaptFiltDeCorr, demonstrate periodic component elim-ination. Use a decorrelating delay of 2 samples.

Suitable calls would be

LVxLMSAdaptFiltDeCorr(0.2,1,100,10,2)

or

MLxLMSAdaptFiltDeCorr(0.2,1,100,10,2)

which result in Figs. 2.10 and 2.11. Much of the amplitude in the input signal is from the twoperiodic components. After the filter converges (very quickly in this case), only the residual whitenoise is left; this is also evident from the respective DFTs, plotted to the right.

After making the script call above, the following information will be printed in the CommandWindow:

Comment =global sound variable names are LMSDeCorDataVec, Err, and FiltOutComment =global sound variable sample rate is 8000

This information can be used to listen to the audio files on demand, as described earlier in thechapter. Note that the sample rate in the script is 1024 Hz, but LabVIEW only accepts standardsample rates such as 8000, 11025, 22050, and 44100. Thus, the sounds produced by LabVIEW willbe higher in pitch by the factor 8000/1024 and shorter in duration by the factor 1024/8000. Whenwriting and running the MATLAB-suited script, you can use the actual sample rate of 1024 anduse pushbuttons on the GUI or use the global variable method described above.

Example 2.7. Show that when the decorrelating delay is zero samples, the script LVxLMSAdapt-FiltDeCorr will no longer decorrelate the filter and algorithm update inputs, and hence the entireinput signal will be cancelled.

Page 82: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

64 CHAPTER 2. APPLIED ADAPTIVE FILTERING

0 200 400

−1

0

1

Am

plitu

de

(a) Sample, Input Data

0 500 1000

−1

0

1

Am

plitu

de

(c) Sample, Output/Error

0 0.5 1

0

10

20

30

40

50

(b) Normalized Frequency

Mag

nitu

de, d

B0 0.5 1

0

10

20

30

40

50

Mag

nitu

de, d

B

(d) Normalized Frequency

Figure 2.10: (a) Test signal, consisting of periodic components of high magnitude and noise of muchlower magnitude; (b) Spectrum of the test signal; (c) Output/Error; (d) Spectrum of Output/Error,showing only noise.

0 100 200 300 400 500

−1

0

1

Am

plitu

de

(a) Sample, LMS Filter Output

0 0.2 0.4 0.6 0.8 1

−20

0

20

40

Mag

nitu

de, d

B

(b) Normalized Frequency

Figure 2.11: (a) The output of the LMS adaptive FIR, showing periodic signals enhanced by the attenu-ation of noise; (b) Spectrum of signal in (a). Note the attenuation of noise, in contrast to the unattenuatednoise-and-signal spectrum shown in Figure 2.10, plot (b).

Page 83: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

2.6. INTERFERENCE CANCELLATION 65

The call

LVxLMSAdaptFiltDeCorr(0.2,1,100,10,0)

which specifies the decorrelating delay as 0, results in the filter being able to cancel both periodicand nonperiodic components, as shown in Fig. 2.12 (this call essentially configures the filter as anActive Noise Canceller).

0 200 400

−1

0

1

Am

plitu

de

(a) Sample, Input Data

0 500 1000

−1

0

1

Am

plitu

de

(c) Sample, Output/Error

0 0.5 1

0

20

40

(b) Normalized Frequency

Mag

nitu

de, d

B

0 0.5 1

0

20

40

Mag

nitu

de, d

B

(d) Normalized Frequency

Figure 2.12: (a) Test signal, consisting of periodic components of high magnitude and noise of muchlower magnitude; (b) Spectrum of the test signal; (c) Output/Error; (d) Spectrum of Output/Error,showing attenuation of both periodic and aperiodic components.

Two scripts, LVxLMSDeCorrHeterodyne and LVxLMSDeCorrHetDualH, are developed in theexercises below to explore periodic component elimination in more detail.

2.6 INTERFERENCE CANCELLATION

Another use for adaptive filtering is to cancel interference from a channel signal which is a mixtureof some desired signal and an interference signal. A reference signal is obtained for the interference,and it is passed through the adaptive filter and the result subtracted from the channel signal. In thegeneral case, the reference signal also contains a component of the desired signal.

Page 84: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

66 CHAPTER 2. APPLIED ADAPTIVE FILTERING

Signal Plus Noise and Noise Plus Signal ChannelsFigure 2.13 shows a basic arrangement for interference cancellation. Interference cancellation isimplemented by using a separate input which serves as a reference signal for the noise (interference)component. The reference signal, ideally, consists only of the noise signal and is unpolluted with thedesired signal component. In many, if not most practical cases, this will not be true, but effectivecancellation is usually possible. The general case is shown here, where some of the desired signalmixes with the noise reference, and of course some of the noise mixes with the desired signal (if itdidn’t, there would be no interference needing to be cancelled).This arrangement works, delay-wise,when the desired signal source is close to M1 (i.e., Path 1 is very short) and the noise source isclose to M2 (i.e., Path 3 is very short), since in this situation the delay of the LMS adaptive filtercan effectively model the delay from the noise source to M1, which is the delay of Path 4. On theother hand, the delay from the desired signal source to M2 (via Path 2) is such that the filter cannotcorrelate the desired signal component in the error signal with the desired signal component in thefilter. Hence, the net result is that the noise signal can be cancelled, and the desired signal cannot becancelled. The other thing to note is that the desired signal distorts the error signal, just as the nearend signal does in an echo canceller. As a result, a Dual-H arrangement or a Single-H arrangementwith no update during desired signal speech must be used.

Referring to Fig. 2.13, the problem of cancelling the Noise Signal component from theOutput signal can be seen as a problem of linear combination. Here we are making the assumption,for purposes of ease of explanation, that all delays associated with Paths 1 through 4 are simple delays(i.e., no echoes are involved) so that each path delay can be simply characterized as z−N where N

represents the path delay as a number of multiples N of the sample period.Using z-transform notation we get

Output (z) = G1S(z)z−�1 + G4N(z)z−�4...

−(G3N(z)z−�3 + G2S(z)z−�2)LMS(z)

where G1, for example, means the gain over Path 1 (Gi = 1 means no attenuation, Gi = 0 meanscomplete attenuation), Output (z) is the z-transform of the output signal, S(z) means the z-transform of the Desired Signal, N(z) means the z-transform of the noise, z−�1represents thedelay of Path 1, etc., and LMS(z) means the z-transform of the LMS adaptive filter.

When the filter has converged, the Noise component that is mixed with the Desired Signalis cancelled in the output, and hence it is possible to say that

G4N(z)z−�4 = G3N(z)z−�3LMS(z)

which simplifies to

LMS(z) = G4N(z)z−�4/(G3N(z)z−�3)

Page 85: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

2.6. INTERFERENCE CANCELLATION 67

Noise Signal Source

Desired Signal Source

M1

M2

AdaptiveFilter

+

OutputPath 1

Path 2

Path 3

Path 4

Error

Interference Cancellation

+

Figure 2.13: A basic interference-cancelling arrangement.

and finally

LMS(z) = G4

G3z−�4+�3

The important thing to note is that the LMS adaptive filter imparts time delay only, not timeadvancement, which means that the quantity (−�4 + �3) must remain negative and have a value(in terms of number of samples) that lies within the capability of the filter.

For example, if �4 = 8 samples, and �3 = 1 sample, then the quantity −�4 + �3 = -7 whichmeans that the LMS adaptive filter must be able to impart a delay of 7 samples. If �3 is greaterthan �4, of course, the filter would be expected to advance signals in time, which it cannot do (if�3 = �4, the filter must have a zero-delay tap).

At proper convergence, the Noise Signal is completely cancelled and the output becomes

Output (z) = G1S(z)z−�1 − G2S(z)z−�2 G4

G3z−�4+�3

which simplifies to

Page 86: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

68 CHAPTER 2. APPLIED ADAPTIVE FILTERING

Output (z) = S(z)(G1z−�P 1 − (G2G4/G3)z

−�2−�4+�3)

What this says is that the Desired Signal is effectively filtered by the interference cancellationprocess. For example, assuming that G1 = G3 = 1 and �1 = �3 = 0, we get

Output (z)/S(z) = (1 − G2G4z−�P 2−�P 4)

which is the transfer function of a modified comb filter.The scripts (see exercises below)

LVxLMSInterferCancel(k,Mu,DHorSH,MuteDesSig,DblTkUpD,NoSampsVOX,VOXThresh,GP2,GP4,DP1,DP2,DP3,DP4,NumTaps)

and

LVxLMSInterferCancShort(k,Mu,DHorSH,MuteDesSig,GP2,GP4,DP1,DP2,DP3,DP4,NumTaps,PrPC)

offer the opportunity to experiment with interference cancellation. A large number of parametersare provided to allow ample experimentation. GP 2 and GP 4 have the same meaning as G2, G4, etc.,used above, and D1 means the same thing as �1, etc. The first seven input arguments of the firstmentioned script above are the same as the first seven arguments of the script LVxLMSAdaptFiltEcho.

For purposes of improving computation speed, the second script, LVxLMSInterferCancShort,uses bandwidth reduced audio files (’drwatsonSR4k.wav’ and ’whoknowsSR4K.wav’) and does notdeal with doubletalk or archive parameters such as the figure of merit (IRLE or Interference ReturnLoss Enhancement). The input argument PrPC is the percentage of the audio files to processwhen running the script (30% to 50% will usually suffice to demonstrate the required results). Moredetail on the latter script, LVxLMSInterferCancShort will be found in the exercises below, where thestudent is guided to create the m-code for the script.

The argument, NumTaps, found in both scripts, allows you to specify the number of taps usedby the adaptive filter. For proper cancellation, it should be equal to at least DP4 - DP3. For example,if DP4 = 42, and DP3 = 7, NoTaps should be 35 or greater.

For purposes of reducing the number of required input arguments, the gains of paths P1 andP3 are assigned the value 1, with realistic values of G2 and G4 being fractions of 1; this correspondsapproximately to the situation in which each microphone is very close to its respective sound source,which results in the highest signal-to-noise ratio for M1, and the highest noise-to-signal ratio forM2, which would result in the smallest amount of comb filtering of the desired signal in the output.

Example 2.8. Simulate interference cancellation using the script LVxLMSInterferCancel. Make acall that shows interference cancellation with little comb filtering, and make a second call that showspronounced comb filtering.

Page 87: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

2.7. EQUALIZATION/DECONVOLUTION 69

For the first task, we make the following call, which has relatively low gain (1/16) for paths 2and 4, and which specifies use of the Dual-H mode with immediate muting to speed convergence,and which results in Fig. 2.14.

LVxLMSInterferCancel(0.02,0.3,1,1,0,50,0.03,1/16,1/16,6,24,6,24,18)

A call requiring less computation, and which plots only the filtered output signal (see exercisesbelow for further details) is

LVxLMSInterferCancShort(0.02,0.3,1,1,1/16,1/16,6,24,6,24,18,30)

The spectrum of the output signal is shown in Fig. 2.15 in plot (a); this may be contrastedwith the spectrum shown in plot (b), which arises from the call

LVxLMSInterferCancel(0.02,0.3,1,1,0,50,0.03,1,1,1,6,1,6,18)

in which the paths P2 and P4 are given gains of 1.0 (i.e., no attenuation). A comb-filtering effect isclearly visible in plot (b), where the cross-paths P2 and P4, like paths P1 and P3, have gains of 1.0.In realistic situations, where the gains of paths P2 and P4 are small compared to those of P1 andP3, the comb-filtering effect is greatly attenuated, as is evident in plot (a) of Fig. 2.15.

Reference [1] gives a basic description of the various LMS filtering topologies discussed inthis chapter.

2.7 EQUALIZATION/DECONVOLUTIONIt is possible to use an adaptive filter to equalize, or undo the result of convolution; that is to say, asignal that has passed through a Plant (being, perhaps, an amplification channel having a filteringcharacteristic such as lowpass or bandpass) will have its characteristics changed since it will have beenconvolved with the Plant’s impulse response. A filter that can undo the effects of convolution mightbe called a deconvolution filter, or an equalization filter, or simply an inverse filter. For simplicity,we’ll only use one of these terms in the discussion below, but all of them apply.

The basic arrangement or topology for channel equalization is as shown in Fig. 2.16.The script

LV xChannEqualiz(ChImp, Mu, tSig, lenLMS)

allows experimentation with channel equalization using an LMS algorithm in the topology shownin Fig. 2.16. The parameter ChImp is the impulse response of the Channel, Mu has the usualmeaning in LMS adaptive filtering, tSig is an integer that selects the type of test signal to passthrough the Channel/LMS Filter serial combination to attempt to train the filter to the inverse ofthe Channel Impulse Response, with 0 yielding random noise, 1 yielding a repeated unit impulsesequence of length equal to the LMS filter, with its phase alternated every repetition, 2 yieldinga repeated unit impulse sequence equal in length to the adaptive filter, and 3 yielding a sinusoidalsequence of half-band frequency. The length of the adaptive filter is specified by lenLMS.

Page 88: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

70 CHAPTER 2. APPLIED ADAPTIVE FILTERING

0 0.5 1 1.5 2 2.5 3 3.5 4

x 104

00.1

(a) Sample, Tap WtsA

mp

0 0.5 1 1.5 2 2.5 3 3.5 4

x 104

−101

Am

p

(b) Sample, Error/Output

0 0.5 1 1.5 2 2.5 3 3.5 4

x 104

0

1

Bin

ary

(c) Sample, Fcn "Coefficient Update Permitted?"

0 0.5 1 1.5 2 2.5 3 3.5 4

x 104

50100

(d) Sample, CurBestIRLE

Mag

, dB

Figure 2.14: (a) All of the adaptive filter coefficients versus sample or iteration, showing good conver-gence and stability; (b) Output/Error signal; (c) Function “Coefficient Update Permitted?”; (d) CurrentBest value of Interference Return Loss Enhancement (IRLE) during the course of algorithm execution.

0 2 4 6 8 10 12

x 104

200

400

600

Mag

(b) Bin (GP2 = 1; GP4 = 1)

0 2 4 6 8 10 12

x 104

200

400

600

Mag

(a) Bin (GP2 = 0.0625; GP4 = 0.0625)

Figure 2.15: (a) Typical output spectrum of signal after noise signal is subtracted, with the conditionthat significant attenuation exists in the crosspaths P2 and P4; (b) Same as (a), except that there was noattenuation in the crosspaths, leading to significant comb filtering effects in the output signal.

Page 89: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

2.7. EQUALIZATION/DECONVOLUTION 71

ChannelImpulse

Response

Channel Equalization / Inverse Filtering / Deconvolution

( Plant )

AdaptiveFilter

Coefficient UpdateLMS

Delay

Error

Output

Signal

−+

Figure 2.16: A basic arrangement for Channel Equalization.

An important thing to note in the script is that the Channel Impulse Response is alwaysmodeled as an FIR, and the adaptive filter is also an FIR. For mathematically perfect inverse filtering,the transfer functions of the two should be the reciprocals of one another. Since both are FIRs, thiscannot be. However, an LMS FIR filter of sufficient length can converge to a truncated version ofan infinite impulse response.

Example 2.9. A Channel has z-transform 1 - 0.7z−1. Determine what length of LMS FIR fil-ter should be used if we arbitrarily require that the truncated inverse include all samples havingmagnitude of 0.01 or greater.

The reciprocal of the Channel’s z-transform is 1/(1 - 0.7z−1) (ROC: |z| > 0.7), and thecorresponding impulse response is [1,0.7,0.49,...0.7n] = (0.7)nu[n].This is an exponentially decayingsequence, and we therefore need the smallest value of n such that

0.7n ≤ 0.01

We compute log(0.01) = n log(0.7) from which n = 12.91 which we round to 13. Thus, anLMS FIR length of 13 or more should be adequate under this criterion.

Example 2.10. Determine if the impulse response of an IIR, truncated to a length-17 FIR, canserve as an adequate inverse for a Channel having its impulse response equal to [1, 1].

Page 90: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

72 CHAPTER 2. APPLIED ADAPTIVE FILTERING

The true inverse system has H(z) = 1/(1 + z−1). This is a marginally stable IIR with anondecaying impulse response equal to (−1)nu[n]. The z-transform of the converged, length-17Channel/LMS Filter system in this case is

(1 + z−1)(1 − z−1 + z−2 − ... + z−16) = 1 + z−17

which is a comb filter, which does not provide a satisfactory equalization of the net channel response.

Example 2.11. For a Channel Impulse Response having z-transform 1 + 0.7z−1, compute a length-17 truncation of the IIR that forms the (approximate) inverse or equalization filter for the Channel.Compute the net equalized Channel impulse response.

Since the z-transform of the Channel Impulse Response is

H(z) = 1 + 0.7z−1

the true inverse would be an IIR having the z-transform

H(z) = 1

1 + 0.7z−1

which equates to a decaying Nyquist rate impulse response with values [1,-0.7,0.49,...(-0.7)n] wheren starts with 0.

The z-transform of the net Channel/LMS Filter is

(1 + 0.7z−1)(1 − 0.7z−1 + 0.49z−2 − ... + (−0.7)16z−16) = 1 + (0.7)17z−17

which simplifies to 1 + 0.0023z−17; this is a severely modified comb filter with a miniscule delay termhaving essentially no effect, i.e., the inverse approximation is a very good one as the net combinationapproaches having a unity transfer function.

The result from making the call

LVxChannEqualiz([1,0.7],2.15,2,17)

(in which Mu is 2.15 and the test signal comprises multiple repetitions of a unit impulse sequencehaving the same length as the filter, 17) is shown in Fig. 2.17.

Example 2.12. Demonstrate Channel Equalization using the script LVxChannEqualiz, with achannel impulse response of [1, 0, 1].

The Channel Impulse Response [1,0,1] is a comb filter with precisely one null at the half-bandfrequency, making it a simple notch filter at that frequency. The IIR which is the perfect inverse

Page 91: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

2.7. EQUALIZATION/DECONVOLUTION 73

0 100 200

−1

−0.5

0

0.5

1

(a) Test Signal

Am

p

0 1 2 30

0.5

1

(b) Ch Imp Resp

Am

p

0 0.5 10

0.5

1

1.5

(c) Freq, Ch Imp

Mag

0 10

−1

−0.5

0

0.5

1

(d) Tap Wts

Am

p

0 0.5 10

0.5

1

(f) Freq, Equal. Chan

Mag

0 0.5 10

1

2

3

(e) Freq, Tap Wts

Mag

Figure 2.17: (a) The test signal; (b) Channel impulse response; (c) Frequency response of the Chan-nel; (d) Converged tap weights of inverse filter; (e) Frequency response of tap weights shown in (d);(f ) Frequency response of net equalized channel.

0 100

−1

−0.5

0

0.5

1

(a) Test Signal

Am

p

0 2 40

0.5

1

(b) Ch Imp Resp

Am

p

0 0.5 10

0.5

1

1.5

2

(c) Freq, Ch Imp

Mag

0 5 10

−0.5

0

0.5

(d) Tap Wts

Am

p

0 0.5 10

0.5

1

(f) Freq, Equal. Chan

Mag

0 0.5 10

1

2

(e) Freq, Tap Wts

Mag

Figure 2.18: (a) The test signal; (b) Channel impulse response; (c) Frequency response of the Chan-nel; (d) Converged tap weights of inverse filter; (e) Frequency response of tap weights shown in (d);(f ) Frequency response of equalized channel.

Page 92: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

74 CHAPTER 2. APPLIED ADAPTIVE FILTERING

filter has an infinite, nondecaying impulse response forming a sinusoid at the half-band frequency(i.e., four samples per cycle), such as [1,0,-1,0,1,0,...]. The call

LVxChannEqualiz([1,0,1],2.4,2,10)

which specifies ChIMp = [1,0,1], Mu = 2.4, tSig = repetitive unit impulse sequence of length 10,and lenLMS = 10, results in Fig. 2.18, which shows that the tap weights have converged to a half-band-frequency sinusoid, but the resultant equalized channel response is that of a comb filter. Thiscan be seen by convolving the Channel’s impulse response and the LMS filter’s impulse response,i.e.,

y = conv([1,0,1],[0,1,0,-1,0,1,0,-1,0,1])% y = [1,0,0,0,0,0,0,0,0,0,1]fr = abs(fft(y,1024)); plot(fr(1,1:513))

which results in Fig. 2.19.

2.8 DECONVOLUTION OF A REVERBERATIVE SIGNALAn occasionally-occurring problem in audio processing is the presence of echo or reverberation,caused by a highly reflective acoustic environment.While small to moderate amounts of reverberationare often imparted to signals deliberately to make them resemble signals in a reverberative room orhall, large amounts of unwanted or unintended reverberation often make an audio signal difficult tocomprehend, in the case of speech, or unpleasant in the case of music.

In the previous section we used a topology for deconvolution that assumed that the uncon-volved (i.e., un-reverberated) signal is available as a reference. In this section, we will attempt todereverberate (deconvolve) an audio signal for which we do not possess the pre-reverberative signal.

In theory, if we can estimate the transfer function between the sound source and the micro-phone (or listener), we can undo all or part of the reverberative (or echo) process. For our study, we’lluse a simple type of reverberation created using an IIR with a single stage of feedback having a longdelay and real scaling factor with magnitude less than one. The echoes in such a signal are evenlyspaced and decay at a uniform rate, making estimation of the parameters relatively easy. We’ll usethe estimated parameters of the IIR (the delay in samples and the gain of the feedback stage) andform a deconvolution or inverse filter to remove the echo from the original signal. We’ll also take alook at the spectral effect of reverberation on the signal. To do all of this, we’ll call on knowledge ofIIRs, the z-Transform, Autocorrelation, and LMS Adaptive filtering.

2.8.1 SIMULATIONWe’ll start by creating a relatively simple reverberant audio signal by passing the “raw” audio signalthrough an IIR having a substantial number of samples of delay (such that the equivalent time delayis about 50 msec or more), and a single feedback coefficient having a magnitude less than 1.0. In thez-domain such an IIR would have the transfer function

Page 93: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

2.8. DECONVOLUTION OF A REVERBERATIVE SIGNAL 75

0 0.2 0.4 0.6 0.8 10

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

2

Mag

nitu

de

Normalized Frequency

Figure 2.19: The frequency response of the net channel impulse response obtained as the convolution ofthe Channel Impulse Response [1,0,1] and a set of converged LMS tap weights which form a truncatedinverse, [0,1,0,-1,0,1,0,-1,0,1].

H(z) = 1

1 − az−n(2.2)

where a is a real number having magnitude less than 1 and n is chosen such that

DesiredEchoPeriod = nTs

where Ts is the sampling period.Next, we’ll estimate the values of a and n in Eq. (2.2) using only the reverberant signal itself,

and then we’ll design a filter which will undo (deconvolve) the original filter’s work and return theaudio signal to one without reverberation. The z-transform of such a filter is just the reciprocal ofthe transfer function that created the reverberant signal, namely

Page 94: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

76 CHAPTER 2. APPLIED ADAPTIVE FILTERING

1 − az−n (2.3)

which is a simple FIR–a modified comb filter (i.e., one having -1 < a < 1).The scripts (see exercises below)

LV xReverbDereverb(Delay, DecayRate, Mu, P eakSep)

and

LV xRevDerevShort (Delay, DecayRate, Mu, P eakSep, P rPC)

typical calls for which might be

LVxReverbDereverb(1450,0.7,0.05,400) (2.4)

and

LVxRevDerevShort(325,0.7,0.04,200,30)

generate a reverberative audio signal using an IIR and then perform an analysis of the signal usingautocorrelation to estimate the delay and adaptive filtering to estimate the decay rate. When thesetwo parameters have been determined, the algorithm deconvolves the reverberative audio signal,returning the original audio signal without reverberation.

The parameter Delay corresponds to n in Eq. (2.2); DecayRate is the factor or coefficient a inEq. (2.2), Mu is used in an LMS adaptive filtering process which estimates the value of DecayRate;and finally, PeakSep specifies the number of samples that must lie between any two adjacent peakvalues which are picked from the autocorrelation sequence to estimate the value of Delay. All ofthese variables will be discussed below in further detail.

Figure 2.20 shows the reverberative audio signal at (a), which was formed by convolving anaudio signal (drwatsonSR4K.wav in this case) with an IIR having a single nonzero delay coefficienthaving DecayRate = 0.7 and Delay =1450 samples. At (b), the autocorrelation sequence of thereverberated audio signal is shown, and at (c), the dereverberated (or deconvolved) signal is shown.The peaks in the autocorrelation sequence are 1450 samples apart.

2.8.2 SPECTRAL EFFECT OF REVERBERATIONNow that we have both the original audio signal and the reverberated version, let’s take a look at thespectral effect of imparting reverberation to the audio signal. One or more delayed versions of anaudio signal, when mixed with the original signal at a given listening point (by acoustic superposition,for example) result in a distortion or filtering of the frequency spectrum of the composite audio signal.The same is true here, where the original audio signal is repeatedly mixed with itself at a basic delayof 1450 samples, for example.

Page 95: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

2.8. DECONVOLUTION OF A REVERBERATIVE SIGNAL 77

0 0.5 1 1.5 2

x 104

−1

0

1

Am

plitu

de

(a) Sample, Conv Signal; Delay = 0.3625 Sec.

0 0.5 1 1.5 2

x 104

−1

0

1

Am

plitu

de

(c) Sample, Deconvolved Audio Signal

0 500 1000 1500 2000 2500 3000 3500 4000−1

0

1

Am

plitu

de

(b) AutoCorr Seq; Est/Delay = 0.3625 Sec.

Figure 2.20: (a) Reverberative audio signal, formed by passing original audio signal through a simpleIIR; (b) Autocorrelation sequence of reverberative audio signal; (c) Audio signal after being deconvolvedusing estimates of the IIR’s delay and decay parameters using autocorrelation and LMS adaptive filtering.

0 20 40 60 80 100 1200

50

Mag

nitu

de

(a) Frequency (Hz), Orig Audio Signal

0 20 40 60 80 100 1200

50

100

Mag

nitu

de

(b) Freq (Hz), Orig Sig Conv w/ Xfer Fcn 1/(1 − 0.7e−jω1450)

0 20 40 60 80 100 1200

2

4

Mag

nitu

de

(c) Freq Resp (Hz), Xfer Fcn 1/(1 − 0.7e−jω1450)

Figure 2.21: (a) Frequency response of original (nonreverberated) audio signal; (b) Frequency responseof reverberated audio signal; (c) Frequency response (evaluated using the z-transform) of IIR filter usedto generate the reverberative signal.

Page 96: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

78 CHAPTER 2. APPLIED ADAPTIVE FILTERING

Plots (a) and (b) of Fig. 2.21 show the frequency response of the signal and the reverberatedsignal up to about 135 Hz.

It can be seen from Fig. 2.21, plots (b) and (c), that the IIR’s frequency response is impartedto the audio signal, as expected. We can observe a comb filter-like effect. Recall that a comb filter isimplemented with an FIR. Here, we see that an IIR can create a peaking effect with its poles, butthe transfer function, without zeros in the numerator, does not drop to zero at any frequency. Youmay also notice that the IIR peaks are narrow, with wide valleys, whereas a comb filter effect hasbroad peaks and narrow valleys. One looks more or less like a vertically flipped version of the other.

2.8.3 ESTIMATING DELAYNext, we’ll discuss estimating the time delay that went into making the reverberative sound. Weare pretending, of course, that we have been given only the reverberative audio signal and theinformation that it was created using a simple IIR. In a realistic situation, of course, the impulseresponse or process that creates the reverberative signal would be much more complex; it is onlyartificial reverberators that can create reverb signals that have perfectly regularly spaced echos. Anactual acoustic environment generates, in general, an infinite number of irregularly spaced echos thedelay times and attenuation factors of which depend on the room dimensions, the room contents, thelocations of the source and listening point, and the reflective properties of the room and everything init. It is very instructive, however, to see how one can estimate a basic delay and attenuation applyingsimple techniques that we have discussed previously.

The autocorrelation sequence is used extensively in speech analysis for generating an accuratefrequency (pitch) estimate of a speech signal with a reasonable amount of computation. First, wecompute the autocorrelation sequence of the reverberative signal, and then take the absolute value,as shown by the following m-code:

b = xcorr(ReverbSnd);bAbs = abs(b);

Plot (b) of Fig. 2.20 shows a portion of the autocorrelation sequence as computed based onthe m-code above. Figure 2.22, plot (a), shows just the most useful portion of the right half of thatsequence.The distance in samples between major peaks is 1450. If the value of DecayRate is negative,every other major peak has a negative value, as shown in Fig. 2.22, plot (b), in which the value ofDecayRate was -0.7 rather than +0.7.

These observations suggest a method of estimating the value of Delay, namely, computing theseparation in samples between the two highest peaks in the absolute value of the autocorrelationsequence, the computation of which is performed by the following lines of m-code:

[locabsPeaks, absvalPeaks] = LVfindPeaks(bAbs,2,PeakSep);absestDelay = abs(locabsPeaks(1) - locabsPeaks(2));

The first line of m-code above finds the two highest magnitude peaks and their locations inthe signal bAbs which are at least PeakSep samples apart; in this case, PeakSep = 400. PeakSep is aparameter which can be estimated by eye if one doesn’t have any idea of the actual value of Delay,

Page 97: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

2.8. DECONVOLUTION OF A REVERBERATIVE SIGNAL 79

0 1000 2000 3000 4000 5000 6000 7000−0.5

0

0.5

1

Am

p

(a) Sample

0 1000 2000 3000 4000 5000 6000 7000

−0.5

0

0.5

1

Am

p

(b) Sample

Figure 2.22: (a) Autocorrelation sequence of reverberated audio signal when DecayRate = +0.7; (b) Au-tocorrelation sequence of reverberated audio signal when DecayRate = -0.7.

which is what we are generally supposing is the case. For purposes of keeping things simple, though,the script allows you to specify the value of PeakSep, which you may do by knowing what valueof Delay you are using. PeakSep must be less than the value of Delay. To estimate PeakSep by eye,you would plot the autocorrelation sequence and examine it for peaks, noting the distance betweenwhat appear to be suitable candidate peaks, and then perform the estimation of Delay by callingLVfindPeaks with PeakSep specified as about half the observed candidate peak separation (details ofthe function LVfindPeaks are in the exercises below).

2.8.4 ESTIMATING DECAYRATEThe process that created ReverbSnd was an IIR filter; therefore, the inverse or deconvolution processis an FIR filter the z-transform of which is the reciprocal the z-transform of the IIR, as shownabove. We have obtained the value for Delay, so the z-transform would be

Output (z)

Input (z)= 1 − az−Delay

which leads to

Page 98: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

80 CHAPTER 2. APPLIED ADAPTIVE FILTERING

Output (z) = Input (z) − a · Input (z) · z−Delay

and after conversion to a difference equation yields

Output[n] = Input[n] − a · Input[n − Delay] (2.5)

or in m-code

DeconOutput(Ctr) = ReverbSnd(Ctr)-a*ReverbSnd(Ctr-estDelay)

This can be treated as an adaptive process in which the deconvolved output, DeconOutput(Ctr),is considered as an error signal arising from the difference between a signal ReverbSnd (Ctr) whichis the convolution of the original signal and an unknown system’s impulse response, and an adaptivefilter output a·ReverbSnd (Ctr - estDelay) in which there is a single weight a to be determined.We aretaking an earlier sample from ReverbSnd (earlier by estDelay) and trying to find the proper weightingcoefficient a which will minimize the error. This works well when the later sample of ReverbSndhas no signal content except the delayed or reverberated earlier sound, which we are attemptingto remove. When the later sample already has content (which is usually the case, except for pausesbetween words), the situation is much the same as in echo cancelling with Near End speech: thereis a signal contributing to the error signal, but which is not also present in the filter input. Thus, aDual-H system is warranted, which automatically detects the absence of content (equivalent to noNear End speech in echo cancelling) by yielding a high value of the ratio of power into the adaptivefilter (in this case samples of the reverberated sound) to power output from the error junction.This issimilar to the ERLE and IRLE parameters discussed earlier in the chapter; here, we’ll call it ReverbReturn Loss Enhancement, or RRLE.

The method we are employing to remove the echo or reverberation from the original signalis very analogous to sparse echo cancellation, which was discussed earlier in the chapter. Note thatwe used correlation to estimate the relevant tap delay, which was (based on a certain amount ofknowledge of the echo or reverberative process), a single delay. We then use only the one tap in theadaptive filter. If we had estimated the delay at, say, 900 samples, we could have used an adaptivefilter having, say, 1000 active taps, covering any echo components occurring in the time delay rangeof 0 to 1000 samples. This, however, not only would result in far more computational overhead, butwould likely produce inferior results.

Figure 2.23, plot (a), shows the sequence of estimates of the single coefficient being estimated,the value of DecayRate during a trial run of script call (2.4), while plot (b) shows the correspondingsequence of values of RRLE.

2.8.5 DECONVOLUTIONAfter both Delay and DecayRate have been estimated, the values are used in a deconvolution filterwhich has as its z-transform the reciprocal of the z-transform of the IIR which was used to generatethe reverberated audio signal. The relevant lines of m-code are:

Page 99: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

2.9. REFERENCES 81

0 0.5 1 1.5 2

x 104

−0.2

0

0.2

0.4

0.6

0.8

(a) Sample, Tap Wt (DecayRate); Est = 0.69958

Am

plitu

de

0 0.5 1 1.5 2

x 104

0

50

100

Mag

, dB

(b) Sample, RRLE. (Max(RRLE) = 84.1036 dB)

Figure 2.23: (a) Estimated value of DecayRate (the amplitude of the pole of the IIR used to formthe reverberative audio signal); (b) Reverb Return Loss Enhancement (RRLE), used to tell the LMSAdaptive filter when to update the coefficient.

deconImp = [1,zeros(1,estDelay-1),-estDecayRate];DereverbSnd = filter(deconImp,1,ReverbSnd)

If all has gone well, the dereverberated audio signal will be identical or nearly so to the originalaudio signal.

2.9 REFERENCES

[1] Bernard Widrow and Samuel D. Stearns, Adaptive Signal Processing, Prentice-Hall, EnglewoodCliffs, New Jersey, 1985.

[2] Paul M. Embree and Damon Danieli, C++ Algorithms for Digital Signal Processing, Prentice-HallPTR, Upper Saddle River, New Jersey, 1999.

[3] Ali H. Sayed, Fundamentals of Adaptive Filtering, John Wiley & Sons, Hoboken, New Jersey,2003.

[4] John R. Buck, Michael M. Daniel, and Andrew C. Singer, Computer Explorations in Signalsand Systems Using MATLAB, Second Edition, Prentice-Hall, Upper Saddle River, New Jersey 07548,2002.

Page 100: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

82 CHAPTER 2. APPLIED ADAPTIVE FILTERING

[5] Vinay K. Ingle and John G. Proakis, Digital Signal Processing Using MATLAB V.4, PWS Pub-lishing Company, Boston, 1997.

[6] U. S. Patent 7,107,303 to Kablotsky et al, Sparse Echo Canceller, September 12, 2006.

[7] U. S. Patent 7,181,001 to Demirekler et al, Apparatus and Method for Echo Cancellation, February20, 2007.

2.10 EXERCISES1. Write the m-code to implement the following script, as described and illustrated in the text. Testit using at least the given test calls.

function LVxLMSANCNorm(PlantCoeffVec,k,Mu,freq,DVMult)% This script implements a 10-tap LMS adaptive filter in an% Active Noise Cancellation (ANC) application, such as cancell-% ing noise in a duct.% PlantCoeffVec is a row vector of 10 coefficients for the FIR that% simulates the Plant (i.e., the Duct impulse response).% k specifies the standard deviation of random noise to be mixed% with two sinusoids having frequencies freq and 3*freq and% respective amplitudes of 1.0 and 0.5;% Mu specifies the tap weight update damping coefficient;% An NLMS algorithm is used; its effectiveness can be tested% by giving the test signal various stepped-amplitude profiles% with the input argument DVMult, which is a vector of% amplitudes to impose on the test signal as a succession of% equally spaced amplitude segments over a test signal of 15% times the filter length.% Test calls:% LVxLMSANCNorm([0,0,1,0,-0.5,0.6,0,0,-1.2,0],2,2,27,[1,2,5,8])% LVxLMSANCNorm([0,0,1,0,-0.5,0.6,0,0,-1.2,0],2,0.2,27,[1,2,5,8])% LVxLMSANCNorm([0,0,1,0,-0.5,0.6,0,0,-1.2,0],0,2,3,[1,2,5,8])

2. Write the m-code to implement the following script, as described and illustrated in the text:

function LVxModelPlant(A, LenLMS, NoPrZs, NoPrPs, Mu,...tSig, NAmp, NoIts)

% A and B are a set of IIR and FIR coefficients to generate the% Plant response LenLMS is the number of LMS adaptive FIR% coefficients to use to model the IIR; it should generally be long% enough to model the impulse response until it has mostly% decayed away% NoPrZs is the number of Prony zeros to use to model the

Page 101: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

2.10. EXERCISES 83

% LMS-derived impulse response (the converged LMS adaptive% filter coefficients, which form a truncated version of the Plant% impulse response); NoPrPs is the number of Prony poles to use% to model the LMS-derived impulse response; Mu is the standard% LMS update term weight; tSig if passed as 0 = white noise;% 1 = DC; 2 = Nyquist; 3 = Half-band; 4 = mixed cosines (DC,% Half-band, and Nyquist)% NAmp = amplitude of noise to add to selected tSig% NoIts is the number of iterations to do in the adaptive process.% NoIts should be at least 10 times LenLMS.

Test the script with at least the following calls:

% LVxModelPlant([1,-0.9],[1],100, 2,2, 0.5,0,0,1000)% LVxModelPlant([1, 0,0.81],[1],100, 3,3,0.5,1, 0,1000)% LVxModelPlant([1,-1.3,0.81],[1],100, 3,3,0.5,1, 0,1000)% LVxModelPlant([1,-1,0.64],[1],100, 3,3, 0.5, 1, 0,1000)

3. Write the m-code for the following script, which provides a basic illustration of echo cancellationusing Single- and Dual-H methods, using a user-selectable percentage of the reduced-bandwidthaudio files ’drwatsonSR4K.wav’ and ’whoknowsSR4K.wav’. Detailed instructions are given after thefunction definition.

function LVxLMSAdptFiltEchoShort(k,Mu,DHorSH,...MuteNrEnd,PrPC)% k is the amount of noise to add to the Far End signal (see below)% Mu is the usual LMS update term weight;% DHorSH yields a Dual-H system if passed as 1, or a Single-H% system if passed as 0; MuteNrEnd mutes the desired signal% (’drwatsonSR4K’) at its very beginning if passed as 1, about% one-third of the way through if passed as 2, or no muting at% all if passed as 0.% The Near End signal consists of the audio file ’drwatsonSR4K.wav’% The Far End signal consists of the audio file ’whoknowsSR4K.wav’% plus random noise weighted by k, limited to the length of the% Near End signal;% The total amount of computation can be limited by processing% only a percentage of the audio signals, and that percentage% is passed as the input variable PrPC.% Using whoknowsSR4K.wav and drwatsonSR4K.wav as test signals,% 100%=22184 samps. The adaptive FIR is ten samples long% and the echo is simulated as a single delay of 6 samples,% for example. The final filtered output signal is interpolated

Page 102: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

84 CHAPTER 2. APPLIED ADAPTIVE FILTERING

% by a factor of 2 using the function interp to raise its sample% rate to 8000 Hz so it may be played using the call sound% (EchoErr,8000) after making the call global EchoErr in% the Command window. A single figure is created showing% the filtered output signal% Test calls:% LVxLMSAdptFiltEchoShort(0.1,0,0,0,50) % Mu = 0, FarEnd hrd% LVxLMSAdptFiltEchoShort(0.1,0.2,0,0,50) % S-H, no Mute% LVxLMSAdptFiltEchoShort(0.1,0.2,0,1,50) % S-H, Mute Imm% LVxLMSAdptFiltEchoShort(0.1,0.2,1,0,50) % D-H, no Mute% LVxLMSAdptFiltEchoShort(0.1,0.2,1,1,50) % D-H, Mute Imm% LVxLMSAdptFiltEchoShort(0.1,0.2,1,2,50) % D-H, Mute del% LVxLMSAdptFiltEchoShort(0.01,0.2,0,0,50) % S-H, no Mute% LVxLMSAdptFiltEchoShort(0.01,0.2,0,1,50) % S-H, Mute Imm% LVxLMSAdptFiltEchoShort(0.01,0.2,1,0,50) % D-H, no Mute% LVxLMSAdptFiltEchoShort(0.01,0.2,1,1,50) % D-H, Mute Imm% LVxLMSAdptFiltEchoShort(0.01,0.2,1,2,50) % D-H, Mute del% LVxLMSAdptFiltEchoShort(0.01,0.005,1,0,60) % Low Mu, can% hear Far End gradually diminish

Since this script is intended to execute as quickly as possible, the detection of Near End speechhas been eliminated, and archiving and plotting of ERLE as well. The code should be written toautomatically play the resultant filtered output at least once. Insert in the m-code the statement

global EchoErr

where EchoErr is the filtered output signal.Then, in the Command Window, type global EchoErrafter the program has run, and you can then play the filtered output at will by making the call

sound(EchoErr, 8000)

In order to avoid retaining older (unwanted) computed values in a global variable, it is goodpractice is to insert the global statement early in the script prior to computing and assigning newvalues to the variable, and to follow the global statement immediately with a statement to empty thevariable, i.e.,

global EchoErrEchoErr = [];

Since the net filtered output is the delayed Far End Signal plus the Near End signal, minusthe FIR output, the delayed Far End signal can be added to the Near End Signal prior to the loopfiltering operation, resulting in only one array access rather than two to compute the filtered output.The coefficient update normalizing term should be two times Mu, divided by the sum of the squares

Page 103: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

2.10. EXERCISES 85

of the signal in the FIR, and this, too, can be computed ahead of the loop and looked up while inthe filtering loop.

Since the audio files were sampled at 4 kHz, the final filtered output signal must be interpolatedby a factor of two so that its net final sample rate is 8000 Hz (LabVIEW only allows certain samplerates to be used in the sound function, namely, 8000, 11025, 22050, and 44100 Hz).

4. Write the m-code to implement the script LVxLMSAdaptFiltEcho, as described and illustratedin the text. The preceding exercise created a much simpler script, which does not use the variablesDblTkUpD, SampsVOX, VOXThresh, and which requires a much smaller computation load. Thisproject adds these variables, and archives ERLE and several other variables for purposes of plotting.The preceding project can serve as a good starting basis for this project.

function LVxLMSAdaptFiltEcho(k,Mu,DHorSH,MuteNrEnd,...DblTkUpD,SampsVOX,VOXThresh)% k is the amount of noise to add to the interference signal% (’drwatsonSR8K.wav’); Mu is the usual LMS update term% weight; DHorSH yields a Dual-H system if passed as 1, or% a Single-H system if passed as 0; MuteNrEnd mutes the% desired signal (’drwatsonSR8K.wav’) at its very beginning% if passed as 1, about one-third of the way through if passed% as 2, or no muting at all if passed as 0. In Single-H mode,% DblTkUpD, if passed as 0, allows coefficient updating any% time, but if passed as 1 prevents coefficient update when% the most recent SampsVOX samples of the desired (Near% End) signal level are above VOXThresh. In Dual-H mode,% DblTkUpD,SampsVOX,VOXThresh are ignored.% The Far End signal is the audio file ’whoknowsSR8K.wav’,% limited to the length of ’drwatsonSR8K/wav’% The adaptive FIR is ten samples long and the echo is% simulated as a single delay of 6 samples, for example.% A figure having four subplots is created. The first subplot% shows the values of all ten FIR coefficients over the entire% course of computation; the second subplot shows the filtered% output signal; the third subplot shows as a binary plot% whether or not coefficient update was allowed at any given% stage of the computation based on the detection of Near% End speech; the fourth subplot shows the figure of merit% (ERLE, as defined in the text) over the course of the% computation. The filtered output audio signal may be played% using the call sound(EchoErr,8000) after making the call% global EchoErr in the Command window. The Near End plus

Page 104: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

86 CHAPTER 2. APPLIED ADAPTIVE FILTERING

% echoed Far End signal (the signal entering the error junction)% can be played with the call sound(NrEndPlusFarEnd,8000)% after making the call global NrEndPlusFarEnd.

Test the script with the following calls, then repeat the calls, changing k to 0.2; compare theaudibility of the Far End signal between the same calls using different amounts of noise as specifiedby k.

% LVxLMSAdaptFiltEcho(0.02,0.2,1,0,[],[],[]) % D-H, no Mute% LVxLMSAdaptFiltEcho(0.02,0.2,1,1,[],[],[]) % D-H, Mute Im% LVxLMSAdaptFiltEcho(0.02,0.2,1,2,[],[],[]) % D-H, del Mute% LVxLMSAdaptFiltEcho(0.02,0.2,0,0,0,50,0.05) % S-H, dbltalk% update, no Mute% LVxLMSAdaptFiltEcho(0.02,0.2,0,1,1,50,0.05) % S-H, no% dbltalk update, Mute Im

5. Write the m-code to implement the following script, as described and illustrated in the text:

function LVxLMSAdaptFiltDeCorr(k,Mu,freq,NoTaps,...DeCorrDelay)% k is an amount of white noise to add to a test% signal which consists of a sinusoid of frequency freq;% Mu is the LMS update term weight;% NoTaps is the number of taps to use in the LMS adaptive filter;% DeCorrDelay is the number of samples to delay the filter input% relative to the Plant or channel delay.

Test the script with at least the following calls:

% LVxLMSAdaptFiltDeCorr(0.1,0.75,200,20,5)% LVxLMSAdaptFiltDeCorr(0.1,0.75,200,20,0)

6. Write the m-code to implement the script LVxLMSInterferCancShort as specified below.

function LVxLMSInterferCancShort(k,Mu,DHorSH,MuteDesSig,...GP2,GP4,DP1,DP2,DP3,DP4,NumTaps,PrPC)% k is the amount of noise to add to the interference signal% (’whoknowsSR4K.wav’); Mu is the usual LMS% update term weight;% DHorSH yields a Dual-H system if passed as 1, or a Single-H% system if passed as 0. MuteDesSig mutes the desired signal% (’drwatsonSR4K.wav’) at its very beginning if passed as 1,% about one-third of the way through if passed as 2, or no muting% at all if passed as 0; GP2 is the gain in the path from the desired% sound source to the noise reference microphone, and GP4 is the% gain in the path from the noise source to the desired signal

Page 105: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

2.10. EXERCISES 87

% microphone;% DP1 is the Delay in samples from the desired sound source to% the desired sound source microphone; DP2 is the Delay from% the desired sound source to the noise microphone; DP3 is the% Delay from the noise source to the noise source microphone;% DP4 is the Delay from the noise source to the desired sound% microphone. NumTaps is the number of Delays or taps to use% in the adaptive filter. The total amount of computation can be% limited by processing only a percentage of the audio signals,% and that percentage is passed as the input variable PrPC.% Using whoknowsSR4K.wav and drwatsonSR4K.wav as test% signals, 100%=22184 samps. The final filtered output signal% is interpolated by a factor of 2 using the function interp to% raise its sample rate to 8000 Hz so it can be played using the% call sound(ActualErr,8000) after running the script and making% the call global ActualErr in the Command window% Typical calls might be:% LVxLMSInterferCancShort(0.02,0.3,1,0,1,1,1,6,1,6,5,30)% LVxLMSInterferCancShort(0.02,0.3,1,0,0.16,0.16,...7,42,7,42,35,30)% LVxLMSInterferCancShort(0.02,0.3,1,0,0.06,0.06,...1,6,1,6,5,30)% LVxLMSInterferCancShort(0.02,0.3,1,0,1,1,6,1,1,6,5,30)% LVxLMSInterferCancShort(0.02,0.3,1,1,1,1,1,6,1,6,5,30)% LVxLMSInterferCancShort(0.02,0.2,0,2,1,1,1,6,1,6,5,30)% LVxLMSInterferCancShort(0.02,0.3,0,1,1,1,1,6,1,6,5,30)

7. Write the m-code to implement the following script, as described and illustrated in the text:

function LVxLMSInterferCancel(k,Mu,DHorSH,MuteDesSig,...DblTkUpD,NoSampsVOX,...% VOXThresh,GP2,GP4,DP1,DP2,DP3,DP4,NumTaps)% k is the amount of noise to add to the interference signal% (’whoknowsSR8K.wav’). Mu is the usual LMS update term% weight; DHorSH yields a Dual-H system if passed as 1, or% a Single-H system if passed as 0. MuteDesSig mutes the% desired signal (’drwatsonSR8K.wav’) at its very beginning% if passed as 1, about one-third of the way through if passed% as 2, or no muting at all if passed as 0; DblTkUpD, if passed% as 0, allows coefficient updating any time, but if passed as 1% prevents coefficient update when the most recent NoSamps

Page 106: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

88 CHAPTER 2. APPLIED ADAPTIVE FILTERING

% VOX samples of the desired signal level are above VOXThresh.% GP2 is the gain in the path from the desired sound source to% the noise reference microphone, and GP4 is the gain in the% path from the noise source to the desired signal microphone;% DP1 is the Delay in samples from the desired sound source% to the desired sound source microphone;% DP2 is the Delay from the desired sound source to the noise% microphone; DP3 is the Delay from the noise source to the% noise source microphone; DP4 is the Delay from the noise% source to the desired sound microphone. NumTaps is the% number of Delays or taps to use in the adaptive filter.% The filtered audio output can be played using the call% sound(ActualErr,8000) after running the script and making% the call global ActualErr in the Command window.

Test the script with at least the following calls:

% LVxLMSInterferCancel(0.02,0.3,1,0,0,50,0.03,1,1,1,6,1,6,5)% LVxLMSInterferCancel(0.02,0.3,1,0,0,50,0.03,0.16,...0.16,7,42,7,42,35)% LVxLMSInterferCancel(0.02,0.3,1,0,1,50,0.03,0.06,...0.06,1,6,1,6,5)% LVxLMSInterferCancel(0.02,0.3,1,0,0,50,0.03,1,1,6,1,1,6,5)% LVxLMSInterferCancel(0.02,0.3,1,1,0,50,0.03,1,1,1,6,1,6,5)% LVxLMSInterferCancel(0.02,0.2,0,2,1,80,0.03,1,1,1,6,1,6,5)% LVxLMSInterferCancel(0.02,0.3,0,1,1,50,0.03,1,1,1,6,1,6,5)

8. A certain signal transmission channel has the transfer function

H(z) = 1 + 0.9z−1 + 0.8z−2

Under ideal convergence conditions, what would be the shortest adaptive FIR that could,when well-converged, result in an equalized channel transfer function having a magnitude responsewithin 0.1 dB of unity?

9. Write the m-code to implement the following script, as described and illustrated in the text,creating a figure with subplots as shown, for example, in Fig. 2.17:

Page 107: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

2.10. EXERCISES 89

function LVxChannEqualiz(ChImp,Mu,tSig,lenLMS)% ChImp is the impulse response of the channel which is to% be equalized by the LMS FIR;% Mu is the usual update weight% tSig designates the type of test signal to be used:% 0 = random noise% 1 = Unit impulse sequence of length lenLMS, repeated at% least ten times, with alternating sign, +, -, +, etc% 2 = Unit impulse sequence of length lenLMS, repeated at% least ten times% 3 = Sinusoid at the half-band frequency% lenLMS is the length of the adaptive LMS FIR

Try two different values for the delay (as shown in Fig. 2.16), namely, (1) one-half the sumof the lengths of the Channel Impulse Response and the LMS FIR, and (2) one-half the length ofthe Channel Impulse Response. In both cases, round the value obtained in (1) or (2) to the nearestintegral value. Test the script with at least the following calls, trying each of the two delay values toobtain the better response.

% LVxChannEqualiz([-0.05,0,-0.5,0,-1,0,1,0,0.5,0,0.05],1.5,2,131)% LVxChannEqualiz([1,0,1],2.1,2,91) % notch% LVxChannEqualiz([1,0,1],2.4,2,191) % notch% LVxChannEqualiz([1 1],2.2,2,65) % lowpass% LVxChannEqualiz([1,0,-1],1.8,2,85) % bandpass% LVxChannEqualiz([1,-1],2.5,2,179) % highpass% LVxChannEqualiz([1,-0.8],2.3,2,35) % highpass% LVxChannEqualiz([1,0.8],2.4,2,35) % lowpass% LVxChannEqualiz([0.3,1,0.3],2.3,2,9) % lowpass% LVxChannEqualiz([1,0.7],2.3,2,25) % lowpass% LVxChannEqualiz([1,0,0,0,0,0,1],1.6,2,179) % comb filter% LVxChannEqualiz([1,0,0,-0.3,0,0,0.05],2.5,2,23)% LVxChannEqualiz([1,0,-1],1.5,2,55)% LVxChannEqualiz( ([1,0,1,0,1,0,1].*hamming(7)’),2.1,2,91)

10. Write the m-code for the script LVxRevDerevShort, which is a shorter version of the scriptLVxReverbDereverb (see exercise immediately following).

function LVxRevDerevShort(Delay,DecayRate,Mu,PeakSep,PrPC)% Delay is the number of samples of delay used in the single% feedback stage to generate the reverberated sound file.% DecayRate is a real number which is feedback gain.% Mu is the usual LMS update weight term.% PeakSep the number of samples by which to separate

Page 108: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

90 CHAPTER 2. APPLIED ADAPTIVE FILTERING

% detected peaks in the autocorrelation sequence of the% reverberated sound. Uses a Dual-H architecture to estimate% DecayRate, and uses the best estimate of DecayRate to% block filter the reverberated sound to produce the% dereverberated sound. Does not archive or plot the values% of RRLE and current best estimate or DecayRate. Plays% the reverberated sound once and the dereverberated% sound once. These sounds may be played again after% running the script by making the calls global ReverbSnd% and global DereverbSnd in the Command window and then% making the calls sound(ReverbSnd,8000) and sound% (DereverbSnd,8000) at will. Processes only a percentage% of the audio file equal to PrPC in the adaptive filtering% loop to estimate DecayRate. Uses the audio file% ’drwatsonSR4K.wav’ as the test signal and at the end% both the reverberative sound and the dereverberated% sound are interpolated to a sample rate of 8000 Hz to% be played by the function sound(x,Fs),which only allows% Fs = 8000, 11025, 22050, or 44100.% Sample call:% LVxRevDerevShort(325,0.7,0.05,200,30)% LVxRevDerevShort(225,0.7,0.12,200,30)% LVxRevDerevShort(625,0.7,0.06,200,47)% LVxRevDerevShort(1625,0.7,0.07,200,47)

You will need to extract peak locations for the autocorrelation; suitable code is found below,and the script is provided with the basic software for this book. Figure 2.24 shows the result of thecall

[Locs,Pks] = LVfindPeaks((cos(2*pi*2*[0:1:63]/64)),3,16)

with the highest three peaks marked with circles. Note that a zone of ineligible values is set uparound each peak that has already been chosen, otherwise peaks might be chosen right next to eachother when that is not the desire.

One thing that makes the adaptive LMS algorithm run much more quickly is to recognizethe fact that the times when RRLE should increase are when the Near End signal is close to zero inmagnitude; such times form a contiguous set of samples.Thus, in stepping through the algorithm, itcan be done by using every fifth sample, for example, rather than every sample, and whenever RRLEincreases, the algorithm goes back to computing the update coefficients for every sample rather thanevery fifth sample. In this way, only a little more than 20% of the samples are used to compute thenext estimate of DecayRate and the corresponding value of RRLE.

Page 109: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

2.10. EXERCISES 91

0 10 20 30 40 50 60 70−1

−0.8

−0.6

−0.4

−0.2

0

0.2

0.4

0.6

0.8

1

Sample

Am

plitu

de

Figure 2.24: Two cycles of a cosine wave over 64 samples with the highest 3 peaks marked with circles(see text).

function [Locs, Pks] = LVfindPeaks(Mat,QuanPks,PeakSep)% Mat is a vector or matrix for which QuanPks peak values% are sought, each of which is at least PeakSep samples% distant from the next closest peak value. Locs tells the% locations of the peak values in the matrix. To make adjacent% samples of Mat eligible to be peaks, use PeakSep = 0.% Sample call:% [Locs, Pks] = LVfindPeaks([0:1:7],8,0)% [Locs, Pks] = LVfindPeaks([0:1:7],8,1)% [Locs, Pks] = LVfindPeaks((cos(2*pi*3*[0:1:63]/64)),4,8)

11. Write the m-code to implement the following script, as described and illustrated in the text:

function LVxReverbDereverb(Delay,DecayRate,Mu,PeakSep)% Delay is the number of samples of delay used in the single

Page 110: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

92 CHAPTER 2. APPLIED ADAPTIVE FILTERING

% feedback stage to generate the reverberated sound file.% DecayRate is a real number which is feedback gain.% Mu is the usual LMS update weight term.% PeakSep the number of samples by which to separate% detected peaks in the autocorrelation sequence of the% reverberated sound. Uses a Dual-H architecture to estimate% DecayRate, and uses the best estimate of DecayRate% from the Dual-H process along with the estimated value% of Delay (from autocorrelation) in an FIR to deconvolve% the entire reverberative signal. Archives and plots the% values of RRLE and current best estimate of DecayRate.% Plays the dereverberated sound once and the reverberative% sound once. These sounds may be played again after% running the script by making the calls global ReverbSnd% and global DereverbSnd in the Command window and then% making the calls sound(ReverbSnd,8000) and sound% (DereverbSnd,8000) at will. The script uses the audio file% ’drwatsonSR8K.wav’ as the test signal.

Test the script with at least the following calls:

% LVxReverbDereverb(650,0.7,0.05,400)% LVxReverbDereverb(3600,0.975,0.05,400)

12. Write a script that generates a test signal by reading the audio file ’drwatsonSR8K.wav’ andadding an interfering sinusoid having amplitude A and frequency Freq to it. The script will thenuse LMS adaptive filtering with a decorrelating delay to remove the sinusoidal tone from the audiosignal. Code suitable to create the test signal is

[y,Fs,bits] = wavread(’drwatsonSR8K.wav’);y = y’; lenFile = length(y);y = (1/max(abs(y)))*y;t = [0:1:lenFile-1]/Fs;LMSDeCorDataVec = y + A*sin(2*pi*t*Freq);

The display figure for the script should be like Fig. 2.25, including four subplots that showthe input signal (audio file corrupted by a large amplitude sinusoid), the filtered output signal, andthe magnitude of the spectrum of each, computed by FFT. Figure 2.25 was generated by makingthe call

LVxLMSDeCorrHeterodyne(’drwatsonSR8K.wav’,1.5,0.08,500,45,1)

Your script should conform to the syntax below; test the script with the given test calls.

Page 111: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

2.10. EXERCISES 93

0 1 2 3 4

x 104

−1

−0.5

0

0.5

1

Am

plitu

de

(a) Sample, Input Sig

0 1 2 3 4

x 104

−0.5

0

0.5

Am

plitu

de

(c) Sample, Output/Err

0 500 1000 1500 2000

−100

−50

0

Mag

, dB

(b) Input Signal Frequency, Hz

0 500 1000 1500 2000

−60

−40

−20

0

Mag

, dB

(d) Output/Error Frequency, Hz

Figure 2.25: (a) Input signal, audio file ’drwatsonSR8K’ corrupted with a large amplitude sinusoid;(b) Spectrum of signal in (a); (c) Filtered (output or error) signal, showing original audio signal with novisible interfering sinusoid; (d) Spectrum of signal in (c). Note the deep spectral notch at the frequencyof the interfering sinusoid as shown in plot (b).

function LVxLMSDeCorrHeterodyne(strWavFile,A,Mu,Freq,...NoTaps,DeCorrDelay)% strWavFile is the audio file to open and use as the test signal% A is the amplitude of an interfering tone (sinusoid) to be added% to the audio file whose file name or path name (including file% name and extension) is specified as strWavFile. Mu is the LMS% update term weight; NoTaps is the number of taps to use in% the LMS adaptive filter; DeCorrDelay is the number of samples% to delay the filter input relative to the Plant or channel delay.% The following plots are created: Frequency response (DTFT)% of the adaptive filter when converged (magnitude v. frequency% in Hz), the test signal (amplitude v. sample), the filtered test% signal (amplitude v. sample), the DTFT of the test signal% (magnitude v. frequency in Hz, and the DTFT (magnitude v.% frequency in Hz) of the last 50 % of the filtered test signal..% Test calls:

Page 112: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

94 CHAPTER 2. APPLIED ADAPTIVE FILTERING

% LVxLMSDeCorrHeterodyne(’drwatsonSR8K.wav’,...0.5,0.08,250,45,1)% LVxLMSDeCorrHeterodyne(’drwatsonSR8K.wav’,...0.5,0.08,500,45,1)% LVxLMSDeCorrHeterodyne(’drwatsonSR8K.wav’,...0.5,0.08,1200,45,1)% LVxLMSDeCorrHeterodyne(’drwatsonSR8K.wav’,...0.5,0.01,75,45,10)%% LVxLMSDeCorrHeterodyne(’drwatsonSR8K.wav’,...1.5,0.08,250,45,1)% LVxLMSDeCorrHeterodyne(’drwatsonSR8K.wav’,...1.5,0.08,500,45,1)% LVxLMSDeCorrHeterodyne(’drwatsonSR8K.wav’,...1.5,0.08,1200,45,1)% LVxLMSDeCorrHeterodyne(’drwatsonSR8K.wav’,...1.5,0.01,75,45,10)%% LVxLMSDeCorrHeterodyne(’drwatsonSR8K.wav’,...4.5,0.08,250,45,1)% LVxLMSDeCorrHeterodyne(’drwatsonSR8K.wav’,...4.5,0.08,500,45,1)% LVxLMSDeCorrHeterodyne(’drwatsonSR8K.wav’,...4.5,0.08,1200,45,1)% LVxLMSDeCorrHeterodyne(’drwatsonSR8K.wav’,...4.5,0.01,75,45,10)%% LVxLMSDeCorrHeterodyne(’drwatsonSR8K.wav’,...0.02,0.08,250,45,1)% LVxLMSDeCorrHeterodyne(’drwatsonSR8K.wav’,...0.02,0.08,500,45,1)% LVxLMSDeCorrHeterodyne(’drwatsonSR8K.wav’,...0.02,0.08,1200,45,1)% LVxLMSDeCorrHeterodyne(’drwatsonSR8K.wav’,...0.02,0.01,75,45,10)

Use the script LVx_AnalyzeModWavFile (developed in the exercises for the chapter devotedto FIR filter design methods in Volume III of the series) to make the test file

drwat8Kplus400HzAnd740Hz.wav

Page 113: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

2.10. EXERCISES 95

for use in the following test call. The amplitudes of the 400 Hz and 740 Hz waves shouldbe specified as 0.02 when creating the file ’drwat8Kplus400HzAnd740Hz.wav’ using the scriptLVx_AnalyzeModWavFile.

% LVxLMSDeCorrHeterodyne(’drwat8Kplus400HzAnd740Hz.wav’,...0,0.01,0,50,10)% LVxLMSDeCorrHeterodyne(’drwat8Kplus400HzAnd740Hz.wav’,...0.02,0.01,1100,50,10)

13. Write a script that is essentially the same as LVxLMSDeCorrHeterodyne with the exceptionthat the new script will use a Dual-H architecture. You may notice that the script LVxLMSDeC-orrHeterodyne does not work well unless the persistent tone is of relatively high amplitude. Whenthe amplitude of the persistent tone is relatively low, performance of the adaptive filter is poor, lead-ing to distortion and inadequate removal of the persistent tone. You should find that the Dual-Harchitecture will function much better with low level persistent tones than does the simple Single-H architecture of the script LVxLMSDeCorrHeterodyne. The test calls presented with the functionspecification below are presented in pairs, with the same parameters being used in both scripts,LVxLMSDeCorrHeterodyne and LVxLMSDeCorrHetDualH, for comparison.

function LVxLMSDeCorrHetDualH(strWavFile,A,Mu,Freq,...NoTaps,DeCorrDelay)% Has the same input arguments and function as% LVxLMSDeCorrHeterodyne, but uses a Dual-H% architectures for better performance.% strWavFile is the audio file to open and use as the test signal% A is the amplitude of an interfering tone (sinusoid) to be% added to the audio file whose file name or path name% (including file name and extension) is specified as% strWavFile.% Mu is the LMS update term weight;% NoTaps is the number of taps to use in the LMS adaptive filter;% DeCorrDelay is the number of samples to delay the filter input% relative to the Plant or channel delay.% The following plots are created: Frequency response (DTFT)% of the adaptive filter when converged (magnitude v.% frequency in Hz), the test signal (amplitude v. sample),% the filtered test signal (amplitude v. sample), the DTFT% of the test signal (magnitude v. frequency in Hz, and% the DTFT (magnitude v. frequency in Hz) of the last% 50 % of the filtered test signal.%% Test calls (pairs to be compared with each other):

Page 114: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

96 CHAPTER 2. APPLIED ADAPTIVE FILTERING

%% LVxLMSDeCorrHeterodyne(’drwatsonSR8K.wav’,...0.012,0.03,850,65,7)% LVxLMSDeCorrHetDualH(’drwatsonSR8K.wav’,...0.012,0.03,850,65,7)%% LVxLMSDeCorrHeterodyne(’drwat8Kplus400HzAnd740Hz.wav’,...0,0.025,0,73,3)% LVxLMSDeCorrHetDualH(’drwat8Kplus400HzAnd740Hz.wav’,...0,0.025,0,73,3)%% LVxLMSDeCorrHeterodyne(’drwat8Kplus400HzAnd740Hz.wav’,...0.02,0.025,1150,73,3)% LVxLMSDeCorrHetDualH(’drwat8Kplus400HzAnd740Hz.wav’,...0.02,0.025,1150,73,3)

The calls

LVxLMSDeCorrHeterodyne(’drwat8Kplus400HzAnd740Hz.wav’,...0.04,0.025,1150,73,3)LVxLMSDeCorrHetDualH(’drwat8Kplus400HzAnd740Hz.wav’,...0.04,0.025,1150,73,3)

result in Figs. 2.26 and 2.27, respectively, which show the spectrum of the test signals and filteredtest signals for the Single-H and Dual-H versions of the persistent-tone removal script.

The call

LVxLMSDeCorrHetDualH(’drwat8Kplus400HzAnd740Hz.wav’,...0.04,0.025,1150,400,7)

results in Fig. 2.28, in addition to the other figures specified in the function description. Note thepeaks in the filter response at the persistent frequencies, and recall that the filtered signal is derivedby subtracting the LMS filter output from the test signal.

Page 115: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

2.10. EXERCISES 97

0 1 2 3 4

x 104

−1

−0.5

0

0.5

1

Am

plitu

de

(a) Sample, Input Sig

0 1 2 3 4

x 104

−0.5

0

0.5

Am

plitu

de

(c) Sample, Output/Err

0 500 1000 1500 2000

0.2

0.4

0.6

0.8

1

Mag

(b) Input Signal Frequency, Hz

0 500 1000 1500 2000

0.2

0.4

0.6

0.8

1

Mag

(d) Output/Error Frequency, Hz

Figure 2.26: (a) Test speech signal having persistent tones added at frequencies of 440 Hz, 740 Hz, and1150 Hz; (b) Spectrum of test signal in (a); (c) Output/Error signal, i.e., the test signal after being filteredby a Single-H LMS system having a decorrelating delay designed to assist in removing persistent tones;(d) Spectrum of signal in (c).

0 1 2 3 4

x 104

−1

−0.5

0

0.5

1

Am

plitu

de

(a) Sample, Input Sig

0 1 2 3 4

x 104

−0.5

0

0.5

Am

plitu

de

(c) Sample, Output/Err

0 500 1000 1500 2000

0.2

0.4

0.6

0.8

1

Mag

(b) Frequency, Hz, Input Signal

0 500 1000 1500 2000

0.2

0.4

0.6

0.8

1

Mag

(d) Frequency, Hz, Output/Error

Figure 2.27: (a) Test speech signal having persistent tones added at frequencies of 440 Hz, 740 Hz,and 1150 Hz; (b) Spectrum of test signal at (a); (c) Output/Error signal, i.e., the test signal after beingfiltered by a Dual-H LMS system using a decorrelating delay designed to assist in removing persistenttones; (d) Spectrum of Output/Error signal. Note the much improved attenuation of the persistent tonescompared to that of the previous figure.

Page 116: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

98 CHAPTER 2. APPLIED ADAPTIVE FILTERING

0 500 1000 1500 2000 2500 3000 3500 4000

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Mag

Adaptive Filter Response, Hz

Figure 2.28: Frequency response of the best set of tap weights for the length-400 LMS adaptive filterthat is computed to remove simultaneous persistent tones at frequencies of 400 Hz, 740 Hz, and 1150Hz.

Page 117: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

99

A P P E N D I X A

Software for Use with this BookA.1 FILE TYPES AND NAMING CONVENTIONSThe text of this book describes many computer programs or scripts to perform computations thatillustrate the various signal processing concepts discussed. The computer language used is usuallyreferred to as m-code (or as an m-file when in file form, using the file extension .m) in MATLAB-related literature or discussions, and as MathScript in LabVIEW-related discussions (the termsare used interchangeably in this book).

The MATLAB and LabVIEW implementations of m-code (or MathScript) differ slightly(Lab-VIEW’s version, for example, at the time of this writing, does not implement Handle Graphics,as does MATLAB).

The book contains mostly scripts that have been tested to run on both MATLAB and Lab-VIEW; these scripts all begin with the letters LV and end with the file extension .m. Additionally,scripts starting with the letters LVx are intended as exercises, in which the student is guided to writethe code (the author’s solutions, however, are included in the software package and will run whenproperly called on the Command Line).

Examples are:

LVPlotUnitImpSeq.m

LVxComplexPowerSeries.m

There are also a small number m-files that will run only in MATLAB, as of this writing.Theyall begin with the letters ML. An example is:

ML_SinglePole.m

Additionally, there are a number of LabVIEW Virtual Instruments (VIs) that demonstratevarious concepts or properties of signal processing. These have file names that all begin with theletters Demo and end with the file extension .vi. An example is:

DemoComplexPowerSeriesVI.vi

Finally, there are several sound files that are used with some of the exercises; these are all inthe .wav format. An example is:

drwatsonSR4K.wav

Page 118: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

100 APPENDIX A. SOFTWARE FOR USE WITH THIS BOOK

A.2 DOWNLOADING THE SOFTWAREAll of the software files needed for use with the book are available for download from the followingwebsite:

http://www.morganclaypool.com/page/isen

The entire software package should be stored in a single folder on the user’s computer, and thefull file name of the folder must be placed on the MATLAB or LabVIEW search path in accordancewith the instructions provided by the respective software vendor.

A.3 USING THE SOFTWAREIn MATLAB, once the folder containing the software has been placed on the search path, any scriptmay be run by typing the name (without the file extension, but with any necessary input argumentsin parentheses) on the Command Line in the Command Window and pressing Return.

In LabVIEW, from the Getting Started window, select MathScript Window from the Toolsmenu, and the Command Window will be found in the lower left area of the MathScript window.Enter the script name (without the file extension, but with any necessary input arguments in paren-theses) in the Command Window and press Return. This procedure is essentially the same as thatfor MATLAB.

Example calls that can be entered on the Command Line and run are

LVAliasing(100,1002)

LV_FFT(8,0)

In the text, many “live” calls (like those just given) are found. All such calls are in boldfaceas shown in the examples above. When using an electronic version of the book, these can usuallybe copied and pasted into the Command Line of MATLAB or LabVIEW and run by pressingReturn. When using a printed copy of the book, it is possible to manually type function calls into theCommand Line, but there is also one stored m-file (in the software download package) per chapterthat contains clean copies of all the m-code examples from the text of the respective chapter, suitablefor copying (these files are described more completely below in the section entitled "Multi-linem-code examples"). There are two general types of m-code examples, single-line function calls andmulti-line code examples. Both are discussed immediately below.

A.4 SINGLE-LINE FUNCTION CALLSThe first type of script mentioned above, a named- or defined-function script, is one in which afunction is defined; it starts with the word "function" and includes the following, from left to right:

Page 119: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

A.5. MULTI-LINE M-CODE EXAMPLES 101

any output arguments, the equal sign, the function name, and, in parentheses immediately followingthe function name, any input arguments. The function name must always be identical to the filename. An example of a named-function script, is as follows:

function nY = LVMakePeriodicSeq(y,N)% LVMakePeriodicSeq([1 2 3 4],2)y = y(:); nY = y*([ones(1,N)]); nY = nY(:)’;

For the above function, the output argument is nY, the function name is LVMakePeriodicSeq,and there are two input arguments, y and N, that must be supplied with a call to run the func-tion. Functions, in order to be used, must be stored in file form, i.e., as an m-file. The functionLVMakePeriodicSeq can have only one corresponding file name, which is

LVMakePeriodicSeq.m

In the code above, note that the function definition is on the first line, and an example callthat you can paste into the Command Line (after removing or simply not copying the percent signat the beginning of the line, which marks the line as a comment line) and run by pressing Return.Thus you would enter on the Command Line the following, and then press Return:

nY = LVMakePeriodicSeq([1,2,3,4],2)

In the above call, note that the output argument has been included; if you do not want thevalue (or array of values) for the output variable to be displayed in the Command window, place asemicolon after the call:

nY = LVMakePeriodicSeq([1,2,3,4],2);

If you want to see, for example, just the first five values of the output, use the above code tosuppress the entire output, and then call for just the number of values that you want to see in theCommand window:

nY = LVMakePeriodicSeq([1,2,3,4],2);nY1to5 = nY(1:5)

The result from making the above call is

nY1to5 = [1,2,3,4,1]

A.5 MULTI-LINE M-CODE EXAMPLESThere are also entire multi-line scripts in the text that appear in boldface type; they may or may notinclude named-functions, but there is always m-code with them in excess of that needed to make asimple function-call. An example might be

Page 120: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

102 APPENDIX A. SOFTWARE FOR USE WITH THIS BOOK

N=54; k = 9; x = cos( 2*pi*k*( 0:1:N-1 )/N);LVFreqResp(x, 500)

Note in the above that there is a named-function (LVFreqResp) call, preceded by m-code todefine an input argument for the call. Code segments like that above must either be (completely)copied and pasted into the Command Line or manually typed into the Command Line. Copy-and-Paste can often be successfully done directly from a pdf version of the book. This often resultsin problems (described below), and accordingly, an m-file containing clean copies of most m-codeprograms from each chapter is supplied with the software package. Most of the calls or multi-linem-code examples from the text that the reader might wish to make are present in m-files such as

McodeVolume1Chapter4.m

McodeVolume2Chapter3.m

and so forth. There is one such file for each chapter of each book, except Chapter 1 of Volume I,which has no m-code examples.

A.6 HOW TO SUCCESSFULLY COPY-AND-PASTE M-CODEM-code can usually be copied directly from a pdf copy of the book, although a number of minor,easily correctible problems can occur. Two characters, the symbol for raising a number to a power,the circumflex ˆ, and the symbol for vector or matrix transposition, the apostrophe or single quotemark ’, are coded for pdf using characters that are non-native to m-code. While these two symbolsmay look proper in the pdf file, when pasted into the Command line of MATLAB, they will appearin red.

A first way to avoid this copying problem, of course, is simply to use the m-code files de-scribed above to copy m-code from. This is probably the most time-efficient method of handlingthe problem—avoiding it altogether.

A second method to correct the circumflex-and-single-quote problem, if you do want to copydirectly from a pdf document, is to simply replace each offending character (circumflex or singlequote) by the equivalent one typed from your keyboard. When proper, all such characters will appearin black rather than red in MATLAB. In LabVIEW, the pre-compiler will throw an error at the firstsuch character and cite the line and column number of its location. Simply manually retype/replaceeach offending character. Since there are usually no more than a few such characters, manuallyreplacing/retyping is quite fast.

Yet a third way (which is usually more time consuming than the two methods described above)to correct the circumflex and apostrophe is to use the function Reformat, which is supplied with thesoftware package. To use it, all the copied code from the pdf file is reformatted by hand into onehorizontal line, with delimiters (commas or semicolons) inserted (if not already present) where lineshave been concatenated. For example, suppose you had copied

Page 121: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

A.7. LEARNING TO USE M-CODE 103

n = 0:1:4;y = 2.ˆnstem(n,y);

where the circumflex is the improper version for use in m-code. We reformat the code intoone horizontal line, adding a comma after the second line (a semicolon suppresses computed outputon the Command line, while a comma does not), and enclose this string with apostrophes (or singlequotes), as shown, where Reformat corrects the improper circumflex and eval evaluates the string,i.e., runs the code.

eval(Reformat(’n=0:1:4;y=2.ˆn;stem(n,y)’))

Occasionally, when copying from the pdf file, essential blank spaces are dropped in the copiedresult and it is necessary to identify where this has happened and restore the missing space. Acommon place that this occurs is after a “for” statement.The usual error returned when trying to runthe code is that there is an unmatched “end” statement or that there has been an improper use ofthe reserved word “end”. This is caused by the elision of the “for” statement with the ensuing codeand is easily corrected by restoring the missing blank space after the “for” statement. Note that thefunction Reformat does not correct for this problem.

A.7 LEARNING TO USE M-CODEWhile the intent of this book is to teach the principles of digital signal processing rather thanthe use of m-code per se, the reader will find that the scripts provided in the text and with thesoftware package will provide many examples of m-code programming starting with simple scriptsand functions early in the book to much more involved scripts later in the book, including scripts foruse with MATLAB that make extensive use of MATLAB objects such as push buttons, edit boxes,drop-down menus, etc.

Thus the complexity of the m-code examples and exercises progresses throughout the bookapace with the complexity of signal processing concepts presented. It is unlikely that the readeror student will find it necessary to separately or explicitly study m-code programming, although itwill occasionally be necessary and useful to use the online MATLAB or LabVIEW help files forexplanation of the use of, or call syntax of, various built-in functions.

A.8 WHAT YOU NEED WITH MATLAB AND LABVIEWIf you are using a professional edition of MATLAB, you’ll need the Signal Processing Toolboxin addition to MATLAB itself. The student version of MATLAB includes the Signal ProcessingToolbox.

If you are using either the student or professional edition of LabVIEW, it must be at leastVersion 8.5 to run the m-files that accompany this book, and to properly run the VIs you’ll need theControl Design Toolkit or the newer Control Design and Simulation Module (which is included inthe student version of LabVIEW).

Page 122: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is
Page 123: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

105

A P P E N D I X B

Vector/Matrix Operations inM-Code

B.1 ROW AND COLUMN VECTORS

Vectors may be either row vectors or column vectors. A typical row vector in m-code might be [3-1 2 4] or [3,-1,2, 4] (elements in a row can be separated by either commas or spaces), and wouldappear conventionally as a row:

[3 −1 2 4

]The same, notated as a column vector, would be [3,-1,2,4]’ or [3; -1; 2; 4], where the semicolon

sets off different matrix rows:⎡⎢⎢⎣

3−124

⎤⎥⎥⎦

Notated on paper, a row vector has one row and plural columns, whereas a column vectorappears as one column with plural rows.

B.2 VECTOR PRODUCTSB.2.1 INNER PRODUCTA row vector and a column vector of the same length as the row vector can be multiplied two differentways, to yield two different results. With the row vector on the left and the column vector on theright,

[1 2 3 4

]⎡⎢⎢⎣

4321

⎤⎥⎥⎦ = 20

corresponding elements of each vector are multiplied, and all products are summed. This is calledthe Inner Product. A typical computation would be

[1, 2, 3, 4] ∗ [4; 3; 2; 1] = (1)(4) + (2)(3) + (3)(2) + (4)(1) = 20

Page 124: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

106 APPENDIX B. VECTOR/MATRIX OPERATIONS IN M-CODE

B.2.2 OUTER PRODUCTAn Outer Product results from placing the column vector on the left, and the row vector on theright:

⎡⎢⎢⎣

4321

⎤⎥⎥⎦ [

1 2 3 4] =

⎡⎢⎢⎣

4 8 12 163 6 9 122 4 6 81 2 3 4

⎤⎥⎥⎦

The computation is as follows:

[4; 3; 2; 1] ∗ [1, 2, 3, 4] = [4, 3, 2, 1; 8, 6, 4, 2; 12, 9, 6, 3; 16, 12, 8, 4]Note that each column in the output matrix is the column of the input column vector, scaled

by a column (which is a single value) in the row vector.

B.2.3 PRODUCT OF CORRESPONDING VALUESTwo vectors (or matrices) of exactly the same dimensions may be multiplied on a value-by-valuebasis by using the notation " .* " (a period followed by an asterisk). Thus two row vectors or twocolumn vectors can be multiplied in this way, and result in a row vector or column vector having thesame length as the original two vectors. For example, for two column vectors, we get

[1; 2; 3]. ∗ [4; 5; 6] = [4; 10; 18]and for row vectors, we get

[1, 2, 3]. ∗ [4, 5, 6] = [4, 10, 18]

B.3 MATRIX MULTIPLIED BY A VECTOR OR MATRIXAn m by n matrix, meaning a matrix having m rows and n columns, can be multiplied from the rightby an n by 1 column vector, which results in an m by 1 column vector. For example,

[1, 2, 1; 2, 1, 2] ∗ [4; 5; 6] = [20; 25]Or, written in standard matrix form:

[1 2 12 1 2

] ⎡⎣ 4

56

⎤⎦ =

[48

]+

[105

]+

[6

12

]=

[2025

](B.1)

An m by n matrix can be multiplied from the right by an n by p matrix, resulting in an m byp matrix. Each column of the n by p matrix operates on the m by n matrix as shown in (B.1), andcreates another column in the n by p output matrix.

Page 125: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

B.4. MATRIX INVERSE AND PSEUDO-INVERSE 107

B.4 MATRIX INVERSE AND PSEUDO-INVERSEConsider the matrix equation

[1 43 −2

] [a

b

]=

[ −23

](B.2)

which can be symbolically represented as

[M] [V ] = [C]

or simply

MV = C

and which represents the system of two equations

a + 4b = −2

3a − 2b = 3

that can be solved, for example, by scaling the upper equation by -3 and adding to the lower equation

−3a − 12b = 6

3a − 2b = 3

which yields

−14b = 9

or

b = −9/14

and

a = 4/7

The inverse of a matrix M is defined as M−1 such that

MM−1 = I

where I is called the Identity matrix and consists of all zeros except for the left-to-right downslopingdiagonal which is all ones. The Identity matrix is so-called since, for example,

Page 126: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

108 APPENDIX B. VECTOR/MATRIX OPERATIONS IN M-CODE

[1 00 1

] [a

b

]=

[a

b

]

The pseudo-inverse M−1 of a matrix M is defined such that

M−1M = I

System B.2 can also be solved by use of the pseudo-inverse[M−1

][M] [V ] =

[M−1

][C]

which yields

[I ] [V ] = V =[M−1

][C]

In concrete terms, we get

[M−1

] [1 43 −2

] [a

b

]=

[M−1

] [ −23

](B.3)

which reduces to

[a

b

]=

[M−1

] [ −23

]

We can compute the pseudo-inverse M−1 and the final solution using the built-in MathScriptfunction pinv:

M = [1,4;3,-2];P = pinv(M)ans = P*[-2;3]

which yields

P =[

0.1429 0.28570.2143 −0.0714

]

and therefore

[a

b

]=

[0.1429 0.28570.2143 −0.0714

] [ −23

]

which yields a = 0.5714 and b = -0.6429 which are the same as 4/7 and -9/14, respectively. A uniquesolution is possible only when M is square and all rows linearly independent.(a linearly independentrow cannot be formed or does not consist solely of a linear combination of other rows in the matrix).

Page 127: DSP for MATLAB™ and LabVIEW™read.pudn.com/downloads154/ebook/684601/1598298992.pdf · Spectral Analysis of Signals Yanwei Wang,Jian Li,and Petre Stoica 2006. ... This book is

109

BiographyForester W. Isen received the B.S. degree from the U. S. Naval Academy in 1971 (majoring in

mathematics with additional studies in physics and engineering), and the M. Eng. (EE) degree fromthe University of Louisville in 1978, and spent a career dealing with intellectual property mattersat a government agency working in, and then supervising, the examination and consideration ofboth technical and legal matters pertaining to the granting of patent rights in the areas of electronicmusic, horology, and audio and telephony systems (AM and FM stereo, hearing aids, transducerstructures, Active Noise Cancellation, PA Systems, Equalizers, Echo Cancellers, etc.). Since retiringfrom government service at the end of 2004, he worked during 2005 as a consultant in databasedevelopment, and then subsequently spent several years writing the four-volume series DSP forMATLAB and LabVIEW, calling on his many years of practical experience to create a book onDSP fundamentals that includes not only traditional mathematics and exercises, but “first principle”views and explanations that promote the reader’s understanding of the material from an intuitive andpractical point of view, as well as a large number of accompanying scripts (to be run on MATLABor LabVIEW) designed to bring to life the many signal processing concepts discussed in the series.