Download - Adaptive Signal Processing ||
Texts and Monographs in Computer Science
Editor
David Gries
Advisory Board F. L. Bauer
J. J. Horning R. Reddy
D. C. Tsichritzis W. M. Waite
Texts and Monographs in Computer Science
Suad A!agi~ Relalional Dala~ TKhnolog,y
Suad Alagi, and Michael A. A!bib The DHign 0( Wt ll-Slruclured and CorrKI Programs
S. Thomas Aleundcr Adapli~e SiGnal Procn.sinK: n,,~"OI"Y and Applkalions
Michael A. A!bib. A. J . Kfoury. aoo Robc:n N. Moll A Ba5is ror TMoRlkal Compuler Sc~n~
Michael A. A!bib aoo EITII'~t G. MallCS Algt braicc Approaches 10 Program Semanl i<:s
F. L. Bauer aoo H. Wiissner AlllO'"ilhmic LanKua~ and Program lltvdOJlmenl
K3a~ Chrislian TM Guide 10 Modula-2
Edsge , W. Dijkstra Selected Wriling~ on ComputinG' A ~rwnal PfnpKtio'e
Nis~im France~
F"air~
Peter W. F~y. Ed. Chess Skill in Man , nd Machine. 2nd t:dition
R. T . Gregory and E. V. Kfi~hnamunhy
Method, and Appl iealiof1$ or F.nOl"-~'r ... CompU Utliotl
David Gries. Ed . Programming Melhodology: A CollK lion 0( Artk~ by Mi:'mben 0( It' IP WG2.J
David Gries The Sc~""e or Prognmming
A. J. Kfoury. Robc:n N. Moll. and Michael A. Amib A Prognmming Approach 10 Compulabilily
E. V. Kris/mamunhy Error-Free Polynomial Malrix CompuUtlions
Franeo P. Prcpardra and Michael Ian Sharnos CompuUtlional Gtomttry: An InlrodlK'lion
Brian Randel!. Ed. The Origins 0( Digila l Compule",: Seledtd Papen
Ano Salomaa and Matti !ini ttol.
Aulomala_Thf<H'<::lk A, pKlS or rCH"mal Power Series
k((~y R Sampson AdaptiYe InrCH"malion PrOtt·.,.inl' An InlrodlK'lCH"y Sun'ey
William M Waile and Gcrttard Goos Compiler Con~lrucllon
Niklaus Winh Programming in Modula ·2
, Adaptive Signal Processing
Theory and Applications
s. Thomas Alexander
With 42 Illustrations
Springer·Verlag New York Berlin Heidelberg London Paris Tokyo
S. Thomas Alexander Department of Electrical and
Computer Engineering North Carolina State University Raleigh, NC 27695-79 11 USA.
Series Editor
David Gries Department of Computer Science Cornell Universi ty Upson Hall Ithaca, NY 14853 U.S.A.
Library of Congress Cataloging in Publication Data Alexander, S. Thomas.
Adaptive signal processing. (Texts and monographs in computer science) Bibliography: p. Includes index. 1. Adaptive signal processing. I. Title.
II. Series. TK5 102.5.A424 1986 621.38'043 86-13956
© 1986 by Springer-Verlag New York Inc. All rights reserved. No part of this book may be Iranslated or reproduced in any form without written pennission from Springer-Verlag, 115 Fifth Avenue, New York, New York 10010, U.S.A.
Typeset by Asco Trade Typesetting Ltd., Hong Kong. Printed and bound by R.R. Donncl1ey & Sons. Harrisonburg, Virginia. Printed in the United States of America.
981 6 5 4 3 2 I
ISBN 0-387-96380-4 Springer-Verlag New York Berlin Heidelberg ISBN 3-540-96380-4 Springer-Verlag Berlin Heidelberg New York
Preface
The creation of the text really began in 1976 with the author being involved with a group of researchers at Stanford University and the Naval Ocean Systems Center, San Diego. At that time, adaptive techniques were more laboratory (and mental) curiosities than the accepted and pervasive categories of signal processing that they have become. Over the lasl 10 years, adaptive filters have become standard components in telephony, data communications, and signal detection and tracking systems. Their use and consumer acceptance will undoubtedly only increase in the future.
The mathematical principles underlying adaptive signal processing were initially fascinating and were my first experience in seeing applied mathematics work for a paycheck. Since that time, the application of even more advanced mathematical techniques have kept the area of adaptive signal processing as exciting as those initial days. The text seeks to be a bridge between the open literature in the professional journals, which is usually quite concentrated, concise, and advanced, and the graduate classroom and research environment where underlying principles are often more important.
In that spirit, this text will be most beneficially used as an introductory tool for anyone interested in learning the fascinating field of adaptive signal processing. Most of the intended audience will be seniors and graduate students in electrical engineering or computer science, although the practicing engineer "gearing up" to work on product development using adaptive techniques will also find the text useful. An understanding of linear systems, digital signal processing, and matrix algebra approximately equivalent to that of an undergraduate electrical engineering curriculum is adequate. The text has been used for a graduate course in adaptive signal processing at North Carolina State University, in which students from a wide variety of backgrounds have actively participated.
vi Preface
The main distinction between this text and others that have appeared on the subject is the inclusion in this text of the timely subject of vector space approaches to fast adaptive filtering. This is currently one of the most active areas of research in signal processing, but the mathematical sophistication required to understand the open literature in the area has been formidable. This text develops the vector space approach through the liberal use of geometrical analogies, which encompasses Chapters 9- t 1. In so doing. the vector space approach becomes actually very easy to understand and possesses a great deal of simple elegance. After completing these chapters, the reader will be well prepared to tackle some of the more specific research problems associated with fast adaptive techniques. The book can be an effective text for a one-semester course in adaptive signal processing or as a reference for the researcher (academic or industrial) to absorb material at his own pace. Problems that have survived the classroom experience are included at the end of the chapters.
This text is approximately the same process by which I became familiar with the different areas of adaptive signal processing. Much of the original material was literally "back of the envelope" information from hearing conference talks and informal discussions. Other portions had their beginning as notes scribbled in the margin of books or papers when something finally jelled.
As with any book, the contributions of many people over many years were instrumental to the entire process. Specifically, J would like to thank the following: Bob Plemmons of North Carolina State for sharing his mastery of linear algebra; Lloyd Griffiths of USC for long runs during which philosophies were discussed; Ed SalOrius of JPL and Joel Trussell of North Carolina Stale for consistently providing honest and, therefore, valuable technical evaluation and discussion on both this text and the Big Picture; John Cioffi of Stanford for his contributions to my own understanding of geometrical approaches and fast adaptive techniques; Nino Masnari, Chairman of the Electrical and Computer Engineering Department at North Carolina State, for helping to foster the professional environment that allowed the time and resources to develop this text; and the editorial staff at Springer-Verlag for providing the assistance and encouragement I needed to successfully complete the manuscript. Additionally, for their unsung efforts in reading and debugging the original drafts and homework problems, I would like to thank the following at North Carolina State: Gary Ybarra, Glenda Poston, Zong Rhee, Daehoon Kim, and Randy Avent. Special thanks are also ex.tended to Peggy Ball, Liz Story, George Winston, and Red Ryder.
Finally, the city of Boston and the season of Winter had a lot to do with the whole process.
Raleigh, NC S.T. ALEXANDER
Contents
Preface
CHAPTER 1 Introduction
1.1 Signal Processing in Unknown Environments 1.2 Two Examples 1.3 Outline of the Text
References
CHAPTER 2 The Mean Square Error (MSE) Performance Criteria
2.1 Introduction 2.2 Mean Square Error (MSE) and MSE Surface 2.3 Properties of the MSE Surface: 2.4 The Normal Equations 2.5 Further Geometrical Properties of the Error Surfaces
Problems References
CHAPTER 3 Linear Prediction and the lattice Structure
3.1 Introduction 3.2 Durbin's Algorithm 3.3 Lattice Derivation
Problems References
,
I 2 5 7
8
8 9
16 22 24 30 32
3' 3' 35 40 44 45
viii
CHAPTER 4 The Method of Steepest Descent
4.1 Int roduction 4.2 Iterative Solution of the Normal Equations 4.3 Weight Vector Solutions 4.4 Convergence Properties of Steepest Descent 4.5 Mean Square Error Propagation
Problems References
CHAPTER S The Leaat Mean Squares (LMS) Algorithm
5. 1 Introduction 5.2 Effects of Unknown Signal Statistics 5.3 Derivation of the LMS Algorithm 5.4 Convergence of the LMS Algorithm 5.5 LM S Mean Sq uare Error Propagation
Problems References
CHAPTER 6 Applications of the LMS Algorithm
6.1 Int roduction 6.2 Echo Cancellation 6.3 Adaptive Waveform Coding 6.4 Adaptive Spectrum Analysis
References
CHAPTER 7 Gradient Adaptive LattIce Methods
7.1 Int roduction 7.2 Lattice Renection Coefficient Computation 7.3 Adaptive Lattice Derivations 7.4 P erformance Example
Problems References
CHAPTER 8 RecursIve Least Squares Signal Processing
8.1 Introduction 8.2 The Recursive Least Squares Filter 8.3 CompUlational Complexity
Problems References
Contents
46
46 47 52 59 62 65 66
68
68 70 73 74 79 83 85
87
87 88 90 93 98
99
99 100 104 107 108 109
III
II I 113 119 12 1 12 1
Contents
CHAPTER 9 Vector Spaces for RLS Filters
9.1 Introduction 9.2 Linear Vector Spaces 9.3 The Least Squares Filter and Projection Matrices 9.4 Least Squares Update Relations 9.5 Projection Matrix Time Update
Problems References
CHAPTER 10 The least Squares lattice Algorithm
10.1 Introduction 10.2 Forward and Backward Prediction Filte rs 10.3 The LS Lattice Structure 10.4 Lattice Order and Time Updates 10.5 Examples of LS Lattice Performance
Problems References
CHAPTER 11 Fast Transversal Filters
11.1 Introduction 11.2 Additional Vector Space Relations 11.3 The Transversal Filter Operator Update 11.4 The FTF Time Updates 11.5 Further Computational Reductions
Problems References
Index
ix
123
123 124 127 133 135 139 140
142
142 143 146 148 150 152 152
154
154 155 161 163 172 174 176
177
