Advanced Textbooks in Control and Signal Processing

Springer-Verlag London Ltd.

Series Editors

Professor Michael J. Grimble, Professor ofIndustrial Systems and DirectorProfessor Michael A. Johnson, Professor ofControl Systems and Deputy DirectorIndustrial Control Centre, Department ofEleetronic and Electrical Engineering,UniversityofStrathdyde, Graham Hills Building, SO George Street, Glasgow GIIQE, U.K.

Other titles published in this series:

Genetic Algorithms: Concepts and DesignsK.F. Man, K.S. Tang and S. Kwong

Model Predictive ControlE. F. Camacho and C. Bordons

Introduction to Optimal EstimationE.W. Kamen and J. Su

Neural Networks for Modelling and Control ofDynamic SystemsM. N0rgaard, O. Ravn, L.K. Hansen and N.K. PoulsenPublication Due March 2000

Modelling and Control ofRobot Manipulators (2nd Edition)

L. Sciavicco and B. SicilianoPublication Due March 2000

Darrell Williamson

Discrete-time Signal Processing An Aigebraic Approach

With 44 Figures

, Springer

Darrell Williamson. PhD

Faculty ofEngineering and Information Technology, Australian National University. Canberra,. ACT 0200. Australia

ISBN 978-1-85233-161-0

British Library Cataloguing in Publication Data Williarnson, Darrell, 1948-

Discrete-time signal processing. - (Advanced textbooks in control and signal processing) I.Signal processing 2.Discrete-time systems I.Tide 621.3'82:2 ISBN 978-1-85233-161-0 ISBN 978-1-4471-0541-1 (eBook) DOI 10.1007/978-1-4471-0541-1

Library of Congress Cataloging-in-Publication Data A catalog record for this book is available from the Library of Congress

Apart li:om any fair dealing for the purposes of researeh or private sludy, or CIiticism or review. as penniued under the Copyright, Designs and Patents A,t 1988, this publication may oo1y be reproduced, stored or transmitted. in any form or by any means, with the prior permission in writing of the publishers. or in the case of reprographie reproduction in accordanee with the terms of Iicences issued by the Copyright Licensing Agency. Enquiries conceming reproduction outside those terms should be sent to the publishers.

© Springer-Verlag London 1999 Original1y publishcd by Springer-Verlag london l.imitcd in 1999

MA TLABII> is the registered trademark ofThe Math Works, Ine., htJp:llwww.mathwork.s.com

The use oe registered narnes. trademarks, ete. in this pub1ication does not imply, even in the absence of a specific statement. that such narnes are exempt from the relevant laws and regulations and therefore free for general use_

The publisher makes no representation. express or implied, with regard 10 the a"unq of the information contained in this book and cannot aeeept any legal responsibility or liability for any enors or omissions that may be made.

Typesetting: Camera ready by author

69/3830-543210 Printed on add-free paper SPIN 10715982

To Jan, Naomi, Pia, Leigh and Zoe

Series Editors' Foreword

The topics of control engineering and signal processing continue toflourish and develop. In common with general scientific investigation, new ideas,concepts and interpretations emerge quite spontaneously and these are thendiscussed, used, discarded or subsumed into the prevailing subject paradigm.Sometimes these innovative concepts coalesce into a new sub-discipline withinthe broad subject tapestry of control and signal processing. This preliminary battlebetween old and new usually takes place at conferences, through the internet andin the journals of the discipline. After a little more maturity has been acquired bythe new concepts then archival publication as a scientific or engineeringmonograph may occur.

The applications of signal processing techniques have grown and grown.They now cover the wide range from the statistical properties of signals and datathrough to the hardware problems of communications in all its diverse aspects.Supporting this range of applications is a body of theory, analysis and techniqueswhich is equally broad. Darrell Williamson has faced the difficult task oforganising this material by adopting an algebraic approach. This uses generalmathematical and systems ideas and results to form a firm foundation for thediscrete signal processing paradigm. Although this may require some extraconcentration and involvement by the student or researcher, the rewards are aclarity of presentation and deeper insight into the power of individual results. Anadditional benefit is that the algebraic language used is the natural language ofcomputing tools like MATLAB and its simulation facility, SIMULINK. Thus, thestep from analysis to demonstration and illustrative simulation is a shorter one.

The special bonus in the book is a rare chapter on finite wordlengthconsiderations. Of course, Darrell Williamson is the ideal author for such acontribution having previously written a specialist text on the subject. Thischapter is where design meets the implications of real-world implementationproblems and it is a very appropriate way to conclude the first signal processingvolume in the Advanced Textbooks in Control and Signal Processing Series.

M.J. Grimble and M.A. JohnsonIndustrial Control CentreGlasgow, Scotland, U.K.

July, 1999

Preface

This text provides an algebraic approach to both the analysis of discretetime signals, and the analysis and design of discrete time signal processingalgorithms. The material is presented with the use of algebraically basedsoftware design packages (such as MATLAB) in mind, and is written forstudents in their third and fourth years of an undergraduate engineeringprogram.

The text assumes that the reader has a working knowledge of complexnumbers, and has completed an introductory course in vector and matrixanalysis. This background is sufficient to cover the material in Chapters 1and 2 which in itself could be used as a introductory course in differenceequations. The material on digital filters presented in Chapter 3 is developedfrom first principles. However the subsequent material on analog filters as­sumes that the reader has completed an introductory course on differentialequations. The study of signal processing in Chapter 4 which begins withthe concepts of norm, inner product and orthogonality would benefit from anearlier course in analysis. Given this background, the coverage of the discreteFourier transform, and the least squares estimation estimation algorithms isthen appropriate. The final chapter on the finite wordlength implementationof a digital filter requires no further prerequisite material although the finaltwo sections of this chapter assumes an advanced understanding in linearalgebra.

Chapter 1 begins with a discussion of a number of applications in digitalsignal processing: market analysis, numerical integration, digital filter design,radar processing and speech compression. As well as providing motivation forsubsequent developments, these applications are useful as illustrations of par­ticular results. This introductory chapter provides some fundamental prop­erties of both discrete and continuous time signal including a brief consider­ation of what is meant by signal processing, and concludes with an overviewof hardware methods for digital-to-analog and analog-to-digital conversion.

Chapter 2 is concerned with the analysis of digital signals that can berepresented in the form of the solution of a linear time invariant differenceequation. The chapter begins with a detailed treatment of first and secondorder linear time invariant difference equations, and introduces the conceptsand significance of order, time invariance and linearity. The chapter then

x

proceeds directly to the consideration of a general linear time invariant dif­ference equation. In particular, it is shown that any linear time invariantdifference equation can be expressed as a system of first order linear timeinvariant difference equations known as the state space representation. Thisrepresentation is most appropriate for the solution of difference equations viaMATLAB.

Chapter 3 introduces the fundamental properties of both recursive (orIIR) and nonrecursive (or FIR) digital filters. All-pass, linear phase, lowpass,bandpass, bandstop and highpass filters are characterized. The first sectionof this Chapter considers some basic principles for the design of FIR digitalfilters. Conditions to guarantee a linear phase response are developed, andwindowing methods are then introduced to help in matching desired magni­tude characteristics. The second section of the chapter then focuses on thedesign of recursive digital filters, and begins with a brief coverage of analogfilters which includes the design of Butterworth, Chebyshev and Elliptic ana­log low pass filters. Based on this introduction, the design of a digital filter asan approximation of an analog filter is then developed. Finally, the design ofa lowpass, bandpass, bandstop or highpass digital filter as a transformationof a given lowpass digital filter is completed. The transformations from ananalog filter to a digital filter are all expressed algebraically as a transfor­mation of a state space representation of an analog filter to a state spacerepresentation of a digital filter, and as a consequence, can be implementedusing MATLAB. Likewise, the transformations from a given lowpass digitalfilter to another lowpass, or a highpass, or a bandpass, or a bandstop digi­tal filter are all expressed algebraically as a transformation of a state spacerepresentation of the given digital filter to a state space representation of therequired digital filter, and so, can also be implemented using MATLAB.

Chapter 4 begins by characterizing signals in terms of their size (or norm).An inner product is defined between two signals which then provides a basisfor characterizing orthogonal signals, and a method for establishing boundson the output signal and the state component signals of a digital filter in termsof a bound on the input signal. In particular, bounds are expressed in termsof the solution of a linear algebraic equation which is readily solvable usingMATLAB. An important representation of both periodic and finite lengthsignals in terms of orthogonal signals known as the Discrete Fourier Seriesrepresentation is then developed. The advantage of such a representation inapplications of signal processing becomes evident following the developmentof the fast Fourier transform algorithm. Finally, the theory of least squaresestimation is developed and applied to problems in filter design, and recursiveand nonrecursive signal estimation. These algorithms are expressed in termsof algebraic equations which can be implemented using MATLAB.

Chapter 5 considers the design of a recursive filter structure for the finitewordlength implementation of a digital filter which takes into account theeffects of input, coefficient and signal quantization errors. After covering the

XI

basic properties of both fixed and floating point arithmetic representations,this chapter considers the implication of the presence of arithmetic quantiza­tions on both the accuracy and the speed of the implementation of a digitalfilter. Performance is considered in terms of arithmetic error and overflow,and in this regard, the structure of the FWL implementation is shown to besignificant. Both low complexity and optimal filter structures are considered.Algebraic methods are developed for selecting the optimal filter structurewhich are readily implemented using MATLAB.

For the most part, the material is developed from first principles, and isbased on lecture notes delivered to undergraduate engineering students atthe Australian National University over a number of years. The material hasappeared in different ways in various texts including those listed below. Insuch cases, no explicit credit is assigned. However, when results are not fullydeveloped, an explicit reference is given. As has already been stated, thepresentation in this text has a strong algebraic flavour. Frequency domaincharacteristics are developed from first principles, but there are no chapterson either the z-transform or the Laplace transform since such material isnot needed. Instead a focus on linear algebraic concepts enables the readerto better integrate the analysis, design and implementation of digital signalprocessing algorithms with numerical simulation and design packages.

Supporting material in vector and matrix analysis can be found in one ormore of the following texts:

• M. Marcus and H. Minc, A Survey of Matrix Theory and Matrix Inequali­ties, Allyn and Bacon, Boston, 1964.

• R. Bellman, Introduction to Matrix Analysis, (2nd ed.), McGraw-Hill, NewYork, 1970.

• S. Barnett, Matrix Methods for Engineers and Scientists, McGraw-Hill,London, 1979.

• B. Noble and J.W. Daniel, Applied Linear Algebra, (3rd ed.), Prentice-Hall,Englewood Cliffs, N.J., 1988.

• G. Strang, Linear Algebra and Its Applications, Brace Jovanovich, SanDiego, 1988.

• R.A. Horn and C.R. Johnson, Topics in Matrix Analysis, Cambridge Uni­versity Press, Cambridge, 1991

Likewise, one or more of the following texts can be consulted for support­ing material in signal analysis, signal processing and filter design:

• L.R. Rabiner and B. Gold, Theory and Applications of Digital Signal Pro­cessing, Prentice-Hall, Englewood Cliffs, N.J., 1975.

• A.V. Oppenheim and R.V. Schafer, Digital Signal Processing, Prentice­Hall, Englewood Cliffs, N.J., 1975.

• S.M. Bozic, Digital and Kalman Filtering, Edward Arnold, 1979.

XII

• H.Y.F Lam, Analog and Digital Filters, Prentice-Hall, Englewood Cliffs,N.J., 1979.

• R.A. Gabel and R.A. Roberts, Signals and Linear Systems, Wiley, 1980.• W.D. Stanley, G.R. Dougherty and R. Dougherty, Digital Signal Process­

ing, Reston Pub!.,1984.• N.S. Jayant and P. Noll, Digital Coding of Waveforms, Prentice-Hall, En-

glewood Cliffs, N.J., 1984.• J.C. Cluley, Transducers for Microprocessors, MacMillan, London 1985.• T.W. Parks and C.S. Burrus, Digital Filter Design, Wiley, 1987.• M.F. Hordeski, Transducers for Automation, Van Nostrand, Reinholt,

N.Y., 1987.• J. R. Johnson, Introduction to Digital Signal Processing, Prentice-Hall, En­

glewood Cliffs, N.J., 1989.• S.S. Soliman and M.D. Srinath, Continuous and Discrete Signals and Sys­

tems, Prentice-Hall, 1990.• H. Kwakernaak and R. Sivan, Modern Signals and Systems, Prentice-Hall,

Englewood Cliffs, N.J., 1991• D. Williamson, Digital Control and Implementation, Prentice-Hall, Engle­

wood Cliffs, N.J., 1991.• R.E. Ziemer, W.H. Tranter and D.R. Fannin, Signals and Systems: Con­

tinuous and Discrete, MacMillan, London, 1993.• C-T Chen, System and Signal Analysis, Saunders College Pub!., 1994.• A.V. Oppenheim and A.S. Wilsky, Signals and Systems, Prentice-Hall, En­

glewood Cliffs, N.J., 1997.• E.W. Kamen and B.S. Heck, Fundamentals ofSignals and Systems, Prentice­

Hall, Englewood Cliffs, N.J., 1997.• P. Lapsley, J. Bier, A. Shoham and E.A. Lee, DSP Processor Fundamentals:

Architecture and Features, IEEE Press, Inc., 1997.

Contents

1. Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.1 Digital and Analog Signals. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21.2 Digital-to-Analog Conversion 91.3 Analog-to-Digital Conversion 121.4 Signal Processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 16Summary 29Exercises 31

2. Digital Signals. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 352.1 First Order. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 35

2.1.1 Zero Input and Forced Response 412.1.2 Steady State and Transient Response. . . . . . . . . . . . . .. 50

2.2 Second Order 532.2.1 Zero Input Response. . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 592.2.2 Unit Impulse Response. . . . . . . . . . . . . . . . . . . . . . . . . . .. 69

2.3 High Order. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 732.3.1 Zero Input Response 732.3.2 Stability Test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 79

2.4 Linear Convolution. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 812.4.1 Forced Response 822.4.2 Steady State Sinusoidal Response 85

2.5 State Space Representation . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 922.5.1 Second Order. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 922.5.2 High Order 1042.5.3 Steady State Sinusoidal Response 110

Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111Exercises 114

3. Digital Filters 1233.1 Overview.............................................. 1233.2 Design of FIR Filters 136

3.2.1 Linear Phase 1363.2.2 Windowing 141

3.3 Design of IIR Filters 146

XIV Contents

3.3.1 All Pass Filter 1473.3.2 Analog Filters 1513.3.3 Analog State Space Representation 1653.3.4 Analog to Digital Transformation 1743.3.5 Digital to Digital Transformation 186

Summary 195Exercises 197

4. Signal Processing 2134.1 Fundamental Properties 213

4.1.1 Signal Norm 2194.1.2 Signal Inner Product 2244.1.3 Orthogonal Signals 2304.1.4 Signal Bounds 241

4.2 Discrete Fourier Transform 2514.2.1 Periodic Signal 2514.2.2 Finite Length Signal 2614.2.3 Properties of the DFT 2634.2.4 Fast Fourier Transform 2684.2.5 Linear Convolution via FFT 277

4.3 Least Squares Estimation 2894.3.1 Linear Phase FIR Filter Design 2924.3.2 Input-Output Signal Model 2964.3.3 State Space Signal Model 3014.3.4 Recursive Estimation 3074.3.5 Applications of Recursive Estimation 310

Summary 314Exercises 315

5. Finite Wordlength IIR Filter Implementation 3275.1 Arithmetic Format 327

5.1.1 Fixed Point 3285.1.2 Floating Point 335

5.2 First Order FWL Filter 3405.2.1 Quantization Errors 3455.2.2 Scaling 351

5.3 Second Order FWL Filter 3565.3.1 Quantization Errors 3565.3.2 Scaling 361

5.4 State Space FWL Filter 3645.4.1 Quantization Errors 3695.4.2 Scaling 378

5.5 Filter Structures 3845.5.1 Error Performance Measure 3845.5.2 Low Complexity Structures 387

Contents XV

5.5.3 Delay Replaced Structures 4005.5.4 Optimal Structures 402

Summary 408Exercises 409

Index 419