An Introduction to Control Theory

From Classical to Quantum Applications

Course Lecture Notes

JM GeremiaPhysics and Control & Dynamical Systems

California Institute of Technology

Heidelberger Graduiertenkurse Physik

6 – 10 October 2003

Ruprecht-Karls-Universitat Heidelberg

Heidelberg, Deutschland

Abstract

Control theory is a vital component of all modern technology because it allows the construction of

high performance devices despite large uncertainty in the individual components used to build them.

This course provides a basic introduction to modern control theory in a series of five lectures, each

intended to be roughly three hours long. It focusses on the development of fundamental concepts,

such as stability, performance and robustness, that lie at the heart of the subject. However, it

also addresses practical issues in controller design and presents techniques for constructing and

improving control systems typically found in an experimental physics setting.

In the first part of the course (Lectures 1-3) we develop basic definitions in classical dynamical

systems and focus on linear feedback control. Experience has shown that a vast majority of physical

systems, even those that are nonlinear, can be well controlled using a feedback theory of this type.

We explore quantitative descriptions of stability, performance and robustness and use them to

develop design criteria for engineering feedback controllers. In Lecture 3, we work through a full

example of this loop-shaping design procedure.

The second part of the course (Lectures 4-5) makes a transition to quantum systems. As

modern devices grow ever smaller, and due to the realization that quantum mechanics opens av-

enues for computation and communication that exceed the limits of classical information theories,

there has been substantial interest in extending the field of feedback control to include quantum

dynamical systems. This process is already being realized in both theoretical and experimental

quantum optics for precision measurement and fundamental tests of quantum measurement theo-

ries. Forefront research in micro- and nano-scale physics has produced mechanical systems that are

rapidly approaching the quantum regime. We will look at the formulation of quantum feedback

control theory for continuously observed open quantum systems in a manner that highlights both

the similarities and differences between classical and quantum control theory. The final lecture

will involve a discussion of special topics in the field and is meant to provide a casual overview of

current experiments in quantum control.

Contents

1 Introduction to Control and Dynamical Systems 1

1.1 State Space Models and Fundamental Concepts . . . . . . . . . . . . . . . . . . . . . 2

1.1.1 State-Space Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

1.1.2 Controlled Dynamics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

1.1.3 Linear Time-Invariant Systems . . . . . . . . . . . . . . . . . . . . . . . . . . 5

1.1.4 Definition of the Control Problem . . . . . . . . . . . . . . . . . . . . . . . . 6

1.2 Classification of Control Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

1.3 Control Performance and Stability . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

1.3.1 Example 1: Open-Loop Control . . . . . . . . . . . . . . . . . . . . . . . . . . 8

1.3.2 Example 2: Feedback and Stability . . . . . . . . . . . . . . . . . . . . . . . . 9

1.4 Lecture 1 Appendices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

1.4.1 Signal Norms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

1.4.2 A Note on References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

2 Feedback and Stability 14

2.1 Stability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

2.1.1 Internal Stability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

2.2 Transfer Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

2.2.1 Properties of Transfer Functions . . . . . . . . . . . . . . . . . . . . . . . . . 18

2.2.2 Bode Plots . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

2.2.3 Transfer Function Poles and Zeros . . . . . . . . . . . . . . . . . . . . . . . . 20

2.3 Stability of the Feedback Control System . . . . . . . . . . . . . . . . . . . . . . . . 22

2.3.1 Closed Loop Transfer Functions . . . . . . . . . . . . . . . . . . . . . . . . . . 23

2.3.2 The Nyquist Criterion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

2.4 Tracking Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

2.4.1 The Sensitivity Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

ii

2.5 Robustness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

2.6 Lecture 2 Appendices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

2.6.1 Review of Laplace and Inverse Laplace Transforms . . . . . . . . . . . . . . . 28

2.6.2 System Norms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

3 Loop Shaping: Balancing Stability, Performance and Robustness 30

3.1 Example Feedback Controller Design . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

3.1.1 Plant Transfer Function: Internal Stability . . . . . . . . . . . . . . . . . . . 32

3.1.2 Feedback Stability and Robustness . . . . . . . . . . . . . . . . . . . . . . . . 33

3.1.3 Gain Margin, Phase Margin and Bandwidth . . . . . . . . . . . . . . . . . . . 34

3.1.4 Loop Shaping: Improving upon Proportional Control . . . . . . . . . . . . . . 36

3.1.5 Finalized Controller Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

3.2 Digital Controller Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

3.2.1 Digital Filters and the Z-Transform . . . . . . . . . . . . . . . . . . . . . . . 40

3.2.2 FIR and IIR Filters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

3.3 Lecture 3 Appendix: Controller Design with Matlab . . . . . . . . . . . . . . . . . . 41

4 Feedback Control in Open Quantum Systems 45

4.1 Similarities Between Classical and Quantum Dynamical Systems . . . . . . . . . . . 46

4.2 Continuously Observed Open Quantum Systems . . . . . . . . . . . . . . . . . . . . 47

4.2.1 Weak Measurements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48

4.3 Quantum Input-Output Formalism . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

4.3.1 Conditional Evolution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52

4.3.2 The Conditional Innovation Process . . . . . . . . . . . . . . . . . . . . . . . 54

4.4 Quantum Feedback Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54

5 Special Topics in Quantum Control 56

iii

List of Figures

1.1 Input-output model of the control system, which acts as a transducer that takes aninput signal, u(t), and imprints its internal dynamics onto an output signal, y(t). . 3

1.2 Solution to the controlled dynamical system in Example 2 for different values of C.As can be seen, the feedback system provides poor tracking of the desired referencesignal, r(t) = 0. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

2.1 Solution to the controlled dynamical system using velocity control. y(t) is plotted forseveral difference choices for the controller gain. . . . . . . . . . . . . . . . . . . . . 15

2.2 When two systems are coupled by a common signal, their transfer functions can bemultiplied to give the transfer function of the composite system, from initial input tofinal output. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

2.3 An example of a Bode plot for a low pass filter with a corner frequency of 1 kHz. Theutility of the Bode plot is that provides an easy to interpret graphical representationof a system transfer function by extracting the frequency dependent gain and phase. 19

2.4 An example of a pole zero plot for the same low-pass filter transfer function depictedin Fig. 2.3 and described in Section 2.2.3. . . . . . . . . . . . . . . . . . . . . . . . 21

2.5 Flow diagram for a simple feedback control system where the r(s) is the referencesignal, y(s) is the system output, e(s) is the error-signal, u(s) is the control signal,and C(s) and P (s) are, respectively, the controller and plant transfer functions. . . 23

2.6 An example of a Nyquist plot for the single-pole low-pass filter transfer functiondescribed in Eq. (2.29). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

2.7 Graphical interpretation of robust stability in the s-plane for a family of transferfunctions generated from the unstructured model, (1 + ∆W2)P (s). . . . . . . . . . . 28

3.1 Schematic for a feedback control system typical of one that might be found in a physicslaboratory, where the objective is to cause the output sense voltage, y = IR, to trackthe programming signal, r. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

3.2 Bode plot for P (s) which is typical of one that you might encounter in the physicslaboratory. It has relatively high low frequency gain, rolls off and higher frequency,and has a complicated phase profile. . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

3.3 Nyquist plot for P (s) which shows that the plant is described by a stable transferfunction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

iv

3.4 Bode plot of T (s) for proportional controllers with three different gains. . . . . . . . 35

3.5 A comparison of the Bode plots for the plant, P (s), optimized controller, C∗(s),and the loop transfer function, L(s) = P (s)C(s) for our feedback controller designedusing loop-shaping techniques. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

3.6 Closed-loop feedback transfer function, T (s) for the finalized control system designedusing loop-shaping techniques. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

3.7 Nyquist plot for the finalized feedback controller showing that is stable. . . . . . . . . 40

4.1 A schematic of weak measurement schemes. A string of meter systems interact withthe quantum system of interest, become entangled with it, and then are detected. Byconstruction, the meters are independent do not interact with one another. . . . . . . 49

AcknowledgementsThese notes would have been completely impossible without countless discussions on the subjectsof classical and quantum control that I have had with Andrew Doherty, John Stockton, MichaelArmen, Andrew Berglund and Hideo Mabuchi over the course of the past several years while in theQuantum Optics group at Caltech.

v

Lecture 1

Introduction to Control and Dynamical Systems

The purpose of this course is to provide an introduction to the branch of science and mathematicsknown as control theory, a field that plays a major role in nearly every modern precision device. Inthe classical engineering world, everything from stereos and computers to chemical manufacturingand aircraft utilizes control theory. In a more natural setting, biological systems, even the smallestsingle-celled creatures, have evolved intricate, life-sustaining feedback mechanisms in the form ofbiochemical pathways. In the atomic physics laboratory (closer to home), classical control theoryplays a crucial experimental role in stabilizing laser frequencies, temperature, and the length ofcavities and interferometers. The new field of experimental quantum feedback control is beginningto improve certain precision measurements to their fundamental quantum noise limit.

Apparently, there are at least two reasons for studying control theory. First, a detailedknowledge of the subject will allow you to construct high precision devices that accurately performtheir intended tasks despite large disturbances from the environment. Certainly, modern electronicsand engineering systems would be impossible without control theory. In many physics experiments,a good control system can mean the difference between an experimental apparatus that works mostof the time and one that hardly functions at all. The second reason is potentially more exciting—it is to develop general mathematical tools for describing and understanding dynamical systems.For example, the mathematics that has been developed for engineering applications, such as flyingairplanes, is now being used to quantitatively analyze the behavior of complex systems includingmetabolic biochemical pathways, the internet, chemical reactions, and even fundamental questionsin quantum mechanics.

There is also substantial interest in extending concepts from classical control theory tosettings governed by quantum mechanics. Quantum control is particularly exciting because itsatisfies both of the reasons for studying control theory. First, there is a good motivation forcontrolling quantum systems. As electronic and mechanical technologies grow ever smaller, we arerapidly approaching the point where quantum effects will need to be addressed; quantum controlwill probably be necessary in ordrer to implement quantum computation and communication in arobust manner. On the other hand, control theory provides a new perspective for testing moderntheories of quantum measurement, particularly in the setting of continuous observation. Many of

1

the mathematical tools used for analyzing classical dynamical systems can be applied to quantummechanics in a manner that is complementary to the traditional tools available from physics. Themathematical analysis in control theory has tended to be more rigorous than typical approachesin physics, and quantum control provides a fantastic setting for demonstrating the power of thesetools.

The defining characteristic of a quantum system, from the perspective of control theory, isthat measurement causes its state to change. This property is referred to as backaction, an inevitableconsequence of the Heisenberg uncertainty principle. In the classical world, it is (in principle)possible to perform a non-invasive measurement of the system as part of the control process. Thecontroller continuously adjusts the system dynamics based on the outcome of these non-invasivemeasurements. However, in quantum mechanics, the very act of observation causes the state of thesystem to wander even farther away from the control target. In this sense, the effect of backactionis to introduce a type of quantum noise into the measurement of the system’s state. Fortuately, itis possible to think of quantum control in a context that is similar to classical control, but with anadditional noise source due to the measurement backaction. Therefore, it is theoretically possibleto adapt existing control knowledge to make it apply in quantum settings, although there aresome mathematical surprises since (unlike classical dynamics) continuous observation of quantumsystems prevents them from achieving steady-state.

For this reason, our course begins with a description of classical systems, particularly thatof linear feedback control. This is not only practical, since it will allow you to construct servos andcontrollers for your common laboratory experiments (like locking cavity mirrors and laser frequen-cies), it is also pedagogical. Understanding the particular details which make quantum systemsdifferent from classical systems requires that we first look at classical control theory. Additionally,in my experience, there is a reasonable amount of misconception regarding servos and simple feed-back systems in the experimental physics community. In part, most of this is likely due to the factthat one has more interesting problems to deal with when constructing a complicated experimentalapparatus. Stabilizing some boring classical detail such as a cavity length in an experiment to dosomething grand, like generate single photons, is often viewed as a necessary evil. Often, this typeof feedback is implemented in a mediocre way rather than with a custom tailored, robust feedbackloop. I assure you, the latter will always perform better!

1.1 State Space Models and Fundamental Concepts

In order to proceed, we require a general mathematical description of dynamics.1 This sectiondevelops a quantitative model of physical dynamical systems and provides the cornerstone uponwhich the rest of these lectures are constructed. The most important abstraction we make is that ofan input–output system model, a concept that is depicted by Fig. 1.1. The entire dynamical systemis schematically reduced to a single box, labelled by x(t). There is a control input, u(t), and anoutput, y(t), that reflects the response of the system’s internal dynamics to the input.

2

1.1.1 State-Space Models

Every dynamical system is described by a collection of time-dependent variables, {x1(t), x2(t), . . . , xn(t)},that we group together into a vector, x(t) ∈ Rn. Physically speaking, x(t) describes the state of thesystem, and it reflects all the information that the system has available to itself in order to propa-gate forward in time. In other words, the future state of the system for, t > 0, can be determinedfrom the initial conditions, x0 = x(t = 0), and an equation of motion,

d

dtx(t) = a

(x[0,t), t

), a : Rn

[0,t) × t → Rn . (1.1)

This model says that the rate of change of the system variables at time, t, can be determined bysome function, a, of the system’s evolution, x[0,t), and the current value of t. Here, the symbol,x[0,t), is shorthand notation for {x(t′), 0 ≤ t′ < t}, i.e. it is the system history starting at time,t = 0, and ending at t. The output of the vector function, a, necessarily has dimension n so thatthe number of variables in x(t) is constant in time. The vector-space, Rn, where every choice ofx(t) lives is referred to as the state-space of the system.

The state-space model, given by Eq. (1.1), reflects information that in principle is onlyknown to the system; it reflects all the physics. In general, we as an external observer can onlygain information about the system by making measurements, and it is often impossible to acquirecomplete information from our measurements. Therefore, we are not always privy to the actualvalues of x(t), but instead, have a partially observed system,

y(t) = c(x(t), t), c : Rn × t → Rk . (1.2)

c(x(t), t) is some function of the state of the system at time, t, and the dimension of y(t) is k,which may be different from n. Physically speaking, y, is the information about the system thatis available to an observer who has performed some experiment to probe the state of the systemat time, t. y(t) and therefore the function, C, provide some access to the inner-workings of thesystem dynamics.

Eqs. (1.1) and (1.2) provide an extremely general mathematical description of dynamics.The evolution and the observation processes are free to be completely nonlinear and the restriction

u(t) y(t)x=f1(x,u,t)

Input

Signal

Output

Signal

Dynamical

System

y=f2(x,u,t)

Figure 1.1: Input-output model of the control system, which acts as a transducer that takes aninput signal, u(t), and imprints its internal dynamics onto an output signal, y(t).

3

to real-valued functions is not at all confining because complex-valued functions, x(t) ∈ Cn, can beredefined in terms of a real state vector, x′(t) = {Re[x(t)], Im[x(t)]} ∈ R2n. There are, however, twoimportant restrictions imposed by the structure of Eqs. (1.1) and (1.2). The first is that we haveassumed a continuous time model (what one would expect for analog systems). In discrete-time(digital) systems, the only difference is that the evolution equations take the form of delay-updateequations,

x(ti) = a(x[t0,...,ti−1], ti

)∆t + x(ti−1) (1.3)

y(ti) = c (x(ti), ti) (1.4)

where ti = ti−1 + ∆t, rather than differentials. In principle, there is little qualitative differencebetween the continuous and discrete evolution equations. For the purpose of this course, I willtend to focus on continuous time models because even when designing a digital controller, it iscommon to perform the design in continuous time and then discretize the process as a second step.Occasionally, I will discuss digital systems when there is a particular point that is not obvious bysimply replacing the continuous system dynamics with a discrete evolution model. There will bemore on this Lecture 3.

The second simplification is that the dynamics are deterministic, in the sense that there areno terms reflecting random processes, such as environmental disturbances or measurement noise.For the time being, we will neglect these things. However, we will certainly return to them alittle later in the course when we address quantum control systems. As can be expected, quantumcontrol must adopt a stochastic, probabilistic model of the dynamics since x will be replaced witha representation of state, such as the wavefunction or density operator, and these are probabilisticquantities. Additionally, the measurement process will inherently introduce quantum fluctuationsinto the dynamics in the form of backaction projection noise.

1.1.2 Controlled Dynamics

The evolution generated by Eq. (1.1) represents the free evolution of the system; there is no externalforcing. We need to add a term in the system evolution that will allow us to affect the dynamicsby applying an input signal, u(t). Following a general approach, as in the above section, this givesus a new equation,

d

dtx(t) = a

(x[0,t), t

)+ b

(x[0,t),u[0,t), t

)(1.5)

where b(x[0,t),u[0,t), t

)can be thought of as the coupling term that describes the interaction be-

tween the input signal, u, and the system. The dimension of u is free to be anything, which wewill call k, but the output of the function b must have dimension, n in order to be consistent withthe system dynamics. Similarly, the observation process, y(t), may be affected by the input signal,so that generally speaking,

y(t) = c (x(t),u(t), t) . (1.6)

4

1.1.3 Linear Time-Invariant Systems

The equations above are extremely general. In some sense that is good because it allows fora theory that includes most possible input signals as well as the majority of physically realisticdynamical systems. The drawback is that with such generality, it is extremely difficult to performany significant analysis. Nonlinear functions cannot generally be solved exactly, and we would thenhave to resort to treating every new control system as a unique problem. Instead, it is beneficialto restrict the form of the functions, a, b, and c such that they still (approximately) reflect a largenumber of physical systems, but can be exactly solved for the entire class.

Such a restriction is provided by the class of dynamical equations referred to as linear time-invariant (LTI) models.4–6 This approximation corresponds to treating the functions, a, b, and c,as linear and time-stationary,

d

dtx(t) = Ax(t) + Bu(t) (1.7)

y(t) = Cx(t) + Du(t) (1.8)

where A ∈ Rn×n, B ∈ Rk×n, C ∈ Rn×m and D ∈ Rk×m. In other words, the functions have beenreplaced by time-invariant matrices.

The best thing about Eqs. (1.7) and (1.8) is that they can be solved exactly for all reasonablechoices of A and B. In fact, doing so only requires basic methods from the field of ordinarydifferential equations,

x(t) = eAtx0 + eAt

∫ t

0e−AτBu(τ) dτ . (1.9)

We see that the system state is a superposition of a complimentary and a particular solution.The former is given by exp(At)x0 and depends upon the initial condition, x0, while the particularsolution is given by a convolution over the input signal. It contains all the physics of the interactionbetween the input signal and the internal system dynamics. Eq. (1.9) can be substituted in Eq.(1.8) in order to obtain an analytic expression for y(t).

At this point, we have to pause for a minute in order to justify the huge approximation thatwe made in reducing the system dynamics to a linear model. One argument for the approximationin Eqs. (1.7) and (1.8) is that these equations are locally true even for nonlinear dynamics. Thiscan be seen from the fact that they reflect a first order Taylor expansion of the full equations inEqs. (1.5) and (1.6). So, even if the full system dynamics are nonlinear, these linear equations willbe true in a neighborhood surrounding each point, x ∈ Rn. Of course, this raises the question ofhow effective a controller design can be if the model use to construct it is only locally applicable.There are two possible responses to this criticism. The first one is simply to state the fact thatengineering control theory survived for 80 years before it even attempted to go beyond LTI models.In that time, it was able to successfully produce controllers for highly nonlinear devices such asairplanes and space missions that made it to the moon and back. In that sense, the justificationcomes in the form of a proof by demonstration. More generally, the answer to this problem isthat once the controller is doing its job, it will generally act to keep the system in a region of itsphase-space where the linear approximation is good. Many examples where this is true can be

5

found in standard control theory texts.7 You will have to take my word for this at the moment,but before long, the point will become clear from the mathematics.

1.1.4 Definition of the Control Problem

By applying a forcing term, u(t), it is possible to drive the system such that it evolves accordingto trajectories that depart from the natural dynamics. In this sense, the function, u(t), steers theoutcome of the evolution. Everything in control theory involves selecting an appropriate signal,u(t), such that the dynamical system evolves in a specified manner. For instance, a control goalmight be to choose a time, t = T , and a target value, y∗t = y(T ), and identify a u∗(t) that achievesthis outcome. Another objective is for the system output, y(t), to track a reference signal, r(t), byminimizing

‖y(t)− r(t)‖ . (1.10)

The signal norm, ‖ · · · ‖ is some appropriate measure of the difference between the two signals (referto the Lecture 1 Appendix for a review of norms).

There are generally a large number of signals, {u∗1(t),u∗2(t), . . .}, that achieve the targetobjective, and each of these is referred to as a permissable control. However, applying u(t) involvesexpending energy in some sense or another, and it is desirable to select the u∗i (t) that requires theleast effort to reach the objective. This can be accomplished by adding an optimality componentto the control problem by minimizing over a signal norm,

u◦(t) = minu∗(t)

‖u∗(t)‖2 = minu∗(t)

J [u(t)] . (1.11)

The 2-norm is most commonly used in this cost functional because it is closely associated with theenergy expenditure.5 All control problems are intrinsically an optimization.

1.2 Classification of Control Systems

We have already seen that control systems can be classified according to their dynamics, as eithercontinuous or discrete and linear or nonlinear. Another important classification involves the methodthat is used to determine the optimal control signal, u◦(t). Roughly speaking, all controllers canbe grouped into one of three classes.

• Open-loop Control: In open loop control, it is assumed that the dynamical model of thesystem is well known, that there is little or no environmental noise and that the controlsignal can be applied with high precision. This approach is generally utilized when there isa target value, y∗(T ), to achieve at a particular final time, T . Under these assumptions, it ispossible to solve the dynamical equations of motion and directly perform the optimization inEq. (1.11). The disadvantage of open-loop control is that the performance of the controlleris highly susceptible to any unanticipated disturbances. Additionally, if there is any error in

6

the model used to describe the system dynamics, the designed control signal is unlikely toperform as expected.

• Feedback Control: In feedback control, continuous or discrete time measurements of thesystem output, y(t), are used to adjust the control signal in real time. At each instant, theobserved process, y is compared to a tracking reference, r(t), and used to generate an errorsignal. The feedback controller interprets this error signal and uses it to generate a u◦(t)that maintains tracking. Feedback controllers have the potential to be adaptive and highlyeffective even if there is large uncertainty in the system model or if there are environmentaldisturbances or measurement noise. Feedback therefore provides the backbone of most moderncontrol applications.

• Learning Control: In learning control, a measurement of the system, y(t), is also usedto design the optimal feedback signal; however, it is not done in real time. Instead, a largenumber of trial control signals are tested in advance, and the one that performs best is selectedto be u◦(t). Learning control has a number of disadvantages compared to feedback, because itdoes not permit as detailed a quantitative analysis of the controller performance and stability.However, it is useful for applications where the dynamics occur too fast for the system to beactuated in a feedback configuration.

1.3 Control Performance and Stability

Regardless of the controller that is used, there are three major objectives that must be consideredwhen assessing the quality of the feedback system.

1. Performance is the ability of the controller to realize its dynamical objective, such as toachieve the target value, y∗(T ), or to follow a reference signal, r(t).

2. Stability is a measure of how well-behaved the control system is. When feedback is utilized,there is the potential for instability. If caution is not taken when designing the feedbacksystem, parasitic signals or environmental disturbances might cause the entire dynamicalsystem to oscillate out of control and potentially experience physical damage.

3. Robustness is the ability of the controller to remain effective despite unanticipated changesin either the system being controlled or the environment.

In general, realistic controllers must achieve a balance between performance, stability androbustness. The particular objective that is most important is application dependent. For example,in an airplane, robustness and stability take higher priority than performance. This is because it ispreferable for a plane to go slightly off course than it is for it to become unstable and potentiallycrash. However, in precision measurement, the opposite is true, and it is better for a laser or aninterferometer to achieve high performance at the expense of having to re-lock it every few days,or when someone drops a screwdriver on the optical table.

7

1.3.1 Example 1: Open-Loop Control

In order to demonstrate these points, it is best to consider several examples of controller design. Inthis first case, we utilize open-loop control to illustrate the concepts of performance and robustness(this following example, and similar variations of it, can be found in most standard discussionsof open-loop control theory). Consider the linear control system governed by the following one-dimensional state-space dynamical model,

x(t) = ax(t) + bu(t) (1.12)

y(t) = cx(t) (1.13)

where a, b, c ∈ R and the objective is to achieve the value, y(T ) = y∗ at time, t = T . This systemis a specialization of the linear time invariant (LTI) model described in Section 1.1.3. Its solutionis given by Eq. (1.9),

y(t) = c

(eatx0 + beat

∫ t

0e−aτu(τ) dτ

). (1.14)

By requiring that u(t) is an admissible control function, we can express the control objective asfollows,

y∗

c= eaT x0 + beaT

∫ T

0e−aτu(τ) dτ (1.15)

which can be rearranged to give,∫ T

0e−aτu(τ) dτ =

c−1y∗ − eaT x0

beaT. (1.16)

Now we need to solve the minimization problem,

u◦(t) = minu(t)

J [u(t)] = minu(t)

∫u2(t) dt (1.17)

which can be accomplished by squaring Eq. (1.16) and utilizing the Cauchy-Schwarz inequality,

[∫ T

0e−aτu(τ) dτ

]2

≤∫ T

0u2(τ) dτ

∫ T

0e−2aτ ′ dτ ′ → J [u(t)]

∫ T

0e−2aτ dτ (1.18)

and therefore,

J [u(t)] ≥(c−1y∗ − eaT x0

)2

b2e2aT∫ T0 e−2aτ dτ

. (1.19)

In order for the minimum to be achieved, we need to find a u∗(t) that satisfies the equality in Eq.(1.19),

J [u(t)] = 2a

(c−1y∗ − eaT x0

)2

b2(e2aT − 1)(1.20)

which (this is the tricky part of the problem) happens if u(t) is proportional to exp(−at),

u∗(t) = Ce−at (1.21)

8

to give, [∫ T

0e−atu ∗ (t) dt

]2

= C2

[∫ T

0e−2at dt

]2

. (1.22)

Using Eq. (1.16), it can be shown that,

C =y∗c−1 − eaT x0

beaT∫ T0 e−2at dt

(1.23)

and finally that,

u◦(t) = 2ay∗c−1 − eaT x0

beaT (1− e−2aT )e−at (1.24)

So it seems that everything worked out fine. We were able to find a solution that minimizesJ [u(t)] and that achieves the control objective. However, what if the value of a, b or c was slightlywrong? In order to address this question, we should ideally evaluate the sensitivity of the controloutcome. This could be accomplished by taking the derivatives, ∂y(t)/∂a|t=T and so on. It wouldbe found that the control outcome is exponentially sensitive to some of the control parameters,meaning that errors in the system model could produce an exponentially large tracking error! Thismethod of controller design is exceptionally non-robust.

1.3.2 Example 2: Feedback and Stability

For the second example, we will try to design a feedback controller to stabilize the following system,

d

dt

x1

x2

=

0 ω

−ω 0

x1

x2

+ u(t)

0

1

(1.25)

y(t) =(

1 0

)

x1

x2

(1.26)

subject to the initial conditions that x0 =(

1 0

)T

.

Our goal is to make y(t) track the reference signal, r(t) = 0. Granted, we do not yet knowanything about feedback control, but let us proceed in the dark and assume a controller that appliesa proportional feedback signal opposite the measured displacement,

u(t) = −C(y(t)− r(t)) = −Cy(t) = −Cx1(t) . (1.27)

This seems like a very reasonable feedback control law. At each time, we measure how far thesystem is from the desired state and then apply a push in the right direction that is stronger if weare farther away from the target and gets smaller as we approach it. The dynamical system, usingthe choice, u(t) = −Cy(t) can be solved analytically (it is an LTI model) to give the followingequation of motion,

y(t) = cosh(√

−ω(C + ω)t)

(1.28)

9

0 2 4 6 8 10 12 14 16 18 20

0

1

2

3

4

Time ωt

Sys

tem

out

put,

y(t) C=-1.01ω

C = -1.001ωC = 0

C = ωC = -ω/2

-1

Proportional Feedback Controller Stability

Figure 1.2: Solution to the controlled dynamical system in Example 2 for different values of C.As can be seen, the feedback system provides poor tracking of the desired reference signal, r(t) = 0.

which shows that y(t) depends very strongly on the choice of C. If C > 0, then the system oscillateswith a frequency that is somewhat larger than ω. However, if C < −ω, then the system blows up,as t →∞, y exponentially increases. Fig. 1.2 shows a plot of y(t) for several different choices of C,and highlights the undamped behavior. Here, C acts as a gain, and the hope would be that somejudicious choice of C would result in stable feedback. However, this is clearly not the case. MakingC = 0 corresponds to no feedback, and increasing C > 0 only makes matters worse by increasingthe oscillation frequency. Certainly, C < 0 is

Well, our first attempt at feedback control was not very successful despite the fact that wedid something entirely reasonable— something that a perfectly well-educated physicist might havetried (in fact, something I mistakenly did myself) in the laboratory. This exercise demonstratesthat stability is not a guarantee when working with feedback, it must be specifically engineered,and that will be the subject of tomorrow’s lecture.

1.4 Lecture 1 Appendices

1.4.1 Signal Norms

A norm is an mathematical measure that provides a generalized measure of the magnitude ofits argument, and therefore provides a way of ranking the relative “size” of different functions.Mathematically, in order for a measure, ‖ · · · ‖, to be considered a norm, it must satisfy the followingrequirements:

(i) ‖u‖ ≥ 0, ∀u(t) which is the requirement that norms be non-negative

(ii) ‖u‖ = 0 iff u(t) = 0, ∀t which is the minimum bound requirement

10

(iii) ‖au‖ = a‖u‖, ∀a ∈ R which is the distributive property for scalars

(iv) ‖u + v‖ ≤ ‖u‖+ ‖v‖ which is the so-called triangle inequality

A set of measures that satisfy these requirements and that are regularly encountered in themathematical analysis used in control theory are the n-norms, defined by

‖u‖n ≡(∫ ∞

−∞|u(t)|n dt

) 1n

, (1.29)

where particulary important n-norms include the 2-norm (or energy norm),

‖u‖2 =(∫

|u(t)|2 dt

) 12

(1.30)

and the ∞-norm (or maximum value norm),

‖u‖∞ = supt|u(t)| . (1.31)

When the argument is a vector function, one can apply the norm to each component of the vectorand then take any combination of those individual norms that satisfies the 4 requirements listedabove. For example,

‖f(t)‖n ≡(∑

i

‖fi(t)‖nn

) 1n

(1.32)

1.4.2 A Note on References

The material covered in this lecture reflects topics that are addressed in a number of standardtexts on the subject of control theory. Anyone looking to learn more about the field is is immedi-ately confronted with a number of possible sources and references. Texts that I have found to beparticularly useful are listed here,

1. Feedback Control Systems8 by Phillips and Harbor is a very good general introduction thefield of feedback control. It has a particularly accessible description of the Nyquist stabilitycriterion.

2. Introduction to Control Theory4 by Jacobs is a good introduction to the field of controltheory. It discusses a large number of topics without going heavily into detail and provides agood starting point.

3. Feedback Control Theory5 by Doyle, Francis and Tannenbaum is a fairly readable introduc-tion to the field of robust control, but not a general introduction to feedback (as the titleimplies). This text requires only moderate background in mathematical analysis.

4. Essentials of Robust Control9 by Zhou and Doyle is a somewhat more involved discussion ofrobust control theory and assumes a stronger background in mathematical analysis.

11

5. Modern Control Systems7 by Dorf and Bishop is a good general introduction that discussesdigital feedback controller design in reasonable detail.

6. Digital Signal Processing10 by Proakis and Manolakis is the standard text on the subject ofdesigning filters in discrete time systems and is an invaluable resource to anyone interestedin digital control.

12

References

[1] L. Perko, Differential Equations and Dynamical Systems (Springer-Verlag, New York, 2001),3rd ed.

[2] G. Chen, G. Chen, and S.-H. Hsu, Linear Stochastic Control Theory (CRC Press, London,1995).

[3] B. Oksendal, Stochastic Differential Equations (Springer Verlag, 1998), 5th ed.

[4] O. L. R. Jacobs, Introduction to Control Theory (Oxford University Press, New York, 1996),2nd ed.

[5] J. Doyle, B. Francis, and A. Tannenbaum, Feedback Control Theory (Macmillan PublishingCo., 1990).

[6] G. E. Dullerud and F. Paganini, A Course in Robust Control Theory: A Convex Approach(Springer Verlag, New York, 2000), 1st ed.

[7] R. C. Dorf and R. H. Bishop, Modern Control Systems (Prentice Hall, Upper Saddle River,NJ, 1991), 9th ed.

[8] C. L. Phillips and R. D. Harbor, Feedback Control Systems (Prentice Hall, Englewood Cliffs,1996), 3rd ed.

[9] K. Zhou and J. C. Doyle, Essentials of Robust Control (Prentice-Hall, Inc., New Jersey, 1997),1st ed.

[10] J. G. Proakis and D. G. Manolakis, Digital Signal Processing: Principles, Algorithms andApplications (Prentice Hall, Upper Saddle River, 1996), 3rd ed.

13

Lecture 2

Feedback and Stability

Yesterday, we attempted to design a feedback controller and were, unfortunately, not particularlysuccessful. The problem we encountered was instability— depending on the controller gain, C, theoutput, y(t), either oscillated or diverged to infinity. For one special case, C = −ω, the value ofy(t) remained constant at y(t) = 1 for all t > 0. This value for the controller gain was (in somesense) more stable, but it failed to achieve tracking since the goal was y(t) = 0.

What went wrong? In order to uncover the problem, we can rewrite the dynamical equationsfrom Eqs. (1.25) and (1.26) in a more convenient form. Making the change of variables, z = x2,we see that the dynamics are effectively a second order differential equation (there is only a littlealgebra involved here)

z(t) = −ω(C + ω)z(t), z(0) = 1 (2.1)

y(t) = z(t) . (2.2)

Suddenly, the reason for the instability is clear. We have a perfect harmonic oscillator with nodamping,

y(t) = cos(√

ω(C + ω)t) (2.3)

when C + ω ≥ 0. Otherwise, we have exponential growth of the signal,

y(t) =12

(e√

ω(C+ω)t + e−√

ω(C+ω)t)

(2.4)

for C + ω < 0.

Physically speaking, this feedback controller operates in the following manner: it looks atthe displacement, y(t) − r(t), and then applies a torque to counteract the motion— it tries toactuate y(t). As a result, feedback provides no damping of the system’s internal dynamics. Withno feedback, C = 0, the system is an oscillator, y(t) = cos(ωt), and the addition of the controlleronly changes the oscillation frequency, not the qualitative behavior. In retrospect, we should havedesigned a controller that was proportional the derivative of the displacement,

u(t) = Cd

dt[y(t)− r(t)] (2.5)

14

0 2 4 6 8 10 12 14 16 18 200.5

0

0.5

1

ωt

y(t)

C = 1/2ω

C = 1ω

C = 2ω

Successful Feedback using Velocity Control

Figure 2.1: Solution to the controlled dynamical system using velocity control. y(t) is plotted forseveral difference choices for the controller gain.

because this would produce an equation of motion for the feedback system,

y(t) = −ω2y(t)− Cy(t) (2.6)

which is solved by a damped sinusoid when C > 0,

y(t) = e−Ct/2ω cos(ωt) . (2.7)

Now, y(t) decays exponentially with time, and therefore rapidly approaches the tracking objective,r(t) = 0. Increasing the controller gain, C, increases the damping coefficient, Γ = C/2ω, and helpsthe feedback achieve its objective even faster. This is illustrated in Fig. 2.1, where y(t) is plottedas a function of time for several different choices of the controller gain.

The essential point is that the feedback controller needed to incorporate information abouty(t) from earlier times, not just t, in forming u(t) (Eq. (2.6) does this through the derivative termwhich involves y(t) at multiple times). In general, the controller signal should involve a convolutionover the past values of y(τ), from τ = 0, . . . , t,

u(t) =∫ t

0F (t− τ)y(τ) dτ . (2.8)

Everything in feedback controller design involves engineering F (t). By the end of this lecture, wewill have a better idea of how to do this by looking at the requirements that F (t) must satisfy inorder to achieve both stability and performance.

Yesterday, we learned that the control problem should also satisfy a minimization over thecost of the driving term, u(t). This optimization is necessary to prevent the controller from utilizingsignals that expend excess energy, or that are infinite. Feedback control provides no exception tothis requirement. Although we will not consider the control signal minimization problem in detail,feedback control of the form in Eq. (2.8) satisfies a cost functional minimization analogous to Eq.

15

(1.11),

J [u(t)] ≡∫ T

0

(xT (t)Px(t) + uT (t)Qu(t)

)dt (2.9)

for appropriate choices of the matrices, P and Q that we will not derive here. For our purposes,it is sufficient to state that feedback control does satisfy all the necessary requirements for signalenergy minimization. More information on deriving feedback control in terms of a cost functionaloptimization can be found in standard texts on the subject.1

2.1 Stability

While the harmonic motion in our first attempt at a feedback controller was better than exponentialgrowth, it certainly failed to achieve tracking because it oscillated around the target indefinitely.In control theory, the term unstable refers to a system that diverges for long time. This includesboth exponential growth and undamped harmonic oscillation, although the latter is typically giventhe distinction of marginal stability because y(t →∞) remains finite, but not zero.

When discussing stability in a feedback control system, there are two questions to address.The first has to do with the intrinsic dynamics of the system (without any feedback), and thesecond involves the effect of closing the loop.

• Internal Stability refers to the dynamical system, x(t) = Ax(t), without feedback. To as-sess internal stability, we ask whether the output, y(t), experiences oscillations, or exponentialgrowth.

• Closed-loop Stability refers to the behavior of the system when feedback is applied. Toassess this question, we look for properties of the feedback loop that lead to oscillation orexponential growth in closed-loop. Feedback loops can be unstable even when its componentsare internally stable, as well as the opposite.

A quantitative analysis of these types of stability is an essential aspect of control theory, and itrequires the development of several mathematical concepts.

2.1.1 Internal Stability

Let us return to our feedback system but focus only on the internal dynamics (those involving xand not u),

x(t) = Ax(t) =

0 ω

−ω 0

x1

x2

. (2.10)

16

Since this system is linear, we know that the solution is given by the form, x(t) = exp(At)x0. Forall reasonable system matrices, we can diagonalize A,

A = OΛO† = O

λ1

. . .

λn

O† (2.11)

where O is normally orthogonal or unitary. O provides a similarity transform that reduces A to adiagonal matrix, Λ, whose elements are the eigenvalues, {λ1, . . . , λn} of A. With this transforma-tion, it is possible to evaluate the matrix exponential,

eAt = exp(OΛO†t) = O

eλ1t

. . .

eλnt

O† (2.12)

which shows that the internal dynamics are governed by a superposition of the eigenvalues expo-nentials (which are in general complex), λi = σi + jωi, j =

√−1. The imaginary components, ωi

of the eigenvectors describe the fundamental frequencies of the linear dynamical system, and thereal components, σi relate to the damping coefficients. In order for the dynamics to be internallystable, it is necessary for all the system eigenvalues to have negative real parts,

Definition: A system is internally stable if and only if, Reλi < 0,∀λi, where λi is an eigenvalueof A. This leads to exponential decay of all the internal dynamics, and the open-loop dynamicalsystem rapidly achieves a steady state, xss → 0, for t > 0.

We can immediately see that the eigenvalues of A for our example system are λi = ±jω, andReλi = 0. A dynamical system with zero real eigenvalues is marginally stable— it is an undampedharmonic oscillator and will not converge to a steady-state for long time. A system matrix whoseeigenvalues all lie in the left half-plane is referred to as Hurwitz, and internally stability is achievedwhen the Hurwitz criterion is satisfied.

2.2 Transfer Functions

In order to address the question of closed-loop stability, we need a tool for analyzing the system’sresponse to a particular input. This can be represented in the following manner,

y(t) = G[u(t)] (2.13)

where the notation, [· · · ], indicates that G takes a function, u(t), as its input. Also note that G

outputs a function, y(t), and therefore serves as a liaison between the input signal used to drivethe system, and its response.

17

For linear systems, it is particularly convenient to analyze G for the case where the inputsare sinusoids, u(t) = a cos(ωt) + b sin(ωt) = r exp(jωt + φ). This is because driving a linear systemwith a sinusoid,

x(t) = x0eat + eat

∫ t

0e−aτrejωτ dτ =

eaT (1 + ax0 − jωx0)a− jω

− rejωt

a− jω(2.14)

produces an output with a term that oscillates at the same frequency, ω, as the driving term plus acomplimentary solution due to the the internal dynamics. Assuming that the system is internallystable, the later component exponentially decays on a timescale that is often faster that 1/ω. Thesystem output, y(t), is therefore,

y(t) =C

a− jωrejωt = Z(ω)u(t) (2.15)

meaning that the output differs from the input only by a change in its magnitude (due to the realpart of a) and a phase shift (due to the imaginary part). The quantity, Z(ω), is generally referredto as the complex frequency response of the linear system at the frequency, ω. This point is madeeven clearer by taking the Laplace transform of the dynamical equations, including all the signalsas well as the differential equations to give,

L[x(t)] → sx(s) = Ax(s) + Bu(s) (2.16)

L[y(t)] → y(s) = Cs(s) (2.17)

where the Laplace variable, s = σ + jω, and we have ignored the terms that correspond to thecomplementary solution to the differential equation.a For internally stable systems, these neglectedterms depend upon the initial conditions and are proportional to the exponentially decaying dy-namics. Therefore, the description given by Eqs. (2.16) and (2.17) are valid in steady state. Underthe approximation that the internal dynamics decay quickly compared to the time-dependence ofu(t), it is possible to algebraically eliminate x(s) from the above equations to give,

y(s) = C (sI−A)−1 Bu(s) (2.18)

or,

G(s) ≡ y(s)u(s)

= C (sI−A)−1 B (2.19)

where G(s) is referred to as the system Transfer Function.

2.2.1 Properties of Transfer Functions

All of the system dynamics are captured by the transfer function, and even though they are definedin terms of sinusoidal inputs, this imposes no restriction since for linear systems,

y(s) = G

(∑

k

uk(s)

)=

∑

k

Guk (2.20)

aI will use the convention that a function, f(s), always corresponds to the Laplace transform of an asso-ciated time-domain function, f(s) = L[f(t)].

18

System 1

u(s) y(s)=G2G1u(s)G1u(s)G

1(s)

System 2

G2(s)

Figure 2.2: When two systems are coupled by a common signal, their transfer functions can bemultiplied to give the transfer function of the composite system, from initial input to final output.

and any input signal can be decomposed according to its Fourier expansion. As can be seen fromEq. (2.19), the transfer function is time-independent (due to the fact that we are using an LTImodel), and it represents the complex frequency response of the system as a function of frequency.

Transfer functions are convenient due to their algebraic simplicity— convolution integrals inthe time-domain correspond to products of functions in the Laplace domain. The product of G(s)and a signal, u(s), is another signal. And, coupled systems, as depicted in Fig. 2.2, with individualtransfer functions, Gk(s), have a total transfer function is given by the product,

G(s) =y(s)u(s)

=∏

k=1

Gk(s) (2.21)

As a matter of convention, system transfer functions are written using uppercase letters, such asG, whereas signals are expressed using lowercase letters, such as u(s).

0

� � �

� � �

� � �

�

���

� � �

���

� � � � � � � � � � � � � � �� � �

� � �

�� �

�� � �

��

� � ! " # $ % & ' ( )

Figure 2.3: An example of a Bode plot for a low pass filter with a corner frequency of 1 kHz. Theutility of the Bode plot is that provides an easy to interpret graphical representation of a systemtransfer function by extracting the frequency dependent gain and phase.

19

2.2.2 Bode Plots

The transfer function, G(s) ≡ G(jω) gives the complex response of the system as a function offrequency.b For linear dynamics, we have already seen that driving a system with a sinusoidalsignal produces a response that oscillates at the same frequency, but with a complex coefficient. Inthe laboratory, it is usually more convenient to think in terms of real numbers that corresponds tophysical quantities rather than complex quantities. Since the frequency of the driving, u(t), andresponse, y(t), signals are the same, the only differences between them can be in the amplitude andphase,

G(jω) = g(ω)ejωt+φ(ω) (2.22)

where g(ω) is the real-valued gain or change in amplitude, and φ(ω) is the real-valued phase-shiftbetween the input and the output. A convenient method for viewing transfer functions is to plottheir frequency-dependent gain and phase, referred to as a Bode plot. As a matter of convention,Bode plots are generated using a log frequency scale, and separate axes for the gain and phasefunctions, as in Fig. 2.3. The gain is plotted in units of dB, defined for signals in relative energyunits as,

GdB = 20 log10 |g(ω)| (2.23)

such that 0 dB is equivalent to unity gain, and every 6 dB corresponds to doubling the signal ampli-tude. There is a related definition for dB in terms of relative power units given by, 10 log10 |gPow(ω)|.

The transfer function depicted by Fig. 2.3 is a Butterworth low pass filter with a cornerfrequency of fc = 1 kHz. The effect of the filter is to allow low frequency components of theinput signal, u(t) to pass with no attenuation and little phase shift; however, higher frequencycomponents are heavily attenuated and also experience a 90◦ phase shift.

2.2.3 Transfer Function Poles and Zeros

It is often convenient to express transfer functions using ratios of polynomials in s = jω. Thisis because the Laplace transform turns differential equations, such as those found in Eq. (1.7),into algebraic equations (refer to the appendix for a review of Laplace transforms). Therefore, thetransfer function, G(s), can be represented,

G(s) =N(s)M(s)

≡ vnsn + vn−1sn−1 + . . . + v1s + v0

wmsm + wm−1wn−1 + . . . + w1s + w0(2.24)

where the {v0, . . . , vn} are coefficients for the numerator polynomial, N(s), and {w0, . . . , wm} arecoefficients for the denominator polynomial, M(s) where n and m are the orders of the polyno-mials.c. Since N(s) and M(s) are polynomials, they can be factored into their n and m roots,

bFor the remainder of this lecture, I will only consider scalar rather than vector functions because thealgebra is a little easier and there is nothing essential to be gained from retaining the vector quantities.

cI apologize for re-using variables, but the n and m correspond to the order of the polynomials and arenot necessarily related to the n an m which provided the dimensions of the matrices, A and B

20

respectively,

N(s) = kn (s− zn) (s− zn−1) · · · (s− z0) (2.25)

M(s) = km (s− pm) (s− pm−1) · · · (s− p0) (2.26)

where the polynomial zeros are complex, pi, zi ∈ C. The transfer function can be represented,

G(s) = K(s− zn) (s− zn−1) · · · (s− z1)(s− pm) (s− pm−1) · · · (s− p1)

(2.27)

where K is a constant gain factor, the {zi} are the zeros of the transfer function, and the {pi} arethe poles. Like the magnitude, g(ω) and phase, φ(ω), the quantities, {K, pi, zi}, are sufficient tocompletely characterize the transfer function.

The usefulness of the pole-zero-gain (PZK) representation is that it is closely connected tothe issue of stability. This can be seen by comparing Eq. (2.19) to Eq. (2.27), and noting thatthe eigenvalues of the dynamical system matrix A are associated with the poles of the transferfunction,

(Is−A)−1 =1

det |Is−A|adj (Is−A) (2.28)

according to the definition of the matrix inverse. Since the transfer function poles include theeigenvalues of A, stability requires that the transfer function, G(s), contains no right-half plane(RHP) poles. If the real parts of the poles of G(s) are all negative, then system is internally stable,otherwise if ∃ pi |Re(pi) ≥ 0, there will be exponential growth of the system dynamics, or marginalstability (harmonic motion) if Re(pi)=0.

-6k -4k -2k 01

0.8

0.6

0.4

0.2

0

0.2

0.4

0.6

0.8

1

1k 800 200400600

Imagin

ary

Axi

s

Real Axis

Figure 2.4: An example of a pole zero plot for the same low-pass filter transfer function depictedin Fig. 2.3 and described in Section 2.2.3.

This stability criterion is elegantly expressed by a pole-zero map, which is simply a plot ofthe locations of the poles and zeros of G(s) on the complex s-plane. It provides a convenient visual

21

tool for assessing the internal stability of the dynamical system. Typically, the symbol, ×, is usedto denote the location of a pole, and ◦ is used to denote a zero, as is illustrated in Fig. 2.4. Thisplot corresponds to the Butterworth filter which we saw in Section 2.2.2 with transfer function,

G(s) =2π × 103

s + 2π × 103(2.29)

expressed according to a pole-zero-gain model.

This transfer function has a single pole at z1 = –1 kHz. Since the pole lies in the left-half of the complex s-plane, it corresponds to a stable transfer function. Evidently, this is why aButterworth low-pass filter is often referred to as a “single-pole low-pass.” There are some usefulcomparisons to make between the Bode plot in Fig. 2.3 and Fig. 2.4. First, the single pole in thePZ map corresponds to the corner frequency, fc, or -3 dB point, on the Bode plot. Evidentally,the LHP pole results in a 6 dB per octave roll-off, or gradual decrease in the gain. It also appearsthat a single pole results in a gradual onset of phase delay that increases from 0◦ to −90◦, withthe inflection point located at the pole-frequency. Although not depicted by this transfer function,a zero has the opposite effect of a pole— it causes the gain to rise at 6 dB per octave and resultsin +90◦ of phase. Finally, it is necessary to define several common phases that are used whendescribing transfer functions,

1. Proper: A transfer function is proper if G(s) is finite for ω →∞.

2. Strictly Proper: A transfer function is strictly proper if G(s) → 0 for ω →∞.

3. Minimum Phase: A transfer function is called minimum-phase if it has no right-half planezeros, Rezi < 0, ∀zi.

2.3 Stability of the Feedback Control System

We finally have enough mathematical equipment to tackle the problem of stability in a closed-loopfeedback control system, and we begin by defining the close-loop feedback transfer function usingthe flow diagram in Fig. 2.5 where the signals are defined as follows,

1. r(s) is the tracking reference signal.

2. y(s) is the system output.

3. e(s) is referred to as the error signal and is given by the difference between the output andthe reference, e(s) = y(s)− r(s).

4. u(s) is the controller signal used to drive the system in response to the error signal.

and the system transfer functions are given by,

22

1. C(s) is the controller transfer function and represents the implementation of Eq. (2.8)

2. P (s) is the plant transfer function, and represents the system that is being controlled. Theterm, “plant” is used generally in control theory, and reflects the fact that control theory washistorically first utilized in optimizing large manufacturing facilities.

2.3.1 Closed Loop Transfer Functions

Using what we know about transfer functions and how they apply to signals, it is possible to makethe following statements about Fig. 2.5:

u(s) = C(s)e(s) ≡ C(s) [r(s)− y(s)] (2.30)

andy(s) = P (s)u(s) (2.31)

These two equations can be combined by eliminating e(s) to solve for the closed loop transferfunction from the tracking signal, r(s), to the system output, y(s),

T (s) ≡ y(s)r(s)

=P (s)C(s)

1 + P (s)C(s). (2.32)

T (s) captures all the necessary response information about the behavior of the feedback loop, andit can be used to assess assess both the system stability and the tracking performance.

Definition: Feedback stability– The feedback system characterized the closed-loop transferfunction, T (s), is stable if and only if it contains no poles in the right half complex s-plane,Re(pi) < 0, ∀pi, where pi is a pole of T (s).

The requirement that T (s) has no RHP poles translates into specifications for the plant, P (s),and controller, C(s), transfer functions. This can be seen by considering the denominator of thefeedback transfer function, 1+PC, a quantity that is sometimes referred to as the return difference.Stability of the feedback system is assured provided that 1+PC contains no right-half plane zeros,which correspond to the poles of (1 + PC)−1.

Controller

+

-r(s) y(s)

Plant

u(s)C(s) P(s)

e(s)

Figure 2.5: Flow diagram for a simple feedback control system where the r(s) is the referencesignal, y(s) is the system output, e(s) is the error-signal, u(s) is the control signal, and C(s) andP (s) are, respectively, the controller and plant transfer functions.

23

Technically, there is second requirement for stability: that there are no pole-zero cancella-tions in the loop transfer function,

L(s) = P (s)C(s) . (2.33)

A pole-zero cancellation occurs when two transfer functions with complimentary roots are multipliedtogether. For example, consider,

G1(s) =1

(s + 2)(s− 1), and G2(s) =

(s + 1)(s− 1)(s + 2)

. (2.34)

Clearly, G1 has a RHP pole at z = 1 and is therefore internally unstable. However, G2 has a RHPzero at exactly the same point, s = 1, and when multiplied, these roots cancel,

G(s) = G1G2 =s + 1

(s + 2)2(2.35)

and it appears as if G(s) is internally stable. Therefore, it is theoretically possible to make anunstable plant into a well-behaved device by engineering the feedback controller to contain a RHPzero to match each unstable pole in the plant. This technique is used in practice all the time despitethe fact that it can be an unsound procedure. For instance, an inherently unstable system thatis made stable by feedback in closed-loop configuration will run wildly out of control if somethingbreaks the loop, such as a malfunction in a sensor or a wire. In control applications where instabilitycould lead to a tragedy (like an airplane) pole-zero cancellation should generally be avoided.

2.3.2 The Nyquist Criterion

In practice, a pole zero plot and a sometimes a Bode plot are sufficient to assess the stabilityof a feedback control loop. However, there is another stability analysis technique that you maycome across in the references and control theory literature referred to as the Nyquist stabilitycriterion. It is somewhat more mathematically involved than the previous discussions and requiresa background in complex analysis to fully appreciate. The Nyquist criterion is not a necessarytechnique for engineering a high performance feedback controller, but is instead most useful for itspower in formal mathematical theorems involving feedback control stability. This section providesonly a brief outline of the procedure because we will use it in passing to describe robust stability ina later section. A longer discussion that is extremely accessible can be found in the text by Phillipsand Harbor.2

The general idea behind the Nyquist criterion involves generating a map in the s-plane of theloop transfer function, L(s), (recall that L = PC is the loop transfer function) which correspondsto an infinite contour in the s-plane. For anyone familiar with concepts such as contour integrationand conformal mapping, this will sound normal; to everyone else, it may seem a bit bizarre. Themap is constructed by tabulating the values of L(s), beginning from the point s = 0, and proceedingup the imaginary axis to jω → +∞. The contour then traces a loop in the right half plane of infiniteradius, σ, to the point, jω = −∞. Finally, the contour returns to the s = 0 point by proceedingup the negative imaginary axis. While traversing the imaginary axis, any Re(s) = 0 poles in L(s)

24

-0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1

0.8

0.6

0.4

0.2

0

0.2

0.4

0.6

0.8

Imagin

ary

Axi

s

Real Axis

(-1, 0)

L(s) Contour

-1

Figure 2.6: An example of a Nyquist plot for the single-pole low-pass filter transfer functiondescribed in Eq. (2.29).

should be traversed by encircling them to the left. This contour can be used to generate a Nyquistplot, such as the one in Fig. 2.6 generated from the low-pass filter transfer function in Eq. (2.29).

Nyquist Stability Criterion A feedback transfer function is stable if its L(s) contour does notencircle the point the (0,−1) in the s-plane if there are no right-half plane poles in L(s). Otherwise,stability is only achieved if the number of counterclockwise encirclements of the s = (−1, 0) pointis equal the number of RHP poles in L(s).

The grounds for the Nyquist criterion crome from the Cauchy principle in complex analysiswhich states that the L(s) contour will encircle the (0,0) point for each RHP zero and pole foran s-plane contour that includes the full right-half s-plane. Since we are interested in the zeros of1 + PC for stability, it is convenient to translate everything by subtracting off the 1 and countingthe encirclements of L = PC around the −1 point rather than the origin. If L(s) has no RHPzeros (which would produce instability) then it should not encircle the -1 point, provided that ithas no RHP poles, in which case it should encircle -1 exactly once for each pole. This point isdemonstrated by Fig. 2.6 for the transfer function in Eq. (2.29), which we know to be stable. As iseasily seen, its contour does not encircle the -1 point.

2.4 Tracking Performance

Until now, we have focussed nearly all our attention on developing quantitative measures of stabilityand have payed little attention to other issues such as performance and robustness. In part, this

25

is because stability is the most difficult property to achieve. However, in any good servo design, itis necessary to balance the three considerations of stability, tracking and robustness (this will bethe subject of tomorrow’s lecture). For the time being, it is only necessary to develop quantitativemeasures of these factors.

As can be expected, the feedback transfer function provides a measure of the tracking sinceit describes the relationship between the reference signal, r(s), and the system output, y(s). Ifperfect tracking is achieved, then these two signals will be identical, y(s) = r(s), so that,

T (s) ≡ y(s)r(s)

= 1 =P (s)C(s)

1 + P (s)C(s). (2.36)

Nominally we can see that tracking is improved by increasing the magnitude of C(s) since rear-ranging Eq. (2.36) yields,

C =1 + PC

P⇒ 1

C= 0 (2.37)

and this leads to the common control mantra that gain is good. Of course, it is not generallypossible to make C(t) arbitrarily large because this may cause the stability to suffer.

2.4.1 The Sensitivity Function

The quantity,

1− T (s) ≡ 1− P (s)C(s)1 + P (s)C(s)

(2.38)

provides a quantitative measure of the tracking error. This new function, 1 − T (s), has a specialinterpretation; it is the transfer function from the reference input, r(s), to the error signal, e(s),and is given the symbol, S(s),

S(s) =e(s)r(s)

=1

1 + P (s)C(s). (2.39)

S(s) is referred to as the sensitivity function because in addition to quantifying the tracking abilityof the feedback system, it provides a measure of how changes in P (s) affect the control system. Inorder to see this, we will take a generalized derivative of T (s) with respect to P ,

lim∆P→0

∆T/T

∆P/P=

dT

dP

P

T=

11 + P (s)C(s)

= S(s) (2.40)

and this is, in fact, the sensitivity function, S(s), as defined in Eq. (2.39). Quantifying a requirementon the tracking performance is accomplished by employing a system norm (refer to the Lecture 2Appendices for a review of system norms),

‖S(s)‖∞ < ε (2.41)

to impose the design requirement that the ratio, e(s)/r(s), must not exceed ε. In practice, it iscommon to express the tracking performance using a potentially frequency dependent weightingfunction, W1(s),

‖W1(s)S(s)‖∞ < 1 (2.42)

to quantify the tracking performance requirements of the feedback controller.

26

2.5 Robustness

The basic idea behind robustness is that sometimes, knowledge of the plant transfer function,P (s), is incomplete or slightly wrong. In other cases, environmental fluctuations such as variabletemperature, air pressure, or similar exogenous factors, can be modelled by altering the planttransfer function. The danger with an uncertain plant is that these fluctuations might cause thefeedback loop, which was stable for P (s), to become unstable for P ′(s) = (1 + ∆(s))P (s). In thelanguage that we have developed, this would mean that although P (s) contains no unstable poles(Re(pi) < 0, ∀pi), it might be the case that the perturbation, ∆(s), causes a LHP pole to driftacross the imaginary axis into the right half of the s-plane. Returning to state-space modes, thedynamical matrix, A, corresponding to to stable plant, P , has no RHP eigenvalues, however, A′

does.

In the field of robust control theory, robustness (particularly robust stability) is achieved byconsidering a family of different plant transfer functions,

P = {P (s)(1 + ∆(s)W2(s)), ‖∆‖∞ ≤ 1} (2.43)

that are generated by considering a collection of different perturbations, ∆W2(s). Here, W2 is fixedand is referred to as the weight, while ∆ generates P. This family is referred to as an unstructuredmultiplicative set because of the fact that ∆ and W2 are allowed to take the form of any stabletransfer function.d Physically, ∆W2 is a multiplicative perturbation to P (s) in the sense that theirtransfer functions multiply, which can be thought of as a fixed nominal plant, P (s) in series witha fluctuating plant, ∆W2.e The convenience of the multiplicative uncertainty in Eq. (2.43) is thatis has a simple interpretation in the s-plane. At each frequency, the value of P(s) in the complexplane is given by a disk of radius ‖W2‖ centered at the point, P (s), as is depicted in Fig. 2.7.Robust stability can be expressed using the Nyquist contour method described in Section 2.3.2 andthe graphical description of the ∆W2 uncertainty illustrated by Fig. 2.7. The entire family, P isstable provided that no member of it violates the Nyquist criterion, which graphically correspondsto the requirement that no uncertainty disk enclose the (−1, 0) point in the s-plane.

Quantitatively, the requirement for robust stability of a feedback control system correspondsto the following inequality,

‖W2T (s)‖∞ ≤ 1 . (2.44)

Although the argument is somewhat mathematically involved, the argument is roughly the folowing:it can be shown from complex analysis that the ∞-norm is a measure of the minimum distancebetween the (−1, 0) point and the Nyquist contour of L(s). Using this fact, along with the diskradius measured by ‖W2L(s)‖∞, it is possible to derive Eq. (2.44), although a formal proof issomewhat involved, and is worked through in greater detail in several robust control texts.1,3,4

dAs before, there is a technical requirement that ∆ and W2 not cancel any unstable poles in P (s).eIn the field of robust control theory, it is also customary to consider other unstructured families of plant

perturbations. For more information, a good source is the text by Doyle, Francis and Tannenbaum3

27

0.8 0.6 0.4 0.2 0 0.2 0.4 0.6 0.8 1

0.8

0.6

0.4

0.2

0

0.2

0.4

0.6

0.8

Imagin

ary

Axi

s

Real Axis

P(s)

||W2P(s)||

Figure 2.7: Graphical interpretation of robust stability in the s-plane for a family of transferfunctions generated from the unstructured model, (1 + ∆W2)P (s).

2.6 Lecture 2 Appendices

2.6.1 Review of Laplace and Inverse Laplace Transforms

The Laplace transform of a time-dependent function is defined by,

f(s) = L[f(t)] =∫ ∞

0e−stf(t) dt, s = σ + jω (2.45)

where σ is a real number that is arbitrary provided that the integral converges, i.e. that σ is largerthan a quantity, α, called the radius of convergence. Similarly, the inverse Laplace transform isgiven by the contour integral,

f(t) =12π

∫ σ+j∞

σ−j∞estf(s) ds (2.46)

where, again, σ must be larger than the radius of convergence. Mathematically, the Laplacetransform is a generalization of the Fourier transform, and it is more general because it allows for aspectral (or weighting) decomposition over all exponential functions, est, rather than only sinusoids.

Of particular mathematical importance is the fact that the Laplace transform changes dif-ferential equations into algebraic equations,

L[

d

dtf(t)

]= sf(s) + f(0) (2.47)

and because of the final value theorem,

lims→0

[sf(s)] = f(0) +∫ ∞

0

d

dtf(t) dt = lim

t→∞ [f(t)] (2.48)

28

which relates the steady-state value of f(t) to the Laplace transform, and serves as the justificationfor using Laplace transforms to construct transfer functions of linear dynamical systems in controltheory.

Table 2.1: Laplace transforms that are commonly encountered in control theory and the study oflinear dynamical systems.

f(t), t ≥ 0 f(s) = L[f(t)]1 s−1

t s−2

tn n!s−n−1

e−at (s + a)−1

cos(at) s(s2 + a2)−1

sin(at) a(s2 + a2)−1

2.6.2 System Norms

The term system norm is generally used to describe a norm over a transfer function, G(s), wheres = jω and j =

√−1. These norms must satisfy a set of requirements analogous to those listedabove in Section 1.4.1 for frequency arguments. Commonly encountered norms for transfer functionsare the H2-norm,

‖G(jω)‖2 =(∫ ∞

−∞G(jω) dω

) 12

(2.49)

and the H∞-norm,‖G(jω)‖∞ = sup

ω|G(jω)| (2.50)

It can be shown using techniques from complex analysis that the 2-norm of G(s) is finite iff G isstrictly proper and has no poles on the imaginary axis. Similarly, the ∞-norm of G(s) is finite iffG is proper and has no poles on the imaginary axis.

References

[1] G. E. Dullerud and F. Paganini, A Course in Robust Control Theory: A Convex Approach(Springer Verlag, New York, 2000), 1st ed.

[2] C. L. Phillips and R. D. Harbor, Feedback Control Systems (Prentice Hall, Englewood Cliffs,1996), 3rd ed.

[3] J. Doyle, B. Francis, and A. Tannenbaum, Feedback Control Theory (Macmillan Publishing Co.,1990).

[4] K. Zhou and J. C. Doyle, Essentials of Robust Control (Prentice-Hall, Inc., New Jersey, 1997),1st ed.

29

Lecture 3

Loop Shaping: Balancing Stability, Performance and

Robustness

We left off yesterday with formal mathematical descriptions of stability, performance and robustnessthat were expressed in terms of the feedback transfer function,

T (s) ≡ P (s)C(s)1 + P (s)C(s)

, (3.1)

the sensitivity function,

S(s) ≡ 11 + P (s)C(s)

= 1− T (s) (3.2)

and a set of frequency-dependent weighting functions, W1(s) and W2(s). We saw that the sensitivityfunction provides a measure of the tracking error, e(s), since S(s) is the transfer function from input,r, to error signal, e,

S(s) =e(s)r(s)

. (3.3)

The requirements for tracking and robust stability where then summarized using ∞-norms, toprovide two inequalities,

‖W1S(s)‖∞ < 1 and ‖W2T (s)‖∞ < 1 (3.4)

for specifying tracking and robust stability, respectively. For all of this to be valid, both T and S

must be strictly proper transfer functions, T (s), S(s) → 0, for t →∞. Of course, S(s) = 1− T (s),and therefore, both the feedback transfer function and sensitivity function cannot be simultaneouslymuch smaller than one. This fact leads to a tradeoff between stability and performance— ideallywe wish to satisfy,

‖ |W1S(s)|+ |W2T (s)| ‖ < 1 (3.5)

and with a little algebra, the inequality can be simplified (by removing the sum), to give,1∥∥∥∥

W1S(s)1 + ∆W2T (s)

∥∥∥∥∞

< 1 . (3.6)

In practice, the plant is provided to you— it is the system that needs to be controlled.From the perspective of a control theorist, you have very little freedom to adjust P (s), because this

30

would require physically altering the plant. Therefore, everything in control theory comes downto engineering C(s) in so that Eq. (3.6) is satisfied. In many cases where the plant is sufficientlywell behaved, it will be possible to implement an aggressive feedback controller that provides bothhigh performance, ‖W1S‖ ¿ 1 and good robustness, ‖W2T‖ ¿ 1, without sacrificing stability.Unfortunately, there are occasions where the plant displays some parasitic behavior that makesit difficult to achieve the desired amount of simultaneous tracking, robustness and stability. It isalways necessary to achieve a balance between these three competing design requirements.1

Today, we will explore the trade-offs between tracking, robustness and stability, and willdevelop a procedure for designing C(s). In general, controller design is an immense and involvedsubject, especially when robustness is required. For the most part, I am going to introduce quali-tative guidelines that are inspired by a mathematically rigorous treatment, but I will not formallyprove anything. In nearly all cases typical of a physics laboratory, the techniques that I describework very well.

Controller

+

-r(s)

y(s)

Current

Supplyu(s)C(s)

Plant

e(s)

y(s)

I

R

Figure 3.1: Schematic for a feedback control system typical of one that might be found in a physicslaboratory, where the objective is to cause the output sense voltage, y = IR, to track the program-ming signal, r.

3.1 Example Feedback Controller Design

Our goal is to engineer a stabilizing feedback controller for the system depicted in Fig. 3.1. Theschematic depicts a very common setup found in many physics laboratories, where the goal is toproduce a time-dependent magnetic field by driving an electromagnet. The coil is powered by acurrent supply that accepts a programming input (in the form of a voltage), and a measurementof the magnetic field is performed by monitoring the current flowing through the magnet (weassume that a calibration of I to B via the inductance of the Helmholz coil is well-characterized).The system output, y is obtained from a current→voltage conversion performed by passing themagnet current through a sense resistor, R. And, the feedback objective is to force y to track theprogramming input r, thus resulting in a magnetic field that also tracks r. Here, the plant consistsof the current supply, the coils and the sense resistor, and the controller will be designed such thatit can be implemented using analog electronics.

31

3.1.1 Plant Transfer Function: Internal Stability

The first step in designing the feedback controller is to measure the plant transfer function, P (s),in open loop. This can be accomplished in the laboratory using a network analyzer- wonderfulinstruments which perform a swept sine measurement of the complex response of the system beingtested— they can be configured to output a Bode plot. A plant transfer function measurement forFig. 3.1 would very likely look like the Bode plot in Fig. 3.2. This transfer function is characterizedby a low-frequency gain of approximately 100, but rolls off to its unity gain point around 100 kHz.The phase shift between the current supply programming input and the measured coil current isa relatively complicated function that varies between 0◦ and −270◦ over the full bandwidth. Thisphase profile is likely the result of stray phase shifts in the current supply and the effect of the coilinductance.

101

102

103

104

105

106

107

270

225

180

135

90

45

0

Ph

ase

(d

eg

) Phase Margin (deg): 22.2 Delay Margin (sec): 5.69x106 At frequency (rad/sec): 68.2 k

-100

-50

0

50

Ma

gn

itud

e (

dB

)

Gain Margin (dB): 6.19

At frequency (rad/sec): 101 k

Frequency (rad/sec)

Figure 3.2: Bode plot for P (s) which is typical of one that you might encounter in the physicslaboratory. It has relatively high low frequency gain, rolls off and higher frequency, and has acomplicated phase profile.

The Bode plot in Fig. 3.2 can be fit to a rational polynomial transfer function model, as inEq. (2.24) to give,

P (s) =1015

s3 + (2.01× 105)s2 + (1.02× 1010)s + 1013(3.7)

and the denominator can be factored to find the poles,

pi = {−103,−105,−105} . (3.8)

Clearly, P (s) has no zeros since the numerator in Eq. (3.7) is a constant. An immediate inspectionof the poles of P (s) shows that they are all located in the left half of the complex plane: P has two

32

real poles at −100/2π kHz and one at −1/2π kHz. Therefore, the plant is stable, and similarly,the transfer function is strictly proper,

limω→∞P (s) = 0 (3.9)

since the order of the denominator polynomial in Eq. (3.7) is larger than that of the numerator.Therefore, the limit of P (s) vanishes for s → ∞. These properties should come as no surprisesince most laboratory devices are internally stable on their own and stop responding at some high-frequency cut-off rendering them strictly proper.

3.1.2 Feedback Stability and Robustness

Now what happens when we add feedback? Since, the plant is internally stable, as a first guess, wemight try proportional control by choosing C(s) to be a constant gain. In feedback configurationthis will give a closed-loop transfer function,

T (s) =P (s)

C−1 + P (s)(3.10)

and we see that making C large will lead to good tracking,

limC→∞

T (s) =P (s)P (s)

= 1 (3.11)

and therefore the sensitivity function, S(s), will be zero and the tracking requirement,

‖W1S(s)‖∞ < 1 (3.12)

will be satisfied for all possible choices of W1. So, initially it appears that we can meet our trackingobjective using a simple amplifier with a large gain for our feedback controller.

However, before we consider our job done, we need to check the stability of our proportionalcontrol strategy. Fig. 3.3 provides Nyquist diagrams for two different choices of the controller gain.In plot (A), where C = 1, it clear that the feedback system is stable since the contour does notencircle the (−1, 0) point. However, in plot (B), where C = 3, something went terribly wrongagain— the Nyquist plot most definitively encircles the (−1, 0) point. It appears that increasingthe gain has caused the feedback system to go unstable. A quick check of the poles of the closed-loop transfer function, T (s) = 3P (s)/(1+3P (s)) shows that increasing the gain by a factor of threepushed the two plant poles at −105, into the right half-plane,

pi = {−2.1767× 105, (.0833 + 1.1743j)× 105, (0.0833− 1.1743j)× 105} (3.13)

to produce the instability. Therefore, our plan to increase the gain for good tracking is inherentlyflawed because the system will go unstable for C & 5/2.

The behavior we just observed is also closely related to the concept of robustness.2,3 If weassume that we set the controller gain to unity, C(s) = 1, because we know that this procedure

33

0 1 2 3

2

0

2

-15 -5 0 5

10

0

10

Real Axis

Ima

gin

ary

Axi

s

Real Axis-10

C=1 C=3

Stable Unstable

Figure 3.3: Nyquist plot for P (s) which shows that the plant is described by a stable transferfunction.

will be stable for the plant transfer function that we measured in Fig. 3.2, we ask the question:How much must the plant change before the system becomes unstable? Without performing anydetailed mathematical analysis, such as by computing ‖W2T (s)‖∞, we already know that increasingthe plant gain by a factor of three will produce instability. We know this because in the feedbacktransfer function,

T (s) =P (s)C(s)

1 + P (s)C(s), (3.14)

an overall change in the gain of C(s) is indistinguishable from an identical change in the gain ofP (s). Therefore, using unity gain proportional feedback to control our system is not particularlyrobust since the plant gain need only drift a few dB before the servo stops working. There areprobably some low-quality, high-power laboratory current supplies whose open-loop gain changesby a factor of 2-3 between the time it is turned on, and the time it has warmed up.

3.1.3 Gain Margin, Phase Margin and Bandwidth

Although proportional feedback is stable for C = 1, the system is (in some sense) very close tobeing unstable since only small changes in the plant or controller cause it to fail. What we needis a measure of how far our feedback system is from the stable/unstable border in order to helpus improve the controller design. These requirements lead to two quantities, known as the gainmargin and phase margin which can be easily identified from a Bode plot of the closed-loop feedbacksystem.

The Bode plots depicted in Fig. 3.4 illustrate the concepts of gain and phase margin, andthese points can be used to identify qualitative reasons why proportional control with C = 1 is

34

-40

-30

-20

-10

0

10

Magnitu

de (

dB

)

104

105

-225

-180

-135

-90

-45

0

Phase

(deg)

Frequency (rad/sec)

C(s) = 0.1

C(s) = 1

C(s) = 3

-180o points

0 dB crossing

Figure 3.4: Bode plot of T (s) for proportional controllers with three different gains.

stable while C = 3 is not. The blue curve (labelled with C = 1) depicts the closed-loop transferfunction, T (s). At low frequency, the tracking is good since the gain is very close to 0 dB [meaningno amplification or attenuation of r(s)] and there is little phase shift. However, as the frequencyincreases up to the point ω = 900 kRad/s, the closed-loop gain rises above 0 dB, achieves amaximum and then drops back below the unity gain level (0 dB). Similarly the phase drops tobelow −180◦ near the same frequency, ω = 900 kRad/s.

The frequency at which the phase passes through −180◦ is an extremely important point onthe Bode plot. With −180◦ of phase, y(s) = −r(s) and negative feedback is replaced with positivefeedback, which certainly leads to oscillations instability since positive feedback produces right-halfplane poles in the system transfer function. If there is any amplification (indicated by |T (s)| > 0db) in regions of the Bode plot where there is more than 180◦ of phase, the feedback loop will beunstable because of positive feedback amplification. Stability requires that regions of the Bode plotwith more than 180◦ of phase correspond to frequency components that are attenuated, |T (s)| < 0dB. This concept leads to the definition of the gain and phase margins:

• Phase margin: If ω0 corresponds to a point where the magnitude of T (s) crosses 0 dB,|T (jω0)| = 0 dB, then the phase margin is φ(ω0 − 180◦. It corresponds to the additionalamount of phase that could be tolerated before the system becomes unstable. Feedbacksystems with positive phase margin are stable while those with negative phase margin areunstable.

• Gain margin: If ω−180 corresponds to a point where the phase of T (s) crosses −180◦, thenthe gain margin is defined as −|T (s)| in dB. It corresponds to the amount of attenuation thatpositive feedback frequency components experience. Systems with positive gain margin arestable, while those with negative gain margin are unstable.

35

With these definitions, it is possible to classify the proportional feedback transfer functionsin Fig. 3.4 according to their phase and gain margins,

1. The C = 0.1 feedback system has a large positive phase margin (nearly 90◦) and a gainmargin of nearly 25 dB. It is clearly stable.

2. The C = 1 feedback system has a phase margin of approximately 2◦ and a gain margin of 1dB. It is barely stable.

3. The C = 3 feedback system has a phase margin of −20◦ and a gain margin of −9 dB. It isunstable.

We see that stability can be engineered into T (s) by maximizing both the gain and phasemargins. However, with the improved stability we also expect a decrease in tracking performance.This too can be seen from the Bode plot in Fig. 3.4 where the most stable controller, with C = 1,loses its tracking ability at a lower frequency than the other two. By 104 kRad/s, |T (s)| for theC = 1 controller has already dropped to −3 dB, i.e. the bandwidth of the controller is 104 kRad/s,whereas the bandwidths for C = 1 and C = 3 exceed 105 kRad/s. The tradeoff for stability ismanifest here as reduced effective feedback bandwidth.

3.1.4 Loop Shaping: Improving upon Proportional Control

The control design approach used until now did not allow for any frequency dependence in C(s),and was therefore heavily constrained. We should expect much better bandwidth, stability androbustness from a frequency dependent controller transfer function. Of course, it is also possibleto do worse by making a poor choice for C(s). We want to “shape” the feedback loop by tailoringthe frequency response of the controller transfer function to optimally improve the gain and phasemargin without substantially reducing the bandwidth.

Loop-shaping techniques form a large fraction of the entire field of control theory. Therefore,you can imagine that it is not something that we can thoroughly explore in only a three hour lectureperiod. In most cases, it is possible to perform a relatively straightforward controller design thatinvolves adjusting the frequency response of the loop transfer function, which we recall is given by,

L(s) = P (s)C(s) . (3.15)

The argument for engineering the controller by adjusting L(s) is purely a matter of convenience— itis a quantity that appears in both the closed-loop transfer function, T (s), and sensitivity function,S(s). Therefore it is often possible to satisfy design requirements on both S and T by adjusting asingle function, L. Once the design is completed, the controller is found by factoring out the planttransfer function, P (s), to yield C(s).

We expect that for the same qualitative design objectives, posed in terms as requirementson T (s) and S(s), every loop-shaping application should qualitatively involve the same target loop

36

transfer function, L(s). In order to see this from a more mathematical perspective, our objective willbe to make T = 1 and S = 0 at low frequencies, and then reverse their roles at high frequencies. Therational behind this qualitative objective is that T = 1, S = 0 corresponds to good tracking. And,we expect to be less concerned about the tracking ability above a a certain maximum frequency,ωmax so it is desirable to turn off the feedback at high frequency, T (ω > ωmax) → 0. Therefore, wequalitatively want T (s) to look like a low-pass transfer function,

T (s) ≡ L(s)1 + L(s)

=ωmax

s + ωmax(3.16)

and conversely, we hope to make S(s) into a high-pass transfer function,

S(s) ≡ 11 + L(s)

=s

s + ωmax. (3.17)

In the end, we will choose ωmax to ensure a sufficient degree of phase and gain margin, thus ensuringadequate stability and robustness.

Straightforward algebra involving Eqs. (3.16) and (3.17) shows that the optimal family ofloop transfer functions, L(s), that satisfy our design objectives is given by,

L(s) = ωmax1s

(3.18)

which has a single pole at the origin, p1 = 0. This is the transfer function of integrator, and it ischaracterized by constant −90◦ phase, infinite gain at s = 0 (infinite DC gain) and a constant 6dB per octave roll-off. L(s) has a unity gain point at ωmax and is strictly proper since s−1 → 0 ass →∞. Most importantly an integrator transfer function has infinite phase and gain margins.

Extracting the optimized controller transfer function from L(s), is accomplished by factoringout the plant,

C∗(s) =L(s)P (s)

=ωmax

sP (s). (3.19)

The only remaining issue is that C∗(s) will not be strictly proper if P (s) rolls-off faster than 6 dbper octave, and in most practical applications, it is desirable to have all strictly proper transferfunctions due to non-ideal behavior and modelling uncertainty at high frequency. Fortunately,C∗(s) can be made proper by adding additional poles,

C∗p =

ωmax

sP (s)

(ωp

s + ωp

)n

(3.20)

where ωp must be chosen sufficiently large that it does not affect T (s) at frequencies where goodtracking performance is required. Here, n > 1 is an integer that should be chosen just large enoughto make C(s) strictly proper. Some caution must always be exercised when selecting ωmax and ωp

to ensure that there is sufficient phase and gain margin to achieve the required degree of stabilityand robustness. In practice, this most conveniently found from inspecting Bode plots of T (s) forthe closed-loop system for different values of the parameters, ωmax and ωp. A good rule of thumbfor high precision laboratory control systems is that there should be 60◦ of phase margin and > 20dB of gain margin to provide a comfortable window of stability with a good deal of robustness.

37

3.1.5 Finalized Controller Design

Using the plant transfer function defined in Eq. (3.7), and selecting the values ωmax = 105 rad/sand ωp = 107 rad/s with n = 3, we can finalize our controller design,

C(s) =1029s3 + (2× 1034)s2 + 1039s + 1042

1015s4 + 3(×1023)s3 + (3× 1031)s2 + 1039s. (3.21)

You should not be concerned with the large exponents in the polynomial coefficients. Rememberthat these numbers correspond to powers of ω, which itself has characteristic magnitudes of 10−108

in common laboratory control applications.

-200

-100

0

100

200

Magnitu

de (

dB

)

102

103

104

105

106

107

108

109

-360

-270

-180

-90

0

90

180

Phase

(deg)

Frequency (rad/s)

C(s)

C(s)

T(s)

T(s)

P(s)

P(s)

Figure 3.5: A comparison of the Bode plots for the plant, P (s), optimized controller, C∗(s), andthe loop transfer function, L(s) = P (s)C(s) for our feedback controller designed using loop-shapingtechniques.

Fig. 3.5 shows Bode plots for the loop-shaped controller, C∗(t), and compares it to theoriginal plant transfer function, P (s), and the target loop transfer function, L(s) (ωp/1 + ωp)

n.The red curve is the target loop transfer function, and it is essentially an integrator with a −90◦

phase shift and a 6 dB per octave up to the cutoff-frequency, ωp, where the n = 3 additional polesincrease the roll-off rate and the phase. The green plot corresponds to C(s) and it is clear that is is(in some sense) the inverse of P (s) with respect to L. In regions of the Bode plot where P (s) has asmaller magnitude or less phase that L, C(s) compensates by increasing its magnitude and phaseabove that of an integrator. The opposite is true for regions where P (s) is too large compared tothe integrator transfer function, L(s).

Finally, Fig. 3.6 shows a plot of of the closed-loop transfer function for our feedback system.The tracking bandwidth, which is set by ωmax, is correct. The -3 dB point in Fig. 3.6 is almostexactly located at ωmax = 105 rad/s. At slightly higher frequencies, T (s) rolls-off at 6 dB per octaveand displays an associated −90◦ of phase. There are no points where φ < −180◦ with greater than

38

103

104

105

106

107

108

-180

-135

-90

-45

0

Ph

ase

(d

eg

)-60

-50

-40

-30

-20

-10

0

Ma

gn

itud

e (

dB

)

Gain Margin (dB): 59

At frequency (rad/sec): 5.77e+007

Frequency (rad/sec)

Figure 3.6: Closed-loop feedback transfer function, T (s) for the finalized control system designedusing loop-shaping techniques.

unity gain. In order to verify the stability, we can check the poles of T (s),

pi = {−105,−9× 107,−(1.05 + 0.08j)× 108,−(1.05− 0.08j)× 108} (3.22)

and see that all have negative real parts. Finally, Fig. 3.7 shows a Nyquist plot of the finalizedfeedback system, and it clear that the (−1, 0) is not encircled.

3.2 Digital Controller Design

As a conclusion to our discussion on classical control problems, we will take a quick look at howto repeat the analog controller design in the above section using digital techniques. When workingwith a digital controller, the considerations for stability, robustness and tracking, are extremelysimilar to the analog case. Just as before, there will be a trade-off between stability, trackingbandwidth and robustness, and there are analogies to the Nyquist plot and to the gain and phasemargin that will apply in a digital system. A full discussion of digital filter design can be found inthe very complete text by Proakis and Manolakis4 although this book is not about control theory.It focusses on the the theory of digital systems and how to faithfully implement filters using digitaltechniques.

39

-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1

0.8

0.6

0.4

0.2

0

0.2

0.4

0.6

0.8

0 dB

20 dB

10 dB

6 dB

4 dB2 dB

20 dB

10 dB

6 dB

4 dB

2 dB

Real Axis

Imagin

ary

Axis

Figure 3.7: Nyquist plot for the finalized feedback controller showing that is stable.

3.2.1 Digital Filters and the Z-Transform

Unlike continuous time dynamical systems whose evolution is described using differential equations,digital systems are described by delay-update equations,

xt+1 = Axt + Bu t, t + 1 = t + ∆t . (3.23)

In order to implement all of the controller design techniques that have been developed for analogsystems in a relatively seamless manner, we require a digital version of the continuous-time transferfunction. This is possible using a mathematical construction known as the z-transform,

X(z) =∞∑

i=0

x(ti)z−i (3.24)

where z = σ + jω is a discrete complex variable analogous to the Laplace variable, s. The z-transform converts a function, x(ti), that is defined on the discrete time-line to a function in thediscrete complex plane, referred to as the z-plane. Just like continuous time transfer functions,z-transforms can be represented as ratios of polynomials in z and it is possible to compute theirpoles and zeros, and for example, the Hurwitz criterion applies to the z-plane. In fact, there areseveral procedures which can be used to convert directly from a Laplace transform to a z-transformsuch as the Tustin procedure.4

Therefore, designing a digital controller can be accomplished in one of two ways. If thedynamics are provided in the form of delay-update equations, then the z-transform can be used tocompute digital transfer functions and all the same guidelines for feedback stability, performanceand robustness can be repeated using discrete instead of continuous transfer functions. However,the usual case is that the system dynamics actually correspond to a continuous time system, in

40

which case the controller design can be performed using the analog techniques introduced in thisLecture. Once the analog controller has been designed, it can be converted into a digital filter.Fortunately, there is a substantial amount of digital design software, including functions in theMatlab control toolbox, that will perform the conversion.

3.2.2 FIR and IIR Filters

When building a digital controller, z-transforms can be implemented using two standard types offilters. These are the so-called finite impulse response (FIR) and infinite impulse response (IIR)circuits. FIR filters operate by constructing an output that is a linear combination of inputs atprevious times,

yFIR(tk) =N∑

p=1

cptk−p (3.25)

where N is the number of these previous inputs, and the cj are weighting coefficients. This circuitgets its name from the fact that it cannot produce an infinite output given a finite input signalbecause its output only depends on the input signal. In contrast, IIRs, also permit previous outputsto contribute to the filter output,

yIIR(tk) =N−1∑

p=0

cptk−p +N−1∑

q=0

dqyk−q (3.26)

and can therefore increase without bound due to positive feedback. The requirement for stabilityof an IIR filter is that its coefficients, dq, must be less than unity in order to avoid an output thatincreases without bound.

3.3 Lecture 3 Appendix: Controller Design with Matlab

The Matlab programming environment is an extremely powerful tool for feedback controller designbecause it has a large number of functions (supplied as part of the Control Theory Toolbox). Thesefunctions include routines for forming and manipulating transfer functions, finding poles and zeros,and generating bode plots, Nyquist plots, and pole-zero plots. I used Matlab to generate all of theplots presented in these Lectures, and the Matlab script that follows is the code that I used to doso.

41

% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

% controllerDesign( )

%

% JM Geremia, 25 September - 2 October, 2003

% California Institute of Technology

%

% This Matlab script generates most of the plots used in Lecture 3 for

% the feedback controller design

% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

clear all;

StartFrequency = 1E1; % Starting Frequency Point for plots

StopFrequency = 1E7; % Ending Frequency Point for plots

% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

% CONTINUOUS TIME (ANALOG) CONTROLLER DESIGN

% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

% Measured Plant Transfer Function in continous time

% The Matlab tf( ) function generates a transfer function

% Type "help tf" for more information

PlantTF = tf( [1E15], [1 201000 1.02E10 1E13] );

pole( TF ) % Compute the poles of the plant TF

zero( TF ) % Compute the zeros of the plant TF

% Target Loop transfer function is an integrator that is rolled

OmegaMax = 1E5; % Closing frequency (rad/s)

OmegaRollOff = 1E7; % Roll-off Frequency (rad/s)

n = 3; % Roll-off order for proper controller TF

RollOffTF = tf( [OmegaRollOff], [1 OmegaRollOff] );

TargetLoopTF = tf( [OmegaMax],[1 0] );

% Compute the designed controller from the target loop

% transfer function by dividing out the plant component

ControllerTF = TargetLoopTF / PlantTF * RollOffTF^n;

% Compute the closed-loop transfer function for the feedback system

ClosedLoopTF = PlantTF * ControllerTF / ...

( 1 + PlantTF * ControllerTF );

42

pole( ClosedLoopTF ) % Compute the poles of T(s)

zero( ClosedLoopTF ) % Compute the zeros of T(s)

% Plot the plant, controller and closed-loop transfer functions

% The bode( ) command generates a Bode plot

% Type "help bode" for more information

figure(1);

bode( PlantTF , ’b’, { StartFrequency, StopFrequency } );

hold on;

bode( ControllerTF, ’g’, { StartFrequency, StopFrequency } );

bode( ClosedLoopTF, ’r’, { StartFrequency, StopFrequency } );

hold off;

% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

% DISCRETE TIME (DIGITAL) CONTROLLER DESIGN

% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

DigitalClockFrequency = 2*pi*100E6; % Clock Speed = 100 MHz

% FIR and IIR filters use a specified number of coefficients in order

% to define the filter. This number depends on the hardware

% implementation of the digital filters

NumberOfCoefficients = 14;

% Digital filters are constructed using a certain number of bits

% to digitize the signals. This number must be large enough to give

% the required precision. As a general rule it should be at least

% twice the number of digital memory bits.

NumberOfBits = 28;

% The digital sampling frequency is the clock rate divided by the

% number of memory bits

SampleFrequency = DigitalClockFrequency / NumberOfBits;

SampleTime = 1 / SampleFrequency;

% Here we convert the continuous time transfer functions into digital

% filters using standard techniques such as the "Tustin" approach to

% compute the Z-transform from the Laplace transfer function

% Type "help c2d" for more information

PlantDigitalFilter = c2d( PlantTF, SampleTime, ’tustin’);

43

ControllerDigitalFilter = c2d( ControllerTF, SampleTime, ’tustin’);

pole( ControllerDigitalFilter )

zero( ControllerDigitalFilter )

LoopDigitalFilter = PlantDigitalFilter * ControllerDigitalFilter;

% Generate Bode Plots for the digital filters

figure(2);

hold on;

bode( ControllerDigitalFilter, ’g’, { StartFrequency, StopFrequency } );

bode( PlantDigitalFilter, ’b’, { StartFrequency, StopFrequency } );

hold off;

% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

References

[1] J. Doyle, B. Francis, and A. Tannenbaum, Feedback Control Theory (Macmillan Publishing Co.,1990).

[2] C. L. Phillips and R. D. Harbor, Feedback Control Systems (Prentice Hall, Englewood Cliffs,1996), 3rd ed.

[3] R. C. Dorf and R. H. Bishop, Modern Control Systems (Prentice Hall, Upper Saddle River, NJ,1991), 9th ed.

[4] J. G. Proakis and D. G. Manolakis, Digital Signal Processing: Principles, Algorithms andApplications (Prentice Hall, Upper Saddle River, 1996), 3rd ed.

44

Lecture 4

Feedback Control in Open Quantum Systems

Today, we begin looking at quantum mechanical systems and we will attempt to adapt our knowl-edge of classical feedback control theory into something that applies when the dynamics are gov-erned by quantum mechanics. Of course, quantum mechanics is a very large subject and I willneed to assume some familiarity with it in order to even get started. Then again, many physicistswho study quantum systems, particularly outside the field of quantum optics, are not necessar-ily familiar with theories on continuous measurement and conditional evolution. Therefore, I willspend some time qualitatively developing the equations of motion for a canonical quantum systemin continuous contact with an external, classical environment. In all practical quantum controlapplications, the person building the controller lives in this external classical world. Therefore,the majority of the subject involves understanding how information about the state of the systempropagates from its little quantum setting to our external classical world.

Observation plays a central role in feedback control— there can be no feedback withoutchecking up on the state of the system to ensure that it is doing the right thing. However, ev-erything we have been taught about measurement in quantum mechanics tells us that observationinduces unavoidable randomness due to the projection postulate. Fortunately, quantum systemscan be measured in many different ways, some of which are less intrusive than others. In the worstcase scenario, a projective measurement instantaneously and completely collapses the quantumsystem into a eigenstate of the measurement operator. This drastic, sudden change in the quantumstate appears to be a poor candidate for for feedback control. Instead, “weak measurements” ofthe quantum system result in less drastic fluctuations and only gradually reduce the system to ameasurement eigenstate. Most importantly, weak measurements can be performed continuouslyand provides the backbone of quantum feedback control. There is a very elegant theory for de-scribing the evolution of a quantum system subject to continuous observation known as quantumtrajectory theory.1–3 The approach I take is similar to those of Gardiner and Doherty; however, Iwill not completely derive quantum trajectory equations because this would require too much ofa background in stochastic calculus. Instead, we will qualitatively construct the dynamical argu-ments, state the mathematical results, and then identify the physical significance of the differentterms in the mathematics.

45

4.1 Similarities Between Classical and Quantum Dynamical Systems

In the first lecture we developed a concept of dynamical systems grounded upon an input/ouptutmodel. Therefore, if we hope to adapt any of the earlier material in the course to the quantumregime, it will be necessary to develop a similar intput/output formalism for quantum dynamics.Back in classical control, we described the system dynamics via state-space variables, x(t), thatencode all the essential system information. In linear models, the matrix, A provides the necessaryphysical information to propagate x(t) forward in time starting from an initial condition, x0.Similarly B describes the effect of the control input, u(t) on the system dynamics, and C providesthe relationship between the system state and the observed output, y(t).

We begin looking at the field of quantum control by making associations between the classicaldynamical quantities, x(t),u(t),y(t),A,B and C, and quantities from quantum mechanics. Oneimmediate similarity with Lectures 1-3 is linearity. Quantum mechanics, in most cases, is describedusing linear operators, and so we expect that there will be dynamical analogies (if not equivalences)between classical and quantum dynamical systems. However, at the same time, we expect somenew complications to arise due to the fact that quantum mechanics is a probabilistic theory whileclassical dynamics can be formulated deterministically.

• State-Space → Hilbert-Space: In classical systems, our state-space is given by Rn, ann-dimensional vector space over the field of real numbers, i.e., x(t) ∈ Rn, ∀t. In quantumdynamical systems, the state-space corresponds to the system Hilbert-space, Hn, where n

denotes the dimension. In both quantum and classical dynamical systems, it is possible forn →∞, such as when describing fields.

• State Vector → Density Operator: In classical systems, x(t), represents all of the neces-sary information for describing the system. Instead, quantum mechanics utilizes a wavefunc-tion, |ψ(t)〉, or more generally a density operator, ρ(t), to accomplish this task. The essentialdifference is that the classical state-vector, x(t), has definite values, while the quantum analogrequires a statistical interpretation.

• System Matrix, A, → Liouvillian, L0: In classical mechanics, the internal dynamics weregenerated by the differential equation,

x(t) = Ax,

whereas in the quantum system, the Liovillian, L0 propagates the state,a

d ρ(t) = L0ρ(t) dt .

In the special case where L is a Hamiltonian, H0(t), the dynamics are described by the vonNeumann equation,b

d

dtρ = −i~[H0, ρ] ,

aCertainly the classical system is also described by a Liovillian, even though we did not explicitly followthis route.

bIn the quantum Lectures, I will use i =√−1, rather than j, because of tradition.

46

but, in general, interactions between the quantum system and its surrounding environmentand due to any measurement process, will lead to irreversible dynamics such as decoherence.

• Forcing Term, Bu(t), → Interaction Hamiltonian, u(t)Hfb In classical control, thematrix, B, described the interaction between the driving term, u(t) and the state-variables.In quantum systems, the control will be implemented through an additional term, Hfb, in thesystem Liovillian,

dρ(t) = L0ρ(t) dt− i~u(t)[Hfb, ρ]

that describes the interaction between some external field, such as a laser or magnetic, andthe system, where u(t) is the time-dependent coupling strength.

• Observation Process, y = Cx, → Conditional Evolution: In classical dynamical sys-tems, it was assumed that observing the system through y(t) did not affect the evolution, i.e.there was no term for y in the expression for x(t). However, in quantum mechanics, we knowthat this cannot be the case since measurements result in backaction.

4.2 Continuously Observed Open Quantum Systems

The largest contrast between the classical and quantum control systems (according to the abovecomparisons) is measurement. As suggested above, direct projective measurements on the quantumsystem are poor candidates for the observation process since they are extremely harsh and theyresult in large discontinuous jumps in the state of the quantum system. If we are going to getquantum feedback control theory to look even remotely like classical feedback control, we need tohave an equation of motion for the evolution of the quantum system under the influence of both thecontroller as well as the measurement backaction. However, in order to even begin discussing topicssuch as equations of motion, we need to have some concept of continuity in the system evolution,and this contradicts what we know about projective measurement.

For example, consider the single-spin system that has been prepared into an equal superpo-sition of spin-up and spin-down,

|ψ〉(t) =1√2

( | ↑ 〉+ | ↓ 〉 ) . (4.1)

We know that if we compute the expectation value of the z-component of the spin angular momen-tum, it should be 〈jz〉 = 0. Therefore, with the knowledge that we prepared the spin particle withthe wavefunction in Eq. (4.1), our knowledge of jz is that we expect 〈jz〉 to be zero. Assuming thatthe spin system evolves according to a Hamiltonian that commutes with jz, we should expect thatthe expectation of jz has remained zero. However, at some point we wish to check up on the stateof the system, so we subject it to a Stern-Gerlach measurement of jz, and this projects the spinsystem into either the up or down eigenstate. Naturally, the statement that 〈jz〉 = 0 correspondsonly to the probabilistic nature that upon repeating this experiment many times, we should expectthe Stern-Gerlach measurement to produce spin-up and spin-down with equal likelihood.

47

When we perform the Stern-Gerlach measurement, there was a drastic change in the stateof the system as the value of jz jumps from 0 to either +1/2~ or −1/2~. Now let us assume thatour feedback control goal is to maintain 〈jz〉 = 0 even if there are environmental factors such as amagnetic field that naturally causes the value of 〈jz〉 to change in time. However, if we perform aprojective measurement each time we want to check up on the system to see if it has drifted awayfrom 〈jz〉 = 0, then we force it make a brutal jump away from the control target. This is not verygood.

Instead, what we want is a measurement scheme that is continuous in some limit, i.e. wewant the result of the measurement to only change the value of 〈jz〉 by a small amount,

|ψ〉(t + ∆t) =

√12

+ ε| ↑ 〉+

√12− ε| ↓ 〉 (4.2)

where ε → 0 implies dynamical continuity (in some sense). Here, the time argument, t + ∆t

reflects the fact that the measurement requires some length of time to perform. If we performthese little measurements back to back, then we should expect the state of the system to changeby a little step, εi, after each ∆t. And, since the results of quantum mechanical measurementsare random, we should expect the actual values of ε to be random. This leads to equations ofmotion that have randomness, or measurement noise, built into them, and our goal for today is toderive these equations. As already mentioned, this type of random evolution has lead the field ofcontinuous measurement in quantum mechanics to adopt the formalism of stochastic differentialequations; however, we attempt to identify the key qualitative features of stochastic quantumdynamics without the complexity of stochastic calculus.

4.2.1 Weak Measurements

A measurement scheme that leads to dynamics like Eq. (4.2) can be implemented through indirectobservation of the quantum system where it is first entangled with a disposable meter (such as anoptical mode) followed by a projective measurement on only the meter. This process still involvesbackaction because of the entanglement; however, for only moderate coupling between the systemand meter, the backaction is much less severe and leads to continuous, stochastic evolution. As such,the field of continuous measurement is based solidly on the foundations of stochastic differentialequations.4 This field allows for rigorous derivations of the system evolution equations. We will usesome stochastic calculus, but will not rigorously derive all the equations— it would take too long.This section provides a qualitative development of continuous measurement theory, hopefully, in amanner that will allow us to use stochastic methods later with minimal confusion.

The process of weak measurement is described schematically in Fig. 4.1. The system, ρS,belongs to a Hilbert space, Hs, and the meters, Ri, to a Hilbert space for the reservoir (the collectionof many meters), HR. Initially, the system and reservoir modes are separable because they havenever interacted,

HS+R = HS ⊗HR . (4.3)

48

Quantum System

Meter ( ith Reservoir Mode)

z=z0

t = 0

Quantum System

Meter i+1

z=z0

t = ti

Quantum Systemz=z0

t = ti+z0 /c

Meter iMeter i+2

Entanglement

Figure 4.1: A schematic of weak measurement schemes. A string of meter systems interact withthe quantum system of interest, become entangled with it, and then are detected. By construction,the meters are independent do not interact with one another.

The initial state of the composite system,

χ(0) = ρS(0)⊗ σi(0) (4.4)

where χ is the density operator for both the system and the meter, ρS is the density operator forthe system alone, and σi is the density operator for the ith reservoir mode.c

At time, t = ti, the ith meter interacts with the system as it travels past it. Typically,the meters are provided by modes of an optical field and therefore travel at the speed of light,so the interaction time, ∆t is very short. During this period, the system and ith meter evolvetogether according to an interaction Hamiltonian, HSR, and therefore become entangled, such thatχ 6= ρ⊗ σi is no longer a product state. At time t = ti + z0/c, the mode is measured by projectingthe combined system-reservoir state onto a measurement operator, C = 1S ⊗ CR. Taking a partial

cI apologize for the fact that ρ is the Greek letter for “R” and that σ corresponds to “S”, possibly a poorchoice of notation, but ρ is traditionally the system density operator.

49

trace over the reservoir leads to a new density operator for the system,

ρc(ti + z0/c) = trR[Cχ(ti + z0/c)], (4.5)

where the subscript, c, on ρc indicates that it is conditioned on the outcome of the measurement.In other words, detection provides information about the particular random measurement outcomethat resulted from the observation process, and therefore, our knowledge of the state is a probabilityconditioned on the measurement outcome. At each point in time, there are many possible randomoutcomes that could occur due to the measurement, and conditioning corresponds to a particularevolution path known as a quantum trajectory.1–3

Our goal is to develop an effective equation of motion for ρc(t) by eliminating the reservoirmodes and relegating their influence on the system to terms in the evolution equations that resemblenoise. Since we are propagating a density operator, and due to the measurement randomness, theevolution equation is referred to as a stochastic master equation. In order to derive it, we need tomake several simplifying assumptions about the meters and their interaction with the system:

1. Time-Independent Interaction: It is assumed that the reservoir modes only interact with thequantum system of interest for a length of time, ∆t, that is short compared to all of theinternal dynamics in the system. Therefore, during that time, the Hamiltonian that couplesthem is effectively constant,

HSR =√

M(sr† + s†r) (4.6)

where M is a coupling or Measurement strength with units of frequency,d s is an operatoron the system Hilbert space, HS and r acts on the meter Hilbert space, HR. After the ∆t

period has elapsed and the ith mode has moved past the system, the interaction ceases.

2. Independent and Non-Interacting Meters: The reservoir consists of a long string of metersthat interact sequentially with the system (refer to Fig. 4.1),

R(t) =∑

i

Ri(t) . (4.7)

Each of the Ri are defined to be essentially orthogonal,

[Ri(t), R†j(t)] = δij (4.8)

and therefore,[R(t), R†(t′)] = δ(t− t′) (4.9)

This assumption corresponds to a Markov approximation of the dynamics in the sense thateach mode interacts with the system only for a small window surrounding ti, and thereforethe evolution of ρc at time ti only depends on ρc(ti).

3. Insignificant Delay : Since the modes are travelling at the speed of light, we will assume thatit is unnecessary to worry about the retardation time, z0/c.

dI have chosen to work in units where ~ = 1.

50

As a matter of convention, we will also choose to adopt an interaction picture representation withrespect to the system Hamiltonian,

χ(t) = e−iHSχeiHS (4.10)

and adopt the convention that operators with an explicit time argument are in the interactionpicture, while those without a time argument are Schrodinger picture operators.

4.3 Quantum Input-Output Formalism

In the interaction picture, the propagator for the quantum system is formally given by,

U(ti + ∆t, ti) = ~T exp[√

Ms

∫ ti+∆t

ti

R†in(t

′) dt′ −√

Ms†∫ ti+∆t

ti

Rin(t′) dt′]

(4.11)

where ~T is the time-ordering operator on the exponential. Rin refers to the reservoir operator priorto the interaction, and has been renamed this in order to develop an input/output perspective.The idea is to express the reservoir operator at detection time in terms of Rin and the interactionevolution by defining an output operator,

Rout(t) ' U †(t, 0)Rin(t)U(t, 0) (4.12)

where the interaction time, ti, lies between t = 0 and t, i.e., 0 < ti < t. Since there is no couplingbetween R and S before ti or after ti + ∆t, the evolution operators can be factored,

U(t, 0) = U(t, ti + ∆t)U(ti + ∆t, ti)U(ti, 0) (4.13)

in accordance with the Markov approximation that we made above. For very small ∆t (eventuallywe will take the limit ∆t → dt) the evolution operators can be expanded to first order,

Rout(t) = U †(t, 0)Rin(t)U(t, 0) (4.14)

= U †(t, 0)(Rin(t) +

√Ms

)U(t, 0) (4.15)

= Rin(t) +√

MU †(t, 0)sU(t, 0) (4.16)

and separated into meter and system components using the commutation relations for R(t) givenby [R(t), R†(t′)] = δ(t − t′). This result shows that the reservoir meter, at the time of detection,is given by a superposition of the input meter and a source term (this is effectively a scatteringtheory) determined by the system operator, s.

The detector performs a measurement on the meter Hilbert space at t ' ti + z0/c byacting on Rout(t). Since the output operator should be interpreted in the Heisenberg picture, themeasurement corresponds to an expectation value,

tr[1s ⊗ Rout(t)χ(0)

]= tr

[√MU †(t, 0)sU(t, 0)χ(0)

]= tr

[√Msχ(t)

]=√

M〈s(t)〉 (4.17)

and the important final result is that the measurement outcome is proportional to the systemoperator, s. Here, χ(t) indicates that the density operator has been expressed in an interaction

51

picture, and therefore the system expectation should also be interpreted in that manner. Notethat this result assumes that expectation values over Rin vanish, which is an assumption motivatedby the fact that this is true in the laboratory for reservoirs in their ground state, or a coherentdisplacement from the vacuum.

Since it is most common in the laboratory for the meters to be modes (local in time) of anoptical field such as a probe laser, the measurement result is often referred to as the photocurrent,

y(t) =√

M〈s(t)〉 (4.18)

since it is acquired using a photoreceiver. Of course, y(t) is the system output that we will use forcontrol.

4.3.1 Conditional Evolution

Eq. (4.17) shows us that the measurements made on the meters provide information about thesystem through 〈s〉. However, while this tells us the value of the system operator at the time ofthe interaction, we still need a method for determining the state of the system immediately afterthe measurement has been performed. To accomplish this, we return to the fact that the evolutionis dissected into a sequence of steps, each ∆t in length, during which the evolution equations areconstant in time, Hint = sR†

in + s†Rin. Therefore, we can integrate over this interaction period todefine a set of course-grained operators,

ci =1√∆t

∫ ti+∆t

ti

Rin(t) dt . (4.19)

A quick check of the commutation relations for these new input operators yields,

[ci, c

†i

]=

1∆t

∫ ti+∆t

ti

∫ ti+∆t

ti

[Rin(t′), R

†in(t

′′) dt′ dt′′]

= 1 (4.20)

and justifies the 1/√

∆t normalization, as it preserves the correct commutators. Propagating theconditional density operator, ρc(t), forward in time is accomplished using time-dependent pertur-bation theory for the full density operator, χ(t) followed by a partial trace over the reservoir Hilbertspace. The Born-Dyson expansion of χ(ti + ∆t) under a Markov approximation is given by,

χ(ti + ∆t) = χ(t) +i

~

[H(ti), χ(ti)

]∆t− 1

~2

[H(ti), [H(ti), χ(ti)]

]∆t2 + · · · . (4.21)

In this expression, it is not yet clear how far we must retain terms in the perturbation theory,however, expanding the commutators,

χ(ti + ∆t) = χ(t)− i√

M[c†i s + s†ci, χ(t)

]√∆t− 2M

[c†i s + s†ci,

[c†i s + s†ci, χ(t)

]]∆t (4.22)

shows that the second order terms are of order ∆t and are therefore important. Computing thirdorder terms would show that they are of order, ∆t3/2, and since we will ultimately take a formallimit where, ∆t → 0, these can be safely ignored. We see that several of the terms here have what

52

seem to be unconventional units, such as the square root of time. Although this arises naturally outof the commutation relations given our definition of the course-grained operators, it is characteristicof stochastic calculus, and rigorous justification for both Eqs. (4.21) and (4.22) requires a theoryof stochastic differential equations.

Extracting an expression for the system alone, by taking a partial trace over the reservoirmodes, requires that we properly order all the operators acting on the full Hilbert space. Thisis made much simpler if we assume that each of the meters is in its ground state, such thatapplying annihilation operators, ci, produces zero. Although we will not justify it here, it turns outthat this restriction is not confining because the photon energy ~ω is often large. As a result, theelectromagnetic field is usually very close to zero temperature. Using, the simplification, 1S⊗ ciχ =0, leads to,

χ(ti + ∆t) = ρ⊗ σi − i√

M∆t(sρ(t)⊗ c†σi − ρ(t)s† ⊗ σici

)(4.23)

+M∆t

(sρ(t)s† ⊗ c†i σici − 1

2s†sρ(t)⊗ cic

†i σi − 1

2ρ(t)s†s⊗ σicic

†i

)(4.24)

where, as a reminder, σi is the density operator for the ith reservoir mode. This master equationis fairly typical of ones that we regularly encounter in quantum optics— it has terms which looklike Linblad decay superoperators,

D[s]ρ(t) = sρ(t)s† − 12

(s†sρ(t) + ρ(t)s†s

)(4.25)

meaning that we should expect the measurement to (if nothing else) cause decoherence.

Taking the partial trace of this expression and evaluating the formal limit ∆t → dt requiresthat we specify the particular measurement, such as photon-counting or quadrature detection inorder to proceed. Since most proposed quantum feedback control experiments are expected to usea coherent laser field in order to perform the weak measurement, we will assume a quadraturedetection model such as proposed by Wiseman and Milburn.2 We now need to take a small leap offaith— mathematically this involves applying stochastic calculus to take the ∆t → dt limit

dρc(t) = MD[s]ρc(t)dt +√

M(sρc(t) + ρc(t)s† − tr[(s + s†)ρc(t)]ρc(t)

)dW (4.26)

= MD[s]ρc(t)dt +√

MH[s]ρc(t)dW (t) (4.27)

to give the conditional master equation. We see that there is a prominent Linblad term thatcorresponds to deterministic decay of the system’s coherence with rate, M . There is also anadditional stochastic term. The dW (t) is referred to as an Ito increment, and it is a random processthat is Gaussian distributed with mean zero, and variance dt. In the language of stochastic calculus,the symbol E[· · · ], refers to an average over many different stochastic trajectories. The stochasticincrements have the property that E[dW (t)] = 0 and that E[dW 2] = dt. The important distinctionbetween normal differential and Ito stochastic calculus is that the chain rules are different. Wewill not pursue stochastic differential equations in any depth in this course, only to say that Eq.(4.26) provides a recipe for propagating the conditional density matrix. In a simulation, one wouldnumerically integrate the stochastic master equation by randomly selecting values of dW (t) from

53

a Gaussian distribution with variance δt (the size of the integration step) during each integrationstep.

4.3.2 The Conditional Innovation Process

We will simply state the quantum trajectory result for the photocurrent,

y(t)dt = 2√

M〈s〉c(t)dt + dW (4.28)

where the dW (t) term now has a physical interpretation. It is the optical shotnoise on the probelaser used to perform the weak measurement, and is defined by,

dW (t) ≡ y(t)dt− 2√

M〈s〉c(t) . (4.29)

Although this definition appears to be circular given Eq. (4.28), Eq. (4.29) should be taken asmore fundamental. It has the following physical interpretation: Given knowledge of the initialsystem state, ρ(0), one can propagate the system density operator forward in time from t = 0 tot = dt according to Eq. (4.26). At this point, a measurement of 〈s〉 is performed; however, dueto backaction and shotnoise, it will not exactly agree with the computed expectation value. Ofcourse, we should trust the measurement over our stochastic differential equation, and thereforewe need to correct the evolution equation by adding in the effect of the residual, or the differencebetween the measurement and computed value, y(t)dt− 2

√M〈s〉c(t)dt. This evolution is known as

the innovation process, because at each time when we compare the measurement to our predictedvalue, we are a little bit surprised that the two do not agree. This surprise, of course, results fromthe inherent randomness of the quantum measurement process.

4.4 Quantum Feedback Control

Most of the hard work has already been done in developing the quantum trajectory equations ofmotion for the system under continuous evolution. We see, that as expected, the measurementproduces a certain randomness to the evolution in the form of a stochastic differential equation anda noisy photocurrent. However, this process does provide a continuous stream of information aboutthe system operator, 〈s〉c(t), making it reasonable to consider a feedback scheme that controls 〈s〉by driving the quantum system with a control signal computed from the photocurrent. The basictheory of feedback control using quantum trajectory theory was introduced in a set of landmarkpapers by Wiseman and Milburn that are summarized in Howard Wiseman’s dissertation.5

This additional driving term appears in the stochastic master equation as a feedback Hamil-tonian, HFb,

dρc(t) = −i[u(t)HFb, ρc(t)

]dt + MD[s]ρc(t)dt +

√MH[s]ρc(t) dW (t) (4.30)

54

where the feedback strength, u(t), is computed from the past history of the photocurrent,

u(t) =∫ t

0F (t− τ)Ic(τ) dτ . (4.31)

Remembering all the way back to Lecture 2, this equation looks very similar to Eq. (2.8). Thedifficulty, however, is that Eq. (4.31) involves a stochastic quantity, and computing the convolutionintegral cannot be relegated to a steady-state transfer-function analysis, as was done in classicalcontrol. The conditional evolution of the quantum system is not a steady-state process, and instead,a quantum feedback controller would need to integrate Eq. (4.31) in real-time. The difficulty isthat performing the feedback convolution integral is computationally difficult because it requirespropagating the stochastic master equation— the evolution reflects a different measurement historyon each trajectory because of inherent randomness. Current proposals for quantum feedback controlattempt to circumvent this problem in two ways: one is to use high-speed digital processing6 and theother is to employ quantum filtering techniques7–10 to simplify the real-time convolution integral.These more advanced quantum control topics are something we may consider discussing tomorrowin the final “special topics” lecture.

References

[1] H. Carmichael, An open-syemtems approach to quantum optics (Springer Verlag, Berlin, 1993).

[2] H. M. Wiseman and G. J. Milburn, Phys. Rev. A 47, 642 (1993).

[3] C. Gardiner, Quantum Noise (Springer Verlag, New York, 1991).

[4] C. W. Gardiner, Handbook of Stochastic Methods (Springer, New York, 1985), 2nd ed.

[5] H. Wiseman, Ph.D. thesis, University of Queensland (1994).

[6] M. A. Armen, J. K. Au, J. K. Stockton, A. C. Doherty, and H. Mabuchi, Phys. Rev. Lett 89,133602 (2002).

[7] H. Mabuchi, Quantum Semiclass. Opt. 8, 1103 (1996).

[8] V. Belavkin, Rep. on Math. Phys. 43, 405 (1999).

[9] F. Verstraete, A. C. Doherty, and H. Mabuchi, Phys. Rev. A 64, 032111 (2001).

[10] J. Geremia, J. K. Stockton, A. C. Doherty, and H. Mabuchi, quant-ph/0306192 (2003).

55

Lecture 5

Special Topics in Quantum Control

This lecture is intended to focus on special topics in quantum control.

1. Adaptive Measurements It was shown by the group of Wiseman that it is possible to reachthe quantum noise limit for phase measurement of a weak optical pulse with unknown initialphase by performing an adaptive measurement.3 This procedure was then demonstrated atCaltech.4

2. Heisenberg Limited Magnetometry Thomsen, Mancini and Wiseman1 have shown thatit is possible to produce spin-squeezing using feedback. This technique can be used, inconjunction with quantum Kalman filtering, to achieve the quantum noise limit for measuringthe magnitude of a magnetic field.2

References

[1] L. K. Thomsen, S. Mancini, and H. M. Wiseman, Phys. Rev. A 65, 061801 (2002).

[2] J. Geremia, J. K. Stockton, A. C. Doherty, and H. Mabuchi, quant-ph/0306192 (2003).

[3] H. Wiseman and R. Killip, Phys. Rev. A 57, 2169 (1998).

[4] M. A. Armen, J. K. Au, J. K. Stockton, A. C. Doherty, and H. Mabuchi, Phys. Rev. Lett 89,133602 (2002).

56

Lecture 5:Feedback and robustness in quantum metrology

adaptive homodyne measurement of optical phaseMike Armen, John Au, Andrew Doherty, John Stockton

loop-shaping for robust phase estimationMike Armen, Ramon van Handel

spin squeezing, Kalman filtering, and atomic magnetometry Andrew Doherty and John Stockton

robust coherent broadband magnetometryJohn Stockton

Hideo Mabuchi

NSF, ONR, ARO/DARPA MURI

Measuring optical phase

ϕ

θ

θ-ϕ

Local oscillator

Signal

approximates canonical phase measurement,given prior knowledge of mean phase

and photon number

Adaptive homodyne measurementH. M. Wiseman, PRL 75, 4587 (1995)

ϕ dsp

Local oscillator

Signalϕ1

ϕ2

ϕ3

…variance within 1% of canonical phase measurement for h n i & 10 photons !

Dolinar, Yamamoto, …

Technical challenges – laser noise

NPRONd:YAG

high-Qresonator(cavity)

τ ~ 16 µs

Adaptive Homodyne Results

PRL 89, 133602 (2002)

quadraturehomodyne!

Balanced homodyne detection

ϕ

θ

Local oscillator

Signal

0

−

2αβ

Robust phase estimation

Balanced homodyne detection is sensitive to optical amplitude noise

• Model amplitude noise as multiplicative uncertainty: α = |α|(1+∆2W2) , where ∆2W2 represents the ‘spectrum’ of uncertainty.

• Treat phase estimation as an input-output tracking problem.

• Design goal: specify an ideal tracking function, Mideal, and require that the closed loop response be within ε of it: || MCL – Mideal ||∞ < ε.

If we consider the linearized phase estimation problem, we can use the machinery of robust control theory to suppress sensitivity to noise

2αβ

C1

C2Pϕϕ

θ

n

ϕ

-

∆2W2

The robust performance problemThis approach yields the H∞ robust performance problem:

• The inequality above guarantees closed loop stability and that the design goal is met, in the presence of model uncertainty.

• The goal is to design a feedback controller, C1, such that Γ < 1.

• The ‘performance weight’, W1, depends only on W2, Mideal, and ε.

• Given Mideal and ε, we can state a necessary condition on W2 for the problem to be solvable, but no algorithmic approach to a solution exists.

However, an algorithmic approach exists for solving the nearby problem:

• By using the Euclidian-like norm, we can transform this problem into the ‘model-matching’ problem, which is solved using interpolation theory(co-prime factorization + controller parametrization + Nevanlinna-Pick).

• We can easily search for the minimal ε (for example) before solving for C1.

Bode Diagram

Frequency (rad/sec)

Phas

e (d

eg)

Mag

nitu

de (d

B)

-150

-100

-50

0

50

100

10-4

10-3

10-2

10-1

100

101

102

103

104

-180

-90

0

90

180

Pφ

Mideal

A specific problem

• Mideal chosen for tracking of a cwsignal at 1 rad/sec, with negligible error.

• Pφ chosen to provide optimal signal-to-noise.

• ε set to 0.01.

Given the above choices of Mideal, Pφ, and ε, a choice of W2 will determine the optimal controller (if it exists).

100

10-4

10-3

10-2

10-1

100

101

Blue: W1 Black: W2 Red: Γerr

Frequency (rads/sec)

Mag

nitu

deLoop Bode Plot

Frequency (rad/sec)

Phas

e (d

eg)

Mag

nitu

de (d

B)

-40

-20

0

20

40

60

100

-270

-180

-90

0

90

180

100

10-4

10-3

10-2

10-1

100

101

Blue: W1 Black: W2 Red: Γerr

Frequency (rads/sec)

Mag

nitu

deLoop Bode Plot

Frequency (rad/sec)

Phas

e (d

eg)

Mag

nitu

de (d

B)

10

20

30

40

50

60

100

-135

-90

-45

0

45

90

135

100

100

101

102

103

Blue: W1 Black: W2 Red: Γerr

Frequency (rads/sec)

Mag

nitu

deLoop Bode Plot

Frequency (rad/sec)

Phas

e (d

eg)

Mag

nitu

de (d

B)

-80

-60

-40

-20

0

20

40

100

-540

-450

-360

-270

-180

-90

0

Some results for various W2

10 15 20 25 30 35-0.2

-0.1

0

0.1

0.2Scaled Interference Fringe (without feedback)

10 15 20 25 30 35-0.2

-0.1

0

0.1

0.2Cyan: True Phase Red: Phase Estimate (w/o feedback) Blue: Phase Estimate (robust controller)

10 15 20 25 30 350

2

4

6

8

x 10-3 Integrated Errors ---- Red: No Feedback Blue: Robust Controller

10 15 20 25 30 35-0.2

-0.1

0

0.1

0.2Scaled Interference Fringe (without feedback)

10 15 20 25 30 35-0.2

-0.1

0

0.1

0.2Cyan: True Phase Red: Phase Estimate (w/o feedback) Blue: Phase Estimate (robust controller)

10 15 20 25 30 350

1

2

3x 10-3 Integrated Errors ---- Red: No Feedback Blue: Robust Controller

Case 2: Broadband amplitude noise (w/o shot noise)

Verification of robust performance

• Shot noise + sinusoidal amplitude noise

• Real-time phase estimates

• The robust controller provides a factor of 10 improvement over no feedback.

Closed loop magnetometry: experiment

Continuous Measurement localization onto “quantum trajectories”

Continuous QND measurement of atomic spin

Conditioning of evolution on observed photocurrent

gives rise to stochastic localization of Jz on

timescale ~ J-1

Spin squeezing and atomic magnetometry

squeezing improves sensitivity…

but how to distinguish Bfrom projection noise?

(further analogy with optical squeezing)

Continuous QND measurement of atomic spin

These equations have the exact same form as a set of classical

equations known as the

Kalman-Bucci Filtering Equations

They ignore all the white noise

The Quantum Kalman Filter

Matrix Riccati Equation

Optimal estimation via Kalman filtering

• optimal “gain scheduling”for noise model, via matrix Riccati equation

• “quantum” Kalman filter is equivalent to Kalman filter for linear model plus stationary noise

• stochastic localization can be absorbed into random initial Jz consistent with coherent state variance

Simple magnetometry is sensitive to shot-to-shot variation in atom number

Photocurrent:

Estimated Slope:

Correct Estimate:

Minimal Error:

Incorrect assumed number:

represents shot-to-shot fluctuations of actual number

Incorrect estimate:

Resulting error:

Open-loop sensitivity to parameter variation

⇒ active magnetic field cancellation (field lock loop)

Broadband atomic magnetometry with feedback

spin=J

(servo)

Jmin

Robustness of closed loop magnetometry (theory)

Transfer Function Measurements: IF BW = 100-300 Hz, N ~ 9.5 x 108

Error bars accumulated from 100 TF measurements

Feedback controller design (simple loop-shaping)

(A) Applied field turned on at 0.5 ms. Resulting Larmor precession, feedback controller acquires its lock in ~ 100 µs

Atom Number ~ 109, Feedback Controller Closes at 86 kHz

Tracking static and non-stationary magnetic fields

Field Tracking Error measured by computing E[(Best-B)2] for 50 samples with P=50µW, ∆=2.0 GHz, Controller Gain = 500 (feedback system closing at 86 kHz)

Demonstration of robust estimation (un-squeezed)