An Introduction to Control Theory

With Applications to Computer Science

Joseph HellersteinAnd

Sujay ParekhIBM T.J. Watson Research

Center{hellers,sujay}@us.ibm.com

2

Example 1: Liquid Level System

H

Input valve control

float

Output valve

Goal: Design the input valve control to maintain a constant height regardless of the setting of the output valve

iq

oqV

(height)

(input flow)

(output flow)

(volume)

R(resistance)

3

Example 2: Admission Control

Users

Administrator

Controller

RPCs

Sensor

Server

Referencevalue

QueueLength

Tuningcontrol

ServerLog

Goal: Design the controller to maintain a constant queue length regardless of the workload

4

Why Control Theory Systematic approach to analysis and design

Transient response Consider sampling times, control frequency Taxonomy of basic controls Select controller based on desired characteristics

Predict system response to some input Speed of response (e.g., adjust to workload

changes) Oscillations (variability)

Approaches to assessing stability and limit cycles

5

Example: Control & Response in an Email Server

Control(MaxUsers)

Response(queue length)

Good

Slow

Bad

Useless

6

Examples of CT in CS Network flow controllers (TCP/IP – RED)

C. Hollot et al. (U.Mass) Lotus Notes admission control

S. Parekh et al. (IBM) QoS in Caching

Y. Lu et al. (U.Va) Apache QoS differentiation

C. Lu et al. (U.Va)

7

Outline Examples and Motivation Control Theory Vocabulary and

Methodology Modeling Dynamic Systems Standard Control Actions Transient Behavior Analysis Advanced Topics Issues for Computer Systems Bibliography

8

Feedback Control System

PlantController)(tu

)(tnDisturbanc

e

)(ty)(tr

)(te

Transducer

)(tb

)()()( tbtrte +

–

Reference Value

9

Controller Design Methodology

Block diagram

construction

Model

Ok?

Stop

Start

Transfer function formulation and

validation

Controller Design

Objective

achieved?

Controller Evaluation

Y

Y

N N

System Modeling

10

Control System Goals Regulation

thermostat, target service levels Tracking

robot movement, adjust TCP window to network bandwidth

Optimization best mix of chemicals, minimize

response times

11

System Models Linear vs. non-linear (differential

eqns) eg, Principle of superposition

Deterministic vs. Stochastic Time-invariant vs. Time-varying

Are coefficients functions of time? Continuous-time vs. Discrete-time

t R vs k Z

xbxbyaya 0201

12

Approaches to System Modeling First Principles

Based on known laws Physics, Queueing theory

Difficult to do for complex systems Experimental (System ID)

Statistical/data-driven models Requires data Is there a good “training set”?

13

The Complex Plane (review)

Imaginary axis (j)

Real axis

jyxu

x

y

r

r

jyxu (complex) conjugate

y

22

1

||||

tan

yxuru

x

yu

14

Basic Tool For Continuous Time: Laplace Transform

Convert time-domain functions and operations into frequency-domain f(t) F(s) (t, s Linear differential equations (LDE) algebraic

expression in Complex plane Graphical solution for key LDE characteristics Discrete systems use the analogous z-

transform

0)()()]([ dtetfsFtf stL

15

Laplace Transforms of Common FunctionsName f(t) F(s)

Impulse

Step

Ramp

Exponential

Sine

1

s

1

2

1

s

as 1

22

1

s

1)( tf

ttf )(

atetf )(

)sin()( ttf

00

01)(

t

ttf

16

Laplace Transform Properties

)(lim)(lim

)(lim)0(

)()()

)(1)(

)(

)0()()(

)()()]()([

0

0

2121

0

2121

ssFtf-

ssFf-

sFsFdτ(ττ)f(tf

dttfss

sFdttfL

fssFtfdt

dL

sbFsaFtbftafL

st

s

t

t

theorem valueFinal

theorem valueInitial

nConvolutio

nIntegratio

ationDifferenti

calingAddition/S

17

Insights from Laplace Transforms What the Laplace Transform says about

f(t) Value of f(0)

Initial value theorem Does f(t) converge to a finite value?

Poles of F(s) Does f(t) oscillate?

Poles of F(s) Value of f(t) at steady state (if it converges)

Limiting value of F(s) as s->0

18

Transfer Function Definition

H(s) = Y(s) / X(s) Relates the output of a linear

system (or component) to its input Describes how a linear system

responds to an impulse All linear operations allowed

Scaling, addition, multiplication

H(s)X(s) Y(s)

19

Block Diagrams Pictorially expresses flows and

relationships between elements in system

Blocks may recursively be systems Rules

Cascaded (non-loading) elements: convolution

Summation and difference elements Can simplify

20

Block Diagram of System

PlantController)(sU

)(sN

Disturbance

)(sY

)(sR)(sE

Transducer

)(sB

+

–

)(1 sG )(2 sG

)(sH

Reference Value

21

Combining Blocks

Combined Block)(sY

)(sR)(sE

Transducer

)(sB

+

–

)(sH

)())()(( 21 sGsNsG

Reference Value

22

Block Diagram of Access Control

M(z)

G(z) N(z) S(z)

Controller Notes Server

Sensor

R(z)+

E(z)U(z)

-

Q(z)

Users

Controller

Sensor

ServerServerLog

23

Key Transfer Functions

)()()(

)(

)(

)(

)(

)(21 sGsG

sE

sU

sU

sY

sE

sY :eedforwardF

)()()()(

)(21 sHsGsG

sE

sB :Loop-penO

)()()(1

)()(

)(

)( :

21

21

sHsGsG

sGsG

sR

sY

Feedback

PlantController)(sU

)(sY

)(sR)(sE

Transducer)(sB

+

–

)(1 sG )(2 sG

)(sH

Reference

24

Rational Laplace Transforms

m

sFsAs

sFsBs

bsbsbsB

asasasA

sB

sAsF

mm

nn

poles # system ofOrder

complex are zeroes and Poles

(So, :Zeroes

(So, :Poles

)0*)(0*)(*

)*)(0*)(*

...)(

...)(

)(

)()(

01

01

25

First Order System

Reference

)(sY)(sR

)(sE

1)(sB

)(sUsT1

1K

sT

K

sTK

K

sR

sY

11)(

)(

26

First Order System

Impulse response

Exponential

Step response

Step, exponential

Ramp response

Ramp, step, exponential

1 sT

K

/1

2 Ts

KT-

s

KT-

s

K

/1

Ts

K-

s

K

No oscillations (as seen by poles)

27

Second Order System

:frequency natural Undamped

where :ratio Damping

(ie,part imaginary zero-non have poles if Oscillates

:response Impulse

J

K

JKBB

B

JKB

ssKBsJs

K

sR

sY

N

cc

NN

N

2

)04

2)(

)(

2

22

2

2

28

Second Order System: Parameters

noscillatio the offrequency the gives

frequency natural undamped of tionInterpreta

0)Im0,(Re Overdamped 1

Im) (Re dUnderdampe

0)Im 0,(Re noscillatio Undamped

ratio damping of tionInterpreta

N

:

0:10

:0

29

Transient Response Characteristics

statesteady of % specified within stays time Settling :

reached is valuepeak whichat Time :

valuestatesteady reachfirst untildelay time Rise :

valuestatesteady of 50% reach untilDelay :

s

p

r

d

t

t

t

t

0.5 1 1.5 2 2.5 3

0.25

0.5

0.75

1

1.25

1.5

1.75

2

rt

overshoot maximumpM

pt stdt

30

Transient Response Estimates the shape of the curve

based on the foregoing points on the x and y axis

Typically applied to the following inputs Impulse Step Ramp Quadratic (Parabola)

31

Effect of pole locations

Faster Decay Faster Blowup

Oscillations(higher-freq)

Im(s)

Re(s)(e-at) (eat)

32

Basic Control Actions: u(t)

:control alDifferenti

:control Integral

:control alProportion

sKsE

sUte

dt

dKtu

s

K

sE

sUdtteKtu

KsE

sUteKtu

dd

it

i

pp

)(

)()()(

)(

)()()(

)(

)()()(

0

33

Effect of Control Actions Proportional Action

Adjustable gain (amplifier) Integral Action

Eliminates bias (steady-state error) Can cause oscillations

Derivative Action (“rate control”) Effective in transient periods Provides faster response (higher sensitivity) Never used alone

34

Basic Controllers Proportional control is often used

by itself Integral and differential control are

typically used in combination with at least proportional control eg, Proportional Integral (PI)

controller:

sTK

s

KK

sE

sUsG

ip

Ip

11

)(

)()(

35

Summary of Basic Control Proportional control

Multiply e(t) by a constant PI control

Multiply e(t) and its integral by separate constants Avoids bias for step

PD control Multiply e(t) and its derivative by separate constants Adjust more rapidly to changes

PID control Multiply e(t), its derivative and its integral by

separate constants Reduce bias and react quickly

36

Root-locus Analysis Based on characteristic eqn of closed-loop

transfer function Plot location of roots of this eqn

Same as poles of closed-loop transfer function Parameter (gain) varied from 0 to

Multiple parameters are ok Vary one-by-one Plot a root “contour” (usually for 2-3 params)

Quickly get approximate results Range of parameters that gives desired

response

37

Digital/Discrete Control More useful for computer systems Time is discrete

denoted k instead of t Main tool is z-transform

f(k) F(z) , where z is complex Analogous to Laplace transform for s-domain

Root-locus analysis has similar flavor Insights are slightly different

0

)()()]([k

kzkfzFkfZ

38

z-Transforms of Common FunctionsName f(t) F(z)

Impulse

Step

Ramp

Exponential

Sine

1

1z

z

2)1( z

z

aez

z

1)(Cos2

Sin2 zaz

az

1)( tf

ttf )(

atetf )(

)sin()( ttf

F(s)

1

s

1

2

1

s

as 1

22

1

s

00

01)(

t

ttf

39

Root Locus analysis of Discrete Systems Stability boundary: |z|=1 (Unit circle) Settling time = distance from

Origin Speed = location relative to Im

axis Right half = slower Left half = faster

40

Effect of discrete poles

|z|=1

Longer settling time

Re(s)

Im(s)

Unstable

Stable

Higher-frequencyresponse

Tsez :Intuition

41

System ID for Admission Control

M(z)

G(z) N(z) S(z)

Controller Notes Server

Sensor

R(z)+

E(z)U(z)

-

Q(z)

ARMA Models

Control Law

Transfer Functions

zz

zK

cz

dzd

az

zbzGzSzN i 1

1)()()(

1

10

1

0

Open-Loop:

zz

zKzG

cz

dzdzS

az

zbzN

i 1

1)(

)(

)(

1

10

1

0

)()1()(

)1()()1()(

)()1()(

101

01

teKtutu

tqdtqdtmctm

tubtqatq

i

42

Root Locus Analysis of Admission Control

Predictions:•Ki small => No controller-induced oscillations•Ki large => Some oscillations•Ki v. large => unstable system (d=2)•Usable range of Ki for d=2 is small

43

Experimental Results

Control(MaxUsers)

Response(queue length)

Good

Slow

Bad

Useless

44

Advanced Control Topics Robust Control

Can the system tolerate noise? Adaptive Control

Controller changes over time (adapts) MIMO Control

Multiple inputs and/or outputs Stochastic Control

Controller minimizes variance Optimal Control

Controller minimizes a cost function of error and control energy

Nonlinear systems Neuro-fuzzy control Challenging to derive analytic results

45

Issues for Computer Science Most systems are non-linear

But linear approximations may do eg, fluid approximations

First-principles modeling is difficult Use empirical techniques

Control objectives are different Optimization rather than regulation

Multiple Controls State-space techniques Advanced non-linear techniques (eg, NNs)

46

Selected Bibliography Control Theory Basics

G. Franklin, J. Powell and A. Emami-Naeini. “Feedback Control of Dynamic Systems, 3rd ed”. Addison-Wesley, 1994.

K. Ogata. “Modern Control Engineering, 3rd ed”. Prentice-Hall, 1997. K. Ogata. “Discrete-Time Control Systems, 2nd ed”. Prentice-Hall, 1995.

Applications in Computer Science C. Hollot et al. “Control-Theoretic Analysis of RED”. IEEE Infocom 2001 (to appear). C. Lu, et al. “A Feedback Control Approach for Guaranteeing Relative Delays in Web

Servers”. IEEE Real-Time Technology and Applications Symposium, June 2001. S. Parekh et al. “Using Control Theory to Achieve Service-level Objectives in

Performance Management”. Int’l Symposium on Integrated Network Management, May 2001

Y. Lu et al. “Differentiated Caching Services: A Control-Theoretic Approach”. Int’l Conf on Distributed Computing Systems, Apr 2001

S. Mascolo. “Classical Control Theory for Congestion Avoidance in High-speed Internet”. Proc. 38th Conference on Decision & Control, Dec 1999

S. Keshav. “A Control-Theoretic Approach to Flow Control”. Proc. ACM SIGCOMM, Sep 1991

D. Chiu and R. Jain. “Analysis of the Increase and Decrease Algorithms for Congestion Avoidance in Computer Networks”. Computer Networks and ISDN Systems, 17(1), Jun 1989