representing and working with uncertainty in dynamical systemspeople.isy.liu.se › en › rt ›...

Complex modeling, Convergence and Uncertainty QuantificationUppsala, Sweden

Representing and working with uncertainty in dynamical systemsThomas Schön, [email protected]

Thomas SchönDivision of Automatic ControlLinköping UniversitySweden

www.control.isy.liu.se/~schon

Representing and working with uncertainty in dynamical systems

Special thanks to: Jeroen Hol (Xsens Technologies), Lennart Ljung (LiU), Johan Kihlberg (Xdin), Fredrik Lindsten (LiU), Henk Luinge (Xsens Technologies), Michael Jordan (Berkeley), Brett Ninness (University of Newcastle), Per-Johan Nordlund (Saab), Simon Tegelid (Xdin) and Adrian Wills (University of Newcastle).

Representing and working with uncertainty in dynamical systems Thomas Schön, [email protected]


Dynamical systems

What is a dynamical system?

“The present state of a dynamical system depends on its history.”

Dynamics

Measurements

Initial state

Known inputState

Measurements

Static parameters

xt+1 | xt ⇠ f✓(xt+1 | xt, ut),

yt | xt ⇠ h✓(yt | xt, ut),

x1 ⇠ µ✓(x1).



Outline

1. Representation - probabilistic state space models (SSM’s)

2. State inference

a) General solutionb) LGSS models and the Kalman filterc) Sensor fusion exampled) Particle filter for general SSM’s via positioning examples

3. Parameter inference

a) Problem formulationb) Bayesian solution - particle MCMCc) System identification example - semiparametric Wiener model

xt+1 | xt ⇠ f✓(xt+1 | xt, ut),


x1 ⇠ µ✓(x1).



Probabilistic models of dynamical systems

We often model a dynamical system using probability density functions (pdf’s)

Dynamics

Measurements

Initial state

Known inputState

Measurements

Static parameters

xt+1 | xt ⇠ f✓(xt+1 | xt, ut),


x1 ⇠ µ✓(x1).

The state process is hidden (latent) and it is observed indirectly via the measurement process.

This type of model is referred to as a state space model (SSM) or a hidden Markov model (HMM).

Model = pdf



Probabilistic models of dynamical systems

Alternative model formulation 1 (common in engineering):

xt+1 = ef✓(xt, ut) + v✓,t,

yt = eh✓(xt, ut) + e✓,t,

x1 ⇠ µ✓(x1).

Uncertainty in the model

Uncertainty inthe measurements

Alternative model formulation 2(graphical model):

The state is a variable that contains all information about the past and the present of a system, which is needed in order to predict the future.

It is the Markov property

that allows for this.

p(xt+1 | x1, . . . , xt) = p(xt+1 | xt)



The use of probabilistic models

The SSM can be used to answer many questions, where two of the most common are:

1. State inference: Infer the states from the available measurements.2. Parameter inference: Infer the static model parameters from the available measurements.

Answering the second question typically involves solving various state inference problems.

System identification deals with the problem of finding a dynamical model based on measurements of the input signal and the output signal,

u1:N = {u1, . . . , uN}, y1:N = {y1, . . . , yN}.

Sensor fusion is the process of using information from several different sensors to learn (estimate) what is happening (this typically includes states of various dynamical systems and various static parameters).



State inference

The aim is to compute a probabilistic representation of our knowledge of the state, based on information that is present in the measurements.

The filtering probability density function (pdf) provides a good representation of the uncertainty about the state at time t, given the measurements up to time t,

p(xt | y1:t)

The obvious question is now, how do we compute this object?

p(xt | y1:t) = p(xt | yt, y1:t�1) =p(yt | xt, y1:t�1)p(xt | y1:t�1)

p(yt | y1:t�1)

=h(yt | xt)p(xt | y1:t�1)

p(yt | y1:t�1)

Bayes’ theorem

Markov property



State inference - summarizing the development

the uncertain information that we have about the state is captured by the filtering pdf, which we compute sequentially using a measurement update

and a time update

p(xt | y1:t) =

z }| {h(yt | xt)

z }| {p(xt | y1:t�1)

p(yt | y1:t�1),

measurement model prediction pdf

p(xt | y1:t�1) =

Zf(xt | xt�1)| {z } p(xt�1 | y1:t�1)| {z } dxt�1,

filtering pdfdynamic model

xt+1 | xt ⇠ f(xt | xt�1),

yt | xt ⇠ h(yt | xt),

We have now showed that for the nonlinear SSM



State inference - simple special case (LGSS)

Consider the following special case (Linear Gaussian State Space (LGSS) model)

or, equivalently,

xt+1 = Axt +But + vt, vt ⇠ N (0, Q),

yt = Cxt +Dut + et, et ⇠ N (0, R).

Gaussian variables and linear transformation implies that the mean and the covariance captures everything there is to know.

xt+1 | xt ⇠ f(xt+1 | xt) = N (xt+1 | Axt +But, Q),

yt | xt ⇠ h(yt | xt) = N (yt | Cxt +Dut, R).



State inference - a sensor fusion example (I/III)

Aim: Estimate the position and orientation of a human (i.e. human motion) using measurements from inertial sensors and ultra-wideband (UWB).

Industrial partner: Xsens Technologies

Sensors Sensor fusion

Position and orientation

Accelerometer

Gyroscope

Magnetometer

World model

Learning (estimation)

Dynamic model

Sensor model

Transmitter

Receiver 1

Receiver 6

...

(17 IMU’s)

(UWB)



State inference - a sensor fusion example (II/III)

Sensor unit integrating an IMU and a UWB transmitter into a single housing.!

• Inertial measurements @ 200 Hz• UWB measurements @ 50 Hz• Mobile transmitter and 6 stationary, synchronized receivers at known positions.• Time-of-arrival (TOA) measurements

The sensors



Outline

1. Representation - probabilistic state space models (SSM’s)

2. State inference

a) General solutionb) LGSS models and the Kalman filterc) Sensor fusion exampled) Particle filter for general SSM’s via positioning examples

3. Parameter inference

a) Problem formulationb) Bayesian solution - particle MCMCc) System identification example - semiparametric Wiener model



State inference - particle filters


Position

Inertial

Terrain elevation

Radar


Dynamic model

Sensor model

Barometer World model

Aim: Find the position, velocity and orientation of a fighter aircraft.

Industrial partner: Saab


p(yt | y1:t�1)

p(xt+1 | y1:t) =Z

f(xt+1 | xt)p(xt | y1:t)dxt




Show movie

“Think of each particle as one simulation of the system state (in the movie we are visualizing the horizontal position). Only keep the good ones.”




The idea in the particle filter (member of the larger family of Sequential Monte Carlo (SMC) methods) is to use the following nonparametric representation of the filtering pdf

”� +

Z!

X”

The weights and the particles are then updated as new measurements becomes available.

This implies that the multidimensional integrals are replaced by finite sums, which is manageable,

p(xt

| y1:t) ⇡NX

i=1

w

i

t

�

x

it(x

t

),NX

i=1

w

i

t

= 1, w

i

t

� 0, 8i




Aim: Compute the position of a person moving around indoors using sensors located in an ID badge.

Industrial partner: Xdin


Position

Inertial

World model

Radio


Dynamic model

Sensor model

Magnetometer World model

Show movie



Parameter inference - system identification

System identification deals with the problem of estimating a dynamical model based on measurements of the input signal and the output signal,

This involves parameter inference (among other things). Two approaches:

u1:N = {u1, . . . , uN}, y1:N = {y1, . . . , yN}.

1. Maximum Likelihood (ML): Computes the point estimate of the parameters that makes the observed measurements as likely as possible,

2. Bayesian: All variables are now assumed to be stochastic, hence the parameters are no longer deterministic variables. Compute

b✓ML

= argmax

✓p✓(y1:N )

p(✓ | y1:N )



Monte Carlo and Markov chain Monte Carlo (MCMC)

Monte Carlo methods provides computational solutions, where the obtained accuracy is only limited by our computational resources.

An MCMC method simulates a Markov chain where the stationary distribution is given by the target distribution of interest.

These samples can then be used to compute various estimates.

There are constructive strategies for doing this and some of the most popular are the Gibbs sampler and the Metropolis Hastings sampler.



Particle MCMC (PMCMC)

The aim in particle Markov chain Monte Carlo (PMCMC) is to compute

when the model is given by

p(✓, x1:T | y1:T )

or some of its marginals distributions, e.g.,

p(✓ | y1:T )

p(x1:T | y1:T )

xt+1 | xt ⇠ f✓(xt+1 | xt, ut),

yt | xt ⇠ h✓(yt | xt, ut).

The fundamental idea is to make use of a sequential Monte Carlo (SMC) sampler to construct a proposal for an MCMC sampler.



Semi-parametric Wiener model

Rather than describing a general solution, let us be very specific and consider an example,

This is a Wiener model (= a dynamic LGSS model followed by a static nonlinearity),

xt+1 = Axt +But + wt, wt ⇠ N (0, Q),

zt = Cxt,

yt = h(zt) + et, et ⇠ N (0, r).

The Wiener game

Fredrik Lindsten and Thomas Schon

May 5, 2012

Lh

⌃ut zt yt

wt et

Figure 1: Wiener model.

xt+1 = Axt +But + wt, wt ⇠ N (0, Q), (1a)

zt = Cxt, (1b)

yt = h(zt) + et, et ⇠ N (0, r). (1c)

Here, xt 2 Rnx denotes the state of the dynamical system, ut 2 Rn

u is the inputsignal, zt 2 R is the output from the linear block and yt 2 R is the measurement.Furthermore, wt and et denotes the process noise and the measurement noise,respectively.

SISO or MISO?Generate a data set consisting of 1500 inputs and outputs from a model

within the class of models (1).



Semi-parametric Wiener model

u1:N = {u1, . . . , uN}, y1:N = {y1, . . . , yN}.

xt 2 Rn,

xt+1 = Axt +But + wt, wt ⇠ N (0, Q),

zt =�1 0 . . . 0

�xt,

yt = h(zt) + et, et ⇠ N (0, r)

Recall that the task is to find the dynamical model based on measurements of the input signal and the output signal,

The red parts of the model below are inferred from data.

A semiparametric Bayesian approach to

Wiener system identification

Fredrik Lindsten

⇤Thomas B. Schon

⇤Michael I. Jordan

⇤⇤

⇤Division of Automatic Control, Linkoping University, Linkoping,

Sweden (e-mail: {lindsten,schon}@isy.liu.se)

⇤⇤Departments of EECS and Statistics, University of California,

Berkeley, USA (e-mail: [email protected])

Abstract: We consider a semiparametric, i.e. a mixed parametric/nonparametric, model of aWiener system. We use a state-space model for the linear dynamical system and a nonparametricGaussian process (GP) model for the static nonlinearity. The GP model is a flexible modelthat can describe di↵erent types of nonlinearities while avoiding making strong assumptionssuch as monotonicity. We derive an inferential method based on recent advances in MonteCarlo statistical methods, known as Particle Markov Chain Monte Carlo (PMCMC). The ideaunderlying PMCMC is to use a particle filter (PF) to generate a sample state trajectory in aMarkov chain Monte Carlo sampler. We use a recently proposed PMCMC sampler, denotedparticle Gibbs with backward simulation, which has been shown to be e�cient even when weuse very few particles in the PF. The resulting method is used in a simulation study to identifytwo di↵erent Wiener systems with non-invertible nonlinearities.

1. INTRODUCTION

Block-oriented nonlinear systems are a family of nonlineardynamical systems which have attracted significant atten-tion in the system identification community, see e.g. [Giriand Bai, 2010]. These systems consist of interconnectedlinear dynamical systems and static nonlinearities. Themost well-known members of this family are the Hammer-stein (static nonlinearity followed by a linear dynamicalsystem) and the Wiener (linear dynamical system followedby a static nonlinearity) systems, introduced by Hammer-stein [1930] and Wiener [1966], respectively.

We are concerned here with the problem of “blind iden-tification” of a Wiener system; i.e., the case when theidentification is carried out in the absence of a known inputsignal. In other words, we wish to identify a model of aWiener system based on the information present in themeasurements y1:T , {y

t

}Tt=1; see Figure 1. This problem

has attracted significant interest, see e.g. [Vanbeylan et al.,2009, Abed-Meraim et al., 1997, Bai, 2002, Wills et al.,2011]. However, it should be noted that the proposedmethod can be generalised straightforwardly to the casein which the system is excited by a known input signal aswell.

G h(·) �

ztwt yt

et

Fig. 1. A blind Wiener system, consisting of a linearsystem G followed by a static nonlinearity h(·). Thesystem noise w

t

and the measurement noise et

areboth unmeasurable.

? This work was supported by: the project Calibrating NonlinearDynamical Models (Contract number: 621-2010-5876) funded by theSwedish Research Council and CADICS, a Linneaus Center alsofunded by the Swedish Research Council.

!2 !1.5 !1 !0.5 0 0.5 1 1.5 2

!1

!0.8

!0.6

!0.4

!0.2

0

0.2

0.4

0.6

0.8

1

z

h(z)

TruePosterior mean99 % credibility

Fig. 2. Nonlinear mapping (non-monotone), the estimatedposterior mean and 99 % credibility interval.

We consider a semiparametric (i.e., a mixed paramet-ric/nonparametric) model of a Wiener system. We use astate-space model for the linear dynamical system and anonparametric Gaussian process (GP) model for the staticnonlinearity. We take a Bayesian approach, modeling theunknown parameters of the model as random variables.The objective in this work is then to provide a method forcomputing p(✓ | y1:T ), the posterior probability densityfunction (PDF) of the unknown parameters ✓ given themeasurements y1:T . The posterior PDF does not allowfor a closed form solution and we will make use of aMarkov Chain Monte Carlo (MCMC) method (see e.g.[Robert and Casella, 2004] for a general introduction) tocompute an approximation of p(✓ | y1:T ). More specifi-cally, we employ the recently proposed particle MCMC(PMCMC) framework by Andrieu et al. [2010]. The basicidea underlying PMCMC is to use a particle filter (PF) togenerate a sample state trajectory, which is then used as acomponent of an MCMC sampler. Here, we use a PMCMCsampler denoted particle Gibbs with backward simulation

!10

0

10

20

Mag

nitude(dB)

0 0.5 1 1.5 2 2.5 3

!50

0

50

100

Frequency (rad/s)

Phase(deg)


Fig. 3. Bode diagram of the linear sys-tem, estimated posterior meanand 99 % credibility interval.

!2 !1.5 !1 !0.5 0 0.5 1 1.5 2

!1

!0.8

!0.6

!0.4

!0.2

0

0.2

0.4

0.6

0.8

1

z

h(z)


Fig. 4. Nonlinear mapping (satura-tion), estimated posterior meanand 99 % credibility interval.

!10

0

10

20

Mag

nitude(dB)

0 0.5 1 1.5 2 2.5 3

!50

0

50

100

Frequency (rad/s)

Phase(deg)



Q = 0.05I4 and R = 0.01. The same system is consideredby Wills et al. [2011] who present a method for maximumlikelihood estimation of blind Wiener systems. The non-linear mapping h(·) is taken as a saturation,

h(z) =

8<

:

1 if z � 0.5,2z if �0.5 z < 0.5,�1 if z < �0.5.

(17)

We generate T = 1000 samples from the system andapply the proposed method for 50000 MCMC iterations(out of which 10000 iterations are considered as burnin),using N = 5 particles in the PG-BSi sampler. Thehyperparameters are set as described in Appendix B.Figure 3 shows the Bode diagram of the linear system andFigure 4 shows the static nonlinearity, along with theirestimates. The gray areas illustrate the 99 % Bayesiancredibility regions, computed from the posterior PDFs.

The method appears to do a good job at identifying boththe linear dynamical system and the nonlinear mapping.Some slight lack of fit arises due to the non-smoothnessof h and the fact that the GP is a smoothness prior.Still, the shape of the nonlinearity is clearly visible fromthe estimated posterior PDF. The uncertainty about thenonlinearity gets larger close to the border of the axis(kzk & 1.5). The reason for this is that there are fewsamples in these regions available in the observed mea-surements.

5.2 4th-order system with non-monotone nonlinearity

To show the flexibility of the GP model, we consider thesame linear system (16), but replace the static nonlinear-ity. Instead of the saturation, we use a non-monotonic non-linear function shown in Figure 2. We generate T = 1000observations from the system and apply the proposedidentification method with the same settings as in Sec-tion 5.1. Note that, due to the nonparametric nature ofthe GP model, we do not need to make any modificationsto the code when we apply it to this modified system.Figure 5 shows the Bode diagram of the linear systemand Figure 2 shows the static nonlinearity, together withthe estimates using 50000 MCMC iterations (out of which10000 iterations are considered as burnin).

Also for this example, the method captures the shape ofthe nonlinearity as well as the linear dynamical system.The uncertainty about the Bode diagram is somewhat

larger than in Section 5.1, which is reflected in the es-timated posterior PDF. This is not surprising, since thenonlinearity illustrated in Figure 2 is quite di�cult todeal with. The reason is that the non-monotonicity of thefunction means that there is an ambiguity of the value ofzt

for a given observation yt

. Basically, for any observationyt

in the range [�1, 1] there are three possible values forzt

which descibe this observation equally well statically.

6. CONCLUSIONS AND FUTURE WORK

We have considered a semiparametric Bayesian model of aWiener system, using a state-space representation (wherethe dimension of the state-space is assumed to be known)of the linear dynamical system and a GP model for thestatic nonlinearity. The posterior parameter distribution isnot available in closed form. This was resolved by makinguse of a particle Markov Chain Monte Carlo method,relying on a particle filter and a backward simualtor toproduce sample state trajectories. The new algorithm wasprofiled on two examples with good results, despite the factthat only 5 particles were used in the underlying particlefilter.

A concern with the current method is that it does notscale well with the number of measurements T , since thecomputational complexity of evaluating the posterior GPis cubic in T . However, this is a well-studied problem inthe GP literature and existing approaches can be used tomitigate this issue.

In the numerical example provided in Section 5.1 weapplied the method to estimate a Wiener model, where thetrue nonlinearity was given by a saturation. This system isin fact not contained in the proposed model class, since theGP that we use is a smoothness prior. Due to this, someproblems arise close to the points of non-di↵erentiabilityof the saturation. Still, the method captures the shapeof this nonlinearity fairly well. An interesting topic forfuture work is to seek some theoretical justification for theapplication of the method, even when the true system liesoutside the treated model class. We may also consider touse the proposed method in a prestudy of the identificationproblem. Once we find the rough shape of the nonlinearity,we can find some suitable parameterisation of it and switchto a fully parametric model.

In this work, we have not considered estimation of theGP hyperparameters from data. However, this can bedone by adding a step to the Gibbs sampler, in which



Semi-parametric Wiener model - representing uncertainty



Fredrik Lindsten

⇤Thomas B. Schon


⇤⇤






1. INTRODUCTION



t



G h(·) �

ztwt yt

et


t




!2 !1.5 !1 !0.5 0 0.5 1 1.5 2

!1

!0.8

!0.6

!0.4

!0.2

0

0.2

0.4

0.6

0.8

1

z

h(z)




!10

0

10

20

Mag

nitude(dB)

0 0.5 1 1.5 2 2.5 3

!50

0

50

100

Frequency (rad/s)

Phase(deg)



!2 !1.5 !1 !0.5 0 0.5 1 1.5 2

!1

!0.8

!0.6

!0.4

!0.2

0

0.2

0.4

0.6

0.8

1

z

h(z)



!10

0

10

20

Mag

nitude(dB)

0 0.5 1 1.5 2 2.5 3

!50

0

50

100

Frequency (rad/s)

Phase(deg)




h(z) =

8<

:

1 if z � 0.5,2z if �0.5 z < 0.5,�1 if z < �0.5.

(17)
















Linear system Static nonlinearity

We used a nonparametric model for the static nonlinearity, more specifically a Gaussian process.

Show convergence using a movie

How do we represent uncertainty for a nonlinear model?!



Conclusions

Throughout the talk I have touched upon a lot of methods that clearly deserves much more time than I gave them in this tutorial presentation.

Tomorrow and on Thursday I am giving an intensive course on this in Brussels, for details, see

http://www.rt.isy.liu.se/~schon/CourseBrussels2012/index.html

• Take home message: Given the computational tools that we have today it can be rewarding to resist the linear Gaussian convenience!

• There are by now a lot of tools that allows us to do this (e.g., SMC, PMCMC).

• There is a lot of interesting research that remains to be done!

• The industrial utility of the sensor fusion technology is growing as we speak!



Sensor unit integrating an IMU and a UWB transmitter into a single housing.!



Fredrik Lindsten

⇤Thomas B. Schon


⇤⇤






1. INTRODUCTION



t



G h(·) �

ztwt yt

et


t




!2 !1.5 !1 !0.5 0 0.5 1 1.5 2

!1

!0.8

!0.6

!0.4

!0.2

0

0.2

0.4

0.6

0.8

1

z

h(z)




!10

0

10

20

Mag

nitude(dB)

0 0.5 1 1.5 2 2.5 3

!50

0

50

100

Frequency (rad/s)

Phase(deg)



!2 !1.5 !1 !0.5 0 0.5 1 1.5 2

!1

!0.8

!0.6

!0.4

!0.2

0

0.2

0.4

0.6

0.8

1

z

h(z)



!10

0

10

20

Mag

nitude(dB)

0 0.5 1 1.5 2 2.5 3

!50

0

50

100

Frequency (rad/s)

Phase(deg)




h(z) =

8<

:

1 if z � 0.5,2z if �0.5 z < 0.5,�1 if z < �0.5.

(17)
















xt+1 | xt ⇠ f✓(xt+1 | xt, ut),


x1 ⇠ µ✓(x1).The Wiener game

Fredrik Lindsten and Thomas Schon

May 5, 2012

L h ⌃ut zt yt

wt et

Figure 1: Wiener model.

xt+1 = Axt +But + wt, wt ⇠ N (0, Q), (1a)

zt = Cxt, (1b)

yt = h(zt) + et, et ⇠ N (0, r). (1c)

Here, xt 2 Rnx denotes the state of the dynamical system, ut 2 Rn

u is the inputsignal, zt 2 R is the output from the linear block and yt 2 R is the measurement.Furthermore, wt and et denotes the process noise and the measurement noise,respectively.

SISO or MISO?Generate a data set consisting of 1500 inputs and outputs from a model

within the class of models (1).


p(yt | y1:t�1),

p(xt+1 | y1:t) =Z

f(xt+1 | xt)p(xt | y1:t)dxt

Special thanks to: Jeroen Hol (Xsens Technologies), Lennart Ljung (LiU), Johan Kihlberg (Xdin), Fredrik Lindsten (LiU), Henk Luinge (Xsens Technologies), Michael Jordan (Berkeley), Brett Ninness (University of Newcastle), Per-Johan Nordlund (Saab), Simon Tegelid (Xdin) and Adrian Wills (University of Newcastle).

p(✓, x1:T | y1:T )

Thank you for your attention!!

representing and working with uncertainty in dynamical systemspeople.isy.liu.se › en › rt ›...

Documents