The Empirical Cross Gramian for ParametrizedNonlinear Systems

Christian Himpe (christian.himpe@uni-muenster.de)Mario Ohlberger (mario.ohlberger@uni-muenster.de)

WWU MünsterInstitute for Computational and Applied Mathematics

MathMod 201519.02.2015

Networks

The brain:A fairly complex network.

Excited by external sensory stimuli.Information propagation due toconnectivity between regions.Usually, no direct measurement ofneuronal activity.

Modelling

Adjacency Matrix:

A =

a11 a12 . . . a1Na21 a22 . . ....

.... . .

aN1 aNN

(aij = Connection from j-th node to i-th node.)

Controlled External InputSmall Number of (Output) SensorsParametrized Connectivity (Components of A)

→ (Parametrized) Control System

Model

(Dynamical) Input-Output System

Linear (Time-Invariant) System:x(t) = Ax(t) + Bu(t)

y(t) = Cx(t)

(x(t) ∈ RN , u(t) ∈ RM , y(t) ∈ RO ,A ∈ RN×N ,B ∈ RN×M ,C ∈ RO×N )

General (Possibly Nonlinear) System:x(t) = f (x(t), u(t))

y(t) = g(x(t), u(t))

(f : R×RN ×RM → RN , g : R×RN ×RM → R

O )

Model Order Reduction

Input-Output Mapping:u ∈ LM

2 [0,∞)→ RN → y ∈ LO

2 [0,∞)

(N � 1, dim(u(t))� N, dim(y(t))� N)

Linear Reduced Order Model:xr (t) = Arxr (t) + Bru(t)

yr (t) = Crxr (t)

(dim(xr (t))� dim(x(t)) and ‖y − yr‖ � 1 in a suitable norm.)

General Reduced Order Model:xr (t) = fr (xr (t), u(t))

yr (t) = gr (xr (t), u(t))

(dim(xr (t))� dim(x(t)) and ‖y − yr‖ � 1 in a suitable norm.)

Projection-Based MOR

Petrov-Galerkin Projection:U ∈ RN×n,V ∈ Rn×N ,VU = 1

(V = UT Galerkin projection)

Linear ROM:xr (t) = VAUxr (t) + VBu(t)

yr (t) = CUxr (t)

(Generally fast.)

General ROM:xr (t) = Vf (Uxr (t), u(t))

yr (t) = g(Uxr (t), u(t))

(Generally slow.)

Hankel Singular Values i.e. [Antoulas’05]

Impulse Response:h(t) := CeAtB

(x0 = 0 and u(t) = δ(t))

Convolution Operator:

S(u)(t) = (h ∗ u)(t) =

∫ ∞0

CeA(t−τ)Bu(τ)dτ = y(t)

(S has infinite spectrum)

Hankel Operator:

H(u)(t) := (S ◦ F )(u)(t)

∫ 0

−∞CeA(t+τ)Bu(τ)dτ

(Time-flip operator F : [0,∞)→ (−∞, 0]; H has finite spectrum)

Cross Gramian [Fernando & Nicholson’83]

Factorization of Hankel Operator:H = OC

(With controllability and observability operator factors)

Cross Gramian:

WX := CO =

∫ ∞0

eAtBCeAtdt

(For square systems: #inputs = #outputs)

Core Property of the Cross Gramian:OC = (OC)∗ ⇒ |λi (WX )| = σi (H)

(For symmetric systems)

Direct Truncation / Approximate Balancing

Singular Value Decomposition of Cross Gramian:

WXSVD= UDV

(Truncated SVD in production code.)

Truncation of singular vectors yields Galerkin projectionU =

(U1 U2

)V1 = UT

1

(Two-sided Petrov-Galerkin is not guaranteed to be stable.)

Cross-Gramian-Based ROM:xr (t) = UT

1 AU1xr (t) + UT1 Bu(t)

yr (t) = CU1xr (t)

(Initial condition is projected too: x0,r = UT1 x0. Equivalent to BT for state-space symmetric systems.)

Empirical Gramians [Lall et al’99]

Snapshots:X =

(x0 . . . xT

), Y =

(y0 . . . yT

)(Discrete state and output trajectories.)

Perturbation Sets:QU = {ei=1...M} × {Rj ∈ RM×M |RjRT

j = 1} × {qk ∈ R}QX = {ei=1...N} × {Rl ∈ RN×N |RlRT

l = 1} × {qm ∈ R}(Perturbing (impulse) input and initial state.)

Empirical Controllability and Observability Gramian:

WC =

|QU |∑q=1

XqXTq , WO =

|QX \{ei}|∑q=1

(Y...q,1, . . . ,Y

...q,N)T (Y

...q,1, . . . ,Y

...1,N)

(WC = WC and WO = WO for linear systems.)

Empirical Cross Gramians [Streif et al’06, H. & Ohlberger’14]

Cross Gramian:

WX =

∫ ∞0

(eAtB)︸ ︷︷ ︸u→x

(CeAt)︸ ︷︷ ︸x→y

dt

(The (inner) product of controllability and observability operator.)

Empirical Cross Gramian:

WX :=1

|QU ||QX |

|QU |∑h=1

|QX |∑i=1

∫ ∞0

Ψhij(t)dt ∈ RN×N ,

Ψhijk,i (t) = (xhj

k (t)− xk)(y ijh (t)− yh)T ∈ R,

(WX = WX for linear systems.)

parametric Model Order Reduction

Parametrized General System:x(t) = f (x(t), u(t), θ)

y(t) = g(x(t), u(t), θ)

(θ ∈ RP = Θ)

Reduced Parametrized General System:xr (t) = fr (xr (t), u(t), θ)

yr (t) = gr (xr (t), u(t), θ)

(dim(xr (t))� dim(x(t)) and ‖y − yr‖ � 1 for all valid θ in a suitable norm.)

Linear Parametrization [Sun & Hahn’06]

Linear Parametric System:

x = Ax + Bu + Fθ → x = Ax +(B F

)( uuθ

)y = Cx

(F ∈ RN×P ; treat parameters as additional inputs.)

Parametric Controllability Gramian:

WC = WC ,0 +P∑

i=1

WC ,i ,

WC ,0 =

∫ ∞0

eAtBBT eAT tdt,

WC ,i>0 =

∫ ∞0

eAtFiFTi eAT tdt

Parametrized Empirical Cross Gramian

Discretized Parameter Space:QΘ =

(θ1 . . . θJ

)(Around a nominal parameter.)

Extended Perturbation Set:QU × QX × QΘ

(As opposed to the linear case where (QU ∪ QΘ)× QX .)

Parametrized Empirical Cross Gramian:

WX :=1

|QU ||QX ||QΘ|

|QU |∑h=1

|QX |∑i=1

|QΘ|∑j=1

∫ ∞0

Ψhij(t)dt ∈ RN×N ,

Ψhijk,i (t) = (xhj

k (t)− xk)(y ijh (t)− yh) ∈ R,

(Parameters are treated like inputs or states and are perturbed.)

Nonlinear Symmetry [Ionescu et al’09]

Linear Symmetry:OC = (OC)∗ ⇔ CeAB = (CeAB)∗

⇔ CA−1B = (CA−1B)∗

⇔ CAkB = (CAkB)∗

(Symmetry property for impulse response, system gain (transfer function) and Markov parameter.)

Nonlinear Symmetry:ImO = Im C+ ⇔ ∃WX : O = C+(WX ) = C+CO

⇔ ∃W +X : C+ = O(W +

X ) = OO+C+

(All SISO systems are symmetric!)

(Nonlinear) RC Ladder Benchmark [Chen’99]

Benchmark Problem [MORwiki http://modelreduction.org]Nonlinear SISO System (!)Variable State DimensionAlso used, for example, in [Condon & Ivanov’04]

(Source: C. Himpe “Nonlinear RC Ladder”, MORwiki, 2014)

(Resistor-capacitor cascade with nonlinear resistors (diodes).)

(Nonlinear) RC Ladder Benchmark II

x(t) =

−g(x1(t))− g(x1(t)− x2(t)) + u(t)g(x1(t)− x2(t))− g(x2(t)− x3(t))

...g(xk−1(t)− xk(t))− g(xk(t)− xk+1(t))

...g(xN−1(t)− xN(t))

,

y(t) = x1(t),

with:g(xi ) = exp(10xi ) + xi − 1

⇒ gθ(xi ) = exp(10xi ) + θixi − 1(dim(x(t)) = 1000⇒ θ ∈ R1000, For experiments: θ ∈ U 1

2 , 32

)

emgr - Empirical Gramian Framework (Version: 2.9, 01.2015)

Gramians:Empirical Controllability GramianEmpirical Observability GramianEmpirical Cross GramianEmpirical Linear Cross GramianEmpirical Sensitivity GramianEmpirical Identifiability GramianEmpirical Joint Gramian

Features:Uniform InterfaceCompatible with MATLAB & OCTAVE (& FREEMAT)Vectorized & ParallelizableOpen-Source licensed

More info at: http://gramian.de

State Reduction

10 20 30 40 50 60 70 80 90 100

10−15

10−10

10−5

100

Reduced States

Re

lative

Ou

tpu

t E

rro

r

Balanced Truncation (L2)

Balanced Truncation (L∞)

Cross Gramian (L2)

Cross Gramian (L∞)

0 50 100 150

WX

BT

Offline Time [s]

Parametrized State Reduction (Offline:3, Online:100 Samples)

10 20 30 40 50 60 70 80 90 100

10−15

10−10

10−5

100

Reduced States

Re

lative

Ou

tpu

t E

rro

r

Balanced Truncation (L2)

Balanced Truncation (L∞)

Cross Gramian (L2)

Cross Gramian (L∞)

0 50 100 150 200 250 300 350 400 450 500

WX

BT

Offline Time [s]

Outlook

Application to neuronal network models,in the context of neuroimaging data inversion (ongoing)

Non-Symmetric Cross Gramian (Preprint: http://arxiv.org/pdf/1501.05519)

Really Large SVDs(Distributed Memory) Parallelization

tl;dl

Cross-gramian-based model reduction,for parametrized nonlinear systems,using the empirical cross gramian,averaged over a discretized parameter space,around a steady state.

http://gramian.de

Thanks!

Get the Companion Code: http://j.mp/mathmod15