![Page 1: Model Order Reduction (for ODE-Constrained Optimization)](https://reader033.vdocuments.mx/reader033/viewer/2022042817/55a6892c1a28ab391e8b47c3/html5/thumbnails/1.jpg)
Model Order Reduction(for ODE-Constrained Optimization)
Christian Himpe ([email protected])Mario Ohlberger ([email protected])
WWU MünsterInstitute for Computational and Applied Mathematics
DCM Workshop24.04.15
![Page 2: Model Order Reduction (for ODE-Constrained Optimization)](https://reader033.vdocuments.mx/reader033/viewer/2022042817/55a6892c1a28ab391e8b47c3/html5/thumbnails/2.jpg)
Disclaimer
The presented methods are subject of ongoing research.I am a mathematician; there will be math!
![Page 3: Model Order Reduction (for ODE-Constrained Optimization)](https://reader033.vdocuments.mx/reader033/viewer/2022042817/55a6892c1a28ab391e8b47c3/html5/thumbnails/3.jpg)
Outline
1 Network Models2 Connectivity Inference3 Dynamic Causal Modelling4 Model Reduction5 Bringing It All Together
![Page 4: Model Order Reduction (for ODE-Constrained Optimization)](https://reader033.vdocuments.mx/reader033/viewer/2022042817/55a6892c1a28ab391e8b47c3/html5/thumbnails/4.jpg)
Motivation
How are regions of the brain connected?How does (sensory) input disperse?How does connectivity change under input?How does the brain learn and unlearn?
![Page 5: Model Order Reduction (for ODE-Constrained Optimization)](https://reader033.vdocuments.mx/reader033/viewer/2022042817/55a6892c1a28ab391e8b47c3/html5/thumbnails/5.jpg)
Procedure
1 Experimental Data2 Forward Model3 Inverse Problem
Forward Problem
Inverse Problem
![Page 6: Model Order Reduction (for ODE-Constrained Optimization)](https://reader033.vdocuments.mx/reader033/viewer/2022042817/55a6892c1a28ab391e8b47c3/html5/thumbnails/6.jpg)
Notation
x(t) State Trajectoryaka Neuronal Activity
y(t) Output Trajectoryaka Measured Response
u(t) Input / Controlaka External Stimulus
θ Parametersaka Connectivity Strength
![Page 7: Model Order Reduction (for ODE-Constrained Optimization)](https://reader033.vdocuments.mx/reader033/viewer/2022042817/55a6892c1a28ab391e8b47c3/html5/thumbnails/7.jpg)
Act I
Network Models
![Page 8: Model Order Reduction (for ODE-Constrained Optimization)](https://reader033.vdocuments.mx/reader033/viewer/2022042817/55a6892c1a28ab391e8b47c3/html5/thumbnails/8.jpg)
Ordinary Differential Equations
Initial Value Problem (IVP) w Ordinary Differential Equation1 (ODE):
x(t) = ax(t)
x(0) = x0
Components:x : R+ → R Solution Trajectoryx = dx
dt Newton Notation for a Time Derivativex0 ∈ R Initial Valuea ∈ R
Solution:
x(t) = eatx0
1Here: Autonomous & Linear
![Page 9: Model Order Reduction (for ODE-Constrained Optimization)](https://reader033.vdocuments.mx/reader033/viewer/2022042817/55a6892c1a28ab391e8b47c3/html5/thumbnails/9.jpg)
Systems of ODEs
IVP with a System of ODEs:
x(t) = Ax(t)
x(0) = x0
Components:x : R+ → RN Solution Trajectoryxi = dxi
dt Component-Wise Derivative
x0 ∈ RN Initial ValueA ∈ RN×N
Solution:
x(t) = eAtx0
![Page 10: Model Order Reduction (for ODE-Constrained Optimization)](https://reader033.vdocuments.mx/reader033/viewer/2022042817/55a6892c1a28ab391e8b47c3/html5/thumbnails/10.jpg)
Control System
Linear Control System2:
x(t) = Ax(t) + Bu(t)
y(t) = Cx(t)
x(0) = x0
Components:y : RN → RO Output TrajectoryA ∈ RN×N System MatrixB ∈ RN×M Input MatrixC ∈ RO×N Output Matrix
Solution:
y(t) = (x ∗ u)(t) = CeAtx0 +
∫ t
0CeAτBu(τ)dτ
2in State-Space Form
![Page 11: Model Order Reduction (for ODE-Constrained Optimization)](https://reader033.vdocuments.mx/reader033/viewer/2022042817/55a6892c1a28ab391e8b47c3/html5/thumbnails/11.jpg)
Network Interpretation
Linear Dynamical System:
x(t) = Ax(t) + Bu(t)
Example (single input, three region network):
x1(t)x2(t)x3(t)
=
a11 a12 a13a21 a22 a23a31 a32 a33
x1(t)x2(t)x3(t)
+
b1b2b3
u(t)
![Page 12: Model Order Reduction (for ODE-Constrained Optimization)](https://reader033.vdocuments.mx/reader033/viewer/2022042817/55a6892c1a28ab391e8b47c3/html5/thumbnails/12.jpg)
Connectivity Parametrization
Connectivity Strength:
θiN+j = aij
Parametrized Linear Dynamical System:
x(t) = A(θ)x(t) + Bu(t)
with Nonlinear Parameter Mapping.
![Page 13: Model Order Reduction (for ODE-Constrained Optimization)](https://reader033.vdocuments.mx/reader033/viewer/2022042817/55a6892c1a28ab391e8b47c3/html5/thumbnails/13.jpg)
General Control System
Possibly Nonlinear Control System:
x(t) = f (x(t), u(t), θ)
y(t) = g(x(t), u(t), θ)
x(0) = x0
(Special Case) Linear Control System:
f (x(t), u(t), θ) = A(θ)x(t) + Bu(t)
g(x(t), u(t), θ) = Cx(t)
(Nonlinear Example) Hyperbolic Network Model:
x(t) = A(θ) tanh(Kx(t)) + Bu(t)
y(t) = Cx(t)
x(0) = x0
![Page 14: Model Order Reduction (for ODE-Constrained Optimization)](https://reader033.vdocuments.mx/reader033/viewer/2022042817/55a6892c1a28ab391e8b47c3/html5/thumbnails/14.jpg)
Act II
Connectivity Inference
![Page 15: Model Order Reduction (for ODE-Constrained Optimization)](https://reader033.vdocuments.mx/reader033/viewer/2022042817/55a6892c1a28ab391e8b47c3/html5/thumbnails/15.jpg)
Data Model
Model-Data Relation:
yd := yθ + ε
Components:yd Measured Outputyθ Parametrized Model Outputε Noise
![Page 16: Model Order Reduction (for ODE-Constrained Optimization)](https://reader033.vdocuments.mx/reader033/viewer/2022042817/55a6892c1a28ab391e8b47c3/html5/thumbnails/16.jpg)
Inverse Problem
ODE Constrained Optimization:
θd = argminθ ‖yθ − yd‖22s.t.:x(t) = f (x(t), u(t), θ)
yθ(t) = g(x(t), u(t), θ)
x(0) = x0
‖ · ‖2 means L2 (Euclidian Distance).This is a least-squares minimization.
![Page 17: Model Order Reduction (for ODE-Constrained Optimization)](https://reader033.vdocuments.mx/reader033/viewer/2022042817/55a6892c1a28ab391e8b47c3/html5/thumbnails/17.jpg)
Difficulties
Well-Posed Problem:A solution existsThe solution is uniqueSolution is stable
A problem not well-posed is ill-posed.
Ill-Conditioned Problem:Small Perturbation result in large errors
![Page 18: Model Order Reduction (for ODE-Constrained Optimization)](https://reader033.vdocuments.mx/reader033/viewer/2022042817/55a6892c1a28ab391e8b47c3/html5/thumbnails/18.jpg)
Regularization
Tikhonov Regularization:
θd = argminθ ‖yθ − yd‖22 + β‖θ‖22
New Components:β Regularization Coefficient‖θ‖22 Regularization Operator
![Page 19: Model Order Reduction (for ODE-Constrained Optimization)](https://reader033.vdocuments.mx/reader033/viewer/2022042817/55a6892c1a28ab391e8b47c3/html5/thumbnails/19.jpg)
Act III
Dynamic Causal Modelling
![Page 20: Model Order Reduction (for ODE-Constrained Optimization)](https://reader033.vdocuments.mx/reader033/viewer/2022042817/55a6892c1a28ab391e8b47c3/html5/thumbnails/20.jpg)
Two-Part Model [Friston et al’03]
Dynamic Sub-Model:Models neuronal activity,and coupling between different brain regionsMultiple-Input-Multiple-Output
Forward Sub-Model:Transforms neuronal activity to measurable outputPhysiologically motivatedSingle-Input-Single-Output
![Page 21: Model Order Reduction (for ODE-Constrained Optimization)](https://reader033.vdocuments.mx/reader033/viewer/2022042817/55a6892c1a28ab391e8b47c3/html5/thumbnails/21.jpg)
Schematic Two-Part Model
![Page 22: Model Order Reduction (for ODE-Constrained Optimization)](https://reader033.vdocuments.mx/reader033/viewer/2022042817/55a6892c1a28ab391e8b47c3/html5/thumbnails/22.jpg)
Model Properties
Models differ forEEG / MEGfMRI / fNIRS
but have commonalities:
Both ...exhibit stable behaviour,(originally) contain nonlinearities,encode connectivity in parameters,are assumed to be deterministic.
![Page 23: Model Order Reduction (for ODE-Constrained Optimization)](https://reader033.vdocuments.mx/reader033/viewer/2022042817/55a6892c1a28ab391e8b47c3/html5/thumbnails/23.jpg)
A Closer Look at ...
... the fMRI Model,
because:the dynamic sub-model is easier to understand,and has less physiological assumptions.
![Page 24: Model Order Reduction (for ODE-Constrained Optimization)](https://reader033.vdocuments.mx/reader033/viewer/2022042817/55a6892c1a28ab391e8b47c3/html5/thumbnails/24.jpg)
Dynamic Sub-Model
Models neuronal activityHidden from (direct) measurementMost likely nonlinear
Encodes couplingInput (External Stimulus)(Connectivity) Parameters
x(t) = f (x(t), u(t), θ)
![Page 25: Model Order Reduction (for ODE-Constrained Optimization)](https://reader033.vdocuments.mx/reader033/viewer/2022042817/55a6892c1a28ab391e8b47c3/html5/thumbnails/25.jpg)
Linear(ized) Model
Using a Taylor series approximation:
x(t) = f (x(t), u(t), θ)
≈ f (0, 0, θ) +dfdx
x(t) +dfdu
u(t)
⇒ ˙x(t) = A(θ)x(t) + B(θ)u(t)
Models effective connectivityParameters θ are the components of A,BStability constraints apply to θ
![Page 26: Model Order Reduction (for ODE-Constrained Optimization)](https://reader033.vdocuments.mx/reader033/viewer/2022042817/55a6892c1a28ab391e8b47c3/html5/thumbnails/26.jpg)
Bilinear Extension
Bilinear approximation [Friston et al’03]:
x(t) = f (x(t), u(t), θ)
≈ f (0, 0, θ) +dfdx
x(t) +dfdu
u(t) +d2f
dxdux(t)u(t)
⇒ ˙x(t) = A(θ)x(t) + B(θ)u(t) +∑i
ui (t)Gi (θ)x(t)
Gi (θ) describes influence of i-th external input on coupling strength(lateral connectivity).
![Page 27: Model Order Reduction (for ODE-Constrained Optimization)](https://reader033.vdocuments.mx/reader033/viewer/2022042817/55a6892c1a28ab391e8b47c3/html5/thumbnails/27.jpg)
Quadratic Extension
Quadratic approximation [Stephan et al’08]:
x(t) = f (x(t), u(t), θ)
≈ f (0, 0, θ) +dfdx
x(t) +dfdu
u(t) +d2f
dxdux(t)u(t) +
d2f
dxdxx(t)x(t)
⇒ ˙x(t) = A(θ)x(t) + Bu(t) +∑i
ui (t)Gi (θ)x(t) +∑j
xj(t)Hj(θ)x(t)
Hj(θ) describes influence of j-th state on coupling strength.
![Page 28: Model Order Reduction (for ODE-Constrained Optimization)](https://reader033.vdocuments.mx/reader033/viewer/2022042817/55a6892c1a28ab391e8b47c3/html5/thumbnails/28.jpg)
Forward-Submodel [Friston’02]
Transforms neuronal activity to an observable BOLD signalIs a nonlinear SISO system
si (t) = κxxi (t)−κssi (t)−κf (1− fi (t))
fi (t) = si (t)
vi (t) = 1κ0
(fi (t)− vi (t)1α )
qi (t) = 1κ0
(fi (t)E(fi (t),κρ)
κρ−vi (t)
1α
qi (t)vi (t) )
yi (t) = k1(1− vi (t)) + k2(1− qi (t)) +
k3(1− qi (t)vi (t) ))
Activity induced signalInflowVenous volume(inflow - outlow)Deoxy. Content(intake - release)BOLD Output(volume + content +concentration)
![Page 29: Model Order Reduction (for ODE-Constrained Optimization)](https://reader033.vdocuments.mx/reader033/viewer/2022042817/55a6892c1a28ab391e8b47c3/html5/thumbnails/29.jpg)
Joint Model
Combining theMIMO dynamic sub-modelSISO forward sub-model
yields a joint nonlinear state-space system:
x(t)z1(t)...
zn(t)
=
Fdyn(x(t), u(t), θ)Fout,1(z1(t), x1(t))
...Fout,n(zn(t), xn(t))
y = g(z1(t), . . . , zn(t))
![Page 30: Model Order Reduction (for ODE-Constrained Optimization)](https://reader033.vdocuments.mx/reader033/viewer/2022042817/55a6892c1a28ab391e8b47c3/html5/thumbnails/30.jpg)
Bayesian Inference
Bayes’ Rule:
P(θ|y) =P(y |θ)P(θ)
P(y)
P(θ|yd) Posterior (Probability of θ given yd)P(yd |θ) Likelihood (Probability of yd given θ)P(θ) Prior (Probability of θ)P(yd) Evidence (Probability of yd)
Proportionality:
P(θ|y) ∝ P(y |θ)P(θ)
![Page 31: Model Order Reduction (for ODE-Constrained Optimization)](https://reader033.vdocuments.mx/reader033/viewer/2022042817/55a6892c1a28ab391e8b47c3/html5/thumbnails/31.jpg)
Gaussian Setting
In case the prior and the noise are gaussian,P(θ) = N (κ,K ), P(ε) = N (0,Λ):
yd = yθ + ε
⇒ ε = yd − yθ
⇒ P(ε) = P(yd − yθ)
⇒ P(θ|yd) ∝ P(yd − yθ)P(θ)
P(yd |θ) ∝ exp(−12‖yθ − yd‖2Λ−1)
P(θ) ∝ exp(−12‖θ − κ‖2K−1)
⇒ P(θ|yd) ∝ exp(−12‖yθ − yd‖2Λ−1 −
12‖θ − κ‖2K−1),
![Page 32: Model Order Reduction (for ODE-Constrained Optimization)](https://reader033.vdocuments.mx/reader033/viewer/2022042817/55a6892c1a28ab391e8b47c3/html5/thumbnails/32.jpg)
Maximum-A-Posteriori
MAP Estimator (Prior incorporated maximum likelihood estimator):
θMAP = argmaxθ∈RP exp(−12‖yθ − yd‖2Λ−1 −
12‖θ − κ‖2K−1
)= argminθ∈RP
(12‖yθ − yd‖2Λ−1 +
12‖θ − κ‖2K−1
).
At second glance this is a regularized least-square problem!
![Page 33: Model Order Reduction (for ODE-Constrained Optimization)](https://reader033.vdocuments.mx/reader033/viewer/2022042817/55a6892c1a28ab391e8b47c3/html5/thumbnails/33.jpg)
Large-Scale Models
The dynamic sub-model (dynamical system):
x(t) = A(θ)x(t) + Bu(t)
dimensions determine the paramter space dimension: P = N2.
The minimization algorithm for such a nonlinear problem computesmany simulations of the system, due to the necessary perturbationsof many directions in the parameter space.
⇒ A few more nodes in the network may prolong the inversionprocedure significantly.
![Page 34: Model Order Reduction (for ODE-Constrained Optimization)](https://reader033.vdocuments.mx/reader033/viewer/2022042817/55a6892c1a28ab391e8b47c3/html5/thumbnails/34.jpg)
Act IV
Model Reduction
![Page 35: Model Order Reduction (for ODE-Constrained Optimization)](https://reader033.vdocuments.mx/reader033/viewer/2022042817/55a6892c1a28ab391e8b47c3/html5/thumbnails/35.jpg)
Model Reduction
System:
x(t) = f (x(t), u(t), θ)
y(t) = g(x(t), u(t), θ)
x(0) = x0
Setting:dim(x(t))� 1 (Many Network Nodes / Brain Regions)
dim(u(t))� dim(x(t)) (Significantly Less Inputs)
dim(y(t))� dim(x(t)) (Significantly Less Outputs)
dim(θ)� 1 (Many Parameters)
Aim:dim(xr (t))� dim(x(t))
dim(θr )� dim(θ)
‖yθ − yr ,θr ‖ � 1
![Page 36: Model Order Reduction (for ODE-Constrained Optimization)](https://reader033.vdocuments.mx/reader033/viewer/2022042817/55a6892c1a28ab391e8b47c3/html5/thumbnails/36.jpg)
Projection-Based Model Reduction
Trajectory Projection:
xr (t) := Ux(t)
x(t) ≈ Vxr (t)
Petrov-Galerkin Projection:
U ∈ Rn×N , V ∈ RN×n, V TU = 1, n� N
Galerkin Projection:
U ∈ Rn×N , V := UT , UTU = 1, n� N
We will only be concerned with Galerkin projection.Petrov-Galerkin can pose issues with stability.
![Page 37: Model Order Reduction (for ODE-Constrained Optimization)](https://reader033.vdocuments.mx/reader033/viewer/2022042817/55a6892c1a28ab391e8b47c3/html5/thumbnails/37.jpg)
State-Space Reduction
General Control System:
x(t) = f (x(t), u(t), θ)
y(t) = g(x(t), u(t), θ)
x(0) = x0
Projection-Based Reduced Order Model (ROM):
xr (t) = Vf (Uxr (t), u(t), θ)
yr (t) = g(Uxr (t), u(t), θ)
xr (0) = Vx0
Aim: ‖y − yr‖ � 1
![Page 38: Model Order Reduction (for ODE-Constrained Optimization)](https://reader033.vdocuments.mx/reader033/viewer/2022042817/55a6892c1a28ab391e8b47c3/html5/thumbnails/38.jpg)
Linear State-Space Reduction
Linear Control System:
x(t) = Ax(t) + Bu(t)
y(t) = Cx(t)
x(0) = x0
Projection-Based ROM:
xr (t) = UAVxr (t) + UBu(t)
yr (t) = CVxr (t)
xr (0) = Vx0
![Page 39: Model Order Reduction (for ODE-Constrained Optimization)](https://reader033.vdocuments.mx/reader033/viewer/2022042817/55a6892c1a28ab391e8b47c3/html5/thumbnails/39.jpg)
(Side Note) PCA / POD / SVD
You may already have done model reduction!
Assume:given a discrete time series,to which a PCA is applied.
This is more or less a centered POD method of snapshots.
For finite dimensional operators PCA and POD are essentially a(sparse) SVD.
![Page 40: Model Order Reduction (for ODE-Constrained Optimization)](https://reader033.vdocuments.mx/reader033/viewer/2022042817/55a6892c1a28ab391e8b47c3/html5/thumbnails/40.jpg)
Parameter Identification
Parameter Space:
θ ∈ RP , P � 1
i.e. P = N2.
Which (linear combination of the) parameter is influencing thebehaviour of the system the most?This is also related to sensitivity analysis.
Parameter (Galerkin) Projection:
θr := Πθ
θ ≈ ΠT θr
ΠTΠ = 1
![Page 41: Model Order Reduction (for ODE-Constrained Optimization)](https://reader033.vdocuments.mx/reader033/viewer/2022042817/55a6892c1a28ab391e8b47c3/html5/thumbnails/41.jpg)
Parameter-Space Reduction
General Control System:
x(t) = f (x(t), u(t), θ)
y(t) = g(x(t), u(t), θ)
x(0) = x0
Projection-Based ROM:
x(t) = f (x(t), u(t),ΠT θr )
y(t) = g(x(t), u(t),ΠT θr )
Aim: ‖yθ − yθr ‖ � 1
![Page 42: Model Order Reduction (for ODE-Constrained Optimization)](https://reader033.vdocuments.mx/reader033/viewer/2022042817/55a6892c1a28ab391e8b47c3/html5/thumbnails/42.jpg)
Combined State and Parameter Reduction
General Control System:
x(t) = f (x(t), u(t), θ)
y(t) = g(x(t), u(t), θ)
Reduced Order Model:
xr (t) = fr (xr (t), u(t), θr )
yr (t) = gr (xr (t), u(t), θr )
Projection-Based ROM:
xr (t) = UT f (Uxr (t), u(t),ΠT θr )
yr (t) = g(Uxr (t), u(t),ΠT θr )
Aim: ‖yθ − yr ,θr ‖ � 1
![Page 43: Model Order Reduction (for ODE-Constrained Optimization)](https://reader033.vdocuments.mx/reader033/viewer/2022042817/55a6892c1a28ab391e8b47c3/html5/thumbnails/43.jpg)
Challenge
How to find U? (MOR)How to find Π? (SYSID)Since y(θ), yr has to be valid for all admissable θ! (pMOR)Since y(θ), θr has to approximate θ well! (COMRED)FYI: My Models are nonlinear! (nMOR)BTW: I have non-affine parameter dependencies!
![Page 44: Model Order Reduction (for ODE-Constrained Optimization)](https://reader033.vdocuments.mx/reader033/viewer/2022042817/55a6892c1a28ab391e8b47c3/html5/thumbnails/44.jpg)
Dual Approach
1 Gramian-Based Combined Reduction
2 Optimization-Based Combined Reduction
![Page 45: Model Order Reduction (for ODE-Constrained Optimization)](https://reader033.vdocuments.mx/reader033/viewer/2022042817/55a6892c1a28ab391e8b47c3/html5/thumbnails/45.jpg)
Gramian-Based Combined Reduction
Based on ...System Theory / Control TheoryLinear Control Systems and their encoded properties
Features:For Nonlinear Systems: Empirical Gramians [Lall et al’99]Combined Reduction: Empirical Cross Gramian and JointGramian [H. & Ohlberger’14]
![Page 46: Model Order Reduction (for ODE-Constrained Optimization)](https://reader033.vdocuments.mx/reader033/viewer/2022042817/55a6892c1a28ab391e8b47c3/html5/thumbnails/46.jpg)
Controllability
![Page 47: Model Order Reduction (for ODE-Constrained Optimization)](https://reader033.vdocuments.mx/reader033/viewer/2022042817/55a6892c1a28ab391e8b47c3/html5/thumbnails/47.jpg)
Observability
![Page 48: Model Order Reduction (for ODE-Constrained Optimization)](https://reader033.vdocuments.mx/reader033/viewer/2022042817/55a6892c1a28ab391e8b47c3/html5/thumbnails/48.jpg)
Balanced Truncation [Moore’81]
For Linear Control SystemsControllability and Observability can be computedas singular values of the System’s Gramian Matrices.Balancing these two matrices yields the so called HankelSingular Values3.
Why HSVs?A state component that is neither controllable nor observableis not contributing to the input-to-output energy transfer.The smaller the HSV, the less important the associated(balanced) state is.
3Singular values of the Hankel operator mapping inputs to outputs.
![Page 49: Model Order Reduction (for ODE-Constrained Optimization)](https://reader033.vdocuments.mx/reader033/viewer/2022042817/55a6892c1a28ab391e8b47c3/html5/thumbnails/49.jpg)
Parameter Observability
Parameter Augmented General Control System:(x(t)
θ(t)
)=
(f (x(t), u(t), θ(t))
0
)y(t) = g(x(t), u(t), θ(t))(
x(0)θ(0)
)=
(x0θ0
)
![Page 50: Model Order Reduction (for ODE-Constrained Optimization)](https://reader033.vdocuments.mx/reader033/viewer/2022042817/55a6892c1a28ab391e8b47c3/html5/thumbnails/50.jpg)
Optimization-Based Combined Reduction
Based on:Greedy AlgorithmLarge-Scale Inverse Problems
Features:Combined Reduction: [Lieberman et al’12]Data-Driven: [H. & Ohlberger (submitted)]
![Page 51: Model Order Reduction (for ODE-Constrained Optimization)](https://reader033.vdocuments.mx/reader033/viewer/2022042817/55a6892c1a28ab391e8b47c3/html5/thumbnails/51.jpg)
Greedy Algorithm
Minimize Maximal Error:
θi+1 = argmaxθ⊥θ0,...i ‖yθ − yθr ‖22 + γ‖θ‖22= argminθ⊥θ0,...i −‖yθ − yθr ‖22 − γ‖θ‖22
Parameter Projection:
Π = [θ0, . . . , θp]
![Page 52: Model Order Reduction (for ODE-Constrained Optimization)](https://reader033.vdocuments.mx/reader033/viewer/2022042817/55a6892c1a28ab391e8b47c3/html5/thumbnails/52.jpg)
Enhanced Greedy Algorithm
Monte-Carlo Parameter Base & Data-Driven Regularization:
M = [P(θ)0, . . . ,P(θ)p]
θi+1 = argmaxMθ⊥Mθ0,...i ‖yθ − yθr ‖22 + γ‖θ‖22 + δ‖yd − yθr ‖
22
Parameter Projection:
Π = [M−10 θ0, . . . ,M
−1p θp]
![Page 53: Model Order Reduction (for ODE-Constrained Optimization)](https://reader033.vdocuments.mx/reader033/viewer/2022042817/55a6892c1a28ab391e8b47c3/html5/thumbnails/53.jpg)
Combined Reduction
Parameter Greedy:
θi+1 = argmaxθ⊥θ0,...i ‖yθ − yr ,θr ‖22 + γ‖θ‖22xi = pod1(x(θi ))
State Projection:
U = [x0, . . . , xn]
![Page 54: Model Order Reduction (for ODE-Constrained Optimization)](https://reader033.vdocuments.mx/reader033/viewer/2022042817/55a6892c1a28ab391e8b47c3/html5/thumbnails/54.jpg)
Act V
Alltogether
![Page 55: Model Order Reduction (for ODE-Constrained Optimization)](https://reader033.vdocuments.mx/reader033/viewer/2022042817/55a6892c1a28ab391e8b47c3/html5/thumbnails/55.jpg)
Back to the Beginning
x(t) = f (x(t), u(t), θ)
y(t) = g(x(t), u(t), θ)
x(0) = x0
Dimensions:dim(x(t))� 1dim(u(t))� dim(x(t))
dim(y(t))� dim(x(t))
dim(θ)� 1
![Page 56: Model Order Reduction (for ODE-Constrained Optimization)](https://reader033.vdocuments.mx/reader033/viewer/2022042817/55a6892c1a28ab391e8b47c3/html5/thumbnails/56.jpg)
Inverse Problem
ODE Constrained Optimization:
θd = argminθ ‖yθ − yd‖22s.t.:x(t) = f (x(t), u(t), θ)
yθ(t) = g(x(t), u(t), θ)
x(0) = x0
Remember:dim(x(t))� 1dim(u(t))� dim(x(t))
dim(y(t))� dim(x(t))
dim(θ)� 1
![Page 57: Model Order Reduction (for ODE-Constrained Optimization)](https://reader033.vdocuments.mx/reader033/viewer/2022042817/55a6892c1a28ab391e8b47c3/html5/thumbnails/57.jpg)
Reduced Order Inverse Problem
ODE Constrained Optimization:
θd = argminθr ‖yθr − yd‖22s.t.:xr (t) = fr (xr (t), u(t), θr )
yr ,θ(t) = gr (xr (t), u(t), θr )
xr (0) = xr ,0
Remember:dim(xr (t))� dim(x(t))
dim(θr )� dim(θ)
‖yθ − yr ,θr ‖ � 1
![Page 58: Model Order Reduction (for ODE-Constrained Optimization)](https://reader033.vdocuments.mx/reader033/viewer/2022042817/55a6892c1a28ab391e8b47c3/html5/thumbnails/58.jpg)
Numerical Results (HNM)
Parameter DimensionState Dimension
1e-3
1e-2
1e-1
1e+0
917
2533
4149
1
57 5749
4133
2517
91
Parameter DimensionState Dimension
1e-3
1e-2
1e-1
1e+0
917
2533
4149
1
57 5749
4133
2517
91
Gramian-Based and Optimization-Based Combined Reduction
![Page 59: Model Order Reduction (for ODE-Constrained Optimization)](https://reader033.vdocuments.mx/reader033/viewer/2022042817/55a6892c1a28ab391e8b47c3/html5/thumbnails/59.jpg)
What does this mean?
State-Space dynamics can be bound to low-dimensionalsub-spaces of the high-dimensional state-space.Identifiable parameters can be restricted to small sub-spaces ofthe high-dimensional parameter-space.State- and parameter-spaces can be reduced jointly,also for a nonlinear system.The inverse problem can be solved on the reduced spaces.Open issue: accurate parameter reconstruction
![Page 60: Model Order Reduction (for ODE-Constrained Optimization)](https://reader033.vdocuments.mx/reader033/viewer/2022042817/55a6892c1a28ab391e8b47c3/html5/thumbnails/60.jpg)
tl;dl
Networks can be modelled by control systemsIn this sense, the parameter inference is an ODE constrainedinverse problemDCM is a flavor of such inverse problem in a bayesian settingModel Reduction approximates large models with smallersurrogate modelsand thus accelerates the inversion / optimization
More Info:Me: http://wwwmath.uni-muenster.de/u/himpeM. Ohlberger: http://wwwmath.uni-muenster.de/u/ohlberger
MoRePaS: http://morepas.orgMORwiki: http://modelreduction.org
Thanks!