inverse problems - applications and solution strategies · 2014. 12. 10. · inverse problems -...

Inverse Problems - Applications and Solution Strategies

Inverse Problems -Applications and Solution Strategies

Barbara Kaltenbacher, Alpen-Adria Universitat Klagenfurt

AD12, Fort Collins, July 2012

Outline

I inverse problems

I some applicationsI regularization of nonlinear problems

I Tikhonov regularizationI iterative methodsI Kaczmarz methodsI expectation maximization

Inverse Problems

Determine causes for

identification

observedor

desired

optimization

effects.

Inverse problems are often unstable:Small perturbations in the data can lead to large errors in the solution!→ regularization necessary

Identifiability question:Are the searched for quantities uniquely determined by the given data?

Inverse Problems

identification

observedor

desired

optimization

effects.

Inverse Problems

identification

observedor

desired

optimization

effects.

Some Application Examples

Material Characterization for Magnetic Flux Measurement

measurement principle:

Faraday’s Law:

moving conductor

in magnetic field

induces electric voltage.

→ identification of the space-dependentmagnetic permeability of the coil core;

→ hysteresis modelling

• Endress + Hauser, Reinach, CH;

• Inst. theor. Elektrotechnik, Univ. Stuttgart;

• Inst. Mechanik u. Mechatronik, TU Wien

Virtual Material Development

cross section of piezoceramicpolarization hysteresis strain hysteresis

→ micromechnaical modelling of ferroelectric polycrystals;→ hysteresis modelling

• joint project COMFEM (Bosch, Siemens, PI Ceramic, CeramTec,Fraunhofer Inst. f. Werkstoffmechanik, KIT)

• Inst. Mechanik u. Mechatronik, TU Wien

Parameter Identification in Systems Biology

→ determination of parameters in technical and biological systems→ application to identification of gene networks (activation/inhibition)

gene network dependency matrix

• Cluster of Excellence SimTech project with Jun.Prof.Dr. Nicole Radde,

Univ. Stuttgart

Identification of Cracks in Piezoceramics

current/voltage measurementsat surface electrodes

potential distributionin material with crack

→ localization of cracksinside piezoelectric devicesfrom surface measurements(nondestructive testing)

→ identifiability→ adaptive discretizetion

• DFG Project with Prof.Dr. Anna-Margarete Sandig, Univ. Stuttgart

All these applications lead toparameter identification problems in PDE/ODE models

Medical Imaging

I e.g., electrical impedance tomography (EIT)

−∇(σ∇φ) = 0 in Ω .

Identify conductivity σ = σ(x) from measurements of theDirichlet-to-Neumann map Λσ, i.e., all possible pairs(φ , σ∂nφ) on ∂Ω.

I e.g., quantitative thermoacoustic tomography (qTAT):

∇×(µ−1∇× E

∂tE + ε

∂t2E = J in Ω .

Identify σ = σ(x) from measurements of the deposited energyσ|E|2 in Ω.

Medical Imaging

I e.g., electrical impedance tomography (EIT)

−∇(σ∇φ) = 0 in Ω .

Identify conductivity σ = σ(x) from measurements of theDirichlet-to-Neumann map Λσ, i.e., all possible pairs(φ , σ∂nφ) on ∂Ω.

I e.g., quantitative thermoacoustic tomography (qTAT):

∇×(µ−1∇× E

∂tE + ε

∂t2E = J in Ω .

Identify σ = σ(x) from measurements of the deposited energyσ|E|2 in Ω.

Parameter identification in PDEs: Model problems

I e.g. “a-example” (transmissivity in groundwater modelling)

−∇(a∇u) = 0 in Ω .

Identify a = a(x) from measurements of u in Ω. [Adrian Sandu, Tue]

I e.g. “c-example” (potential in stat. Schrodinger equation)

−∆u + c u = 0 in Ω .

Identify c = c(x) from measurements of u in Ω.

Parameter identification in PDEs: Model problems

I e.g. “a-example” (transmissivity in groundwater modelling)

−∇(a∇u) = 0 in Ω .

Identify a = a(x) from measurements of u in Ω. [Adrian Sandu, Tue]

I e.g. “c-example” (potential in stat. Schrodinger equation)

−∆u + c u = 0 in Ω .

Identify c = c(x) from measurements of u in Ω.

Forward operator F

I EIT: F : σ 7→ Λσwhere Λσ : φ 7→ σ∂nφ and −∇(σ∇φ) = 0 in Ω .

I qTAT: F : σ 7→ σ|E|2

where ∇×(µ−1∇× E

)+ σ ∂

∂t E + ε ∂2

∂t2 E = J in Ω + bndy.cond.

I a-example: F : a 7→ uwhere −∇(a∇u) = 0 in Ω + boundary conditions

I c-example: F : c 7→ uwhere −∆u + c u = 0 in Ω + boundary conditions

forward operator F often involves solution of PDE,F is usually nonlinear and sometimes also nonsmooth

Forward operator F

)+ σ ∂

∂t E + ε ∂2

Forward operator F

)+ σ ∂

∂t E + ε ∂2

Forward operator F

)+ σ ∂

∂t E + ε ∂2

Forward operator F

)+ σ ∂

∂t E + ε ∂2

Forward operator F

)+ σ ∂

∂t E + ε ∂2

Differentiating F

I large numbers of dependent and independent variables

I directional derivative of state u wrt. parameter often obeyssimilar PDEs to those for the state itself.e.g. a-example −∇(a∇u) = 0; v = ∂u(a; b) solves

−∇(a∇v) = ∇(b∇u)

I derivatives of (cost) functionals wrt. parameters can becomputed from adjoint PDEs derived “by hand”

I . . . but sometimes one would not want to set up and solvethose PDEs

I . . . and correspondence (F ′(a)∗)h = (F ′(a)h)∗ only holds forspecific discretizations [Emre Ozkaya, Tue]

Differentiating F

−∇(a∇v) = ∇(b∇u)

Differentiating F

−∇(a∇v) = ∇(b∇u)

Differentiating F

−∇(a∇v) = ∇(b∇u)

Differentiating F

−∇(a∇v) = ∇(b∇u)

An Example: Ferroelectric Hysteresismicromechanical switching models, e.g. [Huber&Fleck 2001]:(

(M∑I=1

([sE ]I [d]tI

[d]I [εσ]I

M∑I=1

S. . . mechanical strain, D. . . dielectric displacementσ. . . mechanical stress, E. . . electric fieldSiI irreversible strains, Pi

I polarization

M . . . number of polarization directions ξI . . . volume fraction for variant I

ξI =K∑

J=1,J 6=I

(cJIψ(ξJ)− cIJψ(ξI )) . (1)

cIJ = φ(fIJ) fIJ = E ·∆Pi + σt∆Si (driving force)where ∆Pi = Pi

J − PiI ; ∆Si = Si

J − SiI .

ρu− DIV(

[cE ]effDIVTu− Si − [e]effgradϕ)

= 0 (2)

div([e]eff

(Bu− Si

)− [εS ]effgradϕ− Pi

)= 0 , (3)

Inverse Problems as Operator Equations

Nonlinear ill-posed problems

nonlinear operator equation

F (x) = y

!! from now on x is not space variable but parameter function= independent variables

Nonlinear ill-posed problems

nonlinear operator equation

F (x) = y

F : D(F )(⊆ X )→ Y . . . nonlinear operator;F not continuously invertible;

X , Y . . . Hilbert/Banach spaces;

y δ ≈ y . . . noisy data, ‖y δ − y‖ ≤ δ. . . noise level.

regularization necessary

Regularization of Inverse Problems

• unstable operator equation: F (x) = y with F : x 7→ y

• solution x = F−1(y) does not depend continuously on yi.e.,

(∀(yn), yn → y 6⇒ xn := F−1(yn)→ F−1(y)

• only noisy data y δ ≈ y available (measurements): ‖y δ − y‖ ≤ δ

• making∥∥F (x)− y δ

∥∥ small 6⇒ good result for x!

(∀(yn), yn → y 6⇒ xn := F−1(yn)→ F−1(y)

)• only noisy data y δ ≈ y available (measurements): ‖y δ − y‖ ≤ δ

(∀(yn), yn → y 6⇒ xn := F−1(yn)→ F−1(y)

An ExampleIdentification of a source term x in a 1-d differential equation y ′′ = x , δ = 1%

exact and noisy data y , yδ exact x vs x with∥∥F (x)− yδ

∥∥ = 1.e − 14

exact and noisy data y , yδ exact x vs x with∥∥F (x)− yδ

∥∥ = 2δ

(∀(yn), yn → y 6⇒ xn := F−1(yn)→ F−1(y)

• regularization means approaching solution along stable path:given (yn), yn → y construct xn := Rαn(yn) such thatxn = Rαn(yn)→ x = F−1(y)

• regularization parameter choice:trade-off between approximation (small α) and stability (large α)

(∀(yn), yn → y 6⇒ xn := F−1(yn)→ F−1(y)

Reference to noise level

Recall: We wish to solve

F (x) = y given y with∥∥∥y − y δ

∥∥∥ ≤ δ(measurement noise level)

I accuracy requirements for forward evaluation:‖F (x)− Fh(x)‖ ∼ δ to avoid loss of information during inversion.

I On the other hand it does not make sense to make the datamisfit

∥∥F (x)− y δ∥∥ smaller than the noise level δ.

discrepancy principle for choosing α = α∗:∥∥∥F (xδα∗)− y δ∥∥∥ ∼ δ

Parameter identification in a PDE as anonlinear operator equation

F (x) = y

F . . . forward operator: F (x) = (C S)(x) = C (u)where u = S(x) solves

D(x , u) = f . . . PDE

Hilbert spaces X , V , Y : x ∈ XS→ u ∈ V

C→ y ∈ Y

Regularization Methods

Tikhonov Regularization

Minimize jα(x) =∥∥F (x)− y δ

∥∥2+ α ‖x‖2 over x ∈ X ,

or equivalently

Minimize Jα(x , u) =∥∥C (u)− y δ

∥∥2+ α ‖x‖2 over x ∈ X , u ∈ V

under the constraint D(x , u) = f

(one shot formulation)

PDE constrained optimization [Johannes Lotz, Mon], [Andrea Walther, Tue]

∥∥2+ α ‖x‖2 over x ∈ X ,

or equivalently

∥∥2+ α ‖x‖2 over x ∈ X , u ∈ V

∥∥2+ α ‖x‖2 over x ∈ X ,

or equivalently

∥∥2+ α ‖x‖2 over x ∈ X , u ∈ V

Tikhonov Regularization with the Discrepancy Principle

Minimize jα(x) =∥∥∥F (x)− y δ

∥∥∥2+ α ‖x‖2 over x ∈ X ,

Choice of α: discrepancy principle (fixed constant τ ≥ 1)∥∥∥F (xδα∗)− y δ∥∥∥ = τδ

nonlinear 1-d equation φ(α) = 0 for α;evaluation of φ requires minimization of Tikhonov functional

Convergence analysis:

[Engl& Hanke& Neubauer 1996] and the references therein

Tikhonov Regularization and AD

I different levels on which AD might be used(as valid generally in PDE constrained optimization)

I tangential stiffness matrix in FEM solution of nonlinear PDEsfor forward evaluation of F ;

I gradients (Hessians) of cost function Jα and equalityconstraints (=PDE) wrt. parameter x and state u in one shotformulation of Tikhonov regularization;

I gradients (Hessians) of reduced cost function jα wrt.parameter x

I derivative of discrepancy function φ wrt. α.

I convergence with optimal rates requires(F ′(x)∗)h = (F ′(x)h)∗ + O(δ2)when solving via discretized version of Euler equationF ′(x)∗(F (x)− y δ) + α(x − x0) = 0

Tikhonov Regularization and AD

I different levels on which AD might be used(as valid generally in PDE constrained optimization)

I tangential stiffness matrix in FEM solution of nonlinear PDEsfor forward evaluation of F ;

I gradients (Hessians) of cost function Jα and equalityconstraints (=PDE) wrt. parameter x and state u in one shotformulation of Tikhonov regularization;

I gradients (Hessians) of reduced cost function jα wrt.parameter x

I derivative of discrepancy function φ wrt. α.

I convergence with optimal rates requires(F ′(x)∗)h = (F ′(x)h)∗ + O(δ2)when solving via discretized version of Euler equationF ′(x)∗(F (x)− y δ) + α(x − x0) = 0

Further aspects of Tikhonov RegularizationI problem adapted regularization terms, e.g.:

I total variation for sharp edges in imagesI l1 for sparsity

I problem adapted data misfit terms, e.g.:I Kullback-Leibler divergence for modelling Poisson noiseI l1 for robustness wrt. outliers

potential nonsmoothness! [Kamil A. Khan, Mon], [Markus Beckers, Tue], [Andreas Griewank, Tue], [Sabrina Fiege, Tue]. . .

I other parameter choice rules, e.g. a priori choice, generalizedcross-validation, L-curve, balancing principles, quasioptimality, . . .

I stochastic noise modelsI relation to Bayesian estimation via MAP estimatorI iterated Tikhonov regularizationI . . .

I stochastic noise models

I relation to Bayesian estimation via MAP estimatorI iterated Tikhonov regularizationI . . .

I stochastic noise modelsI relation to Bayesian estimation via MAP estimator

I iterated Tikhonov regularizationI . . .

Literature on Tikhonov regularization

I stability and convergence: [Seidman&Vogel 1989]

I convergence rates [Engl&Kunisch&Neubauer 1989] [Neubauer 1999]

[Hofmann&Scherzer 1998]

I analysis in Banach space: [Burger&Osher 2004],[Hofmann&BK&Poschl&Scherzer 2007] [BK&Hofmann 2010],[Hein&Hofmann&Kindermann&Neubauer&Tautenhahn 2009], [. . . ]

I adaptive discretization based on goal oriented error estimators[BK&Kirchner&Vexler 2011], [Don Estep, Thu]

I . . .

Gradient type methodsGradient descent for the minimization of

min 12 ‖F (x)− y‖2 over D(F ) :

xδk+1 = xδk + ωδkF′(xδk )∗(y δ − F (xδk ))

I Landweber iteration: ωδk ≡ 1

I steepest descent and minimal error method:

ωδk :=‖sδk‖

‖F ′(xδk )sδk‖2 , ωδk :=

‖yδ−F (xδk )‖2

‖sδk‖2 ,

where sδk = F ′(xδk )∗(y δ − F (xδk )).

I iteratively regularized Landweber

xδk+1 = xδk + F ′(xδk )∗(y δ − F (xδk ))+βk(x0 − xδk )

Gradient type methods with the Discrepancy Principle

xδk+1 = xδk + ωδkF′(xδk )∗(y δ − F (xδk ))

I stopping index k∗ acts as a regularization parameter

I discrepancy principle:∥∥∥y δ − F (xδk∗)∥∥∥ ≤ τδ < ∥∥∥y δ − F (xδk )

∥∥∥ , 0 ≤ k < k∗ ,

k∗ ∼ δ−1

Literature on gradient type methods

I Landweber for nonlinear inverse problems[Hanke&Neubauer&Scherzer 1995]

I steepest descent and minimal error method [Scherzer 1996],[Neubauer&Scherzer 1995]

I iteratively regularized Landweber [Scherzer 1998]

I generalization to Banach space setting:[Schopfer&Louis&Schuster 2006, Schopfer&Schuster&Louis 2008, BK

&Schopfer&Schuster 2009]

I . . .

Newton type methods

F ′(xδk )(xδk+1 − xδk ) = y δ − F (xδk )

formulation as least squares problem:

minx∈D(F )