bayesian computation approach to inverse problems in heat conduction materials process design and...
TRANSCRIPT
Bayesian Computation Approach to Inverse Bayesian Computation Approach to Inverse Problems in Heat ConductionProblems in Heat Conduction
Materials Process Design and Control LaboratoryMaterials Process Design and Control Laboratory
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
Principal investigator: Prof. Nicholas ZabarasPrincipal investigator: Prof. Nicholas ZabarasPresenter: Jingbo WangPresenter: Jingbo Wang
Materials Process Design and Control LaboratorySibley School of Mechanical and Aerospace Engineering
188 Frank H. T. Rhodes HallCornell University
Ithaca, NY 14853-3801
Email: [email protected]: (607) 255-9104
URL: http://www.mae.cornell.edu/zabaras/
Materials Process Design and Control LaboratoryMaterials Process Design and Control Laboratory
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
Presentation outline
• Introduction to inverse problems in heat conductionIntroduction to inverse problems in heat conduction
• Introduction to Bayesian computationIntroduction to Bayesian computation
----- Fundamentals of Bayesian statistical inference----- Fundamentals of Bayesian statistical inference
----- Markov random field (MRF) and discontinuity adaptive Markov ----- Markov random field (MRF) and discontinuity adaptive Markov
random field (DAMRF)random field (DAMRF)
----- Markov chain Monte Carlo (MCMC) simulation----- Markov chain Monte Carlo (MCMC) simulation
• A few examplesA few examples
----- Inverse heat conduction problem (IHCP)----- Inverse heat conduction problem (IHCP)
----- Heat source reconstruction----- Heat source reconstruction
• Apply Bayesian to computationally intensive problems --- an Apply Bayesian to computationally intensive problems --- an
extension to inverse radiationextension to inverse radiation
• ConclusionsConclusions
Γo
Γg
Γh
* ***
****
** unknown heat flux
known temperature
known heat flux
thermocouples
Inverse problems in heat conduction
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
Materials Process Design and Control Laboratory
***
** **
**
Heat sources
)( TktTC P
in ,
,),( gTtxT ,g
],,0[ maxtt
,),(
hqntxT
k
on
,h),()0,( 0 xTxT
,),(
0qntxT
k
)(unknown ,0
],,0[ maxtt
on ],,0[ maxtt
in ,
on ].,0[ maxtt
+ f
Features of the inverse problem
Materials Process Design and Control LaboratoryMaterials Process Design and Control Laboratory
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
• ill-posedness well-posedness: --- existence --- uniqueness --- continuous dependence of solutions on measurements
stability
A problem is ill-posed if it is not well-posed
identifiability
• uncertainties
random errors
solution errors
measurement errors
model uncertainties
numerical errors
incomplete mathematical representation
perturbations in systemparameters, initial or boundaryconditions …
discretization error
roundoff error
unsolved length scales
Approaches to the inverse problems
Materials Process Design and Control LaboratoryMaterials Process Design and Control Laboratory
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
optimizationobjective
minimum least-squares error
maximum entropy
Maximum Likelihood
Bayesian inference
Gradient methods
(sensitivity and/or adjoint problems need to be solved)
Monte Carlo methods (importance sampling, rejection sampling, Markov chain MC, simulated annealing)
function specification
Tikhonov regularization
(S) future information
iterative regularization
--- conjugate gradient
--- EM method
convexification method
mollification method
regularization
Bayesian computation approach
Materials Process Design and Control LaboratoryMaterials Process Design and Control Laboratory
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
• probabilistic description of inverse solutions
• quantification of various uncertainties
• data driven in nature
• direct simulation in deterministic space
• prior distribution regularization (spatial statistics
models)
• sampling schemes (MCMC, Latin hypercube …)
Advantages:
)|()()|( YppYp
Fundamentals of Bayesian statistics
Materials Process Design and Control LaboratoryMaterials Process Design and Control Laboratory
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
• Bayesian statistics
)|( YθP)(
)()|(YP
θPθYP
• Bayesian inference to estimation (regression)
- interested in distribution of random quantities θ=[θ1, θ2, … θm]T
- ‘a priori beliefs’ about P(θ) - data Y=[Y1, Y2, … Yn]T relevant to θ
Priori pdf
Likelihood
Posterior pdf
• Bayes’ formula
prior + evidence => posterior probability of a hypothesis
The likelihood
Materials Process Design and Control LaboratoryMaterials Process Design and Control Laboratory
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
Y = F(θ) + ω
For a typical system as,
the likelihood is determined by distribution of ω.eg: ω ~ N(0, σ2)
FYθp T
2))((
21exp{)|( Y FY ))}((
• conditional probability of observation (data) Y given the parameter θ• interface to data
prior posterior
The prior
Materials Process Design and Control LaboratoryMaterials Process Design and Control Laboratory
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
• role of a prior pdf --- incorporate known to a priori information --- regularize the likelihood
• a prior distribution can be “informative” or “improper”
• techniques of prior distribution modeling --- accumulated distribution information --- conjugate prior distributions --- physical constraints --- local uniforms --- spatial statistics models
An example of conjugate prior:if Y|θ,σ ~ N(θ, σ), then θ ~ N(θo, σo), σ ~ inv-Gamma (a,b).
A class π of prior distributions is said to form a conjugate family if the posterior density p(θ|X) is in the class for all X whenever the prior density is in π.
More complicated models
Materials Process Design and Control LaboratoryMaterials Process Design and Control Laboratory
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
* A hierarchical structure
)()(),|()|,( ppYpYp
)()|(),|()|,( ppYpYp
)()|( pYp )|( p Ym Yo , )|( Yop
diminish the effect of poor knowledge on hyper-parameters.
* An augmented formulation
quantifying measurement error from data
* A Expectation-Maximization formulation
more robust formulation (iterative regularization) when there are missing data
! Bayesian is adaptive model --- posteriors can be treated as priors for new data
Spatial statistics models
Materials Process Design and Control LaboratoryMaterials Process Design and Control Laboratory
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
Markov random field (MRF)
• having exponential form• explore the spatial and temporal dependence• close related to Tikhonov regularization
}))((exp{)( ~ ji jiijWp
q
x
t
θi
wi
Neighbors of θ
u,
0
)(2
21u
u, else
DAMRF
Monte Carlo simulation
Materials Process Design and Control LaboratoryMaterials Process Design and Control Laboratory
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
Monte Carlo Principle
1. draw an i.i.d. set of samples {x(i)} i = 1:N from a target density p(x)
2. approximate the target density with the following empirical point-mass function
3. approximate the integral (expectation) I(f) with tractable sums IN( f )
N
ixN x
Nxp
i1
)(1
)(
N
iX
NiN dxxpxffIxf
NfI
1
)()()()(1
)(
Markov chain Monte Carlo (MCMC)
Materials Process Design and Control LaboratoryMaterials Process Design and Control Laboratory
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
Initialize xInitialize x00
For i=0:N-1For i=0:N-1
sample u~U(0,1)sample u~U(0,1)
sample sample xx** ~ ~ q(q(xx**|x|xii) )
if u < A(xif u < A(xii, , xx**)=min)=min{1, p(x{1, p(x**)q(x)q(xii|x|x**)/(p(x)/(p(xii)q(x)q(x**|x|xii))}))}
xxi+1i+1=x=x**
else xelse xi+1i+1=x=xii
• Sampling from a complex distribution using Markov chain mechanism
Metropolis-Hastings algorithm
Gibbs sampler
)|(~ 11
ijj
ij xxpx
Initialize x0
For i = 0:N-1 For j = 1:m sample
Bayesian formulation for IHCP
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
Materials Process Design and Control Laboratory
• Parameterization of unknown heat flux q0
m
iii txwq
10 ),( Unknown vector θ
Input θInput θ direct numericalsolver F
direct numericalsolver F
Measurement YMeasurement Y
simulationnoise
Y = F(θ) + ω
• System input and output relation
random
• Likelihood function
FYθp T2
))((2
1exp{)|(
Y FY ))}(( --- known σ
--- unknown σ FYθ,p T2
))((2
1(σ2)-n/2exp{σ2)|(
Y FY ))}((
Assumptions• numerical error much less then measurement noise• ω iid ~ N(0, σ2)
Bayesian formulation for IHCP (cont.)
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
Materials Process Design and Control Laboratory
Markov Random Field (MRF)
}))((exp{)(~ ji
jiijWp
}2
exp{)( 2/ Wp Tm
2
21
)( uu
else
ji
jin
Wi
ij ~
,0
,1
,
Prior distribution modeling
--- Single layer posterior:
--- hierarchical posterior:
TWYYp T2 )
2
1exp(})(
2
1exp{)|(
Y )(
}2
1exp{}
2
)()(exp{),,( 2//2 W
v
YYvvp Tm
T
Tn
TT
}exp{}exp{ 11
)1(0
1 10 TT vv
H H
H H
• Gibbs sampler
Gibbs sampler and modified Gibbs sampler
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
Materials Process Design and Control Laboratory
1
2i
i ii i
b
a a
2
2 21 1
2N N
si s sii ii i p
s s
H Ha W b
s s st t p ji j ik kt i j i k i
Y H W W
),(~| 2iiii N
)|(~ 11
ijj
ij θθpθ
Initialize θ0
For i = 0:N-1 For j = 1:m sample
• modified Gibbs sampler
(i) Initialize θ(0), λ(0) and vT(0)
(ii) For i = 0:N-1 ---For j = 1:m sample ---sample u ~ U(0,1) ---sample λ(*) ~ qλ(λ(*) | λ(i)) ---if u < A(λ(*) , λ(i) ) λ(I+1) = λ(*) ---else λ(I+1) = λ(i) ---sample u ~ U(0,1) ---sample vT
(*) ~ qv(vT(*) | vT
(i)) ---if u < A(vT
(*) , vT(i) )
vT(I+1) = vT
(*) ---else vT
(I+1) = vT(i)
)|(~ 11
ijj
ij θθpθ
xq
dL
Y (d,iΔt)
--- True q in simulationq
0 0.5 0.9
1.0
--- Normalized governing equation
2
2
xT
tT
1t 0 ,0 1x
0),0( xT 1x0
0
LxxT
)(0
tqxTx
,
,
, 1t 0
1t 0,
t1.0
--- Discretization of q(t)
θiθi-1 θi+1
dt
1D IHCP example
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
Materials Process Design and Control Laboratory
0.1
1D IHCP example (results)
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
Materials Process Design and Control Laboratory
d=0.5 σT=0.01 d=0.5 σT=0.001
d=0.1 σT=0.01 d=0.1 σT=0.001
d=0.5 σT=0.01(assume σT unknown )
Reconstruction of piecewise continuous heat source
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
Materials Process Design and Control Laboratory
),())(( txfTxktT
x
5.1
23
4
1
x
xk
0<=x<0.25
0.25<=x<0.5
0.5<=x<0.75
0.75<=x<=1.0
tx
tx
tx
tx
e
ex
ex
e
txf
5.2
)22(
)45(
2
),(
0<=x<0.25
0.25<=x<0.5
0.5<=x<0.75
0.75<=x<=1.0
txetxT),(
with
analytical solution to the direct problem:
2D heat source reconstruction
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
Materials Process Design and Control Laboratory
• Normalized governing equations
,2
2
2
2
fy
T
x
T
t
T
,10 x ,10 y ,0t
,01010
yyxx yT
yT
xT
xT
0T , t = 0.
,0t
unknown
insulated
insulated
insulated
insulated
*
*
*
**
**
*
* **
* **
*
*
*
*
*
*
*
*
*
*
*
2125.02
20)10exp(),,(
ttyxf
2
22
125.02
)725.0()75.0(exp{
yx
2D heat source reconstruction
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
Materials Process Design and Control Laboratory
true heat source
reconstructed heat source when σT=0.005
reconstructed heat source when σT=0.02
Inverse Heat Radiation Problem
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
Materials Process Design and Control Laboratory
S(gray
boundary)thermocouple
participatingmedia
Vheat source
*
**
*
*
What g(t) causes measured T?
2 ( ) ( )p r
TC k T g t G x x y y z zq
t
4( ) ( )
4 bs I I I r d Is
4b
b
TI
4
14 ( ( ) )
4br I I r s dq
0
1( ) ( ) 0b n sI r s I n I r d n ss s
wT T
Direct simulation
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
Materials Process Design and Control Laboratory
A finite element (FE) + S4 method framework
1. Set T(i)guess = T(I-1);
2. Substitute T(i)guess to compute Ib ;
3. Solve intensity eq for I(i);
4. Compute ;
5. Solve temperature eq to update T(i)guess;
6. If the solution converged, set T(i)guess
as T(i) and save I(i); otherwise, go to 2;
7. Go to next time step.
rq
Reduced order modeling --- A POD based approach
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
Materials Process Design and Control Laboratory
• POD eigenfunction problem• reduced order models
( ) ( )
1
1 eNi i
Vie
U U dvN
1
( ) ( ) ( )TK
h Ti i
i
T t r a t r
1
( ) ( ) ( )IK
h Ii i
i
I t r s b t r s
1
( ) 1TK
jj ji i j j T
i
daM H a S Q g t j K
dt
1 1
1I IK K
ji i ji i j Ii i
A b B b D j K
2( )Tj p jV
M C dv T T
ji j iVH k dv
Tj jrVS dvq
( )Tj jV
Q G x x y y z z dv 4
{ ( ) }I I I Iji i j i jVA s d dv
4 4{( ) }I I
ji i jVB d d dv
4
( )I Ij b b jV
D I I d dv
• Solution as linear combination of POD basis
MCMC algorithm --- a cycle design of single component update
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
Materials Process Design and Control Laboratory
• implicit likelihood MH sampler• increasing acceptance probability single component update
Algorithm:
( ) ( 1) ( ) ( ) ( )2
1 1( ) exp{ ( )}
22i i i
j j j j j jqjqj
q
},...,,,...,,{ )()(1
)1(1
)1(2
)1(1
)1( im
ij
ij
iiij
Initialize Initialize θθ00
For i=0:N-1For i=0:N-1
For j=1:mFor j=1:m
sample u~U(0,1)sample u~U(0,1)
sample sample θθ**jj ~ ~ qqjj((θθ **
jj | | θθ ii
-j-j , , θθ ii
jj ) )
if u < A(if u < A(θθ iijj , , θθ **
jj ))
θθi+1i+1==θθ**
else else θθi+1i+1==θθii
A testing example
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
Materials Process Design and Control Laboratory
x
z y
1m
1m
1m
g(t)(0.5m, 0.5m, 0.5m)
800K
800K
800K
800K
800K
O
800K
***
12
3
o t
g(t)
400kW/m3
0.05s0.01s 0.04s
80kW/m3
o t
g(t)
0.02s 0.04s 0.05s
160kW/m3
80kW/m3
g1(t)
g2(t)
Schematic of the example
Profile of testing heat sources
Frame 001 09 Mar 2004 Frame 001 09 Mar 2004
Basis fields
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
Materials Process Design and Control Laboratory
Frame 001 09 Mar 2004 Frame 001 09 Mar 2004 Frame 001 09 Mar 2004 Frame 001 09 Mar 2004
Frame 001 09 Mar 2004 Frame 001 09 Mar 2004 Frame 001 09 Mar 2004 Frame 001 09 Mar 2004 Frame 001 09 Mar 2004 Frame 001 09 Mar 2004
Frame 001 09 Mar 2004 Frame 001 09 Mar 2004
Frame 001 09 Mar 2004 Frame 001 09 Mar 2004 Frame 001 09 Mar 2004 Frame 001 09 Mar 2004
1st, 3rd and 6th Basis of Th
1st, 3rd and 6th Basis of Ih
along direction s =[0.9082483 0.2958759 0.2958759]
1st, 3rd and 6th Basis of Ih
along direction s =[-0.9082483 0.2958759 0.2958759]
Homogeneous temperature solution
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
Materials Process Design and Control Laboratory
Th computed by full model Th computed by reduced order model
Comparison of reduced order solutions atthermocouple locations
Heat source reconstruction
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
Materials Process Design and Control Laboratory
MAP estimates of g1 at different magnitude of noise
MAP estimates of g2 at different magnitude of noise
Posterior mean of g1 when σT =0.005
Posterior mean of g2 when σT =0.005
• Bayesian inference treats the inverse problem as a well-posed problem in an expanded stochastic space• Bayesian inference approach provides statistical distribution as well as point estimates of inverse solution• In seeking point estimates, Bayesian approach regularizes the ill-posedness of inverse problem through prior distribution modeling• MCMC samplers provide accurate estimates for the statistics of inverse solution• Bayesian computation is applicable to complex inverse problems via reduced-order modeling
Conclusions
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
Materials Process Design and Control Laboratory
Futures--- Sequential Bayesian filter--- Stochastic upscaling via Bayesian computation--- Uncertainty quantification in multiscale simulation