multigrid for nonlinear problems ferien-akademie 2005, sarntal, christoph scheit fas, newton-mg,...
TRANSCRIPT
Multigrid for Nonlinear Problems
Ferien-Akademie 2005, Sarntal, Christoph Scheit
FAS, Newton-MG, Multilevel Nonlinear Method
Outline
Motivation Basic Idea of Multigrid Classical MG-Approaches for nonlinear problems
– Newton-Multigrid– FAS– Properties of both approaches
Multilevel Nonlinear Method Conclusions Bibliography
Motivation
To solve (linear) systems of equations arising from the discretization of nowadays engineering science problems fast and robust solvers are needed
A lot of problems arising from engineering science contain nonlinearities
Example: nonlinear diffusion equation
),())(( yxfuug
Basic Ideas of Multigrid
A lot of relaxations schemes smooth the error. Consider Jacobi-relaxation:
nnnn
nnn
nn
BeeBexx
BezBxzexBx
zBxxfDzUDB
fDUxDxfUxDx
UDA
fAx
11
1
11
111
)(
:,:
Basic Ideas of Multigrid
From eigenvalue analysis (Local Mode analysis) or numerical experiments one can see that only high frequency errors are damped (smoothed) efficiently by most relaxation schemes
The basic idea is now, to smooth the error on a grid, on which the error looks high frequent. On a twice as coarse grid, the same mode appears with the double frequency relative to the number of grid points. Therefore high-frequency errors can be smoothed efficiently on the coarser grid.
Basic Ideas of Multigrid
The residual equation:
Using the residual equation, it is possible to compute the error and update the approximate solution
Bringing it all together, one can use coarser grids to efficiently compute a correction for the actual solution
rAe
xAfxAAx
fAx
~~
Basic Ideas of Multigrid
To use a sequence of Grids, transfer operators are needed:
– Restriction to transport the residual to a coarser grid
– Interpolation (or prolongation) to transport the correction back to the finer grid
Using this ideas, one can construct schemes like the well known V-Cycle
Basic Algorithm
MGM_Basic(x,f,l,ν1,ν2,μ)if (l == 1)
return x = exact_sol(x,f)endx_h = preSmoothing(x,f,ν1);r_h = f-Ax;r_H = Restriction(r_h);for (i = 1; i < μ; i++)
x_H = 0;e_H = MGM_Basic(x_H,r_H,l-1,ν1,ν2,μ)
ende_h = Prolongate(e_H);x_h = x_h + e_hx_h = postSmoothing(x_h,f,ν2)return x_h;
end
Nonlinear Problems
For an equation like our resulting operator N for the discretized equation is itself depending on the solution u
How to modify the algorithm for nonlinear problems?– Important, since for nonlinear problems the Residual Equation does
not hold any more (instead, nonlinear Residual Equation / defect equation):
rxNfxNNx
xxNxNNx
fNx
~~)~(~
fuu ³
Nonlinear Approaches for Multigrid
Newton-Multigrid
-> global linearization FAS (Full Approximation Scheme/Storage)
-> local linearization MNM (Multilevel Nonlinear Method)
-> combine local and global method
Newtons Method
First consider Newtons Method for scalar Problems:
If f(x+s) is a solution, then
)(
)(
)(
)()()(0
xf
xfxx
xf
xfsxfsxf
)(²)()()(
0)(
fsxfsxfsxf
xf
Newtons Method
Newtons Method
How to use Newtons Method for nonlinear equation systems? – Analog to the scalar case:
If F(v+s)=F(u) is a solution
Where s is the error of the current approximation, we get:
)(ξFs²F(v)sF(v)s)F(v
0F(u)T
F(v)F(v)][s)vF(sF(v)0 1TT
F(v)F(v)][vv 1T
Newtons Method
What is grad(F(u))?
-> the Jacobian of F(u)
3
3
2
3
1
3
3
2
2
2
1
2
3
1
2
1
1
1
)(
),,(
),,(
),,(
)(
3213
3212
3211
uf
uf
uf
uf
uf
uf
uf
uf
uf
uuuf
uuuf
uuuf
uJuF
Newtons Method
A concrete example for J(v):
Partial derivation yields:
0)(
0)²²2(³²
²1
31
2
u
fhuuhuuhfux
uiiiii
iF
22
21
2
2
32100
10
01
00132
)(
nuh
uh
h
vJ
Newton-Multigrid
Back to Newtons Method for Multigrid:
Using the nonlinear Residual Equation and the truncated taylor serious yields
The last equation is the linearized equation system and has to be computed instead of the original one using multigrid methods.
rvNuN
evJvFevF
)()(
)()()(
revJ
rvNevJvN
)(
)()()(
evu
Newton-Multigrid - Algorithm
v = init_sol();
r = f-N(u);
while (r < tol)
compute J(v);
e = 0;
for i = 0; i < numV-Cycles; i++
e = MGM_Basic(e,r,l,ν1,ν2,μ)
end
v = v + e;
r = f-N(u);
end
FAS
Newtons-Multigrid doesn‘t use Multigrid ideas to solve the nonlinear equation system, but uses a global linearization and an outer iteration with the basic Multigrid Method embeded as a solver for the linearized equation system
Different from the idea of Newtons Method for Multigrid, FAS treats directly the nonlinear equation system, using a nonlinear smoother for local linearization such as Gauss-Newton relaxation
FAS
Back to the nonlinear Residual Equation (defect correction equation):
We can formulate this equation on the coarse grid by:
Where is the injection operator (instead of full weighted restriction)
NvrNurNvNu
hHh
HhHh
HH vINrIuN ˆHhI
FAS - Algorithm (only V-Cycle)
FAS(x,f,l,ν1,ν2)if(l==1)
return x = exact_sol(x,f);endx_h = preSmoothing(x,f,ν1);f_H = restriction(f – A_h x_h)
+ A_h injection(x_h);x_H = injection(x_h); // initial guess for coarse gridFAS(x_H,f_H,l, ν1,ν2);x_h = x_h + prolongation(x_H – injection(x_h));x_h = postSmoothing(x_h,f,ν2);return x_h;
end
FAS – nonlinear relaxation
Instead of a global linearization FAS uses a nonlinear smoother, which is simply obtained by Newtons Method (for scalar problem):
Consider again the nonlinear equation:
Discretized we obtain:
Using Newtons Method yields the following iteration scheme:
fuxu
3
2
2
0)²²2( 13
12
iiiii fhuuhuuh
)32(
)²²2(222
13
12
i
iiiiiii uhh
fhuuhuuhuu
FAS – implementation hints
Start first with a linear problem; then the FAS-Algorithm must yield the same result as the standard MG-Algorithm (except roundoff errors)
For the nonlinear problem considered here, a standard Gauss-Seidel relaxation works also. In general one has to use a nonlinear smoother like presented above
Since FAS does not approximate the error, but directly improves the current solution on the different grid levels, don‘t forget to inject also the boundary condition (for the error in the standard MG this was not necessary, since for Dirichlet b.c. the b.c. for the error is always zero)
Properties of classical approaches
Newton: Fast convergence, often only
a few newton steps For each newton step, the
linearized equation must be solved accurately
A good initial guess is needed to ensure convergence (small attraction basin)
(slow) backtracking to find a good initial guess
FAS: No global linearization is
needed Convergence even for poor
initial guess, if a good approximation for the nonlinear operator is available (large attraction basin)
Converges slower to the solution than Newton-MG
Multilevel Nonlinear Method (MNM)
While Newtons-MG converges fast, we need a good initial guess
While FAS converges not so fast, it converges even for a poor initial guess if we have a good approximation for the nonlinear operator
Idea: Combine the properties of both algorithms, such that the resulting Method converges fast and even for a poor initial guess -> MNM
Use a robust approximation for the dominating operator -> MNM, Galerkin Coarsening
MNM
Once again back to the nonlinear Residual equation:
Now we want to split this equation into a large linear part and a small nonlinear part. The linear part corresponds to Newton-MG while the nonlinear part corresponds to FAS. The nonlinear part should be small, because in this case it would not be so bad, if the approximation of the nonlinear part is not so good (which was required by FAS)
rNvNu
MNM
To obtain this splitting, we add to the left hand side of the nonlinear Residual Equation J(v)e; e = u-v:
Rearranging the terms yields a linear and a nonlinear term:
Obviously, the linear part is O(e), but what about the nonlinear part?
rv)(uvJvuvJNvNu )())((
))(([...]
],),(,,[))((
vuvJNvNuF
rvuvJNvNuFvuvJ
MNM
Consider a Taylor serious:
Hence the nonlinear part is O(e²) and therefor we obtain a splitting with a large linear but a relatively small nonlinear part
evJvNuNeO
eOevJvNevNuN
)()()()(
)()()()()(2
2
MNM
Back to the complete equation:
There are two methods to bring the operators to the coarser grid: Rediscretization Galerkin Coarsening
We will use rediscretization only for the nonlinear part (though rediscretization might yield a bad approximation in case of a PDE with jumping coefficients, the influence for MNM is only O(e²)). Denote rediscretized operators by a head Â:
rv)(uvJNvNuvuvJ )())((
rv)(uvJvNuNvuvJ )(ˆˆˆ))((
MNM
Now we obtain an iteration by defining:
Substituting this into the original equation and rearranging yields the defect equation for MNM:
n
1n
u:v
u:u
nnnnn
1nn1nn1n
)u(uJ)uJ(uuNr
)u(uJ)uJ(uuN
ˆˆ
ˆˆ
MNM
As one can see, the following operators must be defined on the several levels:
While the first two will be simply rediscretized on each level, the third one is obtained by Galerkin coarsening:
To bring the current approximation to the next coarser level, we will use injection (as for FAS)
)J(u),(uJ,N nnˆˆ
hHn
hTHhn
H )I(uJ][I)(uJ
MNM - Algorithm
MNM(u,N,L,f,l,ν1,ν2)if(l == 0)
solve Nu+Lu=f:return u;
endRelax ν1 times equation
Nu+Lu=f;Compute residual
r=f-(Nu+Lu);Construct linearized operator
K = L+J^(u);Initialize coarse grid solution
u_H = injection(u);Galerkin Coarsening for linearized operator
K_H = galerkinCoarsening(K);
MNM – Algorithm(II)
Compute L for coarse gridL_H = K_H + J^_H(u_H);
Compute RHS for coarse gridf_H = restriction(r) + N_H u_H + L_H u_H;
recursive callMNM(u_H,N_H,L_H,f_H,l-1,ν1,ν2);add correction
u = u + prolongation(u_H - injection(u));postsmoothing
Nu+Lu=f;End
Where L :=the linear correction to N
)(ˆ)( nn uJuJ
MNM – Concrete Example
Consider the equation
For the approximated operators we obtain:
Here B is a scaling to ensure compatibility with the linearized coarse grid operator due to Galerkin coarsening
fCeuxu
2
2
])2([]ˆ[
]2[]ˆ[
12
12
12
12
iu
iiiiHH
iu
iiiiHH
uCeHuuHBuK
uCeHuuHBuN
i
i
MNM - Adaptive
Idea: Use parameters to „controll“ how much of FAS and Newton should be used
Consider the complete coarse grid operator:
Two points of view: The first term is the main term, second and third term
are a nonlinear correction The second term is the main term, while the first and
the third term are a linear correction
HHHH KNKN ˆˆ
MNM - Adaptive
Now we can use a weighting of the operators:
a=1, b=0: Newtons Mehtod a=0, b=1:FAS a=1=b=1, MNM
]1,0[,
ˆ)1(ˆ
ba
baba HHHH KNKN
MNM – Results
2-D diffusion problem
where
)()),,(( xfuuuxg x
0,]|)|(1[
|)|(1||1)(
0,1)(21
2
uu
uukugg
uugg
qp
pp
q
MNM - Results
P/α .5 .75 1
1.5 MNM=0.18
FAS=0.26
MNM=0.26
FAS=0.35
MNM=0.97
FAS=0.64
2 MNM=0.13
FAS=0.23
MNM=0.25
FAS=0.33
MNM=0.38
FAS=0.41
2.5 MNM=0.12
FAS=0.21
MNM=0.25
FAS=0.33
MNM=0.41
FAS=0.42
MNM – Implementation hints
Due to the Galerkin coarsening we have the restriction operator acting on the left hand side as well as on the right hand side hence it cancels out. But the prolongation operator is only on the right hand side, therefor we have to introduce a compatible scaling also for the rediscretized operators.
Since N + L is just an approximation of the fine grid operator (nonlinear), a nonlinear relaxation method is needed, such as for FAS (e.g. Gauss-Newton)
Conclusions(I)
FAS and Newton-MG have both advantages and disadvantages
MNM combines the good properties of both methods, but introduces difficulties due to scaling of the coarse grid approximations for the operators
MNM yields usually fastest convergence factor of all three approaches
Sometimes MNM does not converge, than backtracking can be used, but yields poor convergence
Conclusions(II)
Adaptive MNM can be used instead of MNM with backtracking, yielding a quite good convergence factor
The computational cost per V-Cycle for MNM is more expensive than for FAS or Newtons method, but less than the sum of both
MNM is still a research topic MNM is more complicated to implement
Bibliography
I. Yavneh and G Dardyk, A Multilevel Nonlinear Method, Haifa, 2005
W. L. Briggs, V. E. Henson, and S F. McCormick, A Multigrid Tutorial, SIAM, Philadelphia, second ed., 2000
V. E. Henson, Multigrid for nonlinear problems: an overview, Center for Applied Scientific Computing Lawrence Livermore National Laboratory, 2003