consider preconditioning – basic principles basic idea: is to use krylov subspace method (cg,...
DESCRIPTION
Left, Right, and Symmetric Preconditioners Symmetric preconditioning : The matrix in brackets is symmetric positive definite. ( so we can use CG) The matrix in brackets is similar to It is enough to examine the eigenvalues of the nonsymmetric matrix to investigate convergence. No loss of symmetryTRANSCRIPT
Consider
Preconditioning – Basic Principles
bAx
Basic Idea: is to use Krylov subspace method (CG, GMRES, MINRES …) on a modified system such as
bMAxM 11
The matrix need not be formed explicitly; only need to solve whenever needed.
With: requirement is that it should be easy to solve for an arbitrary vector z.
AM 1
zMu
zMu
Left, Right, and Symmetric Preconditioners
Left preconditioning :bMAxM 11
Right preconditioning :
xuMbuAM 11 with
Symmetric preconditioning :
xCubCuACC TT with 11
TCCM
Left, Right, and Symmetric Preconditioners
Symmetric preconditioning :
xCubCuACC TT with )( 11
TCCM
The matrix in brackets is symmetric positive definite. ( so we can use CG)
The matrix in brackets is similar to It is enough to examine the eigenvalues of the nonsymmetric matrix to investigate convergence.
AM 1
AM 1
No loss of symmetry
Example
Example:
nna
a
A
111
1
11
1111
bAx
1000n ,5.0 iaii
1
1
b
CG iterations converges slowly without preconditioner.This is better than direct method
Example
Example:
nna
a
A
111
1
11
1111
bAx
1000n ,5.0 iaii
1
1
b
Preconditioner:)(AdiagM
TCCM MC
r
iterations no
No precond: after 40 iterations, achieves about 5-digit residual reduction.
PCG: after 30 iterations, achieves 15-digits
Practical Implementations
Consider
bAx
11~ ACCA
bCb 1~
2CCCM T
bxA~~~
xCx ~ why No…NO??
Loss of sparsity, matrix multiplication, finding inverse
CG
Practical Implementations
CG PCG-ver1
Practical ImplementationsPCG(ver1) PCG(ver2)
,~ ,~ ,~ ,~ 11
kkkk rCrCppCxxbCb
Practical ImplementationsPCG(ver2) PCG
kk rMz
Good Preconditioner
Good Preconditioner M:
1)Few number of iterations (fast convergence)
2) Cheap to solve the system Mx = y
textbookThe preconditioners used in practice are sometimes as simple as this one (diagonal) , but they are often far more complicated.
Example:
Five items
Item 1: (minimal polynomial with small degree)
DEF:p(x) is the minimal polynomial of the nxn matrix A if p(x) is the monic polynomial of least degree such that p(A)=0.
A = 2.0647 -0.2159 -0.0788 -0.8307 -0.9352 -3.8098 1.3290 -1.0141 -2.8977 -3.2940 4.6193 0.9699 3.3051 4.1618 4.6816 -1.2266 -0.2893 -0.1088 1.9143 -1.2225 1.5148 0.5460 0.2996 1.2249 4.3868
32 )3()2()( xxxc
Characteristic polynomial
minimal polynomial
)3)(2()( xxxp
Example:
Item 1: (minimal polynomial with small degree)
DEF:p(x) is the minimal polynomial of the nxn matrix A if p(x) is the monic polynomial of least degree such that p(A)=0.
A = 2.0647 -0.2159 -0.0788 -0.8307 -0.9352 -3.8098 1.3290 -1.0141 -2.8977 -3.2940 4.6193 0.9699 3.3051 4.1618 4.6816 -1.2266 -0.2893 -0.1088 1.9143 -1.2225 1.5148 0.5460 0.2996 1.2249 4.3868
minimal polynomial
)3)(2()( xxxp
WHY? },{},{ 2100 qqspanArrspan
Five items
Item 2: (A with few distinct eigenvalues)Note and why: seigenvaluedistinct ofnumber iterations ofnumber
Item 3: (small condition number)
A
k
A
k ee )0()(
112
Note: it is only one way ( large condition number )
Example:
Five items
CG
)(inf)(
AnPp
n pVbr
nn
GMRES
1 VVA
Item 4: (residual and error)
Five items
Then .0 and 0 if andˆfor vector residual theis andmatrix nonsigular is
. ofsolution theion toapproximatan is ˆ that suppose
xbxrA
bAxx
br
Ax
xx)(
ˆ
The inequality implies that condition number provide an indication of the connection between the residual vector and the accuracy of the approximation
In general, the relative error is bounded by the product of condition number with the relative residual
Item 5: (A with a good clustering of the eigenvalues)Textbook: A preconditioner M is good if M^(-1)A is not too far from normal and its eigenvalues are clustered.
Five items
Item 6: (M is close enough to A)Textbook: If the eigenvalues of are close to 1 and is small, then any of the iterations we have discussed can be expected to converge quickly.
Five items
AM 1
2
1 IAM
Survey of Preconditioners
Textbook: The preconditioners used in practice are sometimes as simple as this one (diag(A)), but they are often far more complicated.
1) Diagonal Scaling: Choose the preconditioner to be M = diag(c) where c is a suitable vector with nonzero entries.
Problem:
minimized is )( such that find
)(let and ,matrix given 1AMc
cdiagMA
Survey of Preconditioners
2) Incomplete Cholesky Factorization: (IC)
RRM T ~~ Incomplete Cholesky factorization
A Sparse, SPD
Incomplete Cholesky conjugate Gradient MethodICCG
3) Incomplete LU Factorization: (ILU)
ULM ~~ Incomplete LU factorization
A nonsymmetric
Use GMRES with M as preconditioner
Survey of Preconditioners4) Local Approximation
The matrix A represents coupling between elements both near and far from one another.
It may be worth considering M analogous to A but with the longer-range interactions omitted – a short-range approximation A.
In the simplest cases of this kind, M may consist simply of a few of the diagonals of A near the main diagonal, making this a generalization of the idea of a diagonal preconditioner.
BIIBI
IBIIBI
IB
41141
141141
14
B
Survey of Preconditioners5) Block Preconditioners
This is another kind of local approximation, in that local effects within certain components are considered while connections to other components are ignored.
BIIBI
IBIIBI
IB
41141
141141
14
B
Survey of Preconditioners7) Constant-coefficient approximation
PDE with variable coefficient),(),(),( yxfuu yyyxbxxyxa
),( yxfu Use the discritezed matrix as a preconditioner for the first problem
8) symmetric approximation
If a differential equation is not self-adjoint but is close in some sense to a self-adjoint equation that can be solved more easily, then the latter may sometimes sevre as a preconditioner
Survey of Preconditioners6) Domain Decomposition
In which solvers for certain subdomains of a problem are composed in flexible ways to form preconditioners for the global problem.
This method combine mathematical power with natural paralleizability
4
4
28
BII
IIB
A
CBBBABA
ATT21
22
11
4
4
4
21
BIIBI
IBAA
Survey of Preconditioners6) Domain Decomposition
In which solvers for certain subdomains of a problem are composed in flexible ways to form preconditioners for the global problem.
This method combine mathematical power with natural paralleizability
5
5
25
BII
IIB
A
1
43
2
3
2
24 BI
IBA
4
1 2
Note:the problem can be parallized
Survey of Preconditioners7) Low-order discertization
Often a differential or integral equation is discretized by a higher-order method. Bringing a gain in accuracy but making the discretization stencils bigger and the matrix less sparse. A lower-order approximation of the same problem, with its sparser matrix, may be an effective preconditioner.
9-point formula.
jiyxji uδδ h
u ,22
2,1
jiyxyxji uδδδδ h
u ,2222
2, 611
5-point formula.
-4 1
1
1
1
Survey of Preconditioners
8) saddle-point system
0TBBM
A
SM
P0
0BMBS T 1
9) Generalized saddle-point system
CBBM
A T
SM
P0
0
CBMBS T 1
seigenvaluedistinct 3 has 1AP
Survey of Preconditioners
10) Polynomial preconditioner
0
21 )1()1()1(11n
nxxxxx
21 )()( AIAIIA
Survey of Preconditioners
11) Splitting
Many applications involve combinations of physical effects, such as the diffusion and convection that combine tp make up the Navier-Stokes equations of fluid mechanics.
ConvectionDiffusion 1A 2A
Example:Example:
CBBBABA
ATT21
22
11
Example: ADILaplacian in two or three dimensions is composed of analogous operators in each of the dimensions separately. This idea may form the basis of a preconditioner.