the fréchet derivative of a generalized matrix function...vanni noferini the fréchet derivative of...
TRANSCRIPT
-
The Fréchet derivative of a generalized matrix function
Vanni Noferini (University of Essex)
“Network Science meets Matrix Functions”
Univerity of Oxford, 1-2/9/16
September 1st, 2016
Vanni Noferini The Fréchet derivative of a generalized matrix function 1 / 33
-
Classical matrix functions
Classical definition in linear algebra: given
f : C→ C
and a square matrix A ∈ Cn×n we wish to define f (A) in a way that“mimics” the property of f .
For example:A1/2 should solve X 2 = AetA should integrate ˙y(t) = Ay(t), y(0) = v .. . .
Vanni Noferini The Fréchet derivative of a generalized matrix function 2 / 33
-
Classical matrix functions
Classical definition in linear algebra: given
f : C→ C
and a square matrix A ∈ Cn×n we wish to define f (A) in a way that“mimics” the property of f .
For example:A1/2 should solve X 2 = AetA should integrate ˙y(t) = Ay(t), y(0) = v .. . .
Vanni Noferini The Fréchet derivative of a generalized matrix function 2 / 33
-
Defining matrix functions from the eigendecomposition
From Linear Algebra 1:
A Jordan block is a bidiagonal Toeplitz matrix with unit superdiagonal:λ 1 0 00 λ 1 00 0 λ 10 0 0 λ
A matrix is in Jordan canonical form if it is the direct sum of Jordanblocks:
λ 1 0 00 λ 0 00 0 λ 00 0 0 µ
Vanni Noferini The Fréchet derivative of a generalized matrix function 3 / 33
-
Defining matrix functions from the eigendecomposition
Rules:
The function of a Jordan block is an upper triangular Toeplitz matrix:
f
λ 1 0 00 λ 1 00 0 λ 10 0 0 λ
=
f (λ) f ′(λ) f ′′(λ)/2! f ′′′(λ)/3!0 f (λ) f ′(λ) f ′′(λ)/2!0 0 f (λ) f ′(λ)0 0 0 f (λ)
f (A1 ⊕ A2) = f (A1)⊕ f (A2)f (ZAZ−1) = Zf (A)Z−1
Hence f (A) is defined as long as f is defined and sufficiently many timesdifferentiable on the spectrum of A.
Vanni Noferini The Fréchet derivative of a generalized matrix function 4 / 33
-
Defining matrix functions from the eigendecomposition
Rules:
The function of a Jordan block is an upper triangular Toeplitz matrix:
f
λ 1 0 00 λ 1 00 0 λ 10 0 0 λ
=
f (λ) f ′(λ) f ′′(λ)/2! f ′′′(λ)/3!0 f (λ) f ′(λ) f ′′(λ)/2!0 0 f (λ) f ′(λ)0 0 0 f (λ)
f (A1 ⊕ A2) = f (A1)⊕ f (A2)f (ZAZ−1) = Zf (A)Z−1
Hence f (A) is defined as long as f is defined and sufficiently many timesdifferentiable on the spectrum of A.
Vanni Noferini The Fréchet derivative of a generalized matrix function 4 / 33
-
Computing matrix functions
On a computer, it is important to know the sensitivity of the attempetdcomputation.
A backward stable algorithm computes the exact solution of a nearbyproblem:
̂f (A) = f (A+ E ).Whether this is close to f (A) or not depends on the sensitivity of theproblem and, intuitively, on the “first derivative” of f at A.For a scalar, analytic f :
f (A+ E ) = f (A) + f ′(A)E + O(E 2).
Vanni Noferini The Fréchet derivative of a generalized matrix function 5 / 33
-
Computing matrix functions
On a computer, it is important to know the sensitivity of the attempetdcomputation.
A backward stable algorithm computes the exact solution of a nearbyproblem:
̂f (A) = f (A+ E ).
Whether this is close to f (A) or not depends on the sensitivity of theproblem and, intuitively, on the “first derivative” of f at A.For a scalar, analytic f :
f (A+ E ) = f (A) + f ′(A)E + O(E 2).
Vanni Noferini The Fréchet derivative of a generalized matrix function 5 / 33
-
Computing matrix functions
On a computer, it is important to know the sensitivity of the attempetdcomputation.
A backward stable algorithm computes the exact solution of a nearbyproblem:
̂f (A) = f (A+ E ).Whether this is close to f (A) or not depends on the sensitivity of theproblem and, intuitively, on the “first derivative” of f at A.For a scalar, analytic f :
f (A+ E ) = f (A) + f ′(A)E + O(E 2).
Vanni Noferini The Fréchet derivative of a generalized matrix function 5 / 33
-
Derivatives in Banach spaces
Let X ,Y be Banach spaces over R, x ∈ X and
f : X → Y .
If the limit
limt→0
f (x + te)− f (x)t
exists for all e ∈ X then f is Gâteaux differentiable at x .If there exists a bounded R-linear map Lf (x , ·) s.t.
lim‖h‖X→0
‖f (x + h)− f (x)− Lf (x , h)‖Y‖h‖X
exists for all h ∈ X then f is Fréchet differentiable at x .
Vanni Noferini The Fréchet derivative of a generalized matrix function 6 / 33
-
Derivatives in Banach spaces
Let X ,Y be Banach spaces over R, x ∈ X and
f : X → Y .
If the limit
limt→0
f (x + te)− f (x)t
exists for all e ∈ X then f is Gâteaux differentiable at x .
If there exists a bounded R-linear map Lf (x , ·) s.t.
lim‖h‖X→0
‖f (x + h)− f (x)− Lf (x , h)‖Y‖h‖X
exists for all h ∈ X then f is Fréchet differentiable at x .
Vanni Noferini The Fréchet derivative of a generalized matrix function 6 / 33
-
Derivatives in Banach spaces
Let X ,Y be Banach spaces over R, x ∈ X and
f : X → Y .
If the limit
limt→0
f (x + te)− f (x)t
exists for all e ∈ X then f is Gâteaux differentiable at x .If there exists a bounded R-linear map Lf (x , ·) s.t.
lim‖h‖X→0
‖f (x + h)− f (x)− Lf (x , h)‖Y‖h‖X
exists for all h ∈ X then f is Fréchet differentiable at x .
Vanni Noferini The Fréchet derivative of a generalized matrix function 6 / 33
-
Derivatives in Banach spaces
Fréchet differentiable ⇒ Gâteaux differentiable, but 6⇐If f is Fréchet differentiable then
condabs(f ,A) = ‖Lf (A, ·)‖
Vanni Noferini The Fréchet derivative of a generalized matrix function 7 / 33
-
Derivatives in Banach spaces
Fréchet differentiable ⇒ Gâteaux differentiable, but 6⇐
If f is Fréchet differentiable then
condabs(f ,A) = ‖Lf (A, ·)‖
Vanni Noferini The Fréchet derivative of a generalized matrix function 7 / 33
-
Derivatives in Banach spaces
Fréchet differentiable ⇒ Gâteaux differentiable, but 6⇐If f is Fréchet differentiable then
condabs(f ,A) = ‖Lf (A, ·)‖
Vanni Noferini The Fréchet derivative of a generalized matrix function 7 / 33
-
The Fréchet derivative of a classical matrix function
When A is diagonalizable, an explicit formula is known.
Theorem (Dalecǩii and Krěin 1965)
Let A = ZDZ−1 and f differentiable on the spectrum of A. Then
Lf (A,E ) = Z (F ◦ (Z−1EZ ))Z−1
with Fij =f (Dii )−f (Djj )
Dii−Djj if Dii 6= Djj or Fij = f′(Dii ) otherwise.
Vanni Noferini The Fréchet derivative of a generalized matrix function 8 / 33
-
The Fréchet derivative of a classical matrix function
When A is diagonalizable, an explicit formula is known.
Theorem (Dalecǩii and Krěin 1965)
Let A = ZDZ−1 and f differentiable on the spectrum of A. Then
Lf (A,E ) = Z (F ◦ (Z−1EZ ))Z−1
with Fij =f (Dii )−f (Djj )
Dii−Djj if Dii 6= Djj or Fij = f′(Dii ) otherwise.
Vanni Noferini The Fréchet derivative of a generalized matrix function 8 / 33
-
Example
Take
A =
2 0 00 2 00 0 −2
, f (A) = eA,E =1 2 10 0 00 0 −1
.Then
F =
e2 e2 sinh(2)e2 e2 sinh(2)sinh(2) sinh(2) e−2
⇒ Lf (A,E ) =e2 2e2 sinh(2)0 0 00 0 −e−2
.
Vanni Noferini The Fréchet derivative of a generalized matrix function 9 / 33
-
Consequences
The eigenvector matrix Z appears in the Dalecǩii-Krěin theorem. Thisleads to issues caused by non-normality of the argument matrix.
Theorem (N. Higham 2008)
condabs(f ,A) ≤ κ2(Z )2‖F‖.
Vanni Noferini The Fréchet derivative of a generalized matrix function 10 / 33
-
Consequences
The eigenvector matrix Z appears in the Dalecǩii-Krěin theorem. Thisleads to issues caused by non-normality of the argument matrix.
Theorem (N. Higham 2008)
condabs(f ,A) ≤ κ2(Z )2‖F‖.
Vanni Noferini The Fréchet derivative of a generalized matrix function 10 / 33
-
Generalized matrix functions
Given A ∈ Cm×n with SVD A = USV ∗ and a function
f : [0,∞)→ R
we wish to define f �(A) in some sensible way.
Vanni Noferini The Fréchet derivative of a generalized matrix function 11 / 33
-
Gmf: definition
From Linear Algebra 2:
In an SVD A = USV ∗, U,V are unitary and S is diagonal withnonincreasing nonnegative elements.This leads to the CSVD A = UrSrV ∗r where Ur ,Vr are submatrices ofU,V and Sr is square invertible (empty if A = 0).
Vanni Noferini The Fréchet derivative of a generalized matrix function 12 / 33
-
Gmf: definition
Definition (Hawkins and Ben-Israel 1973):
A = UrSrV ∗r 7→ f �(A) = Ur f (Sr )V ∗r .
Equivalent definition:
A = USV ∗ 7→ f �(A) = Uf �(S)V ∗,
where f �(S) is diagonal with diagonal entries
f �(σi ) =
{f (σi ) if σi 6= 00 if σi = 0
Vanni Noferini The Fréchet derivative of a generalized matrix function 13 / 33
-
Why do we care?Among the applications:
complex networks (Arrigo, Benzi, Estrada, Fenu, D. Higham,Klymko... );computing classical matrix functions of structured matrices (DelBuono, Lopez, Politi... );computer vision (Bylow, Kahl, Larsson, Olsson... );the unitary factor of the polar decompisition of a full rank matrix is agmf;the Moore-Penrose pseudoinverse of A is a gmf of A∗.
(So they are familiar to at least some people in the room...)
Vanni Noferini The Fréchet derivative of a generalized matrix function 14 / 33
-
Why do we care?Among the applications:
complex networks (Arrigo, Benzi, Estrada, Fenu, D. Higham,Klymko... );computing classical matrix functions of structured matrices (DelBuono, Lopez, Politi... );computer vision (Bylow, Kahl, Larsson, Olsson... );the unitary factor of the polar decompisition of a full rank matrix is agmf;the Moore-Penrose pseudoinverse of A is a gmf of A∗.
(So they are familiar to at least some people in the room...)
Vanni Noferini The Fréchet derivative of a generalized matrix function 14 / 33
-
ExampleToy matrix:
A =
2 0 00 2 00 0 00 0 0
.
for f (σ) = σ3,
f �(A) =
8 0 00 8 00 0 00 0 0
.for f (σ) = eσ,
f �(A) =
e2 0 00 e2 00 0 00 0 0
.Note that e0 6= 0.
Vanni Noferini The Fréchet derivative of a generalized matrix function 15 / 33
-
ExampleToy matrix:
A =
2 0 00 2 00 0 00 0 0
.for f (σ) = σ3,
f �(A) =
8 0 00 8 00 0 00 0 0
.
for f (σ) = eσ,
f �(A) =
e2 0 00 e2 00 0 00 0 0
.Note that e0 6= 0.
Vanni Noferini The Fréchet derivative of a generalized matrix function 15 / 33
-
ExampleToy matrix:
A =
2 0 00 2 00 0 00 0 0
.for f (σ) = σ3,
f �(A) =
8 0 00 8 00 0 00 0 0
.for f (σ) = eσ,
f �(A) =
e2 0 00 e2 00 0 00 0 0
.Note that e0 6= 0.
Vanni Noferini The Fréchet derivative of a generalized matrix function 15 / 33
-
Basic observations
In spite of their name, gmf do not reduce to classical mf when m = n;
Even when m = n, f �(A) for a polynomial function f is not apolynomial in A in the classical sense;If f (0) 6= 0 and A is rank deficient then f �(A) is not continuous – letalone differentiable– at A.If A = QH is a polar decomposition then f �(A) = Qf �(H).If H is Hpd then f �(H) = f (H); if H is Hpsd this is not generally trueunless f (0) = 0.
Vanni Noferini The Fréchet derivative of a generalized matrix function 16 / 33
-
Basic observations
In spite of their name, gmf do not reduce to classical mf when m = n;Even when m = n, f �(A) for a polynomial function f is not apolynomial in A in the classical sense;
If f (0) 6= 0 and A is rank deficient then f �(A) is not continuous – letalone differentiable– at A.If A = QH is a polar decomposition then f �(A) = Qf �(H).If H is Hpd then f �(H) = f (H); if H is Hpsd this is not generally trueunless f (0) = 0.
Vanni Noferini The Fréchet derivative of a generalized matrix function 16 / 33
-
Basic observations
In spite of their name, gmf do not reduce to classical mf when m = n;Even when m = n, f �(A) for a polynomial function f is not apolynomial in A in the classical sense;If f (0) 6= 0 and A is rank deficient then f �(A) is not continuous – letalone differentiable– at A.
If A = QH is a polar decomposition then f �(A) = Qf �(H).If H is Hpd then f �(H) = f (H); if H is Hpsd this is not generally trueunless f (0) = 0.
Vanni Noferini The Fréchet derivative of a generalized matrix function 16 / 33
-
Basic observations
In spite of their name, gmf do not reduce to classical mf when m = n;Even when m = n, f �(A) for a polynomial function f is not apolynomial in A in the classical sense;If f (0) 6= 0 and A is rank deficient then f �(A) is not continuous – letalone differentiable– at A.If A = QH is a polar decomposition then f �(A) = Qf �(H).
If H is Hpd then f �(H) = f (H); if H is Hpsd this is not generally trueunless f (0) = 0.
Vanni Noferini The Fréchet derivative of a generalized matrix function 16 / 33
-
Basic observations
In spite of their name, gmf do not reduce to classical mf when m = n;Even when m = n, f �(A) for a polynomial function f is not apolynomial in A in the classical sense;If f (0) 6= 0 and A is rank deficient then f �(A) is not continuous – letalone differentiable– at A.If A = QH is a polar decomposition then f �(A) = Qf �(H).If H is Hpd then f �(H) = f (H); if H is Hpsd this is not generally trueunless f (0) = 0.
Vanni Noferini The Fréchet derivative of a generalized matrix function 16 / 33
-
Intuitions on computational advantage
Although you do not compute classical mf via the (unstable?)eigendecomposition, their definition may lead to numerical ill conditioning.
Gmf are defined via the (stable) SVD. Does this imply that they are alwayswell conditioned?
Vanni Noferini The Fréchet derivative of a generalized matrix function 17 / 33
-
Intuitions on computational advantage
Although you do not compute classical mf via the (unstable?)eigendecomposition, their definition may lead to numerical ill conditioning.
Gmf are defined via the (stable) SVD. Does this imply that they are alwayswell conditioned?
Vanni Noferini The Fréchet derivative of a generalized matrix function 17 / 33
-
Gmf: conditions on differentiability
TheoremLet A ∈ Cm×n and f : [0,∞)→ R differentiable on an open subsetcontaining the positive singular value of A. Moreover, ifrankA < min(m, n), suppose that f (0) = 0 and f is right differentiable at0. Then f �(X ) is Fréchet differentiable at X = A.
Important: in this theorem, the Fréchet derivative is the real Fréchetderivative. Generally, generalized matrix functions are not complexdifferentiable.
Vanni Noferini The Fréchet derivative of a generalized matrix function 18 / 33
-
Gmf: conditions on differentiability
TheoremLet A ∈ Cm×n and f : [0,∞)→ R differentiable on an open subsetcontaining the positive singular value of A. Moreover, ifrankA < min(m, n), suppose that f (0) = 0 and f is right differentiable at0. Then f �(X ) is Fréchet differentiable at X = A.
Important: in this theorem, the Fréchet derivative is the real Fréchetderivative. Generally, generalized matrix functions are not complexdifferentiable.
Vanni Noferini The Fréchet derivative of a generalized matrix function 18 / 33
-
Some notation...
Given X ∈ Cm×n, Υ : Cm×n → Cm×n is the following real-linear operator:
Υ (X ) =
[X ∗1X2
]if m > n and X =
[X1X2
];
X ∗ if m = n;
[X ∗1 X2
]if m < n and X =
[X1 X2
].
Vanni Noferini The Fréchet derivative of a generalized matrix function 19 / 33
-
Notation again...Given the singular values of A ∈ Cm×n we introduce two operatorsF ,G ∈ Rm×n:
Fij =
σi f (σi )−σj f (σj )σ2i −σ
2j
if i 6= j , max(i , j) ≤ ν, and σi 6= σj ;σi f ′(σi )+f (σi )
2σiif i 6= j , max(i , j) ≤ ν, and σi = σj 6= 0;
f (σj )σj
if i > n, and σj 6= 0;f (σi )σi
if j > m, and σi 6= 0;f ′(σi ) otherwise;
Gij =
σj f (σi )−σi f (σj )
σ2i −σ2j
if i 6= j , i , j ≤ ν, and σi 6= σj ;σi f ′(σi )−f (σi )
2σiif i 6= j , i , j ≤ ν, and σi = σj 6= 0;
0 otherwise.
Vanni Noferini The Fréchet derivative of a generalized matrix function 20 / 33
-
Notation again...Given the singular values of A ∈ Cm×n we introduce two operatorsF ,G ∈ Rm×n:
Fij =
σi f (σi )−σj f (σj )σ2i −σ
2j
if i 6= j , max(i , j) ≤ ν, and σi 6= σj ;σi f ′(σi )+f (σi )
2σiif i 6= j , max(i , j) ≤ ν, and σi = σj 6= 0;
f (σj )σj
if i > n, and σj 6= 0;f (σi )σi
if j > m, and σi 6= 0;f ′(σi ) otherwise;
Gij =
σj f (σi )−σi f (σj )
σ2i −σ2j
if i 6= j , i , j ≤ ν, and σi 6= σj ;σi f ′(σi )−f (σi )
2σiif i 6= j , i , j ≤ ν, and σi = σj 6= 0;
0 otherwise.
Vanni Noferini The Fréchet derivative of a generalized matrix function 20 / 33
-
Notation again...Given the singular values of A ∈ Cm×n we introduce two operatorsF ,G ∈ Rm×n:
Fij =
σi f (σi )−σj f (σj )σ2i −σ
2j
if i 6= j , max(i , j) ≤ ν, and σi 6= σj ;σi f ′(σi )+f (σi )
2σiif i 6= j , max(i , j) ≤ ν, and σi = σj 6= 0;
f (σj )σj
if i > n, and σj 6= 0;f (σi )σi
if j > m, and σi 6= 0;f ′(σi ) otherwise;
Gij =
σj f (σi )−σi f (σj )
σ2i −σ2j
if i 6= j , i , j ≤ ν, and σi 6= σj ;σi f ′(σi )−f (σi )
2σiif i 6= j , i , j ≤ ν, and σi = σj 6= 0;
0 otherwise.
Vanni Noferini The Fréchet derivative of a generalized matrix function 20 / 33
-
I am lost! Help!
Forgetting about the degenerate cases...
F is f ′(σi ) on the main diagonal, and it is
σi f (σi )− σj f (σj)σ2i − σ2j
off the main diagonal.
G is 0 on the main diagonal, and it is
σj f (σi )− σi f (σj)σ2i − σ2j
off the main diagonal.
Vanni Noferini The Fréchet derivative of a generalized matrix function 21 / 33
-
I am lost! Help!
Forgetting about the degenerate cases...
F is f ′(σi ) on the main diagonal, and it is
σi f (σi )− σj f (σj)σ2i − σ2j
off the main diagonal.
G is 0 on the main diagonal, and it is
σj f (σi )− σi f (σj)σ2i − σ2j
off the main diagonal.
Vanni Noferini The Fréchet derivative of a generalized matrix function 21 / 33
-
I am lost! Help!
Forgetting about the degenerate cases...
F is f ′(σi ) on the main diagonal, and it is
σi f (σi )− σj f (σj)σ2i − σ2j
off the main diagonal.
G is 0 on the main diagonal, and it is
σj f (σi )− σi f (σj)σ2i − σ2j
off the main diagonal.
Vanni Noferini The Fréchet derivative of a generalized matrix function 21 / 33
-
Dalecǩii-Krěin formula for gmf
TheoremGiven A = USV ∗, under the assumptions of the previous theorem,
Lf �(A,E ) = U(F ◦ Ê + iH ◦ Im(Ê ) + G ◦ Υ (Ê )
)V ∗,
where Ê = U∗EV , F ,G are as in the previous slides, and H is diagonalwith Hii = f (σi )/σi − Fii if σi 6= 0 or Hii = f ′(0)− Fii otherwise.
Vanni Noferini The Fréchet derivative of a generalized matrix function 22 / 33
-
Dalecǩii-Krěin formula for real gmf
U,V ,E are real, so that the formula simplifies a little.
TheoremGiven A = USV T , under the assumptions of the previous theorem,
Lf �(A,E ) = U(F ◦ Ê + G ◦ Υ (Ê )
)V T ,
where Ê = UTEV , F ,G are as a few slides ago.
Vanni Noferini The Fréchet derivative of a generalized matrix function 23 / 33
-
Special cases
For special f , formuale may simplify. For example, f = 1 ⇒ unitary factorof a polar decomposition.
(f (0) 6= 0, hence differentiability ⇔ A has full rank).
The theorem then leads to efficient algorithms for the computation of theFréchet derivative, see Arslan, Noferini and Tisseur (in preparation).
Vanni Noferini The Fréchet derivative of a generalized matrix function 24 / 33
-
Example
Toy matrices:
A =
2 0 00 2 00 0 1
,E =1 −1 00 0 01 1 1
.
for f (σ) = σ2,
F =
4 3 7/33 4 7/37/3 7/3 2
,G = 0 1 2/31 0 2/32/3 2/3 0
,
Lf (A,E ) =
4 −3 2/3−1 0 2/37/3 7/3 2
.
Vanni Noferini The Fréchet derivative of a generalized matrix function 25 / 33
-
Example
Toy matrices:
A =
2 0 00 2 00 0 1
,E =1 −1 00 0 01 1 1
.for f (σ) = σ2,
F =
4 3 7/33 4 7/37/3 7/3 2
,G = 0 1 2/31 0 2/32/3 2/3 0
,
Lf (A,E ) =
4 −3 2/3−1 0 2/37/3 7/3 2
.
Vanni Noferini The Fréchet derivative of a generalized matrix function 25 / 33
-
Example
Toy matrices:
A =
2 0 00 2 00 0 1
,E =1 −1 00 0 01 1 1
.
for f (σ) = 1 (polar decomposition),
F =
0 1/4 1/31/4 0 1/31/3 1/3 0
= −G , Lf (A,E ) = 0 −1/4 −1/31/4 0 −1/31/3 1/3 0
.
Vanni Noferini The Fréchet derivative of a generalized matrix function 26 / 33
-
Example
Toy matrices:
A =
2 0 00 2 00 0 1
,E =1 −1 00 0 01 1 1
.for f (σ) = 1 (polar decomposition),
F =
0 1/4 1/31/4 0 1/31/3 1/3 0
= −G , Lf (A,E ) = 0 −1/4 −1/31/4 0 −1/31/3 1/3 0
.
Vanni Noferini The Fréchet derivative of a generalized matrix function 26 / 33
-
Application to conditioning
TheoremAssuming m ≥ n and A ∈ Rm×n,
condabs(f �,A) ≤ max{maxi|Fii |,max
j
-
Application to conditioning
TheoremAssuming m ≥ n and A ∈ Rm×n,
condabs(f �,A) ≤ max{maxi|Fii |,max
j
-
Some more bounds
In practice, only a few singular values may be known:
TheoremIf A ∈ Rm×n has full rank and smallest singular value σr , suppose that M isthe sup norm of f on [σr , ‖A‖2] and that f is Lipschitz continuos on thesame interval with constant K. Then
condabs(f �,A) ≤ max{K ,Mσ−1r }.
TheoremIf A ∈ Rm×n and f (0) = 0, suppose that f is Lipschitz continuos on on[0, ‖A‖2] with constant K. Then
condabs(f �,A) ≤ K .
Vanni Noferini The Fréchet derivative of a generalized matrix function 28 / 33
-
Some more bounds
In practice, only a few singular values may be known:
TheoremIf A ∈ Rm×n has full rank and smallest singular value σr , suppose that M isthe sup norm of f on [σr , ‖A‖2] and that f is Lipschitz continuos on thesame interval with constant K. Then
condabs(f �,A) ≤ max{K ,Mσ−1r }.
TheoremIf A ∈ Rm×n and f (0) = 0, suppose that f is Lipschitz continuos on on[0, ‖A‖2] with constant K. Then
condabs(f �,A) ≤ K .
Vanni Noferini The Fréchet derivative of a generalized matrix function 28 / 33
-
Some more bounds
Again, for special choices of f more can be said.
The following is known but can be recovered as a special case:
Theorem (Kenney and Laub, 1991)
If A ∈ Rm×n has full rank r , then for f (x) = 1
condabs(f �,A) =2
σr + σr−1.
TheoremIf A ∈ Rm×n has full rank r , then for f (x) = ex , defining g(x) = ex/x:
condabs(f �,A) = max{g(σr ), g(‖A‖2), e‖A‖2}.
Vanni Noferini The Fréchet derivative of a generalized matrix function 29 / 33
-
Some more bounds
Again, for special choices of f more can be said.The following is known but can be recovered as a special case:
Theorem (Kenney and Laub, 1991)
If A ∈ Rm×n has full rank r , then for f (x) = 1
condabs(f �,A) =2
σr + σr−1.
TheoremIf A ∈ Rm×n has full rank r , then for f (x) = ex , defining g(x) = ex/x:
condabs(f �,A) = max{g(σr ), g(‖A‖2), e‖A‖2}.
Vanni Noferini The Fréchet derivative of a generalized matrix function 29 / 33
-
Some more bounds
Again, for special choices of f more can be said.The following is known but can be recovered as a special case:
Theorem (Kenney and Laub, 1991)
If A ∈ Rm×n has full rank r , then for f (x) = 1
condabs(f �,A) =2
σr + σr−1.
TheoremIf A ∈ Rm×n has full rank r , then for f (x) = ex , defining g(x) = ex/x:
condabs(f �,A) = max{g(σr ), g(‖A‖2), e‖A‖2}.
Vanni Noferini The Fréchet derivative of a generalized matrix function 29 / 33
-
Relative condition number
..it is the relative condition number that is of interest, but it ismore convenient to state results for the absolute conditionnumber.
Vanni Noferini The Fréchet derivative of a generalized matrix function 30 / 33
-
Relative condition number
..it is the relative condition number that is of interest, but it ismore convenient to state results for the absolute conditionnumber.
Vanni Noferini The Fréchet derivative of a generalized matrix function 30 / 33
-
Some more bounds
Define µ =√
f (‖A‖2)2 + f (σr )2.
TheoremIf A ∈ Rm×n has full rank and smallest singular value σr , suppose that M isthe sup norm of f on [σr , ‖A‖2] and that f is Lipschitz continuos on thesame interval with constant K. Then
condrel(f �,A) ≤√
max(m, n)‖A‖2 max{K/µ, σ−1r }.
TheoremIf A ∈ Rm×n and f (0) = 0, suppose that f is Lipschitz continuos on on[0, ‖A‖2] with constant K. Then
condrel(f �,A) ≤√
max(m, n)K‖A‖2µ−1.
Vanni Noferini The Fréchet derivative of a generalized matrix function 31 / 33
-
Some more bounds
Define µ =√
f (‖A‖2)2 + f (σr )2.
TheoremIf A ∈ Rm×n has full rank and smallest singular value σr , suppose that M isthe sup norm of f on [σr , ‖A‖2] and that f is Lipschitz continuos on thesame interval with constant K. Then
condrel(f �,A) ≤√
max(m, n)‖A‖2 max{K/µ, σ−1r }.
TheoremIf A ∈ Rm×n and f (0) = 0, suppose that f is Lipschitz continuos on on[0, ‖A‖2] with constant K. Then
condrel(f �,A) ≤√
max(m, n)K‖A‖2µ−1.
Vanni Noferini The Fréchet derivative of a generalized matrix function 31 / 33
-
The big picture
If f (0) = 0, then f � has essentially the same condition number as thescalar function f . Unlike classical mf, gmf are never numericallydodgier than the scalar case.
If f (0) 6= 0, trouble may happen only if
maxi|f (σi )/σi | � max
i|f ′(σi )|.
Vanni Noferini The Fréchet derivative of a generalized matrix function 32 / 33
-
The big picture
If f (0) = 0, then f � has essentially the same condition number as thescalar function f . Unlike classical mf, gmf are never numericallydodgier than the scalar case.If f (0) 6= 0, trouble may happen only if
maxi|f (σi )/σi | � max
i|f ′(σi )|.
Vanni Noferini The Fréchet derivative of a generalized matrix function 32 / 33
-
Can trouble happen?If f (0) 6= 0, f � is discontinuos at a rank deficient A. Intuitively, we expectnumerical issues for an ill conditioned A.Let
A =[� 0 00 1 0
], f (x) = 1+ (x − �)2.
Then,condrel(f , �) = 0, condrel(f , 1) = 1+ O(�2).
Yet,
condrel(f �,A) =1�√5+ O(1).
10-8
10-7
10-6
10-5
10-4
10-3
ǫ
10-10
10-8
10-6
10-4
ρ
Vanni Noferini The Fréchet derivative of a generalized matrix function 33 / 33
-
Can trouble happen?If f (0) 6= 0, f � is discontinuos at a rank deficient A. Intuitively, we expectnumerical issues for an ill conditioned A.Let
A =[� 0 00 1 0
], f (x) = 1+ (x − �)2.
Then,condrel(f , �) = 0, condrel(f , 1) = 1+ O(�2).
Yet,
condrel(f �,A) =1�√5+ O(1).
10-8
10-7
10-6
10-5
10-4
10-3
ǫ
10-10
10-8
10-6
10-4
ρ
Vanni Noferini The Fréchet derivative of a generalized matrix function 33 / 33