an improved newton iteration for the weighted moore–penrose inverse

Applied Mathematics and Computation 174 (2006) 1460–1486

www.elsevier.com/locate/amc

An improved Newton iteration forthe weighted Moore–Penrose inverse q

Feng Huang a, Xian Zhang b,*

a Department of Mathematics, Fudan University, Shanghai 200433, People’s Republic of Chinab Institute of Mathematics, Fudan University, Shanghai 200433, People’s Republic of China

Abstract

In this paper, we showed that Newton iteration may be used to compute the weightedMoore–Penrose inverse of an arbitrary matrix. We also take advantage of several newacceleration procedures to reduce the cost of Newton iteration, which extends the earlierwork [R. Schreiber, Computing generalized inverses and eigenvalues of symmetricmatrices using systolic arrays, in: R. Glowinski, J.L. Lious (Eds.), Computing Methodsin Applied Science and Engineering, North-Holland, Amsterdam, 1984; T. Soderstorm,G.W. Stewant, On the numerical properties of an iterative method for computing theMoore–Penrose generalized inverse, SIAM J. Numer. Anal. 11 (1974) 61–74].� 2005 Elsevier Inc. All rights reserved.

Keywords: Newton iteration; Weighted Moore–Penrose inverse; Weighted singular value decom-position

0096-3003/$ - see front matter � 2005 Elsevier Inc. All rights reserved.doi:10.1016/j.amc.2005.05.050

q Project 10471027 supported by the National Natural Science Foundation of China andShanghai Education Committee.* Corresponding author.E-mail addresses: [email protected] (F. Huang), [email protected], 0018123@

fudan.edu.cn (X. Zhang).

mailto:[email protected]

mailto:[email protected]

mailto:0018123@ fudan.edu.cn

mailto:0018123@ fudan.edu.cn

F. Huang, X. Zhang / Appl. Math. Comput. 174 (2006) 1460–1486 1461

1. Introduction and preliminaries

The weighted Moore–Penrose inverse of an arbitrary matrix (including sin-gular and rectangular) has many applications in statistics, prediction theory,control system analysis, curve fitting and numerical analysis [3,4,6,8–22]. Foran arbitrary matrix A 2 Cm�n, M, N are Hermitian positive definite matricesof order m and n, respectively, there is a unique matrix X satisfying the rela-tions [1,5,11]

AXA ¼ A; XAX ¼ X ; ðMAX Þ� ¼ MAX ; ðNXAÞ� ¼ NXA. ð1:1ÞX is commonly known as the weighted Moore–Penrose inverse of A, denotedby Aþ

MN . In particular, when M = Im and N = In, the matrix X satisfying (1.1)is called the Moore–Penrose inverse and is denoted by X = A+.

For x ¼ AþMNb; x

0 2 Cn n fxg arbitrary, it holds

kb� Axk2M ¼ ðb� AxÞ�Mðb� AxÞ 6 kb� Ax0k2M ;and

kb� AxkM ¼ kb� Ax0kM ) kxk2N ¼ x�Nx < kx0k2N .Thus, x ¼ Aþ

MN is the unique minimum N-norm M-least squares solution of aweighted linear squares problem [1,11,19]

kb� AxkM ¼ minz2Cn

kb� AzkM .

The weighted Moore–Penrose inverse x ¼ AþMN can be explicitly expressed from

the weighted generalized singular value decomposition.

Lemma 1.1 [10]. Let A 2 Cm�n and rank(A) = r. There exist U 2 Cm�m,V 2 Cn�n satisfying U*MU = Im and V*N�1V = In such that

A ¼ UD 0

0 0

� �V �. ð1:2Þ

Then, the weighted Moore–Penrose inverse AþMN can be represented as

AþMN ¼ N�1V

D�1 0

0 0

!U �M ; ð1:3Þ

where D = diag(r1,r2, . . . ,rr), r1 P r2 P� � �P rr > 0 and r2i is the nonzero

eigenvalue of N�1A*MA. Furthermore

kAkMN ¼ r1; kAþMNkNM ¼ 1=rr.

We denote A# = N�1A*M as the weighted conjugate transpose matrix of A.Let R(A), N(A), r(A), and q(A) denote the range, the null space, the spectrum

and the spectral radius of A, respectively. The notation k Æ k stands for the spec-

trum norm.

1462 F. Huang, X. Zhang / Appl. Math. Comput. 174 (2006) 1460–1486

2. The Classical Newton Iteration

In this section, we will establish the theory of solving the weighted Moore–Penrose generalized inverse of an arbitrary matrix.

Theorem 2.1. We shall assume that A is an m · n matrix whose weightedsingular value decomposition is given by (1.3). Let 0 < a0 < 2=r21 and let

X 0 ¼ a0A#: ð2:1Þ

Define the sequence of matrices X1,X2, . . . , by

X kþ1 ¼ 2X k � X kAX k. ð2:2ÞThen the Xk converges to Aþ

MN .

Proof. In view of (1.3), to establish this result, we only need to verify that

limk!1

ðV �1NÞX kðM�1ðU �Þ�1Þ ¼ D�1 0

0 0

!. ð2:3Þ

First, we shall show that

ðV �1NÞX kðM�1ðU �Þ�1Þ ¼T k 0

0 0

� �; ð2:4Þ

where the Tk are diagonal,

T 0 ¼ a0D

and

T kþ1 ¼ 2T k � DT 2k . ð2:5Þ

In fact, from (1.2)

ðV �1NÞX 0ðM�1ðU �Þ�1Þ ¼ a0ðV �1NÞA#ðM�1ðU �Þ�1Þ¼ a0ðV �1NÞN�1A�ðMM�1ðU �Þ�1Þ

¼ a0ðV �1NÞN�1VD 0

0 0

� �U �ðMM�1ðU �Þ�1Þ

¼a0D 0

0 0

� �.

Moreover if (2.4) is satisfied, then by (2.2)

ðV �1NÞX kþ1ðM�1ðU �Þ�1Þ ¼ 2ðV �1NÞX kðM�1ðU �Þ�1Þ� ðV �1NÞXkðM�1ðU �Þ�1ÞAðV �1NÞX kðM�1ðU �Þ�1Þ


¼ 2ðV �1NÞX kðM�1ðU �Þ�1Þ � ðV �1NÞX kðM�1ðU �Þ�1ÞU �MU

�D 0

0 0

� �V �NV ðV �1NÞX kðM�1ðU �Þ�1Þ ¼ 2T k � DT 2

k 0

0 0

!;

which establishes (2.5).It follows that T k ¼ diagðsðkÞ1 ; sðkÞ2 ; . . . ; sðkÞr Þ, where

sð0Þi ¼ a0ri ð2:6Þ

and

sðkþ1Þi ¼ sðkÞi ð2� ris

ðkÞi Þ. ð2:7Þ

Now, the sequence generated by (2.6) is the result for applying Newton�smethod to find the zero r�1

i of the function

/ðsÞ ¼ ri � s�1;

starting with sð0Þi . It is easily seen that this iteration converges to r�1i provided

0 < sð0Þi < 2=ri;

which is assured by the condition on a0. Thus, Tk ! R�1, Eq. (2.3) is satisfied,and X k ! Aþ

MN .In fact,

X k ¼ N�1VT k 0

0 0

� �U �MU

D 0

0 0

� �V �.

From

U �MU ¼ Im; V �N�1V ¼ In

then we define

P�1 ¼ N�1V ; P ¼ V �.

Then

X kA ¼ P�1T kD 0

0 0

� �P ¼ P�1

Rk 0

0 0

� �P ; ð2:8Þ

where the

Rk ¼ T kD ¼ diagðqðkÞ1 ;qðkÞ

2 ; . . . ; qðkÞr Þ

and

ðV �1NÞX kðM�1ðU �Þ�1Þ ¼T k 0

0 0

� �;


ðV �1NÞX kþ1ðM�1ðU �Þ�1Þ ¼ 2T k � DT 2k 0

0 0

!.

Then

X kþ1A ¼ N�1V2T k � DT 2

k 0

0 0

!U �MU

D 0

0 0

� �V �

¼ P�1 ð2T k � DT 2kÞD 0

0 0

!P

denote

Rkþ1 ¼ ð2T k � DT 2kÞD ¼ diag qðkþ1Þ

1 ; qðkþ1Þ2 ; . . . ; qðkþ1Þ

r

� �;

for

Rk ¼ T kD;

then

Rkþ1 ¼ ð2T k � DT 2kÞD ¼ 2T kD� D2T 2

k

¼ 2T kD� ðD2T kÞ2 � 1þ 1 ¼ �ð1� T kDÞ2 þ 1

that is for all j, 1 6 j 6 r

1� qðkþ1Þj ¼ ð1� qðkÞ

j Þ2. ð2:9Þ

Clearly, q0j ¼ a0r2

j , from Theorem 2.1, for any a0 < 2=r21 (2.9), from (2.9), for

all j, 1 6 j 6 r, Newton iteration implies the quadratic convergence of qðkÞj , and,

therefore, of X k ! AþMN . The optimum choice of a0 (2.1), which minimizes

kI � X0AkNN by making qð0Þ1 � 1 ¼ 1� qð0Þ

r , is

a0 ¼2

r21 þ r2

r

. ð2:10Þ

In fact

kI � X 0AkNN ¼ kN 1=2ðI � X 0AÞN�1=2k2¼ kI � N 1=2X 0AN

�1=2k2

¼ I � N 1=2N�1VT 0D 0

0 0

� �V �N�1=2

�� 2

¼ I � N�1=2VT 0D 0

0 0

� �V �N�1=2

�� 2

¼ I �T 0D 0

0 0

� �� 2

¼ I � a0D2 0

0 0

� �� 2

;


where N�1/2V and V*N�1/2 are unitary matrix. Since min kI � X 0kNN ¼qð0Þ1 � 1, and qð0Þ

1 P qð0Þr , therefore,

qð0Þ1 � 1 ¼ 1� qð0Þ

r () a0r21 � 1 ¼ 1� a0r

2r .

Thus, (2.10) is established.Let jðAÞ ¼ kAkMN � kAþ

MNkNM ¼ r1=rr, then with the choice of (2.10),

qð0Þr ¼ 2

1þ jðAÞ2;

so that qð0Þr � 1, if j(A) is large. In practice, information about rr is hard to

get, we may instead use a suboptimal but nevertheless safe alternative, such as

a0 ¼1

kAk1 � kAk1. �

3. Convergence acceleration by scaling

In this section, we will present the scaled iteration, given by

X kþ1 ¼ akþ1ð2I � X kAÞX k; k ¼ 0; 1; . . . ; ð3:1Þwhere X0 is given by (2.2). Here, we employ an acceleration parameterak+1 2 [1, 2] chosen so as to minimize that which is bound on the maximum dis-tance of any nonzero weighted singular value of Xk+1 A from 1. Let us assumethat we know bounds rmin and rmax on the singular values of A satisfying

rmin 6 r2r 6 r2

1 6 rmax.

In this case, every eigenvalues of A#A lies in the intervals [rmin,rmax]. (Weassume that this interval has nonzero length, otherwise, A is a scalar multipleof an unitary matrix and X0 is its inverse.) We chose X0 according to (2.1) with

a0 ¼2

rmin þ rmax

. ð3:2Þ

It follows from (1.2), (2.2), (3.1) and (3.2) that for all j and k:

qð0Þj ¼

2r2j

rmin þ rmax

and

qðkþ1Þj ¼ akþ1ð2� qðkÞ

j ÞqðkÞj :

To determine the acceleration parameters ak, for k > 0, we let

qð0Þ ¼ a0rmin ¼2rmin

rmin þ rmax

ð3:3Þ


and

�qð0Þ ¼ a0rmax ¼2rmax

rmin þ rmax

; ð3:4Þ

these being lower and upper bounds on the weighted singular values of X0A.Then take

akþ1 ¼ �qðkþ1Þ ¼ 2

1þ ð2� qðkÞÞqðkÞ ; ð3:5Þ

which is both an acceleration parameter and an upper bound on fqðkþ1Þj g, and

qðkþ1Þ ¼ akþ1ð2� qðkÞÞqðkÞ; ð3:6Þ

which is a lower bound on fqðkþ1Þj g. The definitions (3.3) and (3.4) imply that

for all 1 6 j 6 r, and kP 1,

qðkÞ6 qðkÞ

j 6 �qðkÞ;

and

qðkÞ ¼ 2� �qðkÞ. ð3:7Þ

Note that (3.7) follows immediately from (3.5) and (3.6). the upper boundqðkÞj 6 �qðkÞ is likewise straightforward. Finally, if

0 6 qðkÞ6 qðkÞ

j 6 ð2� qðkÞÞ;

then

ð2� qðkÞÞqðkÞ6 ð2� qðkÞ

j ÞqðkÞj 6 1;

whence, by (3.6) and the definition of qðkÞj , the lower bound qðkÞ

6 qðkÞj follows.

Except for the last few iterations before convergence, q(k) � 1, which impliesthat ak+1 � 2, Thus, qðkþ1Þ

j � 4qðkÞj . Therefore,

�log4qð0Þr � log2½ðjðAÞ

2 þ 1Þ1=2� ¼ log2jðAÞ þ Oð1=jðAÞ2Þ

steps suffice to bring all the singular values of Xk A up to 12, which is half as

many as for the unaccelerated version.

4. Polynomial approximation

We shall now derive a theorem concerning the optimality of the accelerationparameters given by (3.5). Let the symmetric residual matrix initially be

E � I � a0A#A ¼ I � X 0A. ð4:1Þ


It is straightforward to show that Newton�s method, starting with X0 =a0A

#, produces iterates Xk satisfying

X kA ¼ I � Em; ð4:2Þwhere m = 2k. For a nonsingular matrix A and for the choice (3.2) for a0, theeigenvalues of E lie in the open interval (�1,1) and we have that Em ! 0 and,therefore, Xk A! I as k! 1. In fact, the claim (4.2) is clearly when k = 1,2,with

X 1A ¼ ð2X 0 � X 0AX 0ÞA ¼ 2X 0A� ðX 0AÞ2

¼ 2ðI � EÞ � ðI � EÞ2 ¼ I � E2;

X 2A ¼ ð2X 1 � X 1AX 1ÞA ¼ 2X 1A� ðX 1AÞ2

¼ 2ðI � E2Þ � ðI � E2Þ2 ¼ ½2I � ðI � EÞ2�ðI � E2Þ

¼ ðI þ E2ÞðI � E2Þ ¼ I � E4.

Now use induction on k. It is straightforward to establish (4.2).Moreover note that when k = 1,2,

X 1 ¼ 2X 0 � X 0AX 0 ¼ ð2I � X 0AÞX 0

¼ ½2I � ðI � EÞ�X 0 ¼ ðI þ EÞX 0;

X 2 ¼ 2X 1 � X 1AX 1 ¼ ð2I � X 1AÞX 1

¼ ½2I � ðI � E2Þ�ðI þ EÞX 0 ¼ ðI þ E2ÞðI þ EÞX 0

¼ ðI þ E þ E2 þ E3ÞX 0.

Induct on k

X k ¼ 2X k�1 � X k�1AX k�1 ¼ X k�1ð2I � AX k�1Þ ¼ ð2I � X k�1AÞX k�1

¼ ½2I � ðI � E2k�1Þ�Xk�1 ¼ ðI þ E2k�1ÞX k�1 ¼ ðI þ E2k�1ÞðI þ E2k�2ÞX k�2

¼ � � � ¼ ðI þ E þ E2 þ � � � þ Em�1ÞX 0

¼ ðI þ E þ E2 þ � � � þ Em�1Þa0A#.

Thus, Newton�s method is related to the Neumann series expansion

ðI � EÞ�1 ¼ I þ E þ E2 þ � � �We therefore ask whether the accelerated method (2.1), (3.1)–(3.6) is related

to a better polynomial approximation to (I � E)�1. In fact, it is exactly equiv-alent to approximation of this inverse by a Tchebychev polynomial in E, as wenow show. Let

T 2k ðfÞ � cosð2kcos�1fÞ


be the Tchebychev polynomial of degree 2k on (�1,1). Recall that T0(f) = 1,T1(f) = f and

T 2kþ1ðfÞ � 2T 22kðfÞ � 1. ð4:3Þ

The scaled Tchebychev polynomials on (rmin,rmax) are defined by

tkðfÞ �T 2k ðcfþ dÞ

T 2k ðdÞ;

where c = 2/(rmax � rmin) and d = �(rmax + rmin)/(rmax � rmin). Surely, tk is apolynomial of degree 2k, and tk(0) = 1, so that

tkðfÞ ¼ 1� f�tkðfÞ;for some polynomial�tk of degree 2

k � 1. Furthermore, it is a classical result thatamong all such polynomials, tk, minimized the norm suprmin6f6rmax

jtkðfÞj �ktkk1;ðrmin ;rmaxÞ. They also satisfy a recurrence like (4.3). Let bk � T 2k ðdÞ. Thenby (4.3),

bktkðfÞ ¼ 2ðbk�1tk�1ðfÞÞ2 � 1. ð4:4Þ

Theorem 4.1. Let the sequence of matrices Xk, k = 0,1, . . . be generated by (2.1)

and (3.6). Then

X k ¼ �pkðA#AÞA#; ð4:5Þwhere �pkðfÞ is a polynomial of degree 2k � 1. The polynomial 1� f�pkðfÞ is the

scaled Tchebychev polynomial tk(f) of degree 2k on (rmin,rmax).

Remark. Before proving the theorem, we point out that (4.5) implies that

X kA ¼ �pkðA#AÞA#A

and therefore

kI � XkAkNN ¼ max16j6n

j1� r2j�pkðr2

j Þj;

which shows the relevance of the theorem’s conclusion.An analogue of this result also holds for X0 any matrix of the form

r(A#A)A#, with r a polynomial, such that the NN-norm of I � X0A is lessthan 1.

Proof. The claim (4.5) is clearly true when k = 0, with �p0ðfÞ � a0. With thechoice of (3.2), 1� f�p0ðfÞ ¼ 1� 2f=ðrmax þ rminÞ ¼ t0ðfÞ is the appropriatescaled Tchebychev polynomial.

Now use induction on k. A straightforward calculation using (3.5)–(3.7)shows that for all k P 1, 1 < ak < 2 and bk = 1/(ak � 1), or equivalently that


ak = (1 + bk)/bk. From (3.1) and (4.5), after multiplying by A, denote f = A#A,follows from the inductive hypothesis in the form of (4.3)–(4.5)

akð2I�Xk�1AÞXk�1A

¼ akð2I��pk�1ðA#AÞA#AÞ�pk�1ðA#AÞA#A

¼ akð2I��pk�1ðfÞfÞ�pk�1ðfÞf

¼ ak ½2f�pk�1ðfÞ�ðf�pk�1ðfÞÞ2�1þ1�

¼ ak ½�ð1� f�pk�1ðfÞÞ2þ1� due to ak ¼

1þbk

bk

� �¼ 1þbk

bk½�ð1� f�pk�1ðfÞÞ

2þ1� ðas bk � T 2k ðdÞ and (4.3);bk ¼ 2b2k�1�1Þ

¼ 2b2k�1

2b2k�1�1

½�ð1�f�pk�1ðfÞÞ2þ1�

¼�2b2k�1ð1� f�pk�1ðfÞÞ

2þ1�1þ2b2k�1

2b2k�1�1

.

For tkðfÞ ¼ 1� f�pk, and (4.4) then bkð1� f�pkÞ ¼ 2b2k�1ð1� f�pkÞ

2 � 1, then

akð2I � X k�1AÞX k�1A ¼ �bkð1� f�pkðfÞÞ þ 2b2k�1 � 1

2b2k�1 � 1

¼ �bkð1� f�pkðfÞÞ þ bk

bk¼ f�pkðfÞ;

that is,

X kA ¼ akð2I � X k�1AÞX k�1A ¼ akð2I � �pk�1ðfÞfÞ�pk�1ðfÞf ¼ f�pkðfÞ.Thus,

X kA ¼ �pkðA#AÞA#A ) X k ¼ �pkðA#AÞA#

is established, which proves the theorem. h

5. Convergence acceleration with cubic polynomials

In Section 2 we saw that with the unaccelerated Newton method, the conver-gence of qðkÞ

j to one is slow for all j such that qð0Þj lies near zero or two. For

many input matrices, and for moderately large k, the set fqðkÞj gnj¼0 produced

by Newton�s method without acceleration consists of two clusters, one lyingnear zero and the other near one. In this section, we present an alternativeto the acceleration method of Section 2 that, in the case of such a large gapin the spectrum of XkA, results in much faster convergence.


Let X be a fixed matrix satisfying XA = P�1RP, R = diag(q1,q2, . . . ,qn)where 0 6 qj 6 2 for all j. We seek an improved approximation X1 to Aþ

MN ofthe form

X 1 ¼ ðc3ðXAÞ2 þ c2XAþ c1IÞX .

We choose {c1,c2,c3} so that the cubic polynomial c(q) = c1q + c2q2 + c3q

3

satisfies

(i) c(1) = 1,

(ii) c 0(1) = 0,

(iii) c(0) = 0,

(iv) c 0(0) 2.

The idea here is that small singular values are amplified by the factor c 0(0) whilethose near 1 continue to converge. It is quite evident, however, that c(q) willtake large values for some q 2 (0,1), so we must exercise caution.

We begin by finding q > 0 such that we are certain that there are no eigen-values qj in (q, 1 � q) or in (1 + q,2]. Let T = XA and compute T2 andd � kT � T2kF. Now it follows from (2.8) that

d2 ¼Xnj¼1

½qjð1� qjÞ�2; ð5:1Þ

whence, for all 1 6 j 6 n,

qjj1� qjj 6 d.

If d P 14, this provides us with no useful information. If, on the contrary,

0 6 d 614, then we conclude that all the eigenvalues qj lies in the two closed

intervals: [0,q] and ½1� q; 1þ �q�, where

0 6 q � 1

2�

ffiffiffiffiffiffiffiffiffiffiffi1

4� d

r<

1

2;

1

2< 1� q ¼ 1

2þ


4þ d

r;

1 6 1þ �q ¼ 1

2þ 1

2þ


4þ d

r< 1þ q.

(See Fig. 1.) Thus, for j = 1, . . . ,n,

qj 2 ½0; q� [ ½1� q; 1þ �q�. ð5:2Þ

In other words, qj is in neither (q, 1 � q) nor ð1þ �q; 2�. Now, we show how tochoose c(q) so as to satisfy the criteria (i)–(iv), as well as the equally importantcriteria

0 0.2 0.4 0.6 0.8 1 1.2 1.4-0.25

-0.2

-0.15

-0.1

-0.05

0

0.05

0.1

0.15

0.2

0.25

δ

-δ

Fig. 1. The intervals [0,q] and ½1� q; 1þ �q� contain all the qj when d = kXA � (XA)2kF < 1/4.


(v) c : [0,q] ! [0,1],(vi) c : ½1� q; 1þ �qÞ ! ½1; 1þ �qÞ.

To determine c, we enforce (i)–(iii) together with

(vii) c(q) = 1.

The unique solution is

cðqÞ ¼ 1

qðq2 � ð2þ qÞqþ ð1þ 2qÞÞq;

which may be rewritten as

cðqÞ ¼ 1

qððq� 1Þ3 þ ð1� qÞðq� 1Þ2Þ þ 1.

Theorem 5.1. Let 0 6 q 612. If c(q) is the unique cubic satisfying (i)–(iii) and

(vii) above, then (v) and (vi) hold. Noting Fig. 2.

Proof. To show that (v) holds, we recall that a nonvanishing cubic polynomialc(q) has at most two critical points (where c 0(q) � 0). One of these is q = 1(condition (ii), by (i) and (vii), and Rolle�s theorem), the other one is in(q, 1), so c(q) is monotone on [0,q] and must, therefore, map [0,q] onto [0,1].

To establish (vi), note that

cðqÞ ¼ 1þ ðq� 1Þ2

qðq� qÞ.

0 0.2 0.4 0.6 0.8 1 1.2 1.40

0.2

0.4

0.6

0.8

1

1.2

1.4

Fig. 2. The figure of c(q), where q = 2.1443e � 001.


Now let q = 1 � e with jej 6 q < 12. Then c(q)�1 = e2(1 � e � q)/q, and since

the second factor is bounded by 0 and 1, the right-handed sided is boundedby 0 and q, establishing (vi). h

Thus, if d P 14we cannot accelerate. In this case, we let X1 = (2I � T)X and

proceed to the next iteration (i.e., X1 is the result of a Newton step (2.2)). On

the other hand, if d < 14we compute q :¼ 1

2�

ffiffiffiffiffiffiffiffiffiffi14� d

qand let X1 = (1/q)(T2 �

(2 + q)T + (1 + 2q)I)X (i.e., X1 is the result of a cubic step).

6. Suppressing the smaller singular values

In this section, given a matrix A and a positive scalar e, we show how tocompute A(e) and Aþ

MN ðeÞ, where A(e) is obtained from A by suppressing (thatis, setting to zero) all the singular values of A that do not exceed e. (A(e) is theclosest approximation to A by matrices whose rank is that of A(e). In practice,solving least squares often requires the use of this form of regulation; when A isvery badly conditioned, its generalized inverse is largely unknown due to per-turbations in A, but the reduced-rank generalized inverses are much betterconditioned.)

In order to compute these rank-reduced generalized inverses, we require apolynomial c(q) that gives fast convergence to 0 near q = 0 and to 1 nearq = 1. Consider first the iteration [7]


X kþ1=2 ¼ ð2I � X kAÞXk; Xkþ1 ¼ X kþ1=2AX kþ1=2. ð6:1Þ

The eigenvalues of XkA satisfy the equations qðkþ1Þj ¼ ðð2� qðkÞ

j ÞqðkÞj Þ2 for all

j and k. Fig. 3 illustrates the effect of this mapping. The fixed points are q0 ¼ 0,q1 ¼ ð3�

ffiffiffi5

pÞ=2 ¼ 0.3819 . . ., q2 ¼ 1, and q3 ¼ ð3þ

ffiffiffi5

pÞ=2 ¼ 2.618 . . ., which

are the roots of the quartic ð2� qÞ2q2 ¼ q. Consider especially the intervalsfq : 0 < q < q1g and fq : q1 < q < 2� q1g (note that 2� q1 ¼ ð1þ

ffiffiffi5

pÞ=2 ¼

1.618 . . .). Evidently, the eigenvalues from the former interval are sent towardzero and from the latter interval toward one. The convergence is ultimatelyquadratic but is slow near q1 and 2� q1. To compute A+(e), it suffices to applythe iteration (3.1)–(3.6) with appropriate rmin and rmax until the following rela-tions hold:

0 6 qðkÞj < q1 if and only if rj < e; ð6:2Þ

q1 < qðkÞj < 2� q1 otherwise. ð6:3Þ

We satisfy (6.2) for k = 0 by setting a0 ¼ q1=e2, but then (6.3) may not hold.

Therefore, we proceed as follows:

(1) Compute an upper bound rmax on r21.

(2) Set a0 ¼ minf2=ðrmax þ e2Þ; q1=e2g. This ensures that the eigenvalues qð0Þ

j

(of X0A) corresponding to the singular values rj P e (of A) are all closerto 1 than the smaller eigenvalues. Set �qð0Þ ¼ a0rmax; qð0Þ ¼ a0e2.

0 0.5 1 1.5 2 2.5 30

0.5

1

1.5

2

2.5

3

Fig. 3. The mapping (2 � q)2q2.


(3) Apply the iteration (3.1) with parameters given by (3.5) and (3.6) untilqðkÞ P q1.

(4) Set X k :¼ ðq1=qðkÞÞXk.

(5) Apply the iteration (6.1) until the matrix AþMN ðeÞ has been computed with

the desired accuracy.

The scaling at Step (4) is done to ensure that all the small singular values(less than e) are in fact suppressed.

The iteration (6.1) associated with the quartic polynomial (q(2 � q))2 is notthe more efficient way of computing Aþ

MN ðeÞ. The same object can be achievedby using iteration

X kþ1 ¼ ð�2X kAþ 3IÞX kAX k ð6:4Þassociated with the cubic polynomial ~cðqÞ ¼ �2q3 þ 3q2 (see Fig. 4). Note that~cð1Þ ¼ 1;~cð0Þ ¼ ~c0ð0Þ ¼ 0;~cðÞð1

2Þ ¼ 1

2, so that the mapping ~cðqÞ has three non-

negative fixed points 0, 12, and 1. Let q4 ¼ ð1þ

ffiffiffi3

pÞ=2 ¼ 1.366 . . . be the unique

solution to �2q34 þ 3q2

4 ¼ 12greater than 1. The eigenvalues of XkA in the inter-

val fq : 0 6 q < 12g are sent toward zero, and the eigenvalues in the interval

fq : 12< q 6 q4g are sent toward 1; the convergence to 0 and 1 is ultimately

quadratic but is slow near 12and q 6 q4.

The iteration (6.4) is simpler than (6.1): it requires three matrix multiplica-tions, one less than is needed in (6.1).

Remark 6.1. We may also compute A(e) itself since

AðeÞ ¼ AAþMN ðeÞA. ð6:5Þ

0 0.5 1 1.50

0.2

0.4

0.6

0.8

1

1.2

1.4

Fig. 4. The mapping �2q3 + 3q2.


Remark 6.2. We may use either of the methods (6.1) or (6.4) to ‘‘split’’ amatrix A into two better-conditioned matrices A(e) and �AðeÞ such that

A ¼ AðeÞ þ �AðeÞ. ð6:6ÞMoreover, if e is placed where there is a large gap in the singular values of Athen faster convergence is possible. For Aþ

MN may be computed by the formula

AþMN ¼ Aþ

MN ðeÞ þ �AþMN ðeÞ.

The matrices A(e) and �AðeÞ may be much better conditioned than A.

7. Stability of the basic and modified iterations

It is well known that the Newton iteration (2.1) is numerically stable andeven self-correcting if the input matrix A is nonsingular. If A is singular, how-ever, then it is very mildly unstable.

Let A and AþMN have the following SVDs:

A ¼ UD 0

0 0

� �V �; Aþ

MN ¼ N�1VD�1 0

0 0

!U �M :

In order to analyze the propagation of errors by (2.1), we assume that

X k ¼ AþMN þ bE ¼ N�1V

D�1 þ E11 E12

E21 E22

!U �M ; ð7:1Þ

where bE ¼ N�1VEU �M is the current error in Xk. We shall consider whether ornot the Newton iteration amplifies these errors. Throughout this section weshall drop all terms of second order in E.

Using (7.1) it is simple to compute that

X kþ1 ¼ X k þ ðI � X kAÞXk ¼ N�1VD�1 E12

E21 2E22

!U �M .

Due to the block 2E22, the iteration (2.1) is mildly unstable if A is singular (inwhich case the (2,2) block is not empty). After 2 log 2jðAÞ iterations, roundingerrors of order j2(A) can accumulate.

On the other hand, the iteration (6.1) is stable even for singular A. Indeed,by (6.1) and (7.1)

X kþ1=2 ¼ N�1VD�1 E12

E21 2E22

!U �M ;

X kþ1=2A ¼ N�1VI 0

E21D 0

� �V �.


Therefore

X kþ1 ¼ X kþ1=2AX kþ1=2 ¼ N�1VD�1 E12

E21 0

!U �M .

Thus, we deduce that the iteration (6.1) is stable for any matrix A. Similarly, wededuce that the iteration (6.4) is stable for any matrix A. Thus, the methods ofSection 7 have the dual advantage of stability and a well-conditioned solution,in contrast to the use of Newton iteration (or its accelerated form) on A. (If wewish to compute Aþ

MN , then the instability of Newton�s methods can partlyremoved by using a few iterations (6.1) or (6.4) after all the significant singularvalues have converged.)

8. Computing the projection onto a subspace spanned by singular vectors

In this section we discuss a modification of the Newton iterations discussedabove that allows us to compute the oblique projection matrices onto sub-spaces spanned by the singular vectors corresponding to either the dominantor the smallest singular values at less expense than computing the generalizedinverse of A. Important applications to spectral estimation and direction find-ing with antenna arrays were developed.

Let us define the following matrices:

P ðeÞ ¼ AAþMN ðeÞ ¼ U

I 0

0 0

� �U �M ; ð8:1Þ

P �ðeÞ ¼ AþMN ðeÞA ¼ N�1V

I 0

0 0

� �V �; ð8:2Þ

where the matrices U and V are from (1.2), I is the r(e) · r(e) identity block, andr(e) is the number of the singular values of A are not lesser than e. Then P*(e)are the oblique projections onto the subspaces spanned by the first r(e) columnsof the matrices U and V, respectively. Our previous results already gave ussome iterative algorithms for computing AðeÞ ¼ ðAþ

MN ðeÞÞþNM ¼ AAþ

MN ðeÞA, aswell as P(e), but there are simpler and more efficient algorithms that we shallgive shortly. In addition to the signal processing application mentioned above,we may use this technique as an alternative to methods of Section 6 for com-puting A(e), since

AðeÞ ¼ PðeÞA ¼ AP �ðeÞ. ð8:3ÞThe following iteration extends (6.4) and converges to P(e), unless e is a singu-lar value of A. Let

P 0 ¼ a0AA# þ bI ; ð8:4Þ


where we choose a0 > 0, b P 0, to satisfy

a0e2 þ b ¼ 1

2; a0r

21 þ b < q4 ¼ 1.37 . . . ; ð8:5Þ

then iteration as follows:

Pkþ1 ¼ ð�2Pk þ 3IÞP 2k ¼ ðI � 2ðPk � IÞÞP 2

k ; k ¼ 0; 1; . . . ð8:6ÞThe iteration (8.4)–(8.6) converges to P*(e) if we replace AA# by A#A in

(8.4). Furthermore, we may compute A(e) by using (8.3), which is superior tothe solution given in Section 6 because we now need to compute a single gen-eralized inverse(rather than two). Also, each iteration step (8.6) only involvestwo matrix multiplication, and finally, if A is rectangular then one of these iter-ations is less expensive than (6.4) (for example) because it involves smaller sym-metric matrices.

Remark 8.1. The stability analysis of Section 7 can be immediately extendedto the iteration (8.6).

Remark 8.2. We may accelerate the cubic iteration for P(e) as follows. At theearly stages of the iteration, it is more important to move singular values awayfrom 0.5. After a step ~P ¼ 3P 2 � 2P 3, we have the spectrum of ~P that lies in theclosed interval [0, 1]. We then replace ~P by a~P þ bI where aq + b is a line thatmaps [0,1] into ð�ðq4 � 1Þ; q4Þ. To get the best possible speedup by thismethod, we choose such a line and also require that að1

2Þ þ b ¼ 1

2(see Fig. 5).

-0.5 0 0.5 1 1.5-0.5

0

0.5

1

1.5

Fig. 5. Acceleration by scaling and shifting, Pa + bI.


Remark 8.3. Our ability to compute the projectors P(e) allows us, with a littleadditional computation to do the following:

(1) The projector P(e) defines the rank of the matrices A(e) and AþMN ðeÞ, for

rankAðeÞ ¼ rankAþMN ðeÞ ¼ kP ðeÞkFMN ¼ traceðP ðeÞÞ.

This observation may be used as the basis of a bisection strategy for com-puting the singular values of A in polylog time. Indeed, the singular val-ues of A are those of A(e) together with those of A � A(e). We may in thisway reduce the problem to that computing the positive singular value of amatrix with only one positive singular value. This we discuss in point (4).It is straightforward to develop a similar polylog algorithm for the eigen-values of any symmetric matrix.

(2) We may compute projectors P(e1,e2) onto subspaces spanned by singularvectors belonging to all the singular values in [e1, e2), since P(e1, e2) =P(e1) � P(e2).

(3) We may determine easily, for a given vector x, whether or not x2S(e)where S(e) is the span of the singular vectors corresponding to singularvalues greater than or equal to e, by checking whether or not x = P(e)x.

(4) We may rapidly compute any singular value, regardless of multiplicity, aswell as we have found an interval [e1, e2] that contains this singular value,r, and no other. For r is the only singular value of A(e1, e2) = (P(e1) �P(e2))A. Its multiplicity k is given by trace(P(e1) � P(e2)).

9. Algorithm

Algorithm 1. Classical Newton Iteration

INPUT: A, M, NOUTPUT: Aþ

MN

STEP 0 [INITIALIZE]:
Choose X = a0A
#, with a0 given by (2.10);STEP 1 [NEWTON STEP]:

if X is sufficiently close to AþMN , then return(X);

X = X(2I � AX);goto STEP 1.

Algorithm 2. Accelerated Newton Iteration

INPUT: A, M, N

OUTPUT: AþMN


STEP 0 [INITIALIZE]:Choose X = a0A

#, with a0 given by (2.10);Choose qmin = a0rmin, from (3.3);

STEP 1 [NEWTON STEP]:if X is sufficiently close to Aþ

MN , then return(X);a = 2/(1 + (2 � qmin)qmin)qmin = a(2 � qmin)qmin;X = a(2I � AX)X;goto STEP 1.

Algorithm 3. Cube Polynomial Accelerated Newton Iteration

INPUT: A, M, N

OUTPUT: AþMN

STEP 0 [INITIALIZE]:
Choose X = a0A
#, with a0 given by (2.10);T := XA;T2VALID := false;

STEP 1 [NEWTON STEP]:if X is sufficiently close to Aþ

MN , then return(X);X = (2I � T)X;if T2VALID then

T := 2T � T2;else

T := XA;endifT2VALID := false;if traceðT Þ P n� 1

2then

goto STEP 1;STEP 2 [TEST FOR SMALL CHANGE]:

T2 = T2;d := kT � T2kF;if d P 1

4then

T2VALID := true;goto STEP 1;

else

goto STEP 3;STEP 3 [USE ACCELERATION]:

q :¼ 12�

ffiffiffiffiffiffiffiffiffiffi14� d

q;

X := (1/q)(T2 � (2 + q)T + (1 + 2q)I)X;T := XA;goto STEP 1.


10. Numerical examples

Example 1. We generated a random 100 · 80 matrix, and two random Hermi-tian matrices, M 2 C100�100 and N 2 C80�80. We used algorithms ClassicalNewton Iteration and the Accelerated Newton Iteration to compute Aþ

MN to

0 5 10 15 20 25-14

-12

-10

-8

-6

-4

-2

0

2

Fig. 6. The Classical Newton Iteration, log kXA� IkNN .

0 5 10 15-16

-14

-12

-10

-8

-6

-4

-2

0

2

Fig. 7. The Accelerated Newton Iteration, log kXA� IkNN .

Table 1The Classical Newton Iteration

Iteration kXAX � Xk kAXA � Ak k(MAX)* �MAXk k(NXA)* � NXAk1 5.9141e�002 7.0845e+000 2.1898e�016 3.6362e�0162 8.5685e�002 5.0297e+000 2.9436e�016 4.0949e�0163 9.2470e�002 4.7600e+000 4.7903e�016 5.8312e�0164 9.4197e�002 4.4549e+000 7.2380e�016 1.0121e�0155 1.1077e�001 4.0283e+000 1.2085e�015 1.3036e�0156 1.2043e�001 3.4596e+000 1.7435e�015 2.0553e�0157 1.3922e�001 2.8906e+000 2.4262e�015 2.9002e�0158 1.7111e�001 2.3708e+000 3.2353e�015 3.6742e�0159 2.1805e�001 1.9183e+000 3.9131e�015 5.3189e�01510 2.8669e�001 1.4660e+000 5.3034e�015 6.9820e�01511 3.6686e�001 1.0386e+000 6.7099e�015 8.5446e�01512 4.6062e�001 7.3889e�001 8.1423e�015 1.0267e�01413 5.6795e�001 5.5524e�001 9.9435e�015 1.1310e�01414 7.6151e�001 3.6478e�001 1.1243e�014 1.3100e�01415 7.5781e�001 1.8993e�001 1.2651e�014 1.5699e�01416 4.0089e�001 7.1945e�002 1.4801e�014 1.5397e�01417 6.7403e�002 1.0541e�002 1.6277e�014 2.1111e�01418 1.4809e�003 2.2669e�004 1.7927e�014 1.6741e�01419 6.8550e�007 1.0488e�007 1.9245e�014 1.7838e�01420 1.4597e�013 9.9119e�014 1.8765e�014 1.6513e�01421 3.7869e�015 5.6426e�014 1.9512e�014 1.6604e�01422 4.2302e�015 8.8838e�014 1.9128e�014 1.9815e�01423 3.7730e�015 7.7230e�014 2.0902e�014 1.6639e�014

Table 2The Acceleration Newton Iteration

Iteration kXAX � Xk kAXA � Ak k(MAX)* �MAXk k(NXA)* � NXAk1 9.2787e�002 3.3501e+001 3.6309e�016 8.7234e�0162 1.4023e�001 5.6672e+000 9.5945e�016 7.9276e�0163 6.5687e�001 4.3741e+001 2.4065e�015 2.5791e�0154 7.9188e�001 4.0611e+001 4.8785e�015 4.9520e�0155 9.5011e�001 2.9285e+001 1.6086e�014 1.3218e�0146 1.2224e+000 5.8448e+000 4.0986e�014 2.8719e�0147 1.2958e+000 2.8098e+001 5.8023e�014 2.5838e�0148 6.8261e�001 1.0628e+001 5.6092e�014 1.9867e�0149 1.6275e�001 1.0353e+000 4.1421e�014 1.1533e�01410 7.1614e�003 5.0070e�003 4.2224e�014 7.9361e�01511 1.5424e�005 8.2156e�006 3.4458e�014 9.1440e�01512 7.4244e�011 1.1357e�011 3.5097e�014 8.9402e�01513 4.4839e�015 9.4163e�014 3.3545e�014 8.0958e�01514 4.2041e�015 6.5832e�014 3.6587e�014 7.7862e�01515 4.2522e�015 6.5618e�014 3.7792e�014 8.8756e�015


0 10 20 30 40 50 60 70-16

-14

-12

-10

-8

-6

-4

-2

0

2

Fig. 8. The Classical Newton Iteration, log kXA� IkNN .

0 5 10 15 20 25 30 35-16

-14

-12

-10

-8

-6

-4

-2

0

2

Fig. 9. The Cube Polynomial Accelerated Newton Iteration, log kXA� IkNN .


compare the two algorithms. The initial iterate was that of (2.1) and (2.10). Theresults are shown in Figs. 6 and 7; the date are in Tables 1 and 2, which giveskXAX � Xk, kAXA � Ak, k(MAX)* � MAXk and k(NXA)* � NXAk, from thedata and the figs, we can get the fact that the Accelerated Newton Iterationconverges almost twice as fast as the Classical Newton Iteration.

Table 3The Classical Newton Iteration

Iteration kXAX � Xk kAXA � Ak k(MAX)* �MAXk k(NXA)* � NXAk21 4.6695e�002 5.5590e+000 7.2699e�015 4.9330e�01422 9.3390e�002 5.5590e+000 7.3114e�015 4.9466e�01423 1.8678e�001 5.5590e+000 7.2763e�015 4.9548e�01424 3.7356e�001 5.5590e+000 7.2172e�015 4.9508e�01425 7.4712e�001 5.5590e+000 7.4001e�015 4.9579e�01426 1.4942e+000 5.5590e+000 7.3952e�015 4.9566e�01427 2.9885e+000 5.5590e+000 7.4048e�015 4.9520e�01428 5.9769e+000 5.5590e+000 7.4477e�015 4.9611e�01429 1.1954e+001 5.5590e+000 7.3575e�015 4.9700e�01430 2.3907e+001 5.5590e+000 7.4392e�015 4.9678e�01431 4.7813e+001 5.5590e+000 7.4329e�015 4.9568e�01432 9.5621e+001 5.5590e+000 7.3641e�015 4.9481e�01433 1.9122e+002 5.5589e+000 7.5155e�015 4.9360e�01434 3.8235e+002 5.5587e+000 7.4604e�015 4.9404e�01435 7.6437e+002 5.5585e+000 7.4533e�015 4.9450e�01436 1.5274e+003 5.5579e+000 7.4304e�015 4.9543e�01437 3.0494e+003 5.5568e+000 7.5243e�015 4.9513e�01438 6.0771e+003 5.5546e+000 7.4496e�015 4.9413e�01439 1.2068e+004 5.5501e+000 7.6223e�015 4.9346e�01440 2.3797e+004 5.5412e+000 7.6688e�015 4.8967e�01441 4.6264e+004 5.5233e+000 7.5442e�015 4.8834e�01442 8.7446e+004 5.4879e+000 7.6535e�015 4.8507e�01443 1.5632e+005 5.4176e+000 7.5687e�015 4.7943e�01444 2.5087e+005 5.2798e+000 7.3527e�015 4.6692e�01445 3.4545e+005 5.0146e+000 7.0370e�015 4.4468e�01446 4.4120e+005 4.5235e+000 6.4655e�015 4.0046e�01447 5.3655e+005 3.6809e+000 5.3486e�015 3.2568e�01448 6.3231e+005 2.4373e+000 3.3633e�015 2.1505e�01449 6.9803e+005 1.0686e+000 1.6099e�015 9.4683e�01550 8.3258e+005 2.0543e�001 8.6989e�016 1.9881e�01551 1.0368e+006 7.5913e�003 8.3162e�016 9.5248e�01652 1.2457e+006 1.0367e�005 8.0985e�016 1.0398e�01553 1.2014e+006 2.1611e�007 7.6844e�016 1.0307e�01554 1.3132e+006 1.5771e�007 8.0879e�016 1.0144e�01555 1.1873e+006 9.0385e�008 6.8534e�016 1.0336e�01556 5.2725e+005 3.0069e�008 8.4327e�016 9.4254e�01657 6.4893e+004 3.3325e�009 7.5538e�016 9.1749e�01658 8.0393e+002 4.0799e�011 7.1558e�016 9.6346e�01659 1.2022e�001 6.1004e�015 7.6524e�016 1.1281e�01560 3.4771e�009 3.9044e�015 7.6979e�016 1.0235e�01561 2.6565e�009 4.4900e�015 6.9629e�016 8.3881e�01662 2.3412e�009 4.0106e�015 6.8091e�016 8.3152e�01663 2.3868e�009 4.5153e�015 6.0738e�016 8.1560e�01664 2.5029e�009 4.1953e�015 7.3441e�016 8.5071e�01665 2.6002e�009 4.0964e�015 6.8134e�016 1.0882e�015


Table 4The Cube Polynomial Acceleration Newton Iteration

Iteration kXAX � Xk kAXA � Ak k(MAX)* �MAXk k(NXA)* � NXAk1 6.6901e�002 5.8252e+000 2.4210e�016 5.2855e�0172 7.6442e�002 5.7007e+000 3.1226e�016 9.6378e�0173 9.2611e�002 5.6211e+000 4.2955e�016 1.8567e�0164 1.0473e�001 5.5645e+000 7.9510e�016 3.6521e�0165 1.2513e�001 5.5152e+000 1.5877e�015 6.2255e�0166 1.7078e�001 5.4827e+000 3.1769e�015 1.0990e�0157 2.2080e�001 5.4753e+000 6.4359e�015 1.9209e�0158 2.0965e�001 5.4838e+000 1.3737e�014 3.2449e�0159 1.3112e�001 5.4937e+000 3.0640e�014 5.9379e�01510 4.1179e�002 5.4998e+000 6.7588e�014 1.1407e�01411 1.1228e�001 5.5297e+000 8.1633e�013 7.9052e�01412 4.3928e�003 5.5587e+000 8.1556e�013 8.3869e�01413 2.6539e+000 5.5590e+000 7.8509e�013 8.4276e�01414 3.7035e+005 4.8838e+000 4.8603e�011 1.3594e�01215 4.6711e+005 4.2906e+000 4.2699e�011 1.1943e�01216 5.5449e+005 3.3116e+000 3.2956e�011 9.2186e�01317 6.4897e+005 1.9727e+000 1.9632e�011 5.4913e�01318 7.1036e+005 7.0006e�001 6.9669e�012 1.9500e�01319 8.7778e+005 8.8160e�002 8.7745e�013 2.4596e�01420 1.0661e+006 1.3981e�003 1.4014e�014 7.9220e�01621 1.2586e+006 3.5163e�007 9.9444e�016 7.6266e�01622 1.2320e+006 2.0480e�007 1.0555e�015 8.7869e�01623 1.3337e+006 1.4533e�007 1.0228e�015 7.2246e�01624 1.0873e+006 7.7086e�008 9.6293e�016 8.7881e�01625 3.9684e+005 2.1889e�008 9.9947e�016 7.1191e�01626 3.4570e+004 1.7652e�009 1.0481e�015 8.3004e�01627 2.2542e+002 1.1439e�011 1.0309e�015 8.0124e�01628 9.4490e�003 4.1776e�015 1.0172e�015 8.1287e�01629 2.5431e�009 3.6523e�015 1.0109e�015 8.7516e�01630 3.0222e�009 4.4275e�015 1.0987e�015 7.3572e�01631 2.1479e�009 4.1725e�015 9.7709e�016 6.4605e�01632 2.5276e�009 4.0481e�015 8.4361e�016 5.9213e�01633 2.5457e�009 3.7904e�015 1.1030e�015 5.8872e�01634 2.0868e�009 4.1187e�015 9.4860e�016 5.9113e�01635 2.4895e�009 4.0736e�015 1.0547e�015 5.5331e�016


Example 2. We generated a random 64 · 64 matrix, and two random Hermi-tian matrices,M 2 C64�64 and N 2 C64�64, then changed its singular values so asto create an ill-conditioned matrix A whose singular lies in two clusters. Thereare 32 singular value in the interval [1, 7.6] and 32 others in the interval[10�7,10�6]. We used algorithms Classical Newton Iteration and Cube Polyno-mial Accelerated Newton Iteration to compute the Aþ

MN . The initial iterate wasthat of (2.1) and (2.10). The results are shown in Figs. 8 and 9; the date are inTables 3 and 4, which gives kXAX � Xk, kAXA � Ak, k(MAX)* � MAXk andk(NXA)* � NXAk, from the data and the figs, we can get the fact that the Cube


Polynomial Accelerated Newton Iteration converges faster than the ClassicalNewton Iteration for such an ill-conditioned matrix.

11. Concluding remarks

In this paper, we present the Newton iteration for computing the weightedMoore–Penrose inverse of an arbitrary matrix. It is natural to ask if we can ap-ply our method to compute the weighted Moore–Penrose inverse of Toeplitzmatrix [2,15]. This will be a future research topic.

References

[1] A. Ben-Israel, T.N.E. Greville, Generalized Inverses: Theory and Applications, Wiley, NewYork, 1974.

[2] J. Cai, Y. Wei, Displacement structure of weighted pseudo inverses, Appl. Math. Comput. 153(2004) 317–335.

[3] M.E. Gulliksson, X. Jin, Y. Wei, Perturbation bound for constrained and weighted leastsquares problem, Linear Algebra Appl. 349 (2002) 221–232.

[4] M.E. Gulliksson, P.A. Wedin, Y. Wei, Perturbation identities for regularized Tikhonov inverseand weighted pseudo inverse, BIT 40 (2000) 513–523.

[5] X. Jin, Y. Wei, Numerical Linear Algebra and Its Applications, Science Press, Beijing/NewYork, 2004.

[6] V.Y. Pan, R. Schreiber, An improved Newton iteration for the generalized inverse of a matrix,with applications, SIAM J. Sci. Statist. Comput. 12 (1991) 1109–1131.

[7] R. Schreiber, Computing generalized inverses and eigenvalues of symmetric matrices usingsystolic arrays, in: R. Glowinski, J.L. Lious (Eds.), Computing Methods in Applied Scienceand Engineering, North-Holland, Amsterdam, 1984.

[8] T. Soderstorm, G.W. Stewant, On the Numerical properties of an iterative method forcomputing the Moore–Penrose generalized inverse, SIAM J. Numer. Anal. 11 (1974) 61–74.

[9] W. Sun, Y. Wei, Inverse order rule for weighted generalized inverse, SIAM J. Matrix Anal.Appl. 19 (1998) 772–775.

[10] C.F. Van Loan, Generalizing the singular value decomposition, SIAM J. Numer. Anal. 13(1976) 76–83.

[11] G. Wang, Y. Wei, S. Qiao, Generalized Inverses: Theory and Computations, Science Press,Beijing/New York, 2004.

[12] M. Wei, Supremum and Stability of Weighted Pseudo Inverses and Weighted Least SquaresProblems Analysis and Computations, Nova Science Publisher, Inc., 2001.

[13] Y. Wei, Recurrent neural networks for computing weighted Moore–Penrose inverse, Appl.Math. Comput. 116 (2000) 279–287.

[14] Y. Wei, The weighted Moore–Penrose inverse of modified matrices, Appl. Math. Comput. 122(2001) 1–13.

[15] Y. Wei, J. Cai, M.K. Ng, Computing Moore–Penrose inverse of Toeplitz matrices by Newton�siteration, Math. Comput. Modelling 40 (2004) 181–191.

[16] Y. Wei, G. Chen, Perturbation of least squares problem in Hilbert space, Appl. Math.Comput. 121 (2001) 177–183.


[17] Y. Wei, J. Ding, Representations for Moore–Penrose inverse in Hilbert space, Appl. Math.Lett. 14 (2001) 599–604.

[18] Y. Wei, D. Wang, Condition numbers and perturbation of the weighted Moore–Penroseinverse and weighted linear least squares problem, Appl. Math. Comput. 145 (2003) 45–58.

[19] Y. Wei, G. Wang, PCR algorithm for parallel computing the minimum T-norm S-least squaressolution of inconsistent linear equations, Appl. Math. Comput. 133 (2002) 547–557.

[20] Y. Wei, H. Wu, J. Wei, Successive matrix squaring algorithm for parallel computing theweighted generalized inverse Aþ

MN , Appl. Math. Comput. 116 (2000) 289–296.[21] Y. Wei, H. Wu, J. Wei, The representation and approximation for the weighted Moore–

Penrose inverse, Appl. Math Comput. 121 (2001) 17–28.[22] Y. Wei, H. Wu, Expression for the perturbation of the weighted Moore–Penrose inverse,

Comput. Math. Appl. 39 (2000) 13–18.

an improved newton iteration for the weighted moore–penrose inverse

Documents