solving symmetric indefinite systems in an interior...

Mathematical Programming 62 (1993) 15-39 15 North-Holland

Solving symmetric indefinite systems in an interior-point method for linear programming

R o b e r t F o u r e r and Sanjay M e h r o t r a Department of Industrial Engineering and Management Sciences, Northwestern University, Evanston, IL, USA

Received 28 January 1992 Revised manuscript received 27 October 1992

This paper is dedicated to Phil Wolfe on the occasion of his 65th birthday.

We describe an implementation of a primal-dual path following method for linear programming that solves symmetric indefinite "augmented" systems directly by Bunch-Parlett factorization, rather than reducing these systems to the positive definite "normal equations" that are solved by Cholesky factoriz -~ ation in many existing implementations. The augmented system approach is seen to avoid difficulties of numerical instability and inefficiency associated with free variables and with dense columns in the normal equations approach. Solving the indefinite systems does incur an extra overhead, whose median is about 40% in our tests; but the augmented system approach proves to be faster for a minority of cases in which the normal equations have relatively dense Cholesky factors. A detailed analysis shows that the augmented system factorization is reliable over a fairly large range of the parameter settings that control the tradeoff between sparsity and numerical stability.

Key words: Linear programming, interior-point methods, symmetric indefinite systems.

1. Introduction

In t e r io r -po in t a lgor i thms for l inear p r o g r a m m i n g devote the greates t pa r t o f their

c o m p u t a t i o n a l effort to solving s t ruc tured l inear equa t ion systems. In the case of

p r i m a l - d u a l " p a t h fo l lowing" a lgor i thms [16, 25], a s t r a igh t fo rward der iva t ion gives

rise to equa t ions tha t are convenient ly r ega rded as hav ing the fo rm

A D 0 J \ r c / w '

where E and D are posi t ive semi-definite and posi t ive definite d iagona l matr ices,

respect ively, and A is the l inear p r o g r a m ' s cons t r a in t mat r ix . Each i te ra t ion solves

(1.1) wi th one or more r igh t -hand sides (v, w), and uses the solu t ions to const ruct a

Correspondence to: Robert Fourer or Sanjay Mehrotra, Department of Industrial Engineering and Man- agement Sciences, Northwestern University, 2225 North Campus Drive, Evanston, IL, 60208-3119, USA.

This work has been supported in part by National Science Foundation grants DDM-8908818 (Fourer) and CCR-8810107 (Mehrotra), and by a grant from GTE Laboratories (Mehrotra).

[ 6 R. Fourer. S. Mehrotra / Symmetric #uiqfinite systems y

search direction. Then a step in that direction leads to a new iterate and, as a by- product, a new D.

For the most common tbrmulations of the linear program, E is an identity matrix. Thus the first set of equations in (1.1) can be written x = D A V r c - v and substituted

into the second set, reducing the system to the so-called normal equations,

AD2AV rc = ADv + w. (1.2)

Many current implementations [l, 5, 14, 17, 20. 21] solve (1.2) directly by using Gaussian elimination to compute the Cholesky factorization LL r = AD2A T. A major

advantage of this approach is that all elimination pivot orders for finding L are stable. A computationally suitable pivot order can be determined (for example, by using a minimum degree or minimum local fill heuristic) at the outset, and can then be used for all iterations of the interior-point algorithm. This approach has been observed to work well for many linear programs. Nevertheless, it is known to encoun-

ter difficulties in two common situations. The first difficulty occurs when the matrix AD2A v is too dense to permit efficient

computation of a Cholesky factorization, most often due to the presence of "dense columns" in A. Some implementations remove the dense columns from A, and then

correct for their absence by an iterative method [i], a direct method [8], or some combination of the two [17]. All such modifications suffer from numerical instability, however, which may be traced to singularity or near-singularity of ADZA T after the

dense columns have been removed. A more numerically sound alternative is based

on expanded normal equations formed by "splitting" dense columns [18, 28], but the efficiency of this kind of modification is not fully established. In any case, all of

these approaches have the drawback of requiring that dense columns be identified in A in some arbitrary way.

The second difficulty occurs when the linear program contains "free variables"

that are not explicitly bounded above or below. In the equations (1.1) for this case, the matrix E has a zero on the diagonal corresponding to each free variable; as a

result, the first set of equations in (1.1) can no longer be solved for x, and the normal equations (1.2) can no longer be formed. This impediment may be overcome by writing each free variable as the difference of two nonnegative variables, provided a mechanism is supplied to deal with certain numerical difficulties inherent in this approach. Alternatively, each free variable can be given a tentative bound, provided an appropriate change is made if it appears that a bound would become active. Both of these approaches are discussed in [29]. Neither is entirely satisfactory, since each requires that the algorithm be complicated by some sort of heuristic adjustment.

In this paper we report our experience with an alternative approach that avoids the difficulties of the normal equations, by applying elimination directly to factor the larger "augmented system" (1.1). Dense columns in A and zeroes on the diagonal of E are accommodated automatically, through the procedure that chooses a sparse pivot ordering for the entire equation system.

R. Fourer, S. Mehrotra / Symmetric indefinite systems 17 ,r

Because the matrix is merely symmetric indefinite, the equations (1.1) cannot be solved quite so easily as the normal equations. There exists a stable elimination procedure for symmetric indefinite systems, due to Bunch and Parlett [7], but it must allow for certain 2 x 2 pivots along the diagonal in addition to the usual "1 x 1" pivot elements. A sparse version of the Bunch-Parlett procedure has been studied by Duff and Reid [10], but their implementation, known as MA27, is not intended for systems like (1.1) that have many zeroes on the diagonal.

Several recent research reports cite studies of interior-point implementations in which systems like (1.1) are solved directly:

• Turner [27] applies a modification of MA27 to systems like (1.1) that appear in a form of Karmarkar's primal projective algorithm [15] for linear programming. Comparative timings are given for a dozen test problems.

• Gill, Murray, Poncele6n and Saunders [13] substitute out a subset of the variables x from (1.1) to produce a "reduced" system to which MA27 can be applied- without modification. Iteration counts are given for many test problems.

• Vabderbei and Carpenter [30] substitute out all of the variables x except those corresponding to columns of A that are determined in advance to be dense; then they apply ordinary elimination to the resulting reduced system. Extensive tests versus a code that uses a column-splitting approach are reported.

Duff, Gould, Reid, Scott and Turner [9] have meanwhile proposed an improved sparse Bunch-Parlett procedure that would be more appropriate for solving a system like (1.1). They have conducted some tests on matrices from linear programming and other applications.

In this paper' we describe a detailed study of the performance of the augmented system approach. Notable features of our implementation include the following:

• Matrices E and D are chosen to encourage good conditioning of the equations. • A refinement of the sparse Bunch-Parlett elimination procedure of [9] is applied

to the entire system (1.1). • Once a pivot order has been determined by the elimination procedure, it is

reused at subsequent iterations as long as possible.

For purposes of comparison, both the normal equations approach and the augmented system approach are implemented within an efficient primal-dual path following code, described by Mehrotra [21-23]. This code employs predictor-corrector steps and other refinements that have been found to reduce the number of iterations and promote reliability. The reported results are based on runs solving a large number and variety of linear programs, and using a substantial range of values for two parameters that regulate the numerical behavior of the elimination.

l 8 R. Fourer, S. Mehrotra / Symmetric #Tdefinite systems

We are able to conclude that, for parameter values in a comfortably large range, our augmented system version reliably solves all tested problems, and avoids all difficulties encountered by the normal equations approach due to dense columns and free variables. For the dense column cases the augmented system is also, as expected, much less costly to solve than the unmodified normal equations. For other LPs the augmented system approach is usually more expensive by a factor between 1.0 and 2.0, with no clear trend as problem size increases.

Further examination of the results shows that the augmented system approach tends to require fewer arithmetic operations than the normal equations approach to solve for (x, Jr), because it computes sparser factors. Either dense columns or dense rows in A can give rise to an undesirably dense factorization, but only if the above- mentioned pair of numerical parameters is allowed to stray into certain areas outside a safe region.

The interior-point algorithm used in our study is summarized in Section 2 below, after which Section 3 presents the details of the augmented system approach. The computational performance of our approach is compared in Section 4 to the performance of the normal equations approach, and is studied in Section 5 with respect to the relevant numerical parameters. Section 6 makes a few concluding remarks on directions for continuing research in this area.

2. The interior point method

All our experiments employ a second-order version of the primal-dual path following method described more generally by Mehrotra [22], with extensions to the case of free variables as described in [23]. The requisite computations are summarized in this section, while the reader is referred to [22, 23] for a justification of the individual steps.

We assume that the linear program is presented in terms of nonnegative variables x and free variables y:

minimize c Tx + c Ty

subject to A x + Fy = b,

x>~O.

The corresponding dual linear program can thus be written as:

maximize bVrc

subject to AT2: + s = cA,

FTTc = C F ,

s >~O.

Inequality constraints are converted to the above form by adding slack variables, and "ranges" are represented by upper bounds on the slacks. Our implementation

R. Fourer, S. Mehrotra / Symmetric indefinite systems 19

handles upper bounds directly, as described ir~ [22], but for notational convenience we describe the algorithm without bounds in this section.

Starting point. A starting point for the algorithm is generated by using the solutions of a primal least squares problem,

minimize I[xll

subject to A x + Fy = b,

and a dual least squares problem,

minimize Ilsll

subject to AT~ +s = CA,

FT Ir = CF.

Let ~, y and if, ~ be the solutions to these problems. We proceed to take ""

8ix -- max( -3 min {)?j}, 0), 8, = max( - 3 min{gi}, 0),

and compute

~ = S x + _ 1 ( 2 + S x e ) T ( S + a s e) ~s= 8 s + _ 1 ( '2+8":e)T(~+8~e)

2 E, (g~ + 8s) ' 2 ~ j (-~j + 8x)

The starting point is then given by

x(O>=2+gxe, y(O)=y, s(O)=g+~,e, rc(O~=,?.

The least squares problems for determining 92, 37 and r?, g can be solved by use of the same augmented system that is required in the interior-point algorithm. Thus in our implementation the first pivot sequence for the augmented system is determined while generating the starting point.

Iterations. An iteration of the algorithm begins with a solution x(k)> 0, y(~), ~(k~, S (k~ > 0. This solution is not necessarily feasible; denote by ~x = A x (k~ + Fy (~ - b, ~A =

AT~r (k~ +S (k~ -- CA and ~F = FT~ (k~ -- CF the infeasibility of this solution in the primal and dual constraint equations.

Following the development in [22], we consider the trajectory defined by

X ( a ) s ( a ) = aX(k)s (k) + a (1 - a)Z#e,

AW~(a) + s( a) = c~ + a ~ ,

FTTt'(a) = C~+ a~F,

Ax(a ) + F y ( a ) = b + a G ,

2 0 R. Fourer . S. M e h r o t r a / ~ v m m e t r i c imt~f ini te s y s t e m s

where X(a)=diag(x(a)), X"~}=diag(x!~)), and ~l is a centering parameter to';be determined. The current iterate lies at a = 1.

The first derivative of the trajectory is obtained by solving

Hp'-= 0 p{ = / , 2.1)

where S ~k)= diag(s(*)), and by computing p). = ~A- AVP~. Using this derivative as a tentative direction, we find the maximum step

0t,-= rain(l, min{x}k)/(p~,.);] (p!~)j> 0} ),

1 0 , - rain(l, min{slk)/(p).)~l(p~), > 0}),

and compute

( y ( k ) n t I xT,- (k) 1 I -t~xp.~) ~s -O~p~.) f l - - ( x ( k ) ) T s ( k )

We then take the centering parameter as

1 1 2 + _ f,83(X(k))Tslk)/n if (llPxi IIP~, I] 2)/(X(k))TjO < 1.1,

U--~fl3(x(k))vg)/n min(0~ ' 0~) otherwise, ,

where n is the number of nonnegative (x) variables. The choice of a value for the centering parameter makes it possible to determine

the second derivative of the trajectory, by solving

{p2~ V21a(X(k))-,e+2plpl ] Hip2)= I 0 (2.2)

L o

where P~= diag(p~), and by computing p2 = _AVp~. The computations of the first and second derivative require us to solve linear systems in H with two different right hand sides; this is the costly part of an iteration, which will be considered in detail by Section 3.

We proceed to compute the maximum steps for a second-order polynomial as

02 = min( 1, max{ 0 Ix} ~' - O(p'~)j + ~ 02(p~ )j> O} ),

02= min(1, max{0 Is5 k'- O(p).),+ ~ 02(p~),>0} ),

so that the step directions are given by

dr= i ~ , D 2 n - - I t I ,~2 2 . O~p.~- ~,~p-~, d , . - O,.p.~.- :~., .p., . .

0 1 1 I "~ "~ 1 1 ] 2 "~ 4 = xPy- ~O-~pS, d~= O,.p,~- ~O.,.p-~.

R. Fourer, S. Mehrotra / Symmetric indefinite systems

It remains only to determine the step lengths. We first find

a , = x}~)/(dx)tx=min{x}k)/(d,.)jl (dOj> 0},

a, = s}~)/(ds)l, = min{s Ik~/(d,)i [ (d~)i > 0},

and then compute tentative lengths f~, f, such that

(~(k)_.c ~d ~ Ws(k)- = (x (k) ~tx jx" ~ x)t,.j~ t~ a , . (d,)l,) - a x d x ) T ( s ( ~ - a , d , ) / n r a ,

(x~ ~ - a x " (dO¢~)(Slks ~ - f s " (ds)z,) = (x (k) - a~dx)T(s (k) -- asd,) /nTa.

Finally the new iterate is

x (k+l )=x(k ) - -max( fY , 7 0 " d~,

y (k+l )=y(k ) - -max( f x , 7 f ) . dy,

21

s(k+l)=s(k)--max(f~, 7f)" d,,

jr(k+ 1) = zr(k) _ max(Z , 7 0 " d~,

In all the runs described by this paper, we fix the two tuning parameters at 7¢ = 0.9 and 7a = 1/(1 - 7 ¢ ) = 10.

Termination. The algorithm continues until some iterate satisfies

I bTrc (k) -- CA r X (k) -- C~ y(k)[ ~< 10 -p,

1 + IbTjr(k)[

where p is the number of digits of accuracy desired in the solution. All runs reported by this paper employ p = 8.

3. The augmented system approach

In principle, the Bunch-Parlett elimination procedure may be applied directly to the symmetric indefinite systems (2.1) and (2.2), which have the form of our augmented system (1.1). If there are no free variables, and if the elimination procedure pivots first down the diagonal of --(X(k))-lS(k), then it is essentially forming and solving the normal equations. If instead this procedure sets aside certain elements of --(X(k))-lS(k) that line up with dense columns in A, and pivots on these only after all other pivots have been carried out in the usual way, then it accomplishes the effect of the so-called Schur complement modification [8]. Since the Bunch-Parlett procedure is a more general one that can use these orderings, but that admits a vast number of others, it has the potential to produce sparser and more accurate factors.

This section begins by describing the main ideas of our implementation of the augmented system approach: our version of a sparse Bunch-Parlett procedure, and its use within the context of an interior-point algorithm. Rather than apply these ideas directly to the systems (2.1) and (2.2), however, we employ modified scaled systems that have better properties of sparsity and stability. The nature and advantages of the modified systems are discussed in the last part of this section.

22 R. Fourer, S. Mehrotra /' Symmetric indefimte systems

J The elimination procedure. Bunch and Parlett [7] accommodate indefinite systems

by allowing 2 x 2 block pivots to be chosen along the diagonal, in addition to the

I x 1 pivots that would be employed for a positive definite system. To be specific, suppose that at some stage of the elimination we have a k x k reduced (symmetric)

matrix, whose elements are hij. We must next select either a I × I pivot h. #0 , or a 2 x 2 pivot

(3.1)

such that h,j ¢ 0. Adopting the terminology in Duff et ai. [ 9 ] we call a 2 × 2 pivot oxo,

i f hii = hjj = 0 ; t i le , if either h, or hjj = 0, but not both; or full, if h, # 0 and hjj V = O. We wish to select a pivot so as to mainta in both stability and sparsity of the re-

duced matrix. For stability, a 1 × 1 pivot hii is accepted if it satisfies the usual partial-

pivoting condition.

(3.2)

where ilh, llco denotes the largest element in row (or column) i of the reduced matrix. A 2 x 2 pivot (3.1) is accepted if it satisfies an analogous condition,

(3.3)

Values of c~ in the range of 16 -4 to 16 -6 give consistently good factorizations, as the

results in Section 5 will show. To promote sparsity, we employ a modification of the "Markowitz-b" pivot selec-

tion procedure of [9], which is in turn a generalization of a procedure proposed by Markowitz [19] for 1 x 1 pivots. We let ni be the number of nonzero elements in column i of the reduced matrix (the degree of element h,) and let ~u = min(ni + n j - 4, k - 2). For each kind of 2 x 2 pivot we define a "cost" as follows :

This cost is an upper bound on the number of elements in the reduced matrix that could be modified by the "second pivot" as a result of an elimination step using a 2 x 2 pivot. In this respect, it is a generalization of the Markowitz cost ( n i - I ) 2 for a 1 x 1 pivot h,-.

Our pivot selection procedure can now be described. For successively increasing values r = 1, 2 . . . . , we carry out the following two searches until a pivot is accepted:

Step 1. Consider elements h, of degree r. I f any satisfies (3.2), accept it as a stable 1 x 1 pivot. Otherwise, label it an unstable 1 × 1 pivot.

Step 2. From all the unstable 1 x 1 pivots of degree r encountered, consider potential 2 × 2 pivots of the form (3.1) having cost ~< a certain threshold value: ( r - 1) 2,


( r - 1 ) ( 2 r - 3 ) , or (2 r -4 ) 2 for an oxo, tile, or~full pivot, respectively. If any such pivot satisfies (3.3), accept it.

This procedure should choose, among all available numerically acceptable pivots, one of those that is most likely to preserve the sparsity of the reduced matrix.

Use o f the procedure. Our pivot selection procedure refers concurrently to the positions and the values of nonzero matrix elements. Indeed, in contrast to the positive definite case, there is no general way to find a numerically stable pivot order for an indefinite matrix using only symbolic information. Nor does there appear to be any good way to "fix up" a symbolically determined ordering after some pivot is determined to be numerically unacceptable. For these reasons, we carry out the elimination in a single phase that combines the symbolic and numeric computations.

Once a pivot order has been determined, however, we re-use it at subsequent iterations as long as it continues to give satisfactory factorizations and solutions. Because elimination with a known pivot order is much faster than the complete" sparse Bunch-Partett procedure, the re-use of orderings is a key feature of an efficient implementation.

A factorization using a previously determined pivot order is considered satisfactory by our implementation if it satisfies (3.2) and (3.3) with 8 = 16 -12. This is a very weak form of the stability criterion, which is intended only to catch division by zero or by a meaningless small value.

Once a fact0rization has been accepted, then for each system of the form Hz = r that is solved using that factorization, the solution z is deemed satisfactory if

IlHz--rH oo/[Irll o~ < 10 -5.

If a re-used pivot order is found to give an unsatisfactory factorization or solution, then the sparse Bunch-Parlett procedure is re-applied to determine a new order and factorization. Computational experience reported in Section 5 suggests that most linear programs can be solved to optimality using only one or two different pivot orders.

It can happen that even when the factorization has been determined anew by the sparse Bunch-Parlett procedure, it yields an unsatisfactory solution to some linear system by the above criterion. In such a case, we try the elimination procedure again with a larger value of 8 that tends to shift the pivot selection criterion more toward stability, though away from sparsity. Adjustments of this kind are seen very rarely in our test runs; Section 5 provides the details.

Modifications to the equation systems. Clearly the systems (2.1) and (2.2) may be scaled so that the coefficient matrix H becomes

DAAT 1 7 o °SJ f I - 0

ADA FDF

24 R. Fourer, S. Mehrotra Symmetric hufijinite 4vs~cms

where D~ = (X~/°) t 2(S~k/)-a/2. We could leave DF=L but instead choose :;

)1/'2 tl

DF = (Xtk~-Vs(k)- diag(t y(k)l + e),

(where i y~k~[) is a vector of entries /y}k)l ) so that that columns of F are scaled similarly to the columns of A. A small positive value of e would ensure that the scale remains

positive even in the case that some component of y(k~ becomes essentially zero. In fact the tests reported in this paper take e = 0, but no significant difference is observed

when the cases with tree variables are rerun using a = 10 -6 .

Following the recommendation of Bj6rck [6], we further modify the augmented

system to allow for the family of coefficient matrices

DA A T-]

H~ - 0 0 D T . ADA FDF

where IADA i1.2 is the largest magnitude of any element in AD4. and ~c > 0 is a scaling constant used to improve the condition number. In particular Skeel's condition

number [26], ~cs(H~)= iI Igual IH~I II, where !H~-] is a matrix of entries I(H~-)0-I, is minimized by some 0~< ~c ~< 1 (see [3]). Clearly a system in H~- remains symmetric

indefinite, and can be derived from the one in D by a straightforward scaling. The

choice of a value for ,r is discussed in Section 5. There are several advantages to using the scaled matrix H~. rather than H. Scaling

can permit more pivots to satisfy the stability tests (3.2) and (3.3), leading to a sparser elimination; and in fact, for our test problems, the resulting factors of H~ tend to be sparser than those of H. Our tests also indicate that H has a tendency to require two or more refactorizations near the last iteration, in cases where H~ requires

just one refactorization. Finally, we observe that the choice of ~c does significantly affect the implementa-

tion's success. It is closely linked to the choice of the parameter c~ for (3.2) and (3.3),

as Section 5 will explain.

4. Comparison with the normal equations approach

In this section we compare the efficiency and numerical stability of the augmented system and normal equations approaches. All runs of the augmented system approach use initial parameter settings of ,v = 16 .2 and c5 = 16 -6.

Eor the normal equations approach, the pivot order is chosen by a minimum degree procedure (the 1 x 1 analogue of the Markowitz-b procedure described in Section 3). As in most implementations, a symbolic phase is called once at the


beginning, while a numeric phase is run at each~'iteration. Free variables are handled as the difference of two nonnegative variables, and there is no special handling of

dense columns. Our tests use 87 linear programs from the netlib collection [ 12 ], for which statistics

are given in Table A. 1 of Appendix A. As provided by netlib, free variables are found in capri, cycle, greenbeb, perold, pilot4, pilot.ja, pilot.we, stair, tuff and vtp.base; in addition, we slightly modified the definitions of greenbea and greenbeb to explicitly identify a small number of free variables that had been modeled as differences of nonnegative variables. All problems were preprocessed to remove easily identifiable null and fixed variables, using the procedure CLEAN outlined in Adler et al. [2], but no pre-scaling was performed.

Comparative timings, using the same Fortran compiler on a Sun SPARCstation 1+, are summarized in Figure 4.1 and listed in full in Table A.2. Whereas the augmented system approach terminates successfully with 8 digits of accuracy in all cases, the normal equations approach stalls out with a less accurate solution on many of the problems (capri, greenbea, greenbeb, perold, pilot4, pilot.ja, pilot.we, stair) that" contain free variables; these cases are indicated in Table A.2 by the iteration at which the most accurate solution was found, and the number of digits of accuracy in the solution at that stage (in place of the timing). We believe that these failures are primarily due to the numerical difficulties associated with splitting free variables into pairs of nonnegative variables. The augmented system approach also fails to achieve 8 digits of accuracy on greenbea and greenbeb, for example, if we apply it to the original versions of these problems in which some free variables have already been

split.

2.00

"~ 1.50

O z

1.oo

G E

< 0.50

0.0(~

e 8 ~ 8 e

. . . . . . . . . . . . . . . . . . . . , . . . . . . . . . . . . . . . . . . . . . . .

o ~ e w e e ~ e e ~ - - - ~ . . . . . . . . . . . . . . . . . . . . .

® @ D2Q06C O

GANGES @ CYCLE

. . . . . . . . . . . . . . . . . . . . . . . . . . . ~ A G - G . . . . . . . . . . . . . . . . . . . . . . . . . . .

@ ISRAEL

SEBA ~ ® FIT1P © FIT,2P

10 100 1000 10000 100000 1000000

Nonzeroes

Fig. 4.1. Ratio of' total execution time in the augmented system approach to total execution time in the normal equations approach. Problems for which the normal equations approach failed to achieve 8 digits

of accuracy are omitted.

26 R. Fourer, S. Mehrotra /' Symmetric ind(~finile sy.~'Iems

d

From the standpoint of total computational time. the augmented system approach

achieves a predictably great advantage in solving those problems (firlp, fit2p, israel, seba) that have well known subsets of dense columns. We also observe a substantial advantage on another group (agg, cycle, d2q06c, ganges) where our sparse Bunch- Parlett procedure appears to have found a much more efficient elimination ordering than the minimum-degree procedure. For the remaining problems that both

approaches solved successfully, the augmented system approach is only slightly faster or is more expensive by a factor ranging between 1 and 2; the median factor over

all problems is 1.40, with no clear trend as problem size increases. The augmented system approach sometimes takes more or fewer iterations than

the normal equations approach, but the difference shows no pattern and is too small to affect our overall conclusions. Thus almost all of the variation between the

approaches is due to differences in the time per iteration. To help further explain the relative performance of the two approaches, Table A.3

presents the numbers of nonzeroes in the factors, and the numbers of arithmetic

operations required to compute these factors. The normal equations approach solves a smaller equation system, but it also must do the work of forming the equations, and of an extra multiplication by A T when back-solving. Therefore we define the

numbers of nonzeroes and operations as follows to make the counts comparable:

• For the normal equations approach, the number of nonzeroes reported is the number of off-diagonal nonzeroes in one Cholesky factor L, plus the number of nonzeroes in the constraint matrix. The number of arithmetic operations is an estimate of the number of additions and multiplications required to form and factor the

normal equations. • For the augmented system approach, the number of nonzeroes reported is just

the number of nonzeroes in one of the factors, except for nonzeroes in the 1 x 1 and 2 x 2 block pivots. The number of arithmetic operations is an estimate of the number of additions and multiplications required to compute the factors. All statistics are

based on the first factorization.

If there are no free variables and if our implementation pivots first on all the diagonal elements of-~clIADA I1~I in H , , then the remainder of the elimination procedure reduces to minimum degree on the normal equations, and the two approaches have the same counts. This possibility is in fact obse~wed for fitld, jqt2d, scsdl, scsd6, scsdS, ship121 and shipl2s. The execution time per iteration in these cases ranges from 40% to 85% more for the augmented system approach than for the normal equations approach. The difference reflects both the extra direct cost of carrying out the Bunch-Parlett procedure, and the extra overhead imposed by the data structures that are needed for the stability tests.

Figure 4.2 shows that, as we expected~ there is a strong tendency overall toward sparser factors in the augmented system approach. Out of the 87 test problems, 37 have less than 90% as many nonzeroes in the fac tors - by comparison with the

< 0.3

0.3-0.5 E

0.5-0.7 Z

0.7 - 0.9 O

0.9 - 1.1 O

1.1 - 1.3 :7 < 1.3-1.5

> 1.5

P R. Fourer, S. Mehrotra / Symmetric indefinite systems

3 N o n z e r o e s < 0.3

' ' ~ 0.3- 0.5

0 5 - 0 7 z ~ " " 0 . 7 - 0.9 ~

+- i ~ 0.9 - 1.1

~ 1.1-1.3 I

< 1 .3-1 .5 I

> 1.5

27

O p e r a t i o n s

0 10 20 30 40 0 10 20 30 40

Number of Cases Number of C a s e s

Fig. 4.2. Histograms of the ratios between number of nonzeroes (left) and number of elimination operations (right) for the two approaches. Ratios less than 1 correspond to problems for which the augmented

system approach was superior.

analogous figure for the normal equat ions- while only 8 have more than 110% as many nonzeroes.

Operation counts tend to correlate with nonzeroes, but display a greater variance. Out of 87 test problems, 40 have less than 90% as many operations for the augmented system as compared to the normal equations, while only 18 have more than 110%. At the extremes, 11 have less than 50% as many operations (these include the ones with obvious dense columns), while 5 have more than 150%. The only unusual cases are agg2 and agg3, for which nonzeroes are about 80% but operations are over 125%.

5. Performance of the augmented system a p p r o a c h

Section 3 has explained that the stability and sparsity of our augmented system elimination procedure are regulated by two parameters, tc and 6. For the experiments reported in Section 4, we used fixed values of these parameters, ~c = 16 -2 and 6 = 16 -6, which were observed to provide good results overall.

To determine the sensitivity of the results to these parameter settings, we repeated the runs for all 24 combinations of

~:= 1, 16 -1, 16 -2, 16 -3

and

6 = 16 -1, 16 -2, 16 -3, 16 -4, 16 -5, 16 -6.

It is desirable to choose 0 < ~c ~< 1 as explained in Section 3. A choice of 6 = 1 would correspond to strict partial pivoting, in which each 1 x 1 pivot is the largest element of its row or c, olumn; as 6 gets smaller, this stability requirement is relaxed, while the opportunity for sparse pivots is increased. The choice of 6 = 16-6~ 10 -7.2 would

28 R. Fourer. S. Mehrotra / Symmetric indgfinite systems

g be considered a very weak stability requirement in comparison with the settings commonly used in elimination on nonsymmetric sparse matrices.

Appendix B, available separately from the authors as part of [11], tabulates the

following data for all runs: the number of iterations, the number of times a pivot

order was determined (under the rules set forth in Section 3), the first iteration at which a revised pivot order was determined, the number of nonzeroes in the first

factorization, and the maximum number of nonzeroes in any of the t'actorizations.

For certain parameter combinations, the factors become extremely dense. To avoid

wasting time on obviously inefficient runs, we aborted any runs for which the number

of nonzeroes in the factorization exceed 20 times the number of nonzeroes in A.

We first summarize the conclusions that can be drawn from these tests, and then

comment at greater length on the relationship of sparsity to the parameter settings.

Summary of tests. Almost any combination of ~ and 5 values is able to give a

numerically acceptable factorization. The major exception is greenbea, for which only the combinations ~c = 16 -2, a = 16 -1 and ~ = 16 -3, ~ = 16 .2 yield acceptable factors

at later stages of the algorithm. In a few other runs 5 needs to be increased slightly

at refactorization, as indicated by Table 5.1; these cases seem to be flukes, however, as all but one involve intermediate rather than very small values of 5. All adjustments

to K or 5 for purposes of numerical stability are handled automatically in our

implementation; first we try increasing ~ by successive factors of 16, and then if 5 reaches 16 -1 without success we try decreasing ,v by factors of 16. Thus the algorithm

proceeds successfully to optimality in every case.

The sparsity of the factorization is also maintained over a range of parameter settings. For 5 = 16 -6, the sparsity of the factors is insensitive to ~c. I f a more conserv-

ative stability criterion is desired, we can take ~c = 16 -1 or 16 .2 and increase c~ to 16 -4

with little or no loss in sparsity on any of the tested problems. Unacceptably dense factors occur only when ~ is chosen too large.

Tab le 5.1

Miscel laneous a u t o m a t i c adjus tments to the value

of p a r a m e t e r ~ due to unacceptab le factor izat ions

P rob l e m ~c 5 adjus tments

ganges I6 ° 16 -6 to 16 -5

pilot4 16 o 16 -4 to 16 -3

pilot4 16 o 16 -3 to 16 -2

pilot. ja 16 0 16 -4 to 16 -3

pilot. ja 16 o 16 -~ to 16 -2

pilot.ja 16 -~ 16 -3 to 16 -z

For greenbea it was necessary to adjust both

pa rame te r s to ei ther ~c=16 -2, a = 1 6 -~ or ~c=

16 -3 , 8 = 16 .2 in o rde r to achieve an acceptable

factorizat ion.


About half the test problems are solved using the same pivot order throughout.

Most of the other half use just two pivot orders, with the second usually being needed

near the last iteration; in a few instances (degen2, shell) the second ordering is

instead computed near the beginning. Only a few exceptional cases (80bau3b, degen3, greenbea, sierra) exhibit frequent recomputation of the pivot order; there is no clear relationship, however, between ~c or 8 and the number of times a new ordering needs

to be determined. Since the cost of determining a numerically stable pivot order for a symmetric

indefinite matrix is significantly more than the cost of factoring the matrix with a

known pivot order, our implementation benefits greatly by reusing the pivot order at all or most iterations. Based on some informal experiments, we estimate that average execution times would be three to four times greater if at every iteration a

fresh pivot order were computed. When a new pivot order is needed, the number of nonzeroes in the factorization

often increases. However, for ~c= 16 -~ o r 16 .2 and 8 = 16-5or 16 -6, the increase

exceeds 10% in only twelve of the runs, and exceeds 25% in only two. This means

that, as in the case of the normal equations approach, the storage space that will be needed to solve a problem can be estimated at the beginning.

Sparsity and parameter settings. By examining a few particular examples, we can

show how the settings of ~c and 8 act together to influence the sparsity of the factors.

Consider first the problemsfitlp andfit2p, which are known to have a few dense columns. For these problems we observe a sharp increase in the number of nonzeroes

beyond a certain threshold combination of the parameters (Table 5.2): in the runs with ~c8 ~< 16 .5 the factors are sparse, while for to8 >/16 -4 the factors become dense.

Clearly, the latter combination forces a pivot in the augmented system corresponding to a dense column; any alternative sparse pivots must have been rejected as unstable.

Table 5.2

Number of factorization nonzeroes for fitlp and fit2p, under different parameter settings

fitlp fi@ 0 - 1 - 2 - 3 0 - 1 - 2 - 3

- 6 10140 10140 10140 10140 - 6 50583 50583 50583 50583

- 5 10140 10140 10140 10140 - 5 50589 50583 50583 50583 - 4 175744 10140 10140 10140 - 4 - - 50589 50589 50583 - 3 205986 175744 10140 10140 - 3 - - - - 50589 50583

- 2 206019 205986 175744 10140 - 2 - - - - - - 50589 - 1 206019 206018 206198 178527 - 1 . . . .

The tables are labeled by the chosen values of I0g16 fi (left) and 10g16 ~: (top). A - - indicates a run that was aborted because the number of nonzeroes in the factors would have been more that 20 times the number in the constraint matrix A,

30 R. Fourer, S. Mehrotra / Symmetric indefinite systems

To see how the sparsity of the factorization can be tied so closely to ~:6, cons']der the following simplified example:

H =

- to 0 0 0

0 - i c 0 0

0 0 - • 0

0 0 0 - ~

1 16 .2 0 0

1 0 16 .2 0

1 0 0 16 -2

I 1 1

16 -2 0 0

0 16 .2 0

0 0 16 -2

0 0 0

0 0 0

0 0 0

So long as ~:~-~>~ 16 .2 (which is satisfied by the settings in all of our tests), our

procedure can accept the three sparse pivots H22, H33 and//44, yielding the reduced

matrix :

~=

--/,2 Z

1

1

1

11 i] 16-4r -1 0 0 .

0 16-4~; -1 0

0 0 16-4~c -~

Next our procedure can accept the sparse pivots/q22, f~r33 and H44, provided that (16-4 .-i)c~ -i >~ 1 ; that is, provided that ~:~ ~< 16 -4. On the other hand, if K~ > 16 -4,

then our procedure cannot accept any of these pivots; it must choose either the dense

1 x 1 pivot HI~, or a dense 2 x 2 pivot of type full that includes/t1~. The threshold

behavior that we observe with fitlp and fit2p can be attributed to a more complex

example of this phenomenon.

We next consider the problemsfitld andfit2d, which are in a sense duals tofitlp andfit2p, and which consequently have a few dense rows. We observe that the runs with ~:~ -1 >~ 16 0 yield sparse factors, while in the runs for ~c~ -~ ~ 16 -1 the factors are

unacceptably dense (Table 5.3). The latter combination gets into trouble by forcing a pivot in a dense row, after rejecting alternative sparse pivots as unstable. The

Table 5.3

Number of factorization nonzeroes forfitlp andfit2d, in the same format as Table 5.2

fit l d fit2d

0 -1 - 2 - 3 0 - I - 2 - 3

- 6 13699 13699 13699 13699 - 6 129341 129341 129341 - 5 13699 13699 13699 13699 5 129341 129341 129341 - 4 13699 13699 13699 13699 - 4 129341 129341 129341 - 3 13699 13699 13699 13699 3 129341 129341 129341 - 2 13699 13699 13699 386255 - 2 129341 129341 129344 - i 13701 13709 431720 537039 -1 129342 129375 - -

129341 129341 129341 129344


normal equations approach encounters no cotnparable difficulty, since dense rows (unlike dense columns) do not cause the normal equations to fill in.

The relationship between sparsity and ~:8 -~ can also be understood through a simplified example. Consider

H =

"-~: 0 0

0 -~: 0

0 0 -~:

0 0 0

1 1 1

the matrix

0

0

0 --K"

16 -2 :

1

1

1 16-2

. . .

0

So long as xfi-1 >/1 6°, sparse pivots on H~1, Hz: and H33 can be taken. Otherwise these pivots are ~ated out, and we are forced to pivot on H44, yielding the reduced matrix

i 0 0 O r -~c 0

Er= 0 - ~

1 1

1] 1

1 . . . .

16-4tc -1

At this point the pivots on/~11, /~22 and /~33 are still ruled out. It is necessary to choose either the dense 1 x 1 pivot/~44, or a dense 2 x 2 pivot of type full that includes H44.

It is easy to pick out the dense-column and dense-row cases among the test problems, just by looking at the tables of factorization nonzeroes in Appendix B of [ 11 ]. Dense columns give a threshold that depends on ~:~, as seen in boeing1, boeing2, bore3d, f f f f f800 or israel. Dense rows give a threshold that depends on ~:8-1, as in degen3, grow22, nesm, scsdS, shell, ship121, woodlp or woodw. If the matrix exhibits dense rows and dense columns, then both thresholds can be seen. In the case of cycle, a significant growth in nonzeroes is observed for both lc8 >~ 16 -3 and ~c6-1~< 16 -1 (Figure 5.1); other examples are bandm, capri, d2q06c, etamacro and scfxm3. Few of these problems exhibit such obvious dense columns or rows asfitlp,fit2pandfitld, fit2d; the threshold effects may instead be triggered by columns or rows that start out fairly sparse but that rapidly fill in as the elimination proceeds.

Based on this analysis, we can say that tc and 6 should be chosen so that ~:8 is not too large and ~c8-1 is not too small. Our tests show that these requirements are met, for a wide range of test problems, by settings of ~cs[16 -1, 16 .2 ] and ~s [ 16 -4, 16-6]. This explains, in part, the robustness of the implementation.

6. C o n c l u d i n g r e m a r k s

Our results indicate that the augmented system approach offers a natural and reliable way to handle dense columns and free variables within an interior-point me thod-

32 R. Fourer. S. Mehrotra / 'Symmetric hMeJilfite s.rstems

120000 I ~ " 110000

10oo00 £

9000 4 c- O e -

: :¢" • i r

• ":'::: "" / " 0

I ' " : . . . . . 1

!iii .... - 2 -3 -4 log K

log ~ -5 -6

Fig. 5.I. Number of factorization nonzeroes for cycle, under different settings of log~6 d (left) and log~6 (right). A significant growth is observed for both ~c5>~ 16 -3 and ~cc5 -~ ~< 16 ~.

though at a tradeoff in extra cost for solving "easy" sparse problems on which the normal equations approach already works well. The extra cost is often less than 50%, and never grows unmanageably. Nevertheless, the design of a faster but equally reliable "hybrid" approach is a topic for continuing investigation.

The augmented system approach is also likely to play a key role in the extension of primal-dual path following algorithms to linearly constrained quadratic programming. It is well known that essentially identical algorithms can accommodate quadratic terms in the objective, by defining E in (1.1) to have the form (2+ (X(k~)-~S ~k~ where O is the symmetric matrix of the coefficients of the quadratic terms; see, for example, the development in [30]. In this case E is no longer necessarily a diagonal matrix, however, and as a result the formation of the normal equations becomes impractical• In contrast, the Bunch-Parlett elimination procedure described in this paper remains directly applicable when E is any sparse, symmetric matrix.

Appendix A. Summary counts and timings

The problems listed in Table A.1 are taken from the netlib test set [12]. The first three entries give the numbers of rows, columns and nonzeroes in the constraint matrix after conversion of inequality constraints to equalities by addition of slack variables, and removal of inessential variables and constraints using the procedure CLEAN outlined in [2]. In the " B R F " column, a B indicates that the file specifies upper or lower bounds on the variables in addition to the general constraints, an R


indicates that the file specifies additional range~ (upper bounds on slack variables), and an F indicates that the file specifies free variables. In the cases of greenbea and greenbeb, some free variables are represented as the difference of two nonnegative variables in the netlib versions, but we have identified these variables explicitly for our tests.

Objective function values reported by our tests are consistent with those listed in the "objective 'value" column. The latter have been confirmed by previous runs of simplex and interior-point methods [4, 24], but differ in some cases from values that have appeared in the netlib index f i le .

Table A. 1

Test problems

Problem Rows Columns Nonzeroes BRF Objective value

25fv47 793 1849 10566 5.5018458883 e + 3 80bau3b 2196 11424 22401 B 9.8722419241 e + 5 adlittle 55 137 417 2.2549496316 e+ g " afiro 27 5t 102 -4.6475314286 e + 2 agg 391 479 2092 -3.5991767287 e + 7 agg2 514 755 4728 -2 .0239252356 e + 7 agg3 514 755 4744 1.0312115935 e + 7 bandm 246 401 1927 -1.5862801845 e + 2 beaconfd 115 209 2062 3.3592485807 e + 4 blend 74 114 522 -3.0812149846 e+ 1 bnll 618 1503 5280 1.9776295615 e + 3 bnl2 2237 4363 14723 1.8112365404 e + 3 boeingl 346 720 3808 BR -3.3521356751 e + 2 boeing2 140 279 1332 BR -3.1501872802 e + 2 bore3d 140 195 735 B 1.3730803942 e + 3 brandy 134 239 1927 1.5185098965 e + 3 capri 259 454 1778 B F 2.6900129138 e + 3 cycle 1686 3039 18637 B F -5 .2263930249 e + 0 czprob 689 2797 8364 B 2.1851966989 e + 6 d2q06c 2161 5821 33048 1.2278421081 e + 5 degen2 444 757 4201 -1.4351780000 e + 3 degen3 1503 2604 25432 -9.8729400000 e + 2 e226 208 439 2558 -1.8751929066 e + l etamacro 335 670 1998 B -7.5571523337 e + 2 fffffS00 487 991 6227 5.5567956482 e + 5 finnis 474 988 2474 B 1.7279106560 e + 5 f i t ld 24 1049 13427 B -9.1463780924 e + 3 fi t lp 627 1677 9868 B 9.1463780924 e + 3 fit2d 25 10524 129042 B -6.8464293294 e + 4 fit2p 3000 13525 50284 B 6.8464293294 e + 4 forplan 134 460 4470 BR -6.6421896127 e + 2 ganges 1125 1522 6569 B -1.0958573613 e +2 gffd-pnc 590 1134 2393 B 6.9022359995 e + 6 greenbea 1961 4176 23887 B F -7.2555248130 e + 6 greenbeb 1959 4166 23826 B F -4.3022602612 e + 6 growl 5 300 645 5620 B -1.0687094129 e + 8 grow22 440 946 8252 B - 1.6083433648 e+ 8 grow7 140 301 2612 B -4.7787811815 e + 7

34 R. Fourer, S. Mehrotra / Symmetric ilM~finite x)'stcms

Table A. l--continued

Problem Rows Columns Nonzeroes BRF Objective value

israel 174 316 2443 -8.9664482186 e + 5 kb2 43 68 313 B - 1.7499001299 e + 3 lotfi 145 358 1086 -2.5264706062 e + 1 maros 719 1516 7268 13 -5.8063743701 e + 4 nesm 654 2922 13244 BR 1.4076036488 e + 7 perold 601 1399 5782 B F -9.3807552782 e + 3 pilot 1412 4623 42519 B -5.5740430007 e + 2 pilot4 397 1074 5112 B -2.5811392589 e + 3 pilot.ja 870 1870 11776 B F -6.1131364656 e + 3 pilotnov 906 2181 12255 B -4.4972761882 e + 3 pilot.we 705 2819 8879 B F -2.7201075328 e + 6 recipe 75 161 620 B -2.6661600000 e + 2 scl05 104 162 339 -5.2202061212 e + 1 sc205 203 315 663 -5.2202061212 e + l sc50a 49 77 159 -6.4575077059 e + l sc50b 48 76 146 -7.0000000000 e + 1 scagr25 470 670 1717 -1.4753433061 e + 7 scagr7 128 184 457 -2.3313898243 e + 7 scfxml 315 578 2655 1.8416759028 e + 4 scfxm2 630 1156 5315 3.6660261565 e + 4 scfxm3 945 1734 7975 5.4901254550 e + 4 scorpion 360 429 I393 1.8781248227 e + 3 scrs8 456 1234 3147 9.0429695380 e + 2 scsd 1 77 760 2388 8.6666666743 e + 1 scsd6 147 1350 4316 5.0500000078 e+ 1 scsd8 397 2750 8584 9.0499999993 e + 2 sctap 1 300 660 1872 1.4122500000 e + 3 sctap2 1090 2500 7334 1.724807 I429 e + 3 sctap3 1480 3340 9734 1.4240000000 e + 3 seba 449 904 4120 BR 1.5711600000 e + 4 sharelb 112 248 1148 -7.6589318579 e + 4 share2b 96 162 777 -4.1573224074 e + 2 shell 487 1478 2960 B 1.2088253460 e + 9 ship041 325 1963 5771 1.7933245380 e + 7 ship04s 249 1339 3899 1.7987147004 e + 6 ship081 528 3229 9480 1.9090552114 e + 6 ship08s 334 1712 4929 1.9200982105 e + 6 shipl21 692 4330 12667 1.4701879193 e + 6 shipl2s 422 2102 5983 1.4892361344 e + 6 sierra 1227 2735 8001 B 1.5394362184 e + 7 stair 356 532 3813 B F -2.5126695119 e + 2 standata 324 1202 3030 B 1.2576995000 e + 3 standmps 432 1202 3678 B 1.4060175000 e + 3 stocfor 1 106 154 460 -4.1131976219 e + 4 stocfor2 2141 3029 9281 -3.9024408538 e + 4 tuff 286 596 4357 B g 2.9214776509 e - 1 vtp.base 115 189 471 B F 1.2983146246 e + 5 woodl p 172 1803 48580 1.4429024116 e + 0 woodw 712 5368 23158 1.3044763331 e + 0

I n T a b l e A . 2 t i m i n g s a r e in s e c o n d s o n a S u n S P A R C s t a t i o n 1 + . I n t h e c o l u m n

f o r n o r m a l e q u a t i o n t i m i n g s , co i n d i c a t e s t h a t t h e e q u a t i o n s w e r e t o o l a r g e a n d d e n s e


to solve on the computer used; (k) means thatJthe algorithm failed after k digits of accuracy were achieved.

Table A.2

Iteration counts and timings

Problem Normal Augmented Problem Normal Augmented

it. time it. time it. time it. time

25fv47 26 80.56 26 125.19 pilot 33 1452.33 34 80bau3b 41 208.92 40 258.86 pilot4 32 (3) 43 adlittle 10 0.37 10 0.54 pilot.ja 45 (3) 50 afiro 7 0.09 7 0.14 pilotnov 25 179,68 24 agg 25 15.83 25 6.96 pilot.we 34 (3) 43 agg2 22 31.82 22 53.77 recipe 9 0.55 9 agg3 20 29.06 20 41.63 scl05 9 0.39 9 bandm 17 3.93 17 5.26 sc205 11 0.99 11 beaconfd 7 1.18 7 1.79 sc50a 8 0.17 8 blend 10 0.60 10 0.69 sc50b 6 0.16 6 bnl 1 29 25.41 29 34.14 scagr25 17 3.52 17 bnl2 36 556.21 36 546.73 scagr7 13 0,71 13 boeingl 23 10.98 23 13.58 scfxml 18 5.14 18 boeing2 18 3.02 18 3.17 scfxm2 20 11.74 20 bore3d 19 1.91 17 1.70 scfxm3 20 18.07 20 brandy 20 3.66 20 6.20 scorpion 12 1.97 12 capri 18 (7) 19 6.96 scrs8 21 7.92 21 cycle 25 189.00 27 140.34 scsdl 8 1.36 8 czprob 35 22.66 36 36.77 scsd6 10 3.03 10 d2q06c 30 1122.39 30 862.05 scsd8 9 5.86 9 degen2 12 17.01 12 23.91 sctapl 15 2.87 15 degen3 16 313.90 16 420.81 sctap2 13 15.97 13 e226 20 4.54 20 8.53 sctap3 14 21.10 14 etamacro 31 22.46 29 24.78 seba 21 231.65 17 fffff800 38 41.66 37 41.77 sharelb 22 1.78 22 finnis 25 9.48 26 11.96 share2b 12 0.83 12 fitld 18 12.04 18 17.18 shell 20 8.18 20 fitlp * o~ 18 15.35 ship041 12 5.48 12 fit2d 24 140.15 24 214,24 ship04s 13 3.93 13 fit2p * ~ 21 127.12 ship081 14 10.49 14 forplan 23 7.55 23 8,94 ship08s 14 5.59 14 ganges 19 41.47 18 49.78 shipl21 18 17.66 18 gfrd-pnc 16 4.99 16 7.23 ship 12s 16 7.76 16 greenbea 40 (6) 42 215.31 sierra 21 26.71 20 greenbeb 35 (5) 37 161.40 stair 12 (2) 16 growl5 12 6.04 12 10.10 standata 12 4.22 12 grow22 13 10.17 13 15.27 standmps 19 7.98 19 grow7 11 2.69 11 4.14 stocforl 16 0.96 16 israel 24 26,94 24 6.84 stocfor2 24 38.11 24 kb2 19 0.60 19 0.69 tuff 19 11.50 20 lotfi 14 1.65 14 2,10 vtp.base 19 1.20 17 maros 26 25.53 26 56.29 woodlp 29 92.34 29 nesm 33 70.63 31 77.28 woodw 29 95.12 29 perold 30 (4) 35 101.31

1680.13 54.35

340.38 180.06 81.27 0.95 0.61 1.45 0.33, 0.21 4.32 0.93 7.56

16.23 24.35

3.63 10.40 2.50 5.45

10.38 4.50

21.14 27.43

7.41 2.83 1.05

12.43 9.48 6.67

19.23 10.11 31.78 12.86 56.03 16.28 5.55

11.65 1.14

46.43 14.62

1.23 131.93 137.96

36 R. Fourer. S. Mehrolra /' @,mmetric in&~'nite s)'stetns

For the initial factorization of the normal equations or augmented system, Table A.3 reports the number of nonzero elements (nonz.) and the number of arithmetric operations (opers.) as explained in Section 4. In the entries for the normal equations approach, ~c indicates that the equations were too large and dense to solve on the computer used.

Table A.3

Nonzeroes and arithmetic operations

Problem Normal Augmented

Nonz. Opers. Nonz. Opers.

25fv47 43770 2512158 48183 3407863 80bau3b 63251 2485683 60965 2359507 adlittle 766 4722 696 3594 afiro 182 532 156 386 agg 13634 441952 3864 55022 agg2 25757 1139839 20667 1445573 agg3 25773 1140891 20764 1466546 bandm 5732 99856 4728 80646 beaconfd 3674 76576 3743 89471 blend 1446 18188 1106 10942 bnll 16989 498631 16493 515561 bnI2 104111 15141485 93113 12432841 boeingl 10815 213967 91t2 180654 boeing2 3913 69315 2852 43092 bore3d 232I 34935 1485 16095 brandy 4607 104137 4880 145310 capri 6617 159291 5227 119397 cycle 91492 6390408 65687 2722738 czprob 14314 91390 14316 91426 d2q06c 202389 37733415 177415 24063549 degen2 20190 964688 20918 1071268 degen3 144844 15759422 143421 16304269 e226 5741 99423 6639 180079 etamacro 12241 522869 11969 536035 fffffS00 22250 767814 16761 586227 finnis 7676 106598 5757 82015 fitld 13699 182383 13699 182383 fitlp ao eo 10140 115274 fit2d 129341 1632627 129341 1632627 fit2p co co 50583 481447 forplan 7967 183123 7050 137690 ganges 34264 1474436 22005 496721 g~d-pnc 3915 10027 3915 10055 greenbea 74862 2182032 73225 2143711 greenbeb 73841 2045403 71179 1890588 growl5 11410 200090 13548 300718 grow22 16842 296074 19062 400122 grow7 5202 90394 6157 135351 israel 13757 1092231 4231 99277

R. Fourer, S. Mehrotra / Symmetric indefinite systems

Table A.3---continued 3

Problem Normal Augmented

Nonz. Opers. Nonz. Opers.

kb2 773 8559 632 6068 lotfi 2704 31772 2130 19416 maros 22163 537387 27127 1105025 nesm 34892 1285538 31801 1013611 perold 31454 1841272 31929 2285285 pilot 249632 54449178 221610 41999870 pilot4 21263 1034521 14129 503525 pilotja 69252 8461350 57859 5922530 pilotnov 64588 6540410 57614 5345152 pilot.we 24653 740445 24667 758517 recipe 1195 12383 1343 19015 scl05 800 3158 787 3341 sc205 1610 6594 1624 7444 sc50a 354 1298 328 1126 sc50b 340 1326 282 822 scagr25 4195 22995 3099 12495 scagr7 1063 5355 795 2991 scfxml 6976 111282 6916 140996 scfxm2 14161 229803 13751 267389 scfxm3 21399 351089 20564 392596 scorpion 3357 22509 4008 44202 scrs8 7903 107747 6752 71304 scsdl 3703 34355 3703 34355 scsd6 6714 57640 6714 57640 scsd8 14066 108692 14066 108692 sctapl 4195 34241 3713 29909 sctap2 21170 577222 19264 560018 scrap3 27061 606899 24405 568869 seba 57671 11091389 4644 40048 sharelb 2269 20305 2176 19586 share2b I719 17025 1486 14418 shell 6643 70943 6719 77051 ship041 9830 72074 9774 71550 ship04s 6784 51258 6728 50734 ship081 16084 116840 16067 116519 ship08s 8703 67173 8688 66954 shipl21 21476 158564 21476 158564 ship12s 10656 84412 10656 84412 sierra 19751 302371 20886 311552 stair 20484 1284690 12941 477627 standata 5669 49709 5158 37902 standmps 8301 104977 7966 101556 stocf~rI 1261 9411 1098 8156 stocfor2 36159 543573 28119 410929 tuff 12419 378873 11344 291810 vtp.base 1119 7723 705 2901 woodlp 60224 2287278 60775 2342915 woodw 56021 2208219 56641 2302653

37

38

References

R. Fourer, S. Mehrotra / Symmetric ind~finite systems

[I] I. Adler, N. Karmarkar, M.G.C. Resende and G. Veiga, "An implementation of Karmarkar's algorithm for linear programming," Mathematical Programming 44 (1989) 297 336.

[2] I. Adler, N. Karmarkar, M.G.C. Resende and G. Veiga, "'Data structures and programming techniques for the implementation Karmarkar's algorithm," ORSA Journal on Computing 1 (1989) 84-106.

[3] M. Ariolk I.S. Duff and P.P.M. de Rijk, "On the augmented system approach to sparse least-squares problems," Numerische Mathematik 55 (1989) 667-684.

[4] R.E. Bixby, "Implementing the simplex method : The initial basis," ORSA Journal on Computing 4 (1992) 267-284.

[5] P.T. Boggs, P.D. Domich, J.R. Donaldson and C. WitzgaU, '~Optimal 3-dimensional methods for linear programming," MISTIR 89-4225, Center for Computing and Applied Mathematics, National Institute of Standards and Technology (Gaithersburg, MD, 1989).

[6] /~.. Bj6rck, "Iterative refinement of linear least-squares solutions," BIT 7 (1967) 257-278. [7] J.R. Bunch and B.N. Parlett, "Direct methods for solving symmetric indefinite systems of linear

equations," SIAM Journal on Numerical Analysis 8 ( 197 I) 639-655. [8] I.C. Choi, C.L. Monma and D.F. Shanno, "Further development of a primal-dual interior point

method," ORSA Journal on Computing 2 (1990) 304-311. [9] I.S. Duff, N.I.M. Gould, J.K. Reid, J.A. Scott and K. Turner, "The factorization of sparse symmetric

indefinite matrices," IMA Journal of Numerical Analysis 11 (1991) 181-204. [10] I.S. Duff and J.K. Reid, "The multifrontal solution of indefinite sparse symmetric linear equations,"

A CM Transactions on Mathematical Software 9 (1983) 302-325. [i 1] R. Fourer and S. Mehrotra, "Solving Symmetric Indefinite Systems in an Interior-Point Method for

Linear Programming," Technical Report 92-01, Department of Industrial Engineering and Manage- ment Sciences, Northwestern University (Evanston, IL, 1992).

[12] D.M. Gay, "Electronic mail distribution of linear programming test problems," Committee on Algorithms Newsletter I3 (1985) t0-12.

[13J P.E. Gill, W. Murray, D.B. Poncele6n and M.A. Saunders, "Solving reduced KKT systems in barrier methods for linear and quadratic programming," Technical Report SOL 91-7, Systems Optimization Laboratory, Stanford University (Stanford, CA, 1991).

[14] P.E. Gill, W. Murray, M.A. Saunders, J.A. Tomlin and M.H. Wright, "On projected Newton barrier methods for linear programming," Mathematical Programming 36 (1986) 183-209.

[15] N. Karmarkar, "A new polynomial time algorithm for linear programming," Combinatorica4 (1984) 373-393.

[16] M. Kojima, S. Mizuno and A. Yoshise, "A primal-dual interior point algorithm for linear programming," in: N. Megiddo, ed., Progress in Mathematical Programming: Interior-Point and Related Methods (Springer, New York, 1989) pp. 131-158.

[17] I.J. Lustig, R.E. Marsten and D.F. Shanno, "On implementing Mehrotra's predictor-corrector interior-point method for linear programming," SIAM Journal on Optimization 2 (1992) 435 449.

[18] I.J. Lustig, J.M. Mulvey and T.J. Carpenter, "Formulating two-stage stochastic programs for interior point methods," Operations Research 39 (1991) 757-770.

[19] H.M. Markowitz, "The elimination form of the inverse and its application to linear programming," Management Science 3 (1957) 255-269.

[20] K.A. McShane, C.L. Monma and D.F Shanno, "An implementation of a primal-dual interior point method for linear programming," ORSA Journal on Computing 1 (1989) 70-83.

[21] S. Mehrotra, "On the implementation of a primal dual interior point method," SIAM Journal on Optimization 2 (1992) 575-601.

[22] S. Mehrotra, "Higher order methods and their performance," Technical Report 90-16R1, Depart- ment of Industrial Engineering and Management Sciences, Northwestern University (Evanston, IL, 1991).

[23] S. Mehrotra, "Handling free variables in interior methods," Technical Report 91-06, Department of Industrial Engineering and Management Sciences, Northwestern University (Evanston, IL, 1991).

[24] S. Mehrotra and Y. Ye, "On finding the optimal facet of linear programs," Technical Report 91-10. Department of Industrial Engineering and Management Sciences, Northwestern University (Evans- ton, IL, 1991 ).


[25] R.D.C. Monteiro and I. Adler, "Interior path following primal-dual algorithms. Part I: Linear programming," Mathematical Programming 44 (1989) 27-41.

[26] R.D. Skeel, "Scaling for numerical stability in Gaussian elimination," Journal of the Association for Computing Machinery 26 (1979) 494-526.

[27] K. Turner, "Computing projections for the Karmarkar algorithm," Linear Algebra and its Applica- tions 152 (1991) 141-154.

[28] R.J. Vanderbei, "Splitting dense columns in sparse linear systems," Manuscript, AT&T Bell Labora- tories (Murray Hill, NJ, 1990).

[29] R.J. Vanderbei, "ALPO: another linear program solver," Manuscript, AT&T Bell Laboratories (Murray Hill, NJ, 1990).

[30] R.J. Vanderbei and T.J. Carpenter, "Symmetric indefinite systems for interior-point methods," Technical Report SOR-91-7, Department of Civil Engineering and Operations Research, Princeton University (Princeton, NJ, t991).

solving symmetric indefinite systems in an interior...

Documents