towards a strongly polynomial algorithm for strictly convex quadratic programs: an extension of...

Mathematical Programming 46 (1990) 225-236 225 North-Holland

TOWARDS A STRONGLY POLYNOMIAL ALGORITHM FOR STRICTLY CONVEX QUADRATIC PROGRAMS: AN EXTENSION OF TARDOS' ALGORITHM

Frieda G R A N O T and Jadranka S K O R I N - K A P O V *

Faculty of Commerce and Business Administration, The University of British Columbia, Vancouver, B.C., Canada V6T 1Y8

Received 20 July 1986 Revised manuscript received June 1988

In a recent paper Tardos described a polynomial algorithm for solving linear programming problems in which the number of arithmetic steps depends only on the size of the numbers in the constraint matrix and is independent of the size of the numbers in the right hand side and the cost coefficients. In this paper we extend Tardos' results and present a polynomial algorithm for solving strictly convex quadratic programming problems in which the number of arithmetic steps is independent of the size of the numbers in the right hand side and the linear cost coefficients.

Key words: Quadratic programming, strong polynomiality.

1. Introduction

Many problems in economics , statistics and numerica l analysis can be formula ted

as the min imiza t ion of a convex quadrat ic func t ion over a polyhedral set. Moreover,

some algori thms for solving large scale mathemat ica l p rogramming problems mini-

mize a quadra t ic funct ion over a polyhedral set as a subrout ine , e.g., Held et al.

[8], K e n n i n g t o n and Shalaby [10]. The n u m b e r of different algori thms suggested

for min imiz ing a convex quadrat ic funct ion over a polyhedral set is large and we

will not at tempt to survey them here, rather we will restrict our a t tent ion to

po lynomia l ly b o u n d e d algorithms.

A polynomia l algori thm for solving convex quadrat ic p rogramming problems was

first developed by Kozlov, Tarasov and Khachiyan [11]; Helgason, K e n n i n g t o n and

Lall [9] gave a po lynomia l ly b o u n d e d algori thm for a quadrat ic program with a

single constra int while Minoux [12] gave a po lynomia l algori thm for m i n i m u m

quadra t ic cost flow problems. In all the above algori thms, excluding [9], the n u m b e r

of ar i thmetic steps depends on the size of the inpu t numbers , where the size of a

rat ional n u m b e r p~ q is given by [logs(p + 1) ] + [logs(q + 1) ] + l with [a ] being the

smallest integer greater than or equal to a real n u m b e r a. The size of a rat ional

vector or matrix is the sum of the sizes of its entries.

This research was partially supported by the Natural Sciences and Engineering Research Council of Canada Grant 5-83998.

* Currently at W.A. Harriman School for Management and Policy, State University of New York at Stony Brook, Stony Brook, NY, USA.

226 F. Granot, J. Skorin-Kapov / Convex quadratic programs

In [13] Tardos was the first to present a polynomial algorithm for solving linear programming problems in which the number of arithmetic steps depends only on the size of the numbers in the constraint matrix and is independent of the size of the numbers in the right hand side and the cost coefficients.

The aim of this paper is to extend Tardos ' results to convex quadratic programming problems of the form Max{cVx-½xTDx: Ax<~b, x~>0} with D being a positive definite matrix. We assume, without loss of generality, that A and D are integral. We develop a polynomially bounded algorithm for solving the strictly convex quadratic problem where the number of arithmetic steps is independent of c and b but depends on the size of the entries of the matrices A and D. I f in particular the size of the entries in A and D is polynomially bounded in the dimension of the

input, the algorithm is strongly polynomial, e.g, when the quadratic term corresponds to least squares and A is a node arc incidence matrix of a directed graph.

Following Tardos [13] the algorithm presented here finds optimal primal and dual solutions to the quadratic programming problem (if they exist) by solving a sequence of simple quadratic programming problems using the polynomial algorithm for solving quadratic programming problems given in [11] and by checking feasibility of a linear system in time independent of the right hand side using Tardos ' feasibility

algorithm [13].

2. Preliminaries and notation

For simplicity of exposition we will first consider the quadratic programming

problem of the form

Max{cVx -½xTDx: Ax = b, x ~ 0} (1)

where A is an integral m × n matrix with rank(A) = m; D is an integral n × n positive definite matrix; c and x are n-vectors and b is an m-vector.

The validity of the algorithm for quadratic programming problems with inequality constraints will be discussed in Section 4.

Using the fact that D is nonsingular we can substitute z = x - D ~c in (1) resulting

with the following equivalent problem

½cTD l c + M a x { - l z T D z : A z = b - A D - l c , I z - I x = - D - l c , x>~O}. (2)

Recall that positive definiteness of D implies that the objective function of (1) is strictly concave which in turn implies the uniqueness of the optimal solution of (1) (if one exists). Morover, if the set {x: Ax = b, x >~ 0} is not empty, (1) will be bounded. The uniqueness of the optimal value of x implies also the uniqueness of the optimal

value of z; however observe that z is not restricted to be non-negative any longer.

Using now Dorn's duality (see, e.g., [4, 2]), we can state a dual of the maximization

problem given in (2) as

Min{yT(b-AD lc)--vTD 1C+½zTDz: ATy+ Iv+ Dz=O, v~<0}. (3)

F. Granot, J. Skorin-Kapov / Convex quadratic programs 227

Substituting v = - D z - AVy results with

Min{yf(b - A D 1 c) + (yTA + zTD) D 1 c + ½zVDz: A f y + Dz >1 0}

= M i n { y V b - y V A D l c + y V A D lc+zTc+½zTDz: AVy+Dz>~O}.

Finally, after adding slack variables s, we get a dual of (2) of the form

½cTD 1C + Min{cVz + bTy +½zTDz: ATy + Dz - Is = O, s >! 0}. (4)

It is easy to see that replacing z by x - D - I c will give us the following dual to (1),

Min{bTy +½xT Dx: A f y + Dx - Is = c, s >! 0}. (5)

It is important to note that the same slack variables appear both in (4) and (5).

Since the algorithm to be described in the following is significantly simpler when applied to problems with zero right hand side, we will always use the above transformation to replace a pair of primal and dual problems of the form (1) and (5) by an equivalent pair of the form (2) and (4).

Recall that the Karush-Kuhn-Tucker optimality conditions for a pair of problems (1) and (5) will have the form

(i) 0 0) A f - I x,s>~O and xTs O. (6)

At each iteration of the algorithm to follow we will detect at least one new slack variable which is equal to zero in all optimal pairs of primal and dual solutions for the pair of problems (1) and (5). Or equivalently, at each iteration we will detect

at least one new dual constraint which is tight at optimality. After each iteration we will add constraints of the form s; = 0, i c I (where I is the set of slack variables detected to be zero at the current iteration) to the linear system given in (6) and perform Tardos ' feasibility algorithm which will give us a feasible solution (x T, yf , ST). We will check if xXs = 0. If so, the algorithm will terminate since an

optimal pair of primal and dual solutions was determined. If on the other hand xTs ¢ O, we will perform another iteration of our algorithm. In at most n iterations we will find a pair of optimal primal and dual solutions.

As stated above the algorithm will be applied to a minimization problem of the form

Min{c f z+bVy+½zrDz: D z + A X y - l s 0, s~>0}. (7)

Before stating the algorithm we will give two preliminary lemmas. The first one is a direct generalization of Lemma 0.1 in [13], while the second is a special case of Lemma 2 in [5, p. 707].

Lemma 2.1. Replacing (c x ,b x,0 x) in (7) by (c ' t , b ' x , a ' x ) = ( c 3`,b 1,0 .5̀ ) -

pX(D, A T, I) for some n-vector p will not change the set of optimal solutions.

Proof. If (£x, 37T, gf) solves (7), then it also solves the problem obtained from (7) by replacing the linear cost coefficient (c x, b x, 0 x) by (c 'x, b 'T, a 'v) and vice versa.


This since the objective function value remains unchanged. Indeed

( c 'v, b ,T, a ,T) + ½zTDz __ ( C T, b T, 0 T ) _ l z T D z

= --pT(D, A T, - I ) = --pTO = O. []

Lemma 2.2. I f (~T, fiT gT) solves (7), then (a~ T, a~ T, OgS T) solves the quadrat ic problem

in which ( c T, b T, 0 T) in (7) is replaced by ce( c T, b T, 0 T) f o r any scalar a > O.

Proof. See Eaves [5]. []

We are now ready to describe the polynomial quadratic programming algorithm whose number of arithmetic steps is independent of the size of the numbers in the vectors c and b.

3. The quadratic programming algorithm

The quadratic programming algorithm (QPA) described below is a direct generalization to a class of strictly quadratic programming problems of Tardos' linear programming algorithm. It uses as input a strictly convex quadratic program (1), Tardos' feasibility algorithm, which is a polynomial algorithm to check the feasibility of a system of linear inequalities in time independent of the right hand side and if feasible it generates a basic solution, and a polynomial algorithm for solving convex quadratic programming problems, e.g., Kozlov et al.'s algorithm [11]. The output from QPA is a pair of optimal primal and dual solutions for (1).

Let us denote, A = Max{Idet HI: for all square submatrices H of (/9, A v, - I )} .

The quadratic programming algorithm. Step 1. Use Tardos' feasibility algorithm to check whether { A x = b, x>l 0} is

feasible. If not, terminate since (1) is infeasible. If feasible, then the positive definiteness of D guarantees boundedness which in turn implies that the dual constraint set is feasible. Set K = 0.

Step 2. Let D ° x + A ° T y - E ° s = c o denote the equality system D x + A T y - Is = c

together with si = 0 for i c K and let p0 = {(x T, yT, sT): A x = b, D x + ATy -- IS = C,

si = 0 for i~ K, x ~> 0, s ~> 0}. Use Tardos' feasibility algorithm to find a point say (~T,)TT, gT) in pO. If ffTg = 0 terminate with (fiT, ~gT, gT) as an optimal solution to (1) and (5).


Step 3. Find the projection (c 'T, b 'T, a 'T) of (c T, b T, 0 T) onto the null space of (D °, A °T, -E° ) . Since the rows are linearly independent, this can be done using Gaussian elimination. Recall that for a matrix G with full row rank, the projection onto its null space {x: Gx=0} is determined by the idempotent matrix P--- I - G T ( G G T) 1G. I.e., for some vector x its projection onto the null space of G is Px since G ( P x ) = G ( x - G T ( G G T ) - I G x ) = G x - G x = O where P x = x - GT( GGT)-I Gx = x - GTp and p =- ( GGT)-~ Gx. Applying this to our case we obtain

(C tT, b 'T, a 'T) = (e T, b T, 0 T) - pT(D° , a °v, -E° ) .

If (c 'T, b 'T, a 'T) = 0 then solve the problem Min{½zTDz: D°z + A ° T y - E°s = O, s >~ 0} by Kozlov et al.'s [11] algorithm to obtain an optimal solution (xT=zT+ c T D - 1 ' y T sT).

Step 4. Let

ce = (3n + m)(Zn + m)A/l](e 'T, b 'T, a'T) IIo~

where A is a maximal absolute subdeterminant of (/9, A T, - I ) ,

6,= [ctc;], i = l , . . . , n , /~=[ceb;], i = l , . . . , m ,

a i=[~a;] , i = l , . . . , n .

Step 5. Use Kozlov et al.'s [11] algorithm to find an optimal solution (~T, S ) to

Max{-½zTDz: D°~rv - Dz - 6, A°v - b, - -E° fv <~ ~}

which is a dual of

T AOVy _ Min{~rz+bTy+fi~rs+~z Dz D ° z + E ° s = 0 , s~>0}. (8)

Let

I = {i: (--EOTI~)i ~ ~'a~--(2n + m)A}.

Add the set I to K and go to Step 2.

The following lemmas which are extensions of corresponding lemmas in [13] will be used in order to verify the validity of QPA.

Lemma 3.1. Let (2"r, yT, gT) be an optimal solution to (8) where (gT, 6v, ~/T)= r(c T, b v, aT)] of some vector (c z, b T, a T) ¢ ~"+'+", D is' an integral n x n positive

definite matrix and D °, A ° and E ° are integral matrices of appropriate dimensions. Let (~5 T, 2T) be an optimal dual solution for (8). Then for any optimal solution (~T, yT, ~T) to

T I T A O m y _ = M i n { c T z + b T y + a s+~z Dz: D ° z + E°s 0, s>~0} (9)

(i.e., "unrounded" problem), we have

( - E ° T f ) ~ < ~ a i - ( 2 n + m ) A implies gi=0. (10)


Proof. The dual of (8) has the fol lowing set o f constraints

D ° T v - D z = ~ < ~ c + e , A ° v = 6 < ~ b + e , --E°Tv<~a<~a+e,

where e is a vector of ones of appropr ia te dimension. Moreover , since 03 T, 2T) is

an opt imal dual solut ion for (8) and (2T, 37T, f ) an opt imal pr imal solution, we have f rom complemen ta ry slackness that g T ( a + E ° T ~ ) = 0. Coupl ing with the fact

that a <~ a we obtain

( -E°T~5)i<ai<~a~ implies g~=0. (11)

Now, suppose that (2 T, )~W, gT) is an opt imal solut ion to the " u n r o u n d e d " p rob lem (9). Since ( S , 37T, gT) is a feasible solution for (9) we have

cT(2_2)+bT( f i_ f i )+aT(~_g) - - l ^T , - ' .~ , - T ~ - ~ -t~Z UZ--~Z u z ~ O . (12)

Substi tut ing u = 2 - 2, u 1 = 3~ -37, u 2 = g - g in (12) and rearranging terms results with

cTu + bTu 1 + aTu2+½uTDu + 2TDu <~ O. (13)

Since ½uTDu >~ 0 for each u, we obtain f rom (13) that

cTu q- bTu 1 + aTue + 7.TDu ~ O. (14)

We will p rove the validi ty of (10) by contradict ion. Suppose that (10) is not valid, i.e., there exists an index i for which (--E°T~5)i ~ a~ -- (2n + rn)A and gi > 0. N o w

since by the opt imal i ty condi t ions gi = 0 we have that : u~ > 0 . Moreover , u)~>0 for all j with si = 0. I f for some j, u 2 < 0, then ~ > 0 which on the other hand implies (see (11)) that (--E°T#)j ~> aJ or - a J - (E°Tv)j ~ 0. Now observe that the vectors (u T, u ~T, u zT) satisfies the system D°u + A°Tu ~ - E°u 2= 0. Thus there also exists an integral basic solut ion to D°u + A°Tu ~ -- E°u 2 = O, 2 u i > 0 a n d u ~ / > 0 f o r j w i t h ~ = 0 that satisfies (14). Denote this solut ion (a T, a~T, a2T) which by Cram6r ' s rule satisfies II(a T, a 1T, tTzw)lloo ~< a. No te that for the dual constraints D ° V v - Dz = 6 and A°v = 6

cor responding to uncons t ra ined pr imal var iables z and y we have c <~ D°Tv -- Dz <~

c + e and b<~A°v<~b+e which in turn implies [ I - c+D°Vv-DzHoo~a , [ l - b + A°v [[oo <~ 1, and for j with u~ < 0 also I ( - a T - - v T E ° ) j l <~ 1. Combin ing the above facts and the condi t ions on (fiT, a~T, ~:V) we obta in f rom (14) that

0 ~ - - c T t i - - b T t l I - - a T u 2 - - 2 T D u

= ( - c v - eTD)ft -- bTo 1 _ aTft:

= (--C T - ~TD) t7 -- bTt~ 1 _ aTft2+ eV(D°a + A°Ta I _ EOa 2)

= ( - c T - eVD + eVD °) a + ( - b v + eVA° ~) a ~ + ( - a ~ - U E o) a :

<~ E I(--cT--zTD+eTDO)d 10~l+ ~ I(--bT+~TA0~).[ IGI k--1 k = l

+ ~ I(--aT--~TEO)kl [~2]+(--aT--~TEO)ia2 k~l ,k~i

<~ ~ IGI + ~ l a l l - ( 2 n + m ) A + ~ la2J k--1 k = l k~l ,k~i

F. Granot, J. Skorin-Kapov / Convex quadratic programs

<~ nA + mA--(2n + m)A +(n-- 1)A = - - A

which is a cont rad ic t ion .

231

Before s tat ing the fo l lowing l emma (which will ensure the finiteness o f the

a lgor i thm) , we will briefly expla in the ideas used in the proof .

C o n s i d e r a full row rank matr ix H o f the form

and a vec tor x o f the same d imens ion as the co lumns o f /4. Let us par t i t ion

x T = (xT, x~) co r r e spond ing to the pa r t i t ion o f H to H~ and H2. Then the p ro j ec t ion t T tTx x~ , x2 ) o f x v onto the null space o f H is given by (0 w, x~W). In our case this impl ies

that wheneve r s k = 0 is a row of D ° z + A ° T y - E ° s = O for k c { 1 , . . . , n}, the com-

p o n e n t a~ o f the p ro jec t ion a ' de t e rmined in Step 3 is equal to zero.

Next , if x = GTu for some full row rank mat r ix G its p ro j ec t ion to the null space

o f G is zero since

Px ( I - G T ( G G 1) ~ G ) x = G r u - G ' ( G G r) ~ G G T u = G T u - G ' u = O .

Also if the vector y is a pro jec t ion o f some vector w (i.e., y = Pw) onto the null

space o f G, then the p ro jec t ion of y onto the null space of G is the same vector y.

This is true since the p ro jec t ion matr ix P I - G T ( G G r) ~G is i n d e m p o t e n t (i.e.,

P. P = P) and there fore P), = P( Pw) Pw = y. Thus for a vector d = c' + A rt, where

c' is a p ro jec t ion of some vector c onto the null space of a full row rank matr ix A,

Pd = c'.

Lemma 3.2. The set l Jo u n d in Step 5 o f Q P A contains at least one index i sueh that

,si = 0 is not a row o f D ° z + A°Vy - E°s -- O. ( Recall that the latter sys tem is equivalent

to D ° x + A ° S y - E° s = c ° under the tran~sjbrmation z = x - D ~c.)

Proof. The p r o o f will fo l low by cons t ruc t ing a vec tor d and looking at its max-norm.

We will use the no ta t ion f rom Step 4 and Step 5 of Q P A and d is t inguish be tween

two cases.

Case 1. If 5 is not a zero vector, then define the (3n + m) vector d as follows:

d i = - o z c ~ + l S f D ° - f ~ D i , i = 1 , . . . , n,

dn~., = - c e b ; + ~rA~,'r, j 1 , . . . , m,

0 if sk = 0 is a row of D ° z + A ° S y - E ° s = O , dtz+tn+ k = _ o ~ a k~ _ - T E O v k o therwise for k = 1 , . . . , n,

d2 ,+, ,~ ,=(De) j l lD5 .]]~ , , 1 1 , . . . , n .

Since v, z are dual var iables for the " r o u n d e d " p rob l e m (8) we have

oec '-< - D ° T ~ - D 5 . = ¢7= [~C'] ~< c~c'+ 1,

c~b'~ < A%5 = b = loeb'] ~< c~b'+ 1,

- E ° V ~ < ~ a = [ c m ' ] ~ a a ' + l .


Therefore 0 <~ - a c ' + D°Tz5 -- Dff <~ 1, 0 <~ - a b ' + A°~ ~< 1 and - ~ a ' - E°T13 ~< 1. For the

last n componen t s of d we know that - 1 ~< (De) , / I IDelI~<~ 1 for l = L . . . , n. N o w

since all the componen t s of d are b o u n d e d by 1 f r o m above and, with the except ion of componen t s d , + m + ~ , . . . , d2,+m all others are b o u n d e d f rom below by - 1 , the validi ty of the L e m m a will fol low if we can show that H d Iloo ~> (2n + m)A.

To that end note that d can be writ ten as

(o:) - b r d = c ~ , +

A °T - E ° 0 ) T [ ~ )

0 0 ( 1 / I I D e G ) D I \ e l "

It is easy to see that the project ion of d onto the null space of

0 ) A = 0 0 (a/llDelloo)D

is given by the vector c~ = a ( - c 'T, - b 'T, - a 'T, -JlP5[[ooc 'T) (i.e. /qd = 0).

Note also that [l(c 'T, b 'T, a'T)]]~<~ ]](C 'T, b 'y, a 'T, ]]D~ll~oc'V)l]~o and that for any

n-vector x, [Ixll~ >>- (1 /n) l lx l l 2 and IlxII2>~ IIxltoo are valid. Next, since - a ( c 'y, b 'T, a 'r, [ID~llooc 'T) is a project ion of d,

[Id 112 ~> al](c 'T, b 'T, a 'T, [[ D ~ [ [ o o c ' T ) ]12 .

Therefore ,

I ld l l~> (1 / (3n + m))lldl]2

( 3 n + m ) ( 2 n + m ) A

(3n + m)[l(c 'T, b 'T, a'~) I1~

Case 2.

0 dn+.,+k = { - o~a~, -- ~TE°

or, writ ten in matr ix form

(c / d = a - b ' + ( D o A °T --E°)T~3.

\ - - a ' J

The p r o o f now follows a long the same lines as for Case 1.

ii( c ,T, b 'T, a ,T, IID~_llo~c'~r)[[2

rT (2n + m)a II ( c'T, b'T, a , II Ozlloo c'T) 112 >~ (2n + m)A. ]](c 'T, b 'T, a 'T, 11D:g[Iooc'T) [[~

I f ff is a zero vector, then define the (2n + m) vector d as follows:

di = -ac ' i + ~T D °, i = l , . . . , n,

dn+j = r , - - T ,t0T - o e o j ± v z% , j = l , . . . , m ,

if sk = 0 is a row of D ° z + A°Ty - E°s = O,

otherwise for k = l , . . . , n,

[]

F. Granot, 3". Skorin-Kapov / Convex quadratic programs 233

Lemma 3.3. Every optimal pair of primal and dual variables (2T, ~T, gT) satisfies ~ = 0

for i c L

Proof. By Lemma 2.1 replacing (c T, b T, 0 T) by (c '~, b 'T, a 'T) will not change the set of optimal solutions (2 T, f r , gT) for Min{cTz + b Ty + ½zT Dz: D°z + A°Ty -- E°s = 0 °,

s ~>0}. By Lemma 2.2 multiplying the linear part of the objective function by a positive scalar c~, one obtains that the set of optimal solutions (c~ T, c07 T, ag T) and the set of variables that are equal to zero in all optimal solutions is unchanged. Finally, Lemma 3.1 holds with c replaced by crc', b replaced by ceb', and a replaced

by aa ' . []

Lemma 3.4. After at most n iterations of QPA one gets a pair of optimal primal and

dual solutions for problem (1).

Proof. By Lemma 3.3 adding constraints si = 0, i ~ I where I was determined in Step 5 of QPA does not affect the set of optimal solutions (~T, f r , gT). By Lemma

3.2 no more than n iterations of QPA are possible. (In the worst case, where all £, > 0 and Tardos ' feasibility algorithm in every iteraction "missed" the desired face, i.e., face with x rs = 0, and where 1 is a singleton in every iteration, one will have

exactly n iterations.) []

Let us now calculate the complexity of QPA. Denote by T(A) the complexity of Tardos ' feasibility algorithm when applied to the system {Ax = b, x >10}, and by

T( A, D) its complexity when applied to the system { Ax = b, Dx + A ~y - Is - c , x >~ 0, s>~O,s~ Ofor ic K} .Deno teby K(A ,D) thecomplex i t yo fKoz love ta l . ' sa lgor i thm [11]. Note that we apply Kozlov et al.'s algorithm only to quadratic programs for which the linear part of the objective function is integral and polynomially bounded

by the matrices A and D and with right hand side vector which is zero.

Theorem 3.5. The quadratic programming algorithm has running time polynomial in the size of the matrices A and D and independent of the sizes of the vectors c and b.

It runs in

O ( n ( 2 n + m ) 3 + n ( 2 n + m ) l o g ( 2 n + m ) ( 3 n + m ) A + T ( A ) + n T ( A , D ) + n K ( A , D)).

Proof. Step 1 takes T(A) time (i.e., time of Tardos ' feasibility algorithm when

applied to a linear system with constraint matrix A). Consequently Step 2 takes T(A, D) plus at most 2n comparisons to verify xVs = 0. The Gaussian elimination in Step 3 takes O((2n + m) 3) time (see [6]). Step 4 takes O((2n + m) log(2n + m) x ( 3 n + m ) A ) comparisons to find (gT, /~T ~) since II(~ T, U, c i T ) l l ~ = ( 2 n + m ) x

(3n + m)A and one can use binary search to obtain (g T, ~r, tiv). Step 5 takes K(A, D) time. Finally, we need at most n iterations; therefore the claimed complexity

follows. []


Recall that an algorithm is termed strongly polynomial (see, e.g., [7]) if all its operations consist of additions, comparisons, multiplications and divisions and if the number of such steps is polynomially bounded in the dimension of the input, where the dimension of the input is the number of data in the input. Further, when the algorithm is applied to rational input, then the size of the numbers occurring during the algorithm is polynomially bounded in the dimension of the input and the size of the input numbers.

Thus the polynomial algorithm described in this paper becomes strongly polynomial if the size of the entries in A and D are polynomially bounded in the dimension of the data. This clearly provides a strongly polynomial algorithm for, e.g., problems where one minimizes the norm over flow (transportation) type constraints [1, 3], or over multiple choice type constraints [9].

4. The inequality constraint case

The algorithm can be generalized in a straightforward way to work on strictly convex quadratic programs with inequality constraints. To show this consider the quadratic programming problem of the form

Max{crx -½xVDx: Ax <~ b, x >10} (15)

with the same assumptions on the dimension of the problem and the input data as for problem (1).

The dual of (15) has the form

Min{bTy +½xTDx: ATy -- Dx >~ c, y >1 0}. (16)

Using the fact that D is positive definite and applying the transformation z = x - D 1c we obtain the equivalent pair of primal and dual problems

½cTD-lc+Max{--½zTDz: Az+ Iw = b - A D lc,

I z - I x = - D lc, x>~O,w>~O} (17)

and

½cTD lc+Min{cTz+bTy+½zTDz: ATy+Dz- I s=O,y>~O,s~>O} . (18)

Note that an optimal solution (fiT,)sv, j ) for (18) provides an optimal solution x = f + D - l c for (15) and an optimal solution ()~T, fiT) for (16).

The Karush-Kuhn-Tucker optimality conditions for the pair of primal-dual problems (15), (16) are

(i) t:t o i o i ) = A T 0 ' (19)

x ,y , w,s />0 and xVs=O, yVw=O.


In this case, at each iteration the algorithm will detect at least one new variable

y or s which has to equal zero in all optimal pairs of primal and dual solutions for

(15) and (16). After each iteration one will perform Tardos' feasibility algorithm to detect a basic point from the linear system (19). The conditions xTs = 0 and yVw = 0

will then be checked. If satisfied the algorithm will terminate since an optimal pair

of primal and dual solutions were found. Otherwise, another iteration of QPA will

be performed. Instead of problem (7) (in the equality case), we work with the problem

Min{cWz + bWy + ½zTDz: Dz + ATy -- Is = O, y >1 0, s 1> 0}. (20)

The algorithm will have the same form, except that now Step 1 will read "Set K1 = 0

and K 2 = 0 ' ' , in Step 2 the system { D ° x + A ° X y - E ° s = e °, y>~O, s~>0} will be equivalent to the system {Dx + ATy -- IS = c, y/> 0, s ~> 0, si = 0 for i c K1 and yj = 0

for j c K2} , and the polyhedron pO will be

pO= (x x ,yx ,w T,sv): A T 0 - I -- '

1 x~>0, y~>0, w~>0, s~>0, s~=0 for i 6 K ~ , y i = 0 for j~K2 I .

J

Step 5 will now read: "Use Kozlov et al.'s algorithm to find an optimal solution (~v Z T) to

Max{-½z~Dz: D ° V v - D z - ~, A°v<~ b, -E°rv<~ d}

which is the dual of

Min{(Vz +/~Ty + ~tVS +½z~Dz: DOz+ AOTy _ EOs = O, y ~> O, s/> 0}.

Let

and

I = {i: (--E°Wts)i~< c~al (2n + m)A),

J = {j: (A°~)i ~ (xb'j - (2n + m)A}.

Add the set I to K~, the set J to K2 and go to Step 2." The complexity of the algorithm will be affected in the sense that at most (n + m)

iterations are now possible.

Acknowledgement

We are grateful to a referee for helpful comments on an earlier version of this paper.

236

References

F. Granot, J. Skorin-Kapov / Convex quadratic programs

[1] A. Bachem and B. Korte, "An algorithm for quadratic optimization over transportation polytopes," Zeitschrift fiir Angewandte Mathematik und Mechanik 58 (1978) 459-461.

[2] R.W. Cottle, "Symmetric dual quadratic programs," Quarterly of Applied Mathematics 21 (1963) 237-243.

[3] J.L Debiesse and G. Matignon, "Comparison of different methods for the calculation of traffic matrices," Annales des Tdldcommunications 35 (1980) 91-102.

[4] W.S. Dorn, "Duality in quadratic programming," Quarterly of Applied Mathematics" 18 (1960) 155-162.

[5] B.C. Eaves, "On quadratic programming," Management Science 17 (11) (1971) 698-711. [6] J. Edmonds, "System of distinct representatives and linear algebra," Journal of Research of the

National Bureau of Standards, Section B 71 (1967) 241-245. [7] M. Gr6tschel, L Lov~sz and A. Schrijver, Geometric Algorithms and Combinatorial Optimization

(Springer, Berlin, 1988). [8] M. Held, P. Wolfe and H. Crowder, "Validation of subgradient optimization," Mathematical

Programming 6 (1974) 62-88. [9] R. Helgason, J. Kennington and H. Lall, "A polynomially bounded algorithm for a singly constrained

quadratic program," Mathematical Programming 18 (1980) 338-343. [10] J. Kennington and M. Shalaby, "An effective subgradient procedure for minimal cost multi-

commodity flow problems," Management Science 23 (1977) 994-1004. [11] M.K. Kozlov, S.P. Tarasov and L.G. Khachiyan, "Polynomial solvability of convex quadratic

programming," Soviet Mathmaties Doklady 20 (1979) 1108-1111. [ 12] M. Minoux, "A polynomial algorithm for minimum quadratic cost flow problems," European Journal

of Operational Research 18 (1984) 377-387. [13] Eva Tardos, "A strongly polynomial algorithm to solve combinatorial linear programs," Operations

Research 34 (2) (1986) 250-256.

towards a strongly polynomial algorithm for strictly convex quadratic programs: an extension of...

Documents