On the classical logarithmic barrierfunction method for a class ofsmooth convex programming

problems

Report 90-28

D. den HertogC. Roos

T. Terlaky

Technische Universiteit DelftDelft University of Technology

Faculteit der Technische Wiskunde en InformaticaFaculty of Technical Mathematics and Informatics

ISSN 0922-5641

Copyright c 1990 by the Faculty of Technical Mathematics and Informatics, Delft, TheNetherlands.No part of this Journal may be reproduced in any form, by print, photoprint, microfilm,or any other means without permission from the Faculty of Technical Mathematics andInformatics, Delft University of Technology, The Netherlands.

Copies of these reports may be obtained from the bureau of the Faculty of TechnicalMathematics and Informatics, Julianalaan 132, 2628 BL Delft, phone+3115784568.A selection of these reports is available in PostScript form at the Faculty’s anonymousftp-site. They are located in the directory /pub/publications/tech-reports at ftp.twi.tudelft.nl

DELFT UNIVERSITY OF TECHNOLOGYREPORT 90{28A LARGE{STEP ANALYTIC CENTERMETHODFOR A CLASS OF SMOOTH CONVEXPROGRAMMING PROBLEMSD. den Hertog, C. Roos, T. Terlaky

ISSN 0922{5641Reports of the Faculty of Technical Mathematics and Informatics 90{28Delft 1990i

D. den Hertog, C. Roos and T. Terlaky,Faculty of Technical Mathematics and Informatics, Delft University of Technology, P.O. Box 5031,2600 GA Delft, The Netherlands.This work is completed with the support of a research grant from SHELL.The fourth author is on leave from the E�otv�os University, Budapest, and partially supported byOTKA No. 2116.

Copyright c 1990 by Faculty of Technical Mathematics and Infor-matics, Delft, The Netherlands.No part of this Journal may be reproduced in any form, by print,photoprint, micro�lm or any other means without written permis-sion from Faculty of Technical Mathematics and Informatics, DelftUniversity of Technology, The Netherlands. ii

AbstractIn this paper we propose a large{step analytic center method for smooth convex programming. Themethod is a natural implementation of the classical method of centers. It is assumed that theobjective and constraint functions ful�l the socalled Relative Lipschitz Condition, with Lipschitzconstant M > 0. A great advantage of the method, above the existing path{following methods, isthat the steps can be made long by performing linesearches.In our method we do linesearches along the Newton direction with respect to a strictly convexpotential function if we are far away from the central path. If we are su�ciently close to this pathwe update a lower bound for the optimal value. We prove that the number of iterations required bythe algorithm to converge to an �{optimal solution is O((1+M2)pnj ln �j) or O((1 +M2)nj ln �j),dependent on the updating scheme for the lower bound.Key Words: Convex programming, analytic center method, Newton method.

1 IntroductionSince Karmarkar [6] presented his projective method for the solution of the linear programmingproblem in 1984, many other variants have been developed by researchers. Among them are thelarge{step path{following methods such as proposed by Roos and Vial [9], Gonzaga [3], and DenHertog, Roos and Terlaky [1], and the potential reduction methods such as proposed by Ye [12],Freund [2], and Gonzaga [4]. The advantages of these methods are that they don't use projectivetransformations as the projective methods do, and that they don't need to follow the socalledcentral path closely, contrary to the small{step path{following methods.In Jarre [5] and Mehrota and Sun [8] small{step path{following algorithms are proposed for smoothconvex programming problems. Again, the great disadvantage of these methods is that they arebased on very small stepsizes to remain in the vicinity of the central trajectory. This characteristicmakes these methods unattractable for practical use. To accelerate his method, Jarre proposed a(higher order) extrapolation scheme.In this paper we propose a large{step path{following method for smooth convex programmingproblems, which ful�l the socalled Relative Lipschitz Condition. Jarre [5] also uses this condition.Our method is a generalization of our method for linear programming in [1] and is also based onJarre's paper.In our method we do a linesearch along the Newton direction with respect to a certain strictlyconvex potential function. If we are close to the current analytic center we update the lower boundsomehow, whereafter we do linesearches aiming at getting close to the analytic center associatedwith the new lower bound. We proof that after a linesearch the potential value reduces at leastwith a certain constant. Using this result, we proof that the number of iterations required by thealgorithm to converge to an �{optimal solution is bounded by a polynomial in �, the dimension ofthe problem and the Lipschitz constant.We note that Kojima, Mizuno and Yoshise [7] already proposed a primal{dual potential reductionalgorithm for for linear complementarity problems. To our knowledge our algorithm is the �rstlarge{step algorithm for (a class of) smooth convex programming. Our algorithm can also beviewed as a natural implementation of the classical method of centers. In a coming report we willdeal with a natural implementation of the logarithmic barrier function method.This paper is organized as follows. In Section 2 we will do some preliminary work. In Section 3we describe our algorithm. Then, in Section 4 we prove some lemmas, needed for the convergenceanalysis in Section 5.2 PreliminariesWe consider the primal formulation of the smooth convex programming problem:(CP ) min ff0(y) : y 2 Fg;where F denotes the feasible region, which is given byF := fy 2 IRm : fi(y) � 0; 1 � i � ng;1

the functions fi(y), 0 � i � n, are convex functions with continuous �rst and second orderderivatives in F . We assume an additional smoothness condition, namely that the Hessian matrixof fi(y), 0 � i � n, ful�ls the socalled Relative Lipschitz Condition, which will be speci�ed lateron.Moreover, we suppose that the interior of the feasible region F , denoted as F 0, is nonempty andbounded. This assumption is not essential. Without loss of generality we further assume thatf0(y) is linear, i.e. f0(y) = �bTy. If this is not true, one may introduce an additional variableym+1, an additional constraint f0(y)� ym+1 � 0, and minimize ym+1. Consequently, the problemunder consideration becomes (CP ) max fbTy : y 2 Fg:Wolfe's [11] formulation of the dual problem associated with this primal problem is(D) 8>>>><>>>>: min bTy �Pni=1 uifi(y)Pni=1 ui 5 fi(y) = bui � 0:Note that there is no symmetry between the primal and dual problem, as in linear programming,because the dual problem (D) contains both y and u variables. Moreover, the dual problem is notnecessarily convex!However, it is a well{known result that if y is a feasible solution of (CP ) and (y; u) is a feasiblesolution of the dual problem (D), thenbTy � bTy � nXi=1 uifi(y):Due to the assumption that F 0 is nonempty, the Slater condition is satis�ed, and hence (D) has aminimum solution, and the extremum values are equal.We associate the following potential function to (CP )�(y; z) = �q ln(bTy � z)� nXi=1 ln(�fi(y));where z is a lower bound for the optimal value z�, and q a positive integer value, which will bediscussed later on. For q = n this potential function is exactly the same as the one used by Jarre[5].It can be proved that �(y; z) is strictly convex on its domain F (see Jarre [5], p. 8). It alsotakes in�nite values on the boundary of the feasible set. Hence this potential function achievesthe minimal value in its domain (for �xed z) at a unique point, which is denoted by y(z). Thenecessary and su�cient Karush{Kuhn{Tucker conditions for these minima are:8>>>><>>>>: fi(y) � 0; 1 � i � n;Pni=1 uirfi(y) = b; u � 0;�fi(y)ui = bT y�zq ; 1 � i � n: (1)2

Using this it can easily be veri�ed that y(z) lies on the socalled central path of the problem, whichis the set of analytic centers for F \ fy : bTy � �g, where � varies from �1 to z�.We can rewrite �(y; z) as �(y; z) = � n+qXi=1 ln(�fi(y));where �fi(y) = bTy� z for n+1 � i � n+ q. The �rst and second order derivatives of �(y; z) aregiven by g(y; z) := r�(y; z) = n+qXi=1 rfi(y)�fi(y)and H(y; z) := r2�(y; z) = n+qXi=1 "r2fi(y)�fi(y) + rfi(y)rfi(y)Tfi(y)2 # :If no confusion is possible we will write, for shortness sake, g and H instead of g(y; z) and H(y; z).In the sequel of this paper we will also use the quadratic approximation qy(x; z) for �(y; z) whenx is near the point y, de�ned asqy(x; z) := �(y; z) + gT (x� y) + 12(x� y)TH(x� y):We will use the H{norm k:kH to measure closeness of points, and especially closeness to the centraltrajectory. The de�nition of this norm is as follows:kxkH = pxTHx:Because H is positive de�nite, k:kH de�nes a norm.Having introduced this notation we are able to formulate the Relative Lipschitz Condition:9M > 0 : 8v 2 IRm 8y; y + h 2 F 0 :jvT (r2fi(y + h)�r2fi(y))vj �MkhkHvTr2fi(y)v; (2)for all 1 � i � n.This condition is also used by Jarre [5]. In general the condition might be hard to check for agiven problem.3 The algorithmIn our algorithm we don't need to stay close to the central path, as in Jarre [5]. If we are far awayfrom the central path we do a linesearch along the Newton direction with respect to �(y; z). TheNewton direction p(y; z) associated with �(y; z) at y is given byp(y; z) = �H(y; z)�1g(y; z) = �H�1g:If no confusion is possible we will write, for shortness sake, p instead of p(y; z). This processis repeated until we are su�ciently close to the central path. More precisely, we stop doing3

linesearches if kpkH � � , where � is a certain tolerance. This proximity criterion is also used byJarre [5]. In the algorithm we will use � = 18(1+2M), which will appear to be appropriate later on.(Note that kpkH = 0 if and only if y = y(z)). If the proximity criterion is satis�ed, we updatethe lower bound z as follows: z := z + �(bT y � z), for some 0 < � < 1, and the whole process isrepeated again and again until some stopping criterion is satis�ed. Note that z is really a lowerbound, because z < bTy � z�.We can now describe the algorithm.AlgorithmInput:� is the updating factor, 0 < � < 1;� = 18(1+2M) is the proximity tolerance;t is an accuracy parameter, t 2 IN;y0 is a given interior feasible point and z0 < bTy0 is a lower bound for theoptimal value, such that kp(y0; z0)kH(y0;z0) � � and z� � z0 � 1� .beginy := y0; z := z0;while bTy � z > 2�t dobegin (outer step)while kpkH > � dobegin (inner step)�� := argmin�>0 f�(y + �p; z) : y + �p 2 F 0gy := y + ��pend (end inner step)z := z + �(bTy � z);end (end outer step)end.For �nding the initial point that satis�es the input assumptions of the algorithm we refer the readerto Jarre [5] and Mehrota and Sun [8]. Later on the 'centering assumption' will be alleviated.4 Preliminary lemmasIn Section 5 we will prove the complexity result on the Algorithm. In this section we deal withsome lemmas which will be needed to obtain an upper bound for the total number of outer andinner iterations. The lemmas are built up as follows:� Lemma 1 gives an upper bound for the error in the quadratic approximation if the functionsfi(y) are linear or quadratic;� Lemma 2 states the same as Lemma 1, but now for general convex constraint functions;� Lemma 3 states that if the proximity criterion holds then y lies close to the exact centery(z) (with respect to the H{norm); 4

� Lemma 4 states that if we do a linesearch along the Newton direction, then a su�cientdecrease in the potential value can be guaranteed;� Lemma 5 gives an upper bound for the di�erence in potential value of the current iterateand the exact center;� Lemma 6 states that if the lower bound is updated then the potential value increases witha constant;� Lemma 7 gives a relation between the objective value in the exact center and the currentpoint;� Lemma 8 gives an upper bound for the gap between the optimal value and the lower boundz;� Lemma 9 states that the gap bTy(z)� z decreases monotonically if z increases.The following lemma improves Lemma 2.1 of Jarre [5].Lemma 1 If all functions fi(y) are linear or quadratic with positive semi-de�nite Hessian matrix,and if y 2 F 0 and kdkH < 1, then y + d 2 F 0 andj�(y + d; z)� qy(y + d; z)j < kdk3H3(1� kdkH) :Proof:We expand �(y + d; z) in a Taylor series about y:�(y + d; z) = qy(y + d; z) + 1Xi=3 ti; (3)where ti is the i{th order Taylor term in the expansion. Note that � only takes �nite values inF 0. Hence, if P1i=3 ti can be shown to converge for d such that kdkH < 1, then it follows thaty + d 2 F 0. It can be proved that jtij � 1i kdkiH : (4)The proof of this inequality is quite technical. Therefore, it is omitted here. The reader may bereferred to the Appendix of the paper. From (4), and using that kdkH < 1, we derive that1Xi=3 jtij � 1Xi=3 kdkiHi � kdk3H3(1� kdkH) :Substituting this into (3) yieldsj�(y + d; z)� qy(y + d; z)j � 1Xi=3 jtij � kdk3H3(1� kdkH) :Thus the lemma has been proved. 25

Lemma 2 If the functions fi(y) satisfy the Relative Lipschitz Condition with Lipschitz constantM > 0, and if y 2 F 0 and kdkH < minf12 ; 12M1=3g then y + d 2 F 0 andj�(y + d; z)� qy(y + d; z)j < kdk3H3(1� kdkH)(1 + 2M):Proof:Using Lemma 1 one can use the same reasoning as in the proof of Lemma 2.10 of Jarre [5] toobtain the result of the lemma. 2The next lemma simpli�es Lemma 2.16 of Jarre [5].Lemma 3 If kpkH � 18(1+2M) then ky � y(z)kH � 52kpkH :Proof:Let h be arbitrary, such that khkH = 32kpkH . We consider the values on the ellipsoid fy+ p+ h :khkH = 32kpkHg. We havekh+ pkH � khkH + kpkH = 52kpkH < 13(1 + 2M) : (5)With the help of Lemma 2, and using that y + p = argminx qy(x; z), we obtain�(y + p+ h; z) > qy(y + p+ h; z)� 13(1� 13)kp+ hk3H(1 + 2M)� qy(y + p; z) + 12khk2H � 12kp+ hk3H(1 + 2M)� qy(y + p; z) + 98kpk2H � 12516 kpk3H(1 + 2M)� qy(y + p; z) + 9kpk3H(1 + 2M)� 12516 kpk3H(1 + 2M)> qy(y + p; z) + kpk3H(1 + 2M)Using Lemma 2 once more, we also obtain that�(y + p; z) < qy(y + p; z) + 12kpk3H(1 + 2M):Hence �(y + p + h; z) > �(y + p; z). Thus, in the center y + p of the ellipsoid the potential valueis less than the value on its boundary. Therefore by the strict convexity of �, the minimum of �is in the interior of the ellipsoid, which means that ky � y(z)kH � kp + hkH . Now using (5) thelemma follows. 26

Lemma 4 If kpkH � 18(1+2M) then the decrease 4� in the potential function after a linesearchalong the Newton direction p satis�es 4� � 1140(1+ 2M)2 :Proof:Let � be a steplength, such that k�pkH � minf12 ; 12M1=3g: (6)Then, as a consequence of Lemma 2 we have�(y + �p; z) � qy(y + �p; z) + �3kpk3H3(1� �kpkH)(1 + 2M):Now using the de�nition of qy , we obtain�(y; z)� �(y + �p; z) � ��gTp� 12�2pTHp� �3kpk3H3(1� �kpkH)(1 + 2M)= �kpk2H � 12�2kpk2H � �3kpk3H3(1� �kpkH)(1 + 2M):Replacing � by the value 19(1+2M)kpkH , which satis�es (6), yields the lemma. 2Lemma 5 If kpkH � 18(1+2M) then�(y; z)� �(y(z); z) � 4kpk2H : (7)Proof:Let d be de�ned as y(z)� y. Using Lemma 2 we get�(y(z); z) � qy(y + d; z)� kdk3H3(1� kdkH)(1 + 2M)= �(y; z)� pTHd+ 12dTHd� kdk3H3(1� kdkH)(1 + 2M):Using the Cauchy{Schwarz inequality we may write�pTHd � �kpkHkdkH :Also using Lemma 3 we obtain�(y; z)� �(y(z); z) � kpkHkdkH � 12kdk2H + kdk3H3(1� kdkH)(1 + 2M)� kpkHkdkH + kdk3H24(1� kdkH)kpkH� 52kpk2H + 125824(1� 516)kpk2H� 4kpk2H: 27

Lemma 6 Let z be the new lower bound, i.e. z = z + �(bTy � z), where 0 < � < 1, then�(y; z)� �(y; z) = �q ln(1� �):Proof:The proof is simple and straightforward. We havebTy � z = bTy � z � �(bT y � z) = (1� �)(bTy � z):Hence �(y; z)� �(y; z) = �q ln bTy � zbTy � z = �q ln(1� �): 2Lemma 7 If ky � y(z)kH � � then(1� �pq )(bTy � z) � bTy(z)� z � (1 + �pq )(bTy � z):Proof:By de�nition we have�2 � ky � y(z)k2H= (y � y(z))T "q bbT(bTy � z)2 + nXi=1 rfi(y)rfi(y)Tfi(y)2 + r2fi(y)�fi(y) !# (y � y(z))� (y � y(z))Tq bbT(bTy � z)2 (y � y(z))= q (bTy � bTy(z))2(bTy � z)2 :Consequently � �pq (bTy � z) � bTy(z)� bTy � �pq (bTy � z):This means that (1� �pq )(bTy � z) � bTy(z)� z � (1 + �pq )(bTy � z): 2Lemma 8 If ky � y(z)kH � � thenz� � z � (1 + nq )(1 + �pq )(bTy � z):8

Proof:The exact center y(z) minimizes the potential function for z. The necessary and su�cient condi-tions for these minima are (1). From these conditions we derive that (u(z); y(z)) is dual feasible.Moreover, using z� � bTy(z)�Pni=1 ui(z)fi(y(z)), it follows thatz� � bTy(z) � � nXi=1 ui(z)fi(y(z)) = nq (bTy(z)� z):Consequently, (z� � z)� (bTy(z)� z) � nq (bTy(z)� z):This means z� � z � (1 + nq )(bTy(z)� z) � (1 + nq )(1 + �pq )(bTy � z);where the last inequality follows from Lemma 7. This proves the lemma. 2The next lemma generalizes an inequality of Vaidya [10] for the LP{case to the present convexcase.Lemma 9 The gap bT y(z)� z decreases monotonically if z < z� increases.Proof:We have that u(z) and y(z) satisfy the Karush{Kuhn{Tucker conditions (1). Taking the derivativewith respect to z of the last two equations in (1) we obtainnXi=1 u0irfi(y) + nXi=1 uiHiy0 = 0 (8)� u0ifi(y)� uirfi(y)Ty0 = bTy0 � 1q ; i = 1; � � � ; n; (9)where the prime denotes the derivative with respect to z and Hi denotes the Hessian matrix offi(y). The Jacobian of this system of equations is clearly nonsingular for z < z�, and hence, asa consequence of the implicit function theorem, we may conclude that u0 and y0 exist for z < z�.Multiplying (9) with ui and using (1), we getbTy � zq u0i � u2irfi(u)Ty0 = bTy0 � 1q ui:Multiplying this equation with rfi(y) and summing over i and using (8) and (1) results into�bTy � zq nXi=1 uiHiy0 � nXi=1 u2irfi(y)Ty0rfi(y) = bTy0 � 1q b:Now taking the inner product with y0 we obtainbTy0 � 1q bTy0 = �bTy � zq nXi=1(y0)THiy0ui � nXi=1 u2i (rfi(y)Ty0)2 � 0:We conclude that 0 � bTy0 � 1, which means that the derivative of bTy(z)� z, which is equal tobTy0 � 1, is not positive. This proves the lemma. 29

5 Convergence analysisBased on the lemmas in the previous section, we will give upper bounds for the total number ofouter iterations and inner iterations.Theorem 1 Let � � 516(1+2M), then after at mostK = (1 + nq )(1 + �pq )� O(j ln �j)outer iterations, the algorithm �nds an �{optimal solution for (CP ).Proof:Let zk be the lower bound in the k{th outer iteration. Then we havez� � zkz� � zk�1 = z� � (zk�1 + �(bT yk�1 � zk�1))z� � zk�1= 1� �bT yk�1 � zk�1z� � zk�1� 1� �(1 + nq )(1 + �pq ) ;where yk�1 is the iterate at the end of the (k � 1){th outer iteration. The last inequality followsfrom Lemma 8. Hence after K outer iterations we havez� � bTyK � z� � zK+1� 0@1� �(1 + nq )(1 + �pq )1A (z� � zK)� 0@1� �(1 + nq )(1 + �pq )1AK (z� � z0):This means that z� � bTyK � � certainly holds if0@1� �(1 + nq )(1 + �pq)1AK (z� � z0) � �:Taking logarithms, this inequality reduces to�K ln0@1� �(1 + nq )(1 + �pq )1A � j ln �j+ ln(z� � z0):Since � ln(1� v) > v, this will certainly hold ifK > (1 + nq )(1 + �pq )� (j ln �j+ ln(z� � z0)):10

Now using the assumption on z0, i.e. z� � z0 � 1� , the theorem follows. 2From Lemma 8 it follows that it su�ces to take t = O(ln (1+nq )(1+ �pq )� ), i.e. for such t the algorithmends up with a solution y such that z� � bTy � �.Now we give an upper bound for the total number of inner iterations during an arbitrary outeriteration. The approach is similar to Gonzaga's approach for linear programming in [3].Theorem 2 The total number P of inner iterations during an arbitrary outer iteration satis�esP� � 1 + 2��qpq + � + �2q1� �2 ;where � is the guaranteed decrease in each inner iteration, and � � 516(1+2M).Proof:We denote the used lower bound in an arbitrary outer iteration by z, while the lower bound in theprevious outer iteration is denoted by z. The iterates during this outer iteration are denoted byy0; y1; : : : ; yP , where y0 is the iterate at the beginning of the outer iteration. Because of Lemma 4we have �(yP ; z) � �(y0; z)� P�: (10)Because the lower bound was updated at the beginning of the outer iteration we have (see Lemma6) �(y0; z)� �(y0; z) = �q ln(1� �):We also have �(yP ; z)� �(yP ; z) = �q ln bTyP � zbTyP � z= �q ln bTyP � z � �(bT y0 � z)bTyP � z= �q ln 1� � bTy0 � zbT yP � z! :These results are substituted into (10) to obtainP� � �(y0; z)� �(yP ; z)� q ln(1� �) + q ln 1� � bTy0 � zbTyP � z! : (11)Because y0 is almost centered, Lemma 5 imply that�(y0; z)� �(y(z); z) � !;where ! � 116(1+2M)2 . Hence�(y0; z)� �(yP ; z) � ! + �(y(z); z)� �(yP ; z) � ! < 1:11

This is substituted into (11)P� � 1� q ln(1� �) + q ln 1� � bTy0 � zbT yP � z! : (12)From Lemma 7 we obtain bTy0 � z � 11 + �pq (bTy(z)� z);where � � 516(1+2M). Again using Lemma 7 we getbTyP � z = (bTyP � z) + (z � z)� 11� �pq (bTy(z)� z) + �(bTy0 � z)� 11� �pq (bTy(z)� z) + �1� �pq (bTy(z)� z)� 1 + �1� �pq (bTy(z)� z);where the last inequality follows because Lemma 9 states that bTy(z)� z � bTy(z)� z: Hence weobtain bTy0 � zbTyP � z � 1� �pq(1 + �)(1 + �pq ) = pq � �(1 + �)(pq + �) :This is substituted into (12)P� � 1� q ln(1� �) + q ln(1� �(pq � �)(1 + �)(pq + �))= 1 + q ln( 11� � � �(pq � �)(1� �2)(pq + �))= 1 + q ln 1 + 2��pq+�1� �2= 1 + q ln(1 + 2��pq + � ) + q ln(1 + �21� �2 )� 1 + 2��qpq + � + �2q1� �2 : 2From Theorem 1 we know that the total number of outer iterations is at most (1+nq )(1+ �pq )� O(j ln �j).Hence the total number of inner iterations, during the whole process is given by1� (1 + nq )(1 + �pq )(1� + 2�qpq + � + �q1� �2 )O(j ln �j): (13)This makes clear that if we take q = O(n) then12

� if we take � = O( 1pn ), then the algorithm has an O((1 +M)2pnj ln �j) iteration bound.� if we take � = O(1), then the algorithm has an O((1 +M)2nj ln �j) iteration bound.The �rst case corresponds to a small reduction factor �. In this case we can return to the vicinity ofthe central trajectory in O((1 +M)2) steps, while the lower bound must be updated O(pnj ln �j)times. In the path{following algorithm of Jarre [5], the same iteration bound is obtained for� = 1200pn(1+M2) .The second case corresponds to a large reduction factor �. In this case we can return to the vicinityof the central trajectory in O((1 +M)2n) linesearches, while the lower bound must be updatedO(j ln �j) times.Remark 1: We note that the upper bound for the number of iterations is not better than Jarre's.However, while Jarre's bound is more or less exact, our bound can be very pessimistic, because ofthe linesearches involved in the inner iterations. This can also be one of the reasons for the factthat � = O(1) gives a worse bound than � = O( 1pn), while one would expect the contrary.Remark 2: The 'centering assumption' kp(y0; z0)kH(y0;z0) � � can be alleviated to�(y0; z0)� �(y(z0); z0) � O(pnj ln �j)for the �rst case, and to �(y0; z0)� �(y(z0); z0) � O(nj ln �j);for the second case. This follows easily from Lemma 4.Remark 3: Note that the updating factor � is independent fromM , contrary to Jarre's [5] method.Remark 4: For linear programming problems, i.e. M = 0, we can �nd an exact solution if wetake � = 2�L, where L denotes the input length of the problem. In this case our results reducesto an O(pnL) iteration bound if we take � = O( 1pn ) and to an O(nL) iteration bound if we take� = O(1). These results are also obtained by Den Hertog, Roos and Terlaky [1], Gonzaga [3] andRoos and Vial [9].Appendix: Proof of the inequalities (4)Since each function fi is assumed to be linear or quadratic, the k{th order term in its Taylorexpansion has the following form:tk = 1k! n+qXj=1 bk=2cXi=0 ak;i (rfj(y)Td)k�2i(dTr2fj(y)d)i(�fj(y))k�i ; k � 1; (14)where ak;i has to be determined yet. For shortness sake we use the following notations�j := rfj(y)Td�fj(y) ;13

j := dTr2fj(y)d�fj(y) ;D2 := kdk2H :Using these notations (14) becomestk = 1k! n+qXj=1 bk=2cXi=0 ak;i�k�2ij ij: (15)We also have D2 = n+qXj=1(�2j + j):Now we derive a recursive formula for ak;i. Using the chain rule for taking derivatives, we obtainfrom (14) an expression for tk+1:tk+1 = 1(k + 1)! n+qXj=1 24bk=2cXi=0 (k � i)ak;i�k�2i+1j ij + bk=2cXi=0 (k � 2i)ak;i�k�2i�1j i+1j 35 :This can be rewritten astk+1 = 1(k + 1)! n+qXj=1 24bk=2cXi=0 (k � i)ak;i�k�2i+1j ij + bk=2c+1Xi=1 (k � 2i+ 2)ak;i�1�k�2i+1j ij35 :From this the following recursive formula can be derived.a1;0 = 1; a1;i = 0; i 6= 0;ak+1;i = (k � i)ak;i + (k � 2i+ 2)ak;i�1; k � 1: (16)From this recursive scheme we derive an explicit formula for ak;i:ak;i = 8><>: k!(k�i�1)!i!(k�2i)!2i if 0 � i � bk2c;0 otherwise: (17)We prove this formula by using induction. For k = 1; 2 our formula is certainly correct, as followsby inspection. From (16) it readily follows that ak;0 = (k � 1)!; k � 1. This is in accordance with(17), since (k � 1)! = k!(k� 1)!0!k!20 :Now suppose that the formula is correct for some value of k; k � 1: Then, using (16) and (17) onehas, for i � 1;ak+1;i = (k � i)ak;i + (k � 2i+ 2)ak;i�1= (k � i)k!(k� i� 1)!i!(k� 2i)!2i + (k � 2i+ 2) k!(k � i)!(i� 1)!(k� 2i+ 2)!2i�1= (k+ 1)!(k� i)!i!(k� 2i+ 1)!2i : 14

This proves that formula (17) is correct indeed.We proceed by deriving an upper bound for tk . To this end we consider the following optimizationproblem:max 0@ 1k! n+qXj=1 bk=2cXi=0 ak;i�k�2ij ij : n+qXj=1(�2j + j) = D2; �j � 0; j � 0; 1 � j � n1A ;where the maximization is done over �j and j . The nonnegativity of j is an obvious consequenceof its de�nition; the nonnegativity of �j can be assumed, since the constraints are sign independentas far as �j concerns, whereas positive values will give a larger objective than negative values. TheKuhn{Tucker optimality conditions for this problem are given bybk=2c�1Xi=0 (k � 2i)ak;i�k�2i�1j ij + (bk + 12 c � bk2c)ak;b k2 c b k2 c � 2��j ; (18)bk=2cXi=1 iak;i�k�2ij i�1j � �; (19)�j 0@bk=2c�1Xi=0 (k � 2i)ak;i�k�2i�1j ij + (bk + 12 c � bk2 c)ak;b k2 c b k2 c � 2��j1A = 0; (20) j 0@bk=2cXi=1 iak;i�k�2ij i�1j � �1A = 0; (21)where � is the multiplier.From these conditions we will derive that either j or �j must be zero, for each j. Assume thatneither j nor �j is zero. From this we shall derive a contradiction. By multiplying (20) by jand (21) by 2�2j and subtracting, we derivebk=2c�1Xi=0 [(k � 2i)ak;i � 2(i+ 1)ak;i+1]�k�2ij i+1j + (bk + 12 c � bk2 c)ak;b k2 c�j b k2 c+1 = 0:It is easy to see that bk+12 c � bk2c is 1 if k is odd, and 0 if k is even. Furthermore, let�k;i := (k � 2i)ak;i � 2(i+ 1)ak;i+1:We easily obtain that �k;0 = 0 and �k;i > 0, for i � 1, by using the general formula (17) for ak;i:�k;i = (k � 2i)k!(k� i� 1)!i!(k� 2i)!2i � 2(i+ 1) k!(k� i� 2)!(i+ 1)!(k� 2i� 2)!2i+1= k!(k � i� 2)!i!(k� 2i� 2)!2i � k � i� 1k � 2i� 1 � 1�> 0:Hence, we have obtained a contradiction. This means that either �j or j is zero in the maximum.Consequently, the objective function either consists of 'pure' �j terms or 'pure' j terms.15

Now if k is even, it easily follows that the largest objective value is obtained if j = 0, since usingthe general formula (17) we have ak;0 = 2 k2�1ak; k2 ;which means that the coe�cient of the pure �j term is greater than the coe�cient of the pure jterm. Now assume that j > 0 if k is odd. Then (19) holds with equality. By multiplying (19) by2�j and subtracting from (18), we derivebk=2c�1Xi=0 [(k � 2i)ak;i � 2(i+ 1)ak;i+1]�k�2i�1j ij + ak;b k2 c b k2 c = 0: (22)Again, we have obtained a contradiction, since we assumed that j > 0. Consequently, both for kis even and k is odd, the maximum is reached if j = 0.So an upper bound for the maximum of the function is1k! n+qXj=1(k � 1)!�kj = 1k n+qXj=1 �kj � 1k 0@n+qXj=1 �2j1Ak=2 = 1k (D2)k=2 = 1kDk :Hence the proof of (4) is complete.References[1] Den Hertog, D, Roos, C, Terlaky, T. (1990), A Potential Reduction Variant of Renegar'sShort{Step Path{Following Method for Linear Programming, to appear in Linear Algebraand Its Applications.[2] Freund, R.M. (1988), Polynomial{Time Algorithms for Linear Programming Based only onPrimal Scaling and Projected Gradients of a Potential Function, Working Paper OR 182{88,Sloan School of Management, M.I.T., Cambridge, Massachusetts.[3] Gonzaga, C.C. (1989), Large{Steps Path{Following Methods for Linear Programming: BarrierFunction Method, Report ES{210/89, Department of Systems Engineering and ComputerSciences, COPPE{Federal University of Rio de Janeiro, Rio de Janeiro, Brasil.[4] Gonzaga, C.C. (1989), Large{Steps Path{Following Methods for Linear Programming: Po-tential Reduction Method, Report ES{211/89, Department of Systems Engineering and Com-puter Sciences, COPPE{Federal University of Rio de Janeiro, Rio de Janeiro, Brasil.[5] Jarre, F. (1989), The Method of Analytic Centers for Smooth Convex Programs, Dissertation,Institut f�ur Angewandte Mathematik und Statistik, Universit�at W�urzburg, W�urzburg, West{Germany.[6] Karmarkar, N. (1984), A New Polynomial{Time Algorithm for Linear Programming, Combi-natorica 4, 373{395.[7] Kojima, M., Mizuno, S., Yoshise, A. (1988), An O(pn) Iteration Potential Reduction Algo-rithm for Linear Complementarity Problems, Research Report, Department of InformationSciences, Tokyo Institute of Technology, Tokyo, Japan.16

[8] Mehrota, S. and Sun, J. (1988), An Interior Point Algorithm for Solving Smooth Convex Pro-grams Based on Newton's Method, Technical Report 88{08, Department of IE/MS, North-western University, Evaston IL.[9] Roos, C. and Vial, J.{Ph. (1989), Long Steps with the Logarithmic Penalty Barrier Function inLinear Programming, Report No. 89{44, Faculty of Mathematics and Informatics/ComputerScience, Delft University of Technology, Delft, Holland.[10] Vaidya, P.M. (1990), An Algorithm for Linear Programming which Requires O(((m+ n)n2+(m+ n)1:5n)L) Arithmetic Operations, Mathematical Programming 47, 175{201.[11] Wolfe, Ph. (1961), A Duality Theorem for Nonlinear Programming, Quarterly of AppliedMathematics 19, 239{244.[12] Ye, Y. (1988), A Class of Potential Funcions for Linear Programming, Department of Man-agement Sciences, The University of Iowa, Iowa City, Iowa.

17