Bibliography
[lJ T .M. Apostol. Linear Algebra: A First Course, with Application s toDifferential Equations. Wiley, New York , 1997.
[2J E. Asp lund. Cebysev set s in Hilb ert space. Trans acti ons of the American Math ematical Society , 144:235-240, 1969.
[3J J .-P. Aubin. Via bility Th eory. Birkhauser , Boston, 1991.
[4J J .-P . Aubin and A. Cellina. Different ial Inclusions. Springer-Verl ag ,Berlin, 1984.
[5] J. -P. Aubin and H. Frankowska. Set- Valued Analysis. Birkhauser ,Boston, 1990.
[6] M. Avriel. Nonlin ear Programming. Prentice-Hall, Englewood Cliffs,N.J ., 1976.
[7] S. Banach. Sur les operations dans les ensembles abstraits et leurapplication aux equat ions integrales. Fundam enta Mathematicae,3:133- 181, 1922.
[8] H.H. Bau schke, J .M. Borwein , and P. Tseng. Bounded linear regularity, st rong CHIP , and CHIP are distinct properties. Journal ofConvex Analysis, 7:395-412 , 2000.
[9] M.S . Bazaraa and C.M. Shet ty. Nonlinear Programming. Wil ey, NewYork, 1979.
[10] R. Bellm an . Introduction to Matrix Analysis. SIA M , Philad elphia ,1997.
[11] A. Ben-Tal and J . Zowe. Necessary and sufficient condit ions for a classof non smo oth minimization problems. Mathematical Programming,24:70-91 , 1982.
275
276 Bibliography
[12] A. Ben-Tal and J . Zowe. A unified theory of first-order and secondorder cond itions for ext remum problems. Math ematical ProgmmmingSt7tdy, 19:39- 76, 1982.
[13] C. Berge. Espaces Topologiques et Fonctions Multivoqu es. Dunod,Paris , 1959.
[14] D.N. Bessis and F.R. Clarke. Partial subdifferentials , derivates andRademacher 's Theorem. Tmnsactions of the American MathematicalSociety, 351:2899-2926, 1999.
[15] G. Birkhoff. Tres observaciones sobre el algebra lineal. UniversidadNacionale Tucatruin Revista, 5:147-151, 1946.
[16] B. Bollobas , Linear Analysis. Cambridge University Press, Cambridge, U.K. , 1999.
[17] K. Borsuk. Drei Siitze iiber die n-dimensionale Euklidische Sphare.Fundamenta Mathematicae, 21:177-190,1933.
[18] D. Borwein, .J.M. Borwein, and P. Marechal, Surprise maximization.American Mathematical Monthly, 107:517-527, 2000 .
[19] J .M. Borwein. The direct method in semi-infinite programming.Mathematical Programming, 21:301-318, 1981.
[20] J .M. Borwein . Continuity and differentiability properties of convexoperators. Proceedings of the London Mathematical Society, 44:420444 , 1982.
[21] J .M . Borwein . Necessary and sufficient condit ions for quadratic minimality. Num erical Functional Analysis and Optimization, 5:127-140,1982.
[22] J .M. Borwein. A note on the existence of subgradients. Math ematicalProgramming, 24:225-228, 1982.
[23] J .M. Borwein. Adjoint process du ality. Math ematics of OperationsResearch, 8:403-434, 1983.
[24] J.M. Borwein. Norm du ality for convex processes and applications.Journal of Optimi zation Theory and Applications, 48:53-64, 1986.
[25] J .M. Borwein. Stability and regular points of inequality systems .Journal of Optimization Theory and Applications, 48:9-52, 1986.
[26] J .M. Borwein. The linear order-complementarity problem . Math ematics of Operations Research, 14:534-558, 1989.
Bibliography 277
[27] J .M. Borwein. Minimal cuscos and subgradients of Lipschitz funct ions. In J .-B . Baillon and M. Thera, editors, Fixed Point Th eoryand its Applications, Pi tman Lecture Not es in Mathem atics , pages57-82, Essex , U.K., 1991. Longman .
[28] J.M. Borwein . A generalization of Young's lP inequ ali ty. Mathematical In equalit ies and Applications, 1:131- 136, 1997.
[29] J .M. Borwein. Cones and orderings. Techni cal repor t , CECM, SimonFraser University, 1998.
[30] J .M. Borwein . Maximal monotonicity via convex analysis. Journalof Convex Analysis, 2005. To appear.
[31] J .M. Borwein and S. Fitzpatrick. Characterization of Clarke subgradients among one-dimensional multi functions. In Proceedings of theOptimization Min iconference II, pages 61-73. University of BalaratPress, 1995.
[32] J.M. Borwein , S.P. Fitzpatrick, and J .R. Giles . The differen tiabilityof real functions on normed linear spaces using generalized gradients.Journal of Opt imization Th eory and Applications, 128:512-534, 1987 .
[33] J .M. Borwein and J.R. Giles. The proximal normal formula in Banachspaces. Transa ctions of the Am erican Mathematical Society, 302:371381, 1987.
[34] J .M. Borwein and A.S. Lewis. Partially finite convex programming,Part I, Duality theory. Math ematical Programming B, 57:15-48, 1992 .
[35] J .M. Borwein, A.S. Lewis, and D. Noll. Maximum entropy spectralanalysis using first orde r informat ion . Part I: Fisher information andconvex duality. Mathematics of Operations Research, 21 :442~468 ,
1996.
[36] J.M. Borwein, A.S . Lewis , and R. Nussbaum . Entropy minimization,DAD problems and do ubly-stochastic kernels. Journal of Fun ctionalAnalysis, 123:264-307, 1994.
[37] J.M. Borwein, A.S. Lewis, J. Read, an d Q. Zhu . Convex spectralfunctions of compact operat ors. International Journal of Con vex andNonlinear Analysis, 1:17- 35, 1999.
[38] J .M . Borwein and W .B. Moors. Null sets and essent ially smoothLipschitz functions. SIA M Journal on Optimizat ion , 8:309- 323, 1998.
[39] J .M. Borwein and H.M. Strojwas . Tangenti al approximations. Nonlin ear Analysis: Th eory, Methods and Applications, 9:1347-1366 ,1985.
278 Bibliography
[40] J.M. Borwein and H.M. Strojwas. Proximal analysis and boundariesof closed sets in Banach space, Part I: theory. Canadian Journal ofMathematics, 38:431 452, 1986.
[41] J .M. Borwein and H.M. Strojwas . Proximal analysis and boundariesof closed sets in Banach space, Part II . Canadian Journal of Mathematics, 39:428-472 , 1987.
[42] J .M. Borwein and Q. Zhu . A survey of smooth subdifferential calculus with applications . Nonlinear Analysis: Th eory, Methods andApplications, 38:687-773, 1999.
[43] J .M. Borwein and Q. Zhu . Techniques of Variational Analysis.Springer- Verlag, New York , 2005.
[44] J .M. Borwein and D. Zhuang. Super-efficient points in vector optimization. Transactions of the American Mathematical Society ,338:105-122, 1993.
[45] G. Bouligand. Sur les surfaces depourvues de points hyperlimites.Annales de la Societe Polonaise de Mathematique, 9:32-41 , 1930.
[46] S. Boyd, L. El Ghaoui , E. Feron, and V. Balikrishnan. Linear Matrix In equalities in System and Control Theory. SIAM, Philadelphia,1994.
[47] S. Boyd and L. Vandenberghe. Conv ex Optimization. CambridgeUn iversity Press, Cambridge, U.K, 2004.
[48] L.M. Bregman. The method of successive projection for find ing acommon point of convex sets. Sov iet Mathematics Doklady, 6:688692, 1965.
[49] L.E.J. Brouwer . On continuous one-to-one transformations of surfaces into themselves. Proc. Kon. Ned. Ak. V. Wet. Ser. A , 11:788798, 1909.
[50] L.E.J . Brouwer. Uber Abbildungen vom Mannigfaltigkeiten. Mathematische Annalen, 71:97-115 , 1912.
[51] F .E . Browder. Nonexpansive nonlinear operators in a Banach space.Proc. Nat. Acad. Sci. U.S.A ., 54:1041-1044, 1965.
[52] C. Caratheodory, Uber den Variabiletatsbereich der Fourier'schenKonstanten von positiven harmonischen F unktionen. Rendiconti delCircolo Matematico de Palermo, 32:193-217, 1911.
[53] V. Chvatal, Linear Programming. Freeman, New York, 1983.
Bibliography 279
[54] F.H. Clarke. Generalized gradients and applicat ions. Transa ctionsof the American Math ematical Society, 205:247-262, 1975.
[55] F .H. Clarke. Optimization and Nonsmooth Analysis. Wiley, NewYork,1983.
[56] F .H. Clarke, Y.S . Ledyaev, R.I . Stern, and P.R Wolenski. Nonsmooth Analysis and Control Theory. Springer-Ver lag , New York,1998.
[57] G .B. Dantzig. Linear Programming and Its Extensions. PrincetonUniversity Press , Princeton, N.J ., 1963.
[58] C. Davis. All convex invariant functions of hermitian matrices . Archivder Mathematik, 8:276-278, 1957.
[59] H. Debrunner and P. Flor . Ein Erweiterungssatz fiir monotone Mengen. Archiv der Math ematik, 15:445-447, 1964.
[60] V.F. Dem'yanov and V.M . Malozemov. Introduction to Minimax.Dover , New York , 1990.
[61] .I.E. Dennis and RB. Schnabel. Numerical Methods for Unconstrained Optimization and Nonlinear Equations. P rentice-Hall , Englewood Cliffs, N.J ., 1983.
[62] F .R. Deutsch. Best Appro ximation in Inn er Product Spaces. Springer,New York, 200l.
[63] .1. Diestel. Geometry of Banach Spaces-Selected Topics, volume 485of Lecture Notes in Mathematics. Springer-Verlag, New York , 1975.
[64] U. Dini. Fondamenti per la teoria delle funzioni di variabili reali .Pisa, 1878.
[65] A. Dontchev. The Graves theorem revisited. Journal of ConvexAnalysis , 3:45-54, 1996.
[66] A.L . Dontchev, A.S. Lewis, and RT. Rockafellar. The radius of metric regularity. Transactions of the American Math ematical Society,355:493-517, 2003.
[67] .1 . Dugundji. Topology. Allyn and Bacon , Boston, 1965.
[68] .1 . Dugundji and A. Granas. Fixed Point Theory. Polish ScientificPublishers, Warsaw, 1982.
[69] 1. Eke land . On the variational principle. Journal of MathematicalAnalysis and Applications, 47:324-353, 1974.
280 Bibliography
[70] 1. Eke land and R. Temam. Convex Analysis and Variational Problems. North-Holland, Amsterdam, 1976.
[71] .L.C. Evans and RF. Gariepy. Measure Theory and Fin e Propertiesof Functions. CRC Press, Boca Raton, Florida, 1992.
[72] M. Fabian and D. Preiss. On the Clarke's generalized J acobian. Rend.Circ. Mat. Palermo, 14:305-307, 1987.
[73] K. Fan. On a theorem of Weyl concerning eigenvalues of lineartransformations. Proceedings of the National Academy of Sciencesof U.S.A., 35:652-655, 1949.
[74] K. Fan. Fixed po int and minimax theorems in locally convex topological linear spaces. Proceedings of the National Academy of Sciencesof U.S.A. , 38:431-437, 1952.
[75] J . Farkas. Theorie der einfachen Ungleichungen. Journal fur die reineund angewandte Mathematik, 124:1~27 , 1902.
[76] W. Fenchel. On conj ugate convex functions. Canadian Journal ofMath ematics, 1:73-77, 1949.
[77] L.A. Fernandez. On the limits of the Lagrange mu ltiplier ru le. SIAMReview, 39:292-297, 1997.
[78] P.A. Fillmore and J .P. Williams. Some convexity theorems for matrices. Glasgow Math ematical Journal, 12:110~117, 1971.
[79] S. Fitzpatrick. Representing monotone operators by convex functions .Proceedings of the Centre for Math ematical Analysis, Australian National University, 20:59~65 , 1988.
[80] R Fletcher. A new variational result for quasi-Newton formulae.SIAM Journal on Optimization, 1:18-21 , 1991.
[81] D. Ga le. A geometric duality theorem with economic applications.Review of Economic Studies, 34:19-24, 1967.
[82] J .R Giles. Convex Analysis with Application in the Differentiationof Convex Functions. Pitman, Boston, 1982.
[83] K. Goebel and W .A. Kirk. Topics in Metric Fixed Point Theory.Cambridge Univers ity Press, Cambridge, U.K. , 1990.
[84] P. Gordan. Uber die Auft6sung linearer Gleichungen mit reelen Coefficienten. Mathematische Annalen, 6:23-28, 1873.
[85] L.M . Graves. Some mapping theorems. Duke Mathematical Journal,17:111-114, 1950.
Bibliography 281
[86] B. Grone, C .R. Johnson , E. Marques de Sa , and H. Wolkowicz. Positive defini te completions of partial Hermiti an matrices. Lin ear Algebra and Its Applicatio ns, 58:109-124, 1984.
[87] M. Guignard. Generalized Kuhn-Tucker condit ions for mathematical programming in Banach space. SIA1'vl Journal on Control andOpt imization, 7:232-241, 1969.
[88] .1. Had amard. Resolution d 'une question relat ive aux determ inants.Bull. S ci. Math ., 2:240-248, 1893.
[89J J . Had amard. Sur quelques applications de l'indice de Kronecker. InJ . Tannery , Introduc tion a la Th eorie des Fonctions d 'un e Variable,volume II . Hermann, Paris, second edit ion, 1910 .
[90] P.R. Halmos. Finit e-Dimensional Vector Spa ces. Van Nost rand ,P rinceton, N.J ., 1958.
[9] ] G.H. Hardy, J .E. Lit tlewood, and G. Polya , In equalities. CambridgeUnivers ity Press , Cambr idge, U.K., 1952.
[92J W.L. Hare and A.S. Lewis. Identifying active constraints via partialsmoot hness and prox-regularity. Journal of Convex Analysis, ] 1:2.51266 ,2004.
[93] P.T. Harker and J .-S. Pan g. Finite-dimensional vari ational inequali tyand nonlinear complementarity problems: a survey of theory, algorithms and ap plications. Mathem atical Programming, 48:161-220,1990 .
[94] J.-B . Hiriart-Urruty. A short proof of t he variational principle forapprox imate solutions of a minimization problem. American Mathem atical Monthly, 90:206-207, 1983.
[95] J .-B . Hiri art- Urruty. What condit ions are satisfied at points minimizing the maximum of a finit e number of differentiable functions .In Nonsmooth Optim izat ion : Methods and Applications. Gord an andBreach, New York, 1992.
[96] J .-B . Hiriart-Urruty. Ensemble de Tchebychev vs ense mble convexe :l'etat de la situation vu via l'an alyse convexe nonlisse . Annales S cientifiques et Mathematiques du Quebec, 22:47-62, 1998.
[97] J .-B . Hiriart-Urruty and C. Lemarechal . Convex Analysis and Minim ization Algorithms. Springer-Verlag , Berlin, 1993 .
[98J R.B. Holmes. Geom etric Fun ctional Analysis and It s Applications.Springer-Verl ag, New York , 1975.
282 Bibliography
[99] R.A. Horn and C. Johnson. Matrix Analysis. Cambridge UniversityPress, Cambridge, U.K., 1985.
[100] R.A. Horn and C.R. Johnson . Topics in Mat rix Analysis. CambridgeUniversity Press, Cambridge, U.K. , 1991.
[101] A.D. Ioffe. Regular points of Lips chitz functions. Transactions of theAmerican Math ematical Society, 251:61-69 , 1979.
[102] A.D. Ioffe. Sous-differentielles approchees de fonctions numeriques.Comp tes Rendus de l 'Academ ie des Sciences de Paris, 292:675-678 ,1981.
[103] A.D. Ioffe. Approximate subdifferentials and applicat ions. I: The finite dimensional theory. Transactions of the American MathematicalSociety , 281:389-416, 1984.
[104] J . J ah n . Scalarization in multi objective optimization. In P. Serafini ,edi tor , Mathemat ics of Multi Objective Optimization , pages 45-88.Springer-Verlag, Vienn a , 1985.
[105] J . J ahn, An Introduction to the Theory of Nonlin ear Optimizat ion.Springer- Verlag, Berlin, 1996.
[106] G.J.O . Jameson . Topology and Normed Spaces. Chapman and Hall ,New York , 1974.
[107] Fritz John. Extremum problems with inequalities as subsidiary conditions. In Studies and Essays, Courant Anniversary Volume, pages187-204. Int erscience, New York , 1948.
[108] C. Jordan. Memoire sur les form es bilineaires. Journal de Mathematiques Pures et Appliquees, 2:35- 54, 1874 .
[109] S. Kakutani . A generalization of Brouwer 's fixed point theorem . DukeMath ematical Journal , 8:457-459, 1941.
[110] S. Karlin. Math ematical Methods and Theory in Gam es, Programming and Economics. McGraw-Hill , New York , 1960.
[111] W . Karu sh . Minima of functions of several variables with inequalitiesas side conditions. Master 's thes is, Un iversity of Chicago , 1939.
[112] W .A. Kirk. A fixed point theorem for nonexpansive mappings whichdo not increase dis tance. American Math ematical Monthly, 72:10041006, 1965.
[113] E . Klein and A.C . Thompson. Theory of Correspondences, IncludingApplications to Math ematical Economics. Wiley, New York , 1984.
Bibliography 283
[114] B. Knaster, C . Kuratowski , and S. Mazurkiewicz . Ein Beweis des Fixpunktsatzes fur n-dimesionale Simp lexe . Fundamenta Mathematicae,14:132-137, 1929.
[115] D. Konig. Theorie der Endlichen und Unendlichen Graphen. Akademische Verlagsgesellschaft, Leipzig , 1936.
[116] A.Y. Kruger and B.S. Mordukhovich. Extremal points and the Eulerequat ion in nonsmooth optimization. Doklady Akademia Nauk BSSR(Belorussian Academy of Scien ces) , 24:684-687, 1980 .
[117] H.W . Kuhn and A.W . Tucker. Nonlinear programming. In Proceedings of the Second B erkeley Symposium on Mathematical Statisticsand Probability. University of California P ress, Berkeley, 1951.
[118] P. Lax . Change of variables in multiple int egrals. American Mathematical Monthly, 106:497-501 , 1999.
[119] G. Lebourg. Valeur moyenne pour gradient generalise. ComptesRendus de l'Academie des Sciences de Paris, 281:795-797, 1975.
[120] A.S. Lewis. Convex analysis on the Hermitian matrices. SIAM Journal on Opt imization, 6:164-1 77, 1996.
[121] A.S. Lewis . Derivatives of spectral fun ctions. Mathematics of Operations Research, 6:576-588, 1996.
[122] A.S. Lewis . Group invariance and convex matrix analysis. SIAMJournal on Matrix Analysis and Applications, 17:927-949, 1996.
[123] A.S. Lewis . Ill-conditioned convex processes and conic linear systems.Mathematics of Operations Research, 24:829-834, 1999.
[124] A.S. Lewis. Lidskii's theorem via nonsmooth analysis . SIAM Journalon Matrix Analysis, 21:379-381, 1999.
[125] A.S. Lewis . Nonsmooth analysis of eigenvalues. Mathematical Pro gramming, 84:1-24, 1999.
[126] A.S. Lewis. Active sets, nonsmoothness and sensitivity. SIAM Journal on Optimization, 13:702-725, 2003.
[127] A.S. Lewis . The mathematics of eigenvalue optimization. Mathematical Programming, 97:155-176, 2003.
[128] A.S. Lewis and M.L. Overton. Eigenvalue optimization. Acta Numerica, 5:149-190, 1996 .
[129] R. Lipschitz. Lehrbuch der Analysis. Cohen und Sohn, Bonn, 1877.
284 Bibliography
[130] L.A. Liusternik. On the cond it ional extrema of functionals. Matemati cheskii Sbornik, 41:390-401 , 1934 .
[131] D .G . Luenberger. Optimization by Vector Space Methods. Wiley, NewYork, 1969.
[132] D .G . Luenberger. Linear and Nonlinear Programming. AddisonWesley, Reading, Mass. , 1984 .
[133] O .L. Mangasarian and S. Fromovitz. The Fritz John necessary optimality condit ions in the presence of equality and inequality con straints. Journal of Mathematical Analysis and Applications, 17:3747 , 1967.
[134] A.W. Marshall and I. Olkin. Inequalities: Theory of Majorizationand Its Applications. Academic Press , New York, 1979 .
[135] M . Matic, C.KM. Pearce, and J . Pecaric, Improvements on somebounds on entropy measures in informat ion theory. MathematicalIn equalities and Applications, 1:295-304, 1998 .
[136] KJ . McShane. The Lagrange multiplier rule. American MathematicalMonthly, 80:922-924, 1973.
[137] K Michae l. Continuous selections 1. Annals of Mathematics, 63:361382 , 1956.
[138] P . Michel and J.-P. Penot. Ca1cul sous-differentiel pour les fonctionslipschitziennes et non lipschit ziennes. Comptes Rendus de l'Academiedes Sciences de Paris, 298:269-272, 1984 .
[139] P . Michel and J .-P. Penot. A generalized derivative for calm andstable functions. Differential and Integral Equations, 5:433-454, 1992 .
[140] J . Milnor. Analytic proofs of the Hairy ball theorem and the Brouwerfixed point theorem. American Mathematical Monthly, 85:521-524,1978.
[141] H. Minkowski. Geometric der Zahlen. Teubner, Leipzig, 1910.
[142] H. Minkowski. Theorie der konvexen Korper, insbesondere Begriindung ihres Oberflachenbegriffs. In Gesammelte AbhandlungenII. Chelsea, New York, 1967 .
[143] B.S. Mordukhov ich. Maximum principle in the problem of t ime optimal response with nonsmooth constraints. Journal of Applied Mathem atics and Mechanics, 40:960-969, 1976.
Bibliography 285
[144J B.S . Mordukhovich. Nonsmooth analysis with nonconvex generalizeddifferentials and adjoint mappings. Doklady Akadem ia Nauk BSSR,28:976-979, 1984.
[145J B .S. Mordukhovich . Complet e characte rization of openness, metricregularity, and Lipschitzian properti es of mult ifunctions, Transa ctions of the American Mathemat ical Society, 340:1-35, 1993.
[146J J .-J . Moreau, Sur la fonction polaire d 'une fonction semi-cont inuesuperieur ernent. Comptes Rerulus de l'Academic des Sciences deParis, 258:1128-1130, 1964.
[147] J . Nash. Non-cooper ative games. Annals of Math ematics, 54:286295, 1951.
[148] Y. Nesterov and A. Nem irovs kii. Interior-Point Po lynomial A lgorithms in Convex Programming. SIAM Publications, Philadelphia,1994.
[149] G.K. Pedersen . Analysis Now. Springer-Verlag, New York, 1989.
[1 50] J .-P. P enot. Is convexi ty useful for the study of monotonicity. In RP.Agarwal and D. O 'Regan, eds: Nonl in ear Analysis and Applicat ions,pages 807~822 . Kluwer , Dordrecht , 2003.
[1 51] A.L . Peressini . Ordered Topological Vector Spaces. Harper and Row ,New York, 1967.
[152] A.L. Peressini , F .E. Sullivan, and J .J. Uhl. Th e Math ematics of Non lin ear Programming. Springer-Verlag , New York, 1988.
[153] R.R Phelps. Convex Functions, Monotone Operato rs, and Differentiability, volume 1364 of Lecture Not es in Mathematics. SpringerVerlag, New York, 1989 .
[154] RR Phelps. Lectures on maxim al monotone op erators.arXiv :math.FA/9302209, 1993.
[155] RA. Poliquin and RT. Rockafellar. Amena ble functions in optimizat ion . In F . Gianness i, editor , Nonsmootli Optimizat ion Methods andApplications, pages 338-353. Gordan and Breach, Philad elphia, 1992.
[156J RA. Poliquin and RT. Ro ckafella r . Prox-regular functions in variational analysis. Transactions of the American Mathema tical Society,348:1805-1838, 1996.
[157] R.A . Poliquin, RT. Rockafellar, and 1. Thibault . Local differentiability of distance fun ctions. Transactions of the A m erican Math ematicalSociety , 352:5231 -5249, 2000.
286 Bibliography
[158] B.H. Pourciau . Modern multiplier rules . American MathematicalMonthly, 87:433-452, 1980.
[159] B. Pshenichnii . Necessary Conditions for an Extremum. MarcelDekker, New York , 1971.
[160] J. Renegar . Linear programming, complexity theory and element aryfunctional analysis. Math ematical Programming, 70:279-351 , 1995.
[161] S.M . Robinson. Normed convex processes. Transactions of the American Mathematical Society, 174:127-140, 1972.
[162] S.M. Robinson. Regularity and stability for convex multivalued functions. Mathematics of Operations R esearch, 1:130-143, 1976.
[163] S.M . Robinson. Stability theory for systems of inequalities, part II:differentiable nonlinear systems. SIAM Journal on Numerical Analysis, 13:497-513, 1976.
[164] S.M . Robinson. Normal maps induced by line ar transformations.Mathematics of Operations Research, 17:691-714, 1992.
[165] RT. Rockafellar. Level sets and continuity of conjugate convex fun ctions. Transactions of the American Mathematical Society , 123:46-63,1966.
[166] R .T. Rockafellar. Monotone Processes of Convex and Concave Type.Memoirs of the American Mathematical Society, 1967. No. 77.
[167] RT. Rockafellar. Convex Analysis. Princeton University Press,Princeton, N.J. , 1970.
[168] RT. Rockafellar and RJ .-B. Wets. Variational Analysis. Springer,Berlin, 1998.
[169] H.L . Royden. Real Analysis. Macmillan, New York, 1988.
[170] W . Rudin. Real and Complex Analysis. McGraw-Hill, New York,1966.
[171] S. Simons. Minimax and Monotonicity. Springer-Verlag, Berlin, 1998.
[172] S. Simons. Dualized and scaled Fitzpatrick functions . Proceedings ofthe American Mathematical Society, 2005 . To appear.
[173] M. Slater. Lagrange multipliers revisited : a contribution to non-linearprogramming. Cowles Commission Discussion P aper Math. 403,1950 .
Bibliography 287
[174] D.R Smart. Fixed Point Th eorems. Cambridge University Press ,London, 1974.
[175] RJ . Stern and H. Wolkowicz. Indefinite trust region subproblemsand nonsymmetric eigenvalue perturbations. SIAM Journal on Optimizat ion , 5:286-313, 1995.
[176] RE. St euer . Multiple Criteria Optimization : Th eory, Computation.and Application. Wiley, New York , 1986.
[177] K.R Stromberg . An Introduction to Classical Real Analysis. Wad sworth , Belmont , CaL , 1981.
[178] F. E. Su oBorsuk- Ulam implies Brouwer : a direct constru ct ion . American Mathematical Monthly, 109:855- 859, 1997.
[179] C. Swartz. An Introduction to Functional Analysis. Marcel Dekker ,New York, 1992 .
[180] C.M. Theob ald . An inequality for the trace of t he product of two symmet ric matrices . Mathematical Proceedings of the Cam bridge Philosophical Society, 77:265-266, 1975.
[181] J . Van Tiel. Convex Analysis: An Introductory Text . Wiley, NewYork , 1984 .
[182] H. Uzawa. The Kuhn-Tucker theorem in concave programming. InK.J . Arrow, L. Hurwicz, and H. Uzawa, editors, Studi es in Linearand Nonlinear Programming, pages 32-37. Stanford Univers ity P ress,St anford, 1958.
[183J L. Vandenberghe, S. Boyd , and S.-P. Wu. Determinant maximizationwith linear matrix inequality constraints . SIAM Journal on MatrizAnalysis and Applications, 19:499- 533, 1998.
[184] J . von Neumann . Some matrix inequalities and met rizat ion of matricspace. Tom sk University Review, 1:286-300, 1937. In : CollectedWorks, Pergamon, Oxford, 1962 , Volume IV, 205-218.
[185] J . von Neumann and O. Mor genstern. The Th eory of Games andEconomic Behaviour. Princeton Univers ity Press , Princeton, N.J .,1948 .
[186J H. Weyl. Elementare Theorie der konvexen Polyeder. CommentariiMath. Helvetici, 7:290 -306, 1935.
[187J S..1. Wright . Primal-Dual Interior-Po int Methods. SIAM, Philadelphia , 1997 .
288 Bibliography
[188] K. Yosida. Functional Analysis. Springer-Verlag, Berlin, 1995.
[189] E. Zeidler . Nonlinear Functional Analysis and its Applications I.Springer-Verlag, New York, 1986.
Index
Page numbers in italics referto principal references.
absolute ly continuous , 214absorbing set , 244abstract linear program, 110,
111Accessibility lemma, 7, 83act ive
const raint , 30set , 29, 98, 126, 160, 176
adjoint, 3, 12ffprocess , 115-122
affinecombination, 7fun ction, 3, 203
conjugate , 79hull , 7minorant , see minorant ,
affineset , 7
normals to , 19almost homogeneous, 81amenable, ix, 228-232analyt ic centre, 62approximate
crit ical point, see crit icalpoint, approximate
selection, 191-193approximately convex, 224arit hme t ic-geome t ric mean, 5, 12Asplund, 221att ainment, 4
and lower semi continuity,248
289
at tainment (cont .)du al,90in best approximat ion, see
distance fun ct ionattainment
in Fenchel problems, 52in LP and SDP, 109-113primal ,90quadratic program, 175, 205
Bairecategory theorem , 199, 244space, 239
ball , 1Banach
cont raction principle,179- 187
in metric space, 250space, 239-250
barreled, 239, 244base, see cone, base forBasic separation theorem , see
separ ation, BasicBau schke, Hein z, viiiBayes condit ional probability
rule, 86best approximation, 19BFGS update, 21, 62biconjugate, see Fenchel
biconjugateof process , 119
bipolarcone, see cone, bipolarset, 67, 69, 118
Birkhoff's theorem , 10, 12, 74
290
Bish op -Phelps theorem, 249, 250Bol tzm ann-Shan non entropy,
55, 245Bolzano
- Poinoare-Mirandatheorem, 206
- Weierst rass theorem, 3, 4,248
Borel measurable, 214bornology, 240Borsuk, 182
- Ulam theorem , 187-189Borwein- Preiss var iation al
principle, 250Bose-Einst ein entropy, 55Bouli gand, 140boundary, 2
properties, 142, 242bounded , 3ff
converge nce theorem, 215convex fun ctions, 242level set, see level set ,
boundedprocess, 117-122range, 210- 211set of multipliers, 162subd ifferent ial map, 242
Boyd , ixBregm an distance, 39broad crit ical cone, 174-177Brouwer , 182
fixed point theorem, 79,179-192, 206, 209-211
in Banach space, 250Browder-Kirk theorem , 183Bronsted-Rockafellar theorem ,
250
calculus of vari ati ons, viiiCanad ian , ixCaratheo dory's theorem, 25, 75,
225Cartesian product , Iff
tange nt cone to , 141
Index
Cauchy- Schwarz ineq uality, 1,10, 12, 63
and steepest desce nt , 31Cellina approximate selection
t heorem , 191-193,250cent ral path, 112
for LP, 112for SDP, 113
chain rul e, 52, 151, 166, 217,228, 234
change of vari able theorem,180-183
Ch ebyshev set , ix , 218-227Chi, Lily, see HedgehogClarke, ix
directional de rivat ive,124-144
normal cone , 139, 218calc ulus , 158versus lim iting, 170
subdifferential, 124-152 , 218and gene ric
di fferenti ability, 197and prox imal normals,
223as a cusco , 190, 194intrinsic, 133-135, 145,
149, 213-216of composit ion, 151of sum , 125
subgradient, 124unique, 132-134
tangent cone, 137-141, 170and met ric regu lar ity, 167and transvers ality,
158-159closed
fun cti on , see lowersemicontinuous vers usclosed fun ction
graph t heo rem, 118, 249images , 194level set , 76
Index
closed (cont.)multifunction, see
multifunction, closedrange, 241set, 2ffsubdifferential, see
subdifferential, closedmultifunction
subspace, 241closure
of a function, 78of a set , 2
codimensioncountable, 244finite, 246
coercive, 20multifunction, 202, 207
cofinite, 83Combettes, Patrick, viiicompact, 3ff
and Ekeland principle, 157convex hull of, 26convex sets, 67countably, 248general definition, 186, 191images, 194in infinite dimensions, 239level sets, see level set ,
compactoperator, 249polyhedron, 98range of multifunction, 194unit ball, 241weakly, see weakly compact
compactly epi-Lipschitz , 248complement, orthogonal, 3complementarity
problem, 202-208semidefinite, see
semidefinitecomplementarity
strict , 234-237complementary slackness, 29, 43,
45, 113
291
complementary slackness (cont.)in cone programming, 112
complemented subspace, 239complete, 182, 239, 244
and Ekeland's principle, 250composing
convex functions , 6USC multifunctions, 194 ,
200concave
conjugate, 62function , 33
condition number , 122cone, 1, 9ff
and processes , 114base for , 60
compact, 81, 121, 141bipolar, 53, 54, 58, 67, 76,
110, 115, 139contingent, see contingent
conecritical, see critical conedual,27finitely generated, see
finitely generated conegenerating, 120infinite-dimensional, 249lattice, 9nonnegative, 245normal , see normal conenormality of, 120open mapping theorem, 85partial smoothness of, 234pointed, 54, 60, 72, 86 , 99 ,
120, 141polar , vii, 53, 54, 67, 69, 160
of sum and intersection,58
polyhedral , see polyhedralcone
program, 110-113, 115, 116pseudotangent , 142recession, 5, 6, 61, 83 , 143self-dual , see self-dual cone
292
cone (cont.)semidefinit e, see
semidefinite conesums, 58, 245support function of, 55tangent, see t angent conevariational inequality over ,
202conical absorption, 71conjugate, see Fenchel conjugateconnected, 180, 183constraint
active, 30equality, 29, 153, 160error, 168function , 29inequality, 15-32, 160
and partial smoothness,234
convex, 43in infinite dimensions, 248
linear, 16, 19, 21, 52, 53, 62,109, 162
in infinite dimensions, 247qualification, 30, 200, 234
equivalence of Slater andMangasarian et al, 45
in cone programming, 116infinite-dimensional, 250linear independence, 30,
160, 162, 176Mangasarian- Fromovitz,
30-32, 12~ 160-165Mangasarian-Fromowitz,
232Slater, 44-47, 90, 91, 109,
110, 162, 168contingent
cone , 138-144, 153-162,228to feasible region, 160,
232necessary condition, 139,
157, 160, 161sufficiency, 143
Index
continuity, 3ffabsolute, 214and bounded level sets, 78and maximality, 198and USC, 193generic, 199in infinite dimensions ,
239-250of convex fun ctions, 52,
65-69, 76, 241failure, 84univariate, 83
of extensions, 196of linear functionals , 241of multifunctions, 114of partial derivatives, 132of projection, 20, 201of selections, 191-199
continuously differentiable, seedifferentiability,continuous
contour, 82contraction, 179
Banach space, 250non-uniform, 183
control theory, vii , 112convergent subsequence, 3ffconvex
analysis, vii-ixinfinite-dimensional, 68,
249monotonicity via, 209-211polyhedral, 97
approximately, 224calculus, 52, 53, 56, 139
failure , 57sum rule, 52
combination, 2, 5ffconst raint, 43fffunction, 4, 33, 44ff
bounded, 242characterizations, 37composition, 6
Index
convex fun ction (cont .)condit ions for minimizer ,
16cont inuity of, see
continuity of convexfunctions
cr it ical points of, 16, 33difference of, 57 , 108, 219differentiabili ty of, 36directional derivat ive, 16examples, 39exte nded-valued, 33 , 43,
46ffHessian characteri zation ,
see Hessian, andconvexity
of matrices , 40on Banach space, 249recogni zin g, 37 , 38regu larity, 131, 138sy mmetric , 27
growth cond it ions , 7hull, 2, 5ff
and exposed points, 249and extreme p oints, 68and .Gordan 's theo rem, 23of lim iting subdifferenti al,
145, 149im age, 190-199log- , 41midpoint , 80multifunction, 114 , 115order- , see order-convexprocess, see processprogram, 43-48, 51ff
duality, 88Schur-, 25, 27, 38 , 108, 135set, 2ffspectra l fun ction, see
spect ral fun cti onstrictly, see strictl y convex,
essentia llysubd ifferentia l, 131ff
and limiting, 145
293
convex ified represent ative,209-210
convexity, see convexand continuity, see
continuity of convexfun cti ons
and differenti ability, 15and monotonicity, 129in linear spaces, 249in optimization, 33ffof Chebys hev sets, ix ,
220 -227core, 34
in infin ite dimensions, 244versus interior, 37, 67
cost funct ion , 204countable
basis, 244codime nsion, 244
countably compact, 248cover , 191, 248crit ical cone
broad, 174-177narrow, 172-177
crit ica l point , 16approximate, 17st rong , 234- 238unique, 19
curvature, 172cusco, 190- 211 ,219-223
DAD problems, 42, 62Davis ' theor em , 105, 106, 108DC function , see convex
fun ction, difference ofDebrunner-Flor exte ns ion
theorem , 209-211dens e
hyp erplane, 244range, 246, 248subspace, 245
der ivative, see differe nt iabilitydirectional , see direct ion al
de rivative
294
derivative (cont .)Frechet , see Frechet
derivativeGateaux, see
differentiability,Gat eaux
generalized, 123Had amard, see Had amard
derivativestrict , see strict derivativeweak Hadamard, see weak
Hadamard derivativedet erminant , 9, 163, 180, 183
order preservation, 107Deville-Godefroy-Zizler vari
at ional principle, 250differentiability
and pseudoconvexity, 143bornological , 240cont inuous, 132-134, 157,
159, 164approximat ion by, 180
Frechet , see Frechetderivative
Gateaux, 15, 28, 61,130-136, 139, 240-245
generic, 197, 199of convex funct ions , 36, 82of dist ance funct ion , 57,
218,222of Lipschitz functions, ix,
133of spectral funct ions , 105of the conjugate , 225strict , see st rict derivativetwice, 172-176, 231, 233
differential inclusion , 249dimension, 68
infinite, see infinitedimensions
Dinicalc ulus, failure, 128derivative, 127, 214
Index
Dini (cont.)directional derivative, 145,
147and contingent cone , 137cont inuity, 128, 146Lipschitz case , 123, 129,
131subdifferential , 124, 125,
129, 131, 145of distan ce function, 169,
218surjective , 128
subgradient, 124, 146, 148exists densely, 135, 145,
150Dirac, see Fermi-Diracdirecti on al der ivative, 15, 61
and subgradients, 35, 123and tangent cone, 137Clarke, see Clarke
directional derivativeDini , see Dini
subdifferenti alMichel-Penot, see
Michel-Penotdi rectional derivative
of convex fun ction, 34- 42of max-functions, 28, 33, 38sublinear, 34, 123, 124, 137
disjoint op erator ranges, 248distance
Bregm an, 39from feasibility, 168function, 57, 133, 137-144,
218-226attainme nt, 241, 243, 248differ ent iability, 241di rectional derivative , 144regularity, 138, 246subdiffere nt ials, 169to level set, 171
to inconsistency, 119, 122divergence bounds, 63
Ind ex
domainof convex function , 33 , 44,
65of mu lt ifunction, 114of subdifferential, 35
not convex , 40po lyhedral, 97
doubly stochastic , 10, 12, 75pattern , 42
Dowker , 196dua l
attainment , see attainment,dual
cone, 27func t ion, 88linear progr am, 109, 202problem, 88
examples, 91solut ion, 82, 89, 90space, 239value , 52, 88-96
in LP and SD P , 109-113du ality, vii, 2, 76ff
cone program , see coneprogram
duali ty-based algorit hms , viiFenche l, see Fen chel duali tygap, 88-96
Duffin's, 46, 92in LP and SDP, 110-113,
203geometric progr amming, 103in convex programming, 88infinite-dimensional , 91, 249Lagr angian, see Lagrangian
dualityLP, vii, 25, 109-113, 202nonconvex, 93norm, 117process, 114-122quadratic progr amming, 205SDP, vii, 109-113st rict-smoot h , 78, 82
295
du ali ty (cont.)weak
cone program, 109, 110Fenchel, 52-53, 101Lagrangian , 88 , 91
Duffin 's duality gap , see du ali ty,gap , Duffin 's
efficient , 204eigenvalues , 9
derivati ves of, 135functions of, 104isotonicity of, 136lar gest , 162of operators, 249op timizati on of, 106subdifferenti als of, 135sums of, 108
eigenvector , 19, 163Einstein , see Bose- EinsteinEke land variat ional pri nciple, 17,
153-157, 179, 224, 225in metric space, 250
engineering , ixentropy
Boltzmann- Shann on , 55Bose- Einstein , 55Fermi- Dirac, 55maximum, 41, 56, 62
and DAD problems, 42and expected sur prise, 87
epi-Lipschitz-like, 248epigraph, 43ff
as multifunction graph , 193closed , 76, 81normal cone to , 47polyhedral , 97regula rity, 246suppo rt functi on of, 55
equilibrium , 193equivalent no rm , see norm ,
equivalentessent ially smooth, 37, 74, 80
conjugate, 78, 82
296
essentially smooth (cont.)log barriers, 51minimizers, 40spectral functions , 106
essent ially strictly convex, seestrictly convex,essentially
Euclidean space, 1-9, 239subspace of, 24
exact penalization, seepenalization, exact
existence (of optimal solution,minimizer) , 4, 79, 90ff
expected surprise, 86exposed point, 73
strongly, 249extended-valued, 145
convex functions , see convexfunction ,extended-valued
extensioncontinuous, 196maximal monotone, 210
extreme point, 67existence of, 73of polyhedron, 98set not closed, 73versus exposed point , 73
Fan- Kakutani fixed point
theorem, 190-201, 203inequality, 10-14, 104, 105minimax inequality, 205theorem, 10, 13
Farkas lemma, 23-25, 109, 160and first order conditions,
23and linear programming,
109feasible
in order complementarity,205
region, 29, 160
Index
feasible region (cont .)partly smooth, 234
solution, 29, 43, 110Fenchel, ix, 54
- Young inequality, 51,52,71, 105
-Young inequality, 225biconjugate, 49, 55, 76-85,
99, 105, 106, 126and duality, 89and smoothness, 225, 226
conjugate, 23, 49-63and Chebyshev sets, 221and duality, 88and eigenvalues, 104and subgradients, 51examples, 50of affine function, 79of composit ion, 93of exponent ial, 49, 56, 62,
63of indicator function , 55of quadratics, 55of value function, 89self-, 55strict-smooth duality, 78,
82transformations, 51
duality, 52-63, 73, 77, 81,102
and complementarity, 204and LP, 110-113and minimax, 201and relative interior, 74and second order
conditions, 174and strict separation, 70generalized, 102in infinite dimensions,
239,249linear constraints, 53, 62,
71, 100polyhedral, 100, 101symmetric, 62
Index
Fenchel d ua lity (cont.)versus Lagr angian, 93
problem , 52Fermi- Dirac entropy, 55F illmore-W illia ms theorem , 108finite cod imension, 246finite dimensions, 239- 250fini tely generated
cone, 25, 26, 97-99function, 97-101set, 97-101
first order cond it ion (s)and max-functions , 28- 32and the Farkas lemma, 23for optimality, 16Fri t z John, see Fritz John
condit ionsin infinite dimensions, 248Karush-Kuhn-Tucker , see
Karush - Kuhn- Tuckertheorem
linear constraints , 16, 19,21,42
necessary, 15, 16, 29, 139,160,174, 175
sufficient , 16Fish er informati on , 79, 84, 87F itz patrick fun ction, 209fixed point , 179-211
in infin ite dimensions , 250methods, 203property, 184theorem
of Brouwer , see Brouwerfixed point theor em
of Kakutani-Fan, seeKakutani-Fan fixedpoint t heorem
Fourier identification, 246Frechet derivati ve, 132-134, 153,
213-216and continge nt necessary
condition , 139, 157and inversion, 184-185
297
Frechet derivative (cont.)and mult ip liers , 163an d subderivatives , 152in constraint qualification ,
160in infini te d imensions,
240- 250Fri t z J ohn conditions , 29- 31,
130, 165and Gordan 's t heorem, 30nonsmooth, 127second or der , 175
Fubini 's theo rem, 214- 216fun ctional analys is, 239, 248fundam ental theorem of
ca lculus , 214, 216fur thest point , 73, 221, 226-227fuzzy sum rule, 146, 148, 150
G8, 191, 197gamma function , 41Gateaux
derivati ve, see de rivative ,Gateaux
different iable, seedifferentiability,Gateaux
gauge function , 66 , 71, 184genera lized
deriva ti ve, 123Hessian, 217J acobi an, 216-217
generated cuscos, 196gen erating cone , 120generic, 190
cont inuity , 199differenti abili ty, 197, 199single-value d , 199
geometric progr amming, 101,102
global minimizer, see minimizer ,global
Godefroy, see Deville-GodefroyZizler
298
Gordan's theorem, 23-27and Fritz John conditions,
30graph, 114, 190
minimal , 197normal cone to , 150of subdifferential , 145
Graves , 157Grossberg, see Krein-GrossbergGrothendieck space, 243growth condition, 4, 20
cofinite, 83convex, 7multifunction, 201, 208
Guignardnormal cone calculus, 158optimality conditions, 164,
177
Haberer , Guillaume, viiiHadamard, 182
derivative, 240-250inequality, 48, 163
Hahn- Banach extension, 55, 58
geometric version, 248- Kat et ov- Dowker sandwich
theorem, 196Hairy ball theorem, 186-187halfspace
closed, 3, 25ffin infinite dimensions, 246op en, 23, 25support function of, 55
Halmos, viiiHardy et al. inequality, 10-12Hedgehog theorem, 187hemicontinuous, 198Hessian, 17, 172-176
and convexity, 37, 38, 40generalized, 217
higher order optimalityconditions, 175
Hilbert space, 221, 239
Index
Hilbert space (cont.)and nearest points , 249
Hiriart.-Urruty, vii, 25Holder's inequality, 31, 41, 71homeomorphism, 182, 184homogenized
linear system, 109process , 120
hypermaximal, 198, 207, 208hyperplane, 2, 25ff
dense, 244separating, see separation
by hyperplanessupporting, 67, 122,
240-249
identity matrix, 9improper polyhedral function ,
101incomplete, 239inconsistent, 29, 111
distance to, 122indicator function , 33, 67, 137
limiting subdifferential of,146
subdifferential of, 37inequality constraint, see
const raint , inequalityinfimal convolution, 57, 137, 157infimum, 3ffinfinite-dimensional, viii , 79,
157 , 239-250interior, 2ff
relative, see relative interiortangent characterization,
170versus core, see core versus
interiorinterior point methods, vii-ix,
54, 79, 91, 162inverse
boundedness, 120function theorem, 159, 184,
235
Index
inverse (cant .)image, 3, 100Jacobian , 180multifunction , 114- 122
inversion, 221, 226Ioffe, 149isomet ric, 85, 86isotone, 6, 205
cont ingent cone , 141eigenvalues, 136tangent cone, 143
J acobian, generalized , 216-217James t heore m, 243, 249Jordan 's theorem, 163Josephson-Nissenzweig
sequence, 243theorem , 243
Kakut ani- Fan fixed point t heorem,
190- 201, 203minimax t heorem , 96, 206
Karu sh-Kuhn-Thckertheorem, 30-32, 130, 160
convex case, 43- 45, 131infini te-dimension al , 250nonsmooth, 127
vector, 47, 93Katetov, 196Kirchhoff 's law, 20Kirk , see Browder-KirkKlee cavern, 221Knaster- Kur atowski-Mazurkie
wicz principle, 185 , 206Konig, 11Kr ein
- Grossberg t heore m, 120- Rut man theorem, 54, 158-Rutman t heo rem , 230
Kruger, 149Kuhn, see Karush-Kuhn-Tucker
Lagrange mult iplier , 17, 29-32,161
299
Lagrange multiplier (ca nt .)and second order
condit ions , 172-176and subgradients, 43bounded set, 162convex case, 43-47in infinit e d imensions, 249nonexistence, 46, 163, 185
Lagrangian , 29, 172- 176convex, 43, 88du ality, 88-96, 103
infinite-di mension al , 249linear progr amming, 109
necessary condit ions, seenecessary condit ions,Lagrange
sufficient conditions , 43- 48,107
Lambert W-function, 58lat tice
cone, 9ordering, 11, 203
Ledyaev, ixLegendre, 54Lemarechal, Claude, vii , viiilevel set , 3, 13
bounded , 4, 7, 69, 78, 83closed , 76compact, 20, 41, 51, 95, 154
of Lagrangian , 90, 91distan ce to , 168norm al con e to , 47, 171
Ley, Olivier , viiilimit (of seq uence of points) , 2limiting
mean value t heorem , 151norm al cone, see normal
cone, limi tingsubdifferent ial, 145-152
and regul arity, 166-171of compos it ion, 150of distance fun ction , 170sum rule, see nonsmooth
calculus
300
line segme nt, 2, 142lin eality space, 34linear
constraint , see constraint ,linear
fun ctionalcont inuous , 241discontinuous, 248
independence qualifi cation ,see constraintqualification, linearindependence
inequality constraints, 62map,3ff
as process, 120object ive , 109operator, 249progr amming (LP), vii, 54,
91abs t ract, 1l0, Ill , 202and Fenchel du ali ty,
1l0-1l3and processes, ll4and variational
inequaliti es, 203duali ty, see du ali ty, LPpenalized , 91, ll3, 162primal problem , 109
space, 249span , 2subs pace, 2
linearization, 153Lipschitz, ix , 65,66,68, 123-152,
155, 183, 213-217bomological derivatives, 240eigenvalues, 108, 135extension, 157gene ric differentiability, 197non- , 127perturbation, 250
Liustemik, 157t heorem, 156, 158, 160
via inverse funct ions, 159local minimizer , 15-19, 29ff
Index
local minimizer (cont .)strict , 174
localiza tion , 120locally bounded , 65, 66, 68, 78,
190, 194, 196-198locally Lipschitz, see LipschitzLoewn er orde ring , 9log, 5, 13, 49, 55, 62, 92, 104log barrier , 49log det , 13, 15, 20, 21, 32, 37, 40,
46, 48, 49, 55, 69, 92,104-106
log-convex , 41logarithmic homogeneity, 79, 82lower norm, ll7lower semi continuous, 37, 76-81 ,
101and attainment, 248and USC , 193approximate minimizers,
153ca lculus , 146-148gener ic cont inuity, 199in infinite dimensions , 239mul tifuncti on , 114sandwich theorem, 196value fun cti on , 89, 90, 95versus closed fun cti on , 76
LP, see linear programmingLSC (multi function) , ll4-119,
190, 192-196, 204Lucet , Yves , viii
Mangasarian- Fromovitzcons traint qualification,see const raintqu alifica tion ,MangasarianFromovitz
manifold , 233-238mathem ati cal economics , viii ,
ll9, 193matrix , see also eigenvalues
analysis, 104
Index
matrix (cont.)completion , 21, 40optimization, 109
Max formula, 36- 42, 53, 61 , 116,123
and Lagrangian necessarycondit ions , 44
nonsmooth, 124, 125, 139relativizing, 42, 74
max-function(s)and first order condit ions,
28-32directional derivative of, 28subdifferential of, 47 , 59,
125Clarke, 129, 151limiti ng, 151, 171
maximal monotonicity, 190-211maximizer , vii , 3ffmaximum ent ropy, see entropy,
maximumMa zur kiewicz, see
Kn as ter-KuratowskiMazurkiewicz
mean value t heorem, 128, 136,217
infinite-d imensional, 250limi ting , 151
measure t heory, 213-217metric regularity, vii, 153-159,
183 , 184, 229and second order
conditions, 172- 173and sub differentials,
166-171in Banach space, 250in infinite dimensions , 239weak, 154-158
metric space , 250Michael select ion theorem ,
193-196, 204infinite-dimension al, 250
301
Michel-Penotdirectional derivative,
124-144subdifferential, 124-135subgrad ient , 124
unique, 130, 132midpoint convex , 80minimal
cusco, 219 -223graph, 197solut ion in orde r
compleme ntarity, 205minimax
convex-con cave, 95Fan 's inequality, 205Kakutani's theorem, 96, 206von Neumann 's theorem,
see von Neumannminimax t heorem
minimizer , vii , 3ffand differen ti ability, 15and exact penalization, 137app roximate , 153existence , see existenceglobal, 4, 16, 33fflocal, 15-19, 29ffnonexistence, 17of essentially smooth
fun ctions, 40strict , 174subdifferential zeroes, 35 ,
123minimum volume ellipsoid, 32,
40, 48Minkowski , 5, 101
theorem, 68, 73, 98 , 182converse, 73in infinit e dimens ions, 249
minorant , 76affine, 76, 79, 84, 100closed , 78
Miranda, see Bolz ano-PoincareMiranda
monotonically re lated, 209-210
302
rnonotonicityand convexity, 129maximal. 190-211multifunction , 190-211of complementarity
problems, 203, 204of gradients, 40via convex analysis , 209-211
Mordukhovich , 149, 169Moreau, 54
- Rockafellar theorem ,78-83, 249
Motzkin-Bunt theorem, 221multicriteria optimization, 140multifunction, vii, 114-122 ,
190-211closed , 80
and maximal monotone,198
versus USC, 193set-valued map , 114subdifferential, 35
mult iplier , see Lagrangemultipli er
mult iva luedcomplementarity pro blem ,
202variational inequali ty, 200
narrow crit ical cone, 172-177Nash equilibrium, 204, 206near est point , 19, 24, 57, 182,
188and prox-regularity, 228and proximal normals ,
218-223and subdifferent ials, 169and variat iona l inequali ties,
200in epigraph, 135in infinit e dimensions , 239,
248in polyhedron, 62proj ection , 20, 211
Index
nearest point (cont.)selection, 193, 199un iqu e, see also Chebyshev
set , 228necessary condit ion(s) , 125, 138
and subdifferenti als , 123and sufficient, 175and variati on al inequalities,
200contingent , see cont ingent
necessary conditionfirst or der, see first order
condit ion (s), necessaryfor optimality, 16Fritz John, see Fritz John
condit ionsGui gnard , 164, 177high er order , 175Karush -Kuhn-Tucker , see
Karush- Kuhn- Tuckert heore m
Lagrange, 44-46, 49, 89,130, 131
nonsmooth, 126, 130, 139,145, 149, 151
limi ting and Clarke, 170second orde r, 172stronge r, 127, 145
neighbourhood , 2Nemirovski , ixNesterov , ixNewton-type methods, 172Nikodym, see Radon-NikodymNissenzweig, see
J osephson-Nissenzweignoncompact variat ional
inequality , 202nondifferenti able, 18, 33ffnonempty images , 114, 118nonexpansive, 180 , 182, 220
in Ban ach space, 250nonlinear
equation, 179progr am, 160, 177
Index
nonnegative cone , 245nonsmooth
analysis , viiffand metric regul arity, 157infinite-dimensional , 149Lipsch itz , 137
calculus, 125, 128, 139, 155and regul arity, 133equality in , 131failure , 145, 149fuzzy, 146infinite-dimensional , 250limiting, 145, 148-151 ,
167, 169mixed , 134nor me d fun ction, 166sum rule, 125
max formulae, see Maxformula , nonsmooth
necessary conditions, seenecessary condition(s) ,nonsmooth
optimization, seeoptimization,nonsmooth
regularity, see regularnorm, 1
-attaining, 240, 243 , 245,249
-preserving, 10, 12equivalent , 66, 69 , 192lower , 117of linear map, 117of process, 117-122smooth, 188strictly convex, 249subgradients of, 38top ology, 241-243upper , 117
normalproximal , ix, 218-223
normal cone, 15, 16, 18and polarity, 53and relative int erior, 74
303
normal cone (con t.)and subgradients, 37, 56and tange nt cone, 54Clarke, see Clarke normal
coneexamp les, 18limiting, 146, 166-171
and subdifferential , 150proximal , 218to epigraph, 47to graphs, 150to intersecti on , 56, 86to level sets, 47
normal mapping, 200, 203normal problem, 91normal vector, 15, 218normed space, 2:{9, 244null space, 3, 116, 117
objective fun ct ion , 29, 30fflinear , 109
one-sided approx imat ion , 35open, 2
functions and regularity,169, 183
mapping theorem, 71, 82 ,101, 110, 120
for cones, 85for processes, 118in Banach space, 249in infinite dimensions, 239
multifunction, 114-121operator
linear , 249op timal
cont rol, viiisolution , 4ffvalue, 51, 88-96, 100, 174
function , 43in LP and SDP, 109-113,
202primal , 52
optimality conditi ons, vii, 15-22and the Farkas lemma , 24
304
optimality conditions (cont.)and variational inequ alit ies ,
200first order, see first order
condition(s) foroptimality
higher order, 175in Fenchel problems, 56, 82necessary, see necessary
condit ion(s)nonsmooth, 123second order, see second
order conditi onssufficient, see sufficient
condit ion (s)optimizat ion, vii, 3ff
and calculus, 16and convexity, 33and nonlinear equations,
179computational, vii , 162, 172duality in, 76, 88infin ite-dimensional , viii, 79,
157linear , 109matrix, 109mult icriteria, 55nonsmooth, 28, 33, 123-152
infini te-dimensional , 250one-sided approximati on , 35problem , 4, 29ffsubgradient s in , 35, 123vector, 72, 140, 141
order-convex , 59-62, 72, 80, 108-reversing, 49-sublinear , 59- 62, 108, 121-theoreti c fixed point
results, 179complementarity, 203- 205epigraph, 121infimum, 60interval, 120preservation, 11, 72
Index
order preservat ion (cont.)of determinant , 107
statist ic, 129regular ity, 135subdiffer ential , 152
subgradients, 55, 60-62ordered spectral decomposit ion ,
10ordering , 9
lat ti ce, 11orthogonal
complement, 3invariance, 107matrix, 10, 182proj ection , 25similarity t ransformation,
107to subspace, 24
orthonormal basi s, 163
p-norm , 31, 71par acompact , 250Pareto minimization, 72, 204
proper , 141parti tion of uni ty, 191-195, 207partly smooth , ix , 233-238pen alizati on , 91, 113, 162
exact, 137-140, 155, 158,167, 229
qu ad rati c, 164Penot , see Miche l-Penotpermut ati on
matrix, 10, 27, 75, 108perturbati on , 43, 51ffPhelps, see Bishop-Phelpspiecewise linear , 184Poincar e, see Bolzano- Po incare-
Mirandapoin ted , see cone, pointedpointwise maximum, 79polar
calc ulus, 70, 117concave, 85cone, see con e, polar
Index
polar (cont.)set , 67, 69~70
polyhedralalgebr~ 100-101, 116calculus , 101complementarity problem ,
205cone , 98, 102, 110 , 113, 161Fenche l duality, 100fun cti on, 97-102mult ifunction, 114problem, 109, 110process, 116qu asi- , 175set, see pol yhedronvariat ional inequali ty, 203
polyh edron , 3, 9, 11, 58, 97- 102compact, 98in vector optimization, 141infinite-dimensional , 246nearest poin t in , 62partial smoothness of, 237po lyhedral set , 97tangent cone to, 101
polynomialnearest , 21
polytope, 55 , 97-99in infinite dimensions, 246
positive (semi)de finite, 9ffpos it ively hom ogeneous , 33P reiss, see Borwein-Proisspr imal
linear progr am, 109problem , 88recovering soluti ons, 82semidefi nite progr am, 111value, see op timal value
process, 114-122, 249product , see Cartesian produ ctprojecti on, see also nearest poin t
continuity, 223onto subspace, 24orthogonal, 25re laxed, 182
305
properfunct ion , 33, 44, 76, 97, 116P areto minimization , 141point , 142
prox-regular, 228-238proximal nor mal, ix , 218-223pseudoconvex
fun ction, 143set, 142, 143
P shenichn ii- Rockafellarcondit ions, 58
qu adrat icapproximation, 172-175conjugate of, 55path, 173penalizati on , 164program, 91, 175, 205
quasi relative interi or , 244, 248qu asiconcave, 205quasipolyh ed ral , 175quotient space, 247
Rademacher's theorem , ix, 133,197, 213-216, 218
Radon- Nikodyrn prop erty, 249Radst rom cancellation, 5range
closed , 241dense, see dense range
range of mult ifun ction , 114, 191,194, 201
bounded , 210-211rank-one, 122ray, 242, 247Rayleigh quotient , 19real function , 123recession
cone, see cone, recessionfunction, 83
reflexive Banach space, 239- 249regular , 130-136, 138
and generic differe nt iability,197
306
regul ar (cont .)and Rademacher 's theorem,
216regul arity
condit ion, 30, 44, 65, 100,160
epigraphical, 246metric, see metric regul ari typrox- , see prox-regulartange nt ial, see t an genti al
regularityrelative interior , 5-8, 173, 185
and cone calculus, 159and cone programming, 113and Fenchel duality, 74, 102and Max formula, 42calc ulus, 74in infinite dim ensions , 242,
248qu asi , 244, 248
relaxed projection , 182representa tive, 209-210reso lvent , 208retraction, 180, 183reversing, 206Riesz lemma, 188Robinson , 119, 157Rockafellar, vii- ix, 54, 58, 78,
119, 250Ru tman, see Krein-Rutman
saddlepoint, 95, 96, 201Sandwich theorem , 58, 210
Hahn-Katetov-Dowker, 196scalarization, 72, 140, 142Schur
-convexity, see convex,Schur-
space, 243Schwarz , see Cauchy- Schwa rzSDP, see semidefinite progr amsecond order condit ions , 17,
172-177, 237selection , 190-1 99
Index
self map, 179-1 88, 207in Banach space, 250
self-conjugacy, 55self-dual cone , 18, 53, 54, 85,
105, 111selfadjoint , 249semidefini te
complementarity , 108, 208cone, 9, 18, 53, 54, 104, 106,
109,233matrix, 9progr am (SDP) , vii, 54, 92,
109-113, 162central path, 113
Sendov, Hristo, viiisensitivity analysis, 233separable, 62 , 92
and sem icontinuity, 247Banach space, 241-245
separation, 2, 5, 25ffand bipolars , 54, 67and Gordan 's theorem, 23and Hahn-Banach , 248and scalar ization, 142Basic t heorem, 2, 17, 77by hyperplanes, 3in infinit e dimensions, 241nonconvex , 142st r ict, 70strong, 6
set-valued map, seemulti fun ction
Shannon, seeBoltzmann-Shannon
signal reconstruction, 79simplex, 66 , 79simultaneous orde red sp ectral
decomposition , 10, 105single-valued, 190, 197
gene ric, and maximalmon otonicity, 199
sing ular value, 13largest , 163
skew symmetric, 197
Index
Slater condition, see constraintqualification, Slater
smooth Banach space, 241solution
feasible, see feasible solutionoptimal, 4ff
solvability of variationalinequalities, 201-208
spectralconjugacy, 104, 106, 107decomposition, 10, 19differentiability, 105function , 104-108, 133
convex, 105, 106subgradients, 105, 107theory, 249
sphere, 180, 186-189inversion in , 226
square-root iteration , 11st able, 91
Clarke tangent cone, 138steepest descent
and Cauchy-Schwarz, 31Stella's variational principle, 250St ern, ixStiemke's theorem, 26Stone-Weierstrass theorem,
180-183strict
compleme nt ar ity, 234-237derivative, 132-134, 149,
150, 155-167generic, 197
local minimizer , 174separation, 70
strict-smooth duality, 78, 82strictly convex, 4, 38-41
and Hessian, 38conjugate, 78, 82essentially, 35, 40, 84log barriers, 51norm, 249power fun ction, 21spectral functions , 106
307
st rictly convex (cont.)unique minimizer, 19
strictly differentiable, see strictderivative
strongcrit ical point , 234-238
subadditive, 34subcover, 191subdifferential , see also
subgradient(s)and essential smoothness , 74bounded multifun ction , 242calculus, 123Clarke, see Clarke
subdifferentialclosed multifunction, 80,
134, 145, 149, 156, 167compact ness of, 66convex, see convex
subdifferentialDini, see Dini
subdifferentialdomain of, see domain of
subdifferentialin infinite dimensions, 250inverse of, 80limiting, see limiting
subdifferentialmaximality, 240Michel-Perrot , see
Michel-Penotsubdifferential
monotonicity, 190, 197, 198nonconvex, 123nonempty, 36 , 240of distance functions ,
219-223of eigenvalues, 135of polyhedral fun ction, 102on real line, 149smaller , 145support function of, 55versus derivative, 123
subgradient (s) , vii , 35
308
subgradient(s) (cont .)and conjugation, 51and Lagrange mul tipliers, 43and lower semi continuity, 81and normal cone, 37, 56at op timality, 35Clarke, see Clarke
subgradientcons t ru ct ion of, 35Dini, see Dini subgradi entexistence of, 36, 43, 53, 100,
116in infinite dimensions, 239Michel-Penot, see
Michel-Penotsubgradient
of convex functions, 33-42of distance functions,
219-223of max-functions, see
max-function,subdifferenti al of
of maximum eigenvalue , 38of norm, 38of polyhedral function , 98of spectral functions, see
spect ral subgradi entsorder , see order subgradientunique, 36, 242, 245
sublinear, 33, 35, 58, 66, 69, 85,107, 108, 137
and support functions, 77directional derivative, see
directional derivative,sublinear
everywhere-finite, 77order- , 59-62recession functions, 83
subspace, 2closed , 241compleme nted , 239countable-codimensiona l,
244dense, 245
Index
subspace (cont .)finite-codimensional, 246projecti on onto, 24sums of, see sum of
subspacessufficient condit ion(s)
and pseudoconvexity, 143first order , see first order
condit ion(s) , sufficientLagrangian , see Lagrangian
sufficient condit ionsnonsmooth, 149partly smooth, 234second order, 174
sumdirect , 3of cones, see cone sumsof sets , 1of subspaces, 245, 247rule
convex, see convexca lculus
nonsmooth, seenonsmooth calculus
sun, 220-224support fun ction (s), 55, 80, 82
and sublinear fun ctions, 77directional derivative,
124-128of subdifferentials, 125
support poin t , 240-245suppor t ing
functional , 240-245hyperplane, see hyperplane,
supportingsupremum, 3
norm, 243surjective
and growt h, 198, 207and maximal monotone,
198, 208Jacobian, 155, 156, 159,
166, 173, 176, 184linear map , 71, 101, 110
Index
surjective (cont.)process, 114-122
surprise, expected, 86symmetric
convex function, 27function, 104-108matrices, 9-14set , 108
tangency properties, 241tangent cone , 137-144
and directional derivatives,137
as conical approximat ion,137
calculus, 73, 86, 159, 228Clarke, see Clarke tangent
conecoincidence of Clarke and
contingent, 138convex, 54, 74, 138ideal , 143intrinsic descriptions, 138,
140to graphs, 141, 150to polyhedron, 101
t angent space, 157tangent vector field , 186tangential regularity, 138, 156,
158, 229, 233, 246Theobald's condition, 13theorems of the alternative,
23-27, 97Todd, Mike, viiitrace, 9transversality, 158, 164, 228-232,
234trust region, 93Tucker, see
Karush-Kuhn- TUckertwice differentiable, see
differentiability, twic e
Ulam, 182
309
uniformbounded ness theorem, 249convergence, 180, 195multipliers, 176
uniquefixed point, 179, 183minimizer, 19nearest point, 248subgradient, see
subgradient, uniqueupper norm, 117upper semicontinuity (of
multifunctions) , 117Urysohn lemma, 196USC (multifunction), 190-207
value function, 43-48, 52, 88-91,116, 119
polyhedral, 100Vandenberghe, ixvariational
inequality, 200-208principle, 17
in infinite dimensions,239, 250
of Ekeland, see Ekelandvariational principle
vector field, 186-187vector optimization, see
op timization, vectorVille's theorem, 26viscosity subderivative, 149, 151von Neumann, 11
lemma, 13minimax theorem, 79, 81,
201, 204
Wang, Xianfu, viiiweak
-star topology, 241-243duality, see duality, weakHadamard derivative,
240-241
310
weak (cont.)metric regul arity, see metric
regularity, weakminimum, 72topology, 241-243
weakly compac t , 243, 249and nearest points, 249
Weierstrass , see alsoBolzano-Weierstrass ,Stone-Weierstrass
proposition, 4, 17ffWets, vii-ixWeyl, 101Williams, see Fillmore-WilliamsWolenski , ix
Young, see Fenchel-Young
Zizler , see Devill e-GodefroyZizler
Zorn 's lemma, 190, 210
Index