mixed integer conic optimization using julia and jump · mixed integer conic optimization using...

Mixed Integer Conic Optimization using Julia and JuMP

Juan Pablo VielmaMassachusetts Institute of Technology

3rd Los Alamos National Laboratory Grid Science Winter School & Conference,Santa Fe, NM, January, 2019.

Mixed Integer Convex Optimization (MICONV)

min f(x)

s.t.

x 2 C

xi 2 Z i 2 I

Mostly convex f and C.

http://www.gurobi.com/company/example-customers

An MICONV Example• Position described by polynomials:

• Step 1: discretize time into intervals

• Step 2: split domain into “safe polyhedrons” !" = $ ∈ ℝ': )"$ ≤ +"{pi : [Ti, Ti+1] ! R2}Ni=1

0 = T1 < T2 < . . . < TN = 1<latexit sha1_base64="3n8IY9U4DDwmKYRu61lB3KXZHK8=">AAACBXicdVDLSgMxFM34rPVVdamLYBFcDUlb2wotFN24kgp9QVtKJpO2oZkHSUYopRs3/oobF4q49R/c+Tdm2goqeiDh5Jx7ubnHCQVXGqEPa2l5ZXVtPbGR3Nza3tlN7e03VBBJyuo0EIFsOUQxwX1W11wL1golI54jWNMZXcZ+85ZJxQO/psch63pk4PM+p0QbqZc6QmVY6+GSuTKwBDvCDbSC8fO6jHupNLLPs3mMMER2LouK2bwhuIDz6AxiG82QBgtUe6n3jhvQyGO+poIo1cYo1N0JkZpTwabJTqRYSOiIDFjbUJ94THUnsy2m8MQoLuwH0hxfw5n6vWNCPKXGnmMqPaKH6rcXi3957Uj3i90J98NIM5/OB/UjAXUA40igyyWjWowNIVRy81dIh0QSqk1wSRPC16bwf9LI2Ngkc5NLVy4WcSTAITgGpwCDAqiAK1AFdUDBHXgAT+DZurcerRfrdV66ZC16DsAPWG+f1xyVng==</latexit><latexit sha1_base64="3n8IY9U4DDwmKYRu61lB3KXZHK8=">AAACBXicdVDLSgMxFM34rPVVdamLYBFcDUlb2wotFN24kgp9QVtKJpO2oZkHSUYopRs3/oobF4q49R/c+Tdm2goqeiDh5Jx7ubnHCQVXGqEPa2l5ZXVtPbGR3Nza3tlN7e03VBBJyuo0EIFsOUQxwX1W11wL1golI54jWNMZXcZ+85ZJxQO/psch63pk4PM+p0QbqZc6QmVY6+GSuTKwBDvCDbSC8fO6jHupNLLPs3mMMER2LouK2bwhuIDz6AxiG82QBgtUe6n3jhvQyGO+poIo1cYo1N0JkZpTwabJTqRYSOiIDFjbUJ94THUnsy2m8MQoLuwH0hxfw5n6vWNCPKXGnmMqPaKH6rcXi3957Uj3i90J98NIM5/OB/UjAXUA40igyyWjWowNIVRy81dIh0QSqk1wSRPC16bwf9LI2Ngkc5NLVy4WcSTAITgGpwCDAqiAK1AFdUDBHXgAT+DZurcerRfrdV66ZC16DsAPWG+f1xyVng==</latexit><latexit sha1_base64="3n8IY9U4DDwmKYRu61lB3KXZHK8=">AAACBXicdVDLSgMxFM34rPVVdamLYBFcDUlb2wotFN24kgp9QVtKJpO2oZkHSUYopRs3/oobF4q49R/c+Tdm2goqeiDh5Jx7ubnHCQVXGqEPa2l5ZXVtPbGR3Nza3tlN7e03VBBJyuo0EIFsOUQxwX1W11wL1golI54jWNMZXcZ+85ZJxQO/psch63pk4PM+p0QbqZc6QmVY6+GSuTKwBDvCDbSC8fO6jHupNLLPs3mMMER2LouK2bwhuIDz6AxiG82QBgtUe6n3jhvQyGO+poIo1cYo1N0JkZpTwabJTqRYSOiIDFjbUJ94THUnsy2m8MQoLuwH0hxfw5n6vWNCPKXGnmMqPaKH6rcXi3957Uj3i90J98NIM5/OB/UjAXUA40igyyWjWowNIVRy81dIh0QSqk1wSRPC16bwf9LI2Ngkc5NLVy4WcSTAITgGpwCDAqiAK1AFdUDBHXgAT+DZurcerRfrdV66ZC16DsAPWG+f1xyVng==</latexit><latexit sha1_base64="3n8IY9U4DDwmKYRu61lB3KXZHK8=">AAACBXicdVDLSgMxFM34rPVVdamLYBFcDUlb2wotFN24kgp9QVtKJpO2oZkHSUYopRs3/oobF4q49R/c+Tdm2goqeiDh5Jx7ubnHCQVXGqEPa2l5ZXVtPbGR3Nza3tlN7e03VBBJyuo0EIFsOUQxwX1W11wL1golI54jWNMZXcZ+85ZJxQO/psch63pk4PM+p0QbqZc6QmVY6+GSuTKwBDvCDbSC8fO6jHupNLLPs3mMMER2LouK2bwhuIDz6AxiG82QBgtUe6n3jhvQyGO+poIo1cYo1N0JkZpTwabJTqRYSOiIDFjbUJ94THUnsy2m8MQoLuwH0hxfw5n6vWNCPKXGnmMqPaKH6rcXi3957Uj3i90J98NIM5/OB/UjAXUA40igyyWjWowNIVRy81dIh0QSqk1wSRPC16bwf9LI2Ngkc5NLVy4WcSTAITgGpwCDAqiAK1AFdUDBHXgAT+DZurcerRfrdV66ZC16DsAPWG+f1xyVng==</latexit>

(x(t), y(t))t2[0,1]<latexit sha1_base64="wVa52RCd6D1IQ0/dYBkLEyLkkPw=">AAAD93icdZPdatswFMfVeh9d9pVuN4PdiJlBAiXYoWzNYFA2CttNycrSlsVZkGU5EZVlV5K7BsV7ld2MsZsN9hR7hr3NZCeZW9EJZA7n9z9/HR3kMGNUKs/7s7buXLt+4+bGrcbtO3fv3W9uPjiUaS4wGeCUpeI4RJIwyslAUcXIcSYISkJGjsKT1yU/OiNC0pS/V7OMjBI04TSmGCmTGjcftc5bqr01M5/2WKuAcjj0tvxRMW66Xqe3s93r9aDX8apVBt1uz38G/WXGBcvVH2+u/w6iFOcJ4QozJOXQ9zI10kgoihkpGkEuSYbwCZqQoQk5Sogc6eoKBXxqMhGMU2E2V7DKXqzQKJFyloRGmSA1lTYrk1exYa7inZGmPMsV4XhxUJwzqFJYzgNGVBCs2MwECAtqeoV4igTCykytEXDyCadJgnikg71CB+UBYaj3iuIy64c17NvwgCBW4wMb7yOV13jfxm+5qukHm0qiiqE/0gEjsQq06weCTqYqsHWEnGeVkHw0IpuiUNYu85XJ3FKFAtWq1krVtlQ8FcmFlv65BbadaX2h7RptKZuPtdu1W6M8WrhVE4i1X5SFV9wBC1lZuT4McJQqaLysGZyeFvrFSzubSTPfMyRMQFnKy6e/et/w/8Fht+N7Hf/dtrv7avkTbIDH4AloAR88B7vgDeiDAcDgM/gKfoCfzsz54nxzvi+k62vLmofg0nJ+/QUEWViw</latexit><latexit sha1_base64="wVa52RCd6D1IQ0/dYBkLEyLkkPw=">AAAD93icdZPdatswFMfVeh9d9pVuN4PdiJlBAiXYoWzNYFA2CttNycrSlsVZkGU5EZVlV5K7BsV7ld2MsZsN9hR7hr3NZCeZW9EJZA7n9z9/HR3kMGNUKs/7s7buXLt+4+bGrcbtO3fv3W9uPjiUaS4wGeCUpeI4RJIwyslAUcXIcSYISkJGjsKT1yU/OiNC0pS/V7OMjBI04TSmGCmTGjcftc5bqr01M5/2WKuAcjj0tvxRMW66Xqe3s93r9aDX8apVBt1uz38G/WXGBcvVH2+u/w6iFOcJ4QozJOXQ9zI10kgoihkpGkEuSYbwCZqQoQk5Sogc6eoKBXxqMhGMU2E2V7DKXqzQKJFyloRGmSA1lTYrk1exYa7inZGmPMsV4XhxUJwzqFJYzgNGVBCs2MwECAtqeoV4igTCykytEXDyCadJgnikg71CB+UBYaj3iuIy64c17NvwgCBW4wMb7yOV13jfxm+5qukHm0qiiqE/0gEjsQq06weCTqYqsHWEnGeVkHw0IpuiUNYu85XJ3FKFAtWq1krVtlQ8FcmFlv65BbadaX2h7RptKZuPtdu1W6M8WrhVE4i1X5SFV9wBC1lZuT4McJQqaLysGZyeFvrFSzubSTPfMyRMQFnKy6e/et/w/8Fht+N7Hf/dtrv7avkTbIDH4AloAR88B7vgDeiDAcDgM/gKfoCfzsz54nxzvi+k62vLmofg0nJ+/QUEWViw</latexit><latexit sha1_base64="wVa52RCd6D1IQ0/dYBkLEyLkkPw=">AAAD93icdZPdatswFMfVeh9d9pVuN4PdiJlBAiXYoWzNYFA2CttNycrSlsVZkGU5EZVlV5K7BsV7ld2MsZsN9hR7hr3NZCeZW9EJZA7n9z9/HR3kMGNUKs/7s7buXLt+4+bGrcbtO3fv3W9uPjiUaS4wGeCUpeI4RJIwyslAUcXIcSYISkJGjsKT1yU/OiNC0pS/V7OMjBI04TSmGCmTGjcftc5bqr01M5/2WKuAcjj0tvxRMW66Xqe3s93r9aDX8apVBt1uz38G/WXGBcvVH2+u/w6iFOcJ4QozJOXQ9zI10kgoihkpGkEuSYbwCZqQoQk5Sogc6eoKBXxqMhGMU2E2V7DKXqzQKJFyloRGmSA1lTYrk1exYa7inZGmPMsV4XhxUJwzqFJYzgNGVBCs2MwECAtqeoV4igTCykytEXDyCadJgnikg71CB+UBYaj3iuIy64c17NvwgCBW4wMb7yOV13jfxm+5qukHm0qiiqE/0gEjsQq06weCTqYqsHWEnGeVkHw0IpuiUNYu85XJ3FKFAtWq1krVtlQ8FcmFlv65BbadaX2h7RptKZuPtdu1W6M8WrhVE4i1X5SFV9wBC1lZuT4McJQqaLysGZyeFvrFSzubSTPfMyRMQFnKy6e/et/w/8Fht+N7Hf/dtrv7avkTbIDH4AloAR88B7vgDeiDAcDgM/gKfoCfzsz54nxzvi+k62vLmofg0nJ+/QUEWViw</latexit><latexit sha1_base64="wVa52RCd6D1IQ0/dYBkLEyLkkPw=">AAAD93icdZPdatswFMfVeh9d9pVuN4PdiJlBAiXYoWzNYFA2CttNycrSlsVZkGU5EZVlV5K7BsV7ld2MsZsN9hR7hr3NZCeZW9EJZA7n9z9/HR3kMGNUKs/7s7buXLt+4+bGrcbtO3fv3W9uPjiUaS4wGeCUpeI4RJIwyslAUcXIcSYISkJGjsKT1yU/OiNC0pS/V7OMjBI04TSmGCmTGjcftc5bqr01M5/2WKuAcjj0tvxRMW66Xqe3s93r9aDX8apVBt1uz38G/WXGBcvVH2+u/w6iFOcJ4QozJOXQ9zI10kgoihkpGkEuSYbwCZqQoQk5Sogc6eoKBXxqMhGMU2E2V7DKXqzQKJFyloRGmSA1lTYrk1exYa7inZGmPMsV4XhxUJwzqFJYzgNGVBCs2MwECAtqeoV4igTCykytEXDyCadJgnikg71CB+UBYaj3iuIy64c17NvwgCBW4wMb7yOV13jfxm+5qukHm0qiiqE/0gEjsQq06weCTqYqsHWEnGeVkHw0IpuiUNYu85XJ3FKFAtWq1krVtlQ8FcmFlv65BbadaX2h7RptKZuPtdu1W6M8WrhVE4i1X5SFV9wBC1lZuT4McJQqaLysGZyeFvrFSzubSTPfMyRMQFnKy6e/et/w/8Fht+N7Hf/dtrv7avkTbIDH4AloAR88B7vgDeiDAcDgM/gKfoCfzsz54nxzvi+k62vLmofg0nJ+/QUEWViw</latexit>

(x(t), y(t)) = pi(t) t 2 [Ti, Ti+1]<latexit sha1_base64="6XwJ2PWKUuy9QBbVMbLZaCZ/NhE=">AAACcXicdZHfatswFMZlb2u79F+27maMDtEwSEkIdtMk7sWgbDe77CBpC7YxsiInorLsScdjwfi+z7e7vcRu9gKT0wTa0h6Q+PjO+UlHR3EuuAbH+WPZL16+2tjcet3Y3tnd22++eXups0JRNqGZyNR1TDQTXLIJcBDsOleMpLFgV/HN1zp/9ZMpzTM5hkXOwpTMJE84JWCsqHlbBstDfDWLw9LpnY2cget2nZ6zjFr0vb43rALBEmj/asNxd2G2QPHZHI6rz4/5/mA4ODHYaS2GRrgjxzvzqjziBsPBj4JMMQRcYn8c8e44KnnHrcIqarbWNF7TeE1jd9VPC63iImr+DqYZLVImgQqite86OYQlUcCpYFUjKDTLCb0hM+YbKUnKdFgum63wJ+NMcZIpsyTgpXufKEmq9SKNTWVKYK4f52rzqZxfQOKFJZd5AUzSu4uSQmDIcD1+POWKURALIwhV3PSK6ZwoQsF8UsMMYf1S/Ly4POm5ZjLfT1vnX1bj2EIf0BFqIxeN0Dn6hi7QBFH013pnHVofrX/2exvbR3eltrViDtCDsDv/AR/AtH4=</latexit><latexit sha1_base64="6XwJ2PWKUuy9QBbVMbLZaCZ/NhE=">AAACcXicdZHfatswFMZlb2u79F+27maMDtEwSEkIdtMk7sWgbDe77CBpC7YxsiInorLsScdjwfi+z7e7vcRu9gKT0wTa0h6Q+PjO+UlHR3EuuAbH+WPZL16+2tjcet3Y3tnd22++eXups0JRNqGZyNR1TDQTXLIJcBDsOleMpLFgV/HN1zp/9ZMpzTM5hkXOwpTMJE84JWCsqHlbBstDfDWLw9LpnY2cget2nZ6zjFr0vb43rALBEmj/asNxd2G2QPHZHI6rz4/5/mA4ODHYaS2GRrgjxzvzqjziBsPBj4JMMQRcYn8c8e44KnnHrcIqarbWNF7TeE1jd9VPC63iImr+DqYZLVImgQqite86OYQlUcCpYFUjKDTLCb0hM+YbKUnKdFgum63wJ+NMcZIpsyTgpXufKEmq9SKNTWVKYK4f52rzqZxfQOKFJZd5AUzSu4uSQmDIcD1+POWKURALIwhV3PSK6ZwoQsF8UsMMYf1S/Ly4POm5ZjLfT1vnX1bj2EIf0BFqIxeN0Dn6hi7QBFH013pnHVofrX/2exvbR3eltrViDtCDsDv/AR/AtH4=</latexit><latexit sha1_base64="6XwJ2PWKUuy9QBbVMbLZaCZ/NhE=">AAACcXicdZHfatswFMZlb2u79F+27maMDtEwSEkIdtMk7sWgbDe77CBpC7YxsiInorLsScdjwfi+z7e7vcRu9gKT0wTa0h6Q+PjO+UlHR3EuuAbH+WPZL16+2tjcet3Y3tnd22++eXups0JRNqGZyNR1TDQTXLIJcBDsOleMpLFgV/HN1zp/9ZMpzTM5hkXOwpTMJE84JWCsqHlbBstDfDWLw9LpnY2cget2nZ6zjFr0vb43rALBEmj/asNxd2G2QPHZHI6rz4/5/mA4ODHYaS2GRrgjxzvzqjziBsPBj4JMMQRcYn8c8e44KnnHrcIqarbWNF7TeE1jd9VPC63iImr+DqYZLVImgQqite86OYQlUcCpYFUjKDTLCb0hM+YbKUnKdFgum63wJ+NMcZIpsyTgpXufKEmq9SKNTWVKYK4f52rzqZxfQOKFJZd5AUzSu4uSQmDIcD1+POWKURALIwhV3PSK6ZwoQsF8UsMMYf1S/Ly4POm5ZjLfT1vnX1bj2EIf0BFqIxeN0Dn6hi7QBFH013pnHVofrX/2exvbR3eltrViDtCDsDv/AR/AtH4=</latexit><latexit sha1_base64="6XwJ2PWKUuy9QBbVMbLZaCZ/NhE=">AAACcXicdZHfatswFMZlb2u79F+27maMDtEwSEkIdtMk7sWgbDe77CBpC7YxsiInorLsScdjwfi+z7e7vcRu9gKT0wTa0h6Q+PjO+UlHR3EuuAbH+WPZL16+2tjcet3Y3tnd22++eXups0JRNqGZyNR1TDQTXLIJcBDsOleMpLFgV/HN1zp/9ZMpzTM5hkXOwpTMJE84JWCsqHlbBstDfDWLw9LpnY2cget2nZ6zjFr0vb43rALBEmj/asNxd2G2QPHZHI6rz4/5/mA4ODHYaS2GRrgjxzvzqjziBsPBj4JMMQRcYn8c8e44KnnHrcIqarbWNF7TeE1jd9VPC63iImr+DqYZLVImgQqite86OYQlUcCpYFUjKDTLCb0hM+YbKUnKdFgum63wJ+NMcZIpsyTgpXufKEmq9SKNTWVKYK4f52rzqZxfQOKFJZd5AUzSu4uSQmDIcD1+POWKURALIwhV3PSK6ZwoQsF8UsMMYf1S/Ly4POm5ZjLfT1vnX1bj2EIf0BFqIxeN0Dn6hi7QBFH013pnHVofrX/2exvbR3eltrViDtCDsDv/AR/AtH4=</latexit>

∀ - ∃ / 0. 2. 34 2 ∈ !" ∀2 ∈ 54, 5478

Mixed-Integer Disjunctive Polynomial Conic (SDP) Optimization

Initial/Terminal Conditions

Interstitial Smoothing Conditions

Variables = Polynomials : {pi : [Ti, Ti+1] ! R2}Ni=1

min

p

XN

i=1||p000i (t)||2

s.t. p1(0) = X0, p0(0) = X 0

0, p00(0) = X 00

0

pN (1) = Xf , p0N (1) = X 0

f , p00N (1) = X 00

f

pi(Ti+1) = pi+1(Ti+1) 8i 2 {1, . . . , N � 1}p0i(Ti+1) = p0i+1(Ti+1) 8i 2 {1, . . . , N � 1}p00i (Ti+1) = p00i+1(Ti+1) 8i 2 {1, . . . , N � 1}_R

r=1[Arpi(t) br] for t 2 [Ti, Ti+1] 8i 2 {1, . . . , N � 1} Remain in Safe Regions


Initial/Terminal

Conditions

Interstitial

Smoothing

Conditions


min

p

XN

i=1||p000i (t)||2

s.t. p1(0) = X0, p0(0) = X 0

0, p00(0) = X 00

0

pN (1) = Xf , p0N (1) = X 0

f , p00N (1) = X 00

f


r=1[Arpi(t) br] for t 2 [Ti, Ti+1] 8i 2 {1, . . . , N � 1}brj+Mrj (1� zi,r)�Ar

jpi(t) is SOS for t 2 [Ti, Ti+1] 8i, j, rXR

r=1zi,r = 1 8i, z 2 {0, 1}N⇥R

� 0

∀" ∈{1, …,N}, r∈{1, …,R}, j∈{1, 2}


Initial/Terminal Conditions

Interstitial Smoothing Conditions


min

p

XN

i=1||p000i (t)||2

s.t. p1(0) = X0, p0(0) = X 0

0, p00(0) = X 00

0

pN (1) = Xf , p0N (1) = X 0

f , p00N (1) = X 00

f


r=1[Arpi(t) br] for t 2 [Ti, Ti+1] 8i 2 {1, . . . , N � 1}brj+Mrj (1� zi,r)�Ar

jpi(t) is SOS for t 2 [Ti, Ti+1] 8i, j, rXR

r=1zi,r = 1 8i, z 2 {0, 1}N⇥R

∀" ∈{1, …,N}, r∈{1, …,R}, j∈{1, 2}

Sum of Squares (SOS)

• p t = ∑( )(*(,)• . − 1 ×(. − 1) SDP for

degree ≤ . polynomials

Results for 9 Regions and 8 time steps

First Feasible Solution:58 seconds

Optimal Solution:651 seconds

Helicopter Game / Flappy Bird

• 60 horizontal segments, obstacle every 5 = 80 sec. to opt.

Solving Mixed Integer Convex/Conic Optimization Problems

How hard is MICONV: Traveling Salesman Problem ?

“A computer would have to check all these possible routes to find the shortest one.”

Quantum computers may be more of an imminent threat than AI,

Vivek Wadhwa, February 5, 2018

“As the number of cities increases, the problem becomes exponentially complex. It would take a laptop computer 1,000 years to compute the most efficient route between 22 cities, for example.”

MIP = Avoid Enumeration

• Number of tours for 49 cities 48!/2 ≈ '()(• Fastest supercomputer ≈ '('* flops • Assuming one floating point operation per tour:–more than '(+, times the age of the universe!

• How long does it take on an iphone?– Less than a second!–4 iterations of cutting plane method!–Dantzig, Fulkerson and Johnson 1954 did it by hand!–Cutting planes are the key for effectively solving (even NP-

hard) MIP problems in practice.

50+ Years of MIP = Significant Solver Speedups

• Algorithmic Improvements (Machine Independent):

–• v1.2 (1991) – v11 (2007): 29,000 x speedup

–• v1 (2009) – v6.5 (2015): 48.7 x speedup

• Also convex nonlinear:

–• v6.0 (2014) – v6.5 (2015) quadratic: 4.43 x

(V., Dunning, Huchette, Lubin, 2015)

ILOG CPLEX 11.2

CPLEXCPLEX

Gurobi 7.0 Performance Benchmarks


≈ 1.9 x / year

State of MIP Solvers

• Mature: Linear and Quadratic (Conic Quadratic/SOCP)–Commercial:

– “Open Source”

• Emerging: Convex Nonlinear –

CPLEX


SCIP CBC GLPK

Bonmin

MICONV B&B Algorithms

• NLP (QCP) Based B&B • (Dynamic) LP Based B&B – Few cuts = high speed.– Possible slow convergence.

• Lifted LP B&B – Extended or Lifted relaxation.– Static relaxation• Mimic NLP B&B.

– Dynamic relaxation• Standard LP B&B

Lifted or Extended Approximations• Projection = multiply constraints.• V., Ahmed. and Nemhauser 2008:– Extremely accurate, but static and

complex approximation by Ben-Tal and Nemirovski

• V., Dunning, Huchette and Lubin 2015: Simple, dynamic and good approximation:– First talks: May ‘14 (SIOPT), Dec ‘14 IBM– Paper in arxive, May ’15– Adopted in CPLEX v12.6.2, Jun 15’– Gurobi (Oct ‘15), Xpress (May ‘16), SCIP (Mar’ 17)

y2i zi · y0 8i 2 [n]Xn

i=1zi y0

Image from Lipton and Regan, https://rjlipton.wordpress.com

kyk2 y0

Not MICONV but, Mixed Integer Conic Programming (MICP)

• closed convex cones– Linear, SOCP, rotated SOCP, SDP– Exponential cone, power cone, …– Spectral norm, relative entropy,

sum-of-squares, …

Mixed-integer conic form

min2RN

h , i : (M)

k � k 2 Ck 8k 2 [M]

xi 2 Z 8i 2 [I ]

CK+1, . . . , CM are polyhedral cones, e.g. R+, R�, {0}C1, . . . , CK are closed convex nonpolyhedral cones, e.g.

L second-order cone (epi k·k2)E exponential cone (epi cl per exp)

P positive semidefinite cone (S+ on S)

Assume that if M is feasible then its optimal value is attained

Chris Coey (MIT ORC) Conic OA for MICP SIAM Opt. 2017 6 / 19


min2RN

h , i : (M)

k � k 2 Ck 8k 2 [M]

xi 2 Z 8i 2 [I ]







min2RN

h , i : (M)

k � k 2 Ck 8k 2 [M]

xi 2 Z 8i 2 [I ]






• Fast and stable interior point algorithms for continuous relaxation• Geometrically intuitive conic duality guides linear inequality selection• Conic formulation techniques usually lead to extended formulations – MINLPLIB2 instances unsolved since 2001 solved by re-write to MISOCP

Pajarito: A Julia-based MICP Solver

MI-convex model:

CBF, Convex.jl, CVXPY, JuMP

MI-conic solver:

Pajarito

Continuous solver:

CSDP, ECOS,MOSEK, SCS, SDPA

MILP solver:

CBC, CPLEX, GLPK,Gurobi, MOSEK, SCIP

conic interface

conic interfacelinear/quadratic interface

(through JuMP)

Figure 1: Pajarito’s integration with MathProgBase.

coefficient vectors, variable and constraint cones expressed as lists of standardprimitive cones (1-dimensional vector sets) with corresponding ordered row in-dices, and a vector of variable types (each continuous, binary, or general integer).In addition to the basic linear cones (nonnegative, nonpositive, zero, and freecones), Pajarito recognizes three standard primitive nonpolyhedral cones intro-duced in section 1.2: exponential cones (see appendix A.1), second-order cones(see appendix A.2), and positive semidefinite cones (see appendix A.3).19

Friberg [2016] designed the Conic Benchmark Format (CBF) as a file for-mat originally to support mixed-integer second-order cone (SOCP) and positivesemidefinite cone (SDP) instances. In collaboration with Henrik Friberg, weextended the format to support exponential cones in Version 2, and developeda Julia interface ConicBenchmarkUtilities.jl to provide utilities for translatingbetween CBF and MathProgBase conic format.20 One may use Pajarito tosolve any instance in the Conic Benchmark Library (CBLIB), which containsthousands of benchmark problems from a wide variety of sources.

Lubin et al. [2016] demonstrate that all 333 known MI-convex instances inMINLPLib2 [Vigerske, 2018] are representable with linear, second-order, expo-nential, and power cones. Since a power cone constraint is representable withlinear and exponential cone constraints, Pajarito can be used to solve any ofthe MI-convex instances in MINLPLib2. We translated 115 instances from theMINLPLIB2 library to CBF and contributed them to CBLIB.21 Many of the

19As we note in appendix A.2, Pajarito also recognizes rotated second-order cones, but forsimplicity converts them to second-order cones during preprocessing.

20Pajarito’s extensive unit tests rely on small example instances loaded from CBF files.21Lubin et al. [2016] first translated these instances from the MINLPLIB2 library into Con-

vex.jl models. We used ConicBenchmarkUtilities.jl to translate these to CBF. The instances,available at github.com/mlubin/MICPExperiments, are 48 ‘rsyn’ instances, 48 ‘syn’ instances,6 ‘tls’ instances, 12 ‘clay’ instances, and the challenging ‘gams01’ instance.

21

• Early version solved gams01, tls5 and tls6 (MINLPLIB2)

<latexit sha1_base64="o3zegCXig6oKMdKvT/9PyPOk8S4=">AAAT/3ichVjNchu5EeZa+dllfnYdHXNBre0teYtiOKR+VgdWydK6Eq9kWbFkW2UNywXMgCRMDDCLwYikUXPILdfkJXJL5ZpHyTPkEXJJ9wxJaWYkLW1hMN1fN7ob3Q2QLJYisZ3Ofz57sPazn//il59/0fzVr3/z2y+/evi7t4lOTcDfBFpqc8FowqVQ/I0VVvKL2HAaMcnfsckh8t9dcZMIrc7tPOaDiI6UGIqAWiB9ePjgfz7jI6GcpSyV1GROEkm+dVuZO7sEmuSbQ20iavvdQQb0bp3e9oBTIfYGWdO3Ojap5M1vyDfEj1JpBdibRgqVB5lLLLVpwpMM2E3fb/pBJEIU2JDmqett7mQERBMtwX6AOD1BpJMiEjafcWO0yWdTo9UIZ8RZEXGykTwlGUGdC5XNYn2jp247c99mzjcaludMzy61ERCAfjBwe53M6ZgrUoQ3y2D9A60ioTYPDtDK3haMWzh4Hgw9HMjWTg+XWkFfPUNodxvG7R4M3T3E44zsdncq0M3vc70d1PtdGbzjdQrwKf1IjbB684/Hp0ebzw9fnaHM9g6MOyiI6NyS3u5uReLw4HAlsIv6e0sBXId4hek+DUNMoCSmwc1Q9e4PleGJNSKwPMwjdXb44pRsbLU77c7TfD2MUw/D8N3SyJ2FR4enx88vyIbXbe8uwHudnI8xWxq3XXU/l9p8+ers+RHZ2MN12t29TSrjMS107GIAUVF3EZHtPNw+09bqqMgDrsJVrn/46hEoyT+kPvEWk0eNxef0w8Mv/+uHOkgjrmwgaZJcep3YDhw1kNuSQ85DQkMQJ3TEL2GqaMSTgcvLNCNPgBISqBD4U5bk1KYfh0Od2ji1fa9ZkqeBFVcQ+L6zJuUtonSgoxjqlgkp7DxrkaFQVLaINSAh1Khf4CbcqOs33NPrtyEo1abveZ0OsKzhNhj3PZyPjVATmA5cJALYcegVGZbPYh6AxXxm3VKd0mpouArGWdnoc2/g0DtglTmOpnbM5KRKjRLoF+MWPFEswUkyj1gLqVZrmVPsOCrF1vEoHvMfy7TZEOJQJiU2omZuwjKVBgHsX1Imsqhi2cjQeCyCWUWjSJWwFSLTegI5lbSWldNadrsyLqAxdt2KxpTdRr9MeIDEgYslVKVQScU+KyafCgmcScEMuOogbEbMcD9pzJMWodAjp0k74pa2SKwTgTphAzF5bIswUDYyOlVhJRzxaBhLnQdpOYWFXJGBfcWnUPtgUNkkRS0TrOyGwtqDhgED8IQKKwmT9x2VRowbHrbIFb7DE4s1hMKSI+g5sP1dXnE/NeXgXublBNKTpD+kMuEDN4a8NYYPs+YT4oPJAfhpuXEBxAaeUJDEaojDjNgxJywdEQGt/+ojNP9exQfKmOFXUMmSw2mEKv1cO8QkjR2UcAzlY5OASiyOE63Q3CeVDzgAeMnVyI6dDwkDJsMimeu1ezFE04eCQiujiEKT8mHr6HxRo5nz2t2srIAyfcUXqZNMBOjZzbXcgDAu9bQE6SAk31MssBU0olDnzo/mdCaSqQgtrNhpb+36WPTF+08IjbkYjS1K9apCq+xpYvDmBOF/aNMYTtoQmtBcQouD/pJYgkhsdATKFtoY5G8z19O/aRkQi8X6paVbTa6gqY94fkFYJAEUAuwJJ1rJYt1Wcx7Rjxr79TJRWs0ZnMV9r0Xm+Ozgk87gvTlDlKQQxaWZFDoe9E+4H9gx4RRqoNWE/9d+5S5UnUNvsGOCOZjepPCp2yKhoVOisaXaeb/T3kZtsDs3syBlcFBdQlf1oUi5KXoX7Cxe1R55cPCW0VbIkN+CL+iFBNy0IIXxbHL8xzS/CGYO/gHjjNuj6SFoQ6HFM3OPfWgqVI0kNxjpxz5BNIIXkDM7XxCOpiCmrFAprBYsZ0vWAVxNJ5lj+QOWi+iEU8gmC+WYO8LVFfRPlS8f6ZDL3JUmgY8PjXJpblJQQj7094MUalzZfJvQv2a2wMMGlESy5Xoayt3UVh/twy3I6v08q/39odTQzfQMLjnc+hhEcF6NqjpA2tcQ2ii2cwQuJwVnRb6iBhrhGBU0/e853B0Mfwm19Ap6CCgzzg8kxAtb2q1cFkLUwru4hiqIaj7epV2rq3w/ru5BFDvG70KEOsocDnfweSwyh8MdfGiYOnP5eAcCJpnD4a4g4V0BvjjgeBcCWiIgcKwUEmOvs0u43vjYwxhzr6u1w9j7EuB9HXBWApxl1TXESQGQkuGdjFtyQnxjFi8VdeJFFfziHvBxFXx8D/ioCj66B/xDFfzDfeBJDf1hcg/+JKniVQldhisNKeRrDcfoSEHGiJDnvZI8Fo+v36B9+Xg/uGxv89nAeXyWufYWHGy1/QgurjcMzgF3Ud3S4KQMOKkB3pcB72trHJUBR1UNRwsNruPXWMdl2eMa4HUZ8LYGOC0DzmqA52XA85r9jBcIFjley3i74tka73zFO6/L0XTF9fGlCpiu2NMaD1rMSlZHfERriHcrwLsab7bizWq8+Yo3r/E+rXifajy54ska79rRmpdw6BbM4liuC1uRlgC3aGBlFQsb6rC0BktrMAbNeRVZmFfZwYob1HvfisdqvGcr3rMq7xz2vsg+c1ueRDfSJKpnibrBVmktdeHbaXyd3vD1b+Je1goYvu4EFdBhFRRrOa9gTjP8ecCr/hhQn7zttr1O2/vz1qP9g8UPBZ83ft/4urHR8Bq7jf3GnxqnjTeNYI2t/XXtb2t/X//L+j/W/7n+rwL64LOFzHqj9Fn/9/8BqfVBPA==</latexit>

Performance for MISOCP Instances (120 from CBLIB)

Stability of CONIC Interior Point Algorithms is KEY!

• Why? Avoid non-differentiability issues? Stronger theory?

• Industry change in 2018:

– version 11.0 adds support for SOCP constraints

– version 9.0 deprecates nonlinear formulations

and focuses on pure conic (linear, SOCP, rotated SOCP, SDP, exp & power)

Deprecated features

• Advanced sensitivity analysis.• The traditional analysis based on the basis is still available.

• General convex optimization.

min f(x)subject to g(x) 0,

except for explicit quadratic functions.• Dropped the Fusion interface for MATLAB.

• Use a�ne conic constraints instead (Fusion light).

3 / 23

Hypatia: Pure Julia-based IPM Beyond “Standard” Cones

• Extension of methods in CVXOPT and Alfonso

– A customizable homogeneous interior-point solver

for nonsymmetric convex

– Skajaa and Ye ‘15, Papp and Yıldız ‘17, Andersen,

Dahl, and Vandenberghe ‘04-18

• Cones: LP, dual Sum-of-Squares, SOCP, RSOCP,

3-dim exponential cone, PSD, L∞, n-dim

power cone (using AD), spectral norm, …

• Potential:

– flexible number types and linear algebra

– BOB: bring your own barrier (in ∼50 lines of code)

– Alternative prediction steps (Runge–Kutta)

Chris Coey

Early Comparison with Alfonso for LP and SOS

Aug 23Aug 19

chriscoey / Hypatia.jl Private

No one—assign yourself

None yet

No milestone

2 participants

Alfonso matlab code comparison #23 Open chriscoey opened this issue on Aug 5 · 3 comments

Edit New issue

Assignees

Labels

info

speed

tests

Projects

Milestone

Notifications

chriscoey commented on Aug 5 •

running Alfonso 75cba5f with default options against the following Alfonso.m Matlab code with (thesame) default options:

seed=2017;tol=1e-06;results=random_lp(500,1000,tol,seed);seed=2017;tol=1e-06;intParams=ChebInterval(5);results=polyEnv(intParams,2,5,tol,seed);tol=1e-06;intParams=FeketeCube(6,2);results=polyOpt(intParams,'butcher',tol);intParams=FeketeCube(4,4);results=polyOpt(intParams,'caprasse',tol);intParams=FeketeCube(4,3);results=polyOpt(intParams,'lotka-volterra',tol);intParams=PaduaSquare(7);results=polyOpt(intParams,'motzkin',tol);intParams=FeketeCube(3,4);results=polyOpt(intParams,'reaction-diffusion',tol);intParams=PaduaSquare(8);results=polyOpt(intParams,'robinson',tol);

primal and dual objectives for the Julia code and Matlab code match. the iteration counts (very similar)and timings (in seconds) are:

test iters Matlab 75cba5f c9f1eb5 133b422

dense lp 65 5.8 4.1 2.03 1.25

envelope 30 0.085 0.043 0.020 x

butcher 32/30 0.63 0.41 0.357 0.136

caprasse 31/30 1.38 1.87 1.80 0.530

lotka-volt 31/30 0.47 0.38 0.37 0.104

motzkin 41/42 0.35 0.24 x 0.054

reac-diff 29/30 0.32 0.23 0.19 0.075

robinson 29 0.34 0.23 0.17 0.034

edited

chriscoey added speed info labels on Aug 5

chriscoey changed the title from Matlab code comparison to [info] Matlab code comparisonon Aug 5

chriscoey referenced this issue on Aug 6

Aug 5

Linear Optimization

Polynomial Envelope

Polynomial

Minimization

First Hypatia commit : Jul 15

• First Batch of Tests on CBLIB Instances (SDP/SOCP): Only 2 – 10K times

slower than Mosek 8!

Modeling with Conic Optimization

How to get conic representation?

• Relatively mechanical, but understanding details can help performance

MMC

Convex.jl in

3/9/16, 12:08 PMAcademic Page of Juan Pablo Vielma

Page 3 of 3http://www.mit.edu/~jvielma/

Jennifer ChallisE62-571, 100 Main Street,Cambridge, MA 02142(617) 324-4378jchallis at mit dot edu

CollaboratorsShabbir Ahmed, Daniel Bienstock, Daniel Dadush, Sanjeeb Dash, Santanu S. Dey, Iain Dunning, RodolfoCarvajal, Luis A. Cisternas, Miguel Constantino, Daniel Espinoza, Alexandre S. Freire, Marcos Goycoolea,Oktay Günlük, Joey Huchette, Nathalie E. Jamett, Ahmet B. Keha, Mustafa R. Kılınç, Guido Lagos, MilesLubin, Sajad Modaresi, Sina Modaresi, Eduardo Moreno, Diego Morán, Alan T. Murray, George L.Nemhauser, Luis Rademacher, David M. Ryan, Denis Saure, Alejandro Toriello, Andres Weintraub, SercanYıldız, Tauhid Zaman

Affiliations

Links

Back to top© 2013 Juan Pablo Vielma | Last updated: 02/29/2016 00:47:25 | Based on a template design by AndreasViklund

• Using modeling tools like Disciplined Convex Programming (DCP)

• Using standard constructions for standard cones


min2RN

h , i : (M)

k � k 2 Ck 8k 2 [M]

xi 2 Z 8i 2 [I ]






MICP Example 1: MI - Second Order Cone (SOCP)

feasible solutions and fathom nodes.Quesada and Grossmann [1992] and Leyffer [1993] describe subproblem-based

B&B-OA algorithms that solve smooth NLP subproblems at a subset of the nodes.The NLP subproblems provide feasible solutions at which gradient cuts can bederived, however in the case of infeasibility, a second subproblem must be solvedin order to derive the cuts. Bonmin implements this algorithm using the NLPsolver Ipopt, and Bonami et al. [2008] found this B&B-OA method outperformsBonmin’s B&B-NL method. For the special case of MISOCP, Drewes and Ulbrich[2012] propose a conic subproblem-based B&B-OA algorithm that derives cutsfrom subgradients satisfying subproblem KKT optimality conditions, and hencedoes not require smoothness assumptions.

Using the elegant theory of conic duality, it is possible to describe a simplerand more efficient OA algorithm for generic MI-conic problems that uses coniccertificates returned by primal-dual conic solvers, with no need to examine KKTconditions or solve a second modified subproblem in the case of infeasibility.Lubin et al. [2016] propose this idea in an iterative OA algorithm. Howevera B&B algorithm using a single search tree, instead of solving a sequence ofMILP instances each with their own search tree, is more flexible and likely tobe significantly faster in practice. We fill this gap with the first conic-certificate-based B&B-OA algorithm, which we implement in our new MI-conic solverPajarito.2

1.2 Mixed-Integer Conic FormWe use the following general form for a mixed-integer conic (MI-conic) problem:

M

8>><

>>:

inf

x2RNc

Tx :

b�Ax 2 K ⇢ RM

xi 2 Z 8i 2 JIK,

(1a)

(1b)(1c)

where K is a closed convex cone, i.e. a closed subset of RM that contains allconic (nonnegative) combinations of its points [Ben-Tal and Nemirovski, 2001a]:

↵1y1 + ↵2y2 2 K 8↵1,↵2 � 0 8y1,y2 2 K. (2)

The variables in M are represented by the column vector x 2 RN , so the objective(1a) minimizes a linear function of x subject to (denoted by ‘:’) the constraints(1b) and (1c). The index set of integer variables is JIK = {1, . . . , I}, so theintegrality constraints (1c) restrict only the first I variables x1, . . . , xI to the setof integers Z. The conic constraint (1b), which restricts the affine transformationb�Ax of x to K, is a convex constraint, so relaxing the integrality constraints(1c) results in a convex conic optimization problem.

2An early implementation of Pajarito was introduced in Lubin et al. [2016], but this was asimple NLP-based iterative algorithm built to assess the value of MINLP extended formulationsby counting iterations. The new version of Pajarito that we implement for this paper usesconic certificates returned by conic solvers.

5

feasible solutions and fathom nodes.Quesada and Grossmann [1992] and Leyffer [1993] describe subproblem-based

B&B-OA algorithms that solve smooth NLP subproblems at a subset of the nodes.The NLP subproblems provide feasible solutions at which gradient cuts can bederived, however in the case of infeasibility, a second subproblem must be solvedin order to derive the cuts. Bonmin implements this algorithm using the NLPsolver Ipopt, and Bonami et al. [2008] found this B&B-OA method outperformsBonmin’s B&B-NL method. For the special case of MISOCP, Drewes and Ulbrich[2012] propose a conic subproblem-based B&B-OA algorithm that derives cutsfrom subgradients satisfying subproblem KKT optimality conditions, and hencedoes not require smoothness assumptions.

Using the elegant theory of conic duality, it is possible to describe a simplerand more efficient OA algorithm for generic MI-conic problems that uses coniccertificates returned by primal-dual conic solvers, with no need to examine KKTconditions or solve a second modified subproblem in the case of infeasibility.Lubin et al. [2016] propose this idea in an iterative OA algorithm. Howevera B&B algorithm using a single search tree, instead of solving a sequence ofMILP instances each with their own search tree, is more flexible and likely tobe significantly faster in practice. We fill this gap with the first conic-certificate-based B&B-OA algorithm, which we implement in our new MI-conic solverPajarito.2

1.2 Mixed-Integer Conic FormWe use the following general form for a mixed-integer conic (MI-conic) problem:

M

8>><

>>:

inf

x2RNc

Tx :

b�Ax 2 K ⇢ RM

xi 2 Z 8i 2 JIK,

(1a)

(1b)(1c)

where K is a closed convex cone, i.e. a closed subset of RM that contains allconic (nonnegative) combinations of its points [Ben-Tal and Nemirovski, 2001a]:

↵1y1 + ↵2y2 2 K 8↵1,↵2 � 0 8y1,y2 2 K. (2)

The variables in M are represented by the column vector x 2 RN , so the objective(1a) minimizes a linear function of x subject to (denoted by ‘:’) the constraints(1b) and (1c). The index set of integer variables is JIK = {1, . . . , I}, so theintegrality constraints (1c) restrict only the first I variables x1, . . . , xI to the setof integers Z. The conic constraint (1b), which restricts the affine transformationb�Ax of x to K, is a convex constraint, so relaxing the integrality constraints(1c) results in a convex conic optimization problem.

2An early implementation of Pajarito was introduced in Lubin et al. [2016], but this was asimple NLP-based iterative algorithm built to assess the value of MINLP extended formulationsby counting iterations. The new version of Pajarito that we implement for this paper usesconic certificates returned by conic solvers.

5

max ax

s.t.

��Q

1/2x

��2 �

Xn

j=1xj = 1, x 2 Rn

+

xj zj 8j 2 [n]

Xn

j=1zj K, z 2 {0, 1}n

Portfolio Optimization Problems:

A.1.2 Extreme Ray Disaggregation

Suppose we have the E⇤ point (u, v, w). If w = 0, the point is already a non-negative combination of the initial fixed E⇤ points from appendix A.1.1, so wediscard it. If w < 0, we use the E⇤ extreme ray (u,w � w log(

�w/u), w), which

when added to some nonnegative multiple of (0, 1, 0), gives (u, v, w).49

A.1.3 Separation Of An Infeasible Point

Suppose we want to separate a point (r, s, t) /2 E that satisfies the initial fixedcuts. Then r, s � 0 and if r = s = 0 then t 0. If s = 0, then t > 0 andr > 0, and we use the E⇤ extreme ray (

t/r,�2 + 2 log(

2r/t),�2). If s > 0, then

r < s exp(

t/s), and we use the E⇤ extreme ray (1, (

t/s � 1) exp(

t/s),� exp(

t/s)).

A.2 Second-Order ConeFor n � 2, the second-order cone is the epigraph of the homogeneous convex`2-norm:

L1+n= {(r, t) 2 R1+n

: r � ktk2}. (31)

We sometimes drop the dimension 1 + n when implied by context. This cone isself-dual (L⇤

= L). We also define the (self-dual) rotated second-order cone:

V2+n= {(r, s, t) 2 R2+n

: r, s � 0, 2rs � ktk22}. (32)

However, V is an invertible linear transformation of L, since (r, s, t) 2 V2+n ifand only if (r + s, r � s,

p2t1, . . . ,

p2tn) 2 L2+n, so for simplicity we restrict

attention to L.50

A.2.1 Initial Fixed Polyhedral Relaxation

Suppose we have a primitive cone constraint (r, t) 2 L1+n. Ben-Tal and Ne-mirovski [2001b] describe fixed polyhedral OAs of L. First, we note that the`1-norm lower-bounds the `2-norm, since for any t 2 Rn we have:

ktk1 = max

i2JnK|ti| ktk2. (33)

Let e(i) 2 Rn be the ith unit vector in n dimensions. We use the 2n L⇤ extremerays (1,±e(i)), 8i 2 JnK, which imply the conditions r � |ti|, 8i 2 JnK, equivalentto the homogenized box relaxation r � ktk1. Second, we note that the `1-normalso provides a lower bound for the `2-norm, since for any t 2 Rn we have:

ktk1 =

X

i2JnK|ti|

pnktk2. (34)

49This also projects (u, v, w) /2 E⇤ with u > 0, w < 0 onto E⇤.50As noted in section 5.2, Pajarito transforms any V constraints to equivalent L constraints

during preprocessing.

40

⇣�, Q

1/2x

⌘2 L1+n

<latexit sha1_base64="SZ4Z+B6wK6hM7q3Sl89ySjL1gt0=">AAAEQnichVPNbhMxEHaz/JTw0xSOXCyiSKlAYbcgwbH8HDhAaQtJK3VD5HVmE6tee2V7A9F2H4Sn4cIBHoFX4IY4cOGAd7NtswmFkXb1eeabz+OxJ4g508Z1v63UnAsXL11evVK/eu36jbXG+s2elomi0KWSS3UQEA2cCegaZjgcxApIFHDYD46e5fH9CSjNpHhrpjH0IzISLGSUGOsaNB74HELT9jUbReQe3n2Xevc3M/zBV2w0Nhs+E35EzJgSnr7MbPCuyPCg0XQ7bmF4GXglaKLSdgbrtV/+UNIkAmEoJ1ofem5s+ilRhlEOWd1PNMSEHpERHFooSAS6nxany3DLeoY4lMp+wuDCO5+RkkjraRRYZl6qXozlztNYdSsTPu6nTMSJAUFnO4UJx0bivFd4yBRQw6cWEKqYLRbTMVGEGtvRui/gPZVRRMQw9feA8CwtehUE6V6W4X9aC+cJWCRRsCS1TUyi5tW2/69W5vxdUIPJDr1+Wly1nza92eX658i28BswWMagiMk7PS9FAn0mdXyidHxefS38JNCS2/biCeEJVMUCRc7E2idiG1asyhNSRXMH4D1Q5vQQKl9l1QQWl3QiRhxyagEWWCGX9nnNiAXOiTNnlUiB8ZKXw5xWuOr+c7DPWcEre0+vy35ZuhSTLC3+dTsq3uJgLIPeZsdzO97uw+bW03JoVtFtdAe1kYceoS30Au2gLqLoI/qEvqCvzmfnu/PD+Tmj1lbKnFuoYs7vP1qDWcU=</latexit><latexit sha1_base64="SZ4Z+B6wK6hM7q3Sl89ySjL1gt0=">AAAEQnichVPNbhMxEHaz/JTw0xSOXCyiSKlAYbcgwbH8HDhAaQtJK3VD5HVmE6tee2V7A9F2H4Sn4cIBHoFX4IY4cOGAd7NtswmFkXb1eeabz+OxJ4g508Z1v63UnAsXL11evVK/eu36jbXG+s2elomi0KWSS3UQEA2cCegaZjgcxApIFHDYD46e5fH9CSjNpHhrpjH0IzISLGSUGOsaNB74HELT9jUbReQe3n2Xevc3M/zBV2w0Nhs+E35EzJgSnr7MbPCuyPCg0XQ7bmF4GXglaKLSdgbrtV/+UNIkAmEoJ1ofem5s+ilRhlEOWd1PNMSEHpERHFooSAS6nxany3DLeoY4lMp+wuDCO5+RkkjraRRYZl6qXozlztNYdSsTPu6nTMSJAUFnO4UJx0bivFd4yBRQw6cWEKqYLRbTMVGEGtvRui/gPZVRRMQw9feA8CwtehUE6V6W4X9aC+cJWCRRsCS1TUyi5tW2/69W5vxdUIPJDr1+Wly1nza92eX658i28BswWMagiMk7PS9FAn0mdXyidHxefS38JNCS2/biCeEJVMUCRc7E2idiG1asyhNSRXMH4D1Q5vQQKl9l1QQWl3QiRhxyagEWWCGX9nnNiAXOiTNnlUiB8ZKXw5xWuOr+c7DPWcEre0+vy35ZuhSTLC3+dTsq3uJgLIPeZsdzO97uw+bW03JoVtFtdAe1kYceoS30Au2gLqLoI/qEvqCvzmfnu/PD+Tmj1lbKnFuoYs7vP1qDWcU=</latexit><latexit sha1_base64="SZ4Z+B6wK6hM7q3Sl89ySjL1gt0=">AAAEQnichVPNbhMxEHaz/JTw0xSOXCyiSKlAYbcgwbH8HDhAaQtJK3VD5HVmE6tee2V7A9F2H4Sn4cIBHoFX4IY4cOGAd7NtswmFkXb1eeabz+OxJ4g508Z1v63UnAsXL11evVK/eu36jbXG+s2elomi0KWSS3UQEA2cCegaZjgcxApIFHDYD46e5fH9CSjNpHhrpjH0IzISLGSUGOsaNB74HELT9jUbReQe3n2Xevc3M/zBV2w0Nhs+E35EzJgSnr7MbPCuyPCg0XQ7bmF4GXglaKLSdgbrtV/+UNIkAmEoJ1ofem5s+ilRhlEOWd1PNMSEHpERHFooSAS6nxany3DLeoY4lMp+wuDCO5+RkkjraRRYZl6qXozlztNYdSsTPu6nTMSJAUFnO4UJx0bivFd4yBRQw6cWEKqYLRbTMVGEGtvRui/gPZVRRMQw9feA8CwtehUE6V6W4X9aC+cJWCRRsCS1TUyi5tW2/69W5vxdUIPJDr1+Wly1nza92eX658i28BswWMagiMk7PS9FAn0mdXyidHxefS38JNCS2/biCeEJVMUCRc7E2idiG1asyhNSRXMH4D1Q5vQQKl9l1QQWl3QiRhxyagEWWCGX9nnNiAXOiTNnlUiB8ZKXw5xWuOr+c7DPWcEre0+vy35ZuhSTLC3+dTsq3uJgLIPeZsdzO97uw+bW03JoVtFtdAe1kYceoS30Au2gLqLoI/qEvqCvzmfnu/PD+Tmj1lbKnFuoYs7vP1qDWcU=</latexit><latexit sha1_base64="dtOBXy2QyTblq7qfZ1mtsaI66UQ=">AAAD/3ichVPLbtNAFJ0mPIop0K7ZjIgilU1ks4ElCBZsKKWQtFIcVTOT62TUeVgz46DI9Q+wYMOvsEN8CF/ClhnHbeOGwpVsHd8599yH59JccOvi+NdWp3vr9p272/ei+zvRg4ePdndGVheGwZBpoc0JJRYEVzB03Ak4yQ0QSQUc07PX4fx4AcZyrT65ZQ4TSWaKZ5wR512Hp7u9eBDXhjdB0oAeaux0r/M7nWpWSFCOCWLtOIlzNymJcZwJqKK0sJATdkZmMPZQEQl2UtZ1VrjvPVOcaeMf5XDtXY8oibR2KalnSuLm9vpZcF6etVO57MWk5CovHCi2ypQVAjuNQ9d4yg0wJ5YeEGa4LxazOTGEOT+bKFXwmWkpiZqW6REQUZVpSEJpeVRV+J/WxyEAq0LSDakD4gqzrnbwf7Um5u+CFlw1TiZlKiBzadlLUsNnc5feINvHH8FhnYMhLkx6XYpQeyV1fqF0flN9ffyKWi38ePGCiALaYtSQK7H9C7GnXqzNU9rItQbECIy7bMKEr6odwPOGTtRMQKDW4BorE9pfrxWxxoG4craJDLhoeAEGWu2K0jfgr7OBd/4/vW/m5elaLaqyfkd+U5Lre7EJRs8GSTxIPsRoGz1GT9A+StBz9BK9RYdoiBiaoi/oW/dr93v3x2qjOlvNau2hlnV//gEI6kLj</latexit><latexit sha1_base64="0MPnw5HHbSAUhZ/zsWNk3HpMOEA=">AAAEN3ichVPNbhMxEHYbfkooNOXKxSKKlAoUdssBjiA4cIDSFpJWyobI68wmVv2zsr2BaLsPwtNw4QCPwCtwQxy44t1s22xCYaRdfZ755vN47Aljzoz1vO9r67UrV69d37hRv7l56/ZWY3uzZ1SiKXSp4kofh8QAZxK6llkOx7EGIkIOR+HJ8zx+NAVtmJLv7CyGgSBjySJGiXWuYeNRwCGy7cCwsSAP8MH71H+4m+GPgWbjid0JmAwEsRNKePoqc8H7MsPDRtPreIXhVeCXoIlK2x9ur/8ORoomAqSlnBjT973YDlKiLaMcsnqQGIgJPSFj6DsoiQAzSIvTZbjlPCMcKe0+aXHhXcxIiTBmJkLHzEs1y7HceR6rbmWjJ4OUyTixIOl8pyjh2Cqc9wqPmAZq+cwBQjVzxWI6IZpQ6zpaDyR8oEoIIkdpcAiEZ2nRqzBMD7MM/9NaOE/AMhHhitQesYleVNv7v1qZ83dBAzbr+4O0uOogbfrzyw0ukW3ht2CxikETm3d6UYqE5kLq9Ezp9LL6WvhZaBR37cVTwhOoioWaXIi1z8R2nFiVJ5UWCwfgPdD2/BA6X2XVBBaXdCLHHHJqAZZYEVfuec2JBc6Jc2eVSIHxkpfDnFa46sELcM9Zw2t3T2/Kfjm6ktMsLf51Nyr+8mCsgt5ux/c6/oGHNtBddA+1kY8eo6foJdpHXUTRJ/QZfUXfal9qP2o/50O1vlZO1x1UsdqvP3hKWBY=</latexit><latexit sha1_base64="0MPnw5HHbSAUhZ/zsWNk3HpMOEA=">AAAEN3ichVPNbhMxEHYbfkooNOXKxSKKlAoUdssBjiA4cIDSFpJWyobI68wmVv2zsr2BaLsPwtNw4QCPwCtwQxy44t1s22xCYaRdfZ755vN47Aljzoz1vO9r67UrV69d37hRv7l56/ZWY3uzZ1SiKXSp4kofh8QAZxK6llkOx7EGIkIOR+HJ8zx+NAVtmJLv7CyGgSBjySJGiXWuYeNRwCGy7cCwsSAP8MH71H+4m+GPgWbjid0JmAwEsRNKePoqc8H7MsPDRtPreIXhVeCXoIlK2x9ur/8ORoomAqSlnBjT973YDlKiLaMcsnqQGIgJPSFj6DsoiQAzSIvTZbjlPCMcKe0+aXHhXcxIiTBmJkLHzEs1y7HceR6rbmWjJ4OUyTixIOl8pyjh2Cqc9wqPmAZq+cwBQjVzxWI6IZpQ6zpaDyR8oEoIIkdpcAiEZ2nRqzBMD7MM/9NaOE/AMhHhitQesYleVNv7v1qZ83dBAzbr+4O0uOogbfrzyw0ukW3ht2CxikETm3d6UYqE5kLq9Ezp9LL6WvhZaBR37cVTwhOoioWaXIi1z8R2nFiVJ5UWCwfgPdD2/BA6X2XVBBaXdCLHHHJqAZZYEVfuec2JBc6Jc2eVSIHxkpfDnFa46sELcM9Zw2t3T2/Kfjm6ktMsLf51Nyr+8mCsgt5ux/c6/oGHNtBddA+1kY8eo6foJdpHXUTRJ/QZfUXfal9qP2o/50O1vlZO1x1UsdqvP3hKWBY=</latexit><latexit sha1_base64="zW/mNnPCAOW+91Fvcfq/0ZgxwMI=">AAAEQnichVPNctMwEFZjfkr4S+HIRUMmM+nABLsc4Fh+DhygtIWknYlDRlbWiaay5JHkQMb1g/A0XDjAI/AK3BgOXDggO24bJxR2xp5Pu99+Wq20QcyZNq77ba3mXLh46fL6lfrVa9dv3Gxs3OppmSgKXSq5VIcB0cCZgK5hhsNhrIBEAYeD4OhZHj+YgtJMirdmFsMgImPBQkaJsa5h46HPITRtX7NxRO7jvXep92Arwx98xcYTs+kz4UfETCjh6cvMBu+JDA8bTbfjFoZXgVeCJiptd7hR++WPJE0iEIZyonXfc2MzSIkyjHLI6n6iISb0iIyhb6EgEehBWpwuwy3rGeFQKvsJgwvvYkZKIq1nUWCZeal6OZY7T2PVrUz4eJAyEScGBJ3vFCYcG4nzXuERU0ANn1lAqGK2WEwnRBFqbEfrvoD3VEYREaPU3wfCs7ToVRCk+1mG/2ktnCdgkUTBitQOMYlaVNv5v1qZ83dBDSbre4O0uGo/bXrzy/XPkW3hN2CwjEERk3d6UYoE+kzq+ETp+Lz6WvhJoCW37cVTwhOoigWKnIm1T8Q2rViVJ6SKFg7Ae6DM6SFUvsqqCSwu6USMOeTUAiyxQi7t85oTC5wT584qkQLjJS+HOa1w1f3nYJ+zglf2nl6X/bJ0KaZZWvzrdlS85cFYBb2tjud2vD23uf20HJp1dAfdRW3koUdoG71Au6iLKPqIPqEv6Kvz2fnu/HB+zqm1tTLnNqqY8/sPWUNZwQ==</latexit><latexit sha1_base64="SZ4Z+B6wK6hM7q3Sl89ySjL1gt0=">AAAEQnichVPNbhMxEHaz/JTw0xSOXCyiSKlAYbcgwbH8HDhAaQtJK3VD5HVmE6tee2V7A9F2H4Sn4cIBHoFX4IY4cOGAd7NtswmFkXb1eeabz+OxJ4g508Z1v63UnAsXL11evVK/eu36jbXG+s2elomi0KWSS3UQEA2cCegaZjgcxApIFHDYD46e5fH9CSjNpHhrpjH0IzISLGSUGOsaNB74HELT9jUbReQe3n2Xevc3M/zBV2w0Nhs+E35EzJgSnr7MbPCuyPCg0XQ7bmF4GXglaKLSdgbrtV/+UNIkAmEoJ1ofem5s+ilRhlEOWd1PNMSEHpERHFooSAS6nxany3DLeoY4lMp+wuDCO5+RkkjraRRYZl6qXozlztNYdSsTPu6nTMSJAUFnO4UJx0bivFd4yBRQw6cWEKqYLRbTMVGEGtvRui/gPZVRRMQw9feA8CwtehUE6V6W4X9aC+cJWCRRsCS1TUyi5tW2/69W5vxdUIPJDr1+Wly1nza92eX658i28BswWMagiMk7PS9FAn0mdXyidHxefS38JNCS2/biCeEJVMUCRc7E2idiG1asyhNSRXMH4D1Q5vQQKl9l1QQWl3QiRhxyagEWWCGX9nnNiAXOiTNnlUiB8ZKXw5xWuOr+c7DPWcEre0+vy35ZuhSTLC3+dTsq3uJgLIPeZsdzO97uw+bW03JoVtFtdAe1kYceoS30Au2gLqLoI/qEvqCvzmfnu/PD+Tmj1lbKnFuoYs7vP1qDWcU=</latexit><latexit sha1_base64="SZ4Z+B6wK6hM7q3Sl89ySjL1gt0=">AAAEQnichVPNbhMxEHaz/JTw0xSOXCyiSKlAYbcgwbH8HDhAaQtJK3VD5HVmE6tee2V7A9F2H4Sn4cIBHoFX4IY4cOGAd7NtswmFkXb1eeabz+OxJ4g508Z1v63UnAsXL11evVK/eu36jbXG+s2elomi0KWSS3UQEA2cCegaZjgcxApIFHDYD46e5fH9CSjNpHhrpjH0IzISLGSUGOsaNB74HELT9jUbReQe3n2Xevc3M/zBV2w0Nhs+E35EzJgSnr7MbPCuyPCg0XQ7bmF4GXglaKLSdgbrtV/+UNIkAmEoJ1ofem5s+ilRhlEOWd1PNMSEHpERHFooSAS6nxany3DLeoY4lMp+wuDCO5+RkkjraRRYZl6qXozlztNYdSsTPu6nTMSJAUFnO4UJx0bivFd4yBRQw6cWEKqYLRbTMVGEGtvRui/gPZVRRMQw9feA8CwtehUE6V6W4X9aC+cJWCRRsCS1TUyi5tW2/69W5vxdUIPJDr1+Wly1nza92eX658i28BswWMagiMk7PS9FAn0mdXyidHxefS38JNCS2/biCeEJVMUCRc7E2idiG1asyhNSRXMH4D1Q5vQQKl9l1QQWl3QiRhxyagEWWCGX9nnNiAXOiTNnlUiB8ZKXw5xWuOr+c7DPWcEre0+vy35ZuhSTLC3+dTsq3uJgLIPeZsdzO97uw+bW03JoVtFtdAe1kYceoS30Au2gLqLoI/qEvqCvzmfnu/PD+Tmj1lbKnFuoYs7vP1qDWcU=</latexit><latexit sha1_base64="SZ4Z+B6wK6hM7q3Sl89ySjL1gt0=">AAAEQnichVPNbhMxEHaz/JTw0xSOXCyiSKlAYbcgwbH8HDhAaQtJK3VD5HVmE6tee2V7A9F2H4Sn4cIBHoFX4IY4cOGAd7NtswmFkXb1eeabz+OxJ4g508Z1v63UnAsXL11evVK/eu36jbXG+s2elomi0KWSS3UQEA2cCegaZjgcxApIFHDYD46e5fH9CSjNpHhrpjH0IzISLGSUGOsaNB74HELT9jUbReQe3n2Xevc3M/zBV2w0Nhs+E35EzJgSnr7MbPCuyPCg0XQ7bmF4GXglaKLSdgbrtV/+UNIkAmEoJ1ofem5s+ilRhlEOWd1PNMSEHpERHFooSAS6nxany3DLeoY4lMp+wuDCO5+RkkjraRRYZl6qXozlztNYdSsTPu6nTMSJAUFnO4UJx0bivFd4yBRQw6cWEKqYLRbTMVGEGtvRui/gPZVRRMQw9feA8CwtehUE6V6W4X9aC+cJWCRRsCS1TUyi5tW2/69W5vxdUIPJDr1+Wly1nza92eX658i28BswWMagiMk7PS9FAn0mdXyidHxefS38JNCS2/biCeEJVMUCRc7E2idiG1asyhNSRXMH4D1Q5vQQKl9l1QQWl3QiRhxyagEWWCGX9nnNiAXOiTNnlUiB8ZKXw5xWuOr+c7DPWcEre0+vy35ZuhSTLC3+dTsq3uJgLIPeZsdzO97uw+bW03JoVtFtdAe1kYceoS30Au2gLqLoI/qEvqCvzmfnu/PD+Tmj1lbKnFuoYs7vP1qDWcU=</latexit><latexit sha1_base64="SZ4Z+B6wK6hM7q3Sl89ySjL1gt0=">AAAEQnichVPNbhMxEHaz/JTw0xSOXCyiSKlAYbcgwbH8HDhAaQtJK3VD5HVmE6tee2V7A9F2H4Sn4cIBHoFX4IY4cOGAd7NtswmFkXb1eeabz+OxJ4g508Z1v63UnAsXL11evVK/eu36jbXG+s2elomi0KWSS3UQEA2cCegaZjgcxApIFHDYD46e5fH9CSjNpHhrpjH0IzISLGSUGOsaNB74HELT9jUbReQe3n2Xevc3M/zBV2w0Nhs+E35EzJgSnr7MbPCuyPCg0XQ7bmF4GXglaKLSdgbrtV/+UNIkAmEoJ1ofem5s+ilRhlEOWd1PNMSEHpERHFooSAS6nxany3DLeoY4lMp+wuDCO5+RkkjraRRYZl6qXozlztNYdSsTPu6nTMSJAUFnO4UJx0bivFd4yBRQw6cWEKqYLRbTMVGEGtvRui/gPZVRRMQw9feA8CwtehUE6V6W4X9aC+cJWCRRsCS1TUyi5tW2/69W5vxdUIPJDr1+Wly1nza92eX658i28BswWMagiMk7PS9FAn0mdXyidHxefS38JNCS2/biCeEJVMUCRc7E2idiG1asyhNSRXMH4D1Q5vQQKl9l1QQWl3QiRhxyagEWWCGX9nnNiAXOiTNnlUiB8ZKXw5xWuOr+c7DPWcEre0+vy35ZuhSTLC3+dTsq3uJgLIPeZsdzO97uw+bW03JoVtFtdAe1kYceoS30Au2gLqLoI/qEvqCvzmfnu/PD+Tmj1lbKnFuoYs7vP1qDWcU=</latexit><latexit sha1_base64="SZ4Z+B6wK6hM7q3Sl89ySjL1gt0=">AAAEQnichVPNbhMxEHaz/JTw0xSOXCyiSKlAYbcgwbH8HDhAaQtJK3VD5HVmE6tee2V7A9F2H4Sn4cIBHoFX4IY4cOGAd7NtswmFkXb1eeabz+OxJ4g508Z1v63UnAsXL11evVK/eu36jbXG+s2elomi0KWSS3UQEA2cCegaZjgcxApIFHDYD46e5fH9CSjNpHhrpjH0IzISLGSUGOsaNB74HELT9jUbReQe3n2Xevc3M/zBV2w0Nhs+E35EzJgSnr7MbPCuyPCg0XQ7bmF4GXglaKLSdgbrtV/+UNIkAmEoJ1ofem5s+ilRhlEOWd1PNMSEHpERHFooSAS6nxany3DLeoY4lMp+wuDCO5+RkkjraRRYZl6qXozlztNYdSsTPu6nTMSJAUFnO4UJx0bivFd4yBRQw6cWEKqYLRbTMVGEGtvRui/gPZVRRMQw9feA8CwtehUE6V6W4X9aC+cJWCRRsCS1TUyi5tW2/69W5vxdUIPJDr1+Wly1nza92eX658i28BswWMagiMk7PS9FAn0mdXyidHxefS38JNCS2/biCeEJVMUCRc7E2idiG1asyhNSRXMH4D1Q5vQQKl9l1QQWl3QiRhxyagEWWCGX9nnNiAXOiTNnlUiB8ZKXw5xWuOr+c7DPWcEre0+vy35ZuhSTLC3+dTsq3uJgLIPeZsdzO97uw+bW03JoVtFtdAe1kYceoS30Au2gLqLoI/qEvqCvzmfnu/PD+Tmj1lbKnFuoYs7vP1qDWcU=</latexit><latexit sha1_base64="SZ4Z+B6wK6hM7q3Sl89ySjL1gt0=">AAAEQnichVPNbhMxEHaz/JTw0xSOXCyiSKlAYbcgwbH8HDhAaQtJK3VD5HVmE6tee2V7A9F2H4Sn4cIBHoFX4IY4cOGAd7NtswmFkXb1eeabz+OxJ4g508Z1v63UnAsXL11evVK/eu36jbXG+s2elomi0KWSS3UQEA2cCegaZjgcxApIFHDYD46e5fH9CSjNpHhrpjH0IzISLGSUGOsaNB74HELT9jUbReQe3n2Xevc3M/zBV2w0Nhs+E35EzJgSnr7MbPCuyPCg0XQ7bmF4GXglaKLSdgbrtV/+UNIkAmEoJ1ofem5s+ilRhlEOWd1PNMSEHpERHFooSAS6nxany3DLeoY4lMp+wuDCO5+RkkjraRRYZl6qXozlztNYdSsTPu6nTMSJAUFnO4UJx0bivFd4yBRQw6cWEKqYLRbTMVGEGtvRui/gPZVRRMQw9feA8CwtehUE6V6W4X9aC+cJWCRRsCS1TUyi5tW2/69W5vxdUIPJDr1+Wly1nza92eX658i28BswWMagiMk7PS9FAn0mdXyidHxefS38JNCS2/biCeEJVMUCRc7E2idiG1asyhNSRXMH4D1Q5vQQKl9l1QQWl3QiRhxyagEWWCGX9nnNiAXOiTNnlUiB8ZKXw5xWuOr+c7DPWcEre0+vy35ZuhSTLC3+dTsq3uJgLIPeZsdzO97uw+bW03JoVtFtdAe1kYceoS30Au2gLqLoI/qEvqCvzmfnu/PD+Tmj1lbKnFuoYs7vP1qDWcU=</latexit>

Fig. 3.1: Boundary of quadratic cone x1

�p

x2

2

+ x2

3

and rotated quadratic cone 2x1

x2

� x2

3

,x1

, x2

� 0.

3.1.2 Rotated quadratic conesAn n�dimensional rotated quadratic cone is defined as

Qnr =

�

x 2 Rn | 2x1

x2

� x2

3

+ · · · + x2

n, x1

, x2

� 0

. (3.2)

As the name indicates, there is a simple relationship between quadratic and rotated quadraticcones. Define an orthogonal transformation

Tn :=

2

4

1/p2 1/

p2 0

1/p2 �1/

p2 0

0 0 In�2

3

5 . (3.3)

Then it is easy to verify that

x 2 Qn () Tnx 2 Qnr ,

and since T is orthogonal we call Qnr a rotated cone; the transformation corresponds to a

rotation of ⇡/4 in the (x1

, x2

) plane. For example if x 2 Q3 and2

4

z1

z2

z3

3

5 =

2

4

1p2

1p2

01p2

� 1p2

0

0 0 1

3

5 ·2

4

x1

x2

x3

3

5 =

2

4

1p2

(x1

+ x2

)1p2

(x1

� x2

)

x3

3

5

then

2z1

z2

� z23

, z1

, z2

� 0 =) (x2

1

� x2

2

) � x2

3

, x1

� 0,

and similarly we see that

x2

1

� x2

2

+ x2

3

, x1

� 0 =) 2z1

z2

� z23

, z1

, z2

� 0.

Thus, one could argue that we only need quadratic cones Qn, but there are many exampleswhere using an explicit rotated quadratic cone Qn

r is more natural, as we will see next.

21

Chapter 3

Conic quadratic optimization

This chapter extends the notion of linear optimization with quadratic cones. Conic quadraticoptimization, also known as second-order cone optimization, is a straightforward generaliza-tion of linear optimization, in the sense that we optimize a linear function under linear(in)equalities with some variables belonging to one or more (rotated) quadratic cones. Wediscuss the basic concept of quadratic cones, and demonstrate the surprisingly large flexibilityof conic quadratic modeling.

3.1 Cones

Since this is the first place where we introduce a non-linear cone, it seems suitable to makeour most important definition:

A set K ✓ Rn is called a convex cone if

• for every x, y 2 K we have x+ y 2 K,

• for every x 2 K and ↵ � 0 we have ↵x 2 K.

For example a linear subspace of Rn, the positive orthant Rn�0

or any ray (half-line)starting at the origin are examples of convex cones. We leave it for the reader to checkthat the intersection of convex cones is a convex cone; this property enables us to assemblecomplicated optimization models from individual conic bricks.

3.1.1 Quadratic conesWe define the n-dimensional quadratic cone as

Qn =

⇢

x 2 Rn | x1

�q

x2

2

+ x2

3

+ · · · + x2

n

�

. (3.1)

The geometric interpretation of a quadratic (or second-order) cone is shown in Fig. 3.1 fora cone with three variables, and illustrates how the boundary of the cone resembles anice-cream cone. The 1-dimensional quadratic cone simply states nonnegativity x

1

� 0.

20

Lorentz cone or Second Order Cone

MMC


min2RN

h , i : (M)

k � k 2 Ck 8k 2 [M]

xi 2 Z 8i 2 [I ]







min2RN

h , i : (M)

k � k 2 Ck 8k 2 [M]

xi 2 Z 8i 2 [I ]







min2RN

h , i : (M)

k � k 2 Ck 8k 2 [M]

xi 2 Z 8i 2 [I ]







min2RN

h , i : (M)

k � k 2 Ck 8k 2 [M]

xi 2 Z 8i 2 [I ]






MICP Example 1a: Rotated Second Order Cone (RSOCP)

• RSOCP :• Can be used to model geometric mean

! ≤ #$%&

'($

&/'

*&,&, ≤ (& - (,*&,,, ≤ (. - (/

⋮*&,'/,, ≤ ('1& - ('

*,,&, ≤ *&,& - *&,,*,,,, ≤ *&,. - *&,/

⋮*,,'//, ≤ *&,'/,1& - *&,'/,

!, ≤ *21&,& - *21&,,


�p

x2

2

+ x2

3


x2

� x2

3

,x1

, x2

� 0.


Qnr =

�

x 2 Rn | 2x1

x2

� x2

3

+ · · · + x2

n, x1

, x2

� 0

. (3.2)


Tn :=

2

4

1/p2 1/

p2 0

1/p2 �1/

p2 0

0 0 In�2

3

5 . (3.3)





, x2


4

z1

z2

z3

3

5 =

2

4

1p2

1p2

01p2

� 1p2

0

0 0 1

3

5 ·2

4

x1

x2

x3

3

5 =

2

4

1p2

(x1

+ x2

)1p2

(x1

� x2

)

x3

3

5

then

2z1

z2

� z23

, z1

, z2

� 0 =) (x2

1

� x2

2

) � x2

3

, x1

� 0,


x2

1

� x2

2

+ x2

3

, x1

� 0 =) 2z1

z2

� z23

, z1

, z2

� 0.



21

⋯


�p

x2

2

+ x2

3


x2

� x2

3

,x1

, x2

� 0.


Qnr =

�

x 2 Rn | 2x1

x2

� x2

3

+ · · · + x2

n, x1

, x2

� 0

. (3.2)


Tn :=

2

4

1/p2 1/

p2 0

1/p2 �1/

p2 0

0 0 In�2

3

5 . (3.3)





, x2


4

z1

z2

z3

3

5 =

2

4

1p2

1p2

01p2

� 1p2

0

0 0 1

3

5 ·2

4

x1

x2

x3

3

5 =

2

4

1p2

(x1

+ x2

)1p2

(x1

� x2

)

x3

3

5

then

2z1

z2

� z23

, z1

, z2

� 0 =) (x2

1

� x2

2

) � x2

3

, x1

� 0,


x2

1

� x2

2

+ x2

3

, x1

� 0 =) 2z1

z2

� z23

, z1

, z2

� 0.



21

⟺

assume : = 22

MMC

MICP Example 1a: Rotated Second Order Cone (RSOCP)

• Exp-Cone :

• Can be used to model log-sum-exp

Fig. 5.1: The boundary of the exponential cone Kexp

. The red isolines are graphs of x2

!x2

log(x1

/x2

) for fixed x1

, see (5.3).

5.2 Modeling with the exponential cone

Extending the conic optimization toolbox with the exponential cone leads to new types ofconstraint building blocks and new types of representable sets. In this section we list thebasic operations available using the exponential cone.

5.2.1 ExponentialThe epigraph t � ex is a section of K

exp

:

t � ex () (t, 1, x) 2 Kexp

. (5.4)

5.2.2 LogarithmSimilarly, we can express the hypograph t log x, x � 0:

t log x () (x, 1, t) 2 Kexp

. (5.5)

5.2.3 EntropyThe entropy function H(x) = �x log x can be maximized using the following representationwhich follows directly from (5.3):

t �x log x () t x log(1/x) () (1, x, t) 2 Kexp

. (5.6)

39

Chapter 5

Exponential cone optimization

So far we discussed optimization problems involving the major “polynomial” families of cones:linear, quadratic and power cones. In this chapter we introduce a single new object, namelythe three-dimensional exponential cone, together with examples and applications. The ex-ponential cone can be used to model a variety of constraints involving exponentials andlogarithms.

5.1 Exponential cone

The exponential cone is a convex subset of R3 defined as

Kexp

=�

(x1

, x2

, x3

) : x1

� x2

ex3/x2 , x2

> 0 [

{(x1

, 0, x3

) : x1

� 0, x3

0} . (5.1)

Thus the exponential cone is the closure in R3 of the set of points which satisfy

x1

� x2

ex3/x2 , x1

, x2

> 0. (5.2)

When working with logarithms, a convenient reformulation of (5.2) is

x3

x2

log(x1

/x2

), x1

, x2

> 0. (5.3)

Alternatively, one can write the same condition as

x1

/x2

� ex3/x2 , x1

, x2

> 0,

which immediately shows that Kexp

is in fact a cone, i.e. ↵x 2 Kexp

for x 2 Kexp

and ↵ � 0.Convexity of K

exp

follows from the fact that the Hessian of f(x, y) = y exp(x/y), namely

D2(f) = ex/y

y�1 �xy�2

�xy�2 x2y�3

�

is positive semidefinite for y > 0.

38

Chapter 5

Exponential cone optimization

So far we discussed optimization problems involving the major “polynomial” families of cones:linear, quadratic and power cones. In this chapter we introduce a single new object, namelythe three-dimensional exponential cone, together with examples and applications. The ex-ponential cone can be used to model a variety of constraints involving exponentials andlogarithms.

5.1 Exponential cone

The exponential cone is a convex subset of R3 defined as

Kexp

=�

(x1

, x2

, x3

) : x1

� x2

ex3/x2 , x2

> 0 [

{(x1

, 0, x3

) : x1

� 0, x3

0} . (5.1)

Thus the exponential cone is the closure in R3 of the set of points which satisfy

x1

� x2

ex3/x2 , x1

, x2

> 0. (5.2)

When working with logarithms, a convenient reformulation of (5.2) is

x3

x2

log(x1

/x2

), x1

, x2

> 0. (5.3)

Alternatively, one can write the same condition as

x1

/x2

� ex3/x2 , x1

, x2

> 0,

which immediately shows that Kexp

is in fact a cone, i.e. ↵x 2 Kexp

for x 2 Kexp

and ↵ � 0.Convexity of K

exp

follows from the fact that the Hessian of f(x, y) = y exp(x/y), namely

D2(f) = ex/y

y�1 �xy�2

�xy�2 x2y�3

�

is positive semidefinite for y > 0.

38

5.2.4 Relative entropyThe relative entropy or Kullback-Leiber divergence of two probability distributions is definedin terms of the function D(x, y) = x log(x/y). It is convex, and the minimization problemt � D(x, y) is equivalent to

t � D(x, y) () �t x log(y/x) () (y, x,�t) 2 Kexp

. (5.7)

Because of this reparametrization the exponential cone is also referred to as the relativeentropy cone, leading to a class of problems known as REPs (relative entropy problems).Having the relative entropy function available makes it possible to express epigraphs of otherfunctions appearing in REPs, for instance:

x log(1 + x/y) = D(x+ y, y) +D(y, x+ y).

5.2.5 Softplus functionIn neural networks the function f(x) = log(1+ex), known as the softplus function, is used asan analytic approximation to the rectifier activation function r(x) = x+ = max(0, x). Thesoftplus function is convex and we can express its epigraph t � log(1+ ex) by combining twoexponential cones. Note that

t � log(1 + ex) () ex�t + e�t 1

and therefore t � log(1 + ex) is equivalent to the following set of conic constraints:

u+ v 1,(u, 1, x � t) 2 K

exp

,(v, 1,�t) 2 K

exp

.(5.8)

5.2.6 Log-sum-expWe can generalize the previous example to a log-sum-exp (logarithm of sum of exponentials)expression

t � log(ex1 + · · · + exn).

This is equivalent to the inequality

ex1�t + · · · + exn�t 1,

and so it can be modeled as follows:P

ui 1,(ui, 1, xi � t) 2 K

exp

, i = 1, . . . , n.(5.9)

40

5.2.4 Relative entropyThe relative entropy or Kullback-Leiber divergence of two probability distributions is definedin terms of the function D(x, y) = x log(x/y). It is convex, and the minimization problemt � D(x, y) is equivalent to

t � D(x, y) () �t x log(y/x) () (y, x,�t) 2 Kexp

. (5.7)

Because of this reparametrization the exponential cone is also referred to as the relativeentropy cone, leading to a class of problems known as REPs (relative entropy problems).Having the relative entropy function available makes it possible to express epigraphs of otherfunctions appearing in REPs, for instance:

x log(1 + x/y) = D(x+ y, y) +D(y, x+ y).

5.2.5 Softplus functionIn neural networks the function f(x) = log(1+ex), known as the softplus function, is used asan analytic approximation to the rectifier activation function r(x) = x+ = max(0, x). Thesoftplus function is convex and we can express its epigraph t � log(1+ ex) by combining twoexponential cones. Note that

t � log(1 + ex) () ex�t + e�t 1

and therefore t � log(1 + ex) is equivalent to the following set of conic constraints:

u+ v 1,(u, 1, x � t) 2 K

exp

,(v, 1,�t) 2 K

exp

.(5.8)

5.2.6 Log-sum-expWe can generalize the previous example to a log-sum-exp (logarithm of sum of exponentials)expression

t � log(ex1 + · · · + exn).

This is equivalent to the inequality

ex1�t + · · · + exn�t 1,

and so it can be modeled as follows:P

ui 1,(ui, 1, xi � t) 2 K

exp

, i = 1, . . . , n.(5.9)

40

⟺

MMC

MICP Example 2: MI – Semidefinite Programming (+ Exp Cone)

• Find minimum volume ellipsoid that contains 90% of data points

MICP Example 2: MI – Semidefinite Programming (+ Exp Cone)

• Semidefinite constraints:–– Set of positive semidefinite

symmetric matrices:

Chapter 6

Semidefinite optimization

In this chapter we extend the conic optimization framework introduced before with symmet-ric positive semidefinite matrix variables.

6.1 Introduction to semidefinite matrices

6.1.1 Semidefinite matrices and conesA symmetric matrix X 2 Sn is called symmetric positive semidefinite if

zTXz � 0, 8z 2 Rn.

We then define the cone of symmetric positive semidefinite matrices as

Sn+

= {X 2 Sn | zTXz � 0, 8z 2 Rn}. (6.1)

For brevity we will often use the shorter notion semidefinite instead of symmetric positivesemidefinite, and we will write X ⌫ Y (X � Y ) as shorthand notation for (X � Y ) 2 Sn

+

((Y �X) 2 Sn+

). As inner product for semidefinite matrices, we use the standard trace innerproduct for general matrices, i.e.,

hA,Bi := tr(ATB) =X

ij

aijbij.

It is easy to see that (6.1) indeed specifies a convex cone; it is pointed (with origin X = 0),and X, Y 2 Sn

+

implies that (↵X + �Y ) 2 Sn+

, ↵, � � 0. Let us review a few equivalentdefinitions of Sn

+

. It is well-known that every symmetric matrix A has a spectral factorization

A =n

X

i=1

�iqiqTi .

where qi 2 Rn are the (orthogonal) eigenvectors and �i are eigenvalues of A. Using thespectral factorization of A we have

xTAx =n

X

i=1

�i(xT qi)

2,

51

: set of symmetric matrices

Chapter 6





zTXz � 0, 8z 2 Rn.


Sn+

= {X 2 Sn | zTXz � 0, 8z 2 Rn}. (6.1)


+

((Y �X) 2 Sn+


hA,Bi := tr(ATB) =X

ij

aijbij.


+



+


A =n

X

i=1

�iqiqTi .


xTAx =n

X

i=1

�i(xT qi)

2,

51

Fig. 6.1: Plot of spectrahedron S = {(x, y, z) 2 R3 | A(x, y, z) ⌫ 0}.

• Any principal submatrix of A 2 Sn+

(A restricted to the same set of rows as columns)is positive semidefinite; this follows by restricting the Grammian characterization A =V TV to a submatrix of V .

• The inner product of positive (semi)definite matrices is positive (nonnegative). Forany A,B 2 Sn

++

let A = UTU and B = V TV where U and V have full rank. Then

hA,Bi = tr(UTUV TV ) = kUV Tk2

F > 0,

where strict positivity follows from the assumption that U has full column-rank, i.e.,UV T 6= 0.

• The inverse of a positive definite matrix is positive definite. This follows from thepositive spectral factorization A = Q⇤QT , which gives us

A�1 = QT⇤�1Q

where ⇤ii > 0. If A is semidefinite then the pseudo-inverse A† of A is semidefinite.

• Consider a matrix X 2 Sn partitioned as

X =

A BT

B C

�

.

Let us find necessary and sufficient conditions for X � 0. We know that A � 0 andC � 0 (since any principal submatrix must be positive definite). Furthermore, we can

54

Chapter 6





zTXz � 0, 8z 2 Rn.


Sn+

= {X 2 Sn | zTXz � 0, 8z 2 Rn}. (6.1)


+

((Y �X) 2 Sn+


hA,Bi := tr(ATB) =X

ij

aijbij.


+



+


A =n

X

i=1

�iqiqTi .


xTAx =n

X

i=1

�i(xT qi)

2,

51

Chapter 6





zTXz � 0, 8z 2 Rn.


Sn+

= {X 2 Sn | zTXz � 0, 8z 2 Rn}. (6.1)


+

((Y �X) 2 Sn+


hA,Bi := tr(ATB) =X

ij

aijbij.


+



+


A =n

X

i=1

�iqiqTi .


xTAx =n

X

i=1

�i(xT qi)

2,

51

⟺

Example 6.1. As a more interesting example, consider the symmetric matrix

A(x, y, z) =

2

4

1 x yx 1 zy z 1

3

5 (6.4)

parametrized by (x, y, z). The set

S = {(x, y, z) 2 R3 | A(x, y, z) 2 S3

+

},

(shown in Fig. 6.1) is called a spectrahedron and is perhaps the simplest bounded semidef-inite representable set, which cannot be represented using (finitely many) linear orquadratic cones. To gain a geometric intuition of S, we note that

det(A(x, y, z)) = �(x2 + y2 + z2 � 2xyz � 1),

so the boundary of S can be characterized as

x2 + y2 + z2 � 2xyz = 1,

or equivalently as

xy

�T

1 �z�z 1

�

xy

�

= 1 � z2.

For z = 0 this describes a circle in the (x, y)-plane, and for �1 z 1 it characterizesan ellipse (for a fixed z).

6.1.2 Properties of semidefinite matricesMany useful properties of (semi)definite matrices follow directly from the definitions (6.1)-(6.3) and their definite counterparts.

• The diagonal elements of A 2 Sn+

are nonnegative. Let ei denote the ith standard basisvector (i.e., [ei]j = 0, j 6= i, [ei]i = 1). Then Aii = eTi Aei, so (6.1) implies that Aii � 0.

• A block-diagonal matrix A = Diag(A1

, . . . Ap) is (semi)definite if and only if eachdiagonal block Ai is (semi)definite.

• Given a quadratic transformation M := BTAB, M � 0 if and only if A � 0 and B hasfull rank. This follows directly from the Grammian characterization M = (V B)T (V B).For M ⌫ 0 we only require that A ⌫ 0. As an example, if A is (semi)definite then sois any permutation P TAP .

53

Fig. 6.1: Plot of spectrahedron S = {(x, y, z) 2 R3 | A(x, y, z) ⌫ 0}.

• Any principal submatrix of A 2 Sn+

(A restricted to the same set of rows as columns)is positive semidefinite; this follows by restricting the Grammian characterization A =V TV to a submatrix of V .

• The inner product of positive (semi)definite matrices is positive (nonnegative). Forany A,B 2 Sn

++

let A = UTU and B = V TV where U and V have full rank. Then

hA,Bi = tr(UTUV TV ) = kUV Tk2

F > 0,

where strict positivity follows from the assumption that U has full column-rank, i.e.,UV T 6= 0.

• The inverse of a positive definite matrix is positive definite. This follows from thepositive spectral factorization A = Q⇤QT , which gives us

A�1 = QT⇤�1Q

where ⇤ii > 0. If A is semidefinite then the pseudo-inverse A† of A is semidefinite.

• Consider a matrix X 2 Sn partitioned as

X =

A BT

B C

�

.

Let us find necessary and sufficient conditions for X � 0. We know that A � 0 andC � 0 (since any principal submatrix must be positive definite). Furthermore, we can

54

MMC

MICP Example 2: SDP representation of ellipsoid take 1

! − # $% ! − # ≤ 1⟺ !$%! − 2!$%# + #$%# ≤ 1⟺ !$%! − 2!$%# + + ≤ 1

#$%# ≤ +⟺ !$%! − 2!$%# + + ≤ 1

+ #$# %,- ≽ 0

!

0 = {3 ∈ ℝ6: 3 − # $% 3 − # ≤ 1}, % ∈ :;6

Vol(0) ∝ Det(%),-/F

3.2. WHAT CAN BE EXPRESSED VIA LMI’S? 143

Since Z ⌫ 0 (see (3.2.2.b)), we have Sk

(Z) Tr(Z), and combining these inequalities we get

Sk

(X) Tr(Z) + sk.

The latter inequality, in view of (3.2.2.a)), implies Sk

(X) t, and (i) is proved.

To prove (ii), assume that we are given X, t with Sk

(X) t, and let us set s = �k

(X).

Then the k largest eigenvalues of the matrix X�sIm

are nonnegative, and the remaining are

nonpositive. Let Z be a symmetric matrix with the same eigenbasis as X and such that the

k largest eigenvalues of Z are the same as those of X � sIm

, and the remaining eigenvalues

are zeros. The matrices Z and Z � X + sIm

are clearly positive semidefinite (the first by

construction, and the second since in the eigenbasis of X this matrix is diagonal with the first

k diagonal entries being 0 and the remaining being the same as those of the matrix sIm

�X,

i.e., nonnegative). Thus, the matrix Z and the real s we have built satisfy (3.2.2.b, c). In

order to see that (3.2.2.a) is satisfied as well, note that by construction Tr(Z) = Sk

(x)� sk,

whence t� sk � Tr(Z) = t� Sk

(x) � 0.

In order to proceed, we need the following highly useful technical result:

Lemma 3.2.1 [Lemma on the Schur Complement] Let

A =✓B CT

C D

◆be a symmetric matrix with k⇥ k block B and `⇥ ` block D. Assume that B is positive definite.Then A is positive (semi)definite if and only if the matrix

D � CB�1CT

is positive (semi)definite (this matrix is called the Schur complement of B in A).

Proof. The positive semidefiniteness of A is equivalent to the fact that

0 (xT , yT )✓B CT

C D

◆✓xy

◆= xTBx+ 2xTCT y + yTDy 8x 2 Rk, y 2 R`,

or, which is the same, to the fact that

infx2Rk

hxTBx+ 2xTCT y + yTDy

i� 0 8y 2 R`.

Since B is positive definite by assumption, the infimum in x can be computed explicitly for everyfixed y: the optimal x is �B�1CT y, and the optimal value is

yTDy � yTCB�1CT y = yT [D � CB�1CT ]y.

The positive definiteness/semidefiniteness of A is equivalent to the fact that the latter ex-pression is, respectively, positive/nonnegative for every y 6= 0, i.e., to the positive definite-ness/semidefiniteness of the Schur complement of B in A.




Sk

(X) Tr(Z) + sk.





(X).










�X,



(x)� sk,


(x) � 0.



A =✓B CT

C D


D � CB�1CT



0 (xT , yT )✓B CT

C D

◆✓xy



infx2Rk


i� 0 8y 2 R`.




3.7. EXTREMAL ELLIPSOIDS 213

3.7.2.2 Outer ellipsoidal approximation of a finite set

Let S be a polyhedral set given as a convex hull of a finite set of points:

S = Conv{x1

, ..., xm}.

Proposition 3.7.2 Assume that S is a full-dimensional polytope (i.e., possesses a nonemptyinterior). Then the smallest volume ellipsoid containing S is

E = {x | (x� c⇤)TD⇤(x� c⇤) 1},

where c⇤, D⇤ are given by an optimal solution (t⇤, Z⇤, z⇤, s⇤) to the semidefinite program

maximize ts.t.(a) t (DetZ)1/n,(b) Z ⌫ 0,

(c)✓s zT

z Z

◆⌫ 0,

(d) xTi Zxi � 2xTi z + s 1, i = 1, ...,m,

(Out)

with the design variables Z 2 Sn, z 2 Rn, t, s 2 R via the relations

D⇤ = Z⇤; c⇤ = Z�1

⇤ z⇤.

Note that (Out) indeed is a semidefinite program, cf. Proposition 3.7.1.

Proof. Indeed, let us pass in the description (3.7.2) from the “parameters” D, c to the param-eters Z = D, z = Dc, thus coming to the representation

E = {x | xTZx� 2xT z + zTZ�1z 1}. (!)

The ellipsoid of the latter type contains the points x1

, ..., xm if and only if

xTi Zxi � 2xTi z + zTZ�1z 1, i = 1, ...,m,

or, which is the same, if and only if there exists s � zTZ�1z such that

xTi Zxi � 2xTi z + s 1, i = 1, ...,m.

Recalling Lemma on the Schur Complement, we see that the constraints (Out.b�d) say exactlythat the ellipsoid (!) contains the points x

1

, ..., xm. Since the volume of such an ellipsoid is(DetZ)�1/2, (Out) is the problem of maximizing a negative power of the volume of an ellipsoidcontaining the finite set {x

1

, ..., xm}, i.e., the problem of finding the smallest volume ellipsoidcontaining this finite set. It remains to note that an ellipsoid is convex, so that it is exactly thesame – to say that it contains a finite set {x

1

, ..., xm} and to say that it contains the convex hullof this finite set.

We see that if S is a polytope given as a convex hull of a finite set, then the problem ofthe best outer ellipsoidal approximation of S is an explicit semidefinite program and as suchcan be e�ciently solved. In contrast to this, if S is a polytope given by a list of inequalityconstraints, then the problem of the best outer ellipsoidal approximation of S is “computationallyintractable” – in this case, it is di�cult just to check whether a given candidate ellipsoid containsS.

Schur Complement Lemma:

⟺




S = Conv{x1

, ..., xm}.


E = {x | (x� c⇤)TD⇤(x� c⇤) 1},



(c)✓s zT

z Z

◆⌫ 0,

(d) xTi Zxi � 2xTi z + s 1, i = 1, ...,m,

(Out)


D⇤ = Z⇤; c⇤ = Z�1

⇤ z⇤.



E = {x | xTZx� 2xT z + zTZ�1z 1}. (!)





xTi Zxi � 2xTi z + s 1, i = 1, ...,m.


1


1


1



MICP Example 2: SDP representation of ellipsoid take 2

! − # $% ! − # ≤ 1⟺ !$%! − 2!$%# + #$%# ≤ 1⟺ !$%! − 2!$%# + + ≤ 1

#$%# ≤ +⟺ !$%! − 2!$, + + ≤ 1

,$%-., ≤ +⟺ !$%! − 2!$, + + ≤ 1

+ ,$, % ≽ 0

!

1 = {4 ∈ ℝ7: 4 − # $% 4 − # ≤ 1}, % ∈ ;<7

Vol(1) ∝ Det(%)-./G

, = %#




Sk

(X) Tr(Z) + sk.





(X).










�X,



(x)� sk,


(x) � 0.



A =✓B CT

C D


D � CB�1CT



0 (xT , yT )✓B CT

C D

◆✓xy



infx2Rk


i� 0 8y 2 R`.







Sk

(X) Tr(Z) + sk.





(X).










�X,



(x)� sk,


(x) � 0.



A =✓B CT

C D


D � CB�1CT



0 (xT , yT )✓B CT

C D

◆✓xy



infx2Rk


i� 0 8y 2 R`.







S = Conv{x1

, ..., xm}.


E = {x | (x� c⇤)TD⇤(x� c⇤) 1},



(c)✓s zT

z Z

◆⌫ 0,

(d) xTi Zxi � 2xTi z + s 1, i = 1, ...,m,

(Out)


D⇤ = Z⇤; c⇤ = Z�1

⇤ z⇤.



E = {x | xTZx� 2xT z + zTZ�1z 1}. (!)





xTi Zxi � 2xTi z + s 1, i = 1, ...,m.


1


1


1



Schur Complement Lemma:

⟺




S = Conv{x1

, ..., xm}.


E = {x | (x� c⇤)TD⇤(x� c⇤) 1},



(c)✓s zT

z Z

◆⌫ 0,

(d) xTi Zxi � 2xTi z + s 1, i = 1, ...,m,

(Out)


D⇤ = Z⇤; c⇤ = Z�1

⇤ z⇤.



E = {x | xTZx� 2xT z + zTZ�1z 1}. (!)





xTi Zxi � 2xTi z + s 1, i = 1, ...,m.


1


1


1



(!)

MICP Example 2: SDP representation of determinant part 1

! ""# $

≽ 0,

diag " = diag(!)" is upper triangular! is diagonal

8

9 = {; ∈ ℝ>: ; − A #$ ; − A ≤ 1}, $ ∈ EF>

Vol(9) ∝ Det($)JK/M

NOPK

>!O,O ≤ Det($)

⟺

(!)

MICP Example 2: SDP representation of determinant part 2a

!

" = {% ∈ ℝ(: % − + ,- % − + ≤ 1}, - ∈ 23(

Vol(") ∝ Det(-)=>/@ ABC>

(DB,B ≤ Det(-)

E ≤ ABC>

(DB,B

>/( F>,>@ ≤ D>,> G D@,@F>,@@ ≤ DH,H G DI,I

⋮F>,(/@@ ≤ D((=>,(=>) G D(,(

E@ ≤ FK=>,> G FK=>,@⋯

⟺

(!)

MICP Example 2: SDP representation of determinant part 2a

!

" = {% ∈ ℝ(: % − + ,- % − + ≤ 1}, - ∈ 23(

Vol(") ∝ Det(-)=>/@ ABC>

(DB,B ≤ Det(-)

(!)

min Vol(")

max Det(-) >/(

max JK. J.

(")(#) SDP

SDP

RSOCP

⟺ ⟺

SDP

MICP Example 2: SDP representation of determinant part 2b

!

" = {% ∈ ℝ(: % − + ,- % − + ≤ 1}, - ∈ 23(

Vol(") ∝ Det(-)=>/@ ABC>

(DB,B ≤ Det(-)

E ≤FBC>

(Log (IB,B) ≤ Log Det(-)

E ≤FBC>

(Log (IB,B)

E ≤FBC>

(EB

IB,B , 1, EB ∈ JKLM

⟺ (!)

MICP Example 2: SDP representation of determinant part 2b

!

" = {% ∈ ℝ(: % − + ,- % − + ≤ 1}, - ∈ 23(

Vol(") ∝ Det(-)=>/@ ABC>

(

DB,B ≤ Det(-)

(!)

min Vol(")

max Log Det(-)

max LM. L.

(")(#) SDP

SDP

EXP-Cone

⟺ ⟺

SDP + EXP

MICP Example 2: MI part = choose points

!

" = {% ∈ ℝ(: % − + ,- % − + ≤ 1}, - ∈ 23(

Vol(") ∝ Det(-)=>/@

!,-! − 2!,B + D ≤ 1 ∀ ! ∈ F

D B,

B -≽ 0

(!)

!,-! − 2!,B + D ≤ 1 +I 1 − JK ∀ ! ∈ F

∑K∈M JK ≥ 0.9 F

JK ∈ 0,1 ∀ ! ∈ F

D B,

B -≽ 0

(")

References:

• Conic Optimization: – http://www2.isye.gatech.edu/~nemirovs/LMCO_LN.pdf– https://web.stanford.edu/~boyd/cvxbook/– https://themosekblog.blogspot.com/2018/05/new-modeling-cookbook.html

• MI-Conic Optimization:– https://arxiv.org/abs/1808.05290

http://www2.isye.gatech.edu/~nemirovs/LMCO_LN.pdf

https://web.stanford.edu/~boyd/cvxbook/

https://themosekblog.blogspot.com/2018/05/new-modeling-cookbook.html

https://arxiv.org/abs/1808.05290

mixed integer conic optimization using julia and jump · mixed integer conic optimization using...

Documents