lecture 2 - university of illinoisangelia/ie598ns_lect2.pdf · a. nedich and u. shanbhag lecture 2...

Lecture 2

Topology of Sets in Rn

August 27, 2008

A. Nedich and U. Shanbhag Lecture 2

Outline

• Vectors, Matrices, Norms, Convergence

• Open and Closed Sets

• Special Sets: Subspace, Affine Set, Cone, Convex Set

• Special Convex Sets: Hyperplane, Polyhedral Set, Ball

• Special Cones: Normal Cone, Dual Cone, Tangent Cone

• Some Operations with Sets:

• Set Closure, Interior, Boundary

• Convex Hull, Conical Hull, Affine Hull

• Relative Closure, Interior, Boundary

• Functions

• Special Functions: Linear, Quadratic

Game Theory: Models, Algorithms and Applications 1


Notation: Vectors and Matrices

We consider the space of n-dimensional real vectors denoted by Rn

• Vector x ∈ Rn is viewed as a column: x =

x1...

xn

• The prime denotes a transpose: x′ = [x1, . . . , xn]

• For a matrix A, we use aij or [A]ij to denote its (i, j)-th entry

• The matrix A is positive semidefinite (A ≥ 0) when

x′Ax ≥ 0 for all x ∈ Rn

• The matrix A is positive definite (A > 0) when

x′Ax > 0 for all x ∈ Rn, x 6= 0



Inner Product and Norms

• For vectors x ∈ Rn and y ∈ Rn, the inner product is

x′y =n∑

i=1

xiyi

• Norm (length) of a vector:

• Euclidean norm ‖x‖ =√

x′x =√∑n

i=1 x2i (l2-norm)

• Sum-norm ‖x‖1 =∑n

i=1 |xi| (l1-norm)

• Max-norm ‖x‖∞ = max1≤i≤n |xi| (l∞-norm)

• Matrix norm induced by (Euclidean) vector norm

‖A‖ = supx 6=0

‖Ax‖‖x‖

= sup‖x‖=1

‖Ax‖‖x‖



Norm Properties, Distance, Orthogonal Vectors

• Properties of a norm

• Nonnegativity: ‖x‖ ≥ 0 for every x ∈ Rn

• Uniqueness of the zero-value: ‖x‖ = 0 if and only if x = 0

• Homogeneity: ‖λx‖ = |λ| ‖x‖ for any scalar λ and any x ∈ Rn

• Triangle relation: ‖x + y‖ ≤ ‖x‖+ ‖y‖ for every x and y ∈ Rn

• (Euclidean) Distance between two vectors x ∈ Rn and y ∈ Rn

‖x− y‖ =

√√√√ n∑i=1

(xi − yi)2

• Vectors x and y are orthogonal when x′y = 0

• For orthogonal vectors x and y:

Pythagorean Theorem ‖x + y‖2 = ‖x‖2 + ‖y‖2

• For any vectors x and y:

Schwartz Inequality: |x′y| ≤ ‖x‖ ‖y‖



Vector Sequence

Given a vector sequence {xk} ⊂ Rn, we say that

• The sequence is bounded when for some scalar C, we have

‖xk‖ ≤ C for all k

• The sequence converges to a vector x ∈ Rn when

limk→∞ ‖xk − x‖ = 0 (xk → x)

• Vector y is an accumulation point of the sequence when there is a

subsequence {xki} such that xki → y as i →∞

• Example: xk = (1 + (−1)ka)k with 0 < a < 1. Accum. points?

• Bolzano Theorem

A bounded sequence {xk} ⊂ Rn has at least one limit point



Set TopologyTypically denoted by capital letters C, X, Y , Z, etc.

Let X be a set in Rn.

• A vector x is an accumulation (or a limit) point of the set X when

there is a sequence {xk} ⊆ X such that xk → x

• The set X is closed when it contains all of its accumulation points

• The set X is open when its complement set Xc = {x ∈ Rn | x 6∈ X}is closed

• Rn and ∅ are the only sets that are both open and closed

• The set X is bounded when for some C: ‖x‖ ≤ C for all x ∈ X

• (Th.) The set X is compact when it is both closed and bounded

Let {Xi | i ∈ I} be a family of closed sets Xi ⊆ Rn

• The intersection ∩i∈IXi is closed

• When the set I is finite (I = {1, . . . , m}) the union ∪i∈IXi is closed



Special SetsLet X be a set in Rn. We say that X is:

• subspace when αx + βy ∈ X for every x, y ∈ X and any α, β ∈ R• Example: {x | x′e = 0} where e ∈ Rn with ei = 1 for all i

• Is a subspace a closed set? Bounded?

• affine when it is a translated subspace:

X = x + S for an x ∈ X and a subspace S

• cone when λx ∈ X for every λ ≥ 0 and every x ∈ X

• convex when αx + (1− α)y ∈ X for any x, y ∈ X and α ∈ [0,1]



Convex Sets

Is a subspace convex set? Affine set?

Special convex sets:

• (Euclidean) Ball centered at x0 ∈ Rn with a radius r > 0

{x ∈ Rn | ‖x− x0‖ ≤ r} (closed ball)

{x ∈ Rn | ‖x− x0‖ < r} (open ball)

• A closed ball is convex; An open ball is convex



More Convex Sets• Half-space with normal vector a 6= 0: {x | a′x ≤ b}• Hyperplane with normal vector a 6= 0: {x | a′x = b} (b is a scalar)

• Polyhedral set (given by finitely many linear inequalities):

{x | a′ix ≤ bi, i = 1, . . . , m} = {x | Ax ≤ b}

A is an m× n matrix and b ∈ Rm



Dual ConeLet K ⊆ Rn be a cone.

Def. A dual cone of K is the following set

K∗ = {d ∈ Rn | d′y ≥ 0 for all y ∈ K}

• Dual cone appears in formulations of complementarity problems

• Dual cone is always closed and convex (regardless of K)



Normal Cone of a SetLet X ⊆ Rn be a nonempty set, and let x ∈ X.

Def. A normal cone of the set X at the point x is the following set

N(x;X) = {y ∈ Rn | y′(x− x) ≤ 0 for all x ∈ X}Vectors in this set are called normal vectors to the set X at x

X

N(x ; X )

x

^

x

N(x ; X ) = {0}

~

~

• Normal cone plays an important role in optimality conditions

• Normal cone is always closed and convex (regardless of X)



Tangent Cone of a Set

Let X ⊆ Rn be a nonempty set, and let x ∈ X.

Def. A tangent cone of the set X at the point x is the following set

T (x;X) ={y ∈ Rn | y = lim

k→∞

xk − x

λk

with {xk} ⊂ X, {λk} ⊂ R++

s. t. xk → x, λk → 0}

Vectors in this set are called tangent vectors to the set X at x

Prop. The tangent cone T (x;X) is equivalently given by

T (x;X) =

{βy ∈ Rn | y = lim

k→∞

xk − x

‖xk − x‖with {xk} ⊂ X,

xk → x, xk 6= x, β ≥ 0}



Tangent Cone Illustration

X

x

x~

T(x ; X )^

Tangent cone T (x;X) for any set X and any x ∈ X:

• plays an important role in optimality conditions

• always closed regardless of the properties of X

• when X is convex, then T (x;X) is convex



Tangent and Normal Cone

Let X ⊂ Rn and let x ∈ X. Then, we have −N(x;X) ⊆ T (x;X)∗

X

x

x~

N(x ; X )^

T(x ; X )*^

Prop. If X is convex, then

T (x;X)∗ = −N(x;X) for all x ∈ X



Set Interior, Closure and Boundary

Let X ⊆ Rn.

• A vector x ∈ X is an interior point of X if there exist a ball B(x, r)

such that B(x, r) ⊂ X

• The interior of the set X is the set of all interior points of X; it is

denoted by int(X)

• A vector x ∈ X is a accumulation point of X if there exist a sequence

{xk} ⊂ X converging to x

• The closure of the set X is the set of all accumulation points of X; it

is denoted by cl(X)

• The boundary of the set X is the intersection of the closure set of X

and the closure set of its complement (Rn \X), i.e.,

bd(X) = cl(X) ∩ cl(Rn \X)

Example: X = {(x1, x2) ∈ R2 | 0 ≤ x1 ≤ 1, 0 < x2 < 1}



Convex and Conical HullsA convex combination of vectors x1, . . . , xm is a vector of the form

α1x1 + . . . + αmxm αi ≥ 0 for all i and∑m

i=1 αi = 1

The convex hull of a set X is the set of all convex combinations of the

vectors in X, denoted conv(X)

A conical combination of vectors x1, . . . , xm is a vector of the form

λ1x1 + . . . + λmxm with λi ≥ 0 for all i

The conical hull of a set X is the set of all conical combinations of the

vectors in X, denoted by cone(X)



Affine Hull

An affine combination of vectors x1, . . . , xm is a vector of the form

t1x1 + . . . + tmxm with∑m

i=1 ti = 1, ti ∈ R for all i

The affine hull of a set X is the set of all affine combinations of the vectors

in X, denoted aff(X)

The dimension of a set X is the dimension of the affine hull of X

dim(X) = dim(aff(X))



Relative Interior, Closure, and Boundary

Let X ⊆ Rn.

• A vector x ∈ X is a relative interior point of X if there exist a ball

B(x, r) such that B(x, r) ∩ aff(X) ⊂ X

• The relative interior of the set X is the set of all interior points of X;

it is denoted by rint(X)

• The relative boundary of the set X is the intersection of the affine

hull aff(X) and the boundary bd(X)

Example: X = {(x1, x2) ∈ R2 | 0 ≤ x1 ≤ 1, x2 = 1}

aff(X) =?



Mappings

Let X ⊆ Rn.

• We write F : X → Rm to denote a mapping F from the set X to Rm

• F is a function when F : X → R (m = 1)

• We refer to X as a domain of the mapping F

• The image of X under mapping F is the following set

F (X) = {y ∈ Rm | y = F (x) for some x ∈ X}

• The inverse image of Y ⊆ Rm under mapping F is the following set

F−1(Y ) = {x ∈ X | F (x) ∈ Y }



Special Mappings and Functions

A mapping F : X → Rm with X ⊆ Rn is

• affine when for all x ∈ X,

F (x) = Ax + b for some matrix A and a vector b ∈ Rm

Example: Space translation (by a given x0 ∈ Rn) is an affine mapping

from Rn to Rn: F (x) = x + x0 for all x ∈ Rn

• linear when for all x ∈ X, we have F (x) = Ax for a matrix A

Example: For a coordinate subspace S = {x ∈ Rn | xi = 0, for i ∈ I},where I is a subset of coordinate indices (I ⊆ {1, . . . , n}, the projection

on S is a linear mapping from Rn to Rn:

F (x) = Px with [P ]ii = 1 for i 6∈ I and Pij = 0 otherwise



Special Functions

A function f : Rn → R is

• quadratic when for all x

f(x) = x′Qx + a′x + b for a matrix Q, a ∈ Rn and b ∈ R

• affine when for all x

f(x) = a′x + b for a vector a ∈ Rn and b ∈ R

• linear when for all x

f(x) = a′x for a vector a ∈ Rn



References for this lecture

The material for this lecture:

• (B) Bertsekas D.P. Nonlinear Programming

• See Appendix there for topology in Rn and linear algebra

• (BNO) Bertsekas, Nedic, Ozdaglar Convex Analysis and Optimization

• Chapter 1 for convex sets and functions

• see also lecture slides for Convex Optimization (Lectures 2–5) at

https://netfiles.uiuc.edu/angelia/www/angelia.html

• (FP) Facchinei and Pang Finite Dimensional ..., Vol I

• Chapter 1 for Normal Cone, Dual Cone, and Tangent Cone

• NOTE!!!

• Normal cone definition used in (FP) is different than that of (B,BNO)


lecture 2 - university of illinoisangelia/ie598ns_lect2.pdf · a. nedich and u. shanbhag lecture 2...

Documents