Projection onto simplicial cones by Picard’s method

Jorge Barriosa,1, Orizon P. Ferreiraa,2,∗, Sandor Z. Nemethb,3

aIME/UFG, Campus II- Caixa Postal 131, Goiania, GO, 74001-970, BrazilbSchool of Mathematics, The University of Birmingham, The Watson Building,

Edgbaston, Birmingham B15 2TT, United Kingdom

Abstract

By using Moreau’s decomposition theorem for projecting onto cones, theproblem of projecting onto a simplicial cone is reduced to finding the uniquesolution of a nonsmooth system of equations. It is shown that Picard’smethod applied to the system of equations associated to the problem of pro-jecting onto a simplicial cone generates a sequence that converges linearly tothe solution of the system. Numerical experiments are presented making thecomparison between Picard’s and semi-smooth Newton’s methods to solvethe nonsmooth system associated with the problem of projecting a pointonto a simplicial cone.

Keywords: Projection, simplicial cones, Moreau’s decomposition theorem,Picard’s method2000 MSC: 90C33, 15A48, 90C20

1. Introduction

The interest in the subject of projection arises in several situations, havinga wide range of applications in pure and applied mathematics such as ConvexAnalysis (see e.g. [1]), Optimization (see e.g. [2, 3, 4, 5, 6, 7]), NumericalLinear Algebra (see e.g. [8]), Statistics (see e.g. [9, 10, 11]), Computer

∗Corresponding author.Email addresses: numeroj@gmail.com (Jorge Barrios), orizon@ufg.br (Orizon P.

Ferreira), nemeths@for.mat.bham.ac.uk (Sandor Z. Nemeth)1The author was supported in part by CAPES.2The author was supported in part by FAPEG, CNPq Grants 471815/2012-8,

305158/2014-7 and PRONEX–Optimization(FAPERJ/CNPq).3The author was supported in part by the Hungarian Research Grant OTKA 60480.

Preprint submitted to Linear Algebra and its Applications March 11, 2015

arX

iv:1

503.

0275

7v1

[m

ath.

OC

] 1

0 M

ar 2

015

Graphics (see e.g. [12] ) and Ordered Vector Spaces (see e.g. [13, 14, 15,16, 17, 18]). More specifically, the projection onto a polyhedral cone, whichhas as a special case the projection onto a simplicial one, is a problem ofhigh impact on scientific community4. The geometric nature of this problemmakes it particularly interesting and important in many areas of science andtechnology such as Statistics (see e.g. [11]), Computation (see e.g. [19]),Optimization (see e.g.[20, 7]) and Ordered Vector Spaces (see e.g. [16]).

The projection onto a general simplicial cone is difficult and computa-tionally expensive, this problem has been studied e.g. in [21, 22, 5, 23, 16, 7].It is a special convex quadratic program and its KKT optimality conditionsform the linear complementarity problem (LCP) associated with it, see e.g[24, 23, 7]. Therefore, the problem of projecting onto simplicial cones can besolved by active set methods [25, 26, 27, 24] or any algorithms for solvingLCPs, see e.g [25, 24] and special methods based on its geometry, see e.g[23, 24]. Other fashionable ways to solve this problem are based on the clas-sical von Neumann algorithm (see e.g. the Dykstra algorithm [28, 10, 29]).Nevertheless, these methods are also quite expensive (see the numerical re-sults in [20] and the remark preceding section 6.3 in [30]).

In this paper we particularize the Moreau’s decomposition theorem forsimplicial cones. This leads to an equivalence between the problem of pro-jecting a point onto a simplicial cone and one of finding the unique solutionof a nonsmooth system of equations. We apply Picard’s method to find aunique solution of the obtained associated system. Under a mild assumptionon the simplicial cone we show that the method generate a sequence thatconverges linearly to the solution of the associated system of equations. Nu-merical experiments are presented making the comparison between Picard’sand semi-smooth Newton’s methods for solving the nonsmooth system asso-ciated with the problem of projecting a point onto a simplicial cone.

The organization of the paper is as follows. In Section 2, some notations,basic results used in the paper and the statement of the problems that weare interested are presented, in particular, the problem of projecting ontosimplicial cone. In Section 3 we present some results about projection ontosimplicial cones. In Section 4 we present two different Picard’s iterations forsolving the problem of projecting onto simplicial cone. In Section 5 theoret-

4see the popularity of the Wikimization page Projection on Polyhedral Cone atwww.convexoptimization.com/wikimization/index.php/Special:Popularpages

2

ical and numerical comparisons between Picard’s methods and semi-smoothNewton’s method for solving the problem of projecting onto simplicial cone[31] are provided. Some final remarks are made in Section 6.

2. Preliminaries

Consider Rm endowed with an orthogonal coordinate system and let 〈·, ·〉be the canonical scalar product defined by it. Denote by ‖ · ‖ be the normgenerated by 〈·, ·〉. If a ∈ R and x = (x1, . . . , xm) ∈ Rm, then denotea+ := max{a, 0}, a− := max{−a, 0} and

x+ :=((x1)+, . . . , (xm)+

), x− :=

((x1)−, . . . , (xm)−

), |x| :=

(|x1|, . . . , |xm|

).

For x ∈ Rm, the vector sgn(x) will denote a vector with components equal to1, 0 or −1 depending on whether the corresponding component of the vectorx is positive, zero or negative. We will call a closed set K ⊂ Rm a cone ifthe following conditions hold:

(i) λx+ µy ∈ K for any λ, µ ≥ 0 and x, y ∈ K,

(ii) x,−x ∈ K implies x = 0.

Let K ⊂ Rm be a closed convex cone. The polar cone and the dual cone ofK are, respectively, the sets

K⊥:={x ∈ Rm | 〈x, y〉≤0,∀ y∈K}, K∗:={x ∈ Rm | 〈x, y〉≥0,∀ y∈K}. (1)

It is easy to see that K⊥ = −K∗. The set of all m × m real matricesis denoted by Rm×m, I denotes the m ×m identity matrix and diag(x) willdenote a diagonal matrix corresponding to elements of x.

For an M ∈ Rm×m consider the norm defined by

‖M‖ := maxx 6=0{‖Mx‖ : x ∈ Rm, ‖x‖ = 1},

this definition implies

‖Mx‖ ≤ ‖M‖‖x‖, ‖LM‖ ≤ ‖L‖‖M‖, (2)

for any m×m matrices L and M .

3

Denote Rm+ = {x = (x1, . . . , xm) ∈ Rm : x1 ≥ 0, . . . , xm ≥ 0} the nonneg-

ative orthant. Let A ∈ Rm×m be a nonsingular matrix. Then, the cone

K := ARm+ = {Ax : x = (x1, . . . , xm) ∈ Rm, x1 ≥ 0, . . . , xm ≥ 0}, (3)

is called a simplicial cone or finitely generated cone. Let z ∈ Rm, then theprojection PK(z) of the point z onto the cone K is defined by

PK(z) := argmin {‖z − y‖ : y ∈ K} .

From the definition of simplicial cone associated with the matrix A thisdefinition is equivalent to

PK(z) :=argmin

{1

2‖z − Ax‖2 : x = (x1, . . . , xm)∈Rm, x1≥ 0,. . .,≥ xm≥ 0

}.

Remark 1. It is easy to see that PRm+

(z) = z+. It is well know that theprojection onto a convex set is continuous and nonexpansive, in particular,we have ‖z+ − w+‖ ≤ ‖z − x‖ for all z, w ∈ Rm, see [1].

The above remark shows that projection onto the nonnegative orthant is aneasy problem. On the other hand, the projection onto a general simplicialcone is difficult and computationally expensive, this problem has been studiede.g. in [21, 5, 31, 7, 16, 22]. The statement of the problem that we areinterested is:

Problem 1 (projection onto a simplicial cone). Given A ∈ Rm×m anonsingular matrix and z ∈ Rm, find the projection PK(z) of the point zonto the simplicial cone K = ARm

+ .

The problem of projection onto a simplicial cone has many different formu-lations which allow us develop different techniques for solving them. In thenext remark we present some of these formulations.

Remark 2. Let A ∈ Rm×m be a nonsingular matrix and z ∈ Rm. Fromthe definition of the simplicial cone associated with the matrix A in (3),the problem of projection onto a simplicial cone K = ARm

+ may be statedequivalently as the following quadratic problem

Minimize1

2‖z − Ax‖2, subject to x ≥ 0.

4

Hence, if v ∈ Rm is the unique solution of this problem then we have PK(z) =v. The above problem is equivalent to the following nonnegative quadraticproblem

Minimize1

2x>Qx+ x>b+ c, subject to x ≥ 0, (4)

by taking Q = A>A, b = −A>z and c = z>z/2. The optimality conditionfor the problem (4) implies that its solution can be obtained by solving thefollowing linear complementarity problem

y = Qx+ b, x ≥ 0, y ≥ 0, x>y = 0. (5)

where y is a column vector of variables in Rm. It is easy to establish that cor-responding to each nonnegative quadratic problems (4) and each linear com-plementarity problems (5) associated to symmetric positive definite matrixes,there are equivalent problems of projection onto simplicial cones. Therefore,the problem of projecting onto simplicial cones can be solved by active setmethods [25, 26, 27, 24] or any algorithms for solving LCPs, see e.g [25, 24]and special methods based on its geometry, see e.g [23, 24]. Other fashionableways to solve this problem are based on the classical von Neumann algorithm(see e.g. the Dykstra algorithm [28, 10, 29]). Nevertheless, these methodsare also quite expensive (see the numerical results in [20] and the remarkpreceding section 6.3 in [30]).

As we will see in the next section, by using Moreau’s decomposition theo-rem for projecting onto cones, solving Problem 1 is reduced to solving thefollowing problem.

Problem 2 (nonsmooth equation). Given A ∈ Rm×m a nonsingular ma-trix and z ∈ Rm, find the unique solution u of the nonsmooth equation(

A>A− I)x+ + x = A>z. (6)

In this case, PK(z) = Au+ where K = ARm+ .

Since x+ = (x+ |x|)/2 the Problem 2 is equivalent to the following problem:

Problem 3 (absolute value equation). Given A ∈ Rm×m a nonsingularmatrix and z ∈ Rm, find the unique solution u of the absolute value equation(

A>A+ I)x+

(A>A− I

)|x| = 2A>z. (7)

In this case, PK(z) = Au+ where K = ARm+ .

5

We will show in Section 4 that Problem 2 and Problem 3 can be solved byusing Picard’s method. We end this section with the Banach’s fixed pointtheorem which will be used for proving our main result, its proof can befound in [32] (see Theorem 5.1− 2 pag. 300 and Corollary 5.1− 3 pag. 302).

Theorem 1 (Banach’s fixed point theorem). Let (X, d) be a non-emptycomplete metric space, 0 ≤ α < 1 and T : X → X a mapping satisfyingd(T (x), T (y)) ≤ αd(x, y), for all x, y ∈ X. Then there exists an unique x ∈ Xsuch that T (x) = x. Furthermore, x can be found as follows: start with anarbitrary element x0 ∈ X and define a sequence {xn} by xn+1 = T (xn), thenlimn→+∞ xn = x and the following inequalities hold:

d(x, xn+1) ≤ α

1− αd(xn+1, xn), d(x, xn+1) ≤ αd(x, xn), n = 0, 1, . . . .

3. Moreau’s decomposition theorem for simplicial cones

In this section we present some results about projection onto simplicialcones. We recall the following result due to Moreau [33]:

Theorem 2 (Moreau’s decomposition theorem). Let K,L ⊆ Rm be twomutually polar cones in Rm. Then, the following statements are equivalent:

(i) z = x+ y, x ∈ K, y ∈ L and 〈x, y〉 = 0,

(ii) x = PK(z) and y = PL(z).

Remark 3. Let K be a cone in Rm. Note that from Moreau’s decomposi-tion theorem, definition of the polar cone and the dual cone in (1) and therelationship K⊥ = −K∗ it follows that

PK(z) = z + PK∗(−z), ∀ z ∈ Rm.

Hence the problem of projecting onto K is equivalent to problem of projectingonto K∗.

The following result follows from the definition of the polar, see [13].

Lemma 1. Let A ∈ Rm×m be a nonsingular matrix. Then,

(ARm+ )⊥ = −(A>)−1Rm

+ .

6

The following result has been proved in [13] by using Moreau’s decompositiontheorem and Lemma 1.

Lemma 2. Let A ∈ Rm×m be a nonsingular matrix and K = ARm+ the

corresponding simplicial cone. Then, for any z ∈ Rm there exists a uniquex ∈ Rm such that the following two equivalent statements hold:

(i) z = Ax+ − (A>)−1x−, x ∈ Rm,

(ii) Ax+ = PK(z) and −(A>)−1x− = PK⊥(z).

The following result is a direct consequence of Lemma 2, it shows that solvingProblem 1 is reduced to solving Problem 2.

Lemma 3. Let A ∈ Rm×m be a nonsingular matrix, K = ARm+ the corre-

sponding simplicial cone and z ∈ Rm arbitrary. Then, equations (6) and(7) have a unique solution u and PK(z) = Au+, i.e., to solve Problem 1 isequivalent to solving either Problem 2 or Problem 3.

Proof. Since A is an m × m nonsingular matrix, multiplying by A>, theequality in item (i) of Lemma 2 is equivalently transformed into

A>Ax+ − x− = A>z.

As −x− = x−x+, the above equality is equivalent to (6). Therefore, equation(6) is equivalent to the equation in item (i) of Lemma 2. Hence, we concludefrom Lemma 2 that equation (6) has a unique solution u and PK(z) = Au+.Since the equations (6) and (7) are equivalent the result follows. 2

4. Picard’s Method

In this section we will present two different Picard’s iterations, one ofthem for solving Problem 2 and the other one for solving Problem 3.

4.1. Picard’s Method for solving Problem 2

The Picard’s method for solving Problem 2 is formally defined by

xk+1 = −(A>A− I

)x+k + A>z, k = 0, 1, 2, . . . . (8)

The sequence {xk} with starting point x0 ∈ Rm, called the Picard’s sequencefor solving Problem 2. The next theorem provides a sufficient condition forthe linear convergence of the Picard’s iteration (8).

7

Theorem 3. Let A ∈ Rm×m be a nonsingular matrix, K = ARm+ the corre-

sponding simplicial cone and z ∈ Rm arbitrary. If

‖A>A− I‖ < 1, (9)

then the Picard’s sequence {xk} for solving Problem 2 converges to the uniquesolution u of equation (6) from any starting point x0 ∈ Rm, PK(z) = Au+

and the following error bound holds

‖u− xk‖ ≤‖A>A− I‖

1− ‖A>A− I‖‖xk − xk−1‖, ∀ k = 1, 2 . . . . (10)

Moreover, the sequence {xk} converges linearly to u as follows

‖u− xk+1‖ ≤ ‖A>A− I‖‖u− xk‖, k = 0, 1, 2, . . . . (11)

Proof. Define the function F : Rm → Rm as

F (x) = −(A>A− I

)x+ + A>z. (12)

Since Remark 1 implies ‖x+ − y+‖ ≤ ‖x− y‖ for all x, y ∈ Rm, from (12) iteasy to conclude that

‖F (x)− F (y)‖ ≤ ‖A>A− I‖‖x− y‖, ∀ x, y ∈ Rm.

Therefore, as by assumption ‖A>A− I‖ < 1 we may apply Theorem 1 withX = Rm, T = F , d(x, y) = ‖y− x‖ for all x, y ∈ Rm and α = ‖A>A− I‖, forconcluding that the Picard’s Method (8) or equivalently, the sequence

xk+1 = F (xk), k = 0, 1, . . . ,

converges to a unique fixed point u of F , which from (12) is the solution ofthe Problem 2, i.e., (

A>A− I)u+ + u = A>z,

and by using Lemma 3 we have PK(z) = Au+. Moreover, Theorem 1 impliesthat the inequalities (10) and (11) hold. 2

8

4.2. Picard’s Method for solving Problem 3

The Picard’s method for solving Problem 3 is formally defined by(A>A+ I

)xk+1 = −

(A>A− I

)|xk|+ 2A>z, k = 0, 1, 2, . . . . (13)

The sequence {xk} with starting point x0 ∈ Rm, called the Picard’s sequencefor solving equation (7) or for projecting a point z ∈ Rm onto the simplicialcone K. From now on we will refer this method as Picard 2.

Since A ∈ Rm×m is a nonsingular matrix we conclude that A>A is sym-metric and positive definite. Hence, A>A + I is nonsingular. Then for sim-plifying the notations define

C :=(A>A+ I

)−1 (A>A− I

). (14)

Let λ1, . . . λm and σ1, . . . σm be the eigenvalues of A>A and C, respectively.As λi > 0, for i = 1, 2, . . .m, it easy to conclude that

‖C‖ = max {|σ1|, . . . |σm|} < 1, where σi =1− λiλi + 1

, i = 1, 2, . . .m.

The next theorem provides the convergence of the Picard’s iteration (13).

Theorem 4. Let A ∈ Rm×m be a nonsingular matrix, K = ARm+ the corre-

sponding simplicial cone and z ∈ Rm arbitrary. The Picard’s sequence {xk}for solving Problem 3 is well defined and converges to the unique solutionu of equation (7) from any starting point x0 ∈ Rm, PK(z) = Au+ and thefollowing error bound holds

‖u− xk‖ ≤‖C‖

1− ‖C‖‖xk − xk−1‖, ∀ k = 1, 2 . . . . (15)

Moreover, the sequence {xk} converges linearly to u as follows

‖u− xk+1‖ ≤ ‖C‖‖u− xk‖, k = 0, 1, 2, . . . . (16)

Proof. Since the matrix A>A+I is nonsingular, the function F : Rm → Rm,

F (x) := −(A>A+ I

)−1 (A>A− I

)|x|+ 2

(A>A+ I

)−1A>z, (17)

is well defined. Since ‖|x| − |y|‖ ≤ ‖x− y‖ for all x, y ∈ Rm, from (17) and(14) we conclude that

‖F (x)− F (y)‖ ≤ ‖C‖‖x− y‖, ∀ x, y ∈ Rm.

9

Therefore, as ‖C‖ < 1 we may apply Theorem 1 with X = Rm, T = F ,d(x, y) = ‖y − x‖ for all x, y ∈ Rm and α = ‖C‖, for concluding that thePicard’s Method (13) or equivalently, the sequence

xk+1 = F (xk), k = 0, 1, . . . ,

converges to a unique fixed point u of F , which from (17) is the solution ofthe Problem 3, i.e.,(

A>A+ I)u+

(A>A− I

)|u| = 2A>z,

and by using Lemma 3 we have PK(z) = Au+. Moreover, Theorem 1 impliesthat the inequalities (15) and (16) hold. 2

5. Comparison between Picard’s and Newton’s methods

In this section theoretical and numerical comparisons of above Picard’smethods and semi-smooth Newton’s method studied in [31] are provided.Also Picard’s method (13) is applied to solve an specific example.

5.1. Theoretic comparison

In this section theoretical comparisons between Picard’s methods andsemi-smooth Newton’s method for solving Problem 1 will be provided.

It is shown in [31] that the semi-smooth Newton method applied to equa-tion (6), namely,((

A>A− I)

diag(sgn(x+k )) + I

)xk+1 = A>z, k = 0, 1, 2, . . . , (18)

is always well defined and under the assumption

‖A>A− I‖ < b <1

3, (19)

on the matrix A defining the simplicial cone K = ARm+ , the generated se-

quence {xk} converges linearly to the unique solution u of Problem 2 fromany starting point and, as a consequence of Lemma 3 we have PK(z) = Au+

for any z ∈ Rm, which implies that u solves Problem 1.Problem 1, i.e., the problem of projecting a point z ∈ Rm onto a simpli-

cial cone K = ARm+ is equivalent, by Lemma 3, to solving either Problem 2

10

or Problem 3. Note that solving Problems 2 by Picard’s method (8) assump-tion (9) on the matrix A (see Theorem 3) is less restrictive than assumption(19). When solving Problem 3 we only need the invertibility of the matrix Afor Picard’s method (13) to converge (see Theorem 4). Therefore, Picard’smethod (13) is theoretically more robust than Picard’s method (8) and con-sequently than semi-smooth Newton method (18). In the next section we willpresent an example, where according to the established theory, only Picard’smethod (13) can be applied.

The main drawbacks of Picard (13) and semi-smooth Newton (18) is thatboth require the solution of a linear system in each iteration which constitutethe largest computational effort of these methods. Picard’s method (8) donot have to solve a linear system, avoiding more complicated calculations,which is particularly interesting for large scale problems. We will investigatethe efficiency of these methods in section 5.2.1.

5.1.1. Example

Consider the monotone nonnegative cone, which is a simplicial cone Kdefined by

K :={x = (x1, . . . , xm) ∈ Rm, x1 ≥ x2 ≥ · · · ≥ xm ≥ 0

}. (20)

The monotone nonnegative cone and the projection onto it occurs in variousimportant practical problems such as the problem of map-making from rela-tive distance information e.g., stellar cartography (see web page5 and Section5.13.2 in [34]) and isotonic regression [35, 36, 37, 38]. The isotonic regres-sion [39, 40, 41, 42] is a very important topic in statistics with hundreds ofpapers and several books dedicated to this topic. This section provides adifferent view about projecting onto the monotone nonnegative cones via Pi-card’s method (13) which is related to the iterative theory of bidiagonal andtridiagonal matrices, and the Fibonacci numbers. The dual of the monotone

5 www.convexoptimization.com/wikimization/index.php/Projection_on_Polyhedral_Convex_Cone

11

nonnegative cone is K∗ = ARm+ , where A ∈ Rm×m is the nonsingular matrix

A =

1−1 1

−1 1. . . . . .

−1 1

, A>A =

2 −1−1 2 −1

−1. . . . . .. . . 2 −1−1 1

.

From Remark 3, the problem of projecting a point onto K∗ is equivalent toprojecting onto K. Let λ1, . . . λm be the eigenvalues of A>A. From [43] wehave that the eigenvalues of matrix A>A are given by

λi = 2 + 2 cos

(2iπ

2m+ 1

), i = 1, 2, . . . ,m. (21)

Hence from (21) we conclude that

0 < λi < 4, limm→∞

λm = 0, limm→∞

λ1 = 4, limm→∞

‖A>A− I‖ = 3.

Since ‖A>A − I‖ > 1 for all m ≥ 2, for projecting a point onto the coneK∗, we can not apply semi-smooth Newton method studied in [31] neitherPicard’s iteration (8). However Picard’s method (13) can be used. In orderto reduce the computational cost of this method, for solving the linear systeminvolved in each iteration, we suggest the following triangular decomposition

A>A+ I =

d1 −1

d2 −1. . . . . .

dm−1 −1dm

1

− 1d2

1

− 1d3

. . .

. . . 1− 1dm

1

,

where dm = 2, di = 3− 1/di+1 for i = m− 1, · · · , 1. Another alternative forsolving the linear system would be to compute the matrices

R =(A>A+ I

)−1 (A>A− I

), S =

(A>A+ I

)−1A>.

By using the recursion formulas for a tridiagonal matrix from [44], which arebased on the results of [45, 46, 47], after some algebraic manipulations and

12

taking into account that R is symmetric we obtain

Rij =

−2F2iF2m−2j+1

F2m+1

if 1 < i < j < m,

F2iF2m−2i − F2i−2F2m−2i+1

F2m+1

if 1 < i = j < m,

2F2m−2j+1

F2m+1

if 1 = i < j < m,

− 2F2i

F2m+1

if 1 < i < j = m,

− 2

F2m+1

if i = 1, j = m,

F2m−2

F2m+1

if i = j = 1,m.

Sij =

−F2iF2m−2j+2

F2m+1

if i < j,

F2j−1F2m−2i+1

F2m+1

if 1 < j ≤ i,

F2m−2i+1

F2m+1

if 1 = j ≤ i.

where Fi is the Fibonacci sequence defined by F0 = 0, F1 = 1 and Fi+2 =Fi + Fi+1.

5.2. Computational results

In this section we present two numerical experiments. In the first, nu-merical comparisons between Picard’s methods (8), (13) and semi-smoothNewton’s method (18) for solving Problem 1 will be provided. In the sec-ond one, we study the behavior Picard’s method (13) solving the problemdescribed in Section 5.1.1. All programs were implemented in MATLAB Ver-sion 7.11 64-bit and run on a 3.40GHz Intel Core i5 − 4670 with 8.0GB of

13

RAM. All MATLAB codes and generated data of this paper are available inhttp://orizon.mat.ufg.br/pages/34449-publications.

General considerations:

• In order to accurately measure the method’s runtime for a problem,each of them was solved 10 times and the runtime data collected. Then,we defined the corresponding method’s runtime for a problem as themedian of these measurements.

• We consider that the method converged to the solution and stoppedthe execution when, for some k, the condition

‖u− xk‖‖u‖

< RelativeTolerance,

is satisfied.

5.2.1. Numerical experiment I

In this experiment, we study the percentage of problems for which amethod was the fastest one (efficiency) to compare them. With the aimthat methods (8), (13) and (18) find solutions on 1000 generated randomtest problems of dimension m = 1000, we construct the matrix A (definingthe simplicial cone K = ARm

+ ) in each problem satisfying the condition (19).We assume that a method is the fastest one for a problem, if the cor-

responding runtime is less than or equal to 1.01 times the best time of allmethods to find the solution.

Each test problem was generated as follows:

(i) To construct the matrix A ∈ Rm×m satisfying the condition (19), wefirst chose a random number b from the standard uniform distributionon the open interval (0, 1/3). Then, we chose a random number bfrom the standard uniform distribution on the open interval (0, b). Wecomputed the matrices S, V andD, respectively, from the singular valuedecomposition of a m × m generated real matrix containing randomvalues drawn from the uniform distribution on the interval [−106, 106].Finally we computed

A = S

(sqrt

(I +

b

νV

))D,

were ν is the largest singular value of V and sqrt(I + bνV ) is the square

root of the diagonal matrix I + bνV .

14

(ii) We chose the solution u ∈ Rm containing random values drawn from theuniform distribution on the interval [−106, 106] and computed z ∈ Rm

from equation (6). Finally we chose a starting point x0 ∈ Rm containingrandom values drawn from the uniform distribution on the interval[−106, 106].

In order to provide information for the analysis of the large test problemsset considered, we use the performance profiles (see [48]). The performanceprofile for a method is the cumulative distribution function for a performancemetric. In this case we use the ratio of the method’s runtime versus the bestruntime of all of the methods as the performance metric. Efficiency can bechecked in the value of the profile function at 1.

Figure 1 shows the performance profiles of the three methods for differentrelative tolerance values. These graphs reveal that Picard’s method (8) wasthe most efficient for low and medium accuracy, while semi-smooth Newton’smethod (18) was the most efficient for high accuracy requirements. However,since semi-smooth Newton’s method (18) requires at each step the solutionof a system of linear equations, which may become unreasonably expensivecomputationally as the problem dimension increases, these results suggestthat for large scale problems Picard’s method (8) is recommended.

(a) RelativeTolerance=10−7 (b) RelativeTolerance=10−10 (c) RelativeTolerance=10−13

Figure 1: Performance profiles on [1,4] for different accuracies. Picard, Picard2 and ss-Newton denotes the methods (8), (13) and (18), respectively.

On the other hand, Picard’s method (13) was always the worst, except inthe low accuracy case. It can be inferred from Figure 2, where convergencemean time for each problem consumed by Picard’s method (13) is less thanconsumed by semi-smooth Newton’s method (18).

15

Figure 2 shows, as one would expect, the number of iterations on semi-smooth Newton method is less than Picard’s methods (8) and (13) for solvingthe same set of problems and only for certain tolerance semi-smooth Newtonmethod consumes less time.

Figure 2: Total overall iterations and total time in seconds, performed and consumed, re-spectively by each method to solve the 1000 test problems for different accuracies. Picard,Picard2 and ssNewton denotes the methods (8), (13) and (18), respectively.

5.2.2. Numerical experiment II

In this experiment, we study the behavior of Picard’s method (13) solvingthe problem described in Section 5.1.1 on sets of 100 generated random testproblems of dimension m = 100, 500, 1000, 1500, 2000, respectively.

Each m−dimensional test problem was generated as follows: We con-structed the matrix A (defining the simplicial cone K∗ = ARm

+ ) as is definedin Section 5.1.1. We chose the solution u ∈ Rm, computed z ∈ Rm and chosea starting point x0 ∈ Rm as we described in the previous Section 5.2.1.

The computational results obtained are reported in Table 1. From these,it can be noted that for the same dimension, to achieve higher accuracy, themethod needs to perform a greater number of iterations and consequentlyconsume more runtime. The same behavior occurs when, for the same accu-racy, the dimension of the problem increases.

16

Dimension m Total Iterations Total Time

100 4927 7475 10036 1.096 1.624 2.180

500 6613 10333 14055 66.183 103.411 140.812

1000 8120 12873 17640 449.507 717.310 984.274

1500 8159 12924 17732 1358.698 2151.743 2952.247

2000 8814 14054 19359 3098.215 4955.041 6820.121

Relative Tolerance 10−7 10−10 10−13 10−7 10−10 10−13

Table 1: Total overall iterations and total time in seconds, performed and consumed,respectively by Picard’s method (13) to solve the 100 test problems of each dimension fordifferent accuracies.

6. Conclusions

In this paper we studied the problem of projection onto a simplicial conewhich, via Moreau’s decomposition theorem for projecting onto cones, is re-duced to finding the unique solution of a nonsmooth system of equations.Our main results show that, under a mild assumption on the simplicial cone,we can apply Picard’s method for finding a unique solution of the obtainedassociated system and that the generated sequence converges linearly to thesolution for any starting point. Note that in Theorem 4 we do not make anyassumption on the simplicial cone, on the other hand, we have to solve alinear equation in each iteration. It would be interesting to see whether theused technique can be applied for finding the projection onto more generalcones. As has been shown in [7], the problem of projection onto a simplicialcone is reduced to a certain type of linear complementarity problem (LCP).Numerical comparisons between Picard’s methods (8,13) and semi-smoothNewton’s method (18) for solving Problem 1 was provided in Section 5. Itwould also be interesting to compare these methods with the methods pro-posed in [22, 20, 7] and the Lemke’s method for LCPs.

References

References

[1] J.-B. Hiriart-Urruty, C. Lemarechal, Convex analysis and minimizationalgorithms: Fundamentals. I, Vol. 305 of Grundlehren der Mathema-

17

tischen Wissenschaften [Fundamental Principles of Mathematical Sci-ences], Springer-Verlag, Berlin, 1993.

[2] H. H. Bauschke, J. M. Borwein, On projection algorithms for solvingconvex feasibility problems, SIAM Rev. 38 (3) (1996) 367–426. doi:

10.1137/S0036144593251710.URL http://dx.doi.org/10.1137/S0036144593251710

[3] Y. Censor, T. Elfving, G. T. Herman, T. Nikazad, On diagonally relaxedorthogonal projection methods, SIAM J. Sci. Comput. 30 (1) (2007/08)473–504. doi:10.1137/050639399.URL http://dx.doi.org/10.1137/050639399

[4] Y. Censor, D. Gordon, R. Gordon, Component averaging: an efficientiterative parallel algorithm for large and sparse unstructured problems,Parallel Comput. 27 (6) (2001) 777–808. doi:10.1016/S0167-8191(00)00100-9.URL http://dx.doi.org/10.1016/S0167-8191(00)00100-9

[5] H. Frick, Computing projections into cones generated by a matrix, Bio-metrical J. 39 (8) (1997) 975–987. doi:10.1002/bimj.4710390808.URL http://dx.doi.org/10.1002/bimj.4710390808

[6] H. D. Scolnik, N. Echebest, M. T. Guardarucci, M. C. Vacchino, In-complete oblique projections for solving large inconsistent linear sys-tems, Math. Program. 111 (1-2, Ser. B) (2008) 273–300. doi:10.1007/s10107-006-0066-4.URL http://dx.doi.org/10.1007/s10107-006-0066-4

[7] M. Ujvari, On the projection onto a finitely gener-ated cone (2007). arXiv:PreprintWP2007-5,MTASZTAKI,

LaboratoryofOperationsResearchandDecisionSystems,Budapest.

[8] G. W. Stewart, On the perturbation of pseudo-inverses, projections andlinear least squares problems, SIAM Rev. 19 (4) (1977) 634–662.

[9] R. Berk, R. Marcus, Dual cones, dual norms, and simultaneous inferencefor partially ordered means, J. Amer. Statist. Assoc. 91 (433) (1996)318–328.

18

[10] R. L. Dykstra, An algorithm for restricted least squares regression, J.Amer. Statist. Assoc. 78 (384) (1983) 837–842.URL http://links.jstor.org/sici?sici=0162-1459(198312)78:

384<837:AAFRLS>2.0.CO;2-D&origin=MSN

[11] X. Hu, An exact algorithm for projection onto a polyhedral cone, Aust.N. Z. J. Stat. 40 (2) (1998) 165–170. doi:10.1111/1467-842X.00018.URL http://dx.doi.org/10.1111/1467-842X.00018

[12] J. D. Foley, A. van Dam, S. K. Feiner, J. F. Hughes, Computer Graphics:Principles and Practice, Addison-Wesley systems programming series,1990.

[13] M. Abbas, S. Z. Nemeth, Solving nonlinear complementarity problemsby isotonicity of the metric projection, J. Math. Anal. Appl. 386 (2)(2012) 882–893.

[14] G. Isac, A. B. Nemeth, Monotonicity of metric projections onto positivecones of ordered Euclidean spaces, Arch. Math. (Basel) 46 (6) (1986)568–576. doi:10.1007/BF01195027.URL http://dx.doi.org/10.1007/BF01195027

[15] G. Isac, A. B. Nemeth, Isotone projection cones in Euclidean spaces,Ann. Sci. Math. Quebec 16 (1) (1992) 35–52.

[16] A. B. Nemeth, S. Z. Nemeth, How to project onto an isotone projectioncone, Linear Algebra Appl. 433 (1) (2010) 41–51.

[17] S. Z. Nemeth, Characterization of latticial cones in Hilbert spaces byisotonicity and generalized infimum, Acta Math. Hungar. 127 (4) (2010)376–390. doi:10.1007/s10474-009-9145-3.

[18] S. Z. Nemeth, Isotone retraction cones in Hilbert spaces, Nonlinear Anal.73 (2) (2010) 495–499. doi:10.1016/j.na.2010.03.044.

[19] T. Huynh, C. Lassez, J.-L. Lassez, Practical issues on the projection ofpolyhedral sets, Ann. Math. Artificial Intelligence 6 (4) (1992) 295–315,artificial intelligence and mathematics, II. doi:10.1007/BF01535523.URL http://dx.doi.org/10.1007/BF01535523

19

[20] P. M. Morillas, Dykstra’s algorithm with strategies for projecting ontocertain polyhedral cones, Appl. Math. Comput. 167 (1) (2005) 635–649.doi:10.1016/j.amc.2004.06.136.URL http://dx.doi.org/10.1016/j.amc.2004.06.136

[21] K. S. Al-Sultan, K. G. Murty, Exterior point algorithms for nearestpoints and convex quadratic programs, Math. Programming 57 (2, Ser.B) (1992) 145–161. doi:10.1007/BF01581078.URL http://dx.doi.org/10.1007/BF01581078

[22] A. Ekart, A. B. Nemeth, S. Z. Nemeth, Rapid heuristic projection onsimplicial cones (2010). arXiv:arXiv:1001.1928.

[23] K. G. Murty, Y. Fathi, A critical index algorithm for nearest pointproblems on simplicial cones, Math. Programming 23 (2) (1982) 206–215. doi:10.1007/BF01583789.URL http://dx.doi.org/10.1007/BF01583789

[24] K. G. Murty, Linear complementarity, linear and nonlinear program-ming, Vol. 3 of Sigma Series in Applied Mathematics, Heldermann Ver-lag, Berlin, 1988.

[25] M. S. Bazaraa, H. D. Sherali, C. M. Shetty, Nonlinear programming, 3rdEdition, Wiley-Interscience [John Wiley & Sons], Hoboken, NJ, 2006,theory and algorithms. doi:10.1002/0471787779.URL http://dx.doi.org/10.1002/0471787779

[26] Z. Liu, Y. Fathi, An active index algorithm for the nearest point problemin a polyhedral cone, Comput. Optim. Appl. 49 (3) (2011) 435–456.doi:10.1007/s10589-009-9303-0.URL http://dx.doi.org/10.1007/s10589-009-9303-0

[27] Z. Liu, Y. Fathi, The nearest point problem in a polyhedral set andits extensions, Comput. Optim. Appl. 53 (1) (2012) 115–130. doi:10.

1007/s10589-011-9448-5.URL http://dx.doi.org/10.1007/s10589-011-9448-5

[28] F. Deutsch, H. Hundal, The rate of convergence of Dykstra’s cyclicprojections algorithm: the polyhedral case, Numer. Funct. Anal. Optim.15 (5-6) (1994) 537–565. doi:10.1080/01630569408816580.URL http://dx.doi.org/10.1080/01630569408816580

20

[29] S. Xu, Estimation of the convergence rate of Dykstra’s cyclic projectionsalgorithm in polyhedral case, Acta Math. Appl. Sinica (English Ser.)16 (2) (2000) 217–220. doi:10.1007/BF02677683.URL http://dx.doi.org/10.1007/BF02677683

[30] M. Tan, G.-L. Tian, H.-B. Fang, K. W. Ng, A fast EM algorithm forquadratic optimization subject to convex constraints, Statist. Sinica17 (3) (2007) 945–964.

[31] O. P. Ferreira, S. Z. Nemeth, Projection onto simplicialcones by a semi-smooth Newton method, Optimization Lettershttp://dx.doi.org/10.1007/s11590-014-0775-1.

[32] E. Kreyszig, Introductory functional analysis with applications, JohnWiley & Sons, New York-London-Sydney, 1978.

[33] J. J. Moreau, Decomposition orthogonale d’un espace hilbertien selondeux cones mutuellement polaires, C. R. Acad. Sci. 255 (1962) 238–240.

[34] J. Dattorro, Convex Optimization and Euclidean Distance Geometry,Mεβoo publishing, v2014.04.08, 2014.

[35] S. J. Grotzinger, C. Witzgall, Projections onto order simplexes, Appl.Math. Optim. 12 (3) (1984) 247–270. doi:10.1007/BF01449044.URL http://dx.doi.org/10.1007/BF01449044

[36] O. Burdakov, I. Kapyrin, Y. Vassilevski, Monotonicity recovering andaccuracy preserving optimization methods for postprocessing finite el-ement solutions, J. Comput. Phys. 231 (8) (2012) 3126–3142. doi:

10.1016/j.jcp.2011.12.041.URL http://dx.doi.org/10.1016/j.jcp.2011.12.041

[37] A. B. Nemeth, S. Z. Nemeth, How to project onto the monotone non-negative cone using pool adjacent violators type algorithms (2012).arXiv:arXiv:1201.2343.

[38] A. Guyader, N. Jegou, A. B. Nemeth, S. Z. Nemeth, A geometrical ap-proach to iterative isotone regression, Appl. Math. Comput. 227 (2014)359–369. doi:10.1016/j.amc.2013.11.048.URL http://dx.doi.org/10.1016/j.amc.2013.11.048

21

[39] R. E. Barlow, D. J. Bartholomew, J. M. Bremner, H. D. Brunk, Sta-tistical inference under order restrictions. The theory and applicationof isotonic regression, John Wiley & Sons, London-New York-Sydney,1972, wiley Series in Probability and Mathematical Statistics.

[40] T. Robertson, F. T. Wright, R. L. Dykstra, Order restricted statis-tical inference, Wiley Series in Probability and Mathematical Statis-tics: Probability and Mathematical Statistics, John Wiley & Sons, Ltd.,Chichester, 1988.

[41] M. J. Best, N. Chakravarti, Active set algorithms for isotonic regression;an unifying framework, Math. Programming 47 (1990) 425–439.

[42] P. S. Puri, H. Singh, On recursive formulas for isotonic regression usefulfor statistical inference under order restrictions, J. Statist. Plann. Infer-ence 24 (1) (1990) 1–11. doi:10.1016/0378-3758(90)90012-J.URL http://dx.doi.org/10.1016/0378-3758(90)90012-J

[43] A. R. Willms, Analytic results for the eigenvalues of certain tridiagonalmatrices, SIAM J. Matrix Anal. Appl. 30 (2) (2008) 639–656. doi:

10.1137/070695411.URL http://dx.doi.org/10.1137/070695411

[44] C. M. da Fonseca, On the eigenvalues of some tridiagonal matrices, J.Comput. Appl. Math. 200 (1) (2007) 283–286. doi:10.1016/j.cam.

2005.08.047.URL http://dx.doi.org/10.1016/j.cam.2005.08.047

[45] C. F. Fischer, R. A. Usmani, Properties of some tridiagonal matrices andtheir application to boundary value problems, SIAM J. Numer. Anal. 6(1969) 127–142.

[46] R. A. Usmani, Inversion of a tridiagonal Jacobi matrix, in: Proceedingsof the 3rd ILAS Conference (Pensacola, FL, 1993), Vol. 212/213, 1994,pp. 413–414. doi:10.1016/0024-3795(94)90414-6.URL http://dx.doi.org/10.1016/0024-3795(94)90414-6

[47] R. A. Usmani, Inversion of Jacobi’s tridiagonal matrix, Comput. Math.Appl. 27 (8) (1994) 59–66. doi:10.1016/0898-1221(94)90066-3.URL http://dx.doi.org/10.1016/0898-1221(94)90066-3

22

[48] E. D. Dolan, J. J. More, Benchmarking optimization software with per-formance profiles, Math. Programming 91 (2) (2002) 201–213. doi:

10.1007/s101070100263.URL http://dx.doi.org/10.1007/s101070100263

23