globally convergent jacobian smoothing inexact newton methods for ncp

Comput Optim Appl (2008) 41: 243–261DOI 10.1007/s10589-007-9104-2

Globally convergent Jacobian smoothing inexactNewton methods for NCP

Nataša Krejic · Sanja Rapajic

Received: 3 August 2006 / Revised: 16 October 2006 / Published online: 31 October 2007© Springer Science+Business Media, LLC 2007

Abstract A new smoothing algorithm for the solution of nonlinear complementarityproblems (NCP) is introduced in this paper. It is based on semismooth equation refor-mulation of NCP by Fischer–Burmeister function and its related smooth approxima-tion. In each iteration the corresponding linear system is solved only approximately.Since inexact directions are not necessarily descent, a nonmonotone technique is usedfor globalization procedure. Numerical results are also presented.

Keywords Nonlinear complementarity problems · Semismooth systems ·Modification of Newton method

1 Introduction

Nonlinear complementarity problems arise in many mathematical models from econ-omy, engineering, technology and optimization theory. One comprehensive reviewof the main models is presented in [13]. Such problems are usually solved applyingiterative methods to equivalent nonsmooth systems of nonlinear equations. Variousmethods for solving NCP have been developed. Many of them are based on general-ized derivatives and Newton method for nonlinear systems.

Reformulation of NCP leads to a system of nonlinear equations. The correspond-ing mapping is nonsmooth, more precisely semismooth and thus it is possible to de-velop generalized Newton method and its modifications, based on some generalizedJacobian. Different approaches are presented in the literature, see [6, 7, 9, 11, 12, 19,22, 24]. As in the classical smooth case, each iteration consists of finding a solution

Research supported by Ministry of Science, Republic of Serbia, grant No. 144006.

N. Krejic (�) · S. RapajicDepartment of Mathematics and Informatics, University of Novi Sad, Trg Dositeja Obradovica 4,21000 Novi Sad, Serbiae-mail: [email protected]

mailto:[email protected]

244 N. Krejic, S. Rapajic

of linear system which may be cumbersome if one is solving a large-scale problem.The inexact approach is one way to overcome this difficulty. The main idea is to solvethe linear system only approximately. The accuracy level of approximate solution iscontrolled by the so-called forcing parameter which links the norm of residual vectorto the norm of mapping at the current iterate. The sequence of forcing parametersalso influences the rate of convergence.

The local convergence of inexact Newton method in the smooth case is provedin [8] with forcing parameters bounded by any t < 1. The same result is obtainedin [26] with the affine-invariant (Ypma’s) condition. This criterion links the normof residual vector with the norm of mapping, but in a norm defined by the Jacobianat the solution. It has the advantage of being affine-invariant in the range space, butthe weight matrix that defines the norm complicates its practical usage unless someadditional properties are known.

Generalizations of classical inexact Newton method for solving nonsmooth equa-tions are introduced in many papers [7, 9, 11, 17, 19, 22]. The local convergence fornonsmooth problems is proved in [22] but for some t < 1 in the case of Dembo–Eisenstat–Steighaugh (DES) forcing parameters, while the better result is obtainedin the case of Ypma’s criterion, allowing forcing parameters to any t < 1. The samekind of result for block-angular semismooth problems is reported in [19]. For a lo-cally Lipschitzian but nonsmooth mapping the limit of forcing parameters t can notbe extended to 1 as in the smooth case, see [22]. One globalization procedure for thesmooth case is given in [10], and the nonsmooth case is considered in [7, 9, 11]. Glob-ally convergent inexact Newton method for nonsmooth equations proposed in [9] isa generalization of the method for smooth equations from [10], obtained by requir-ing the inexact Newton condition with a sufficient function decreasing. Some vari-ants of the generalized inexact Newton methods for complementarity problems arepresented in [7], with global convergence results and numerical experiments. Glob-alization strategy in [7] is based on general line-search for minimizing related meritfunction.

As we mentioned before, a large class of iterative methods for solving semismoothsystems is developed in recent years. The first class, known as nonsmooth methodsuses some kind of generalized Jacobian (see [6, 7, 9, 11, 17, 19, 22–24]). The secondone are smoothing methods where the nonsmooth function is replaced by a smoothoperator and a sequence of smooth problems is solved (see [3, 16]). The third class,in which we are interested here, is the class of Jacobian smoothing methods. Suchmethods try to solve the mixed Newton equation which combines the original semi-smooth function with the Jacobian of a smooth operator. The smoothing procedureis governed by a sequence of smoothing parameters that converges to zero. This wayeach iteration requires a solution of linear system determined by uniquely definedsmooth Jacobian. The name—Jacobian smoothing comes from Kanzow and Pieper[18]. Jacobian smoothing methods for NCP, first introduced in [4], successfully over-come the difficulties due to semismoothness of the mapping. Globally convergentJacobian smoothing Newton method with a line-search is presented in [4]. A similarJacobian smoothing algorithm, applicable to any general complementarity problemis defined in [18] and its global convergence is proved. An inexact Newton methodfor semismooth equations, as a modification of generalized Newton method is also

Globally convergent Jacobian smoothing inexact Newton 245

introduced by Kanzow [17]. The main objective of that paper was to show that themethod may be applied quite successfully to large-scale problems. Numerical testspointed fairly good results and practical efficiency of proposed method, specially forlarge-scale problems with up to million variables.

Global convergence of methods for solving nonlinear systems is usually obtainedby minimizing a merit function. If a search direction is descent, monotone techniquesare recommended. In this paper we describe a globally convergent Jacobian smooth-ing inexact Newton method for NCP. The inexact direction is not in general descentdirection, so a sufficient reduction of the merit function is not always possible usingmonotone strategy. This is the reason why we use a nonmonotone technique for glob-alization, similar to the one given in [20] for the semismooth case, and the one givenin [1] for the smooth case.

This paper is organized as follows. In Sect. 2 we give an overview of basic prop-erties of the smooth approximation of NCP based on Fischer–Burmeister function.The algorithm is formulated in Sect. 3. Convergence results are analyzed in Sect. 4.Numerical experiments are presented in Sect. 5, comparing our method with inexactmethods for NCP from [7].

2 Preliminaries

A few words about notation. For continuously differentiable mapping F : Rn → Rn

the Jacobian of F at x is denoted by F ′(x), whereas for smooth mapping Fi : Rn → R

we denote by ∇Fi(x) the gradient of Fi at x. For a given matrix A ∈ Rn,n and anonempty set of matrices A ∈ Rn,n, the distance between A and A is denoted bydist(A, A) = infB∈A ‖A − B‖. Landau symbols o(·) and O(·) are defined in usualway.

The nonlinear complementarity problem (NCP) consists of finding a vector x ∈ Rn

such that

x ≥ 0, F (x) ≥ 0, x�F(x) = 0,

where F : Rn → Rn is a smooth mapping, F(x) = (F1(x),F2(x), . . . ,Fn(x))�. Theequivalent semismooth nonlinear system given in [14] is

�(x) = 0, � : Rn → Rn, (1)

�(x) = (φ(x1,F1(x)),φ(x2,F2(x)), . . . , φ(xn,Fn(x))

)�,

with

φ(a, b) =√

a2 + b2 − a − b.

The corresponding smoothing problem was introduced by Kanzow [16] and is definedfor a parameter μ > 0 as

�μ(x) = 0, �μ : Rn → Rn,

�μ(x) = (φμ(x1,F1(x)),φμ(x2,F2(x)), . . . , φμ(xn,Fn(x))

)�


where

φμ(a, b) =√

a2 + b2 + 2μ − a − b.

Obviously φμ(a, b) is a smooth mapping in R2 and therefore �μ : Rn → Rn issmooth for μ > 0. The properties of such smoothing are analyzed in [18] in detail.We will cite some of the results which are necessary for our convergence analysis.

The C-subdifferential of � at x is defined by

∂C�(x) = ∂�1(x) × ∂�2(x) × · · · × ∂�n(x), �i(x) = φ(xi,Fi(x)),

where ∂�i(x) is a generalized derivative in the sense of Clark [5]. As it is well known,

∂C�(x) = Da(x) + Db(x)F ′(x),

where Da(x) = diag(a1(x), . . . , an(x)), Db(x) = diag(b1(x), . . . , bn(x)) are diago-nal matrices with elements

ai(x) = xi√x2i + F 2

i (x)

− 1, bi(x) = Fi(x)√

x2i + F 2

i (x)

− 1,

when (xi,Fi(x)) = (0,0) and

ai(x) = ξi − 1, bi(x) = ρi − 1, (ξi , ρi) ∈ R2, ‖(ξi, ρi)‖ ≤ 1,

for (xi,Fi(x)) = (0,0).

The main properties of � and �μ which will be used in this paper are given in thefollowing statements.

Lemma 1 [18] The function �μ satisfies the inequality

‖�μ1(x) − �μ2(x)‖ ≤ √2n

∣∣√μ1 − √μ2

∣∣

for all x ∈ Rn and μ1,μ2 ≥ 0. In particular,

‖�μ(x) − �(x)‖ ≤ √2nμ

holds for all x ∈ Rn and μ ≥ 0.

Lemma 2 [18] Assume that {xk} ⊆ Rn is any convergent sequence with limit pointx∗ ∈ Rn. If function � is semismooth then

∥∥�(xk) − �(x∗) − Hk(xk − x∗)

∥∥ = o(‖xk − x∗‖)

holds for any Hk ∈ ∂C�(xk).

Lemma 3 [12] Let � : Rn → Rn be a semismooth function and let x∗ ∈ Rn be asolution of �(x) = 0 such that all elements of ∂�(x∗) are nonsingular. Suppose


that two sequences {xk} and {dk} are given in such a way that {xk} → x∗ and‖xk + dk − x∗‖ = o(‖xk − x∗‖). Then

‖�(xk + dk)‖ = o(‖�(xk)‖).

Lemma 4 [23] Suppose that x∗ is the solution of (1). If � is semismooth at x∗ andall elements of ∂B�(x∗) are nonsingular, then there exists a neighborhood of x∗ suchthat x∗ is the unique solution of (1) in it.

Let us denote

�0(x) = limμ→0

�′μ(x).

It is shown in [18] that

limμ→0

dist(�′

μ(x), ∂C�(x)) = 0

holds for any x ∈ Rn and therefore �0(x) ∈ ∂C�(x), so the function �μ has theJacobian consistency property. This property is closely related with directional dif-ferentiability. The smoothing procedure is governed by a sequence of smoothing pa-rameters {μk}. One definition of the threshold value for the smoothing parameter isgiven in [18].

We also mention P0 and uniform P -functions that will be used later.

Definition 1 A function F : Rn → Rn is a:

• P0-function if for every x, y ∈ Rn, with x = y, there is an index i such that

xi = yi, (xi − yi)(Fi(x) − Fi(y)) ≥ 0;• P -function if for every x, y ∈ Rn, with x = y, there is an index i such that

(xi − yi)(Fi(x) − Fi(y)) > 0;• uniform P -function if there exists a positive constant c such that for every

x, y ∈ Rn, there is an index i such that

(xi − yi)(Fi(x) − Fi(y)) ≥ c‖y − x‖2.

Lemma 5 [20] If F from NCP is a P0-function then the Jacobian �′μ(x) is nonsin-

gular matrix for every μ > 0 and any x ∈ Rn.

3 The algorithm

We describe an inexact Newton methods for NCP which belong to the class of Ja-cobian smoothing methods using the reformulation of NCP based on function � andits smooth approximation �μ. The mixed Newton equation is solved approximately


in every iteration, so we call this algorithm Jacobian Smoothing Inexact Newtonmethod. The globalization strategy is based on the natural merit function � : Rn → R

where

�(x) = 1

2‖�(x)‖2 (2)

and its related function �μ : Rn → R,

�μ(x) = 1

2‖�μ(x)‖2.

If NCP is solvable then solving NCP is equivalent to finding a global minimum of (2).The main idea of all global algorithms is to solve NCP by minimizing merit function� , which is continuously differentiable even if � is semismooth.

Lemma 6 [18] The merit function � is continuously differentiable with ∇�(x) =HT �(x) for an arbitrary H ∈ ∂C�(x).

Lemma 7 [12] Suppose that F from NCP is a P0-function. Then every stationarypoint of � is a global minimizer of � .

Lemma 8 [12] Suppose that F from NCP is an uniform P -function. Then the levelsets of �

L(α) = {x ∈ Rn, �(x) ≤ α}are bounded.

We can now state the algorithm as follows.

Algorithm JSIN: Jacobian Smoothing Inexact Newton method

S0: Choose σ,α, ξ ∈ (0,1), 0 < τmin < τmax < 1, γ > 0, t, θ ∈ [0,1), such that t <1−α1+α

−σ(1 − θ)(1 +α), ε ≥ 0 and x0 ∈ Rn. Let {ηk} > 0 be a sequence such that∑∞

k=0 ηk < ∞ and {tk}, 0 ≤ tk ≤ t . Set β0 = ‖�(x0)‖, μ0 = (αβ0

2√

2n)2 and k = 0.

S1: If ‖�(xk)‖ ≤ ε STOP.S2: Compute dk from the inexact mixed Newton equation

�′μk

(xk)dk = −�(xk) + rk, (3)

where

‖rk‖ ≤ tk‖�(xk)‖.S3: Set α = 1. If

�μk(xk + αdk) ≤ (1 + ασ (θ − 1))2�μk

(xk) + ηk, (4)

set αk = α and xk+1 = xk + αkdk .

If (4) is not satisfied choose αnew ∈ [ατmin, ατmax], set α = αnew and repeat (4).


S4: If

‖�(xk+1)‖ ≤ max

{ξβk,

1

α‖�(xk+1) − �μk

(xk+1)‖}

(5)

then

βk+1 = ‖�(xk+1)‖and choose μk+1 such that

0 < μk+1 ≤ min

{(αβk+1

2√

2n

)2

,μk

4,

μ2k

‖�μk(xk+1)‖2

, μ(xk+1, γβk+1)

}. (6)

If (5) doesn’t hold then

βk+1 = βk, μk+1 = μk.

S5: Set k := k + 1 and return to step S1.

The main difficulty in proving global convergence of inexact Newton (IN) meth-ods is the fact that the inexact direction is not descent direction in general. We usea nonmonotone technique to ensure global convergence of our method. One way forglobalization of IN method for NCP is performing an Armijo-type line-search usinglocal inexact search direction and negative gradient step in the case when inexact di-rection is not descent direction, see [7]. But introducing the negative gradient stepcould slow down the algorithm and could also complicate convergence analysis insmoothing methods. The inexact Newton method for nonsmooth equations given in[9] requires sufficient decrease condition on function, and could break down. Our al-gorithm is well defined because (4) holds for α > 0 sufficiently small, since ηk > 0. Itmeans that the backtracking step S3 is well defined and completed in every iteration.

Let us define sets

K = {0} ∪{k, k ∈ N; ‖�(xk)‖ ≤ max{ξβk−1,

1

α‖�(xk) − �μk−1(x

k)‖}},

K1 ={k, k ∈ K; ξβk−1 ≥ 1

α‖�(xk) − �μk−1(x

k)‖},

K2 ={k, k ∈ K; ξβk−1 <

1

α‖�(xk) − �μk−1(x

k)‖}.

We prove the next theorem in a similar way as in [4, 18].

Theorem 1 The sequence {xk} generated by Algorithm JSIN remains in the level set

L0 ={x ∈ Rn; �(x) ≤

(α + (α + 1)

α

2

)2

�(x0)

+ 2√

η

(α + (α + 1)

α

2

)√�(x0) + η

},


where η is an upper bound of∑∞

k=0 ηk and

α = 1 + ασ (θ − 1). (7)

Proof From the definition of sets K , K1 and K2 there follows that K = {0}∪K1 ∪K2.Assume that K , which is not necessary infinite set, consists of k0 = 0 < k1 < k2 < · · ·.Let k ∈ N be an arbitrary, fixed integer and kj be the largest number in K such thatkj ≤ k. Then

μk = μkj, βk = βkj

. (8)

By (4) from step S3 of Algorithm JSIN for kj ≤ k holds

�μkj(xk) ≤ α2�μkj

(xkj ) + ηkj,

where α is given with (7). Using the definition of �μ and previous inequality it fol-lows for kj ≤ k that

‖�μkj(xk)‖ ≤

√α2‖�μkj

(xkj )‖2 + 2ηkj≤ α‖�μkj

(xkj )‖ +√

2ηkj. (9)

Then (8, 9) and Lemma 1 imply

‖�(xk)‖ ≤ ‖�μk(xk)‖ + ‖�(xk) − �μk

(xk)‖= ‖�μkj

(xk)‖ + ‖�(xk) − �μkj(xk)‖

≤ α‖�μkj(xkj )‖ +

√2ηkj

+√

2nμkj

≤ α(‖�μkj

(xkj ) − �(xkj )‖ + ‖�(xkj )‖) +√

2ηkj+

√2nμkj

≤ αβkj+ (1 + α)

√2nμkj

+√

2ηkj.

Therefore for j ≥ 0 holds

‖�(xk)‖ ≤ αβkj+ (1 + α)

√2nμkj

+√

2ηkj. (10)

If j = 0 then kj = k0 = 0, βkj= β0 and μkj

= μ0, so

‖�(xk)‖ ≤ αβ0 + (α + 1)√

2nμ0 + √2η0 = αβ0 + (α + 1)

α

2β0 + √

2η0,

≤(

α + (1 + α)α

2

)‖�(x0)‖ + √

2η. (11)

If j ≥ 1 then from step S4 of Algorithm JSIN there follows

μkj≤ 1

4μkj −1 = 1

4μkj−1 and βkj

≤ ξβkj −1 = ξβkj−1 , (12)


for kj ∈ K1 or

βkj≤ 1

α

∥∥�μkj −1(xkj ) − �(xkj )

∥∥ ≤ 1

α

√2nμkj −1

= 1

α

√2nμkj−1 ≤ 1

2βkj−1 , (13)

for kj ∈ K2.Let r = max{ 1

2 , ξ}. Then from (12) and (13) for j ≥ 1 holds

βkj≤ rβkj−1 .

By the definition of μ0 and β0 we have

μkj≤ 1

4jμ0 = 1

4j

(αβ0

2√

2n

)2

= α2

2n4j+1β2

0 = α2

2n4j+1‖�(x0)‖2,

so

√μkj

≤ 1

2j+1

α√2n

‖�(x0)‖, (14)

holds for j ≥ 1. Furthermore, for j ≥ 1

βkj≤ ξβkj−1 ≤ ξ2βkj−2 ≤ · · · ≤ rj‖�(x0)‖. (15)

Using (10, 14, 15), for j ≥ 1 we obtain

‖�(xk)‖ ≤ αβkj+ (1 + α)

√2nμkj

+√

2ηkj

≤ αrj‖�(x0)‖ + (1 + α)α

2j+1‖�(x0)‖ +

√2ηkj

≤ rj

(α + (1 + α)

α

2

)‖�(x0)‖ +

√2ηkj

≤(

α + (1 + α)α

2

)‖�(x0)‖ + √

2η. (16)

So (11) and (16) imply

‖�(xk)‖ ≤(

α + (1 + α)α

2

)‖�(x0)‖ + √

2η (17)

for j ≥ 0. Finally, from the definition of � and the previous inequality follows

�(xk) ≤(

α + (α + 1)α

2

)2

�(x0) + 2√

η

(α + (α + 1)

α

2

)√�(x0) + η

so the sequence {xk} remains in the level set L0. �


4 Convergence results

In this section we are going to prove that Jacobian Smoothing Inexact Newton methodis globally convergent. In order to do it we need some preliminary results from [18],which also hold for our algorithm.

Lemma 9 [18] Let {xk} be a sequence generated by Algorithm JSIN. Then the fol-lowing statements hold

‖�(xk) − �μk(xk)‖ ≤ α‖�(xk)‖, for k ≥ 0,

dist(�′

μk(xk), ∂C�(xk)

) ≤ γ ‖�(xk)‖, for k ≥ 1, k ∈ K.

Lemma 10 [18] Let {xk} be a sequence generated by Algorithm JSIN. Assume that{xk} has an accumulation point x∗ which is a solution of NCP. Then the index set K

is infinite and {μk} → 0.

Using the same idea as in [18] the next lemma can be proved.

Lemma 11 Let sequence {xk} be generated by Algorithm JSIN. Assume that the setK is infinite. Then each accumulation point of the sequence {xk} is a solution of NCP.

Proof Let x∗ be an accumulation point of {xk} and {xk}L be a subsequence whichconverges to x∗. Since K is infinite we have kj → ∞ and j → ∞, so from (16) therefollows

‖�(x∗)‖ = limk∈L

‖�(xk)‖ ≤ limj→∞

(rj

(α + (1 + α)

α

2

)‖�(x0)‖ +

√2ηkj

)

≤ limj→∞

(rj

(α + (1 + α)

α

2

)‖�(x0)‖ + √

2ηj

)= 0,

because r < 1 and ηj → 0 when j → ∞. Thus x∗ is a solution of NCP. �

Let us now prove the global convergence of Jacobian Smoothing Inexact Newtonmethod.

Theorem 2 Assume that F is an uniform P -function and {xk} is a sequence gen-erated by Algorithm JSIN. Then every accumulation point of the sequence {xk} is astationary point of � .

Proof We distinguish two cases. The first one is when the set K is infinite. In thiscase the statement of theorem follows immediately from Lemma 11.

The second one is the case when K is a finite set. Let k be the largest index in K .Then by Algorithm JSIN

μk = μk, βk = βk = ‖�(xk)‖, (18)

‖�(xk)‖ > ξβk = ξ‖�(xk)‖ > 0, α‖�(xk)‖ > ‖�μk(xk) − �(xk)‖ (19)


hold for every k > k. Let {xk}L be a subsequence which converges to x∗, with x∗being an accumulation point of {xk}. It can be assumed, without loss of generality,that k /∈ K , for every k ∈ L, because K is finite. The assumption that F is an uniformP -function and Lemma 5 imply that for every x ∈ L0 there exists a constant M > 0such that ‖�′

μk(x)−1‖ ≤ M . Then from the mixed inexact Newton equation (3) and

(17) there follows

‖dk‖ ≤ M1,

where M1 is a positive constant. Since limk∈L,k→∞ αkdk = 0 we consider two differ-

ent possibilities:

(a) αk → α > 0, k ∈ L;(b) αk → 0, k ∈ L.

(a) If {αk}L → α > 0, then there exist a subset L ⊂ L and a constant α > 0 suchthat αk ≥ α > 0 for all k ∈ L. So

�μk(xk+1) ≤ (1 + ασ (θ − 1))2�μk

(xk) + ηk (20)

holds for any k ∈ L. Since k ∈ L then k /∈ K and μk = μk = μ. Adding all inequali-ties (20) for k ∈ L we obtain

�μ(xk+1)+ (1− (1+ ασ (θ −1))2)∑

k∈L

�μ(xk) ≤ (1+ ασ (θ −1))2�μ(x0)+∑

k∈L

ηk.

Therefore,

∑

k∈L

�μ(xk) ≤ (1 + ασ (θ − 1))2�μ(x0) + ∑k∈L

ηk

1 − (1 + ασ (θ − 1))2and

limk∈L

�μ(xk) = 0

because the previous sum is bounded. Since limk∈L,k→∞ xk = x∗ there follows

�μ(x∗) = 0 and ‖�μ(x∗)‖ = 0. (21)

Taking into account (21), the first statement of Lemma 9 and passing to the limits wehave

‖�(x∗)‖ ≤ α‖�(x∗)‖ < ‖�(x∗)‖,since α < 1. Therefore ‖�(x∗)‖ = 0 and x∗ is a solution of NCP.

(b) Now let us consider the second possibility, {αk}L → 0. Suppose on the contrarythat x∗ is not a stationary point of � , so ∇�(x∗) = 0. The choice of αnew impliesthat for k ∈ L sufficiently large, there exists α′

k > αk , α′k ∈ [ αk

τmax,

αk

τmin] such that

limk∈L

α′k = 0 and

�μk(xk + α′

kdk) > (1 + α′

kσ (θ − 1))2�μk(xk) + ηk,


so

�μk(xk + α′

kdk) − �μk

(xk) >(2α′

kσ (θ − 1) + (α′kσ )2(θ − 1)2)�μk

(xk).

Therefore, taking the limits we obtain

limα′

k→0

�μk(xk + α′

kdk) − �μk

(xk)

α′k

≥ limα′

k→0

(2σ(θ − 1) + α′

kσ2(θ − 1)2)�μk

(xk),

and

∇�μk(xk)�dk ≥ 2σ(θ − 1)�μk

(xk). (22)

By (18, 19) and the definitions �(xk) and �μk(xk) for every k > k holds

∇�μk(xk)�dk = �μk

(xk)��′μk

(xk)dk

= �μk(xk)��′

μk(xk)

(−�′μk

(xk)−1�(xk) + �′μk

(xk)−1rk)

= −�μk(xk)��(xk) + �μk

(xk)�rk

= −2�(xk) − (�μk

(xk) − �(xk))�

�(xk)

+ (�μk

(xk) − �(xk))�

rk + �(xk)�rk

≤ −2�(xk) + ∥∥�(xk) − �μk(xk)

∥∥‖�(xk)‖+ ∥∥�(xk) − �μk

(xk)∥∥‖rk‖ + ‖�(xk)‖‖rk‖

≤ −2�(xk) + 2α�(xk) + (1 + α)tk‖�(xk)‖2

≤ +(−2(1 − α) + 2t (1 + α))�(xk). (23)

Then, from (22) and (23) there follows

2σ(θ − 1)�μk(xk) ≤ ∇�μk

(xk)�dk ≤ (−2(1 − α) + 2t (1 + α))�(xk). (24)

By (19) and (24), since μk = μk for k > k, we have

−∇�μk(xk)�dk ≤ 2σ(1 − θ)�μk

(xk) = σ(1 − θ)‖�μk(xk)‖2

≤ σ(1 − θ)(‖�(xk)‖ + α‖�(xk)‖)2

= 2σ(1 − θ)(1 + α)2�(xk). (25)

Now (24) and (25) imply

2(1 − α) − 2t (1 + α)�(xk) ≤ −∇�μk(xk)�dk ≤ 2σ(1 − θ)(1 + α)2�(xk).

Taking the limits when k → ∞, k ∈ L we obtain

(2(1 − α) − 2t (1 + α))�(x∗) ≤ −∇�μ∗(x∗)�d∗ ≤ 2σ(1 − θ)(1 + α)2�(x∗). (26)


On the other hand we have �(x∗) > 0, because otherwise K would be infinite byLemma 10 and we consider the case when K is a finite set. Thus (26) implies

2(1 − α) − 2t (1 + α) ≤ 2σ(1 − θ)(1 + α)2 and

t ≥ 1 − α

1 + α− σ(1 − θ)(1 + α).

This is a contradiction with the condition on t in step S0 of the Algorithm JSIN. So,under the assumption that x∗ is not a stationary point of � we get the contradiction.The proof of theorem is now completed. �

Since we assume that F is an uniform P -function, Lemma 7 and Theorem 2 implythat every accumulation point of the sequence generated by Algorithm JSIN is astationary point of � and its global minimum, so it is also a solution of NCP.

Now we are going to establish the superlinear convergence of Jacobian smoothinginexact Newton method with special choice of the sequence {ηk}.

Theorem 3 Assume that F is an uniform P -function. Let x∗ be an accumulationpoint of a sequence {xk} generated by Algorithm JSIN with sequence {ηk} defined by

ηk = (2 + σ(θ − 1))2nμk + (2 + σ(θ − 1))√

2nμk(1 + σ(θ − 1))‖�μk(xk)‖.

Assume that all elements of ∂C�(x∗) are nonsingular and let limk→∞ tk = 0. Thenx∗ is a solution of NCP and the sequence {xk} converges to x∗ q-superlinearly.

Proof It is clear that x∗ is a solution of NCP by Theorem 2. Since ∂B�(x∗) ⊆∂C�(x∗) from Lemma 4 there is a neighborhood of x∗ such that x∗ is the uniquesolution in it. Since x∗ is an accumulation point of the sequence {xk} generated bythe Algorithm JSIN and also is a solution of NCP, Lemma 10 implies that K is aninfinite set. Then there is a subsequence K0 of K such that {xk}K0 → x∗.

First, we will show that this special choice of sequence {ηk} satisfies∑∞

k=0 ηk < ∞.Let A = (2 + σ(θ − 1))2n, and B = (2 + σ(θ − 1))

√2n(1 + σ(θ − 1)). Then

∞∑

k=0

ηk = A

∞∑

k=0

μk + B

∞∑

k=0

√μk‖�μk

(xk)‖. (27)

From Lemma 1 and Algorithm JSIN it follows that

∥∥�μk(xk) − �μk−1(x

k)∥∥ ≤ √

2n(√

μk−1 − √μk ),

so

‖�μk(xk)‖ ≤ ∥∥�μk

(xk) − �μk−1(xk)

∥∥ + ‖�μk−1(xk)‖

≤ √2n(

√μk−1 − √

μk ) + ‖�μk−1(xk)‖.


Since K is an infinite set, (6) implies

√μk‖�μk

(xk)‖ ≤ √2n

√μk(

√μk−1 − √

μk) + √μk‖�μk−1(x

k)‖≤ √

2n√

μk

1

2k

√μ0 + √

μk‖�μk−1(xk)‖

≤ √2n

√μk

1

2k

√μ0 + μk−1. (28)

Using (6, 27, 28) and Lemma 1 we get

∞∑

k=0

ηk ≤ Aμ0

∞∑

k=0

1

4k+ B

∞∑

k=0

√2n

√μk

1

2k

√μ0 + B

∞∑

k=1

μk−1

= Aμ0

∞∑

k=0

1

4k+ B

√2nμ0 + B

√μ0

∞∑

k=1

√2nμk

2k+ B

∞∑

k=1

μk−1

= Aμ0

∞∑

k=0

1

4k+ B

√2nμ0 + B

√2nμ0

∞∑

k=1

1

4k+ Bμ0

∞∑

k=1

1

4k−1

= (A + B)μ0

∞∑

k=0

1

4k+ B

√2nμ0

(

1 +∞∑

k=1

1

4k

)

< ∞,

because constants A,B,μ0 and previous sums are bounded.So, for this special choice of {ηk} there holds

∑∞k=0 ηk < ∞.

Lemma 5 implies that �′μk

(xk) is nonsingular. It can be shown in the same way asin [4] that

‖�′μk

(xk)−1‖ ≤ M (29)

for all k ∈ K0 large enough and some positive constant M . The set ∂C�(xk) is non-empty and compact, so there exists Hk ∈ ∂C�(xk) such that

dist(�′

μk(xk), ∂C�(xk)

) = ‖�′μk

(xk) − Hk‖. (30)

From the second statement of Lemma 9 it follows

‖�′μk

(xk) − Hk‖ ≤ γβk, k ∈ K0. (31)

Lipschitz continuity of �, Lemma 2, (29, 31), the assumption that limk→∞ tk = 0and the construction of Algorithm JSIN imply

‖xk + dk − x∗‖ = ‖xk − x∗ − �′μk

(xk)−1(�(xk) − rk)‖≤ ‖�′

μk(xk)−1‖(∥∥�(xk) − �(x∗) − Hk(x

k − x∗)∥∥

+ ‖Hk − �′μk

(xk)‖‖xk − x∗‖ + ‖rk‖)

≤ ‖�′μk

(xk)−1‖(∥∥�(xk) − �(x∗) − Hk(xk − x∗)

∥∥

+ ‖Hk − �′μk

(xk)‖‖xk − x∗‖ + tk‖�(xk)‖)


≤ M(o(‖xk − x∗‖) + γβk‖xk − x∗‖ + tk‖�(xk) − �(x∗)‖)

= o(‖xk − x∗‖) (32)

for k ∈ K0 sufficiently large, since βk → 0, where Hk ∈ ∂C�(xk) is chosen such that(30) holds. Hence by (32) and Lemma 3 for k ∈ K0, k → ∞ follows

‖�(xk + dk)‖ = o(‖�(xk)‖). (33)

Now, we are going to prove that there exists an index k ∈ K0 such that for everyk ≥ k, k ∈ K0 the index k + 1 belongs to set K0 and xk+1 = xk + dk . Since (33) isvalid for k sufficiently large then

‖�(xk + dk)‖ ≤ (1 + σ(θ − 1))‖�(xk)‖. (34)

From Lemma 1 and (34) we have

‖�μk(xk + dk)‖ ≤ ‖�(xk + dk)‖ + ∥∥�μk

(xk + dk) − �(xk + dk)∥∥

≤ (1 + σ(θ − 1))‖�(xk)‖ + √2nμk

≤ (1 + σ(θ − 1))(‖�μk

(xk)‖ + ‖�(xk) − �μk(xk)‖) + √

2nμk

≤ (1 + σ(θ − 1))‖�μk(xk)‖ + (2 + σ(θ − 1))

√2nμk,

and using this choice of sequence {ηk} and previous inequality it follows that

�μk(xk + dk) ≤ (1 + σ(θ − 1))2�μk

(xk) + ηk,

so the condition (4) in S3 of Algorithm JSIN is satisfied for αk = 1 and then xk+1 =xk + dk . Let σ ≤ ξ−1

θ−1 . Then (34) gives

‖�(xk+1)‖ = ‖�(xk + dk)‖ ≤ (1 + σ(θ − 1))‖�(xk)‖ ≤ ξ‖�(xk)‖.The last inequality and the definition of K together with (32) imply that k + 1 ∈ K0.Finally, we can conclude that for k ∈ K0 large enough there holds xk+1 = xk + dk

and k + 1 ∈ K0. Repeating this it may be proved that for all k ≥ k we state k ∈ K0and xk+1 = xk + dk , so this fact with (32) implies q-superlinear convergence. �

Our method with special choice of the sequence {ηk} and the inexact Newtonmethods for semismooth systems presented in [7, 9] are q-superlinearly convergent.The assumption in the algorithm that F is a P0-function implies nonsingularity of thesmoothing Jacobian during the iterative process. It also implies that every stationarypoint of � is a global minimizer. In order to get bounded level set we assumed thatF is a uniform P -function. The assumption A2 given in [9] and R-regular assump-tion from [7] also imply solvability of linear systems during iterative procedure. Themethod from [7] in principle has better chances for global convergence due to the factthat it converges to stationary points of the squared Fischer–Burmeister function. Theassumption about uniform P -function is not satisfied in all test examples presented in


the next section but our method was rather competitive with the one from [7]. Com-bining nonmonotone technique for globalization we can overcome difficulties arisingfrom the fact that the inexact search direction might be nondescent.

5 Numerical experiments

In this section we present some numerical results obtained by our Jacobian Smooth-ing Inexact Newton method, Jacobian Smoothing Newton method (as its special casewith tk = 0), and semismooth method with inexact Fischer–Qi direction from [7]. Al-gorithms are implemented in MATLAB 7.0. The test problems are generated in theway proposed by Gomes-Ruggiero et al. [15].

Let f (x) = (f1(x), f2(x), . . . , fn(x))� be a differentiable nonlinear mappingfrom Rn to Rn and let x∗ = (1,0,1,0, . . .)� ∈ Rn. For i = 1,2, . . . , n set

Fi(x) ={

fi(x) − fi(x∗), if i is odd or i > r,

fi(x) − fi(x∗) + 1, otherwise,

where r ≥ 0 is an integer. For function F defined in this way, vector x∗ is a solutionof NCP, but not necessarily its unique solution. If r < n, x∗ is a degenerate solutionof NCP, while for r = n it is a nondegenerate solution. Function f is defined by usingall test problems proposed in [21], problems 2, 4, 6, 7, 12, 13, 25 and 27 from [25]and problems 1.1–1.3 and 1.5 from [2]. We tested three cases of forcing parametersfor both inexact methods: tk = 0.5, tk = 2−k and tk = ‖�(xk)‖. More precisely, wetested seven methods:

• JSN—Jacobian Smoothing Newton method.• JSIN1—Jacobian Smoothing Inexact Newton method with tk = 0.5.• JSIN2—Jacobian Smoothing Inexact Newton method with tk = 2−k .• JSIN3—Jacobian Smoothing Inexact Newton method with tk = ‖�(xk)‖.• GIN1—Semismooth method with inexact Fischer–Qi direction and tk = 0.5.• GIN2—Semismooth method with inexact Fischer–Qi direction and tk = 2−k .• GIN3—Semismooth method with inexact Fischer–Qi direction and tk = ‖�(xk)‖.

On all the test problems we considered three dimensions, n = 10, n = 100, n =1000, and two different starting points suggested in [2, 7, 21, 25]. The first startingpoint denoted by x0 is from papers dealing with smooth problems [2, 21, 25] and thesecond one denoted by x0 is defined by

x0i =

{10x0

i , if x0i = 0,

10, otherwise.

For each dimension of problem we consider one degenerate case (r = n/2) and thenondegenerate case (r = n). The main stopping criteria is

‖�(xk)‖ ≤ 10−5√n,

but if it is not satisfied, the algorithms are stopped after kmax = 200 iterations. We usethe same parameters value for GIN1, GIN2, GIN3 as in [7], and also set σ = 10−4,


α = 0.5, ξ = 0.5, γ = 20, θ = 0.8, τmin = 0.3, τmax = 0.8 in methods JSN, JSIN1,JSIN2 and JSIN3. For all inexact methods, GMRES is used as the linear solver.

Further indices are used to collect the data which are compared: the index of ro-bustness, the efficiency index and the combined robustness and efficiency index.

The robustness index is defined by

Rj = tj

nj

,

the efficiency index is

Ej =m∑

i=1,rij =0

(rib

rij

)/tj ,

and the combined index is

Ej × Rj =m∑

i=1,rij =0

(rib

rij

)/nj ,

where rij is the number of iterations required to solve the problem i by the method j ,rib = minj rij , tj is the number of successes by method j and nj is the number ofproblems attempted by method j .

Tables 1–4 report the numerical results of tested methods.The best possible value for all indices is 1. As inexact methods have good theoret-

ical background, they are suitable for solving large-scale complementarity problems.On the other hand, their numerical behavior is worse than the behavior of related ex-act methods, which can be seen comparing JSIN1, JSIN2, JSIN3, with JSN method.

Numerical results obtained by JSIN2 are better than the results of other consideredinexact methods on this collection of test problems. Superlinear convergence of ourmethod is proved only for uniform P -function F. Some of the test functions do not

Table 1 Nondegenerate case (r = n) with starting point x0

JSN JSIN1 JSIN2 JSIN3 GIN1 GIN2 GIN3

R 0.875 0.6667 0.7639 0.5694 0.6667 0.7361 0.6111

E 0.9468 0.7723 0.8756 0.9383 0.7786 0.8716 0.9187

E × R 0.8284 0.5149 0.6688 0.5343 0.51913 0.6416 0.5742

Table 2 Nondegenerate case (r = n) with starting point x0


R 0.8472 0.75 0.9028 0.4167 0.6944 0.8194 0.4861

E 0.9051 0.6954 0.8854 0.9222 0.6685 0.838 0.8611

E × R 0.7668 0.5215 0.7993 0.3843 0.4642 0.6867 0.4186


Table 3 Degenerate case (r = n/2) with starting point x0


R 0.7606 0.6761 0.7746 0.4648 0.6197 0.7606 0.4366

E 0.9721 0.5851 0.8265 0.9803 0.5855 0.8134 0.9157

E × R 0.7393 0.3956 0.6402 0.4556 0.3628 0.6187 0.3998

Table 4 Degenerate case (r = n/2) with starting point x0


R 0.8382 0.6618 0.7941 0.3382 0.6176 0.8088 0.5441

E 0.9383 0.5381 0.8397 0.8197 0.5654 0.7925 0.8306

E × R 0.7865 0.3561 0.6668 0.2773 0.3492 0.6410 0.452

satisfy this assumption but numerical results are not seriously affected by this fact.In fact, efficiency coefficients for JSIN2 are significantly better in comparison withJSIN1 both in nondegenerate and degenerate case. Clearly JSIN2 is closer to the exactNewton method (JSN, [18]) since forcing parameters tend to zero. Therefore we thinkthat the absence of uniform P condition is compensated with the beneficial similaritywith the exact Newton method.

From the theoretical point of view, methods JSIN3 and GIN3 should have the bestproperties. However, numerical experiments report that this is not true in practice,because of a large number of failures of these algorithms. The reason for this mightbe the fact that GMRES linear solver is quite often not the best choice for solving alinear system in our test collection. However we used it in all cases since the objectiveof our investigation was not concerned with linear algebra issues. We can concludethat efficiency and robustness of these inexact methods depends very much on theavailability of a good iterative linear solver.

The fact which is also interesting is that there are some problems for which differ-ent algorithms converge to different solutions. That fact perhaps might be explainedby the influence of gradient step in GIN methods and nonmonotone rule in JSINmethods. It also can be remarked that all methods failed on problem 4.6 from [21],with starting point x0, but with a new stopping criteria

‖�(xk)‖ ≤ 10−6,

method JSIN1 applied to this problem with the dimension n = 10, found the solution.Comparing our method with semismooth method from [7], we can see that they

have similar numerical behavior. Numerical examples pointed out that in some casesin which our method doesn’t converge the method from [7] converges, but with gra-dient steps. That corresponds well with better theoretical expectations of the methodfrom [7]. On the other hand in some other cases JSIN methods manage to find a solu-tion faster than GIN methods since nonmonotone technique does not require gradientsteps which might slow down the process.


Acknowledgements The authors are grateful to two anonymous referee for valuable suggestions andcomments.

References

1. Birgin, E., Krejic, N., Martínez, J.M.: Globally convergent inexact quasi-Newton methods for solvingnonlinear systems. Numer. Algorithms 32, 249–260 (2003)

2. Bus, J.C.P.: Numerical Solution of Systems of Nonlinear Equations. Mathematical Centre Tracts, vol.122. Mathematical Centre, Amsterdam (1980)

3. Chen, B., Chen, X.: A global and local superlinear continuation-smoothing method for P0 + R0 andmonotone NCP. SIAM J. Optim. 7(2), 403–420 (1997)

4. Chen, X., Qi, L., Sun, D.: Global and superlinear convergence of the smoothing Newton methodand its applications to general box constrained variational inequalities. Math. Comput. 67, 519–540(1998)

5. Clarke, F.H.: Optimization and Nonsmooth Analysis. Wiley, New York (1983)6. De Luca, T., Facchinei, F., Kanzow, C.: A semismooth equation approach to the solution of nonlinear

complementarity problems. Math. Program. 75, 407–439 (1996)7. De Luca, T., Facchinei, F., Kanzow, C.: A theoretical and numerical comparison of some semismooth

algorithms for complementarity problems. Comput. Optim. Appl. 16, 173–205 (2000)8. Dembo, R., Eisenstat, S.C., Steihaug, T.: Inexact Newton methods. SIAM J. Numer. Anal. 19, 400–

408 (1982)9. Dingguo, P., Weiwen, T.: Globally convergent inexact generalized Newton’s methods for nonsmooth

equations. J. Comput. Appl. Math. 138, 37–49 (2002)10. Eisenstat, S.C., Walker, H.F.: Globally convergent inexact Newton’s method. SIAM J. Optim. 4, 393–

422 (1994)11. Facchinei, F., Kanzow, C.: A nonsmooth inexact Newton method for the solution of large scale com-

plementarity problems. Math. Program. 76, 493–512 (1997)12. Facchinei, F., Soares, J.: A new merit function for nonlinear complementarity problems and a related

algorithm. SIAM J. Optim. 7, 225–247 (1997)13. Ferris, M., Pang, J.S.: Engineering and economic applications of complementarity problems. SIAM

Rev. 39, 669–713 (1997)14. Fischer, A.: A special Newton-type optimization method. Optimization 24, 269–284 (1992)15. Gomes-Ruggiero, M.A., Martínez, J.M., Santos, S.A.: Solving nonsmooth equations by means of

quasi-Newton methods with globalization. In: Recent Advances in Nonsmooth Optimization, pp. 121–140. World Scientific, Singapore (1995)

16. Kanzow, C.: Some noninterior continuation methods for linear complementarity problems. SIAM J.Matrix Anal. Appl. 17, 851–868 (1996)

17. Kanzow, C.: Inexact semismooth Newton methods for large-scale complementarity problems. Optim.Methods Softw. 19, 309–325 (2004)

18. Kanzow, C., Pieper, H.: Jacobian smoothing methods for general nonlinear complementarity prob-lems. SIAM J. Optim. 9, 342–373 (1999)

19. Krejic, N., Martínez, J.M.: Inexact-Newton methods for semismooth systems of equations with block-angular structure. J. Comput. Appl. Math. 103, 239–249 (1999)

20. Li, D., Fukushima, M.: Globally convergent Broyden-like methods for semismooth equations andapplications to VIP, NCP and MCP. Ann. Oper. Res. 103, 71–97 (2001)

21. Lukšan, L.: Inexact trust region method for large sparse systems of nonlinear equations. J. Optim.Theory Appl. 81, 569–590 (1994)

22. Martínez, J.M., Qi, L.: Inexact Newton’s method for solving nonsmooth equations. J. Comput. Appl.Math. 60, 127–145 (1995)

23. Qi, L.: Converges analysis of some algorithms for solving nonsmooth equations. Math. Oper. Res. 18,227–244 (1993)

24. Qi, L., Sun, J.: A nonsmooth version of Newton’s method. Math. Program. 58, 353–367 (1993)25. Spedicato, E., Huang, Z.: Numerical experience with Newton-like methods for nonlinear algebraic

systems. Computing 58, 69–89 (1997)26. Ypma, T.J.: Local convergence of inexact Newton methods. SIAM J. Numer. Anal. 21, 583–590

(1984)

globally convergent jacobian smoothing inexact newton methods for ncp

Documents