honglei˜xu˜· song˜wang soon-yi˜wu editors optimization

Honglei Xu · Song WangSoon-Yi Wu Editors

Optimization Methods, Theory and Applications

Optimization Methods, Theory and Applications

Honglei Xu • Song Wang • Soon-Yi WuEditors

Optimization Methods,Theory and Applications

123

EditorsHonglei XuDepartment of Mathematics and StatisticsCurtin UniversityPerth, WA, Australia

Soon-Yi WuDepartment of MathematicsNational Cheng Kung UniversityTainan, Taiwan

Song WangDepartment of Mathematics and StatisticsCurtin UniversityPerth, WA, Australia

ISBN 978-3-662-47043-5 ISBN 978-3-662-47044-2 (eBook)DOI 10.1007/978-3-662-47044-2

Library of Congress Control Number: 2015942905

Springer Heidelberg New York Dordrecht London© Springer-Verlag Berlin Heidelberg 2015This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part ofthe material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation,broadcasting, reproduction on microfilms or in any other physical way, and transmission or informationstorage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodologynow known or hereafter developed.The use of general descriptive names, registered names, trademarks, service marks, etc. in this publicationdoes not imply, even in the absence of a specific statement, that such names are exempt from the relevantprotective laws and regulations and therefore free for general use.The publisher, the authors and the editors are safe to assume that the advice and information in this bookare believed to be true and accurate at the date of publication. Neither the publisher nor the authors orthe editors give a warranty, express or implied, with respect to the material contained herein or for anyerrors or omissions that may have been made.

Printed on acid-free paper

Springer-Verlag GmbH Berlin Heidelberg is part of Springer Science+Business Media (www.springer.com)

Preface

The 9th International Conference on Optimization: Techniques and Applications(ICOTA9) was held in National Taiwan University of Science and Technology,Taipei, during December 12–16, 2013. As a continuation of the ICOTA series, thegoal of the 9th ICOTA is to provide a forum for scientists, researchers, softwaredevelopers, and practitioners to exchange ideas and approaches, to present researchfindings and state-of-the-art solution techniques, to share experiences on potentialsand limits, and to open new avenues of research and developments on all issues andtopics related to optimization and its applications.

This conference consisted of 2 keynote addresses, 13 plenary lectures, and56 technical sessions, and this book contains 10 chapters on recent advances inoptimization and optimal control presented at the conference. Each of the chapterswas accepted after a stringent peer review process by at least two independentreviewers to ensure that the works are of high quality.

Chapter 1 establishes a mathematical technique to analyze human walking behav-ior using dynamic optimization. The method works well for a complex movementthat involves a change in the dynamics from single support phase to double supportphase. Numerical results can replicate human walking motions and calculate theoptimal joint torques to produce the resulting motions. Chapter 2 studies two opti-mization problems related to a class of elliptic boundary value problems on smoothbounded domains of RN . These optimization problems are formulated as solvableminimum and maximum problems related to the rearrangements of given functions.Chapter 3 proposes a multiobjective optimization method that supports agile andflexible decision-making to handle complex and diverse decision environments.Chapter 4 investigates the existence of solutions in connection with variational-likehemivariational inequalities in reflexive Banach spaces. Conditions for the existenceof solutions of the variational-like hemivariational inequalities involving lowersemicontinuous set-valued maps are established. Chapter 5 develops an inertialalgorithm and proves its weak convergence for solving the split common fixed-pointproblem for demicontractive mappings in Hilbert space. It provides an efficient wayto study the split common fixed-point problem. Chapter 6 investigates a class of

v

vi Preface

multiobjective optimization problems with inequality, equality, and vanishing con-straints. It shows that under mild assumptions, some constraint qualifications, suchas Cottle constraint qualification, Slater constraint qualification, and Mangasarian-Fromovitz constraint qualification, are not satisfied. New Karush-Kuhn-Tucker-typenecessary optimality conditions are developed accordingly. Chapter 7 proposes anew hybrid global optimization technique, where a gradient-based method withBFGS update is combined with an Artificial Bee Colony, to solve an Archieparameter estimation problem. This global optimization technique has both thefast convergence of gradient descent algorithm and the global convergence ofswarm algorithm. Chapter 8 considers the regularization problem of a nonlinearprogram. It examines inner connections among exact regularization, normal coneidentity, and the existence of a weak sharp minimum for certain associated nonlinearprograms. Chapter 9 presents a mathematical methodology that optimally solvesan inverse mixing problem when both the composition of the source componentsand the amount of each source component are unknown. The model is used foranalyzing longitudinal proton magnetic resonance spectroscopy (1H MRS) datagathered from the brains of newborn infants. It shows that the method can providemore specific and accurate assessments of the brain cell types during early braindevelopment in neonates. It is also beneficial to study a wide range of physicalsystems that involve mixing of unknown source components. Finally, Chap. 10considers an optimal design problem of a DFT filter bank subject to subchannelvariation constraints. The design problem is transformed to a minimax optimizationproblem, which is equivalent to a semi-infinite optimization problem. Moreover,a computational procedure is proposed to solve such a semi-infinite optimizationproblem. Simulations and comparisons also show the effectiveness of the results.

We would like to thank the organizing institutions and sponsors of the confer-ence and this book, including the National Nature Science Foundation of China(11171079, 11410301010) and Natural Science Foundation of Hubei Provinceof China (2014CFB141). In editing this book, we have been assisted by manyvoluntary colleagues, particularly the anonymous referees. Thus, we take thisopportunity to thank all the referees for their efforts and valuable comments. Wewould also like to thank the authors of these chapters for their contributions andpatience. Last but not least, we would like to express our gratitude to the Springerstaff including Grace Guo, Emmie Yang, and Toby Chai for their professionalismand help and to all who have, in one way or another, contributed to the publicationof this book.

Perth, WA, Australia Honglei XuPerth, WA, Australia Song WangTainan, Taiwan Soon-Yi Wu

Contents

1 Analysing Human Walking Using Dynamic Optimisation . . . . . . . . . . . . . 1Meiyi Tan, Leslie S. Jennings, and Song Wang

2 Rearrangement Optimization Problems Related to a Classof Elliptic Boundary Value Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35Chong Qiu, Yisheng Huang, and Yuying Zhou

3 An Extension of the MOON2/MOON2R Approachto Many-Objective Optimization Problems. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51Yoshiaki Shimizu

4 Existence of Solutions for Variational-LikeHemivariational Inequalities Involving LowerSemicontinuous Maps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67Guo-ji Tang, Zhong-bao Wang, and Nan-jing Huang

5 An Iterative Algorithm for Split Common Fixed-PointProblem for Demicontractive Mappings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85Yazheng Dang, Fanwen Meng, and Jie Sun

6 On Constraint Qualifications for MultiobjectiveOptimization Problems with Vanishing Constraints . . . . . . . . . . . . . . . . . . . 95S.K. Mishra, Vinay Singh, Vivek Laha, and R.N. Mohapatra

7 A New Hybrid Optimization Algorithm for the Estimationof Archie Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137Jianjun Liu, Honglei Xu, Guoning Wu, and Kok Lay Teo

8 Optimization of Multivariate Inverse Mixing Problemswith Application to Neural Metabolite Analysis . . . . . . . . . . . . . . . . . . . . . . . . 155A. Tamura-Sato, M. Chyba, L. Chang, and T. Ernst

vii

viii Contents

9 Exact Regularization, and Its Connections to NormalCone Identity and Weak Sharp Minima inNonlinear Programming . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175S. Deng

10 The Worst-Case DFT Filter Bank Design with SubchannelVariations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183Lin Jiang, Changzhi Wu, Xiangyu Wang, and Kok Lay Teo

Chapter 1Analysing Human Walking Using DynamicOptimisation

Meiyi Tan, Leslie S. Jennings, and Song Wang

Abstract A mathematical model to simulate human walking motions and studythe dynamics behind walking is developed which is adjustable to accommodatedifferent cases such as the single and double support phases of walking. We firstpropose a technique for estimating joint moments and position coordinates of bodysegments using the method of inverse dynamics. The estimates are then used asinitial joint torques for solving the model as an optimal control problem with thesetup of appropriate objective functions and constraints. Numerical experiments onthe developed model and solution technique have been performed and the numericalresults show that the model is able to replicate human walking motions and theoptimal joint torques can be calculated to produce the resulting motions.

1.1 Introduction

The study of human motion has been of considerable interest in the field ofbiomechanics. It provides detailed information to understand the human movementsthat enable certain motions to be improved or made safer. The desire to understandthe mechanics behind walking has spurred the study of human locomotion (McGeer1988) but due to its complex nature, modelling and understanding human walkingcontinues to be a challenging research problem in multibody systems (Hardt et al.1999). Movement patterns can be predicted as best as possible using mathematicalmodels, however models can become very complicated while trying to model ahuman body and its movements as closely as possible due to a body’s complexity(Alexander 1996, 2003).

In order to replicate human walking motion realistically, a complete gait cycle,comprising of two continuous steps, should be considered. Each step is made up

M. Tan • L.S. JenningsSchool of Mathematics & Statistics, The University of Western Australia, 35 Stirling Highway,Crawley, WA 6009, Australia

S. Wang (�)Department of Mathematics and Statistics, Curtin University, GPO Box U1987, Perth,WA 6845, Australiae-mail: [email protected]

© Springer-Verlag Berlin Heidelberg 2015H. Xu et al. (eds.), Optimization Methods, Theory and Applications,DOI 10.1007/978-3-662-47044-2_1

1

2 M. Tan et al.

of two phases namely, single support phase and double support phase. The singlesupport phase occurs when one foot contacts the ground while the other leg isswinging from rear to front, starting from the rear foot toe-off and ending whenthe swinging foot lands on the ground with a heel strike. The double support phasebegins with the heel strike of the forward swing foot and ends with the toe-off of therear foot. As such, Bessonnet et al. (2004) described single support phase as movinglike an open tree-like kinematic chain while double support phase is kinematicallyclosed and overactuated.

The kinematic configuration of the model biped may change going throughone phase to the other during the collision of the foot with the ground whichresults in jump conditions on the velocities (Hardt et al. 1999). This could havebeen a contributing reason as to why most early research only considered singlesupport or assumed an instantaneous double support phase (Ren et al. 2007).Instantaneous double support phase was being considered for walking simulationsin later research, followed by studies which considered a complete step consisting ofboth the single support and double support phases (Xiang et al. 2010). In addition,the foot segment was often neglected or assumed to be flat on the floor duringstance.

In order to achieve realistic human walking motions, both phases are necessaryand should be incorporated in the model. Modelling a biped with feet will allowthe modelling of the double support phase, from swing heel strike to stance toe off.Hardt et al. (1999) suggested that feet bring about the addition of ankle torques andliftoff force that is produced as the heel comes off the ground. They contemplatedthat at a higher speed, the biped cannot walk effortlessly without the inclusion ofa foot. The addition of ankle actuation generated a smoother walking motion andallowed torque inputs at hip to be distributed to the knees and ankles. Anotheradvantage of having a foot as a segment was the ability to distribute center ofpressure from the rear to the front of the foot during ground contact (Bullimoreand Burn 2006).

The double support phase of the gait cycle is deemed a common example of aclosed-loop problem. In a closed-loop model such as the double support phase, thenumber of actuating torques is usually more than the number of degrees of freedom,presenting a redundancy problem for inverse dynamics and control applications toresolve (Ünver et al. 2000).

The inverse dynamics problem of biomechanics has been the most commonmethod used to estimate muscle forces during locomotion (Anderson and Pandy2001) since it is computationally inexpensive and solutions can be obtainedrelatively quickly on single-processor computers. However, Marshall (1985) andSelles et al. (2001) both acknowledged that a major problem in the inverse dynamicsapproach is the need for numerical differentiation of potentially noisy position data.Even though inverse dynamics have been commonly used to estimate joint torquesduring locomotion (Anderson and Pandy 2001), they produce poor results in thepresence of noisy measurements (Kuo 1998). In addition, to model a complete gaitcycle, both single and double support phases have to be considered. However, theredundancy problem of the double support phase, leads to a state of indeterminacy,

1 Analysing Human Walking Using Dynamic Optimisation 3

unless ground reaction forces are known. The inverse dynamics method is not ableto directly solve the equations of motion for that phase as seen in Ren et al. (2007)model.

The dynamic optimisation method integrates the equation of motion during anoptimisation process to simulate motions and solve for the optimal joint forces ofthe model (Chow and Jacobson 1971; Anderson and Pandy 2001; Pandy 2001). Esti-mating the trajectories of joint torques using this method is practical as the methodapplies a forward simulation to reproduce a best observed motion (Chao and Rim1973). A major disadvantage of dynamic optimisation is that it is more expensivecomputationally (Yamaguchi and Zajac 1990), and hence has led to solutions forwalking being greatly simplified. A large amount of computation is required tocompute the trajectory of joint torques when using dynamic optimisation due tothe choice of initial guesses of torque value and the mathematical sophisticationrequired to understand the technique (Koh and Jennings 2003). However, as ageneral rule of dynamic optimisation, an initial estimate set of torque trajectories isrequired to start the optimisation. In Koh (2001), Koh used the conventional methodof inverse dynamics to find the torque trajectories as initial estimates to speed upthe convergence of the numerical process (Chao and Rim 1973). Koh had managedto incorporate the use of both methods by applying the inverse dynamics methodto determine the initial torque estimates to be used in the dynamic optimisationmethod.

Most research tends to avoid modelling double support phase as it, being aclosed loop problem, produces redundancy problem and complicates the modelling.Studies that consider modelling both single and double support phases have chosento employ the method of inverse dynamics and optimsation to solve the system ofboth phases, and had two different algorithms to deal with each phase. The purposeof this study is to develop a generalised model applying optimal control theory, tosimulate normal walking motion through both the single support and double supportphases, and to gain insight into the mechanics that are involved in the overall motion.Since normal walking can be assumed to have symmetric and cyclic characteristics,only one step of the gait cycle needs to be modelled (Hardt et al. 1999; Anderson andPandy 2001; Xiang et al. 2009) with appropriate semi-periodic boundary conditions.A combination of both inverse dynamics method and dynamic optimisation methodwill be used to solve the equations of motion to model the human motion. Jointtorques and forces obtained will be studied to understand the mechanics behindcertain movements.

One major contribution is to have one model able to solve for one step cycle inwalking. The model formulated in this study can be adjusted according to cases,hence the single support and double support phases can be solved for individuallyor together. As the aim of the study is to simulate walking for one step cycle, bothphases are considered and solved.

An advantage the study brings is the application of the conventional inversedynamics method to obtain initial estimates of joint torques to be used in dynamicoptimisation, which is the approach taken in this study to produce the observedmotion and improve joint torques estimates. An initial joint torques “guess” from

4 M. Tan et al.

applying the inverse dynamics method can reduce computation time in computingjoint torque trajectories. In addition, dynamic optimisation solves the problem byintegrating the equation of motion forward in time, so simulation is performedin a manner consistent with the development of motion in humans, allowing theresearch to evaluate the effects of changes in ‘muscle activity’ on the outcome ofthe movement.

The rest of this paper is organised as follows. In the next section, we will first setup the geometry and the mathematical model of the walking model. In Sect. 1.3, wewill establish the objective function, its corresponding constraints and discuss theinverse analysis used to obtain the initial joint estimates. In Sect. 1.4, we presentthe analysis of numerical experiments to confirm that the mathematical modelformulated replicated human walking motions.

1.2 Model Development

1.2.1 Geometry

We formulate a mathematical model for an efficient and versatile computation ofthe forward dynamics for a two-dimensional (2D) system of planar linked rigidbodies. The formulation of the model was adopted from Koh (2001), whose studywas to optimise performance on a Yurchenko vault. It is similar to Kuo (1998) exceptwith more formalised and simplified notations with respect to the topology of themodel. A link segment model was used to represent the human body (Fig. 1.2). Theseven link segments (n D 7) representing the human body were assumed to movein the two-dimensional (2D) sagittal plane. The head, arms and trunk (HAT) wererepresented by one segment, assuming that arms did not swing excessively (Cavagnaet al. 1976; Dean 1965), so as to change the points of center of mass (CoM) of HAT.Each of the feet was represented by a fixed triangle of appropriate shape and theforward point of the triangle was placed about 0.04 m in front of the metatarsal jointto partially compensate for the toe action during the later stages of pushoff. Jointsbetween segments represent the ankles, knees and hip. The link segment model issimilar to that used in Onyshko and Winter (1980), which is deemed to provide agood compromise between complexity and the accurate representation of the realsituation. An increase in the number of segments will increase the complexity ofresulting equations of motion rapidly. However, less than seven segments greatlyreduce the accuracy of the model.

Anthropometric data for the model, including segment masses and centre ofmass (CoM) positions were determined by using the anthropometric proportionsand regression equations given by Winter (2009). As we did not use any subjects,we assumed an adult height of 1.7 m and weight of 65 kg. We assumed each segmentto be similar to a rod and calculated moments of inertia of each segment based on


the equation used to calculate moments of inertia of a rod, with the axis of rotationat the CoM of the segment.

The ith segment has length `i, mass mi and moment of inertia Ii about its centerof mass (CoM)(within the segment), which is a distance ri from proximal .xp

i ; ypi /

and distance li � ri from the distal .xdi ; yd

i / end (Fig. 1.1). All segments are labelledto have a proximal and distal end. The proximal end of a segment is situated closestto point of contact while the distal end of a segment is situated furthest from pointof contact. In this case, we have considered the point of contact to be the ball of thestance foot. There is a global coordinate system, XOY, which has an origin fixed inan infinite mass “ground”. Each segment’s position is known from its CoM (xi, yi)and the angle �i that the segment makes with the positive x-axis. The CoM of thewhole body, (X, Y), is given by

�XY

�D 1

M

�PniD1 mixiPniD1 miyi

�where M D

nXiD1

mi:

Fig. 1.1 An ith segment diagram

6 M. Tan et al.

Segment 6(Swing Foot)(x

6d,y

6d)

Segment 5(Swing Shank)

Segment 4(Swing Thigh)

Segment 3(Stance Thigh)

Segment 2(Stance Shank)

Segment 1(Stance Foot)

Segment 7(Head, Arms, Trunk)

5

4 3

2

1

7 6

(x1

p,y1

p)

X

Y

Walk Direction

Fig. 1.2 Seven-segment model

For a 2D connected body, knowing all �i, i D 1; : : : ; n, and either one segment’spositional coordinate or the CoM (X, Y) gives the position of every segment.Defining .xp

1; yp1/ as the coordinates of the proximal end of the first segment

(Segment 1) allows a convenient and concise way to describe the positions of theCoM of the segments, as this point is fixed throughout the movement. The positionalequations for a chain of segments (with a multiple branch on joint 4 where Segment3, Segment 4 and Segment 7 join, see Fig. 1.2) are

x D xp1e C LDce; y D yp

1e C LDse;


where x D .x1; : : : ; xn/t, y D .y1; : : : ; yn/

t, e D .1; 1; : : : ; 1/t. Here, (xi; yi) denotesthe position of CoM for segment i, where i D 1; : : : ; n, n denoting the number ofsegments.

Dc D diag.cos �1; : : : ; cos �n/; Ds D diag.sin �1; : : : ; sin �n/

and

L D

26666666664

r1 0 0 0 0 0 0

l1 r2 0 0 0 0 0

l1 l2 r3 0 0 0 0

l1 l2 l3 r4 0 0 0

l1 l2 l3 l4 r5 0 0

l1 l2 l3 l4 l5 r6 0l1 l2 l3 0 0 0 r7

37777777775:

The position of CoM of the whole system (MX, MY) can likewise be written inmatrix-vector form as MX D mtx and MY D mty, where mt D .m1; : : : ;mn/. Hencethe relations, using mte D M,

MX D mtx D Mxp1 C mtLDce and MY D mty D Myp

1 C mtLDse:

The distal end of Segment 6 of the chain of segments, has co-ordinates

xd6 D xp

1 C6X

iD1li cos �i and yd

6 D yp1 C

6XiD1

li sin �i:

The segments are ordered so that L is lower triangular despite having multiplebranches.

1.2.2 The Topology

To describe the topology of the body, each joint is labelled with a number, k, k D1; : : : ; j where j D n C 1 for a body with no loops, in other words, a tree-structuredbody. Some of these joints will be in contact with the ground while some are freeor constrained on a curve. This allows for externally applied forces on any joint, inparticular, from the infinite mass ground. The proximal incidence matrix is a j � nmatrix Ap where

Apki D

(�1; if segment i has proximal end at joint k;

0; otherwise:

8 M. Tan et al.

The distal incidence matrix Ad is similarly defined,

Adki D

(1; if segment i has distal end at joint k;

0; otherwise:

These two matrices define the topology of the body and Ap C Ad defines the vertex-arc, or joint-directed segment incidence matrix of a digraph (directed graph). Thejoint-external contact incidence matrix B (j � e) is defined as

Bki D(1; if joint k contacts the ground at external contact i;

0; otherwise;

where e is the number of external contacts. The possible contacts considered are atjoints numbered 1, 2, 6 and 7 which are the toes and heels of the body, where formodel simplicity, the heels are a rigid extension of the ankles.

Suppose we have a joint k, which has two proximal ends, namely i and b incidenton it and one distal end a incident on it. An external force f e

k acts on the joint. Thereaction forces on segment i come from joints k proximal and .k C 1/ distal and aredenoted f p

i and f di for the proximal and distal forces respectively. It is these forces

which supply the rotational and translational motions to segment i and are given by

f ek D

�f exk

f eyk

�; f p

i D�

f pxi

f pyi

�; f d

i D�

f dxi

f dyi

�:

The balance of forces at any joint k takes the form, “the sum of all reaction forcesat joint k equals the external force at joint k”. From Fig. 1.3,

�f pxi

f pyi

�C�

f pxb

f pyb

�C�

f dxa

f dya

�C��f ex

k

�f eyk

�D 0:

Using the incidence matrices, these equations for all joints can be combined intomatrix equations:

�Apf px C Adf dx C B.�f ex/ D 0;�Apf py C Adf dy C B.�f ey/ D 0;

where f px is a vector of proximal x-component reaction forces for each segmentand f dx is the corresponding distal x component reaction forces, similarly the y-components. The vectors f ex and f ey are the joint external forces. The two equations

above can be simplified to�

A D ��Ap j B j Ad� �

Af x D 0;

Af y D 0;


Fig. 1.3 Forces on joint k

where the proximal, distal and external forces are ordered such that

f x D24 f px

�f ex

f dx

35 ; f y D

24 f py

�f ey

f dy

35 :

When there are no external contacts at any joint, the external forces are zero.

1.2.3 Translational Equations of Motion

The translational equations of motion for an n-segment model can be derived bydifferentiating the equations of positions of the CoM of each segment twice to obtainthe acceleration, multiplied by the mass and equating to the forces on each segment.

Let Pxp1 D u1, so Rxp

1 D Pu1 and Pyp1 D v1, Ryp

1 D Pv1. The translational equations canbe written as

�mPu1 C Jy P! C Sf x D �Jx!2;

�m Pv1 � Jx P! C Sf y D gm � Jy!2:

10 M. Tan et al.

To cater for both contact and free flight dynamics, where free flight dynamics isdescribed when there is no external contact to the ground, during contact at .xp

1; yp1/,

.Pu1; Pv1/ is zero and non-zero when not in contact. Hence the translational equationsduring contact are:

Jy P! C Sf x D �Jx!2;

�Jx P! C Sf y D gm � Jy!2:

In the case of non-contact, we would require the dynamic equations, Rxp1 D Pu1 and

Ryp1 D Pv1 to be included.

We have defined: � D .�1; : : : ; �n/t, ! D P� , P! D R� , !2 D .!21 ; : : : ; !

2n /

t,S D ŒIn; 0; In�.

Note that Jx D DmLDc and Jy D DmLDs, where Dm D diag.m/.

1.2.4 Rotational Equations of Motion

For the general case, the moment equations of each segment i about the segmentCoM are given by

Ik P!i D �pi C �d

i C f pxi ri sin �i � f dx

i .li � ri/ sin �i � f pyi ri cos �i C f dy

i .li � ri/ cos �i;

where i D 1 to n. The moment equation can similarly be expressed in matrix form as

J P! C Mxf x C Myf y D T�;

where J D diag.I1; I2; : : : ; In/,

Mx D Ds��Dr j 0 j Dl � Dr

�,

My D Dc�

Dr j 0 j �.Dl � Dr/�,

Dr D diag.r1; r2; : : : ; rn/, Dl D diag.l1; l2; : : : ; ln/.

The external forces do not appear explicitly in these equations as they act on thejoints and transfer the forces to the segments attached to the joint.

The vector � is a vector of the proximal torques appropriately ordered. Thesetorques can be considered as making the angle between the segments larger orsmaller. It is possible for a torque to act on a segment separated by other segments.In this case, we assume that only one torque acts between two segments about acommon joint and that the torques are given functions of time.

A consistent notation to specify the torques needs to be established and is givenby the matrix T, which is a matrix of zeros, ones and negative ones that describeswhich segments have torques acting on them. A positive torque on the distal end of asegment contributes a negative angular acceleration to the segment, while a positivetorque on the proximal end of a segment contributes a positive angular acceleration.


This holds true up till Segment 6, as Segment 7 is part of a multiple branch at joint4, together with Segment 3 and Segment 4 (Fig. 1.2). At joint 4 (hip joint), we havea torque between Segment 3 and Segment 7, and another between Segment 4 andSegment 7, which will be opposite of each other.

The matrix T with a torque acting between Segment 6 and the external world isgiven by

T D

seg1seg2seg3seg4seg5seg6seg7

�1 �2 �3 �4 �5 �6 �7

Œ �1 0 0 0 0 0 0 �

Œ 1 �1 0 0 0 0 0 �

Œ 0 1 0 0 0 �1 0 �

Œ 0 0 �1 0 0 0 1 �

Œ 0 0 1 �1 0 0 0 �

Œ 0 0 0 1 �1 0 0 �

Œ 0 0 0 0 0 1 �1 �

:

Note that the sum of the each column and row has to be zero with the exception of�5 and Segment 6 in the case of one contact at Segment 1. �1 acts between Segment1 and the external world and there is no torque between Segment 6 and the externalworld.

1.2.5 Dynamics of the Model

The equations for the second order variables and forces are looked at, taking accountof variables which are zero for a time interval.

The complete equations where eight rows of the equations are included or not,to specify the cases, are presented. Hence all dynamic variables and all forces areincluded, whether zero or not. But first, the two row vectors lt

s and ltc are defined so

as to relate distal Segment 6 (toe, or joint 7) to proximal Segment 1 (toe or joint 1),or proximal Segment 2 (ankle (heel) or joint 2),

1lt D Œl1; l2; l3; l4; l5; l6; 0�; or 2lt D Œ0; l2; l3; l4; l5; l6; 0�;

1lts D 1ltDs; or 2lt

s D 2ltDs;

1ltc D 1ltDc; or 2lt

c D 2ltDc;

where for proximal Segment 1

�xd6

yd6

�D�

xp1

yp1

�C�1lt

ce

1ltse

�;

� Pxd6

Pyd6

�D� Pxp

1

Pyp1

�C��1lt

s!

1ltc!

�;

and renaming the velocities as u and v, scripted appropriately,

12 M. Tan et al.

� Pud6

Pvd6

�D� Pup

1

Pvp1

�C��1lt

s P!1lt

c P!�

C��1lt

c!2

�1lts!

2

�:

Similarly for distal Segment 6 measured from proximal Segment 2.The complete equations for non-heel contact are:

P� D !;

� Pxp1

Pyp1

�D�

u1v1

�;

� Pxd6

Pyd6

�D�

u6v6

�;

n 1 1 1 1 n n � 1 n n � 1

n Œ J 0 0 0 0 Mpx Mdx Mpy Mdy �

1 Œ lts �1 0 1 0 0t 0t 0t 0t �

1 Œ �ltc 0 �1 0 1 0t 0t 0t 0t �

.a/ 1 ŒŒ 0t 1 0 0 0 0t 0t 0t 0t ��

.a/ 1 ŒŒ 0t 0 1 0 0 0t 0t 0t 0t ��

.b/ 1 ŒŒ 0t 0 0 1 0 0t 0t 0t 0t ��

.b/ 1 ŒŒ 0t 0 0 0 1 0t 0t 0t 0t ��

n Œ Jy �m 0 0 0 I I 0 0 �

n � 2 Œ 0 0 0 0 0 �Ap Ad 0 0 �

n Œ �Jx 0 �m 0 0 0 0 I I �

n � 2 Œ 0 0 0 0 0 0 0 �Ap Ad �

266666666666664

P!Pu1Pv1Pu6Pv6

f px

f dx

f py

f dy

377777777777775

D

26666666666666666664

T�

�ltc!

2

�lts!

2

0

0

0

0

�Jx!2

0

�Jy!2 C mg0

37777777777777777775

:

The above can be adjusted further to accommodate different cases such as thesingle and double support phase.

The cases are broken down into:

• Case 1 (double-support phase) where Case 1A considers a scenario of twoexternal contacts at joint 2 and 7, and Case 1B considers a scenario of twoexternal contacts at joint 1 and 6

• Case 2 (free flight) considers a scenario where there is no contact with the ground• Case 3 (single support phase with one contact at swing heel) where Case 3A

considers a scenario of one contact point at proximal Segment 2 and Case 3Bconsiders a scenario of one contact point at proximal Segment 6

• Case 4 (single support phase with one contact at stance toe) where Case 4Aconsiders a scenario of one contact point at proximal Segment 1 and Case 4Bconsiders a scenario of one contact point at proximal Segment 6.

These cases, showing how the matrices are modified depending whether theweight is on the heel or toe of the foot can be found in the Appendix.


1.3 Methods and Procedures

This study applied the conventional inverse dynamics (Winter 2009) to obtain initialestimates of joint torques which were used in dynamic optimisation. Dynamicoptimisation approach was adopted to compute the trajectory of joint torques and toproduce the observed motion. Since we were only concerned with the single supportphase and double support phase of walking and would be considering Segment 1 tobegin from stance foot, hence our focus would be on Case 4A for single supportphase and Case 1B for double support phase.

An alternative formulation of the conventional method of inverse dynamics wasused to determine the initial estimates of joint torques, and can be found in detaillater in this section. It is used so that a more realistic set of joint torques historiesis generated. Using torque histories specific to the movement pattern of the subjectimproves the convergence of the optimisation (Chao and Rim 1973). The bounds foreach control are determined using the maximum and minimum estimates obtainedfrom the inverse dynamics approach. Since these estimates provided approximatetorque histories that were specific to the movement pattern, it would ensure that theoptimised torque trajectories are realistic.

For the present study, 18 states (x D Œx1; x2; : : : ; x18�>), 15 system parameters(z D Œz1; z2; : : : ; z15�>), and 7 controls (� D Œ�1; �2; : : : ; �7�

>) namely joint torques,were set up in MISER3.3 (Jennings et al. 2000). The 18 states consist of the angulardisplacements from Segment 1 to Segment 7 (xi D �i; i D 1; : : : ; 7), angularvelocity (xi D �i; i D 8; : : : ; 14), coordinate and velocity of proximal end ofsegment one, (.x15; x16; x17; x18/ D .xp

1; yp1; Pxp

1; Pyp1/). The system parameter consists

of the initial segment angular orientation (�i.0/ D zi; i D 1; : : : ; 7), initial angularvelocity at start of single support phase (!i.0/ D z7Ci) and z15 is the step lengthwhich is twice the distance of initial distance of xp

1.0/ and xd6.0/ and hence dependent

on (z1; : : : ; z6). Placing tight bounds on the system parameters allows the initialconditions to vary by a small amount about the initial values of the data.

The variables �1; �2; �3 describe the angular displacements of the stance legbeginning with the foot, shank, and thigh segments respectively; �4; �5; �6 describethe angular displacements of the swing leg from thigh, shank and foot segmentsrespectively and �7 describes the angular displacement of the trunk segment.!1; : : : ; !7 are the segments’ corresponding velocities; and (xp

1; yp1) are the coor-

dinates of the proximal end of segment one (toe of stance foot) which remainsstationary on the ground during one step of the walk cycle.

There were two parts involved in the experiment to simulate normal walking andobtaining more precise joint torque estimates. For the first part, forward dynamicsof seven segment model during the single support phase (Case 4A) was optimisedfor the joint torques and initial values of � and ! such that computed �.�; z; t/trajectories produced motion similar to normal walking. In doing so, the body hadto be kept upright and from falling under gravity. This was done by keeping they-coordinate CoM close to the initial y-coordinate CoM (at t D 0). Our initialobjective function is thus given by,

14 M. Tan et al.

G0.�; z/ DZ T1

0

.CoMypos � CoMyinit/2 dt

where T1 (D 0:386 s) is the duration of the single support phase, CoMypos is thecenter of mass of y-coordinate, a function of �.�; z; t/, and CoMyinit is the initialcenter of mass of y-coordinate, a function of z, as calculated by MISER3.3.

The objective function is subject to constraints in the canonical form:

Gk.u; z/ D �k.x.tk; z//CZ tk

0

gk.t; x.t/;u.t/; z/ dtD��0; k D 1; : : : ; ngc;

where ngc is the total number of canonical constraints, and tk 2 .0; tf � is aknown constant and is referred to as the ‘characteristic time’ associated with theconstraint Gk. All-time constraints h.t; x;u; z/ � 0 and constraints involving systemparameters gk.z/ as well, are converted by MISER3.3 to canonical constraints. SeeJennings et al. (2000) for more details.

The objective function is subject to the following constraints:

• Two terminal state equality constraints at the end of a single support phase (T1 D0:386), to position the swing foot such that the swing toe is pivoting upwards andthe heel strikes the ground, marking the end of single support phase, these aregiven by,

g1 � 0; �1.�.T1/; z/ D a211 C a212 D 0;

where; a11 D 0:4794�5X

iD1li cos.�i.T1//C 0:1 cos.�6.T1/� 1:57/;

a12 D �5X

iD1li sin.�i.T1//C 0:1 sin.�6.T1/ � 1:57/I

g2 � 0; �2.�.T1/; z/ D a221 C a222 D 0;

where; a21 D 0:7039�6X

iD1li cos.�i.T1//;

a22 D 0:0751�6X

iD1li sin.�i.T1//:

Remark. The sum of squares of two constraints equal to zero was used toreduce the total number of constraints and to allow more leeway in reducingthe individual constraints to zero.


• Two all-time constraints on the swing toe and swing heel, such that the swingfoot does not penetrate the ground, are given by,

�3 � 0; h3.�.t/; z; t/ D6X

iD1li sin.�i.t// � 0; and

�4 � 0; h4.�.t/; z; t/ D5X

iD1li sin.�i.t//C 0:1 sin.�6.T1/ � 1:57/ � 0:

• Two all-time constraints on knee, to prevent the knee from hyperextending, aregiven by,

�5 � 0; h5.�.t/; z; t/ D �3.t/ � �2.t/ � 0; and

�6 � 0; h6.�.t/; z; t/ D �4.t/ � �5.t/ � 0:

Remark. There is a possibility that segmented rigid body models moving underthe effect of gravity can spin past a natural limit during optimisation computationin the line search. Said et al. (2006) found that hyperextension modelling done“automatically” in the dynamics, introduced a large force to be exerted to restorethe joint if it gets close to hyperextension. However, this was not used in thisresearch as it creates “stiff” differential equations.

• An all-time constraint on trunk, to prevent the trunk from falling forward orbackward, is given by,

�7 � 0; h7.�.t/; z; t/ D 0:32112 � .�7.t/ � 1:5376/2 � 0:

• Four all-time constraints on ankles, to prevent the ankle from hyperextending,are given by,

�8 � 0; h8.�.t/; z; t/ D �1.t/ � �2.t/ � 0;

�9 � 0; h9.�.t/; z; t/ D �2.t/ � �1.t/C 1:57 � 0;

�10 � 0; h10.�.t/; z; t/ D �6.t/ � �5.t/ � 0; and

�11 � 0; h11.�.t/; z; t/ D �5.t/ � �6.t/C 1:57 � 0:

• Two constraints involving system parameters, to ensure stance foot is flat on theground and to position swing foot at time t D 0, is given by.

gz1.z/ D l1 sin.z1/C 0:1 sin.z1 C 1:57/ D 0; and

gz2.z/ D �0:7647�6X

iD1li cos.zi/ D 0:

16 M. Tan et al.

The optimisation process was usually CPU-time-consuming with numerousfailures along the way. At each failure, the constraints were checked to determineif they were satisfied before the optimisation was restarted. It was noted that thelarge number of ankle constraints made it difficult to obtain satisfactory resultshence a method proposed by Rehbock et al. (1996) was to add the constraint tothe objective function as a penalty on being negative. The new objective functionwas thus given by,

ankle1 D�1.�; z; t/ � �2.�; z; t/; if ™1 � ™2 < 0;

0; if ™1 � ™2 � 0;

ankle2 D�2.�; z; t/ � �1.�; z; t/C 1:57; if ™2 � ™1 C 1:57 < 0;

0; if ™2 � ™1 C 1:57: � 0;

ankle3 D�6.�; z; t/ � �5.�; z; t/; if ™6 � ™5 < 0;

0; if ™6 � ™5 � 0;

ankle4 D�5.�; z; t/ � �6.�; z; t/C 1:57; if ™5 � ™6 C 1:57 < 0;

0; if ™5 � ™6 C 1:57 � 0;

G0.�; z/DZ T1

0

�.CoMypos � CoMyinit/

2 C 1;000.jankle1j C jankle2j C jankle3j C jankle4j/� dt

where (ankle1; ankle2; ankle3; ankle4) are the ankle constraints and only the con-straints that are not satisfied are taken in the objective function. The final set ofcontrols from this optimisation was then used as the initial joint torques estimatesfor the optimisation studies in the second part of the experiment.

In the second part of the experiment, the second phase of the walk cycle, doublesupport phase (Case1B), was incorporated with the first part for the rest of the timeinterval (T D 0:486 s). Forward dynamics of seven segment model was optimisednow to simulate normal walking for both single support and double support phaseof walking. However, as the forward dynamics changed from single support phaseto double support phase, the collision of the foot with the ground resulted in jumpconditions on the model velocities (Hardt et al. 1999). The dynamics in the modelcomputed in MISER3.3 allowed the state to jump at particular times �j, hence thestate equations have a form:

Px.t/ D

8ˆ<ˆ:

f 1.t; x;u; z/; t 2 Œts; �1/; x.ts/ D x0.z/;f 2.t; x;u; z/; t 2 Œ�1; �2/; x.�1/ D h1.x.��

1 /; z/;:::

f p.t; x;u; z/; t 2 Œ�p�1; �p/; x.�p�1/ D hp�1.x.��p�1/; z/;

where x.�j/ are the new states at time �j. As the ankle constraints were satisfiedin the first part of the experiment, they were removed from the objective function.


Thus, the objective function is now similar to the original function, given by,

G0.�; z/ DZ Tf

0

.CoMypos � CoMyinit/2 dt

with the following state equations:

Px.t/ D

f 1.t; x;u; z/; t 2 Œ0;T1/; x.0/ D x0.z/;f 2.t; x;u; z/; t 2 ŒT1;Tf /; x.T1/ D h1.x.T�

1 /; z/;

where T1 .D 0:386 s/ is the duration of the single support phase, Tf (D 0:486 s) isthe duration of a step (single support and double support phase) and h1.x.T�

1 /; z/defines the new states governing the start of double support phase. However, asonly the angular velocities experience jumps, angular displacements and proximalSegment 1 position and velocity are defined as xi.T1/ D xi.T�

1 /; i D 1; : : : ; 7,and i D 15; : : : ; 18, while xi.T1/; i D 8; : : : ; 14 are newly defined. The objectivefunction is now subjected to similar all-time constraints as before except it appliesnow to the full time interval from T D 0 to T D Tf .0:486 s/. It is however subjectto different terminal constraints:

• A terminal constraint at the end of single support phase (T1 D 0:386 s) todetermine heel-strike of swing foot, such that xd

6.Tf / is twice the step length ofxp1.0/ and xd

6.0/ at initial time 0, is given by,

g1 � 0; �1.�.T1/; z/ D a211 C a212 D 0;

where; a11 D .z15 � 0:2368/�5X

iD1li cos.�i.T1//C 0:1 cos.�6.T1/� 1:57/;

a12 D �5X

iD1li sin.�i.T1//C 0:1 sin.�6.T1/� 1:57/:

Remark. Foot length was defined to be from heel to toe (D 0:2368m) andwas estimated from segment parameters, Segment 1/6 and length of heel (fromankle), since the foot segment was assumed to take on the shape of a right-angledtriangle.

• Two terminal constraints at the end of double support phase (Tf D 0:486 s), suchthat the swing foot is on the ground and does not slide, are given by,

g2 � 0; �2.�.Tf /; z/ D b211 C b212 D 0

where; b11 D .z15 � 0:2368/�5X

iD1li cos.�i.Tf //C 0:1 cos.�6.Tf / � 1:57/;

18 M. Tan et al.

b12 D �5X

iD1li sin.�i.Tf //C 0:1 sin.�6.Tf / � 1:57/I

g3 � 0; �3.�.Tf /; z/ D b221 C b222 D 0

where; b21 D z15 �6X

iD1li cos.�i.Tf //;

b22 D �6X

iD1li sin.�i.Tf //:

• A terminal constraint at the end of double support phase (Tf D 0:486 s) onthe trunk angular displacement, as in order to obtain periodicity of the walkingmotion, the trunk itself must have a periodic motion, thus the constraint is givenby,

g4 � 0; �4.�.Tf /; z/ D z7 � �7.Tf / D 0:

• Constraints on system parameters are similar as in single support phase, with aslight change to the constraint that positions the swing foot which is given by,

gz2.z/ D z15 �6X

iD1li cos.zi/ D 0:

1.3.1 Inverse Analysis

In order to solve for initial joint torques for dynamic optimisation, the conventionalmethod of inverse dynamics (Winter 1990) was adopted. The dynamics equationin Case 4A and Case 1B were rearranged to solve for joint moments and reactionforces from kinematic data of segments.

Rearranging Case 4A:

2666664

�T Mpx Mdx Mpy Mdy

0 I I 0 0

0 �Ap Ad 0 0

0 0 0 I I0 0 0 �Ap Ad

3777775

2666664

�

f px

f dx

f py

f dy

3777775

D

2666664

�J P!�Jx!2 � Jy P!

0

�Jy!2 C mg C Jx P!0

3777775:


The coefficient matrix is now a block upper triangular and � was obtained easilyat each discrete data time, ti (static inverse analysis), after solving for reactionforce vectors. As the matrix is square and invertible, a unique set of torques wascomputed. This meant that for n segments, there should be n torques betweensegments. However, this was not the same for Case 1B.

Even though the dynamic equations in Case 1B is a square matrix, whenrearranged, it becomes a non-invertible matrix with two rows of zeros.

Rearranging Case 1B:

26666666664

�T 0 0 Mpx Mdx Mpy Mdy

0t 0 0 0t 0t 0t 0t

0t 0 0 0t 0t 0t 0t

0 0 0 I I 0 0

0 e5 0 �Ap Ad 0 0

0 0 0 0 0 I I0 0 e5 0 0 �Ap Ad

37777777775

26666666664

�

�f ex6

�f ey6

f px

f dx

f py

f dy

37777777775

D

26666666664

�J P!�4lt

c!2 � 4lt

s P!�4lt

s!2 C 4lt

c P!�Jx!2 � Jy P!

0

�Jy!2 C mg C Jx P!0

37777777775

:

As there were more unknowns than equations, the system is an underdeterminedsystem and a generalised inverse method was adopted to find a set of suitabletorques. This was done by considering the equations to be of the form Ax D y,where we simply ignored the two zero row equation (the two entries in the RHSshould be zero), and compute

x D At.AAt/�1y DW ACy;

where AC is called the pseudo-inverse of A. The solution obtained was a uniquesolution being the smallest normed solution from the infinite number of solutionsto an underdetermined system. However, we only required an initial estimate of thejoint torques so that a more realistic set of joint torques could be generated fromdynamic optimisation. If the external forces were known from force plate data, thecorrect torques and forces could be uniquely computed.

1.4 Numerical Results

Experiment Main solved the problem by minimising the objective, G0, to minimisey-CoM displacement through the entire walk cycle subjected to constraints. Theseconstraints include four terminal state equality constraints, one at the end of a singlesupport phase on swing foot heel (�1), and three at the end of double support phase(�2; �3; �4), two on swing foot and one on trunk angular displacement. Nine all-timeconstraints (hi, where i D 5; : : : ; 13) on swing toe and heel, knees, trunk and anklesand two system parameter constraints (gz1; gz2) to position stance foot and swingfoot at t D 0.

20 M. Tan et al.

Figure 1.4 shows the resulting stick diagram of a 2D human walking on levelground, including the motion in single support phase and double support phase.The algorithm produced 137 time points for the optimised simulation but for clarity,only 21 were chosen at equal time intervals (t D 0:025 s) to depict the movement forcomparative purposes. A linear interpolation was done on the .x; y/ data for plottingpurposes to evenly distribute the intervals as the single support phase held a largerpercentage of the walk cycle time than the double support phase. Figure 1.5 plots

1 0.8 0.6 0.4 0.2 0 0.2 0.4 0.6 0.8 10.2

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

met

res

meters

Fig. 1.4 Simulation of optimised walking motion – experiment main

1 0.8 0.6 0.4 0.2 0 0.2 0.4 0.6 0.8 10.2

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

0.486red is swing

met

res

meters

Fig. 1.5 Start and end of walk cycle – experiment main. Dotted figure is the initial configurationwith final swing toe and initial stationary toe at the same position


Table 1.1 Constraint values and corresponding � value

Constraints G1 G2 G3 G4

Values 4:56e�8 6:60e�7 2:92e�7 4:37e�7Constraints G5 G6 G7 G8 G9 G10 G11 G12 G13

g�.h/ 0 �1:15e�8 0 0 0 0 0 0 0

� value 1e�6 9:86e�7 1e�6 1e�6 1e�6 1e�6 1e�6 1e�6 1e�6

the first and last position of the walk cycle, and showed that periodicity conditionsmight have to be worked on for the last position to more accurately mirror the startposition.

As initial joint moment estimates derived from an alternate inverse dynamicsformulation were not very close to the real optimum, a large number of iterationswere involved. The computation was eventually able to come to a satisfactorysolution, satisfying constraint conditions. The final values of the canonical con-straints are given in Table 1.1. G1 to G4 are defined as terminal time constraintswhich were required to converge to 0. Their values were small enough to beapproximated to 0 and considered satisfied by MISER3.3. G5 to G13 are all timeconstraints, and had to satisfy the algorithm. Constraint value of 0 occurs whenh � � .D 10�6/ ) g�.h/ D 0 and are satisfied. G6 constraint had value h ash � ��, however it was still considered satisfied. The g� value of G6 constraint notbeing 0 could be explained from when swing heel struck the ground at t D 0:386 s tot D 0:486 s, the y-position of swing heel had to be 0 or in this case approximately 0.

A closer look at the forces acting on the hip can be seen in Fig. 1.6. It wasobserved that forces from the two thighs (Segment 3 and Segment 4) had to equateto balance out the force from Segment 7. A larger force is required in the horizontaldirection from t D 0:386 s onwards as it prevented the body from continuallymoving forward during double support phase.

External forces on swing ankle occurred at the start of double support phasewhen swing heel struck the ground. During double support phase, the swingankle experienced three forces acting on it, namely distal and proximal forcefrom Segment 5 and Segment 6 respectively, and external force from the ground.Figure 1.7 presents the forces acting on the ankle during double support. Externalforces acting on the swing ankle were observed to be equal and opposite to theproximal and distal forces in both x � y direction. It was also observed that most ofthe forces were contributed from Segment 5 rather than Segment 6 mainly becauseSegment 5 was carrying the main weight of the body while Segment 6 only had thefoot.

Figure 1.8 depicts the components of the vertical and horizontal forces on thestance toe for a one step cycle. As no ground force plate was use in this experiment,ground reaction force cannot be accurately predicted. However, a vertical reactionforce was observed and a 9th order polynomial fit was plotted against it whichhad a familiar double-peak, also known as the “M-shaped” pattern. Distributionof the model’s weight from the stance toe to the swing ankle could be seen at

22 M. Tan et al.

0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.52000

1500

1000

500

0

500

1000

1500

Time(s)

Hor

izon

tal f

orce

(N

)Forces (N) on hip

x3dx4px7p

0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5800

600

400

200

0

200

400

600

Time(s)

Ver

tical

forc

e (N

)

y3dy4py7p

Student Version of MATLAB

Fig. 1.6 Forces on hip at distal and proximal end of x and y at segments 3, 4 and 7. (xd3; y

d3)

denotes (x; y)-coordinate of distal segment 3, (xp4; y

p4) denotes (x; y)-coordinate of proximal segment

4, (xp7; y

p7) denotes (x,y)-coordinate of proximal segment 7

Fig. 1.7 External forcesfrom angle to ground duringdouble support phase.(fe6x; fe6y) denotes externalforces acting on(x; y)-coordinate of proximalsegment 6 (ankle)

0.38 0.4 0.42 0.44 0.46 0.48 0.5400

200

0

200

400

600

800

1000

Time(s)

Ext

erna

l For

ce (

N)

External Force on swing ankle (N) during double support

fe6xfe6y


0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4100

0

100

200

300

400

500

600

700

800

Time(s)

For

ce (

N)

Force on stance toe (N) during single support

x1py1p9th order polyfit

0.38 0.4 0.42 0.44 0.46 0.48 0.51400

1200

1000

800

600

400

200

0

200

400

Time(s)

For

ce (

N)

Force on stance toe (N) during double support

x1py1p

Fig. 1.8 Forces on stance toe (xp1; y

p1). (xp

1; yp1) denotes (x; y)-coordinate of proximal segment 1

(toe)

24 M. Tan et al.

0 0.1 0.2 0.3 0.4150

100

50

0

50

100Torque on segments, model 4

Tor

que

(Nm

)

0.4 0.45 0.5200

0

200

400

600

800

1000Torque on segments, model 1

Tor

que

(Nm

)

0 0.1 0.2 0.3 0.4250

200

150

100

50

0

50

100

Forces on segments, model 4

Time(s)

For

ces

(N)

0.4 0.45 0.51000

800

600

400

200

0

200

Forces on segments, model 1

Time(s)

For

ces

(N)

1 2 3 4 5 6 7

Fig. 1.9 Torques and forces acting on segments

heel-strike from end of single support phase, through to double support phase. Ahigh negative horizontal force kept the stance toe in position and prevented it frommoving especially during double support phase.

Figure 1.9 presents the torques and forces acting on segments to produce angularacceleration. The torques and forces acting on each segment, including externalforces, were computed from the dynamics and plotted. It was observed that thetorques and forces on each segment for each phase were similar in pattern but wereopposite in direction to each other.

The final results of MISER3.3 and the data were plotted out and compared.Figure 1.10 presents the .x; y/ center of mass (CoM) trajectories and velocities,while Fig. 1.11 presents the segment angular displacements of MISER3.3 plottedagainst the ones from the original data. Slight changes were observed between theoriginal angular displacement data and the optimised results, however the patternremains consistent. This was observed for the CoM trajectories and velocities as


0 0.1 0.2 0.3 0.4 0.50.4

0.2

0

0.2

0.4

Hor

izon

tal d

ispl

acem

ent (

m)

CoM X

0 0.1 0.2 0.3 0.4 0.50.5

0

0.5

1

1.5

2

Time(s)

Hor

izon

tal v

eloc

ity (

ms

1 )

CoM X vel

0 0.1 0.2 0.3 0.4 0.51.1

1.12

1.14

1.16

1.18

Ver

tical

dis

plac

emen

t (m

)

CoM Y

0 0.1 0.2 0.3 0.4 0.51

0.5

0

0.5

1

Time(s)

Ver

tical

vel

ocity

(m

s1 )

CoM Y vel

MISER3data

Fig. 1.10 Optimised and data CoM displacement and velocity

well except for the sudden jump in velocities seen in the MISER3.3 data due to thejump condition implemented when the heel struck the ground to account for abruptvelocity change.

Comparisons between the results obtained by MISER3.3 and inverse analysiswere made as well. The optimised set of joint torques obtained from MISER3.3is illustrated in Figs. 1.12 and 1.13, and plotted against the initial joint torques’estimates. It was noted that the joint torques in MISER3.3 had to be made piecewiseconstant for computation due to the jump condition occurring at t D 0:386 s.Piecewise linear approximation was not possible as simulation could not convergeto �.0:386�/ D �.0:386C/. Inverse analysis estimated joint torques for singlesupport and double support phases separately, while MISER3.3 considered the onestep walk cycle when solving for joint torques, thus requiring the jump condition.As observed, though the joint torques required to reproduce the walking motionin Fig. 1.4, as estimated by MISER3.3, were different to those obtained by theconventional method of inverse dynamics, they however still followed a similarpattern. Figure 1.14 illustrates the vertical reaction force, obtained using the resultsof MISER3.3 and inverse analysis, on the stance toe during the single support phase.A double peak pattern that was noticed in Fig. 1.13 was also observed when a 9thorder polynomial was fitted to data obtained by inverse analysis.

26 M. Tan et al.

0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.51

2

3S

tanc

e fo

ot

0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.50

1

2

Sta

nce

shan

k

0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.51

1.5

2

Sta

nce

thig

h MISER3data

0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.54.5

5

5.5

Sw

ing

thig

h

0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.52

4

6

Sw

ing

shan

k

0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.50

5

10

Sw

ing

foot

0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.51

1.5

2

HA

T

Time (s)

Fig. 1.11 Optimised (MISER3.3) and data segment angular displacement (rad) of a one step cycle

1.5 Conclusions

In this paper, we have proposed a mathematical method to analyse human walkingbehaviour using dynamic optimisation. A main advantage of the method developedin the paper, is that it works well even for a complex movement that involves a


0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4500

0

500S

tanc

e to

e

0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4500

0

500

Sta

nce

ankl

e

0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4200

0

200

Sta

nce

knee

MISER3data

0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.450

0

50

Sw

ing

knee

0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.410

0

10

Sw

ing

ankl

e

0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4200

0

200

Hip

0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4100

0

100

Hip

Time(s)

Fig. 1.12 Joint torque trajectories of optimised (MISER3.3) and inverse dynamics (data) for singlesupport phase

change in the dynamics from single support phase to double support phase. Theoverall research was able to simulate normal walking motion for a full walk cycle,based on the model developed using MISER3.3.

28 M. Tan et al.

0.38 0.4 0.42 0.44 0.46 0.48 0.54000

2000

0S

tanc

e to

e

0.38 0.4 0.42 0.44 0.46 0.48 0.54000

2000

0

Sta

nce

ankl

e

0.38 0.4 0.42 0.44 0.46 0.48 0.54000

2000

0

Sta

nce

knee

0.38 0.4 0.42 0.44 0.46 0.48 0.5500

0

500

Sw

ing

knee

MISER3data

0.38 0.4 0.42 0.44 0.46 0.48 0.51

0

1

Sw

ing

ankl

e

0.38 0.4 0.42 0.44 0.46 0.48 0.52000

0

2000

Hip

0.38 0.4 0.42 0.44 0.46 0.48 0.51000

0

1000

Hip

Time(s)

Student Version of MATLAB

Fig. 1.13 Joint torque trajectories of optimised (MISER3.3) and inverse dynamics (data) fordouble support phase


Fig. 1.14 Vertical reactionforce of optimised(MISER3.3) and inversedynamics (data) during singlesupport phase

0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4100

0

100

200

300

400

500

600

700

800

900

Time(s)

For

ce (

N)

on y

1p

MISER3

9th order polyfit

data

9th order polyfit

Appendix A: Model Cases

1 Case 1A: Two External Contacts, at Joint 2 and Joint 7

The dynamic equations are:

24

P�Pxp1

Pyp1

35 D

24 !

u1v1

35 ;

n 1 1 1 1 n � 1 n � 1 n � 1 n � 1

n Œ J 0 0 0 0 Mpx Mdx Mpy Mdy �

1 Œ 3lts 0 0 0 0 0t 0t 0t 0t �

1 Œ 1lts �1 0 0 0 0t 0t 0t 0t �

1 Œ �3ltc 0 0 0 0 0t 0t 0t 0t �

1 Œ �1ltc 0 0 �1 0 0t 0t 0t 0t �

n Œ Jy �m 0 0 0 I I 0 0 �

n � 2 Œ 0 0 e1 0 0 �Ap Ad 0 0 �

n Œ �Jx 0 0 �m 0 0 0 I I �

n � 2 Œ 0 0 0 0 e1 0 0 �Ap Ad �

266666666666664

P!Pu1

�f ex2

Pv1�f ey

2

f px

f dx

f py

f dy

377777777777775

D

266666666666664

T�

�3ltc!

2

�1ltc!

2

�3lts!

2

�1lts!

2

�Jx!2

0

�Jy!2 C mg0

377777777777775

:

Note that this is a square matrix of order 5n. There are n C 2 second derivativesand 4n � 2 force components. Unfortunately the order 2n � 1 zero-one blockson the diagonal are not invertible, so the usual block inversion cannot be fol-lowed.

30 M. Tan et al.

2 Case 1B: Two External Contacts, at Joint 1 and Joint 6


P� D !;

� Pxp1

Pyp1

�D�

u1v1

�D 0;

n 1 1 n n � 2 n n � 2

n Œ J 0 0 Mpx Mdx Mpy Mdy �

1 Œ 4lts 0 0 0t 0t 0t 0t �

1 Œ �4ltc 0 0 0t 0t 0t 0t �

n Œ Jy 0 0 I I 0 0 �

n � 2 Œ 0 e5 0 �Ap Ad 0 0 �

n Œ �Jx 0 0 0 0 I I �

n � 2 Œ 0 0 e5 0 0 �Ap Ad �

26666666664

P!�f ex

6

�f ey6

f px

f dx

f py

f dy

37777777775

D

26666666664

T�

�4ltc!

2

�4lts!

2

�Jx!2

0

�Jy!2 C mg0

37777777775

:

Note that this is a square matrix of order 5n � 2. There are n second derivativesand 4n � 2 force components.

3 Case 2: No Contacts

Now the forces at the joints 1 and 7, f px1 , f py

1 , f dx6 , f dy

6 , are zero. The dynamicequations are:

P� D !;

� Pxp1

Pyp1

�D�

u1v1

�;

n 1 n � 1 n � 2 1 n � 1 n � 2

n Œ J 0 Mpx Mdx 0 Mpy Mdy �

n Œ Jy �m I I 0 0 0 �

n � 2 Œ 0 0 �Ap Ad 0 0 0 �

n Œ �Jx 0 0 0 �m I I �

n � 2 Œ 0 0 0 0 0 �Ap Ad �

26666666664

P!Pu1f px

f dx

Pv1f py

f dy

37777777775

D

2666664

T�

�Jx!2

0

�Jy!2 C mg0

3777775:

This latter system has 5n �4 equations in n C2 second derivatives and the 4n �6force components are not necessarily zero. There are 2n C 4 differential equations


in total. If the position, velocity and acceleration of the distal point of Segment 6 areneeded, an extra four differential equations can be added with the four position andvelocity variables, but these can be computed from a knowledge of the variablesalready in the system, that is they are dependent on the variables already in theequations above.

4 Case 3A: One Contact at Swing Heel, Proximal Segment 2


24

P�Pxp1

Pyp1

35 D

24 !

u1v1

35 ;

n 1 1 n � 1 1 n � 2 n � 1 1 n � 2

n Œ J 0 0 Mpx 0 Mdx Mpy 0 Mdy �

1 Œ 5lts �1 0t 0t 0 0 0t 0 0t �

1 Œ �5ltc 0 �1 0t 0 0t 0t 0 0t �

n Œ Jy �m 0 I 0 I 0 0 0 �

n � 2 Œ 0 0 0 �Ap e1 Ad 0 0 0 �

n Œ �Jx 0 �m 0 0 0 I 0 I �

n � 2 Œ 0 0 0 0 0 0 �Ap e1 Ad �

266666666666664

P!Pu1Pv1

f px

f ex2

f dx

f py

f ey2

f dy

377777777777775

D

26666666664

T�

�5ltc!

2

�5lts!

2

�Jx!2

0

�Jy!2 C mg0

37777777775

:

Note that this is a square matrix of order 5n � 2. There are n C 2 second orderderivatives and 4n � 4 force components.

5 Case 3B: One Contact at Swing Heel, Proximal Segment 6

The case presented here is similar to the one before except here the swing heel atproximal Segment 6 is considered. The dynamic equations are:

24

P�Pxp1

Pyp1

35 D

24 !

u1v1

35 ;

32 M. Tan et al.

n 1 1 n � 1 1 n � 2 n � 1 1 n � 2

n Œ J 0 0 Mpx 0 Mdx Mpy 0 Mdy �

1 Œ 4lts �1 0t 0t 0 0 0t 0 0t �

1 Œ �4ltc 0 �1 0t 0 0t 0t 0 0t �

n Œ Jy �m 0 I 0 I 0 0 0 �

n � 2 Œ 0 0 0 �Ap e1 Ad 0 0 0 �

n Œ �Jx 0 �m 0 0 0 I 0 I �

n � 2 Œ 0 0 0 0 0 0 �Ap e1 Ad �

266666666666664

P!Pu1Pv1

f px

f ex6

f dx

f py

f ey6

f dy

377777777777775

D

26666666664

T�

�4ltc!

2

�4lts!

2

�Jx!2

0

�Jy!2 C mg0

37777777775

:

Note that this case and the case before are similar, the only difference falls in thenumbering of segments. This system, as before, is a square matrix of order 5n � 2.There are n C 2 second derivatives and 4n � 4 force components.

6 Case 4A: One Contact at Proximal Segment 1


P� D !;

n n n � 2 n n � 2n Œ J Mpx Mdx Mpy Mdy �

n Œ Jy I I 0 0 �

n � 2 Œ 0 �Ap Ad 0 0 �

n Œ �Jx 0 0 I I �

n � 2 Œ 0 0 0 �Ap Ad �

2666664

P!f px

f dx

f py

f dy

3777775

D

2666664

T�

�Jx!2

0

�Jy!2 C mg0

3777775:

This gives n second order derivative variables and 4n � 4 force components inthe latter equation.

7 Case 4B: One Contact at Distal Segment 6


24

P�Pxp1

Pyp1

35 D

24 !

u1v1

35 ;


n 1 1 n � 1 n � 1 n � 1 n � 1

n Œ J 0 0 Mpx Mdx Mpy Mdy �

1 Œ 1lts �1 0 0t 0t 0t 0t �

1 Œ �1ltc 0 �1 0t 0t 0t 0t �

n Œ Jy �m 0 I I 0 0 �

n � 2 Œ 0 0 0 �Ap Ad 0 0 �

n Œ �Jx 0 �m 0 0 I I �

n � 2 Œ 0 0 0 0 0 �Ap Ad �

26666666664

P!Pu1Pv1

f px

f dx

f py

f dy

37777777775

D

26666666664

T�

�1ltc!

2

�1lts!

2

�Jx!2

0

�Jy!2 C mg0

37777777775

:

The system has 5n � 2 equations in n C 2 second derivatives and 4n � 4 forcecomponents. Note that Case 4A and Case 4B are similar, as they should be, onlydifference being in the numbering of segments.

References

Alexander RM (1996) Walking and running. Math Gaz 80(488):262–266Alexander RM (2003) Modelling approaches in biomechanics. Philos Trans Biol Sci

358(1437):1429–1435Anderson FC, Pandy MG (2001) Dynamic optimization of human walking. J Biomech Eng

123:381–390Bessonnet G, Chessé S, Sardain P (2004) Optimal gait synthesis of a seven-link planar biped. Int J

Robot Res 23:1059–1973Bullimore SR, Burn JF (2006) Consequences of forward translation of the point of force

application for the mechanics of running. J Theor Biol 238(1):211–219Cavagna GA, Thys H, Zamboni A (1976) The sources of external work in level walking and

running. J Physiol 262(3):639–657Chao EY, Rim K (1973) Application of optimization principles in determining the applied moments

in human leg joints during gait. J Biomech 6:479–510Chow CK, Jacobson DH (1971) Studies of human locomotion via optimal programming. Math

Biosci 10:239–306Dean GA (1965) An analysis of the energy expenditure in level and grade walking. Ergonomics

8:31–47Hardt M, Kreutz-Delgado K, Helton JW (1999) Optimal biped walking with a complete dynamical

model. In: Proceedings of the 38th conference on decision and control, Phoeniz, pp 2999–3004Jennings LS, Fisher ME, Teo KL, Goh CJ (2000) MISER3 optimal control software (version 3):

theory and user manual. Centre of Applied Dynamics and Optimization, The University ofWestern Australia

Koh MTH (2001) Optimal performance of the Yurchenko layout vault. PhD thesis, University ofWestern Australia

Koh MTH, Jennings LS (2003) Dynamic optimization: inverse analysis for the Yurchenko layoutvault in women’s artistic gymnastics. J Biomech 36(8):1177–1183

Kuo A (1998) A least-squares approach to improving the precision of inverse dynamics computa-tions. J Biomech Eng 120(1):148–159

Marshall RN (1985) Biomechanical performance criteria in normal and pathological walking. PhDthesis, University of Western Australia

McGeer T (1988) Stability and control of two-dimensional biped walking. Technical report CSS-ISTR 99-01, Centre for Systems Science, Simon Fraser University, Burnaby

34 M. Tan et al.

Onyshko S, Winter DA (1980) A mathematical model for the dynamics of human locomotion. JBiomech 13(4):361–368

Pandy MG (2001) Computer modeling and simulation of human movement. Annu Rev BiomedEng 3:245–273

Rehbock V, Teo KL, Jennings LS (1996) Optimal and suboptimal feedback controls for a class ofnonlinear systems. Comput Math Appl 31(6):71–86

Ren L, Jones RK, Howard D (2007) Predictive modelling of human walking over a complete gaitcycle. J Biomech 40(7):1567–1574

Said M, Jennings LS, Koh MT (2006) Computational models satisfying relative angle constraintsfor 2-dimensional segmented bodies. Anziam J 47:541–554

Selles RW, Bussmann JBJ, Wagenaar RC, Stam HJ (2001) Comparing predictive validity of fourballistic swing phase models of human walking. J Biomech 34(9):1171–1177

Ünver NF, Tümer ST, Özgören MK (2000) Simulation of human gait using computed torquecontrol. Technol Health Care 8(1):53–66

Winter DA (1990) Biomechanics and motor control of human movement, 2nd edn. Wiley, HobokenWinter DA (2009) Biomechanics and motor control of human movement, 4th edn. Wiley, HobokenXiang Y, Arora JS, Rahmatalla S, Abdel-Malek K (2009) Optimization-based dynamic human

walking prediction: one step formulation. Int J Numer Methods Eng 79(6):667–695Xiang Y, Arora JS, Abdel-Malek K (2010) Physics-based modeling and simulation of human

walking: a review of optimization-based and other approaches. Struct Multidiscip Optim42(1):1–23

Yamaguchi GT, Zajac FE (1990) Restoring unassisted natural gait to paraplegics via functionalneuromuscular stimulation: a computer simulation study. IEEE Trans Biomed Eng 37(9):886–902

Chapter 2Rearrangement Optimization Problems Relatedto a Class of Elliptic Boundary Value Problems

Chong Qiu, Yisheng Huang, and Yuying Zhou

Abstract In this paper, we investigate two optimization problems related to a classof elliptic boundary value problems on smooth bounded domains of R

N . Theseoptimization problems are formulated as minimum and maximum problems relatedto the rearrangements of given functions. Under some suitable assumptions, weshow that both problems are solvable. Moreover, we obtain a representation resultof the optimal solution for the minimization problem and show that this solution isunique and symmetric if the domain is a ball centered at the origin.

Keywords Existence and uniqueness • Optimization • Eigenvalue • Rearrange-ments

2.1 Introduction

After the Burton fundamental work (Burton 1987, 1989) on theory of rearrange-ments, the rearrangement optimization problems in addressing questions suchas existence, uniqueness, symmetry and some qualitative properties of optimalsolutions have been investigated by a number of authors, see for example (Burton1989; Kurata et al. 2004; Del Pezzo and Bonder 2009; Zivari-Rezapour 2013; Cuccuet al. 2006a,b, 2009; Marras 2010; Marras et al. 2013; Emamizadeh and Zivari-Rezapour 2007; Emamizadeh and Fernandes 2008; Emamizadeh and Prajapat 2009;Chanillo et al. 2000; Chanillo and Kenig 2008; Nycander and Emamizadeh 2003;Anedda 2011; Qiu et al. 2015) and the references therein.

This work was supported by Natural Science Foundation of China (11471235, 11171247,11371273) and GIP of Jiangsu Province (CXZZ13_0792).

C. Qiu • Y. Huang • Y. Zhou (�)Department of Mathematics, Soochow University, Suzhou 215006, People’s Republic of Chinae-mail: [email protected]


35

36 C. Qiu et al.

Let˝ be a smooth bounded domain of RN .N � 3/. We say that two measurablefunctions f .x/ and g.x/ defined in ˝ are rearrangements of each other if

meas .fx 2 ˝ W g.x/ � ag/ D meas .fx 2 ˝ W f .x/ � ag/ ; 8a 2 R:

The rearrangement optimization problems related to the following eigenvalueproblem

.Lh/

( ��u D h.x/u in ˝;

u D 0 on @˝

or boundary value problem

.Pf /

( ��u D f .x/ in ˝;

u D 0 on @˝;

including their similar problems involving p-Laplacian, have been studied by someauthors, see for example (Burton 1987, 1989; Cuccu et al. 2006a, 2009; Marras2010; Marras et al. 2013; Emamizadeh and Zivari-Rezapour 2007; Emamizadehand Fernandes 2008), where 0 < h 2 L1.˝/, f 2 Lq.˝/ with q > 2N=.N C2/.

Recently, a rearrangement optimization problem related to the following quasi-linear elliptic boundary value problem has been considered in Qiu et al. (2015):

.P/

( ��pu C h.x; u/ D f .x/ in ˝;

u D 0 on @˝

where 1 < p < 1, h.x; t/ W ˝ � R 7! R is a Carathéodory function satisfying

suitable growth conditions, f 2 Lq.˝/ with some 1 � q < 1. In Qiu et al.(2015), we showed that the minimum and maximum optimization problems relatedto .P/ are solvable in both cases of 1 < p � N and p > N, which extended thecorresponding results in Burton (1987, 1989) with p D 2 and Cuccu et al. (2006a)and Marras (2010) with 1 < p < 1.

In this paper, we will investigate two rearrangement optimization problemsrelated to the following elliptic boundary value problem:

.Ph;f /

( ��u � h.x/u D f .x/ in ˝;

u D 0 on @˝:

.Ph;f / is actually a model of the deformation problem for an elastic membrane made

out of some materials with prescribed quantities h, subject to a fixed vertical forcef . The usual goal is to identify a force function selected from R.f /, in such a way

2 Rearrangement Optimization Problems Related to a Class of Elliptic. . . 37

that the total displacement of the membrane is as small as possible. More precisely,let I W H1

0.˝/ ! R be the energy functional corresponding to the problem .Ph;f /,which is given by

I.u/ D 1

2

Z˝

jruj2dx �

2

Z˝

hu2dx �Z˝

fudx; (2.1)

and let 0 < h0 2 L1.˝/ and f0 2 Lq.˝/ with q > 2N=.N C 2/ be two givenfunctions, then we will study the following minimum and maximum optimizationproblems:

.Optm/ Find Oh 2 R.h0/; Of 2 R.f0/ such that I.uOh;Of / D infh2R.h0/;f 2R.f0/ I.uh;f /

and.OptM/ Find Nf 2 R.f0/ such that I.uh0;Nf / D supf 2R.f0/ I.uh0;f /,

where R.h0/ and R.f0/ respectively denote the sets of all rearrangement of h0 andf0, uh;f (uh0;f ) is the unique solution of the problem .Ph;f / (.Ph0;f /) (the existenceand uniqueness of uh;f (uh0;f ) will be obtained in Propositions 2.1 of Sect. 2.3). Wewill show that there exists � > 0 such that for all 2 .0; �/, both problems.Optm/ and .OptM/ are solvable.

We note that the optimization problem considered in all the papers mentionedabove is constrained by a rearrangement set which is generated by just one fixedfunction. The minimum optimization problem considered here is however con-strained by two rearrangement sets generated by two fixed independent functions.Moreover, Problem .Ph;f / contains .Lh/ and .Pf / as special cases, the costfunctional used in our problem is more complicated than that used in the above twoproblems, therefore our case needs special handling. We point out that an essentialassumption in Qiu et al. (2015) is that h.x; u/ being non-decreasing with respect tothe second variable u for almost all x 2 ˝ . But in the present paper, since we assumethat 0 < and 0 < h.x/; a:e: x 2 ˝ , the term �h.x/u in the problem .Ph;f /

would be decreasing with respect to u for almost all x 2 ˝ and then it violatesthe essential assumption given in paper Qiu et al. (2015). So the conditions andresults for the maximization problem (OptM) here are different from those obtainedin Qiu et al. (2015). To the best of our knowledge, the results obtained in this paperare new.

This paper is organized as follows. In Sect. 2.2, we give some preliminaries. InSect. 2.3, we show that the problem .Ph;f / has a unique solution. Section 2.4 isdevoted to discuss the minimization problem (Optm) in detail. Firstly, we prove thatthe minimization problem (Optm) is solvable in the case of 0 < < �, then weobtain a representation result of the optimal solution for the minimization problemand show that the problem (Optm) has unique solution with some symmetricproperties if ˝ is a ball centered at the origin. In Sect. 2.5, we show that themaximization problem (OptM) is solvable.

38 C. Qiu et al.

2.2 Preliminaries

We denote by Lr.˝/ .1 � r � 1/ and H10.˝/ the usual Sobolev spaces endowed

with the norms kukLr D R˝

jujrdx�1=r

if 1 � r < 1, kuk1 D ess supx2˝ ju.x/j and

kuk D R˝

jruj2dx�1=2

, respectively. Throughout the paper C will denote a positive(possibly different) constant.

Definition 2.1. By a solution u of the problem .Ph;f / we mean that u 2 H10.˝/

satisfying

Z˝

.rurv � huv � fv/ dx D 0; 8v 2 H10.˝/:

Let I be given in (2.1). If I 2 C1.H10.˝/; R/, then we have

I0.u/v DZ˝

.rurv � huv � fv/ dx; 8v 2 H10.˝/:

In this case, u 2 H10.˝/ is a weak solution if and only if I0.u/v D 0, 8v 2 H1

0.˝/.The following lemmas will be used through the proofs of our main results.

Lemma 2.1 (Burton 1989, Lemma 2.1). Assume that 1 � r < 1 and given f 2Lr.˝/, then for any g 2 R.f / we have g 2 Lr.˝/ and kgkLr D kf kLr .

Lemma 2.2 (Burton 1989, Lemma 2.2). Assume that 1 � r < 1 and given f 2Lr.˝/, denote by R.f /w the weak closure of R.f / in Lr.˝/, then R.f /w is convexand weakly compact in Lr.˝/.

Lemma 2.3 (Burton 1989, Lemma 2.9 or Cuccu et al. 2009, Lemma 2.1). Letf ; g W ˝ 7! R be measurable functions and suppose that for each t 2 R, the levelset of g at t, i.e., fx 2 ˝ W g.x/ D tg, has zero measure. Then there exists anincreasing (decreasing) function ' such that ' ı g is a rearrangement of f where' ı g denotes a composite function defined by

.' ı g/.x/ D '.g.x//; 8x 2 ˝:

Lemma 2.4 (Burton 1989, Lemma 2.4 or Cuccu et al. 2009, Lemma 2.2). Forany 1 � r < 1 define r0 D r

r�1 if r > 1 and r0 D 1 if r D 1. Let f 2 Lr.˝/ and

g 2 Lr0

.˝/. Suppose that there exists an increasing (decreasing) function ' W R 7!R such that ' ı g 2 R.f /. Then ' ı g is the unique maximizer (minimizer) of thelinear functional

R˝

hgdx, relative to h 2 R.f /w.

Lemma 2.5 (Burton 1987, Theorem 5). For any 1 � r < 1 define r0 D rr�1 if

r > 1 and r0 D 1 if r D 1. Let f 2 Lr.˝/ and g 2 Lr0

.˝/. Suppose that the linearfunctional L.l/ D R

˝lgdx has a unique maximizer (minimizer) Of relative to R.f /

then there exists an increasing (decreasing) function ' W R 7! R such that 'ıg D Of .


Lemma 2.6 (Emamizadeh and Prajapat 2009, Lemma 2.3). Suppose that f 2Lr.˝/ and g 2 Lr0

.˝/, then there exists Of 2 R.f / which maximizes (minimizes) thelinear functional

R˝

hgdx, relative to h 2 R.f /w.

Lemma 2.7 (Leoni 2009, Theorem 16.9). Suppose that B is a ball centered at theorigin, then

ZB

fgdx �Z

Bf �g�dx;

for any non-negative measurable functions f and g, where f � and g� are respectivelythe Schwarz symmetric decreasing rearrangements of f and g, defined in thefollowing.

Definition 2.2 (Leoni 2009, Definition 16.5). Let f W ˝ 7! Œ0;1/ be a measurablefunction. The Schwarz symmetric decreasing rearrangement of f is the functionf � W B.0; r/ 7! Œ0;1/, defined by

f �.x/ D inf˚t 2 Œ0;1/ W f .t/ � !N jxjN

�;8x 2 B.0; r/

where !N denotes the volume of the unit ball in N-dimensions, r WD.meas.˝/=!N/

1=N and f W R 7! Œ0;1/ is the distribution function of f defined by

f .t/ D meas.fx 2 ˝ W f .x/ > tg/:

It is well known that f � D g� for each g 2 R.f /.

Lemma 2.8 (Leoni 2009, Theorem 16.10). Suppose that B is a ball centered atthe origin, u W B 7! Œ0;1/ is a measurable function and � W Œ0;1/ 7! Œ0;1/ is aBorel function, then

ZB� ı u�dx �

ZB� ı udx:

The following result can be deduced from Lemmas 2.3 and 3.2 and Theorem 1.1 ofBrothers and Ziemer (1988).

Lemma 2.9. Suppose that B is a ball centered at the origin. If u 2 W1;p0 .B/ with

1 < p < 1 and u � 0 then u�1.˛;1/ is a translation of u��1.˛;1/ for every˛ 2 Œ0; ess supx2B u.x// and

ZB

jrujpdx �Z

Bjru�jpdx: (2.2)

If the equality holds in (2.2) and the setnx 2 B W ru.x/ D 0; 0 < u.x/ < ess sup

y2Bu.y/

o

has zero measure, then u D u�.

40 C. Qiu et al.

It is well known that the first eigenvalue 1.h/ of the problem .Lh/ can becharacterized by

1.h/ D infv2H1

0 .˝/;v 6D0

R˝

jrvj2dxR˝

hv2dx: (2.3)

By Cuccu et al. (2009, Theorem 3.1), if 0 < h0.x/ 2 L1.˝/, then there existsNh 2 R.h0/ (the set of all rearrangements of h0) such that

0 < � WD 1.Nh/ D infh2R.h0/

1.h/ D infh2R.h0/

infv2H1

0 .˝/;v 6D0

R˝

jrvj2dxR˝

hv2dx: (2.4)

2.3 Existence and Uniqueness for the Solution of theProblem .Ph;f /

In this section, we will obtain the existence and uniqueness for the solution of theproblem .Ph;f /.

Proposition 2.1. Fix 0 < h.x/ 2 L1.˝/, and f 2 Lq.˝/ with q > 2N=.N C2/ and0 < < 1.h/, where 1.h/ is the first eigenvalue of the problem .Lh/. Then theproblem .Ph;f / has a unique solution uh;f 2 H1

0.˝/ and I.uh;f / D infv2H10 .˝/

I.v/.Moreover, if in addition f .x/ > 0 a.e. x 2 ˝ , then uh;f > 0.

Proof. First, we show that the problem .Ph;f / has a solution.By the Hölder inequality and the Sobolev embedding inequality, we have

ˇˇZ˝

fudx

ˇˇ � kf kLq kukLq0

� Ckuk (2.5)

for all u 2 H10.˝/ since now 1 < q0 WD q=.q � 1/ < 2� where 2� WD 2N=.N � 2/.

Hence we deduce from (2.1), (2.3) and (2.5) that

I.u/ � 1

2.1 �

1.h//kuk2 � Ckuk ! 1

as kuk ! 1, which shows that the functional I is coercive.It is easy to see that the functional I is weakly lower semi-continuous (which

we will denote by w.l.s.c for short). So that the functional I has a minimizer uh;f 2H10.˝/ with I.uh;f / D infv2H1

0 .˝/I.v/. Using a standard argument (cf. Willem 1996,

Lemma 2.16), we can easily show that I 2 C1.H10.˝/;R/, therefore uh;f is a solution

of the problem .Ph;f / satisfying


I0.uh;f /v DZ˝

ruh;f rv � huh;fv � fv�

dx D 0; 8v 2 H10.˝/: (2.6)

Next, we show that uh;f is the unique solution of the problem .Ph;f /.Assume that wh;f 2 H1

0.˝/ is another solution of the problem .Ph;f / and uh;f 6Dwh;f , then

kuh;f � wh;f k > 0:

By 0 < h.x/ 2 L1.˝/, we have

Z˝

h.wh;f � uh;f /2dx > 0: (2.7)

From (2.6) and Definition 2.1 we get that for every v 2 H10.˝/,

Z˝

ruh;f rv � huh;fv�

dx DZ˝

fvdx;

Z˝

rwh;f rv � hwh;f v�

dx DZ˝

fvdx:

Therefore,

Z˝

.hwh;f � huh;f /vdx DZ˝

.rwh;f � ruh;f /rvdx; 8v 2 H10.˝/: (2.8)

Let v D wh;f � uh;f . Note that 0 < < 1.h/, then from (2.3), (2.7) and (2.8) weobtain

Z˝

.jrwh;f � ruh;f j2/dx

DZ˝

h.wh;f � uh;f /2dx

<

Z˝

1.h/h.wh;f � uh;f /2dx

�Z˝

.jrwh;f � ruh;f j2/dx;

a contradiction. Therefore we have proved that uh;f is the unique solution of theproblem .Ph;f /.

42 C. Qiu et al.

Finally, if f .x/ > 0 then we can easily check that I.juh;f j/ � I.uh;f /, which showsthat juh;f j is also a minimizer of I and thus a solution of the problem .Ph;f /. Thenuh;f D juh;f j � 0 by the uniqueness of the solution. Since

��uh;f .x/ D f .x/C h.x/uh;f .x/ > 0; a:e: x 2 ˝;

we have uh;f .x/ > 0; a:e: x 2 ˝ (cf. Vázquez 1984, Theorem 5). utRemark 2.1. In the case of D 1.h/, if uh;f 2 H1

0.˝/ is a solution of the problem.Ph;f /, then for any t 2 R and v 2 H1

0.˝/,

Z˝

r.uh;f C t'/rv � h.uh;f C t'/v � fv�

dx

DZ˝

ruh;f rv � huh;fv � fv�

dx D 0;

where ' is the eigenfunction of .Lh/. That is, uh;f Ct' is the solution of the problem.Ph;f /. Therefore, in order to obtain the unique solution of the problem .Ph;f /, weonly consider the case of 0 < < 1.h/ in the following.

2.4 Existence of Solution of the Problem .Optm/

Theorem 2.1. Suppose that 0 < h0.x/ 2 L1.˝/, f0 2 Lq.˝/with q > 2N=.NC2/,and 0 < < �, where � is given by (2.4). Then there exists Oh 2 R.h0/; Of 2 R.f0/which solves the problem .Optm/, i.e.,

I.Ou/ D infh2R.h0/;f 2R.f0/

I.uh;f /;

where Ou D uOh;Of is the unique solution of .POh;Of /.

Proof. Clearly, � < 1.h/, 8h 2 R.h0/. By Proposition 2.1, the problem .Ph;f /

has a unique solution uh;f 2 H10.˝/. Let

A D infh2R.h0/;f 2R.f0/

I.uh;f /

then A is well-defined. Indeed, for each h 2 R.h0/; f 2 R.f0/, from (2.3) wehave

�Z˝

hu2h;f dx � 1.h/Z˝

hu2h;f dx �Z˝

jruh;f j2dx;


and then

I.uh;f / D 1

2

Z˝

jruh;f j2dx �

2

Z˝

hu2h;f dx �Z˝

fuh;f dx

� 1

2.1 �

�/kuh;f k2 � Ckf kLq kuh;f k:

(2.9)

By Lemma 2.1, kf kLq D kf0kLq , we deduce that A must be finite.Let f.hi; fi/g be a minimizing sequence, i.e.,

hi 2 R.h0/ and fi 2 R.f0/;8i 2 N

and

A D limi!1 I.ui/

where ui D uhi;fi . It follows from (2.9) that fuig is bounded in H10.˝/, then it

has a subsequence (still denoted fuig) which weakly converges to u 2 H10.˝/

and strongly converges to u in Lq0

.˝/ with 1 < q0 D q=.q � 1/ <

2�. Since kfikLq � kf0kLq , ffig contain a subsequence (still denoted ffig)

converging weakly to some Nf 2 R.f0/w, the weak closure of R.f0/ in Lq.˝/.Then ˇ

ˇZ˝

.fi � Nf /udx

ˇˇ ! 0 as i ! 1

since u 2 Lq0

.˝/. It follows from the Hölder inequality that

ˇˇZ˝

.fiui � Nf u/dx

ˇˇ �

ˇˇZ˝

fi.ui � u/dx

ˇˇC

ˇˇZ˝

.fi � Nf /udx

ˇˇ

� kfikLq kui � ukLq0 CˇˇZ˝

.fi � Nf /udx

ˇˇ ! 0

(2.10)

as i ! 1. Since khik1 � kh0k1, fhig is bounded in L1.˝/, it must containa subsequence (still denoted fhig) converging weakly to some Nh 2 R.h0/w,the weak closure of R.h0/ in Lr.˝/.r > N=2/. Similarly as (2.10) we have

limi!1

Z˝

.hiu2i � Nhu2/dx D 0: (2.11)

By (2.10) and (2.11) and the weak lower semi-continuity of the norm in the H10.˝/,

we obtain that

A D limi!1 I.ui/ � 1

2

Z˝

jruj2dx �

2

Z˝

Nhu2dx �Z˝

Nf udx: (2.12)

44 C. Qiu et al.

From Lemma 2.6 we infer the existence of Of 2 R.f0/ which maximizesthe linear functional

R˝

ludx, relative to l 2 R.f0/w. As a consequence,Z˝

Nf udx �Z˝

Of udx:

Similarly we have there exists Oh 2 R.h0/ which maximizes the linear functionalR˝

lu2dx, relative to l 2 R.h0/w. So that

Z˝

Nhu2dx �Z˝

Ohu2dx:

Combining with (2.12), we get

A � 1

2

Z˝

jruj2dx �

2

Z˝

Ohu2dx �Z˝

Of udx: (2.13)

By Proposition 2.1,

I.Ou/ D infv2H1

0 .˝/

Z˝

1

2jrvj2 �

2Ohv2dx � Of v

�dx

� 1

2

Z˝

jruj2dx �

2

Z˝

Ohu2dx �Z˝

Of udx:

(2.14)

It follows from (2.13) and (2.14) that I.Ou/ � A.On the other hand, recall that A D infh2R.h0/;f 2R.f0/ I.uh;f /, we must have

A � I.Ou/. So that A D I.Ou/. utWe now obtain a representation result of the optimal solution .Oh; Of / for the

problem .Optm/.

Theorem 2.2. Under the assumptions of Theorem 2.1 and moreover suppose thatmeas.fx 2 ˝ W f0.x/ D 0g/ D 0. Then there exist increasing functions � and 'such that

Oh D �.Ou2/ a:e: in ˝;

Of D '.Ou/ a:e: in ˝;(2.15)

where Ou D uOh;Of is the solution of .POh;Of /.

Proof. Since Ou is the solution of .POh;Of /, I.Ou/ � I.uh;Of /, 8h 2 R.h0/. Therefore

1

2

Z˝

jr Ouj2dx �

2

Z˝

OhOu2dx �Z˝

Of Oudx � 1

2

Z˝

jr Ouj2dx �

2

Z˝

hOu2dx �Z˝

Of Oudx;


i.e.,

Z˝

hOu2dx �Z˝

OhOu2dx; 8h 2 R.h0/:

So that Oh is a maximizer of the linear functional L.h/ WD R˝

hOu2dx, relative toh 2 R.h0/.

We claim that Oh is the unique maximizer of L.h/. If not, suppose that Nh is anothermaximizer of L.h/. Then

Z˝

OhOu2dx DZ˝

NhOu2dx:

Thus

I.Ou/ D 1

2

Z˝

jr Ouj2dx �

2

Z˝

OhOu2dx �Z˝

Of Oudx

D 1

2

Z˝

jr Ouj2dx �

2

Z˝

NhOu2dx �Z˝

Of Oudx

� I.uNh;Of /

� I.Ou/:

So that

1

2

Z˝

jr Ouj2dx �

2

Z˝

NhOu2dx �Z˝

Of Oudx D I.uNh;Of /:

By the uniqueness of the minimizer of the functional I, we obtain Ou D uNh;Of . Then

Z˝

ruNh;Of rvdx � Z˝

NhuNh;Of vdx DZ˝

Ofvdx;

Z˝

r Ourvdx � Z˝

OhOuvdx DZ˝

Ofvdx; 8v 2 H10.˝/:

So thatZ˝

.Nh � Oh/Ouvdx D 0; 8v 2 H10.˝/;

which implies that

.Nh.x/ � Oh.x//Ou.x/ D 0; a:e: x 2 ˝: (2.16)

46 C. Qiu et al.

By the assumption, meas.fx 2 ˝ W f0.x/ D 0g/ D 0, we have meas.fx 2 ˝ WOf .x/ D 0g/ D 0, since Of 2 R.f0/. Thus meas.fx 2 ˝ W Ou.x/ D 0g/ D 0.Combining with (2.16) we have Nh.x/ D Oh.x/, a.e. x 2 ˝ . Therefore, Oh is the uniquemaximizer of L.h/. Note that L.h/ D R

˝hOu2dx, so by using Lemma 2.5, there exists

an increasing function � such that

�.Ou2/ D Oh; a:e: in ˝:

Similarly, we can show that Of is the unique maximizer of the linear functional l.f / WDR˝

f Oudx, relative to f 2 R.f0/. Also from Lemma 2.5, there exists an increasingfunction ' such that

'.Ou/ D Of ; a:e: in ˝:

We complete the proof. utTheorem 2.3. Under the assumptions of Theorem 2.1 and if ˝ is a ball centeredat the origin, f0.x/ > 0, a.e. x 2 ˝ , then the problem .Optm/ has a unique solution.Oh; Of / and Oh D h�

0 , Of D f �0 where h�

0 (f �0 ) is the Schwarz symmetric decreasing

rearrangement of h0 (f0).

Proof. Denote by Ou� the Schwarz symmetric decreasing rearrangement of Ou, whereOu D uOh;Of is the solution of .POh;Of /.

Similar to the proof of Theorem 4.5 in Qiu et al. (2015), we obtainZ˝

jr Ou�j2dx DZ˝

jr Ouj2dx (2.17)

and

meas

(x 2 ˝ W r Ou D 0; 0 < Ou.x/ < ess sup

y2˝Ou.y/

)!D 0: (2.18)

Now, by using Lemma 2.9, and noting (2.17) and (2.18), we see that Ou D Ou�.By (2.15) in Theorem 2.2, Oh D � ı .Ou�/2 and Of D ' ı Ou� are spherically symmetricdecreasing functions. It follows that Oh coincides its Schwarz rearrangement, i.e.,Oh D Oh� D h�

0 , so is Of . ut

2.5 Existence of Solution of the Problem .OptM/

We now consider the problem .OptM/. Our results for the problem .OptM/ are thefollowing.

Theorem 2.4. Let 0 < h0.x/ 2 L1.˝/, and let f0 2 Lq.˝/ with q > 2N=.N C 2/.Suppose that 0 < < 1.h0/, where 1.h0/ is given by (2.3) and f0.x/ � 0, thenthere exists a unique Nf 2 R.f0/ which solves the problem .OptM/, i.e.,


I.uh0;Nf / D supf 2R.f0/

I.uh0;f /;

where uh0;f denotes the unique solution of the problem .Ph0;f /.

By using Proposition 2.1, we can define a functionalˆ W Lq.˝/ 7! R by

ˆ.f / D I.uh0;f /; (2.19)

where uh0;f denotes the unique solution of the problem .Ph0;f /.Before proving Theorem 2.4, we shall show the following lemmas.

Lemma 2.10. Suppose that all the assumptions of Theorem 2.4 are satisfied.Then

(I) The functional˚ jR.f0/wis weakly continuous;

(II) The functional˚ jR.f0/wis strictly concave;

(III) The functional˚ is Gâteaux differentiable at each f 2 R.f0/w with derivative�uf ,

where ˚ W Lq.˝/ 7! R is given by (2.19).

Proof. Since 0 < < 1.h0/ and 1.h0/ D infv2H10 .˝/;v 6D0

R˝ jrvj2dxR˝ h0v2dx

,

1.h0/Z˝

hu2h0;f dx �Z˝

jruh0;f j2dx:

We get

I.uh0;f / � 1

2.1�

1.h0//kuh0;f k2 � Ckf kLq kuh0;f k:

The rest proof of (I), (II) and (III) is similar to the proof of Lemma 4.1 in Qiuet al. (2015), we omit it. utSimilar to Lemma 4.2 in Qiu et al. (2015), we obtain

Lemma 2.11. Under the assumptions of Theorem 2.4, there exists a unique Qf 2R.f0/w which maximizes ˚ jR.f0/w

. Moreover,

Z˝

QuQf dx �Z˝

Qugdx; 8g 2 R.f0/w; (2.20)

where Qu D uQf .

Lemma 2.12. Let Qf and Qu be as in Lemma 2.11, and let S.Qf / D fx 2 ˝ W Qf .x/ > 0g:Set

� D ess supx2S.Qf /

Qu.x/; ı D ess infx2˝nS.Qf /

Qu.x/:

Then � � ı.

48 C. Qiu et al.

Proof. If not, we assume that � > ı. Then we can choose � > �1 > �2 > ı. Since� > �1, there exists a set A � S.Qf /, with positive measure, such that Qu � �1 in A.Similarly, there exists a set B � ˝ n S.Qf /, with positive measure, such that Qu � �2in B. Without lose of generality we may assume that meas.A/ D meas.B/. Thenthere exists a measure preserving map T W A ! B: So that we can define a particularrearrangement of Qf as following:

Nf .x/ D

8<ˆ:

Qf .Tx/; x 2 A

Qf .T�1x/; x 2 B

Qf .x/; x 2 ˝ n .A [ B/:

Thus Z˝

QuQf dx �Z˝

QuNf dx DZ

A[BQuQf dx �

ZA[B

QuNf dx

DZ

AQuQf dx �

ZB

QuNf dx

� �1

ZA

Qf dx � �2Z

B

Nf dx

D .�1 � �2/Z

A

Qf dx > 0:

Therefore,R˝ QuQf dx >

R˝ QuNf dx, a contradiction. ut

Proof of Theorem 2.4. Let Qf and Qu be as in Lemma 2.11. It is clear that the level setsof Qu, restricted to S.Qf /, have measure zero. Therefore applying Lemma 2.3, thereexists a decreasing function Q� such that Q� ı Qu is a rearrangement of Qf relative to theset S.Qf /. Now, define

�.t/ D( Q�; t � �;

0; t > �;

where � is given in Lemma 2.12. Then � is a decreasing function.In the following, we will prove that � ı Qu is a rearrangement of Qf . By the definition

of the rearrangement, it is sufficient to prove

meas.f� ı Qu.x/ � ag/ D meas.fQf .x/ � ag/ (2.21)

holds for each a 2 R. Clearly,

f� ı Qu.x/ � ag D fQf .x/ > 0; � ı Qu.x/ � ag[

fQf .x/ D 0; � ı Qu.x/ � ag;

fQf .x/ � ag D fQf .x/ > 0; Qf .x/ � ag[

fQf .x/ D 0; Qf .x/ � ag;(2.22)


By the definition of Q� , � and � , we get

meas.fQf .x/ > 0; � ı Qu.x/ � ag/ D meas.fQf .x/ > 0; Q� ı Qu.x/ � ag/D meas.fQf .x/ > 0; Qf .x/ � ag/:

(2.23)

By (2.22) and (2.23), in order to prove (2.21) we only need to show that

meas.fQf .x/ D 0; � ı Qu.x/ � ag/ D meas.fQf .x/ D 0; Qf .x/ � ag/: (2.24)

By Lemma 2.12, we can deduce easily that

meas.fQf .x/ D 0; Qu.x/ < �g/ D 0:

Then the left side of the equality (2.24) can be rewritten as

meas.fQf .x/ D 0; � ı Qu.x/ � ag/ D meas.fQf .x/ D 0; Qu.x/ > �; � ı Qu.x/ � ag/C meas.fQf .x/ D 0; Qu.x/ D �; � ı Qu.x/ � ag/:

(2.25)By the definition of � , we see that

fQf .x/ D 0; Qu.x/ > �; � ı Qu.x/ � ag D fQf .x/ D 0; 0 � ag D fQf .x/ D 0; Qf .x/ � ag:(2.26)

Since

��Qu.x/ � h.x/Qu.x/ D Qf .x/; a:e: in fQf .x/ D 0; Qu.x/ D � > 0g;

meas.fQf .x/ D 0; Qu.x/ D �; � ı Qu.x/ � ag/ D 0: (2.27)

It follows from (2.25), (2.26) and (2.27) that (2.24) holds, and then (2.21) holds.Therefore, � ı Qu is a rearrangement of Qf .

Hence, applying Lemma 2.4, we can deduce that � ı Qu is the unique minimizerof the linear functional

R˝

gQudx, relative to g 2 R.f0/w. This and (2.20) obviouslyimply Qf D � ı Qu 2 R.f0/. We complete the proof by choosing Nf D Qf . ut

Acknowledgements The authors would like to thank the referees for the valuable suggestionswhich have improved the early version of the manuscript.

References

Anedda C (2011) Maximization and minimization in problems involving the bi-Laplacian. Annalidi Matematica 190:145–156

Brothers JE, Ziemer WP (1988) Minimal rearrangements of Sobolev functions. J Reine AngewMath 384:153–179

50 C. Qiu et al.

Burton GR (1987) Rearrangements of functions, maximization of convex functionals and vortexrings. Math Ann 276:225–253

Burton GR (1989) Variational problems on classes of rearrangements and multiple configurationsfor steady vortices. Ann Inst Henri Poincaré 6:295–319

Chanillo S, Kenig C (2008) Weak uniqueness and partial regularity for the composite membraneproblem. J Eur Math Soc 10:705–737

Chanillo S, Grieser D, Kurata K (2000) The free boundary problem in the optimization ofcomposite membranes. Contemp Math 268:61–81

Cuccu F, Emamizadeh B, Porru G (2006a) Nonlinear elastic membrane involving the p-Laplacianoperator. Electron J Differ Equ 2006:1–10

Cuccu F, Emamizadeh B, Porru G (2006b) Optimization problems for an elastic plate. J Math Phys47:1–12

Cuccu F, Emamizadeh B, Porru G (2009) Optimization of the first eigenvalue in problems involvingthe p-Laplacian. Proc Am Math Soc 137:1677–1687

Del Pezzo LM, Bonder JF (2009) Some optimization problems for p-Laplacian type equations.Appl Math Optim 59:365–381

Emamizadeh B, Fernandes RI (2008) Optimization of the principal eigenvalue of the one-Dimensional Schrödinger operator. Electron J Differ Equ 2008:1–11

Emamizadeh B, Prajapat JV (2009) Symmetry in rearrangemet optimization problems. Electron JDiffer Equ 2009:1–10

Emamizadeh B, Zivari-Rezapour M (2007) Rearrangement optimization for some elliptic equa-tions. J Optim Theory Appl 135:367–379

Kurata K, Shibata M, Sakamoto S (2004) Symmetry-breaking phenomena in an optimizationproblem for some nonlinear elliptic equation. Appl Math Optim 50:259–278

Leoni G (2009) A first course in Sobolev spaces. Graduate studies in mathematics. AmericanMathematical Society, Providence

Marras M (2010) Optimization in problems involving the p-Laplacian. Electron J Differ Equ2010:1–10

Marras M, Porru G, Stella VP (2013) Optimization problems for eigenvalues of p-Laplaceequations. J Math Anal Appl 398:766–775

Nycander J, Emamizadeh B (2003) Variational problem for vortices attached to seamounts.Nonlinear Anal 55:15–24

Qiu C, Huang YS, Zhou YY (2015) A class of rearrangement optimization problems involving thep-Laplacian. Nonlinear Anal Theory Methods Appl 112:30–42

Vázquez JL (1984) A strong maximum principle for some quasilinear elliptic equations. Appl MathOptim 12:191–202

Willem M (1996) Minimax theorems. Birkhauser, BaselZivari-Rezapour M (2013) Maximax rearrangement optimization related to a homogeneous

Dirichlet problem. Arab J Math (Springer) 2:427–433

Chapter 3An Extension of the MOON2/MOON2R

Approach to Many-Objective OptimizationProblems

Yoshiaki Shimizu

Abstract A multi-objective optimization (MUOP) method that supports agile andflexible decision making to be able to handle complex and diverse decision envi-ronments has been in high demand. This study proposes a general idea for solvingmany-objective optimization (MAOP) problems by using the MOON2 or MOON2R

method. These MUOP methods rely on prior articulation in trade-off analysis amongconflicting objectives. Despite requiring only simple and relative responses, thedecision maker’s trade-off analysis becomes rather difficult in the case of MAOPproblems, in which the number of objective functions to be considered is largerthan in MUOP. To overcome this difficulty, we present a stepwise procedure that isextensively used in the analytic hierarchy process. After that, the effectiveness of theproposed method is verified by applying it to an actual problem. Finally, a generaldiscussion is presented to outline the direction of future work in this area.

Keywords Many-objective optimization • MOON2 • MOON2R • Pairwise com-parison • AHP • Neural network

3.1 Introduction

A multi-objective optimization (MUOP) method that supports flexible and adaptivedecision making for application in complex, diverse, and competitive environmentshas been in high demand. Notably, MUOP applies to problems involving incom-mensurable objectives that conflict or compete with each other. Although Paretooptimal solutions represent a rational norm in MUOP, there can be an infinite

This paper was presented at ICOTA 2013 held in Taipei., Taiwan.

Y. Shimizu (�)Department of Mechanical Engineering, Toyohashi University of Technology,Toyohashi, Aichi 441-8580, Japane-mail: [email protected]


51

52 Y. Shimizu

number of members of this class. The set of optimal solutions is known as the Paretofront. Generally speaking, however, decision making as an engineering task aims atobtaining a limited number of candidates for the final decision.

From this viewpoint, this study proposes a general idea for solving the many-objective optimization (MAOP) problem in which more than several objectivefunctions are considered simultaneously. Effort is devoted to obtain a uniquesolution known as the preferentially optimal solution or the best-compromisesolution. This approach is notably different from that of multi-objective evolutionaryalgorithms (MOEA), which attempt to derive only the Pareto front (Coello 2001;Czyzak and Jaszkiewicz 1998; Deb et al. 2000; Jaeggi et al. 2005; Robic and Filipic2005). However, recent studies have revealed that even in MOEA, conventionalmethods are not necessarily effective for dealing with MAOP problems (Hughes2005; Sato et al. 2010).

In this context, we extend our previously proposed methods, named MOON2 andMOON2R (Shimizu and Kawada 2002; Shimizu et al. 2004), to be able to handleMAOP problems. Although MOON2 and MOON2R require only simple and relativeresponses, handling the decision makers’ (DMs’) responses in trade-off analysisbecomes rather difficult in MAOP. To overcome this difficulty, this study proposesan approach that is easily applicable to MAOP. Consequently, the proposed idea canextend the applicability and practicality of existing methods to the complex decisionmaking environments mentioned above.

The rest of this chapter is organized as follows. In Sect. 3.2, the generalprocedures of MOON2 and MOON2R are explained. Section 3.3 extends thisprocedure to MAOP. In Sect. 3.4, the validity and effectiveness of the proposedmethod is verified by applying it to an actual problem. A general discussion is alsopresented in that section to give a definite and comprehensive outline of the directionof future work in this area. A conclusion is given in Sect. 3.5.

3.2 MOON2 and MOON2R for MUOP and MAOP

General MUOP problems are described as follows.

.p: 1/ Min f .x/ D ff1 .x/ ; f2 .x/ ; : : : ; fN .x/g subject to x 2 X ;

where x denotes a decision variable vector; X, is a feasible region; and f is anobjective function vector, some elements of which are incommensurable and conflictwith one another. When n > 3, this problem is commonly referred to as a MAOPproblem. The abbreviation MOP is used below in cases where the distinctionbetween MUOP and MAOP is irrelevant.

As a particular characteristic of MOP, in addition to the mathematical procedures,we need some information on the DM’s preference to obtain the best-compromisesolution as a final goal. The solution methods of MOP problems are generally classi-fied as prior articulation methods or interactive methods (Shimizu 2010). Naturally,

3 An Extension of the MOON2/MOON2R Approach to Many-Objective. . . 53

each of these conventional methods has both advantages and disadvantages. Forexample, since in the former method a value function is derived separately fromthe search process, the DM does not need to perform repeated interactions duringthe search process, whereas such interactions are required in the latter method.On the other hand, although the latter method allows for elaborate articulation ofattainability among the conflicting objectives, such articulation is difficult to obtainwith the former method. Consequently, the derived solution may sometimes differsubstantially from the best-compromise solution provided by the DM.

MOEA methods, which differ substantially from the two methods mentionedabove, have been developed recently. However, these methods require further stepsbefore attaining the final solution because the DM has to find the best solutionamong a potentially large number of candidates scattered along the Pareto front. Incontrast, MOON2 and MOON2R can readily derive the best-compromise solutionwhile being free from the requirement of repeated responses during the search,without giving up elaborate trade-off analysis. Therefore, MOON2 and MOON2R

are expected to serve as powerful tools for enabling flexible decision making inagile engineering under diverse customer requirements.

Because MOON2 and MOON2R belong to the prior articulation methods in MOP,they have to identify the value function of the DM in advance. Such modeling canbe performed with a suitable artificial neural network to deal with the non-linearitycommonly seen in the value function. A back-propagation network (BPN) is usedin MOON2, while MOON2R employs a radial-basis function network (RBFN).

To train the neural network, training data representing the preferences of theDM should be gathered by an appropriate means. These methods use pairwisecomparison among the appropriate trial solutions, which are spread over the searcharea in the objective-function space. It is natural to constrain this modeling spaceto within the convex hull enclosed by the utopia and nadir solutions, which aredefined as f * D (f1(x utop), f2(x utop), : : : , fN(x utop))T and f* D (f1 (x nad), f2(x nad),: : : , fN(x nad)) T, respectively, where x utop and x nad are the respective utopia andnadir solutions in the decision variable space.

Then, the DM is asked to indicate the preferred solution and the spacingbetween each pair of trial solutions, for example, f i D f(xi) and f j D f(x j), xi, x j2X.These responses are provided in the form of linguistic statements, which are latertransformed into scores denoted as aij (Table 3.1), similarly to the analytic hierarchyprocess (AHP) (Saaty 1980). For example, when the answer is such that f i isstrongly preferable to f j, aij takes a value of 5 (Table 3.1).

Table 3.1 Conversion tablefor linguistic statements

Linguistic statement aij

Equally 1Moderately 3Strongly 5Demonstrably 7Extremely 9Intermediate values between adjacent statements 2, 4, 6, 8

54 Y. Shimizu

Fig. 3.1 Learning processusing RBFN

Table 3.2 Pairwisecomparison

f 1f 1 f 2 f 3 f k

f 2

f 3

f k1

1

1

1

1aij =

a ij

By performing such pairwise comparisons over k trial solutions, we can obtaina pairwise comparison matrix (PCM) (Table 3.2). Element aij represents the degreeof preference of f j compared to f i. Note that although aij is defined as the ratio ofrelative degrees of preference, it does not necessarily mean that f i is aij times morepreferable to f j. According to the same conditions as AHP, such that aii D1 and aji

D1/aij, the DM is required to provide k(k-1)/2 responses for the pairs highlighted inTable 3.2. Under these conditions, it is also easy to examine the consistency of suchpairwise comparisons from the consistency index CI used in AHP.

Since information on the preferences of the DM is embedded in the PCM,we can derive a value function based on it. However, in general, it is almostimpossible to give a mathematically definite form of the value function, as it islikely to be highly nonlinear. Unstructured modeling techniques that use neuralnetworks are suitable for modeling in such situations. All objective values of eachpair f i and f j (8i, j2f1, 2, : : : , kg) are used as 2 N inputs of the neural network,and aij is the single output. Hence, PCM provides a total of k2 training data setsfor the neural network. Eventually, the trained neural network can be viewed asan implicit function mapping the 2 N dimensional space to the scalar space (i.e.,VNN W f i; f j

� 2 R2N ! aij 2 R).Next, looking at the relations in Eq. (3.1), we can easily compare the preferences

for any pair of solutions. Therefore, by fixing one of the input vectors of the neuralnetwork at an appropriate reference vector f R, we can evaluate any solution fromthe output of the neural network (Eq. (3.2)). In other words, VNN can serve as avalue function. We can nominate some candidates for the reference point f R, such


as utopia, nadir, a center of gravity between them, or the point where the total sumof distances from all trial points is a minimum.

VNN

f i; f k� D aik > VNN

f j; f k

� D ajk () f i � f j (3.1)

VNN

f .x/ ; f R� D axR > VNN

f .y/ ; f R� D ayR () f .x/ � f .y/; 8x; y 2 X(3.2)

Once the value function is identified, the original MOP problem is transformedinto an ordinal single-objective problem.

.p:2/ Max VNN

f .x/; f R�

subject to x 2 X

Because the value function is built separately from the search process, a DMcan carry out trade-off analyses whatever pace is desired without having to provideimmediate responses or wait for queries, as is often required in interactive methods.In addition, because the required responses are simple and relative, the load on theDM in such interaction is rather small. These are some of the notable advantagesof this approach. Moreover, the following proposition supports the validity of theabove formulation.

[Proposition] The optimal solution of Problem (p. 2) is a Pareto optimal solutionof Problem (p. 1) if the value function is chosen so as to satisfy the relation givenby Eq. (3.1).

(Proof) Letbf �i , (i D 1, : : : , N) be the values of the objective functions for the

optimal solutionbx� of Problem (p. 2), so thatbf �i D fi

bx��. Here, let us assume for

contradiction thatbf �is not a Pareto optimal solution. Then there exists a certain

f 0 such that for 9j, f 0j < bf �j � �fj;

�fj > 0

�) and f 0j � bf �

i , (i D 1, : : : , N,

i ¤ j). Because the DM apparently prefers f 0 tobf �, it holds that VNN

f 0; f R

�>

VNN

�bf �; f R

�. This contradicts thatbf �

is the optimal solution of Problem (p. 2).

Hence,bf �must be a Pareto optimal solution.

Once x is given, we can readily evaluate any candidate solution through VNN .Hence, it is possible to choose the most appropriate method from among a variety ofconventional single-objective optimization methods. In addition to direct methods,meta-heuristic methods such as genetic algorithms, simulated annealing and tabusearch are also applicable. At the same time, it is almost impossible to apply any ofthe interactive methods of MOP due to the large number of interactions during thesearch, which are likely to make the DM rather careless in providing responses.

When this approach is applied with an algorithm that requires the gradients ofthe objective function, such as nonlinear programming, we need to obtain thesegradients by numeric differentiation. The derivative of the value function withrespect to a decision variable is calculated by using the following chain rule.

56 Y. Shimizu

Fig. 3.2 Flowchart of theproposed method

Start

Generate trial sols.

Perform pair comparisons

Select Optimization Method

Need gradients?

Incorporate Numerical differentiation

Apply Optimization algorithm

Limit the space

No

No

Yes

Yes

Yes

No

Set utopia/nadir & Searching space

Consistent?

Identify VNN by NN

Incorporate Numerical differentiation

Satisfactory?

END

Limit the space

No

No

Yes

Yes

No

@VNNf .x/ ; f R

�@x

D @VNN

f .x/ ; f R

�@f .x/

! @f .x/@x

�(3.3)

The derivative can be calculated from the analytic form of the second part in theright-hand side of Eq. (3.3) and the following numeric differentiation. Since mostnonlinear programming software supports numeric differentiation, the algorithm canbe realized without any special concerns.

@VNN@fi

Š VNN.�� ;fi.x/C�fi;�� I f R/�VNN.�� ;fi.x/;�� I f R/� fi

(3.4)

The proposed procedure can be summarized as follows (Fig. 3.2).

Step 1: Generate several trial solutions in the objective-function space.


Step 2: Extract the preferences of the DM through pairwise comparison betweenevery pair of trial solutions.

Step 3: Train the neural network with the preference information obtained fromthe above responses. This network serves as a value function VNN by selecting acertain reference solution f R.

Step 4: Apply an appropriate optimization method to solve the resulting Prob-lem (p. 2).

Step 5: If the DM is not satisfied with the result obtained in the above process, limitthe search space around that result and repeat the same procedure until he or sheaccepts the result.

3.3 Procedure for MAOP

Because the aforementioned methods are natural and easy to work with for valueassessments by humans, we have applied them to various problems and confirmedtheir effectiveness (Shimizu et al. 2005, 2006, 2010, 2012a; Shimizu and Tanaka2003; Shimizu and Nomachi 2008). However, the case of MAOP is different if weconsider the limit to the abilities of humans to perform assessment. As the numberof objective functions increases, the difficulty of such value assessment throughpairwise comparison increases rapidly. For example, suppose that a customerintends to buy a ticket for transportation in a certain situation. It seems rather easyto choose between a pair of candidates if they are evaluated on only two objectives,such as travel time and expense. According to the procedure outlined above, in thiscase, the customer has to make a pairwise comparison between the pair of solutions(i, j) in terms of the objectives (time i, cost i) and (time j, cost j), respectively.However, what will happen if there are more objectives to be compared? Supposethat the customer has to compare a pair of candidates in terms of four objectives:time, cost, service and comfort. Undoubtedly, the difficulty of assessment will growsubstantially, and the customer may often give up on the comparison altogether,except in special cases.

For MAOP, therefore, it is impractical to deploy the proposed idea whilemaintaining the portability of the previous method. The basic idea of the proposedprocedure involves replacing the pairwise comparison on many objectives with acomparison on a scalar objective. Assuming independence of the objective functionsof (p.1), this procedure can be realized by the following steps.

Step 1: Determine the relative importance among the objective functions as weightswk; .k D 1; : : : ; N/ ; such that

Xk

wk D 1, through pairwise comparison and

eigenvalue calculation, as in AHP. Repeat if the pairwise comparison fails theconsistency test.

Step 2: Narrowing the focus to the kth objective function only, ask the DM to givea preference for every pair of trial solutions f i D f

xi�

and f j D fx j�, ffk(xi),

58 Y. Shimizu

fk(x j)g .8i; j; i > j/, and obtain the preference intensity sik;8i by calculating

the eigenvalues of this PCM. Repeat this process for every objective function.

Step 3: Calculate the total preference of the ith trial as Si DNX

kD1wksi

k;8i.

Step 4: Finally, calculate aij, which is the PCM element corresponding to thepreference between f i and f j, as aij D Si=Sj.

Step 5: Similarly to the previous step, identify the value function of the DM from fi

and f j as the inputs and aij as the output of the neural network.

The above procedure can be easily implemented by a DM who is familiar withAHP, and does not introduce additional complexity to the original procedures ofMOON2 and MOON2R.

3.4 Case Study

3.4.1 Evaluation Method

To verify the feasibility of our approach, we applied it to a problem assuming avirtual DM whose value function is given by Eq. (3.5) as a reference. We comparedthe result obtained by the proposed method with that from the optimization problemby using the following comprehensive objective function of (p.1):

U .f .x// D(

NXk

wk

fk .x/ � fk�

f �k � fk�

�t) 1.

t; (3.5)

where f �k and fk� denote the utopia and nadir values of the kth objective function,

respectively. Moreover, wk and t are a weight representing relative importance anda norm parameter, respectively. Hence, U( f(x)) represents the attainability ratio forutopia and takes a value of 1.0 for the utopia and 0.0 for the nadir.

We carried out the experiment along with the following procedures that corre-spond to those in Sect. 3.3.

Step 1: Determine a set of weights wk .P

wk D 1/, each of which stands forthe relative importance of the corresponding objective function regarding thepreference.

Step 2: Instead of interactive pairwise comparison, for the virtual DM, obtain the

preference index sik of the ith trial for the kth objective as si

k D�

f ik�fk�

f �

k �fk�

�t,

(8k; 8i).

Step 3: Calculate the total preference score Si as Si D

NXk

wksik

!1.t, .8i/.


Fig. 3.3 Welded beamdesign

Ll

h

t

P

bLl

h

t

P

b

Step 4: Obtain the ijth element of the PCM as aij D Si=Sj

Step 5: By using the data obtained above, train the neural network so that the relationVNN

f i; f j

� D aij; .8i; j/ is satisfied. Then, select an appropriate referencesolution f R.

Step 6: From the above steps, make Problem (p. 2) definite and solve it by anappropriate ordinal optimization method.

Step 7: Compare the above result with that of another optimization problem, suchas Max(Eq. (3.5)) subject to x 2 X.

3.4.2 Welded Beam Design Problem

We considered a welded beam design problem (Fig. 3.3) and described it as afour-objective optimization Problem (p. 3). This is originally studied in (Erfani andUtyuzhnikov 2012) as a bi-objective optimization problem.

(p. 3) Min ff1, f2, f3, f4g subject to Eqs. (3.6), (3.7), (3.8), (3.9), (3.10), (3.11),(3.12), (3.13), (3.14), (3.15), (3.16), and (3.17–3.20)

3.4.2.1 Objective Functions

f1 WD 1:105h2l C 0:048tb .L C l/ ! min.Cost/ (3.6)

f2 WD ı D 4PL3

Et3b! min .Deflection/ (3.7)

f3 WD � Dr� 02 C � 0� 00 l

RC � 002 ! min .Shear stress/ (3.8)

f4 WD D 6PL

t2b! min .Bending stress/ (3.9)

60 Y. Shimizu

3.4.2.2 Constraints

h � b (3.10)

Pc � P (3.11)

Pc D 64746:02 .1 � 0:3t/ tb3 (3.12)

� 0 D Pp2hl

(3.13)

� 00 D P .L C 0:5l/R=J (3.14)

R Dr0:25

�l2 C .h C t/2 (3.15)

J D p2hl

l2

12C .h C t/2

4

!(3.16)

0:125 � b � 5; 0:1 � t � 10; 0:1 � l � 10; 0:125 � h � 5; (3.17–3.20)

3.4.2.3 Decision Variables

h [m]: welding thickness; l [m]: welding length; t [m]: beam width; b [m]: beamthickness

3.4.2.4 Parameters

P D 6000:0 Œlb� ; L D 14:0 Œin� ; E D 3:0 E8 Œpsi�

3.4.3 Numerical Results

First, we described the objective tree as shown in Fig. 3.4. Then, we generated sixrandom trials within the hyper-rectangular space enclosed by the utopia and nadir,which are shown in Table 3.3 together with the test trials. Next, we set the weights


Bending stress

Design1 Design 3 Design 5Design 4 Design6

Beam design

Cost Deflection Shear stress

Design 2 Design 3

Fig. 3.4 Hierarchy of evaluation factors

Table 3.3 Specification of each trial with utopia and nadir

Design 1 Design 2 Design 3 Design 4 Design 5 Design 6 Utopia Nadir

Cost 20.00 9.27 16.49 11.52 13.75 11.13 5.00 20.00Deflection 4.11 E-03 5.56 E-03 6.66 E-03 6.82 E-03 2.32 E-03 3.80 E-03 1.00

E-038.00E-03

Shearstress

7281.69 8071.22 7333.68 5421.39 12214.13 9246.70 3200.00 13600.00

Bendingstress

29256.52 19066.19 18203.62 22550.54 29709.86 27521.33 15000.00 30000.00

Table 3.4 PCM (t D 1)

Design 1 Design 2 Design 3 Design 4 Design 5 Design 6

Design 1 1 0.51 0.84 0.61 0.67 0.57Design 2 1.95 1 1.63 1.18 1.30 1.10Design 3 1.20 0.61 1 0.72 0.80 0.68Design 4 1.65 0.85 1.38 1 1.10 0.94Design 5 1.50 0.77 1.25 0.91 1 0.85Design 6 1.76 0.91 1.48 1.07 1.18 1

representing the relative importance as w D (0.4, 0.3, 0.2, 0.1), which are the sameas those given for the reference value function in Eq. (3.5). Then, the preferenceintensity of every trial with respect to each objective function was derived from theformula given in Step 2. Finally, the total preference was calculated as S D (0.293,0.570, 0.350, 0.484, 0.439, 0.517) for t D 1. In Step 4, Si/Sj was calculated to derivethe elements of the PCM shown in Table 3.4. Based on that procedure, we built thevalue function VNN(f(x); f R) of the neural network.

Letting f R D f �, we solved (p. 3) under this value function by the modifiednonlinear simplex method (Nelder and Mead 1965) so that it can accommodate theconstraints. In Table 3.5, the result is compared with that obtained by optimizingProblem (p. 3) under the objective function in Eq. (3.5). This problem is solved byusing the commercial software package LINGO (Ver. 13.0).

62 Y. Shimizu

Table 3.5 Results of MUOP (Independent: t D 1)

Decision variable Objective function valuel t b h Cost Deflection Shear stress Bending stress

This work 2.540 3.328 2.650 1.135 10.614 2.249E-03 11262.08 17179.24LINGO 2.982 3.329 2.789 1.154 11.955 2.134E-03 9657.292 16307.03Gap [%] 14.82 0.03 4.98 1.65 11.22 5.39 16.62 5.35

(Input layer: 8 neurons; hidden layer: 10 neurons; learning rate: 0.5; momentum: 0.1; RSME:3.33 � 10�4)Gap D j This work � LINGO j /LINGO � 100

Table 3.6 PCM (Independent: t D 2)

Design 1 Design 2 Design 3 Design 4 Design 5 Design 6

Design 1 1 0.69 1.00 0.77 0.79 0.76Design 2 1.45 1 1.45 1.11 1.14 1.11Design 3 1.00 0.69 1 0.77 0.79 0.77Design 4 1.31 0.90 1.30 1 1.03 1.00Design 5 1.27 0.88 1.27 0.98 1 0.97Design 6 1.31 0.90 1.31 1.00 1.03 1

Table 3.7 Result of MUOP (Independent: t D 2)

Decision variable Objective function valuel t b h Cost Deflection Shear stress Bending stress

This work 1.857 3.329 2.736 1.149 9.637 2.176E-03 14463.65 16628.19LINGO 2.118 3.330 3.000 1.102 10.573 1.982E-03 13600.00 15151.22Gap [%] 12.32 0.03 8.80 4.27 8.85 9.79 6.35 9.75

(Input layer: 8 neurons; hidden layer: 10 neurons; learning rate: 0.5; momentum: 0.1; RSME:3.60 � 10�4)

In a similar manner, we had S D (0.408, 0.592, 0.409, 0.534, 0.520, 0.535) fort D 2 and obtained the results in Tables 3.6 and 3.7. Close correspondence can beobserved between the results, with a few exceptions.

3.4.4 Discussion

A definite basis for evaluating subjective decisions in a well-defined manner thatis acceptable to everyone does not exist. This fact causes considerable difficultywhen attempting to perform a general evaluation to obtain the best-compromisesolution found by the mathematical process of MOP. As it often happens, what oneperson considers the best compromise may not be acceptable to others since eachDM has a different value system. We are confident that the procedure outlined hereis applicable in such situations since the final preference is evaluated on the basis of


Table 3.8 Comparison ofvalue function values

t This work LINGO Gap [%]

1 0.627 0.633 0.922 0.692 0.691 0.18

a b

Fig. 3.5 Post-optimal analysis in terms of MUOP. (a) Result of elite-induced MOEA. (b) Resultobtained with "-constraint method

an implicitly embedded value function, such as Eq. (3.5). This is also a basic normof utility theory (Fishburn 1970).

Although some results in Tables 3.5 and 3.7 seem to be somewhat far from thereference solution, we can account for this weakness if we compare the resultsin terms of the above aspects. Both results in Table 3.8 are so similar that theDM cannot distinguish between them. Moreover, we confirmed that the best-compromise solution could not be outperformed by any of 200 solutions obtainedwith NSGA-II (Deb et al. 2000) after convergence. This numerically validates theproposition in Sect. 3.2, which asserts that the proposed method can derive a Paretooptimal solution.

In addition, we can use the result obtained for the post-optimal analysis combinedwith a classical multi-objective analysis method, such as the " constraint method,or recent approaches such as elite-induced evolutionary multi-objective analysis(Shimizu et al. 2012b). As illustrated in Fig. 3.5, by producing several solutionsaround the optimal result, we can move on to the next stage by choosing amongthose candidates to make a final decision for actual execution. Based on the abovediscussion, we again emphasize the validity of the proposed approach.

3.5 Conclusion

A MAOP method that supports flexible and adaptive decision making for complex,diverse and competitive decision environments has been in high demand. Fromthis viewpoint, this study proposed a general idea for solving MAOP problems byextending our previously proposed MUOP methods (MOON2 and MOON2R).

64 Y. Shimizu

Although MOON2 and MOON2R require only simple and relative responses,handling the DM’s responses in trade-off analysis becomes rather difficult in MAOP,where more than a few objective functions are to be considered simultaneously. Toovercome this difficulty, this study proposed an approach that is easily applicablein such cases. After presenting the general procedure, the effectiveness of theproposed method was verified by applying it to an actual problem. The experimentalresults showed that the proposed method is moderately more complex than previousmethods but maintains flexibility and adaptability. Finally, the general discussionprovided a definite and comprehensive outline of the direction of future work inthis area.

References

Coello CAC (2001) A short tutorial on evolutionary multiobjective optimization. In: Zitzler E et al(eds) Lecture notes in computer science. Springer, Berlin, pp 21–40

Czyzak P, Jaszkiewicz AJ (1998) Pareto simulated annealing-a meta-heuristic technique formultiple-objective combinatorial optimization. J Multi-Criteria Decis Anal 7:34–47

Deb K, Agrawal S, Pratap A, Meyarivan T (2000) A fast elitist non-dominated sorting geneticalgorithm for multi-objective optimization: NSGA-II. In: Proceedings Parallel Problem Solvingfrom Nature VI (PPSN-VI). Paris, France, pp 849–858

Erfani T, Utyuzhnikov SV (2012) Control of robust design in multiobjective optimization underuncertainties. Struct Multidiscip Optim 45:247–256

Fishburn PC (1970) Utility theory for decision making. Wiley, New YorkHughes EJ (2005) Evolutionary many-objective optimization. Many once or one many? In:

Proceedings IEEE Congress on Evolutionary Computation (CEC2005). Edinburgh, UK,pp 222–227

Jaeggi D, Parks G, Kipouros T, Clarkson J (2005) A multiobjective tabu search algorithmfor constrained optimization problems. In: Evolutionary multi-criterion optimization, thirdinternational conference, EMO 2005, LNCS 3410. Guanajuato, Mexico, pp 490–504

Nelder JA, Mead R (1965) A simplex method for functional minimization. Comput J 7:308–313Robic T, Filipic B (2005) DEMO: differential evolution for multi-objective optimization. In: Evo-

lutionary multi-criterion optimization, third international conference (EMO 2005), Guanajuato.LNCS 3410, pp 520–533

Saaty TL (1980) The analytic hierarchy process. McGraw-Hill, New YorkSato H, Aguirre H, Tanaka K (2010) Many-objective evolutionary optimization by self-controlling

dominance area of solutions. Trans Jpn Soc Evol Comput 1(1):32–41Shimizu Y (2010) An enhancement of learning optimization engineering – workbench for smart

decision making. CORONA Publishing Co., LTD. (In Japanese)Shimizu Y, Kawada A (2002) Multi-objective optimization in terms of soft computing. Trans Soc

Instrum Control Eng 38(11):974–980Shimizu Y, Nomachi T (2008) Integrated product design through multi-objective optimization

incorporated with meta-modeling technique. J Chem Eng Jpn 41(11):1068–1074Shimizu Y, Tanaka Y (2003) A practical method for multi-objective scheduling through soft

computing approach. JSME Int J Ser C 46(1):54–59Shimizu Y, Tanaka Y, Kawada A (2004) Multi-objective optimization system on the internet.

Comput Chem Eng 28(5):821–828Shimizu Y, Miura K, Yoo J-K, Tanaka Y (2005) A progressive approach for multi-objective design

through inter-related modeling of value system and meta-model. J JSME Ser C 71(712):296–303. (In Japanese)


Shimizu Y, Yoo J-K, Tanaka Y (2006) A design support through multi-objective optimization awareof subjectivity of value system. J JSME Ser C 72(717):1613–1620. (In Japanese)

Shimizu Y, Kato Y, Kariyahara T (2010) Prototype development for supporting multiobjectivedecision making in an ill-posed environment. J Chem Eng Jpn 43(8):691–697

Shimizu Y, Waki T, Sakaguchi T (2012a) Multi-objective sequencing optimization for mixed-model assembly line considering due-date satisfaction. J Adv Mech Des Syst Manuf 6(7):1057–1070

Shimizu Y, Takayama M, Ohishi H (2012b) Multi-objective analysis through elite-inducedevolutionary algorithm – in the case of PSA. Trans Jpn Soc Evol Comput 3(2):22–30. (InJapanese)

Yoshiaki Shimizu is a Professor in Department of Mechanical Engineering, Toyohashi Universityof Technology, Japan. He was responsible for the head of department of Production SystemsEngineering during 2006–2009.

He was graduated from Kyoto University in Japan, and earned Doctor of Engineering in 1982.His teaching and research interests include production systems and supply chain management,multi-objective optimization and applied operations research. He is the author of more than 200academic and technical papers and books. See more detail on his home page (URL http://ise.me.tut.ac.jp/). His email address is [email protected]

Chapter 4Existence of Solutions for Variational-LikeHemivariational Inequalities Involving LowerSemicontinuous Maps

Guo-ji Tang, Zhong-bao Wang, and Nan-jing Huang

Abstract The main aim of this chapter is to investigate the existence of solutions inconnection with a class of variational-like hemivariational inequalities in reflexiveBanach spaces. Some existence theorems of solutions for the variational-likehemivariational inequalities involving lower semicontinuous set-valued maps areproved under different conditions. Moreover, a necessary and sufficient conditionto guarantee the existence of solutions for the variational-like hemivariationalinequalities is also given.

Keywords Variational-like hemivariational inequality • Generalized monotonic-ity • Set-valued map • Existence • Mosco’s alternative

2010 Mathematics Subject Classification: 49J40; 49J45; 47J20.

4.1 Introduction

Different from the fact that the variational inequality is mainly concerned withconvex energy functions, the hemivariational inequality, first introduced by Pana-giotopulos (Panagiotopoulos 1983, 1985, 1991, 1993) in the early 1980s, is closely

G.-j. TangSchool of Science, Guangxi University for Nationalities, Nanning, Guangxi 530006, People’sRepublic of Chinae-mail: [email protected]

Z.-b. WangDepartment of Mathematics, Southwest Jiaotong University, Chengdu, 610031, People’sRepublic of Chinae-mail: [email protected]

N.-j. Huang (�)Department of Mathematics, Sichuan University, Chengdu, Sichuan 610064, People’sRepublic of Chinae-mail: [email protected]


67

68 G.-j. Tang et al.

concerned with nonsmooth and nonconvex energy functions. This type of inequal-ities and their generalization play a crucial role in describing many importantproblems arising in mechanics and engineering, such as unilateral contact problemsin nonlinear elasticity, thermoviscoelastic frictional contact problems and obstaclesproblems (see, for example, Carl et al. 2007; Naniewicz and Panagiotopoulos 1995;Motreanu and Radulescu 2003; Panagiotopoulos 1985, 1993 and the referencestherein). The derivative of hemivariational inequality is based on the generalizeddirectional derivative introduced by Clarke (1983). In the past of almost 30 years, thetheory of hemivariational inequalities has been developed a great deal of importantresults both in pure and applied mathematics as well as in other fields such asmechanics and engineering sciences, since it allowed mathematical formulationsfor some interesting problems (Carl 2001; Carl et al. 2005; Costea and Radulescu2009, 2010; Costea and Lupu 2010; Costea 2011; Costea et al. 2012; Costea andRadulescu 2012; Liu 2008; Migórski and Ochal 2004; Motreanu and Radulescu2000; Xiao and Huang 2009, 2008; Xiao et al. 2014; Zhang and He 2011).

On the other hand, Parida et al. (1989) introduced another new type of variationalinequality, called variational-like inequality, and showed that it can be relatedto some mathematical programming problems. For more related work regardingvariational-like inequalities, we refer to Fang and Huang (2003), Bai et al. (2006),Ansari and Yao (2001) and the references therein.

Let K be a nonempty, closed and convex subset in a real reflexive Banach spaceX. Assume that A W K ⇒ X� is a set-valued map, � W K � K ! X is a single-valuedmap and � W X ! R[fC1g is a convex and lower semicontinuous functional suchthat K� WD K \ dom� ¤ ;, where dom� WD fx 2 X W �.x/ < C1g is the effectivedomain of �. Let � be a bounded open set in R

N and j.x; y/ W � � Rk ! R be a

function. Let T W X ! Lp.�IRk/ be a linear and continuous mapping, where 1 <p < 1. We shall denote Ou WD Tu and denote by jı.x; yI h/, the Clarke’s generalizeddirectional derivative of a locally Lipschitz mapping j.x; / at the point y 2 R

k withrespect to the direction h 2 R

k, where x 2 �. We are interested in finding solutionsfor the following problem:

(P) Find u 2 K� such that

8u� 2 A.u/ W hu�; �.v; u/i C �.v/ � �.u/

CZ�

jı.x; Ou.x/I Ov.x/� Ou.x//dx � 0; 8v 2 K; (4.1)

which is related closely to the following problem:

Find u 2 K� such that

9u� 2 A.u/ W hu�; �.v; u/i C �.v/ � �.u/

CZ�

jı.x; Ou.x/I Ov.x/ � Ou.x//dx � 0; 8v 2 K: (4.2)

It is clear that a solution of problem (P) is necessarily the solution of problem (4.2)and the converse relation is not true in general. Particularly, if A is a single-valued

4 Existence of Solutions for Variational-Like Hemivariational Inequalities. . . 69

mapping, then problems (P) coincides with (4.2) . Sometimes, a solution of problem(P) is called as a strong solution of problem (4.2) (the similar notions can be referredto Costea et al. 2012; Tang et al. 2014). To our best knowledge, the strong solutionof hemivariational inequalities involving set-valued maps (for example, a solutionof problem (P) other than (4.2)) was considered in few papers (Tang et al. 2014).Moreover, we would like to mention that problems (P) and (4.2) are more generalones because they include some problems as special cases such as:

• If j is a constant on��Rk, then problems (P) and (4.2) become, respectively, as

follows:

8u� 2 A.u/ W hu�; �.v; u/i C �.v/ � �.u/ � 0; 8v 2 K (4.3)

and

9u� 2 A.u/ W hu�; �.v; u/i C �.v/ � �.u/ � 0; 8v 2 K; (4.4)

which were considered by Costea et al. (2012). If, in addition, �.v; u/ D v � u,then problems (4.3) and (4.4) become to

8u� 2 A.u/ W hu�; v � ui C �.v/ � �.u/ � 0; 8v 2 K (4.5)

and

9u� 2 A.u/ W hu�; v � ui C �.v/ � �.u/ � 0; 8v 2 K; (4.6)

which were called (generalized) mixed variational inequalities and studiedextensively by many authors (see, for example, Tang and Huang (2014, 2013a)and the references therein).

If �.v; u/ D v � u, problem (4.2) reduces to the following problem:

9u� 2 A.u/ W hu�; v � ui C �.v/ � �.u/

CZ�

jı.x; Ou.x/I Ov.x/ � Ou.x//dx � 0; 8v 2 K; (4.7)

which is called variational hemivariational inequality (see, for example, Costeaand Lupu 2010; Tang and Huang 2013b).

• If A is single-valued and � D IK , the indicator function on the constraint set K,then problems both (P) and (4.2) reduce to the problem:

hA.u/; �.v; u/i CZ�

jı.x; Ou.x/I Ov.x/� Ou.x//dx � 0; 8v 2 K; (4.8)

which was considered by Costea and Radulescu (2009).


• If A is single-valued and �.v; u/ D v�u, then problems both (P) and (4.2) reduceto the problem:

hA.u/; v�uiC�.v/��.u/CZ�

jı.x; Ou.x/I Ov.x/�Ou.x//dx � 0; 8v 2 K; (4.9)

which was studied by Motreanu and Radulescu (2000). If, in addition, � D IK ,then problem (4.9) becomes to

hA.u/; v � ui CZ�

jı.x; Ou.x/I Ov.x/� Ou.x//dx � 0; 8v 2 K;

which was introduced and studied by Panagiotopoulos et al. (1999).

Extensive attention has been paid to the existence results for some types ofhemivariational inequalities by many researchers in recent years (see, for example,Carl 2001; Carl et al. 2005, 2007; Xiao and Huang 2009; Migórski and Ochal2004; Park and Ha 2008, 2009; Goeleven et al. 1998; Liu 2008; Zhang and He2011; Tang and Huang 2013b; Costea and Lupu 2010; Xiao and Huang 2008;Costea and Radulescu 2009 and the references therein). In particular, some authorsconsidered some classes of variational-like hemivariational inequalities (see, forexample Costea and Radulescu 2009; Xiao and Huang 2008). It is also worthmentioning that, under some generalized monotonicity assumptions, Costea et al.(2012) investigated some results concerned with the existence of solutions forproblems (4.3) and (4.4) involving set-valued mappings.

In this chapter, we continue to study the existence of solutions for problem (P)involving lower semicontinuous set-valued maps in reflexive Banach spaces. Weprove the existence of solutions for problem (P) when K is compact convex andbounded closed convex, respectively. In the case when K is unbounded, we studythe existence of solutions and the boundedness of the solution set for problem (P)under some coercivity conditions. Moreover, a necessary and sufficient condition tothe existence of solutions for problem (P) is also derived. We would like to pointout that the results presented in this chapter generalize and improve some knownresults due to Costea and Radulescu (2009), Costea and Lupu (2010), Costea et al.(2012), Motreanu and Radulescu (2000), Panagiotopoulos et al. (1999), and Tangand Huang (2013b).

4.2 Preliminaries

Let X be a reflexive Banach space with the norm denoted by k k, X� be its dualspace. For a nonempty, closed and convex subset K of X and every r > 0, we define

Kr WD fu 2 K W kuk � rg and K�r WD fu 2 K W kuk < rg:


Let T W X ! Lp.�IRk/ be a linear compact operator, where 1 < p < 1 and k � 1,and� be a bounded open set in R

N . Denote by q the conjugated exponent of p, i.e.,1p C 1

q D 1.Recall that f ı.xI v/ denotes Clarke’s generalized directional derivative of the

locally Lipschitz mapping f W X ! R at the point x 2 X with respect to the directionv 2 X, while @f .x/ is the Clarke’s generalized gradient of f at x 2 X (see, forexample Clarke 1983), i.e.,

f ı.xI v/ D lim supy!x;t!0C

f .y C tv/ � f .y/

t

and

@f .x/ D f� 2 X� W h�; vi � f ı.xI v/; 8v 2 Xg:

Lemma 4.1 (Proposition 2.1.1 of Clarke 1983). Let f W K ! R be Lipschitz ofrank M near x. Then

(i) The function v ! f ı.xI v/ is finite, positively homogeneous and subadditiveon X, and satisfies

jf ı.xI v/j � MkvkI

(ii) f ı.xI v/ is upper semicontinuous as a function of .x; v/ and, as a function of valone, is Lipschitz of rank M on X;

(iii) f ı.xI �v/ D .�f /ı.xI v/.In order to solve problem (P), we need the following hypotheses about j.

(Hj) Let j W � � Rk ! R be a function which satisfies:

(i) For every y 2 Rk, j.; y/ W � ! R is measurable;

(ii) For all x 2 �, the mapping j.x; / is locally Lipschitz;(iii) There exists C > 0 such that

jzj � C.1C jyjp�1/; 8x 2 �;8z 2 @j.x; y/:

Now we consider the mapping J W Lp.�IRk/ ! R defined by

J.'/ DZ�

j.x; '.x//dx: (4.10)

Under the hypotheses (Hj), we can apply the Aubin-Clarke theorem (see e.g. Aubinand Clarke 1979 or Motreanu and Radulescu 2003) to conclude that the functionalJ defined above is locally Lipschitz and

Jı.wI z/ �Z�

jı.x;w.x/I z.x//dx; 8w; z 2 Lp.�IRk/:


Consequently,

Jı.OuI Ov/ �Z�

jı.x; Ou.x/I Ov.x//dx; 8u; v 2 X: (4.11)

Definition 4.1. Let E and F be two Hausdorff topological spaces. A set-valued mapT W E ⇒ F is said to be

(i) Lower semicontinuous at x0 iff, for any open set V � F such that T.x0/\ V ¤;, we can find a neighborhood U of x0 such that T.x/ \ V ¤ ; for all x 2 U;

(ii) Lower semicontinuous iff, it is lower semicontinuous at each x 2 E;(iii) Lower semicontinuous iff, the restriction of T to every line segment of K is

lower semicontinuous.

We denote by G.T/ WD f.x; y/ W x 2 E and y 2 T.x/g the graph of T. It is wellknown that there is an equivalent characterization for a lower semicontinuous maps(see, for example, item (i) of Proposition 2.1 of Costea et al. 2012).

Lemma 4.2. Let E and F be two Hausdorff topological spaces. Then a set-valuedmap T W E ⇒ F is lower semicontinuous iff, for any pair .x; y/ 2 G.T/ and anynet fxg2I � E converging to x, we can determine, for each 2 I, an elementy 2 T.x/ such that y ! y.

The following result is a fixed point theorem for set-valued maps due to Ansariand Yao (1999), which plays an important role in proving the existence of solutionsof problem (P) in the case of compact convex subsets in reflexive Banach spaces.

Lemma 4.3. Let K be a nonempty, closed and convex subset of a Hausdorfftopological vector space E and let S;T W K ⇒ K be two set-valued maps. Assumethat

• For each x 2 K, S.x/ be nonempty and convfS.x/g � T.x/;• K D [y2K intKS�1.y/;• If K is not compact, there exists a nonempty, compact and convex subset C0 of K

and a nonempty and compact subset C1 of K such that, for each x 2 KnC1, thereexists Ny 2 C0 with the property that x 2 intKS�1.Ny/.

Then there exists x0 2 K such that x0 2 T.x0/.

The next lemma is known as Mosco’s Alternative (see Mosco 1976) and plays acrucial role in proving the existence theorems for problem (P) in the next section.

Lemma 4.4 (Mosco’s Alternative). Let K be a nonempty, compact and convexsubset of a topological space E and assume � W E ! R

n [ fC1g is a proper,convex and lower semicontinuous function such that K� ¤ ;. Let �; � W E � E ! R

be two functions such that

• �.x; y/ � �.x; y/ for all x; y 2 E;• For each x 2 E, the map y 7! �.x; y/ is lower semicontinuous;• For each y 2 E, the map x 7! �.x; y/ is concave.


Then, for each 2 R, the following alternative holds true: either there exists y0 2K� such that

�.x; y0/C �.y0/ � �.x/ � ; for all x 2 E;

or there exists x0 2 E such that �.x0; x0/ > .

Definition 4.2. Let � W K � K ! X and ˛ W X ! R be two single-valued maps.A set-valued map T W K ⇒ X� is said to be relaxed � � ˛ monotone iff, for allu; v 2 K, all v� 2 T.v/ and all u� 2 T.u/, one has

hv� � u�; �.v; u/i � ˛.v � u/: (4.12)

Remark 4.1. If ˛ D 0 in (4.12), then T is said to be ��monotone. If �.u; v/ D u�vin (4.12), then T is said to be relaxed ˛ monotone. If �.u; v/ D u � v and ˛.z/ Dkkzkp with constants k > 0 and p > 1 in (4.12), then T is said to be p�monotone,if, in addition, p D 2, then T is called strongly monotone. If �.u; v/ D u � v and˛ D 0 in (4.12), then T is said to be monotone.

For some examples related to relaxed �-˛ monotone mappings, the readers canbe referred to Costea et al. (2012).

4.3 Existence Theorems

In order to prove our existence results, we shall use some of the followinghypotheses, which have ever been extensively used in recent literatures (see, e.g.Costea and Radulescu 2009, 2012; Costea et al. 2012):

(H1A) A W K ⇒ X� is a set-valued mapping which is lower semicontinuous from K

with the strong topology into X� with the weak topology, and has nonemptyvalues;

(H2A) A W K ⇒ X� is a set-valued mapping which is lower semicontinuous from K

with the strong topology into X� with the weak topology, and has nonemptyvalues;

(H1�) � W K � K ⇒ X is such that

(i) For all v 2 K, the map u 7! �.v; u/ is continuous;(ii) For all u; v;w 2 K and w� 2 A.w/, the map v 7! hw�; �.v; u/i is convex

and hw�; �.v; u/i � 0;

(H2�) � W K � K ⇒ X is such that

(i) �.u; v/C �.v; u/ D 0 for all u; v 2 K;(ii) For all u; v;w 2 K and w� 2 A.w/, the map v 7! hw�; �.v; u/i is convex

and lower semicontinuous;


(H�) � W X ! R[ fC1g is a proper, convex and lower semicontinuous functionalsuch that K� WD K \ D.�/ is nonempty;

(H˛) ˛ W X ! R is weakly lower semicontinuous functional such thatlim sup#0

˛.v/

� 0 for all v 2 X.

In the sequel, we shall study three cases regarding the constraint set K:

1. K a nonempty, compact and convex subset of a real reflexive Banach space X;2. K a nonempty, bounded, closed and convex subset of a real reflexive Banach

space X;3. K a nonempty, unbounded, closed and convex subset of a real reflexive Banach

space X.

Theorem 4.1. Let X be a real reflexive Banach space and K a nonempty, compactand convex subset of X. Assume that (H1

A), (H1�), (H�) and (Hj) hold. Then problem

(P) admits at least one solution.

Proof. Arguing by contradiction, let us assume that problem (P) has no solution.Then, for each u 2 K� , there exist Nu� 2 A.u/ and v D v.u; Nu�/ 2 K such that

hNu�; �.v; u/i C �.v/ � �.u/CZ�

jı.x; Ou.x/I Ov.x/� Ou.x//dx < 0: (4.13)

Now we define a functional J W Lp.�IRk/ ! R as follows

J.'/ DZ�

j.x; '.x//dx:

Thus, combining (4.13) and (4.11), we have

hNu�; �.v; u/i C �.v/ � �.u/C Jı.OuI Ov � Ou/ < 0: (4.14)

Clearly, the element v for which (4.14) takes place satisfies v 2 D.�/, thereforev 2 K� . We consider next a set-valued map F W K� ⇒ K� defined by

F.u/ D fv 2 K� W hNu�; �.v; u/i C �.v/ � �.u/C Jı.OuI Ov � Ou/ < 0g;

where Nu� 2 A.u/ is some element that satisfies (4.14).

Claim 1. For each u 2 K� , the set F.u/ is nonempty and convex.

Let u 2 K� be arbitrarily fixed. Then (4.14) implies that F.u/ is nonempty. Letv1; v2 2 F.u/ and define w D v1 C .1� /v2 with 2 .0; 1/. By item (ii) of (H1

�),we have

hNu�; �.w; u/i � hNu�; �.v1; u/i C .1� /hNu�; �.v2; u/i: (4.15)


Since f ı.I / is positively homogeneous and subadditive (see item (i) of Lemma 4.1)and T is linear, we get

Jı.OuI Ow � Ou/ D Jı.OuI. Ov1 � Ou/C .1� /. Ov2 � Ou//� Jı.OuI Ov1 � Ou/C .1 � /Jı.OuI Ov2 � Ou/: (4.16)

Combining (4.15), (4.16) and the convexity of �, we conclude that

hNu�; �.w; u/i C �.w/ � �.u/C Jı.OuI Ow � Ou/D ŒhNu�; �.v1; u/i C �.v1/ � �.u/C Jı.OuI Ov1 � Ou/�

C.1 � /ŒhNu�; �.v2; u/i C �.v2/ � �.u/C Jı.OuI Ov2 � Ou/�� 0; (4.17)

which shows that w 2 F.u/. Therefore, F.u/ is a nonempty and convex subset of K� .

Claim 2. For each v 2 K� , the set F�1.v/ D fu 2 K� W v 2 F.u/g is open.

Let us fix v 2 K� . Taking into account that

F�1.v/ D fu 2 K� W 9Nu� 2 A.u/ s.t. hNu�; �.v; u/iC�.v/��.u/CJı.OuI Ov� Ou/ < 0g;

we shall prove

ŒF�1.v/�cDfu 2 K� W hu�; �.v; u/iC�.v/��.u/CJı.OuI Ov�Ou/�0 for all u� 2 A.u/g

is a closed subset of K� . Let fug2I � ŒF�1.v/�c be a net converging to someu 2 K� . Then, for each 2 I, we have

hu�; �.v; u/i C �.v/ � �.u/C Jı. OuI Ov � Ou/ � 0 for all u�

2 A.u/: (4.18)

By item (i) of (H1�), one has

�.v; u/ ! �.v; u/: (4.19)

For each u� 2 A.u/ and 2 I, applying item (i) of Lemma 4.2, we can determineu� 2 A.u/ such that

u� * u�

since A is lower semicontinuous from K with the strong topology into X� with theweak topology. This, together with (4.19), shows that

hu�; �.v; u/i ! hu�; �.v; u/i: (4.20)


Since T is linear and u ! u and u ! u, we know that Ou ! Ou and Ov� Ou ! Ov� Ou.By item (ii) of Lemma 4.1, we have

lim sup

Jı. OuI Ov � Ou/ � Jı.OuI Ov � Ou/: (4.21)

Using (4.18), (4.20), (4.21) and the lower semicontinuity of �, one has

0 � lim supŒhu�; �.v; u/i C �.v/ � �.u/C Jı. OuI Ov � Ou/�

� lim suphu�; �.v; u/i C �.v/ � lim inf�.u/C lim sup Jı. OuI Ov � Ou/

� hu�; �.v; u/i C �.v/ � �.u/C Jı.OuI Ov � Ou/; (4.22)

which shows that u 2 ŒF�1.v/�c and so ŒF�1.v/�c is a closed subset of K� .

Claim 3. K� D [v2K� intK�F�1.v/.

Since F�1.v/ is a subset of K� for all v 2 K� , it is easy to see that[v2K� intK�F�1.v/ � K� . Now we prove that K� � [v2K� intK�F�1.v/. For eachu 2 K� , there exits v 2 K� such that v 2 F.u/ (such a v exists since F.u/ isnonempty) and so

u 2 F�1.v/ � [v2K�F�1.v/ D [v2K� intK�F�1.v/:

The compactness of K and the above claims ensure that all the conditions ofLemma 4.3 are satisfied for S D T D F. Thus, we deduce that the set-valued mapF W K� ⇒ K� admits a fixed point u0 2 K� , i.e., u0 2 F.u0/. This can be rewrittenequivalently as

0 D h Nu0�; �.u0; u0/i C �.u0/� �.u0/C Jı. Ou0I Ou0 � Ou0/ < 0:

Thus, we get a contradiction which completes the proof.

Remark 4.2. If j is a constant on � � Rk, then Theorem 4.1 reduces to the

corresponding result of Theorem 3.2 presented by Costea et al. (2012).

Theorem 4.2. Let K be a nonempty, bounded, closed and convex subset of the realreflexive Banach space X. Let A W K ⇒ X� be a relaxed �-˛ monotone map andassume that (H2

A), (H2�), (H˛), (H�) and (Hj) hold. Then problem (P) admits at least

one solution.

Proof. In order to prove the conclusion, we shall apply Mosco’s Alternative for theweak topology. First, we note that K is weakly compact as it is a bounded, closedand convex subset of the real reflexive space X and � W X ! R [ fC1g is weaklylower semicontinuous as it is convex and lower semicontinuous. Now we definethree functionals J W Lp.�IRk/ ! R and �; � W X � X ! R as follows:


J.'/ DZ�

j.x; '.x//dx;

�.v; u/ D � infv�2A.v/

hv�; �.v; u/i � Jı.OuI Ov � Ou/C ˛.v � u/;

and

�.v; u/ D supu�2A.u/

hu�; �.u; v/i � Jı.OuI Ov � Ou/:

Let us fix u; v 2 X and choose Nv� 2 A.v/ such that

h Nv�; �.v; u/i D infv�2A.v/

hv�; �.v; u/i:

For arbitrary fixed u� 2 A.u/, we have

�.v; u/� �.v; u/

D supu�2A.u/

hu�; �.u; v/i C infv�2A.v/

hv�; �.v; u/i � ˛.v � u/

� hu�; �.u; v/i C h Nv�; �.v; u/i � ˛.v � u/

D hNv� � u�; �.v; u/i � ˛.v � u/

� 0: .by the relaxed � � ˛ monotonicity/

Clearly,

� infv�2A.v/

hv�; �.v; u/i D supv�2A.v/

hv�; �.u; v/i:

It follows from conditions (H2�) and (H˛) that the map defined by

u 7! � infv�2A.v/

hv�; �.v; u/i C ˛.v � u/

is weakly lower semicontinuous. Furthermore, since T is a linear and compactoperator, we know that un * u implies Oun ! Ou and so lim sup

n!1Jı. OunI Ov � Oun/ �

Jı.OuI Ov � Ou/ by item (ii) of Lemma 4.1. Therefore, u 7! �.v; u/ is weakly lowersemicontinuous. By the fact that T is linear and item (i) of Lemma 4.1, we concludethat v 7! Jı.OuI Ov � Ou/ is convex. This, together with assumption (H2

�), implies thatv 7! �.v; u/ is concave. Since �.v; v/ D 0 for all v 2 X, by Mosco’s Alternative for D 0, we conclude that there exists u0 2 K� such that

�.v; u0/C �.u0/ � �.v/ � 0; for all v 2 X:


A simple computation shows that, for each w 2 K, we have

hw�; �.w; u0/i C Jı. Ou0I Ow � Ou0/C �.w/� �.u0/ � ˛.w � u0/; for all w� 2 A.w/:(4.23)

Let us fix v 2 K� and define w D u0 C.v� u0/ with 2 .0; 1/. By the convexityof K� , we know that w 2 K� . Then, for each w�

2 A.w/, from (4.23), we have

˛..v � u0//

� hw�; �.w; u0/i C Jı. Ou0I Ow � Ou0/C �.w/ � �.u0/

� hw�; �.v; u0/i C .1 � /hw�

; �.u0; u0/i C Jı. Ou0I Ov � Ou0/C.1 � /Jı. Ou0I Ou0 � Ou0/C �.v/C .1 � /�.u0/ � �.u0/

D Œhw�; �.v; u0/i C Jı. Ou0I Ov � Ou0/C �.v/ � �.u0/�;

which leads to

˛..v � u0//

� hw�

; �.v; u0/i C Jı. Ou0I Ov � Ou0/C �.v/ � �.u0/: (4.24)

For each u�0 2 A.u0/, combining the fact that w ! u0 as # 0 with the fact that

A is semicontinuous, we deduce that, for each 2 .0; 1/, we can find w� 2 A.w/

such that w� * u�

0 as # 0. Taking the superior limit in (4.24) as # 0 andkeeping (H˛) in mind, we get

0 � lim sup#0

˛..v � u0//

� lim sup#0

Œhw�; �.v; u0/i C Jı. Ou0I Ov � Ou0/C �.v/ � �.u0/�

D hu�0 ; �.v; u0/i C Jı. Ou0I Ov � Ou0/C �.v/ � �.u0/ (4.25)

� hu�0 ; �.v; u0/i C �.v/ � �.u0/C

Z�

jı.x; Ou0.x/I Ov.x/ � Ou0.x//dx: (by (4.11))

Therefore, we have

8u�0 2 A.u0/ W hu�

0 ; �.v; u0/i C �.v/ � �.u0/

CZ�

jı.x; Ou0.x/I Ov.x/ � Ou0.x//dx � 0; 8v 2 K� :

If v 2 KnD.�/, then �.v/ D C1 and thus the inequality above holdsautomatically. This, together with the inequality above, shows that u0 2 K� is asolution of problem (P). Thus, the proof is complete.


Remark 4.3. Theorem 4.2 generalizes some recent results in the followingaspects:

(i) If j is a constant on � � Rk, then Theorem 4.2 reduces to the corresponding

result of Theorem 3.3 presented by Costea et al. (2012);(ii) From the relation of solutions between problems (P) and (4.2), we know that,

under the same assumptions as Theorem 4.2, problem (4.2) necessarily admitsat least one solution. In this case, if, in addition, �.u; v/ D u � v and ˛./ D 0,then this conclusion reduces to Theorem 2 of Costea and Lupu (2010).

(iii) If A is single-valued, �.u; v/ D u � v and ˛./ D 0, then Theorem 4.2 reducesto Theorem 2 of Motreanu and Radulescu (2000);

(iv) If A is single-valued, �.u; v/ D u � v, � D IK and ˛./ D 0, then Theorem 4.2reduces to Corollary 3.3 of Costea and Radulescu (2009) (or see Theorem 2 ofPanagiotopoulos et al. 1999).

Let us turn our attention to the case when K is an unbounded, closed and convexsubset of X. In order to establish the existence results of problem (P), we need tointroduce the following coercivity conditions:

(C1) There exists r0 > 0 such that, for each u 2 K�nKr0 , we can find v 2 K� withkvk � kuk such that

hu�; �.v; u/iC�.v/��.u/CJı.OuI Ov� Ou/ � 0; for all u� 2 A.u/I (4.26)

(C2) There exists r0 > 0 such that, for each u 2 K�nKr0 , we can find v 2 K� withkvk � kuk such that

hu�; �.v; u/i C �.v/ � �.u/

CZ�

jı.x; Ou.x/I Ov.x/� Ou.x//dx < 0; for all u� 2 A.u/:

Remark 4.4. (i) It is obvious that the implication (C2))(C1) holds as (4.11).(ii) The conditions (C1) and (C2) can be regarded as generalization of some

coercivity conditions proposed recently by some authors. For example,

• If j is a constant on � � Rk, then condition (C1) reduces to condition (H2)

of Theorem 3.5 presented by Costea et al. (2012);• If �.v; u/ D v�u, then conditions (C1) and (C2) reduce to conditions (B) and

(C) presented in Proposition 4.1 of Tang and Huang (2013b), respectively;if, in addition, � D IK , then they become to conditions (B) and condition (C)presented in Proposition 3.1 of Zhang and He (2011), respectively.

Theorem 4.3. Assume that all the assumptions of Theorem 4.2 hold exceptthe condition that K is bounded. If, in addition, the condition (C1) holdsfor the functional J defined as (4.10), then problem (P) admits at least onesolution.


Proof. Let us fix r > r0. Applying (4.25) of Theorem 4.2 as Kr is bounded, closedand convex, we deduce that there exists ur 2 Kr \ D.�/ such that

8u�r 2 A.ur/ W hu�

r ; �.v; ur/i C �.v/ � �.ur/C Jı. OurI Ov � Our/ � 0; 8v 2 Kr:

(4.27)

(i) If kurk D r, then kurk > r0. By condition (C1), we can find v0 2 K� withkv0k < kurk such that

hu�r ; �.v0; ur/i C �.v0/� �.ur/C Jı. OurI Ov � Our/ � 0; 8u� 2 A.u/: (4.28)

Let v 2 K� be arbitrarily fixed. Since kv0k < kurk D r, we know that thereexists t 2 .0; 1/ such that vt WD v0 C t.v � v0/ 2 Kr \ D.�/. Note that T isa linear mapping and � is convex. It follows from (4.27), item (i) of (H2

�) anditem (i) of Lemma 4.1 that

0 � hu�r ; �.vt; ur/i C �.vt/� �.ur/C Jı. OurI Ov � Our/ (by (4.27))

� tŒhu�r ; �.v; ur/i C �.v/ � �.ur/C Jı. OurI Ov � Our/�

C.1 � t/Œhu�r ; �.v0; ur/i C �.v0/ � �.ur/C Jı. OurI Ov � Our/�

� tŒhu�r ; �.v; ur/i C �.v/ � �.ur/C Jı. OurI Ov � Our/�: (by (4.28))

(4.29)

Therefore, this together with t 2 .0; 1/ implies that


r ; �.v; ur/iC�.v/��.ur/CJı. OurI Ov� Our/ � 0; 8v 2 K�:(4.30)

(ii) If kurk < r, then for each v 2 K� , there is some t 2 .0; 1/ such that vt WDur C t.v � vr/ 2 Kr \ D.�/. Note that T is a linear mapping and � is a convexfunction. It follows from (4.27) and item (i) of Lemma 4.1, we have

0 � hu�r ; �.vt; ur/i C �.vt/� �.ur/C Jı. OurI Ov � Our/ (by (4.27))

� tŒhu�r ; �.v; ur/i C �.v/ � �.ur/C Jı. OurI Ov � Our/�: (4.31)

Therefore, this, together with t 2 .0; 1/, shows that (4.30) also holds.

Since J.'/ D R� j.x; '.x//dx, by (Hj) and (4.11), we conclude that


r ; �.v; ur/i C �.v/ � �.ur/

CZ�

jı.x; Our.x/I Ov.x/ � Our.x//dx � 0; 8v 2 K� : (4.32)


When v 2 KnD.�/, we have �.v/ D C1 and thus the inequality in (4.32)holds automatically. This fact, together with (4.30), shows that ur 2 Kr \ D.�/ is asolution of problem (P). This completes the proof.

Remark 4.5. (i) If j is a constant on � � Rk, then Theorem 4.3 reduces to the

corresponding result of Theorem 3.5 due to Costea et al. (2012);(ii) Compared with Theorem 4.2 of Tang and Huang (2013b), the problem con-

sidered in the present paper is more general and the condition regarding theset-valued map A is also different.

If the constraint set K is bounded, then the solution set of problem (P) isobviously bounded. In the case when the constraint set K is unbounded, thesolution set of problem (P) may be unbounded. In the sequel, we provide asufficient condition to the boundedness of the solution set of problem (P) whenK is unbounded. The following theorem also generalizes corresponding results ofTang and Huang (2013b) and Zhang and He (2011).

Theorem 4.4. Assume that all the assumptions of Theorem 4.2 hold except thecondition that K is bounded. If, in addition, the condition (C2) holds, then thesolution set of problem (P) is nonempty and bounded.

Proof. Applying Theorem 4.3 as the implication relation (C2))(C1), we know thatthe solution set of problem (P) is nonempty. Now we prove that the solution set ofproblem (P) is bounded. Assuming that the solution set is unbounded, then for anypositive r0, there exists u0 2 K� with ku0k > r0 such that

8u�0 2 A.u0/ W hu�

0 ; �.v; u0/i C �.v/ � �.u0/

CZ�

j0.x; Ou0.x/I Ov.x/� Ou0.x//dx � 0; 8v 2 K: (4.33)

Since ku0k > r0, by the condition (C2), we know that there exists v0 2 K� withkv0k < ku0k such that

8u�0 2 A.u0/ W hu�; �.v0; u0/iC�.v0/��.u0/C

Z�

j0.x; Ou0.x/I Ov0.x/� Ou0.x//dx<0;

which contradicts with (4.33). Thus, it follows that the solution set is bounded,completing the proof.

Using a similar technique to the one used in Panagiotopoulos et al. (1999), Costea(2011), and Tang and Huang (2013b), we can provide a necessary and sufficientcondition for problem (P) and get the following result.

Theorem 4.5. Let T W X ! Lp.�IRk/ be a linear compact operator, where 1 <p < 1, k � 1 and � is a bounded open set in R

N. Let K be a nonempty, closedand convex subset of X. Assume that assumptions (H2

�), (H�) and (Hj) hold. Then a


necessary and sufficient condition for problem (P) to have a solution is that thereexists a constant r > 0 with the property that at least one solution of the problem:

(Pr) find ur 2 Kr \ D.�/ and such that


r ; �.v; ur/i C �.v/ � �.ur/

CZ�

jı.x; Our.x/I Ov.x/� Our.x//dx � 0; 8v 2 Kr; (4.34)

satisfies the inequality kurk < r.

Proof. The necessity is obvious.Now we show the sufficiency. Suppose that there exists a solution ur of problem

(Pr) with kurk < r. We shall prove that ur is a solution of problem (P). For anyfixed v 2 K, since kurk < r, we can choose " > 0 small enough such that w Dur C ".v � ur/ satisfies kwk < r. By assumption (H2

�), we have

hu�r ; �.w; ur/i � "hu�

r ; �.v; ur/i C .1 � "/hu�r ; �.ur; ur/i D "hu�

r ; �.v; ur/i: (4.35)

It follows from item (i) of Lemma 4.1 and the linearity of T that

Z�

jı.x; Our.x/I Ow.x/ � Our.x//dx

� "

Z�

jı.x; Our.x/I Ov.x/� Our.x//dx C .1 � "/

Z�

jı.x; Our.x/I Our.x/� Our.x//dx

D "

Z�

jı.x; Our.x/I Ov.x/� Our.x//dx: (4.36)

Applying (4.34) for v D w and assumption (H�) and combining (4.35) and (4.36),one has

8u�r 2 A.ur/ W "Œhu�

r ; �.v; ur/i C �.v/ � �.ur/

CZ�

jı.x; Our.x/I Ov.x/ � Our.x//dx� � 0; 8v 2 K:

Dividing by " > 0, it follows that ur is a solution of problem (P). The proof iscomplete.

Remark 4.6. For a suitable choice of maps and functionals such as A; � and j, it iseasy to see that Theorem 4.5 can be reduced to Theorem 4.4 of Tang and Huang(2013b) and Theorem 3 of Panagiotopoulos et al. (1999).

Acknowledgements This work was supported by the National Natural Science Foundation ofChina (11171237), Guangxi Natural Science Foundation (2013GXNSFBA019015), ScientificResearch Foundation of Guangxi Department of Education (ZD2014045), Outstanding Young and


Middle-aged Backbone Teachers Training Project of Guangxi Colleges and Universities (Gui-Jiao-Ren 2014-39) and Open Fund of Guangxi Key Laboratory of Hybrid Computation and IC DesignAnalysis (HCIC201308).

References

Ansari QH, Yao JC (1999) A fixed point theorem and its applications to a system of variationalinequalities. Bull Aust Math Soc 59:433–442

Ansari QH, Yao JC (2001) Iterative schemes for solving mixed variational-like inequalities. JOptim Theory Appl 108:527–541

Aubin JP, Clarke FH (1979) Shadow prices and duality for a class of optimal control problems.SIAM J Control Optim 17:567–586

Bai MR, Zhou SZ, Ni GY (2006) Variational-like inequalities with relaxed �-˛ pseudomonotonemappings in Banach spaces. Appl Math Lett 19:547–554

Carl S (2001) Existence of extremal solutions of boundary hemivariational inequalities. J DifferEqn 171:370–396

Carl S, Le VK, Motreanu D (2005) Existence and comparison principles for general quasilinearvariational-hemivariational inequalities. J Math Anal Appl 302:65–83

Carl S, Le VK, Motreanu D (2007) Nonsmooth variational problems and their inequalities,comparison principles and applications. Springer, New York

Clarke FH (1983) Optimization and nonsmooth analysis. Wiley, New YorkCostea N, Radulescu V (2009) Existence results for hemivariational inequalities involving relaxed�-˛ monotone mappings. Comm Appl Anal 13:293–304

Costea N, Radulescu V (2010) Hartman-Stampacchia results for stably pseudomonotone operatorsand nonlinear hemivariational inequalities. Appl Anal 89:175–188

Costea N, Lupu C (2010) On a class of variational-hemivariational inequalities involving set valuedmappings. Adv Pure Appl Math 1:233–246

Costea N (2011) Existence and uniqueness results for a class of quasi-hemivariational inequalities.J Math Anal Appl 373:305–315

Costea N, Ion DA, Lupu C (2012) Variational-like inequality problems involving set-valued mapsand generalized monotonicity. J Optim Theory Appl 155:79–99

Costea N, Radulescu V (2012) Inequality problems of quasi-hemivariational type involving set-valued operators and a nonlinear term. J Glob Optim 52:743–756

Fang YP, Huang NJ (2003) Variational-like inequalities with generalized monotone mappings inBanach spaces. J Optim Theory Appl 118:327–338

Goeleven D, Motreanu D, Panagiotopoulos D (1998) Eigenvalue problems for variational-hemivariational inequalities at resonance. Nonlinear Anal 33:161–180

Liu ZH (2008) Existence results for quasilinear parabolic hemivariational inequalities. J Differ Eqn244:1395–1409

Migórski S, Ochal A (2004) Boundary hemivariational inequality of parabolic type. NonlinearAnal 57:579–596

Mosco U (1976) Implicit variational problems and quasi-variational inequalities. In: Gossez JP,LamiDozo EJ, Mawhin J, Waelbroek L (eds) Nonlinear operators and the calculus of variations.Lecture notes in mathematics, vol 543. Springer, Berlin, pp 83–56

Motreanu D, Radulescu V (2000) Existence results for inequality problems with lack of convexity.Numer Funct Anal Optim 21:869–884

Motreanu D, Radulescu V (2003) Variational and non-variational methods in nonlinear analysisand boundary value problems. Kluwer Academic, Boston/Dordrecht/London

Naniewicz Z, Panagiotopoulos PD (1995) Mathematical theory of hemivariational inequalities andapplications. Marcel Dekker, New York


Panagiotopoulos PD (1983) Nonconvex energy functions, hemivariational inequalities and substa-tionarity principles. Acta Mech 42:160–183

Panagiotopoulos PD (1985) Inequality problems in mechanics and applications, convex andnonconvex energy functions. Birkhäser, Basel

Panagiotopoulos PD (1991) Coercive and semicoercive hemivariational inequalities. NonlinearAnal 16:209–231

Panagiotopoulos PD (1993) Hemivariational inequalities, applications in mechnics and engineer-ing. Springer, Berlin

Panagiotopoulos PD, Fundo M, Radulescu V (1999) Existence theorems of Hartman-Stampacchiatype for hemivariational inequalities and applications. J Glob Optim 15:41–54

Parida J, Sahoo M, Kumar A (1989) A variational-like inequality problem. Bull Aust Math Soc39:225–231

Park JY, Ha TG (2008) Existence of antiperiodic solutions for hemivariational inequalities.Nonlinear Anal 68:747–767

Park JY, Ha TG (2009) Existence of anti-periodic solutions for quasilinear parabolic hemivaria-tional inequalities. Nonlinear Anal 71:3203–3217

Tang GJ, Huang NJ (2013a) Gap functions and global error bounds for set-valued mixed variationalinequalities. Taiwan J Math 17:1267–1286

Tang GJ, Huang NJ (2013b) Existence theorems of the variational-hemivariational inequalities. JGlob Optim 56:605–622

Tang GJ, Huang NJ (2014) Strong convergence of an inexact projected subgradient method formixed variational inequalities. Optimization 63:601–615

Tang GJ, Wang X, Wang ZB (2014) Existence of variational quasi-hemivariational inequalitiesinvolving a set-valued operator and a nonlinear term. Optim Lett. doi:10.1007/s11590-014-0739-5

Xiao YB, Huang NJ (2008) Generalized quasi-variational-like hemivariational inequalities. Non-liear Anal 69:637–646

Xiao YB, Huang NJ (2009) Sub-super-solution method for a class of higher order evolutionhemivariational inequalities. Nonliear Anal 71:558–570

Xiao YB, Yang XM, Huang NJ (2014) Some equivalence results for well-posedness of hemivaria-tional inequalities. J Glob Optim. doi:10.1007/s10898-014-0198-7

Zhang YL, He YR (2011) On stably quasimonotone hemivariational inequalities. Nonlinear Anal74:3324–3332

Chapter 5An Iterative Algorithm for Split CommonFixed-Point Problem for DemicontractiveMappings

Yazheng Dang, Fanwen Meng, and Jie Sun

Abstract Inspired by the inertial proximal algorithms for finding a zero of amaximal monotone operator, we propose an inertial iteration algorithm for solvingthe split common fixed point problem for demicontractive mappings. We provethe asymptotical convergence of the algorithm under certain mild conditions.The results extend the result of Dang and Gao (Inverse Probl, 27:015007, 2011)and Moudafi (Inverse Probl 26:055007, 6pp, 2010. doi:10.1088/0266-5611/26/5/055007).

Keywords Split common fixed point problem • Inertial technique • Demicontrac-tive mapping • Asymptotical convergence

5.1 Introduction

Consider the convex feasibility problem (CFP) (Chinneck 2004), which is to find acommon point in the intersection of finitely many convex sets. CFP has extensiveapplications in many areas such as approximation theory (Deutsch 1992), imagereconstruction from projections (Censor 1998; Herman 1980), optimal control (Gao2009), and so on. A popular approach to the CFP is the so-called projection

Y. Dang (�)College of Computer Science and Technology (Software College), Henan Polytechnic University,454000, Jiaozuo, People’s Republic of China

School of Management, University of Shanghai for Science and Technology, 200093, Shanghai,People’s Republic of China.e-mail: [email protected]

F. MengNational Healthcare Group, Singapore City, Singaporee-mail: [email protected]

J. SunDepartment of Mathematics and Statistics, Curtin University, 6102, Bentley, WA, Australiae-mail: [email protected]


85

86 Y. Dang et al.

algorithm which employs orthogonal projection onto a set, see Bauschke andBorwein (1996). An important special case of CFP is the split feasibility problem(SFP), which deals with the case of finding a point in both the domain and the rangeof a given linear operator. Namely, SFP is to find a point x satisfying

x 2 C; Ax 2 Q; (5.1)

where C and Q are nonempty convex subsets in H1 and H2, respectively, andA W H1 ! H2 is a linear operator, and H1;H2 are real Banach spaces. The SFPwas originally introduced in Censor and Elfving (1994) and can be applied toimage reconstruction, signal processing, and radiation therapy, for examples. Manyprojection methods have been developed for solving the SFP, see Byrne (2004),Censor et al. (2005), Dang and Gao (2011), Qu and Xiu (2008, 2005), and Yang(2004). In Byrne (2002), Byrne introduced the so-called CQ algorithm, which takesan arbitrary initial point x0 and computes the iterative step as:

xkC1 D PCŒ.I � �AT.I � PQ/A/.xk/�; (5.2)

where PC denotes the usual orthogonal projection onto C; that is, PC.x/ Darg miny2C kx � yk, for any x 2 C; 0 < � < 2=�.ATA/, and �.ATA/ is the spectralradius of ATA.

Another algorithm, the KM algorithm, was proposed initially for solving fixedpoint problem (Crombez 2005), Byrne (2004) first applied KM iteration to theCQ algorithm for solving the SFP. Subsequently, Zhao and Yang (2005) appliedKM iteration to a perturbed CQ algorithm, Dang and Gao (2011) combined theKM iterative method with the modified CQ algorithm to construct a KM-CQ-Likealgorithm for solving the SFP. All these algorithms only use current iteration to findthe next iteration, so they tend to have slow convergence in practice.

The problem of finding a zero of a maximal monotone operator G in Euclideanspace RN is

Find x 2 RN such that 0 2 Gx:

One of the fundamental approaches to solving it is the proximal method, whichgenerates the next iteration xkC1 by solving the subproblem

0 2 kG.x/C .x � xk/; (5.3)

where xk is the current iteration and k is a regularization parameter. In 2001,Attouch and Alvarez applied the inertial technique to the above algorithm (5.3)to obtain an inertial proximal method for solving the problem of finding zero ofa maximal monotone operator. It works as follows. Given xk�1; xk 2 RN and twoparameters �k 2 Œ0; 1/; k > 0; find xkC1 2 RNsuch that

0 2 kG.xkC1/C xkC1 � xk � �k.xk � xk�1/: (5.4)

Here, the inertia is induced by the term �k.xk � xk�1/.

5 An Iterative Algorithm for Split Common Fixed-Point Problem for. . . 87

It is well known that the proximal iteration (5.3) may be interpreted as an implicitone-step discretization method for the evolution differential inclusion

0 2 dx

dt.t/C G.x.t// a:e: t � 0; (5.5)

where a.e. stands for almost everywhere. While the inspiration for (5.4) comes fromthe implicit discretization of the differential system of the second-order in time,namely

0 2 d2x

dt2.t/C �

dx

dt.t/C G.x.t// a:e: t � 0; (5.6)

where � > 0 is a damping or a friction parameter. It gives rise to various numericalmethods (for monotone inclusions and fixed problems) related to the inertialterminology (first introduced in Alvarez and Attouch 2001), all these methods,as (5.4), achieve nice convergence properties (Alvarez 2000, 2004; Alvarez andAttouch 2001; Mainge 2007, 2008) by incorporating second order information.

Inspired by the inertial proximal point algorithm for finding zeros of a maximalmonotone operator, in this paper, we apply the inertial technique to the algorithmpresented by Moudafi in 2010 to propose an inertial iterative algorithm to solvethe split common fixed-point problem for demicontractive mappings. Under somesuitable conditions, the asymptotical convergence is proved.

The paper is organized as follows. In Sect. 5.2, we recall some preliminaries.In Sect. 5.3, we present an inertial iterative algorithm and show its convergence.Section 5.4 summarizes the paper by making some concluding remarks.

5.2 Preliminaries

Throughout the rest of the paper, I denotes the identity operator, Fix.T/ denotes theset of the fixed points of an operator T i.e., Fix.T/ WD fx j x D T.x/g:

An operator T W H ! H is called demicontractive (see for example Maruster andPopirlan 2008) if there exists a constant ˇ 2 Œ0; 1/ such that

kTx � zk2 � kx � zk2 C ˇkx � Txk2;8.x; z/ 2 H � Fix.T/; (5.7)

which is equivalent to

hx � Tx; x � zi � 1 � ˇ2

kx � Txk2;8.x; z/ 2 H � Fix.T/ (5.8)

and

hx � T.x/; z � T.x/i � 1C ˇ

2kx � T.x/k2;8.x; z/ 2 H � Fix.T/: (5.9)

88 Y. Dang et al.

An operator T W H ! H is called

(i) nonexpansive if kTx � Tyk � kx � yk for all .x; y/ 2 H � HI(ii) quasi-nonexpansive if kTx � zk � kx � zk for all .x; z/ 2 H � Fix.T/I

(iii) strictly pseudocontractive if

kTx � Tyk2 � kx � yk2 C ˇkx � y � .Tx � Ty/k2 for all .x; y/

2 H � H . for some ˇ 2 Œ0; 1/ /:

Let us also recall that T is called demi-closed at the origin, if for any sequencefxkg � H and x 2 H, we have

xk ! x weakly and .I � T/.xk/ ! 0 strongly ) x 2 Fix.T/:

In the following, an operator satisfying (5.7) will be called ˇ-demicontractivemapping. Obviously, the class of demicontractive mappings contains quasi-nonexpansive mappings and strictly pseudocontractive mappings with fixed points.It is well known that the nonexpansive operators are demi-closed, which are bothquasi-nonexpansive and strictly pseodocontractive mappings.

The following Lemmas are important for the convergence analysis in the nextsection.

Lemma 5.1. Let T˛ WD .1 � ˛/I C ˛T, where ˛ 2 .0; 1�, T is a ˇ-demicontractiveself-mapping on H with Fix.T/ ¤ ;. Then,

kT˛x � zk2 � kx � zk2 � ˛.1 � ˇ � ˛/kTx � xk2: (5.10)

Proof. For any arbitrary element .x; q/ 2 H � Fix.T/, we have

kT˛x � zk2 D kx � zk2 � 2˛hx � z; x � Txi C ˛2kTx � xk2;

which, according to (5.8), yields

kT˛x � zk2 � kx � zk2 � ˛.1 � ˇ � ˛/kTx � xk2:

From Lemma 5.1, it is easy to see that T˛ is quasi-nonexpansive if ˛ 2Œ0; 1 � ˇ�I Fix.T/ D Fix.T˛/ if Fix.T/ ¤ ;. Hence, Fix.T/ is then a closedconvex subset of H.

Lemma 5.2 (Mainge 2008). Assume 'k 2 Œ0;1/ and ık 2 Œ0;1/ satisfy:

(1) 'kC1 � 'k � �k.'k � 'k�1/C ık;

(2)PC1

kD1 ık < 1;

(3) f�kg � Œ0; ��; where � 2 Œ0; 1/:Then, the sequencef'kg is convergent with

PC1kD1 Œ'kC1 � 'k�C < 1; where Œt�C WD

maxft; 0g for any t 2 R:


5.3 The Inertial Algorithm and Its Asymptotic Convergence

In what follows, we will focus our attention on the following general two-operatorsplit common fixed-point problem (SCFP):

find x� 2 C such that Ax� 2 Q: (5.11)

where A W H1 ! H2 is a bounded linear operator, U W H1 ! H1 and T W H2 ! H2

are two demicontractive operators with nonempty fixed-point sets Fix.U/ D C andFix.T/ D Q. Denote the solution set of the two-operator SCFP by

� D fy 2 C j Ay 2 Qg:

5.3.1 The Inertial Algorithm

Now, we give a description of the inertial algorithm and then present its asymptoticconvergence.

Algorithm 5.1Initialization: Let x0 2 H1 be arbitrary.Iterative step: For k 2 N, set u D I C �AT .T � I/A, and let

yk D xk C �k.xk � xk�1/

xkC1 D .1� ˛k/u.yk/C ˛kU.u.yk//;

where ˛k 2 .0; 1/ and � 2 .0;1�

/ with being the spectral radius of the operator AT A; �k 2

Œ0; 1/.

5.3.2 Asymptotic Convergence of the Inertial Algorithm

In this subsection, we establish the asymptotic convergence of Algorithm 5.3.1.

Lemma 5.3 (Opial 1967). Let H be a Hilbert space and let fxkg be a sequence inH such that there exists a nonempty set S � H satisfying:

(1) For every x�, limk kxk � x�k exists.(2) Any weak cluster point of the sequence fxkg belongs to S. Then, there exists

z 2 S such that fxkg weakly converges to z.

Theorem 5.1. Given a bounded linear operator A W H1 ! H2, let U W H1 ! H1

be ˇ-demicontractive operator with nonempty Fix.U/ D C, T W H2 ! H2 be -demicontractive operator with nonempty Fix.T/ D Q. Assume that U � I and T � I

90 Y. Dang et al.

are demiclosed at 0. If � ¤ ;, then any sequence fxkg generated by Algorithm 5.3.1weakly converges to a split common fixed point x� 2 � , provided that we chooseparameter �k satisfying �k 2 Œ0; N�k�with N�k WD minf�; 1=.kkxk�xk�1k/2g; � 2 Œ0; 1/,� 2 .0; 1�

/ and ˛k 2 .ı; 1 � ˇ � ı/ for a small enough ı > 0.

Proof. Taking z 2 � , using (5.10), we obtain

kxkC1 � zk2 D k.1 � ˛k/u.yk/C ˛kU.u.yk// � zk2

� ku.yk/� zk2 � ˛k.1 � ˇ � ˛k/kU.u.yk// � u.yk/k2: (5.12)

On the other hand, we have

ku.yk/� zk2 D kyk C �AT.T � I/.Ayk/ � zk2D kyk � zk2 C �2kAT.T � I/.Ayk/k2

C2�hyk � z;AT.T � I/.Ayk/i� kyk � zk2 C �2k.T � I/.Ayk/k2

C2�hAyk � Az; .T � I/.Ayk/i;

that is,

ku.yk/�zk2 � kyk �zk2C�2k.T �I/.Ayk/k2C2�hAyk �Az; .T �I/.Ayk/i: (5.13)

Setting � WD 2�hAyk � Az; .T � I/.Ayk/i, from (5.9), we get

� D 2�hAyk � Az; .T � I/.Ayk/iD 2�hAyk � Az C .T � I/.Ayk/� .T � I/.Ayk/; .T � I/.Ayk/iD 2�.hAyk � Az; .T � I/.Ayk/i � k.T � I/.Ayk/k2/

� 2�.1C

2k.T � I/.Ayk/k2 � k.T � I/.Ayk/k2/

D ��.1� /k.T � I/.Ayk/k2:

Combining the inequality above with (5.11), (5.13), it yields that

kxkC1 � zk2 � kyk � zk2

��.1� � �/k.T � I/.Ayk/k2 � ˛k.1 � ˇ � ˛k/kU.u.yk// � u.yk/k2: (5.14)

Define the auxiliary real sequence 'k WD 12kxk � zk2. Therefore, from (5.14), we

have


'kC1 � 1

2kyk � zk2

�12�.1�� /k.T � I/.Ayk/k2 � 1

2˛k.1�ˇ � ˛k/kU.u.yk//� u.yk/k2: (5.15)

Then we have

1

2kyk � zk2 D 1

2kxk C �k.x

k � xk�1/� zk2

D 1

2kxk � zk2 C �khxk � z; xk � xk�1i C �2k

2kxk � xk�1k2

D 'k C �khxk � z; xk � xk�1i C �2k2

kxk � xk�1k2:

It is easy to verify that 'k D 'k�1 C hxk � z; xk � xk�1i � 12kxk � xk�1k2. Hence

1

2kyk � zk2 D 'k C �k.'k � 'k�1/C �k C �2k

2kxk � xk�1k2: (5.16)

Putting (5.16) into (5.14), we get

'kC1 � 'k C �k.'k � 'k�1/C �k C �2k2

kxk � xk�1k2

�12�.1 � � �/k.T � I/.Ayk/k2 � 1

2˛k.1 � ˇ � ˛k/kU.u.yk// � u.yk/k2

By the assumption on ˛k, we have

'kC1 � 'k C �k.'k � 'k�1/C �k C �2k2

kxk � xk�1k2

�12�.1� � �/k.T � I/.Ayk/k2 � 1

2ı2kU.u.yk// � u.yk/k2: (5.17)

Since � 2 .0; 1�/ and �2k � �k, from (5.17), we derive

'kC1 � 'k C �k.'k � 'k�1/C �kkxk � xk�1k2: (5.18)

From the assumption on �k, we have

�kkxk � xk�1k2 � 1

k2;

92 Y. Dang et al.

and

C1XkD1

�kkxk � xk�1k2 < 1: (5.19)

Let ık WD �kkxk � xk�1k2 in Lemma 5.2. We deduce that the sequence fkxk � zkgis convergent (hence fxkg is bounded) with

PC1kD1 Œkxk � zk2 � kxk�1 � zk2�C < 1.

From (5.17), we have

1

2��.1� � �/k.T � I/.Ayk/k2 � 'k � 'kC1 C �k.'k � 'k�1/C �kkxk � xk�1k2;

and

1

2ı2kU.u.yk// � u.yk/k2 � 'k � 'kC1 C �k.'k � 'k�1/C �kkxk � xk�1k2:

Hence,

C1XkD1

1

2��.1� � �/k.T � I/.Ayk/k2 < 1

and

C1XkD1

ı2kU.u.yk// � u.yk/k2 < 1:

Therefore,

k.T � I/.Ayk/k2 ! 0 (5.20)

and

kU.u.yk// � u.yk/k2 ! 0: (5.21)

Suppose that x� is a weak-cluster point of fxkg, let fxk g be a subsequence of fxkg.Obviously,

w � lim

yk D w � lim

xk D x�: (5.22)

Then, from (5.20) and the demiclosedness of T � I at 0, we obtain

T.Ax�/ D Ax�; (5.23)

from which it follows that Ax� 2 Q:


Now, setting uk D yk C �AT.T � I/.Ayk/; it follows that w � lim uk D x�. Bythe demiclosedness of U � I at 0, it follows from (5.21) that

U.x�/ D x�: (5.24)

Hence x� 2 C, and therefore x� 2 �: By using Lemma 5.3 with S D � , we obtainthe weak convergence of the whole sequence fxkg.

5.4 Concluding Remarks

The paper developed an inertial algorithm and proved its weak convergence forsolving the split common fixed-point problem for demicontractive mappings inHilbert space. To some extent, the proposed algorithm and obtained results areextensions of corresponding work in Dang and Gao (2011) and Moudafi (2010).The inertial technique paves the way for investigating more effective and feasiblealgorithm for the split common fixed-point problem. The strong convergence of thealgorithm is a possible future research topic.

Acknowledgements This work was partially supported by National Science Foundation of China(under grant No.11171221), Basic and Frontier Research Program of Science and TechnologyDepartment of Henan Province (under grants No.112300410277 and No.082300440150), ChinaCoal Industry Association Scientific and Technical Guidance to Project (under grant MTKJ-2011-403), the NSTIP strategic technologies program in the Kingdom of Saudi Arabia – Award No.(11-MAT1916-02), and research grant 71901 from Faculty of Science and Engineering, CurtinUniversity.

References

Alvarez F (2000) On the minizing property of a second order dissipative dynamical system inHilbert spaces. SIAM J Control Optim 39:1102–1119

Alvarez F, Attouch H (2001) An inertial proximal method for maximal monotone operators viaDiscretization of a nonlinear oscillator with damping. Set-Valued Anal 9:3–11

Alvarez F (2004) Weak convergence of a relaxed and inertial hybrid projection-proximal pointalgorithm for maximal monotone operators in Hilbert space. SIAM J Optim 3:773–782

Bauschke HH, Borwein JM (1996) On projection algorithms for solving convex feasibilityproblems. SIAM Rev 38:367–426

Byrne C (2002) Iterative oblique projection onto convex sets and the split feasibility problem.Inverse Probl 18:441–453

Byrne C (2004) An unified treatment of some iterative algorithm algorithms in signal processingand image reconstruction. Inverse Probl 20:103–120

Chinneck JW (2004) The constraint consensus method for finding approximately feasible pointsin nonlinear programs. INFORMS J Comput 16:255–265

Censor Y (1998) Parallel application of block iterative methods in medical imaging and radiationtherapy. Math Progr 42:307–325

94 Y. Dang et al.

Censor Y, Elfving T (1994) A multiprojection algorithm using Bregman projections in a productspace. Numer Algorithms 8:221–239

Censor Y, Elfving T, Kopf N, Bortfeld T (2005) The multiple-sets solit feasibility problem and itsapplications for inverse problems. Inverse Probl 21:2071–2084

Crombez G (2005) A geometrical look at iterative methods for operators with fixed points. NumerFunct Anal Optim 26:137–175

Deutsch F (1992) The method of alternating orthogonal projections. In: Sampat Pal S (ed) Approx-imation theory, spline functions and applications. Kluwer Academic, Dordrecht, pp 105–121

Dang Y, Gao Y (2011) The strong convergence of a KM-CQ-Like algorithm for split feasibilityproblem. Inverse Problems 27:015007

Gao Y (2009) Determining the viability for a affine nonlinear control system (in Chinese). J ControlTheory Appl 29:654–656

Herman GT (1980) Image reconstruction from projections: the fundamentals of computerizedtomography. Academic, New York

Moudafi A (2010) The split common fixed-poiny problem for demicontractive mappings. InverseProbl 26:055007 (6pp). doi:10.1088/0266-5611/26/5/055007

Maruster S, Popirlan C (2008) On the Mann-type iteration and convex feasibility problem. JComput Appl Math 212:390–396

Mainge PE (2007) Inertial iterative process for fixed points of certain quasi-nonexpansivemappings, Set-valued Analysis 15:67–79

Mainge PE (2008) Convergence theorem for inertial KM-type algorithms. J Comput Appl Math219:223–236

Opial Z (1967) Weak convergence of the sequence of successive approximations for nonexpansivemappings. Bull Am Math Soc 73:591–597

Qu B, Xiu N (2008) A new halfspace-relaxation projection method for the split feasibility problem.Linear Algebra Appl 428:1218–1229

Qu B, Xiu N (2005) A note on the CQ algotithm for the split feasibility problem. Inverse Probl21:1655–1665

Yang Q (2004) The relaxed CQ algorithm solving the split feasibility problem. Inverse Probl20:1261–1266

Zhao J, Yang Q (2005) Several solution methods for the split feasibility problem. Inverse Probl21:1791–1799

Chapter 6On Constraint Qualifications for MultiobjectiveOptimization Problems with VanishingConstraints

S.K. Mishra, Vinay Singh, Vivek Laha, and R.N. Mohapatra

Abstract In this chapter, we consider a class of multiobjective optimizationproblems with inequality, equality and vanishing constraints. For the scalar case, thisclass of problems reduces to the class of mathematical programs with vanishing con-straints recently appeared in literature. We show that under fairly mild assumptionssome constraint qualifications like Cottle constraint qualification, Slater constraintqualification, Mangasarian-Fromovitz constraint qualification, linear independenceconstraint qualification, linear objective constraint qualification and linear constraintqualification do not hold at an efficient solution, whereas the standard generalizedGuignard constraint qualification is sometimes satisfied. We introduce suitablemodifications of above mentioned constraint qualifications, establish relationshipsamong them and derive the Karush-Kuhn-Tucker type necessary optimality condi-tions for efficiency.

Keywords Constraint qualifications • Multiobjective optimization problems •Vanishing constraints • Efficiency • Optimality conditions

S.K. MishraDepartment of Mathematics, Banaras Hindu University, Varanasi-221005, Indiae-mail: [email protected]

V. SinghDepartment of Mathematics, National Institute of Technology, Chaltlang,Izawal-796012, Mizoram, Indiae-mail: [email protected]

V. Laha (�)

Varanasi-221005, Indiae-mail: [email protected]

R.N. MohapatraDepartment of Mathematics, University of Central Florida, 4000 Central Florida Blvd.,Orlando, FL 32816, USAe-mail: [email protected]


95

Department of Mathematics, Faculty of Science, Banaras Hindu University,

96 S.K. Mishra et al.

6.1 Introduction

In the multiobjective optimization problems, the constraint qualifications play animportant role for the existence of Lagrange multipliers so that the Karush-Kuhn-Tucker (KKT) necessary optimality conditions hold, which in turn are importantto design various optimization algorithms. The constraint qualifications are therestrictions imposed on the constraints in order to remove the degenerate cases fromthe problem (see, e.g. Abadie 1967; Guignard 1969; Mangasarian 1969; Gould andTolle 1971; Peterson 1973). Maeda (1994) introduced generalized Guignard typeconstraint qualifications in the differentiable multiobjective optimization problemswith inequality constraints and derived the Kuhn-Tucker type necessary optimalityconditions for efficiency ensuring the existence of positive Lagrange multipliers.Later using the results of Maeda (1994) many authors have derived necessary opti-mality conditions and duality results for efficiency in multiobjective optimizationproblems both for smooth and nonsmooth cases (see, e.g., Bigi and Pappalardo1999; Preda and Chitescu 1999; Li 2000; Aghezzaf and Hachimi 2001, 2004; Lianget al. 2003; Maeda 2004; Mishra et al. 2005). We refer to Chinchuluun and Pardalos(2007) and the references therein for more details in the field of multiobjectiveoptimization problems.

Recently, Achtziger and Kanzow (2008) introduced a special class of optimiza-tion problems known as the mathematical programs with vanishing constraints(MPVC). It was described in Achtziger and Kanzow (2008) that the MPVCs areclosely related to the class of mathematical programs with equilibrium constraints(MPECs) (see, e.g. Luo et al. 1996; Outrarata et al. 1998; Facchinei and Pang 2003)and a MPVC can always be reformulated as an MPEC. But, this reformulationincreases the dimension of the problem and involves a non-uniqueness of the solu-tion. Moreover, studying MPVC as a MPEC does not take into account the specialstructure of the MPVC. Hence, it is worth studying the properties of the MPVC.We refer to Hoheisel et al. (2007, 2010), Hoheisel and Kanzow (2008, 2009), andIzmailov and Solodov (2009) for more details related to MPVC literature.

It was also described in Achtziger and Kanzow (2008) that many problemsfrom structural topology optimization can be reformulated as a MPVC and thus thecomplexities, nonlinearities, and singularities of the realistic stress constraints canbe incorporated into the mathematical problem formulations. Since, in the optimaldesign of structures, one has to consider several conflicting design objectivessimultaneously, multiobjective optimization methodology must be applied withinthe frame work of structural topology optimization. Stadler (1984) introduced thefield of multiobjective optimization problems in mechanics and later it was used asa tool to solve various engineering problems including structural design problems(see, e.g., Eschenauer et al. 1990; Koski 1993; Min et al. 2000; Lin et al. 2011).

The above mentioned works in the fields of multiobjective optimization problemsand mathematical programs with vanishing constraints are the main motivationsof this chapter. In this chapter, we study the class of multiobjective optimizationproblems with vanishing constraints (MOPVC) and provide suitable modificationsof several known constraint qualifications like Guignard constraint qualification,

6 Multiobjective Optimization Problems with Vanishing Constraints 97

Abadie constraint qualification, Cottle constraint qualification, Slater constraintqualification, linear objective constraint qualification, Mangasarian–Fromovitz con-straint qualification, linear independence constraint qualification and linear con-straint qualification for the MOPVC to establish necessary Karush-Kuhn-Tuckertype optimality conditions for efficiency.

The outline of this chapter is as follows: in Sect. 6.2, we give some knowndefinitions and results which will be used in the sequel. In Sect. 6.3, we discussthe standard GGCQ at an efficient solution of the MOPVC and derive KKT typenecessary optimality conditions. In Sect. 6.4, we observe that some constraintqualifications do not hold under fairly mild assumptions at an efficient solution ofthe MOPVC, and hence we modify them to serve as sufficient conditions for thestandard GGCQ to hold. In Sect. 6.5, we give some suitable modifications of somemore constraint qualifications, like GGCQ, GACQ, ACQ, to establish a weakerKKT type necessary optimality condition for a feasible solution to be an efficientsolution of the MOPVC, and establish relationships among them. In Sect. 6.6, weconclude the results of this chapter and discuss some future research work.

6.2 Preliminaries

Consider the following multiobjective optimization problem (MOP):

min Qf .x/ WD . Qf1.x/; : : : ; Qfm.x//s:t: Qgi.x/ � 0;8i D 1; 2; : : : ; Qp; (6.1)

Qhi.x/ D 0;8i D 1; 2; : : : ; Qq;

where all the functions ; Qfi; Qgi; Qhi W Rn ! R are continuously differentiable. Thefeasible set of the MOP (6.1) is given by

QX W fx 2 Rn W Qgi.x/ � 0.i D 1; 2; : : : ; Qp/; Qhi.x/ D 0.i D 1; 2; : : : ; Qq/g: (6.2)

Solving the MOP (6.1) is to find a local efficient solution or an efficient solutionwhich are defined as follows:

Definition 6.1. Let x� 2 QX be a feasible solution of the MOP (6.1). Then, x� is saidto be a local efficient solution of the MOP (6.1), iff there exists a number ı > 0 suchthat, there is no x 2 QXTB .x�I ı/ satisfying

Qfi.x/ � Qfi.x�/;8i D 1; : : : ; Qm;Qfi.x/ < Qfi.x�/; at least one i;

where B .x�I ı/ denotes the open ball of radius ı and centre x�:


Definition 6.2. Let x� 2 QX be a feasible solution of the MOP (6.1). Then, x� is saidto be an efficient solution of the MOP (6.1), iff there is no x 2 QX satisfying

Qfi.x/ � Qfi.x�/;8i D 1; : : : ; Qm;Qfi.x/ < Qfi.x�/; at least one i:

The following concept of tangent cones is well known in optimization (see, e.g.Rockafellar 1970; Bajara et al. 1974; Clarke 1983).

Definition 6.3. Let QQ be a nonempty subset of Rn: The tangent cone to QQ at x� 2cl QQ is the set T

QQI x�� defined by

T. QQI x�/ WD

d 2 Rnj9fxng � QQ; ftng # 0 W xn ! x�andxn � x�

tn! d

�;

where cl QQ denotes the closure of QQ:The tangent cone T. QQI x�/ is a nonempty closed cone and if QQ is convex, then thecone T. QQI x�/ is also convex. Let x� 2 QX be a feasible solution to the MOP (6.1),and suppose that IQf ; IQg and IQh are the set of indices given by

IQf WD f1; 2; : : : ; Qmg ;IQg WD fi 2 f1; 2; : : : ; Qpg j Qgi.x

�/ D 0g ; (6.3)

IQh WD f1; 2; : : : ; Qqg :

For each k D 1; : : : ; Qm; the nonempty sets QQk and QQ are given as follows

QQk WD fx 2 Rnj Qgi.x/ � 0;8i D 1; 2; : : : ; Qp;Qhi.x/ D 0;8i D 1; 2; : : : ; Qq; (6.4)

Qfi.x/ � Qfi.x�/;8i D 1; 2; : : : ; Qm; and i ¤ kg;and

QQ WD fx 2 Rnj Qgi.x/ � 0;8i D 1; 2; : : : ; Qp;Qhi.x/ D 0;8i D 1; 2; : : : ; Qq; (6.5)

Qfi.x/ � Qfi.x�/;8i D 1; 2; : : : ; Qmg:

For scalar objective optimization problems, QQ1 D QX:The following concept of an approximating cone to the set QQ was introduced in

Maeda (1994) for a multiobjective optimization problem with inequality constraints,and is of significant importance for the subsequent analysis.


Definition 6.4. The linearizing cone to QQ at x� 2 QQ is the set L QQI x�� given by

L QQI x�� WD fd 2 Rnj r Qgi.x

�/Td � 0;8i 2 IQg;

r Qhi.x�/Td D 0;8i 2 IQh;

rQfi.x�/Td � 0;8i 2 IQf g:

The following constraint qualification was considered in Maeda (1994) for amultiobjective optimization problem with inequality constraints as a generalizationof the Guignard constraint qualification appeared in Guignard (1969).

Definition 6.5. Let x� 2 QX be an efficient solution of the MOP (6.1). Then, thegeneralized Guignard constraint qualification (GGCQ) holds at x� iff

L QQI x��

Qm\kD1

clcoT QQkI x�� ;

where clcoT QQkI x�� denotes the closure of the convex hull of T

QQkI x�� :The following constraint qualifications are sufficient conditions for the GGCQ to

hold at an efficient solution of the MOP (6.1).

Definition 6.6. Let x� 2 QX be an efficient solution of the MOP (6.1). Then,

(a) The Abadie constraint qualification (ACQ) holds at x� iff

L QQI x�� T

QQI x�� I

(b) The generalized Abadie constraint qualification (GACQ) holds at x� iff

L QQI x��

Qm\kD1

T QQkI x�� I

(c) The Cottle constraint qualification (CCQ) holds at x� iff for each k D1; 2; : : : ; Qm; the system

rQfi.x�/Td < 0;8i 2 IQf ; i ¤ k;

r Qgi.x�/Td < 0;8i 2 IQg;


has a solution d 2 RnI


(d) The Slater constraint qualification (SCQ) holds at x�; iff the objective functionsand the inequality constraints

Qfi .i D 1; 2; : : : ; Qm/ ;Qgi .i D 1; 2; : : : ; Qp/

are all convex on Rn, the equality constraints

Qhi .i D 1; 2; : : : ; Qq/

are all affine on Rn; and for each k D 1; 2; : : : ; Qm; the system

Qfi.x/ < Qfi.x�/;8i D 1; 2; : : : ; Qm; and i ¤ k;

Qgi.x/ < 0;8i D 1; 2; : : : ; Qp;Qhi.x/ D 0;8i D 1; 2; : : : ; Qq

has a solution x 2 Rn:

(e) The linear constraint qualification (LCQ) holds at x� iff the objective functionsQfi.i D 1; 2; : : : ; Qm/; the inequality constraints Qgi.i D 1; 2; : : : ; Qp/; and theequality constraints Qhi.i D 1; 2; : : : ; Qq/; are all affine.

(f) The linear objective constraint qualification (LOCQ) holds at x�; iff theobjective functions Qfi.i D 1; 2; : : : ; Qm/; are all affine, and the system

rQfi.x�/Td � 0;8i 2 IQf ;



has a solution d 2 RnI(g) The Mangasarian-Fromovitz constraint qualification (MFCQ) holds at x�; iff

the gradients

rQfi.x�/�

i 2 IQf�

r Qhi.x�/i 2 IQh

�

are linearly independent, and the system

rQfi.x�/Td D 0;8i 2 IQf ;




has a solution d 2 Rn:

(h) The linear independence constraint qualification (LICQ) holds at x�; iff thegradients

rQfi.x�/�

i 2 IQf�;

r Qgi.x�/i 2 IQg

�;

r Qhi.x�/i 2 IQh

�

are linearly independent.

The relationships among above mentioned constraint qualifications are given inthe following Fig. 6.1.

It is clear that GGCQ is the weakest among all the constraint qualifications, andwhen it is satisfied the KKT type necessary optimality conditions for efficiency wasgiven in Maeda (1994, Theorem 3.2) as follows:

Theorem 6.1. Let x� 2 QX be an efficient solution of the MOP (6.1), suchthat the GGCQ holds at x�. Then, there exist Lagrange multipliers Qi 2R .i D 1; 2; : : : ; Qm/ ; Qi 2 R.i D 1; 2; : : : ; Qp/; Q�i 2 R.i D 1; 2; : : : ; Qq/; suchthat the following first order optimality conditions hold

QmXiD1

Qir Qfi.x�/CQpX

iD1Qir Qgi.x

�/CQqX

iD1Q�ir Qhi.x

�/ D 0; (6.6)

and

Qi > 0;8i D 1; 2; : : : ; Qm; Qi � 0; Qi Qgi.x�/ D 0;8i D 1; 2; : : : :; Qp: (6.7)

Fig. 6.1 Relationshipsamong constraintqualifications of MOP


6.3 Constraint Qualifications in Multiobjective OptimizationProblems with Vanishing Constraints

We consider a constrained multiobjective optimization problem as follows:

min f .x/ WD .f1.x/; : : : ; fm.x//

s:t: gi.x/ � 0;8i D 1; 2; : : : ; p;

hi.x/ D 0;8i D 1; 2; : : : ; q; (6.8)

Hi.x/ � 0;8i D 1; 2; : : : ; r;

Gi.x/Hi.x/ � 0;8i D 1; 2; : : : ; r;

where all the functions fi; gi; hi;Hi;Gi W Rn ! R are assumed to be continuouslydifferentiable. The problem (6.8) is called as a multiobjective optimization problemwith vanishing constraints (MOPVC). For the scalar case the MOPVC (6.8)reduces to a special class of optimization problems known as the mathematicalprograms with vanishing constraints (MPVC), which was introduced in Achtzigerand Kanzow (2008), and further studied in Hoheisel et al. (2007, 2010), Hoheiseland Kanzow (2008, 2009), and Izmailov and Solodov (2009).

The class of MOPVCs can be interrelated with the class of multiobjectiveoptimization problems with equilibrium constraints (MOPEC), see, Mordukhovich(2004, 2006, 2009), Bao and Mordukhovich (2007), and Bao et al. (2007, 2008)and the references therein for more details. By introducing slack variables si; i D1; 2; : : : ; r; the MOPVC (6.8) is equivalent to the following MOPEC in the variablesz WD .x; s/ W

minx;s

f .x/ WD .f1.x/; : : : ; fm.x//

s:t: gi.x/ � 0;8i D 1; 2; : : : ; p;

hi.x/ D 0;8i D 1; 2; : : : ; q;

Gi.x/ � si � 0;8i D 1; 2; : : : ; r;

Hi.x/ � 0;8i D 1; 2; : : : ; r;

si � 0;8i D 1; 2; : : : ; r;

Hi.x/si D 0;8i D 1; 2; : : : ; r:

The above reformulation of the MOPVC (6.8) as a MOPEC is always possible,but it increases the dimension of the problem and involves a non-uniqueness of thesolution. Moreover, studying MOPVC as a MOPEC does not take into account thespecial structure of the MOPVC. Hence, it is worth studying the properties of theMOPVC directly.


In this section, we discuss the GGCQ for the MOPVC (6.8) under which theKarush-Kuhn-Tucker (KKT) type necessary optimality conditions for a feasiblesolution to be an efficient solution will be given. Suppose that the set X definedby

X WD fx 2 Rn W gi.x/ � 0;8i D 1; 2; : : : ; p;

hi.x/ D 0;8i D 1; 2; : : : ; q;

Hi.x/ � 0;8i D 1; 2; : : : ; r;

Gi.x/Hi.x/ � 0;8i D 1; 2; : : : ; rg

is the feasible set of the MOPVC (6.8), and x� 2 X is an efficient solution. Theindex sets x� are defined as follows

If WD f1; 2; : : : ;mg;Ig WD fi 2 f1; 2; : : : ; pgjgi.x

�/ D 0g;Ih WD f1; 2; : : : ; qg; (6.9)

IC WD fi 2 f1; 2; : : : ; rgjHi.x�/ > 0g;

I0 WD fi 2 f1; 2; : : : ; rgjHi.x�/ D 0g:

The index set IC.x�/ can be further divided into the following subsets

IC0 WD fi 2 f1; 2; : : : ; rgjHi.x�/ > 0;Gi.x

�/ D 0g;IC� WD fi 2 f1; 2; : : : ; rgjHi.x

�/ > 0;Gi.x�/ < 0g: (6.10)

Similarly, partitioning the index set I0.x/ can be done as follows

I0C WD fi 2 f1; 2; : : : ; rgjHi.x�/ D 0;Gi.x

�/ > 0g;I00 WD fi 2 f1; 2; : : : ; rgjHi.x

�/ D 0;Gi.x�/ D 0g; (6.11)

I0� WD fi 2 f1; 2; : : : ; rgjHi.x�/ D 0;Gi.x

�/ < 0g:

Also, consider the following function

�i.x/ WD Gi.x/Hi.x/;8i D 1; 2; : : : ; r (6.12)

and the gradient is given by

r�i.x/ D Gi.x/rHi.x/C Hi.x/rGi.x/;8i D 1; 2; : : : ; r: (6.13)

The definition of index sets (6.9)–(6.11) provides the following


r�i.x�/ D

8<ˆ:0; if i 2 I00;

Gi.x�/rHi.x�/; if i 2 I0C [ I0�;Hi.x�/rGi.x�/; if i 2 IC0:

(6.14)

For each k D 1; 2; : : : ;m; the nonempty sets Qk and Q are defined as follows

Qk WD fx 2 Rnj gi.x/ � 0;8i D 1; 2; : : : ; p;

hi.x/ D 0;8i D 1; 2; : : : ; q;

Hi.x/ � 0;8i D 1; 2; : : : ; r; (6.15)

Gi.x/Hi.x/ � 0;8i D 1; 2; : : : ; r

fi.x/ � fi.x�/;8i D 1; 2; : : : ;m; i ¤ kg;

and

Q WD fx 2 Rnj gi.x/ � 0;8i D 1; 2; : : : ; p;

hi.x/ D 0;8i D 1; 2; : : : ; q;

Hi.x/ � 0;8i D 1; 2; : : : ; r; (6.16)

Gi.x/Hi.x/ � 0;8i D 1; 2; : : : ; r;

fi.x/ � fi.x�/;8i D 1; 2; : : : ;mg:

The following result gives the standard linearizing cone to Qk; k D 1; 2; : : : ;m;at an efficient solution x� 2 X of the MOPVC (6.8).

Lemma 6.1. Let x� 2 X be an efficient solution of the MOPVC (6.8). Then, thelinearizing cone to Qk; k D 1; 2; : : : ;m; at x� is given by

L.QkI x�/ D fd 2 Rnj rfi.x�/Td � 0;8i 2 If ; i ¤ k;

rgi.x�/Td � 0;8i 2 Ig;

rhi.x�/Td D 0;8i 2 Ih; (6.17)

rHi.x�/Td D 0;8i 2 I0C;

rHi.x�/Td � 0;8i 2 I0� [ I00;

rGi.x�/Td � 0;8i 2 IC0g:

Proof. Suppose that �i; i D 1; 2; : : : ; r; is the function from (6.12). Then, using thedefinitions of the index sets from (6.9)–(6.11), and in view of Definition 6.4, thelinearizing cone to Qk; k D 1; : : : ;m at x� 2 Qk is given by


L.QkI x�/ D fd 2 Rnj rfi.x�/Td � 0;8i 2 If ; i ¤ k;


rhi.x�/Td D 0;8i 2 Ih;

rHi.x�/Td � 0;8i 2 I0;

r�i.x�/Td � 0;8i 2 I0 [ IC0

�:

Now, using the expression of the r�i.x�/ for i 2 I0 [ IC0 from (6.14), andrearranging the terms involved, we get the required representation (6.17) for thelinearizing cone to Qk; k D 1; : : : ;m at x� 2 Qk: utRemark 6.1. The linearizing cone to Qk; k D 1; : : : ;m at x� 2 Qk is a nonempty,closed and convex cone in Rn:

Moreover, for the scalar case it reduces to the linearized cone of the MPVCgiven in Achtziger and Kanzow (2008, Lemma 4). Also, it is clear by the expres-sions (6.15)–(6.17), that L .QI x�/ D \m

kD1LQkI x�� : Alternatively, the expression

for L .QI x�/ follows immediately from Achtziger and Kanzow (2008, Lemma 4),by viewing it as a linearized cone to an MPVC with the constraints

gi.x/ � 0;8i D 1; 2; : : : ; p;

fi.x/ � fi.x�/ � 0;8i D 1; 2; : : : ;m;

hi.x/ D 0;8i D 1; 2; : : : ; q;

Hi.x/ � 0;8i D 1; 2; : : : ; r;

Gi.x/Hi.x/ � 0;8i D 1; 2; : : : ; r;

The following result gives the KKT type necessary optimality conditions for effi-ciency, when the standard GGCQ holds at an efficient solution of the MOPVC (6.8).

Theorem 6.2. Let x� 2 X be an efficient solution of the MOPVC (6.8). Ifthe standard GGCQ holds at x�; then there exist Lagrange multipliers i 2Ri 2 If

�; i 2 R .i D 1; : : : ; p/ ; �i 2 R .i 2 Ih/ ; �

Hi ; �

Gi 2 R .i D 1; : : : ; r/ ; such

that

mXiD1

irfi.x�/C

pXiD1

irgi.x�/ C

qXiD1

�irhi.x�/

�rX

iD1�H

i rHi.x�/C

rXiD1

�Gi rGi.x

�/ D 0; (6.18)


and

i > 0;8i D 1; 2; : : : ;m;

gi.x�/ � 0; i � 0; igi.x

�/ D 0;8i D 1; 2; : : : ; p;

hi.x�/ D 0;8i D 1; 2; : : : ; q; (6.19)

�Hi D 0 .i 2 IC/ ; �H

i � 0 .i 2 I00 [ I0�/ ;

�Hi free .i 2 I0C/ ; �H

i Hi.x�/ D 0;8i D 1; 2; : : : ; r;

�Gi D 0 .i 2 I0 [ IC�/ ; �G

i � 0; .i 2 IC0/ ; �Gi Gi.x

�/ D 0;8i D 1; 2; : : : ; r:

Proof. Suppose that x� 2 X is an efficient solution of the MOPVC (6.8) suchthat GGCQ holds at x�: Then, by Theorem 6.1, there exists Lagrange multipliersi 2 R .i D 1; : : : ;m/ ; i 2 R .i D 1; : : : ; p/ ; �C

i ; ��i 2 R .i D 1; : : : ; q/ ; ˛i 2

R .i D 1; : : : ; r/ ; ˇi 2 R .i D 1; : : : ; r/ ;such that the following conditions hold:

mXiD1

irfi.x�/C

pXiD1

irgi.x�/C

qXiD1

�Ci rhi.x

�/�qX

iD1��

i rhi.x�/

�rX

iD1˛irHi.x

�/CrX

iD1ˇir�i.x

�/ D 0; (6.20)

and

i > 0;8i D 1; 2; : : : ;m;

gi.x�/ � 0; i � 0; igi.x

�/ D 0;8i D 1; 2; : : : ; p;

hi.x�/ � 0; �C

i � 0; �Ci hi.x

�/ D 0;8i D 1; 2; : : : ; q;

�hi.x�/ � 0; ��

i � 0; ��i

�hi.x�/� D 0;8i D 1; 2; : : : ; q; (6.21)

�H.x�/ � 0; ˛i � 0; ˛i�Hi.x

�/� D 0;8i D 1; 2; : : : ; r;

�i.x�/ � 0; ˇi � 0; ˇi�i.x

�/ D 0;8i D 1; 2; : : : ; r;

where �i; i D 1; 2; : : : ; r denotes the function from (6.12). Now, using therepresentation (6.13) of the gradient of �i; and setting

�Ci � ��

i WD �i;8i D 1; 2; : : : ; q;

˛i � ˇiG.x�/ WD �H

i ;8i D 1; 2; : : : ; r; (6.22)

ˇiHi.x�/ WD �G

i ;8i D 1; 2; : : : ; r;

we get the required KKT type necessary optimality conditions (6.18) and (6.19).ut


Fig. 6.2 The feasible regionof Example 6.1

For the scalar case, the above KKT type necessary optimality conditions for theMOPVC (6.8) under GGCQ reduces to the KKT conditions for the MPVC underthe standard Abadie constraint qualification given in Achtziger and Kanzow (2008,Theorem 1). The next corollary is a direct consequence of the fact that the tangentcones T

QkI x�� ; k D 1; 2; : : : ;m; contain the origin 0 2 Rn:

Corollary 6.1. Let x� 2 X be an efficient solution of the MOPVC (6.8) suchthat L .QI x�/ D f0g: Then, there exists Lagrange multipliers satisfying (6.18)and (6.19).

Now, we give an example which verifies Corollary 6.1 with I00 ¤ �:

Example 6.1. Consider the following MOPVC given by

min f .x1; x2/ WD x1 C x2;�x1 C x22

�;

s:t: H1.x/ WD x31 C x2 � 0;

G1.x/H1.x/ WD x2x31 C x2

� � 0;

which is a MOPVC of the form (6.8) with n D 2;m D 2; p D q D 0 and r D 1: Itis easy to see that the origin x� WD .0; 0/ 2 R2 is a feasible solution of the MOPVCand I00 D f1g: Also, x� WD .0; 0/ 2 R2 is an efficient solution of the MOPVC overthe feasible region given by Fig. 6.2. Using Lemma 6.1, one has

LQI x�� D ˚

.0; 0/ 2 R2�:

Now, for any Lagrange multipliers 1 � 0; 2 � 0 (not all zero) �H1 � 0 and �G

1 ;

one has

.0; 0/ D 1rf1.x�/C 2rf2.x

�/� �H1 rH1.x

�/C �G1 rG1.x

�/

D 1 � 2; 1 � �H

1 C �G1

�;

and

�H1 � 0; �H

1 H1.x�/ D 0:


Fig. 6.3 The objectivefunctions of Example 6.1


Thus 1 D 0 implies 2 D 0; and vice versa. Hence, we have 1 > 0 and 2 > 0;

and Corollary 6.1 is satisfied (Fig. 6.3).

Now, we give an example in which GGCQ does not hold for the MOPVC (6.8) withI00 ¤ �


min f .x1; x2/ WD x1;�x1 C x22

�;

s:t: H1.x/ WD 1 � x21 � x22 � 0;

G1.x/H1.x/ WD �x21 � x21 � x22

� � 0;

which is MOPVC (6.8) with n D 2; m D 2; p D q D 0; and r D 1: It is easy tosee that the point x� WD .�1; 0/ is a feasible solution of the MOPVC and I00 D f1g:Also, x� is an efficient solution of the MOPVC over the feasible region given byFig. 6.4. Using Lemma 6.1 and the definitions of tangent cones T

QkI x�� ; k D 1; 2;

one has



LQI x�� D ˚

d 2 R2jd1 D 0�;

TQ1I x�� D ˚

d 2 R2j2d1 � d21 � d22 � 0;�d22d1 � d21 � d22

� � 0;�d1Cd22 � 0�;

TQ2I x�� D ˚

d 2 R2j2d1 � d21 � d22 � 0;�d22d1 � d21 � d22

� � 0; d1 � 0�:

Observe that

LQI x�� 6�

2\kD1

clcoTQkI x�� ;

and hence the standard GGCQ does not hold at x�: Now, for any Lagrangemultipliers 1 � 0; 2 � 0; not both zero, and �H

1 � 0; �G1 � 0; one has

.0; 0/ D 1rf1.x�/C 2rf2.x

�/� �H1 rH1.x

�/C �G1 rG1.x

�/

D 1 � 2 � 2�H

1 ;��G1

�;

which does not satisfy the necessary optimality conditions (6.18) and (6.19).

Examples 6.1 and 6.2 show that GGCQ is not always violated when I00 ¤ �; butit may not hold sometimes when I00 ¤ �: Following example shows that GGCQ isnot a sufficient condition for the existence of positive Lagrange multipliers for theMOPVC (6.8) (Fig. 6.5).


min f .x1; x2/ WD x1; x

21 C x2

�;

s:t: H1.x/ WD x1 C x2 � 0;

G1.x/H1.x/ WD x2 .x1 C x2/ � 0;


Fig. 6.6 Feasible region ofthe Example 6.3

Fig. 6.7 Objective functionsof Example 6.3

which is MOPVC (6.8) with n D 2;m D 2; p D q D 0 and r D 1: It is easy tosee that origin x� WD .0; 0/ is feasible solution of the MOPVC and I00 D f1g : Also,x� is an efficient solution of the MOPVC over the feasible region given by Fig. 6.6.Using Lemma 6.1, one has L .QI x�/ D f0g:

Now, for any Lagrange multipliers 1 � 0; 2 � 0; not both zero, and �H1 �

0; �G1 � 0; one has

.0; 0/ D 1rf1.x�/C 2rf2.x

�/� �H1 rH1.x

�/C �G1 rG1.x

�/

D 1 � �H

1 ;��H1 C �G

1

�:

Observe that for �H1 D �G

1 � 0; 1 D 0 implies 2 � 0; which violets the existenceof positive Lagrange multipliers (Fig. 6.7).


6.4 Sufficient Conditions for the Generalized GuignardConstraint Qualification

It was shown in Achtziger and Kanzow (2008) that some constraint qualificationslike LICQ and MFCQ do not hold under fairly mild assumptions, whereas someconstraint qualifications like ACQ may not hold sometimes at a local minimumof the MPVC. In this section, we investigate some more constraint qualificationslike CCQ, SCQ, LOCQ and LCQ for the MOPVC (6.8) and modify them wherenecessary to use them as sufficient conditions for the GGCQ to hold at an efficientsolution of the MOPVC (6.8). The next result shows that under fairly reasonableassumptions CCQ is not satisfied at an efficient solution of the MOPVC (6.8).

Lemma 6.2. Let x� 2 X be an efficient solution of the MOPVC (6.8) with I00 [I0C ¤ �: Then, the standard CCQ is not satisfied at x�:

Proof. Suppose that CCQ is satisfied at x�: Then, for each k D 1; : : : ;m; the system

rfi.x�/Td < 0;8i 2 If ; i ¤ k;

rgi.x�/Td < 0;8i 2 Ig;

rhi.x�/Td D 0;8i 2 Ih; (6.23)

rHi.x�/Td > 0; i 2 I0C [ I00 [ I0�;

�i.x�/Td < 0; i 2 I0C [ I00 [ I0� [ IC0;

has a solution d 2 Rn: Using the gradient of �i from (6.14) in (6.23), one has

0 D r�i.x�/ < 0;8i 2 I00;

and

rHi.x�/Td D 1

Gi.x�/r�i.x

�/d < 0;8i 2 I0C;

a contradiction, and hence CCQ is not satisfied at x�: utAs a direct consequence of Lemma 6.2, and in view of Maeda (1994, Lemmas 4.3),we obtain the following result, which is multiobjective analog of Achtziger andKanzow (2008, Lemma 3).

Corollary 6.2. Let x� be an efficient solution of the MOPVC (6.8) with I00 [ I0C ¤�: Then, the standard MFCQ is not satisfied at x�:

We also obtain the following corollary as a direct consequence of Lemma 6.2 inview of Maeda (1994, Lemmas 4.4).


Corollary 6.3. Let x� be an efficient solution of the MOPVC (6.8) with I00 [ I0C ¤�: Then, the standard SCQ is not satisfied at x�:

Since LICQ implies MFCQ, the following result is a direct consequence ofCorollary 6.2, and is a multiobjective analog of Achtziger and Kanzow (2008,Lemma 2).

Corollary 6.4. Let x� be an efficient solution of the MOPVC (6.8) with I00 [ I0C ¤�: Then, the standard LICQ is not satisfied at x�: Moreover, if I0 ¤ �; then also thestandard LICQ is not satisfied at x�:

The proof of the following result is similar to the proof of Lemma 6.2, and itshows that LOCQ is also not satisfied at an efficient solution of the MOPVC (6.8)when I00 [ I0C ¤ �:

Lemma 6.3. Let x� be an efficient solution of the MOPVC (6.8) with I00[I0C ¤ �:

Then, the standard LOCQ is not satisfied at x�:

The above results show that under fairly mild assumptions most of the constraintqualifications are violated at an efficient solution of the MOPVC (6.8), and hencewe introduce some constraint qualifications as modifications of the standard CCQ,MFCQ, SCQ, LICQ and LOCQ for the MOPVC (6.8).

Definition 6.7. Let x� be an efficient solution of the MOPVC (6.8). Then theCottle-Type constraint qualification for the MOPVC (6.8), denoted by CCQ-MOPVC, holds at x� iff for each k D 1; : : : ;m; the system

rfi.x�/Td < 0;8i 2 If ; i ¤ k;

rgi.x�/Td < 0;8i 2 Ig;

rhi.x�/Td D 0;8i 2 Ih; (6.24)

rHi.x�/Td D 0;8i 2 I0C [ I00;

rHi.x�/Td > 0;8i 2 I0�;

rGi.x�/Td < 0;8i 2 IC0;


It is clear that CCQ-MOPVC is different from the standard CCQ for theMOPVC (6.8), and is a fair assumption. We now show that CCQ-MOPVC is asufficient condition for the GGCQ provided that the critical index set I00 D �:

Lemma 6.4. Let x� be an efficient solution of the MOPVC (6.8) with I00 D �: IfCCQ-MOPVC is satisfied at x� then the standard GGCQ is also satisfied at x�:

Proof. Suppose that x� is an efficient solution of the MOPVC (6.8) with I00 D �:

Then, the MOPVC (6.8) is locally equivalent to the following MOP:


min f .x/ WD .f1.x/; : : : ; fm.x//

s:t: gi.x/ � 0;8i 2 Ig;

hi.x/ D 0;8i 2 Ih;

Hi.x/ D 0;8i 2 I0C; (6.25)

Hi.x/ � 0;8i 2 I0�;

Gi.x/ � 0;8i 2 IC0:

Now, when CCQ-MOPVC is satisfied at x� with I00 D �; then the standard CCQfor the MOP (6.25) will also hold at x�; and hence the standard GGCQ for theMOP (6.25) will also be satisfied at x�; that is,

L� OQI x�

��

m\kD1

clcoT� OQkI x�

�;

where OQk; k D 1; : : : ;m; and OQ are defined as

OQk WD fx 2 Rnj fi.x/ � fi.x�/;8i 2 If ; i ¤ k;

gi.x/ � 0;8i 2 Ig;

hi.x/ D 0;8i 2 Ih; (6.26)

Hi.x/ D 0;8i 2 I0C;

Hi.x/ � 0;8i 2 I0�;

Gi.x/ � 0;8i 2 IC0g;

and

OQ WD fx 2 Rnj fi.x/ � fi.x�/;8i 2 If ;

gi.x/ � 0;8i 2 Ig;

hi.x/ D 0;8i 2 Ih; (6.27)

Hi.x/ D 0;8i 2 I0C;

Hi.x/ � 0;8i 2 I0�;

Gi.x/ � 0;8i 2 IC0g:

Also, the linearizing cone L� OQI x�

�is given by


L� OQI x�� D fd 2 Rn W rfi.x

�/Td � 0;8i 2 If ;


rhi.x�/Td D 0;8i 2 Ih; (6.28)


rHi.x�/T � 0;8i 2 I0�;

rGi.x�/T � 0;8i 2 IC0g:

which in view of Lemma 6.1 is nothing but the linearizing cone L .QI x�/ with

I00 D �: Now, since OQ � Qk, k D 1; 2; : : : ;m; it follows that T� OQkI x�

��

TQkI x�� ; k D 1; 2; : : : ;m; which implies that

m\kD1

clcoT� OQkI x�

��

m\kD1

clcoTQkI x�� ;

and hence the standard GGCQ holds at x�: utNow, we give an example which verifies that CCQ-MOPVC may not imply GGCQif I00 ¤ �:


min f .x1; x2/ WD �x1 C x22; x21 C x2

�;

s:t: H1.x/ WD x1 C x2 � 0;

G1.x/H1.x/ WD x1 .x1 C x2/ � 0;

which is a MOPVC (6.8) with n D 2;m D 2; p D q D 0 and r D 1: It is easy to seethat the origin x� WD .0; 0/ is an efficient solution of the MOPVC over the feasibleregion given by Fig. 6.8 and I00 D f1g: Using Lemma 6.1, and the definition of thetangent cones T

QkI x�� ; k D 1; 2; one has

LQI x�� D ˚

d 2 R2jd1 � 0; d2 � 0; d1 C d2 � 0� I

TQ1I x�� D ˚

d 2 R2jd1 C d2 � 0; d1 .d1 C d2/ � 0; d21 C d2 � 0� I

TQ2I x�� D ˚

d 2 R2jd1 C d2 � 0; d1 .d1 C d2/ � 0;�d1 C d22 � 0�:

Observe that

LQI x�� 6�

2\kD1

clcoTQkI x�� ;




and hence the GGCQ-MOPVC is not satisfied at x� for the given MOPVC. But, thesystem given by (6.24) is solvable for x�; and hence CCQ-MOPVC holds at x� forthe given MOPVC (Fig. 6.9).

Now, we give a constraint qualification for the MOPVC (6.8), which is a modifiedversion of the standard MFCQ, and is a multiobjective analog of VC-MFCQintroduced in Achtziger and Kanzow (2008).

Definition 6.8. Let x� 2 X be an efficient solution of the MOPVC (6.8). Then,the Mangasarian-Fromovitz constraint qualification (MFCQ) for the MOPVC (6.8),denoted by MFCQ-MOPVC, holds at x� iff the gradients


rfix�� i 2 If

�;

rhix�� .i 2 Ih/ ;

rHix�� .i 2 I00 [ I0C/ ;

are linearly independent, and the system

rfix��T

d D 0;8i 2 If ;

rgix��T

d < 0;8i 2 Ig;

rhix��T

d D 0;8i 2 Ih; (6.29)

rHix��T

d D 0;8i 2 I0C [ I00;

rHix��T

d > 0;8i 2 I0�;

rGix��T

d < 0;8i 2 IC0;


The following result gives the relationship between the CCQ-MOPVC and theMFCQ-MOPVC.

Lemma 6.5. Let x� 2 X be an efficient solution of the MOPVC (6.8). If MFCQ-MOPVC holds at x� 2 X; then CCQ-MOPVC also holds at x�:

Proof. Suppose that the MFCQ-MOPVC holds at x�; but the CCQ-MOPVC doesnot hold at x�: Then, there exists k 2 f1; : : : ;mg such that the system (6.24) has nosolution d 2 Rn: By Motzkin’s theorem of the alternative Mangasarian (1969), thereexist real numbers i � 0

i 2 If ; i ¤ k

�; i � 0

i 2 Ig

�; �H

i � 0 .i 2 I0�/ ; �Gi �

0 .i 2 IC0/ W not all zero, and �i 2 R .i 2 Ih/ ; e�iH 2 R .i 2 I0C

SI00/ ; such that

mXiD1i¤k

irfi.x�/C

Xi2Ig

irgi.x�/C

Xi2Ih

�irhi.x�/

�X

i2I00S

I0C

e�iHrHi.x

�/ �Xi2I0�

�Hi rHi.x

�/CX

i2IC0

�Gi rGi.x

�/ D 0: (6.30)

Suppose that d 2 Rn solves the systems (6.29), then from (6.30), one has

Xi2Ig

irgi.x�/Td �

Xi2I0�

�Hi rHi.x

�/Td CX

i2IC0

�Gi rGi.x

�/Td D 0:


Using (6.29) the above equation implies that

i D 0;8i 2 Ig;

�Hi D 0;8i 2 I0�;

�Gi D 0;8i 2 IC0:

Substituting the values in (6.30), one has

mXiD1i¤k

irfi.x�/C

Xi2Ih

�irhi.x�/�

Xi2I00

SI0C

e�iHrHi.x

�/ D 0:

Since

rfix�� i 2 If

�;


rHix�� .i 2 I00 [ I0C/

are all linearly independent, one has

i D 0;8i 2 If ; i ¤ k;

�i D 0;8i 2 Ih;

e�iH D 0;8i 2 I00 [ I0C;

a contradiction to the existence of not all zero Lagrange multipliers, and hence theresult. ut

The following constraint qualification is a modification of the standard SCQ forthe MOPVC (6.8) and serves as a sufficient condition for the CCQ-MOPVC to hold.

Definition 6.9. Let x� 2 X be an efficient solution of the MOPVC (6.8). Then,the Slater-type Constraint Qualification for the MOPVC (6.8), denoted by SCQ-MOPVC, holds at x� 2 X; iff the functions

fii 2 If

�;

gii 2 Ig

�;

Gi .i 2 IC0/ ;

are all convex on Rn;

Hi .i 2 I0�/


are all concave on Rn; and

Hi .i 2 I00 [ I0C/ ;

hi .i 2 Ih/

are all affine on Rn; and for each k D 1; 2; : : : ;m the system

fi .x/ < fix��8i 2 If ; i ¤ k;

gi .x/ < 0;8i 2 Ig;

hi .x/ D 0;8i 2 Ih; (6.31)

Hi .x/ D 0;8i 2 I0C [ I00;

Hi .x/ > 0;8i 2 I0�;

Gi .x/ < 0;8i 2 IC0;

has a solution x 2 Rn:

The following result gives the relationship between the CCQ-MOPVC and theSCQ-MOPVC.

Lemma 6.6. Let x� 2 X be an efficient solution of the MOPVC (6.8). If SCQ-MOPVC holds at x�; then CCQ-MOPVC also holds at x�:

Proof. Suppose that SCQ-MOPVC holds at x�: Then, for each k D 1; : : : ;m; thereexists an xk 2 Rn such that

fixk�< fi

x�� ;8i 2 If ; i ¤ k;

gixk�< 0;8i 2 Ig;

hixk� D 0;8i 2 Ih; (6.32)

Hixk� D 0;8i 2 I0C [ I00;

Hixk�> 0;8i 2 I0�;

Gixk�< 0;8i 2 IC0:

Since, the function fii 2 If

�; gi

i 2 Ig

�; Gi .i 2 IC0/ are all convex on

Rn; Hi .i 2 I0�/ are all concave on Rn; and Hi .i 2 I00S

I0C/ ; hi .i 2 Ih/ are allaffine on Rn; one has

rfix��T

xk � x�� fixk� � fi

x�� < 0;8i 2 If ; i ¤ k;

rgix��T

xk � x�� gixk� � gi

x�� < 0;8i 2 Ig;

rhix��T

xk � x�� D hixk� � hi

x�� D 0;8i 2 Ih; (6.33)


rHix��T

xk � x�� D Hixk� � Hi

x�� D 0;8i 2 I0C [ I00;

rHix��T

xk � x�� Hixk� � Hi

x�� > 0;8i 2 I0�;

rGix��T

xk � x�� Gixk� � Gi

x�� < 0;8i 2 IC0;

Setting xk � x� WD dk; (6.33) implies that the CCQ-MOPVC holds at x�: utThe following constraint qualifications are modifications of LCQ and LOCQ for

the MOPVC (6.8).

Definition 6.10. Let x� 2 X be an efficient solution of the MOPVC (6.8). Then, theLinear Constraint Qualification for the MOPVC (6.8), denoted by LCQ-MOPVC,holds at x�; iff the functions

fii 2 If

�gii 2 Ig

�;

hi .i 2 Ih/ ;

Hi .i 2 I0/ ;

Hi .i 2 IC0/ ;

are all affine.

Definition 6.11. Let x� 2 X be an efficient solution of the MOPVC (6.8). Then, theLinear Objective Constraint Qualification for the MOPVC (6.8), denoted by LOCQ-MOPVC, holds at x�; iff the functions

fii 2 If

�

are all affine, and the system

rfix��T

d � 0;8i 2 If ;

rgix��T

d < 0;8i 2 Ig;

rhix��T

d D 0;8i 2 Ih; (6.34)

rHix��T

d D 0;8i 2 I0C [ I00;

rHix��T

d > 0;8i 2 I0�;

rGix��T

d < 0;8i 2 IC0;



The following result gives the relationship between the LCQ-MOPVC and thestandard GGCQ.

Lemma 6.7. Let x� 2 X be an efficient of the MOPVC (6.8) such that I00 D �: ifLCQ-MOPVC holds at x�; then the standard GGCQ also holds at x�:

Proof. Suppose that x� 2 X is an efficient solution of the MOPVC (6.8) with I00 D�: Then, the MOPVC (6.8) is locally equivalent to the MOP (6.25). Hence, forI00 D �; LCQ-MOPVC is identical to the standard LCQ of the MOP (6.25.) Since,LCQ of the MOP (6.25) holds at x�; it follows that GGCQ of the MOP (6.25) alsoholds at x�; and proceeding as in Lemma 6.4, we get the required result. ut

The proof of the following result is similar to the proof of Lemma 6.7.

Lemma 6.8. Let x� 2 X be an efficient solution of the MOPVC (6.8) such thatI00 D �. If LOCQ-MOPVC holds at x�; then GGCQ also holds at x�:

Now, we give a constraint qualification of the MOPVC (6.8), which serves as asufficient condition for the MFCQ-MOPVC to hold, and is a multiobjective analogof VC-LICQ introduced in Achtziger and Kanzow (2008).

Definition 6.12. Let x� 2 X be an efficient solution of the MOPVC (6.8). Then,the linear independence constraint qualification of the MOPVC (6.8), denoted byLICQ-MOPVC, holds at x�; iff for each k D 1; 2; : : : ;m; the gradients

rfix�� i 2 If ; i ¤ k

�;

rgix�� i 2 Ig

�;


rHix�� .i 2 I0/ ;

rGix�� .i 2 IC0/ ;

are linearly independent.

The next result is a direct consequence of Definitions 6.8 and 6.12.

Lemma 6.9. Let x� 2 X be an efficient solution of the MOPVC (6.8). If LICQ-MOPVC holds x�; then MFCQ-MOPVC also holds at x�:

The summary of the above results are given in Fig. 6.10, and we have thefollowing theorem.

Theorem 6.3. Let x� 2 X be an efficient solution of the MOPVC (6.8) withI00 D �: If any of the constraint qualifications given by Definitions 6.7–6.12 holdsat x� then the standard GGCQ holds at x� and there exist Lagrange multiplierssatisfying (6.18) and (6.19).


LOCQ-MOPVC

MFCQ-MOPVC CCQ-MOPVC

SCQ-MOPVC

LCQ-MOPVC

LICQ-MOPVC

ACQ GACQ

GGCQ

Fig. 6.10 Relationships among modified constraint qualifications

6.5 A Modified Generalized Guignard ConstraintQualification

It was observed in Sect. 6.3 that the standard GGCQ may or may not hold at anefficient solution of the MOPVC (6.8) when I00 ¤ �: In this section, we introducea suitable modification of the GGCQ of the MOPVC (6.8), and use it to provenecessary optimality conditions for efficiency in MOPVC (6.8), that are differentfrom the standard KKT conditions given by Theorem 6.2. We also provide varioussufficient conditions for the modified GGCQ to hold.

In order to define a modified Guignard constraint qualification, we intro-duce a nonlinear multiobjective optimization problem (NLMOP) derived from theMOPVC (6.8) depending on an efficient solution x� 2 X as follows

min f .x/ WD .f1.x/; : : : ; fm.x//

s:t: gi.x/ � 0;8i D 1; 2; : : : ; p;

hi.x/ D 0;8i D 1; 2; : : : ; q; (6.35)

Hi.x/ D 0;Gi.x/ � 0;8i 2 I0C;

Hi.x/ � 0;Gi.x/ � 0;8i 2 I0� [ I00 [ IC0 [ IC�:


Also, define the sets Qk

and Q as follows

Qk WD fx 2 Rnj fi.x/ � fi

x�� ;8i D 1; 2; : : : ;m; i ¤ k;

gi.x/ � 0;8i D 1; 2; : : : ; p;

hi.x/ D 0;8i D 1; 2; : : : ; q; (6.36)

Hi.x/ D 0;Gi.x/ � 0;8i 2 I0C;

Hi.x/ � 0;Gi.x/ � 0;8i 2 I0� [ I00 [ IC0 [ IC�g

and

Q WD fx 2 Rnj fi.x/ � fix�� ;8i D 1; 2; : : : ;m;

gi.x/ � 0;8i D 1; 2; : : : ; p;

hi.x/ D 0;8i D 1; 2; : : : ; q; (6.37)

Hi.x/ D 0;Gi.x/ � 0;8i 2 I0C;

Hi.x/ � 0;Gi.x/ � 0;8i 2 I0� [ I00 [ IC0 [ IC�g:

The linearizing cone Qk

at x� 2 X is given by

L.QkI x�/ D fd 2 Rnj rfi.x

�/Td � 0;8i D 1; : : : ;m; i ¤ k;


rhi.x�/Td D 0;8i 2 Ih; (6.38)


rHi.x�/Td � 0;8i 2 I00 [ I0�;

rGi.x�/Td � 0;8i 2 I00 [ IC0g:

The linearizing cone to Q at x� 2 Q given by

LQI x�� D

m\kD1

L�

QkI x�� : (6.39)

The following lemma gives the relationship between the tangent cones

T�

QkI x�

�; k D 1; 2; : : : ;m; and the linearizing cone L.QI x�/:

Lemma 6.10. Let x� 2 be an efficient solution of the MOPVC (6.8). Then, we have

m\kD1

clcoT�

QkI x�

�� L

QI x�� :


Proof. Suppose that x� 2 X is an efficient solution of the MOPVC (6.8). By Maeda(1994, Lemma 3.1), we always have

m\kD1

clcoTQkI x�� L

QI x�� ; (6.40)

and

m\kD1

clcoT�

QkI x��

m\kD1

L�

QkI x�� D L

QI x�� : (6.41)

Also, since Qk � Qk, 8k D 1; : : : ;m; one has

T�

QkI x�

�� T

QkI x�� ;8i D 1; : : : ;m; (6.42)

and

L�

QkI x�

�� L

QkI x�� ;8i D 1; : : : ;m: (6.43)

Combining (6.40)–(6.43), we have

m\kD1

clcoT�

QkI x�

�� L

QI x�� ; (6.44)

and hence the result. utIn view of Lemma 6.10, we are now in a position to define some modified

constraint qualifications of the MOPVC (6.8).

Definition 6.13. Let x� 2 X be an efficient solution of the MOPVC (6.8). Then, amodified GGCQ of the NLMOP (6.35), denoted by GGCQ-NLMOP, is said to holdat x�; iff

LQI x��

m\kD1

clcoT�

QkI x�� :

Definition 6.14. Let x� 2 X be an efficient solution of the MOPVC (6.8). Then, amodified GGCQ of the MOPVC (6.8), denoted by GGCQ-MOPVC, is said to holdat x�; iff

LQI x��

m\kD1

clcoTQkI x�� :


The following result gives the relationship between the GGCQ-NLMOP and theGGCQ-MOPVC.

Lemma 6.11. Let x� 2 X be an efficient solution of the MOPVC (6.8). If theGGCQ-NLMOP holds at x�; then the GGCQ-MOPVC also holds at x� 2 X:

Proof. Suppose that x� 2 X is an efficient solution of the MOPVC (6.8) such thatGGCQ-NLMOP holds at x�; then one has

LQI x��

m\kD1

clcoT�

QkI x�

�: (6.45)

Since, Qk � Qk; 8k D 1; 2; : : : ;m; it follows that T

�Q

kI x��

�TQkI x�� ; 8k D 1; 2; : : : ;m; and hence

m\kD1

clcoT�

QkI x�

��

m\kD1

clcoTQkI x�� : (6.46)

Also, we always have L�

QkI x�

�� L

QkI x�� ; 8k D 1; : : : ;m; which follows

that

m\kD1

L�

QkI x�

��

m\kD1

LQkI x�� : (6.47)

Combining (6.45)–(6.47), one has

LQI x��

m\kD1

clcoTQkI x�� ;

which implies that the GGCQ-MOPVC holds at x�: utRemark 6.2. Let x� 2 X be an efficient solution of the MOPVC (6.8) such that thestandard GGCQ holds at x�: Then, GGCQ-MOPVC is also satisfied at x�; since wealways have

LQI x�� L

QI x��

m\kD1

clcoTQkI x�� :

The following example says that GGCQ-MOPVC is strictly weaker than thestandard GGCQ.



min f .x1; x2/ WD .x1 C x2; x2 � x1/ ;

s:t: H1.x/ WD x1 C x2 � 0;

G1.x/H1.x/ WD x1 .x1 C x2/ � 0;

which is a MOPVC (6.8) with n D 2; m D 2; p D q D 0 and r D 1: It is easyto see that the origin x� WD .0; 0/ is an efficient solution of the MOPVC over thefeasible region X given by

X˚x 2 R2 W x1 C x2 � 0; x1 .x1 C x2/ � 0

�;

and I00 D f1g: Also, the sets Q1; Q2; Q and Q are given by

Q1 D ˚x 2 R2 W x1 C x2 � 0; x1 .x1 C x2/ � 0; x2 � x1 � 0

�;

Q2 D ˚x 2 R2 W x1 C x2 � 0; x1 .x1 C x2/ � 0; x1 C x2 � 0

�;

Q D ˚x 2 R2 W x1 C x2 � 0; x1 .x1 C x2/ � 0; x1 C x2 � 0; x2 � x1 � 0

�;

Q D ˚x 2 R2 W x1 C x2 � 0; x1 � 0; x1 C x2 � 0; x2 � x1 � 0

�:

It is clear that

TQ1; x�� D ˚

d 2 R2 W d1 C d2 � 0; d1 .d1 C d2/ � 0; d2 � d1 � 0�;

TQ2; x�� D ˚

d 2 R2 W d1 C d2 � 0; d1 .d1 C d2/ � 0; d1 C d2 � 0�;

LQ; x�� D ˚

d 2 R2 W d1 C d2 � 0; d2 � d1 � 0; d1 C d2 � 0�;

LQ; x�� D ˚

d 2 R2 W d1 C d2 � 0; d2 � d1 � 0; d1 C d2 � 0; d1 � 0�:

which implies that

LQ; x�� 6�

2\kD1

clcoTQk; x�� ;

whereas

LQ; x��

2\kD1

clcoTQk; x�� ;

hence the GGCQ-MOPVC holds whereas the standard GGCQ is not satisfied at x�:

When GGCQ-MOPVC holds at an efficient solution x� 2 X of the MOPVC (6.8),the KKT conditions of Theorem 6.2 may not hold, since GGCQ-MOPVC is weakerthan the standard GGCQ. Hence, in the following result, we derive KKT typenecessary optimality conditions for efficiency under GGCQ-MOPVC.


Theorem 6.4. Let x� 2 X be an efficient solution of the MOPVC (6.8) such thatthe GGCQ-MOPVC holds at x�. Then, there exist Lagrange multipliers i 2 R.i D1; : : : ;m/; i 2 R.i D 1; : : : ; p/; �i 2 R.i D 1; : : : ; q/; �H

i ; �Gi 2 R.i D 1; : : : ; r/;

such that

mXiD1

irfi.x�/C

pXiD1

irgi.x�/C

qXiD1

�irhi.x�/

�rX

iD1�H

i rHi.x�/C

rXiD1

�Gi rGi.x

�/ D 0; (6.48)

and

i > 0;8i D 1; 2; : : : ;m;

gi.x�/ � 0; i � 0; igi.x

�/ D 0;8i D 1; 2; : : : ; p;

hi.x�/ D 0;8i D 1; 2; : : : ; q; (6.49)

�Hi D 0 .i 2 IC/ ; �H

i � 0 .i 2 I00 [ I0�/ ;

�Hi free .i 2 I0C/ ; �H

i Hi.x�/ D 0;8i D 1; 2; : : : ; r;

�Gi D 0 .i 2 I0C [ I0� [ IC�/ ; �G

i � 0; .i 2 I00 [ IC0/ ;

�Gi Gi.x

�/ D 0;8i D 1; 2; : : : ; r:

Proof. Suppose that x� is an efficient solution of the MOPVC (6.8). We will firstshow that the system

rfix��T

d � 0;8i D 1; 2; : : : ;m;

rfix��T

d < 0; at least one i;

rgix��T

d � 0;8i 2 Ig;

rhix��T

d D 0;8i 2 Ih; (6.50)

rHix��T

d D 0;8i 2 I0C;

rHix��T

d � 0;8i 2 I00 [ I0�;

rGix��T

d � 0;8i 2 I00 [ IC0;

has no solution d 2 Rn: Suppose to the contrary that there exists d 2 Rn such that thesystem (6.50) is solvable, then d 2 L

QI x�� : Without loss of generality, we may

assume that


rfix��T

d < 0; for some k; 1 � k � m;

rfix��T

d � 0;8i D 1; : : : ;m; i ¤ k:

Since, the GGCQ-MOPVC holds at x�; d 2 clcoTQiI x�� ; 8i D 1; : : : ;m: In

particular, we have

d 2 clcoTQkI x�� ; 1 � k � m:

Hence, there exists a sequence fdlg � coTQkI x�� ; 1 � k � m; such that

dl ! d: Now, for each dl; l D 1; 2; : : : ; there exist number �l; lj � 0; anddlj 2 T

QkI x�� ; 1 � k � m; j D 1; 2; : : : ; �l; such that

�lXjD1

lj D 1;

�lXjD1

ljdlj D dl:

Since, dlj 2 TQkI x�� ; 1 � k � m; for each l D 1; 2; : : : ; and j D 1; 2; : : : ; �l;

there exist sequencesnxs

lj

o� Qk; 1 � k � m; and

ntslj

o# 0 such that xs

lj ! x� andxs

lj�x�

xslj

! dlj: Setting dslj WD xs

lj�x�

xslj; for all s D 1; 2; : : : ; we have

fixs

lj

� D fix� C ts

ljdslj

� � fix�� ;8i D 1; : : : ;m; i ¤ k;

gixs

lj

� D gix� C ts

ljdslj

� � 0 D gix�� ;8i 2 Ig;

hixs

lj

� D hix� C ts

ljdslj

� D 0 D hix�� ;8i 2 Ih; (6.51)

Hixs

lj

� D Hix� C ts

ljdslj

� � 0 D Hix�� ;8i 2 I0� [ I00 [ I0C;

�ixs

lj

� D �ix� C ts

ljdslj

� � 0 D �ix�� ;8i 2 I0� [ I00 [ I0C [ IC0:

Also, since x� is an efficient solution of the MOPVC (6.8), for all s D 1; 2; : : : ; wehave

fkxs

lj

� D fkx� C ts

ljdslj

� � fkx�� ; 1 � k � m: (6.52)

From (6.51) and (6.52), we have

rfkx��T

dlj � 0; 1 � k � m;

rfix��T

dlj � 0;8i D 1; : : : ;m; i ¤ k;

rgix��T

dlj � 0;8i 2 Ig;

rhix��T

dlj D 0;8i 2 Ih;

rHix��T

dlj � 0;8i 2 I0C [ I00 [ I0�;

r�ix��T

dlj � 0;8i 2 I0C [ I00 [ I0� [ IC0:


By linearity and continuity of the inner product and using the expression of ther�i .x�/ from (6.14), it follows that

rfkx��T

d � 0; 1 � k � m;

rfix��T

d � 0;8i D 1; : : : ;m; i ¤ k;

rgix��T

d � 0;8i 2 Ig;

rhix��T

d D 0;8i 2 Ih;

rHix��T

d D 0;8i 2 I0C;

rHix��T

d � 0;8i 2 I00 [ I0�;

rGix��T

d � 0;8i 2 IC0;

which in view of Lemma 6.1 implies that rfk .x�/T d D 0; 1 � k � m; sinced 2 L

QI x�� L .QI x�/ ; a contradiction to our assumption, and hence the

system (6.50) has no solution d 2 Rn: Now, by the Tucker’s theorem of thealternative (Mangasarian 1969), there exist Lagrange multipliers

i 2 R .1; : : : ;m/ ; i 2 Ri 2 Ig

�; �i 2 R .1; : : : ; q/ ;

�Hi 2 R .i 2 I0C [ I00 [ I0�/ ; �G

i 2 R .i 2 I00 [ IC0/ ;

such that

mXiD1

irfi.x�/ C

Xi2Ig

irgi.x�/C

qXiD1

�irhi.x�/

�Xi2I0

�Hi rHi.x

�/CX

i2I00[IC0

�Gi rGi.x

�/ D 0;

and

i > 0;8i D 1; 2; : : : ;m;

� 0;8i 2 Ig;

�i 2 R;8i D 1; 2; : : : ; q;

�Hi 2 R;8i 2 I0C; �H

i � 0;8i 2 I00 [ I0�;

�Gi � 0;8i 2 I00 [ IC0:


Setting

�Hi D 0;8i 2 IC;

�Gi D 0;8i 2 I0C [ I0� [ IC�;

the required necessary optimality conditions (6.48) and (6.49) follow. utFor m D 1; the above necessary optimality conditions will reduce into the VC-

KKT conditions of the MPVC (Achtziger and Kanzow 2008, Theorem 4) and arecalled as the KKT-MOPVC conditions.

We now give some constraint qualifications which assure that the GGCQ-MOPVC holds.

Definition 6.15. Let x� 2 X be an efficient solution of the MOPVC (6.8). Then, amodified GACQ of the MOPVC (6.8), denoted by GACQ-MOPVC, holds at x�; iff

LQI x��

m\kD1

TQkI x�� :

Remark 6.3. As we always have LQI x�� L .QI x�/ ; the GACQ-MOPVC holds,

whenever the standard GACQ is satisfied, but the converse is not true in general.Hence, the standard GACQ serves as a sufficient condition for the GACQ-MOPVCto hold.

The following constraint qualification assures that the GACQ-MOPVC holds.

Definition 6.16. Let x� 2 X be an efficient solution of the MOPVC (6.8). Then, amodified GACQ of the NLMOP (6.35), denoted by GACQ-NLMOP, is said to holdat x�; iff

LQI x��

m\kD1

T�

QkI x�� :

The following result is a direct consequence of Definitions 6.14–6.16.

Lemma 6.12. Let x� 2 X be an efficient solution of the MOPVC (6.8). If the GACQ-MOPVC holds at x�, then the GGCQ-MOPVC is satisfied. Moreover, if the GACQ-NLMOP holds at x�, then the GACQ-MOPVC and the standard GACQ both aresatisfied at x�:

Now, we give some modifications of the standard ACQ which assure that theGACQ-MOPVC holds. The following constraint qualification can be treated asa multiobjective analog of the VC-Abadie constraint qualification introduced inAchtziger and Kanzow (2008).


Definition 6.17. Let x� 2 X be an efficient solution of the MOPVC (6.8). Then, amodified ACQ of the MOPVC (6.8), denoted by ACQ-MOPVC, is said to hold atx�; iff

LQI x�� T

QI x�� :

The ACQ-MOPVC is weaker than the standard ACQ as we always haveLQI x�� L .QI x�/ :

Definition 6.18. Let x� 2 X be an efficient solution of the MOPVC (6.8). Then, amodified ACQ of the MOPVC (6.8), denoted by ACQ-NLMOP, holds at x�; iff

LQI x�� T

QI x�� :

The following result is a direct consequence of Definitions 6.15, 6.17 and 6.18.

Lemma 6.13. Let x� 2 X be an efficient solution of the MOPVC (6.8). If the ACQ-MOPVC holds at x�; then the GACQ-MOPVC is also satisfied at x�: Moreover, ifACQ-NLMOP holds at x� then the ACQ-MOPVC and the standard ACQ both aresatisfied at x�:

We now give some more sufficient conditions which assure that the GGCQ-MOPVC holds at an efficient solution of the MOPVC (6.8).

Theorem 6.5. Let x� be an efficient solution of the MOPVC (6.8), and consider thefollowing conditions:

1. The standard GGCQ holds for the NLMOP (6.35) at x�I2. For each k D 1; : : : ;m; there exists a vector Od 2 Rn satisfying

rfix��T Od < 0;8i D 1; 2; : : : ;m; i ¤ k;

rgix��T Od < 0;8i 2 Ig;

rhix��T Od D 0;8i 2 Ih; (6.53)

rHix��T Od D 0;8i 2 I0C;

rHix��T Od > 0;8i 2 I00 [ I0�;

rGix��T Od < 0;8i 2 I00 [ IC0I

3. The functions fi .i D 1; : : : ;m/, gi .i D 1; : : : ; p/ ; Gi .i 2 I0� [ I00 [ IC0 [ IC�/are all convex, Gi .i 2 I0C/ ; Hi .i 2 I0� [ I00 [ IC0 [ IC�/ are all concave andhi .i D 1; : : : ; q/ ; Hi .i 2 I0C/ are all affine, and for each k D 1; : : : ;m; thereexists a vector Od 2 Rn satisfying


fi .x/ < fix�� ;8i D 1; 2; : : : ;m; i ¤ k;

gi .x/ < 0;8i D 1; 2; : : : ; p;

hi .x/ D 0;8i D 1; 2; : : : ; q; (6.54)

Hi .x/ D 0;Gi .x/ > 0;8i 2 I0C;

Hi .x/ > 0;Gi .x/ < 0;8i 2 I0� [ I00 [ IC0 [ IC�I

4. The functions fi; gi; hi;Gi;Hi are all affine;5. The functions fi .i D 1; : : : ;m/ are all affine, and there exists a vector Od 2 Rn

satisfying

rfix��T Od � 0;8i D 1; 2; : : : ;m;


rhix��T Od D 0;8i 2 Ih; (6.55)

rHix��T Od D 0;8i 2 I0C;

rHix��T Od > 0;8i 2 I00 [ I0�;


6. The gradients

rfix�� .i D 1; : : : ;m/ ;

rhix�� .i D 1; : : : ; q/ ;

rHix�� .i 2 I0C/ ;

are linearly independent, and that there exists a vector Od 2 Rn satisfying

rfix��T Od D 0;8i D 1; 2; : : : ;m;

rhix��T Od D 0;8i D 1; 2; : : : ; q;

rHix��T Od D 0;8i 2 I0C; (6.56)


rHix��T Od > 0;8i 2 I00 [ I0�;



7. The gradients

rfix�� .i D 1; 2; : : :m/ ;

rgix�� i 2 Ig

�;

rhix�� .i D 1; : : : q/ ;

rHix�� .i 2 I0/ ;

rGix�� .i 2 I00 [ IC0/ ;

are all linearly independent.

Then, GGCQ-MOPVC holds at x� and there exist Lagrange multipliers satisfy-ing (6.48) and (6.49).

Proof. (1) Since the standard GGCQ holds for the NLMOP (6.35), we have

LQI x��

m\kD1

clcoT�

QkI x�� :

Also, Qk � Qk; k D 1; 2; : : : ;m; which implies that T

�Q

kI x��

� TQkI x�� ; k D

1; 2; : : : ;m; and hence

m\kD1

clcoT�

QkI x��

m\kD1

clcoTQkI x�� :

Combining above inclusions, we have

LQI x��

m\kD1

clcoTQkI x�� ;

that is, the GGCQ-MOPVC is satisfied at x�I(2)–(7) the assumptions are the standard CCQ, SCQ, LCQ, LOCQ, MFCQ andLICQ of the NLMOP (6.35), respectively, and hence the standard GGCQ holds forthe NLMOP (6.35) at x�; which implies by (1) above that the GGCQ-MOPVC issatisfied at x�:

Hence, by Theorem 6.4, there exist Lagrange multipliers satisfying (6.48)and (6.49). utThe results of this section can be summarized in Fig. 6.11, and we have the followingtheorem.

Theorem 6.6. Let x� 2 X be an efficient solution of the MOPVC (6.8). If,any of the constraint qualifications given by Definitions 6.13–6.18 holds at x�;then the GGCQ-MOPVC also holds at x�; and there exist Lagrange multiplierssatisfying (6.48) and (6.49).


ACQACQ-NLMOP

GACQ-NLMOP

GGCQ-NLMOP

ACQ-MOPVC GACQ-MOPVC

GGCQ-MOPVC

GGCQ

GACQ

Fig. 6.11 Sufficient conditions for the GGGCQ-MOPVC

6.6 Conclusions

In this chapter, we introduced a new class of multiobjective optimization problemscalled the multiobjective optimization problems with vanishing constraints as anextension of the mathematical programs with vanishing constraints from the scalarcase. We showed that under fairly mild assumptions some constraint qualificationslike Cottle constraint qualification, Slater constraint qualification, Mangasarian-Fromovitz constraint qualification, linear independence constraint qualification,linear objective constraint qualification and linear constraint qualification do nothold at an efficient solution, whereas the standard generalized Guignard constraintqualification is sometimes satisfied. We gave various constraint qualifications,as modifications of the standard constraint qualifications, which assure that thegeneralized Guignard constraint qualification holds at an efficient solution. Wealso introduced a suitable modification of the generalized Guignard constraintqualification, gave sufficient conditions which assure that it holds and deriveKarush-Kuhn-Tucker type necessary optimality conditions for efficiency.

Acknowledgements The authors are thankful to the anonymous referees for their valuablecomments and suggestions which helped to improve this chapter in its present form. This workwas done when Vinay Singh was a Post Doctoral Fellow of National Board of Higher Mathematics(NBHM), Department of Atomic Energy (DAE), Government of India and Vivek Laha was aSenior Research Fellow of the Council of Scientific and Industrial Research (CSIR), New Delhi,Ministry of Human Resources Development, Government of India at Department of Mathematics,Banaras Hindu University.

Currently, Vivek Laha is supported by the Postdoctoral Fellowship of National Board of HigherMathematics, Department of Atomic Energy, Government of India (Ref. No. 2/40(47)/2014/R &D-II/1170).


References

Abadie JM (1967) On the Kuhn-Tucker theorem. In: Abadie JM (eds) Nonlinear programming.Wiley, New York, pp 21–36

Achtziger W, Kanzow C (2008) Mathematical programs with vanishing constraints: optimalityconditions and constraint qualifications. Math Progr 114:69–99

Aghezzaf B, Hachimi M (2004) Second order duality in multiobjective programming involvinggeneralized type I functions. Numer Funct Anal Optim 25(7–8):725–736

Aghezzaf B, Hachimi M (2001) Sufficiency and duality in multiobjective programming involvinggeneralized .F; ��/convexity. J Math Anal Appl 258:617–628

Bajara BS, Goode JJ, Nashed MZ (1974) On the cones of tangents with applications tomathematical programming. J Optim Theory Appl 13:389–426

Bigi G, Pappalardo M (1999) Regularity conditions in vector optimization. J Optim Theory Appl102(1):83–96

Bao TQ, Mordukhovich BS (2007) Existence of minimizers and necessary conditions in set-valuedoptimization with equilibrium constraint. Appl Math 52:453–472

Bao TQ, Gupta P, Mordukhovich BS (2007) Necessary conditions in multiobjective optimizationwith equilibrium constraints. J Optim Theory Appl 135:179–203

Bao TQ, Gupta P, Mordukhovich BS (2008) Suboptimality conditions for mathematical programswith equilibrium constraints. Taiwan. J Math 12(9):2569–2592

Chinchuluun A, Pardalos PM (2007) A survey of recent developments in multiobjective optimiza-tion. Ann Oper Res 154:29–50

Clarke FH (1983) Optimization and nonsmooth analysis. Wiley-Interscience, New YorkEschenauer H, Koski J, Osyczka A (eds) (1990) Multicriteria design optimization: procedures and

applications. Springer, BerlinFacchinei F, Pang J-S (2003) Finite-dimensional variational inequalities and complementarity

problems. Springer, New YorkGould FJ, Tolle JW (1971) A necessary and sufficient qualification for constrained optimization.

SIAM J Appl Math 20:164–172Guignard M (1969) Generalized Kuhn-Tucker conditions for mathematical programming problems

in a Banach space. SIAM J Contr 7:232–241Hoheisel T, Kanzow C (2007) Würzburg: first and second order optimality conditions for

mathematical programs with vanishing constraints. Appl Math 52(6):495–514Hoheisel T, Kanzow C (2008) Stationary conditions for mathematical programs with vanishing

constraints using weak constraint qualifications. J Math Anal Appl 337:292–310Hoheisel T, Kanzow C (2009) On the Abadie and Guignard constraint qualifications for mathemat-

ical programmes with vanishing constraints. Optimization 58(4):431–448Hoheisel T, Kanzow C, Outrata JV (2010) Exact penalty results for mathematical programs with

vanishing constraints. Nonlinear Anal 72:2514–2526Izmailov AF, Solodov MV (2009) Mathematical programs with vanishing constraints: optimality

conditions, sensitivity, and a relaxation method. J Optim Theory Appl 142:501–532Koski J (1993) Multicriteria optimization in structural design: state of the art. In: Proceedings of

the 19th design automation conferences, Albuquerque. ASME, pp 621–629Li XF (2000) Constraint qualifications in nonsmooth multiobjective optimization. J Optim Theory

Appl 106(2):373–398Liang Z-A, Huang H-X, Pardalos PM (2003) Efficiency conditions and duality for a class of

multiobjective fractional programming problems. J Global Optim 27:447–471Lin P, Zhou P, Wu CW (2011) Multiobjective topology optimization of end plates of proton

exchange membrane fuel cell stacks. J Power Sources 196(3):1222–1228Luo Z-Q, Pang J-S, Ralph D (1996) Mathematical programs with equilibrium constraints.

Cambridge University Press, CambridgeMaeda T (1994) Constraint qualifications in multiobjective optimization problems: differentiable

case. J Optim Theory Appl 80(3):483–500


Maeda T (2004) Second order conditions for efficiency in nonsmooth multiobjective optimizationproblems. J Optim Theory Appl 122(3):521–538

Mangasarian OL (1969) Nonlinear programming. McGrawHill, New YorkMin S, Nishiwaki S, Kikuchi N (2000) Unified topology design of static and vibrating structures

using multiobjective optimization. Comput Struct 75:93–116Mishra SK, Wang, S-Y, Lai KK (2005) Optimality and duality for multiple-objective optimization

under generalized type I univexity. J Math Anal Appl 303:315–326Mordukhovich BS (2004) Equilibrium problems with equilibrium constraints via multiobjective

optimization. Optim Methods Softw 19:479–492Mordukhovich BS (2006) Variational analysis and generalized differentiation, II: applications.

Grundlehren series (fundamental principles of mathematical sciences), vol 331. Springer,Berlin

Mordukhovich BS (2009) Multiobjective optimization problems with equilibrium constraints.Math Progr Ser B 117:331–354

Outrarata JV, Kocvara M, Zowe J (1998) Nonsmooth approach to optimization problems withequilibrium constraints. Kluwer Academic, Dordrecht

Peterson DW (1973) A review of constraint qualifications in finite-dimensional spaces. SIAM Rev15:639–654

Preda V, Chitescu I (1999) On constraint qualifications in multiobjective optimization problems:semidifferentiable case. J Optim Theory Appl 100(2):417–433

Rockafellar RT (1970) Convex analysis. Princeton University Press, PrincetonStadler W (1984) Multicriteria optimization in mechanics. Appl Mech Rev 37:277–286

Chapter 7A New Hybrid Optimization Algorithm forthe Estimation of Archie Parameters

Jianjun Liu, Honglei Xu, Guoning Wu, and Kok Lay Teo

Abstract Archie formula, which contains three fundamental parameters (a, m, n),is the basic equation to compute the water saturation in a clean or shaly formation.These parameters are known as Archie parameters. To identify accurately the watersaturation for a given reservoir condition, it depends critically on the accurateestimates of the values of Archie parameters (a, m, n). These parameters areinterdependent and hence it is difficult to identify them accurately. So we presenta new hybrid global optimization technique, where a gradient-based method withBFGS update is combined with an intelligent algorithm called Artificial Bee Colony.This new hybrid global optimization technique has both the fast convergence ofgradient descent algorithm and the global convergence of swarm algorithm. It isused to identify Archie parameters in carbonate reservoirs. The results obtainedare highly satisfactory. To further test the effectiveness of the new hybrid globaloptimization method, it is applied to ten non-convex benchmark problems. Theoutcomes are encouraging.

Keywords Archie parameters • Hybrid global optimization • ABC algorithm •Gradient-based method

7.1 Introduction

An accurate identification of oil reserve in either an undeveloped or a developedreservoir is a significant task for a petro-physicist and reservoir engineer. Tocalculate the hydrocarbon reserve in its formation, it is required to know the amountof the water saturation. Inaccurate calculation of the amount of the water saturationwill lead to a large error in the estimation of the hydrocarbon reserve.Archie

J. Liu (�) • G. WuCollege of Science, China University of Petroleum, Beijing 102249, Chinae-mail: [email protected]

H. XuDepartment of Mathematics and Statistics, Curtin University, Perth, WA 6845, Australia

K.L. TeoSchool of Mathematics and Statistics, Curtin University, Perth, WA, 6845, Australia


137

138 J. Liu et al.

equation, which is the underlying foundation for analyzing water saturation inpotential oil and gas zones, is commonly used in the calculation of the amount of thewater saturation of a reservoir rock, and hence providing an estimate of the initialhydrocarbon reserve of the reservoir. The values of the input parameters of Archiewater saturation model in a clean or shaly formation must be estimated as accuratelyas possible. From field experiments, it is observed that the values of the followingthree input parameters, the cementation exponent m, the saturation exponent n,and the tortuosity factor a, depend critically on the petro-physical properties of agiven rock. Thus, these parameters will take different values for different fields.Furthermore, the values of a, m and n in Archie formula are interdependent.Therefore, it is of critically importance that the estimations of the values of theparameters a, m and n are done accurately (Mabrouk et al. 2013; Michael et al.2013; Makar and Kamel 2012).

There are several techniques available in the literature for estimating the valuesof Archie parameters a;m and n. A conventional technique is to determine nseparately, independent of a and m. This approach may not be valid in realsituations, as it may lead to large error in the estimation of the amount of thewater saturation (Archie 1942). In (Maute et al. 1992), a data analysis approachis proposed to determine the Archie parameters m; n and a based on standardresistivity measurements on the core samples. The simplex method is applied in(Chen et al. 1995) to identify the three parameters of Archie equation. In (Hamadaet al. 2002, 2013), Archie parameters are being estimated by an approach basedon a three-dimensional (3D) regression plot involving water saturation, formationresistivity and porosity. In (Godarzi et al. 2012), an intelligent algorithm, GA, isapplied to estimation of Archie parameters.

In reality, the parameters a, m and n in Archie equation are known to be closelyinterdependent. In this paper, a new hybrid optimization technique is proposed toestimate these Archie parameters. Several carbonate reservoirs are chosen to carryout comparative experiments using the proposed hybrid optimization technique andother existing techniques.

The rest of the paper is organized as follows. In Sect. 7.2, the model of Archieequation is described, where the three crucial parameters, which are required to beestimated accurately, are clearly indicated. In Sect. 7.3, a new hybrid optimizationmethod is developed. Its properties are being revealed. In Sect. 7.4, the new hybridoptimization technique is applied to estimate the values of Archie parameters. Tobetter appreciate the effectiveness of the proposed hybrid optimization technique,it is applied to solve some benchmark global optimization problems in Sect. 7.5.Finally, some concluding remarks are made in Sect. 7.6.

7.2 Estimations of Archie Parameters

7.2.1 Archie Equation

Archie equation, which relates the resistivity of the formation to the porosity, thewater saturation and the formation water resistivity, is expressed as:

7 A New Hybrid Optimization Algorithm for the Estimation of Archie Parameters 139

Sw D

F Rw

Rt

� 1n

(7.1)

where Sw is the amount of the water saturation (fraction or percentage), n is thesaturation exponent, Rw is the formation water resistivity (� m), Rt is the trueformation resistivity (�m), and F is the formation resistivity factor (dimensionless).

It is known that the formation resistivity factor F is closely related to the porosityof the formation. From the well-known Archie formula reported in (Archie 1942),F can be expressed approximately as:

F D a

ˆm(7.2)

where ˆ is the porosity (dimensionless), a is the tortuosity factor (dimensionless),and m is the cementation exponent (dimensionless).

Archie Eq. (7.1), with ˆ being obtained from independent porosity logs, iscommonly used to estimate the amount of the water saturation Sw provided thatthe values of the parameters a, m and n are known. Here, Archie parameters, i.e., a,m and n, are the most important parameters to be estimated accurately, as they willaffect the accuracy of the estimation of the amount of the water saturation.

The cementation factor, m, varies for different rock types. Its value ranges from1.2 to 3. However, it is usually assumed to have a value of 2. The saturationexponent, n, is a critical parameter in petro-physics, determining a quantitativerelationship between the electrical properties of a reservoir rock and the amountof the water saturation in its formation. In (Ara et al. 2001), the saturation exponentn is reported to be less than 2 (for strongly water-wet rocks) and above 25 (stronglyoil-wet rocks). Like the cementation factor, it is assumed conventionally to be equalto 2. The tortuosity factor, a, is usually assumed to be equal to the value of unity.Obviously, incorrect assumptions of the values of these parameters can lead to largeerrors in the estimation of the amount of the water saturation, and consequently theestimation of the hydrocarbon reserve in a formation.

7.2.2 The Model of CAPE (Core Archie-ParametersEstimation)

Among the various methods available in the literature, Core Archie-ParametersEstimation (CAPE) (Maute et al. 1992; Enikanselu and Olaitan 2013), which isan analysis method, is to estimate the Archie parameters a, m and n by minimizingthe error between the computed water saturation and the measured water saturation.

It is observed that the regressions of IR D S�nw vs Sw and F vs ˆ based on their

plots are not the optimum way to obtain the Archie parameters. For the conventionalmethod, the errors, expressed as least squares form, are minimized with respect tothe parameters such as the formation factor, F, and the resistivity index, IR: For

140 J. Liu et al.

CAPE, the error function, expressed as the difference between the computed andthe measured water saturations given by

f .m; n; a/ DMX

jD1

NXiD1

Swij �

aRwij

ˆjmRtij

� 1n

!2(7.3)

is minimized subject to 1:2 � m � 3:0, 1:0 � n � 3:0; and 0:5 � a � 1:5

to obtain the values of the Archie parameters, where j is core index, i is index foreach of the core j measurements, Swij is ith laboratory measured water saturation forcorej (fraction), Rtij is ith laboratory measured resistivity for core j, and � j is corejporosity (fraction). This function f (m, n, a) may hold multiple stationary points withthe different values of Swij , Rtij and � j, so it would be likely to has several optima.

The three parameters in the function f (m, n, a) are interdependent. For a specificcarbonate corn sample, three surfaces of f (m, n, a) as a function of n and a withm D 2, m and a with n D 2, and m and n with a D 1, are depicted in Fig. 7.1. Fromthe figures, we can see the correlation of the three parameters m, n and a in Archieequation.

The validity of CAPE is under the following two assumptions: (i) Archie formulais valid for the carbonate core sample concerned; and (ii) the core sample used is avalid representative of the zone under consideration. Based on both assumptions andthe model of CAPE, we shall develop a new hybrid optimization method to solvethe minimization problem (3) as a box-constrained optimization problem.

7.3 A New Hybrid Algorithm

The methods to solve the global optimization problems (GOPs) can be classified intotwo main classes: deterministic methods, and stochastic methods. The first class,which makes use of the some deterministic information to solve GOPs, includesthe tunneling method (Levy and Gomez 1985), and filled function methods (Lianget al. 2007; Liu and Xu 2004). The second class, which relies on probabilistictechniques, includes Ant Colony Optimization algorithm (Toksari 2006), GeneticAlgorithm (Goldberg 1989; Michaelewicz 1996), Simulated Annealing algorithm(Kirpatrick et al. 1983), and Particle Swarm Optimization (Eberhart and Kennedy1995; Kennedy and Eberhart 1995; Shi and Eberhart 1998). Especially, ArtificialBee Colony (ABC) holds better performance than most of the stochastic algorithmsmentioned above (Karaboga and Basturk 2007, 2008). The deterministic algorithmswhich are gradient-based converge rapidly. However, they will get stuck in localminima of a multimodal function. On the other hand, stochastic optimization algo-rithms, which tend to perform global optimization, are computationally expensivedoing random searches. Therefore, an approach that combines thestrengths of


Fig. 7.1 The three surfacesof f (m, n, a) as a D 1, m D 2and n D 2, respectively

m=2

8

6

4

2

03

2 2 3

nn=2

1 1a

1

2

3

4

5

6

40

30

20

10

03

3

21

1

a

a=1

m2

5

10

15

20

25

30

35

20

15

10

5

03

322

1 1m n

2

4

6

8

10

12

14

142 J. Liu et al.

stochastic and deterministic optimization schemes but avoids their weaknesses isof much interest. For details on such an approach, see (Noel 2012; Yiu et al. 2004;Garcia-Palomares et al. 2006).

In this section, a hybrid algorithm, which combines the strengths of a gradient-based optimization technique and ABC algorithm, will be presented.

Let f (x) be a twice continuously differentiable non-convex function defined on

the set � Dnx 2 R

nˇˇ a � x � b

o, where a and b 2 R

n. We assume that all the

minima of f (x) are isolated minima and that there is a finite number of them. Weconsider the problem of finding the global minimum of f (x) on the set �.

The new hybrid descent method may be formally stated as follows:

Step 1. Initialization. Generate x0 randomly and evaluate f (x0). Set k: D 0.Step 2. Local search. Search for a local minimum of f (x) by using a gradient-based

algorithm with xk as the initial point. Let it be denoted as xkC1.

If xjkC1 < lbj; then xj

kC1 D lbj and else if xjkC1 > ubj; then xj

kC1 D ubj.Then set x.kC1/� D xkC1.If f

�x.kC1/�

�< f

�xk�

�, then x� D x.kC1/� , else x� D xk�. Goto Step 3.

Step 3. Find a better solution by carrying out ABC algorithm.Set yk WD x� as the current minimizer and then execute the ABC iterations

until the stopping criteria of the ABC algorithm are met. Output a new globalminimizer y*.

Step 4. Set k: D k C 1, xk D y�. Return to Step 2 until convergence.

In Step 2 of the algorithm, the quasi-Newton algorithm with BFGS update isused to perform the local search. In Step 3, the search of ABC composes of threekey steps (Karaboga and Basturk 2007, 2008). They are:

1. Sending the employed bees into the food sources and then measuring their nectaramounts;

2. Selecting the food sources by the onlookers after sharing the information ofemployed bees and determining the nectar amount of the foods;

3. Determining the scout bees and then sending them onto possible food sources.

In ABC algorithm, the artificial bee colony consists of three kinds of bees:employed bees, onlookers and scouts. Half of the colony is made up of employedbees, and the other half includes onlooker bees and scouts. Employed bees searchfor the food around the food source in their memory; meanwhile they share theirfood information with onlookers. Onlooker bees tend to select better food sourcesfrom those found by employed bees, and then further search for the food aroundthe selected food source. Scouts abandon these food sources and search for newones. Whenever a scout or onlooker bee finds a food source, it becomes anemployed bee. Whenever a food source is exploited fully, all the employed beesassociated with it abandon it, and become scouts or onlookers. Scout bees performthe job of exploration, whereas employed and onlooker bees perform the job ofexploitation.


In ABC algorithm, the position of a food source is a potential solution of theoptimization problem and the nectar amount of a food source corresponds to thefitness of the associated solution. The number of employed bees (N) is equal tothe number of food sources (SN) because it is assumed that for every food source,there is only one employed bee. After generating a randomly distributed initialpopulation of size SN of solutions, each of the employed and onlooker bees exerts aprobabilistically modification on the solution (the position of a food source) forfinding a new solution (new food source position) and tests the fitness (nectaramount) of this new solution (new food source).

Suppose each solution consists of D parameters and let Yti D

yti1; y

ti2; ; yt

iD

�denotes to the i-th solution generated in cycle t with parameter valuesyt

i1; yti2; ; yt

iD. In the ABC algorithm, every employed bee produces a new solutionVt

i D vt

i1; vti2; ; vt

iD

�in a D-dimensional search space, from the old one Yt

iaccording to the following equation

vtij D yt

ij C ' tij

yt

ij � ytkj

�(7.4)

where j 2 f1; 2; : : : ; Dg, and k is selected randomly fromf1, 2, : : : , Ng suchthat k ¤ i. ®t

ij is a random scaling factor. When all employed bees have finishedtheir searching process, they share the fitness (nectar) information of their solution(food sources) with the onlookers. Then each of these onlookers selects a solutionaccording to a probability proportional to the fitness value of that solution. Equation(7.4) is applied again to generate a new solution by an onlooker bee based on theold solution in her memory and the selected one. If the fitness amount of the newsolution is better than the old one, the bee memorizes the new position and forgetsthe old one. The probability value, pi, by which an onlooker bee chooses a foodsource is calculated according to the following equation:

pi D fitiXSN

jD1fiti(7.5)

where fiti is the fitness value of the solution i and SN is the number of food sources.When the nectar of the food source is abandoned by employed bees, the scout

bees replace it with a new one. In the ABC algorithm, if the quality of a solutioncannot be improved after a predetermined number of cycles called “limit”, the scoutbee replaces the abandoned solution with a new one chosen randomly. In such acondition, the new solution is constructed according to the following equation

ytij D yjmin C rand .0; 1/

yjmax � yjmin

�(7.6)

whereyjminand yjmax are, respectively, the lower and upper bounds on the value of thejth parameter.

In this paper, the search mechanism, proposed in (Karaboga and Basturk 2007),is chosen to escape from a local solution in the new hybrid method. The search

144 J. Liu et al.

strategy is determined by the parameters, SN, the number of the food sources whichis equal to the number of employed or onlooker bees; limit, a predetermined numberof cycles; and the maximum cycle number, MCN. The ABC algorithm can beimplemented as follows:

Initiation. Set SN, limit, MCN, and cycle D 1. Generate an initial population of thepotential solutions Yi; i D 1; 2; : : : ; SN, based on x*, and Evaluate f (Yi).

repeat

1. Produce new solutions Vi for the employed bees by using Eq. (7.4)2. Assume that the greedy selection process is adopted by the employed bees3. Calculate the probability values pi for the solutions Yi by using Eq. (7.5)4. With the probability values pi, produce the new solutions Vi for the onlookers

from the solutions Yi

5. Assume that the greedy selection process is adopted by the onlookers6. Determine the abandoned solution for the scout, if exists, and replace it with

a new randomly produced solution Yi by using Eq. (7.6)7. Memorize the best solution achieved so far8. cycle D cycle C 1

until cycle D MCN

In the proposed hybrid algorithm, BFGS algorithm is executed from initial pointx0 to find a local minimum of f (x) with high-speed descent. Then the ABC algorithmis used to escape from the local solution to find a better solution, y*, which will betaken as the new starting point for the BFGS algorithm in the next cycle. A betterminimum x* found by the BFGS algorithm is taken as the best solution (food source)in the ABC algorithm and is memorized. If the best solution y* found by the ABCalgorithm has a better value than the former memorized solution x*, then y* is usedas the starting point for the BFGS algorithm. This guarantees that a local searchoperates in the neighborhood of the best solution found by the proposed algorithmin all previous iterations.

Incidentally, the proposed algorithm can be generalized by using more powerfullocal search techniques to refine the best solution found in the ABC. Derivative-freetechniques like Nelder–Mead Simplex method or Hooke-Jeeves (Fei Kang et al.2011) method can be used for the local search when the objective function is notcontinuously differentiable.

7.4 Estimation of the Archie Parameters

In this section, the focus is on the estimation of the Archie parameters for carbonatereservoirs. The carbonate reservoirs are of great importance because they containalmost 60 % of world’s oil reserves. The accuracy of Eq. (7.1) depends on theaccuracy of the estimates of the input parameters Rw, Rt, and F.


Table 7.1 The values ofArchie parameters obtainedfrom three techniques

Method a m n

3D method (Hamada et al. 2013) 0.28 2.34 2.12CAPE (a, m, n) (Hamada et al. 2013) 0.23 2.15 2.87BFGSABC 1.14 1.62 2.02

Table 7.2 Error analysis on the determined Archie parameters by three techniques

Absolute errorMethod Ea Emin Emax Erms S

3D method (Hamada et al. 2013) 0.102 0.002 0.51 0.14 0.10CAPE (a, m, n) (Hamada et al. 2013) 0.095 0.001 0.33 0.12 0.08BFGSABC 0.035 0.004 0.026 0.08 0.07

Note: Ea the average absolute relative error, Emin/Emax the minimum/maximum absolute error, Sthe standard deviation. Erms the root mean square error

Hereinafter, 29 carbonate core samples taken from (Hamada et al. 2013) areselected as the simulations of certain wells, that is N D 29. For each core sample, theelectrical resistivity Rw and Rt at different water saturation percentages are measuredat room temperature, that is M D 30. Set x D .m; n; a/, then BFGSABC algorithmcan be applied to solve the model indicated by Eq. (7.3).

Based on 30 independent core samples measurements, the data obtained by theproposed hybrid method are compared with the data computed by 3D method andCAPE.

Typical values of the Archie parameters obtained using the CAPE method,the 3D method (Hamada et al. 2002, 2013) and the proposed hybrid method areas shown in Table 7.1. Table 7.2 demonstrates the average error, the root meansquare error and standard deviation of the water saturation computed by threetechniques.

The average error, the root mean square error and standard deviation are shownin Fig. 7.2. Figure 7.3 displays the linear regression estimation for three techniques.Typical results of the measured water saturation and the estimated water saturationprofiles for different Archie parameters obtained by using CAPE, 3D and theproposed hybrid method (BFGSABC) are illustrated in Fig. 7.4a. Figure 7.4bdepicts water saturation relative error profiles calculated by the three options againstselected interval for core samples.

From Figs. 7.2, 7.3, and 7.4, the measured water saturation and the estimatedwater saturation profiles calculated by different methods are clearly demonstrated.These profiles support the accuracy analysis in regards of the performance ofdifferent techniques in order to get the most accurate Archie parameters. Note thatwater saturation computed by the proposed technique has a better matching with themeasured water saturation than other two methods.

146 J. Liu et al.

0.102

0.14

0.10.095

0.12

0.08

0.035

0.080.07

average error RMS error standard deviation

3D Method CAPE (a,m,n) BFGSABC

Fig. 7.2 The average error, RMS error and standard deviation between the three techniques

0.9BFGSABC3D MethodCAPE (a,m,n) Method

0.8

0.7

0.6

0.5

0.4

com

pute

d S

w

measured Sw

0.3

0.2

0.10.2 0.25 0.3 0.35 0.4 0.45 0.5 0.55 0.6

Fig. 7.3 The linear regression estimation for three techniques

7.5 The Performance of the Proposed Algorithm on SomeTest Functions

In order to test the performance of the proposed algorithm, it is applied to severalrepresentative benchmark functions chosen from (Andrei 2008; Jamil and Yang2013). Those test functions can be classified into two classes. (1) Unimodal func-tions, which have no other minimum, except one global minimum. The following aresome of such unimodal functions: bowl-shaped Sphere function(f1), valley-shapedRosenbrock function (f2), steep drops Easom function(f7), other shaped Goldstein-Price function (f8), and Branin function(f9). (2) Multimodal functions, which have


1a b

3

5

7

9

11

13

Sw-MeasuredSw-CAPESw-3D

Sw-BFGSABC

Sw-CAPESw-3D

Sw-BFGSABC

15

17

No.

of c

ore

sam

ples

No.

of c

ore

sam

ples

19

21

23

25

27

290.2 0.4 0.6 0.8

Ws1 1.2

1

3

5

7

9

11

13

15

17

19

21

23

25

27

290 50

Relative Error

100

Fig. 7.4 (a) Comparison between the measured water saturation with the calculated watersaturation for three techniques, and (b) Relative error between three techniques

many local minima. The following are some of such multimodal functions: Rastriginfunction(f3), Ackley function(f4), Griewank function(f5), Schaffer function(f6) andLevy function (f10).The ten test functions, their dimensions and modalities are listedin details in the table of the Appendix.

In the experiments of BFGSABC on test problems, the number of maximumgenerations are 50, 100 and 500 for the dimensions of 2, 10 and 50, and thepopulation sizes are 20, 20 and 200, respectively. Because BFGS is the deter-ministic algorithm, only ABC and BFGSABC performed for 50 independent runson 10 functions. The calculation is done within the Matlab 7.70 environment.The computer was functioned with double cores 2.5 GHz CPU PC running inwindows 7.

148 J. Liu et al.

Table 7.3 The global minima of test functions in 2D found by BFGS, ABC and BFGSABC

Best value MeanFun. Dim. BFGS ABC BFGSABC ABC BFGSABC

f1 2 1.256E-16 1.12e-12 5.59E-20 1.05e-09 2.29E-20f2 4.30E-14 3.32E-04 1.94E-18 0.1437 1.32E-18f3 9.949 8.84E-11 0.0 0.0017 0.0f4 17.612 5.98E-07 8.88E-16 5.43E-05 3.23E-15f5 97.449 1.05E-05 0.0 0.0101 0.0f6 0.494 9.91E-05 0.0 0.0127 0.0f7 0.0 �0.9886 �1.0 �0.1835 �1.0f8 84.000 3.0000 2.99 3.5450 2.99f9 0.397 0.3979 0.39 0.3983 0.39f10 1.844E-15 1.35E-17 4.822E-20 2.47E-12 7.22E-20

Table 7.4 The global minima found by BFGS, ABC and BFGSABC for 10 dimensions


f1 10 1.12E-16 3.5834E-04 4.55E-17 0.0883 8.51E-17f2 1010.67 1.0431 5.91E-11 21.9949 6.91E-11f3 97.50 1.0878 1.13E-9 5.1409 5.12E-9f4 19.20 0.2565 3.60E-5 1.9434 6.89E-5f5 0.147 0.0436 2.65E-14 0.2768 4.46E-14f10 7.85E-11 2.7850E-05 4.79E-14 0.0011 9.62E-14

Table 7.5 The global minima found by BFGS, ABC and BFGSABC for 50 dimensions


f1 50 7.94E-16 4.13E-06 3.58E-16 4.0423E-05 5.45E-16f2 6.47 E C 6 52.3644 7.6 E-3 225.8868 8.6 E-3f3 4.64 E C 2 28.6034 5.96E-1 40.8226 9.96E-1f4 1.93 E C 2 1.8572 3.233E-8 3.3657 6.73E-7f5 1.60E-2 0.0230 2.90E-13 0.4623 2.90E-12f10 8.46E-7 5.92E-04 1.39E-11 0.0670 1.39E-11

All the ten functions considered are in 2 dimensions and some of these functionsare in higher dimensions. The results of experiments are listed in following tables.Bold fonts in Tables 7.3, 7.4, and 7.5 indicate that the BFGS algorithm fails to solvethe problems because of being trapped into a local minimum.

After comparison of the data in Table 7.3, for 10 test functions of 2 dimensions, itcan be show that the global minimums of f3�f8 cannot be found by BFGS. AlthoughABC finds the approx global minimums, their accuracy of minimal function valuesand the mean value for all of 10 functions is lower than BFGSABC.


105a b

c d

initial point

after BFGS search

After ABC search

initial point

after BFGS search

After ABC search

initial point

after BFGS search

After ABC search

initial point

after BFGS search

After ABC search100

10-5

Loga

rithm

of f

unct

ion

valu

eLo

garit

hm o

f fun

ctio

n va

lue

Loga

rithm

of f

unct

ion

valu

e

10-10

10-15

102 100

10-5

10-10

10-15

10-20

100

10-2

10-4

1 2 3 4

Iteration

5 6 7 1 2 3 4

Iteration

5 6 7

105

100

10-5

Loga

rithm

of f

unct

ion

valu

e

10-10

10-15

0 5 10 150 2 4

Iteration Iteration6 8 10

Fig. 7.5 Typical convergence history for the new algorithm for multimodal functions in 2dimensions. (a) f3 Rastrigin function; (b) f4 Ackley function; (c) f5 Griewank function; (d) f6

Schaffer function

From Tables 7.4 and 7.5, it is observed that the hybrid method has betterperformance than both BFGS and ABC in terms of the best values found for higherdimensions.

Monotonic convergence, which is a very desirable property, is observed forthe proposed hybrid method. See, for example, the typical convergence historiesfor the algorithm on the test functions f3, f4, f5 and f6 in 2 dimensions, whichare displayed in Fig. 7.5. Since the ABC method is mainly used for bypassingthe previously converged local minimum and discovering the descent point, thedecrease in function value after executing each ABC search might be small.

In addition, we study two multimodal functions f5 and f10 in 1,000 dimensions.The maximum numbers of generations are 2,000 and the population size is 200.These two functions have many local minima, which are regularly distributed.Table 7.6 shows the best and mean values, CPU time and numbers of functionevaluation.

150 J. Liu et al.

Table 7.6 The optimal information on two 1,000 dimensions test functions by the proposedmethod

Function Best value Mean CPU time (s) Number of function evaluation

f5 1.33E-8 7.90E-15 1,755 6.3E C 7f10 1.01E-5 5.22E-2 2,713 6.6E C 7

From Tables 7.4, 7.5, and 7.6, it can be observed that the proposed hybrid methodcan find the best “global” minima when compared with BFGS and ABC methodsavailable for the ten test functions. Furthermore, the success rate of finding the“global” minima is 100 % for the new proposed hybrid method.

It can be concluded that the hybrid method proposed in this paper has betterperformance in solving global optimization problems, especially in the rate ofconvergence speed, reliability and the quality of the solution obtained.

7.6 Conclusions

For an accurate estimation of the Archie parameters, a new hybrid global searchmethod, which combines the well-known quasi-Newton algorithm (BFGS) and thepopulated global search algorithm (ABC), is proposed. The ABC technique playsthe role of escaping from a local minimum to a better descent point from whichthe local search can restart to find a better minimum. The hybrid method inheritsboth the convergent rate and accuracy of the BFGS and the capability of escapingfrom local minima of the ABC. Numerical results on ten benchmark problems haveshown that global minimum, especially for multimodal continuous functions, canbe sought using this hybrid descent method with very nice monotonic convergencehistory. The results obtained from the simulation experiments on carbonate coresamples show that the proposed method has better performance than other methodsin terms of the accurate estimates of the Archie parameters. From the experimentresults, we observe that the water saturation computed by the proposed methodmatches well with the measured water saturation when compared with the othertwo methods.

Acknowledgements The authors gratefully acknowledge the financial support from the NationalNatural Science Foundation of China (Grant No. 11371371, No. 11171079) and the Foundationsof China University of Petroleum (No. KYJJ2012-06-03, KYJJ2012-12).


App

endi

x:E

xpre

ssio

nsan

dP

rope

rtie

sof

Ten

Tes

tP

robl

ems

Fun

Exp

ress

ion

Ran

geO

ptim

also

luti

onO

ptim

alva

lue

Mod

alit

ies

f 1f S

phD

n X iD1

x2 i�100

�x i

�100

. 0;��

�;0/

0U

ni

f 2f R

osD

n�1 X iD1

h 100

� x i�1

�x2 i

� 2 C. x

i�1/2i

�30

�x i

�30

. 0;��

�;0/

0U

ni

f 3f R

asD10n

Cn X iD1

� x2 i�10

cos. 2�

x i/�

�5:12

�x i

�5:12

. 0;��

�;0/

0M

ulti

f 4f A

ckD20

Ce

�20e�

1 5

v u u u u t1 n

n X iD1

x2 i

�e�

1 n

n X iD1

cos.2�

x i/

�32:768

�x i

�32:768

. 0;��

�;0/

0M

ulti

f 5f G

riD

14000

n X iD1

x2 i�

n Y iD1

cos� x i

p

i� C1

�600

�x i

�600

. 0;��

�;0/

0M

ulti

f 6f S

chD0:5

Csi

n2q x2 1

Cx2 2

�0:5

� 1C0:001 x2 1

Cx2 2�� 2

�100

�x i

�100

(0,0

)0

Mul

ti

f 7f E

asD

�cos

x 1co

sx 2

e.�

x 1��/2

�. x2��/2

�100

�x i

�100

. ��;�/

�1U

ni (con

tinu

ed)

152 J. Liu et al.

(con

tinu

ed)

Fun

Exp

ress

ion

Ran

geO

ptim

also

luti

onO

ptim

alva

lue

Mod

alit

ies

f 8

f GP

Dh 1

C. x1

Cx 2

C1/2 19

�14x 1

C3x2 1

�14x 2

C6x 1

x 2C3x2 2�i

h 30

C. 2

x 1�3x 2/2 18

�32x 1

C12x2 1

C48x 2

�36x 1

x 2C3x2 2�i

�2�

x i�2

. 0;�1/

3U

ni

f 9f B

raD

a x 2–b

x2 1C

cx1

�r� 2

Cs. 1

�t/

cos. x1/

Cs;

wit

ha

D1;b

D5:1

4�2;

cD

5 �;

rD6;

sD10;

tD

1 8�

�5�

x 1�100

�x 2

�15

. ��;12:275/;

(�,2

.275

),(9

.424

78,2

.475

)

0.39

7887

Uni

f 10

f Lv

eD

sin2. �

y 1/

Cn�

1 X iD1

h . yi�1/2 1C

10si

n2. �

y iC1/�i

C.y n

�1/2 1C

10si

n2. 2�

y n/� ;

whe

rey i

D1

Cx i

�1

4:

�5�

x i�10

(1,1

,:::,

1)0

Uni


References

Andrei N (2008) An unconstrained optimization test functions collection. Adv Model Optim10(1):147–161

Ara TS, Talabani S, Atlas B, Vaziri HH, Islam MR (2001) In-depth investigation of the validity ofthe Archie Equation in carbonate rocks, SPE 67204, pp 1–10

Archie GE (1942) The electrical resistivity log as an aid in determining some reservoir character-istics. Trans AIME 146:54–62

Chen DS et al (1995) Novel approaches to the determination of Archie parameters I: simplexmethod. SPE Adv Technol Ser 3(1):39–43

Eberhart RC, Kennedy J (1995) A new optimizer using particle swarm theory. In: Proceedings ofthe 6th symposium on micro machine and human science, Nagoya, Japan, pp 39–43

Enikanselu PA, Olaitan OO (2013) Determination of Archie parameters and the effect on watersaturation over “honey” field, Niger-delta. Can J Comput Math Nat Sci Eng Med 4(4):306–314

Fei Kang, Junjie Li, Zhenyue Ma, Haojin Li (2011) Artificial bee colony algorithm with localsearch (HJ)for numerical optimization. J Softw 6(3):490–497

Garcia-Palomares UM, Gonzalez-Castan FJ, Burguillo-Rial JC (2006) A Combined Global &Local Search (CGLS) approach to global optimization. J Glob Optim 34:409–426

Godarzi AA et al (2012) The simultaneous determination of Archie’s parameters by applicationof modified genetic algorithm and HDP methods: a comparison with current methods via twocase studies. Pet Sci Technol 30(1):54–63

Goldberg DE (1989) Genetic algorithms in search, optimization and machine learning. Mas-sachusetts: Addison-Wesley

Hamada GM, Al-Awad MNJ, Alsughayer AA (2002) Water saturation computation from labora-tory, 3D Regression. Oil Gas Sci Technol Rev. IFP 57(6):637–651

Hamada GM, Almajed AA, Okasha TM et al (2013) Uncertainty analysis of Archie’s parametersdetermination techniques in carbonate reservoirs. J Pet Explor Prod Technol 3(1):1–10

Karaboga D, Basturk B (2007) A powerful and efficient algorithm for numerical functionoptimization: artificial bee colony (ABC) algorithm. J Glob Optim 39(3):459–471

Karaboga D, Basturk B (2008) On the performance of artificial bee colony (ABC) algorithm. ApplSoft Comput 8(1):687–697

Kennedy J, Eberhart RC (1995) Particle swarm optimization. In: Process of IEEE internationalconference on neural networks, Piscataway, pp 1942–1948

Kirpatrick S, Gelatt CD, Vecchi MP (1983) Optimization by simulated annealing. Science, NewSeries 220(4598):671–680

Levy AV, Montalvo A (1985) The tunneling algorithm for the global minimization of functions.SIAM J Sci Stat Comput 6(1):15–29

Liang YM, Zhang LS, Li MM, Han BS (2007) A filled function method for global optimization. JComput Appl Math 205:16–31

Liu X, Xu W (2004) A new filled function applied to global optimization. Comput Oper Res31:61–80

Mabrouk WM, Soliman KS, Anas SS (2013) New method to calculate the formation waterresistivity (Rw). J Pet Sci Eng 10:49–52

Makar KH, Kamel MH (2012) An approach for velocity determination from merging Archie andRaymer–Hunt–Gardner transform in reservoir of clean nature. J Pet Sci Eng 86–87:297–301

Maute RE, Lyle WD, Sprunt E (1992) Improved data-analysis method determines Archieparameters from core data. J Pet Technol 44(1):103–107

Michael R, Collett TS et al (2013) Large-scale depositional characteristics of the Ulleung Basin andits impact on electrical resistivity and Archie-parameters for gas hydrate saturation estimates.Mar Pet Geol 47:222–235

Michaelewicz Z (1996) Genetic algorithms C Data structures D Evolution programs. Springer,Berlin

154 J. Liu et al.

Momin Jamil, Xin-She Yang (2013) A literature survey of benchmark functions for globaloptimization problems. Int J Math Model Numer Optim 4(2):150–194

Noel MM (2012) A new gradient based particle swarm optimization algorithm for accuratecomputation of global minimum. Appl Soft Comput 12:353–359

Shi Y, Eberhart RC (1998) A modified particle swarm optimizer. In: Proceedings of the IEEECongress on Evolutionary Computation (CEC 1998), Piscataway, pp 69–73

Toksari MD (2006) Ant colony optimization for finding the global minimum. Appl Math Comput176(1):308–316

Yiu KFC, Liu Y, Teo KL (2004) A hybrid descent method for global optimization. J Glob Optim28:229–238

Chapter 8Optimization of Multivariate Inverse MixingProblems with Application to Neural MetaboliteAnalysis

A. Tamura-Sato, M. Chyba, L. Chang, and T. Ernst

Abstract A mathematical methodology is presented that optimally solves aninverse mixing problem when both the composition of the source componentsand the amount of each source component are unknown. The model is usefulfor situations when the determination of the source compositions is unreliableor infeasible. We apply the model to longitudinal proton magnetic resonancespectroscopy (1H MRS) data gathered from the brains of newborn infants. 1HMRS was used to study changes in five metabolite concentrations in two brainregions of nine healthy term neonates. Measurements were performed three timesin each infant over a period of 3 months, starting from birth, for a total of 27scans. The methodology was then used to translate the metabolite concentrationdata into measures of relative density for two major brain cell type populationsby fitting a matrix of metabolite concentration per unit density to the data. Onecell type, reflecting neuronal density, increased over time in both regions studied,but especially in the frontal regions of the brain. The second type, characterizedprimarily by myoinositol, reflecting glial cell content, was found to decrease inboth regions over time. Our new method can provide more specific and accurateassessments of the brain cell types during early brain development in neonates. Themethodology is applicable to a wide range of physical systems that involve mixingof unknown source components.

A. Tamura-Sato (�) • M. ChybaDepartment of Mathematics, University of Hawai‘i at Manoa, 2565 McCarthy Mall,96822 Honolulu, Hawai‘ie-mail: [email protected]

L. Chang • T. ErnstDepartment of Medicine, University of Hawai‘i at Manoa, Honolulu, Hawai‘i


155

156 A. Tamura-Sato et al.

8.1 Introduction

Many scientific problems can be described as combining multiple sources to form afinal mixture; this is denoted herein as a multivariate mixing problem. Each sourcemay contain an amount of certain components we are interested in tracking. Forexample, if the salt content of two different source solutions of salt water is known,then one can determine the salt content of mixtures made from the two sources, aslong as the amount of each source added to the mixtures is known. Similarly, givenseveral bronze alloy sources made of different amounts of copper, lead, and zinc,one could determine the copper, lead, and zinc content of a final product obtainedby mixing the sources. This can be modeled by the matrix equation AX D C, wherethe composition matrix A contains information on the amount of each componentper unit of source, and the population matrix X contains the contribution of eachsource to each final product. Then C is naturally a matrix which gives the amountof each component in each final product.

The goal of the inverse mixing problem is to reverse this calculation. Now, weknow the amount of each of the components in the final products, but wish todetermine how much of each source was used to create them. In other words, weknow C, but need to solve for X. In many applications, the composition matrix Ais known or can be determined by secondary experiments, but this is not alwaysfeasible. In this paper, we will analyze the situation where the composition matrixis unknown.

When calculating X in an inverse mixing problem, it is frequently the case thatthe system is overdetermined and no X can be found that perfectly fits the equationAX D C. Instead, X is calculated using a least squares method that minimizeskEk D kC � AXk. Such optimization techniques have been used and refined fora variety of scientific applications, including chemistry and geophysics (Cantrell2008; Snieder and Trampert 1999). In these applications, the composition matrixA is known. However, in some situations, A cannot be determined or the datacollected for A is unreliable or ultimately insufficient for the analysis. For instance,an attempted analysis on streamwater was inconclusive due to insufficient data onmeasured source compositions (Christopherson et al. 1990). Therefore in our model,we attempt to find a best fit solution for A and X simultaneously since we lackinformation about the matrix A. To guide our fit, we use certain constraints on thematrices A and X that reflect the physical reality of the system.

While principal component analysis (PCA) is sometimes used as a methodto extract suitable parameter values for A from data, it is not sufficient for thistask. PCA can provide the rank of A that should be used, but provides littleinformation useful in determining what the composition of the sources should be(Christopherson and Hooper 1992). In addition, PCA principal components must beorthogonal to each other and the first principal component is chosen to explain asmuch of the variance as possible. Such restrictions may not be realistic or desired.Our approach allows us to explain most of the variance without forcing the columnsof the composition matrix to be orthogonal, and without assuming that one particularsource causes most of the variance in the data.

8 Optimization of Multivariate Inverse Mixing Problems with Application to. . . 157

It is our goal to apply an optimization procedure that minimizes residual valuesusing an interior point algorithm to a situation where the composition matrix isunknown. Of importance to our methodology is the enforcing of constraints on theindividual components of A and X. This differentiates our approach from PCA orsparse PCA techniques, which can suppose some constraints, but not on individualcomponents (Hunter and Takane 2002; Takane and Hunter 2001; Zou et al. 2006).Thus, our methodology optimizes A and X to deliver the best fit to known data,subject to certain constraints.

Our approach is motivated by an application where composition data are impossi-ble to determine. Specifically, we will apply our methodology to the analysis of datagathered by proton magnetic resonance spectroscopy (1H MRS) from the neonatalbrain in order to determine the relative level of density of multiple populationsof cells in different regions of the brain. To our knowledge, this is the first timesuch an analysis has been performed. 1H MRS is a non-invasive spectroscopictechnique that allows the measurement of several brain metabolites, and has beenused to evaluate the early developing brain (Kreis et al. 2002; Pouwels et al. 1999).One of the strengths of MRS is its promise to identify and characterize variouscellular compartments in the developing brain, such as neurons, glial cells (Brandet al. 1993; Guimaraes et al. 1995), and possibly neural stem cells (Manganas et al.2007). In the clinical setting, 1H MRS has found application in the evaluation anddiagnosis of hypoxia (Ancora et al. 2010; Cheong et al. 2006; van Doormaal et al.2012), shaken baby syndrome (Haseler et al. 1997), metabolic diseases (Befroy andShulman 2011), the effects of preterm delivery on the brain (Wang et al. 2008), andmany other brain disorders. 1H MRS has also been used to study brain biochemicalchanges and maturation in healthy adults and neonates (Kirov et al. 2008). However,the interpretation of MRS findings in terms of anatomical and physiological aspectsof the brain can be difficult since the data reflect a heterogeneous distribution ofcells within a specific region of interest.

In this particular application, we cannot determine the composition matrix Asince pure populations of specific neural cell types are not found in vivo, andinvasive measurements are impossible. We will assess for two major cell types,characterized by tentative metabolic markers for glia and neurons, and attempt todetermine the density level of each type in two separate regions of the neonatal brain.Further work is ongoing to examine several other regions and disease conditions.Our methodology therefore seeks to simultaneously determine A, the matrix ofmetabolic concentration per unit of brain cell density, and X, the matrix of braincell density in each MRS experiment, given C, a matrix of metabolic concentrationsmeasured by MRS. The goal is to optimize the norm of a percentage error matrixcalculated from E D C � AX, subject to certain known constraints, such as non-negativity for elements of A and X.

In this paper, we will first introduce our methodology in general terms.We will then briefly describe the experimental procedure for the MRS study,before applying the methodology to the specific data acquired and analyzing theresults.


8.2 Methodology

We introduce the mathematical model to translate measured component values(such as total metabolite or chemical ion concentrations) into relative contributionsof sources (such as cell density or chemical solutions). We assume conservationof all components in the mixture. For example, there are no chemical reactionsor precipitation. Let us consider a general case with m measurements taken of pcomponents, with n sources.

We introduce �i D .�1i ; �2i ; : : : ; �

pi / the component spectrum per unit of source

i. Thus, �21 represents the amount of component 2 per unit of source 1. Givenmeasurement k, the contribution of source i will be denoted xk

i .Our model is based on the assumption that for a given measurement k, the

measured value for component j, ck;j, is obtained as a linear superposition ofthe component spectra �i adjusted by the contribution of each source. In matrixnotation, AX D C where A is the matrix of component spectra and X the matrix ofcontributions by each source.

In practice, we usually have an overdetermined system. If the composition matrixA is known, a least square approach is used, and it is well known that the bestsolution to AX D C is given by X� D .ATA/�1ATC D A�C where A� is the Moore-Penrose pseudo-inverse (i.e. if c represents a column vector of the matrix C then theminimal residual to c � Ax is given by x D A�c).

However, when A is unknown and we can impose the number of sources, wemust take a different approach. To account for the fact that some components mayhave a significantly smaller value than others, we will minimize the percentage errorand not the absolute error. A normalization constraint is imposed on the columns ofA, to prevent scaling from becoming an issue in the optimization. Several differentconstraints can be used, such as setting the sum of squares to 1 or requiring columnsof A to be orthogonal, similar to the PCA procedure. Then our problem becomes:Given the measured component values for our set of m measurements, find thebest fit in terms of component spectra and source contribution that minimizes thepercentage residual.

To find a solution, X must solve the normal equations associated with theequation AX D C:

ATAX D ATC (8.1)

The matrix ATA is a symmetric n � n matrix, and is invertible provided thatdet.ATA/ ¤ 0 (which is assumed in the sequel). Our problem can then bereformulated as follows. For a given C, determine A such that we minimize the normof the percentage residual matrix E% with respect to C, where E% is calculated fromthe absolute error matrix, E, given by:

AA�C � C D E (8.2)

Then the entries of E% are given by e%k;j D 100% � ek;j

ck;j for metabolite j andmeasurement k.


Geometrically, this can be interpreted as follows. The operator P D AA� is theorthogonal projection (i.e. P2 D Id) onto the space generated by the columns of A.Our goal is therefore to identify a matrix A that minimizes the sum of the square ofthe percentage residuals (orthogonal to the span of A) over all the measurements(i.e. when applied to all column vectors of the matrix C). Once a matrix A isknown we can determine the matrix X of source contributions using X D A�C.This methodology is in contrast to the technique used in Christopherson and Hooper(1992), which is limited to the space spanned by the most relevant axes given by aPCA analysis. Conversely, our optimization evaluates the entire space.

Depending on the situation, partial information may be known about the valuesof A that can be incorporated into the model. Frequently, for example, negative �or x values do not occur in nature; for instance, when concentrations are involved.It may also be known that certain components are absent from a particular source.These can be introduced as constraints on the optimization problem.

To simplify our notations further, we introduce y D .�11 ; �21 ; : : : ; �

12 ; �

22 ; : : : ; �

pn / 2

Rnp. Let E%.y/ denote the percentage error matrix obtained for A determined by y.

Then we wish to minimize f .y/ D kE%.y/k, the L2-norm of E%, subject to equalityand inequality constraints on A. Our optimization problem becomes:

miny

f .y/; f .y/ D kE%.y/k (8.3)

subject to the following constraints:

hi D 0 (8.4)

gj � 0 (8.5)

with equality constraints hi and inequality constraints gj.This constrained optimization problem can be solved with a variety of available

computer software. We use MATLAB R2010b and its optimization tool withthe fmincon solver and the interior point algorithm. The interior point algorithmattempts to solve the constrained optimization problem by first replacing theinequality constraints with equality constraints by introducing slack variables, s, andintroducing a new term to the cost function. For example, the inequality constraintin Eq. 8.5 becomes gj C sj D 0. Then for each > 0, Eq. 8.3 is approximated by

miny;s

f.y; s/ D miny;s

f .y/ � X

q

ln.sq/ (8.6)

and this is subject to only equality constraints, which is an easier class of problemto solve. Note that as approaches 0, the minimum in Eq. 8.6 should equalthe minimum in Eq. 8.3. Solving the approximated problem is done by taking asequence of steps using one of two methods. The first, and default, method linearizesthe Lagrangian of the approximated problem and attempts to find a solution thatsatisfies the Karush-Kuhn-Tucker (KKT) conditions (Byrd et al. 2000). If this


fails, then a conjugate gradient method is used, which attempts to minimize aquadratic approximation to the problem within a trust region (a neighborhood ofradius R defined by the user and shrunk if no good solution can be found) (Byrdet al. 1999; Waltz et al. 2006). This minimization is done subject to linearizedconstraints. These two methods are repeated until a solution satisfying the stoppingcriterion is determined. More specifics can be found in the MATLAB documentation(Constrained Nonlinear Optimization Algorithms 2014).

Verification of the local optimization can be done by checking sufficient con-ditions for local minima. The Lagrangian function associated with our problem(assuming no further constraints) is given by L .y; / D f .y/ �P

i ihi � Pj jgj

and the second order sufficient condition for a local minimum is that there existLangragian multipliers �

i such that

DyL .y�; �/ D 0 where �i � 0 8i; �

j gj.y�/ D 0 (8.7)

and

zTr2L .y�; �/z > 0 8z 2 T 0; z ¤ 0

where T 0 WD fv W rhi.y�/v D 08igThe Lagrangian multipliers can be calculated numerically using MATLAB or

several other software programs.When using software to numerically solve optimization problems, it is important

to recognize that most solvers determine only local solutions, not global ones. Totest if our local minima are in fact global minima, MATLAB’s GlobalSearch solverwas used. Simply put, the GlobalSearch algorithm generates a number of test pointsto use as initial starting points for the fmincon solver. The algorithm assumes thatany local minima found by the fmincon solver have spherical basins of attraction,with radius equal to the Euclidean distance from the local minimum to its associatedstarting point. As the algorithm steps through the list of test points, it discards anythat are found to be in existing basins. At the end, it reports the local minimumwith the smallest cost function among the solutions calculated from the test points.More details can be found in the MATLAB documentation (How GlobalSearch andMultiStart Work 2014).

8.3 Application to Analysis of Brain Metabolites

In this section we apply our methodology to the analysis of brain metabolite data.Numerous studies have measured the concentrations of major brain metabolitesusing 1H MRS, to evaluate brain biochemical changes and maturation in adults andneonates (Kreis et al. 1993). However, the interpretation in terms of the health orspecific status of the brain cells is unclear since the data are obtained from a regionwith a heterogeneous mixture of cells.


Fig. 8.1 Age division for each subject visit. Box and whisker plots represent the minimum, secondquartile, median, third quartile, and maximum value of all subject data

8.3.1 Human Subject Studies

Nine healthy newborns were studied, two boys and seven girls. Pregnant motherswere recruited from the maternity ward at the Queen’s Medical Center in Honoluluand through physician referrals. Each parent or legal guardian signed an informedconsent form approved by our Institutional Review Board, and completed detailedinterviews regarding their medical and drug use histories. Mothers were 18 years orolder at the time of giving birth, and had minimal or no drug use or any other prenatalcomplications during pregnancy or perinatal problems during delivery. All babieswere at or near full term for gestational age (36 weeks or more), and were evaluatedthoroughly to ensure they were healthy. Each neonate was scanned three times:within 1 week of birth, and at approximately 1 and 2 months thereafter. Althoughmany more neonates were studied, only nine infants with complete datasets for allfive metabolites and good quality data for both brain regions studied are presented.The age distribution of these nine infants at each visit is shown in Fig. 8.1.

8.3.2 MRI and Localized 1H MRS

MRI studies were performed on a Siemens Trio 3.0 T scanner while the infantsslept (typically after nursing) and were unsedated. All babies had a sagittal3D-magnetization prepared rapid acquisition by gradient echo (MP-RAGE)sequence. Also, a T2-weighted 3D-SPACE sequence was acquired to ensure nolesions were present. Based on anatomical landmarks in the MP-RAGE scan,spectra were acquired in the right basal ganglia (BGR, 6.0 cm3) and frontalwhite matter, right side (FWR, 5.0 cm3); see Fig. 8.2. A short echo-time PointRESolved Spectroscopy (PRESS) acquisition sequence (relaxation time/echotime = 3,000/30 ms, 2.5 min acquisition) was used (Chang et al. 1996), andmetabolite concentrations for five major metabolites were determined as described


Fig. 8.2 Neonate spectroscopic voxel locations shown on the MPRAGE images in all threeorientations (top left: coronal; bottom left: sagittal; top and bottom right: axial)

previously (Kreis et al. 1993, 2002). For each subject and each time point, acomplete MRS data set was available, which included the concentrations of:

• N-acetyl compounds (NAA): This metabolite is exclusively found in the nervoussystem (peripheral and central) and is detected in both grey and white matter. Itis thought to be a marker of neuronal and axonal viability and density. Decreasedconcentration of NAA is a sign of neuronal loss or degradation (Cheong et al.2006; Urenjak et al. 1992, 1993).

• Total creatine (tCR): The role of tCR is to supply energy to all cells in the body,including brain cells (Cheong et al. 2006).

• Choline-containing compounds (CHO): A marker of nerve signaling, myelin, andcellular membrane turnover which can also reflect cellular proliferation. ReducedCHO may be related with delayed myelination or apoptosis (Cheong et al. 2006).


• Myoinositol (MI): This sugar moiety is considered a glial marker because it isprimarily synthesized in glial cells, both in microglia and in astrocyglia cells.Elevated MI occurs with proliferation of glial cells or with increased glial cellsize, as found in inflammation, and may reflect glial activation accompanyingneuronal dysfunction or loss. MI is also thought to play an important role withits high concentration in normal fetal brain development (Blüml et al. 2013).

• Glutamate+glutamine (GLX): The GLX signal represents a combination ofglutamate (GLU) and glutamine (GLN), but is dominated by GLU, which is animportant excitatory neurotransmitter found throughout the brain (Mangia et al.2012).

8.4 MRS Findings

Table 8.1 and Fig. 8.3 show the mean values and standard deviation for themetabolite concentrations in the basal ganglia, right side (BGR) and data for thefrontal white matter, right side (FWR) is also provided in Table 8.1. We note thatwith the exception of [MI] in visits II and III, the metabolite concentrations arehigher in the BGR compared to the FWR. We can compare the measurements fromsubjects in Visit I to those obtained in Kreis et al. (2002) for full term neonates(38 weeks < GA < 43 weeks). In Kreis et al. (2002), all metabolite concentrationsin the ROI placed in the centrum semiovale for developing white matter (based on 11subjects) were higher compared to our values in the frontal white matter (based onnine subjects). Indeed, in their paper they obtained [NAA]:3:5˙0:5, [tCR]:4:8˙0:6,[CHO]: 2:3˙ 0:1, [mI]:5:9˙ 0:7 and [GLX]:6:3˙ 1:1.

Table 8.1 Mean and standard deviation of metabolic concentrations in the two regions of interestby visit

Visit I

Region [NAA] [tCR] [CHO] [MI] [GLX]

BGR 4:23˙ 0:36 5:17˙ 0:58 2:19˙ 0:13 4:80˙ 0:39 7:80˙ 1:10

FWR 2:85˙ 0:33 3:10˙ 0:24 1:83˙ 0:21 4:54˙ 0:49 5:90˙ 0:85

Visit II


BGR 4:85˙ 0:19 5:59˙ 0:28 1:90˙ 0:18 4:00˙ 0:64 7:96˙ 1:15

FWR 3:71˙ 0:52 3:40˙ 0:52 1:62˙ 0:21 4:06˙ 0:71 5:95˙ 0:78

Visit III


BGR 5:06˙ 0:51 5:62˙ 0:55 1:73˙ 0:22 3:13˙ 0:60 8:01˙ 1:03

FWR 4:70˙ 0:49 3:76˙ 0:58 1:53˙ 0:17 3:28˙ 0:70 7:00˙ 0:77


Fig. 8.3 Metabolite Concentrations in the BGR Region. Nine subjects were each sceanned threetimes. Visit I was scanned between 39.6 and 41.1 weeks (postmenstrual age). Visit II was scannedbetween 43.9 and 46.7 weeks. Visit III was scanned between 48.0 and 52.7 weeks

8.4.1 Regional Variations and Age Dependence

Graphs for each of the five metabolite concentrations over time for the two ROIs aredisplayed in Fig. 8.4. The average rate of growth for each metabolite on the graphis shown in Table 8.2. There is a greater increase over time in [NAA], [tCR], and[GLX] levels in the FWR compared to the BGR, and a greater decrease in [CHO]and [MI] levels in the BGR compared to the FWR. We note the smallest difference ingrowth between the two regions is for [CHO], and the greatest difference in growthrate is for [GLX].

A two-factor (age and region) repeated measures ANOVA analysis was con-ducted on the metabolite data (Table 8.3). All metabolites except CHO had asignificant regional dependence, and all metabolites except GLX had significant agedependence. Only NAA showed a significant interaction effect between region andage.


Fig. 8.4 Graphs for each of the five metabolite concentrations over time. Each of the five verticalplanes displays graphs of a specific metabolite concentration vs time for the two ROI. Since atraditional line of best fit does not take into account the dynamics of the data within a subjectover time, we use a new fitting procedure. We calculate the slope between sequential data points(subjects with three measurements in the same ROI have two pairs of sequential points). We setan initial point using the average age and average metabolite concentration of Visit I scans. Tocreate the fit, we use a variation of Euler’s method: taking a small step along the x-axis (age), andusing the average of the calculated slopes at that age to find the change in the y-value (metaboliteconcentration), then repeating the process to create a piecewise linear function that fits the data

Table 8.2 Metabolite concentration growth rates. Average rate of change for metabolite concen-tration vs age in each region [mM/week]

NAA tCR CHO MI GLX

BGR 0:0701 0:0378 �0:0403 �0:1731 0:0116

FWR 0:1862 0:0715 �0:0263 �0:1389 0:1486

Table 8.3 Repeated measures 2-Factor ANOVA, p values

Metabolite [NAA] [tCR] [CHO] [MI] [GLX]

Region <0.001 <0.001 <0.001 0.9098 <0.001

Age <0.001 0.001 <0.001 <0.001 0.0538

Region � age 0.0042 0.5950 0.4671 0.5711 0.1393

8.5 Cell Types and Distribution in Each Brain Region

We apply our methodology to a simple model with two populations, differentiatedbased on some of their metabolite characteristics, and will calculate their relativecellular density in the BGR and FWR as a function of age. The term “relative cellulardensity” is rather vague at this stage, discussion on a possible interpretation for ourmodel is included in the results section.


Since NAA and MI are theorized to be markers for neurons and glia (Brandet al. 1993 and Guimaraes et al. 1995), this supports an approach with these twopopulations of cells. We will call them type I and type II, respectively. Cells of typeI are characterized by a negligible concentration of NAA, a common assumption forglia, and cells of type II are characterized by negligible MI concentration, a commonassumption for neurons.

We have a total of m D 27measurements for the BGR and m D 27measurementsfor the FWR. In our model, each measurement is treated independently even thoughit might represent a repeat measurement in a given subject. Based on our analysisof metabolite concentrations in Sect. 8.4 and Table 8.3, we will consider region-specific composition matrices. Thus, each region will have its own compositionmatrix A. The methodology for setting up the model for each region is essentiallythe same, however.

The matrix C is a 5 � m matrix as follows. Each column of C correspondsto one measurement, and therefore contains five values: one for each metaboliteconcentration. We order the concentrations as follows: [NAA], [tCR], [CHO], [MI],[GLX]; therefore, �11 reflects the NAA concentration per unit of density in type Icells. Because of the exclusivity of the NAA and MI markers, we set �11 D 0 and�42 D 0, and can therefore remove them as variables.

Since n D 2, the composition matrix A is 2 � 5. Since negative values wouldbe unrealistic, we add the non-negative constraints that � j

i � 0 and xki � 0 for

each i; j; k. We can compute explicitly the elements of the residual matrix E D.ei;j.�//1�i�5;1�j�m defined by Eq. (8.2)

ei;j.�/ D5X

kD1

cj;k

det.ATA/

"� k1

� i1 � � i

2

5XlD1

� l1�

l2

!C � k

2

�� i

1

5XlD1

� l1�

l2 C � i

2

!#�ci;j;

(8.8)As in Eq. (8.3), our optimization problem becomes:

miny

f .y/; f .y/ DX

i;j

.ei;j%.y//

2 (8.9)

subject to the constraints:

h1.y/ D4X

iD1.yi/

2 � 1 D 0 (8.10)

h2.y/ D8X

iD5.yi/

2 � 1 D 0 (8.11)

gi.y/ D yi � 0; i D 1; 8 (8.12)


G1;j.y/ D 1

det .ATA/

5XkD1

ck;j

a1k � a2k

5XmD1

a1ma2m

!� 0 (8.13)

G2;j.y/ D 1

det .ATA/

5XkD1

ck;j

a2k � a1k

5XmD1

a1ma2m

!� 0 (8.14)

where Eqs. (8.10)–(8.11) are the L2-normalization for each column of A, Eq. (8.12)is the non-negativity constraint on elements of A, and Eqs. (8.13)–(8.14) are thenon-negativity constraints on cell density.

We solve the above optimization problem numerically using MATLAB 2010b.The results of our numerical calculations can be found in Table 8.4.

The Lagrangian multipliers were calculated numerically for all metabolitespectra matrices in Table 8.4 as well as the Hessian of the Lagrangian using Matlab’sfmincon solver. The Hessians were found to be strictly positive definite, thereforesatisfying the sufficient condition for local minima. We then checked for globaloptima with MATLAB’s GlobalSearch. Across all regions and all starting pointschosen, the GlobalSearch solver reported the same minima and these minima wereconsistent with the local minima found by the fmincon solver.

Inspection of optimized regional A values in Table 8.4 (see also Fig. 8.5) showsthat the metabolite concentration per unit of cellular density is similar for [CHO] inboth type I and type II cells for both regions examined. [tCR] is higher in the FWR

Table 8.4 Optimized valuesfor the region-specificcomposition matrices A

Ex: Aregion

Type I Type II ABGR AFWR

�NAA1 �NAA

2 0 0:46589 0 0:61583

� tCR1 � tCR

2 0:12945 0:48874 0:25013 0:3628

�CHO1 �CHO

2 0:23189 0:0925 0:22591 0:08927

�MI1 �MI

2 0:95397 0 0:83338 0

�GLX1 �GLX

2 0:13936 0:7318 0:43803 0:69366

Fig. 8.5 Metabolite concentration per unit cell density for each region. The left graph shows theconcentrations per unit type I density, and the right graph shows the concentrations per unit typeII density. Note the NAA concentrations for type I cells and the MI concentrations for type II arezero across both regions due to constraints


than the BGR in type I cells, but in type II cells [tCR] levels are higher in BGR thanFWR. The neuronal marker [NAA] has a higher value per unit of cellular density inthe frontal region compared to the deep grey matter, while the glial marker [MI] ishigher in the BGR compared to the FWR, suggesting a greater dependence on eachmarker for their respective cell type. [GLX] has a substantially higher concentrationper unit density in the FWR compared to BGR for type I cells, whereas in type IIcells BGR has a slightly greater [GLX] value than FWR.

8.5.1 Cellular Density

The matrix of relative cellular density X can now be computed from the equalityX D A�C where A� represents the Moore-Penrose pseudo-inverse regional matrices(see Table 8.5). From this Table it is clear that for the cells of type I, almost all theinformation is embedded in the MI concentration. For the cells of type II, however,the interplay between the various metabolites is more pronounced but dominated bythe concentrations of NAA and GLX.

The results show a general decrease in density of type I cells over time, and anincrease in density of type II cells. The frontal region shows a greater increase indensity for type II cells than the basal ganglia, see Fig. 8.6 and Table 8.6. Repeated

Table 8.5 Pseudo-inverse regional matrices, where A� represents the Moore-Penrose pseudo-inverse matrix of matrix A

A�BGR A

�FWR

-0.09012 0.03959 0.22237 0.98842 0.00283 -0.30849 0.12036 0.22813 1.00653 0.18156

0.48271 0.48135 0.05099 -0.18454 0.73127 0.74378 0.31288 -0.0053 -0.41747 0.61835

Fig. 8.6 Graphs of density levels for type I and type II vs age

Table 8.6 Average rate ofgrowth for cell types I and II

BGR FWR

Type I �0.195 �0.172

Type II 0.084 0.283


Table 8.7 p values of repeated measure 2-Factor ANOVA of density levels

Type I Type II

Region 0:00240 2:678 � 10�7

Age 1:045 � 10�6 8:683 � 10�6

Region x age 0.8840 0.0134

Table 8.8 Mean and standard deviation of control residual values, in percent

Residual values region [NAA] [tCR] [CHO] [MI] [GLX]

BGR �0:11˙ 9:6 0:37˙ 7:8 �1:1˙ 11 0:19˙ 1:6 1:0˙ 7:0

FWR 0:033˙ 7:7 �0:096 ˙ 7:7 �1:5˙ 13 0:37˙ 3:2 0:75˙ 5:4

measures 2-factor ANOVA on the density levels shows both populations have asignificant dependence on region and age, but only type II cells have a significantinteraction effect on region and age, see Table 8.7.

8.5.2 Residuals

Finally, to validate our model with the specific A and X matrices obtained, wecan recover C� D AX and compare it with the measured data C. Recall that ouroptimization process seeks to minimize the sum of the squares of the percentagedifferences between our calculated C� values and the metabolite data from the MRSexperiments C. Table 8.8 shows the mean and standard deviation of the residuals.The measured data C are subject to error due to noise in the MRS experiments, sowe expect some error in our results. The standard deviations of residual values areapproximately 10 % or less, which is consistent with typical errors associated within vivo MRS metabolite levels.

When the methodology is applied to all of the data together, rather than separatelyfor each region, the residuals are larger. Thus, the region-specific A matrices give amore accurate representation.

8.5.3 Interpretation of Results

Our study demonstrates that alterations in brain metabolite profiles on 1H MRSduring early brain maturation can be represented as a multivariate mixing problemwith only two sources (cell types). Type I cells are designed to have no NAA,and have strong contributions of MI. Therefore, type I cells most likely representthe glial population. Interestingly, GLX had a substantial contribution to typeI (glial) compartment in the frontal white matter, but not in the basal ganglia.


Conversely, type II cells are designed to have no MI, and were found to have similarcontributions of NAA, tCR, and GLX. Therefore, type II cells most likely representthe neuronal population.

Importantly, over the age range evaluated, a simple 2-source model is able toexplain most of the variance in the metabolite data, with residuals that approachtypical errors associated with in vivo MRS measurements (approximately 10 %).Furthermore, a 3-source model did not result in substantial improvements in fittingaccuracy, suggesting that a glial and neuronal compartment are sufficient forrepresenting the measured 1H MRS data within experimental errors.

Assuming the type I compartment represents glial cells and the type II com-partment represents the neuronal compartment, our findings are in agreement withthose of prior studies. Specifically, both regions examined showed a pronouncedincrease in the neuronal compartment with age, probably representing neuronalmaturation during the first few months of life. Of note, the neuronal compartmentof the frontal white matter increased at over three times the rate compared to thatof the basal ganglia. This suggests that the basal ganglia are more mature at birthcompared to white matter, in agreement with the literature (Kostovic and Jovanov-Miloševic 2006). In parallel, there is a substantial decrease in the glial (type I)compartment over the first few months of life, perhaps due to replacement of theglial cell compartment with maturing neurons or osmotic effects.

8.5.4 Comparison to PCA Analysis

We used MATLAB 2010b’s princomp tool to conduct a principal componentanalysis (PCA) on the MRS data to compare with our results. The results are shownin Table 8.9 and Fig. 8.7. The percentage of variance for the basal ganglia regionexplained by each principal component, in decreasing order, is 47.4 %, 36.0 %,10.6 %, 4.4 %, and 1.6 %. For the FWR region, it is 68.1 %, 20.5 %, 8.1 %, 2.6 %,and 0.7 %. Approximately 80–90 % of the variance can be explained by the first twocomponents in both regions. Of note, most of the variance occurs in NAA, GLX,and MI.

There are some major differences between our model and PCA. First, we notethat the principal components cannot be used to represent the composition of our

Table 8.9 Coefficients of first and second principal components in BGR and FWR

BGR FWRMetabolite Comp. 1 Comp. 2 Metabolite Comp. 1 Comp. 2

NAA 0.18166 �0.33287 NAA 0.62662 �0.03408

Cr 0.2051 �0.16135 Cr 0.30277 0.28121

CHO �0.01854 0.15223 CHO �0.02391 0.14161

MI �0.27759 0.84329 MI �0.37771 0.84766

GLX 0.92062 0.35897 GLX 0.61028 0.42566


1MI MI

CHO CHO

Cr

Cr

NAA

NAA

GLX GLX

1

0.5

0.5

0

0Component 1

Com

pone

nt 2

Com

pone

nt 2

BGR FWR

-0.5

-0.5-1

1

0.5

0

-0.5

-1-1 10.50Component 1

-0.5-1

Fig. 8.7 Reduction of system to two PCA Components for BGR and FWR. Contribution of fivemeasured metabolites to each component shown as vectors

Table 8.10 Calculated values for A without non-negativity constraints and zero constraints

BGR FWRMetabolite Type I Type II Metabolite Type I Type II

NAA 0:02263 0:46374 NAA �0:16187 0:61143

Cr 0:15187 0:48950 Cr 0:17478 0:36445

CHO 0:23401 0:09748 CHO 0:22052 0:09245

MI 0:94423 �0:02221 MI 0:90005 0:01410

GLX 0:17348 0:73167 GLX 0:29074 0:69612

sources (matrix A in the model), since negative composition values are unrealistic.For instance, the potential glial principal component of the BGR region, having ahigh loading of MI (Table 8.9), would also have a substantial negative contribution.�0:33/ of NAA, which is physically impossible in a mixing situation. Conversely,the calculated A matrices from our methodology produce results that are non-negative, and the residual values using the model results are smaller than the PCAresults.

The principal components in PCA are calculated to optimally explain variance inthe data. Our calculated A matrices are chosen to minimize residual values, but stillexplain most of the variance in the data. In the BGR, type I and type II compositions(the first and second columns of A) explain 30:1% and 37:6% of the variance,respectively. In the FWR, type I and type II compositions explain 20:4% and 60:4%of the variance, respectively.

If the non-negative constraints and zero constraints on NAA and MI are removedfrom our analysis, we obtain the solutions in Table 8.10. Comparing the first andsecond PCA components in Table 8.9 to the Type II and Type I results, respectively,for the model with reduced constraints, the FWR results are similar, while the BGRresults are less comparable.


8.6 Conclusion

In this paper, we have applied a new methodology to solve an inverse mixingproblem when source composition is largely unknown or uncertain. The modelallows us to calculate an optimal solution in terms of residual values, whileincorporating known information or constraints on source composition.

In comparison to other techniques that are used to solve inverse mixing problems,the model presented has advantages in specific situations. One major strength of thismodel is that the composition of the sources does not need to be known. This isuseful in situations where such measurements are infeasible or when experimentalmeasurements are found to be insufficient or suspect. The model can thereforebe used to create an “ideal” composition matrix. This “ideal” matrix has thefurther benefit of being reported in terms of the components of interest, unlikePCA approaches which tend to determine composition in terms of the new basisdetermined by the PCA analysis. This makes understanding the composition matrixsimpler, as the contribution of each source to the final product is known. Thus this“ideal” matrix is easy to compare with known data, if any exist. The model alsohas the advantage of allowing both equality and inequality constraints. Even if onedoes not know the exact composition of the sources, the constraints allow one toincorporate what information is known about the situation. They can be used toensure certain proportions of components, or the absence of certain components insome sources, or upper or lower bounds of components. Finally, the optimizationapproach used gives a best fit to data, even when a significant amount of randomerror is present due to noise.

In ongoing work, we are expanding our analysis to MRS data gathered inadditional brain regions and in a much larger sample size of normally developingneonates. We are also applying this novel method to study neonates whose motherssmoked tobacco cigarettes or used illicit drugs during their pregnancies in order tocompare the brain development between drug-exposed and non-exposed neonates.This new analysis method can be applied to all MRS studies, especially thoseinvolving the same brain metabolites evaluated here.

Acknowledgements We are grateful to the research participants in this study. We also thank allof our clinical and technical research staff who helped with the data collection (S. Buchthal, A.Hernandez, E. Cunningham, H. Johansen, J. Skranes, R. Yamakawa).

Funding: This work was supported by the National Institute on Drug Abuse (K24-DA016170;K02-DA016991; 1R01 DA021146), the National Institute on Minority Health and Health Dis-parities (8G12-MD007601-27), the National Institute of Neurological Diseases and Stroke (U54-NS056883) and the Office of National Drug Control Policy. M. Chyba and A. Tamura-Sato werepartially supported by the National Science Foundation (NSF) Division of Graduate Education,award #0841223, and the NSF Division of Mathematical Sciences, award #1109937.


References

Ancora G, Soffritti S, Lodi R, Tonon C, Grandi S, Locatelli C, Nardi L, Bisacchi N, Testa C, TaniG, Ambrosetto P, Faldella G (2010) A combined a-EEG and MR spectroscopy study in termnewborns with hypoxic-ischemic encephalopathy. Brain Dev 32:835–842

Befroy DE, Shulman GI (2011) Magnetic resonance spectroscopy studies of human metabolism.Diabetes 60(5):1361–1369

Blüml S, Wisnowski JL, Nelson Jr. MD, Paquette L, Gilles FH, Kinney HC, Panigrahy A (2013)Metabolic Maturation of the Human Brain from Birth Through Adolescece: Insights from InVivo Magnetic Resonance Spectroscopy. Cerebral Cortex 23(12):2944–2955

Brand A, Richter-Landsberg C, Leibfritz D (1993) Multinuclear NMR studies on the energymetabolism of glial and neuronal cells. Dev Neurosci 15:289–298

Byrd RH, Hribar M, Nocedal J (1999) An interior point algorithm for large-scale nonlinearprogramming. SIAM J Optim 9(4):877–900

Byrd RH, Gilbert JC, Nocedal J (2000) A trust region method based on interior point techniquesfor nonlinear programming. Math Program 89(1):149–185

Cantrell, CA (2008) Technical note: review of methods for linear least-squares fitting of data andapplication to atmospheric chemistry problems. Atmos Chem Phys 8:5477–5487

Chang L, Ernst T, Poland RE, Jenden DJ (1996) In vivo proton magnetic resonance spectroscopyof the normal aging human brain. Life Sci 58(22):2049–2056

Cheong JLY, Cady EB, Penrice J, Wyatt JS, Cox IJ, Robertson NJ (2006) Proton MR spectroscopyin neonates with perinatal cerebral hypoxic-ischemic injury: metabolite peak-area ratios,relaxation times, and absolute concentrations. Am J Neuroradiol 27:1546–1554

Christopherson N, Hooper RP (1992) Multivariate analysis of stream water chemical data: theuse of principal components analysis for the end-member mixing problem. Water Resour Res28(1):99–107

Christopherson N, Neal C, Hooper RP, Vogt RD, Andersen S (1990) Modelling streamwaterchemistry as a mixture of soilwater end-members: a step toward second-generation acidificationmodels. J Hydrol 116:307–320

Constrained Nonlinear Optimization Algorithms. The MathWorks Inc. Web. Accessed 6Aug. 2014. http://www.mathworks.com/help/optim/ug/constrained-nonlinear-optimization-algorithms.html

Guimaraes AR, Schwartz P, Prakash MR, Carr CA, Berger UV, Jenkins BG, Coyle JT, GonzálezRG (1995) Quantitative in vivo 1H nuclear magnetic resonance spectroscopic imaging ofneuronal loss in Rat brain. Neuroscience 69(4):1095–1101

Haseler LJ, Arcinue E, Danielsen ER, Bluml S, Ross BD (1997) Evidence from proton magneticresonance spectroscopy for a metabolic cascade of neuronal damage in shaken baby syndrome.Pediatrics 99(1):4–14

How GlobalSearch and MultiStart Work. The MathWorks Inc. Web. Accessed 6 Aug. 2014. http://www.mathworks.com/help/gads/how-globalsearch-and-multistart-work.html#bsc9eec

Hunter MA, Takane Y (2002) Constrained principal component analysis: various applications. JEduc Behav Stat 27(2):105–145

Kirov I, Flaysher L, Fleysher R, Patil V, Liu S, Gonen O (2008) The age dependence of regionalproton metabolites T2 relaxation times in the human brain at 3 Tesla. Magn Reson Med60(4):790–795

Kostovic I, Jovanov-Miloševic N (2006) The development of cerebral connections during the first20–45 weeks’ gestation. Assess Brain Funct Perinat Period 11(6):415–422

Kreis R, Ernst T, Ross BD (1993) Development of the human brain: in vivo quantification ofmetabolite and water content with proton magnetic resonance spectroscopy. Magn Reson Med30(4):424–437

Kreis R, Hofmann L, Kuhlmann B, Boesch C, Bossi E, Hüppi PS (2002) Brain metabolitecomposition during early human brain development as measured by quantitative in vivo 1Hmagnetic resonance spectroscopy. Magn Reson Med 48(6):949–958


Manganas L, Zhang X, Li Y, Hazel RD, Smith SD, Wagshul ME, Henn F, Benveniste H, Djuric PM,Enikolopov G, Maletic-Svatic M (2007) Magnetic resonance spectroscopy identifies neuralprogenitor cells in the live human brain. Science 318(5852):980–985

Mangia S, Giove F, DiNuzzo M (2012) Metabolic pathways and activity-dependent modulation ofglutamate concentration in the human brain. Neurochem Res 37(11):2544–2561

Pouwels P, Brockmann K, Kruse B, Wilken B, Wick M, Hanefeld F, Frahm J (1999) Regional agedependence of human brain metabolites from infancy to adulthood as detected by quantitativelocalized proton MRS. Pediatr Res 46:474–474

Snieder R, Trampert J (1999) Inverse problems in geophysics. In: Wirgin A (ed) Wavefield inver-sion. International centre for mechanical sciences, vol 398. Springer, New York, pp 119–190

Takane Y, Hunter MA (2001) Constrained principal component analysis: a comprehensive theory.Appl Algebra Eng Commun Comput 12(5):391–419

Urenjak J, Williams SR, Gadian DG, Noble M (1992) Specific expression of N-acetylaspartatein neurons, oligodendrocyte-type-2 astrocyte progenitors, and immature oligodendrocytes invitro. J Neurochem 59(1):55–61

Urenjak J, Williams SR, Gadian DG, Noble M (1993) Proton nuclear magnetic resonancespectroscopy unambiguously identifies different neural cell types. J Neurosci 13(3):981–989

van Doormaal P, Meiners L, ter Horst H, van der Veere C, Sijens P (2012) The prognostic valueof multivoxel magnetic resonance spectroscopy determined metabolite levels in white and greymatter brain tissue for adverse outcome in term newborns following perinatal asphyxia. Eur JRadiol 22(4):772–778

Waltz RA, Morales JL, Nocedal J, Orban D (2006) An interior algorithm for nonlinear optimizationthat combines line search and trust region steps. Math Program 107(3):391–408

Wang ZJ, Vigneron DB, Miller SP, Mukherjee P, Charlton NN, Lu Y, Barkovich AJ (2008) Brainmetabolite levels assessed by lactate-edited MR spectroscopy in premature neonateswith andwithout pentobarbital sedation. Am J Neuroradiology 29:798–801

Zou H, Hastie T, Tibshirani R (2006) Sparse principal component analysis. J Comput Graph Stat15(2):265–286

Chapter 9Exact Regularization, and Its Connections toNormal Cone Identity and Weak Sharp Minimain Nonlinear Programming

S. Deng

Abstract The regularization of a nonlinear program is exact if all solutions of theregularized problem are also solutions of the original problem for all values ofthe regularization parameter below some positive threshold. In Deng (Pac J Optim8(1):27–32, 2012), we show that, for a given nonlinear program, the regularizationis exact if and only if the Lagrangian function of a certain selection problemhas a saddle point, and the regularization parameter threshold is inversely relatedto the Lagrange multiplier associated with the saddle point. The results in Deng(Pac J Optim 8(1):27–32, 2012) not only provide a fresh perspective on exactregularization but also extend the main results in Friedlander and Tseng (SIAMJ Optim 18:1326–1350, 2007) on a characterization of exact regularization of aconvex program to that of a nonlinear (not necessarily convex) program. In thispaper, we will examine inner-connections among exact regularization, normal coneidentity, and the existence of a weak sharp minimum for certain associated nonlinearprograms. Along the way, we illustrate by examples, how to obtain both new resultsand reproduce many existing results from a fresh perspective.

Keywords Saddle point • Lagrangian function • Exact regularization • normalcone identity • and weak sharp minima

9.1 Introduction

To understand basic ideas of exact regularization, let us begin by considering thefollowing examples.

Example 9.1. Given an underdetermined system of linear equations

Ax D b;

S. Deng (�)Department of Mathematical Sciences, Northern Illinois University, DeKalb, IL, USAe-mail: [email protected].


175

176 S. Deng

where A is an m � n matrix, x 2 Rn, and b 2 Rm with m < n, we assume thatthe system is consistent. In signal processing, the following linear programmingproblem

min jjxjj1 subject to Ax D b (9.1)

has been widely studied.

Why? The problem (9.1) is closely connected to the problem of finding sparsesignal representation,

min jjxjj0 subject to Ax D b;

where jjxjj0 denotes the number of nonzero components in x.As a linear programming problem, (9.1) has multiple solutions in general. It

is desirable to select a solution with additional properties, e.g. a least two-normsolution.

Example 9.2. The forward model in many data acquisition scenarios can beformulated as follows

y D Ax0 C w;

where y 2 Rm are the observations, x0 2 Rn the unknown signal to recover, wthe noise, and A is a linear operator which maps the signal domain Rn into theobservation domain Rm with m � n.

Even when m D n, A is in general ill-conditioned or singular. This makes thelinear inverse problem of finding a good approximation of x0 is ill-posed. A usefulway to deal with this challenge is to solve the following optimization problemparametrized by ı:

minx2Rn 1=2jjy � Axjj22 C ıR.x/; (9.2)

where R is an appropriate regularization term through which some regularity isenforced on the recovered signal, and ı > 0.

For noiseless observations, i.e. w = 0, by letting ı # 0, we see that (9.2) is closelyrelated to the following constrained minimization problem

min R.x/ subject to Ax D y: (9.3)

The role of problem (9.3) is to analyze problem (9.2) through certain regulariza-tion process (see Definition 9.1 for more details).

The technique of regularization is a common approach used to solve an ill-posed nonlinear optimization problem with non-unique solutions by constructinga related problem whose solution is well behaved and deviated only slightly from a

9 Exact Regularization, and Its Connections to Normal Cone Identity and. . . 177

solution of the original problem. Deviations from solutions of the original problemare generally accepted as a trade-off for obtaining solutions with other desirableproperties. However, it would be more desirable if solutions of the regularizedproblem were also solutions of the original problem. In a recent paper by Friedlanderand Tseng (2007), the authors presented a systematic study for exact regularizationof a convex program. The term exact regularization was coined in Friedlander andTseng (2007); according to Friedlander and Tseng (2007) the regularization is exactif the solutions of perturbed problems are also solutions of the original problemfor all values of penalty parameters below some positive threshold value. In Deng(2012), we demonstrate that the main results of Friedlander and Tseng (2007) canbe extended to non-convex programs thereby the application domain of this exactregularization technique has been significantly expanded. In this paper, for convexprograms, we examine inner-connections among exact regularization, normal coneidentity, and the existence of a weak sharp minimum for certain associated nonlinearprograms. Specifically, we show that strongly exact regularization is equivalent tonormal cone identity, and weak sharp minima implies normal cone identity. Alongthe way, we illustrate by examples, how to obtain both new results and reproducemany existing results from a fresh perspective.

The notation used in this note is standard. See e.g. Rockafellar and Wets (1998).

9.2 Review of Main Results in Deng (2012)

Examples in the Introduction section suggest to us to consider the following generalnonlinear program

.P/ min g.x/ s:t: x 2 C;

where g W Rn ! R is a continuous function, and C is a closed set in Rn. Let Sbe the set of all optimal solutions and suppose that the solution set S of (P) isnonempty, and denote its optimal value by p�. When (P) has multiple solutionsor is very sensitive to data perturbations, a popular way to regularize the problemis to modify the objective function by adding a new function f . This leads to thefollowing regularized problem

.P.ı// min g.x/Cıf .x/ s:t: x 2 C;

where f W Rn ! R is a continuous function and ı is a nonnegative regularizationparameter. Let Sı be the set of optimal solutions. For Example 9.1, we have g.x/ D0, f .x/ D jjxjj1, and ı D 1, and for Example 9.2, we have g.x/ D 1=2jjy � Axjj22,and f .x/ D R.x/. The regularization function f may be nonlinear, non-convex ornon-differentiable. A popular choice, commonly known as Tikhonov regularization,of f is jjxjj22, which can be used to select a least two-norm solution. Another popular

178 S. Deng

choice is l1 regularization with f .x/ D jjxjj1. More examples of f , applications ofexact regularization of convex programs can be found in Friedlander and Tseng(2007).

Now we can introduce a key definition of the paper.

Definition 9.1. For (P) with a given function f , we say that regularization is exactif the solutions of (P.ı/) are also solutions of (P) for all values of ı below somepositive threshold value Nı; that is, Sı � S for all ı � Nı.

As in Deng (2012) and Friedlander and Tseng (2007), a key to analysis is arelated nonlinear program that selects solutions of (P) of the least f �value:

.Q/ min f .x/ s:t x 2 C; g.x/ � p�;

where p� denotes the optimal value of (P). Let SQ be the set of optimal solutions of(Q), and suppose that SQ 6D ;. Let the Lagrangian function of (Q) be

L.x; y/ D f .x/C y.g.x/� p�/

for x 2 C and y � 0. We say that a pair of vector .Nx; Ny/ 2 C � RC gives a saddlepoint of the Lagrangian L on C � RC if

L.Nx; y/ � L.Nx; Ny/ � L.x; Ny/ 8x 2 C � Rn;8y 2 RC:

Problem (Q) may not have a Lagrange multiplier even for the convex case asillustrated by the following example.

Example 9.3. Let g.x/ D x2, C D R, and f .x/ D x. Then S D argminx2Cg D f0g.For L.x; y/ D x C yx2, there are no saddle points for L over R � RC. In fact, fory � 0,

infx2C

L.x; y/ D(

�1 if y D 0;

� 14y if y > 0:

A saddlepoint condition characterization for (Q) is given in Deng (2012).Note that this characterization is not true for standard nonlinear programs. Acharacterization for standard nonlinear programs can be found in Rockafellar (1993)

Theorem 9.1 (Theorem 3 of Deng (2012)). For problem (Q), a pair .Nx; Ny/ 2C � RC is a saddle point of the Lagrangian L if and only if the pair satisfies theconditions:

(1) Nx 2 S;(2) Nx is a minimizer of L.; Ny/ over C.

In particular, Nx is an optimal solution of (Q).


If (Q) is a convex program, then the existence of Lagrange multiplies for (Q) isequivalent to the existence of a saddle point for L, and the set of Lagrange multipliesis the same for any solutions of (Q). Hence Theorem 9.1 generalizes one of mainresults in Friedlander and Tseng (2007).

The following theorem generalizes the main results (Theorem 2.1) of Friedlanderand Tseng (2007) on exact regularization to the non-convex case.

Theorem 9.2 (Theorem 6 of Deng (2012)). Consider problems (P), (Q), and(P.ı/). Then the following statements are true.

(a) For any ı > 0, S \ Sı � SQ.(b) If there exists a saddle point .Nx; Ny/ of L for (Q) with Nx 2 SQ, then S \ Sı D SQ

for all ı 2 .0; 1=Ny�. Here we use the convention 1=Ny D C1 when Ny D 0.(c) If there exists Nı > 0 such that S \ SNı 6D ;, then .Nx; 1= Nı/ is a saddle point of L

for (Q) with any Nx 2 S \ Sı D SQ for all ı 2 .0; Nı�.(d) If there exists Nı > 0 such that S \ SNı 6D ;, then Sı � S for all ı 2 .0; Nı/.

A direct consequence is the following characterization of exact regularization interms of saddle point of L.

Corollary 9.3. If there is some Ny > 0 such that .Nx; Ny/ is a saddle point of L, thenNx 2 Sı where ı D 1=Ny. Conversely if Sı \ S 6D ;, then for any Nx 2 Sı \ S, .Nx; Ny/ is asaddle point of L where Ny D 1=ı.

Example 9.4. Let g.x1; x2/ D maxf�x1 C x2; 0g, C D f.x1; x2/ j x1 � 0; x2 �0; x1 C x2 � 4g, and f .x1; x2/ D �.x1 � 4/2 � .x2 � 4/2. Then the set S of (P)is the convex hull of the points .0; 0/, .4; 0/ and .2; 2/ and p� D 0. Since f isa concave function, the optimal value of (Q) is achieved at an extreme point ofS due to the concavity of f and the convexity of S. An easy computation showsthat SQ D f.0; 0/g. For any Ny � 0, .0; 0/ is a minimizer of L.; Ny/ over C whereL.x; Ny/ D f .x/C Ny.g.x/� p�/. We conclude that ..0; 0/; Ny/ is a saddle point of L byTheorem 9.1.

9.3 Strongly Exact Regularization, Normal Cone Identity,and Weak Sharp Minima

In this section, we assume that C is a closed convex set, and f ; g are finite convexfunctions. We begin with two definitions.

Definition 9.2. We say that strongly exact regularization holds for (P) if for anyf with SQ 6D ;, (P) is exactly regularized with respect to f in the sense ofDefinition 9.1.

Definition 9.3. We say that the normal cone identity holds for (P) if, for each x 2 S,

NS.x/ D NC.x/C [email protected]/;

180 S. Deng

where @g.x/ is the subdifferential of g at x, and NS.x/;NC.x/ are normal cones of Sand C at x respectively.

The equivalence of strongly exact regularization and normal cone identityfollows.

Theorem 9.4. For (P), strongly exact regularization holds if and only if the normalcone identity holds for (P).

Proof. ())Let Nx 2 S be given. For any v 2 NS.Nx/, let f .x/ D 1=2jjx � .v C Nx/jj2:Then Nx is the unique minimizer for f over S, and rf .Nx/ D �v. Since (P) is exactlyregularized with respect to f , by Theorem 9.2, there is some Ny � 0 such that .Nx; Ny/ isa saddle point of L. So

f .x/C Ny.g.x/� p�/ � f .Nx/ 8x 2 C:

Hence,

�rf .Nx/ 2 [email protected]/C NC.Nx/Ithat is, v 2 NC.Nx/ C [email protected]/, which implies that NS.Nx/ � NC.Nx/ C [email protected]/. Onthe other hand, as S D fx jg.x/ � p�g \ C, one always has

NS.Nx/ � Nfx jg.x/�p�g.Nx/C NC.Nx/ � [email protected]/C NC.Nx/:

Thus, the inclusion NS.Nx/ � NC.Nx/C [email protected]/ always holds. Therefore, the normalcone identity holds.

(() Let f be a finite convex function. Suppose that SQ 6D ;. Then for Nx 2 S,

0 2 @f .Nx/C NS.Nx/:As NS.Nx/ D [email protected]/C NC.Nx/;

0 2 @f .Nx/C [email protected]/C NC.Nx/:

So there are some v 2 @f .Nx/, w 2 @g.Nx/, and Ny � 0 such that 0 2 v C Nyw C NC.Nx/.Hence < v C Nyw; x � Nx >� 0 for all x 2 C. Since f and g are convex,

f .x/C Nyg.x/� .f .Nx/C Nyg.Nx// �< v C yw; x � Nx >� 0

for all x 2 C. It follows that, for each x 2 C,

L.x; Ny/� L.Nx; Ny/ D f .x/C Ny.g.x/� p�/ � f .Nx/ � Ny.g.Nx/� p�/ � 0:

Since g.Nx/ D p�, for any y 2 RC,

L.Nx; y/ D f .Nx/C y.g.Nx/ � p�/ D f .Nx/ D f .Nx/C Ny.g.Nx/� p�/ D L.Nx; Ny/:


This shows that .Nx; Ny/ is a saddle point of L. By Theorem 9.2, exact regularizationholds for (P) with respect to f . Since f is any finite convex function, strongly exactregularization holds for (P). This completes the proof.

For (P), the normal cone identity property is closely related to the notionof weak sharp minima, which has found a number of important applications inmathematical programming. See Burke and Ferris (1993) and Burke and Deng(2002) and references therein. Recall Burke and Ferris (1993) and Burke and Deng(2002) that S is said to be a set of weak sharp minima for g over the set C withmodulus ˛ > 0 if

˛dist.x; S/ � g.x/� p� 8x 2 C;

where dist.x; S/ is the Euclidean distance between x and S.

Theorem 9.5. If S is a set of weak sharp minima for g over C with modulus ˛ > 0,then the normal cone identity holds for (P)

Proof. Since g is a finite convex function,

@.g C ıC/.x/ D @g.x/C NC.x/;

where ıC is the indicator function of the set C. By Part 2 of Theorem 2.3 in Burkeand Deng (2002), for any x 2 S,

˛B \ NS.x/ � @g.x/C NC.x/;

where B is the Euclidean unit ball. Then

NS.x/ D RC.˛B \ NS.x// � [email protected]/C NC.x// � [email protected]/C NC.x/:

On the other hand, as S D fz jg.z/ � p�g \ C, one always has

NS.x/ � Nfz j g.z/�p�g.x/C NC.x/ � [email protected]/C NC.x/:

This shows that NS.x/ D [email protected]/C NC.x/; that is, the normal cone identity holds.This completes the proof.

We conclude the paper with an example which shows that the normal coneidentity holding for (P) does not imply that S is a set of weak sharp minima forg over C in general.

Example 9.5. For (P), let g W R3 ! R be given by g.x/ D x3, and C DŒ0; 1�3 \ .\1

kD2Ck/; where Ck D fx 2 R3 j x1 � .k � 1/x2 � k2x3 � 1=kg: Then theoptimal value p� D 0 and S D fx 2 C j x3 D 0g. It was shown in the Appendix ofFriedlander and Tseng (2007) that S is not a set of weak sharp minima for g over C,and that for any given z 2 Rn and f .x/ D 1=2jjx� zjj2, exact regularization holds for

182 S. Deng

(P) with respect to f . The proof of necessary part of Theorem 9.4 shows that thisimplies the normal cone identity holds; that is, NS.x/ D NC.x/C [email protected]/ for eachx 2 S.

Acknowledgements We wish to thank the referees for their useful comments, which helped usimprove the presentation of the paper.

References

Burke JV, Ferris M (1993) Weak sharp minima in mathematical programming. SIAM J ControlOptim 31:1340–1359

Burke JV, Deng S (2002) Weak sharp minima revisited, part I: basic theory. Control Cybern31:439–469

Deng S (2012) A saddle point characterization of exact regularization of non-convex programs.Pac J Optim 8(1):27–32

Friedlander M, Tseng P (2007) Exact regularization of convex programs. SIAM J Optim18:1326–1350

Rockafellar RT (1993) Lagrange multipliers and optimality. SIAM Rev 35:183–238Rockafellar RT, Wets R (1998) Variational analysis. Springer, Berlin/Heidelberg

Chapter 10The Worst-Case DFT Filter Bank Design withSubchannel Variations

Lin Jiang, Changzhi Wu, Xiangyu Wang, and Kok Lay Teo

Abstract In this paper, we consider an optimal design of a DFT filter banksubject to subchannel variation constraints. The design problem is formulated asa minimax optimization problem. By exploiting the properties of this minimaxoptimization problem, we show that it is equivalent to a semi-infinite optimizationproblem in which the continuous inequality constraints are only with respect tofrequency. Then, a computational scheme is developed to solve such a semi-infiniteoptimization problem. Simulation results show that, for a fixed distortion level, thealiasing level between different subbands is significantly reduced, in some cases upto 28 dB, when compared with that obtained by the bi-iterative optimization methodwithout consideration of the subchannel variations.

10.1 Introduction

Filter banks play an important role in a wide range of signal processing applicationssuch as echo cancellation (Kellermann 1988), microphone arrays (de Haan et al.2003), speech enhancement and equalization (Vaidyanathan 1993), as well as imageand speech processing. Owing to their wide range of applications, they have beenextensively studied in the past two decades (Dam et al. 2005; de Haan et al. 2001,2003; Harteneck et al. 1999; Kellermann 1988; Kha et al. 2009; Mansour 2007;Nguyen 1994; Sturm 1999; Vaidyanathan 1993; Wilbur et al. 2004; Wu and Teo2010, 2011; Wu et al. 2008, 2013; Yiu et al. 2004; Zhang et al. 2008). In multiratedigital signal processing, an analysis filter is used to divide the signal to be processedinto subbands. They are then decimated according to the new bandwidth of thesubbands. The decimation process causes aliasing of the subband signals. It is

L. Jiang (�)School of Mathematics, Anhui Normal University, Wuhu, 241000, China

C. Wu • X. WangAustralasian Joint Research Centre for Building Information Modelling, School of BuiltEnvironment, Curtin University, Perth, WA 6845, Australia

K.L. TeoSchool of Mathematics and Statistics, Curtin University, Perth, WA, 6845, Australia


183

184 L. Jiang et al.

possible to cancel this aliasing through the design of a synthesis filter bank in sucha way that the whole multirate chain yields no distortion; the total transfer functionis reduced to a simple delay. This is often referred to as the perfect reconstruction(PR) property (Harteneck et al. 1999).

However, any filtering operation in the subbands will cause phase and amplitudechanges, thereby altering this property. Thus, aliasing may be caused in thereconstructed output of the subband adaptive filter. To overcome this problem,optimization methods are often used in the design of filter banks, where boththe aliasing effect and the distortion levels in the filter bank are optimized. Inde Haan et al. (2003), the design of a uniform discrete Fourier transform (DFT)filter bank is solved by a two-step optimization problem. In the first step, theanalysis filter bank is designed in such way that the aliasing terms in each subbandare minimized individually, contributing to minimal aliasing at the output withoutaliasing cancellation. In the second step, the synthesis filter bank is designed tomatch the analysis filter bank where the analysis-synthesis response is optimizedwhile all aliasing terms in the output signal are individually suppressed, rather thanaiming at aliasing cancellation. Since the analysis and synthesis filter banks aredesigned separately, this method does not produce a good result. To improve thismethod, a bi-iterative method is used in Dam et al. (2005), where the design ofthis filter bank is formulated as a constrained fourth order polynomial optimizationproblem with continuous constraints. This optimization problem is hard to solve.Thus, a bi-iterative computational scheme is incorporated to solve the formulatedoptimization problem. More specifically, the analysis filter bank is fixed whensolving the synthesis filter bank, while the synthesis filter bank is fixed whensolving the analysis filter bank. In this way, only quadratic optimization problemis required to be solved during the design process of this filter bank. Since theoriginal optimization problem is a fourth order polynomial optimization problem,this bi-iterative scheme does not provide a global optimal solution. To overcomethis problem, a global optimization method based on the filled function method isintroduced to solve the corresponding optimization problem in Wu et al. (2008).By using the global method, better results are obtained. In Yiu et al. (2004), thedesign problem has been formulated as a multicriteria optimization problem. Then, anonlinear programming methods can be used to solve such an optimization problem.In Wilbur et al. (2004), the design of a generalized DFT filter bank is formulated as acone program with a combination of linear, second-order, and semi-definite cones. Itis solved by using an existing convex optimization software package (Sturm 1999).

Although the DFT filter bank has been extensively studied, the actual filteringoperation in each frequency band has only been taken into limited consideration.In this paper, we propose a new formulation which includes a filtering operation ineach subband in addition to the optimization and control of each individual criterion.The aliasing effects for the filter bank are minimized subject to the constraints onthe distortion level for all the frequencies. The formulation is such that it includes aterm measuring the deviation from the nominal value in each subband. Comparingwith the earlier formulations, our formulation is more ideal for use in adaptive filtersor speech enhancement applications since the filter operation with distortion in each

10 The Worst-Case DFT Filter Bank Design with Subchannel Variations 185

subband has been taken into consideration. In this problem formulation, the studyis to provide simultaneous optimization on both the analysis and synthesis filterbanks, while maintaining robustness against deviation from unity in each subband.The advantage of this problem formulation is that it provides less overall distortionwhen the subchannels are subject to distortions. This filter bank design problem isformulated as a constrained minimax optimization problem.

The second contribution of this paper is that we proved that the formu-lated constrained minimax optimization problem was equivalent to a semi-infiniteoptimization problem where the continuous constraints are only with respect tofrequency. Although a minimax optimization problem is actually a semi-infiniteoptimization problem, we cannot use available methods for solving semi-infiniteoptimization to solve it directly. This is due to the fact that the semi-infiniteoptimization problem is known to suffer from the curse of dimensionality (Lopezand Still 2007). In this paper, we will show that this minimax optimization problemis equivalent to a standard semi-infinite optimization problem with non-smoothconstraints. Then, an iterative computational scheme is developed to solve thisoptimization problem. Some simulation examples are presented to illustrate themethod proposed. Simulation results show that, for a fixed distortion level, thealiasing between different subbands is significantly reduced, in some cases up to28 dB, when compared with those obtained by using the bi-iterative optimizationmethod developed in Dam et al. (2005) which is not taken into the consideration ofthe subband variations.

10.2 Analysis and Synthesis Filter Banks

Uniformly modulated filter banks are employed where the filter banks are formedby modulated versions of the analysis and synthesis prototype filters. Denote h DŒh.0/; ; h.La � 1/�T as the prototype filter of length La for the analysis filterbank with the corresponding transfer function H.z/ D hT�a.z/, where �a.z/ DŒ1; z�1; ; z�.La�1/�. Similarly, denote g D Œg.0/; ; g.Ls � 1/�T as the prototypefilter of length Ls for the synthesis filter bank with the transfer function G.z/ DgT�s.z/, where �s.z/ D Œ1; z�1; ; z�.Ls�1/�. For a system with M subbands, thesubband filters Hm.z/ and Gm.z/, 0 � m � M � 1, are obtained from the prototypefilters H.z/ and G.z/, respectively, as follows:

Hm.z/ D H.zWmM/ and Gm.z/ D G.zWm

M/ (10.1)

where WM D e�j2�=M . A possible realization of an analysis and synthesis filter bankis given in Fig. 10.1. The input signal X.z/ is filtered by the analysis filter Hm.z/ anddecimated by a factor D, D � M, according to

Xm.z/ D 1

D

D�1XdD0

H.z1=DWmMWd

D/X.z1=DWd

D/ (10.2)

186 L. Jiang et al.

( )0H z( )X z ( )0X zD↓ D↓ D↓ D↓ D↑ ( )0Y z ( )0G z

( )1Y z( )1H z ( )1X zD↓ D↓ D↓ D↓ D↑ ( )1G z

( )1MH z− D↓ D↓ D↓ D↓ D↑ ( )1MG z−( )1MX z− ( )1MY z−

+

+( )Y z

Fig. 10.1 Analysis and synthesis filter banks

where WD D e�j2�=D. Denote �m.z/ as an application dependent filtering operationfor the mth subband. The output of the filtering operation for the mth subband thenbecomes

Ym.z/ D �m.z/Xm.z/: (10.3)

In the synthesis filter bank, the subband signals Ym.z/ are interpolated by a factor Dand then added to form the output signal Y.z/ given by

Y.z/ D 1

D

D�1XdD0

X.zWdD/

M�1XmD0

�m.zD/H.zWm

MWdD/G.zWm

M/: (10.4)

The termM�1PmD0

�m.zD/H.zWmMWd

D/G.zWmM/ can be viewed as the transfer function

which contributes to the aliasing terms in the output signal for 1 � d � D�1 and tothe desired output signal for d D 0. Normally, the analysis and synthesis filter banksare designed for the case �m.zD/ D 1 as filtering operation is unknown. However,this performs poorly in real situations when �m.zD/ can take on arbitrary values.Thus, the optimization should be performed subject to an allowable variation for�m.zD/ D 1C ım; where ım is a random variation. Thus the design of this filter bankshould include this variation constraint. Let ı D Œı0; ı1; ; ıM�1�T : We supposethat the random vector ı is restricted to the following box constrained set

U D ˚ı D Œı0; ı1; ; ıM�1�T 2 R

M W jıij � "i�;

where "i; i D 0; ;M � 1; are small constants.

10.3 Worst-Case Prototype Filter Design

The objective is to optimize the analysis and synthesis prototype filters with respectto both the aliasing power and the distortion for the filter bank. The aliasing powerfor all the aliasing terms in (10.4) for a frequency ! 2 Œ��; �� is given by


A.!/ D 1

Dj

D�1XdD1

M�1XmD0

�m.zD/H.ej!Wm

MWdD/G.e

j!WmM/j2: (10.5)

This can be rewritten in terms of the analysis and synthesis prototype FIR filtercoefficients as follows:

A.!/ D 1

Dj

D�1XdD1

M�1XmD0

�m.zD/hTˆm;d.e

j!/gj2 (10.6)

where

ˆm;d.ej!/ D �a.e

j!WmMWd

D/�Ts .e

j!WmM/: (10.7)

Thus, the total aliasing power for all the frequencies ! 2 Œ��; �� is defined as

�.h; g; ı/ D 1

2�

Z �

��A.!/d!

D 1

2�D

Z �

��j

D�1XdD1

M�1XmD0

�m.zD/hTˆm;d.e

j!/gj2d!

D 1

2�D

Z �

��j

M�1XmD0

.1C ım/

D�1XdD1

hTˆm;d.ej!/gj2d!; (10.8)

and the analytical form of �.h; g; ı/ is given in Appendix 1.However, we would also like to constrain the distortion on the solution so that the

solution obtained will possess the property of having a tight bound on distortion. Assuch, denote �d as the desired total delay for the filter bank. The desired frequencyresponse for the total system is given as Td.ej!/ D e�j!�d : It follows from (10.4)that the transfer function of the filter bank is

T.z/ D 1

D

M�1XmD0

�m.zD/H.zWm

M/G.zWmM/ D hT‰.z; ı/g; (10.9)

where

‰.z; ı/ D 1

D

M�1XmD0

.1C ım/�a.zWmM/�

Ts .zWm

M/: (10.10)

Since the analysis and synthesis prototype filters are to be designed subject to anallowable small distortion from a desired response, the worst case scenario can beformulated using the mini-max constraint on the total response of the filter bankbelow,

jhT‰.ej!; ı/g � Td.ej!/j � �; 8! 2 Œ��; ��;8ı 2 U ; (10.11)

188 L. Jiang et al.

where � is a specified small error. To proceed further, the modulus con-straints (10.11) are replaced by their real and imaginary part constraints givenbelow.

hTRe˚‰.ej!; ı/

�g � cos .�d!/

� �=p2; 8! 2 Œ��; ��;8ı 2 U ; (10.12)

cos .�d!/ � hTRe˚‰.ej!; ı/

�g

� �=p2; 8! 2 Œ��; ��;8ı 2 U ; (10.13)

hTIm˚‰.ej!; ı/

�g C sin .�d!/

� �=p2; 8! 2 Œ��; ��;8ı 2 U ; (10.14)

� hTIm˚‰.ej!; ı/

�g� sin .�d!/

� �=p2; 8! 2 Œ��; ��;8ı 2 U ; (10.15)

where Re fzg and Im fzg denote the real part and the imaginary part of z; respectively.Now the optimal design for the worst-case scenario of this filter bank can be posedas the following optimization problem:

Problem 10.1.

minh;g

maxı2U �.h; g; ı/ (10.16)

subject to the constraints (10.12)–(10.15).

10.4 Solution Strategy

As a result of the presence of the maxı2U operator, the cost function (10.16) is non-

smooth. Thus, Problem 10.1 cannot be solved by the gradient-based optimizationmethods. By introducing a new variable �; Problem 10.1 can be reformulated as thefollowing semi-infinite optimization problem:

Problem 10.2.

minh;g;�

�

subject to the constraints (10.12)–(10.15) and

�.h; g; ı/ � �;8ı 2 U : (10.17)


As we can see, Problem 10.2 is a semi-infinite optimization problem. However, inthis semi-infinite optimization problem, the argument ı is an M-dimensional vector.Thus, it is extremely difficult, if not impossible, to solve Problem 10.2 using anyavailable method for semi-infinite optimization problems. In the following, we showthat Problem 10.2 is equivalent to a semi-infinite optimization problem, where thecontinuous inequality constraints are only with respect to w: We first give somedefinitions and then obtain the maximizer �.h; g; ı/ on U for each fixed h and g:

Convex set: A nonempty set B in Rn is said to be convex if x C .1 � / y 2 B

for any x; y 2 B and 0 � � 1.Convex combination: A convex combination of a finite number of points

x1; ; xk in Rn is a point of the form

kPiD1ixi; where i � 0; and

kPiD1i D 1:

Convex hull: Let B be a nonempty subset of Rn: The convex hull of B, which isdenoted as co .B/ ; is defined by

co .B/ D(

kXiD1

ixi W xi 2 B; i � 0;

andkX

iD1i D 1; k D 1; 2;

):

Extreme point: Let B be a convex set in Rn: If x is an extreme point of B; then

there do not exist two distinct points y; z ¤ x in B such that x is expressed as aconvex combination of y and z:

Define

D D ˚ı D Œı0; ı1; ; ıM�1�T 2 U W jıij D "i

�: (10.18)

Clearly, D has 2M distinct extreme points. Let these 2M distinct points be denoted asNı1; ; Nı2M : For the set U , we have the following theorems:

Theorem 10.1. U is a convex set and Nı1; ; Nı2M are extreme points of U :

Furthermore, U D co Nı1; ; Nı2M

�:

Proof. See Appendix 2.

Theorem 10.2. For each given h and g; the maximum of �.h; g; ı/ in U is attainedat one of the 2M extreme points Nı1; ; Nı2M :


By Theorem 10.2, it is clear that the constraint

�.h; g; ı/ � �;8ı 2 U ;

190 L. Jiang et al.

is satisfied if and only if the following 2M constraints are fulfilled.

�.h; g; Nıi/ � �; i D 1; ; 2M: (10.19)

For notational simplicity, denote

r;m .!;h; g/ D Re˚hT�a.e

j!WmM/�

Ts .e

j!WmM/g

�;

and

i;m .!;h; g/ D Im˚hT�a.e

j!WmM/�

Ts .e

j!WmM/g

�:

Now, we have the following theorem.

Theorem 10.3.

maxı2U hTRe

˚�.ej!; ı/

�g

D 1

D

M�1XmD0

. r;m .!;h; g/C "m j r;m .!;h; g/j/ ; (10.20)

minı2U hTRe

˚�.ej!; ı/

�g

D 1

D

M�1XmD0

. r;m .!;h; g/� "m j r;m .!;h; g/j/ ; (10.21)

maxı2U hTIm

˚�.ej!; ı/

�g

D 1

D

M�1XmD0

. i;m .!;h; g/C "m j i;m .!;h; g/j/ ; (10.22)

and

minı2U hTIm

˚�.ej!; ı/

�g

D 1

D

M�1XmD0

. i;m .!;h; g/� "m j i;m .!;h; g/j/ : (10.23)


The conclusion given in the following theorem follows readily from Theo-rems 10.2 and 10.3.


Theorem 10.4. Problem 10.2 is equivalent to Problem 10.3 which is defined asfollows.

Problem 10.3.

minh;g;�

� (10.24)

subject to

�.h; g; Nıi/ � �; i D 1; ; 2M; (10.25)

1

D

M�1XmD0

. r;m .!;h; g/C "m j r;m .!;h; g/j/

� cos .�d!/ � �=p2; 8! 2 Œ��; ��; (10.26)

� 1

D

M�1XmD0

. r;m .!;h; g/� "m j r;m .!;h; g/j/

C cos .�d!/ � �=p2; 8! 2 Œ��; ��; (10.27)

1

D

M�1XmD0

. i;m .!;h; g/C "m j i;m .!;h; g/j/

C sin .�d!/ � �=p2; 8! 2 Œ��; ��; (10.28)

� 1

D

M�1XmD0

. i;m .!;h; g/� "m j i;m .!;h; g/j/

� sin .�d!/ � �=p2; 8! 2 Œ��; ��: (10.29)

Remark 10.1. Since (10.26) and (10.27) can be re-written as

1

D

M�1XmD0

r;m .!;h; g/� cos .�d!/

� �p2 � 1

D

M�1XmD0

"m j r;m .!;h; g/j ;

192 L. Jiang et al.

and

� 1

D

M�1XmD0

r;m .!;h; g/� cos .�d!/

!

� �=p2 � 1

D

M�1XmD0

"m j r;m .!;h; g/j ;

respectively, we haveˇˇˇ1

D

M�1XmD0

r;m .!;h; g/� cos .�d!/

ˇˇˇ

� �=p2 � 1

D

M�1XmD0

"m j r;m .!;h; g/j :

Thus, a necessary condition for the solvability of Problem 10.1 is that there exits hand g; such that

1

D

M�1XmD0

"m j r;m .!;h; g/j � �=p2;8! 2 Œ��; ��: (10.30)

Taking Q" D min f"0; "1; ; "M�1g and ! D 0 in (10.12) and (10.13), we obtain

Q"�1 � �=

p2�

� Q"D

ˇˇM�1XmD0

r;m .0;h; g/

ˇˇ � Q"

D

ˇˇM�1XmD0

r;m .0;h; g/

ˇˇ

� Q"D

M�1XmD0

j r;m .0;h; g/j � 1

D

M�1XmD0

"m j r;m .0;h; g/j � �=p2: (10.31)

If Q" � p2 � 1; then Q" � � by (10.31). This illustrates the relationship between

the disturbance and the tolerance in (10.11). Actually, from our computationalexperience, � should be much larger than max f"0; "1; ; "M�1g :

Let

F1 .!/ Dˇˇ 1D

M�1XmD0

r;m .!;h; g/� cos .�d!/

ˇˇ (10.32)

C 1

D

M�1XmD0

"m j r;m .!;h; g/j ;


F2 .!/ Dˇˇ 1D

M�1XmD0

i;m .!;h; g/C sin .�d!/

ˇˇ (10.33)

C 1

D

M�1XmD0

"m j i;m .!;h; g/j :

Then, according to Remark 10.1, the constraints (10.26)–(10.29) are equivalent toF1 .!/ � �=

p2 and F2 .!/ � �=

p2:

Note from Theorem 10.4 that the continuous inequality constraints in Problem10.3 are only with respect to w: However, some of the constraint functions arenon-smooth since they appear in the form as absolute value functions. Thus,gradient-based optimization methods cannot be applied directly. In order to removethis obstacle, we introduce the following smoothing approximation (Teo and Goh1988):

'� .y/ D8<:

y; if y � �;�2 C y2

�= .2�/ ; if jyj < �;

�y; if y � ��;(10.34)

where � > 0 and y 2 R. Now we replace j r;m .!;h; g/j and j i;m .!;h; g/jin the continuous inequality constraints (10.26)–(10.29) and obtain the followingoptimization problem

minh;g;�

� (10.35)

subject to

�.h; g; Nıi/ � �; i D 1; ; 2M; (10.36)

1

D

M�1XmD0

r;m .!;h; g/C "m'� . r;m .!;h; g//

�(10.37)

� cos .�d!/ � �=p2; 8! 2 Œ��; ��;

� 1

D

M�1XmD0

r;m .!;h; g/ � "m'� . r;m .!;h; g//

�(10.38)

C cos .�d!/ � �=p2; 8! 2 Œ��; ��;

194 L. Jiang et al.

1

D

M�1XmD0

i;m .!;h; g/C "m'� . i;m .!;h; g//

�(10.39)

C sin .�d!/ � �=p2; 8! 2 Œ��; ��;

� 1

D

M�1XmD0

i;m .!;h; g/ � "m'� . i;m .!;h; g//

�(10.40)

� sin .�d!/ � �=p2; 8! 2 Œ��; ��:

Let the corresponding problem be referred to as Problem .P�/: For any �2 ��1 > 0, since

'�2 .y/ � '�1 .y/ � jyj ;

it follows that any feasible solution of Problem .P�2/ is a feasible solution ofProblem .P�1/: It is also a feasible solution of Problem 10.3. According to Teoand Goh (1988), the solution of Problem .P�/ converges to the solution of Problem10.3 as � ! 0. Furthermore, the feasible region of Problem .P�/ is increased as � isdecreased. Hence, Problem 10.3 can be solved through solving Problem .P�/ whichcan be solved by many available methods, such as the one given in Lopez and Still(2007). Now the algorithm is summarized as follows.

Algorithm 10.1• Step 1: Initialize �1 > 0 and k D 1: Set ıi D 0; i D 0; 1; � � � ;M � 1; and use the bi-iterative

optimization method in Dam et al. (2005) to design the filter bank. Let the solution obtained bedenoted by

h�

0 ; g�

0

�.

• Step 2: Solve Problem .P�k / with the initial conditionh�

k�1; g�

k�1

�. Let the cost and solution

obtained be denoted by ��

k andh�

k ; g�

k

�; respectively.

• Step 3: Set �kC1 D �k=L; where L > 1 is a pre-specified number.• Step 4: If ��

k�1 � ��

k � κ; where κ > 0 is a prescribed small number, stop. Otherwise, setk D k C 1, go to Step 2.

There are three parameters �1; L and κ in Algorithm 10.1. �1 determineshow close the approximate problem .P�1/ is to Problem 10.3. The smaller the�1; the closer the approximate problem .P�1 / is to Problem 10.3. However, theconstraints become less smooth. L determines the required iteration. The largerthe L; the lesser iterations are required. However, the approximate problem willbecome less smooth for smaller k: κ determines the accuracy of the approximation.The smaller the κ; the more iterations are required. In our simulation, we take�1 D 10�3; L D 10; κ D 10�6. Such parameters can achieve a good performancefor most optimization problems judging from our computational experience.


10.5 Numerical Examples

In this section, we will use our developed algorithm to design the analysis and syn-thesis prototype filters with subchannel variations. In the following discussion, � D10�2: Furthermore, the continuous constraints interval Œ��; �� is discretized into512 equally spaced frequency points for the optimization Problem .P�/. Considerthe case with M D 4; D D 2; " D Œ"0; "1; "2; "4�

T D Œ0:001; 0:002; 0:004; 0:004�T :

Let La D Ls D 4M C 1; �d D 4M.First, we consider the case without subchannel variations, i.e., ıi D 0; i D

0; 1; ;M � 1: For such a filter bank design, the bi-iterative algorithm developedin Dam et al. (2005) is introduced to design the initial analysis and synthesisprototype filters. The cost obtained is �167:0414dB. Let the prototype filters h andg obtained be collected together and denote as x.0/�:We substitute x.0/� into (10.16)and find that the largest value of �.h; g; ı/ is �90:7572dB which is obtained atthe extreme point Œ0:001; 0:002; 0:004; 0:004�T of U . The maximum violation ofthe constraints (10.12)–(10.15) is 0:003973: From these results, we can see that thealiasing �.h; g; ı/will have a large increase if the subchannels with variations. Thus,it is necessary to consider the robust optimal filter bank design.

Now, we consider the case of the subchannels with variations. We use theprototype filters x.0/� obtained by setting ıi D 0 as the initial condition and useAlgorithm 10.1 to optimize Problem .P�/ with �1 D 10�3; L D 10; κ D10�6. After three iterations, the optimal solution is obtained. The optimal cost is�106:4557dB which is obtained at the extreme point Œ0:001; 0:002; 0:004; 0:004�T

of U . Clearly, there is not only about 16 dB improvement by using the developedrobust optimization method when compared with the result obtained by the bi-iterative algorithm developed in Dam et al. (2005), but also maintaining theconstraints (10.12)–(10.15) when the subchannels with variations. However, itshould be noted that the problem considered in Dam et al. (2005) is an optimizationproblem of a uniform FIR filter bank with group delay specification but without theconsideration of the variations in subchannels. The coefficients of the prototypeanalysis and synthesis filters are presented in Table 10.1 and the correspondingfrequency responses are plotted in Figs. 10.2 and 10.3, respectively. The frequencyresponse of T

ej!�

with ı D Œ0; 0; 0; 0�T in (10.9) is plotted in Fig. 10.4 and F1 .w/and F2 .w/ are plotted in Fig. 10.5. If we take all the "i to be the same, i.e., "i D0:005: Then the analysis and synthesis prototype filters designed by our method areplotted in Fig. 10.6 and F1 .!/ and F2 .!/ are depicted in Figs. 10.7 and 10.8. Thecost obtained is �109:7622dB. Thus, there is about 19 dB improvement by usingour design method. Furthermore, the filter bank design by our method satisfies theconstraints (10.12)–(10.15) for any ı in U since the constraints (10.26)–(10.29)are satisfied. From Fig. 10.7, we can see that this is not the case if the bi-iterativealgorithm developed in Dam et al. (2005) is used.

Now let M D 8;D D 4; " D Œ0:002; 0:001; 0:002; 0:001; 0:002; 0:001; 0:002;

0:001�T . La D Ls D 4M; �d D 4M. We use the bi-iterative algorithm in Damet al. (2005) to design the initial analysis and synthesis prototype filters with

196 L. Jiang et al.

Table 10.1 The coefficientsof the prototype analysis andsynthesis filters with " DŒ0:001; 0:002; 0:004; 0:004�

T

h g

0:002229646148184 0:004295059400797

0:008688331557603 �0:0057736374359930:002888438166081 �0:035983297514096

�0:023517760474806 �0:058922903112834�0:044869395983114 �0:024054015169344�0:003348418428374 0:09822147184615

0:126091278937254 0:279096672564763

0:28163363523314 0:44123638388896

0:351454776609997 0:506386191221335

0:279700679819365 0:442412199910485

0:123916445314385 0:28101672864404

�0:004236799526732 0:099853376280495

�0:044285867995809 �0:023793500365253�0:022506213122295 �0:0603234310589720:003492532067841 �0:0380622126695150:008946437740113 �0:0072667187156550:0022092084606 0:003686028054542

0 0.2 0.4 0.6 0.8 1−90

−80

−70

−60

−50

−40

−30

−20

−10

0

10

The normalized frequency

Fre

quen

cy r

espo

nse

in d

B

Analysis prototype filter

Fig. 10.2 The frequency response of the analysis prototype filter with " D Œ0:001; 0:002;

0:004; 0:004�T


0 0.2 0.4 0.6 0.8 1−80

−70

−60

−50

−40

−30

−20

−10

0

10


Fre

quen

cy r

espo

nse

in d

B

Synthesis prototype filter

Fig. 10.3 The frequency response of the synthesis prototype filter with " D Œ0:001; 0:002;

0:004; 0:004�T

−1 −0.5 0 0.5 10

0.5

1

1.5

2

2.5

3

3.5x 10

−3


Fre

quen

cy r

espo

nse

in d

B

Fig. 10.4 The frequency response of Tejw�

with ı D Œ0; 0; 0; 0�T and " D Œ0:001; 0:002;

0:004; 0:004�T

198 L. Jiang et al.

−1 −0.5 0 0.5 10

1

2

3

4

5

6

7

8x 10

−3

F1(w)

F2(w)

Fig. 10.5 The figures of F1 .!/ and F2 .!/ with " D Œ0:001; 0:002; 0:004; 0:004�T

0 0.2 0.4 0.6 0.8 1−100

−80

−60

−40

−20

0

20


Fre

quen

cy r

espo

nse

in d

B

Analysis prototype filterSynthesis prototype filter

Fig. 10.6 The frequency responses of the analysis and synthesis prototype filters with " DŒ0:005; 0:005; 0:005; 0:005�T


−1 −0.5 0 0.5 10

0.002

0.004

0.006

0.008

0.01

0.012

0.014F

1(w)

F2(w)

Fig. 10.7 The figures of F1 .!/ and F2 .!/ by the method in Dam et al. (2005)

−1 −0.5 0 0.5 10

1

2

3

4

5

6

7

8x 10

−3

F1(w)

F2(w)

Fig. 10.8 The figures of F1 .w/ and F2 .w/ with " D Œ0:005; 0:005; 0:005; 0:005�T

ıi D 0; i D 0; 1; ; 7. The cost obtained is �105:75 dB. For these two prototypefilters, �.h; g; ı/ achieved the largest value �70:6954dB at the extreme point

Œ0:002; 0:001; 0:002; 0:001; 0:002; 0:001; 0:002; 0:001�T

of U : Now we use Algorithm 10.1 to solve Problem 10.3 with the obtainedprototype filters as initial guess. After two iterations, the optimal cost �98:5855 dBis obtained. The analysis and synthesis prototype filters are plotted in Fig. 10.9. The

200 L. Jiang et al.

0 0.2 0.4 0.6 0.8 1−100

−80

−60

−40

−20

0

20


Fre

quen

cy r

espo

nse

in d

B

Analysis prototype filterSynthesis prototype filter

Fig. 10.9 The frequency response of the synthesis prototype filter with M D 8, D D 4, " DŒ0:002; 0:001; 0:002; 0:001; 0:002; 0:001; 0:002; 0:001�

T

−1 −0.5 0 0.5 10

1

2

3

4

5

6

7

8x 10

−3


Fre

quen

cy r

espo

nse

in d

B

F1(w)

F2(w)

Fig. 10.10 The figures of F1 .!/ and F2 .!/ with " D Œ0:002; 0:001; 0:002; 0:001; 0:002; 0:001;

0:002; 0:001�T

corresponding F1 .!/ and F2 .!/ are plotted in Fig. 10.10. From the results obtained,we can see that the aliasing can achieve about 28 dB improvement by using ourrobust design method.


10.6 Conclusions

We have proposed a new formulation of the DFT filter bank problem in which thesubchannel are with variations. Comparing with the earlier formulations, our formu-lation is more realistic since the filter operation with distortion in each subband hasbeen taken into consideration. It is in the form of a minimax optimization problemwith continuous inequality constraints. Although this minimax optimization can bereformulated as a semi-infinite optimization problem by introducing an additionalvariable, it is still cannot be solved directly by any existing method for semi-infiniteoptimization problems. This is because the continuous constraints are not only withrespect to frequency, but also with respect to variations in subchannels. However, byexploiting its properties, we proved that such a semi-infinite optimization problem isequivalent to a semi-infinite optimization problem where the continuous constraintsare only with respect to frequency. Then, an approximate computation scheme isdeveloped to solve the transformed semi-infinite optimization problem. Simulationresults showed that the new method achieved a very high aliasing suppression whilemaintaining the distortion under variations in the different filter bands to be a smalllevel.

Acknowledgements Changzhi Wu was partially supported by Australian Research CouncilLinkage Program, Natural Science Foundation of China (61473326), Natural Science Foundationof Chongqing (cstc2013jcyjA00029 and cstc2013jjB0149).

Appendix 1

�.h; g; ı/ D 1

D

M�1XmD0

M�1XnD0

.1C ım/ .1C ın/

D�1XdD1

D�1XlD1

hTˆm;n;d;l .g/ h; (10.41)

whereˆm;n;d;l .g/ is a La � La matrix. The .i; j/-th element of ˆm;n;d;l .g/ is given by

Œˆm;n;d;l .g/�i;j DLs�1XtD0

Ls�1XsD0

cos

�.m � n/ .i C t � 2/ 2�

M

C .d .i � 1/� l .j � 1// 2�D

�ı .i C t � j � s/ g .t/ g .s/ ;

where ı ./ is the delta function, i.e.,

ı .t/ D1; if t D 0;

0; if t ¤ 0:

202 L. Jiang et al.

Appendix 2

Proof of Theorem 10.1. Clearly, U is a convex set and Nı1; ; Nı2M are extremepoints of U : It remains to show that U D co

Nı1; ; Nı2M

�: For any ı D

Œı0; ı1; ; ıM�1�T 2 co Nı1; ; Nı2M

�; there exists i � 0; i D 1; ; 2M; such

that ı D2MPiD1i

Nıi and2MPiD1i D 1: Then, ık D

2MPiD1i

Nıi;k; where Nıi;k denotes the kth

element of Nıi: From (10.18), we have

jıkj Dˇˇˇ2MXiD1

iNıi;k

ˇˇˇ �

2MXiD1

i

ˇ Nıi;k

ˇ D2MXiD1

i"k D "k:

Thus, ı 2 U ; and hence, co Nı1; ; Nı2M

� � U . On the other hand, let ı DŒı0; ı1; ; ıM�1�T with jıij � "i; i D 0; 1; ;M � 1: Since jı0j � "0; thereexists a 0; 0 � 0 � 1; such that ı0 D 0"0 � .1 � 0/ "0: Since co

Nı1; ; Nı2M

�is convex and

Œı0; "1; ; "M�1�T D 0 Œ"0; "1; ; "M�1�T

C .1 � 0/ Œ�"0; "1; ; "M�1�T ;

we have

Œı0; "1; ; "M�1�T 2 co Nı1; ; Nı2M

�:

Since jı1j � "1; there exists a 1 such that ı1 D 1"1 � .1 � 1/ "1 and 0 � 1 � 1:

Thus,

Œı0; ı1; "2; ; "M�1�T D 1 Œı0; "1; "2; ; "M�1�T

C .1 � 1/ Œı0;�"1; "2; ; "M�1�T 2 co Nı1; ; Nı2M

�:

Continuing this process, we can show that ı D Œı0; ı1; ; ıM�1�T2co Nı1; ; Nı2M

�:

Hence, U � co Nı1; ; Nı2M

�: Therefore, U D co

Nı1; ; Nı2M

�:

Proof of Theorem 10.2. From (10.8), we see that �.h; g; ı/ is in quadratic form withrespect to ı. Furthermore, �.h; g; ı/ � 0 for any ı: Thus, we can write �.h; g; ı/ inthe form below.

�.h; g; ı/ D ıTQı C qT ıC q;


where Q D Q1=2

�TQ1=2 is a semi-positive definite matrix, q and q are correspond-

ing vector and constant. For any ; 0 � � 1, ı1 and ı2; we have

�.h; g; ı1/C .1 � / �.h; g; ı2/

��.h; g; ı1 C .1 � / ı2/

D ıT1Qı1 C .1 � / ıT

2Qı2 � 2ıT1Qı1

� .1 � /2 ıT2Qı2 � 2 .1 � / ıT

1Qı2

D .1 � /ıT1Qı1 C ıT

2Qı2 � 2ıT1Qı2

�

D .1 � /h

Q1=2ı1�T

Q1=2ı1

C Q1=2ı2

�TQ1=2ı2 � 2 Q1=2ı1

�TQ1=2ı2

i

� 0:

Thus, �.h; g; ı/ is a convex function with respect to ı. Suppose that ı� Darg max

ı2U �.h; g; ı/ and N� D max˚�.h; g; Nı1/; ; �.h; g; Nı2M /

�: From Theo-

rem 10.1, we know that there exists i � 0; 0 � i � 2M; with2MPiD1i D 1;

such that ı� D2MPiD1i Nıi: Since �.h; g; ı/ is a convex function with respect to ı, we

have

�.h; g; ı�/ D �.h; g;2MXiD1

i Nıi/

�2MXiD1

i�.h; g; Nıi/ �2MXiD1

i N� D N�:

Thus, the maximum of �.h; g; ı/ in U is attained at one of Nı1; ; Nı2M :

Proof of Theorem 10.3. Since

hTRe˚‰.ej!; ı/

�g

D 1

D

M�1XmD0

.1C ım/ r;m .!;h; g/

D 1

D

M�1XmD0

r;m .!;h; g/C 1

D

M�1XmD0

ım r;m .!;h; g/ :

204 L. Jiang et al.

Hence,

maxı2U hTRe

˚‰.ej!; ı/

�g

D 1

D

M�1XmD0

r;m .!;h; g/C maxı2U

1

D

M�1XmD0

ım r;m .!;h; g/ :

For any ı D Œı0; ı1; ; ıM�1�T 2 U , we have

ˇˇ 1D

M�1XmD0

ım r;m .!;h; g/

ˇˇ

� 1

D

M�1XmD0

jımj j r;m .!;h; g/j

� 1

D

M�1XmD0

"m j r;m .!;h; g/j :

Hence,

maxı2U

1

D

M�1XmD0

ım r;m .w;h; g/ � 1

D

M�1XmD0

"m j r;m .!;h; g/j : (10.42)

On the other hand, taking Qı Dh Qı0; Qı1; ; QıM�1

iT; where Qım D "m r;m .!;h; g/ =

j r;m .!;h; g/j ; yields

1

D

M�1XmD0

Qım r;m .!;h; g/

D 1

D

M�1XmD0

"m . r;m .!;h; g//2 = j r;m .!;h; g/j

D 1

D

M�1XmD0

"m j r;m .!;h; g/j :

Combining (10.42) and (10.43), we obtain

maxı2U

1

D

M�1XmD0

ım r;m .!;h; g/ D 1

D

M�1XmD0

"m j r;m .!;h; g/j : (10.43)

Thus, (10.20) is obtained. The validity of (10.21)–(10.23) can be establishedsimilarly. Thus, the proof is complete.


References

Dam HH, Nordholm S, Cantoni A (2005) Uniform FIR filterbank optimization with group delayspecification. IEEE Trans Signal Process 53(11):4249–4260

de Haan JM, Grbic N, Claesson I, Nordholm S (2001) Design of oversampled uniform DFT filterbanks with delay specification using quadratic optimization. In: Proceedings of ICASSP’2001,Salt Lake City, pp 3633–3636

de Haan JM, Grbic N, Claesson I, Nordholm S (2003) Filter bank design for subband adaptivemicrophone arrays. IEEE Trans Speech Audio Process 11(1):14–23

Harteneck M, Weiss S, Stewart RW (1999) Design of near perfect reconstruction oversampled filterbanks for subband adaptive filters. IEEE Trans Circuits Syst 46:1081–1085

Kellermann W (1988) Analysis and design of multirate systems for cancellation of acoustic echoes.In: Proceedings of ICASSP’88, New York, pp 2570–2573

Kha HH, Tuan HD, Nguyen TQ (2009) Efficient design of cosine-modulated filter banks via convexoptimization. IEEE Trans Signal Process 57(3):966–976

Lopez M, Still G (2007) Semi-infinite programming. Eur J Oper Res 180:491–518Mansour MF (2007) On the optimization of oversampled DFT filter banks. IEEE Signal Process

Lett 14(6):389–392Nguyen TQ (1994) Near-perfect-reconstruction pseudo-QMF banks. IEEE Trans Signal Process

42(1):65–75Sturm JF (1999) Using SeDuMi 1.02, a Matlab toolbox for optimization over symmetric cones.

Optim Methods Softw 11–12:625–653Teo KL, Goh CJ (1988) On constrained optimization problems with nonsmooth cost functionals.

Appl Math Optim 18:181–190Vaidyanathan PP (1993) Multirate systems and filter banks. Prentice-Hall, Englewood CliffsWilbur MR, Davidson TN, Reilly JP (2004) Efficient design of oversampled NPR GDFT

filterbanks. IEEE Trans Signal Process 52(7):1947–1963Wu CZ, Teo KL (2010) A dual parametrization approach to Nyquist filter design. Signal Process

90:3128–3133Wu CZ, Teo KL (2011) Design of discrete Fourier transform modulated filter bank with sharp

transition band. IET Signal Process 5:433–440Wu CZ, Teo KL, Rehbock V, Dam HH (2008) Global optimum design of uniform FIR filter bank

with magnitude constraints. IEEE Trans Signal Process 56(11):5478–5486Wu CZ, Gao D, Teo KL (2013) A direct optimization method for low group delay FIR filter design.

Signal Process 93:1764–1772Yiu KFC, Grbic N, Nordholm S, Teo KL (2004) Multicriteria design of oversampled uniform DFT

filter banks. IEEE Signal Process Lett 11(6):541–544Zhang ZJ, Shui PL, Su T (2008) Efficient design of high-complexity cosine modulated filter banks

using 2Mth band conditions. IEEE Trans Signal Process 56(11):5414–5426

honglei˜xu˜· song˜wang soon-yi˜wu editors optimization

Documents