This article was downloaded by: [University North Carolina - Chapel Hill]On: 29 October 2014, At: 08:21Publisher: Taylor & FrancisInforma Ltd Registered in England and Wales Registered Number: 1072954Registered office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK
International Journal ofMathematical Education in Scienceand TechnologyPublication details, including instructions for authors andsubscription information:http://www.tandfonline.com/loi/tmes20
Nonlinear optimization ofconstrained functions using tabusearchJ.A. Bland aa Department of Mathematics, Statistics and OperationalResearch , The Nottingham Trent University, Burton Street ,Nottingham NG1 4BU, EnglandPublished online: 09 Jul 2006.
To cite this article: J.A. Bland (1993) Nonlinear optimization of constrained functions usingtabu search, International Journal of Mathematical Education in Science and Technology, 24:5,741-747, DOI: 10.1080/0020739930240515
To link to this article: http://dx.doi.org/10.1080/0020739930240515
PLEASE SCROLL DOWN FOR ARTICLE
Taylor & Francis makes every effort to ensure the accuracy of all the information(the “Content”) contained in the publications on our platform. However, Taylor& Francis, our agents, and our licensors make no representations or warrantieswhatsoever as to the accuracy, completeness, or suitability for any purpose of theContent. Any opinions and views expressed in this publication are the opinions andviews of the authors, and are not the views of or endorsed by Taylor & Francis. Theaccuracy of the Content should not be relied upon and should be independentlyverified with primary sources of information. Taylor and Francis shall not be liablefor any losses, actions, claims, proceedings, demands, costs, expenses, damages,and other liabilities whatsoever or howsoever caused arising directly or indirectly inconnection with, in relation to or arising out of the use of the Content.
This article may be used for research, teaching, and private study purposes. Anysubstantial or systematic reproduction, redistribution, reselling, loan, sub-licensing,systematic supply, or distribution in any form to anyone is expressly forbidden.
Terms & Conditions of access and use can be found at http://www.tandfonline.com/page/terms-and-conditions
Dow
nloa
ded
by [
Uni
vers
ity N
orth
Car
olin
a -
Cha
pel H
ill]
at 0
8:21
29
Oct
ober
201
4
INT. J. MATH. EDUC. SCI. TECHNOL., 1993, VOL. 24, NO. 5, 741-747
Nonlinear optimization of constrained functionsusing tabu search
by J. A. BLANDDepartment of Mathematics, Statistics and Operational Research,
The Nottingham Trent University, Burton Street,Nottingham NG1 4BU, England
(Received 20 July 1992)
A direct search algorithm is presented which is based on the tabu searchmethodology. It is shown how the algorithm may be used to solve constrainednonlinear optimization problems. The major merit of the presented approach isthe ability of the algorithm to escape local optima in the search for a globaloptimim.
1. IntroductionThis article is concerned with the optimization of functions with constraints. An
algorithm is presented that is quite general in the sense that it is applicable to bothlinear and nonlinear functions of an arbitrary number of independent variableswhich may be continuous or discrete. Furthermore, it is flexible enough to cope •withboth linear and nonlinear constraints.
Over the past few decades many optimization methods have been proposed, forreviews see references [1] and [2]. These methods, together with the ever increasingspeed and capacity of electronic computers, have enabled large and complexoptimization problems to be solved. In general, proposed optimization procedureshave fallen into two broad classes; search algorithms, which utilize functionevaluations only [3,4], and gradient-based techniques which require both functionand derivative information [5,6].
In this article a direct search algorithm is presented which adapts the tabu searchmethodology [7,8] to optimize constrained nonlinear functions.
2. Tabu searchOver the past few years the tabu search optimization procedure has grown
in popularity although, in general, it has been applied to combinatorial problems[9-11]. In this article it is applied to functions with independent variables that maybe continuous, or, if they are discrete, need not take integer values.
The tabu search methodology has been fully described (in a combinatorialcontext) in [10] and [12]; here a brief outline of the algorithm is given in terms ofunconstrained minimization.
The basic operation of the tabu search procedure is similar to a simple descentalgorithm in the sense that solutions with ever decreasing objective function valuesare continually accepted as the current minimum solution, until no furtherimprovements (i.e. reductions) are obtained. However, tabu search incorporates ashort-term memory of forbidden (or tabu) solutions that enables the procedure to
0020-739X/93 $10-00 © 1993 Taylor & Francis Ltd.
Dow
nloa
ded
by [
Uni
vers
ity N
orth
Car
olin
a -
Cha
pel H
ill]
at 0
8:21
29
Oct
ober
201
4
742 J. A. Bland
escape from local minima, which usually cause simple descent algorithms toterminate.
The short-term memory facility is a characteristic of tabu search and consists of alist (called the tabu list) of arbitrary length whose elements are previously acceptedsolutions. In the present context an accepted solution is one whose objective functionvalue is the smallest of those associated with a defined neighbourhood of solutions,and it is not currently in the tabu list.
The tabu list operates as a first in-first out queue of accepted solutions whichenables the algorithm to continue searching while retaining knowledge of the currentminimum solution. In this manner, tabu search with a tabu list of length L, avoidsboth entrapment by local minima and cycling (cycles of order L, at least) throughpreviously accepted solutions.
3. Constrained minimizationA basic algorithm which adapts the tabu search methodology to the minimization
of functions whose independent variables are continuous (or non-integer discrete) isnow presented.
Consider a function/of n independent variables, x, where x={xt}, i = l, 2 , . . .n.Ifxo = {xOi}, i = l , 2 , . . . « , is an initial solution in n-dimensional space then/ 0 is thefunction value, where/0=/(a;0). In the framework of the tabu search technique x0
becomes the first element of an initially empty tabu list. Furthermore, the currentminimum solution and function value, arbest and/best, respectively, are such that xbest
= x0 and / b e s t =/ 0 . Next a neighbourhood of x0 is constructed by considering thesolutions that are generated when each coordinate of x0 is, in turn, slightly displaced(positively and negatively) by a specified amount; xoi±hh i = 1,2,. . . n, where ht areconstants. At each neighbourhood solution the function is evaluated and a move ismade from the initial solution to the neighbourhood solution with the smallestfunction value (ties are broken arbitrarily). If xt denotes the neighbourhoodsolution with the smallest function value, flt -where f1—f(x1), then the searchprogresses from x0 to xt and xt enters the tabu list. If / t < / 0 then the currentminimum solution and function value are updated; *best = ;x;i ar>d/best = / i > respec-tively. Note that if / t > / 0 no updating takes place but the search, nonetheless,progresses to x1 even though, with strict inequality, x0 is a local minimum.Furthermore, at the next move, the search cannot return to x0 because it is in the tabulist of forbidden solutions.
In general terms, a move from solution xt to xt + j is based on/j + j being the lowestfunction value in the neighbourhood of x( and that xi+1 is not in the tabu list. As thesearch progresses the tabu list becomes full, after which its length is maintained byreleasing the least recent solution when the current solution is entered. In thismanner the tabu list provides the search with the capability to escape local minimaand avoid cycling through solution cycles of order less than or equal to the list length.Because of its ability to avoid termination at a local minimum tabu search willprogress continually through n-dimensional space in the search for a globalminimum; hence a stopping condition, such as a specified maximum number ofmoves, is required.
The preceeding explanation gives an outline of a basic (unconstrained) tabusearch algorithm which may be enhanced in the following ways. First, the searchmay incorporate a second sweep, where a sweep consists of a specified number ofmoves. If the tabu list length is greater than the number of moves in a sweep and the
Dow
nloa
ded
by [
Uni
vers
ity N
orth
Car
olin
a -
Cha
pel H
ill]
at 0
8:21
29
Oct
ober
201
4
Nonlinear optimization using tabu search 743
search in the second sweep starts at xbest, then the search is forced to investigate newsolutions. A second enhancement is the incorporation of sweeps within a zoomfacility, whereby, after the sweep phase, the search starts at xbest, but some (or all) ofthe step lengths, hh are reduced by setting h—hjcj, where c{^\,i—1,2,.. . w. Hence,by using a sequence of zooms which contain sweeps, the presented algorithm has, bymeans of the tabu search methodology, the capability to determine a globallyminimum solution of a function with continuous and/or discrete independentvariables.
With k constraints the optimization (here minimization) problem may be statedin general terms as follows,
minimize f(x) (3.1)
subject to gi(x)~^0 i '=l,2, ...k (3.2)
where objective function,/(x), and constraint functions, gi(x), i= 1,2.. .k, may belinear or nonlinear functions of n independent variables, x. In order to utilize thepresented algorithm a new objective function, F(x), is formulated, where
F(x)=f(x)+ £ a,ft(:e) (3.3)
with0 iigi(x)>0 (3.4)N ifg:(x)<0 (3.5)
Note that, with N assigned to be a suitably large negative number, the second termon the right-hand side of equation (3.3) acts as a penalty function which contributessignificantly (i.e. overwhelmingly) to the value of F(x) when one or more constraintsare violated. Hence the optimization problem now becomes the unconstrainedminimization of F(x), which achieves the aim of minimizing f(x) subject togi(x)'^0,
4. Example
As an example, consider the following constrained optimization problem:
minimize
x\ + 2^1-0-3 cos(3;rx1)-0-4cos(47K<;2) + 0-7 (4.1)subject to
x2<l-0-75sin(^x1) (4.2)
( * , - l ) 2 + ;c|>(0-75)2 (4.3)
(4.4)
In other words, the objective function is
/(*) = *?+ 2x!-0-3cos(37rx,)-0-4cos(47rx2) + 0-7 (4.5)
and the constraints are
(4.6)
(4.7)
(4.8)
Dow
nloa
ded
by [
Uni
vers
ity N
orth
Car
olin
a -
Cha
pel H
ill]
at 0
8:21
29
Oct
ober
201
4
744 J. A. Bland
Figure 1. Contours of equation (4.5) in the region — 1 ̂ * t < 1, — 1 «S x2
Figure 2. Contours of equation (4.5) with constraints (4.6), (4.7) and (4.8) in the region
Dow
nloa
ded
by [
Uni
vers
ity N
orth
Car
olin
a -
Cha
pel H
ill]
at 0
8:21
29
Oct
ober
201
4
Nonlinear optimization using tabu search 745
Hence the (unconstrained) objective function F(x), given by equation (3.3), may beconstructed with k = 3. The value of N in equation (3.5) was taken to be —10000.The (constrained) objective function f(x), given by equation (4.5), is investigatedbecause it has a number of local minima (the global minimum is at the origin, whereF(0,0)=/(0,0) = 0). Furthermore, since f(x) has two independent variables itscontours may be visualized, as shown in Figure 1 (without constraints) and Figure 2(with constraints (4.6), (4.7) and (4.8) included).
In order to ensure (for purposes of illustration) that the search encountered localminima, a step length of h = 05 was used with each variable. The results of the searchwhen various starting points, x0, are used is given in Table 1, where M denotes thenumber of moves the search took to encounter the global minimum. In all cases thetabu list length was set to the same value as the maximum number of moves, i.e. 150.
Table 1. Tabu search results.
(1,1)(1,-1)(-1,-1)(-1,1)(0,1)(1,0)(0,-1)(-1,0)
F(*o)
3-610003-610003-6
3-62-01-620
5626-6
M
100120848617
so8820
Figure 3. Search path with xo = (— 1, 1).
Dow
nloa
ded
by [
Uni
vers
ity N
orth
Car
olin
a -
Cha
pel H
ill]
at 0
8:21
29
Oct
ober
201
4
746 J. A. Bland
Figure 4. Search path with 3CO = (O, —1).
To illustrate the capability of the search to escape local minima the search pathfor starting points at ( — 1,1) and (0,-1) are shown in Figure 3 and Figure 4,respectively.
5. Concluding remarksIn this article a direct search algorithm is presented which may be used to
optimize a given function subject to constraints. The presented algorithm is based onthe tabu search methodology which was originally developed to solve (un-constrained) combinatorial problems.
The presented optimization method is quite general in the sense that it may beapplied to both linear and nonlinear functions of an arbitrary number of variables(which may be continuous or discrete). Moreover it is flexible enough to accommo-date both linear and nonlinear constraints.
As stated in the previous section, with respect to the example problem, the globalminimum was obtained with all starting points (not all of them feasible). Fur-thermore, the search path was visualized in two instances, Figure 3 and Figure 4.These two diagrams clearly show the ability of the algorithm to escape local minima;a characteristic of tabu search.
References[1] POWELL, M. J. D., 1970, Asurvey of numerical methods for unconstrained optimization,
SI AM Rev., 12,79-91.[2] Box, M. J., 1966, A comparison of several current optimisation methods, and the use of
transformations in constrained problems, The Computer J, 9, 67-77.[3] HOOKE, R., and JEEVES, T. A., 1961, Direct search solution of numerical and statistical
problems, J. Assn. Comp. Mach., 8, 212-229.
Dow
nloa
ded
by [
Uni
vers
ity N
orth
Car
olin
a -
Cha
pel H
ill]
at 0
8:21
29
Oct
ober
201
4
Nonlinear optimization using tabu search 747
[4] NELDER, J. A., and MEAD, R., 1965, A simplex method for function minimisation, TheComputer J., 7, 308-313.
[5] FLETCHER, R., and REEVES, C. M., 1964, Function minimisation by conjugate gradients,The Computer J., 7, 149-154.
[6] POWELL, M. J. D., 1971, On the convergence of the variable metric algorithm, J. Inst.Math. AppL, 7, 21-36.
[7] GLOVER, F., 1977, Heuristics for integer programming using surrogate constraints,Decision Sci., 8, 156-166.
[8] GLOVER, F., 1986, Future paths for integer programming and links to artificialintelligence, Comput. Operations Res., 13, 533-549.
[9] KNOX, J. E., 1989, The application of tabu search to the symmetric travelling salesmanproblem, PhD. Thesis, University of Colorado, USA.
[10] BLAND, J. A., and DAWSON, G. P., 1991, Tabu search and design optimisation,Computer-Aided Des., 23, 195-201.
[11] SKORIN-KARPOV, J., 1990, Tabu search applied to the quadratic assignment problem,ORSA J. Comp., 2, 33–45.
[12] GLOVER, F., 1989, Tabu search; Part I, ORSA J. Comp., 1, 190-206.
Dow
nloa
ded
by [
Uni
vers
ity N
orth
Car
olin
a -
Cha
pel H
ill]
at 0
8:21
29
Oct
ober
201
4
