applying multiple objective tabu search to continuous optimization problems with a simple...

INTERNATIONAL JOURNAL FOR NUMERICAL METHODS IN ENGINEERINGInt. J. Numer. Meth. Engng 2006; 65:406–424Published online 1 September 2005 in Wiley InterScience (www.interscience.wiley.com). DOI: 10.1002/nme.1455

Applying multiple objective tabu search to continuousoptimization problems with a simple neighbourhood strategy

Adil Baykasoglu∗,†

Department of Industrial Engineering, University of Gaziantep, 27310 Gaziantep, Turkey

SUMMARY

One of the first multiple objective versions of the tabu search (TS) algorithm is proposed by the author.The idea of applying TS to multiple objective optimization is inspired from its solution structure.TS works with more than one solution (neighbourhood solutions) at a time and this situation givesthe opportunity to evaluate multiple objectives simultaneously in one run. The selection and updatingstages are modified to enable the original TS algorithm to work with more than one objective. In thispaper, the multiple objective tabu search (MOTS) algorithm is applied to multiple objective non-linearoptimization problems with continuous variables using a simple neighbourhood strategy. The algorithmis applied to four mechanical components design problems. The results are compared with severalother solution techniques including multiple objective genetic algorithms. It is observed that MOTSis able to find better and much wider spread of solutions than the reported ones. Copyright � 2005John Wiley & Sons, Ltd.

KEY WORDS: multiple objective optimization; tabu search; non-linear programming; design optim-ization

1. INTRODUCTION

Most real life problems involve multiple objectives. In the optimization literature, these problemsare known as ‘Multiple Objective Optimization’ (MOO) problems. Generally, a MOO problemis formally given as follows:

min or max F(X)

such that

X ∈ S = [X|X ∈ An, gi(X) � ai, hj (X) = bj ] i = 1, 2, . . . , m, j = 1, 2, . . . , n

(1)

∗Correspondence to: Adil Baykasoglu, Department of Industrial Engineering, University of Gaziantep, 27310Gaziantep, Turkey.

†E-mail: [email protected]

Received 18 February 2005Revised 25 May 2005

Copyright � 2005 John Wiley & Sons, Ltd. Accepted 22 July 2005

MULTIPLE OBJECTIVE TABU SEARCH 407

where X is an n-dimensional vector of the decision variables; F(X) = {f1(X), f2(X), . . . , fk(X)}is the set of objective functions; and S is the set of feasible solutions, bounded by m inequalityconstraints (gi) and n equality (hj ) constraints, ai and bj are constants. For continuous variablesA = �, for discrete variables A contains set of permissible values.

Objectives are generally conflicting in MOO problems. In general, existence of conflictingobjectives results in a number of optimal solutions. The concept of Pareto optimality is generallyused to characterize optimal solutions to a MOO problem. The Pareto-optimal (non-dominated)solution is defined as follows: a solution X∗ ∈ S is Pareto optimal if and only if there existsno X ∈ S such that fi(X) � fi(X

∗) for i = 1, 2, 3, . . . , k with fi(X) < fi(X∗) for at least one

value of i. In other words, the solution X∗ is Pareto optimal if no objective function can beimproved without worsening at least one other objective function. Since not one Pareto-optimalsolution can be said to be better than the other without further consideration, it is desired tofind as many such Pareto-optimal solutions as possible.

Unfortunately, classical techniques impose several limitations on solving mathematical pro-gramming models including the multiple objective ones [1–3]. The problem is mainly relatedto inherent solution mechanisms of these techniques. Their solution strategies are generallydependent on the type of objective and constraint functions (linear, non-linear, etc.) and thetype of variables used in the problem modelling (integer, real, etc.) [3], their efficiency is alsodependent on the size of the solution space, number of variables and constraints used in theproblem modelling and the structure of the solution space (convex, non-convex, etc.) [1, 2].They also do not offer a general solution strategy that can be applied to problem formulationswhere, different type of variables, objective and constraint functions are used [3]. For example,simplex Algorithm can be used to solve models with linear objective and constraint functions;geometric programming can be used to solve non-linear models with a polynomial or signomialobjective function, etc. However, most of the real life problems require different types of vari-ables, objective and constraint functions simultaneously in their formulation. Therefore, classicoptimization procedures are generally not adequate or easy to use for their solution [3, 4].

Researchers have spent a great deal of effort in order to adapt many engineering designproblems to the classic optimization procedures. Many examples can easily be found in theliterature [5]. It is not easy to formulate a real life problem that suits a specific solution proce-dure. In order to achieve this, it is necessary to make some modifications and/or assumptionson the original problem parameters (rounding variables, softening constraints, etc.) [3]. Thissituation certainly affects the solution quality. A new set of problem and model independentheuristic optimization techniques were proposed by researchers to overcome drawbacks of theclassical optimization procedures. These techniques are efficient and flexible [3, 4]. They canbe modified and/or adapted to suit specific problem requirements as portrayed in Figure 1.Three of these commonly accepted and applied techniques are known as Genetic Algorithms[6, 7], Tabu Search [8] and Simulated Annealing [9].

The idea of applying TS to MOO comes from its solution structure, in working with morethan one solution (neighbourhood solutions) at a time. In fact, any solution methodology thatworks with more than one solution vector at a time can be effectively used for MOO likegenetic algorithms [1]. Due to its population-based search characteristic, the genetic algorithmsare frequently applied to MOO problems. To enable the TS algorithm to work with morethan one objective, selection and updating stages of the basic TS are redefined. Other stagesare similar to the original tabu search algorithm. In contrast to original TS algorithm, MOTSalgorithm has two more lists in addition to the tabu list. The first one is the Pareto list,

Copyright � 2005 John Wiley & Sons, Ltd. Int. J. Numer. Meth. Engng 2006; 65:406–424

408 A. BAYKASOGLU

PROBLEMSOLUTION

TECHNIQUEModifications

PROBLEMSOLUTION

TECHNIQUEModifications

ClassicalOptimisationTechniques

ModernHeuristicOptimisationTechniques

Not easy

Easy

IF Linear Programming Model USE(Simplex);IF Integer Programming Model USE(Branch and Bound);...

Any model USE (Genetic Algorithms)Any Model USE (Tabu Search)Any Model USE (Simulated Annealing)...

Generalpurpose

Problemdependent

Figure 1. A pictorial comparison of classical and meta-heuristic optimization strategies [3].

which collects selected non-dominated solutions found by the algorithm. The second one isthe candidate list, which collects all other non-dominated solutions, which are not selected asPareto optimal solutions in the current iteration. These solutions may become seed solutions ifthey maintain their non-dominated status in later iterations. The candidates list plays also animportant role, it gives the opportunity to diversify the search [1, 3].

There are also several other approaches that applied TS to MOO. Hansen [10] proposed amethod that uses a set of solutions, each having its own tabu list. Weights are defined foreach solution to force search towards the non-dominated frontier. He applied his algorithm toknapsack and project scheduling problems. Gandibleux et al. [11] proposed a TS based MOOalgorithm for combinatorial optimization that is based on weighting and scalarizing functions.There are also some other recent attempts to apply TS to combinatorial MOO problems.Ehrgott and Gandibleux [12] gave an excellent survey of these approaches along with othermeta-heuristic approaches to combinatorial MOO. The present approach is entirely differentfrom these approaches in a number of ways. In the present approach, there is no need toapply a weighting scheme, the objective functions are not combined into a single function byusing some kind of utility functions. Moreover, in the present approach there is not any extraTS parameter to set for running the optimization procedure, this is not the case in the otherapproaches.

In this study, the focus is on applying MOTS to multiple objective non-linear optimizationproblems with continuous decision variables by using simple neighbour generation strategies.TS is generally applied to combinatorial and integer programming problems. Although someguidelines are given in the literature [13, 14] on the application of TS to continuous optimizationproblems, the performance of MOTS is not evaluated. In this paper, the efficiency of MOTS



with simple neighbourhood functions is investigated in finding diverse Pareto-optimal frontin a number of multiple objective mechanical design optimization problems with continuousvariables. The quality of solutions obtained from the MOTS is compared with the results ofseveral other techniques including genetic algorithms in the paper. It is observed that MOTSis eligible to solve problems with continuous decision variables.

2. AN OVERVIEW OF THE MOTS ALGORITHM

The elements of the MOTS algorithm for finding Pareto-optimal solutions in MOO problemsare defined as follows:

Initial solution: A randomly generated or feasible solution vector is initial solution.Generation of neighbourhood solutions: To generate neighbour solutions, the following simple

neighbour functions are used:

Integer variable x∗i = xi + integer[(2 ∗ random( ) − 1) ∗ stepii]

Zero-one variable x∗i =

{1 if xi = 00 if xi = 1

Discrete variable x∗i =d(l+integer[(2 ∗ random( )−1) ∗ stepdi ]) if xi = dl

Continuous variable x∗i = xi + (2 ∗ random( ) − 1) ∗ stepci

(2)

where xi is the value of the ith variable prior to the neighbourhood move, x∗i is the value of

the ith variable after the neighbourhood move. random( ): Random number generator, whererandom( ) ∈ (0, 1). stepii , stepdi , stepci : Step size for integer, discrete and real variables. dl isthe lth element of the discrete variable subset Xd . integer[ ] is the function to convert a realvalue to an integer value. According to the types of variables used in the model, the appropriatemovement strategies are used to generate a previously determined number of feasible, non-tabu, neighbourhood solutions from the current seed solution. Neighbourhood solutions mustalso be non-dominated by the current seed solution. It is also possible to use several othermore advanced strategies for neighbourhood generation, like variable neighbourhood strategy asdiscussed in Reference [15], path relinking as advised by Laguna and Marti [16] and adaptiveprocedures [17]. The implementation and testing of these strategies is beyond the scope ofthis paper. However, implementation of these algorithms within the proposed MOTS can be auseful research contribution in multiple objective optimization.

Selection of the seed solution: Selection of the seed solution is performed using the Paretooptimality logic (domination and non-domination). Pareto optimality is an economics termfor describing a solution for multiple objectives. It is generally used to characterize optimalsolutions to a MOO problem. The Pareto optimal (non-dominated) solution is defined as follows:a solution x∗ ∈ s is Pareto optimal if and only if there exists no x ∈ s such that fi(x) � fi(x

∗)for i = 1, 2, 3, . . . , m with fi(x) < fi(x

∗) for at least one value of i. In other words, thesolution x∗ is Pareto optimal if no objective function can be improved without worsening atleast one other objective function. Based on the Pareto optimality logic, the selection of thebest neighbourhood solution as the new seed solution is performed in the following manner:

(i) For each neighbourhood solution vector, the corresponding objective function values arecalculated. In the example given below, the neighbourhood size is three and there are two


410 A. BAYKASOGLU

continuous variables and two objective functions to be maximized.

3 Neighbourhood solutions CorrespondingSeed solution followed (non-tabu, feasible and not objective functionby objective dominated by the values offunction values seed solution) neighbourhood solutions

(6.3 6.1) → (60.08 47.09)(4.80 4.60)(52.40 40.93) → (6.0 6.0) → (58.79 46.54)

(6.4 6.0) → (60.39 46.86)Variable Objective Variable values Objective valuesvalues values

(ii) Candidate seed solutions within the neighbourhood solutions are identified. Candidateseed solutions should not be dominated by other neighbourhood solutions, solutions in thePareto list or solutions in the candidate list. This process is illustrated below.

Neighbourhood solutionsSeed solution followed (non-tabu, feasible and not Objective functionby objective dominated by the values offunction values seed solution) neighbourhood solutions

(6.3 6.1) → (60.08 47.09) ©(4.8 4.6) (52.4 40.93) → (6.0 6.0) → (58.79 46.54)

(6.4 6.0) → (60.39 46.86) ©Pareto list Candidate list

(0.0 0.0) (0 0) x (4.0 3.0) (46.93 33.98) x(0.5 0.5) (16.97 13.44) x (3.0 4.0) (42.64 36.93) x(1.0 1.0) (24 19) x(2.0 2.0) (33.94 26.87) x(3.0 3.0) (41.57 32.91) x(3.8 3.6) (46.57 36.26) x(4.8 4.6) (52.4 40.93)

(©: Candidate solutions, x: Eliminated solution from previous iterations)

(iii) One of the candidate solutions is randomly selected as the new seed solution. If thereare no candidate solutions in the current neighbourhood, the oldest solution from the candidatelist is selected as the seed solution.

It can be seen from the above selection strategy that the dominated solutions are not takeninto consideration, because the purpose is to find the Pareto optimal solutions, which do notdominate each other. MOTS algorithm works with two more dynamic lists namely Paretolist and Candidate list. The Candidate list (which collects potential candidate Pareto optimalsolutions and updates their status dynamically) enables the search process to avoid abandoningwhile searching and diversify the search. Pareto list collects the seed (or currently selected)potential Pareto optimal solutions and dynamically updates their status.



Updating the lists: The initial feasible solution is recorded as the first known Pareto solution.The solutions, which are dominated by any neighbourhood solution, are removed from bothPareto and candidate lists in each iteration. Then the seed solution is added to the Pareto list,and other candidate solutions are put into the candidate list. This process is shown on the sameexample below.

Pareto list Candidate list(0 0) (0 0) x (4 3) (46.93 33.98) x(0.5 0.5) (16.97 13.44) x (3 4) (42.64 36.93) x(1 1) (24 19) x (6.3 6.1) (60.08 47.39)(2 2) (33.94 26.87) x(3 3) (41.57 32.91) x(3.8 3.6) (46.57 36.26) x(4.8 4.6) (52.4 40.93) x

Selected seed solutions for an arbitrarily defined number of previous moves are considered astabu, since reusing one of them may trap the algorithm into cycling through recent moves.In the present algorithm, the tabu list contains m solutions, corresponding to the last m seedsolutions. The tabu list is circular, i.e. when it is full a new item replaces the head of the list.

Aspiration criteria: In combinatorial optimization problems, solution vectors are gener-ally generated indirectly using several features of the problem at hand. For example in amanufacturing cell formation problem a new solution can be generated by randomly reassign-ing part–machine pairs to different cells in each iteration in order to obtain a new solution[18]. In such a case instead of putting the whole solution vector in the tabu list as a tabusolution, only indices (features) of randomly selected and reassigned part–machine pairs are putinto the tabu list. However, reselection of these features for new solution generation in lateriterations might generate different solution vectors, as these features themselves are not thesolution vectors. Therefore, optimal solutions may be missed if these features are consideredstrictly as tabu. In order to prevent this situation in TS applications an aspiration criterionneeds to be defined to override the tabu status of features when necessary. However, if theentire solution vector is put into the tabu list, then it is not necessary to define an aspirationcriterion, which is generally the case in continuous optimization problems.

Termination: If a previously determined number of iterations is reached, or if the candidatelist is empty and the algorithm cannot find any new candidate solutions, the program terminates.The general flowchart of the algorithm is given in Figure 2.

2.1. Guidelines for the determination of tabu search parameters

One of the main drawbacks of the meta-heuristic algorithms including the MOTS is tuning thealgorithm parameters [12]. Unfortunately, there is no known optimal strategy for finding thebest possible combination of these parameters. Determination of these parameters is generallyproblem dependent. It is a common practice to solve the problem with different set of theseparameters to find a good combination of the parameter set. The present MOTS does not requireany additional parameter than the original TS algorithm in contrast to other TS approachesto MOO [12]. Tabu list size, number of iterations, neighbourhood size and step size forthe variables constitute tabu search parameters in the present MOTS algorithm. Within these


412 A. BAYKASOGLU

Start

Stop

Initial SolutionRandomly select a feasible solution s as the seed * Add s to the Tabu list and the Pareto list* Empty the candidate list

Neighbourhood GenerationApply variable-movement strategies to s to generate n feasible neighbours not belonging to the tabu list andnot dominated by the seed solution. If a solutionbelongs to the tabu list then for that solution check theaspiration criteria.

Identify Candidate SolutionsThe following conditions must be satisfied for aneighbourhood solution to become a candidate:* A neighbour solution must not be dominated by anyother neighbourhood solution* A neighbour solution must not be dominated by anysolutions in the Pareto list or the candidate list

Is there a candidate solutionin the neighbourhood?

Selection of new seed solutionselect the oldest solution from the candidate list as thenew seed

Add s to the Pareto and tabu lists, remove fromcandidate list

Selection of new seed solutionRandomly select a candidate solution from theneighbourhood to become the new seed s

Update taboo, Pareto and candidate lists* Eliminate all solutions from the Pareto and candidatelists which are dominated by any neighbourhoodsolution* Add s to the Pareto list and the tabu list* Add remaining candidate neighbours to the candidatelist

Is the stoppingcondition reached?

Print the Pareto list (i.e. Paretooptimal solutions have beenfound by the program)

Yes

No

Yes

No

Figure 2. The flowchart of the MOTS algorithm.



parameters, determination of step sizes and neighbourhood size for real and integer variables isparticularly important. If the range of variables are too wide and neighbourhood size is smalland the corresponding step sizes are chosen very small then computational time increasesconsiderably. With the same conditions, if step sizes are chosen too big then it may bepossible to miss the optimal solution. As guidance for setting up the tabu search parameters,the following rule can be used [3].

MOTS parameters setting rule: If the ranges of variables are too wide then use big stepsizes otherwise prefer smaller step sizes. If step sizes are big then increase the number ofneighbourhood solutions in each iteration, otherwise smaller number of neighbourhood solutionscan safely be used. Make sure that, converge criteria is satisfied before terminating, thereforenumber of iterations should be big enough to assure convergence.

This parameter-setting rule is used in the example problems presented below. It is observedthat promising solutions can be obtained by following the rule with some trials. In the experi-mental work, the effect of the tabu list size on solution quality was negligible. The parametersthat are given in the examples below are modified within the boundary of ±10 percent and nosignificant difference is observed in the solution quality.

3. TEST PROBLEMS

MOTS algorithm is applied to four non-linear continuous mechanical design optimization prob-lems. In each application, the algorithm successfully found good solutions in comparison to thereported solutions. The MOTS algorithm is programmed using C++. The program is an objectoriented one and uses advanced linked list constructs. The program is tested on a Pentium IVmodel PC at 1.60 GHz (256 MB RAM).

3.1. Multiple disc brake design

Osyczka and Kundu [19] used plain stochastic method and genetic algorithms (GA) to solvedesign optimization problem of a multiple disc brake. The problem contains a mixture ofcontinuous and integer variables, for details of the problem formulation, see Reference [19].The model is given as follows:

min f1(x) = 4.9 ∗ 10−5(x22 − x2

1 )(x4 − 1)

min f2(x) = (9.82 ∗ 106(x22 − x2

1 ))

(x3x4(x32 − x3

1))

min f3(x) = x3

s.t.

(x2 − x1) − 20 � 0

30 − 2.5(x4 + 1) � 0

0.4 − x3/3.14(x22 − x2

1 ) � 0


414 A. BAYKASOGLU

Table I. Comparison of the extreme points obtained from plain stochasticmethod, GA and MOTS.

Method Minima F(X) = [f1(x), f2(x), f3(x)]TPlain stochastic method min f1(x) [1.79, 2.77, 2920.9]

min f2(x) [3.76, 2.24, 2948.4]min f3(x) [3.25, 2.80, 2309.2]

Genetic algorithms (GA) min f1(x) [1.66, 2.87, 2982.4]min f2(x) [3.25, 2.11, 2988.3]min f3(x) [3.91, 2.86, 2255.1]

MOTS min f1(x) [0.131156, 41.3532, 1183.29]min f2(x) [2.16656, 2.15, 2981.64]min f3(x) [1.15309, 10.8508, 1000.03]

1 − 2.22 ∗ 10−3x3(x32 − x3

1)

(x22 − x2

1 )2� 0

2.66 ∗ 10−2x3x4(x32 − x3

1)

(x22 − x2

1 )− 900 � 0

55 � x1 � 80

75 � x2 � 110

1000 � x3 � 3000

2 � x4 � 20

x1, . . . , x3 ∈ �x4 integer (3)

Osyczka and Kundu [19] reported 19 Pareto optimal solutions for the multiple disc brakeusing the plain stochastic method, and 133 solutions with GAs method after 20 000 evaluations.For solving the multiple disc brake problem the MOTS algorithm parameters are set as follows:neighbourhood size = 20, tabu list size = 20, step size for real variables 0.01, step size forinteger variable = 1 found 5964 Pareto optimal solutions after 20 000 evaluations around 10 minexecution time. Osyczka and Kundu [19] also reported the extreme points (i.e. minimum valuepoints for each separate criterion) obtained from both methods in their paper. Comparison ofthe extreme points obtained by three methods is shown in Table I.

Clearly, MOTS generates better extreme points with much more Pareto optimal solutions.The trade-off surface obtained from the MOTS for multiple disc brake problem is shown inFigure 3. The extreme points are also shown in Figure 4. The extreme points obtained fromMOTS dominate the plain stochastic method and are much better than GA. The spread of theMOTS solutions are also much better than GA and plain stochastic method as it is shownin Figure 4.



Figure 3. MOTS solutions for multiple disc brake design problem.

Figure 4. Comparison of the extreme points (circle for MOTS, rectangle forplain stochastic method and line for GA).

3.2. Two bar truss design

The two bar truss design problem was originally studied by Deb et al. [20] using the ∈-constraint method. The truss has to carry a certain load without elastic failure (see Figure 5).


416 A. BAYKASOGLU

100 kN

4 m 1 m

y

A

C

B

x1x2

Figure 5. Two-bar truss design.

Thus, in addition to the objective of designing the truss with minimum volume, there areadditional objectives of minimizing stresses in each of the two members AC and BC. The fol-lowing two-objective optimization problem with three continuous variables, y (vertical distancebetween B and C in m), x1 (length of AC in m) and x2 (length of BC in m) is constructed byDeb et al. [20].

min f1(x) = x1

√16 + y2 + x2

√1 + y2

min f2(x) = max(�AC, �BC)

s.t.

max(�AC, �BC) � 1 ∗ 105

1 � y � 3

x1, x2, y � 0

x1, x2, y ∈ �

�AC = 20√

16 + y2

yx1

�BC = 80√

1 + y2

yx2

(4)

For solving the two bar truss design problem, MOTS parameters are set as follows: neigh-bourhood size = 150, tabu list size = 20, step sizes for the first, second, and third continuousvariables are taken as 2, 0.01, 0.01, maximum number of iterations is set to 5000. MOTS found651 Pareto optimal solutions after 1562 iterations in 18 s by using this parameter set. Paretooptimal solutions are shown in Figure 6. The comparison of the best values of objectives



0

20000

40000

60000

80000

100000

120000

0 0.030.020.01 0.04 0.05 0.06

f1(x)

f2(x

)

Figure 6. MOTS solutions for two-bar truss problem.

obtained from MOTS and other techniques presents that solutions of MOTS are not domi-nated. Moreover, MOTS found much more Pareto-optimal solutions than the other techniques(∈-constraint method [21] and Deb et al.’s GA-1 and GA-2 [20]), and the spread of the MOTSsolutions are much wider than the ∈-constraint method see Reference [20]. The solutions ofMOTS are spread in the following range {0.005902 m3, 99 757 kPa}, {0.056623 m3, 8432 kPa}.Whereas the spread of solutions of Deb et al.’s GA-2 [20] are in the {0.00407 m3, 99 755 kPa},{0.05304 m3, 8439 kPa} range. The spread of the solutions of ∈-constraint method [21] arein the {0.004445 m3, 89 983 kPa}, {0.004833 m3, 83 268 kPa} range. If the minimization of thestress is important, MOTS finds a solution with stress as low as 8432 kPa, whereas the GA-2of Deb et al. [20] has found 8439 kPa, ∈-constraint method has found 83 268 kPa. As a result,the quality of solution obtained by MOTS and GA-2 of Deb are nearly equal. However, MOTSfound much more Pareto-optimal solutions in the Pareto-optimal front. Both MOTS and GA-2dominated ∈-constraint method in terms of solution quality and number of Pareto-optimalsolutions found. Deb’s GA-2 [20] found 93 Pareto optimal solutions, Palli et al.’s ∈-constraintmethod [21] found five Pareto optimal solutions, MOTS found 651 Pareto optimal solutions.The extreme points (spread of the solutions) that are obtained from the three methods areshown in Figure 7. As it can be seen from Figure 7 MOTS generated the widest spread ofthe solutions in the Pareto frontier. The spread of the solutions obtained from the GA is alsocomparable to MOTS. However, the ∈-constraint method is not able to cover the Pareto optimalfront successfully.

3.3. Machine tool spindle design

Eschenauer et al. [22] modelled the machine tool spindle problem (see Figure 8). Coello [23]remodelled this problem as a MOO problem and proposed a genetic algorithm (MOSES) forits solutions. They also compared their results with four other MOO techniques with respectto best results obtained for each objective function. In this paper MOTS is used to solve the


418 A. BAYKASOGLU

0.00 0.01 0.02 0.03 0.04 0.05 0.06f1(x)

0

10000

20000

30000

40000

50000

60000

70000

80000

90000

100000

f2(x)

Figure 7. Comparison of the extreme points (rectangle for MOTS, circlefor ∈-constraint method and line for GA).

cb

caF

dodbda

l a

Figure 8. Sketch of the machine tool spindle.

problem. The model is given in Equation (5).

min f1 = �

4[a(d2

a − d2o ) + l(d2

b − d2o )]

min f2 = Fa3

3EIa

(1 + l

a

Ia

Ib

)+ F

ca

[(1 + a

l

)2 + caa2

cbl2

]

Ia = 0.049(d4a − d4

0 )

Ib = 0.049(d4b − d4

0 )

ca = 35400|�ra|1/9d10/9a

cb = 35400|�rb|1/9d10/9b



0

0.005

0.01

0.015

0.02

0.025

0.03

0.035

0.04

0 500000 1000000 1500000 2000000

f1

f2

Figure 9. MOTS solutions for machine spindle design problem.

s.t.l − lg � 0

lk − l � 0

da1 − da � 0

da − da2 � 0

db1 − db � 0

db − db2 � 0

dom − do � 0

p1do − db � 0

p2db − da � 0∣∣∣�a + (�a − �b)a

l

∣∣∣ − � � 0

do, l � 0

do, l ∈ �da, db discrete (5)

In the machine spindle design model da and db are discrete variables and da should beselected from the following set {80, 85, 90, 95}, and db from the set {75, 80, 85, 90}. For solvingthe spindle design problem, MOTS parameters are set as follows: neighbourhood size = 10, tabulist size = 20, step sizes for the first (do), second (l), third (da) and fourth (db) variables aretaken as 5, 5, 4, 4 maximum number of iterations is set to 1000. The second and third variablesare continuous. MOTS found 128 Pareto optimal solutions after 348 iterations in 9 s by usingthis parameter set. The Pareto optimal solutions are shown in Figure 9. The comparison ofthe best values of the objectives obtained from the MOTS and other techniques reported in


420 A. BAYKASOGLU

Table II. Comparison of the best results for eachobjective of the machine spindle design problem.

Techniques f1(x) f2(x)

Monte Carlo 606765.47 0.032463Monte Carlo 1457748.36 0.019242GA(binary) 494015.44 0.038087GA(binary) 1643777.68 0.016613GA(floating point) 1124409.37 0.017951GA(floating point) 1637052.38 0.016615Literature 531183.70 0.030215Literature 694200.03 0.023101MOTS 497644.1 0.037839MOTS 1485169 0.016894

For each method the best results for f1(x) and f2(x) areshown in boldface.

0 500000 1000000 1500000 2000000

f1(x)

0.01

0.02

0.03

0.04

f2(x)

Figure 10. Comparison of the extreme points (circle for Monte Carlo, rectangle for literature, cross forMOTS, horizontal line for GA-binary, vertical line for GA-floating point).

Reference [23] presents that solutions of MOTS are not dominated. These comparisons are alsoshown in Table II and Figure 10. As it can be seen from Figure 10 MOTS is able to find agood range of Pareto optimal solutions. The figure also presents that solutions obtained fromthe Monte Carlo method are dominated by the other techniques.

3.4. I -beam design

Osyczka [24] modelled the design optimization problem for a simply supported I-beam that isshown in Figure 11. Coello and Christiansen [25] remodelled the problem as MOO problemand solved the model using genetic algorithm, which is known as MOSES. They also made



x4

x1x3

x2

L=200

L/2

P

Q

Y

Z

Figure 11. Sketch of the simply supported I-beam.

an extensive comparison of their results with some other MOO techniques with respect to bestresults obtained for each objective function. The mathematical model is given in Equation (6).

min f1(x) = 2x2x4 + x3(x1 − 2x4)

min f2(x) = 60 000

x3(x1 − 2x4)3 + 2x2x4(4x24 + 3x1(x1 − 2x4))

s.t.

16 − 180 000x1

x3(x1 − 2x4)3 + 2x2x4(4x24 + 3x1(x1 − 2x4))

− 15 000x2

(x1 − 2x4)x33 + 2x4x

32

� 0

10 � x1 � 80

10 � x2 � 50

0.9 � x3 � 5

0.9 � x4 � 5

x1, x2, x3, x4 � 0

x1, x2, x3, x4 ∈ �

(6)

MOTS is applied to I-beam design problem with the following parameter set: neighbourhoodsize = 10, tabu list size = 20, step sizes for the first, second, third and fourth continuous variablesare taken as 5, 5, 2, 2, maximum number of iterations is set to 1000. Using this parameter setMOTS found 92 solutions after 178 iterations in 8 seconds. The Pareto optimal solutions areshown in Figure 12. The comparison of the best values of objectives obtained from MOTS and


422 A. BAYKASOGLU

0

0.005

0.01

0.015

0.02

0.025

0.03

0.035

0.04

0 100 200 300 400 500 600 700 800

f1(x)

f2(x

)

Figure 12. MOTS solutions for I-beam design problem.

Table III. Comparison of the best results for eachobjective of the machine spindle design problem.

Techniques f1(x) f2(x)

Monte Carlo 188.65 0.06175Monte Carlo 555.22 0.00849Min–Max 316.85 0.01697Min–Max 326.49 0.01636GA(binary) 128.27 0.05241GA(binary) 848.41 0.00591GA(floating point) 127.46 0.06034GA(floating point) 850 0.0059Literature 128.47 0.06Literature 850 0.0059MOTS 143.52 0.037MOTS 678.21 0.00664

For each method the best results for f1(x) and f2(x) areshown in boldface.

other techniques reported in Reference [25] presents that solutions obtained from the MOTSare not dominated. These comparisons are also shown in Table III and Figure 13. The extremepoints obtained from the Monte Carlo method are dominated with techniques.

4. CONCLUSIONS

Tabu search is generally applied to combinatorial and integer programming problems. Its appli-cation to continuous optimization problems is limited. In this research paper, an effort is madeto present the efficiency of the MOTS algorithm on solving design optimization problems withcontinuous decision variables. Four multiple objective design optimization problems with



100 200 300 400 500 600 700 800 900f1(x)

0.00

0.01

0.02

0.03

0.04

0.05

0.06

0.07

f2(x)

Figure 13. Comparison of the extreme points (circle for Monte Carlo, rectangle for MOTS, tilted linefor literature, vertical line for GA-binary, horizontal line for min–max, cross for GA-floating point).

continuous variables are solved with the MOTS algorithm. The performance of the MOTS onthese problems is compared with other classical techniques and GAs. It is found out that MOTSwith a simple neighbourhood function is able to find many good-quality Pareto optimal solutionsin comparison to compared algorithms. The range of Pareto optimal front is also as good as thereported algorithms. It is concluded that the MOTS can also be applied to multiple objectiveoptimization problems with continuous variables. The performance of the present algorithmscan also be improved by using more advanced neighbourhood functions and adaptive strategies.In the present MOTS algorithm design optimization problems up to three objective functionsare studied. There are some discussions in the literature to test meta-heuristic algorithmsincluding MOTS with more than three objectives in order to prove their suitability in higherorder dimensions [12, 26]. Testing the present MOTS with more than three objective functionscan be a valuable future research.

REFERENCES

1. Baykasoglu A, Owen S, Gindy N. A tabu search based approach to find the Pareto optimal set in multipleobjective optimization. Journal of Engineering Optimization 1999; 31:731–748.

2. Baykasoglu A, Owen S, Gindy N. Solution of goal programming models using a basic taboo searchalgorithm. Journal of the Operational Research Society 1999; 50:960–973.

3. Baykasoglu A. Goal programming using the multiple objective tabu search. Journal of the OperationalResearch Society 2001; 52:1359–1369.

4. Dhingra AK, Lee BH. A genetic algorithm approach to single and multiobjective structural optimizationwith discrete-continuous variables. International Journal for Numerical Methods in Engineering 1994; 37:4059–4080.

5. Sonmez AI, Baykasoglu A, Dereli T, Filiz IH. Dynamic optimization of multipass milling operations viageometric programming. International Journal of Machine Tools and Manufacture 1999; 39:297–320.

6. Holland JH. Genetic algorithms. Scientific American 1992; 267(1):44–50.


424 A. BAYKASOGLU

7. Goldberg DE. Genetic Algorithms in Search, Optimization and Machine Learning. Addison-Wesley: Reading,MA, 1989.

8. Glover F. Tabu search: a tutorial. Interfaces 1990; 20:74–94.9. Kirkpatrick S, Gelatt Jr CD, Vecchi MP. Optimisation by simulated annealing. Science 1983; 220:671–680.

10. Hansen MP. Tabu search for multiobjective combinatorial optimization: TAMOCO. Control and Cybernetics2000; 29:799–818.

11. Gandibleux X, Mezdaoui N, Freville A. A tabu search procedure to solve multiobjective combinatorialoptimization problems. In Advances in Multiple Objective and Goal Programming, Caballero R, Ruiz F,Steuer R (eds). Lecture Notes in Economics and Mathematical Systems, vol. 455. Springer: Berlin, 1997;291–300.

12. Ehrgott M, Gandibleux X. Approximate solution methods for multiobjective combinatorial optimization. Top2004; 12:1–89.

13. Siarry P, Berthiau G. Fitting of tabu search to optimize functions of continuous variables. InternationalJournal for Numerical Methods in Engineering 1997; 40:2449–2457.

14. Franze F, Speciale N. A tabu search algorithm for continuous multiminima problems. International Journalfor Numerical Methods in Engineering 2001; 50:665–680.

15. Mladenovic N, Hansen P. Variable neighbourhood search. Computers and Operations Research 1997; 24:1097–1100.

16. Laguna M, Marti R. GRASP and path relinking for 2-layer straight line crossing minimization. INFORMSJournal on Computing 1999; 11:44–52.

17. Resende MGC, Ribeiro CC. Greedy randomized adaptive search procedures. In Handbook of Metaheuristics,Glover F, Kochenberger G (eds). Kluwer Academic Publishers: Dordrecht, 2003; 219–249.

18. Baykasoglu A, Gindy NNZ. MOCACEF 1.0: capability based approach to form part-machine groups forcellular manufacturing applications. International Journal of Production Research 2000; 38:1133–1161.

19. Osyczka A, Kundu S. A modified distance method for multicriteria optimization using genetic algorithms.Computers and Industrial Engineering 1996; 30:871–882.

20. Deb K, Patrap A, Moitra S. Mechanical component design for multi-objective using Elitist non-dominatedsorting GA. KanGAL Report 200002, Indian Institute of Technology Kanpur, India, 2000.

21. Palli N, Azram S, McCluskey P, Sundararajan R. An interactive multistage ∈-inequality constraint methodfor multiple objectives decision making. Journal of Mechanical Design (ASME) 1999; 120:678–686.

22. Eschenauer H, Koski J, Osyczka A. Multicriteria Design Optimization. Springer: Berlin, 1990.23. Coello CCA. An empirical study of evolutionary techniques for multiobjective optimization in engineering

design. Ph.D. Thesis, Department of Computer Science, Tulane University, New Orleans, LA, 1996.24. Osyczka A. Multicriteria optimization for engineering design. In Design Optimization, Gero JS (ed.). Academic

Press: New York, 1985; 193–227.25. Coello CCA, Christiansen AD. MOSES: a multiple objective optimization tool for engineering design. Journal

of Engineering Optimization 1999; 31:337–368.26. Marett R, Wright M. A comparison of neighbourhood search techniques for multi-objective combinatorial

problems. Computers and Operations Research 1996; 23:465–483.


applying multiple objective tabu search to continuous optimization problems with a simple...

Documents