Download - A hybrid approach using TOPSIS, Differential Evolution, and Tabu Search to find multiple solutions of constrained non-linear integer optimization problems

1

3

4

5

6

7

8 Q1

91011

12

1 4

1516171819

2021 Q322232425

2 6

40

41

42

43

44

45

46

47

48

49

50

51

52

53

54

55

56

57

58

59

Q2

Knowledge-Based Systems xxx (2014) xxx–xxx

KNOSYS 2761 No. of Pages 9, Model 5G

18 March 2014

Contents lists available at ScienceDirect

Knowledge-Based Systems

journal homepage: www.elsevier .com/ locate /knosys

A hybrid approach using TOPSIS, Differential Evolution, and Tabu Searchto find multiple solutions of constrained non-linear integer optimizationproblems

http://dx.doi.org/10.1016/j.knosys.2014.02.0150950-7051/� 2014 Elsevier B.V. All rights reserved.

⇑ Corresponding author. Tel.: +55 2799475139.E-mail addresses: [email protected] (E.R.F.A. Schneider), krohling.renato@

gmail.com (R.A. Krohling).

Please cite this article in press as: E.R.F.A. Schneider, R.A. Krohling, A hybrid approach using TOPSIS, Differential Evolution, and Tabu Search to find msolutions of constrained non-linear integer optimization problems, Knowl. Based Syst. (2014), http://dx.doi.org/10.1016/j.knosys.2014.02.015

Erick R.F.A. Schneider a, Renato A. Krohling b,⇑a Graduate Program in Computer Science, PPGI, UFES – Federal University of Espírito Santo, Av. Fernando Ferrari, 514, CEP 29075-910 Vitória, Espírito Santo, ES, Brazilb Department of Production Engineering & Graduate Program in Computer Science, PPGI, UFES – Federal University of Espírito Santo, Av. Fernando Ferrari, 514, CEP 29075-910 Vitória,Espírito Santo, ES, Brazil

a r t i c l e i n f o a b s t r a c t

2728293031323334353637

Article history:Received 1 April 2013Received in revised form 19 February 2014Accepted 25 February 2014Available online xxxx

Keywords:Constrained non-linear integer optimizationDifferential EvolutionTabu SearchTOPSIS

38

This paper presents a novel method to find multiple solutions of multi-modal constrained non-linearinteger optimization problems. First, the constrained optimization problem is cast into a bi-objectiveoptimization problem, where the constraints are inserted as another objective function. Next, the novelmethod to solve multi-objective optimization problems is developed and applied to solve the reformu-lated problem. The novel method developed to solve multi-objective optimization problems is basedon the Technique for Order Preference by Similarity to Ideal Solution (TOPSIS) whereas the multi-objec-tive problem is cast in single-objectives problems. The Differential Evolution (DE) algorithm in its threeversions (standard DE, DEbest and DEGL) are used as optimizer. Since the solutions found by the DE algo-rithms are continuous, a Tabu Searh (TS) is employed to find integer solutions during the optimizationprocess. Experimental results show the effectiveness of the proposed method.

� 2014 Elsevier B.V. All rights reserved.

39

60

61

62

63

64

65

66

67

68

69

70

71

72

73

74

75

76

77

78

79

1. Introduction

There is a growing interest in methods that solve optimizationproblems effectively and efficiently. Depending on the nature ofthe problem, there are methods that ensure that the optimal solu-tion is found, for instance, the Simplex algorithm for linear pro-gramming problems, and the Branch and Bound algorithm forinteger linear programming problems [1,16]. However, the existingmethods to deal with non-linear integer programming problemscannot ensure that the optimal solution is always found, thus, itis necessary to use approximate methods. In recent years, methodssuch as the biologically inspired algorithms have gained impor-tance to tackle these types of optimization problems [8].

There are some approaches to deal with multi-modal non-linearoptimization problems in which some of the variables arecontinuous and other are discrete (MINLP) as in [3,11,25,26].Many researchers have proposed to use filled functions[30,29,28,10,22,18]. One of the main disadvantages of suchapproach is that the addition of a filled function introduces extraparameters which need to be tuned and they generally are

80

81

82

83

dependent on the problem to be optimized. In [20] an algorithmfor solving non-linear optimization problems with linear con-straints has been presented. In [24] an algorithm for solving non-linear integer optimization problems by means of linearization ofthe objective function and constraints has been proposed.

Generally, Evolutionary Algorithms (EA) and Particle SwarmOptimization (PSO) converge to a single final solution when usedfor optimization purposes. However, many optimization problemsare multimodal and present many solutions [13]. Niching methodshave been developed to find solution of multimodal optimizationproblems with multiple optima. The most relevant niching meth-ods include [13]: fitness sharing, derating, restricted tournamentselection, crowding, deterministic crowding, clustering, clearing,parallelization, speciation, among others. These niching methodsplay an important role when incorporated into evolutionary algo-rithms to maintain multiple solutions within a stable population.Unfortunately, most of the methods listed above present difficul-ties for solving multi-modal optimization problems with multiplelocal or global optima because is necessary to specify, a priori,the niching parameters, which are difficult to tune and they typi-cally are dependent on the problem to be optimized.

An interesting approach recently proposed in [13], which doesnot require specification of any niching parameters uses a PSO withring topology. In this case, the ‘‘local memory’’ of the individual

ultiple

http://dx.doi.org/10.1016/j.knosys.2014.02.015

mailto:[email protected]




http://www.sciencedirect.com/science/journal/09507051

http://www.elsevier.com/locate/knosys


84

85

86

87

88

89

90

91

92

93

94

95

96

97

98

99

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126127

129129

130

131

132

133134

136136

137

138

139

140

141

142

143

144

145

146

147

148

149

150

151

152

153

154

155

156

157

158

159

160

161

162

163

164

165

166167

169169

170

171

172

173

174

175

176177179179

180

181

182183

185185

186

188188

189

190

191192

194194

195

197197

2 E.R.F.A. Schneider, R.A. Krohling / Knowledge-Based Systems xxx (2014) xxx–xxx


18 March 2014

particles of the PSO are able to maintain the best positions found sofar, while the particles explore the search space. In the niching PSOwithout niching parameters proposed in [13] it is shown that largepopulations using PSO with ring topology is capable of forming sta-ble niche and able to find multiple local and global optima. Thepromising results suggest that this method presents good resultswithout requiring parameters to tune. The need of large popula-tions in niching PSO without niching parameters is its main disad-vantage. Nonetheless, the method developed in [13], motivated usto extend the approach to Differential Evolution (DE). In fact, DEwith ring topology [5,6] were applied to multimodal problemswith promising results to find a single solution of optimizationproblems. However, for problems with multiple optima, DE withring topology called DEGL [5] has not been used yet. The DEGL pro-vides a good balance between exploration and exploitation duringthe search improving the performance of the algorithm.

In this work, inspired by PSO with ring topology, the DEGL isused to explore the search space, but since DEGL has no memory,this motivated us to combine the algorithm with Tabu Search(TS). In this paper, we propose a method which uses only the valueof the objective function and the constraints, without no assump-tions regarding linearity, continuity, and convexity of the objectivefunctions and the constraints. The proposed method consists ofthree stages, where each stage uses the DE (Differential Evolution)algorithm hybridized with the TOPSIS (Technique for Order Prefer-ence by Similarity to Ideal Solution). The three stages are based onTOPSIS [14] to transform a k-objective optimization problem insingle-objectives optimization problem. In the last stage, a hybrid-ization with the Tabu Search (TS) algorithm to find integer solu-tions from the real solutions obtained previously is performed.

The remainder of this paper is organized as follows. In Section 2,we present the mathematical formulation of the optimizationproblem. The three algorithms used in our approach are describedin Section 3. In Section 4, we propose a novel hybrid method to findmultiple solutions of constrained optimization problems. Compu-tational results for several benchmarks are provided in Section 5to show the suitability of the method. In Section 6, we give someconclusions with directions for further works.

2. Mathematical formulation

A usual notation is used here: Z denotes the set of integers, R

the set of real numbers, and Zn ¼ Z� � � �n �Z. The problem of inter-est in this paper is the constrained non-linear integer optimizationproblem, which, without loss of generality, can be defined as:

min f ðxÞs:t: : giðxÞ � 0; with i ¼ 1; . . . ;m

lj � xj � uj; with j ¼ 1; . . . ;n

x 2 Zn

ð1Þ

where f : Zn ! R is the objective function, gi : Zn ! R are inequal-ity constraints; lj and uj represents the lower and upper bounds ofthe integer search space, respectively.

The reformulated problem can be defined as:

min ðf ðxÞ; gðxÞÞs:t: : lj � xj � uj with j ¼ 1; . . . n

x 2 Zn

ð2Þ

where g(x) = max(0, g1(x), . . . , gm(x)).

3. Algorithms description

To deal with the problem defined in Section 2, the constraintswere incorporated into the objective function. A method to solve

Please cite this article in press as: E.R.F.A. Schneider, R.A. Krohling, A hybrid appsolutions of constrained non-linear integer optimization problems, Knowl. Bas

multi-objective optimization problems is developed, whereas theTOPSIS is used to formulate the multi-objective problem inmono-objective problems. Next, the DE and TOPSIS are used tosolve the resulting mono-objective problems. At the last stage ofthe proposed method, the TS algorithm is executed to find integersolutions provided by DE algorithm. Next, we describe the threealgorithms used in this novel method.

3.1. The technique for order preference by similarity to ideal solution

TOPSIS [27] is a technique to find the best alternative(s) for amulti-criteria decision making (MCDM) problem consisting ofalternatives and criteria (benefit or cost). The algorithm selectsthe best alternative within the set of alternatives so that there isa compromise between the criteria. The algorithm calculates thepositive ideal solution (PIS) and the negative ideal solution (NIS)and chooses the alternative that best approximates the PIS and atthe same time increase the distance to the NIS. The PIS is definedas the solution containing the best values of the criteria withinthe set of alternatives, whereas the NIS contains the worst valueswithin the set of alternatives.

First, we define a matrix A = (xi,j)m�2 which contains in each linean alternative. In this paper, an alternative means a possiblesolution. The matrix A is composed of two columns, namely, twocriteria. The first criterion is the objective function for eachcandidate solution (or alternative), while the second criterion isthe maximum value of the constraint violation. The matrix A is de-scribed as:

A ¼

f1 g1

..

. ...

fm gm

2664

3775

where fi ¼ f ðX!iÞ and gi ¼maxð0; g1ðX!

iÞ; . . . ; gmðX!

iÞÞ.The weight vector composed by individual weights wj(j = 1, 2)

to each criteria Cj satisfiesP2

j¼1 wj ¼ 1: In this paper, the matrixA is normalized for each criterion Cj throughpij ¼

xij

MAX �xj; with j ¼ 1;2 and MAX �xj represents the maximum va-

lue for each criterion Cj. Thus, the decision matrix B representsthe normalized relative rating of alternatives and is described by:

B ¼ ðpi;jÞm�2

The TOPSIS [12] begins with the calculation of the positive idealsolutions B+(benefits) and negative ideal solutions B�(cost) asfollows:

Bþ ¼ ðpþ1 ; pþ2 ; . . . ;pþn Þ where pþj ¼ maxi

pij; j 2 J1; mini

pij; j 2 J2

� �ð3Þ

B� ¼ ðp�1 ; p�2 ; . . . ;p�n Þ where p�j ¼ mini

pij; j 2 J1; maxi

pij; j 2 J2

� �ð4Þ

where J1 and J2 represent the benefit and cost criteria, respectively.Secondly, compute the Euclidean distances between Bi and B+

(benefits) and between Bi and B� (cost) as follows:

dþ ¼ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiXn

j¼1wjðdþij Þ

2r

where dþij ¼ pþj � pij; with i

¼ 1; . . . ;m: ð5Þ

d� ¼ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiXn

j¼1wjðd�ij Þ

2r

where d�ij ¼ p�j � pij; with i

¼ 1; . . . ;m: ð6Þ

roach using TOPSIS, Differential Evolution, and Tabu Search to find multipleed Syst. (2014), http://dx.doi.org/10.1016/j.knosys.2014.02.015


198

199200

202202

203

204

205

206

207

208

209

210

211

212

213

214

215

216

217

218

219

220

221

222

223

224225227227

228

229

230

231

232

233

234

235

236

237

238

239

240

241242

244244

245

246

247

248249

251251

252

253

254

255

256

257

258

259

260

261

262

263

264

265

266

267

268

269

270

271

272273

275275

276

277

278

279

280

281

282283

285285

286

287

288

289290

292292

293

294295

297297

298

299

300

301302

304304

305

306

E.R.F.A. Schneider, R.A. Krohling / Knowledge-Based Systems xxx (2014) xxx–xxx 3


18 March 2014

Next, one calculates the relative closeness coefficient ni to eachalternative Bi in relation to the positive ideal B+ as:

ni ¼d�i

dþi þ d�i� ð7Þ

Finally, ranking according to the relative closeness coefficient.The best alternatives are those that have the greatest value ni

and should be chosen because they are closer to the positive idealsolution.

The TOPSIS pseudo-code is shown in Fig. 1.In this paper, TOPSIS is used to select the best solution within a

population of candidate solutions. In order to obtain a compromiseamong the set of criteria, which in this case are the value of theobjective function and the maximum constraint violation.

3.2. Differential Evolution

The optimization algorithm Differential Evolution (DE) wasintroduced by Storn and Price [19]. Similar to other EvolutionaryAlgorithms (EAs), DE is based on the idea of evolution of popula-tions of possible candidate solutions, which undergoes the opera-tions of mutation, crossover and selection [5]. The parametervectors are denoted by xi = [xi,1, xi,2, . . . , xi,n]T with components xi,j.The lower and upper bounds of xi,j are lj and uj, respectively. The in-dex i = 1, . . . , NP represents the individual’s index in the populationand j = 1, . . . , n is the position in D-dimensional individual. NP andn stand for population size and problem dimension, respectively.

First, the population of candidate solutions is initialized asfollows:

xi;j ¼ lj þ Uð0;1Þi;jðuj � ljÞ j ¼ 1; . . . ;n; i ¼ 1; . . . ;NP ð8Þ

where U(0,1)i,j is a uniformly distributed random number within[0,1].

The DE algorithm undergoes the following operators:

� Mutation.� Crossover.� Seletion.

Mutation is seen as a change or perturbation with a random ele-ment. In DE-literature, a parent vector from the current generationis called target vector, a mutant vector obtained through the differ-ential mutation operation is known as donor vector and finally anoffspring formed by recombining the donor with the target vectoris called trial vector. The mutation operation of the standard DE iscalculated as follows:

V!

i ¼ X!

r1;i þ FðX!r2;i � X!

r3;iÞ ð9Þ

where F is a scaling factor between 0.4 and 1, and r1, r2, r3 e [1, NP],such that r1 – r2 – r3.

The next variant of the DE studied is the DEbest. The mutationoperation of the DEbest is calculated as follows:

V!

i ¼ X!

r1;i þ FðX!r2;i � X!

gbest;iÞ ð10Þ
307
input: matrix AB = normalization of input matrix ACalculation of the PIS and NIS Solutions according Calculation of Euclidean distances between iB aaccording to (5) and (6), respectivelyCalculation of the relative closeness coefficient iξ toRanking according to the relative closeness coefficieReturn the alternative with the greatest value of iξ

Fig. 1. TOPSIS ps


where gbest is the best individual of the population.The DEGL algorithm differs from the standard DE due the insert-

ing of a ring neighborhood topology [6] as shown in Fig. 2. This ringneighborhood topology was initially proposed in PSO [13] and ex-tended to other bio-inspired algorithms. It favors a balance be-tween exploration and exploitation improving the capabilities ofDE algorithm. The donor vector is calculated by performing a linearcombination of the local donor vector L

!i, created using members

of a neighborhood, and between the global donor vector G!

i, cre-ated using members of the population. This linear combination ismade by a weighting factor which weights exploration versusexploitation. The smaller the value of the weight, the higher isthe exploration capacity of the algorithm. On the other side, thelarger the weight, the higher is the exploitation capacity of thealgorithm.

The mutation operator is calculated in three steps as follows:

(1) Local donor vector creation.(2) Global donor vector creation.(3) Combination of local and global donor vectors.

The local donor vector is calculated as follows:

L!

i ¼ X!

i þ aðX!besti� X!

iÞ þ bðX!r1 � X!

r2 Þ ð11Þ

where X!

bestiis the best neighbor of X

!i, and the neighborhood is de-

fined by a ring topology.The neighborhood size is defined by a parameter k, where in the

case of Fig. 2, it was set to k = 2. X!

i is the i-th individual of theneighborhood, r1 and r2 are chosen within the neighborhood sothat i – r1 – r2.

The global donor vector G!

i is calculated as follows:

G!

i ¼ X!

i þ aðX!gbest � X!

iÞ þ bðX!q1� X!

q2Þ ð12Þ

where X!

gbest is the best individual in the population, q1 and q2 arechosen in the population such that i – q1 – q2.

The donor vector V!

i is calculated as a linear combination of L!

i

and G!

i as:

V!

i ¼ r G!

i þ ð1� rÞ L!

i ð13Þ

where r is a weighting factor that weights exploration versusexploitation and is calculated as:

r ¼ number of iterationsmax iterations

: ð14Þ

The operation of crossover and selection are the same for allvariants of the DE.

In crossover step, a recombination is generated between thevector X

!i and the vector V

!i as follows:

ui;j ¼v i;j if Uð0;1Þ 6 Cr or j ¼ jrand

xi;j otherwise:

�ð15Þ

where Cr is the crossover rate, and jrand is a uniformly distributedrandom value between 1 and n to ensure that at least one compo-nent of V

!i is part of the vector U

!i.

to (3) and (4)nd B+ (benefits) and between iB and B− (cost)

each alternative iB according to (7)nt

eudo-code.



308

309

310

311

312313

315315

316

317318

319

320

321

322

323

324

325

326

327

328

329

330

331

332

333

334

335

336

337

338

339

340

341

342

343

344

345

346

347

348

349

350

351

352

353

354

355

356

357

358

359

360

361

362

363

364

365

366

367

368

369370372372

373

374

375

376

377

378

379

380

381

382

383

384

385

386387

389389

390

391

392393395395

396398398

399401401

402403

405405

406

408408

409411411

412

413

414

Fig. 2. Ring topology.



18 March 2014

The selection phase consists in the choice of individuals fromthe current population to the next, while keeping as members ofthe population the best individuals. Each individual i in the popu-lation X

!is compared with its respective individual i of the popula-

tion U!

as:

X!

i;Gþ1 ¼U!

i;G if U!

i;G is better than X!

i;G

X!

i;G otherwise:

(ð16Þ

An individual is considered better than another [7]:� If both individuals are feasible, is chosen the individual with

better fitness.� If one individual is feasible and the other infeasible, is chosen

the feasible individual.� If both individuals are infeasible, is chosen the one that least

violates the constraints.

The pseudo-code of DEGL is presented in Fig. 3 for uncon-strained optimization problems as similarly described in [6]. Inour case, the constraint handling is done using TOPSIS (DEGL andDEbest) and Deb [7].

3.3. Tabu Search

The Tabu Search, proposed by Glover [9], is a local search algo-rithm which uses a structure of memory to perform movementsthat allow the algorithm to escape from local optima. This struc-ture of memory, called tabu list, stores the most recent move-ments. Then, the algorithm examines the neighbors of a currentpoint, and selects the best neighbor not taboo (not on the list), stor-ing the previous move in the tabu list, which acts as a queue offixed size, i.e., the first in is the first out. This process is repeateduntil a maximum number of iterations is reached. The operationof the algorithm depends on a starting point from which it per-forms movements. The closer the starting point is from the opti-mum, the faster the convergence of the algorithm to thisoptimum is.

Since the solutions found in the previous stage are real values,firstly they are converted to integer by means of rounding as de-scribed in pseudo-code of Fig. 4. Next, the rounded solutions areused as initial solutions for the Tabu Search algorithm as describedin pseudo-code of Fig. 5 [17].

A solution is considered better than another in the same man-ner as in the selection operation of DE algorithm.

The Tabu Search algorithm calls the Tabu Move routine in its in-ner loop as presented in Fig. 6. In order to test the feasibility of the


candidate integer solutions is used the same procedure for con-straint handling as previously described by Deb [7].

The arguments of this algorithm are the current solution X!

i, thebest solution found X

!�i , the current iteration j, and the tabu vector

t. The Tabu information is kept in the vector t and tc holds the iter-ation at which the variable c has been updated. The beginning ofthe algorithm prevents the block of the search in which all themoves are potentially taboo. Then, a random movement is selectedand a value within its limits is chosen for the movement. Theneighborhood is searched by means of a breadth first search [17].With breadth first all the neighbor solutions are checked, and thebest is returned.

4. Proposed method

The proposed method to solve multi-objective problems con-sists in three stages, where in each stage is used the DE + TOPSISto solve mono-objective optimization problems. To treat thesemono-objective optimization problems, it is reformulated asfollows:

minðFðxÞ;GðxÞÞ ð17Þ

where F: Rn ? R is the objective function of the stage in questionand G: Rn ? R is the constraint set of the stage in question.

The DEGL used is similar to that presented in [6]. However, forthe choice of local or global best solution, is used the TOPSIS,whereas each candidate solution corresponds to an alternative,and each objective function corresponds to a criterion, which isconsidered either a benefit or cost. Benefit and cost criteria are re-lated to maximizing and minimizing the objective function, respec-tively. Since the problem consists in minimizing the objectivefunctions, then both criteria are considered as cost. Thus, the objec-tive function and the constraints are the criteria, so the TOPSIS finda compromise solution minimizing both criteria.

The multi-objective optimization problem is mathematicallyformulated as:

minðf1ðxÞ; f2ðxÞ; . . . ; fiðxÞ; . . . ; fkðxÞÞs:t: : x 2 X

where X ¼ fx; lj 6 xj 6 uj with j ¼ 1; . . . ;n and giðxÞ 6 0with i ¼ 1; . . . ;mg

ð18Þ

The first stage [14] consists in finding the solutions PIS and NISof the problem to be optimized. The PIS and NIS are defined asfollows:

PIS : ðmin f 1ðxÞ; . . . ;min f kðxÞÞ ¼ ðf �1 ; . . . ; f �k Þ ð19aÞ

NIS : ðmax f 1ðxÞ; . . . ;max f kðxÞÞ ¼ ðf�1 ; . . . ; f�k Þ ð19bÞ

x 2 X:

For the second stage, two functions are defined as follows [14]:

dðxÞPIS ¼Xk

j¼1

w2j

ðfjðxÞ � f �jf�j � f �j

" #28<:

9=;

12

ð20aÞ

dðxÞNIS ¼Xk

j¼1

w2j

f�j � fjðxÞf�j � f �j

" #28<:

9=;

1=2

ð20bÞ

x 2 X:

where wj is the weight of the j-th objective function. In this paper,we use equal weights in this stage. In this stage, DEGL + TOPSISalgorithm is used to solve the problems (20a) and (20b).



415

416

417

418

419

420

421

422

423

424425

427427

428

430430

431

432

433434436436

437

Fig. 3. DEGL pseudo-code.

Fig. 4. Rounding of real to integer solutions.

Fig.5. Tabu Search algorithm.



18 March 2014


The goal is to minimize d(x)PIS and to maximize d(x)NIS. To thisend a compromise solution is sought. The compromise solution isfound using membership functions. However, before finding thecompromise solution, (dPIS)⁄ and (dNIS)⁄ are calculated using theDEGL + TOPSIS as follows:

(dPIS)⁄ = mind(x)PIS, such that x e X, with solution xp,(dNIS)⁄ = maxd(x)NIS, such that x e X, with solution xn,(dPIS)0 = d(xn)PIS and (dNIS)0 = d(xp)NIS.

The membership functions are described as:

l1ðxÞ ¼

1 if dðxÞPIS< ðdPISÞ

�

1� dðxÞPIS�ðdPISÞ�

ðdPISÞ0�ðdPISÞ

� if ðdPISÞ0P dðxÞPIS P ðdPISÞ

�

0 if dðxÞPIS> ðdPISÞ

0:

8>>><>>>:

ð21aÞ

l2ðxÞ ¼

1 if dðxÞNIS> ðdNISÞ

�

1� ðdNISÞ

��dðxÞNIS

ðdNISÞ��ðdNISÞ

0 if ðdNISÞ06 dðxÞNIS

6 ðdNISÞ�

0 if dðxÞNIS< ðdNISÞ

0:

8>>><>>>:

ð21bÞ

To minimize (21a) and maximize (21b), we use the max–minoperator proposed by Bellman and Zadeh [2] and extended by Zim-mermann [31]. So, the problem is reformulated as:

lDðx�Þ ¼ maxfminðl1ðxÞ;l2ðxÞÞg: ð22Þ

The max–min solution is illustrated in Fig. 7.



438439

441441

442

443

444445

446

447

448

449

450

451

452

453

454

455

456

457

458

Fig. 6. Tabu Move algorithm.

Fig. 7. Compromise solution obtained by means of membership functions.

Fig. 8. Illustration of the proposed approach.



18 March 2014


This problem is reformulated as follows:

max as:t: : l1ðxÞ � a

l2ðxÞ � ax 2 X:

ð23Þ

At this point, the third stage begins, which consists of solvingthe problem (23). We use the algorithm DE + TOPSIS setting thea-level of satisfaction in the first objective function and constraintsin the second objective function. We alternate the DE + TOPSISalgorithm and Tabu Search algorithm. After a fixed number of iter-ations of the DE + TOPSIS, the Tabu Search algorithm is executed toconvert the problem of real to integer and to perform a local searchnear of the solutions found by the DE + TOPSIS algorithm. Next,DE + TOPSIS is executed again accompanied by the Tabu Search,i.e., the algorithms are alternated to solve the problem.

It is known that the solution of the problem (23) is also thesolution of the problem (18) [14]. So, the end of the third stage pro-vides a solution of the problem (18). The proposed approach isshown in Fig. 8.

To solve the problem (1), the constraints are inserted as anadditional objective and the proposed method can be applied tosolve the resulting multi-objective problem.

Table 1Para Q5meter setup for the algorithms used in the experiments.

Population size 40Number of iterations of DE 100Number of iterations of the TS 1000Number of alternations between DE and TS 10Crossover rate 0.9a Parameter 0.8b Parameter 0.8Scaling factor F 0.8Neighborhood size (k) 2



459

460

461

462

463

464

465

466

467

468

469

470

471

472

473

474

Table 2Success rate for the six benchmarks.

Problem Global solution DEGL (%) DEbest (%) Standard DE (%)

1 (1,1) 100 100 1002 (16,22,5,5,7) 100 70 603 (2,0) 100 100 1004 (0,�1,�2) 100 100 1005 (3,1,0) 100 100 1006 (0,1,2,�1) 100 100 100

Table 3Average execution time for the six benchmarks in minutes.

Prob. 1 Prob. 2 Prob. 3 Prob. 4 Prob. 5 Prob. 6

DEGL 3.11 5.77 2.75 2.77 3.09 4.54DEbest 2.18 4.58 1.86 1.97 2.20 3.58DE 2.26 4.51 1.85 2.02 2.23 3.62

Table 4Parameter setup for the algorithms used in the experiments.

Population size 40Number of iterations of DE 100 to problem 7 and 200 to

problem 8Number of iterations of TS 500Number of alternations between DE

and TS50

Crossover rate 0.9a Parameter 0.8b Parameter 0.8Scaling factor F 0.8Neighborhood size (k) 2

Table 5Success rate for the Problem 7.

Solutions of the Problem 7: DEGL DEbest Standard DE

(2353011) 20 (100%) 16 (80%) 16 (80%)(2253111) 20 (100%) 4 (20%) 7 (35%)(2452011) 20 (100%) 19 (95%) 20 (100%)(2254011) 20 (100%) 20 (100%) 19 (95%)(2352111) 20 (100%) 5 (25%) 6 (30%)(2252211) 20 (100%) 8 (40%) 13 (65%)

Table 6Success rate for the Problem 8.

Solutions of the Problem 8: DEGL DEbest Standard DE

(85422993064581) 19 (95%) 17 (85%) 10 (50%)(85413003064590) 18 (90%) 10 (50%) 3 (15%)

Fig. 9. Success rate varying the

Fig. 10. Success rate varying the number of alternations between DE and TS for afixed number of iterations of TS set to 10.




18 March 2014


5. Computational results

In this paper, we use six constrained non-linear integer bench-marks to evaluate the proposed method, which are described inAppendix A.

The parameter setup for the algorithms DE (standard), DEbest

and DEGL used in the experiments is given in Table 1. The valueof DE parameters was suggested by [5,6], who studied this issuein-depth. It is out of scope of this paper to investigate the optimalparameters of DE. Obviously, we have carried out preliminaryexperiments to our particular problems investigated here in orderto adopt such DE parameters as listed in Table 1.

The TOPSIS algorithm does not have any parameters, however,weights for the two criteria are generated using a uniform proba-bility distribution between 0 and 1, since r1 + r2 = 1, then r1 -= rand(0, 1) and r2 = 1 � r1. This choice allows a betterdistribution of solutions in the search space.

number of iterations of TS.



475

476

477

478

479

480

481

482

483

484

485

486

487

488

489

490

491

492

493

494

495

496

497

498

499

500

501

502

503

504

505

506

507

508

509

510

511

512

513

514

515

516

517

518

519

520

521

522

523

524

525

526

527

528

529

530

531

532

533

534

535

536

537

538

539

540

541

542543

545545

546

547

548549

551551

552

553

554555


Table 7Average execution time for the two multi-modal problems in minutes.

Problem 7 Problem 8

DEGL 11.82 26.66DEbest 8.23 18.84DE 8.17 17.72



18 March 2014

The parameters of the TS are the number of executions, and thelist size is equal to the problem dimension (number of variables).

The weighting factor r is as given by (9).The hybrid algorithm has been applied to the benchmark prob-

lems and executed 20 times. The solutions obtained during theoptimization process and the success rate are shown in Table 2.Numbers in bold mean the best value obtained.

Table 3 shows the average execution time for the six bench-marks investigated.

From Table 2, we notice that the proposed approach with DEGL,DEbest and standard DE was effective for the first six benchmarkproblems as expected, since these benchmarks have only oneoptimum.

In order to test the capacity of the proposed hybrid approachwith DEGL to find multiple solutions, two additional benchmarkshave been included, which are described in Appendix A as prob-lems 7 and 8.

In a similar way to the first experiment, the parameters setupused in the second experiment is given in Table 4.

The hybrid algorithm has been applied to the problems 7 and 8and executed 20 times. The solutions obtained during the optimi-zation process and the success rate are shown in Tables 5 and 6,respectively.

Simulation experiments have been carried out to investigate theperformance of the proposed method with respect to two parame-ters: the number of iterations of TS and the number of alternationsbetween DE and TS. In order to address the influence of the numberof iterations of TS we carry out an in-depth simulation study fordetermining the success rate varying this parameter. We depictthe simulation results in Fig. 9. In the same way, the number ofinterleaved iterations between DE and TS was also investigatedand the success rate with respect to this parameter is depicted inFigs. 10–12. We notice that for both problems investigated theknown optima solutions are found for different parameterscombinations.

Through Fig. 9, we note that above 100 iterations for the TabuSearch algorithm the results are satisfactory, because for bothbenchmarks investigated the average success rate is greater than80%. Through Figs. 10–12, we note that a value equal to interleaves


50 provides an average success rate of 100% for both benchmarksinvestigated.

Table 7 shows the average execution time for the two multi-modal problems investigated.

From Tables 5 and 6, we observe that the proposed approachwith DEGL is appropriate to find multiple optima of multi-modaloptimization problems, because the ring topology of the DEGL pro-vides a better diversity of the population contributing to find mul-tiple solutions, while the DEbest and the standard DE convergeprematurely just to one solution. From Tables 3 and 7, we noticethat the proposed approach with DEGL demands a little more com-putational time than DEbest and standard DE.

6. Conclusions

In this paper, we propose a novel method for solving multi-modal constrained non-linear integer optimization problems. Thismethod has the advantage that only the value of the objectivefunction and the constraints are needed. The effectiveness of thealgorithms investigated to find solutions of the benchmarks prob-lems with just one known optimal solution was very similar. How-ever, the effectiveness of the DEGL to find multiple solutionsoutperforms DEbest and the standard DE, since both algorithmsconverge prematurely just to one solution, while the DEGL pro-vides better diversity of the population. In the future, we intendto extend the approach to find solutions of multi-modal con-strained non-linear integer bi-level optimization problems.

Appendix A. Test problems

In the following, we list all the benchmarks used in theexperiments.

Problem 1 [10]:

Minimize : f ðxÞ ¼ 100ðx2 � x21Þ

2 þ ð1� x1Þ2

x21 þ x2

2 � 2513

x1 þ x2 � �0:1

subject to � 13

x1 þ x2 � �0:1

� 10 � x1 � 100 � x2 � 10x1; x2 2 Z:

The optimum solution is (1,1) with objective function valueequal to 0.

Problem 2 [15]:

Minimize : f ðxÞ ¼ x21 þ x2

2 þ 3x23 þ 4x2

4 þ 2x25

� 8x1 � 2x2 � 3x3 � x4 � 2x5

x1 þ 2x2 þ 2x3 þ x4 þ 6x5 � 8002x1 þ x2 þ 6x3 � 200x3 þ x4 þ 5x5 � 200x1 þ x2 þ x3 þ x4 � 48x2 þ x4 þ x5 � 34

subject to 6x1 þ 7x5 � 10455 � x1 þ x2 þ x3 þ x4 þ x5 � 4000 � xi � 99xi 2 Z 8i 2 f1;2;3;4;5g:

The optimum solution is (16,22,5,5,7) with objective functionvalue equal to 807.

Problem 3 [4]:



557557

558

559

560561

563563

564

565

566567

569569

570

571

572573

575575

576

577

578579

581581

582

583

584

585586

588588

589

590

591

592

593594595596597598599600601602603604605606607608609610611612Q4613614615616617618619620621622623624625626627628629630631632633634635636637638639640641642643644645646647648649650651652653654655656657658659660

661



18 March 2014

Maximize : f ðxÞ ¼ ð1� ð1� 0:98Þx1 ð1� 0:92Þx2 Þ11x1 þ 5x2 � 234x1 þ 6x2 � 12

subject to x1 þ x2 � 10 � x1; x2 � 10x1; x2 2 Z:

The optimum solution is (2,0) with objective function valueequal to 0.9996.

Problem 4 [21]:

Minimize : f ðxÞ ¼ x21 � 2x1 þ x2

2 þ 3x2 þ x23 þ 4x3

x21 þ 2x2

2 þ 3x23 � 17

subject to � 10 � xi � 0xi 2 Z 8i 2 f1;2;3g:

The optimum solution is (0,�1,�2) with objective function va-lue equal to �6.

Problem 5 [21]:

Maximize : f ðxÞ ¼ 13x1 � 5x22 þ 30:2x2 � x2

1 þ 10x3 þ 2:5x23

2x1 þ 4x2 þ 5x3 � 10x1 þ x2 þ x3 � 5

subject to 0 � xi � 30xi 2 Z 8i 2 f1;2;3g:

The optimum solution is (3,1,0) with objective function valueequal to 55.2.

Problem 6 [21]:

Minimize : f ðxÞ ¼ x21 þ x2

2 þ 2x23 þ x2

4 � 5x1 � 5x2 � 21x3 þ 7x4X4

i¼1

x2i þ

X3

i¼0

ð�1Þixiþ1 � 8

x1 � 1þ 2x22 þ x2

3 þ x4ð2x4 � 1Þ � 10subject to 2x1ðx1 þ 1Þ þ x2ðx2 � 1Þ þ x2

3 � x4 � 5� 5 � xi � 5xi 2 Z 8i 2 f1;2;3;4g:

The optimum solution is (0,1,2,�1) with objective function va-lue equal to �44.

Problem 7 [23]:

Maximize : f ðxÞ¼x1þx2þx3

20y1þ30y2þx1þ2x2þ2x3�18030y1þ20y2þ2x1þx2þ2x3�150

subject to �60y1þx1�0�75y2þx2�00�x1;x2;x3�100 with x1;x2;x3 2Z

and y1;y2 2f0;1g:

The optima solutions are (23,53,0,1,1), (22,53,1,1,1),(24,52,0,1,1), (22,54,0,1,1), (23,52,1,1,1) and (22,52,2,1,1) withobjective function value equal to 76.

Problem 8 [23]:

Maximize : f ðxÞ ¼ 10ðx1 þ x2 þ x3Þ þ 8ðy1 þ y2 þ y3Þ� 4ðx1 þ y1Þ � 5ðx2 þ y2Þ � 6ðx3 þ y3Þ

x1 þ y1 � 400x2 þ y2 � 500x3 þ y3 � 3000:2ðx1 þ x2 þ x3Þ � x1

subject to 0:1ðx1 þ x2 þ x3Þ � x2

0:2ðx1 þ x2 þ x3Þ � x3

0:4ðy1 þ y2 þ y3Þ � y1

0:5ðy1 þ y2 þ y3Þ � y3

0 � x1; x2; x3; y1; y2; y3 � 1000x1; x2; x3; y1; y2; y3 2 Z:


The optima solutions are (85,42,299,306,458,1) and(85,41,300,306,459,0) with objective function value equal to4516.

References

[1] M.S. Bazaraa, H.D. Sherali, J.J. Jarvis, Linear Programming and Network Flows,John Wiley & Sons Inc., New Jersey, USA, 2009.

[2] R.E. Bellman, L.A. Zadeh, Decision-making in a fuzzy environment, Manage. Sci.B 17 (1970) 141–164.

[3] M.F. Cardoso, R.L. Salcedo, S. Feyo de Azevedo, D. Barbosa, A simulatedannealing approach to the solution of MINLP problems, Comput. Chem. Eng. 21(1997) 1349–1364.

[4] M.S. Chern, R.H. Jan, Reliability optimization problems with multipleconstraints, IEEE Trans. Reliab. 35 (1986) 431–436.

[5] S. Das, P. Suganthan, Differential evolution: a survey of the state of-the-art,IEEE Trans. Evol. Comput. 15 (2011) 4–31.

[6] S. Das, A. Abraham, U.K. Chakrabort, A. Konar, Differential evolution using aneighborhood-based mutation operator, IEEE Trans. Evol. Comput. 3 (2009)526–553.

[7] K. Deb, An efficient constraint handling method for genetic algorithms,Comput. Methods Appl. Mech. Eng. 186 (2000) 311–338.

[8] R. Eberhart, Y. Shi, Computational Intelligence: Concepts to Implementations,Morgan Kaufman Publisher, San Francisco, USA, 2007.

[9] F. Glover, Future paths for integer programming and links to artificialintelligence, Comput. Oper. Res. 13 (1986) 533–549.

[10] Y.H. Gu, Z.Y. Wu, A new filled function method for nonlinear integerprogramming problem, Appl. Math. Comput. 173 (2006) 938–950.

[11] V. Hanagandi, M. Nikolaou, A hybrid approach to global optimization using aclustering algorithm in a genetic search framework, Comput. Chem. Eng. 22(1998) 1913–1925.

[12] J. Huang, Combining entropy weight and TOPSIS method for informationsystem selection, IEEE Conf. Cybernet. Intell. Syst. (2008) 1281–1284.

[13] X. Li, Niching without niching parameters: particle swarm optimization usinga ring topology, IEEE Trans. Evol. Comput. 14 (2010) 150–169.

[14] Y.J. Lai, T.Y. Liu, C.L. Hwang, TOPSIS for MODM, Eur. J. Oper. Res. 76 (1994)486–500.

[15] C. Mohan, H.T. Nguyen, A controlled random search technique incorporatingthe simulated annealing concept for solving integer and mixed integer globaloptimization problems, Comput. Optim. Appl. 14 (1999) 103–132.

[16] G.L. Nemhauser, L.A. Wolsey, Integer and Combinatorial Optimization, JohnWiley & Sons, New York, USA, 1999.

[17] J.P. Pedroso, Tabu Search for Mixed Integer Programming, Technical ReportSeries DCC-2004-02, LIACC, Universidade do Porto, Portugal, 2004.

[18] G. Rempu, C. Huang, A continuous approach to nonlinear integerprogramming, Appl. Math. Comput. 34 (1989) 39–60.

[19] R. Storn, K.V. Price, Differential Evolution – A Simple and Efficient AdaptiveScheme for Global Optimization Over Continuous Spaces, Technical Report TR-95-012, ICSI, Berkeley, CA, USA, 1995.

[20] M.S. Sabbagh, A partial enumeration algorithm for pure nonlinear integerprogramming, Appl. Math. Model. 32 (2008) 2560–2569.

[21] V.K. Srivastava, A. Fahim, A two-phase optimization procedure for integerprogramming problems, Comput. Math. Appl. 42 (2000) 1585–1595.

[22] X.L. Sun, Logarithmic-exponential penalty formulation for integerprogramming, Appl. Math. Lett. 12 (1999) 73–77.

[23] J.F. Tsai, M.H. Lin, Y.C. Hu, Finding multiple solutions to general integer linearprograms, Eur. J. Oper. Res. 184 (2008) 802–809.

[24] V. Vassilev, K. Genova, An approximate algorithm for nonlinear integerprogramming, Eur. J. Oper. Res. 74 (1994) 170–178.

[25] C. Wang, H. Quan, X. Xu, Optimal design of multi-product batch chemicalprocess using genetic algorithm, Indust. Eng. Chem. Res. 35 (1996) 3560–3566.

[26] L. Yan, K. Shen, S. Hu, Solving mixed integer nonlinear programming problemswith line-up competition algorithm, Comput. Chem. Eng. (2004) 2647–2657.

[27] K. Yoon, C.L. Hwang, Multiple Attribute Decision Making Methods andApplications, Springer Verlag, Berlin, Germany, 1980.

[28] Y. Yongjian, L. Yumei, A new discrete filled function algorithm for discreteglobal optimization, J. Comput. Appl. Math. 202 (2007) 280–291.

[29] W. Zhu, M.M. Ali, Discrete dynamic convexized method for nonlinearlyconstrained nonlinear integer programming, Comput. Oper. Res. 36 (2009)2723–2728.

[30] W. Zhu, An approximate algoritm for nonlinear integer programming, Appl.Math. Comput. 98 (1998) 183–193.

[31] H.J. Zimmermann, Fuzzy programming and linear programming with severalobjective functions, Fuzzy Sets Syst. 1 (1978) 45–55.



Download - A hybrid approach using TOPSIS, Differential Evolution, and Tabu Search to find multiple solutions of constrained non-linear integer optimization problems

Top Related