cultural algorithms: a tabu search approach for the optimization of engineering design problems

Soft ComputDOI 10.1007/s00500-013-1169-5

METHODOLOGIES AND APPLICATION

Cultural algorithms: a Tabu search approach for the optimizationof engineering design problems

Mostafa Z. Ali · Robert G. Reynolds

© Springer-Verlag Berlin Heidelberg 2013

Abstract Cultural Algorithms and Tabu search algorithmsare both powerful tools to solve intricate constrained engi-neering and large-scale multi-modal optimization problems.In this paper, we introduce a hybrid approach that combinesCultural Algorithms and Tabu search (CA–TS). Here, TabuSearch is used to transform History Knowledge in the BeliefSpace from a passive knowledge source to an active one. Ineach generation of the Cultural Algorithm, we calculate thebest individual solution and then seek the best new neigh-bor of that solution in the social network for that popula-tion using Tabu search. In order to speed up the convergenceprocess through knowledge dissemination, simple forms ofsocial network topologies were used to describe the con-nectivity of individual solutions. This can reduce the num-ber of needed generations while maintaining accuracy andincreasing the search radius when needed. The integrationof the Tabu search algorithm as a local enhancement processenables CA–TS to leap over false peaks and local optima.The proposed hybrid algorithm is applied to a set of com-plex non-linear constrained engineering optimization designproblems. Furthermore, computational results are discussedto show that the algorithm can produce results that are com-parable or superior to those of other well-known optimizationalgorithms from the literature, and can improve the perfor-mance and the speed of convergence with a reduced commu-nication cost.

Communicated by Y.-S. Ong.

M. Z. Ali (B)Jordan University of Science and Technology,Ar Ramtha, Jordane-mail: [email protected]

R. G. ReynoldsWayne State University, Detroit, USAe-mail: [email protected]

Keywords Knowledge diffusion · Constrained engineeringoptimization design · Knowledge integration ·Cultural Algorithms · Tabu search

1 Introduction

Over the last five decades, several authors have developeda substantial number of methods to solve engineering opti-mization problems in different areas. Many mathematical-based programming techniques have been developed to han-dle this category of problems such as integer programming,dynamic programming, linear and non-linear programming.When considering simple models, these numerical meth-ods can provide a valuable approach to find the absoluteoptimum by using gradient information to search the solu-tion space. The solution obtained via these gradient-basedmethods is more accurate and the convergence process isfaster, compared to stochastic methods (Kaveh and Talata-hari 2010a). However, the effectiveness of such gradient-based approaches depends on a good initial starting pointin the search space (Salehi 2008). The nature of numerousengineering optimization design problems is highly complex,with many imposed restrictions on domain variables, non-convex cost functions and prohibited regions. This makes itdifficult for these traditional mathematical-based methods tosolve such types of problems (Kaveh and Talatahari 2010a).Although some of these mathematical-based approaches canbe modified to provide some capability in solving multi-modal and non-convex optimization problems, the computa-tion endeavor involved is often too high (Kaveh and Talata-hari 2010b).

As an alternative to the mathematical programming opti-mization algorithms, meta-heuristics have been proposed toprovide a better solution for complex engineering problems.

123

M. Z. Ali, R. G. Reynolds

These meta-heuristics imitate natural phenomena by combin-ing rules and randomness in their searching methods. In gen-eral, meta-heuristics compete in the sense of their explorationcapabilities, and finding promising regions, which alleviatesthe need for continuous cost functions and other restrictionsimposed by mathematical-based approaches. Natural phe-nomena that inspired such meta-heuristics include the bio-logical evolution process [evolutionary algorithms proposedby Fogel et al. (1966), De Jong (1975), and Koza (1990),Genetic Algorithms (GA) proposed by Holland (1975), Cul-tural Algorithms (CA) proposed by Reynolds (1994), and dif-ferential evolution (DE) proposed by Storn and Price (1997)],animal behavior inspired algorithms like ant colony opti-mization proposed by Dorigo et al. (1996), Tabu search(TS) proposed by Glover (1989), the particle swarm opti-mization (PSO) proposed by Eberhart and Kennedy (1995),and simulated annealing proposed by Kirkpatrick et al.(1983).

In this paper we describe a hybrid system that consistsof Cultural Algorithm with an embedded local Tabu searchcomponent. In this work, interleaving such CA with Tabusearch intends to improve diversity in the population ofsearching agents, shun premature convergence, and escapestagnation states with fewer computations. The motivationfor the hybrid system is as follows. One of the knowledgesources used in the Belief Space to direct problem solversin the population space is History Knowledge. In previousversions of Cultural Algorithms it has been used to matchpast behavior with that of the present in a passive way. Here,Tabu search actively encapsulates the History Knowledgeand uses it to select appropriate solution pathways. Thus,Tabu search transforms History Knowledge from a passiveknowledge component to an active one. The remainder ofthis paper is organized as follows. In Sect. 2 we providea brief overview of Tabu search. An overview of CulturalAlgorithms and details of the Cultural knowledge sourcesthat are adopted in this research, in addition to interpretingknowledge diffusion in this social context are given in Sect.3. Section 4 introduces the proposed hybrid algorithm thatembeds Tabu search within Cultural Algorithms. Experimen-tal settings for the benchmark problems and strategies of thecarried simulation are discussed in Sect. 5. The results arepresented and analyzed in Sect. 6. Finally, our conclusionsand future work are given in Sect. 7.

2 Tabu search

Tabu search algorithm is a meta-heuristic local searchapproach, superimposed on another heuristic, which was cre-ated by Glover (1989), and is often used to deal with com-binatorial optimization problems. It was first proposed tohelp hill-climbing algorithms overcome premature conver-

gence to local optima in the search space. The approachuses memories to avoid entrapment in cycles by penaliz-ing any new attempts by the individuals to direct the searchpath in the next generation to regions in the landscape thathave been visited earlier. It records previously investigatedstates in a designated list so it avoids revisiting them again.Hence, Tabu search is viewed as hill-climbing enhanced withstructures of short-term memory. Tabu search uses long-termoccurrence memory to strengthen attractive modules. Thealgorithm pursues the search when the optimization processencounters local optima, where cycling back to formerlyused solutions is prohibited through the use of memory listscalled “Tabu lists”. Tabu lists trace the recent search his-tory. The pseudo-code for Tabu search algorithm is shown inTable 1.

The tabu list will hold the a record of all previously vis-ited states. The algorithm starts by searching for an opti-mal solution (X∗) until the stopping criteria is reached. Thefunction findBestSol() will search through the candidate list(X_List) and return the best solution based on fitness score.The neighborhood of the initial solution in N (X) will thenbe checked for a better solution to determine the next move.Any neighboring solution X in N (X) that is checked andfound not to contain elements in the Tabu list T then itwill be added to the solution list X_List via the functionaddFeatures().

If the fitness of the candidate solution is higher than thatof the current solution then it will be chosen as the newbest X∗. If the Tabu list is full at any time (size exceedsthe TabuListMaxSize value), then some elements will bedeleted according to the order they were first added to thelist.

Table 1 Tabu search algorithm

123

Cultural algorithms: a Tabu search approach

3 Cultural Algorithms

Cultural Algorithm is an evolutionary algorithm inspiredfrom models of the cultural evolution process (Reynolds1994). The basic components of the algorithm include thepopulation space, a belief space, and the basic protocols thatare used for the communication between both components.The selected experiences of the successful agents during thepopulation evolution will produce knowledge that can becommunicated to the belief component, where it gets manip-ulated and used to affect the evolution process of the popu-lation of agents. The interaction between both componentsyields a dual inheritance structure in which the evolution ofthe agents and their evolved beliefs take place in parallel overtime.

In this framework, five classes of cultural knowledge havebeen recognized (Reynolds and Peng 2005). These knowl-edge sources can be summarized as follows:

• Situational knowledge: exemplar individuals or reminis-cences of all types of solutions (successful and unsuccess-ful).

• Normative knowledge: acceptable ranges for individuals’behavior.

• Topographical knowledge: geographically-recorded beha-viors of the individuals (spatial patterns).

• Domain knowledge: formed knowledge of domain enti-ties, their formed interactions and relationships.

• History knowledge: a time series of individuals’ behavior(temporal patterns). The interpretation of History Knowl-edge is a little different in this hybrid system as will beexplained in the next subsection.

Reynolds has proposed the completeness of this set of knowl-edge sources since any piece of knowledge can be expressedas a blend of some subset of the knowledge sources set(Reynolds and Peng 2005). Table 2 below shows the pseudo-code for Cultural Algorithms.

Individuals get assessed with the performance functionobj(), every generation. The acceptance function accept()then decides which agents in the population space Pt

s are

allowed to use their experiences to inform the belief spaceBt

s . Experience of those best agents are used to update thebelief component Bt

s using the function update(). Knowledgein the belief space is then allowed to enhance those individ-uals selected to the next generation through the influence()function.

Cultural Algorithms replicates this process every iterationuntil we reach the determined stopping condition. This allowsthe main components of the Cultural Algorithm system tosupport and communicate with each other, in a way that goessimilarly to how it is in the human culture’s evolution.

3.1 Details of knowledge

Five knowledge sources compose the heart of the belief spacecomponent in the CA system (Reynolds and Peng 2005). Thedifferent knowledge sources keep track of information aboutdifferential evolution and use this recorded information toaffect the evolution process in future generations. The follow-ing subsections describe only the Normative and Topograph-ical knowledge sources in detail, while History knowledgeis expressed in the context of Tabu search, since the con-sidered problems have static nature with multi-dimensions,and hence the other knowledge sources will have minimalor no influence on the evolution process here and will onlyincrease computation time (Reynolds and Peng 2005). “His-tory” or Temporal Knowledge has been traditionally usedin CA as a “passive” device. Just comparing new sequenceswith old to predict a new action. Here, History Knowledge arepast states, where we reject new solutions that relate to thesepast states. The “Tabu List” in this hybridization scheme isHistory Knowledge, and hence its discussion is going to bepresented while discussing how Tabu search is adapted tocontinuous spaces in Sect. 4.

Table 3 shows the average number of controlled indi-viduals (based on 25 runs using 100 individuals) using thefive knowledge sources, when tested on a randomly chosenbenchmark (spring tension/compression). This is an indica-tion of the success of the knowledge source based on itsshare of the roulette wheel. Information about the averagerate of accepted individuals to the next generation that are

Table 2 Cultural Algorithmsframework

123


Table 3 Performance metric of the five knowledge sources on Bench-mark 1 (spring tension/compression) based on 25 runs using 100 agents

Knowledgesource

Avg. no. ofcontrolled agents

Avg. rate ofaccepted agents

Situational 15 0.08

Normative 34 0.4

Domain 3 0

Topographic 41 0.48

History 7 0.04

controlled by each of the five KSs out of the 25 % of thewhole population is also recorded.

The structure of the modified knowledge interaction inthe context of our new influence function will impose betterdiversity among the population and hence an improved searchradius.

3.1.1 Normative knowledge

Normative knowledge keeps a memory of the feasiblesearch regions and ranges of acceptable individual’s behav-ior (Reynolds and Peng 2005). The normative knowledgestructure, VB, is described as follows:

{V B} = 〈V1, V2, . . . , Vn〉 (1)

For the ith variable Vi , the normative data structure contains[li , ui ], where ui , li indicate the upper bound and lower boundof the i th dimension. Moreover, this structure contains theperformance of the individuals in the lower and upper limitsof the i th dimension, and is given as PLi , and PUi , respec-tively.

The update process is given by:

P Lt+1i =

⎧⎪⎨

⎪⎩

f (xk), xi,k ≤ lti

f (xk), f (xk) < P Lti

P Lti , otherwise

(2)

lt+1i =

⎧⎨

⎩

xti,k, xt

i,k ≤ lti

x ti,k, f (xk) < P Lt

ilti , otherwise

(3)

where, the kth agent can affect the lower limit for the ithvariable, lt

i denotes the lower limit for the ith variable at timet , and the score of the performance is denoted by PLt

i .The update process of the upper limit of the i th variable

is described as follows:

C − C −

C − C

s P

−

− =

tC tC

tC

tC tC

tC

11t 1

2t

13t

141

1

t

t

1 2

33_1 3_2

3_3 3_4tC

a

b

( )

2

+

b( )

2

+ a b( )

2

+

b

( )

2

+* ( 3 )

2a b

a b a b

a b

a bM

+=

*M

Fig. 1 First pass of topographic knowledge for a 2-dimensional prob-lem

ut+1i =

⎧⎪⎨

⎪⎩

xi,k, xi,k ≥ uti

xi,k, f (xk) > PU ti

uti , otherwise

(4)

PU t+1i =

⎧⎨

⎩

f (xk), xi,k ≥ uti

f (xk), f (xk) > PU ti

PU ti , otherwise

(5)

Here, PU ti is the performance score for this limit.

3.1.2 Topographic knowledge

Topographic is responsible for updating the distribution offeasible solutions and promising regions in the problemsearch space (Saleem 2001). It uses the search space found byNormative Knowledge to uniformly divide the search regioninto sub-regions alongside each of the dimensions of theproblem. This progression results in 2n cells in every gen-eration of the optimization process for the n-dimensions ofthe problem. Assuming we have a 2-dimensional problemwith {V B} = < [a, b] , [a, b]> then the process by whichTopographic knowledge works is shown in Fig. 1. If the bestsolution found after t generations is located in cellCt−1

4 , thenthat cell’s area is subdivided and its depth is updated accord-ingly.

The set of cells of the search space is organized as a hier-archical tree structure, and can be described as follows:

{hostCell}t = ⟨Ct

1, Ct2, . . . , Ct

k

⟩(6)

where hostCell is the cell that hosts agent xi at a certainpoint in time, and the initial k cells form the root level of theexploration process. The cell in the feasible distribution canbe described as follows:

Cti = ⟨

Lti , U t

i , sti , dt

i , pti

⟩(7)

where the interval lower limit and the interval upper limit fora certain cell are represented as follows:

Lti = 〈l1, l2, . . . , ln〉

U ti = 〈u1, u2, . . . , un〉 (8)

123


sti is the status of the cell at a given time, and is given as

follows

sti =

⎧⎪⎪⎨

⎪⎪⎩

P+, f (x∗i ) > f̄ (x∗t )

U, ∀x ∈ A : (xtx , xt

y)|∀t = Cti (x, y)

P−, f (x∗i ) ≤ f̄ (x∗t )

(9)

where P+ means that the cell is the most promising cell inexploring further and enhancing the solution, and U meansthat the cell has not been explored yet, and hence there areno agents at time t with coordinates (xt

x , xty) equal to the

coordinates of this ith cell (Cti (x, y)). f (x∗

i ) is the fitnessof the best agent in the ith cell, and f̄ (x∗t ) is the averagefitness of the fittest agent in all cells. Any cell is exploredin the order of how promising it is expected to be and hencethe lowest order is that of these cells marked with P−. dt

i isthe depth of the ith cell, which shows how many times thesearch region has been divided to explore further. Since theused memory increases while we progress with the searchas cells get partitioned further, we use the branching restric-tion dt

i ≤ d̄. pti is a pointer to the children of the ith cell,

or otherwise it is a leaf cell. The update process is givenin Eq. 10.⟨Ct+1

1 , Ct+12 , . . . , Ct+1

k

⟩

=

⎧⎪⎪⎨

⎪⎪⎩

⟨Ct

1, Ct2, . . . , Ct

k

⟩ ∪⟨Ct+1

j_1 , Ct+1j_2 , Ct+1

j_3 , Ct+1j_4

⟩|∀ j∈branch(t),

i f (hostC(xti ) = Ct

j & f (xti ) > f (x∗

i )⟨Ct

1, Ct2, . . . , Ct

k

⟩, otherwise

(10)

where⟨Ct+1

1 , Ct+12 , . . . , Ct+1

k

⟩represents the set of cells in

the feasible distribution at time t + 1, and⟨Ct+1

j_1 , Ct+1j_2 , Ct+1

j_3 , Ct+1j_4

⟩are the sub-cells that are formed

from branching cell Ctj , knowing that the branching process

may occur at more than one cell at the same time step.

3.2 The evolution of knowledge-production

The adopted population space is an evolutionary program-ming module. The choice of parent solutions in the muta-tion process is dependent on the Normative and Topographicknowledge sources in the knowledge-inducing process.

During the knowledge-inducing step of the whole opti-mization process, Normative knowledge plays an importantrole in performing a global search and wide exploration inthe problem landscape via the subset of the population spaceunder its effect. If the parent agent is found in the feasibleregion of the ith dimension for Normative knowledge, thenthe mutation operator is used to obtain the child agent fromthis parent agent. Otherwise, the agent is generated at randombased on a uniform distribution in that feasible search region

of Normative knowledge. The mutation operator affected bythis type of knowledge is given by

xtci =

⎧⎪⎪⎨

⎪⎪⎩

xtpi + ε(ut

i − lti )( f (xt

pi )

/∑n

i=1 f (xtpi )), xt

pi ∈ Vi

lti + ε(ut

i − lti ), xt

pi /∈ Vi

(11)

Here, xtpi represents the ith parent agent before it gets

mutated. xtci is the ith child agent after the mutation hap-

pens, and ε ∼ U (0, 1) a random variable from the uniformdistribution.

Topographic knowledge is more concerned with localsearch and exploiting the region in a cell-based manner. Ifthe parent agents are found in cells that are marked with P−or U , then the new parent individuals are selected randomlyfrom better-attributed cells. These new parent agents are usedto generate new child agents. Otherwise, children are gener-ated via the mutation operator in these cells. The mutationoperator for this type of knowledge is given by

xtci =

⎧⎪⎨

⎪⎩

xtji + ε(ut

j − ltj ), (xt

pi /∈ Ctj ) ∧ (st

k = P−)

xtpi + ε

√f (xt

pi )

m , (xtpi ∈ Ct

j ) ∧ (stk = P+)

(12)

3.3 Knowledge diffusion

Information about the evolution at any time is exchangedusing the knowledge sources. A simple mode of interaction,derived from Reynolds and Ali (2008), is added to the indi-viduals for the sake of speeding up the evolution processat predefined intervals. Each individual is affected in twoways. The first is through the direct knowledge source oper-ator in the belief component (family-like effect). The secondway is through its immediate neighbors in the communica-tion topology (community-like effect). In this simple form ofinteraction, there will be no hops in propagating knowledgeand the whole population is involved in one topology, and thenet effect knowledge source on any individual is based on themajority of neighbors of the individual. In case of a tie, thedirect affecting knowledge source is chosen to resolve thisconflict. Hence, knowledge diffusion is affected by the cho-sen topology of interaction, and knowledge-diffusion win-dow at which knowledge is migrated between the agents inthe population space.

Knowledge-diffusion window (α) is defined to be thenumber of generations between two exodus operations tomigrate knowledge. It is obvious that if the value of α issmall then knowledge is exchanged more frequently andhence this results in losing the diversity aspect of the wholepopulation, which results in getting trapped easily in localminima through search. On the other hand, if the value ofα is large, then knowledge is exchanged at a slower rate

123


and hence the convergence is a little slow. The total num-ber of generation can be expressed in terms of this factor asfollows

Ngen = α × s, s ∈ Z+ (13)

where α > 0.

Knowledge diffusion topology determines how the indi-viduals are connected with each other for knowledge-sharingand the direction of knowledge-diffusion. We will experi-ment with the basic topologies taken from earlier work wherethe authors build networks of agents in a hierarchical contextwith aggregation (Reynolds and Ali 2008). In this work wejust use a simple form of interaction based on the used topol-ogy with no hierarchies, using ring, square, and global topolo-gies. The complexity of the needed communication betweenall the individuals is O(2(N − 1)!) but we will assume thatonly immediate neighbors are to be contacted at the timeof sharing and there are no hops, which reduces this com-plexity to O(N ). Used topologies in CA–TS are illustratedin Fig. 2.

The process of coordinating the movement of knowl-edge among the individuals is kept abstract and sim-ple. Since each individual is affected by knowledge intwo ways unlike the basic CA does, then the choicemethod should decide the final knowledge operator that theindividual should follow. The individual will always fol-low the fittest knowledge source among those affecting it(belief space decision) at a certain time. This is shown asfollows:

K s(xti ) =

⎧⎪⎪⎪⎨

⎪⎪⎪⎩

K sσ , σ ∼ U (1, 2)

K sd (xti ), f (K sd (xt

i )) > Max( f (K sn(xtj ))| j=1→nr

)

K sn(xtj ), ∀ j ∈ (1, nr ) : f (K sn(xt

j )) > f (K sd (xti ))

(14)

Here, σ is a discrete uniform distribution, andK sσ is a ran-domly chosen knowledge source from the adopted knowl-edge sources in this paper. K sd(xt

i ) is the direct knowledgesource influencing individual xt

i , and f (K sd(xti )) is the fit-

ness of this knowledge source at time t . K sn(xtj ) is the knowl-

edge source of an immediate neighbor individual that is con-

nected through the topology to individual xti , and nr is the

number of the neighbors of individual xti .

4 A Cultural Algorithms-Tabu search learning scheme

A combination of both Cultural Algorithms and Tabu search(CA–TS) is proposed to accelerate the convergence of CAin minimum computation time. Most of the cited papers inliterature are used to solve discrete optimization applications(Sha and Hsu 2006; Shen et al. 2008) and they use a com-bination of Genetic Algorithms, Differential Evolution, orParticle Swarm Optimization (Thangaraj et al. 2011).

Similar to other evolutionary algorithms, in highly multi-modal landscapes, Cultural Algorithms might have most ofthe population individuals attracted to a certain local optimaand hence loses its chance of exploring new regions espe-cially if the search radius is less than a pre-specified value.Moreover, the most important flaw of Tabu search is thatit has a strong reliance on the initial solution and its qual-ity. Thus we believe that the quality of solutions producedby Cultural Algorithms when reinforced with an algorithmthat is best known to be able to find promising solutions forfuture steps, and avoid cycling, will distinguish the optimiza-tion process. The proposed combination will also reduce thecomputation time needed to find new fruitful results com-pared to other algorithms that used Tabu search as an auxil-iary search engine while their individuals wander the terrainsearching for the global optima.

4.1 Adapting TS to continuous spaces

TS is first adapted to the continuous spaces as in Siarry andBerthiau (1997). Our algorithm uses the initial solution gen-erated from CA, X = XC A. This solution is considered asthe current solution and is used to generate a set of neigh-bors N (X). In order to escape the situation of having cycles,N (X) that belong to a successively distinct tabu list, are thor-oughly eliminated. Each generated solution in N (X) is eval-uated using the objective function and the current solutionis set to the best neighbor of X , even if it is worse thanX . When we reach a point where the tabu list is full, the

Fig. 2 Topologies ofknowledge-diffusion in thesystem. a Ring. b Square. cGlobal

(a) (b) (c)

123


algorithm updates the list by removing the first entered solu-tion in the history of previously maintained solutions. Thisprocedure continues until the stopping criteria is reached.The stopping criteria is usually based on reaching a certainnumber of iterations without any enhancement or wheneverthe algorithm escapes from the stagnation point reached byCA.

To state clearly how we discretize the solution space, weagain refer to the general setting out in Siarry and Berthiau(1997), where the ‘ball’ concept is used to define the neigh-borhood. A solution X with a radius r will contain the set ofsolutions X ′ in a ball B(X , r), such that

∥∥X ′ − X

∥∥ ≤ r (15)

where ‖·‖ denotes the Euclidean norm. To explore the spacehomogeneously, a set of balls with a center at X will be cho-sen, each with radius r0, r1, . . . , rk . This space becomes par-titioned into concentric crowns Cri (X, ri−1, ri ) and contourregions, such that

Cri (X, ri−1, ri ) = {X ′ ∣∣ri−1 ≤ ∥

∥X ′ − X∥∥ ≤ ri

}. (16)

The k neighbors are randomly selected, one inside eachcrownCri . In this context we use hyper-rectangles insteadof crowns as selecting a point inside a hyper-rectangularregion is much easier than selecting it from a crown.Figure 3 illustrates this adopted partitioning for selectingthese k new solutions, where for instance the new solutionX∗

n = X ′m+4.

Our algorithm accepts the newly generated solution if itis not tabu. The algorithm checks to see that none of tabuballs with centers that are stored in the tabu list contains thisnewly generated neighbor.

Xm*

X’m+1

X’m+2

X’m+3

X’m+4

Fig. 3 The process of partitioning the neighborhood of the currentsolution

4.2 Detailed presentation of the algorithm

Any information about good and bad experiences should beplaced in the belief space. This is necessary to incorporate thisnew knowledge to the list of experiences from a successfulsolution. Promising and tabu lists can be thought of as beingeach an n × d matrix, where n is the number of individualsand d is the number of dimensions.

Since we have three lists, one for the newly generatedpoints, one for the tabu list, and one for the promising list,we merge their current contents to form a list with n × 3individuals. This will be used to update the tabu and promis-ing lists. We record the trail of the best individuals from thebeginning of the run till the end and we denote the set of allthese points that constitute the path as P.

The set of all individuals that is considered in the exteriorof the hyper-rectangular neighborhood of X∗ are not consid-ered in the set. This can be formulated as

Xi ∈ (X∗ ± 3 × i × max{σ }), i = 1, 2, . . . , d. (17)

where σ is standard deviation of positions of all individualsfor the i th dimension. The number of neighbors is chosen tobe k = 4 for all the experiments.

The size of tabu balls is allowed to dynamically changethroughout the run. This change is based on the distribution ofthe tabu and promising positions in the search space. We usethe notion of half the length of the minimum displacementfrom the promising points and tabu points. And hence thetabu ball radius (T Br ) is

T Br = 0.5 × min{

xtj − x p

j

}, j = 1, . . . , n (18)

Such that xtj is a point in the tabu list, x p

j is a point in thepromising list, and n is the number of individuals. Figure 4illustrates the framework of the new system.

Table 4 presents the algorithm of the amalgamated sys-tem (CA–TS). Terms used here are similar to those used toexplain Tables 1 and 2. The topology of communication isconstructed in the function construct_topology(Pt

S), and thefunction generate (Pt

S , influence(BtS)) is where the current

population space is allowed to communicate, and the finalknowledge source operator for each individual is produced.The best solution produced, at each iteration, by CulturalAlgorithms will be checked to see if it is good enough solu-tion using the mean squared error (MSE) according to thefollowing formula (Franti et al. 1998):

M SE = E[(θ̂ − θ)2]

= 1

kST

k∑

i=1

E(i)2 (19)

123


Fig. 4 The Cultural Algorithms–Tabu search (CA–TS) learning scheme

where, k is the number of samples in the neighborhood, ST isthe sampling time, θ̂ is the measured output, θ is the desiredoutput, and E(i) is the difference between them.

M SE = V ar(θ̂) + (Bias(θ̂ , θ))2 (20)

If the quality and fitness of the output from CA (XC A) is fineand there is no stagnation (same solution for α generationsas a threshold) then the output is produced (the optimal X∗ =XC A), otherwise a new neighborhood solution will be createdand evaluated and the individual will be picked based on theaspiration criterion and Tabu search conditions. The last stepthen is to update the tabu list T preparing for the next iterationuntil the stopping criterion is satisfied.

5 Experimental settings

5.1 Initialization and parameters

Some of the most well-known problems in engineering opti-mization will be used as benchmarks to test the performanceof our algorithm. The same set of test problems can also befound in Hu and Eberhart (2002), Hu et al. (2003), Kayhanet al. (2010); Lee and Geem (2005), Kaveh and Talatahari(2010a,b) and Sun et al. (2011). The current instantiation wasimplemented in Java on a Quad-Core Intel Xeon running at3.2 GHZ with 32 GB memory—under OS X Mountain Lion.We ran each problem for 30 independent runs, and set themaximum number of objective function evaluations (FEs)to be 35,000 to prove the power of exploration of embed-ding a tabu search component within the Cultural Algorithmframework in a way that enhances its knowledge dissemina-tion. Competent algorithms from the literature with numbershigher than this limit of FEs have results that are reported

as they were reported by the corresponding authors in theresults section and were not implemented by us. The popu-lation size (Ps) is set as 20, and the size of the tabu list isselected to be 5.

The proportion of agents in the population space to beselected into the belief structure to revise the experiences ofthe knowledge sources (Pa), with a possible value in the set[0, 1], is set in this work to 0.25. The whole population isuniformly distributed over the whole search space. Table 5summarizes these parameters.

In this work, the initialization procedure is asymmetricsimilar to the procedure adopted in Angeline (1998). Thisprocedure limits the search to be in a segment of the feasi-ble region that is equal to half the length from the upper limitalong any axes to the origin. Therefore, as the dimensionalityof optimization problems increases, the asymmetric initial-ization procedure will have the volume of its initializationregion decrease exponentially, as compared to the symmet-ric version that has given limits for a particular problem. Thetest benchmark problems are presented in the following sub-sections.

5.2 Application: optimization of constrained engineeringdesign problems

5.2.1 Design problem 1: spring tension/compression

The problem is a minimization of the weight of a ten-sion/compression spring with respect to constraints that areset on shear tension, surge-rate of recurrence, and minimumdigression as depicted in Fig. 5.

Design variables for this problem are: mean coil diameterx1(=D); line diameter x2(=d), and number of functioningcoils x3(=N ). This problem can be formulated mathemati-cally as follows

123


Table 4 The new hybridCultural Algorithms-Tabusearch framework (CA–TS)

Minimize:

F(X) = x21 x2x3 + 2x2

1 x2, (21)

Subject to:

g1(x) = 1 − x32 x3

71785x41

≤ 0,

g2(x) = 4x22 − x1x2

12566(x2x31 − x4

1 )+ 1

5108x21

− 1 ≤ 0,

g3(x) = 1 − 140.45x1

x22 x3

≤ 0,

g4(x) = x1 + x2

1.5− 1 ≤ 0. (22)

The bounds of the design space are:

0.25 ≤ x1 ≤ 1.3,

0.05 ≤ x2 ≤ 2,

2 ≤ x3 ≤ 15. (23)

123


Table 5 CA–TS main parameters

Function evalu-ations (FEs)

Runtimes

Mutationprobability

Selectionproportion(Pa)

15,000 30 0.06 0.25

Populationssize (Ps)

Tabu listsize

Diffusionwindow

CommunicationTopology

20 5 Tested:(1, . . . , 10)

Ring, square,global

N

Fig. 5 A tension/compression spring

Fig. 6 Structure of the welded beam

5.2.2 Design problem 2: welded beam design

The welded beam design, as depicted in Fig. 6, is a practi-cal engineering problem that is usually used as a gauge forassessing the performance of optimization algorithms by dif-ferent researchers (Coello and Montes 2002; He and Wang2007; Montes and Coello 2003). This problem aims at min-imizing the overall cost of the welded beam. The overallprocess is subject to different constraints on shear tension(τ ), buckling load (Pc), bending tension (σ ), end deviation(δ), and side constraints. Design variables of this engineeringdesign problem are: x1(=h), x2(=l), x3 (=t), x4 (=b).

The problem can be formulated as followsMinimize:

F(X) = 1.10471x21x2 + 0.04811x3x4(14.0 + x2). (24)

Subject to:

g1(x) = τ({x}) − τmax ≤ 0,

g2(x) = σ({x}) − σmax(x) ≤ 0,

g3(x) = x1 − x4 ≤ 0,

g4(x) = 0.10471x21 + 0.04811x3x4(14.0 + x2)−5.0 ≤ 0,

g5(x) = 0.125 − x1 ≤ 0,

g6(x) = δ({x}) − δmax(x) ≤ 0,

g7(x) = P − Pc({x}) ≤ 0, (25)

where

τ(X) =√

(τ ′)2 + 2τ ′τ ′′ x2

2R+ (τ ′′)2,

τ ′ = P√2x1x2

, τ ′′ = M R

J,

M = P(

L + x2

2

), R =

√

x22

4+

(x1 + x3

2

)2

,

J = 2

{√2x1x2

[x2

2

12+

(x1 + x3

2

)2]}

,

σ (X) = 6 P L

x4x23

, δ(X) = 4 P L3

Ex33 x4

,

Pc(X) = 4.013E√

x23 x6

436

L2

(

1 − x3

2L

√E

4G

)

,

P = 6, 000lb, L = 14 in,

E = 30 × 106psi, G = 12 × 106 psi (26)

And the variable bounds are as follows:

0.1 ≤ x1 ≤ 2,

0.1 ≤ x2 ≤ 10,

0.1 ≤ x3 ≤ 10,

0.1 ≤ x4 ≤ 2. (27)

5.2.3 Design problem 3: pressure vessel design

Hemispherical heads are used to clap a cylindrical vesselat both trimmings as depicted in Fig. 7. The objective inthis engineering problem is to minimize the overall cost thatincludes the cost of material, welding, and forming (Sandgren1988).

Fig. 7 Structure of the pressure vessel

123


The main variables of this design problem are: the shell’sthickness x1(=Ts), the head’s thickness x2(=Th), the innerradius x3(=R), and the length of columnar part of the mainvessel x4(=L) excluding the head component. R and L arecontinuous. Th and Ts values are usually in the form n ×0.0625 in. where n is an integer.

The mathematical formulation of this problem is given by:

F(X) = 0.6224x1x3x4 + 1.7781x2x23 + 3.1661 x2

1 x4

+ 19.84x21 x3, (28)

Subject to:

g1(X) = −x1 + 0.0193x3 ≤ 0,

g2(X) = −x2 + 0.00954x3 ≤ 0,

g3(X) = −π x23 x4 − 4

3π x3

3 + 1,296,000 ≤ 0,

g4(X) = x4 − 240 ≤ 0. (29)

And the design space is given by:

0 ≤ x1 ≤ 99,

0 ≤ x2 ≤ 99,

10 ≤ x3 ≤ 200,

10 ≤ x4 ≤ 200. (30)

6 Numerical results and performance analysis

To validate the efficiency and feasibility of CA–TS, the algo-rithm’s performance will be tested and compared on thebenchmark engineering design problems based on differentparameters. The algorithm will then be compared to differentother well-known algorithms from the literature.

6.1 Performance analysis

The process of knowledge dissemination is a crucial fac-tor in the success of the algorithm to figure out the dis-tribution of promising regions in the search landscape, asquickly as possible. For this purpose, the performance ofthe algorithm will be tested on two important related para-meters: (1) How often knowledge is exchanged among theindividuals and (2) the topology of how these individualsshare this information. When testing these parameters, whatwe do is to vary the parameter under study and fix therest. Details of these tests are discussed in the followingsubsections.

6.1.1 Analyzing the performance on varying diffusionintervals

Diffusion window has a great impact on how fast knowledge-update propagates in the whole system. Table 6 presents theperformance of the algorithm at several values of the diffu-sion window (α).

The performance of the algorithm was analyzed basedon two indices. PM1 is a performance measure that showsthe average optimal solution over 30 runs. PM2 shows theaverage objective function evaluations per run, as the optimalsolution was obtained. The total number of objective functionevaluations for all the problems was limited by 35,000 perrun.

Results from the conducted simulations show that atα = 6, the algorithm achieved the best optimal solutionwith the fastest convergence. A small value of α indi-cates that the communication cost is high since informa-tion is exchanged at a higher rate. On the other hand, alarge value of α indicated a slower exchanged rate of infor-mation in which case it has less impact on the overallperformance.

6.1.2 Analyzing the performance on varying interactiontopologies

Communication topologies have a direct impact on the direc-tion of knowledge-diffusion and the propagation-range of

Table 6 Performance comparison for the algorithm with different α

values

Problems F1 F2 F3

α = 1

PM1 0.012898 1.743790 6,115.0134

PM2 14,623 14,899 14,945

α = 2

PM1 0.012739 1.739891 6,098.6666

PM2 14,512 14,824 14,974

α = 4

PM1 0.012671 1.734176 6,091.3119

PM2 14,482 14,796 14,942

α = 6

PM1 0.012665 1.724816 6,059.0134

PM2 14,145 14,758 14,916

α = 8

PM1 0.012669 1.729901 6,065.2703

PM2 14,701 14,902 14,953

α = 10

PM1 0.012673 1.735210 6,094.5511

PM2 14,767 14,918 14,990

123


Table 7 Performance comparison for the algorithm with differenttopologies

Problems F1 F2 F3

Ring

PM1 0.012826 1.739024 6,309.2111

PM2 14,636 14,833 14,912

Square

PM1 0.012665 1.724816 6,059.0134

PM2 14,145 14,758 14,916

Global

PM1 0.012992 1.749981 7,356.2616

PM2 14,728 14,923 14,996

influence. Three communication topologies were tested andthe performance of the algorithm using each one of thesetopologies is shown in Table 7. The algorithm performedbest when we used the square topology, with faster con-vergence and best optimal values. Using the mesh topologygave the individuals enough time to evolve with acceptablecost that is not as good as ring topology but much less thanglobal topology. This advantage makes square topology thebest compared to the two other used topologies.

6.2 Comparing the performance with other algorithms

6.2.1 Results of design problem 1

Belegundu (1982) has reported using eight various opti-mization algorithms that were used to solve this designproblem. The authors in Coello and Montes (2002) used agenetic algorithm (GA) based model to solve this engineer-ing problem. Recently the authors in Montes and Coello(2008) used an enhanced ant colony optimization (ACO)technique for the optimization of this problem. Table 4 com-pares the results obtained by CA–TS to the results reportedby other researchers. Our work will be compared with thebasic CA (Reynolds and Peng 2005), Basic Tabu (Glover1989), an implementation of hybrid PSO and tabu that canbe found in Li et al. (2010), SiC-PSO (Cagnina et al. 2008),

V (μ + 1) − E S (Montes and Coello 2008), and dynamicmulti-swarm particle swarm optimizer (DMS-PSO) (Zhaoet al. 2011). From Table 8, it can be easily seen, by thosefamiliar with this type of engineering problems, that theresults obtained using CA–TS are better than those obtainedby the other algorithms or reported by other researchers, interms of the best, average, worst observed solutions, and theaverage needed FEs (avg. FE) to obtain the optimal solu-tion. Although SiC-PSO scored similar results in terms ofthe found best solution (0.012665), on average CA–TS wasbetter as it has smaller variation between the results andneeded much less computations. The computations that CA–TS needed were way less than any of the other competentalgorithms.


Same methods that we compared the performance of ourresults with in the tension/compression problem will be usedhere as well (Reynolds and Peng 2005; Glover 1989; Li et al.2010; Cagnina et al. 2008; Montes and Coello 2008; Zhao etal. 2011). The statistical simulation results for this problemare summarized in Table 9. The best (1.724816) and aver-age (1.729901) and the worst (1.755654) obtained results byCA–TS are the best among all competent algorithms. More-over, the standard deviation of the results by CA–TS in the 30runs is very small compared to the other algorithms. Again,SiC-PSO obtained a close best value result (1.724852) com-pared to our work (1.724816), but on average CA–TS had avalue of (1.729901) compared to (2.057400) as reported bySiC-PSO. CA–TS needed (14,758) compared to the neededcomputations to obtain the best reported result in the litera-ture by Sic-PSO (24,032).


Statistical simulation results for this problem are given inTable 10. The results show that the average on all the 30 runsusing CA–TS was better than all the competent algorithms,with a value of 6,065.1865. CA–TS was able to get a betterresults for this problem with fewer number of computations,

Table 8 Statistical results ofdifferent algorithms for thetension/compression springdesign

NA not available

Algorithm Best Mean Worst SD Avg. FE

Basic CA 0.012867 0.013191 0.015226 4.178252E−04 25,412

Basic Tabu 0.012935 0.013961 0.015629 4.376190E−03 35,000

Hybrid PSO-Tabu 0.012699 0.012707 0.016898 6.962231E−03 32,021

V (μ + 1) − E S 0.012698 0.013461 0.016485 9.66E−4 25,156

SiC-PSO 0.012665 0.0126 NA 4.1E−04 24,364

DMS-PSO 0.012690 0.012718 0.012928 9.482092E−05 15,000

This work (CA–TS) 0.012665 0.012670 0.012749 2.451731E−05 14,145

123


Table 9 Statistical results ofdifferent algorithms for thewelded beam design

NA not available


Basic CA 1.729813 1.746211 1.790013 0.012827 23,121

Basic Tabu 1.729999 1.749018 1.793249 0.015003 27,265

Hybrid PSO-Tabu 1.725677 1.736644 1.786648 0.009278 22,109

V (μ + 1) − E S 1.737300 1.813290 1.994651 0.070500 21,008

SiC-PSO 1.724852 2.057400 NA 0.215400 24,032

DMS-PSO 1.725489 1.735725 1.773972 0.007985 15,987

This work (CA–TS) 1.724816 1.729901 1.755654 0.001868 14,758

Table 10 Statistical results ofdifferent algorithms for thepressure vessel design

NA not available


Basic CA 6,073.0030 6,091.5371 6,142.5812 038.9999 29,124

Basic Tabu 6,098.6258 6,164.8388 6,521.3836 162.5249 30,089

Hybrid PSO-Tabu 6,071.2647 6,085.6538 6,132.4612 027.4657 22,090

V (μ + 1) − E S 6,059.7456 6,850.0049 7,332.8798 426.0000 25,011

SiC-PSO 6,059.7143 6,092.0498 NA 012.1725 24,213

DMS-PSO 6,072.6294 6,089.5833 6,135.5425 035.4801 18,254

This work (CA–TS) 6,059.1312 6,065.1865 6,090.1113 011.0035 14,916

making it superior to the rest of the algorithms and thosereported in the literature as well (He and Wang 2007; Cagninaet al. 2008; Montes and Coello 2008; Sandgren 1988).

Note that only those approaches that produced comparablybest results without violating the problem constraints wereconsidered for comparison here.

7 Conclusion

In this research, we improved Cultural Algorithms work byembedding Tabu search as a local search algorithm whichgets activated when there is stagnation based on pre-specifiedthreshold. Tabu search is able to guide efficient search for theoptimum when provided with an appropriate initial solutionwhich it derives from Cultural Algorithms here.

Experimental results show that CA–TS makes a noticeableimprovement in the minimization quest of each of the engi-neering problem designs in terms of the best, average, worstvalues obtained over all the runs and most importantly withless number of computations. The standard deviation over allthe runs for these problems was also very small compared tothe other comparable algorithms. For future work we plan tointroduce niche technology to the hybridization of both algo-rithms and enhance the exploration range of Topographicalknowledge further to see how it affects the searching powerand radius of exploration. This technique will be analyzedand applied to other challenging problems such as grid taskscheduling.

References

Angeline PJ (1998) Evolutionary optimization versus particle swarmoptimization: philosophy and the performance difference. In: PortoVW, Waagen D (eds) EP. LNCS, vol 1447, pp 84–89

Belegundu AD (1982) A study of mathematical programming methodsfor structural optimization. PhD thesis. University of Iowa, Iowa,USA

Cagnina LC, Esquivel SC, Coello CA (2008) Solving engineering opti-mization problems with the simple constrained particle swarm opti-mizer. Informatica 32:319–326

Coello CAC, Montes EM (2002) Constraint-handling in genetic algo-rithms through the use of dominance-based tournament selection.Adv Eng Inform 16(3):193–203

De Jong KA (1975) Analysis of the behavior of a class of genetic adap-tive systems. PhD thesis. University of Michigan, Ann Arbor, MI,USA

Dorigo M, Maniezzo V, Colorni A (1996) The ant system: optimizationby a colony of cooperating agents. IEEE Trans Syst Man Cybern B26(1):29–41

Eberhart RC, Kennedy J (1995) A new optimizer using particle swarmtheory. In: Proceedings of the sixth international symposium onmicro machine and human science, pp 39–43

Franti P, Kivijarvi J, Nevalainen O (1998) Tabu search algorithmfor codebook generation in vector quantization. Pattern Recogn.31(8):1139–1148

Fogel LJ, Owens AJ, Walsh MJ (1966) Artificial intelligence throughsimulated evolution. Wiley, New York

Glover F (1989) Tabu search—part I. ORSA J Comput 1(3):190–206He Q, Wang L (2007) An effective co-evolutionary particle swarm opti-

mization for constrained engineering design problems. Eng ApplArtif Intell 20:89–99

Holland JH (1975) Adaptation in natural and artificial systems. Univer-sity of Michigan Press, Ann Arbor

Hu X, Eberhart R (2002) Solving constrained nonlinear optimizationproblems with particle swarm optimization. In: Proceedings of the

123


sixth world multiconference on systemics, cybernetics and informat-ics

Hu X, Eberhart R, Shi Y (2003) Engineering optimization with particleswarm. In: Proceedings of 2003 IEEE swarm intelligence sympo-sium, pp 53–57

Kaveh A, Talatahari S (2010a) A novel heuristic optimization method:charged system search. Acta Mech 213(3–4):267–289

Kaveh A, Talatahari S (2010b) An improved ant colony optimization forconstrained engineering design problems. Eng Comput 27(1):155–182

Kayhan A, Ceylan H, Ayvaz MT, Gürarslan G (2010) PSOLVER: a newhybrid particle swarm optimization algorithm for solving continuousoptimization problems. Expert Syst Appl 37(10):6798–6808

Kirkpatrick S, Gelatt C, Vecchi M (1983) Optimization by simulatedannealing. Science 220(4598):671–680

Koza JR (1990) Genetic programming: a paradigm for geneticallybreeding populations of computer programs to solve problems.Stanford University, Computer Science Department technical reportSTAN-CS-90-1314

Lee KS, Geem ZW (2005) A new meta-heuristic algorithm for contin-uous engineering optimization: harmony search theory and practice.Comput Methods Appl Mech Eng 194(36–38):3902–3933

Li J, Pan Q, Xie S, Jia B, Wang Y (2010) A hybrid particle swarm opti-mization and tabu search algorithm for flexible job-shop schedulingproblem. IJCTE 2(2):189–194

Montes EM, Coello CAC (2003) Adding a diversity mechanism to asimple evolution strategy to solve constrained optimization prob-lems. In: Proceedings of the congress on evolutionary computation2003 (CEC 2003), vol 1, pp 6–13

Montes EM, Coello CAC (2008) An empirical study about the useful-ness of evolution strategies to solve constrained optimization prob-lems. Int J Gen Syst 37(4):443–473

Reynolds R (1994) An introduction to cultural algorithms. In: The 3rdannual conference on evolutionary programming, pp 131–139

Reynolds R, Peng B (2005) Cultural algorithms: computational model-ing of how cultures learn to solve problems: an engineering example.Cybern Syst 36(8):753–771

Reynolds RG, Ali MZ (2008) The social fabric approach for a betterknowledge integration in cultural algorithms. In: The IEEE worldcongress on computational intelligence, pp 1–6

Saleem SM (2001) Knowledge-based solution to dynamic optimiza-tion problems using cultural algorithms. PhD thesis, Wayne StateUniversity, USA

Salehi M (2008) Reverse engineering of temporal gene expression datausing dynamic Bayesian networks and evolutionary search. MScthesis, Queen’s University, Kingston, Ontario, Canada

Sandgren E (1988) Nonlinear integer and discrete programming inmechanical design. In: Proceedings of the ASME design technol-ogy conference, pp 95–105

Sha D, Hsu C (2006) A hybrid particle swarm optimization for job shopscheduling problem. Comput Ind Eng 51(4):791–808

Shen Q, Shi WM, Kong W (2008) Hybrid particle swarm optimizationand tabu search approach for selecting genes for tumor classificationusing gene expression data. Comput Biol Chem 32:53–60

Siarry P, Berthiau G (1997) Fitting of Tabu search to opti-mize functions of continuous variables. Int J Numer MethodsEng 40:2449–2457. doi:10.1002/(SICI)1097-0207(19970715)40:13<2449:AID-NME172>3.0.CO;2-O

Storn R, Price K (1997) Differential evolution—a simple and efficientheuristic for global optimization over continuous spaces. J GlobalOptim 11(4):341–359

Sun C, Zeng J, Pan J (2011) A modified particle swarm optimizationwith feasibility-based rules for mixed-variable optimization prob-lems. Int J Innov Comput Inf Control 7(6):3081–3096

Thangaraj R, Pant M (2011) Particle swarm optimization: hybridiza-tion perspectives and experimental illustrations. Appl Math Comput217(12):5208–5226

Zhao S, Suganthan P, Pan Q-K, Tasgetiren M (2011) Dynamic multi-swarm particle swarm optimizer with harmony search. Expert SystAppl 38(4):3735–3742

123

http://dx.doi.org/10.1002/(SICI)1097-0207(19970715)40:13<2449:AID-NME172>3.0.CO;2-O

http://dx.doi.org/10.1002/(SICI)1097-0207(19970715)40:13<2449:AID-NME172>3.0.CO;2-O

cultural algorithms: a tabu search approach for the optimization of engineering design problems

Documents