bee system- an improved genatic algorithm

7/27/2019 Bee System- An Improved Genatic Algorithm

1/6

Bee System: Finding Solutionby a Concentrated Search

TOMOYAAT0 A N D MASAFUMIHAGIWARADepartment of Electrical Engineering, Faculty of Science and TechnologyKeio University

3-14-1 Hiyoshi, Kouhoku-Ku, Yokohama 223, JAPANe-mail: [email protected]

A b s t r a c tIn this pap er we propose an improved genetic algo-r i thm named Bee System. The concept of the BeeSystem com es from beha vior of bees: first, a beefinds feed and then it notifies the information to theother many bees by dance to work together. In theproposed Bee System, each chromosome tries t o findgood solution individually. When some chromosomeis regarded as superior one, other chromosomes tryto find solution around there using multiple popula-

tions. Such a procedure is repeated. The Bee Systememploys some new operations such as concentratedcrossover, an d Pseudo-Simplex Method. By computersimulations it is confirmed that the Bee System hasbetter performance than the conventional genetic al-gorithm.1 Introduction

Nowadays man y kinds of systems have been becom-ing more and more com plicated. Ad justm ent of theparam eters in such a system is a very importa nt prob-lem. It can be regarded as an optimization problemof mu ltivaria te functions. Tra ditiona l optimizationmethod s[l ] such as Quasi-Newton method have someshortcomings.

0 They t end to suff er from excessively slow conver-gence.Many of them require some special informationsuch as grad ient of the objective function.

0 If they fall into a local optim um , it is difficult toescape from the point .Genetic Algorithms (GA S) have recently attr act edmuch at tent ion as a new optimization technique.Many applications using GAS have been proposed.For example, gas pipeline control, design planningof airplane, image pattern matching and so on[2]-[9].Techniques combining GAS with neural networks orfuzzy systems have been also studied[2][11]. Thus itis thought that GAS are ones of t he very impor tan toptimization me t hods.

0-7803-4053-1/97/$l0.001997EEE

It is certain th at GAS have good global search abil-ity, however, they lack the local search ability [2]-[9][14][15][19]. To im prove this proble m severa l meth-ods have been proposed before.0 Combining GA with Quasi-Newton method[l5]:One of the shortcomings of this method is that

it requires gradient of the objective function.Forking GA[16]: One of the shortcomings of thismetho d is complexity of the algorithms to ob tainsuperior schem ata.

0 GAMA S[14]: Th e idea of GAMAS is very inter-esting, however, a l i t t le improvement is reportedfor local search.I t i s thought tha t these s tudies are not sufficient toovercome the shortcoming.Artificial Life, which is considered as a largerframework of G AS, also has been studie d actively[l2].For instance, Marco Dorigo et al. proposed Ant Sys-tem which originated from an analogy with coop-erative work of ants[lO]. We believe that observa-tion of na tura l system can b e a n invaluable source ofinsp irati on[ 101.In this pap er, we propose a n improved GA inspiredby the bee colonys function. We call i t Bee System.T h e bee colonys function which we refer is as follows.First, each bee belonging to a colony looks for th e feedindividually. When a bee finds feed, then it notifiesthe information to the other many bees by dance,and they engage in a job to carry the feed. Whenthey finish the work, each bee tries to find new oneindividually again.In the proposed Bee System, global search is donefirst, and some chromosomes with pretty high fit-ness are obtained using the simple GA. These chro-mosomes are called superior chromosomes. Second,many chromosomes obta in the inform ation of supe-rior chromosomes by th e concen trated crossover, andthey search intensively around there using multiplepopulations. In addition, we modify the SimplexMetho d[l?] which is popular as one of the optimiza-tion techniques and combine it with the Bee System.The features of the Bee System are as follows:

3954
mailto:[email protected]:[email protected]


2/6

Probability of falling into a local optim um is lowbecause of the combination of local search andglobal search.Proposed c oncen trated crossover can concen tratemany chrom osomes on th e areas where the globaloptimum might exist.Modified Simplex Method named Pseudo-Simplex Method contributes to enhance th e localsearch ability of the Bee System.

Th e remainder of this paper is organized as follows:Section 2 reviews Genetic Algorithms; Section 3 pro-vides details of the Bee System; Section 4 hows theresults of com puter simulations.2 Genetic Algorithms

GAS are th e effective algorithnls for search and op-t imization problems based on an analogy with theprocesses of evolution and adaptation of natural l ife.Fig.1 shows a flowchart of the con ventional GAS. InGASeach can didate for solution for the prob lem is en-coded in to a linear list of th e symb ols, which is calledchromosome. T he G A generates a group of chromo-somes called population, and three basic operations,selection, crossover, and mutation are applied to it .Selection is a process to keep chromosom es with highfitness value. Crossover is a process for exchanginggenes of two chromosomes to create higher fit ones.Mutation is the occasional alternation of some genevalues in chromosomes.Compared with the other opt imizat ion methods,GAS have the following features[l3]:

Since GAS work in parallel on a num ber of searchpoints , they are no t easily caught in a local op-t imum.

e GAS do not need derivative of th e objective func-t ion but only need to evaluate each candidate forsolution.For these feature s, i t is possible to find th e most suit-able values or near suitable values for various prob-lems effectively.It is said th at , however, the disadvantage of GAS istheir lack of the local search ability. Th e reasons arethought as follows:

Since the search is based on bit strings repre-sented as chromosomes, the search points arenot necessarily proximal each other on the ac-tual search space.Generally sp eaking, since the local search abilityconflicts with the global search ability, the globalsearch ability degrades if we try to improve thelocal search ability without consideration of th ebalance.

LStartjInitializing population I

CrossoverMutation

Figure 1: Genet ic Algori thm.

We should consider the se points t o improve th e localsearch ability.3 Bee System

As we mentioned above, the Bee System is basedon th e functions of bees colony. T he p urpose of th eBee System is to improve the local search ability ofGAS without degrading t he global search ability.T h e Bee System uses multiple populations as shownin Fig.2; one pop-G for global search, and some

pop-L;( i = 1 , . .n ) for local search. Fir st, globalsearch is done by using pop-G. If one chromo-some which is regarded to have pretty high fitnessvalue is found, it is kept for local search as SuperiorChromosome. I t corresponds to a bee which findsfeed. After finding n Superior Chromosomes (SCI-SC,), local search starts. In the local search, allof th e chromosomes in POp-Lk t ry to search aroundthe SCk intensively. To realize this concentratedsearch, we introduce concentrated crossover corre-sponding to bees dance. Basically multiple popu-lations (pop-L] - pop&) work independently, buta little exchange is desired for multiple populationmodels [14]. Thus we apply Migration t o pop-L . W ealso introduce Pseudo-Simplex Method t o enhance thelocal search ability. When th e search around SCs isover, the best solution found is judged whether it sat-isfies the prede termined c onditions or not. If not, theBee System re tu rns to the global search mode again.

Now we explain details of Superior Chromosome,concentrated crossover, Migrataon, and Pseudo-Simplex Method.3. 1 Superior Chromosome

In the Bee System, first global search is done bypop-G. Th e purpose is to search as widely as possibleto avoid falling into a local optimum. In the pop-G,simple GA is applied in every generations.

3955


3/6

Local search

Figure 2: Bee System.

If one chromosome is the best for successive G,, en -erations, it is considered as very good one aroundwhich there may be the global optim um . We callit Superior Chromosome (SC), and keep it for localsearch. If one chromosome is regarded as SC, pop-Gis initialized a nd the search begins again. Such a pro-cedure is repeated n times. Fig.3 shows the flowchartof global search, where mdc s the counter represent-ing the number of generations in which max fitnessdid not change, and k is the counter representing th enumber of Superior Chromosomes saved already.In p o p - G , in order to keep variety of genes, muta-t ion r ate is set at comparatively high value and theelitist strategy[U] is not employed.3.2 Concentrated crossover

At the beginning of local search, all of the chromo-somes in pop& make couple with sck and crossoveroperation is applied. In t he conventional crossovereach pair is made randomly, while in this concen-trated crossover all of the chromosomes make pairwith sck.This concentrated crossover transmits in-formation abo ut th e kth Superior Chromosome sckto all of the chromosomes in the kth populationP O P - L k . So POp-Lk can try to search concentratedlyaround sck.3.3 Migration

As we mentioned above, basically all of the pop-ulations are independent each other. But i t is moreeffective to comm unicate with other populations. Th eBee System selects one individual per predeterminedgeneration Gmig, nd transfers i t t o the neighboringpopulation. It is called M igra t ion . For this Migra-tion, each population tries to search independentlyand cooperatively.3.4 Pseudo-Simplex Method

For more effective search, a simplified SimplexMethod named Pseudo-Simplex Method is intro-duced. Here we briefly explain the Simplex M ethodfirst. T h e geometrical figure formed by a set of n +1

I Crossover I k = k + l 1I 1I

Saving the superiorchromosome as SCtMutation I

IReset : m s c = O 1I1( Local search 3

Figure 3: Flowchart of global search.

points in th e n-dimensional space is called a simplex.In the two-dimensional case, the simplex' becomes atriangle. T he basic idea of the Simplex Method isto move the simplex gradually toward th e op timumpoint by an iterative process using th ree operations:reflecteon, expansion, and contraction. One of themost at trac tive features of the Simplex Metho d isthat it does not require derivative of the object ivefunction[l7].For the sake of simplicity in th e proposed Pseudo-Simplex Me thod , we always use jus t 3 poin ts even ifthe dimensions of the objective function are highert ha n 2. One of the authors proposed a G A combinedwith a simplified Simplex Method[lS]. Since it con-sidered only re f l e c t ion , however, it might go passthe optimum point . On the other hand , the proposedPseudo-Simplex Method utilizes not only re f l e c t ionbut a lso contrac t ion for more effective search.

In every generation, the following algorithm isapplied. Fig.4 shows the schematic expression ofthe proposed Pseudo-Simplex Method in a two-dimensional case.1) Pick up the best three chromosomes, and name

them C1,C2, an d Cs in order of fitness.2) Translate them into vectors; X l ,X 2 , and Xs ,respectively.3) Calculate the middle point of XI and X2, an dmake XO s follows:

3956


4/6

Figure 4: Pseudo-Simplex Method in a two dimen-sional case. (a=0.5,/3 =0.3)

Mutat ionra te

4) Calculate X,,, as follows:X,,f =(1+a )X , -ax,. (2)

This s tep corresponds to re flection. Where a isa cons tant which represents th e reflection ratio.

pop-G 0.05pop-L 0.005

(3 )

GmigG*Cnumber of pop-L :nparameters for aPSM p

5) Calculate Xcont s follows:Xcont =(1-P)XO+PX3- (4)

This s tep corresponds to contraction. Where pis a constant which represents the contraction

5203

0.40.4

(5)6 ) Exchange X,,j and Xcontnto chromosomes andmake them Crefand Ccontrespectively.7) Introduce Cl,Cref nd Cconto th e original pop-ulation to which crossover and m utation have al-ready applied.

3.5 Return to global searchAfter passing the predetermined generations, thelocal search stops . If th e best solution found sofar does not suffice the endin g condition, the globalsearch starts again and the algorithm is repeated.Th e flowchart of the local search is summarized inFig.5.

4 SimulationsWe have done a series of computer simulations toconfirm th e validity of the Bee System. We comparedit with the conventional GA. In addition, we exam-ined the effects of the c once ntrated crossover and th ePseudo-Simplex Met hod.

3957

+-,Migration

Pseudo sim lexmethod-Figure 5: Flowchart of local search.

Tab le 1: Parameters used in simulations.

4.1 Simulation conditionsWe used gray-code coding, two-point crossover andproportionate selection in both the Bee System andthe conventional GA. M utat ion rate was 0.005, andthe elitist strategy was employed in the conventionalGA. Other parameters used in the Bee System aresummarized in Table 1 . These p arameters were deter-mined by preliminary sim ulations, they are not neces-sarily the best. For each function, twenty trials weredone to obtain averaged data .We used nine test functions fl-f9, which are sum -marized in Table 2. They are widely used in GAS

comm unity[ 181[191 20].To compare the performance of t h e Bee Systemwith the one of the conventional G A fairly, chromo-somes were evaluated tota l 15000 times per on e trialfor fl-f5, and total 50000 times per one trial for f6-D,n both the Bee System and the conventional GA.For the same purpose, the Bee System did not returnto a global search mode ag ain, tha t is, only one cyclewas executed in each trial.


5/6

Table 2: Test functions used in simulations.Functions

-3 2 -16 0 16 32 -32 -16 ...-32 -32 -32 -32 -32 -16 -16 .. . 32 32 32

conventionalGA PW\ I

5 f6 fl f8 f9Figure 6: Simu lation Results. (For each function,the left bar shows the resul t by the conventional GAand the r ight bar shows the one by the proposed BeeSystem.)

We introduce the Normalized error to evaluatetheir performances. W hen the number of chromo-some evaluat ions reaches the predetermined number,we calculate the normalized error E, of the best so-lution as follows:x 100 (%).-utI,", - ot, =

Where x is the best solution found in the trial, Otis t he Theore t ica l Opt im um , and I,,, is the Aver-aged best solution in the Initial population. By thenormalizing if the theoretical optimum is found, thenormalized error E, becomes 0% .4.2 Simulation results

Fig.6 compares the Bee System with the conven-tional GA in respect of the success rate. For every

without pseud simplex method

~ 0 %f l f2 f3 f4 f5 f6 t7 f8 f9

Figure 7: Effect of the concentrated crossover andthe Pseud eSimplex method. (For each funct ion, theleft bar shows the result by the Bee System withoutthe concentrated crossover, the center bar shows theone by the Bee System without the Pseudo-SimplexMethod, and th e r ight bar shows the one by the com-plete Bee System.)

function, the proposed Bee Sysiem shows better per-formance than th e conventional GA. Let us take f8,for example, about 30% of the trials by the conven-tional G A could n ot find a solution whose norm alizederror was within 2.0%. And the opt imum solut ion,whose normalized error w as 0% , was found in only25 % of the trials. While th e proposed Bee Systemcould find the optimum solution in all of the trialsunder t he sam e condi t ions.Since f l is a very simple function, however, the con-ventional GA shows the sam e performance as th e BeeSystem. I t i s thought tha t the proposed Bee Systemis more effective especially for highly complex m ulti-variate functions.

3958


6/6

In order to examine the effect of the concentratedcrossover and the Pseudo-Simplex Method, we triedth e Bee System withou t, using each of them . We com-pared the resul ts wi th the complete Bee System. Fig.7shows the results. It can be seen that t he Bee Sys-te m without either the concentrated crossover or thePseudo-Simplex Method, has lower ability than thecomplete system. Th us we could verify their effec-tiveness.

5 ConclusionIn this paper we have proposed an improved Ge-netic Algorithm named Bee System. It is based onthe bees colony function. In the proposed Bee Sys-tem, f i rst p ret ty good chromosomes are found b y us -ing a population for the global search, then the con-centrated search around them is carried out by usingsome populations for th e local search. If the solu-tion found by one cycle is not satisfactory, the globalsearch is repeated . We have introduced new oper-ations: the concentrated crossover and the Pseudo-Simplex M ethod. Because of these techniques and

good balance between global search and local search ,the proposed Bee System can obtain high ability forlocal search without degrading t he global search abil-ity.We have confirmed the v alidity of th e proposed BeeSystem by computer s imulat ions. I t i s found thatth e Bee System is mo re effective especially for highlycomplex multivariate functions.

AcknowledgmentThis work is par t ly suppor ted by th e Ja pan Soci-ety for the Promotion of Science (Research for theFuture Prog ram , JSP S-R FT F 96 I 00102).

ReferencesT.Ibaraki and M.Fukushima: OptimizatinMethods, Kyoritsushupan, 1993. (in Japanese)M.Hagiwara: Ne uro, Fuzzy, and G enetic Algo-rithms, Sangyotosho, 1994. (in Japane se)H.Kitano: Genetic Algorithms, Sangyotosho,1993. (in Ja panese )H.Kitano: Genetic Algorithms &, Sangyoto-sho, 1995. (in Japanese )T.Agui and T.Nagao: Genetic Algorithms,Shokodo, 1993. (in Japane se)L.Davis: Handbook o f Genetic Algorithms,Moriki tashuppan, 1994. ( in Japanese)Y.Davidor: G ene tic Algorighm s and robotics,Baifukan, 1996. (in Japanese )Y.Yonezawa: Genetic Algorithms, Moriki-tashup pan, 1993. ( in Japanese)

[9] HKawanishi and h4.Hagiwara: A Shape De-tection Method using Improved Genetic Algo-r i thm, Trans. IEE of Japan, vol.llG-C, No.8,pp.891-897, Aug. 1996.[ lo] M.Dorigo, V.Maniezzo, and A.Colorni: AntSystem: Optimization by a Colony of Cooper-at ing Agents, IEEE Trans. Syst., Man, andCybern. PartB: Cybern., ~0 1 . 2 6 ,No.1, pp.29-41,

Feb. 1996.[I 13 H.Ishiuchi, K.Nozaki, N.Yamamoto, andH.Tanaka: Selectiong Fuzzy If-Then Rulesfor Classification Problems Using Genetic Al-gori thms, IEEE Trans. Fuzzy Systems, vo1.3,No.3, pp.260-270, Aug. 1995.[l2] K.Ha t tori: The world of artificial life,Ohmsha, 1994. (in Japanese)[13] D.E.Goldberg: Genetic Algorithms in Search ,Optimization & Machine Learning, AddisonWesley, 1989.[14] J.C .Po tts, T.D.G iddens, and S.B.Yadav: Th eDevelopment and Evaluation of an Improved Ge-netic Algorighm Based on Migration and Arti-

ficial Selection, IE EE Trans. on Syst., Man, andCybern. vo1.24, pp.73-86, J a n . 1994.[15] J.M.Renders and S.P.Flasse: Hybrid Meth-ods Using Genetic Algorithms for Global Op-t imizat ion, IE EE T rans. Syst., Man, and Cy-bern. PartB: Cybern., vo1.26, No.2, pp.243-258,Apr ~ 9 9 6 .[16] S.Tsutsui and K.Fujimoto: Forking GA , Jour-nal of Japanese Society f o r Artificial Intelli-gence, vol.9, No.5, pp.741-747, 1994.[17] S.S.Rao: Optimization theory a nd applications(second edition), John Wiley & Sons, pp.292-300, 1984.[18] K.A.De Jon g: Analysis of th e behavior of aclass of genetic adaptive systems, TechnicalReport No.185, Department of Computer andCommunication Sciences, University of Michi-gan, 1975.1191 M.H agiwara: Pseudo -Hill Climbing Gen etic Al-gori thm (PHGA) for Funcion Optimizat ion,

Proceedings of 1993 International Joint Con-ference on Neural Networks. vol.1, pp.713-716,1993.

[20] H.M uhlenbein, M.Schomisch, a nd J.Born: Th eParallel Genetic Algorithm as Function Opti-mizer, Proceedings of the fourth internationalconference on Genetic Algorithms, pp.271-278,1991.

3959

bee system- an improved genatic algorithm

Documents