[IEEE 2010 Third International Workshop on Advanced Computational Intelligence (IWACI) - Suzhou, China (2010.08.25-2010.08.27)] Third International Workshop on Advanced Computational Intelligence - An efficient Symbiotic Taguchi-based Differential Evolution for neuro-fuzzy network design

Download [IEEE 2010 Third International Workshop on Advanced Computational Intelligence (IWACI) - Suzhou, China (2010.08.25-2010.08.27)] Third International Workshop on Advanced Computational Intelligence - An efficient Symbiotic Taguchi-based Differential Evolution for neuro-fuzzy network design

Post on 08-Mar-2017




2 download

Embed Size (px)


<ul><li><p> AbstractIn this paper, we proposed a functional-link-based </p><p>neural fuzzy network to improve the traditional TSK-type neural fuzzy network. Besides, an efficient evolutionary learning algorithm, called the Symbiotic Taguchi-based Modified Differential Evolution (STMDE), is proposed for the neural fuzzy networks design. Firstly, in order to avoid trapping in a local optimal solution and to ensure the searching capability of near global optimal solution, the STMDE adopts the Taguchi method to effectively search towards the best individual and employs an adaptive parameter control to adjust scaling factor which is called the Taguchi method. Moreover, the proposed STMDE introduces the concept of symbiotic evolution to improve the individual structure. Unlike the traditional individual that uses each one in a population as a full solution to a given problem, symbiotic evolution assumes that each individual in a population represents only a partial solution, while complex solutions combine several individuals in the population. </p><p>I. INTRODUCTION S inheriting the advantages of fuzzy system and neural network simultaneously, neural fuzzy networks (NFNs) </p><p>have been demonstrated their advantage in lots aspects of research. In other words, NFNs have the inference characteristic of the fuzzy system and with the learning ability of the neural network to adjust fuzzy rules automatically. Therefore, NFNs have become a popular research target progressively, and been applied to various applications, such as in the fields of control, prediction, classification and pattern recognition. </p><p>In NFNs, it is necessary to apply some learning algorithm for network parameter adjusting. Many NFN approaches were implemented by using traditional backpropagation (BP) learning algorithm which is based on gradient descents that are known to be easily trapped at local minima. The other drawback of applying the BP algorithm is it will increase the complexity of solving the problem. Recently, the approach of evolutionary computation has been designed to optimize parameters of NFNs and achieved great success. Many researches have been successfully utilized evolutionary algorithms to solve a lot of problems. Several evolutionary algorithms, such as genetic algorithm (GA) and Immune Algorithm (IA), are able to efficiently explore the desired </p><p> Manuscript received March 22, 2010. C.-J. Lin, C.-H. Hsu, S.-Y. Wu and C.-C. Peng are with the Department of </p><p>Computer Science and Information Engineering, National Chin-Yi University of Technology, Taichung County, Taiwan, R.O.C. (e-mail: cjlin@ncut.edu.tw, chhsu828@gmail.com, SiaoYin28@gmail.com, goudapeng@gmail.com) </p><p>global search space, but the drawbacks of local minimum and premature convergence remain the same. Therefore, technologies that can be used to train the system parameters and find the global solution while optimizing the overall structure are generally required. Simultaneously, a new optimization algorithm, called the differential evolution (DE), is an evolutionary computation approach that was developed by Rainer and Kenneth in 1995 [1]. Recently, DE has emerged as a robust numerical optimization algorithm and been successfully applied to tackle various difficult optimization problems [2]. Basically, DE is a fast and easy-to-use method, which is not only astonishingly simple, but also performs extremely well on a wide variety of applications. However, DE sometimes explores too many search points before locating the global optimum. In addition, though DE is particularly simple to work with, i.e., having only a few control parameters, proper choice of these parameters is often critical to the performance of DE [3]. </p><p>In this paper, a learning algorithm, called the symbiotic Taguchi-based modified differential evolution (STMDE) is proposed for designing of a functional-link-based neural fuzzy network (FNFN). Firstly, the proposed STMDE has two crucial ideas to balance the exploration abilities. The proposed STMDE adopts a method to effectively search towards the best individual and employs an adaptive parameter control procedure to adjust the scaling factor of the traditional DE algorithm. Therefore, STMDE does not only explore the search space by randomly chosen individuals, but also exploits the search capability of a near global optimal solution by the best individual currently. In addition, we use the Taguchi method to obtain the better evolutionary direction. Taguchi method was developed by Taguchi and Konishi in 1950 [4], while these techniques have been utilized widely in engineering analysis to optimize the performance characteristics within the combination of design parameters [5]. Tsai et al. [5] proposed a hybrid Taguchi-genetic algorithm (HTGA) which inherits both the merits of powerful global exploration capability of the traditional GAs and exploiting the optimum offspring of the Taguchi method. Taguchi technique is a powerful tool for the design of high quality systems. It introduces an integrated approach which is simple and efficient to find the best range of designs for quality, performance, and computational cost. It is designed for simply value analysis via orthogonal arrays. The characteristic of Taguchi method is to utilize fewer experiment sets and to grant more important information. Furthermore, the Taguchi method can point out the trend of optimization. On the other hand, the STMDE introduces the </p><p>An Efficient Symbiotic Taguchi-based Differential Evolution for Neuro-Fuzzy Network Design </p><p>Cheng-Jian Lin, Chia-Hu Hsu, Siao-Yin Wu, and Chun-Cheng Peng </p><p>A </p><p>179</p><p>Third International Workshop on Advanced Computational Intelligence August 25-27, 2010 - Suzhou, Jiangsu, China </p><p>978-1-4244-6337-4/10/$26.00 @2010 IEEE</p></li><li><p>concept of symbiotic evolution [6] to improve the traditional structure of the DE. Different to the traditional DE that uses each individual in a population as a full solution to a specific problem, symbiotic evolution assumes that each individual in a population represents only a partial solution to a problem. As the result, complex solutions combine several individuals in the population. </p><p>II. STRUCTURE OF THE FUNCTIONAL-LINK-BASED NEURAL FUZZY NETWORK </p><p>This section describes the structure of the FNFN model. The FNFN model adopted the functional link neural network (FLNN) generating complex nonlinear combination of the input variables as the consequent part of the fuzzy rules. Each fuzzy rule corresponds to a sub-FLNN, comprising a functional link. Fig. 1 presents the structure of the proposed FNFN model. The FNFN model realizes a fuzzy if-then rule in the following form. Rule-j: </p><p>1 1 2 2IF is and is ... and is ... and is ,i ij N Njj jx A x A x A x A</p><p>1</p><p>1 1 2 2</p><p>THEN </p><p>... ,</p><p>M</p><p>j kj kk</p><p>Mj Mj j</p><p>y w</p><p>w w w</p><p> =</p><p>=</p><p>= + + +</p><p> (1) </p><p>where xi and jy are the input and local output variables, respectively, Aij is the linguistic term of the precondition part with Gaussian membership function, N the number of input variables, wkj is the link weight of the local output, k the basis trigonometric function of the input variables, M the number of basis function, and Rule-j the j-th fuzzy rule. </p><p>1y 2y 3y</p><p>1 2 M</p><p> Fig. 1. Structure of proposed FNFN model. </p><p>The operation functions of the nodes in each layer of the FNFN model are now described. In the following description, u(l) denotes the output of a node in the l-th layer. </p><p>Layer 1 (Input layer): No computation is performed in this layer. Each node in this layer is an input node, which corresponds to one input variable, and only transmits input values to the next layer directly: </p><p> (1) iiu x= (2) Layer 2 (Membership function layer): Nodes in this </p><p>layer correspond to a single linguistic label of the input variables in Layer 1. Therefore, the calculated membership </p><p>value specifies the degree to which an input value belongs to a fuzzy set in layer 2. The implemented Gaussian membership function in layer 2 is </p><p> (1) 2(2)2 ,</p><p>[ ]exp ijiijij</p><p>u mu </p><p>= (3) </p><p>where and ij are the mean and variance of the Gaussian membership function, respectively, of the j-th term of the i-th input variable xi. </p><p>Layer 3 (Rule layer): Nodes in this layer represent the preconditioned part of a fuzzy logic rule. Here, the product operator described above is adopted to perform the IF-condition matching of the fuzzy rules. As a result, the output function of each inference node is (3) (2) ,j ij</p><p>iu u= (4) </p><p>where (2)uiji of a rule node represents the firing strength of </p><p>its corresponding rule. Layer 4 (Consequent layer): Nodes in this layer are </p><p>called consequent nodes. The input to a node in layer 4 is the output from layer 3, and the other inputs are nonlinear combinations of input variables from a FLNN, as depicted in Fig. 1. For such a node, </p><p> (4) (3)1</p><p>,M</p><p>kj kj jk</p><p>u u w =</p><p>= (5) where wkj is the corresponding link weight of functional link neural network andk is the functional expansion of input variables. The functional expansion uses a trigonometric polynomial basis function, given by </p><p>sin ( ) cos ( ) sin ( ) cos( )1 1 1 2 2 2x x x x x x for </p><p>two-dimensional input variables. Therefore, M is the number of basis functions, i.e., 3M N= , where N is the number of input variables. </p><p>Layer 5 (Output layer): Each node in this layer corresponds to a single output variable. The node integrates all of the actions recommended by layers 3 and 4 and acts as a defuzzifier with </p><p>(3)(4) (3)</p><p>1 11 1(5)</p><p>(3) (3) (3)</p><p>1 1 1</p><p>,</p><p>R MR R</p><p>jkj kjj jj kj j</p><p>R R R</p><p>j j jj j j</p><p>u wu u yy u</p><p>u u u</p><p>= == =</p><p>= = =</p><p> = =</p><p> = =</p><p> (6) </p><p>where R is the number of fuzzy rules, and y is the output of the FNFN model. III. THE PROPOSED SYMBIOTIC TAGUCHI-BASED MODIFIED </p><p>DIFFERENTIAL EVOLUTIONARY ALGORITHM In a general evolution algorithm, a single individual is </p><p>responsible for the overall performance, with a fitness value assigned to that individual according to its performance, it can be found in the traditional DE. It also applies each individual to represent a population as a full solution to a problem. As stated previously, symbiotic evolution assumes that each individual in a population represents only a partial solution to a problem; the goal of each individual is to form a partial solution that can be combined with other partial </p><p>180</p></li><li><p>solutions currently in the population to build an effectively full solution. The general structure of the individuals in the symbiotic evolution is shown in Fig. 2. </p><p>j2j1 ij</p><p> Fig. 2. The representation of a fuzzy system by STMDE. </p><p>The learning process of the STMDE includes the coding, initialization, fitness evaluation, parameter learning and solution aging mechanisms. The flowchart of the proposed STMDE algorithm is shown in Fig. 3(a) and Fig. 3(b), while the whole learning process is described phase-by- phase as follows. </p><p>A. Initialization Phase The coding step is related with the membership functions </p><p>and fuzzy rules of a fuzzy system that represent sub-individuals suitable for symbiotic evolution. The initialization step assigns the population values before the evolution process begins. </p><p> Coding Step The foremost step in the STMDE is the coding of the </p><p>individual (rule) into a fuzzy system. Fig. 4 shows an example of the coding of parameters of the neural fuzzy network into a fuzzy system where i and j represent the i-th input variable and the j-th rule, respectively. ijm and ij are the mean and variance of a Gaussian membership function, respectively, and kjw represents the corresponding link weight of the consequent part that is connected to the j-th rule node. In this study, a real number is used to indicate the element of each rule. </p><p>Create Initial Population Before the STMDE method is applied, every individual gix , must be created randomly within the range [0, 1], where </p><p>i=1, 2, , PS, g is the generation index and PS the population size. </p><p> Fig. 3(a). Flowchart of the proposed STMDE designs method. </p><p>A</p><p>B</p><p>Create a mutated individual</p><p>Crossover</p><p>MutationBuilding a orthogonal arrays for the Taguchi </p><p>experiment </p><p>Set up two levels for three factors</p><p>The end of all experiments?</p><p>Compute the SNR of this experiment</p><p>Record the best levels</p><p>No</p><p>Yes</p><p> Fig. 3(b). Flowchart of the Taguchi method in the TMDE algorithm. </p><p>j1 j2 ij Fig. 4. Coding FNFN model into a fuzzy system in the STMDE. </p><p>B. Evaluation Phase The fitness value of a fuzzy system is computed by the </p><p>fitness values of all the feasible combinations of that rule with all other randomly selected rules. The details of assigning the fitness value are described as follows. Step 1: Randomly select R fuzzy rules (individuals) from each subgroup with size PS for composing the fuzzy system. Step 2: Evaluate every fuzzy system that is generated from step1 to obtain a fitness value. The fitness function is defined as follows. </p><p>( )2</p><p>1</p><p>1 ,11</p><p>tNd</p><p>k kt k</p><p>fitness value</p><p>y yN =</p><p>=</p><p>+ </p><p> (7) </p><p>where yk represents the model output of the k-th data, dky the desired output of the k-th data, and Nt the number of the training data. Step 3: Compare fitness value of each current fuzzy system with the one of the best fuzzy system. If the current fitness value is higher than the best fuzzy system has, then the best </p><p>181</p></li><li><p>fuzzy system will be replaced with the current fuzzy system. Step 4: Repeat the above steps until each rule (individual) in each subgroup has been selected a sufficient number of times. </p><p>C. Parameter Learning Phase In this phase, the STMDE performs the parameter learning </p><p>based on a sub-population, so all individuals are updated using the proposed TMDE method. Each sub-population allows the individual (rule) itself to evolve by evaluating the composed fuzzy system. Fig. 5 shows the structure of the individual in the rule-based symbiotic modified differential evolution. </p><p> Fig. 5. Structure of the individual in the STMDE. </p><p>The TMDE comprises of two major sub-phases, i.e., the procedures of parents choice and reproduction, which are summarized as follows. </p><p>1. Sub-phase of Parent Choice with Taguchi method In order to obtain the better evolutionary direction, we have </p><p>used the Taguchi method to determine the best parent combination according to some experimental results. The Taguchi method is based on orthogonal array experiments that provide many reduced variances and with optimal setting of control parameters. In this paper the two-level orthogonal array has been applied. The general symbol for two-level standard orthogonal arrays is defined as </p><p> ( ),ca</p><p>L b (8) </p><p>where L represents the symbol of Latin square, a the number of the experimental, b the level of each factors, and c the number of factors. </p><p>Experimental Factors Determination and Orthogonal Array Construction In traditional DE algorithm, each individual in the current </p><p>generation is allowed to breed through mating with other randomly selected individuals from the population. Specifically, for each individual ,i gx , i=1, 2, , PS, where g denotes the current g...</p></li></ul>