SICE-ICASE International Joint Conference 2006Oct.18-21, 2006 in Bexco, Busan, Korea

Evolutionary design of Self-Organizing Fuzzy Polynomial Neural Networks for modeling andprediction of NOx emission process

Ho-Sung Park', Kyung-Won Jang', Sung-Kwun Oh2, and Tae-Chon Ahn'Department of Electrical Electronic and Information Engineering, Wonkwang University, Chon-Buk, Korea

(Tel: +82-63-850-6342; E-mail: {neuron, jjang, tcahn} gwonkwang.ac.kr)2 Department of Electrical Engineering, Suwon University, Gyeonggi-do, Korea

(Tel: +81-31-222-6544; E-mail: ohskgsuwon.ac.kr)Abstract: In this study, we proposed genetically dynamic optimized self-organizing fuzzy polynomial neural networkwith information granulation based FPNs (gdSOFPNN), develop a comprehensive design methodology involvingmechanisms of genetic optimization. The proposed gdSOFPNN gives rise to a structurally and parametrically optimizednetwork through an optimal parameters design available within FPN (viz. the number of input variables, the order of thepolynomial, input variables, the number of membership functions, and the apexes of membership function). Here, withthe aid of the information granulation, we determine the initial function being used in the premised and consequencepart of the fuzzy rules respectively. The performance of the proposed gdSOFPNN is quantified through experimentationthat exploits standard data already used in fuzzy modeling

Keywords: genetically dynamic optimized self-organizing fuzzy polynomial neural network, information granulation,FPNs, Genetic algorithm.

1. INTRODUCTION

When the dimensionality of the model goes up (thenumber of system's variables increases), so do thedifficulties. In the sequel, to build models with goodpredictive abilities as well as approximation capabilities,there is a need for advanced tools[1].To help alleviate the problems, one among the first

approaches along systematic design of nonlinearrelationships between system's inputs and outputscomes under the name of a Group Method of DataHandling (GMDH)[2]. GMDH-type algorithms havebeen extensively used since the mid-1970's forprediction and modeling complex nonlinear processes.

While providing with a systematic design procedure,GMDH comes with some drawbacks. To alleviate theproblems associated with the GMDH, Self-OrganizingNeural Networks (SONN, called "SOFPNN") were

introduced by Oh and Pedrycz [3-5] as a new categoryof neural networks or neuro-fuzzy networks. Althoughthe SOFPNN has a flexible architecture whose potentialcan be fully utilized through a systematic design, it isdifficult to obtain the structurally and parametricallyoptimized network because of the limited design of thenodes located in each layer of the SOFPNN.

In this study, in considering the above problemscoming with the conventional SOFPNN, we introduce a

new structure and organization of fuzzy rules as well as

a new genetic design approach. The determination ofthe optimal values of the parameters available within an

individual FPN (viz. the number of input variables, theorder of the polynomial, a collection of preferred nodes,the number of MF, and the apexes of membershipfunction) leads to a structurally and parametricallyoptimized network through the genetic approach.

2. SOFPNN WITH FUZZY POLYNOMIALNEURON AND ITS TOPOLOGY

The FPN consists of two basic functional modules.

The first one, labeled by F, is a collection of fuzzy setsthat form an interface between the input numericvariables and the processing part realized by the neuron.The second module (denoted here by P) is about thefunction - based nonlinear (polynomial) processing.The detailed FPN involving a certain regressionpolynomial is shown in Table 1.

Table 1 Different forms of regression polynomialbuilding a FPN.

N.of inputs

The polyn1 2 3

Order FPN0 Type 1 Constant Constant Constant1 Type 2 Linear Bilinear Trilinear2 Type 3 Quadratic Biquadratic-1 Triquadratic-1

Type 4 Biquadratic-2 Triquadratic-21: Basic type, 2: Modified type

Proceeding with the SOFPNN architecture essentialdesign decisions have to be made with regard to thenumber of input variables and the order of thepolynomial forming the conclusion part of the rules aswell as a collection of the specific subset of inputvariables.

In Table2, notation A: Vector of the selected inputvariable (x1, X2, .., xi), B: Vector of the entire systeminput variables (xI, x2, ..., xi, xi, ...), Type T :1(A)=f(x1,x2, ..., xi)- type of a polynomial function standing in theconsequence part of the fuzzy rules, Type T*: J(B)=f(x1,.x2 , xj ...) - type of a polynomial function occurring

in the consequence part of the fuzzy rules

Table 2 Polynomial type according to the number ofinput variables in the conclusion part of fuzzy rules.

Inputs vector Selected input Selected inputType ofthe variables in the variables in theConsequence polynom premise part consequence part

Type T A AType T * A B

89-950038-5-5 98560/06/$10 © 2006 ICASE3796

3. THE STRUCTURAL OPTIMIZATION OFGDSOFPNN

3.1 Information granulation by means of HardC-Means clustering method

Information granulation is defined informally aslinked collections of objects (data points, in particular)drawn together by the criteria of indistinguishability,similarity or functionality [6]. Granulation ofinformation is a procedure to extract meaningfulconcepts from insignificant numerical data and aninherent activity of human being carried out with intendof better understanding of the problem. We extractinformation for the real system with the aid of HardC-means clustering method[7], which deals with theconventional crisp sets. Through HCM, we determinethe initial location (apexes) of membership functionsand initial values of polynomial function being used inthe premise and consequence part of the fuzzy rulesrespectively.

The fuzzy rules of gdSOFPNN is as followings.

Ri :If xl is Aji and * Xk is Ajk then yj -M

sub-chromosome contains the number of input variables,the 2nd sub-chromosome involves the order of thepolynomial of the node, the 3rd sub-chromosomecontains input variables, and the 4th sub-chromosome(remaining bits) involves the number of MF coming tothe corresponding node (FPN).[Step 5] Design of structurally optimized gdSOFPNN.In this step, we design the structurally optimizedgdSOFPNN by means ofFPNs that obtained in [Step 4].[Step 6] Identification of membership value usingdynamic searching method of GAs.

(1)=fj {(xl vl),(X2 Vj2)' ,(Xk Vjk)}

Where, A1k mean the fuzzy set, the apex of which isdefined as the center point of information granule(cluster). Mj and Vjk are the center points of new createdinput-output variables by information graunle.

3.2 Genetic optimization of gdSOFPNNThe main features of genetic algorithms concern

individuals viewed as strings, population-basedoptimization and stochastic search mechanism(selection and crossover). In order to enhance thelearning of the gdSOFPNN and augment itsperformance, we use genetic algorithms to obtain thestructural optimization of the network by optimallyselecting such parameters as the number of inputvariables (nodes), the order of polynomial, inputvariables, and the number of MF within a gdSOFPNN.Here, GAs uses serial method of binary type,roulette-wheel as the selection operator, one-pointcrossover, and an invert operation in the mutationoperator [8].

4. THE ALGORITHM AND DESINGPROCEDURE OF GDSOFPNN

The framework of the design procedure of thegdSOFPNN with aid of the Information granulation(IG) comprises the following steps.[Step 1] Determine system's input variables.[Step 2] Form training and testing data.[Step 3] Decide initial information for constructing thegdSOFPNN structure.[Step 4] Decide FPN structure using genetic design.We divide the chromosome to be used for geneticoptimization into four sub-chromosomes. The 1St

Fig. 1 Identification of membership value usingdynamic searching method.

[Step 7] Design of parametrically optimizedgdSOFPNN.The fitness function reads asF(fitness function) = 1/E (2)Where, E means the objective function with weightingfactor(E= 0<PI+(l -O)xEPI).

5. EXPERIMENTAL STUDY

We illustrate the performance of the network andelaborate on its development by experimenting withdata coming from the NOx emission process of gasturbine power plant [9]. The input variables include AT,CS, LPTS, CDP, and TET. The output variable is NOx.We consider 260 pairs of the original input-output data.130 out of 260 pairs of input-output data are used aslearning set; the remaining part serves as a testing set.To come up with a quantitative evaluation of network,we use the standard MSE performance index.

3797

Table 3 Computational aspects of the genecticoptimization ofgdSOFPNN.

Parameters 1St layer | 2nd layer 3rd layerMaximum generation 100Total population size 300xNo. of 1st layer node

GA Crossover rate 0.65Mutation rate 0.1String length 90

Maximal no.(Max) of 1 < / 1 < I< 1 < /<inputs to be selected Max(2-3) Max(2-3) Max(2-3)

Polynomial type (TypegdSO T) ofthe consequent 1 <T* <4 1 < T < 4 1 < T < 4FPNN part of fuzzy rules

Membership Function Triangular Triangular Triangular(MF) type Gaussian Gaussian Gaussian

No. ofMFs per input 2 or 3 2 or 3 2 or 3I, T, Max: integers

Table 4 shows the performance index of the proposedgdSOFPNN.

Table 4 Performance index ofgdSOFPNN for theNOx process data.

Layer 3rd layerModel M.F Triangular MF Gaussian MF

Max PI EPI PI EPI2 0.016 0.068 0.012 0.180

gdSOFP 3 0.014 0.036 0.004 10.134gdSOFP Triangular MF* Gaussian MF*

2 0.003 0.017 0.002 0.0243 0.002 0.008 0.001 0.023

Fig. 2 illustrates the detailed optimal topologies ofthe gdSOFPNN for 3 layers (PI=0.002, EPI=0.008).

Tamb-

LPT . 2X.)yPcd w

Texh

Fig. 2 gdSOFPNN architecture.

0.03

0.0250

a)c 0.02

bfl

.5 0.015

.HC)nni

0.005

50 100 150 200 250 300 350 400 450 500 550

Generation

(a) Training data error

0.6IG_gSOFPNN gdSOFP

0.5

0.4

0.3

0.2

0.1

C0 50 100 150 200 250 300 350 400 450 500 550

Generation

(b) Testing data errorFig. 3 The optimization process quantified by the values

of the performance index.

Fig. 3 illustrates the different optimization processbetween IG_gSOFPNN and the proposed gdSOFPNNby visualizing the values of the performance indexobtained in successive generations of GA when usingType T*.

Table 5 Comparative analysis of the performance of thenetwork; considered are models reported in the

literature.Model PIS EPIs

Regression model 17.68 19.23

GA Simplified 7.045 11.264FNNmodel[GA Linear 4.038 6.028

FNN model[10] SHybrid miplified 6.205 8.868Hyrd Linear 3.830 5.397

Multi-FNNs[II] Linear 0.720 2.025T 3rd layer 0.008 0.082

gHFPNN[12] Max=2 l5th ayer 0.008 0.081

Proposed G ~~~'dlaer 0.002 0.0324

G 5th layer 0.016 0.116lG 1S3]N Max=-2 T 3 rd IayerI 0.002 O0U045

Proposed G1x 3rd 10, 0.002 0.024gdSOFPNN TaxT 0.002 0.008Ma 3 G I______0.001 10.023

6. CONCLUDING REMARKS

In this study, we introduced and investigated a newarchitecture and comprehensive design methodology ofgenetically dynamic optimized Self-Organizing FuzzyPolynomial Neural Networks with InformationGranulation based FPNs (gdSOFPNN), and discussedtheir topologies. gdSOFPNN is constructed with the aidof the algorithmic framework of information granulation.The design methodology comes as a structural andparametrical optimization being viewed as twofundamental phases of the design process. In the designof gdSOFPNN, the characteristics inherent to entireexperimental data being used in the construction of theIG_gSOFPNN architecture is reflected to fuzzy rulesavailable within a FPN. Therefore Informationgranulation based on HCM(Hard C-Means) clustering

3798

gdSOFP-<NN- IG_gSOFPNN -

0-035

U- .II.

U.U1 I

method was adopted. With the aid of the information Intelligence," Neurocomputing, Vol. 64, pp.granulation, we determine the initial location (apexes) 397-431, 2005.of membership functions and initial values of [13] H. S. Park, D. H. Park and S. K. Oh, "Geneticallypolynomial function being used in the premised and optimized Self-Organizing Fuzzy Polynomialconsequence part of the fuzzy rules respectively. Neural Networks Based on Information

Granulation," International Symposium on NeuralNetworks, pp. 410-415, 2005.

Acknowledgement. This work was supported by theKorea Research Foundation Grant funded by the KoreanGovernment(MOEHRD)(KRF-2005-041 -D00713)

REFERENCES

[1] Nie. J. H and Lee. T. H, "Rule-based modeling:Fast construction and optimal manipulation," IEEETrans. Syst. Man, Cybern, Vol. 26, pp. 728-738,1996.

[2] A. G. Ivakhnenko, "Polynomial theory of complexsystems," IEEE Trans. on Systems, Man andCybernetics, Vol. SMC-1, pp. 364-378, 1971.

[3] S. K. Oh and W. Pedrycz, "The design ofself-organizing Polynomial Neural Networks,"Information Science, Vol. 141, pp. 237-258, 2002.

[4] S. K. Oh, W. Pedrycz and B. J. Park, "PolynomialNeural Networks Architecture: Analysis andDesign," Computers and Electrical Engineering,Vol. 29, Issue 6, pp. 703-725, 2003.

[5] S. K. Oh and W. Pedrycz, "Fuzzy PolynomialNeuron-Based Self-Organizing Neural Networks,"International Journal of General Systems, Vol. 32,pp. 237-250, 2003.

[6] Zadeh. L. A, "Toward a theory of fuzzyinformation granulation and its centrality in humanreasoning and fuzzy logic," Fuzzy Sets and Systems,Vol. 90, pp. 111-117, 1997.

[7] J. C. BEZDEK, Pattern Recognition with FuzzyObjective Function Algorithms, New York, Plenum,1981.

[8] D. Jong, K. A, Are Genetic Algorithms FunctionOptimizers?. Parallel Problem Solving from Nature2, Manner, R. and Manderick, B. eds.,North-Holland, Amsterdam, 1972.

[9] G. Vachtsevanos, V. Ramani, and T. W. Hwang.:Prediction of gas turbine NOx emissions usingpolynomial neural network. Technical Report,Georgia Institute of Technology. Atlanta, 1995.

[10] S. K. Oh, W. Pedrycz and H. S. Park, "HybridIdentification in Fuzzy-Neural Networks", FuzzySets and Systems, Vol. 138, Issue 2, pp. 399-426,2003.

[11] S. K. Oh, W. Pedrycz and H. S. Park, "Multi-FNNidentification based on HCM clustering andevolutionary fuzzy granulation," SimulationModelling Practice and Theory, Vol. 11, pp.627-642, 2003.

[12] S. K. Oh, W. Pedrycz and H. S. Park, "Multi-layerHybrid Fuzzy Polynomial Neural Networks: ADesign in the Framework of Compuational

3799