47-nn3 jimenez methodology

8/6/2019 47-NN3 Jimenez Methodology

1/3

AbstractThis paper present the model used for forecast the11 time series from the NN3 Competition 2007. This model is a

new type-2 fuzzy inference system for treatment of

uncertainties with learning automatic, denominated Type-2

Hierarchical Neuro-Fuzzy Model (T2-HNF). This new model

combines the paradigms of modelling of the type-2 FIS and

neural networks (NN) with techniques of recursive BSP

partitioning. This model, besides to have the capacity to create

and to expand its structure automatically, to reduce the

limitation referred to the number of inputs and to extract rules

of knowledge from a dataset, is also able to model and to

manipulate existing uncertainties in real systems, diminishing

the effects of these and presenting, consequently, a better

performance. Also, this model provides an interval of

confidence for its output, that constitutes an important

information for real applications. In this context, this modelsurpasses the limitations of the type-2 fuzzy inference system

(type-2 FIS) and the type-1 fuzzy inference systems (type-1

FIS).

The developed model was evaluated in diverse databases

benchmark and real applications of forecast and approximation

of functions. The results obtained were compared with others

models, demonstrating that T2-HNF model offers next results

and in several cases superior to the best results provided by the

models used for comparison. In terms of computational time, its

performance also is very good. Also, the obtained intervals of

confidence for the defuzzified outputs always show to be

coherent and offer greater credibility in most of cases when

compared with intervals of confidence obtained by traditional

methods.

Index TermsType-2 fuzzy logic systems, interval type-2

fuzzy sets, lower and upper membership functions, hierarchical

neuro-fuzzy models, uncertainties.

I. INTRODUCTIONHE concept of information is intimately related to the

one of uncertainties. Uncertainties are the result of

deficient and not reliable information, of the randomness

in the data and the process that generate them (random

processes), or of the modelling of variant systems in the time

of not known form. Uncertainty is a inherent part to fuzzyinference systems (FIS) used in real applications. The

following sources of uncertainties can be present in FIS [1]:

- uncertainties in relation to the meaning of the words

used in the antecedents and consequents of the linguistic

Manuscript received May 14, 2007. This work was supported in part by

CAPES (), Brazil.

R. Jimenz C., M. M. B. R. Vellasco, and R. Tanscheit are with the

Electrical Engineering Department, Pontifical Catholic University of Rio de

Janeiro (PUC-Rio), Rio de Janeiro 22453-900, Brazil (e-mail:

[email protected]; [email protected], [email protected]).

rules (fuzziness);

- uncertainties in relation to the consequent of one rule

(strife);

- uncertainties in relation to the noisy data that activate

the FIS and that are used to tune the parameters of this FIS

(not specificity);

- uncertainties in relation to the specification of the

membership functions of each variable (example: center,

end-points).

The type-1 FIS use type-1 fuzzy sets characterized by two

dimensions and precise membership functions, where the

membership grade is a number in the interval [0,1]. This

way, they have limited capacity to model uncertainties totally

and directly [1],[2],[3].The type-2 FIS use type-2 fuzzy sets characterized by

membership functions fuzzy, where the membership grade

for each element of this set is a fuzzy set in [0.1]. This way,

the type-2 membership functions have three dimensions; this

new third dimension provides an additional degree of

freedom. Besides, these membership functions include a

footprint of uncertainty doing possible the quantification and

the direct modelling of uncertainties.

The type-2 FIS provide a fundamental measure of

dispersion, similar to the variance, to capture more

information about its uncertainties in the design of a model.

This measure of dispersion constitutes the new direction for

FIS. Also, the output set of a type-2 FIS, called type-reducedset, is truely unique and provides an interval of confidence

for its output, contributing, this way, more information that

an precise output [1],[2],[3].

The type-2 FIS have demonstrated better performance in

specific applications in which uncertainty exists, or in

applications that have greater complexity, such as non-

linearity, non-stationarity or time-variability.

Unfortunately many of the elaborated type-2 FIS present

limitations with regard to the reduced number of inputs

allowed and to the limited (or nonexistent) form to create

their own structure and rules. On the other hand the type-1

FIS have limited capacity to model uncertainties completely,

and they do not provide intervals of confidence for theiroutputs.

In function of the limitations of the type-1 FIS and the

type-2 FIS existing, the main objective of this work is to

create and to develop a hybrid model type-2 neuro-fuzzy that

besides to have the capacity to create and to expand his

structure automatically, to reduce the limitation referred to

the number of inputs and to extract rules of knowledge from

a dataset, is also able to model and to manipulate existing

uncertainties in real systems, diminishing the effects of these

and presenting, consequently, a better performance. In

Hierarchical Type-2 Neuro-Fuzzy Model

Roxana Jimnez Contreras, Marley Maria Bernardes Rebuzzi Vellasco, Ricardo Tanscheit.

T


2/3

addition, this model must provide an interval of confidence

for its output, that constitutes a information more rich than

the contained in precise output. Of this form the new T2-

HNF model is defined.

The remainder of this paper this organized of the

following form. Section 2 introduces T2-HNF model,

describing his basic cell, its architecture and his method of

learning. Finally, section 3 presents the conclusions.

II. TYPE-2HIERARCHICAL NEURO-FUZZY MODELThis model is made up of basic cells, calls type-2 neuro-

fuzzy cell. These cells are arranged in a hierarchical structure

in the form of a binary tree, in which the cell in the highest

level of the hierarchy generates the systems output, while

the cells in the lowest level of the hierarchy behave as

consequents of the higher hierarchy cells [4].

In this new hybrid model, the type-2 fuzzy rules are

generated by an automatic process of partitioning of the

input space, using an extension of BSP partitioning for type-

2 fuzzy regions. The hierarchical aspect is related to the fact

that each partition of the input space defines a type-2

subsystem, that, as well, may have a type-2 subsystem with

the same structure as its consequent (recursiveness).

A. Basic Type-2 Neuro-Fuzzy CellAn type-2 neuro-fuzzy cell [4]is a type-2 neuro-fuzzy mini

system that performs binary fuzzy partitioning of to certain

space, through its input variable x, according to the type-2

sigmoid membership functions, ~ (low) and ~ (high),described in Fig. 1. In this figure, )()(~ xx = ,

)()(~ xx =, )()(~ xx = , )()(~ xx =

are the upper

and lower membership grades of the interval type-2 fuzzy

sets low ~

and high ~

, respectively.

The primary membership functions of the sigmoid type-2

membership functions - ~ e ~ - are given by (1) e (2):

)(1

1],,[)(

bxae

xbasigx

+

== , ],[ 21 bbb (1)

)(1)( xx = (2)

In T2-NF cell, the type-2 membership functions ~ and

~ are implemented so that:

1)()( =+ xx ; 1)()( =+ xx . (3)

Equation (1) describes the sigmoid primary membership

function of ~ and ~ with inclination a, of fixed value,and middle point of the transition with uncertainty that

assumes values in [b1, b2]. In T2-NF cell, )(x and )(x

have like middle point of the transition b1'; on the other

hand )(x and )(x have like middle point of transition

b2', in agreement shown in the Fig 1.

The linguistic interpretation of the mapping implemented

by the T2-NF cell is given by the following set of rules:

rule 1: IFx~ then 1=y ; ],[ 11 211 cc= .

(partition 1)

or

rule 2: IFx~ then 2=y ; ],[ 22 212 cc= .

(partition 2)

Each rule corresponds to one of the two partitions

generated by BSP partitioning. Each partition can in turn besubdivided into two parts by means of another T2-NF cell.

This way, T2-NF cell represents rules whose antecedents

are defined by both sigmoid interval type-2 fuzzy sets

associated to the input variable x. The value of the input

variable is inferred in the sigmoid interval type-2 fuzzy sets

of the antecedents. If the antecedent is true, the rule is shot.

The consequents, symbolized by i , are interval type-1

fuzzy sets and to be represented by their left and right end-

pointsi

c1 , ic2 , respectively, where i indicates i-th rule of

the system.

Therefore, the type-2 rules in this model let us

simultaneously account for uncertainty about antecedentmembership functions and consequent parameter values.

Each consequent ],[ 21 ii cci =corresponds to one of the

three possible consequents: a interval type-1 fuzzy set; a

linear combination of inputs with interval type-1 fuzzy sets;

The output of a cell of a previous level.

T2-NF cell generates an output that is a interval type-1

fuzzy sets, represented by its left and right end-points

ly , ry , respectively.

B. T2-HNF ArchitectureT2-HNF model can be created based on the

interconnection of several T2-NF cells. These cells arearranged in a hierarchical structure in the form of a tree; only

in the root cell is made a defuzzification.

Fig. 2 illustrates an small T2-HNF architecture along with

the respective partitioning of the input space. In this

architecture, the initial partitions 1 and 2 (BSP-T2 0 cell)

have been subdivided; hence, the consequents of its type-2

rules are the outputs (interval type-1 fuzzy sets) of subsystem

1 and subsystem 2, i.e., ],[ 111 rl yyy = and ],[ 222 rl yyy = ,

respectively. In turn, subsystem 1 has as consequents

Fig. 1. Profiles of the type-2 membership functions of the T2-NF

cell.


3/3

],[1111 2111

cc= and ],[ 121212 rl yyy = , while subsystem 2 has

],[2121 2121

cc= and ],[2222 2122

cc= as its consequent. The

interval type-1 fuzzy set, consequent ],[121212 rl yyy = , is the

output of the BSP-T2 12 cell.

Each ],[ 21 ii cci =corresponds to a interval type-1 fuzzy

sets, in the case of using consequents of Sugeno of order 0,

or corresponds to one linear combination of the inputs with

interval type-1 fuzzy sets, in the case of using consequents of

Sugeno of order 1.

C. Learning AlgorithmIn the neuro-fuzzy literature the learning process is

generally divided in two parts: the identification of the

structure and the adjustments of parameters. The T2-HNF

model follows the same process. However, only one

algorithm carries out both learning tasks simultaneously. The

T2-HNF model has a training algorithm based on the

gradient descent method [4] for learning the structure of the

model (the linguistic type-2 rules) and its fuzzy weights. The

parameters that define the profiles of the type-2 membership

functions of the antecedents and the left and right end-points

of the consequents are regarded as the fuzzy weights of theT2-HNF model. Thus, the ],[ 21 ii cci=

and the parameters a,

b1 and b2 are the fuzzy weights of the model. In order to

prevent the models structure from growing indefinitely, a

parameter, named decomposition rate ; was used. It is adimensionless parameter and acts as a limiting factor for the

decomposition process.

III. CONCLUSIONSThe study of cases, made with different "benchmark"

databases and real applications in different areas, confirm the

good applicability of this model in the task of forecast and

approximation of functions.

The T2-HNF model present better performance in the task

of forecast and approximation of functions in cases where it

exists greater complexity and greater uncertainties.

The performance of this model in relation to the

computational time of processing also was very good,

presenting a smaller computational cost in comparison with

the other models like MLP NN.

The T2-HNF model has the advantage to model

uncertainties in form totally new, being able to give an

interval of confidence - important in real applications - for

their outputs, contrary to others models that have limitations

in the modelling of present uncertainties and the not to give

an interval of confidence for their outputs; since these

models only provide precise outputs.

The intervals of confidence obtained of automatic form by

the T2-HNF model are coherent when they are compared

with intervals of confidence obtained by traditional methods.

Also is observed that these intervals offer greater credibility

in majority of cases, once they are more narrow than the

traditional intervals of confidence.

REFERENCES

[1] J. M. Mendel, Uncertain Rule-Based Fuzzy Logic Systems:Introduction and new directions, Ed. Prentice Hall, USA, 2000.

[2] J. M. Mendel and R. I. Bob John, Type-2 Fuzzy Sets Made Simple,IEEE Transactions on Fuzzy Systems, Vol. 10, No. 2, April 2002.

[3] J. M. Mendel, Type-2 Fuzzy Sets: Some Questions and Answers,IEEE Neural Networks Society, pp, 10-13, August 2003.

[4] R. Jimnez. C. Modelos Neuro-Fuzzy Hierarquicos do Tipo 2. Tese deDoutorado. DEE-Puc-Rio, 2007.

(a)

(b)

Fig. 2. (a) Example of T2-HNF architecture. (b) BSP Partitioning of

the T2-HNF in fig. 2(a).

47-nn3 jimenez methodology

Documents