Method of rules extraction for expert systems based on artificial neural networks

Shvarts Alexander

Saratov State Technical University

Intellectual systems

Medicine

Technical diagnostics

Oil and gas industry

Transport

and many others…

Basis of intellectual systems

RulesDecision treesRegression analysisArtificial neural networks

Multilayer perceptrons Radial-basis functions networks Kohonen self-organizing networks Recurrent networksetc.

Existing problemsDisadvantages of expert systems based on artificial neural networks:

Difficulties in explaining the decision making process Problems in validation “Missing exceptions” mistakes

X1

X2

X3

X4

X5

X6

X7

y1

y2

y3

Purpose of study:

to introduce and develop new method, that could transform network structure into classification rules in order to

Easily validate expert system Develop explaining module Handle “missing exceptions” mistakes

Stages of research

1. Analysis of existing methods of rules extraction

2. Introducing new method with following characteristics:

a) Structure – multilayer perceptron

b) No network pruning and re-training

3. Testing the method on trained network

Introduced method

Att

ribu

te 0 X00

X01

X02

X10

X11

X12

X20

X21

X22

H0

H1

H2

H3

C0

C1

C2

Wij,n

Wn,m

Att

ribu

te 1

Att

ribu

te 2

Feedforward multilayer perceptron

• One hidden layer

• Hyperbolic tangent as the activation function of hidden layer

• Sigmoid as the activation function of output layer

• Input neurons are grouped into attributes

Index of importance

SIGNMINWIMAXWI

WIAVGWIII im

mi

,

C0

H

0hhmihim WWWI

Weight index

imWIMAXWI max

imWIMINWI min

Xi

H0

H1

H2

H3

Cm

XI

X1

Attributes calculations

Cm

Xi

H0

H1

H2

H3

XI

X1

Xi-1

Xi+1

Attribute a

aXIIMINII imiam ),min( ,

aXIIMAXII imiam ),max( ,

aXIIAVGII imiam ),avg( ,

Importance threshold

avgavgmax SSSKL

where

A

0aamMAXIISmax

A

0aamavg AVGIIS

1K0

and

- reliance coefficient

Attribute Importance indices

0

1

… …

A

RANGED3980II7140II870II 121110 .,.,. ,,,

120II670II910II 161514 .,.,. ,,,

3090II8890II 1I11I .,. ,),(

Attribute Importance indices

1

A

0

…

7

3980II7140II870II 121110 .,.,. ,,,

120II670II910II 161514 .,.,. ,,,

3090II8890II 1I11I .,. ,),(

IIS

10II1980II240II 115114113 .,.,. ,,,

Combinations graph14II , 15II , 16II ,

115II ,

11III ),( 1III ,

114II ,113II ,

Attribute 1

Attribute 0

Attribute 7

aMAXIImax

aMAXIImin

Rules generation

121I14 IIIIII ,,, ,...,

111I15 IIIIII ,,, ,...,

1211I16 IIIIII ,),(, ,...,

IF [Attribute 1]=[Value 4] AND [Attribute A]=[Value I] AND … AND [Attribute 0]=[Value 2] THEN [Class m]

IF [Attribute 1]=[Value 5] AND [Attribute A]=[Value I] AND … AND [Attribute 0]=[Value 1] THEN [Class m]

IF [Attribute 1]=[Value 6] AND [Attribute A]=[Value (I-1)] AND … AND [Attribute 0]=[Value 2] THEN [Class m]

Method application

Expert system for predicting arrhythmia, based on multilayer perceptron: 15 inputs 1 neuron in hidden layer 2 classes 92% fidelity

Experimental data

Thank you!