01001110 01100101 01110101 01110010 01101111 01101110 01101111 01110110 01100001 00100000 01110011...

01001110 0110010101110101 0111001001101111 0110111001101111 0111011001100001 0010000001110011 0110101101110101 0111000001101001 0110111001100001 0010000001101011 0110000101110100 0110010101100100 0111001001111001 0010000001110000 0110111101100011 0110100101110100 0110000101100011 0111010100101100 0010000001000110 0100010101001100 0010000001000011 0101011001010101 0101010000101100 0010000001010000 0111001001100001 0110100001100001 00000000

Automatic Method for Data Preprocessing for the GAME Inductive Modelling Method

Miroslav Čepekcepekm1@fel.cvut.cz

Miloslav Pavlicek, Pavel Kordik

Miroslav Šnorek

Computational Intelligence GroupDepartment of Computer Science and Engineering

Faculty of Electrical EngineeringCzech Technical University in Prague

ICIM 2008

International Conference on Inductive Modelling, Kyiv 2008

Miroslav Cepek, cepekm1@fel.cvut.cz

Automatic preprocessing The GAME Neural Network (as all others data

mining methods) heavily depends on data preprocessing.

Preprocessing involves selection, setup and ordering of preprocessing methods.

We want to automate this stage. We will use genetic algorithm to find optimal

sequence of methods.

International Conference on Inductive Modelling, Kyiv 2008

Miroslav Cepek, cepekm1@fel.cvut.cz

GAME Neural Network Group of Adaptive Method Evolution (GAME)

uses inductive modelling. The structure of the model is created in

inductive way (data driven modelling).

International Conference on Inductive Modelling, Kyiv 2008

Miroslav Cepek, cepekm1@fel.cvut.cz

Main Ideas of Automatic Preprocessing

The main idea is to use genetic algorithms to find optimal order and optimal setup of data preprocessing methods.

In the first stage we will to use simple genetic algorithm.

Because we want to find sequence which will the most fits the GAME ANN we will use reduced GAME ANN for fitness function evaluation.

International Conference on Inductive Modelling, Kyiv 2008

Miroslav Cepek, cepekm1@fel.cvut.cz

Single individual in automatic preprocessing

The individuals in our automatic consists of list of preprocessing methods. Each method can be applied to different attributes. Each method have different setup. Methods are applied one by one. Some methods changes structure of the dataset

(PCA) and must be treated separately.

International Conference on Inductive Modelling, Kyiv 2008

Miroslav Cepek, cepekm1@fel.cvut.cz

GA for Automatic Preprocessing Genetic algorithm goes in standard way as

shown below.

International Conference on Inductive Modelling, Kyiv 2008

Miroslav Cepek, cepekm1@fel.cvut.cz

GA Properties Selection – tournament selection

Several individuals are selected at random from population and individual with the highest fitness is selected.

Cross over – standard one-point cross over. Mutation

adds or removes preprocessing methods from individual.

changes order of methods. changes configuration of methods.

International Conference on Inductive Modelling, Kyiv 2008

Miroslav Cepek, cepekm1@fel.cvut.cz

Fitness Recalculation Fitness is average accuracy of several simple

GAME models generated from data preprocessed by given individual. Accuracy of models is not always the same due to

genetic algorithm involved in training. Using several models allows more consistent

results. We assume that better simple model also

means better complex models.

International Conference on Inductive Modelling, Kyiv 2008

Miroslav Cepek, cepekm1@fel.cvut.cz

Outline of the Experiment Complete dataset is split into training and

testing part. From training data given portion of values is

removed. Several GAME models are created on raw data. Instances with missing values are removed. Then

several GAME models are created. Automatic preprocessing is performed. The best

individual is selected and preprocessing methods are applied and several GAME models are created.

International Conference on Inductive Modelling, Kyiv 2008

Miroslav Cepek, cepekm1@fel.cvut.cz

Artificial data

International Conference on Inductive Modelling, Kyiv 2008

Miroslav Cepek, cepekm1@fel.cvut.cz

Best Chromosomes

The best individuals for selected amount of missing values. Part a) shows the best chromosome 1% of missing values. Part b) shows individual for 5% of missing values and c) shows 20% of missing values.

International Conference on Inductive Modelling, Kyiv 2008

Miroslav Cepek, cepekm1@fel.cvut.cz

Best Chromosomes Chromosomes for simple problems (low

number of missing values) are quite simple. Chromosomes for complicated problems (high

number of missing values) are quite complicated.

In this sense our algorithm works.

International Conference on Inductive Modelling, Kyiv 2008

Miroslav Cepek, cepekm1@fel.cvut.cz

Manually vs Automatically selected methods.

International Conference on Inductive Modelling, Kyiv 2008

Miroslav Cepek, cepekm1@fel.cvut.cz

Results Graph shows that GAME is unable to handle

missing values. Results of RAW data are quite poor.

When instances with missing data are removed, accuracy increase rapidly.

When automatic preprocessing is used accuracy is even better.

International Conference on Inductive Modelling, Kyiv 2008

Miroslav Cepek, cepekm1@fel.cvut.cz

Conclusion We proposed algorithm for automatic selection

and ordering of data preprocessing methods. We performed the first experiment with our

method. It works for artificial data and in future we have

to prove that it work also for more complicated and real-world data.

International Conference on Inductive Modelling, Kyiv 2008

Miroslav Cepek, cepekm1@fel.cvut.cz

Thank You for Your attention.

cepekm1@fel.cvut.cz