Organizational Texts Classification Using Artificial Immune Recognition Systems

Nafiseh Forouzideh Kish International Campus,

University of Tehran, Kish, Iran

n.forouzideh@gmail.com

Maryam Tayefeh Mahmoudi 1 Knowledge Management & Elect. Organizations Group, IT Research

Faculty, Iran Telecom Research Center, (Education and Research

Institute for ICT) 2 School of Electrical and Computer

Eng., University of Tehran, Iran mahmodi@itrc.ac.ir

Kambiz Badie

Knowledge Management & Elect. Organizations Group, IT Research

Faculty, Iran Telecom Research Center, (Education and Research

Institute for ICT) K_badie@itrc.ac.ir

Abstract—This paper outlines the use of Artificial Immune Recognition System (AIRS) within the field of text/document classification. Various versions of AIRS including AIRS1, AIRS2, Parallel AIRS and Modified AIRS with Fuzzy KNN are applied to classify the mode of a text’s content which is organized for helping users with their organizational tasks. In this regard, 7 major features as inputs with 3 nominal values of Low, Medium, and High are chosen to classify texts into 6 organizational functionality classes. Results of experimentation on a dataset including 540 data show the fact that different versions of AIRS, performs better compared to multi-layer perceptron and radial basis function as simple neural approaches. Due to the high performance of this approach, it is expected to be successfully applicable to a wide range of content mode classification issues in decision support environment.

Keywords- text/ document classification; immune-inspired algorithm; artificial immune recognition system (AIRS); neural network; organizational task; content mode.

I. INTRODUCTION In recent years, text/ document categorization has

become one of the prime issues in text mining [1,2,3,4]. One major aspect in text classification is to identify the type of a text’s content, e.g. its mode/style, its peculiarities/ characteristics, the category it belongs to, as well as the peculiarities of the environment within which it has been prepared. Pattern recognition and evolutionary computing techniques have a wide range of applications in this issue [1,2,5,6]. Due to the distributed characteristics of a text, and the complexity of mode/style which may exhibit itself in an aggregation of a variety of considerations in its different parts/ components, evolutionary methods are expected to be particularly workable in this regard.

Based on the above point, we present in this paper an approach for classifying the mode of a text’s content using evolutionary techniques. Due to the significance of

comprehensive texts in making efficient decisions in organizations, organizational task of texts can be appropriate criteria to classification in our approach.

In this paper, an immune-inspired supervised learning algorithm called artificial immune recognition system (AIRS) has been considered for text classification [7]. AIRS performs good on very different problems such as large dimensioned feature space problems, problems with many classes and etc. [8]. Moreover, its own characteristics such as self-regulation, performance, generalization, parameter stability etc. makes it appropriate among the evolutionary algorithms for text classification purposes [9]. Besides, as each text can belong to various classes with different membership degrees, a modified AIRS for fuzzy classification is proposed. This algorithm can help in finding the appropriate features for text classification where each text may belong to various organizational functionality classes [10].

II. EXISTING APPROACHES TO TEXT CLASSIFICATION SYSTEMS

Text/document classification indicates that unclassified texts are assorted to predefined categories according to a specified principle. Within this context, classification can be performed based on the type of content, subject/issue, qualification level and style of document/text and even its authors’ specifications. Text classification can also ease the organization of increasing textual information, in particular Web pages and other electronic form of documents [11]. It usually consists of two parts: feature selector and text classifier.

Feature selector selects the features which are essential to classifying the text’s content; in terms of a feature vector. The classifier then assigns the feature vector to the appropriate class (es) based on the feature vectors. Many techniques can be used in feature selection to improve accuracy as well as to reduce the dimensions of the feature vector and thus reduce the time for computation. Feature

978-1-4244-9897-0/11/$26.00 ©2011 IEEE

selection mostly adopts various assessment functions such as document frequency, information gain, mutual information, and statistics (CHI) to calculate the weights [12]. There also exist some bio-inspired feature selectors that facilitate incremental feature selection on data streams like immune-inspired one [13]. Many classifiers have also been applied to classify texts [14,15], including k-nearest neighbor (KNN), Naïve Bayes, decision trees, neural networks, linear discriminate analysis (LDA), logistic regression, , support vector machines, rule learning algorithms, relevance feedbacks, vector space model, etc. Several kinds of competitive networks are used in text classification, including learning vector quantization (LVQ) and self-organizing maps (SOM) network. Besides, back propagation (BP) and radial basis function (RBF) networks are two successful examples for classification [2].

As with many other problems involving classification or decision making in a complex, distributed system, solutions have been inspired by a variety of biologically metaphors, such as imperialistic competitive algorithm [16], particle swarm optimization [5], genetic programming [6], ant colony [17], genetic algorithm [18], memetic algorithm [19], artificial immune systems [20,21,22], etc.

III. THE PROPOSED APPROACH Due to the distributed characteristics of a text and the fact

that its mode/style may exhibit itself in an aggregation of several considerations in its different parts/components, evolutionary computing algorithms specifically immune-inspired algorithms are expected to function more efficiently.

Based on the above point, in this paper, we present an approach for classifying the organizational texts using different versions of artificial immune recognition systems including AIRS1, AIRS2 and parallel AIRS. AIRS performs good on either large dimensioned feature space problems or problems with many classes or both. For this purpose, a dataset has been prepared on the basis of the existing technical reports at software research institute of our university at Kish campus. It has to be noted that, some of the major tasks important for an organization are: Planning/ Scheduling, Research, Innovation, Development/ optimization/ Improvement, Education/ Promotion, Analysis/ Assessment/ Assurance, Guidance, and Justification. This ontology can just as well be used by researchers, innovators, developers, planners, analyzers, and others to organize and disseminate results of their works in terms of appropriate texts/documents. In this paper, documents will be categorized into six of these output classes: Research, Development/ Planning, General Learning, Justification, Innovation and Analysis/ Assessment, and other.

A. Basics In this paper, the focus is on classification of text/

document using artificial immune recognition system approach. The recognition and learning capabilities of the natural immune system have been an inspiration for researchers developing algorithms for a wide range of application. The power of AIRS comes from the manner in which the algorithm derives candidate memory cells for the

pool that eventually make up the resulting classifier and the memory cell selection decision making process within AIRS is investigated empirically [9].

It is to be noted that AIRS are not solely designed by expert knowledge but are at least partly learned from experiential data. If no a-priori knowledge is available, fuzzy models do not make any sense and neural classification methods outperform. However, if accuracy is not the only ultimate goal and instead an understanding of the functioning of the process is desired, then fuzzy and neuro-fuzzy models are appropriate [23,24]. Whenever the accuracy and speed both are significant, artificial immune recognition system may appeal as a potential candidate for text classification.

In this respect, features of each text’s functionality can be identified and valued. These functionalities are considered to be the 6 classes of organizational tasks discussed above. In this regard, 7 major features have been chosen as inputs for our neural and AIRS approaches based on the expert’s idea and existing approaches [24,25]. Those major features are: “General Background”, “Existing viewpoints”, “Key issue”, “Proposed approach realization/ implementation”, “Validation/Verification”, “Comparative analysis & capability interpretation”, “Conclusion & prospect anticipation”. The values of each of the features, have been determined by experts as nominal values (L: Low, H: High, M: Medium). Table I illustrates these features with their prospected values. It is obvious that, these features have been realized to be consistent for a wide range of contents which are to be created for helping users with their tasks in organizations.

Obviously, based on the type of a task, a limited number of the labels may be activated. Nominal values “L”, “M”, and “H” associated with the labels of key segments indicate the extent according to which linguistically significant notions such as “What”, “Who”, “Whom”, “Where”, “Which”, “When”, “How”, and “Why”, can be addressed to create a petit content for each key segment in the content [25].

Taking this point into account, the feature vector of input content is structured based on the afore-mentioned features and the nominal values (Table I).

The dataset used in this research would include the data from text/ content’s labels that belong to 6 output classes on the basis of the existing technical reports. It contains 540 samples with 7 attributes. After normalizing the input data and making the test and train data, classification would start.

In this paper, for classification purpose, multi-layer perceptron (MLP) and radial basis function (RBF) are compared with various versions of artificial immune recognition system including AIRS1, AIRS2 and Parallel AIRS. As each text can belong to various classes with different membership degrees, a corresponding modified AIRS for fuzzy classification is also proposed respectively.

TABLE I. INPUT FEATURES AND OUTPUT CLASSES OF PROPOSED SYSTEM

B. Artificial Immune Recognition System(AIRS)Algorithms Artificial Immune Recognition System (AIRS) is a

supervised learning algorithm, inspired by the human immune system. Until 2001, most of the studies in artificial immune system were focused on unsupervised learning algorithms. Therefore, Watkins decided to show that artificial immune systems could be used for classification problems [7]. The only exception was Carter’s study in 2000 [26] who introduced a complex classification system based on AIS.

The AIRS classification algorithm uses the resource-limited approach [27] and the clonal selection principles [28]. After the first AIRS algorithm, a modified AIRS (AIRS2) was proposed that had a lower complexity but a slight reduction of accuracy [29,30]. Respectively, a parallel AIRS was developed based on AIRS2 [31]. AIRS algorithm has powerful features which are listed as follows [9]:

• Generalization: The algorithm does not need all the dataset for generalization and it has data reduction capability.

• Parameter stability: Even though user-defined parameters are not optimized for the problem, the decline of its performance is very small.

• Performance: There has been demonstrated that its performance is excellent for some datasets and totally remarkable.

• Self-regulatory: There is no need to choose a topology before training.

AIRS includes two major parts: data reduction through an optimization process and classification. In the first part, some representatives called memory cells, for each class based on training data are evolved, which leads to a considerable reduction in number of training data. These evolved representatives act as new training data for the next part which is classification using KNN method. The resultant memory cells can better represent the problem space rather than main training data which improve classification accuracy. It can be evaluated by test data classification using new training data.

The immune metaphors used in AIRS are: antibody-antigen binding, affinity maturation, clonal selection process, resource competition and memory acquisition. Timmis et al. introduced the concept of an artificial recognition ball (ARB) [27], which has the same representation as a B cell, but may stand for any number of identical B cells. Each ARB represents a certain number of the B cells, or resources, and the total number of resources of the system is bounded. Watkins modeled AIRS mainly on the mechanisms followed by the B cells of the biological immune system. All the B cells having similar features are represented together as ARBs. Within AIRS, each element (ARB) corresponds to a vector of n dimensions and a class to which the data belong. Additionally, each ARB has an associated stimulation level as defined in (1), where x is the feature vector of the ARB, S* is the stimulation of an ARB, x, y is the training antigen, and affinity(x,y) is the Euclidean distance between the two feature vectors:

S* = 1 ( , ),

( , ),

affinity x y if classof x classof y

affinity x y otherwise

− ≡ (1)

Affinity threshold (AT) is used to control the quality of

the memory cells maintained as classifier cells in the system. AT is the average Euclidean distance between each item in the training data set (as described in (2)).

1 1

1 1( , )

( 1)2

n n

i j i jaffinity ag ag

Affinitythresholdn n

− −

= ==−

� � (2)

Where n is the number of training data items (antigens), agi and agj are the ith and jth training antigen.

In the AIRS, ARBs compete with each other for a fixed resource number. This allows ARBs, which have higher affinities to the training antigen to improve. The memory cells (M) formed after the whole training antigens have been presented, are used to classify test Antigens.

1) AIRS1 The AIRS1 algorithm is defined by following steps: 1. Initialization: All items in the data set are normalized such that the Euclidean distance between any items is in the range of [0,1]. After normalization, the affinity threshold (AT) is calculated as defined in (2). A set of cells called the memory pool (M) and the ARB pool (P) is created from randomly selected training data. 2. Antigenic Presentation: For each antigenic pattern do: a. Clonal expansion: For each element of M determines their

affinity to the antigenic pattern, which resides in the same class. Select the highest affinity memory cell (mc) and clone mc in the proportion to its antigenic affinity to add to the set of ARBs (P).

b. Affinity maturation: Mutate each ARB descendant of this highest affinity mc. Place each mutated ARB into (P).

c. Metadynamics of ARBs: Process each ARB through the resource allocation mechanism. This will result in some

Output Classes

Input Content Features

Res

earc

h

Dev

elop

men

t/ Pl

anni

ng

Gen

eral

L

earn

ing

Just

ifica

tion

Inno

vatio

n A

naly

sis/

Ass

essm

ent

General Background H M L L M L Existing viewpoints H M H L M L

Key issue H M M M M M Proposed approach

realization/ implementation H M L M L M

Validation/ Verification H M L M L H

Comparative analysis & capability interpretation H M L L L L

Conclusion & prospect anticipation H H L L L L

ARB death, and ultimately controls the population. Calculate the average stimulation for each ARB, and check for termination condition.

d. Clonal expansion and affinity maturation: Clone and mutate a randomly selected subset of the ARBs left in (P) based in proportion to their stimulation level.

e. Cycle: If the average stimulation value of each ARB class group is less than a given stimulation threshold repeat from step 2(c).

f. Metadynamics of memory cells: Select the highest affinity ARB of the same class as the antigen from the last antigenic interaction. If the affinity of this ARB with the antigenic pattern is better than that of the previously identified best memory cell mc then add the candidate (mc-candidate) to memory set M. Additionally, if the affinity of mc and mc-candidate are below the affinity threshold, then remove mc from M.

3. Cycle: Repeat step 2 until all antigenic patterns have been presented. 4. Classification: After training has completed, the evolved memory cells are available for use for classification. The classification is performed in a k-nearest neighbor (K-NN) approach. Each memory cell is presented with a data item for stimulation. The system’s classification of a data item is determined by using a majority vote of the outputs of the k most stimulated memory cells. After training, test data are presented only to memory cells. K-NN approach in a described manner is used to determine the classes in test phase. A more detailed description of the algorithm and results can be found in [7, 30]. 2) AIRS2

AIRS1 is the first version of the algorithm, which is extremely complex. That’s why, a second version was introduced. AIRS2 has a lower complexity, a higher data reduction percentage and a small decrease in accuracy [29,30]. The main steps of AIRS1 and AIRS2 are generally similar but there are some minor differences. AIRS2 is simpler than AIRS1 and has a better data reduction percentage. AIRS2 has a less performance degradation and a lower complexity than AIRS1.

AIRS1 uses the ARB pool as a permanent resource, whereas AIRS2 uses it as a temporary resource. AIRS2 approach is more accurate, because of the higher similarity between the antigens which have same class label and the ARBS. Previous ARBs which come from other ARB improvement transitions do not need to be evaluated continuously as in AIRS1. Classes of clones can be changed after mutation process in AIRS1, but this is not allowed in AIRS2. AIRS1 uses user-defined mutation parameter but AIRS2 uses the concept of “somatic hypermutation”, where the amount of clone mutation is proportional to its affinity with the antigenic pattern, the higher the affinity, the smaller the mutation rate, and vice-versa.

3) Parallel AIRS

The parallel version of AIRS2 behaves in the following manner [9,31]:

• Read in the training data at the root process. • Scatter the training data to the available processes. • Execute, on each process, steps 1 through 3 from the

serial version of AIRS2 on the portion of the training data obtained.

• Gather the developed memory cells from each process back to the root.

• Execute step 4 with members of evolved memory cell pool

If the dataset is not divided into very small subsets, this approach provides a faster solution while the performance is preserved.

Successful classification of texts/ documents using AIRS relies on appropriate feature selection and in the intervening similarity measure definition that reveals how close a text is to a memory cell using the predefined features.

4) Modified AIRS for fuzzy classification

As it has been mentioned before, it doesn’t seem logically to differentiate text/documents crisply in independent classes. In this regard, multi-label classification approaches are proposed [32]. In these approaches each text/document belongs with the same membership degree to different classes while it does not have compatibility with the nature of text/documents which may belong to different classes with different membership degrees. To reveal a text’s membership degree to each of the output classes and distinguish its relevance membership degree to each functionality class, a modified AIRS approach is proposed. For this purpose, classic KNN is replaced with fuzzy KNN in AIRS1. The proposed modified AIRS may not outperform the other mentioned AIRS algorithms but it is important for us from the view point of finding membership degree of text to each output functionality classes.

Keller et al. proposed fuzzy K-NN classification algorithm based on the K-NN algorithm and concepts of the fuzzy set theory [33]. Class memberships are assigned to the instance, as a function of the instance’s distance from its K-NN training instances. The advantages of fuzzy K-NN classifier are its capability of taking account the ambiguous nature of the neighbors and assigning a membership to each instance to be classified not in a binary form. However the crisp results form of AIRS with Fuzzy K-NN can be evaluated based on maximum membership degree to be considered for comparison.

The results of this method suit better the classification by human experts in the real world. Therefore the proposed method is more reliable than the other existing ones. Also as it inherently includes more information so it can be a good tool for selecting and modifying appropriate features. For instance, if numerous documents that belong to one class, in the meantime have a membership degree above a threshold to another class and vice versa, then it reveals that the input features may not be appropriate and have to be modified. Determination of a proper threshold for membership degree to each class has a significant role in having a useful classification.

C. Experimental Results To evaluate the capability of the proposed approaches,

their performance has been compared to that of neural nets for the same tasks. In this respect, multi-layer perceptron (MLP) and radial basis function (RBF) are implemented for simple neural classification and AIRS1 AIRS2 and parallel AIRS are implemented for immune inspired classification. A modified version of AIRS using fuzzy K-NN is also proposed for finding membership degree of texts to each output classes of organizational functionalities.

The dataset used in this research would include the data from text/ content’s labels that belong to 6 output classes. It contains 540 samples with 7 attributes. After normalizing the input data and making the test and train data, classification would start. 70% of data are considered for training and 30% for test. The presented results are the average of 20 iterations for each algorithm. 1) Classification using MLP

In this respect, a feed forward MLP has been used with 1 hidden layer and a variation of hidden neurons. The optimal number of neurons in this respect was found to be 20 based on testing neural network with various number of nodes and hidden layers for proposed classification purpose. Since we have 6 output classes, binary sub-networks of output classes are trained with normalized input data in order to increase convergence rate. The best resultant specifications of these binary networks are as follows:

Number of neurons: 20; Train parameter epochs: 100; DivideParam.trainRatio = 0.7; DivideParam.testRatio = 0.15; DivideParam.valRatio = 0.15; Train Param. max_ fail = 30;

After training each network separately, the total network output is computed and is transformed from binary into decimal to have checked its status of belonging to the existing classes. Reconstructing test data for outputs and comparing the real classes with the network outputs yields realization of the whole classification process.

The average classification error for 20 repetitions of this method for the validation phase (test data) is 12.34% while the accuracy of the correct classification on test data is: 90.74%.

2) Classification using RBF

Another neural method for text classification which has been used in this experiment is RBF whose basis function is Gaussian and is applied under the following conditions in its optimal form: goal = 0; spread = 1; MaxNeurons = 30; displayInterval = 2.

The average classification error for 20 repetitions of this method for the validation phase (test data) is 11.11% while the average accuracy of the correct classification on test data is: 88.89%.

3) Classification Using Artificial Immune Recognition Systems

Various versions of AIRS including AIRS1, AIRS2, Parallel AIRS and modified AIRS using fuzzy K-NN are applied in this paper on the same data set.

One advantage of AIRS is that it is not necessary to know the appropriate settings for the classifier. The most important element of the classifier is Self-regulatory/ self-adjusting [8]. So, the parameters shown in Table II and used in our experiments have little effect on system performance.

TABLE II. PARAMETERS OF AIRS CLASSIFICATION ALGORITHMS

Parameters of AIRS Values Number of seed cells 1 Hyper clonal rate 2 Clonal rate 10 Number of resources 250 Stimulation threshold 0.9 Mutation rate 0.2 Affinity threshold scalar (ATS) 0.1 k-Value for nearest neighbor 3

The average classification results for 20 repetitions of the mentioned algorithms show that among 162 test data as input, 5.56% were classified incorrectly by AIRS1, 6.79% by AIRS2, and 4.32% by Parallel AIRS.

Also fuzzy experimental results for using modified AIRS with fuzzy KNN shows that 8.02% were classified incorrectly based on its crisp outputs which were resultant from maximum membership degrees.

4) Comparison and Analyzing the Results

In order to evaluate and compare the proposed classification methods, accuracy is not enough a criterion. In this regard, some other criteria are considered for better analyzing including sensitivity or recall, specificity and precision. These criteria are measured for each class separately. It has to be noted that all of the mentioned parameters are measured and compared on test data set for all considered algorithms in this research. The presented results are the average of 20 iterations for each algorithm.

The classification accuracy of the various classification algorithms are measured using (3) for the test dataset and represented in Table III [7]. Accuracy is the overall correctness of the model and is calculated as the sum of correct classifications divided by the total number of classifications.

1 ( )( ) ,

Tii

i

assess taccuracy T t T

T== ∈� (3)

1, if classify(t) = t.c assess(t) = 0, otherwise where T is the set of data items to be classified (the test

set), t is a member of T, t.c is the class of item t, and classify (t) returns the classification of t by classification algorithms.

As it is seen in Table III, Parallel AIRS outperforms the other algorithms. AIRS1, AIRS2 and AIRS with fuzzy K-NN follow Parallel AIRS respectively. Although the accuracy of AIRS1and AIRS2 are so closed, but the speed of AIRS2 is higher than AIRS1. It is obvious that all the versions of AIRS outperform RBF and MLP.

It is to be noted that, sensitivity or recall, specificity and precision are measured for each class separately and represented for various classification algorithms in Tables IV-VI.

Precision is a measure of the accuracy provided that a specific class has been predicted. It is defined by (4):

Precision = TP/ (TP + FP) (4)

where TP, TN, FP and FN denote true positives, true negatives, false positives, and false negatives, respectively [34].

As it is illustrated in Table IV, various versions of AIRS have higher precision in comparison with RBF and MLP. Moreover, the differences between precisions among various classes are smaller using AIRS methods. This shows the ability of AIRS in learning and predicting output classes equally that is realized by Parallel AIRS as well.

Recall or sensitivity is a measure of the ability of a prediction model to select instances of a certain class from a data set. It is defined by (5):

Recall = Sensitivity = TP/ (TP+FN) (5)

Sensitivity of various algorithms of AIRS is higher to the

organizational functionality classes rather than other classification methods. Moreover, the sensitivity of each AIRS classifier, especially parallel AIRS, with respect to each functionality classes are so similar that show make them prior than other existing algorithms.

Specificity is a measure which is calculated based on the proportion of those instances that do not belong to a specific class and are classified in other classes. It is defined by (6):

Specificity = TN/ (TN + FP) (6)

TABLE III. CLASSIFICATION ACCURACY(%) FOR DIFFERENT CLASSIFICATION ALGORITHMS

Method RBF MLP AIRS1 AIRS2 Parallel AIRS

AIRS-Fuzzy K-NN

Classification Accuracy 88.89 90.74 94.44 93.21 95.68 0.9198

TABLE IV. CLASSIFICATION PRECISION(%) FOR DIFFERENT CLASSIFICATION ALGORITHMS PER EACH ORGANIZATIONAL

FUNCTIONALITY CLASS

method class RBF MLP AIRS

1 AIRS

2 Parallel AIRS

AIRS-Fuzzy K-NN

Research 96.30 96.55 100 96 100 100 Development/

Planning 100 100 100 100 100 100

General Learning 100 96.77 86.96 86.96 90.91 90.91

Justification 67.86 79.17 93.75 90.91 93.94 84.85 Innovation 86.21 100 93.10 92.86 93.33 93.10

Analysis/ Assessment 85.71 71.43 91.67 91.67 95.65 83.33

TABLE V. CLASSIFICATION SENSITIVITY(%) FOR DIFFERENT CLASSIFICATION ALGORITHMS PER EACH ORGANIZATIONAL

FUNCTIONALITY CLASS

method class RBF MLP AIRS

1 AIRS

2 Parallel AIRS

AIRS-Fuzzy K-NN

Research 100 100 100 100 100 100 Development/

Planning 100 100 100 96.67 100 100

General Learning 86.67 96.77 90.91 90.91 90.91 90.91

Justification 79.17 70.37 93.75 93.75 96.88 87.50 Innovation 100 96.43 90 86.67 93.33 90 Analysis/

Assessment 72.73 80 91.67 91.67 91.67 83.33

TABLE VI. CLASSIFICATION SPECIFICITY(%) FOR DIFFERENT CLASSIFICATION ALGORITHMS PER EACH ORGANIZATIONAL

FUNCTIONALITY CLASS

method class RBF MLP AIRS

1 AIRS

2 Parallel AIRS

AIRS-Fuzzy K-NN

Research 99.26 99.25 100 99.28 100 100 Development/

Planning 100 100 100 100 100 100

General Learning 100 99.24 97.86 97.86 98.57 98.57

Justification 93.48 96.30 98.46 97.69 98.46 96.15 Innovation 97.08 100 98.48 98.48 98.48 98.48 Analysis/

Assessment 96.90 94.16 98.55 98.55 99.28 97.10

As it is seen, AIRS classifiers usually can become more

specialized in each class over existing methods. Therefore the specificity of AIRS algorithms per functionality classes is better than that of the other algorithms.

As a conclusion, it can be mentioned that, better performance of AIRS in classification is related to its ability in learning the classes separately. It can easily be seen AIRS2 and parallel AIRS have stronger related potential because of their internal improvements. In almost all the cases parallel AIRS outperforms the other methods. AIRS with fuzzy KNN represents acceptable results with respect to MLP and RBF but weaker than AIRS1, AIRS2, parallel AIRS.

The comparison between the classification rates respectively belonging to Parallel AIRS, AIRS1, AIRS and parallel AIRS. As it is seen from the experimental results, AIRS has classified better on test data compared to MLP and RBF.

To show the experimental results of AIRS using fuzzy KNN, the membership relevance of each document to each class are evaluated. Due to the high number of results, all of them can not be presented in this paper (Table VII).

Analyzing the achieved results reveals that, the selected input features may not specifically distinguish between “justification” class and “Analysis” class. That may be because of the similarity between the concepts of mentioned classes. In this regard, this method may be appropriate for modifying the input features iteratively, which will have a significant role in classification.

TABLE VII. MEMBERSHIP VALUES OF 3 SAMPLE DOCUMENT S

As a conclusion, artificial immune recognition system

reveals better performance on the given dataset compared to the other mentioned algorithms.

This is at first glance because a text’s content has generally a multi-class or multi-modal nature, and thus due to its simultaneous affiliation to different classes (modes), classification approaches based on a sort of uncertainty handling logic can perform far better compared with those without such a basis. Moreover, as an immune-inspired classification algorithm, makes AIRS more successful given its very peculiar ability in speeding up the training procedure as well as incorporating with many kinds of prior knowledge (nominal values for the input features in our case), and also its insensitivity toward curse of high dimensionality and its self-adjusting characteristic to the features of its architecture.

IV. CONCLUDING REMARKS In this paper, the performance of multi-layer perceptron

and radial basis function as simple neural approach, and various versions of AIRS as immune-inspired supervised learning algorithm, was evaluated for classifying the mode of a text’s content, which is basically designed for helping users with their organizational tasks.

Experimental results on an initial dataset including the data belonging to 540 texts, demonstrate the fact that all versions of AIRS especially parallel AIRS performs far better compared to simple neural approaches. This, as was discussed, is mainly due to ability of this approach in classifying the patterns of texts, which are somewhat multi-class or multi-modal in nature.

Moreover, to find appropriate features for classification, the proposed AIRS with fuzzy KNN can have a significant role. As it is able to result the membership degree of each document to each class thus can find the classes which have more overlap because of having some inappropriate features.

As a final discussion, the presented classification algorithms based on AIRS can be particularly useful for organizing texts in decision support environments, where enriching the existing texts for supporting the human elements with their decisions is of particular significance.

REFERENCES [1] A. Danesh, B. Moshiri, and O. Fatemi, “Improve text

classification accuracy based on classifier fusion methods”, 10th Intl. Conf. on Information Fusion, 2007, pp.1-6.

[2] Z. Wang, Y. He, and M. Jiang, “A comparison among three neural networks for text classification”, IEEE Intl. Conf. on Signal Processing ,2006, pp.1883-1886.

[3] S. S. Lin, “A document classification and retrieval system for R&D in semiconductor industry - A hybrid approach”, Expert Systems with Applications, Vol. 36, No. 3, 2009, pp.4753-476.

[4] P. Gu, Q. S. Zhu, and C. Zhang, “A multi-view approach to semi-supervised document classification with incremental naive bayes”, Computers and Mathematics with Applications, Vol. 57, Issue 6, 2009, pp. 1030-1036.

[5] Z. Wang, Q. Zhang, and D. Zhang, “A PSO-based web document classification algorithm”, 8th ACIS Intl. Conf. on Software Engineering, Artificial Intelligence, Networking, and Parallel/Distributed Computing, 2007, pp. 659-664.

[6] M. Segond, C. Fonlupt, and D. Robilliard, “Genetic programming for protein related text classification”, The Genetic and Evolutionary Computation Conf. (GECCO’09), July 2009, pp. 1099-1105.

[7] A. Watkins, “AIRS: A resource limited artificial immune classifier”, Master Thesis, Mississippi State University, 2001.

[8] D. E. Goodman, L. Boggess, and A. Watkins, “An investigation into the source of power for AIRS, an artificial immune classification system”, Intl. joint Conf. on Neural Networks (IJCNN’03), 2003, pp. 1678-1683.

[9] J. Brownlee, “Artificial immune recognition system: a review and analysis”, Technical Report 1-02, Swinburne University of Technology, 2005.

[10] J. Gelernter, D. Cao, R. Lu, E. Fink, and J. G. Carbonell, “Creating and visualizing fuzzy document classification”, IEEE Intl. Conf. on Systems, Man and Cybernetics (SMC 2009), 2009, pp. 672-679.

[11] A. Secker, A. A. Freitas, and J. Timmis, “AISIID: An artificial immune system for interesting information discovery on the web”, Applied Soft Computing, Vol. 8, Issue 2, 2008, pp. 885-905.

[12] M. Srinivas, K. P. Supreethi, E. V. Prasad, and S. A. Kumari, “Efficient text classification using best feature selection and combination of methods”, Intl. Conf. on Human Computer Interaction (HCI’09), 2009.

[13] X. Yue, H. Mo, and Z. X. Chi, “Immune-inspired incremental feature selection technology to data streams”, Applied Soft Computing, Vol. 8, Issue 2, 2008, pp. 1041-1049.

[14] F. Sebastiani, “Machine learning in automated text categorization”, ACM Computing Surveys, Vol. 34, Issue 1, 2002, pp. 1-47.

[15] A. Khan, B. Baharudin, L. H. Lee, and K. khan, “A review of machine learning algorithms for text-documents classification”, Journal of Advances in Information Technology, Vol. 1, No. 1, 2010, pp. 4-20.

[16] M. T.Mahmoudi, N. Forouzideh, C. Lucas, and F. Taghiyareh, “Artificial neural network weights optimization based on imperialist competitive algorithm”, 7th Intl. Conf. on Computer Science and Information Technologies (CSIT 2009), 2009, pp. 244,247.

[17] M. J. Moayed, A. H. Sabery, and A. Khanteymoory, “Ant colony algorithm for web page classification”, Intl. Symp. on Information Technology (ITSim 2008), 2008, pp. 1-8.

[18] C. H. Liu, C. C. Lu, and W. P. Lee, “Document categorization by genetic algorithms”, IEEE Intl. Conf. on Systems, Man, and Cybernetics, Vol.5, 2000, pp. 3868-3872.

[19] X. Sun, Z. Wang, and D. Zhang, “A MA-based web document classification algorithm”, IEEE Intl. Symp. on IT in Medicine and Education, 2008, pp. 952- 955.

[20] J. Twycross, “An immune system approach to document classification”, Master's Thesis, COGS, University of Sussex, U.K., August 2002.

Output Classes R

esea

rch

Dev

elop

men

t/ Pl

anni

ng

Gen

eral

L

earn

ing

Just

ifica

tion

Inno

vatio

n

Ana

lysi

s/ A

sses

smen

t

Membership values of doc. 20 0 0 0 0.4809 0 0.5191

Membership values of doc. 35 0 0 0 0.3439 0.1124 0.5437

Membership values of doc. 56 0 0 0 0.5651 0.4349 0

[21] H. Kopackova, and L. Kopacek, “Text-based decision making with artificial immune systems”, 6th WSEAS Intl. Conf. on Simulation, Modelling and Optimization, 2006, pp. 432-437.

[22] J. Greensmith, and S. Cayzer, “An artificial immune system approach to semantic document classification”, 2nd Intl. Conf. on Artificial Immune Systems (ICARIS 2003), LNCS 2787, Springer, Edinburgh, UK, 2003, pp 136-146.

[23] O. Neles, “Nonlinear system identification”, Springer Pub., 2001.

[24] M. T. Mahmoudi, K. Badie, B. N. Araabi, and N. Forouzideh, “Classifying content mode of organizational texts using simple neural & neuro-fuzzy approaches”, accepted in The Second International Conference on Creative Content Technologies (CONTENT 2010), 2010, in press (unpublished).

[25] K. Badie, M. Kharrat, M. T. Mahmoudi, M. S. Mirian, S. Babazadeh, and T. M. Ghazi, “Creating contents based on inter-play between the ontologies of content’s key segments and problem context”, The First Intl. Conf. on Creative Content Technologies (CONTENT 2009), 2009, pp. 626-631.

[26] J. H. Carter, “The immune system as a model for pattern recognition and classification”, Journal of American Medical Informatics Association 7 (1), 2000, pp. 28-41.

[27] J. Timmis, and M. Neal, “Investigating the evolution and stability of a resource limited artificial immune systems”, Genetic and Evolutionary Computation Conference, Nevada, 2000, pp. 40-41.

[28] L. N. De Castro, and F. J. Von Zubben, “The clonal selection algorithm with engineering applications”, Genetic and Evolutionary Computation Conference, Nevada, 2000, pp. 36–37.

[29] A. Watkins, and J. Timmis, “Artificial immune recognition system: revisions and refinements”, 1st International Conference on Artificial Immune Systems (ICARIS 2002), 2002. pp. 173-181.

[30] A. Watkins, J. Timmis, and L. Boggess, “Artificial immune recognition system (AIRS): an immune-inspired supervised learning algorithm”, Genetic Programming and Evolvable Machines, 5, 2004, pp. 291-317.

[31] A. Watkins, “Exploiting immunological metaphors in the development of serial, parallel, and distributed learning algorithms”, Ph.D. Thesis, Mississippi State University, 2005.

[32] A. F. De Souza, F. Pedroni, E. Oliveira, P. M. Ciarelli, W. F. Henrique, L. Veronese, and C. Badue, ”Automated multi-label text categorization with VG-RAM weightless neural networks”, Neurocomputing, Vol. 72, Issue 10-12, 2009, pp. 2209-2217.

[33] J. M. Keller, R. Gray and J. A. J. R. Givens, “A fuzzy k-nearest neighbor algorithm”, IEEE Trans. Systems Man Cybernet, Vol. 15, Issue 4, 1985, pp. 580–585.

[34] R. Kohavi, and F. Provost, “Glossary of terms, special issue on applications of machine learning and the knowledge discovery process”, Machine Learning, Vol. 30, 1998, pp. 271-274.