Intrusion detection by machine learning: A review

Download Intrusion detection by machine learning: A review

Post on 21-Jun-2016




2 download


  • A-Y


    tern nIn les.e le

    reviews 55 related studies in the period between 2000 and 2007 focusing on developing single, hybrid,

    systems by machine learning are present and discussed. A number of future research directions are also

    daily l, suchlar, Intess m

    and computer systems usage, whereas the conventional rewallcan not perform this task. Intrusion detection is based on theassumption that the behavior of intruders different from a legaluser (Stallings, 2006).

    In general, IDSs can be divided into two categories: anomalyand misuse (signature) detection based on their detection ap-

    niques have been used, what experiments have been conducted,and what should be considered for future work based on the ma-chine learnings perspective.

    This paper is organized as follows. Section 2 provides anoverview of machine learning techniques and briey describesa number of related techniques for intrusion detection. Section 3compares related work based on the types of classier design,the chosen baselines, datasets used for experiments, etc. Conclu-sion and discussion for future research are given in Section 4.

    * Corresponding author. Tel.: +886 5 2720411; fax: +886 5 2720859.

    Expert Systems with Applications 36 (2009) 1199412000

    Contents lists availab


    .eE-mail address: (W.-Y. Lin).2007). For the business operation, both business and customers ap-ply the Internet application such as website and e-mail on businessactivities. Therefore, information security of using Internet as themedia needs to be carefully concerned. Intrusion detection is onemajor research problem for business and personal networks.

    As there are many risks of network attacks under the Internetenvironment, there are various systems designed to blockthe Internet-based attacks. Particularly, intrusion detection sys-tems (IDSs) aid the network to resist external attacks. That is,the goal of IDSs is to provide a wall of defense to confront theattacks of computer systems on Internet. IDSs can be used ondetect difference types of malicious network communications

    developed based on many different machine learning techniques(c.f. Section 3). For example, some studies apply single learn-ing techniques, such as neural networks, genetic algorithms,support vector machines, etc. On the other hand, some systemsare based on combining different learning techniques, such ashybrid or ensemble techniques. In particular, these techniquesare developed as classiers, which are used to classify or recognizewhether the incoming Internet access is the normal access or an at-tack. However, there is no a review of these different machinelearning techniques over the intrusion detection domain.

    Therefore, the goal of this paper is to review 55 related studies/systems published from 2000 to 2007 by examining what tech-1. Introduction

    The Internet has become a part oftoday. It aids people in many areasment and education, etc. In particuan important component of busin0957-4174/$ - see front matter 2009 Elsevier Ltd. Adoi:10.1016/j.eswa.2009.05.029provided. 2009 Elsevier Ltd. All rights reserved.

    ife and an essential toolas business, entertain-ernet has been used asodels (Shon & Moon,

    proaches (Anderson, 1995; Rhodes, Mahaffey, & Cannady, 2000).Anomaly detection tries to determine whether deviation fromthe established normal usage patterns can be agged as intrusions.On the other hand, misuse detection uses patterns of well-knownattacks or weak spots of the system to identify intrusions.

    In literature, numbers of anomaly detection systems areand ensemble classiers. Related studies are compared by their classier design, datasets used, andother experimental setups. Current achievements and limitations in developing intrusion detectionReview

    Intrusion detection by machine learning:

    Chih-Fong Tsai a, Yu-Feng Hsu b, Chia-Ying Lin c, WeiaDepartment of Information Management, National Central University, TaiwanbDepartment of Information Management, National Sun Yat-Sen University, TaiwancDepartment of Accounting and Information Technology, National Chung Cheng UniversdDepartment of Computer Science and Information Engineering, National Chung Cheng

    a r t i c l e i n f o

    Keywords:Intrusion detectionMachine learningHybrid classiersEnsemble classiers

    a b s t r a c t

    The popularity of using Inmajor research problem isecure internal networks.machine learning techniqurent status of using machin

    Expert Systems

    journal homepage: wwwll rights

    ang Lin d,*

    Taiwanersity, Taiwan

    net contains some risks of network attacks. Intrusion detection is oneetwork security, whose aim is to identify unusual access or attacks toiterature, intrusion detection systems have been approached by variousHowever, there is no a review paper to examine and understand the cur-arning techniques to solve the intrusion detection problems. This chapter

    le at ScienceDirect

    ith Applications

    lsevier .com/locate /eswa

  • Pattern recognition is the action to take raw data and activity on

    chines, articial neural network, decision trees, self-organizing

    on-line trains the examples and nds out k-nearest neighbor of

    h Apthe new instance.

    2.2.2. Support vector machinesSupport vector machines (SVM) is proposed by Vapnik (1998).

    SVM rst maps the input vector into a higher dimensional featurespace and then obtain the optimal separating hyper-plane in thehigher dimensional feature space. Moreover, a decision boundary,i.e. the separating hyper-plane, is determined by support vectorsrather than the whole training samples and thus is extremely ro-bust to outliers.

    In particular, an SVM classier is designed for binary classica-tion. That is, to separate a set of training vectors which belong totwo different classes. Note that the support vectors are the trainingsamples close to a decision boundary. The SVM also provides a userspecied parameter called penalty factor. It allows users to make atradeoff between the number of misclassied samples and thewidth of a decision boundary.

    2.2.3. Articial neural networksThe neural network is information processing units which to

    mimic the neurons of human brain (Haykin, 1999). Multilayer per-ceptron (MLP) is the widely used neural network architecture inmaps, etc.) have been used to solve these problems.

    2.2.1. K-nearest neighborK-nearest neighbor (k-NN) is one of the most simple and tradi-

    tional nonparametric technique to classify samples (Bishop, 1995;Manocha & Girolami, 2007). It computes the approximate dis-tances between different points on the input vectors, and then as-signs the unlabeled point to the class of its K-nearest neighbors. Inthe process of create k-NN classier, k is an important parameterand different k values will cause different performances. If k is con-siderably huge, the neighbors which used for prediction will makelarge classication time and inuence the accuracy of prediction.

    k-NN is called instance based learning, and it is different fromthe inductive learning approach (Mitchell, 1997). Thus, it doesnot contain the model training stage, but only searches the exam-ples of input vectors and classies new instances. Therefore, k-NNdata category (Michalski, Bratko, & Kubat, 1998). The methods ofsupervised and unsupervised learning can be used to solve differ-ent pattern recognition problems (Theodoridis & Koutroumbas,2006, 2006). In supervised learning, it is based on using the train-ing data to create a function, in which each of the training datacontains a pair of the input vector and output (i.e. the class label).

    The learning (training) task is to compute the approximate dis-tance between the inputoutput examples to create a classier(model). When the model is created, it can classify unknown exam-ples into a learned class labels.

    2.2. Single classiers

    The intrusion detection problem can be approached by usingone single machine learning algorithm. In literature, machinelearning techniques (e.g. k-nearest neighbor, support vector ma-2. Machine learning techniques

    2.1. Pattern classication

    C.-F. Tsai et al. / Expert Systems witmany pattern recognition problems. A MLP network consists ofan input layer including a set of sensory nodes as input nodes,one or more hidden layers of computation nodes, and an outputlayer of computation nodes. Each interconnection has associatedwith it a scalar weight which is adjusted during the training phase.

    In addition, the backpropagation learning algorithm is usuallyused to train a MLP, which are also called as backpropagation neu-ral networks. First of all, random weights are given at the begin-ning of training. Then, the algorithm performs weights tuning todene whatever hidden unit representation is most effective atminimizing the error of misclassication.

    2.2.4. Self-organizing mapsSelf-organizing map (SOM) (Kohonen, 1982) is trained by an

    unsupervised competitive learning algorithm, a process of selforganization. The aim of SOM is to reduce the dimension of datavisualization. That is, SOM projects and clusters high-dimensionalinput vectors onto a low-dimensional visualized map, usually 2 forvisualization. It usually consists of an input layer and the Kohonenlayer which is designed as two-dimensional arrangement of neu-rons that maps n dimensional input to two dimensions. KohonensSOM associates each of the input vectors to a representative out-put. The network nds the node closest to each training case andmoves the winning node, which is the closest neuron (i.e. the neu-ron with minimum distance) to the training case.

    That is, SOM maps similar input vectors onto the same or sim-ilar output units on such a two-dimensional map. Therefore, out-put units will self-organize to an ordered map and those outputunits with similar weights are also placed nearby after training.

    2.2.5. Decision treesA decision tree classies a sample through a sequence of deci-

    sions, in which the current decision helps to make the subsequentdecision. Such a sequence of decisions is represented in a treestructure. The classication of a sample proceeds from the rootnode to a suitable end leaf node, where each end leaf node repre-sents a classication category. The attributes of the samples are as-signed to each node, and the value of each branch is correspondingto the attributes (Mitchell, 1997).

    A well-known program for constructing decision trees is CART(Classication and Regressing Tree) (Breiman, Friedman, Olshen,& Stone, 1984). A decision tree with a range of discrete (symbolic)class labels is called a classication tree, whereas a decision treewith a range of continuous (numeric) values is called a regressiontree.

    2.2.6. Nave bayes networksThere are many cases where we know the statistical dependen-

    cies or the causal relationships between system variables. How-ever, it might be difcult to precisely express the probabilisticrelationships among these variables. In other words, the priorknowledge about the system is simply that some variable mightinuence others. To exploit this structural relationship or casualdependencies between the random variables of a problem, onecan use a probabilistic graph model called Nave Baysian Networks(NB).

    The model provides an answer to questions like What is theprobability that it is a certain type of attack, given some observedsystem events? by using conditional probability formula. Thestructure of a NB is typically represented by a directed acyclicgraph (DAG), where each node represents one of system variablesand each link encodes the inuence of one node upon another(Pearl, 1988). Thus, if there is a link from node A to node B, A di-rectly inuences B.

    2.2.7. Genetic algorithms

    plications 36 (2009) 1199412000 11995Genetic algorithms (GA) use the computer to implement thenatural selection and evolution (Koza, 1992). This concept comesfrom the adaptive survival in natural organisms. The algorithm

  • starts by randomly generating a large population of candidate pro-grams. Some type of tness measure to evaluate the performanceof each individual in a population is used. A large number of iter-ations is then performed that low performing programs are re-placed by genetic recombinations of high performing programs.That is, a program with a low tness measure is deleted and doesnot survive for the next computer iteration.

    2.2.8. Fuzzy logicFuzzy logic (or fuzzy set theory) is based on the concept of the

    fuzzy phenomenon to occur frequently in real world. Fuzzy set the-ory considers the set membership values for reasoning and the val-

    numbers of the 55 articles using single, ensemble, and hybrid

    creases gradually after that. Due to the recent developments inintrusion detection, it is now very difcult to design a singleapproach which outperforms the existing ones. On th other hand,the hybrid approaches have moved from marginalization to main-stream in the recent years. A strong supporting evidence is thatthere are 10 research publications based on hybrid approaches in


    6al. (2007), Bridges and Vaughn (2000), Chavan), Chen et al. (2007), Depren et al. (2005), Eskin), Florez et al. (2002), Giacinto and Roli (2003),(2006), Joo et al. (2003), Kayacik et al. (2007),(2007), Lee and Stolfo (1998, 2000), Liu and Yiet al. (2007, 2004), Luo and Bridgest (2000),

    d Zulkernine (2004), Ozyer et al. (2007),igari et al. (2007), Shon et al. (2006), Shon and7); Stein et al. (2005), Toosi and Kahani (2007),

    Abadeh et al. (2007), Giacinto et al.(2006, 2008), Giacinto and Roli(2003), Han and Cho (2003),Kang et al. (2005), Mukkamala et al.(2005); Peddabachigari et al. (2007)









    of a



    11996 C.-F. Tsai et al. / Expert Systems with Applications 36 (2009) 1199412000ues range between 0 and 1. That is, in fuzzy logic the degree oftruth of a statement can range between 0 and 1 and it is not con-strained to the two truth values (i.e. true, false). For examples,rain is a commonly natural phenomenon, and it may have veryerce change. Raining may be able to convert the circumstancesfrom slight to violent (Zimmermann, 2001).

    2.3. Hybrid classiers

    In the development of an IDS, the ultimate goal is to achieve thebest possible accuracy for the task at hand. This objective naturallyleads to the design of hybrid approaches for the problem to besolved. The idea behind a hybrid classier is to combine severalmachine learning techniques so that the system performance canbe signicantly improved. More specically, a hybrid approachtypically consists of two functional components. The rst one takesraw data as input and generates intermediate results. The secondone will then take the intermediate results as the input and pro-duce the nal results (Jang, Sun, & Mizutani, 1996).

    In particular, hybrid classiers can be based on cascading differ-ent classiers, such as neuro-fuzzy techniques. On the other hand,hybrid classiers can use some clustering-based approach to pre-process the input samples in order to eliminate unrepresentativetraining examples from each class. Then, the clustering resultsare used as training examples for classier design. Therefore, t...


View more >