chapter 9 chapter 9 neural network neural network

Download Chapter 9 Chapter 9 Neural Network Neural Network

Post on 30-Dec-2015

229 views

Category:

Documents

9 download

Embed Size (px)

TRANSCRIPT

  • Chapter 9Neural Network

    Data Warehouse and Data Mining

    Understand how feed-forward networks are used to solve estimation problems. Know how input and output data conversions are performed for neural networks. Understand how feed-forward neural networks learn through backpropagation. Know how genetic learning is applied to train feed-forward neural networks. Know how self-organizing neural networks perform unsupervised clustering. List the strengths and weaknesses of neural networks.Chapter Objective

    Data Warehouse and Data Mining

    Feed-Forward Neural Network

    Data Warehouse and Data Mining

    Feed-Forward Neural Network

    Data Warehouse and Data Mining

    Neural Network Training: A Conceptual View

    Data Warehouse and Data Mining

    Neural Network Training: A Conceptual View

    Data Warehouse and Data Mining

    Neural Network Training: A Conceptual View

    Data Warehouse and Data Mining

    Neural Network ExplanationSensitivity analysis is a technique that has been successfully applied to gain insight into the effect individual attributes have on neural network output. The general process consists of the following steps:1. Divide the data into a training set and a test dataset.2. Train the network with the training data.

    Data Warehouse and Data Mining

    Neural Network Explanation3. Use the test set data to create a new instance I. Each attribute value for I is the average of all attribute values within the test data.4. For each attribute:a. Vary the attribute value within instance I and present the modification of I to the network for classification.b. Determine the effect the variations have on the output of the neural network.c. The relative importance of each attribute is measured by the effect of attribute variations on network output.

    Data Warehouse and Data Mining

    General ConsiderationsThe following is a partial list of choices that affect the performance of a neural network model:What input attributes will be used to build the network?How will the network output be represented?How many hidden layers should the network contain?How many nodes should there be in each hidden layer?What condition will terminate network training?

    Data Warehouse and Data Mining

    Neural Network Training: A Detailed View

    Data Warehouse and Data Mining

    Neural Networks Advantagesprediction accuracy is generally highrobust, works when training examples contain errorsoutput may be discrete, real-valued, or a vector of several discrete or real-valued attributesfast evaluation of the learned target functionCriticismlong training timedifficult to understand the learned function (weights)not easy to incorporate domain knowledge

    Data Warehouse and Data Mining

    A NeuronThe n-dimensional input vector x is mapped into variable y by means of the scalar product and a nonlinear function mapping

    Data Warehouse and Data Mining

    Network TrainingThe ultimate objective of training obtain a set of weights that makes almost all the tuples in the training data classified correctly StepsInitialize weights with random values Feed the input tuples into the network..... one by oneFor each unitCompute the net input to the unit as a linear combination of all the inputs to the unitCompute the output value using the activation functionCompute the errorUpdate the weights and the bias

    Data Warehouse and Data Mining

    Multi-Layer PerceptronOutput nodesInput nodesHidden nodesOutput vectorInput vector: xiwij

    Data Warehouse and Data Mining

    Network Pruning and Rule ExtractionNetwork pruningFully connected network will be hard to articulateN input nodes, h hidden nodes and m output nodes lead to h(m+N) weightsPruning: Remove some of the links without affecting classification accuracy of the network

    Data Warehouse and Data Mining

    Network Pruning and Rule ExtractionExtracting rules from a trained networkDiscretize activation values; replace individual activation value by the cluster average maintaining the network accuracyEnumerate the output from the discretized activation values to find rules between activation value and outputFind the relationship between the input and activation value Combine the above two to have rules relating the output to input

    Data Warehouse and Data Mining

    A neural network is parallel computing system of several interconnected processor nodes. The input to individual network nodes is restricted to numeric values falling in the closed interval range [0,1].Because of this, categorical data must be transformed prior to network training.Chapter Summary

    Data Warehouse and Data Mining

    Chapter SummaryDeveloping a neural network involves first training the network to carry out the desired computations and then applying the trained network to solve new problems. During the learning phase, training data is used to modify the connection weights between pairs of nodes so as to obtain a best result for the output node (s).The feed-forward neural network architecture is commonly used for supervised learning. Feed-forward neural networks contain a set of layered nodes and weighted connections between nodes in adjacent layers.

    Data Warehouse and Data Mining

    Feed-forward neural networks are often trained using a backpropagation learning scheme. Backpropagation learning works by making modifications in weight values starting at the output layer then moving backward through the hidden layers of the network. Genetic learning can also be applied to train feed-forward networks.Chapter Summary

    Data Warehouse and Data Mining

    Chapter SummaryThe self-organizing Kohonen neural network architecture is a popular model for unsupervised clustering. A self-organizing neural network learns by having several output nodes complete for the training instances. For each instance, the output node whose weight vectors most closely match the attribute values of the input instance is the winning node.

    Data Warehouse and Data Mining

    Chapter SummaryAs a result, the winning node has its associated input weights modified to more closely match the current training instance. When unsupervised learning is complete, output nodes winning the most instances are saved. After this, test data is applied and the clusters formed by the test set data are analyzed to help determine the meaning of what has been found.

    Data Warehouse and Data Mining

    Chapter SummaryA central issue surrounding neural network is their inability to explain what has been learned. Despite this, neural network have been successfully applied to solve problems in both the business and scientific world. Although we have discussed the most popular neural network models, several other architectures and learning rules have been developed. Jain, Mao, and Mohiuddin (1996) provide a good starting point for learning more about neural networks.

    Data Warehouse and Data Mining

    Key TermsAverage member technique. An unsupervised clustering neural network explanation technique where the most typical member of each cluster is computed by finding the average value for each class attribute.Backpropagation learning. A training method used with many feed-forward networks that works by making modifications in weight values starting at the output layer then moving backward through the hidden layer.Delta rule. A neural network learning rule designed to minimize the sum of squared errors between computed and target network output.

    Data Warehouse and Data Mining

    Key TermsEpoch. One complete pass of the training data through a neural network.Feed-forward neural network. A neural network architecture where all weights at one layer are directed toward nodes at the next network layer. Weights do not cycle back as inputs to previous layers.Fully connected. A neural network structure where all nodes at one layer of the network are connected to all nodes in the next layer.Kohonen network. A two-layer neural network used for unsupervised clustering.

    Data Warehouse and Data Mining

    Key TermsNeural network. A parallel computing system consisting of several interconnected processors.Neurode. A neural network processor node. Several neurodes are connected to form a complete neural network structure.Sensitivity analysis. A neural network explanation technique that allows us to determine a rank ordering for the relative importance of individual attributes.Sigmoid function. One of several commonly used neural network evaluation functions. The sigmoid function is continuous and outputs a value between 0 or 1.

    Data Warehouse and Data Mining

    Key TermsLinearly separable. Two classes, A and B, are said to be linearly separable if a straight line can be drawn to separate the instances of class B.Perceptron neural network. A simple feed-forward neural network architecture consisting of an input layer and a single output layer.

    Data Warehouse and Data Mining

    ReferenceData Mining: Concepts and Techniques (Chapter 7 Slide for textbook), Jiawei Han and Micheline Kamber, Intelligent Database Systems Research Lab, School of Computing Science, Simon Fraser University, Canada