1 introduction to neural networks and their applications

1

Introduction to Neural Introduction to Neural Networks And Their Networks And Their

ApplicationsApplications

2

Table of ContentsTable of Contents

I. Introduction of Neural NetworksII. Application of Neural NetworksIII. Theory of Neural NetworksIV. A Neural Network Demo

3

What is neural networks ?What is neural networks ?

http://www.youtube.com/watch?v=DG5-UyRBQD4&feature=rellist&playnext=1&list=PL4FA5D71B0BA92C1C

4

It is simulation of human brain It is the most well known artificial

intelligence techniques We are using them: voice

recognition system, reading hand writes, door rocks et al.

It is a called black box

I. Introduction of Neural I. Introduction of Neural NetworksNetworks

5

Neural Networks simulate human brain Learning in Human Brain

Neurons Connection Between Neurons

Neural Networks As Simulator For Human Brain Processing Elements or Nodes Weights

It is a simulator for human brainIt is a simulator for human brain

6

II. Applications of Neural II. Applications of Neural NetworksNetworks Prediction of Outcomes

Patterns Detection in Data

Classification

7

Accounting Identify tax fraud Enhance auditing by finding irregularities

Finance Signatures and bank note verifications Foreign exchange rate forecasting Bankruptcy prediction Customer credit scoring Credit card approval and fraud detection* Stock and commodity selection and trading Forecasting economic turning points Pricing initial public offerings* Loan approvals

Business ANN Applications -1Business ANN Applications -1

8

Business ANN Applications -2Business ANN Applications -2

Human Resources Predicting employees’ performance and behavior Determining personnel resource requirements

Management Corporate merger prediction Country risk rating

Marketing Consumer spending pattern classification Sales forecasts Targeted marketing, …

Operations Vehicle routing Production/job scheduling, …

9

Neural Computing is a problem solving methodology that attempts to mimic how human brain functions

Artificial Neural Networks (ANN)

Machine Learning/Artificial Intelligence

III. Theory of Neural III. Theory of Neural NetworksNetworks

10

The Biological AnalogyThe Biological Analogy

Neurons: brain cells Nucleus (at the center) Dendrites provide

inputs Axons send outputs

Synapses increase or decrease connection strength and cause excitation or inhibition of subsequent neurons

11

Biological ArtificialSoma <-> NodeDendrites <-> InputAxon <-> OutputSynapse <-> Weight

Artificial Neural Networks (ANN)Artificial Neural Networks (ANN)

Three Interconnected Artificial Neurons

12

Basic structure of Neural NetworksBasic structure of Neural Networks

Network Structure : Layers, Nodes and Weights

Input Layer Hidden Layer Output Layer

13

ANN FundamentalsANN Fundamentals

14

Processing Information by the Network Inputs Outputs Weights Summation Function

Figure 15.5

ANN Fundamentals: how ANN Fundamentals: how informatio is processed in ANNinformatio is processed in ANN

15

Learning in NN(Neural Network) Learning in NN(Neural Network) is finding the best numeric is finding the best numeric values (X), representing input values (X), representing input (4) and output(8) relationship (4) and output(8) relationship ( ex: 4 * X = 8 )( ex: 4 * X = 8 )*Try with x= 1, x= 2, x=3, …… When x=4, it solve the problem.*Try with x= 1, x= 2, x=3, …… When x=4, it solve the problem.

1. Compute outputs2. Compare outputs

with desired targets

3. Adjust the weights and repeat the process

16

Neural Network ArchitectureNeural Network Architecture

There are several ANN architectures :feed forward, recurrent, Hopfield et al.

17

Neural Network ArchitectureNeural Network Architecture

Feed forward Neural Network: Multi Layer Perceptron, - Two, Three, sometimes Four or Five Layers, But normally 3 layers are common structure.

18

Step function evaluates the summation of input values

Calculating outputs Measure the error (delta) between outputs

and desired values Update weights, reinforcing correct resultsAt any step in the process for a neuron, j, we

getDelta(Error) = Zj - Yj

where Z and Y are the desired and actual outputs, respectively

How a Network LearnsHow a Network Learns

19

1. Initialize the weights2. Read the input

vector3. Generate the output4. Compute the error

Error = Output – Desired output

5. Change the weights

Drawbacks: A large network can take a very long time to

train May not converge

BackpropagationBackpropagation

20

Training A Neural NetworksTraining A Neural Networks

Neural Networks learn from data Learning is finding the best weights

values which represent the input and output relationship in Neural Networks

(ex: 4*X= 8)-> finding the value for X

21

Collect data and separate it into Training set (50%), Testing set (50%) Training set (60%), Testing set (40%) Training set (70%), Testing set (30%) Training set (80%), Testing set (20%) Training set (90%), Testing set (10%)

Use training data set to build model Use test data set to validate the trained

network

training data set and test data training data set and test data setset

22

Prediction with New DataPrediction with New Data

If the Neural Network's performance in test is good , it can be used to predict outcome of new unseen data

If the performance with test is not good, you should collect more data, add more input variables

23

Terms in Neural Networks

How does Neural Network work for prediction?

24

Demo – How does Neural Network work for prediction?

25

ANN Development ToolsANN Development Tools

26

Why use Neural Networks in Why use Neural Networks in Prediction? - major benefits of Prediction? - major benefits of

Neural NetworksNeural Networks

27

Benefits of ANNBenefits of ANN

Advantages:Non-linear model leads to better performanceIt works generally good when data size is smallIt works generally good when there are noises in dataIt works generally good when there are missing in data (incomplete data set)Fast decision making

Diverse Applications:Pattern recognitionCharacter, speech and visual recognition

28

Limitations of ANNLimitations of ANN

Black box that is hardly understood by human

Lack of explanation capabilities Training time can be excessive

and tedious

29

IV. A Neural Networks DemoIV. A Neural Networks Demo

How do neural networks learn? : trials and errorshttp://www.youtube.com/watch?v=0Str0Rdkxxo