network objects, data, and

8/3/2019 Network Objects, Data, and

1/16

Network Objects, Data, and

Training Styles


2/16

Introduction

The work flow for the neural network design process has seven primarysteps:

1. Collect data

2. Create the network

3. Configure the network4. Initialize the weights and biases

5. Train the network

6. Validate the network

7. Use the network

This chapter shows how to format the data for presentation to thenetwork. It also explains network configuration and the two forms ofnetwork training: incremental training and batch training.


3/16

Introduction

There are four different levels at which the Neural Network Toolboxsoftware can be used.

The first level is represented by the GUIs and can be launched by using nnstart . These provide a quick way to access the power of thetoolbox for many problems of function fitting, pattern recognition,

clustering and time series analysis. The second level of toolbox use is through basic command-line

operations. The command-line functions use simple argument listswith intelligent default settings for function parameters.

A third level of toolbox use is customization of the toolbox. This

advanced capability allows you to create your own custom neuralnetworks, while still having access to the full functionality of thetoolbox.

The fourth level of toolbox usage is the ability to modify any of the M-files contained in the toolbox.


4/16

MODEL

Simple Neuron

The fundamental building block

for neural networks is the single-input

neuron, such as this example.

There three processes whichtaking place are : the weight function,

the net input function and the transfer

function.


5/16

Transfer Functions

Many transfer functions are included in the Neural Network Toolboxsoftware. Two of the most commonly used functions are shown below.

Linear Transfer Function

Neurons of this type are used inthe final layer of multilayer networks

that are used as function approximations.


6/16

Neural Network DESIGN One-Input Neuron

Alter the weight, biasand input by draggingthe triangular shapedindicators.

Pick the transferfunction w ith theF menu.

Watch the change tothe neuron functionand its output.

Chapter 2

1

pw

b

a

Input Line ar Ne uron: a = pure lin(w *p+b)

F:

-2 0 2

w

-2 0 2

b

-4 -2 0 2 4-4

-2

0

2

4

p

a

Neural Network DESIGN One-Input Neuron

Alter the weight, bias

and input by dragging

the triangular shaped

indicators.

Pick the transferfunction with the

F menu.

Watch the change to

the neuron functionand its output.

Chapter 2

1

pw

b

a

Input Line ar Ne uron: a = pur elin(w *p+b)

F:

-2 0 2

w

-2 0 2

b

-4 -2 0 2 4-4

-2

0

2

4

p

a


7/16

Neuron with Vector Input

The simple neuron can be extended to handle inputs that are vectors. Aneuron with a singleR-element input vector is shown below. Here theindividual input

elements are multiplied by weights

and the weighted values are fed to

the summing junction.

This expression can, of course,

be written in MATLAB code as


8/16

Abbreviated Notation


9/16

Network Architectures

One Layer of Neurons


10/16

Multiple Layers of Neurons

To describe networks having multiple layers, the notationmust be extended. Specifically, it needs to make adistinction between weight matrices that are connected toinputs and weight matrices that are connected betweenlayers. It also needs to identify the source and destinationfor the weight matrices.

We will call weight matrices connected to inputs inputweights; we willcall weight matrices connected to layeroutputs layer weights.

Further, superscripts are used to identify the source(second index) and the destination (first index) for thevarious weights and other elements of the network.


11/16

the weight matrixconnected to the input

vector p is labeled as aninput weight matrix

(IW1,1) having a source1 (second index) and adestination 1 (firstindex). Elements of layer

1, such as its bias, net input,and output have asuperscript 1 to say thatthey are associated with the

first layer.


12/16


13/16


14/16

The layers of a multilayer network play different roles. Alayer that produces the network output is called an outputlayer. All other layers are calledhidden layers. The three-layer network shown earlier has one output layer (layer 3)

and two hidden layers (layer 1 and layer 2). Some authorsrefer to the inputs as a fourth layer. This toolbox does notuse that designation.


15/16

Input and Output Processing Functions

Input Processing FunctionsNetwork inputs might have associated processing functions. Processingfunctions transform user input data to a form that is easier or moreefficient for a network.

Mapminmax transforms input data so that all values fall into theinterval [1, 1]. This can speed up learning for many networks.

removeconstantrows removes the rows of the input vector thatcorrespond to input elements that always have the same value, becausethese input elements are not providing any useful information to thenetwork.

fixunknowns which recodes unknown data (represented in the usersdata with NaN values) into a numerical form for the network. Fixunknowns preserves information about which values are known andwhich are unknown.


16/16

Output Processing Functions

Output processing functions are used to transform user-providedtarget vectors for network use. Then, network outputs are reverse-

processed using the same functions to produce output data with thesame characteristics as the original user-provided targets.

Both mapminmax and removeconstantrows are often associatedwith network outputs. However, fixunknowns is not. Unknown values

in targets (represented by NaN values) do not need to be altered fornetwork use.

network objects, data, and

Documents