perceptrons michael j. watts
DESCRIPTION
ANN Revision Artificial neurons o mathematical abstractions of biological neurons o three functions associated with each neuron input activation output o McCulloch and Pitts neuron the first to be designedTRANSCRIPT
![Page 1: Perceptrons Michael J. Watts](https://reader033.vdocuments.mx/reader033/viewer/2022050917/5a4d1b867f8b9ab0599bcce2/html5/thumbnails/1.jpg)
Perceptrons
Michael J. Watts
http://mike.watts.net.nz
![Page 2: Perceptrons Michael J. Watts](https://reader033.vdocuments.mx/reader033/viewer/2022050917/5a4d1b867f8b9ab0599bcce2/html5/thumbnails/2.jpg)
Lecture Outline
• ANN revision• Perceptron architecture• Perceptron learning• Problems with perceptrons
![Page 3: Perceptrons Michael J. Watts](https://reader033.vdocuments.mx/reader033/viewer/2022050917/5a4d1b867f8b9ab0599bcce2/html5/thumbnails/3.jpg)
ANN Revision
• Artificial neuronso mathematical abstractions of biological neuronso three functions associated with each neuron
input activation output
o McCulloch and Pitts neuron the first to be designed
![Page 4: Perceptrons Michael J. Watts](https://reader033.vdocuments.mx/reader033/viewer/2022050917/5a4d1b867f8b9ab0599bcce2/html5/thumbnails/4.jpg)
ANN Revision
• Artificial Neural Networkso Collections of connected artificial neuronso Each connection has a weighting value attached
to ito Weight can be positive or negativeo Usually layeredo Signals are propagated from one layer to another
![Page 5: Perceptrons Michael J. Watts](https://reader033.vdocuments.mx/reader033/viewer/2022050917/5a4d1b867f8b9ab0599bcce2/html5/thumbnails/5.jpg)
Perceptron Architecture
• An ANN using McCulloch and Pitts neurons• Proposed by Rosenblatt in 1958• Developed to study theories about the visual
system• Layered architecture
![Page 6: Perceptrons Michael J. Watts](https://reader033.vdocuments.mx/reader033/viewer/2022050917/5a4d1b867f8b9ab0599bcce2/html5/thumbnails/6.jpg)
Perceptron Architecture
• Feedforwardo signals travel only forwardo no recurrent connections
• Inputs are continuous• Outputs are binary• Used for classification
![Page 7: Perceptrons Michael J. Watts](https://reader033.vdocuments.mx/reader033/viewer/2022050917/5a4d1b867f8b9ab0599bcce2/html5/thumbnails/7.jpg)
Perceptron Architecture
• Three neuron layerso first is a buffer, usually ignoredo second is the input or "feature" layero third is the output or "perceptron" layer
![Page 8: Perceptrons Michael J. Watts](https://reader033.vdocuments.mx/reader033/viewer/2022050917/5a4d1b867f8b9ab0599bcce2/html5/thumbnails/8.jpg)
Perceptron Architecture
• Buffer neurons are arbitrarily connected to feature neuronso usually one-to-one
each buffer neuron is connected only to the corresponding feature neuron
o can combine multiple buffer values into one feature
• Connection weight values from buffer to feature layer are fixed
![Page 9: Perceptrons Michael J. Watts](https://reader033.vdocuments.mx/reader033/viewer/2022050917/5a4d1b867f8b9ab0599bcce2/html5/thumbnails/9.jpg)
Perceptron Architecture
• Feature neurons are fully connected to perceptron neuronso Each feature neuron is connected to every
perceptron neuron• Weights from feature to perceptron layer are
variable• One modifiable connection layer
o single layer network
![Page 10: Perceptrons Michael J. Watts](https://reader033.vdocuments.mx/reader033/viewer/2022050917/5a4d1b867f8b9ab0599bcce2/html5/thumbnails/10.jpg)
Perceptron Architecture
![Page 11: Perceptrons Michael J. Watts](https://reader033.vdocuments.mx/reader033/viewer/2022050917/5a4d1b867f8b9ab0599bcce2/html5/thumbnails/11.jpg)
Perceptron Architecture
• Threshold neurons are used in the perceptron layer
• Threshold can be a step function or a linear threshold function
• Biases can be used to create the effect of a threshold
![Page 12: Perceptrons Michael J. Watts](https://reader033.vdocuments.mx/reader033/viewer/2022050917/5a4d1b867f8b9ab0599bcce2/html5/thumbnails/12.jpg)
Perceptron Architecture
• Perceptron recallo propagate a signal from buffer to perceptron layero signal elements are multiplied by the connection
weight at each layero output is the activation of the output (perceptron)
neurons
![Page 13: Perceptrons Michael J. Watts](https://reader033.vdocuments.mx/reader033/viewer/2022050917/5a4d1b867f8b9ab0599bcce2/html5/thumbnails/13.jpg)
Perceptron Learning
• Supervised learning algorithm• Minimises classification errors
![Page 14: Perceptrons Michael J. Watts](https://reader033.vdocuments.mx/reader033/viewer/2022050917/5a4d1b867f8b9ab0599bcce2/html5/thumbnails/14.jpg)
Perceptron Learning
1. propagate an input vector through the perceptron2. calculate the error for each output neuron by subtracting the actual output from the target output:3. modify each weight4. repeat steps 1-3 until the perceptron converges
![Page 15: Perceptrons Michael J. Watts](https://reader033.vdocuments.mx/reader033/viewer/2022050917/5a4d1b867f8b9ab0599bcce2/html5/thumbnails/15.jpg)
Perceptron Learning
• Other error measures used• Widrow-Huff learning rule• Used in a system call ADALINE
o ADAptive LINEar neuron
![Page 16: Perceptrons Michael J. Watts](https://reader033.vdocuments.mx/reader033/viewer/2022050917/5a4d1b867f8b9ab0599bcce2/html5/thumbnails/16.jpg)
Problems with Perceptrons
• Minsky and Papert (1969)o Mathematically prove limitations of perceptronso Caused most AI researchers of the time to return
to symbolic AI• The “Linear Separability” problem
o Perceptrons can only distinguish between classes separated by a plane
![Page 17: Perceptrons Michael J. Watts](https://reader033.vdocuments.mx/reader033/viewer/2022050917/5a4d1b867f8b9ab0599bcce2/html5/thumbnails/17.jpg)
Linear Separability
![Page 18: Perceptrons Michael J. Watts](https://reader033.vdocuments.mx/reader033/viewer/2022050917/5a4d1b867f8b9ab0599bcce2/html5/thumbnails/18.jpg)
Linear Separability
• XOR problem is not linearly separable• Truth table
![Page 19: Perceptrons Michael J. Watts](https://reader033.vdocuments.mx/reader033/viewer/2022050917/5a4d1b867f8b9ab0599bcce2/html5/thumbnails/19.jpg)
Linear Separability
![Page 20: Perceptrons Michael J. Watts](https://reader033.vdocuments.mx/reader033/viewer/2022050917/5a4d1b867f8b9ab0599bcce2/html5/thumbnails/20.jpg)
Linear Separability
• This problem killed research into ANN• ANN field was moribund for >10 years• Reasons for this problem
o Hyperplanes
![Page 21: Perceptrons Michael J. Watts](https://reader033.vdocuments.mx/reader033/viewer/2022050917/5a4d1b867f8b9ab0599bcce2/html5/thumbnails/21.jpg)
Linear Separability
• A hyperplane is a plane of >3 dimensionso in 2 dimensions it is a line
• Each output neuron in a perceptron corresponds to one hyperplane
• Dimensionality of the hyperplane is determined by the number of incoming connections
![Page 22: Perceptrons Michael J. Watts](https://reader033.vdocuments.mx/reader033/viewer/2022050917/5a4d1b867f8b9ab0599bcce2/html5/thumbnails/22.jpg)
Linear Separability
• The position of the hyperplane depends on the weight values
• Without a threshold the hyperplane will pass through the origino causes problems with learning
• Threshold / bias allows the hyperplane to be positioned away from the origin
![Page 23: Perceptrons Michael J. Watts](https://reader033.vdocuments.mx/reader033/viewer/2022050917/5a4d1b867f8b9ab0599bcce2/html5/thumbnails/23.jpg)
Linear Separability
• Learning involves positioning the hyperplane• If the hyperplane cannot be positioned to
separate all training examples, then the algorithm will never convergeo never reach 0 errors
![Page 24: Perceptrons Michael J. Watts](https://reader033.vdocuments.mx/reader033/viewer/2022050917/5a4d1b867f8b9ab0599bcce2/html5/thumbnails/24.jpg)
Advantages of Perceptrons
• Simple models• Efficient• Guaranteed to converge over linearly
separable problems• Easy to analyse• Well known
![Page 25: Perceptrons Michael J. Watts](https://reader033.vdocuments.mx/reader033/viewer/2022050917/5a4d1b867f8b9ab0599bcce2/html5/thumbnails/25.jpg)
Disadvantages of Perceptrons
• Cannot model non-linearly separable problems
• Some problems may require a bias or threshold parameter
• Not taken seriously anymore
![Page 26: Perceptrons Michael J. Watts](https://reader033.vdocuments.mx/reader033/viewer/2022050917/5a4d1b867f8b9ab0599bcce2/html5/thumbnails/26.jpg)
Summary
• Perceptrons are the oldest neural network • One of the first uses of the McCulloch and
Pitts neuron• Supervised learning algorithm• Still useful for some problems• Cannot handle non-linearly separable
problems