neural networks
DESCRIPTION
TRANSCRIPT
![Page 1: Neural Networks](https://reader033.vdocuments.mx/reader033/viewer/2022061112/5457be1aaf795946138b6a37/html5/thumbnails/1.jpg)
NEURAL NETWORKS
& Machine Learning
Justin ChowLevon Mkrtchyan
Eric Su Senior Project
5/16/07
![Page 2: Neural Networks](https://reader033.vdocuments.mx/reader033/viewer/2022061112/5457be1aaf795946138b6a37/html5/thumbnails/2.jpg)
What are Neural Nets?
– Refers to a computing paradigm that is modeled after the structure of the brain.
– Inspired by examination of the central nervous system and the neurons
• In Neuroscience, refers to physically collected neurons in our brains.
– Is a network because the function f(x) executed by a node is a composition of other functions, which are in turn defined as compositions of other function.
![Page 3: Neural Networks](https://reader033.vdocuments.mx/reader033/viewer/2022061112/5457be1aaf795946138b6a37/html5/thumbnails/3.jpg)
What is Machine Learning?
– Learning – Given a task to solve and a class of functions F, learning means using a set of observations to find an optimal solution that is an element of F
– Requires a cost function to determine how close we are to the optimal solution.
• Learning Paradigms– Supervised learning – Unsupervised learning – Reinforcement learning
– Training employs many cutting edge mathematical theories
– The NN has a learning algorithm, which you train with thousands of examples.
![Page 4: Neural Networks](https://reader033.vdocuments.mx/reader033/viewer/2022061112/5457be1aaf795946138b6a37/html5/thumbnails/4.jpg)
Relations to A.I.
– Marvin Minsky, one of the founding fathers of A.I., built first neural network learning machine and wrote Perceptrons, foundational work of artificial neural networks.
- Neural network is one of the main methods for developing computational intelligence. They often have very strong pattern
recognition capabilities.
![Page 5: Neural Networks](https://reader033.vdocuments.mx/reader033/viewer/2022061112/5457be1aaf795946138b6a37/html5/thumbnails/5.jpg)
What is currently being done?
• IBM is funding a four-year program called “systems neurocomputing” – Developing neural networks to recognize patterns and
avoid the “superposition catastrophe”. Is now using this research to recreate a person’s ability to perceive a broken line.
• Aston Martin, Daimler Chrysler, and other car companies are developing ANN models to detect cylinder misfires in engines.
• Georgia Tech introduced a neural network that combines living and robotic elements. – uses neural networks of cultured rodent brain cells and
robotic body• Recent advances in VLSI circuits, optical computing,
fuzzy logic, and protein-based computing have moved the field closer to realizing massively parallel hardware.
![Page 6: Neural Networks](https://reader033.vdocuments.mx/reader033/viewer/2022061112/5457be1aaf795946138b6a37/html5/thumbnails/6.jpg)
Learning
• Associative mapping – network learns to produce a particular pattern on the set of input units whenever another particular pattern is applied on the set of input units.
• Regularity detection - units learn to respond to particular properties of the input patterns.
![Page 7: Neural Networks](https://reader033.vdocuments.mx/reader033/viewer/2022061112/5457be1aaf795946138b6a37/html5/thumbnails/7.jpg)
How do they work?
• The neuron– Model biological
neurons– many inputs, one output– have weights, a bias,
and a threshold (activation) function
• The network architecture– Three interconnected
layers– Input layer partitions– processing (hidden)
layer analyzes– output layer … outputs– programmer uses
previous knowledge to ease training
![Page 8: Neural Networks](https://reader033.vdocuments.mx/reader033/viewer/2022061112/5457be1aaf795946138b6a37/html5/thumbnails/8.jpg)
How do they work?
• Tissues– networks like neural tissue– output may be input to another network– hidden layer may consist of a number of
such tissues
• Training– weight adjustments– recognizing key part of input– hard to see what the network “learned”
![Page 9: Neural Networks](https://reader033.vdocuments.mx/reader033/viewer/2022061112/5457be1aaf795946138b6a37/html5/thumbnails/9.jpg)
How do they work?
• Firing rule – determines how one calculates whether a neuron should fire for an input – Ex: take a collection of data, some
which causes firing and some which don’t. If new data is inputted, elements most in common with firing data will then cause firing.
![Page 10: Neural Networks](https://reader033.vdocuments.mx/reader033/viewer/2022061112/5457be1aaf795946138b6a37/html5/thumbnails/10.jpg)
How do they work?
• Feed forward architecture– Signals traveling one
way, from input to output (associates input with output)
• Feed backward architecture- Signals can travel both ways with loops. The state continually changes until equilibrium is reached
![Page 11: Neural Networks](https://reader033.vdocuments.mx/reader033/viewer/2022061112/5457be1aaf795946138b6a37/html5/thumbnails/11.jpg)
Successes
Strengths of neural networks:
• Pattern recognition
• Unclear algorithm
• No existing algorithm
• Large amounts of test data
![Page 12: Neural Networks](https://reader033.vdocuments.mx/reader033/viewer/2022061112/5457be1aaf795946138b6a37/html5/thumbnails/12.jpg)
Successes
• 20q– Based on a word game– Learns from users– Correct 80% of the time
• Image recognition– Recognizing objects– Categorizing images– Rendering images searchable
![Page 13: Neural Networks](https://reader033.vdocuments.mx/reader033/viewer/2022061112/5457be1aaf795946138b6a37/html5/thumbnails/13.jpg)
Successes
• Signature analysis– First large-scale use in US– Compares with stored signatures– 97% accuracy over old 83%– old four-way classification more difficult
• Face recognition– seeking to distinguish people– takes 100-200 of training pictures per person– average recognition rate of over 95%– more training does not guarantee better recognition
![Page 14: Neural Networks](https://reader033.vdocuments.mx/reader033/viewer/2022061112/5457be1aaf795946138b6a37/html5/thumbnails/14.jpg)
Current Implementation
• Instant physician– Developed in 1980s, trained a NN to
store a large amount of medical records. After training, could be presented with symptoms, and could then present the best diagnosis
![Page 15: Neural Networks](https://reader033.vdocuments.mx/reader033/viewer/2022061112/5457be1aaf795946138b6a37/html5/thumbnails/15.jpg)
Current implementations
• Business– Marketing control of seat allocation on an
airplane using feed-forward mechanism– Credit models and mortgage screening
boosted profitability of HNC by 27%
• Medicine– NN used to model cardiovascular system.
Build a NN of a patient and compare to actual patient. Can detect medical conditions before they happen.
![Page 16: Neural Networks](https://reader033.vdocuments.mx/reader033/viewer/2022061112/5457be1aaf795946138b6a37/html5/thumbnails/16.jpg)
Current implementations
• User interfaces– Handwriting analysis tools, text-to-
speech conversion (IBM, Babel)
• Industrial processes– control machinery, adjust
temperature settings, and diagnose malfunctions in robotic factories (Alyuda Research Factory)
![Page 17: Neural Networks](https://reader033.vdocuments.mx/reader033/viewer/2022061112/5457be1aaf795946138b6a37/html5/thumbnails/17.jpg)
Problems Encountered
• Applications using neural networks have little or no data available for training on fault conditions, so fuzzy logic is used, based on an expert’s definition of certain rules.
• “Neural network programs sometimes become unstable when applied to larger problems.” – the larger the problem, the more neural
networks must draw information from to obtain a solution, making neural networks very problem specific.
![Page 18: Neural Networks](https://reader033.vdocuments.mx/reader033/viewer/2022061112/5457be1aaf795946138b6a37/html5/thumbnails/18.jpg)
Problems Encountered
• Larger datasets require more extensive training time to reach a predictive solution, and there is the possibility of overtraining, in which there was low training error but high actual testing error. Unknown data necessary for the solution will also cause a high error rate, sometimes by affecting the weighted values used in determining a solution.
![Page 19: Neural Networks](https://reader033.vdocuments.mx/reader033/viewer/2022061112/5457be1aaf795946138b6a37/html5/thumbnails/19.jpg)
Future
• Simple systems which have learned to recognize simple entities (e.g. walls looming, or simple commands like Go, or Stop) may have neural network chips implanted to help in decision-making. Japanese are already using fuzzy logic for this purpose.
• Use of neural networks to put labels on what is determined to be in the pictures, for use in medical searches
• User-specific systems for education and entertainment based on readings taken of the user.
![Page 20: Neural Networks](https://reader033.vdocuments.mx/reader033/viewer/2022061112/5457be1aaf795946138b6a37/html5/thumbnails/20.jpg)
Future
• Development of integration of man and machine, such as with retinal and cochlear implants
• Generally the development of use of neural networks in more everyday and diverse applications, such as in retail and manufacturing, to help make accurate decisions.
![Page 21: Neural Networks](https://reader033.vdocuments.mx/reader033/viewer/2022061112/5457be1aaf795946138b6a37/html5/thumbnails/21.jpg)
Questions?