pattern recognition
DESCRIPTION
TRANSCRIPT
![Page 1: Pattern recognition](https://reader033.vdocuments.mx/reader033/viewer/2022050815/547853a8b4af9f8f4f8b4607/html5/thumbnails/1.jpg)
CPR 2007-2008 1
Armando Vieira&
Bernardete Ribeiro
2007
/200
8
Artificial Intelligence&
Pattern Recognition
Introduction
![Page 2: Pattern recognition](https://reader033.vdocuments.mx/reader033/viewer/2022050815/547853a8b4af9f8f4f8b4607/html5/thumbnails/2.jpg)
CPR 2007-2008 2
Program • Introduction to Artificial Intelligence• Soft Artificial Intelligence• Artificial Neural Networks: theory, training,
applications• Supervised Learning: Mulilayer Perceptron• Unsupervised Learning: Self-Organized Kohonen
Maps• Genetic Algorithms• Applications• Project
![Page 3: Pattern recognition](https://reader033.vdocuments.mx/reader033/viewer/2022050815/547853a8b4af9f8f4f8b4607/html5/thumbnails/3.jpg)
CPR 2007-2008 3
Pattern Recognition?
• A pattern is an object, process or event
• A class (or category) is a set of patterns that share common attribute (features) usually from the same information source
• During recognition (or classification) classes are assigned to the objects.
• A classifier is a machine that performs such task
“The assignment of a physical object or event to one of several pre-specified categories” -- Duda & Hart
![Page 4: Pattern recognition](https://reader033.vdocuments.mx/reader033/viewer/2022050815/547853a8b4af9f8f4f8b4607/html5/thumbnails/4.jpg)
CPR 2007-2008 4
What is a pattern?What is a pattern?“A pattern is the opposite of a chaos; it is an entity vaguely
defined, that could be given a name.”
![Page 5: Pattern recognition](https://reader033.vdocuments.mx/reader033/viewer/2022050815/547853a8b4af9f8f4f8b4607/html5/thumbnails/5.jpg)
CPR 2007-2008 5
Examples of PatternsCristal Patterns: atómic or molecular
Their structures are represented by 3D graphs and can be described by deterministic grammars or formal languages
![Page 6: Pattern recognition](https://reader033.vdocuments.mx/reader033/viewer/2022050815/547853a8b4af9f8f4f8b4607/html5/thumbnails/6.jpg)
CPR 2007-2008 6
Patterns of Constellations
Patterns of constellations are represented by 2D planar graphs
Human perception has strong tendency to find patterns from anything. We see patterns from even random noise --- we are more likely to believe a hidden pattern than denying it when the risk (reward) for missing (discovering) a pattern is often high.
Examples of Patterns
![Page 7: Pattern recognition](https://reader033.vdocuments.mx/reader033/viewer/2022050815/547853a8b4af9f8f4f8b4607/html5/thumbnails/7.jpg)
CPR 2007-2008 7
Biological Patterns ---morphology
Landmarks are identified from biologic forms and these patterns are then represented by a list of points. But for other forms, like the root of plants,Points cannot be registered crossing instances.
Applications: Biometrics, computacional anatomy, brain mapping, …
Examples of Patterns
![Page 8: Pattern recognition](https://reader033.vdocuments.mx/reader033/viewer/2022050815/547853a8b4af9f8f4f8b4607/html5/thumbnails/8.jpg)
CPR 2007-2008 8
Biological Patterns
Landmarks are identified from biologic forms and these patterns are then represented by a list of points.
Examples of Patterns
![Page 9: Pattern recognition](https://reader033.vdocuments.mx/reader033/viewer/2022050815/547853a8b4af9f8f4f8b4607/html5/thumbnails/9.jpg)
CPR 2007-2008 9
Music Patterns
Ravel Symphony?
Examples of Patterns
![Page 10: Pattern recognition](https://reader033.vdocuments.mx/reader033/viewer/2022050815/547853a8b4af9f8f4f8b4607/html5/thumbnails/10.jpg)
CPR 2007-2008 10
People Recognition
Pat
tern
s B
ehav
ior
?
Funny, Funny
Examples of Patterns
![Page 11: Pattern recognition](https://reader033.vdocuments.mx/reader033/viewer/2022050815/547853a8b4af9f8f4f8b4607/html5/thumbnails/11.jpg)
CPR 2007-2008 11
Discovery and Association of Patterns
Statistics show connections between the shape of one’s face (adults) and his/her Character. There is also evidence that the outline of children’s face is related to alcohol abuse during pregnancy.
Examples of Patterns
![Page 12: Pattern recognition](https://reader033.vdocuments.mx/reader033/viewer/2022050815/547853a8b4af9f8f4f8b4607/html5/thumbnails/12.jpg)
CPR 2007-2008 12
What are the features?Statistics show connections between the shape of one’s face (adults) and his/her Character.
Examples of PatternsDiscovery and Association of Patterns
![Page 13: Pattern recognition](https://reader033.vdocuments.mx/reader033/viewer/2022050815/547853a8b4af9f8f4f8b4607/html5/thumbnails/13.jpg)
CPR 2007-2008 13
We may understand patterns of brain activity and find relationships between brain activities, cognition, and behaviors
Patterns of Brain Activity
Examples of Patterns
![Page 14: Pattern recognition](https://reader033.vdocuments.mx/reader033/viewer/2022050815/547853a8b4af9f8f4f8b4607/html5/thumbnails/14.jpg)
CPR 2007-2008 14
Variation Patterns: 1. Expression – geometric deformation 2. illumination--- Photometric deformation 3. Transformation –3D pose 3D 4. Noise and Occlusion
Examples of Patterns
![Page 15: Pattern recognition](https://reader033.vdocuments.mx/reader033/viewer/2022050815/547853a8b4af9f8f4f8b4607/html5/thumbnails/15.jpg)
CPR 2007-2008 15
A broad range of texture patterns are generated by stochastic processes.
Examples of Patterns
![Page 16: Pattern recognition](https://reader033.vdocuments.mx/reader033/viewer/2022050815/547853a8b4af9f8f4f8b4607/html5/thumbnails/16.jpg)
CPR 2007-2008 16
. How are these patterns represented in human mind?
Examples of Patterns
![Page 17: Pattern recognition](https://reader033.vdocuments.mx/reader033/viewer/2022050815/547853a8b4af9f8f4f8b4607/html5/thumbnails/17.jpg)
CPR 2007-2008 17
Speech signals and Hidden Markov models
Examples of Patterns
![Page 18: Pattern recognition](https://reader033.vdocuments.mx/reader033/viewer/2022050815/547853a8b4af9f8f4f8b4607/html5/thumbnails/18.jpg)
CPR 2007-2008 18
Natural Language and stochastic grammar..
Examples of Patterns
![Page 19: Pattern recognition](https://reader033.vdocuments.mx/reader033/viewer/2022050815/547853a8b4af9f8f4f8b4607/html5/thumbnails/19.jpg)
CPR 2007-2008 19
Object Recognition
Pat
tern
s ev
eryw
her
e ?
Examples of Patterns
![Page 20: Pattern recognition](https://reader033.vdocuments.mx/reader033/viewer/2022050815/547853a8b4af9f8f4f8b4607/html5/thumbnails/20.jpg)
CPR 2007-2008 20
Maps Recognition
Pat
tern
s o
f G
lob
al W
arm
ing
?
Examples of Patterns
![Page 21: Pattern recognition](https://reader033.vdocuments.mx/reader033/viewer/2022050815/547853a8b4af9f8f4f8b4607/html5/thumbnails/21.jpg)
CPR 2007-2008 21
Finacial Series Pattern Recognition
Examples of Patterns
![Page 22: Pattern recognition](https://reader033.vdocuments.mx/reader033/viewer/2022050815/547853a8b4af9f8f4f8b4607/html5/thumbnails/22.jpg)
CPR 2007-2008 22
Ho
w t
o T
rad
e C
har
t P
atte
rns
?Examples of Patterns
![Page 23: Pattern recognition](https://reader033.vdocuments.mx/reader033/viewer/2022050815/547853a8b4af9f8f4f8b4607/html5/thumbnails/23.jpg)
CPR 2007-2008 23
Pattern Recognition in Medical Diagnosis
TomografiaTomografia
projecçã
oproj
ecção
retroproje
cção
retroproje
cção
ReconstruçãoReconstrução
Tomos (=corte) +grafos (=escrita, imagem, gráfico)
f(x,y,z)f(x,y,z)
projecçõesprojecçõesp(r,p(r,,z),z)
p(r,p(r,,z),z)f(x,y,z)f(x,y,z)
Examples of Patterns
![Page 24: Pattern recognition](https://reader033.vdocuments.mx/reader033/viewer/2022050815/547853a8b4af9f8f4f8b4607/html5/thumbnails/24.jpg)
CPR 2007-2008 24
Optical Character Recognition
Examples of Patterns
![Page 25: Pattern recognition](https://reader033.vdocuments.mx/reader033/viewer/2022050815/547853a8b4af9f8f4f8b4607/html5/thumbnails/25.jpg)
CPR 2007-2008 25
Gra
ph
ic A
rts
Escher, who else?
Examples of Patterns
![Page 26: Pattern recognition](https://reader033.vdocuments.mx/reader033/viewer/2022050815/547853a8b4af9f8f4f8b4607/html5/thumbnails/26.jpg)
CPR 2007-2008 26
Human Genome
Bea
uti
ful
Pat
tern
s!
Examples of Patterns
![Page 27: Pattern recognition](https://reader033.vdocuments.mx/reader033/viewer/2022050815/547853a8b4af9f8f4f8b4607/html5/thumbnails/27.jpg)
CPR 2007-2008 27
• Optical Character
Recognition (OCR)
• Biometrics
• Diagnostic systems
• Military applications
• Handwritten: sorting letters by postal code, input device for PDA‘s.
• Printed texts: reading machines for blind people, digitalization of text documents.
• Face recognition, verification, retrieval. • Finger prints recognition.• Speech recognition.
• Medical diagnosis: X-Ray, EKG analysis.• Machine diagnostics, waster detection.
• Automated Target Recognition (ATR).
• Image segmentation and analysis (recognition from aerial or satelite photographs).
Examples of Applications
![Page 28: Pattern recognition](https://reader033.vdocuments.mx/reader033/viewer/2022050815/547853a8b4af9f8f4f8b4607/html5/thumbnails/28.jpg)
CPR 2007-2008 28
Approaches
• Statistical PR: based on underlying statistical model of patterns and pattern classes.
• Neural networks: classifier is represented as a network of cells modeling neurons of the human brain (connectionist approach).
• Structural (or syntactic) PR: pattern classes represented by means of formal structures as grammars, automata, strings, etc.
![Page 29: Pattern recognition](https://reader033.vdocuments.mx/reader033/viewer/2022050815/547853a8b4af9f8f4f8b4607/html5/thumbnails/29.jpg)
CPR 2007-2008 29
An example of Pattern Recognition Classification of fish into two classes: salmon and Sea Bass by discriminative method
•“Sorting incoming Fish on a conveyor according to species using optical sensing”
![Page 30: Pattern recognition](https://reader033.vdocuments.mx/reader033/viewer/2022050815/547853a8b4af9f8f4f8b4607/html5/thumbnails/30.jpg)
CPR 2007-2008 30
Problem Analysis
– Set up a camera and take some sample images to extract features
• Length• Lightness• Width• Number and shape of fins• Position of the mouth, etc…
This is the set of all suggested features to explore for use in our classifier!
![Page 31: Pattern recognition](https://reader033.vdocuments.mx/reader033/viewer/2022050815/547853a8b4af9f8f4f8b4607/html5/thumbnails/31.jpg)
CPR 2007-2008 31
Pattern Recognition Phases
• Preprocess raw data from camera
• Segment isolated fish
• Extract features from each fish (length,width, brightness, etc.)
• Classify each fish
![Page 32: Pattern recognition](https://reader033.vdocuments.mx/reader033/viewer/2022050815/547853a8b4af9f8f4f8b4607/html5/thumbnails/32.jpg)
CPR 2007-2008 32
Pattern Recognition Phases
• Preprocessing– Use a segmentation operation to isolate
fishes from one another and from the background
• Information from a single fish is sent to a feature extractor whose purpose is to reduce the data by measuring certain features
• The features are passed to a classifier
![Page 33: Pattern recognition](https://reader033.vdocuments.mx/reader033/viewer/2022050815/547853a8b4af9f8f4f8b4607/html5/thumbnails/33.jpg)
CPR 2007-2008 33
• Classification
Select the length of the fish as a possible feature for discrimination
![Page 34: Pattern recognition](https://reader033.vdocuments.mx/reader033/viewer/2022050815/547853a8b4af9f8f4f8b4607/html5/thumbnails/34.jpg)
CPR 2007-2008 34
Features and Distributions
The length is a poor feature alone!
Select the lightness as a possible feature.
![Page 35: Pattern recognition](https://reader033.vdocuments.mx/reader033/viewer/2022050815/547853a8b4af9f8f4f8b4607/html5/thumbnails/35.jpg)
CPR 2007-2008 35
“Customers do not want sea bass in their cans of salmon”
• Threshold decision boundary and cost relationship
• Move our decision boundary toward smaller values of lightness in order to minimize the cost (reduce the number of sea bass that are classified salmon!)
Task of decision theory
![Page 36: Pattern recognition](https://reader033.vdocuments.mx/reader033/viewer/2022050815/547853a8b4af9f8f4f8b4607/html5/thumbnails/36.jpg)
CPR 2007-2008 36
• Adopt the lightness and add the width of the fish
Fish x = [x1, x2]
Lightness Width
![Page 37: Pattern recognition](https://reader033.vdocuments.mx/reader033/viewer/2022050815/547853a8b4af9f8f4f8b4607/html5/thumbnails/37.jpg)
CPR 2007-2008 37
Decision/classification Boundaries ?
![Page 38: Pattern recognition](https://reader033.vdocuments.mx/reader033/viewer/2022050815/547853a8b4af9f8f4f8b4607/html5/thumbnails/38.jpg)
CPR 2007-2008 38
![Page 39: Pattern recognition](https://reader033.vdocuments.mx/reader033/viewer/2022050815/547853a8b4af9f8f4f8b4607/html5/thumbnails/39.jpg)
CPR 2007-2008 39
• We might add other features that are not correlated with the ones we already have. A precaution should be taken not to reduce the performance by adding such “noisy features”
• Ideally, the best decision boundary should be the one which provides an optimal performance such as in the following figure:
![Page 40: Pattern recognition](https://reader033.vdocuments.mx/reader033/viewer/2022050815/547853a8b4af9f8f4f8b4607/html5/thumbnails/40.jpg)
CPR 2007-2008 40
![Page 41: Pattern recognition](https://reader033.vdocuments.mx/reader033/viewer/2022050815/547853a8b4af9f8f4f8b4607/html5/thumbnails/41.jpg)
CPR 2007-2008 41
• However, our satisfaction is premature because the central aim of designing a classifier is to correctly classify novel input
Issue of generalization!
![Page 42: Pattern recognition](https://reader033.vdocuments.mx/reader033/viewer/2022050815/547853a8b4af9f8f4f8b4607/html5/thumbnails/42.jpg)
CPR 2007-2008 42
![Page 43: Pattern recognition](https://reader033.vdocuments.mx/reader033/viewer/2022050815/547853a8b4af9f8f4f8b4607/html5/thumbnails/43.jpg)
CPR 2007-2008 43
Occam’s Razor
Entities are not to be multiplied without necessity
William of Occam (1284-1347)
![Page 44: Pattern recognition](https://reader033.vdocuments.mx/reader033/viewer/2022050815/547853a8b4af9f8f4f8b4607/html5/thumbnails/44.jpg)
CPR 2007-2008 44
A Complete PR System
![Page 45: Pattern recognition](https://reader033.vdocuments.mx/reader033/viewer/2022050815/547853a8b4af9f8f4f8b4607/html5/thumbnails/45.jpg)
CPR 2007-2008 45
Problem Formulation
Measurements Preprocessing
ClassificationFeaturesInputobject
ClassLabel
Basic ingredients:•Measurement space (e.g., image intensity, pressure)•Features (e.g., corners, spectral energy)•Classifier - soft and hard•Decision boundary•Training sample•Probability of error
![Page 46: Pattern recognition](https://reader033.vdocuments.mx/reader033/viewer/2022050815/547853a8b4af9f8f4f8b4607/html5/thumbnails/46.jpg)
CPR 2007-2008 46
Design Cycle
1. Feature selection and extraction --- What are good discriminative features?2. Modeling and learning 3. Dimension reduction, model complexity4. Decisions and risks5. Error analysis and validation.6. Performance bounds and capacity.7. Algorithms
![Page 47: Pattern recognition](https://reader033.vdocuments.mx/reader033/viewer/2022050815/547853a8b4af9f8f4f8b4607/html5/thumbnails/47.jpg)
CPR 2007-2008 47
![Page 48: Pattern recognition](https://reader033.vdocuments.mx/reader033/viewer/2022050815/547853a8b4af9f8f4f8b4607/html5/thumbnails/48.jpg)
CPR 2007-2008 48
• Data Collection
How do we know when we have collected an adequately large and representative set of examples for training and testing the system?
![Page 49: Pattern recognition](https://reader033.vdocuments.mx/reader033/viewer/2022050815/547853a8b4af9f8f4f8b4607/html5/thumbnails/49.jpg)
CPR 2007-2008 49
• Feature Choice
Depends on the characteristics of the problem domain. Simple to extract, invariant to irrelevant transformation, insensitive to noise.
![Page 50: Pattern recognition](https://reader033.vdocuments.mx/reader033/viewer/2022050815/547853a8b4af9f8f4f8b4607/html5/thumbnails/50.jpg)
CPR 2007-2008 50
• Model Choice
Unsatisfied with the performance of our linear fish classifier and want to jump to another class of model
![Page 51: Pattern recognition](https://reader033.vdocuments.mx/reader033/viewer/2022050815/547853a8b4af9f8f4f8b4607/html5/thumbnails/51.jpg)
CPR 2007-2008 51
• Training
Use data to determine the classifier. Many different procedures for training classifiers and choosing models
![Page 52: Pattern recognition](https://reader033.vdocuments.mx/reader033/viewer/2022050815/547853a8b4af9f8f4f8b4607/html5/thumbnails/52.jpg)
CPR 2007-2008 52
• Evaluation
Measure the error rate (or performance) and switch from one set of features & models to another one.
![Page 53: Pattern recognition](https://reader033.vdocuments.mx/reader033/viewer/2022050815/547853a8b4af9f8f4f8b4607/html5/thumbnails/53.jpg)
CPR 2007-2008 53
• Computational Complexity
What is the trade off between computational ease and performance?
(How an algorithm scales as a function of the number of features, number or training examples, number patterns or categories?)