Introduction

Mohammad BeigiDepartment of Biomedical Engineering

Isfahan UniversityMajid.beigi@eng.ui.ac.ir

Pattern recognition and Machine Learning

Syllabus Introduction,Linear Models for classificationNeural Networks (MLP, RBF, SOM, LVQ, ADALINE) Kernel Methods & Support Vector Machines Statistical Pattern Recognition ? (HMM,EM, Clustering and unsupervised learning ? Feature Selection and Dimension reduction ?

Pattern recognition and Machine Learning

TextsR. O. Duda, P. E. Hart, D. G. Stork,

Pattern Classification, 2nd edition, John Wiley & Sons, Inc., 2000.

M. Bishop, Pattern Recognition and Machine Learning, Springer, 2006.

• Midterm 25%• Final 40%• Computer assignments 10%• Final Programming Project 15%• Seminar 10%

Evaluation

Human Perception

Humans have developed highly sophisticated skills for sensing their environment and taking actions according to what they observe, e.g

Understanding spoken wordsreading handwritingdistinguishing fresh food from its smell

We would like to give similar capabilities to machines

What is Pattern Recognition? A pattern is an entity, vaguely defined, that could be given a name, e.g.,

fingerprint image, handwritten word, human face, speech signal, DNA sequence,

Pattern recognition is the study of how machines can observe the environment, learn to distinguish patterns of interest, make sound and reasonable decisions about the categories of the

patterns.

Human and Machine Perception

We are often influenced by the knowledge of how patterns are modeled and recognized in nature when we develop pattern recognition algorithms. Research on machine perception also helps us gain deeper understanding and appreciation for pattern recognition systems in nature. Yet, we also apply many techniques that are purelynumerical and do not have any correspondence in naturalsystems.

Pattern Recognition Applications

Pattern Recognition Applications

Pattern Recognition Applications

Pattern Recognition Applications

Pattern Recognition Applications

Pattern Recognition Applications

Pattern Recognition Applications

Pattern Recognition Applications

Pattern Recognition Applications

Figure 9: Clustering of Microarray Data

Pattern Recognition Applications

Figure 10: Brain Control Interface

Regression: Polynomial Curve Fitting

t is continuous

Sum-of-Squares Error Function

* min ( )w Arg E w Optimization Problem

0th Order Polynomial

1st Order Polynomial

3rd Order Polynomial

9th Order Polynomial

Over-fitting

Root-Mean-Square (RMS) Error:

Polynomial Coefficients

Data Set Size: 9th Order Polynomial

Data Set Size: 9th Order Polynomial

Regularization ;ridge regression

Penalize large coefficient values

Shrinkage: reduce the order of method

~* min ( )w Arg E w

Regularization:

Regularization:

Regularization: vs.

Polynomial Coefficients

Optimization Problem: Finding optimum ,M

Classification example: Handwritten Digit Recognition

28*28 Pixel image : 784 real numbers, training set: 1{ ,.... }Nx x1x

( ),y x t {1,..,9}t

Pattern recognition approaches

Statistical Pattern recognition

Statistical Pattern recognition

Structural Pattern Recognition

Neural Pattern Recognition