handwritten identification technique based on wavelet transform science/ms.c/… · ·...

Handwritten Identification

Technique Based on Wavelet Transform

A Thesis Submitted to the College of Science

University of Baghdad In Partial Fulfillment of the Requirements

for the Degree of Master of Science in Computer

By Ali Hadi Hussein

Supervised By Prof. Dr. Saleh M. Ali

2006 A.D. 1427H.

I would like to express my deep gratitude to my supervisor

Professor Dr. Saleh Mahdi Ali for his following-up, guidance and

care throughout all the stages of preparing this thesis.

I would like to give my sincere thanks to the head of Computer

department Assistant Professor Msc. Makiya K. Hamed.

Special thanks goes to staff members of Remote Sensing Unit.

I would like to dedicate my work to my father, mother , and wife

as a gift for their efforts and supports all along my way up to this

point.

Also, I give my thanks to the people who supplied me with the

different references directly or indirectly. Many thanks for my

colleagues and friends, Who gave me there advises or helps at

different stages of my work. Many thanks to the people who assisted

me in the finalization of this thesis.

Ali Al-Niemi

Acknowledgment

In this research, a system that has the ability to recognize and

identify persons from his handwriting is designed. Wavelet transform algorithm was utilized to analyze the individual characters of certain person's writing. Complex moments is used as tool for the differentiation between different persons.

This involves process that requires isolating the individual

characters from the input images, because adopted writing involved series of characters; e.g. capital English letters "A through Z".

Only the 1st order decomposition level of the wavelet transform is

utilized. This utilization is adopted to perform the recognition and identification process quickly and adequately.

The test was implemented on 40 samples of 10 persons

voluntaries. Some of the samples are used for verification tests. Minimum distance classifier (based on Euclidian distance) is used

to test the similarity or closeness between matched samples. Three handwriting versions, for each person, has been adopted and used as reference data reserved in the constructed database.

Complex moments of four decomposed sub images (i.e. LL, LH,

HL, HH) are computed and used as featured vectors to be compared in the recognition and identification processes.

The developed system utilized the power of the Visual Basic 6 for

performing the required operations involved in our designed system.

Abstract

List of Contents

Chapter One General Introduction

1-1 Introduction 1 1-2 Biometric Keys 2 1-2-1 Signature Recognition System 2 1-3 Features Generation and Selection 2 1-4 Literature Survey 3 1-5 Aim of the Thesis 4 1-6 Outline of the Thesis 5 Chapter Two

Wavelet Transforms 2-1 Introduction 6 2-2 Wavelet Transform 7 2-2-1 The Continuous Wavelet Transform 13 2-2-2 The Discrete Wavelet Transform 13 2-2-2-1 Fast Algorithm for the DWT 14 2-2-3 Harr Wavelet Transform 16 2-2-4 The Daubechies D4 Wavelet Transform 20 2-3 Haar vs. Daubechies D4 Transform 24 Chapter Three

Moments and Pattern Rcognition 3-1 Introduction 25 3-2 Invariant Moments Theory 26 3-2-1 Regular (Geometrical) Moment Invariants (RM) 27 3-2-1-1 Properties of Low Order Moments 28 3-2-2 Complex Moments (CM) 29 3-3 Pattern Recognition (PR) System 30 3-3-1 Designation of Pattern Recognition System 30 3-3-2 Designation Methodology 34 3-3-2-1 Heuristic Methods 34 3-3-2-2 Mathematical Methods 34 3-3-2-3 Linguistic (Syntactic) Methods 35 Chapter Four

Handwritten Identification System 4-1 Introduction 36 4-2 The Tested Samples 36 4-3 Training Phase 36 4-3-1 Handwritten Image Acquisition 37 4-3-2 Image Binarization Process 39

4-3-2-1 Threshold Value's effects on the Extraction process 39

4-3-3 Character's Isolation (Segmentation) process 40 4-3-3-1 Projection Technique 40 4-3-4 Feature Extraction 40 4-3-4-1 Discrete Wavelet Transform 40 4-3-4-2 Computation of the Complex Moments 42 4-3-4-3 Analyzing the Complex Moments Values 46 4-4 Recognition Phase 47 4-5 Matching and Euclidean Distance Measures 47 4-6 Experimental Results 49 4-7 Results 56 Chapter five

Conclusions and suggestions for future work

5-1 Conclusions 58 5-2 Suggestions for future work 59 References 60


1

Chapter One

General Introduction 1-1 Introduction

The need for improved information systems has become more conspicuous, since information is an essential element in decision making, and the world is generating increasing amounts of information in various forms with different degrees of complexity. One of the major problems in the design of modern information systems is automatic pattern recognition [1].

Advances in computer technology have stimulated new approaches to deal with different image processing subjects, ranging from processing of simple images to the more complicated aspects of image analysis and pattern recognition [2].

Recognition is regarded as a basic attribute of human beings, as well as other living organisms. A pattern is the description of an object. One of the first learning activities that the human brain is involved in is pattern recognition. In the course of their daily routine, human beings recognize and perform complex pattern recognition tasks routinely and effortlessly. However, this capability to recognize has proved to date to be extremely difficult to automate on a conventional computer. Human beings are constantly receiving data, processing it and taking decisions based on their interpretation of data. In the recent years of information explosion, there has arisen an increasing need for automatic data handling and interpretation. Automatic pattern recognition systems provide an engineering solution to data interpretation and decision making.

The field of pattern recognition overlaps with several other existing disciplines such as: artificial or machine intelligence, psychology, optimization and estimation theory, adaptive signal processing and so on.

Applications of pattern recognition techniques can be found in computer vision and robotics, target recognition of radar images, speech and speaker recognition, character and handwriting analysis, image processing and segmentation. In all these subjects, a priori information and training data (which are application specific) are used to develop a model of a pattern recognition system that is then used for the classification of new data.

The common objective of all pattern recognition systems is to categorize the observed data into known classes. The definition of a 'class ’ varies from one application problem to another. All patterns in a class are in some way similar to each other. For instance, in weather predication, the classes are sunny, cloudy, snow, etc [1,3].


2

1-2 Biometric Keys

Recent advantages in technology have made it possible for biometrics to be used in the user authorization process. Biometrics are the measurements of personal characteristics such as voice prints, finger prints, retinal prints, hand geometry, signature analysis [4], and handwritten identification.

In the literature, the handwritten identification has not found to be used as a biometric key, but there is a great similarity between signature recognition. We present work followed the signature recognition procedure to perform the handwritten identification technique. 1-2-1 Signature Recognition System

As the available computing power is eventually increasing and computer algorithms become smartest, tasks that a few years ago seamed completely unfeasible, now come again to focus. This partly explains why a considerable amount of research effort is being recently devoted in designing algorithms and techniques associated with the problems like human signature recognition and verification [5].

The design of any signature recognition system requires the solutions for four types of problems: data acquisition, preprocessing, feature extraction, and comparison process. Automatic signature recognition requires a representation of the handwritten signature that is suitable for computer processing. There are basically two approaches to obtain such a representation: Dynamic (on-line) approach and Static (off-line) approach. Dynamic systems use a digitizer or an electronic pen to generate signals; static systems produce an image of a signature with the help of a camera or a scanner. In the first approach, the signature is considered as one or several signals varying with time, u(t), gives a representation of the written signature. In the second approach, the signature written on paper appears as a 2-D image which can be picked up by optical means.

In processing step, the raw data are preprocessed to remove noise information, to filter the significant signals or images, and to validate the acquisition. The next step is referred to as the feature extraction process. Specific discriminating functions or parameters are computed from the filtered input data and are used to represent the signature. Prior to performing comparisons, a signature reference set must be generated for each user of the system, which is registered in the reference database during the enrollment process. At the comparison stage, the features of the tested signature just collected and then compared to all the references of the database. Finally a decision process evaluates the comparison criteria with respect to specific threshold and the signature is either accepted or rejected [6]. 1-3 Features Generation and Selection

Feature generation, is the transformation of the selected features (or low data) into a format that provides, for example, a higher class separation. Feature generation includes linear transforms, such as the discrete Fourier transform,


3

discrete wavelet transform, and others. An appropriately chosen transform can exploit and remove information redundancies, which usually exist in the set of samples obtained by the measuring device.

Human beings commonly use physical and structural features in the recognition of patterns, because these features are easily detected by touch or by the eye or other sensory organs. Color and fragrance are examples of physical features, shape, texture, and other geometrical properties of patterns are considered structural features.

When a machine is designed to recognize patterns, the effectiveness of adopted features in recognition process may be sharply reduced since the capabilities of human sensory organs are generally difficult to imitate in most practical situations. On the other hand, machines can be designed to extract mathematical features of patterns, which a human being may have some difficulty in determining without mechanical aid. In automatic pattern recognition, physical and structural features are used primarily in the area of image processing.

Like any pattern recognition problem, in signature recognition ( like handwritten identification) distinctive features can be extracted from a set of original signatures. These features can be in the form of functions of time or global parameters.

The problem of feature selection is related to the number of features at the disposal of the designer of a classification system is usually very large. This number can easily become of the order of dozens or hundreds. A subset of features out of universe of all the available features can be selected according to some criterion, such as statistical hypothesis testing or class separability, thus reducing the quantity of features to be used to a few [7]. 1-4 Literature Survey

To provide an overview of previous work and to provide a basic theoretical understanding of the considered subject, some papers presented by various authors are reviewed and quoted in this section.

Some authors have applied specific transform techniques to signals, and they use as features a set of coefficients derived from these mathematical transforms. As the first stage for a signature represented in the frequency domain via a Walsh transform, Zimmermann and Varady have considered the first forty low-frequency coefficients of the power spectra as features [8]. Similarly, Hale and Paganini have applied Haar transform to force patterns and use, among other parameters, the first fifteen coefficients as a feature vector [9].

Al-Emami et. al. [10] reported work on an on-line recognition of handwritten Arabic characters. Handwritten words were entered into an IBM PC via a graphic tablet and a segmentation process applied to the points; the length and the slope of each segment were then found. And the slope categorized to one of four directions. In the learning process, specification of the stroke of each character are fed to the computer. In the recognition process, the parameters of each stroke found special rule applied to select the collection of strokes, which


4

best matches, the features of one of the stored characters. The results were promising, and suggestions for improvements leading to 100٪ recognition as were proposed.

Murib [11] proposed a conventional method of image recognition, by using the wavelet transform to generate a resolution projection.. This projection combine the first order statistic to extract common features moments for distinct between the images, also he applied a direct method which is based on generating recursive family matrices.

Bajaj and Chaudhury (1997) [12] proposed a signature recognition and verification system based on using of multiple neural networks supplied by three sets of global features including projection moments .

Kadhim [13] proposed a signature recognition and verification system, in which he computed complex-moment's values (up to 10-th order ) for each signature' s image. To minimize the variation in the final results, he normalized all signatures in the position and size before extracting the complex-moment values. Three types of coordinates normalization were tested (centre of image, corner of image, center of mass). Many statistical parameters were tested and it was found that the mean and standard deviation offer better results in recognition system in comparison with the results associated with min, max median.

Dudani, Breedng and Mcghee [14] proposed an on time recognition system for the purpose of identification of three-dimensional objects. In their work they used central moments as features descriptors. Two distinct decision rules were used in the classification experiments : a Bayes decision rule and a distance-weighted k-nearest neighbor rule.

Ismail et. al. (2000) [15] developed recognition technique based on a multistage classifier and a combination of global and local features. New algorithms for signature verification based on fuzzy concepts have also described and tested.

Abd Alrazak, Raghad S. in (2004) [16] proposed a signature recognition using wavelet transform based on using discrete wavelet transform and complex moment. In this system the wavelet transform is used to decompose the signature images up to the 3-rd level of decomposition. Complex moments were computed from the wavelet coefficients to describe the signatures. 1-5 Aim of the Thesis

The objective of this work is to design an off-line handwritten identification system based on wavelet transform and invariant moment.

In this work, we have used many handwritten samples for a group of persons and after that compare a new handwritten pattern with the saved samples to decide if the new pattern belongs to someone in the group or not. Also, we decide whether a specific handwritten pattern belongs to the same person or not.

Discrete wavelet transform are used to decompose each handwritten image up to the 1-level. From each result sub images from wavelet decomposition


5

(LL1,LH1,HL1,HH1), four complex moments are calculated. All of the values of complex moment are selected to form the features vector for each handwritten image entered to the system.

The proposed system is trained by saving the nine complex moments for each character(four features for LH, HL and HH respectively). Minimum distance measures is used to classify the tested handwritten images. During this work, the effect of some related control parameters are also tested on the recognition accuracy of the proposed system. These parameters are investigate and treated carefully. 1-6 Outline of the Thesis In addition to chapter one, the organization of this thesis is as follows: Chapter two provides the theoretical issues that are later used in the design of the proposed identification scheme. A background information of the wavelet transform, explanation of the discrete wavelet transform, explanation of the two types of wavelet : haar and daubechies wavelet . Chapter three provides the theoretical issues that are later used in the design of the proposed identification scheme. Presents an introduction to the invariant moment and pattern recognition system, A background information of the invariant Moments Theory, A background information of the pattern recognition concepts, the stages of the typical pattern recognition system are presented. Chapter four presents the proposed handwritten identification system. It includes some details about the implementation issues, where some algorithms and flowcharts are presented, presents the test results of the proposed system. Chapter five presents some derived conclusions and a list of suggestions for the future work.

Chapter Two Wavelet Transforms

6


2-1 Introduction Transform is a procedure, equation or algorithm, which changes one group

of data into another group of data [17]. The transforms considered here provide information regarding the spatial domain frequency content of an image. In general, a transform maps image data into a different mathematical space; i.e. mapping the image data from spatial domain to the frequency domain, illustrated in the figure 2-1. Different types of signal's domains may be encountered; i.e.

· Time domain: The signal collected in time sequence; i.e. presented as f(t),

where f(t) referred to signal value at time t. · Frequency domain: The signal presented in term of number of times

repeated itself in given time period; presented as f(n), where n=1/T and T is the period. Usually, the transformed function (e.g. Fourier transform) is presented in this mode [18].

· Spatial domain: The signal is presented in term of distance or position. Therefore, the frequency of such representation called spatial frequency which indicates the number of times certain length of signal repeated it self at given distance.

Generally, transformations are used as tools in many areas of engineering and science, including computer imaging. Transformations may be presented either as continuous form or as discrete (sampled) forms. The discrete form of the transformation is created by sampling the continuous form of the functions on which these transforms are based; i.e. the basis functions [18].

· Basis functions: The set of waveforms that decomposition uses. For instance,

the basis functions for the Fourier decomposition are unity amplitude sine and cosine waves [17]. The one-dimensional (1-D) case provides us with the basic vector, while the two-dimensional (2-D) output is in matrix or image form.

Input Image Output Image

All Pixels

Transform Equation

Figure 2-1: All pixels in the input image contribute to each value in the output transformed image [18].


7

The general form of the transformation equation, assuming an N × N image

file, is given by:

åå-

=

-

=

=1

0

1

0),;,(),(),(

N

r

N

cvucrBcrIvuT (2.1)

Here u and v represent the spatial frequency variables, T(u, v) are the transform coefficients, and B(r, c; u, v) corresponds to the basis images. The notation B(r, c; u, v) defines a set of basis images, corresponding to each different value for u and v, and the size of each is r by c. the transform coefficients T(u,v) are the projections of I(r,c) onto each B(u,v).

To obtain the image from the transformed coefficients, the inverse

transform equation should be applied; i.e.

),;,(),()],([),(1

0

1

0

11 vucrBvuTvuTTcrIN

u

N

våå

-

=

-

=

-- == (2.2)

Here the T-1[T(u,v)] represents the inverse transform, and the B-1(r,r;u,v) represents the inverse basis images [18]. 2-2 Wavelet Transform

Wavelet theory involves representing general functions in terms of simpler, fixed building blocks at different scales and positions. This has been found to be a useful approach in several different areas; e.g. for signal and image forms. In the early 80's, Stromberg discovered the first orthogonal wavelets. In the early to mid 80's, Alex Grossmann and Jean Morlet suggested the word "wavelet", they studied the wavelet transform in its continuous form [19].

Wavelets are mathematical functions that cut up data into different frequency components, and then study each component with a resolution matched to its scale. They have advantages over traditional Fourier methods in analyzing physical situations, where the signal contains discontinuities and sharp spikes. Wavelets were developed independently in the fields of mathematics, quantum physics, electrical engineering, and seismic geology. Interchanges between these fields during the last years have led to many new wavelet applications such as image compression, turbulence, human vision, radar, and earthquake prediction. Wavelets are functions that satisfy certain mathematical requirements and are used in representing data or other functions. This idea is not new; approximation using superposition of functions has existed since the early 1800's, when Joseph Fourier discovered that he could superpose Sine and Cosine to represent other functions. However, in wavelet analysis, the scale that we use to look at data plays a special role. Wavelet algorithms process data at different scales or resolutions. If we look at a signal with a large "window", we would notice gross features. Similarly, if we look at a signal with a small "window", we would notice small features as shown in figure 2-2. A way to achieve this is to have short high-frequency fine scale functions and long low-


8

frequency ones. This approach is known as multi-resolution analysis. This makes wavelets interesting and useful. The wavelet analysis procedure is to adopt a wavelet prototype function, called an analyzing wavelet or mother wavelet. Temporal analysis is performed with a contracted, high-frequency version of the prototype wavelet, while frequency analysis is performed with a dilated, low-frequency version of the same wavelet. Because the original signal or function can be represented in terms of a wavelet expansion (using coefficients in a linear combination of the wavelet functions), data operations can be performed using just the corresponding wavelet coefficients. And if you further choose the best wavelets adapted to your data, or truncate the coefficients below a threshold, your data is sparsely represented. This sparse coding makes wavelets an excellent tool in the field of data compression [20]. The term wavelets refers to sets of function of the form

÷øö

çèæ -

=-

abtt a

abyy

21

,)(

(2.3) , i.e., sets of function formed by dilation and translation (a,b) of a single function

)(ty , called as the mother wavelet [21].

A family of wavelets is constructed by translations and dilations performed

on a single fixed function called the mother wavelet. A wavelet j j is derived from its mother wavelet j by

( ) ÷÷ø

öççè

æ -=

j

jj d

mxz ff (2.4)

Figure 2-2 : Wavelet Transform [20].


9

where its translation factor mj and its dilation factor dj are real numbers (dj > 0). We are concerned with modeling problems, i.e. with the fitting of a data set by a finite sum of wavelets [22].

The main idea is to select a mother wavelet, a function that is nonzero in some intervals, and use it to exploit the properties of f(t) in the interval. The importance of the WT as a multi-resolution analysis come from its decomposition of the image into multi-level of the independent information with changing the scale like a geographical map in which the image has non-redundant information due to the changing of scale. Using this fact, every image will be transform in each level of decomposition to a one low information sub image and three details sub image horizontal, vertical and diagonal axis sub image, also the low information sub Image can be decomposed into another four sub image. This approach of decomposition process provides a number of unrealizable features for the original image, which appear in these levels after applying transformation [23].

The wavelet transform break the image down into four sub sampled or decimated images. The result consists of one image that has been high pass filtered in both horizontal and vertical direction (HH), one that has been high pass filter in the vertical and low pass filter in horizontal (HL),one that has been low pass filter in vertical and high pass filter in horizontal (LH), and one that has been low pass filter in both direction [24].

Low pass filter attenuate or remove high-frequency components, but high pass filter attenuate or remove low-frequency components [25], see figure 2-3.

A high pass filter allows the high frequency components of a signal through while suppressing the low frequency components. For example, the differences that are captured by the Haar wavelet function represent high frequency change between an odd and an even values. A low pass filter suppresses the high frequency components of a signal and allows the low frequency components through. The Haar scaling function calculates the average of an even and an odd elements, which results in a smoother, low pass signal[26].

This wavelet transform can be repeated L times, where L is a parameter to form L level (or resolution) of subbands. Resolution L-1 is the original image (resolution 4 is shown in figure 2-4) and resolution 1 is the lowest-frequency subband [16].

Figure 2-3: Low pass and high pass filter [17]


10

According to above procedure, the original image can be transformed into

four sub-images, namely:

1- LL sub-image: both horizontal and vertical directions have low frequency. 2- LH sub-image: the horizontal direction has low frequency and the vertical one

has high frequency.. 3- HL sub-image: the horizontal direction has high frequency and the vertical

one has low frequency.. 4- HH sub-image: both horizontal and vertical directions have high

frequency[24]. · The Scale

The parameter scale in the wavelet analysis is similar to the scale used in maps. As in the case of maps, high scales correspond to a non-detailed global view (of the signal), and low scales correspond to a detailed view. Similarly, in terms of frequency, low frequencies (high scales) correspond to a global information of a signal (that usually spans the entire signal), whereas high frequencies (low scales) correspond to a detailed information of a hidden pattern in the signal (that usually lasts a relatively short time). Cosine signals corresponding to various scales are given as examples in the figure 2-5.

LH 1

HH 1 HL 1

LH 2

HL 2 HH 2

LH 3

HL 3 HH 3

Level 4

Level 2

Level 1

LL4

HL4

LH4

HH4

Level 3

Figure 2-4 : Four-Level decomposition image by wavelet transform.


11

Scaling, as a mathematical operation, either dilates or compresses a signal. Larger scales correspond to dilated (or stretched out) signals and small scales correspond to compressed signals. All of the signals given in the figure are derived from the same cosine signal, i.e., they are dilated or compressed versions of the same function. In the above figure, s=0.05 is the smallest scale, and s=1 is the largest scale.

In terms of mathematical functions, if f(t) is a given function f(st) corresponds to a contracted (compressed) version of f(t) if s > 1 and to an expanded (dilated) version of f(t) if s < 1 [27] .

The scaling function produces a smoother version of the data set, which is half the size of the input data set. Wavelet algorithms are recursive and the smoothed data becomes the input for the next step of the wavelet transform. The Haar wavelet scaling function is

21++

= iii

ssa (2.5)

where ai is a smoothed value [26]. · Analyzing Wavelet Functions

Fourier transforms deal with just two basis functions (sine and cosine), while there are an infinite number of wavelet basis functions. The freedom of the analyzing wavelet is a major difference between the two types of analyses and is important in determining the results of the analysis.

Figure 2-5 : Cosine signals corresponding to various scales [27]


12

The “wrong” wavelet may be no better (or even far worse than) than the Fourier analysis. A successful application presupposes some expertise on some users. Some prior knowledge about the signal must generally be known in order to select the most suitable distribution and adapt the parameters to the signal. Some of the more common ones are shown in figure 2-6. There are several wavelets in each family, and they may look different than those shown in figure 2-6. Somewhat longer in duration than these functions, but significantly shorter than infinite sinusoids is the cosine packet shown in figure 2-6 [20].

Figure 2-6 : Sample Wavelet Analysis [20]


13

2-2-1 The Continuous Wavelet Transform The wavelet analysis described in the introduction is known as the

continuous wavelet transform or CWT. More formally it is written as: dtttfs s )()(),( *

,ò= tytg (2.6) where * denotes complex conjugation. This equation shows how a function f(t) is decomposed into a set of basis functions , called the wavelets. The variables s and , scale and translation, are the new dimensions after the wavelet transform. For completeness sake (2.7) gives the inverse wavelet transform. I will not expand on this since we are not going to use it:

dsdtstf s tytg t )(),()( ,òò= (2.7) The wavelets are generated from a single basic wavelet (t), the so-called

mother wavelet, by scaling and translation:

÷øö

çèæ -

=s

ts

tstyy t

1)(, (2.8)

In (2.8) s is the scale factor, is the translation factor and the factor s-1/2 is for energy normalization across the different scales.

It is important to note that wavelet basis functions are not specified. This is a difference between the wavelet transform and the Fourier transform, or other transforms. The theory of wavelet transforms deals with the general properties of the wavelets and wavelet transforms only. It defines a framework within one can design wavelets to taste and wishes [28]. 2-2-2 The Discrete Wavelet Transform

In many practical applications the signal of interest is sampled. In order to use the results we have achieved so far with a discrete signal we have to make our wavelet transform discrete too. Remember that our discrete wavelets are not time-discrete, only the translation and the scale step are discrete. Simply implementing the wavelet filter bank as a digital filter bank intuitively seems to do the job [28]. The general form of an L-level DWT is written in terms of L detail sequences, dj(k ) for j=1,2,….,L , and the L−th level approximation sequence, cL(K) as follows:

)()()()()(1

tkctkdtx Lk Lj

L

jk j jy ååå +=

=

(2.9)

In jL (t) is the L−th level scaling function and ψj (t ) for j L =1,2,…,L are wavelet function sequences for L different levels.

In order to work directly with the wavelet transform coefficients, the

relationship between the detailed coefficients at a given level in terms of those at previous level is used. In general, the discrete signal is assumed the highest achievable approximation sequence, referred to as 0-th level scaling coefficients. The approximation and detail sequences at level j +1 are related to the approximation sequence at level j by :


14

)()2()( 01 mckmhkc jm

j -= å+

and (2.10) )()2()( 11 mckmhkd j

mj -= å+

As it obvious, the approximation sequence at higher scale (lower level index), along with the wavelet and scaling filters, ho(k) and h1(k) respectively, can be used to calculate the detail and approximation sequences (or discrete wavelet transform coefficients) at lower scales.

In practice, a discrete signal, at its original resolution is assumed the 0-th level approximation sequence; i.e., C0(k)=X(k ). For a given wavelet system, with known wavelet filters ho(k) and h1(k), it is possible to use equations shown in (2.10), in a recursive fashion, to calculate the discrete wavelet transform coefficients at all desired lower scales (higher lever). In most engineering applications, the wavelet systems are chosen such that the two wavelet filters have finite number of non-zero coefficients. In signal processing terminology, these filters are referred to as finite impulse response (FIR) filters. Under this assumption, and by using ideas from multirate signal processing literature, it is possible to calculate the two summations in (2.10) by using two FIR filters [29].

2-2-2-1 Fast Algorithm for the DWT

Calculating wavelet coefficients at every possible scale is a fair amount of work, it turns out, that if we choose scales and positions based on powers of two – so – called dyadic scales and positions – then our analysis will be more efficient and just such an analysis form the discrete wavelet transform (DWT).

An efficient way to implement this scheme using filters was developed in 1988 by Mallat. This is a practical filtering algorithm yields a fast wavelet transform [30].

For many signals, the low frequency content is the most important part. It is what gives the signal its identity. For example, consider the human voice; If you remove the high frequency components, the voice sounds different, but you can still tell what’s being said. While, in case if an enough low–frequency components are removed [12].

The decomposition process can be made by filtering the signal by LPF and HPF, then the signal is down-sampled by two [31]. The reconstruction process can be made by up-sampling f and d in equations (2.11), and filtering them with g(n) and h(n) [32].

)2( )()( )()1( nkhkfndk

jj -= å-

å -=-

k

jj nkgkfnf )2( )()( )()1( (2.11)

where f ( j ) = the signal. f ( j -1) = the approximation. d(j-1) = the details. g(n), h(n) = the LPF,HPF filters impulse responses, respectively.


15

The aim of the DWT is to decompose the discrete time signal into basis functions, called the wavelets, to give us a good analytic view of the analyzed signal. The decomposition process is divided into stages, called levels or depths. At each depth, different time and frequency resolution is taken (high frequency resolution means lower time resolution and vise versa). This variable resolution is done using building blocks, or wavelets, which are derived from an original wavelet, called the mother wavelet. The signal is decomposed using dilated and shifted versions of the mother wavelet. Figure 2-7 shows the wavelet decomposition and reconstruction tree [33].

Figure 2-7 : Wavelet decomposition and reconstruction tree [33]

(b) Reconstruction

g'(n)

h'(n)

g'(n)

h'(n)

g'(n)

h'(n)

2

2

2

2

2

2 2

(a) Decomposition

Signal x(n)

g(n)

h(n)

g(n)

h(n)

g(n)

h(n)

2

2

2

2

2

2


16

2-2-3 Harr Wavelet Transform The Haar wavelet algorithms are applied to time series where the number

of samples is a power of two (e.g., 2, 4, 8, 16, 32, 64...) The Haar wavelet uses a rectangular window to sample the time series. The first pass over the time series uses a window width of two. The window width is doubled at each step until the window encompasses the entire time series. Each pass over the time series generates a new time series and a set of coefficients. The new time series is the average of the previous time series over the sampling window. The coefficients represent the average change in the sample window. For example, if we have a time series consisting of the values v0, v1, ... vn, a new time series, with half as many points is calculated by averaging the points in the window. If it is the first pass over the time series, the window width will be two, so two points will be averaged: for (i = 0; i < n; i = i + 2) Si = (Vi + Vi+1)/2;

In lifting scheme terms the wavelet calculates the difference between a prediction and an actual value. If we have a data sample Si, Si+1, Si+2... the Haar wavelet equations is

21+-

= iii

ssc (2.12)

Where Ci is the wavelet coefficient. The wavelet Lifting Scheme uses a slightly different expression for the Haar wavelet :

iii evenoddc -= (2.13) The scaling function produces a smoother version of the data set, which is

half the size of the input data set. Wavelet algorithms are recursive and the smoothed data becomes the input for the next step of the wavelet transform. The Haar wavelet scaling function is

21++

= iii

ssa (2.14)

where ai is a smoothed value [26]. The low pass filter can be expressed

)1,1(2

1 (2.15)

In the Haar case and when we average the data, we move this filter along our input data. The differencing corresponds to high pass filtering. It removes low frequencies and responds to details of an image since details correspond to high frequencies. It also responds to noise in an image, since noise usually is located in the high frequencies. The high pass filter can be expressed as

)1,1(2

1- (2.16)

In the Haar case and when we differentiate the data, we simply move this filter along our input data. The low pass and high pass filters make up what in signal processing is referred to as a filter bank. The method of averaging and


17

differencing is referred to as analysis. The reverse procedure is called synthesis [34]. The wavelet coefficients are calculated along with the new average time series values. The coefficients represent the average change over the window. If the windows width is two this would be: for (i = 0; i < n; i = i + 2) Ci = (Vi - Vi+1)/2; The negative values mean that the time series is moving upward Vi is less than Vi+1, so Vi - Vi+1 will be less than zero. Positive values mean the the time series is going down, since Vi is greater than Vi+1 [26]. To calculate the Haar transform of an array of n samples:

1-Find the average of each pair of samples. (n/2 averages) 2-Find the difference between each average and the samples it was calculated

from. (n/2 differences) 3-Fill the first half of the array with averages. 4-Fill the second half of the array with differences. 5-Repeat the process on the first half of the array.

(The array length should be a power of two) Average / Difference Two samples, l and r, can be expressed as an average, a, and a difference, d, like in mid-side coding: a = (l + r) / 2 d = l- a = a - r This is reversible: l = a + d r = a - d Example Eight elements:

7 1 6 6 3 -5 4 2 Averages: (7 + 1) / 2 = 4 (6 + 6) / 2 = 6 (3 + -5) / 2 = -1 (4 + 2) / 2 = 3 Differences: (7 - 4) = ( 4 - 1) = 3 (6 - 6) = ( 6 - 6) = 0 (3 - -1) = (-1 - -5) = 4 (4 - 3) = ( 3 - 2) = 1

4 6 -1 3 3 0 4 1 Four elements:


18

4 6 -1 3 3 0 4 1 Averages: ( 4 + 6) / 2 = 5 (-1 + 3) / 2 = 1 Differences: ( 4 - 5) = (5 - 6) = -1 (-1 - 1) = (1 - 3) = -2

5 1 -1 -2 3 0 4 1 Two elements:

5 1 -1 -2 3 0 4 1 Averages: (5 + 1) / 2 = 3 Differences: (5 - 3) = (3 - 1) = 2

3 2 -1 -2 3 0 4 1

We can't recurse any further. Note that the first value in the resulting array is the average value of all the samples in the original array [35]: (7 + 1 + 6 + 6 + 3 - 5 + 4 + 2) / 8 = 3. Figure 2-8 shows A two level Haar wavelet transform [34].


19

The Haar transformation, previously described based on the calculation of

the averages and differences. Given two adjacent pixels a and b the principles is to calculate the averages s=(a+b)/2 and differences d=b - a. If a and b were similar, s will be similar to both and d will be small; i. e. require fewer bits to represent. This transform is reversible, since a=s - d / 2 and b=s + d / 2, and

1),(21

21

11),(),(),,(

121

121

),(),( -=úúû

ù

êêë

é

-==úúú

û

ù

êêê

ë

é -= Adsdsbababads

(2.17)

Figure 2-8 : A two level Haar wavelet transform [34]


20

The main idea in this operation is to begin with a pair of consecutive pixels a and b. they are replaced with their average s and difference d by first replacing a with d=b - a that replacing a with s=a+d/2 (since d=b-a, a+d/2=a+(b-a) equals (a+b)/2) [23]. 2-2-4 The Daubechies D4 Wavelet Transform

A particular set of wavelets is specified by a particular set of numbers, called wavelet filter coefficients [36]. The Daubechies wavelet transform is named after its inventor, the mathematician Ingrid Daubechies. The Daubechies D4 transform has four wavelet and scaling function coefficients. The scaling function coefficients are

2431

0+

=h

2433

1+

=h (2.18)

2433

2-

=h

2431

3-

=h

Each step of the wavelet transform applies the scaling function to the data input. If the original data set has N values, the scaling function will be applied in the wavelet transform step to calculate N/2 smoothed values. In the ordered wavelet transform the smoothed values are stored in the lower half of the N element input vector. The wavelet function coefficient values are:

g0 = h3, g1 = -h2, g2 = h1, and g3 = -h0 Each step of the wavelet transform applies the wavelet function to the input

data. If the original data set has N values, the wavelet function will be applied to calculate N/2 differences (reflecting change in the data). In the ordered wavelet transform the wavelet values are stored in the upper half of the N element input vector.

The scaling and wavelet functions are calculated by taking the inner product of the coefficients and four data values. The equations (2.19,2.20) are shown :

Each iteration in the wavelet transform step calculates a scaling function

value and a wavelet function value. The index i is incremented by two with each iteration, and new scaling and wavelet function values are calculated.

(2.19)

(2.20)


21

In the case of the forward transform, with a finite data set (as opposed to the mathematician's imaginary infinite data set), i will be incremented until it is equal to N-2. In the last iteration the inner product will be calculated from s[N-2], s[N-1], s[N] and s[N+1]. Since s[N] and s[N+1] don't exist (they are beyond the end of the array) ), this presents a problem.. This is shown in the transform matrix figure 2-9 .

Note that this problem does not exist for the Haar wavelet, since it is calculated on only two elements, s[i] and s[i+1]. Figure 2-10 is shown forward transform step of the lifting scheme version of the Daubechies D4.

The split step divides the input data into even elements which are stored in

the first half of an N element array section ( S0 to Shalf-1) and odd elements which are stored in the second half of an N element array section (Shalf to SN-1). In the forward transform equations below the expression S[half+n] references an odd element and S[n] references an even element. Although the figure 2-10 shows two normalization steps, in practice they are folded into a single function, as

Figure 2-9 : Daubechies D4 forward transform matrix for an 8 elements signal [37]

Figure 2-10 : Daubechies D4 forward wavelet transform [37]


22

mentioned in the following procedure which represent the forward transformation [37].

A similar problem exists in the case of the inverse transform. Here the

inverse transform coefficients extend beyond the beginning of the data, where the first two inverse values are calculated from s[-2], s[-1], s[0] and s[1]. This is shown in the inverse transform matrix figure 2-11.

The inverse transform works on N data elements, where the first N/2 elements are smoothed values and the second N/2 elements are wavelet function

Figure 2-11 : Daubechies D4 inverse transform matrix for an 8- element transform result [37]


23

values. The inner product that is calculated to reconstruct a signal value is calculated from two smoothed values and two wavelet values. Logically, the data from the end is wrapped around from the end to the start. In the comments the "coef. val" refers to a wavelet function value and a smooth value refers to a scaling function value. Figure 2-12 is shown inverse transform step of the lifting scheme version of the Daubechies D4

The merge step interleaves elements from the even and odd halves of the vector (e.g., even0, odd0, even1, odd1, ...) [37]. The inverse transform steps summarized in the following procedure [37].

Figure 2-12 : Daubechies D4 inverse wavelet transform [37]


24

2-3 Haar vs. Daubechies D4 Transform When I first started studying wavelets, one of the many questions I had was

"How does one decide which wavelet algorithm to use?" There is no absolute answer to this question. The choice of the wavelet algorithm depends on the application. The Haar wavelet algorithm has the advantage of being simple to compute and easier to understand. The Daubechies D4 algorithm has a slightly higher computational overhead and is conceptually more complex. As the matrix forms of the Daubechies D4 algorithm above show, there is overlap between iterations in the Daubechies D4 transform step. This overlap allows the Daubechies D4 algorithm to pick up detail that is missed by the Haar wavelet algorithm.

In the Haar transform, the coefficients show the average change between odd and even elements of the signal. Since the large changes fall between even and odd elements in this sample, these changes are missed in this wavelet coefficient spectrum. These changes would be picked up by lower frequency (smaller) Haar wavelet coeffient bands. The figure 2-13 below represents samples of wavelet functions transformed by Harr and Daubechies methods[37].

Figure 2-13 : Sample Wavelet Functions [20]

Chapter Three Moments and Pattern Rcognition

25


3-1 Introduction The ability to extract invariant features from an image is important in the

field of pattern recognition. An object within an image can be identified independent of its position, size and orientation. One of the most promising features for extracting such invariance are invariant moment, Hu [38], has shown results relating two-dimensional moment to be invariant to image translation, rotation, and scaling [39]. Two-dimensional moments have been used with success for a number of image processing tasks. In the robotics field, Goshorn [40] had used moments for motion tracking and for object orientation calculations. As recognition features, moments have been applied to a variety of image processing problems, this includes, aircraft identification [14], character recognition [41]. Casey [42] had used moments as a preprocessing tool to normalize patterns.

In general, moments describe numeric quantities at some distance from a reference point or axis [43]. The operation of the image feature vector extraction by moments is one of the common techniques used these days, where each moment order has different information for the same image. These moments are also divided into orthogonal (e.g. Zernike and Legendre), non orthogonal such as Regular moments, and Complex moments. The theory of moments provides an interesting and sometimes useful alternative for series expansions for representing objects and shapes [44].

The use of moments for image analysis is straightforward if we consider a binary or gray level image segment as a two-dimensional density distribution function. In this way, moments may be used to characterize an image segment and extract properties that have analogies in statistics [43]. Operations such as rotation, translation, scaling, and reflection may exist for each image. They are also called transformations. These transformations cause changes for each order of image moments. The solutions were introduced to keep the moments constant or invariants, which are called moment invariants [45].

Pattern Recognition is primarily concerned with the description and classification of measurement taken from physical or mental process. Pattern recognition is also referred to as image pattern recognition in the context of pattern recognition of images. From the respective of designing pattern recognition systems the image processing of the input image is called preprocessing. The image analysis is also called feature extraction of pattern recognition [2].

Pattern Recognition concepts have become increasingly recognition as an important factor in the design of modern computerized information systems. Interest in this area is still growing at a rapid rate, having a subject of interdisciplinary study and research in such varied fields as engineering computer science, information science, statistics, physics, chemistry, linguistics, psychology, biology, physiology, and medicine. Each group emphasizes certain


26

aspects of the problem. Pattern Recognition can be defined as "the categorization of input data into identifiable classes via the extraction of significant features or attributes of the data from a background of irrelevant detail" [1].

The field of pattern recognition includes a number of application that has been implemented and studied, i. e., radar detection, speech recognition, finger print identification [46], and handwritten characters recognition.

The recognition of handwritten characters has been a topic widely studied during the recent decades because of both its theoretical values in pattern recognition [47]. 3-2 Invariant Moments Theory

Various types of moment's theory have been used to recognize image patterns in a number of applications. Moment descriptors (moment invariants) are used in many pattern recognition applications.[14,38], The idea of using moments in shape recognition gained prominence in 1961, when Hu[35] derived a set of invariants using the theory of algebraic invariants[48].

Image or shape feature invariants remain unchanged if that image or shape undergoes any combination of the following changes:- 1. Change of size (Scale), 2. Change of position (Translation), 3. Change of orientation (Rotation), and 4. Reflection

The moment invariants are very useful way for extracting features from

two-dimensional images [49]. The moment invariants can be subdivided into skew and true moment invariants, where the skew moment invariants are invariant under change of size, translation, and rotation only but the true moment invariants are invariant under all of the previous changes including reflection. The diagram in figure 3-1 shows the categorization of the moment invariants.

Hu[35] described how fundamental transformations such as translation, rotation, and scale change could be achieved with the moment image representation [50]. He introduced the concept of moment invariants, which are nonlinear absolute combinations of moment values (Regular Moments) that are invariant to image rotation [51]. His observations are based on the theories of algebraic invariants which deal with the properties of certain classes of algebraic expressions that remain invariant under general linear transformations. Hu recognized that rotation invariance is the most difficult to achieve and proposed two different methods for computing rotationally invariant moments. The first method known as “principal axis method” uses the second order moments to compute the major and the minor axes of an ellipse that completely encloses the object. The second technique is based on a normalization procedure that achieves rotation invariance of moments. The algebraic moment invariants up to


27

third order have been used for the recognition of different types of shapes and images [52].

3-2-1 Regular (Geometrical) Moment Invariants (RM) The definition of the Regular moments RM has the form of the projection

of the f(x,y) function onto the nominal xp yq. Unfortunately, the basis set xp yq is not orthogonal. Regular moments have by far been the most popular type of moments[25]. Similar to definition of moments from classical mechanics, the two dimension (p + q)th order moments of an image irradiance (continuous image intensity function ) f(x , y) are defined by[53] :

dxdyyxfyxM qppq ),(ò ò

¥

¥-

¥

¥-

= (3.1)

where: p , q = 0, 1, 2, … The two-dimensional moment for a ( M ´ N ) discretized image, f(x , y ),

i.e. the digital images, the integrals are replaced by summations and Mpq becomes

Moment Invariants

Skew True

Regular and Complex Moment

Algebraic Moment Invariants

Orthogonal Moments Invariant

Scale Invariant Central Moment

Rotated or Reflected

Hu Moment Invariant

Zernike, Legendre, and Hermit

Moment Invariant

Figure 3-1 : Classification of Moment Invariants [45]


28

),(

1

0

1

0yxfyxM q

M

x

N

y

ppq åå

-

=

-

=

= (3.2)

3-2-1-1 Properties of Low Order Moments The low order moment values represent well-known[54], fundamental

geometric properties of a distribution or body. To illustrate these properties and show their applicability to object representation, consider the moments of distribution function that is binary and continuous [55]. · Zeroth Order Moments

The definition of the zeroth order moment, M00, of the distribution, f(x,y)

ò ò¥

¥-

¥

¥-

= dxdyyxfM ),(00 (3.3)

represent the total mass of the given distribution function or image f(x,y). when computed for a binary image of a segmented object, the zeroth moment represents the total object area [54,55].

· First Order Moments The two first order moments, M10 and M01, are used to locate the center of mass of the object. They are defined as follows:

ò ò¥

¥-

¥

¥-

= dxdyyxxfM ),(10

ò ò¥

¥-

¥

¥-

= dxdyyxyfM ),(01 (3.4)

The center of mass defines a unique location with respect to the object that may be used as a reference point to describe the position of the object within the field of view. If an object is positioned such that its center of mass is coincident with the origin of the field of view (mean of x and y), then the moments computed for that object are referred to as central moments are shown on latter.

· Second Order Moments The second order moments M02, M11, and M20 may be used to determine several useful object features, and these called moments of inertia (mean square value or average energy ).

ò ò¥

¥-

¥

¥-

= dxdyyxxyfM ),(11

ò ò¥

¥-

¥

¥-

= dxdyyxfxM ),(220 (3.5)

ò ò¥

¥-

¥

¥-

= dxdyyxfyM ),(202

In special case Hu[38] defines seven values, computed from central moments through order three, that are invariant to object scale, position, and orientation. In terms of the central moments, the seven moment invariants are given by


29

02201 mm +=M

( ) 211

202202 4mmm +-=M

( ) ( )20321

212303 33 mmmm -+-=M

( ) ( )20321

212304 mmmm -++=M (3.6)

( )( ) ( ) ( )[ ] ( )( ) ( ) ( )[ ]20321

21230032103

20321

21230123012305 321333 mmmmmmmmmmmmmmmm +-++-++-++-=M

( ) ( ) ( )[ ] ( )( )03211230112

03212

123002206 4 mmmmmmmmmmm ++++-++=M( )( ) ( ) ( )[ ] ( )( ) ( ) ( )[ ]2

03212

1230032112302

03212

1230123003217 3333 mmmmmmmmmmmmmmmm +-++-++-+++=M The functions M1 … M7 can be normalized to make them invariant under a scale change by substituting the normalized central moments hpq or mpq in Equations (3.6).

3-2-2 Complex Moments (CM)

Pattern recognition feature such as moment invariants are considered "good" if their values are sensitive to the identity of the pattern being recognized (discrimination) but not to the noise encountered [55]. The notation of complex moments is very simple and quite powerful in providing an analytic characterization for moment invariants. The CMs of order (p+q) are defined as [56,57]:

dxdyyxfiyxiyxC q

x y

ppq ),()()(

0 0

-+= ò ò¥

=

¥

=

(3.7)

Where p, q=0,1….., and 1-=i , and f(x, y) is the real image intensity function, which is continuous and has bounded support. For digital image the CM equation is:

åå= =

-+=N

x

qM

y

ppq yxfiyxiyxC

0 0),()()( (3.8)

Where we have assume the function f(x,y) is a two dimensional function. N×M is the dimensional of f(x,y).


30

3-3 Pattern Recognition (PR) System The objective of Pattern Recognition System is to determine, on the basis

of the observed information, the Pattern class responsible for generating a set of measured characteristics similar to the observed data( a pattern class is a category determined by some given common attributes. A pattern is the description of any number of a category representing a pattern class). Correct recognition will depend on the amount of discriminating information contained in the measurements and effective utilization of this information. Recognition is regarded as a basic attribute of human being and other living organisms.

We may divide our acts of recognition into two major types according to the nature of patterns to be recognized; i.e. the recognition of concrete items and that of abstract items. As an example, we recognize characters, pictures and the objects around us, which includes visual and aural pattern recognition. The other category examples are physical objects, speech, waveforms, signatures, fingerprint [1], and the human-perception from their handwriting.

The study of pattern recognition problems may be logically divide into two

major categories :

1. The study of the pattern recognition capability of human beings and other living organisms.

2. The development of the theory and techniques for the design of devices capable of performing a given recognition task for a specific application.

The second area deals primarily with engineering, computer, and information science. In simple language, pattern recognition can be defined as the categorization of input data into identifiable classes via the extraction of significant feature or attributes of the data from a background of irrelevant details [1]. 3-3-1 Designation of Pattern Recognition System

The pattern recognition system consists of the following stages [3,58] : · Step 1. Data Representation

The first step in PR is to choose an appropriate mathematical presentation of the sensor information. A pattern is usually taken to be a vector of raw measurement data from an appropriate transducer. We shall use the term "pattern" to denote the p-dimensional data vector x=(x1,….xp)T of measurements (T denotes vector transpose), whose components x1 are measurements of the features of an object. Thus the features are the variables specified by the investigator to be important for classification. In discrimination, we assume that there exist C groups or classes, denoted by w1,…,wc, and associated with each pattern x is a categorical variable z that denotes the class or group membership; that is, if z=i, then the pattern belongs to wi, iÎ{1,….C}.


31

· Step 2. Feature Vector Selection In general, the dimension of patterns can be very large, for instance an x-

ray image of size 512 ´ 512 generates a pattern vector with n=262144!. A speech signal of 1 second duration sampled at 25 KHz sampling frequency generates a pattern space of dimension 25000. It is therefore common practice to map the pattern vectors onto a lower dimensional feature space of feature vectors. For example, in recognition of faces, does the human brain store the entire vector such as the hair color, eye color, shape of the nose, etc. ? Similar questions arise in the classification of image data such as (fingerprints, satellite images of different terrain, x-ray images in medical diagnosis, etc.).

In computer implementation of pattern recognition of such data, particularly in real time, the amount of memory needed for storing information is important. Hence, one of the uses of features is data compression and memory reduction. In general, feature selection process serves to:

1. Compress Data. 2. Eliminate Redundancy. 3. Retain discriminatory information. 4. Invariance to measure environments.

The objective of feature selection is to identify attributes that have some

correlation with class membership, such as 1. Similar attributes of patterns from the same class. 2. Dissimilar attributes of patters from different classes.

As an example the discrimination of features are: 1. In the analysis of handwritten characters 'a' and 'b', the aspect ratio

obtained by dividing the length by width can be chosen as the one-dimensional feature.

2. In medical diagnosis, the patients symptoms, blood pressure, temperature could serve as features and diagnosis of illness is the class.

Feature selection process quantifies relevant information and identifies

these attributes into a feature vector. It can be said that feature selection introduces another level of complexity in the overall problem of the pattern recognition. The choice of features depends strongly on the type of pattern data and the nature of classes. Selection of features could be selected according to individual understanding of an application problem, or could be mathematically computed. When the data is a speech, or similar waveform where it is not possible to handpick suitable features, it is useful to select features using mathematical transformations. Mathematical features are obtained by computing the Fourier, the wavelet or any kind of signal transforms. Other commonly used mathematical features are the moments of two-dimensional images. · Invariant features


32

Another important attribute that should be taken into consideration in the stage of selecting the appropriate discrimination feature is the invariance versus measurement conditions. The need for invariant features rises in many practical problems as illustrated in the following examples:

1. Speech Recognition: the adopted features should be independent

of speakers. Hence, features should be insensitive to speaker dependent data.

2. Speaker Recognition: features should be invariant to spoken speech.

3. Image Recognition: features should be invariant under rotation, translation, scaling and the illumination angle used in generating the image.

For the overall classification performance to be independent of the

experimental measurement parameters, the computed features should be invariant under such variations. In general, feature selection is an important step in designing a pattern recognition system and involves a significant amount of thought and care. The task of feature selection depends on the application problem, the nature of data and nature of classes.

· Step 3. Classification

The objective of classification is to make a decision about class membership of a pattern, as shown in figure 3-2.

Figure 3-2 : Class membership of a pattern The output could be

1. An unambiguous decision XÎ class C. 2. Doesn't know (not enough data). 3. List of possibilities with associated probabilities

XÎC1 with probability P1 XÎC2 with probability P2

F P C


33

In the last case , a post-processing step is required to evaluate the possibilities to make a final decision . This step could use contextual information to make a decision.


34

Figure 3-3 shows the overall of a generic pattern recognition system.

Outside World

Sensor Measurement

Preprocessing

Feature Extraction

Classifier

Post Processing

Output / Decision

Figure 3-3 : Overall schematic of a generic pattern Recognition system

What type of data to measure?

(Noise filtering, segmentation)

Extract relevant features or attributes · Reduce dimensionality · Extract features · Invariance properties

Rotating, Scaling, Translation, Illumination, Frequency Gain instrument settings, Material properties, Transducer characteristics,…..

Makes decision about class membership of pattern · An ambiguous

x belongs to wi · Doesn't know or no decision · List of possibilities with

associated probabilities: x Î w1 with probability p1 x Î w2 with probability p2


35

3-3-2 Designation Methodology The basic design concepts for automatic pattern recognition described

above may be implemented according to one of the three principal categories of methodology: heuristic, mathematical, and linguistic or syntactic. It is common to find a combination of these methods in a pattern recognition system [1].

3-3-2-1 Heuristic Methods

The heuristic approach is based on human intuition and experience, making use of the membership-roster and common property concepts. The structure and performance of a heuristic system will depend on a large degree of the cleverness and experience of the system designers. 3-3-2-2 Mathematical Methods

The mathematical approach is based on classification rules, which are formulated and derived in a mathematical framework, making use of the common-property and clustering concepts. This approach may be subdivided into two categories: deterministic and statistical.

The deterministic approach is based on a pure mathematical framework, without employing explicitly the statistical properties of the pattern classes under consideration.

The statistical approach is based on mathematical classification rules, which are formulated and derived in a statistical framework. · Minimum Distance Classifier

A minimum distance classifier computes the distance from a pattern X of unknown class to several patterns of known classes. The classifier then assigns the unknown pattern the same identity as that pattern with the minimum distance [48].

The Euclidean distance between an arbitrary patterns vector x and the ith prototype is given by [1]:

21

2

1, ),(

úúû

ù

êêë

é-= å

=

n

jqjijqi xxxxd (3.9)

d: is any distance measure. x: is the unknown sample to be classified. · Likelihood Function

Suppose that, in a game between nature and the classifier, nature selects class wi and produces a pattern x. The probability that x comes from wi is written as P(wi/x). If the classifier decides that x came from wi when it actually came from wi, it incurs a loss equal to Lij. Since pattern x may belong to any of the M classes under consideration, the expected loss incurred in assigning observation x to class wj is given by :

)/()(1

xwPLxr i

M

iijj å

=

= (3.10)

where wi: is called the a prior probability of class wi, i=1 to M . Lij: the M by N matrix L=( Lij) with elements Lij=L(yi,zi), Y=( y1, y2 …. YM) and Z=( z1, z2 …. zN)


36

which is often referred to as the conditional average risk or less in decision theory terminology. The classifier has M possible categories to choose from for each pattern given by nature. If it computes the quantities r1(x), r2(x)..., rM(x), for each x, and assign each pattern to class with smallest conditional loss, it is clear that the total expected loss with respect to all decision will also be minimized. The classifier which minimizes the total expected loss is called the Bayes classifier. From a statistical point of view, the Bayes classifier represents the optimum measure of performance [1].

Using Bayes' formula, (3.11)

we my express Eq. (3.10) in the form :

)()/()(

1)(1

ii

M

iijj wPwxPL

xPxr å

=

= (3.12)

Where P(x/wi) is called the likelihood function of class wi. Since 1/P(x) is a common factor in the evaluation of rj(x), j=1,2,…,M, it may be dropped from Eq. (3.12). The expression for the average loss then reduces to

)()/()(1

ii

M

iijj wPwxPLxr å

=

= (3.13)

3-3-2-3 Linguistic (Syntactic) Methods Characterization of patterns by primitive elements (subpatterns) and the

relationships suggests automatic pattern recognition by the linguistic or syntactic approach, making use of the common-property concept. A pattern can be described by a hierarchical structure of subpatterns analogous to the syntactic structure of languages. This permits application of formal language theory to the pattern recognition problem. A pattern grammar is considered as constructing of finite sets of elements called variables, primitives, and productions [1]. Minimum Distance Classifier has been chosen because it is the most common used classifying method , and represents one of the most easiest way to test the dissimilarities between compared patterns.

Chapter Four Handwritten Identification System

36


4.1 Introduction

In this chapter the procedures for identifying the persons from their handwriting will be presented. The operations can be divided into the following:

1. If the persons were defined priory in the created database., or 2. Deciding whether a specific handwritten pattern belongs to the

same person or not.

In the first point, at least three samples should be taken for each person. These samples then analyzed and their features should be selected. These features are saved in the database in order to use them for matching processes with any newly defined handwriting, using minimum distance criterion.

In the second point, the aim is to recognize two different handwritings as to belong to the same person or not. This recognition process, however, needs a comparison between the features of the two handwritings.

The presented handwritten identification based on discrete wavelet transform and complex moment descriptors. The block diagram summarizing all the adopted operation will be presented too. As will be shown, the implementation of the suggested system will mainly be consisted of training and recognition phases. The identification phase involves the following: 4.2 The Tested Samples

Our investigation has been performed on 30 samples, belong to 10 persons; 3 samples for each. A blank sheet of paper is divided into six rows, shown in figure 4-1a, was given to each person. As it is obvious, the first row occupy the name of the writing person, the second row shows a typed samples of alphabetic from 'A' to 'Z' , the other rows (from three to six), to be used by the volunteers, as shown in the figure 4-1b. 4.3 Training Phase

In the training phase, the writing samples of each person are fed to the computer by using page scanner (300 DPI: Dotes Per Inch). Each sample then manipulated, separately, by denosing them firstly and then binarizing each isolated letter separately. For English alphabets adopted in this research, we should have 26 characters (from 'A' to 'Z'), illustrated in figure 4-1a below. The features value of each isolated letter can then be computed (i.e. complex moments), stored in a reference database to be used in matching operation.


37

It should be noted that the mentioned processes will be discussed in the coming sections. Figure 4-2 presents the block diagram of the training phase.

4-3-1 Handwritten Image Acquisition The first step in the handwritten identification system is the image acquisition. It consists of the utilization of the scanner to convert the written sheets into a numerical data in an image form, which is suitable for input into a digital computer. Each image, then, has been stored as 24-bits BMP image file in JPEG format.

a) Blank sheet of paper

b) Filled sheet of paper Figure 4-1: Showing samples of blank and filled sheets


38

Start

Input handwritten image and the name of person

Extract writing body from the image

Perform image binarization

Perform image segmentation to split the image into 26 sub-images, each represents an individual character

Perform Wavelet transform to produce four sub-images representing the (LL,LH,HL,HH) sub-bands

Compute the complex moments (m1, m2, m3, m4) for each sub image

Insert the moment's features in a one dimensional vector

Saving the feature vector in the reference database

Is there more images

End

Figure 4-2: Flowchart for the training phase

Yes

No


39

4-3-2 Image Binarization Process In this step, the handwritten is converted into a binary image, whose pixel's

values are set to one or zeros depending on the used threshold value; i.e. all points having values larger than the threshold is considered to represent part of the letter, while other points are background. The threshold value first suggested equal 128, during the coursework, this threshold found to produce a lot of missing in detected character's body. To overcome this problem, the threshold value is increased to 220 to 225, which found to cope well with the problem. This is true because the image gray level values have been inverted so that; the characters are presented as brighter gray.

4-3-2-1 Threshold Value's effects on the Extraction process

As it has been mentioned before, the threshold value used for the binarization process has great effects on the extracted features of the characters. Figure 4-3 illustrates the effects of changing the threshold value.

Figure 4-3: The effect of the threshold binarization value on handwritten image

a - Original image

b - Binarized image by using 220 as a threshold value

c - Binarized image by using 200 as a threshold value

d - Binarized image by using 185 as a threshold value


40

4-3-3 Character's Isolation (Segmentation) process The adopted method for character's isolation would definitely, affecting,

the performance of the recognition algorithm. In our present research, the projection technique is used to isolate the written characters from the image of the fill sheet. Unfortunately, characters are not presented in tall descending or ascending order, therefore the raster search detects the tallest characters first. Therefore, our search started vertically (i.e. column projection) to separate the characters from each other. An extra row's projection is then performed on each isolated character to enclose this character in a certain window's size. Thus, as a character is isolated, their horizontal and vertical coordinates will be defined to be extracted for successive operation; i.e. performing wavelet transform. The horizontal and vertical projections will be discussed in the coming section.

4-3-3-1 Projection Technique

The word projection in image processing refers to mapping an image into waveform whose values are the sums of the values of the image points along particular directions [59]. In order to split the image into writing lines, parallel-rays method, illustrated in figure 4-3. In our present research, the parallel form projection is utilized in discrete image form, using intensity summing operation; i.e. horizontally and vertically, the character will be bounded by two summing zero values, respectively. Finally, the isolated characters are surrounded by certain window's size which prepared for further manipulations. Figure 4-4 demonstrates the isolation performance by the projection method.

4-3-4 Feature Extraction The concepts of features extraction and their selection have been explained

in previous chapters. Here, the procedures used in our identification system (i.e. feature's extraction and selection) will be discussed in some details. 4-3-4-1 Discrete Wavelet Transform

The 2D discrete wavelet transform; i.e. floating point wavelet (9/7), is adopted here to perform the recognition operation. It should be noted that only one level wavelet transformation is performed on the images of the individual extracted characters. Number of transformation levels is not extended because it

1 2

3 4

Figure 4-4: Illustrate handwritten isolation by projections


41

h~

is constrained by the size of the extracted character's sub-images; i.e. higher transformation levels require larger size of elements.

We consider the popular (9/7) filter pair. The analysis filter has 9 coefficients, while the synthesis filter h has 7 coefficients [60].

In fact, the (9/7) wavelet transform is representing the core of the well known JPEG-2000 standard. Figure 4-5 shows a row of seven symbols "ABCDEFG" and their left-right extended form, the characters in figure 4-4 may represent a pixel value. This extending operation is, in fact, representing the first step of the (9/7) wavelet transform.

The row of the extended pixels in an image can now be denoted by pk, pk+1, through pm. Since the pixels have been extended, indexing values below k and above m can be used [61]. The (9/7) floating-point wavelet transform can now be computed by executing four "lifting" steps followed by two "scaling" steps on the extended pixel values pk through pm. Each step is performed on all the pixels of the row before the next step starts, as illustrated below: Step 1 : c(2i+1) = p(2i+1) + a[p(2i)+p(2i+1)], for k-3£2i+1<m+3 Step 2 : c(2i) = p(2i+1) + b[c(2i-1)+c(2i+1)], for k-2£2i<m+2 Step 3 : c(2i+1) = c(2i+1) + g[c(2i)+c(2i+2)], for k-1£2i+1<m+1 Step 4 : c(2i) = p(2i) + d[c(2i-1)+c(2i+1)], for k£2i<m Step 5 : c(2i+1) = -G´c(2i+1) , for k£2i+1<m Step 6 : c(2i) = (1/G)´c(2i) , for k£2i<m

Where the five constants (wavelet filter coefficients), used by JPGE 2000, are a=-1.586134342, b=-0.052980118, g=0.882911075, d=0.0443506852, and G=1.230174105. After applying wavelet to each row data, then the transform should be repeated on each column of the row transformed data. This wavelet transform can be repeated L times, where L is referring to the level of the transformation, see figure 2-6.

….. DEFGFEDCB ABCDEFG FEDCBABCD …..

k m

Figure 4-5: Extending a row of characters.


42

The following summarizes the steps that should be followed to perform the algorithm of (9/7) floating-point wavelet transform, for one-level decomposition: Step 1 : Set Y=0. Step 2 : Read a block of data (p(x)) of size w from the handwritten image file,

w denotes the width of the image. Step 3 : p(x)=p(-x) for x=1,2,3.

p(w+x)=p(w-2-x) for x=0,1,2. Step 4 : For all x where (-3£x£w+3) compute:

c(x)=p(x)+a[p(x-1)+p(x+1)], x here denotes the odd index in the wavelet coefficients array c.

Step 5 : For all x where (-2£x<w+2) compute : c(x)=p(x)+b[c(x-1)+c(x+1)], x here denotes the even index in the

wavelet coefficients array c. Step 6 : For all x where (-1£x<w+1) compute :

c(x)=c(x)+g[c(x-1)+c(x+1)], x here denotes the odd index. Step 7 : For all x where (0£x<w) compute :

c(x)=c(x)+d[c(x-1)+c(x+1)], x here denotes the even index. Step 8 : For all x where (0£x<w) compute :

c(x)=-G´c(x), x here denotes the odd index. Step 9 : For all x where (0£x<w) compute :

c(x)=(1/G)´c(x), x here denotes the even index. Step 10: Push the array c in the buffer. Step 11: Increment (Y). Step 12: If (Y<H) then go to step 2, H denotes the height of the image. Step 13: Rearrange the image columns by making the columns with the odd positions in

the left half of the image and the columns with the even positions in the right half of the image.

Step 14: Repeat the steps from 3 to 9, but this time these steps will be performed on the image columns.

Step 15: Rearrange the image rows by making the rows with the odd positions in the above half of the image and the rows with the even positions in the below half of the image.

Step 16: Eliminate the extended rows and columns from the output file that contains the four sub bands LL,LH,HL,HH.

Step 17: Stop. 4-3-4-2 Computation of the Complex Moments

The computation of the complex moment involves the calculation of its real and imaginary components. The kth-order complex moment Mi for an individual isolated character handwritten image of size n´m is calculated according to the following equation:

),(),(1

0

1

0yxdsyxMZM

n

x

m

y

ini ´= åå

-

=

-

= (4.1)


43

Where i indicates moment's order, Zn = xn + iyn is a complex number, M(x,y) represents a pixel value at position (x,y), ds(x,y) is the distance between the position (x,y) and the center of image, xn and yn are the normalized coordinates which can be determined by using the following equations:

c

cn

c

c

cn

c

yyyy

Hy

xxxx

Wx

-=

-=

-=

-=

21

21

(4.2)

It is well known that the moment sets can offer a powerful description of the geometrical distribution of the material within any region of interest [54]. The low order of complex moments has meanings which is very relevant to some well-known physical quantities; i.e.

1. The 0th-order moment represents the total mass (i.e. total number of 'on' pixels within an image).

2. 1st - order moment refers to the image's center of mass. 3. 2nd - order moment represents the image's moment of inertia. 4. The 3rd-order and 4th-order moments are used to compute the

statistical quantities known as skews and kurtosis, respectively. The computation of complex moments may require a relatively long time,

and this time is increased with the increase of moment's order, because this will lead to increase the number of additions and multiplication process implied within the calculations. In order to reduce the computation redundancy which may occur during the trial of computing, each nth-order complex moment separately computed recursively (from Zeroth-order up to any nth_order moments); i.e.

ZZZ nn .1-= (4.3) Where ; Z=x+ i.y is the complex number. By assuming that the real and imaginary parts of Zn & Z n-1 complex numbers could be written as,

nnn iIRZ +=

111

--- += nn

n iIRZ (4.4) Keeping in to consideration that the computation started from Z0 & Z1, it

is simple to conclude that R0=1; I0=0; R1=x; I1=y. Substituting the values Zn ,Z n-

1 & Z1 in equations (4.4), we get: yIxRR nnn 11 -- -=

yRxII nnn 11 -- += (4.5)


44

The equations above indicating that knowing the components of Zn-1 will be used to compute the components of Zn .

Before the computation of the complex moments take place, the

coordinates (x, y) which are entered in this computation must be normalized first. The coordinates normalization is a process of mapping the actual discrete image coordinates (x, y) into normalized set (xn, yn), so that the new set of coordinates (xn, yn) can be considered as a "standard" version of the original coordinates (x, y). The idea of coordinate normalization used in this work is to map the original signature image coordinates to standard range (i.e., [-1, 1]). The purpose of this step is to keep domain of image coordinates fixed and irrelevant to the original image size. To perform the normalization, a linear mapping method is adopted; i.e.

Normalized coordinates = scale ´ discrete image coordinate + offset (4.6) In this work, the centralized normalization is adopted, accordingly the pixel

coordinates (x, y) will be mapped to the range [-1, 1]. The coordinate origin point will be at the center of the image, as shown in Figure 4-6.

H

+X

+1

-X

-1

Figure 4-6: Central coordinates normalization.

+1 +Y

-1 -y


45

The computation of complex moments involves the following steps:

Step 1 : Set Mrj=0, Mij=0 , for j=1, 2 …… m; m is the number of moments, H is the height of the handwritten image., W is the width of the handwritten image, Mj is the complex moments of order j.

Step 2 : Set y=0 : x0=H/2 : y0=W/2 Step 3 : Compute the normalized coordinates yn by :

11

2-

-=

Hyyn

Step 4 : Set x=0 Step 5 : Compute the normalized coordinates xn by :

11

2-

-=

Wxxn

Step 6 : Set ynew = yn ; xnew = xn ; j=1 Step 7 : Read from the image file a data row G Step 8 :

8.1: Compute the moment of order j Mrj= Mrj + xnew ´ G[x] ´ ABS(x0-x) ; Mij= Mij + ynew ´ G[x] ´ ABS(y0-y); Set n=j ; j=j+1. If (j>4) then go to step 10 8.2: Compute the coordinates of the complex moments of order

higher than j xnew = xnew ´ xn - ynew ´ yn ; ynew = xn ´ yn + ynew ´ xn ; Set n=n-1 .

8.3 if (n>1) then go to step 8.2 Step 9 : if (j£4) then go to step 8 Step 10 : INC (x)

If (x<W) then go to step 7. Step 11 : INC (Y)

If (y<H) then go to step 6. Step 12 : For all j values /j=1, 2, 3, 4/

( ) ( )( )22ijrjj MMM +=

Step 13 : Stop.


46

4-3-4-3 Analyzing the Complex Moments Values The complex moment features have been computed for the 104-sub images

that resulted from the wavelet transform process (each handwritten image divided into 26 images (from A to Z) and each image divided into 4 sub-images: LL,LH,HL,HH). Table 4-1 below demonstrates the arrangements of the computed complex moment values.

Table 4-1 : Complex moments values for sub-images (LL,LH,HL,HH)

Handwritten images

Moment order LLi LHi HLi HHi

A

1 LL1 LH1 HL1 HH1 2 LL2 LH2 HL2 HH2 3 LL3 LH3 HL3 HH3 4 LL4 LH4 HL4 HH4

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

Z

1 LL1 LH1 HL1 HH1 2 LL2 LH2 HL2 HH2 3 LL3 LH3 HL3 HH3 4 LL4 LH4 HL4 HH4

Where LLi , LHi , HLi , and HHi (for i=1,2,3,4) are the complex moments values of ith orders to the sub-image LL, LH, HL, and HH, respectively, obtained from the wavelet decomposition, (i=1,2,3,4). After that, each of these values are normalized by dividing them by the low-low value.

i

iLHi LL

LHM =

i

iHLi LL

HLM = (4.7)

i

iHHi LL

HHM =

Where i is the order of complex moments (i=1,2,3,4), and then select the minimum value of complex moments for each sub-image (MHLi, MLHi, MHHi). The features vector is the three minimum values for the sub-images (LH, HL, HH), as shown in Table 4-2.


47

Table 4-2 :The features vector for the one handwritten pattern

Handwritten characters

Minimum moment of LH

(MLHi)

Minimum moment of HL

(MHLi)

Minimum moment of HH

(MHHi) A MLH MHL MHH B MLH MHL MHH . . . . . . . . . . . . . . . . Z MLH MHL MHH

Where MLHi is the minimum value of complex moments for sub-image LH, MHLi is the minimum value of complex moments for sub-image HL, MHHi is the minimum value of complex moments for sub-image HH. 4-4 Recognition Phase

In this phase most of the operations mentioned in features preparing phase (i.e. scanning, binarization, isolation, wavelet transform, complex moment's calculation and analyzing) will be repeated for other adopted handwritten samples, whose ID has to be recognized. The matching test then is performed by utilizing the minimum distance classifier mechanism, in which the identification is based on adopting the reference class whose features has the closest distance compared with the features of the previously created Database. The classification processes will be given down. 4-5 Matching and Euclidean Distance Measures

The objective of handwritten identification machine is to identify writer from his handwriting. Euclidean minimum distance metric has been used in this work to perform the desired identification, as follows: Let C be the features vector representing the tested handwritten. The components of C are the values of complex moments for LH, HL, HH sub-images. Let T be the reference features vector. The measure of closeness between the test handwritten and the reference can be performed by utilizing the Euclidean distance metric, given by:

D(C,T) = ( )

n

TCn

iii

2

1å

=

- (4.8)


48

Where n is the number of features, Ci is the value of the ith feature in the feature's vector C, Ti is the ith feature of the reference vector T. The smallest value of D(C,T), the greater the similarity between C and T, and therefore between the test handwritten represented by C and the reference handwritten T. Therefore, the test handwritten represented by C belongs to the same class to which the reference handwritten (represented by T) belongs. Figure 4-7: illustrates the block diagram of the handwritten identification phase.

Figure 4-7: Block diagram for handwritten identification phase

Start

Input unknown handwritten image

Extract writing body from the image

Image binarization

Segment the handwritten image into 26 images, one for each character from 'A' to 'Z'

Wavelet decomposition up to 1-level, decomposition the image into four sub

images (LL,LH,HL,HH)

Calculated the complex moments (m1,m2,m3,m4) for each sub image

Analyzing the moment values

Classification using minimum distance

Features vector

Logical decision

Reference set

Matching "Identification"


49

4-6 Experimental Results In this work, 40 handwritten images have been collected from 10 persons,

30 handwritten images were used as a checking set, while the remaining 10 handwritten images were used for testing purposes. Below, the experimental results illustrating the handwritten image and the recognition results are summarized by the following steps: 1 - The unknown handwritten image is entered to the system, Figure 4-9a. 2 - Convert the handwritten image into binary image, Figure 4-9b. 3 - Perform segmentation process to isolate individual characters from the

handwritten image, Figure 4-9c. 4 - Apply the discrete 1st-level wavelet transform to each isolated character,

Figure 4-9d. 5 – Features extraction: For each isolated character, four moment values are

computed; i.e. 416 values are produced and then select 78 values as a feature vector. Table 4-3 lists the measured minimum moment values computed for the sub-images shown in figures above.

Figure 4-9: illustrating the recognition procedures.

d – One level wavelet transform decomposition

c – Isolate handwritten characters

a - Original image

b - Binarized image


50

Table 4-3: Presents the minimum complex moment values

Minimum

HH Minimum

HL Minimum

LH Minimum

LL Sample

NO. Handwritten

character 5.068365916 8.120516651 3.532795766 7.737402678 1

A 3.468728863 5.222080748 3.255625615 5.282238211 2 3.858951026 6.679482953 1.587161338 6.772321486 3 3.617168695 5.604191408 2.842573757 5.511199283 1

B 2.449068205 5.380153414 2.213093919 5.144408969 2 4.566062109 5.172550021 4.17798611 5.3193825 3 2.256234102 6.039136168 2.206267401 3.168756507 1

C 2.408859065 3.473173209 2.058872561 4.615169738 2 2.052756442 3.953214571 2.315965015 4.330689149 3 3.835334439 6.996059939 4.90964805 7.189109375 1

D 4.289348817 5.566292396 4.696921613 5.624304361 2 2.208906339 4.291052189 2.177374787 4.18617357 3 2.18099419 4.320095144 3.102839022 5.75188073 1

E 2.55812623 4.057796451 1.508359872 4.29006402 2 2.71936476 4.914800343 3.053173451 3.988988974 3 3.58863994 4.610037441 3.047566353 5.023378802 1

F 2.424197797 3.866769201 1.914980991 3.674063377 2 2.830694717 3.870634557 3.264519491 4.037077037 3 3.432496378 4.425045232 2.363988814 4.859014593 1

G 3.987229609 6.252016821 2.494195557 5.40957031 2 4.786937426 5.173739963 1.789390735 6.628886349 3 3.317793367 4.553702911 2.610538575 4.638329255 1

H 1.910746848 4.731072889 3.057858929 4.807360068 2 3.197763917 5.218291219 3.591737143 5.209958509 3 3.297967239 2.193295922 3.180274707 2.883046023 1

I 3.781403774 2.153627953 4.03666757 3.111183003 2 3.672051387 3.585386323 4.109229097 4.10766992 3 4.67384384 5.177793741 4.355322044 5.113852627 1

J 4.804012157 5.214604377 4.867948043 5.323569463 2 4.744525494 5.429713947 5.045572218 5.519729305 3 2.534790736 4.637839849 3.150196479 4.63634562 1

K 1.956194272 6.033882749 2.996285293 6.144537683 2 2.439051576 5.747791382 1.620624027 5.731453858 3 2.909771569 6.557608485 2.137924911 6.380210314 1

L 3.137796237 3.616396156 2.659757785 2.8554032 2 3.522733474 3.87994222 2.458272964 3.713363141 3 10.47343699 10.41299991 4.296728489 11.09263304 1

M 8.005315115 11.11632814 7.019383727 11.22190882 2 6.32369122 10.48844454 4.666257816 10.40908851 3 4.879781428 7.658633051 4.270114618 7.764680202 1

N 3.067928428 7.307956236 4.500318651 7.178186548 2 3.550738885 7.166247987 4.254957831 7.012490671 3 3.193207229 4.995475618 4.004977722 5.069242625 1 O


51

2.534383228 4.627406374 1.501417497 4.712944843 2 4.748690283 4.979909813 3.572141685 4.734429132 3 3.479075351 3.963069943 1.530461544 4.105877102 1

P 2.612348242 4.255614744 2.663622368 3.97805886 2 2.361317504 4.230777982 3.360390218 3.925417724 3 5.423812429 6.856320619 5.518996751 6.940011554 1

Q 3.080206227 6.76484552 4.058233448 6.574231494 2 5.310503521 6.621768473 3.270350603 6.478644528 3 2.387834792 5.336043657 1.740669875 5.490510012 1

R 3.113320419 6.532328971 4.238794601 6.53589373 2 3.250413377 6.762286971 4.114892911 6.887594822 3 3.840529918 5.788204173 3.586879845 5.874310264 1

S 2.321524352 5.29068024 1.006262558 5.46871539 2 2.770023108 5.661934635 2.695588271 5.69270801 3 3.479970641 8.101974871 3.934094978 8.720980722 1

T 4.351858617 5.395985178 2.376472437 6.815420286 2 3.811977955 7.032264349 4.192484857 6.729912673 3 3.508903944 5.526740592 4.186850512 5.578332897 1

U 1.41407249 4.533542213 2.793423342 4.45085257 2 4.143629241 4.984150981 3.203220447 5.084701707 3 2.262906936 4.735445495 2.969288992 4.995766856 1

V 1.458901182 3.754909522 2.292440388 3.753164067 2 1.652363186 4.906004257 1.227648567 4.68084135 3 5.547330533 10.5687689 6.980139906 10.16510732 1

W 3.114808721 10.87333128 3.763960731 10.96221707 2 5.946176498 8.372217875 6.680666328 8.657083092 3 2.525173951 4.964672675 2.657784466 5.173271575 1

X 3.191494965 5.265896744 2.057894566 5.365455587 2 2.208484194 4.844217084 1.636574648 5.021131415 3 2.704263479 5.735786099 3.153527311 5.889174338 1

Y 4.146474474 5.344777047 3.431259623 5.366255932 2 2.616787046 5.079601942 3.082430453 4.971035749 3 2.30889125 10.42218894 5.834579154 9.094057244 1

Z 4.925879092 9.846061326 4.270346385 9.996299748 2 3.336362509 10.09272493 2.156249357 9.435200322 3

The three minimum complex moment values (i.e. LH, HL, and HH) are

then be normalized by dividing each of them by the LL minimum moment, as described in Eqs.(4.7), and listed in Table 4-4.


52

Table 4-4: Represents the normalized moments of those listed in

table(4-3) Normalized moments of

HH

Normalized moments of HL

Normalized moments of

LH

Sample No.

Handwritten character

0.652477406 1.040098536 0.447734825 1 A 0.581077369 0.985814671 0.569336167 2

0.535669064 0.986291476 0.229697758 3 0.523559189 1.016873301 0.326254309 1

B 0.476064057 1.031055444 0.318281778 2 0.678005566 0.972396706 0.594224091 3 0.382785251 1.499540473 0.393475801 1

C 0.517515547 0.752555899 0.375109393 2 0.47400226 0.912837296 0.418909926 3 0.533492292 0.973146961 0.63188669 1

D 0.73257017 0.989685486 0.748435806 2 0.464119916 1.021724973 0.395860084 3 0.220102463 0.75107523 0.365007092 1

E 0.452941177 0.945859183 0.351593791 2 0.535162306 1.189324915 0.336648494 3 0.608646236 0.917716466 0.570989928 1

F 0.308564324 1.021707789 0.4606919 2 0.387120625 0.958555818 0.39867915 3 0.703207408 0.907427236 0.21898381 1

G 0.724610165 1.112409745 0.461070919 2 0.66672057 0.780484035 0.177866306 3 0.635698905 0.968317032 0.520886375 1

H 0.329337999 0.977411695 0.535065654 2 0.613779152 0.987776395 0.648002115 3 0.691852323 0.760756473 0.850215697 1

I 0.716636972 0.692221561 1.014012253 2 0.30806498 0.872851615 0.869162923 3 0.910243343 1.012503511 0.834051368 1

J 0.898609048 0.9720962 0.9144143 2 0.81440448 0.979572356 0.860185929 3 0.47312246 0.990279941 0.550170574 1

K 0.164844975 0.978225226 0.487633968 2 0.365130493 0.999181699 0.282759674 3 0.456062015 1.018378173 0.335086902 1

L 0.601281061 0.478125098 0.931482386 2 0.70704749 1.044859356 0.662007154 3 0.930412791 0.938725113 0.381109539 1

M 0.697729225 0.990591558 0.617609447 2 0.603192876 1.006235831 0.433653309 3 0.628458778 0.985690047 0.549940823 1

N 0.427396029 1.015359373 0.57921478 2 0.455737365 1.021926206 0.548661806 3 0.598366627 0.980882262 0.752117534 1 O


53

0.244030993 0.981360327 0.31857311 2 0.958916995 1.021987271 0.716977999 3 0.326197886 0.941054511 0.37274899 1

P 0.388698941 1.027252542 0.66957842 2 0.601545535 1.050289769 0.856059267 3 0.766844047 0.987096007 0.778209149 1

Q 0.23924056 1.01696887 0.56369003 2 0.777046046 1.017773185 0.504789326 3 0.434902183 0.962035472 0.302069281 1

R 0.377509946 0.997115518 0.64854093 2 0.455972599 0.981806733 0.597435392 3 0.653783975 0.978092605 0.607139977 1

S 0.279176346 0.96212655 0.184003461 2 0.443041739 0.987786225 0.357679336 3 0.388708828 0.920681532 0.447296195 1

T 0.614268614 0.788989241 0.348690519 2 0.566423093 1.042968944 0.622962743 3 0.594166423 0.986020588 0.750103038 1

U 0.316458375 1.018578383 0.592873136 2 0.699995185 0.980224852 0.522428579 3 0.283854063 0.947891611 0.537982541 1

V 0.388712339 0.999463932 0.475957476 2 0.353005595 1.038886508 0.207975996 3 0.541914043 1.038290468 0.675409346 1

W 0.284140398 0.991891623 0.323244687 2 0.674829928 0.96709455 0.743949467 3 0.245277664 0.956416088 0.468603723 1

X 0.50778879 0.974278695 0.383545168 2 0.438050162 0.964766042 0.325937426 3 0.238883651 0.973954203 0.356599215 1

Y 0.754688641 0.992332266 0.639414084 2 0.499783049 1.016316212 0.451409142 3 0.20638402 1.138533398 0.628735481 1

Z 0.492530806 0.984970597 0.411486878 2 0.353608021 1.066460898 0.186410945 3

6- Classification and Identification Procedures: To classify an unknown person

from its handwritten image, Euclidean distance measure is used to evaluate the differences between the features vector of the unknown handwritten image with the reference features vector sets preserved in the Database. The unknown image then assigned as to belong to certain person depending on the minimum distance obtained from the comparison. Moreover, if two different samples were processed by our system then it is possible to identify if both the samples belong to the same person or not. However, the decision may be taken according to the degree of similarity between the compared handwriting samples. In this case, a threshold value should be decided to differentiate between True or False decision (i.e. True means same person, while False


54

indicating different persons). It is our opinion, from the trained results, that the suitable threshold value for the differentiating is £0.97.

Table 4-5 represents the minimum distance measures between different persons. As it obvious, minimum values existed for the same persons; i.e. the minimum error values refer to higher degree of similarity.

The following algorithm is used to compute the minimum distance for unknown handwritten image:

Let (k) refers to the number of samples stored in the reference database. Let (I=1 to 26) refers to the number of characters in each handwritten image. Let (J=1 to 3) refers to the number of complex moments. LET (A_LH[26], A_HL[26], A_HH[26]) arrays refer to the reference features vector. LET (C_LH[26], C_HL[26], C_HH[26]) arrays refer to the extracted features for unknown

handwritten image. Let (ER_LH, ER_HL, ER_HH, MER, ERROR) are temporary buffers to preserve

minimum distance. BEGIN

WHILE (k<number of samples) DO WHILE (I<26) DO

BEGIN ER_LH=ER_LH+ABS((A_LH[I]–C_LH[K])) ER_HL=ER_HL+ABS((A_HL[I]–C_HL[K])) ER_HH=ER_HH+ABS((A_HH[I]–C_HH[K]))

IF I=1 THEN MER= ER_LH END IF (ER_LH>ER_HL THEN ER_LH=ER_HL IF (ER_LH>ER_HH THEN ER_LH=ER_HH ERROR=ER_LH IF MER < ERROR THEN

ERROR = MER END IF

END.

Table 4-5 represents an example of comparison between handwriting image of the adopted each sample with other samples stored in our database.


55

Table 4-5: Represents minimum distance results

Name of persons

Ali Abd Al munem Ali Hadi Ali Noore Farah Haitham Lamee Manal Rasha Saif Ziyad

Ali Abd Al

munem 0.00000148 0.017323058 0.01166823 0.00609750 0.00411784 0.00800061 0.00441559 0.00622177 0.00736547 0.01291037

Ali Hadi 0.01473033 0.00005411 0.01221734 0.00739674 0.01547233 0.01171158 0.01667246 0.00685768 0.00489167 0.00611468

Ali Noore 0.00892473 0.00746771 0.00000198 0.01293812 0.00725613 0.01296780 0.01407296 0.01405094 0.00330929 0.00792758

Farah 0.000001123 0.01126745 0.01231931 0.00005721 0.01512243 0.01562237 0.00897180 0.00000885 0.00972056 0.00746469

Haitham 0.01700588 0.01609278 0.01756632 0.00626562 0.00000242 0.01478826 0.01093006 0.01139562 0.00467887 0.01551103

Lamee 0.01069786 0.00796898 0.00967685 0.00911468 0.00547535 0.00000207 0.00750255 0.01111641 0.00658764 0.00680312

Manal 0.00848903 0.01243273 0.01053442 0.01182890 0.01209322 0.01179953 0.01676880 0.00342569 0.01013460 0.01304832

Rasha 0.00566603 0.00615498 0.01031481 0.01669577 0.00931119 0.01150859 0.00880202 0.00000157 0.01492894 0.00533851

Saif 0.00959612 0.00980051 0.00582674 0.01356850 0.00336738 0.01200962 0.01137238 0.00939700 0.00010733 0.00655065

Ziyad 0.00807068 0.00936700 0.00431920 0.01335296 0.00988460 0.00618531 0.00706116 0.01267107 0.00708597 0.00000314


56

4-7 Results As it has been mentioned before, set of samples of 40 handwritten images

representing 10 persons have been selected to perform the validity of our designed recognition system. For each person, 4 handwritten images were chosen, 3 of them used as references. Table 4-6 represents the achieved recognition results performed on the adopted handwriting persons.

Table 4-6 : Presents the success or fail of recognition for each classes

Class number ID Class Success

1 Ali Hadi True 2 Ali munem True 3 Ali Noore True 4 Farah False 5 Haithm True 6 Lamee True 7 Manal False 8 Rasha True 9 Saif True 10 Ziyad True

From the above results we may conclude that further improvement procedures are required to enhance the system output.

The following figure 4-10 illustrates how the users can deal with this Handwritten Identification System


57

Figure 4-10 : illustrates this Handwritten Identification System

Chapter Five Conclusions and suggestions for future work

58

Chapter five Conclusions and suggestions for future work

5-1 Conclusions:

The following points are drawn form this work: 1- The complication associated with our present research was that; the proposed

handwriting doesn’t carry inherent biological property as those existed in the finger print, voice print, retina print etc. Therefore, the obtained results of our designed system (i.e as given in the tables) could be regarded as to be well but not perfect. However, the result may be improved if the suggestions given in the future work are considered.

2- The fidelity of the proposed system may be improved if temporal handwriting

images (i.e. images acquired in different times) are used. Temporally, persons may be affected by certain environments circumstances.

3- Since our system selected the minimum normalized moment's distance for

each person, therefore, increasing number of samples would definitely enhance the system's output.

4- The success rate may be improved if certain features in related to the

psychological behavior of the voluntaries are adopted; i.e. instantaneous effects pressure of the person.

5- Extracting the correct character's body from the original image (i.e.

segmentation or isolation processes) has a great effects on the recognition and identification operations. Pin's pressing, character's size, blank's distance between characters…etc. all affecting the results.

6- The trained results showed instability of complex moment's values as

moments order were increased. Therefore, the computation in our designed system adopted lower moments up to 4th order only.

7- The wavelet transform produced sub-images of sizes that can't be suffered

higher than 4th order moments computation. To increase number of adopted moment's order, scanning resolution should be increased to more than 300dpi. However, increasing the wavelet's decomposition levels will slowing the execution time and reduce expected features within the sub-images.

Chapter Five Conclusions and suggestions for future work

59

5-2 Suggestions for future work: Based on the obtained results, the following suggestions may contribute in

improving the system's output: 1- The implemented wavelet transform may be replaced by other type of wavelet

transformation that has compaction results better than the Daubechies; e.g. Tap -4 or Tap-6 versions.

2- Multi types of wavelet may be performed in each decomposition would increase the system sensitivity and may improve the results.

3- Other types of the moments may be adopted to test the similarity between compared samples; e.g. zernik moment which has proven to be superior to other moment functions in terms of feature representation capabilities .

4- The proposed system may be implemented on different patterns; e.g. signature, voice print, etc.

5- Extraction or isolation processes may be performed by adopting certain adequate edge detection technique.

م. تصميم نظام ل�ه ق�درة التع�رف عل�ى األش�خاص م�ن خ�الل الخ�ط الي�دوي لك�ل م�نه في بحثنا هذا سنتناول

لتحلي�ل الخ�ط الي�دوي ألش�خاص مح�ددين ،إض�افة wavelet transformحيث يستخدم التحوي�ل الم�وجي

لتحديد خصائص التمييز بين األشخاص المختلفين. Complex moments لذلك يستخدم الـ

ل ص�ور تمث�ل الخ�ط الي�دوي لك�ل ش�خص، فصل بين األحرف المدخل�ة م�ن خ�الأجراء عملية تطلب يهذا و

"A"الكبيرة للغة اإلنكليزي�ة م�ن (مثال األحرف ألنه الخط اليدوي للشخص عبارة عن سلسلة من األحرف

.)"Z" إلى

1st order decomposition level of theنقتصر عل�ى اس�تخدام المس�توى األول للتحوي�ل الم�وجي س

wavelet transform.يعتمد هذا االستخدام لغرض إنجاز عملية التمييز والتعرف بشكل سريع ودقيق .

عين��ة لعش��رة متط��وعين، وبع��ض ه��ذه العين��ات س��وف تس��تخدم ألج��راء ٤۰ختب��ار عل��ى وس��يجرى ه��ذا اال

اختبارات التحقق من األشخاص.

Euclidianوف�ق مس�افة اقلي�دس minimum distance classifierيستخدم معي�ار المس�افة الص�غرى

distance .ث��الث عين��ات لنم��اذج الخ��ط الي��دوي لك��ل ش��خص ف��ي فح��ص التش��ابه ب��ين العين��ات المتطابق��ة

.reference databaseالمخزونة في قاعدة البيانات معلوماتكمرجع للستستخدم

س�وف يحس�ب ويس�تخدم كأن�ه (LL,LH,HL,HH)ألج�زاء الص�ور األربع�ة complex momentsال�ـ

للمقارنة في عملية التمييز والتعرف. featured vectorمتجه خصائص

ألنحاز العمليات المطلوبة التي يتض�منها ه�ذا النظ�ام Visual Basic Ver. 6هذا النظام يستخدم لغة الـ

المصمم.

الخالصة

66

References

1 R. C. Gonzalez and J. T. Tou, Pattern Recognition Principles, Addison-Wesley Publishing Company,1974.

2 Mantas, J. ,"Methodologies in Pattern Recognition and Image Analysis-A Brief Survey", Pattern Recognition, Vol. 20, No. 1, PP. 1-6, 1987.

3 "Introduction to Pattern Recognition" http://www.display-system.tuc.gr/courses/pr/pr-contents.pdf

4 Eric A. Fisch, Gregory B. White, Secure Computers and Networks: Analysis, Design, and Implementation, Publisher: Boca Rton, Fla: CRC Press, 2000.

5 Baltzakis, H. Papamarkos, N., "A New Signature Verification Technique Based on A Two-Stage Neural Network", Engineering Application of Artificial Intelligence, Vol. 14,PP. 95-103,2001.

6 Palamondon, R., Lortte, G., "Automatic Signature Verification and Writer Identification- The State of Art", Pattern Recognition, Vol. 22, No. 2,PP.107-131, 1989.

7

Martineze-R,L. Aleantara-S, R., "On-Line Signature Verification Based on Optical Feature Representation and Neural-Network-Driven Fuzzy Reasoning", Institute of Technology of Aguasclients, Mexico, November 20,2002.

8 Zimmermann, K. P., Varandy, M. J., "Hand Writer Identification From One-bit Quantized", Pattern Recognition, Vol. 18, PP.63-72,1985.

9 Hale, W. J., Paganini, B. J., "An Automatic Personal System Based on Signature Writing Habits", Proc. 1980 Carnahan Conf. on Crime Countermeasures, PP. 121-125, Univ. of Kentucky, Lexington, KY (1980).

10 Samir Al-Emami and Mike Usher, "On-Line Recognition of Handwritten Arabic Characters", IEEE Transaction on Pattern Analysis and Machine Intelligence, Vol. 12, No. 7, PP. 704-710, 1990.

11 Murib, A. W., "Image Recognition using Wavelet Transform", M.S.C., Thesis, University of Baghdad Electrical Engineering Dept., 1999.

12 I. Daubechies, Ten Lectures On Wavelets , © 1992 By Siam Press.

13 Kadhim, A., "Signature Recognition Using Complex Moments", M.S.C.. Thesis, University Of Baghdad, 1997.

14 S. A. Dudani, K. J. Breeding, and R. B. Mcghee, Aircraft Identification By Moment Invariants, IEEE Trans. Comput. C-26, No. 1, PP. 39-45, Jan. 1977.

15 M. A. Ismail and Samia Gad, "Off-Line Arabic Signature Recognition and Verification", Pattern Recognition, Vol. 33, Pp. 1727-1740,2000.

16 Abd Alrazak, Raghad S., "Signature Recognition using Wavelet Transform', Ph. D. ,Thesis, University of Technology,2004.

17 Steven W. Smith , The Scientist and Engineer's Guide to Digital Signal Processing, Second Edition, California Technical Publishing, 1999.

67

18 Scott E Umbaugh, Ph. D., Computer Vision and Image Processing: A Practical Approach Using Cviptools, Prentice Hall PTR,1998.

19 Bjorn Jawerth and Wim Sweldens, “An Overview Of Wavelet Based Multiresolution Analyses”, http://cm.bell-labs.com/who/wim/papers/overview.pdf

20 Bruce, Donoho, and Gao., “TRANSFORMS / WAVELETS”, IEEE Spectrum Article, Wavelet Analysis ,October 1996, http://www.tscm.com/wavelet.pdf#search='transforms%20/%20wavelets'

21 “Wavelet Analysis: Theory & Implementation”, http://www.cs.ualberta.ca/~minghong/wavelet.doc

22

Y. Oussar, I. Rivals, L. Personnaz, G. Dreyfus, ”Training Wavelet Networks for Nonlinear Dynamic Input-Output Modeling”. http://www.neurones.espci.fr/francais.docs/dossier_recherche /articles_ps/ondelettes.pdf

23 Ali Mohamed Al-Aofy, "Color Image Compression using Harr Wavelet Transform", M.S.C., Thesis ,The Institute for Higher Studies in Informatics,2003.

24 Wisam H. Fatheel, "Texture Analysis and Classification by using Wavelet Transform", M.S.C., Thesis, Baghdad University, 2002.

25 Rafael C. Gonzalez & Richard E. Woods, Digital Image Processing, Addison-Wesley, Fifth Indian Reprint,2000.

26 Ian Kaplan , “Applying the Haar Wavelet Transform to Time Series Information”, July 2001-2004, http://www.bearcave.com/misl/misl_tech/wavelets/haar.html#introduction

27 Robi Polikar ,”The Wavelet Tutorial”, Part Iii, 1996, Iowa State University, U.S.A., http://users.rowan.edu/~polikar/wavelets/wttutorial.html

28 C. Valens, "A Really Friendly Guide to Wavelets",1999-2004, http://perso.wanadoo.fr/polyvalens/clemens/wavelets/wavelets.html, http://perso.wanadoo.fr/polyvalens/clemens/download/arfgtw_26022004.pdf

29 Yousef M. Hawwar, Ali M. Reza, Robert D. Turney, “Filtering (Denoising) in the Wavelet Transform Domain”, University of Wisconsin-Milwaukee, http://www.xilinx.com/products/logicore/dsp/denoise_wavelet.pdf

30 Y. Lee and K. W. Hwang, "Selecting Good Speech Features for Recognition", ETRI Journal, Vol. 18, No. 1, April 1996.

31 S. Mallat, "Wavelets for A Vision", Proceeding of IEEE, Vol. 84, No. 4, April 1996.

32 E. C. Ifeachor, B. W. Jervis, Digital Signal Processing: A Practical Approach, Addison – Wesley, 1996.

33 Riyadh Fakher Khalaf Al-Hashimi, ““TTeexxtt –– IInnddeeppeennddeenntt SSppeeaakkeerr RReeccooggnniittiioonn uussiinngg WWaavveelleett aanndd NNeeuurraall NNeettwwoorrkkss””, M.S.C., Thesis, University of Baghdad,2002.

68

34 Kristian Sandberg ,”The Haar Wavelet Transform”, University of Colorado at Boulder,2000, http://amath.colorado.edu/courses/4720/2000spr/labs/haar/haar.html

35 Emil Mikulic, “Haar Wavelet Transform”, 2004 http://Dmr.Ath.Cx/Gfx/Haar/

36 Daubechies, I. Communications on Pure and Applied Mathematics, Vol. 41, PP. 909–996,1988.

37 Ian Kaplan, “The Daubechies D4 Wavelet Transform”,2002, http://www.bearcave.com/software/java/wavelets/daubechies/index.html

38 M. Hu, "Visual Pattern Recognition by Moment Invariants", IRE, Transactions on Information Theory, Vol. It-8, PP. 179-87, Feb. 1962.

39 Jurgen. A. and Herbert J. P. R., “A Fast Correlation Method for scale-and Translation-Invariant Pattern Recognition”, IEEE Trans. on Pattern Anal. Mac Intell. Vol. PAMI-6,No. 1, PP.46-57, January 1984.

40 L. A. Goshorn, “Vision System Eye Real-Time Speeds Through Multiprocessor Architectures”, Electronics Vol. 56,No. 25, PP. 137-140, 1983.

41 G. L. Cash and M. Hatamian, “Optical Character Recognition by the Method of Moments”, Computer Vision, Graphics, and Image Processing No. 39, PP. 291-310, 1987.

42 R. G. Casey, “Moment Normalization of Hand Printed Characters”, IBM J. Res Development. 14, PP. 548-557, Sep. 1970.

43

Richard J. Prokop and Anthony P. Reeves, “A Survey of Moment – Based Techniques for Unoccluded Object Representation and Recognition”, Graphical Models and Image Processing, Vol. 54, No. 5, September, PP. 438 - 460, 1992.

44 Yhoi - Yulkin, Binary Normalization for scale invariance with Zernike Moments, IEEE Transaction, 1992.

45 S. O. Belkasim, “Image Description using Moments”, http://pami.uwaterloo.ca/mkamel/sd776/notes/note2.pdf#search='dr.%20s.%20belkasim'

46 J. Sklansky, " Pattern Recognition: Introduction and Foundation", Publisher Dowden, Hutehinson & Ros,1973.

47 C. Y. Suen, M. Berthod and S. Mori. "Automatic Recognition of Handprinted Characters- The Stat of Art", Proceeding of the IEEE, 68(4),PP. 469-487, 1980.

48 S. O. Belkasim, M. Shridhar, and M. Ahmadi, "Pattern Recognition with Moment Invariants: A Comparative Study and Results", Pattern Recognition Society, Vol. 24, No. 12, PP.1117-1138, 1991.

49 Tien C. Hsia, "A Note on Invariant Moments in Image Processing", IEEE Trans. on Sys. Man, Sybern, Vol. SMC-11,No.12, PP. 831-834, Dec. 1981.

50 Yajun Li, "Reforming the Theory of Invariant Moments for Pattern Recognition', Pattern Recognition Society, Vol. 25, No. 7, PP. 723-930, 1992.

51 Anil K. Jain, Fundamentals of Image Processing, Prentice Hall, Inc. 1989.

69

52 Yajun Li, "A New Theory of Invariant Moments for Pattern Recognition”, IEEE Transaction on Pattern Recognition and Machine Intelligence, July 1991.

53

Anthony P. Reeers,R. J. Prop,Siisan E. anderews, and Frank P. Kuhl, "Three Dimensional Shape Analysis using Moments and Fourier Description", IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 10,No. 6, November 1980.

54 Michael Reed Teague, "Image Analysis Via the general Theory of Moments", J. Opt. Soc. Amer., Vol. 70, PP. 920-930, Aug. 1980.

55 Cho-Huak The and R. Chin, "On Image Analysis by the Methods of Moments", IEEE Transaction on Pattern Recognition and Machine Intelligence, Vol. 10, No. 4, PP. 496-512, July 1988.

56 Abo-Mostafa, Y. S., Psaltis, D., "Recognition Aspects of Moment Invariants", IEEE Transaction on Pattern Analysis and Machine Intelligence, Vol. PAMI-6,No. 6, PP. 698-706, November 1984.

57 Abo-Mostafa, Y. S., Psaltis, D., " Image Normalization by Complex Moments ", IEEE Transaction on Pattern Analysis and Machine Intelligence, Vol. 1,PAMI-7,No. 1,PP. 46-55, January 1985.

58 "Introduction to Statistical Pattern Recognition". http://medianwiley.com/product-data/experst/39/04708451/0470845139.pdf

59 T. Pavlidis, Algorithms for Graphics and Image Processing, Berlin-Heidelbery, Springer-Verlage, 1982.

60 Ingrid Daubechies and Wim Sweldens, “Factoring Wavelet Transforms into Lifting Steps”, Sept. 1996.

61 ISO and ITU, "Information Technology-JPGE 2000 Image Coding System", JPGE 2000 Final Committee Draft Ver. 1.0, 16 March 2000. http://www.newauto.com/xatb/241/24.html

تقنية التّعريف على الخط اليدوي باالعتماد على التحويل الموجي

رسالة مقدمة إلى كلية العلوم / جامعة بغداد

علوم في وهي جزء من متطلبات نيل درجة ماجستير الحاسبات

من قبل علي هادي حسين

بأشراف أ. د. صالح مهدي علي

م۲۰۰٦شباط / هـ ۱٤۲۷محرم /

handwritten identification technique based on wavelet transform science/ms.c/… · ·...

Documents