LAPPEENRANTA UNIVERSITY OF TECHNOLOGY

DEPARTMENT OF INFORMATION TECHNOLOGY

AUTOMATED HELIOTEST INSPECTION USING

MACHINE VISION

The topic of the master’s thesis has been accepted in the departmental council of

Department of Information Technology, November 26, 2003.

Supervisors: Professor Heikki Kälviäinen, Dr. Joni-Kristian Kämäräinen

Lappeenranta, December 17th, 2003

Albert Sadovnikov

Karankokatu 4 A 18

53810 Lappeenranta

Tel. 0 468 109 468

Albert.Sadovnikov@lut.fi

ABSTRACT

Lappeenranta University of TechnologyDepartment of Information TechnologyAlbert Sadovnikov

Automated Heliotest inspection using machine vision

Thesis for the Degree of Master of Science in Technology200372 pages, 19 figures, 8 tables and 1 appendix.Examiners: Professor Heikki Kälviäinen, Dr. Joni-Kristian KämäräinenKeywords: Heliotest, visual quality inspection, paper printability, paper making,machine vision, image processing

The main subject of this master’s thesis was to develop methods for automatedHeliotest inspection using machine vision techniques. Heliotest is one of paperprintability tests and it measures ability of paper to reproduce halftones in ro-togravure printing. Test print result consists of three differently evaluated areasand the most interesting area in this thesis is a dotted strip printed in the middle ofa tested paper sample. Quality measurement is made by observing the number ofmissing dots from the dotted strip. The size of a printed dot is small and difficultto evaluate by human experts which is, however, the current practice. Thus, anautomatic method would have an important practical impact.

In this work two frequency domain filtering methods and one model based spa-tial feature extraction approach are presented. All the proposed methods exploitregularity properties of the dot pattern structure.

In conducted experiments on real data all methods performed well. The presentedautomated Heliotest evaluation methods seem to be on a solid theoretical founda-tion and based on the practical results it seems possible to perform the Heliotestevaluation automatically in industry.

ii

TIIVISTELMÄ

Lappeenrannan teknillinen yliopistoTietotekniikan osastoAlbert Sadovnikov

Automated Heliotest inspection using machine visionDiplomityö200372 sivua, 19 kuvaa, 8 taulukkoa ja 1 liite.Tarkastajat: Professori Heikki Kälviäinen, TkT Joni-Kristian KämäräinenHakusanat: Heliotest, visuaalinen laadunvalvonta, paperin painettavuus, paperin-valmistus, konenäkö, kuvankäsittelyKeywords: Heliotest, visual quality inspection, paper printability, paper making,machine vision, image processing

Tässä työssä käsitellään konenäkömenetelmiä automaattisen Heliotest-mittauksensuorittamiseksi. Heliotest-mittaus on yksi paperin painettavuutta mittaavaa testija se mittaa erityisesti paperin kykyä tuottaa syväpainettuja rasterikuvia. Testiävarten tehtävä testipainatus sisältää kolme eri tavalla mitattua aluetta ja tässä työssäkeskitytään rasteripisteisiin, jotka ovat keskimmäisellä alueella paperinäytteessä.Painatuksen laatua mitataan tutkimalla alueelta puuttuvien rasteripisteiden määrääpainokuviossa. Rasteripisteiden koko on hyvin pieni ja niiden manuaalinen tutkimi-nen on hankalaa, mutta kuitenkin yleisin teollisuudessa käytetty menetelmä. Tämäntyön tarkastelun kohteena olevalla menetelmällä tulisi toimiessaan olemaan huo-mattava käytännöllinen merkitys.

Tässä työssä esitellään kaksi taajuutasoon perustuvaa menetelmää ja yksi spati-aalisessa tasossa olevaan malliin perustuva menetelmä. Kaikki kolme menetelmäähyödyntävät rasteripisteiden muodostamaa toistuvaa rakennetta.

Suoritetuissa kokeissa todellisella kuva-aineistolla kaikki esitetyt menetelmät me-nestyivät hyvin. Käytetty teoreettinen perusta näyttää olevan oikea ongelmanratkaisemisen kannalta ja käytännön tuloksien perusteella Heliotestin automati-soiminen teollisuudessa näyttää mahdolliselta.

iii

Preface

This master’s thesis was done in Lappeenranta University of Technology in De-

partment of Information Technology during the summer and the autumn semester

of 2003. The thesis was part of a larger project called "PAPVISION - Paper and

board printability tests using machine vision" which has been done in Informa-

tion Processing Laboratory of Lappeenranta University of Technology. Espe-

cially I would like to thank European Union and National Technology Agency of

Finland (TEKES) for financial support in the Papvision project (TEKES Project

No. 70049/03). Moreover, industrial partners UPM-Kymmene, Stora Enso, Myl-

lykoski Paper, Metso Paper, Future Printing Center (FPC: Raisio Chemicals, Hansa-

print, and Omya), and Labvision Technologies are greatly acknowledged for their

scientific and financial support.

I would like to thank my supervisors professor Heikki Kälviäinen and Joni-Kristian

Kämäräinen for support and overall encouragement. My project colleagues Pasi

Saarinen, Lasse Lensu, Jarkko Vartiainen, Petja Salmela and Alexander Drobchenko

also receive my sincere gratitude for information and help in various project as-

pects.

Special thanks to all the people who made the IMPIT program possible, particu-

lary professor Jan Voracek and Nina Kontro-Vesivalo. Whithout the existence of

the IMPIT program nothing could have been done.

This work could not be possible without backing from family, friends and many

other people and I express my thankfulness to them.

Lappeenranta

iv

Contents

1 Introduction 4

1.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

1.2 Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

1.3 Structure of the thesis . . . . . . . . . . . . . . . . . . . . . . . . 5

2 Paper printabiliy 6

2.1 Printing papers . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

2.2 Printability tests . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

2.3 Heliotest . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

3 Machine vision methods 13

3.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

3.2 Image acquisition and enhancement . . . . . . . . . . . . . . . . 16

3.3 Feature extraction . . . . . . . . . . . . . . . . . . . . . . . . . . 18

3.3.1 General feature picture . . . . . . . . . . . . . . . . . . . 18

3.3.2 Fourier domain filtering . . . . . . . . . . . . . . . . . . 19

3.3.3 Filtering using discrete cosine transform . . . . . . . . . . 23

3.3.4 Gabor filters . . . . . . . . . . . . . . . . . . . . . . . . 28

3.3.5 Pattern modelling . . . . . . . . . . . . . . . . . . . . . 29

3.4 Classification . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

3.4.1 Image segmentation . . . . . . . . . . . . . . . . . . . . 32

3.4.2 Spatial features and Gabor features . . . . . . . . . . . . 32

4 Automated Heliotest inspection 34

4.1 Common to all methods . . . . . . . . . . . . . . . . . . . . . . . 34

4.2 FFT based method . . . . . . . . . . . . . . . . . . . . . . . . . 36

4.2.1 Preprocessing . . . . . . . . . . . . . . . . . . . . . . . . 36

4.2.2 Extraction of the irregular component . . . . . . . . . . . 37

4.2.3 Classification . . . . . . . . . . . . . . . . . . . . . . . . 38

4.3 DCT based method . . . . . . . . . . . . . . . . . . . . . . . . . 39

4.4 Pattern modelling with spatial features method . . . . . . . . . . . 40

4.4.1 Grid extraction . . . . . . . . . . . . . . . . . . . . . . . 40

1

4.4.2 Classification . . . . . . . . . . . . . . . . . . . . . . . . 42

5 Experiments and results 43

5.1 Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

5.1.1 Heliotest samples . . . . . . . . . . . . . . . . . . . . . . 43

5.1.2 Training and testing data . . . . . . . . . . . . . . . . . . 43

5.2 Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44

5.2.1 FFT based method . . . . . . . . . . . . . . . . . . . . . 45

5.2.2 DCT based method . . . . . . . . . . . . . . . . . . . . . 46

5.2.3 Pattern modelling with spatial features method . . . . . . 48

5.3 Results and discussions . . . . . . . . . . . . . . . . . . . . . . . 49

6 Conclusion 50

2

List of Symbols and Abbreviations

F{·} Fourier transform

F−1{·} inverse Fourier transform

H image filtering operator

M(x, y) mask function

µ image mean value

τ threshold value

ψ(x, y) 2-d Gabor filter in spatial domain

γ sharpness of Gabor filter along major axis

η sharpness of Gabor filter perpendicular to major axis

θ orientation of Gabor filter

χ characteristic function

f frequency of Gabor filter

I(x, y) unit function

rξ(x, y; f, θ) response of 2-d Gabor filter

S minimal area value

T{·} threshold operator

ξ(x, y) 2-d image in spatial domain

Ξ(u, v) 2-d image in frequency domain

u first frequency coordinate

v second frequency coordinate

x first spatial coordinate

y second spatial coordinate

2-d two dimensional

DCT discrete cosine transform

DFT discrete Fourier transform

FFT fast Fourier transform

MV machine vision

PCA principal component analysis

ROI region of interest

3

1 Introduction

1.1 Background

The paper making technology has a very long history ongoing from the distant

past. During the last 150 years it has been developed greatly and has arrived

at significant progress nowadays. In the 19th century Keller invented the wood

pulp and a little bit later the cellulose treatment process was made-up which has

allowed technology to be developed up to today’s state. The paper is the ma-

jor polygraphic material and meets all technological, consumer and economic re-

quirements showed to such materials.

There is a research project, called "PAPVISION - Paper and board printability

tests using machine vision" [1] in Lappeenranta University of Technology. The

goal of this project is to provide new innovative solutions for paper and printing

industry and to establish top knowledge on image processing in this industry. The

project gathers information about problems which can be solved using machine vi-

sion and image processing techniques and implements automatic applications for

the use of industrial partners. The project is carried out in intensive co-operation

with the industrial partners. The main objective is to serve industrial partners as

a main research resource and organization for machine vision and other scientific

computation solutions. [1]

Heliotest is one of tests which are intended to measure paper printability in lab-

oratory conditions. Printablility is the quality of paper in terms of its behavior

with ink, water and other liquids, as well as drying in the printing process. It is

affected by factors such as fiber furnish, coatings and surface finishing [2]. He-

liotest shows how good printing quality is, i.e., how many dots should have been

printed to halftone image, but due to some reasons were not printed. This partic-

ular work is intended to summarize and propose new approaches and methods for

automated Heliotest inspection.

4

1.2 Objectives

The objective of this master’s thesis is to develop a method or several methods

for automated Heliotest inspection. The developed method is not intended to be

’working in all cases’, though the topic should be researched for possible lim-

itations/circumstances in which the developed method should work at least not

worse comparing to if the test was conducted by a human expert.

1.3 Structure of the thesis

The thesis focuses on the measurement problem of paper printability, particulary

Heliotest. It starts with the paper printablility measurement problem raising and

describing it together with existing Heliotest methodology, and proposes several

new approaches based on machine vision and image processing methods.

The structure of the thesis is following. Section 2 is an introduction to a gen-

eral problem state. It includes a survey on paper printability, paper printability

testing and thorough Heliotest description. Section 3 deals with machine vision

methods for Heliotest inspection. This part contains several approaches on image

pattern removal and further classification methods. Section 4 presents algorithm

design using methods from the Section 3. It focuses on the implementation de-

tails describing practical properties omitted in Section 3. Section 5 represents

real experiments. In this part methods are tested and statistical data is collected,

also comparison of the methods is conducted. Section 6 finishes the thesis with

a review of key points, conclusions on performed research and propositions on

possible work to be undertaken in the future.

5

2 Paper printabiliy

2.1 Printing papers

Printed paper in its different manifestations is familiar to and used by everybody.

Products truly represent the much sought-after every citizen interface to infor-

mation. The time spent each day browsing printer matter for work, education,

information and leisure is considerable. Needs fulfilled are diverse, as are also

the user situations. In other words, printed matter is an essential part of the fabric

of our daily lives. Print production technologies have changed immensly during

the last forty years, more so than during the preceding four hundred years. This

is mainly due to the digitalization of the basic raw materials of prints, namely in-

formation. Previous progress was made largely through mechanisation. However,

during this time the appearance of printed matter has not changed radically.[3]

The term "paper grade" denotes papers that share the most relevant criteria. The

main criteria used to classify printing papers include following matters [3]:

• Fiber furnish of the base of paper.

• Coating.

• Type of surface finish.

• End use.

2.2 Printability tests

Printability of paper is a paper property that tells how paper behaves during a

printing process. Printability depends on interactions between paper and printing

ink and printing process variables. Good printability of paper means that the paper

6

is not very sensitive to variations in different process variables and easily gives

good print quality. Print quality describes the final result of printing or the quality

of the printed image. Print quality definition does not have absolute terms. It

depends on the print density, resolution, and eveness of the printed image. Many

other properties are also of interest. [4]

Printability tests depend on the printing process involved. Certain types of test

methods aim at testing general print quality. These tests are useful for all types

of papers regardless of the printing methods used. Some test methods are only

suitable for a certain printing process.

Printability and print quality depend on many different factors. The paper and

its properties are important components as is the printing process with its many

variables. Due to the complex nature of the printing process and different interac-

tions influencing it, testing of printabiliy and print quality can consist of different

techniques.

Papermakers and printers have searched for a fast, simple, and effective test meth-

ods to predict paper behavior in printing without actually printing the paper. Every

printing process has its own features that often require a separate testing device

and a specific test. In many cases, measuring similar properties after different pro-

cesses is possible. Table 1 lists laboratory printing tests commonly used. Some

tests may be used with all the listed printing methods, and some are more specific

to a certain printing process [4]. As it can be seen from Table 1, missing dots

testing (Heliotest) is performed only for rotogravure printing.

Rotogravure printing is an intaglio process meaning engraved or carved. The im-

pressions are achieved by the transfer of ink from cells or depressions of varying

depths, etched into the print cylinder to a web of paper at high speeds. The pro-

cess of printing involves coating the etched cylinder into an enclosed fountain or

trough of ink and the etched cells are filled with ink. While the cells fill with

ink, the surface of the cylinder (non-image area) also becomes coated with ink.

This non-wanted ink is removed by a doctor blade or knife which wipes all of

7

the surface ink from the cylinder. The printing cylinder comes in contact with

the paper and the ink which remains in the cells is transferred to the paper. High

cylinder cost generally limits gravure to run lengths of over 1 millions impres-

sions and thus, gravure is a long run process. Gravure presses are also much

wider than other printing type presses. Unlike letterpress or offset, the ink used is

very fluid and is usually solvent based which in today’s environment is undesir-

able. Typical rotogravure printed products would include packaging, catalogs and

magazines. [5]

Table 1: Laboratory test methods for predicting the printability and print qualityin different printing methods. [4]

Offset Digital Rotogravure FlexoPrinting Printing Printing Printing

Coldset Heatset Sheet Ink-jet LaserPrint density x x x x x x xInk requirement x x x x xInk set-off x x x xRub-off x xPrint through x x xInk Gloss x x x x xDry pick x xWet pick x xBlack trap pick x xMottle x xFiber roughening x x xToner adhesion xMissing dots xResidual solvent x xDot gain x x x x x xDots geometry x x x x x

8

2.3 Heliotest

The greatest problems in rotogravure printing involve reproduction of light and

medium tones. Two different types of defects are missing dots and waving. Miss-

ing dot defect means that the ink was not transferred to the paper while printing

due to the bad quality of that paper area, which did not allow ink to glue to the

paper. Missing dots are inevitable at 5% half tone but disastrous when occuring

at 20% and 30% half tones [4]. Literally, half tone means the tone which is in

between the paper color and the ink color. Needed half tone reproduction can be

achieved by transferring a certain amount of ink. So in dark areas (close to ink

color) missing dots are more fatal. The number of missing dots in a certain area is

a traditional measure of rotogravure printablity of a paper. In laboratory printing,

Heliotest indicates the number of missing dots. The Heliotest fitting, developed

by Centre Technique du Papier [6] is used with an IGT AC2 laboratory printer [7]

(see Figure 1).

Printing disk Doctoring system

Paper sample Gravure ink

Figure 1: Heliotest accessory.

The Heliotest attachment consists of an engraved printing disk, a doctoring system

9

and a special, not quick drying gravure ink. Some drops of the ink are put on the

printing disk, the surplus of ink is wiped off and a print is made on the substrate

which has been attached to the sector. The disk contains three types of engraving

as follows:

• Halftone screen area (see Figure 2): Disk carves in this part are the same

in size, but their depth changes from the beginning till the end in order

to reproduce less intensive halftones. More deep engraves produce larger

dots and less deep ones produce smaller. In this area the distance from

the beginning of the print till the twentieth missing dot is measured. The

longer the distance, the smoother the paper. This part of the disk is the most

important part.

• A conventional screen area: See dark rectangles on the top and bottom sides

of the Heliotest print result in Figure 2. This part is used for visual assess-

ment and is of low importance.

• Two lines of dots on the both sides of the halftone screen area (see Figure

2): In these lines the total number of missing dots is counted. This is done in

case the distance measured in the variable halftone screen area is too small,

that is when the paper is very rough. The more missing dots, the rougher

the paper.

After printing the test strips shown in Figure 2, an expert should make an as-

sessment of it, which is made according to IGT recommendation leaflet in the

following way [8]:

1. Starting from the side with the largest dots in the variable screen count the

missing dots until the 20th one.

2. Measure the distance in millimeters from the 20th missing dot to the begin-

ning of the variable screen area.

10

3. In the case the distance in Step 2 is very small (only some millimeters) count

the total number of missing dots in the four dotted lines on both sides of the

variable half tone.

4. Repeat Steps 1 and 2 or 3 for each test strip.

5. Compute the average and if required the standard deviation. In some cases

it may be useful to mention the highest and the lowest value as well.

6. If needed assess the printing quality visually in the conventional screen area.

11

Halftone screen area

Missing dots

Lines of dots

Conventional Screen Area

Figure 2: Detailed description of Heliotest print result.

12

3 Machine vision methods

3.1 Background

Machine vision (MV) is the ability of a computer to "see". Combining high speed,

high definition cameras and powerful embedded processing engines, machine vi-

sion has become a crucial part of production processes in many industries. With

the growing of personal computer power, new applications for machine vision

techniques arise [9]. MV is used in various industrial and medical applications for

example as follows [10]:

• Electronic component analysis.

• Signature identification.

• Optical character recognition.

• Handwriting recognition.

• Materials inspection.

• Currency inspection.

• Medical image analysis.

Among such applications is computer-based reading of graphical test results. Au-

tomated visual inspection of industrial materials such as textile, paper, plastic,

and, in particular, automated test print evaluation require adaptive solutions that

can be executed in real time. Currently, the quality assurance of paper printablity

is mainly carried out by manual inspection. However, manual inspection is labor

intensive and insufficient. Advantages of using a machine vision system in paper

printability testing can be defined as follows [11]:

• Less deviations in measurements.

13

• Better measurement accuracy.

• New printability parameters.

• Shorter measurement times.

• Less manpower to monotonic measurements.

• Many quality parameters by one system.

• Automatic data transfer to mill level information systems.

Therefore, automation of visual inspection tasks can increase the efficiency of

production lines and improve quality as well.

Standard machine vision routine includes several major stages as illustrated in

Figure 3. The most interesting step in this routine is feature extraction. Therefore,

quite a large part of the thesis work is devoted to this particular topic. Printed

Heliotest strip shown in Figure 4 can be considered as one of a regular texture.

It is clearly seen in Figure 4 that in the given local part, sizes of dots, angles and

distances remain nearly constant. This structure regularity allows to consider the

given problem as a particular case of a larger problem class, called defect de-

tection in textured materials [12, 13, 14, 15]. This problem domain is relatively

novel, and has not been studied thoroughly enough. However, there are a number

of general approaches which have been applied in this field (see Table 2).

Image

Aquisition

Image

Enhancement

Feature

Extraction Classification

Results

Output

Figure 3: Standard machine vision routine.

Textures are frequently seen in ordinary life and it is a common practice to think

of images as of textures. When one speaks about a texture, a textile fabric is often

14

associated. Hence, everybody has an idea of the term texture, but at the mean

time it is hard to define it precisely, and the definition is unfortunately ambigu-

ous. Nevertheless, there have been several attempts to define the term texture for

image processing needs. For example, in [16] the definition of texture is given

where texture is defined as "the feature, which describes spatial ordering of pixel

intensity in a region." Another definition given in [17] emphasizes structural prop-

erties of the texture. According to this definition "the term texture generally refers

to repetition of basic texture elements, called texels. Their placement can be pe-

riodic, quasi-periodic, or random." Image texture can be defined by the number

and types of its primitives together with the spatial organization or layout of those

primitives. The spatial organization may be random, may contain a pairwise de-

pendence of one primitive on a neighbouring primitive. The dependence itself

may be structural, probabilistic or functional, e.g., a linear relationship. On the

other hand, the notation of texture should admit to not rigid description.

Figure 4: Part of halftone screen area.

According to the texture regularity of Heliotest halftone screen area, the most

promising methods are, probably, those that include spectral analysis idea. In this

particular work three frequency domain approaches are proposed: using discrete

Fourier transform, using discrete cosine transform, and using Gabor filters. The

fourth model-based method presented also uses spectral analysis techniques.

15

Table 2: Methods for the detection of defects in textured materials. [12]

Approach Method1. Fractal dimension2. Bi-level thresholding3. Gray level statistics4. Morphological operations

Statistical 5. Edge detection6. Normalized cross-correlation7. Co-occurrence matrix8. Eigenfilters9. Local linear transforms10. Rank-order functions1. Discrete Fourier transform2. Optical Fourier transform3. Windowed Fourier transform

Spectral 4. Gabor filter5. Real Gabor function6. Wavelet transform7. Wavelet packets8. Wigner distribution1. Gauss Markov random field

Model-based 2. Poissonian model3. Model-based clustering

3.2 Image acquisition and enhancement

In the common machine vision routine, image acquisition is not the major part,

but it plays crucial role in the whole process. To find an appropriate acquisition

device one must think about several parameters, to make the whole method fast,

robust, and easy to use, which are defined as follows:

1. Minimal resolution needed to operate. This one is the base value for the

whole algorithm.

2. Color information needed. This is the parameter which tells how many color

halftones needed for optimal performance.

16

3. Ease of use. This one is quite simple, since for example using flatbed scan-

ner is less complicated than using an electronic microscope.

4. Price. If there are several devices with approximately same characteristics

described above, then cheapest one is, of course, more attractive.

Generally, resolution is measured in the number of dots per inch. While inves-

tigating the problem, it was found that with resolution of 600dpi, the width of a

dot on the halftone screen area is approximately 4 pixels. Presented methods can

work with this resolution quite well. However, further decreasing of resolution

seems not to be possible due to already small size of the dot.

Color information for that type of image is not necessary, and can be easily

changed to the grey-level. By losing color information during acquisition it is

possible to make the process faster and less memory dependent.

In the most machine vision laboratories there are two main acquisition devices.

First one is a digital camera, and second one is a flatbed scanner. The camera is

a very fast device, but however, it lacks resolution to take all the image in one

time with the given resolution (600dpi). This problem can be solved by taking

the image part by part, and processing separately. The flatbed scanner is a slow

device, but by losing time it is possible to win resolution needed. In the project

case, the flatbed scanner has been chosen because of the ease of use. Of course,

the price comparison of a flatbed scanner and a digital camera gives a significant

advantage to a scanner.

Since the project has an industrial goal, it is worth to design a special device

for reading test prints. From the thesis part, two parameters for such device are

defined: minimum resolution (600dpi) and grayscale color information (8bits).

Image enhancement is a subject of further research, since it seems clear that the

input image should be somehow cleared and standartised to fit processing needs.

For example, median filtering [18] can be used to remove minor noise from the

image.

17

3.3 Feature extraction

3.3.1 General feature picture

As it was mentioned before, feature extraction is the most interesting part in image

processing, from a scientific point of view. Properly extracted features are the base

of correct classification.

Let us consider Heliotest halftone screen area image as a 2-d functionξ(x, y)

which value in the given point(x, y) is a pixel intensity.{(xd, yd)} is a set of

missing dots centroid coordintes. LetH is an ideally designed image filter, then

the response of this filter should be

H{ξ(x, y)} = χ({(xd, yd)})

whereχ is a characteristic function of the missing dots set. However, filter re-

sponse for a certain image pixel is a valueH{ξ(x0, y0)} ∈ [0, 1] which describes

the amount of pixel belonging to a missing dot. So, the filter response is definitely

containing noise, caused by filter non-ideality (see Figure 5). The areas of miss-

ing dots on the filter response get larger, according to the actual dot size, which is

approximately4 × 4 pixels for the 600dpi resolution and 8x8 pixels for 1200dpi.

The response of such filter is called general feature picture, meaning that extracted

features (intensity values in this case) are attached to the image coordinate grid.

18

Figure 5: Original image and the corresponding filter response.

Following chapters will describe actual image filter design.

3.3.2 Fourier domain filtering

Fourier theorem states that any signal can be represented by the sum of sine and

cosine waves with various amplitudes and frequencies. That is, the relation-

ship between a repetitive, regular, and uniform dotted area pattern in the image

space and its spectrum in the spatial frequency can be linked by operating a two-

dimensional Fourier transform. Let a two dimensional image beξ(x, y) which is

real function representing the grey-level inx, y spatial coordinates, and let the im-

age height and image width are N and M, correspondingly. Then two-dimensional

discrete Fourier transform is given by the equation [19]

Ξ(u, v) = F{ξ(x, y)} =1

MN

M−1∑x=0

N−1∑y=0

ξ(x, y)e−j2π(ux/M+vy/N) (1)

This expression must be computed for values ofu = 0, 1, 2, ...,M−1, and also for

v = 0, 1, 2, ..., N − 1. Similarly, givenΞ(u, v), we obtainξ(x, y) via the inverse

19

Fourier transform, given by the expression [19]

ξ(x, y) = F−1{Ξ(u, v)} =M−1∑u=0

N−1∑v=0

Ξ(u, v)ej2π(ux/M+vy/N) (2)

for x = 0, 1, 2, ...,M − 1 andy = 0, 1, 2, ..., N − 1. Equations 1 and 2 comprise

the two-dimensional, discrete Fourier transform (DFT) pair [17]. The variablesu

andv are the frequency variables, andx andy are the spatial or image variables.

Let us consider Fourier spectra of the given texture part image (Figure 4). Accord-

ing to the complex nature of the Fourier spectra, it is useful to visualize it using

only its magnitude|Ξ(u, v)|.

In Figure 6 it is possible to see distinctive frequency peaks, which in turn are

responsible for the texture structure of the initial image. It is clear that all the peaks

are based on harmonic frequencies. Thus, for the separation of major and minor

frequencies in the Fourier spectra it is needed to multiply the complex Fourier

image by the maskM, which can be extracted from the Fourier magnitude image.

Figure 6: Original image and its Fourier spectra magnitude.

20

Finally, initial signal can be separated into so-called regular and irregular parts

ξ(x, y) = F−1{Ξ(u, v)} =

= F−1{M(u, v)Ξ(u, v) + (I(u, v)−M(u, v))Ξ(u, v)} =

= F−1{M(u, v)Ξ(u, v)}+ F−1{(I(u, v)−M(u, v))Ξ(u, v)}(3)

whereξ(x, y) is the spatial image,F andF−1 are forward and inverse discrete

Fourier transforms,M(u, v) is a mask (real valued function of the same definition

domain asΞ(u, v)), I(x, y) is a unit function. The decomposition in Eq. (3) is

possible according to that addition operation in spatial and frequency domain is

identical.

Let us consider maskM(u, v). To remove non-peak frequencies information it

should have ones in the frequency peak points (and harmonics) and zeros in other

frequencies. At first, corresponding peaks should be found at the frequency mag-

nitude picture. In practice, major peaks are the closest (among harmonic set) to

the (0, 0) frequency. Due to that fact, harmonics grid can be estimated (see Fig-

ure 7). After harmonics grid is found, it is worth to make the descent form the

grid points to other points smooth. Proposed instrument for that is to estimate 2-d

Gaussian in the each grid point (see Figure 8). Now when the mask is proposed, it

is possible to separate image into regular part and irregular one, according to Eq.

(3) as follows:

ξreg(x, y) = F−1{M(u, v)Ξ(u, v)}ξirr(x, y) = F−1{(I(u, v)−M(u, v))Ξ(u, v)} (4)

It is seen in Figure 9 that irregular partxiirr(x, y) looks like previously mentioned

general feature picture.

21

u

v

50 100 150 200 250 300 350

50

100

150

200

250

300

350

Figure 7: Fourier spectra magnitude and corresponding major frequencies withharmonics grid.

u

v

50 100 150 200 250 300 350

50

100

150

200

250

300

350

Figure 8: Mask with the Gaussians estimated in the grid points.

22

However, it needs to be said, that both regular and irregular parts contain compen-

sation noise near the borders which makes missing dots areas estimation harder.

This happens because the initial signalξ(x, y) has finite length, and during FFT

calculation it assumed equal to zero outside its definition domain.

Figure 9: Regular and irregular image parts.

3.3.3 Filtering using discrete cosine transform

The discrete cosine transform (DCT) is closely related to the discrete Fourier

transform. It is a separable linear transformation; that is, the two-dimensional

transform is equivalent to a one-dimensional DCT performed along a single di-

mension followed by a one-dimensional DCT in the other dimension. The def-

inition of the two-dimensional DCT for an input image A and output image B

is [20]

23

Bpq = αpαq

M−1∑m=0

N−1∑n=0

Amn cosπ(2m+1)p2M

cosπ(2n+1)q2N

,0 ≤ p ≤ M − 1

0 ≤ q ≤ N − 1

αp =

{1/√

M, p = 0√2/M, 1 ≤ p ≤ M − 1

αq =

{1/√

N, q = 0√2/N, 1 ≤ q ≤ N − 1

(5)

where M and N are the row and column size of A, respectively. If you apply DCT

to real data, the result is also real. DCT tends to concentrate information, making

it useful for image compression applications. The inverse transform for DCT is

defined as follows [20]:

Amn =M−1∑p=0

N−1∑q=0

αpαqBpq cosπ(2m+1)p2M

cosπ(2n+1)q2N

,0 ≤ m ≤ M − 1

0 ≤ n ≤ N − 1

αp =

{1/√

M, p = 0√2/M, 1 ≤ p ≤ M − 1

αq =

{1/√

N, q = 0√2/N, 1 ≤ q ≤ N − 1

(6)

In Figure 10 it is shown how certain frequency responses are much higher than

other. This fact leads to an obvious conclusion that these frequencies are respon-

sible for the texture structure.

However, the process of separating regular and irregular parts is not so simple.

According to energy conservation in image compression techniques (JPEG [21])

regular pattern can be removed as follows: sort all the DCT coefficients in the

descending order (see Figure 11) then apply some step function (see Figure 12) to

separate valuable frequencies from the others.

24

Figure 10: The DCT coefficients magnitude of the image presented in Figure 4.

25

0 0.5 1 1.5 2 2.5 3

x 104

101

102

103

104

Number of DCT coefficients

Mag

nitu

de (

log)

Figure 11: First 20% of absolute values of DCT coefficients sorted in descendingorder (presented in the logarithmic scale).

0 0.5 1 1.5 2 2.5

x 104

−1

−0.5

0

0.5

1

1.5

2

Number of DCT coefficients

Ste

p fu

nctio

n va

lue

Figure 12: Step function example.

26

Finding a proper step function is a matter of a trial method. While testing this

approach one phenomenon was found. As it can be seen in Figure 13, while

moving a step function threshold towards smaller coefficients, irregular picture

becomes less sharp, but minor noise remains nearly the same. This fact was used

to improve a step function to the appearance

T (t) =

0, t < n

1, n ≤ t ≤ m

0, t > m

(7)

wheret is a coefficient number,n is a lower coefficient number limit, andm is an

upper coefficient number limit. Tht idea of such threshold is not just to remove

major coefficients, but to cut minor coefficients tail also. In other words, to leave

only certain number of coefficients from n% to m%.

Figure 13: Initial image after removing 0.5%, 2.0%, and 5.0% DCT coefficients.

Using the DCT filtering gives less periodical noise at the borders than the DFT

filtering (compare Figure 9 and Figure 14). Also, it has to be mentioned that DCT

is significantly faster in computation than FFT.

27

Figure 14: Regular and irregular parts after the DCT filtering.

3.3.4 Gabor filters

A 2-dimensional Gabor filter can be defined as [22]

ψ(x, y; f, θ) = e−( f2

γ2 x′2+ f2

η2 y′2)ej2πfx′

x′ = x cos θ + y sin θ

y′ = −x sin θ + y cos θ

(8)

wheref is frequency,θ orientation, andγ andη control the frequency and orienta-

tion bandwidths. The normalized response of the Gabor filter,ψ(x, y), for image

ξ(x, y)

rξ(x, y, ; f, θ) =f 2

πγηψ(x, y, ; f, θ) ∗ ξ(x, y) (9)

can be used to construct a Gabor feature at any location(x, y) = (x0, y0). If

several frequencies and orientations of Gabor filter are used, a feature matrix of

filter responses at a single point can be constructed as

rξ(x0, y0, ; f0, θ0) . . . rξ(x0, y0, ; f0, θN−1)...

.. ....

rξ(x0, y0, ; fM−1, θ0) . . . rξ(x0, y0, ; fM−1, θN−1)

(10)

28

In the case of Heliotest the following frequencies and orientations were used (se-

lected empirically)f0,...,2 = 1

4, 1

8, 1

16

θ0,...,3 = 0, π4, 2π

4, 3π

4

(11)

Thus, in each point of initial image a feature vector of 12 complex values was

extracted. It is worth to mention that the extraction procedure takes quite a long

time in comparison with previously described methods (FFT, DCT).

3.3.5 Pattern modelling

One of the possible solutions for the missing dots inspection is to model the ideal

texture picture. The difference between the ideal pattern and the actual one will

show missing dots locations. This method performs pattern removal directly in

the spatial domain, not alike previously described methods which remove pattern

indirectly in frequency domain.

Let P(x,y) is an ideally modelled pattern. Then output result of pattern modelling

filtering will be

H{ξ(x, y)} = P (x, y)− ξ(x, y) = χ({xd, yd}) (12)

whereξ(x, y) is the actual image,χ({xd, yd}) is feature picture (characteristic

function of missing dots set with unknown noise). The main problem of this

approach is that an actual pattern is not enough ideal as it might seem (see Figure

4). A suggested method is to estimate pattern dots grid. Since it has generally

parallel nature, then it is worth to assign two grid basis vectors, let us name them

dx anddy. Having defined the starting point(x0, y0) (can be any actually printed

dot), the grid can be defined as

(xg, yg) = (x0, y0) + kdx + ldy (13)

29

wherek andl are integers such that(xg, yg) remains in theξ(x, y) definition do-

main.

Having estimated the dots grid it is possible to reproduce printed dot in each grid

knot. A dot can be modelled by a 2-d Gaussian curve, 2-d sine curve, or some

other hat-like function. If the dot modelling function in the grid knot(xg, yg) is

defined asD(xg ,yg)(x, y), then pattern model will look like

P (x, y) = max(xj ,yj)∈{(xg ,yg)}

D(xj ,yj)(x, y) (14)

which means that the pattern model is a maximum combination of all dot mod-

elling functions.

However, another approach seems to be more promising. Since the dots grid is

already defined, then it is better to compute some filter response based feature in

each grid knot, i.e., the average intensity of pixel neighborhood, etc. This method

will reduce filter response computations significantly, for example in a case of the

Gabor filters. Also, with the estimated grid, it is possible to operate not on the

actual image pixel level, but on the grid knots. This will eliminate the problem of

segmentation.

It is seen that the cornerstone of pattern modelling task is the dots grid estimation.

Finding dots grid in spatial domain seems to be a rather complicated problem.

However, dot coordinates can be achieved as an accompanying result of frequency

domain filtering methods, presented before (see Sections 3.3.2 and 3.3.3). While

applying these filters, regular and irregular image components are extracted. Let

us consider regular component. It models the ideal pattern structure quite well.

On the one hand, it is possible to subtract initial image from the pattern model

and to get the feature picture, but this is the same approach that was used in Sec-

tions 3.3.2 and 3.3.3 (subtraction was made directly in the frequency domain),

on the other hand it is possible to extract actual dots grid. This operation can

be performed using segmentation techniques. The first operation to be made is

thresholding, the second is labelling white areas, and the third is computing areas

30

centroid coordinates. These coordinates will comprise a grid (see Figure 15).

a b

c d

Figure 15: Pattern modelling: a) Initial image; b) Regular part from FFT filtering;c) Regular part thresholded; d) Initial image with the estimated grid (circles onthe grid knots).

This pattern modelling technique with the use of the frequency domain works

well. However, further research of pattern modelling only in the spatial domain is

also of interest.

31

3.4 Classification

3.4.1 Image segmentation

Filtering methods using DFT and DCT produce features as images of the same

size as the input one, and thus there should be more image processing techniques

applied, instead of proceeding directly to classification. As it can be seen in ir-

regular part of Figure 9 and Figure 14, the missing dot areas are large in size and

have more intensity. So, it is worthwhile to recognize missing dot by its intensity

and area. The proposed method consists of four following stages:

1. Binarize the image to a black and white image using a certain threshold.

2. Remove white areas which are smaller than a certain number of pixels.

3. Label all the separate areas.

4. Compute areas centroid coordinates and output these as missing dots loca-

tions.

This method requires the following parameters that have to be defined: a threshold

value and a minimal area. These values depend on the method implementation and

can be hard ones (numerical value), or dependent on the certain characteristics of

the given feature picture, i.e., on the image mean/minimum/maximum values.

3.4.2 Spatial features and Gabor features

Term spatial features in this context defines a set of values calculated for each

pixel in the spatial domain, which comprise a real valued feature vector for every

image point. In the case of Gabor filters are used for the feature extraction, several

complex values are produced by the filter, instead of getting a single value, like in

previously described methods. Generally speaking, for each vector (which in turn

32

represents single pixel on the image) two possible classes are defined: missing dot

pixel class and a regular pixel class.

Let us consider the classifier which produces a probability of pixel belonging

to the missing dots class. These probabilities comprise, in turn, another image,

which appearance is like the appearance of the general feature picture. This causes

another turn of segmentation and classification, using the techniques described in

Section 3.4.1.

Using Gabor features or spatial features in combination with pattern modelling

approach will eliminate need of further processing. In this case grid points, repre-

senting dots, will be classified, instead of classifying each image pixel.

For each grid knot features classification a subspace classifier presented in [23]

was used. Though, the subspace basis vectors were found using principal compo-

nent analysis (PCA) [24]. For the classification procedure following parameters

have to be defined:

• Subspace dimension.

• Maximum projection length to consider feature vector as belonging to a

missing dot class.

33

4 Automated Heliotest inspection

In this section designed algorithms are described. The description is quite thor-

ough, though presented in a more practical way. Before proceeding to the actual

methods, it needs to be said that presented algorithms are aimed at the detection

of missing dots in the halftone screen area. It is assumed, that the mentioned area

is already located and resized to the constant dimensions.

4.1 Common to all methods

Presented methods use the spectral analysis technique in different ways. This

means that they depend on the image regularity. So, it is worth to make the image

as regular as possible before applying the methods. Observing the halftone screen

area it was found that missing dots sizes change from the top of the strip to the

bottom. Actually, sizes do not change because printing cylinder of Heliotest ac-

cessory contains constant cell sizes, though the depth of the engraves varies which

means that print result intensity changes (see Figure 16). This fact leads to a an

idea to filter image parts separately, trying to preserve image regularity.

However, to keep certain frequency responses high input image should contain

enough regular information, meaning that it should be large enough. Due to this

fact, it was decided to part image in squares with the side equal to the halftone

screen area width. Each square is called a window. Such partial processing re-

quires some details to be considered. At first, as it was described in Sections 3.3.2

and 3.3.3, frequency domain filtering output lacks sharpness on the borders. This

means that a certain area near the borders of the image should be considered as

a false response area and not used in the classification. Thus, the actual region

of interest (ROI) is a bit smaller than the window size. The difference between

ROI and window is called window padding. Further, by cutting image into win-

dows, it is seen that there are gaps between the ROIs of two neighbor windows and

consequently there is a probability of losing some missing dots while performing

34

the inspection. To remove such gaps two neighbor windows should overlap. The

graphical presentation of such partitioning can be seen in Figure 17.

a b

Figure 16: Pattern difference of the halftone screen area: (a) At the top; (b) At thebottom.

Window

Window Padding

ROI Overlap

Figure 17: Sample partitioning.

After the image has been processed and missing dots have been detected, collected

information should be combined since there is a probability that some missing

dots were detected twice (in the overlap area). After the data is summarized, it is

ready for further considerations, i.e., for determining the quality of paper.

35

4.2 FFT based method

This method is based on the Fourier domain filtering presented in Section 3.3.2.

Designed filter outputs irregular image component which, in turn, is processed

using segmentation techniques described in Section 3.4.1. The method has been

summarized in Algorithm 1.

Algorithm 1 FFT based method

1. Produce digital image.

2. Preprocess the image.

3. Perform image partitioning.

4. With each image part do:

4.1 Extract the irregular component using FFT method.

4.2 Classify the irregular component.

4.3 Compute missing dots coordinates.

5. Combine missing dots coordinates from each image part.

6. Compute sample quality.

4.2.1 Preprocessing

The preprocessing stage includes simple median filtering [18] with a small domain

size (in comparison with the dot size). This filtering removes minor noise from

the image. Then the image is being normalized to the intensity level[0, 1] and

afterwards the image mean intensity value is being subtracted. Normalization is

required to reduce differences in intensity between samples. The subtraction of

the mean value prevents getting a high response in the central frequency(0, 0).

36

4.2.2 Extraction of the irregular component

The extraction of irregular image components was described in Section 3.3.2.

Some practical details need to be mentioned such as mask extraction. This proce-

dure includes following stages:

1. FFT magnitude computation.

2. Search of the maximum response frequency.

3. Estimation of harmonic frequencies set.

4. Computation of 2-d Gaussians in the grid points.

The first two steps are clear enough. However, the other two need explanation. As

it was experimentally found, the maximum frequency response ((uM , vM) on the

frequency plane) is the closest to the central frequency(0, 0) among the harmonic

set. With the mirrored frequency at(−uM , vM) they comprise the basis of the

harmonic set. So, each point of the harmonic set can be defined as

(uh, vh) = k(uM , vM) + l(−uM , vM) (15)

wherek, l ∈ Z. However, since the frequency plane is discrete, this harmonics set

estimation is not accurate. So, harmonics grid points need to be adjusted to actual

local maxima. This adjustment is performed by looking for a local maximum

in a certain neighborhood. This neighborhood was defined as a rectangular area

with the first approximation point in the centre which (area) should contain only

one local maximum from the harmonics set, i.e., not exceed sizesuM/2× vM/2.

Then, the first approximation point is replaced with the corresponding local area

maximum. In practice, the first approximation is already so good that it does not

miss the actual harmonics set points more than 5 pixels. So, the adjustment areas

can be reduced significantly. Gaussian estimation in the found points is based on

37

the values ofuM andvM and can be defined as:

Gh(u, v) = exp (−uM(u− uh)2 + vM(v − vh)

2

2σ) (16)

whereσ was found empirically, according to filtering results. It was defined as

σ =

√u2

M+v2M

10.

Filter output (feature picture) is adjusted before proceeding to the next algorithm

step. At first, all the values which are lower than the image mean are made equal

to mean value, and then image is normalized to fit intensity interval[0, 1]. This is

made to unify filter response for different inputs.

4.2.3 Classification

As it was stated in Section 3.4.1, the classification procedure consists of four

steps. This process requires two parameters to be defined: a threshold valueτ to

be applied and a certain areaS for the opening procedure [9]. The thresholding

operatorT is defined as follows:

T{ξ(x, y)} =

{0, ξ(x, y) < τ

1, ξ(x, y) ≥ τ(17)

This is actually a hard threshold, i.e.,τ is a certain intensity level. In this problem

case, such threshold is inappropriate. For example, if feature picture contains

no irregularities, which happens when there no missing dots in the input image,

then hard thresholding gives false information and the following result will be

incorrect. This problem can be solved by using proportional threshold coefficient

τ , based on the image mean valueµξ

T{ξ(x, y)} =

{0, ξ(x, y) < τµξ

1, ξ(x, y) ≥ τµξ(18)

38

Black-white opening is like the thresholding, but it removes areas in the picture

which are less than S pixels in size. These two parametersτ andS were estimated

using cross-validation technique. At first, a definition domain was assigned for

both parameters empirically

τ ∈ 1, 1.5, 2, 2.5...20;

S ∈ 1, 2, 3...20.(19)

Then the method was executed using the training set, and the best matching pa-

rameters were found. It should be noticed that each image part contains its own

parameters.

4.3 DCT based method

This method is based on the DCT based filtering presented in Section 3.3.3. The

whole approach is similar to the one described in Section 4.2, except for the use

of the DCT filter described in Section 3.3.3. The method has been summarized in

Algorithm 2.

Algorithm 2 DCT based method

1. Produce digital image.

2. Preprocess the image.

3. Perform image partitioning.

4. With each image part do:

4.1 Extract the irregular component using DCT method.

4.2 Classify the irregular component.

4.3 Compute missing dots coordinates.

5. Combine missing dots coordinates from each image part.

39

6. Compute sample quality.

4.4 Pattern modelling with spatial features method

This method uses the pattern modelling technique described in Section 3.3.5. Pre-

processing stage is identical to the one presented in Section 4.2. This method is

considered to be the most promising one. An important part of the method is the

grid extraction part. The classifier is made according to the proposal in Section

3.4.2. The method has been summarized in Algorithm 3.

Algorithm 3 Pattern modelling with spatial features method

1. Produce digital image.

2. Preprocess the image.

3. Perform image partitioning.

4. With each image part do:

4.1 Extract the pattern grid.

4.2 Classify each pattern grid knot.

4.3 Compute missing dots coordinates.

5. Combine missing dots coordinates from each image part.

6. Compute sample quality.

4.4.1 Grid extraction

The grid extraction part of this method is based on the technique presented in

Section 3.3.2. Though, instead of the irregular part of the input image, it considers

40

the regular component. The regular part from the FFT filter is better than the one

achieved by DCT filtering. Somehow, it contains less noise and therefore it is

more suitable for the grid extraction.

The grid extraction consists of the following stages:

1. Forward FFT transform.

2. FFT magnitude computation.

3. Mask extraction.

4. Mask application.

5. Inverse FFT transform.

6. Threshold application.

7. Segmentation.

8. Areas centroids computation.

The mask extraction procedure is identical to the one described in Section 4.2.

The only difference is that not the original mask is applied, but the inverse one

M = I(u, v) − M(u, v). When the filtering is made the image is converted to

black and white using a certain threshold. In this case a proportional threshold is

used, based on the output image mean. Since the regular output of the FFT filter

contains nearly no noise a segmentation procedure does not include noise removal.

So, after thresholding has been done, each white area on the image is labelled and

its centroid coordinates are computed. The actual dots grid is determined in(x, y)

spatial coordinates of each dot center estimation.

41

4.4.2 Classification

After dots grid has been estimated it is possible to observe each dot for making

a decision, whether the dot is missing or not. In Figure 18 it is possible to see

several dot observations.

a

e

b

f

c

g

d

h

Figure 18: Dot samples: (a)-(c) regular dots; (d) regular dot expectation; (e)-(g)missing dots; (h) missing dot expectation.

It was decided to use each grid knotn×n neighborhood as a feature vector andn

was selected the way it covers all the dot area. Finally, feature vector consists of

n2 values, each is in[0, 255] interval (8bit greyscale).

On the other hand, it is possible to extract not just spatial features in each grid

knot, but to use another class of features, namely Gabor features, described in

3.3.4.

42

5 Experiments and results

Every method proposal should be finalized with practical experiments. In this

section real data experiments are described and actual algorithmic decisions are

covered.

5.1 Data

5.1.1 Heliotest samples

Data for this project consist of a set of Heliotest print results which were printed

by the author and Dr. J.-K. Kämäräinen in the quality control lab of a paper mill.

Total number of the test prints obtained is 150 which include 10 different types

of coated paper. Printing of each sample took from 10 to 40 seconds, depending

on the ink consumption. Test prints were made on one printing device, using one

IGT printing cylinder which is actually not good in the sense of generality.

Original samples were digitized using a flatbed scanner. For the purposes of fur-

ther research it was decided to digitize as much information from test prints as

possible. Therefore, the resolution of 1200dpi and color depth of 24bit were de-

fined. One image scan occupies approximately 55-70 megabytes of hard drive

space. Scanning process took about 8 minutes for each sample. In practice we

need less information so the scanning process can consume significantly less time.

5.1.2 Training and testing data

This work is concentrated on the missing dots detection in the halftone screen

area. Therefore, this part of digital image must be located, whether automatically

or manually. Also a certain amount of missing dots has to be picked for testing and

43

training data. For the aforementioned needs the author developed an application

(see Figure 19) which allows to locate halftone screen area on the scanned sample

and to mark missing dots. This application was developed in Visual C++, using

MFC [25], and works under MS Windows operation system. It saves area corners

coordinates and missing dots locations in a plain text file. The half of the samples

were fully examined and missing dots locations were marked. Another half was

partially marked, i.e., only the first 20 dots from the beginning.

Figure 19: Screen shot of an application for halftone screen area location andmissing dots picking.

5.2 Experiments

The training set contained 75 images, fully marked with missing dot locations.

Experiments were tested on another set of 70 images with only first 20 missing

dots marked. However, all the statistical results used combined information of the

performance of the method on both training and testing sets. Before proceeding

44

to the actual experiments, it needs to be said that all the data sets were unified,

i.e., acquired images were resized to the region of interest (halftone screen area)

using corresponding coordinates. The training and testing sets (manually marked

missing dots) were recalculated according to the new ROI position. Original ROI

dimensions were changed to 5000 pixels (height) by 400 pixels (width).

5.2.1 FFT based method

Using supervised learning the following method parameters were esimated (see

Table 3)

Table 3: Segmentation parameters of the FFT based method.

Window number Threshold coefficient (τ ) Area (S)1 8.5 122 9 143 9.5 144 9.5 115 11 96 9 117 11 88 12 69 10 810 11.5 411 12.5 412 13 413 12 414 10 415 10 4

45

The method performance was measured in several parameters:

1. Number of missing dots detected out of the first 20 manually marked dots.

2. Number of false alarms (falsely detected dots) in the area before 20th man-

ually marked dot.

3. Actual quality (distance in millimeters from the beginning of the halftone

screen area till the 20th missing dot detected).

4. Difference between the results achieved by the method and the manual eval-

uation.

All results for each sample can be seen in Appendix 1. The performance summary

has been presented in Table 4.

Table 4: Performance summary of the FFT based method.

Average number of missing dots detected out of the first 20 marked17,90344828Average number of false alarms before the first 20 marked 6,282758621Mean square error in quality measurement 4,300962202Time needed for one sample processing ≈90 seconds

5.2.2 DCT based method

Using supervised learning the method parameters were estimated as shown in

Table 5.

Full data for each sample observation can be seen in Appendix 1. In this part, the

method performance is presented in Table 6.

46

Table 5: Segmentation parameters of the DCT based method.

Window Number Threshold coefficient (τ ) Area (S)1 7.5 122 8 143 8.5 144 8.5 115 10 96 8 117 10 88 11 69 9 810 10.5 411 11.5 412 12 413 11 414 9 415 9 4

Table 6: Performance summary of the DCT based method.

Average number of missing dots detected out of the first 20 marked17,69655172Average number of false alarms before the first 20 marked 3,075862069Mean square error in quality measurement 2,539949014Time needed for one sample processing ≈40 seconds

47

5.2.3 Pattern modelling with spatial features method

During observation the method parameters were estimated as shown in Table 7).

Table 7: Classification parameters for the pattern modelling with spatial featuresmethod.

Subspace dimensionMaximum projection length25 50

All results for each sample observation can be obtained from Appendix 1. In thispart, method performance summary (see Table 8) is presented.

Table 8: Performance summary for the pattern modelling with spatial featuresmethod.

Average number of missing dots detected out of first 20 marked17,71034483Average number of false alarms before first 20 marked 0,95862069Mean square error in quality measurement 2,390654698Time needed for one sample processing ≈120 seconds

48

5.3 Results and discussions

The results are considered to be quite good. It is seen that the DCT based method

performs a little better than the FFT based one. Even though, they both use nearly

the same frequency based approach. Also the DCT based algorithm works faster

since it requires less processing stages (no mask estimation, etc.) and generally,

DCT implementations are faster than FFT.

Although the DCT and FFT based methods perform well, best results are achieved

by the pattern modelling method. It is seen that classification in which pattern el-

ements are considered instead of image units gives significantly less false alarms,

and, in turn, produces better measurement accuracy.

More detailed methods comparison is of some interest. For example, do the false

alarms of both methods intersect, and if it is true then in what image parts they

respond a missing dot. It is maybe also worthwhile to combine all methods results

to achieve better measurement accuracy.

It is also clear that for the better results more testing data is needed. Data samples

should cover different types of Heliotest print results of paper and board with

different quality.

49

6 Conclusion

In this work several machine vision techniques for automated Heliotest inspection

were presented. It was found that spectral analysis idea works well in the case of

missing dots detection. All of the presented methods use this approach in different

ways. The best results were achieved with the pattern modelling with spatial

features method. This is because it classifies missing dots in a natural way, like if

the test is conducted by a human expert.

Future research will include:

• The current methods improvement.

• Data set increase.

• New methods development.

• Comparison with human expert results.

• Machine vision system design.

This work can be considered as the first step in the Heliotest automation research.

On this stage it is possible to say that machine vision system implementation in

printability testing process has a potential chance of success.

50

References

[1] PAPVISION project official website. Lappeenranta Uni-

versity of Technology, Information Technology Department,

http://www.it.lut.fi/project/papvision/, 2003. Referred on 12.2003.

[2] H. Juslin and E. Hansen.Strategic Marketing in the Global Forest Industries.

Authors Academic Press, 2003. ISBN 0-9703333-7-4.

[3] P. Oittinen and H. Saarelma.Printing. Papermaking Science and Technol-

ogy. Fapet Oy, 1998. ISBN 952-5216-13-6.

[4] J.-E. Levlin and L. Söderbjelm.Pulp and Paper Testing. Papermaking Sci-

ence and Technology. Fapet Oy, 1999. ISBN 952-5216-17-9.

[5] International Paper Corporation.IP coated official website. International

Paper, http://ipcoated.com/, 2003. Referred on 12.2003.

[6] Centre Technique du Papier.The Pulp and Paper Research and Technical

Centre. WWW, http://www.webctp.com/, 2003. Referred on 12.2003.

[7] IGT Testing Systems.IGT official website. WWW, http://www.igt.nl/, 2003.

Referred on 12.2003.

[8] IGT Testing Systems.IGT Information leaflet W41. IGT, 2003.

[9] R. C. Gonzalez and R. E. Woods.Digital Image Processing. Prentice-Hall,

Inc., 2002. ISBN 0-201-18075-8.

[10] Techtarget Network. Technical Dictionary. Techtarget Inc.,

http://whatis.techtarget.com/, 2003. Referred on 12.2003.

[11] H. Kälviäinen, P. Saarinen, P. Salmela, A. Sadovnikov, and A. Drobchenko.

Inspection on paper by machine vision.Proceedings of SPIE Conference on

Intelligent Robots and Computer Vision XXI: Algorithms, Techniques and

Active Vision, 2003. Providence, Rhode Island, USA.

51

[12] A. Kumar and G.K.H. Pang. Defect detection in textured materials using

optimized filters. IEEE Transactions on Systems, Man and Cybernetics,

32:553–570, October 2002.

[13] D. Chetverikov. Structural defects: general approach and application to tex-

tile inspection.Proceedings of the 15th International Conference on Pattern

Recognition, 1:521–524, September 2000.

[14] L.H. Monteiro and A. Conci. Multifractal characterization of texture-based

segmentation.International Conference on Image Processing, 1:792–795,

September 2000.

[15] L. Macaire and J.G. Postaire. Automated visual inspection of galvanized and

painted metallic strips.CompEuro ’93. ’Computers in Design, Manufactur-

ing, and Production’, Proceedings., pages 8–15, May 1993.

[16] IEEE Standard 610.4-1990.IEEE Standard Glossary of Image Processing

and Pattern Recognition Terminology.

[17] A. K. Jain.Fundamentals of Digital Image Processing. Prentice Hall, 1989.

ISBN 0-13-336165-9.

[18] Y. Zou and W.T.M. Dunsmuir. Generalized max/median filtering.Proceed-

ings of International Conference on Image Processing, 1:428–431, October

1997.

[19] B. Jähne. Digital Image Processing:Concepts, Algorithms and Scientific

Applications. Springer-Verlag, 1991. ISBN 3-540-53782-1.

[20] The MathWorks Inc. The MathWorks official website. WWW,

http://www.mathworks.com, 2003. Referred on 12.2003.

[21] JPEG Group.Official JPEG website. http://www.jpeg.org, 2003. Referred

on 12.2003.

[22] J.-K. Kämäräinen, V. Kyrki, H. Kälviäinen, M. Hamouz, and J. Kittler. In-

variant gabor features for evidence extraction.Proceedings of the IAPR

Workshop on Machine Vision Applications, pages 228–231, 2002.

52

[23] E. Oja and T. Kohonen. The subspace learning algorithm as a formalism for

pattern recognition and neural networks.IEEE International Conference on

Neural Networks, 1:277–284, 1988.

[24] R. O. Duda, P. E. Hart, and D. G. Stork.Pattern Classification. Wiley-

Interscience, 2001. ISBN 0-471-05669-3.

[25] Microsoft. Microsoft Developers’ Network. Microsoft Corp.,

http://msdn.microsoft.com/, 2003. Referred on 12.2003.

53