i

AUTOMATIC COMPUTER VISION SYSTEM FOR INDUSTRIAL PRODUCT

QUALITY INSPECTION

NOR NABILAH SYAZANA BINTI ABDUL RAHMAN

This Report Is Submitted In Partial Fulfilment Of Requirements For The Bachelor

Degree of Electronic Engineering (Industrial Electronic)

Fakulti Kejuruteraan Elektronik dan Kejuruteraan Komputer

Universiti Teknikal Malaysia Melaka

June 2016

ii

UNIVERSTI TEKNIKAL MALAYSIA MELAKA

FAKULTI KEJURUTERAAN ELEKTRONIK DAN KEJURUTERAAN KOMPUTER

BORANG PENGESAHAN STATUS LAPORAN

PROJEK SARJANA MUDA II

Tajuk Projek :

Sesi Pengajian

: 1 5 / 1 6

NOR NABILAH SYAZANA BINTI ABDUL RAHMAN Saya …………………………………………………………………………………………………..

(HURUF BESAR) mengaku membenarkan Laporan Projek Sarjana Muda ini disimpan di Perpustakaan dengan syarat-syarat kegunaan seperti berikut:

1. Laporan adalah hakmilik Universiti Teknikal Malaysia Melaka.

2. Perpustakaan dibenarkan membuat salinan untuk tujuan pengajian sahaja.

3. Perpustakaan dibenarkan membuat salinan laporan ini sebagai bahan pertukaran antara institusi

pengajian tinggi.

4. Sila tandakan ( √ ) :

SULIT*

*(Mengandungi maklumat yang berdarjah keselamatan atau

kepentingan Malaysia seperti yang termaktub di dalam AKTA

RAHSIA RASMI 1972)

TERHAD**

**(Mengandungi maklumat terhad yang telah ditentukan oleh

organisasi/badan di mana penyelidikan dijalankan)

TIDAK TERHAD

Disahkan oleh:

__ ________________________ ___________________________________

(TANDATANGAN PENULIS) (COP DAN TANDATANGAN PENYELIA)

AUTOMATIC COMPUTER VISION SYSTEM FOR INDUSTRIAL

PRODUCT QUALITY INSPECTION

iii

“I hereby declare that the work in this project is my own except for summaries and

quotations which have been duly acknowledge.”

Signature : .......................................................

Author : NOR NABILAH SYAZANA BINTI ABDUL RAHMAN

Date : ....................................................

iv

“I acknowledge that I have read this report and in my opinion this report is sufficient

in term of scope and quality for the award of Bachelor of Electronic Engineering

(Industrial Electronics) with Honours.”

Signature : .......................................................

Supervisor’s Name : NORHASHIMAH BINTI MOHD SAAD

Date : .......................................................

v

To my beloved father and mother

vi

ACKNOWLEDGEMENT

First and for most, praise to Allah S.W.T, because of His bless that giving me

the strength to complete my full project report. The special thanks go to my helpful

supervisor, Dr. Norhashimah Binti Mohd Saad for the valuable guidance and advice.

She inspired me greatly to work on this project. Her willingness to motivate me

contributed tremendously to my project for showing me some example that related to

the topic of my project. I am also very thankful to my co-supervisor Dr. Abdul Rahim

Bin Abdullah for his guidance, advice, and motivations.

I also take this opportunity to express a deep sense of gratitude to Universiti

Teknikal Malaysia Melaka for giving me a chance to undergo this Final Year Project

as partial fulfilment for my degree. Thanks again to Universiti Teknikal Malaysia

Melaka for providing me with a good environment and facilities to complete this

project.

Finally, an honourable mention goes to my families and my fellow

undergraduate for their understandings and supports in completing this project. My

sincere appreciation also extends to all my colleagues and others who have provided

assistance on various occasions. Their views and tips are useful indeed.

vii

ABSTRACT

Product quality inspection is the life business enterprises, only the objective, and strict

inspection can promise the best quality of products left the factories. Therefore, the

product quality inspection is a key job of business enterprises. Product quality

inspection becomes a major issue in production and industrial. The problem is

associated with a manual inspection that is done by a human. In facts, human as an

inspector is slower and their efficiency is affected by their states of tiredness, illness

or other human being shortcomings. This project presents an automatic computer

vision system for industrial product quality inspection as a solution from the issue that

has been raised. Soft drink is a product that is tested for quality inspection in this

system. The offline system was designed to inspect the product based on color and

water level classification using Visual Studio 2010 software. There are several image

processing technique are used which are thresholding, morphological operation and

segmentation. The system used Otsu’ method for threshold the image and quadratic

distance classifier to classify the color of the soft drink based on color classification.

The water level will read the value of a white pixel that shows the height of the bottle.

It is used to measure the range of water level. This system is also do in real-time testing

that using conveyor, sensor and webcam. The experiment result shows that the system

is 100% accurate for both offline and online system.

viii

ABSTRAK

Pemeriksaan kualiti produk adalah hayat sesuatu perusahaan atau perniagaan dan

hanya objektif yang berkaitan, dan pemeriksaan yang ketat boleh menjanjikan sesuatu

produk berkualiti setelah keluar daripada sesebuah kilang. Oleh itu, pemeriksaan

kualiti produk adalah tugas utama sesebuah perusahaan atau perniagaan. Pemeriksaan

kualiti produk menjadi isu utama dalam pengeluaran sesebuah industri. Masalah ini

berkaitan dengan pemeriksaan manual yang dilakukan oleh manusia. Fakta telah

menunjukkan bahawa manusia sebagai pemeriksa adalah lebih perlahan dan

kecekapan mereka dipengaruhi oleh keletihan, sakit atau lain-lain kelemahan. Projek

ini membentangkan sistem penglihatan secara automatik oleh komputer untuk

pemeriksaan kualiti produk industri sebagai penyelesaian daripada isu yang telah

dibangkitkan. Minuman ringan adalah produk yang telah diuji untuk pemeriksaan

kualiti dalam sistem ini. Sistem luar talian telah direka untuk memeriksa produk

berdasarkan klasifikasi warna dan paras air yang menggunakan perisisan Visual Studio

2010. Terdapat beberapa teknik pemprosesan imej yang digunakan seperti

pengambangan, operasi morfologi dan segmentasi. Sistem ini menggunakan kaedah

Otsu ' untuk ambang imej dan pengelas jarak kuadratik untuk mengelaskan warna

minuman ringan berdasarkan klasifikasi warna. Paras air akan membaca nilai piksel

putih yang menunjukkan ketinggian botol. Ia digunakan untuk mengukur julat paras

air. Sistem ini juga dilakukan dalam talian yang menggunakan penghantar, pengesan

dan kamera web. Keputusan eksperimen menunjukkan bahawa sistem ini adalah 100%

tepat untuk kedua-dua luar talian dan sistem dalam talian

ix

TABLE OF CONTENTS

CHAPTER TITLE PAGES

PROJECT TITLE i

REPORT STATUS FORM ii

DECLARATION iii

SUPERVISOR DECLARATION iv

DEDICATION v

ACKNOWLEDGEMENT vi

ABSTRACT vii

ABSTRAK viii

TABLE OF CONTENTS ix

LIST OF TABLES xii

LIST OF FIGURES xiii

LIST OF ABBREVIATION xv

LIST OF APPENDIX xvi

I INTRODUCTION

1.1 Project Background 1

1.2 Problem Statement 2

1.3 Objectives 3

1.4 Scope of Work 3

1.5 Research Methodology 4

1.6 Thesis Organization 4

x

II LITERATURE REVIEW

2.1 Introduction 6

2.2 Theory and Basic Principles 7

2.3 RGB Color Model 9

2.4 Classification System 10

2.5 Level Classification 11

2.6 Hardware Setup 12

2.7 Commercial Value and Market Survey for 13

Product Inspection

2.8 Review of Previous Related Work 14

III PROJECT METHODOLOGY

3.1 Introduction 16

3.2 Visual Studio 2010 17

3.2.1 Integrated Development Environment 17

(IDE)

3.2.2 New Project 18

3.2.3 Solution Explorer 19

3.2.4 Toolbox 20

3.2.5 Editors and Designers 21

3.2.6 Properties Window 22

3.2.7 Build and Debug Tools 23

3.3 Project Development Workflow 24

3.4 Analysis Framework 27

3.5 Image Acquisition 28

3.6 Pre-Processing 28

3.6.1 RGB to HSV 28

3.7 Color Segmentation 30

3.7.1 Thresholding (Otsu’ Method) 31

3.7.2 Morphological Operation 31

3.7.3 Region of Interest (ROI) 32

3.8 Histogram Analysis 32

xi

3.9 Quadratic Distance Classifier 33

3.10 Water Level Classifier 33

IV RESULTS AND DISCUSSION

4.1 Introduction 34

4.2 Experimental Setup 35

4.3 Color Classification Analysis 35

4.4 Water Level Analysis 43

4.5 System Accuracy Analysis 45

V CONCLUSION AND RECOMMENDATION

5.1 Conclusion 47

5.2 Sustainability and Implementation 48

5.3 Recommendation for Future Development 48

VI REFERENCES 49

VII APPENDIXES 53

xii

LIST OF TABLES

NO TITLE PAGE

2.1 Hardware Component 13

2.2 Different Method Review 14

4.1 Total Sample Image 35

4.2 Red Histogram Value of Reference Image 38

4.3 Green Histogram Value of Reference Image 38

4.4 Blue Histogram Value of Reference Image 39

4.5 RGB Histogram Value of Reference Image 39

4.6 Red Histogram Value of Test Image 40

4.7 Green Histogram Value of Test Image 41

4.8 Blue Histogram Value of Test Image 41

4.9 RGB Histogram Value of Test Image 42

4.10 Threshold Value for PASS Color 43

xiii

LIST OF FIGURES

NO TITLE PAGE

2.1 Different Levels Image Processing 8

2.2 Schematic of the RGB Color Cube 10

2.3 RGB 24-Bit Color Cube 10

2.4 Bottle Level Filling System 12

3.1 IDE of Visual Basic 17

3.2 “New Project” From Start Page 18

3.3 “New Project” From the File Menu 18

3.4 New Project 19

3.5 Solution Explorer 20

3.6 Toolbox 20

3.7 Graphical User Interface 21

3.8 Graphical User Interface Designer 21

3.9 Code Editor 22

3.10 Properties Window 22

3.11 Build Tool 23

xiv

3.12 Error Message 23

3.13 Debug Tool 24

3.14 Project Workflow 26

3.15 Analysis Framework 27

3.16 RGB Color Cube 29

3.17 Top View of RGB Cube 29

3.18 HSV Color 29

4.1 Color Process of Reference Image 28

4.2 Color Process of Test Image 29

4.3 Red Histogram of Reference Image 30

4.4 Green Histogram of Reference Image 30

4.5 Blue Histogram of Reference Image 31

4.6 RGB Histogram of Reference Image 31

4.7 Red Histogram of Test Image 32

4.8 Green Histogram of Test Image 33

4.9 Blue Histogram of Test Image 33

4.10 RGB Histogram of Test Image 34

4.11 Water Level Pass 36

4.12 Water Level Overfill 36

4.13 Water Level Underfill 37

4.14 GUI before Running 38

4.15 GUI after Running 38

4.16 Accuracy System Bar Chart of Test Image 39

xv

LIST OF ABBREVIATION

IKS – Industri Kecil Sederhana

RGB – Red, Green, and Blue

HSV – Hue, Saturation and Value

VB – Visual Basic

VS – Visual Studio

3D – 3 Dimension

2D – 2 Dimension

LED – Light Emitting Diode

ROI – Region of Interest

GUI – Graphical User Interface

IDE – Integrated Development Environment

xvi

LIST OF APPENDIX

APPENDIX TITLE PAGE

A Analysis data of RGB value for Reference 53

image and Test image

1

CHAPTER 1

INTRODUCTION

1.1 Project Background

Product quality inspection is the life business enterprises, only the objective,

and strict inspection can promise the best quality of products left the factories.

Therefore, the product quality inspection is a key job of business enterprises. Visual

inspection is a sub-item from machine vision that contributed so much modern

manufacturing field. In the old days, this can be done with human inspection those

using human eyes. In facts, human as an inspector is slower and their efficiency is

affected by their states of tiredness, illness or other human being shortcomings [1]. In

the modern days, visual inspection systems reduce human interaction with the

inspected goods, classify generally faster than human beings and tend to be more

consistent in their product classification. Visual inspection also has broad application

in industrial automation and covers the full range of technical difficulty in computer

vision [2].

The technique of automated visual inspection has an extensively applied

foreground in the most modern automatic inspection of product quality [3]. This

technique is based on computer vision is kind of method does not touch with inspecting

2

objects. This method uses integrated technology of image processing, exactitude

measure, pattern recognition, artificial intelligence and so on. It is basic principle is to

analyze the target images obtained by the computer vision system in order to get

necessary measuring information, and judge whether the measured target is the norm

or not according to the known knowledge. The image processing, process, and judges

the collected images using the image processing and inspecting software and gets a

conclusion whether the inspected object is in accord with the beforehand established

request or not. Such inspection system extracts and analyzes a large number of features

and automatically reject products that do not satisfy the set quality standards. As a

result, automated visual inspection is undergoing substantial growth in manufacturing

because of its cost effectiveness, consistency, superior speed, and accuracy.

The global beverage industry is growing constantly and the people are

spending more and more money on it. The growing demand requires more efficient

production. At the same time also demands on quality are increasing. Therefore, the

manufacturers are forced to invest in faster machines and plants respectively. In the

beverage industry, this has already started with the production of the preform and of a

bottle and repeats itself, when the empty bottle is recycled and has become a new

bottle.

Industri Kecil Sederhana (IKS) is defined as manufacturing companies that

providing manufacturing related services with an annual turnover not exceeding RM25

million and employs a staff of full-time and does not exceed 150 people. Industri Kecil

Sederhana (IKS) is also a manufacturing company that uses a small capital to produce

their own product. This report will describe a software framework that used for this

project and developing an automatic visual inspection system for inspecting the quality

of beverage product.

1.2 Problem Statement

There exists no industrial vision system capable of handling all tasks in every

application field. Global consumption of soft drink is increasing year by year. It is very

3

important to take care of the quality of the product delivered. To maintain the quality

of such larger number of bottles is not an easy job. There are some IKS that still used

manual inspection in their quality inspection process. With manual inspection, it is

impossible maintaining the easy manufacturing flow. In the IKS, they are searching

new ways to package and deliver their products to minimize their impact on the

environment and reduce costs.

Based on the manual inspection and quality of product issues, this project is

introducing an automatic computer vision system for industrial product quality

inspection. It is because the vision applications have achieved a significant role in the

packaging industry. It is also a much more efficient way of checking products and

easier to be used. This project is proposed to overcome the problem of manual

inspection by a human that is not efficient anymore and to reduce cost by industry. The

quality control processes are based on color and level classification of soft drink bottle.

1.3 Objectives

There are several objectives that need to be achieved in this project. These

objectives are:-

i. To implement image processing technique for the detection and

classification of color and level of the beverage product.

ii. To develop the system based on classification method.

iii. To verify the performance of the system.

1.4 Scope of Work

This project mainly focuses on the software development that is designing an

automated system to classify the soft drink bottle based on color segmentation and

4

level classification of each bottle. The hardware design is not involved in this project.

The system will be designed using the Visual Studio software. The input image of soft

drink bottle will be captured using a webcam. Then, the image will be analyzed,

segmented and classified using the VS software. This system will be able to inspect of

soft drink bottle based on color segmentation and level classification. This project is

also focusing on five different flavors of soft drink, which are Strawberry, Grapes,

Orange, Tropical and Root beer.

1.5 Research Methodology

This project only discusses on software development which is involving the

algorithm development. The research methodology is an important part that shows the

work procedure in order to complete the project. There are some research

methodologies:-

Data collection: An input image of soft drink bottle was taken by using

a webcam as a sample image.

Pre-processing: Color was analyzed by using RGB component, HSV

component, and saturation component.

Color segmentation: Color is segmented using the process of

thresholding (Otsu’ method), morphological operation and region of

interest (ROI).

Color classification: Quadratic Distance Classifier was used to

calculate the threshold value between two sample images.

Level classification: Each bottle has been set by a certain range of water

level, which is passed, overfill and underfill.

Performance verification: It shows the accuracy of the system.

Field testing: Real-time testing.

5

1.6 Thesis organization

The report of this project is divided into five main chapters. There are Chapter

1: Introduction, Chapter 2: Literature Review, Chapter 3: Methodology, Chapter 4:

Results and Discussion, Chapter 5: Conclusion and Recommendation.

Chapter 1 was explained about project background or project overview of the

automated visual system inspection system. The problem statement of the project has

been described and the reason for developing this project. The objectives are stated is

related to the problem statement of this project. The scope of work is described the

project requirement and research methodology is described what method used in this

project. Thesis organization shows the report structure and organization.

Chapter 2 is an important part that needs to be understood by previous research

was done that related to this project. This chapter will gather information about method

and approaches that suitable to be used in this project from any resources. The previous

research will be reviewed, discussed and summarized.

Chapter 3 will be discussed a method that used in this project. This chapter has

explained the procedures that have been made in order to complete this project. The

method used are RGB and HSV color model, Otsu’ method and Quadratic Distance

Classifier. All of this method will be applied in this project.

Chapter 4 is about finding the results and discussion of this project. This

chapter was explained about the results of a simulation using VS software. The

algorithm has been developed in the previous chapter and it will use to figure out the

result in this chapter. The results will be discussed the acceptable or unacceptable of

the beverage product, based on color segmentation and level classification.

Chapter 5 was described conclusion and recommendation in this project. An

overall conclusion of this project has been discussed in this chapter. A specific

recommendation has been recommended due to the issue that has been raised by the

previous researchers.

6

CHAPTER 2

LITERATURE REVIEW

2.1 Introduction

In order to gain enough information to recommend the appropriate method that

can be used to complete the research, comparison between certain techniques is

required to detect and classify of color and level of the beverage product. This chapter

explains and discussed related to the theory and basic principles, review of previous

related work, and summary of this project. The research covered image acquisition,

RGB color model, and level classification method.

The review of previous related works focuses on image processing technique

for automatic product quality inspection that designed by respective researchers. The

commercial value and market survey this project will also be discussed in this chapter

so that the project will be beneficial to the market.

7

2.2 Theory and Basic Principles

The computerized method of image analysis is recently widely used in industry

and is attracting a lot of attention and interest. It is caused by an increasing number of

production requirements and development of computer technology [4]. Computer

vision is a disciplined study how to reconstruct, interpret and understand a 3D scene

from its 2D images in terms of the properties of the structures present in the scene.

Besides, the ultimate goal of computer vision is to model, replicate and, more

importantly, exceed the human vision using computer software and hardware at

different levels. Computer vision systems are commonly used in many fields such as

Science, engineering, and industry. Their applications allow automating many

scientific and technological process.

Image processing is a technique to enhance raw images received from cameras

or sensors placed on satellites, space probes and aircraft or pictures taken in normal

day to day life for various applications. Image processing systems are becoming

popular due to easy availability of powerful personal computers, large size memory

device, and graphic software. There are two methods available for image processing

which is analog image processing and digital image processed. But, this project was

focused on digital image processing. The term digital image processing generally

refers to the processing of a two-dimensional picture by a digital computer [5]. In

digital image processing, a digital computer is used to process the image. The image

will be converted to digital form and then process it. It is defined as subjecting

numerical representations of objects to a series of operations in order to obtain the

desired result.

Image processing or analysis involves a series of steps, which can be broadly

divided into three levels: low-level processing, intermediate level processing, and

high-level processing as indicated in figure 2.1.

8

Figure 2.1: Different Levels Image Processing

Figure 2.1 shows Low-level processing includes image acquisition and pre-

processing. Before any video or image processing can commercially an image must be

captured by the camera and converted into a manageable entity. This is a process

known as image acquisition. Image pre-processing refers to the initial processing of

the raw image data for correction of geometric distortions, removal of noise, graylevel

correction and correction for blurring. Pre-processing aims to improve image quality

by suppressing undesired distortions or by the enhancement of the important features

of interest. Intermediate level processing involves image segmentation, and image

representation and description.

Image segmentation is typically defined as an exhaustive partitioning of an

input image into regions, each of which is considered to be homogenous with respect

to any image property of interest. For example intensity, color or texture. This process

is considered an essential component of any image analysis system and the problem

associated with it have received considerable attention [6], [7]. Image representation

and description are critical for successful detection and recognition of objects in a

scene. After an image has been segmented into object and background regions, one

intends to represent and describe them in characteristic feature for computer

processing during pattern recognition or in quantitative codes for efficient storage

during image compression. Image description deals with the extraction of quantitative

information from the previously segmented image regions.

High-level processing involves recognition and interpretation. It is using

statistical classifiers or multi-layer neural networks of the region of interest. These