BILLING SYSTEM

Submitted To:

Ms. Adwitiya Sinha

Submitted By:

Akshit Bansal(10104669)

Saumya Tripathi(9910103583)

USING HAND GESTURES

OUR PROJECT: BILLING SYSTEM USING

HAND GESTURES

Billing system is developed taking into consideration problems faced

by a disabled person

The person can go to the shop

Select items from the menu

Enter it’s quantity using hand gesture

Confirm/Cancel order

Add more items as per the requirement

Pay the bill

Give his feedback

OBJECTIVES

To make a unique billing system that is developed for people with

hearing and/or speech disability

It is based on gesture recognition because the disabled person

lacks the ability to communicate

The system generates the bill

The person can also give his feedback

KEY FEATURES

The Billing system is being implemented using openCV in Microsoft

Visual Studio 2010

Person enters a shop, if he wants to place an order he has to select

an item from the menu

Selection will be based on color recognition. He has to select the

items using blue color

Person can enter the quantity of the selected items. The quantity will

be entered using gesture recognition

Algorithms and techniques used are:

1) Contouring

2) Convex Hull Detection

3) Defects calculation

4) Thresholding

5) Fingers Earth’s Movers Distance(FEMD)

Person can perform the following functions:

1) Cancel order

2) Confirm order

3) Add more items

4) Give feedback

ADVANTAGES TO THE DISABLED PERSONS

People with some disabilities can communicate easily with the helpof sign languages

Disabled people can easily place their orders using our billing system and it can be understood by any person

The disabled person can:

1) Go to the shop

2) Select the item he wants to order

3) Enter it’s quantity

4) Confirm/Cancel order

5) Pay the bill with the help of the system

He can avoid the tedious and time consuming task of explaining himself to the sales person

SIGNIFICANCE OF PROBLEM

Now the disabled person is no more dependent on anyone else like

an interpreter for explaining himself to the other people.

The system does not depend on anyone particular sign language.

So anyone can use this system with some basic English language

knowledge.

No use of database for this system reduces the image processing

time and makes the system more efficient.

ALGORITHMS USED

Hand segmentation

1) Is the detection of hand from the whole image

2) Background subtraction is used to find the hand area

3) The system acquires image of the background

4) Then it starts capturing the image when hand is shown in front of

the camera

5) Now the previous image is subtracted from the new one

6) Hence the background is removed and we obtain image of the

hand

Thresholding

1) Used to separate out the regions of image as per the users

requirement

2) The RGB image is converted into gray scale image

3) The gray scale is then converted into black and white image

Contouring

1) Contour is a curve that connects points of an image with same

color or intensity

2) To identify hand gestures we detect contours of all objects that

exist in the threshold image

3) Identify the contour with the maximum area

Fingertip detection

1) No of fingers can be detected by constructing a polygon of the

hand contour

2) Then a convex hull is made around the polygon

3) Now the defects are calculated

4) No of fingers = defects – 1

FEMD algorithm

1) No of fingers can be detected by measuring the distance of all

contour points from the centre

2) The angle between the fingers are calculated

3) No of fingers = angles + 1

TOOLS AND TECHNOLOGIES USED

openCV

1) Has C++, C, Python and Java interfaces and supports Windows, Linux, Mac OS, iOS and Android.

2) openCV was designed for computational efficiency and with astrong focus on real-time applications.

3) The library can take advantage of multi-core processing.

MICROSOFT VISUAL STUDIO 2010

It is used to develop console and graphical user interface applications along with Windows Forms or WPF applications, web sites, web applications, and web services in both native code together with managed code for all platforms supported by Microsoft Windows.

PROJECT REQUIREMENTS

Functional Requirements

1) The system shall be able to select the item as pointed by the user

2) Acquire and process the images of the numerals

3) Show the list of items available in the shop

4) confirm or cancel the order

5) Calculate the bill of the user and display the results

Non-Functional Requirements

1) System requirements - 32 bits windows, webcam and blue colored

objects

2) Performance - The system shall not consume much time to process

the image and response

3) Security - The system has no security constraint

4) Reliability - The system shall be able to process the images almost

accurately

5) Maintainability - System is easy to maintain

FINDINGS

1) Background of an image plays a very important role while

developing any system based on image processing.

2) The efficiency of most of the image processing algorithms and

techniques depends significantly on the background.

3) The color of an image, its brightness and the distance of the objectfrom the webcam in an image also need to be paid full heed to

maintain the accuracy.

Billing system using “Hand Gestures” has

been divided into a number of modules

List of modules is as follows:

Load image of background.

background subtraction.

convert RGB-black and white.

contouring

polygon formation

Convex hull detection

defects calculation

fingers = defects - 1

quantity determined.

FEMD algorithm

angle calculation

fingers = angles + 1

select item to buy(tea, coffee, samosa)

selection via(centroid formation)

enter the quantity

confirm/cancel order

add more items

price = quantity * price per unit

give feedback

CONCLUSION

A billing system based on the gesture recognition has been

developed that serves the basic purpose of placing an order for a disabled person.

The person can select the item of his choice, show the quantity

needed, confirm the order, add more items if he wishes to and get

the bill he has to pay all just with the help of gestures.

The person also has the option of cancelling the order if in case he

changes his mood and wants to cancel it.

He can also give his valuable feedback thus helping the place improve its quality and standard.