design of hand glove for wireless gesture control of robot · design of hand glove for wireless...

Design of Hand Glove for

Wireless Gesture Control of

Robot

Akshunya Mishra1, Saksham Malhotra2,

Hemender Pal Singh3, Ruchira4

Electrical &Electronics Engineering

Department

Amity University,

Uttar Pradesh1,2,3,4

[email protected]

[email protected]

[email protected]

[email protected]

Apr 14-15, 2017

Abstract

These days electronic devices are taking over the

world, and have become a huge part of our life. The

use of smartphone has become a daily necessity and

new groundbreaking innovations allow physically

challenged people to be able to live a better life.

These electronic devices can be controlled in various

ways, but the most popular method for control

nowadays is gesture control. In the present work

control of wheel based robots by hand gestures has

been done with the use of accelerometer, flex

sensors and metal contacts between fingers of the

glove, and then controlling the robot wirelessly

using radio frequency module. The device designed

in the present work can sense upto four gestures for

various operations of the wheel based robot.

International Journal of Pure and Applied MathematicsVolume 114 No. 8 2017, 69-79ISSN: 1311-8080 (printed version); ISSN: 1314-3395 (on-line version)url: http://www.ijpam.euSpecial Issue ijpam.eu

69

Application of this device in robotics can vary from

pick & drop to bomb dismantling. However, this

technique can also be used for the control of other

electronic devices such as smartphones, laptops,

entertainment equipment in our houses as well.

Keywords— NRF24L01; AtMega32 microcontroller;

Accelerometer; flex sensors.

1. Introduction

Ever since electronics was discovered, it has become

an integral part of our life. Over the years control of

electronic devices such as smartphones and robots have

seen groundbreaking research. Nowadays we are

moving towards automation where electronic devices

can think on their own.[1]

However sometimes we need a human to control

the robot instead of relying on a piece of code. This

control of robots may be achieved by either wired or

wireless method. Both have their pros and cons. Among

these, the recent method of gesture control has become

quite popular. The reason behind it being that human

hand gestures are natural and hence give a more

intuitive form of interaction with the robot. By making

this interface wireless using radio frequency, it becomes

possible to control the robot very easy and user friendly

[2-4].

Nowadays number of different wireless robots are

being manufactured and used in different fields of

industry. This paper describes a wireless gesture control

for robots using AtMega328 microcontroller,

AtMega2560 microcontroller NRF24L01 wireless

module, A gesture sensing glove (consisting of a

accelerometer, flex sensors and finger contacts) and a

two wheel drive robot.

In the present work, the glove uses accelerometer for

sensing the motion of the wrist, the flex sensors for

motion of fingers and thumb and finger contacts for

sensing the gesture of touching two or more fingers. A

single or combination of above motions is used as a

gesture which is transmitted by the glove using NRF

transceiver to the robot unit, which processes that

signal to decide how to move based on the gesture made.

International Journal of Pure and Applied Mathematics Special Issue

70

The robot uses 2 PMDC motors driven by L298n dual H-

bridge motor driver module and controlled by another

AtMega328 microcontroller.

This gesture control technique has been

applied for different purposes. For example in medical

field it can be used by disabled patients to send

messages by doing a subtle gesture, it can also be used

to control all the electronic devices such as LEDs, fans,

TV, Air conditioners etc, it can help in giving feedback

in Augmented reality devices, it can also be paired

with a smart phone to provide a more interactive

platform to control and interact with it, it can also be

used to control a modern electric vehicle via pairing it to

the master control of that vehicle using Human-

Machine-Interface technology and hidden Markov

method and it can be used as an interactive input device

for computers[5-14].

2. Hardware Design And Implementation

The hardware design of the present work involves the design of two

main parts i.e. gesture sensing glove and robot.

Design of Gesture Sensing Glove Fig. 1 shows the picture of the designed glove. The

hardware design of gesture sensing glove uses the three main components -

sensor, microcontroller and transmitter. The components used for the

design of glove have been listed in Table No. 1.

Table 1. List of components used to design the Gesture Sensing Glove.

S.No. Equipment Name Quantity

1. Flex Sensor 5

2. Contact Sensor 5

3. Accelerometer

Sensor

1

4. NRF24L01

Wireless Module

1

5. AtMega2560

Microcontroller

1

6. Microcontroller

shield

1

7. Connecting wires As

required

8. Glove 1


71

Fig.1 Gesture Sensing Glove

1. Sensor The sensing unit consists of three sensors discussed below -

Accelerometer, which is used to sense the position of glove. This senses the

position in all three axes and gives its corresponding analog values to the

microcontroller [20].

Flex sensors, which are employed for each finger to detect the amount of

bend a finger makes and then its internal resistance changes accordingly to

give output to microcontroller.

Contact sensors, which are placed between fingers for sensing position of

finger with respect to each other.

2. Microcontroller The microcontroller employed for the gesture detecting

glove is AtMega2560 microcontroller. It receives the information from each

sensor and processes the data to recognize the gesture. It then sends a

control integer to the receiver. In the current work we have used four

actions which are Forward, Backward, Right and Left and have set the

values of control integer according to it.

3. Transmitter We have used NRF24L01 wireless transceiver module as a

transmitter in the design of our glove. It works on 2.4 GHz (radio) frequency

band. It has a range of 100m as tested which can be extended to more than

500m if an antenna is employed. Other features of the NRF radio frequency

module include that it does not require line of sight and is able to

communicate even through obstacles, it has low cost, easily available and

universally applicable[21]. It transmit the control integer to the AtMega328

microcontroller used in robot for further processing.


72

Fig. 2 Flowchart describing the design of Gesture Sensing Glove.

3. Hardware Design of the Robot

Fig. 2 shows the picture of the robot. The robot consists of

majorly three components i.e. the receiver, the microcontroller

and the actuating unit (2 PMDC motors). The list of components

used to design the Gesture Sensing Glove has been tabulated in

Table 2.

Receiver We have used NRF24L01 wireless transceiver module as

a receiver in our design of robot. It is the same module which has

also been used in the design of glove and has been discussed

there. It receives the control integer from the gesture sensing

glove and sends it to the AtMega328 microcontroller used in

robot for further processing.

Fig. 2 Wheel based Robot prototype

Table 2. List of components used to design the Gesture Sensing

Glove.

S .No. Equipment Name Quantity

1. AtMega328

Microcontroller

5


73

2. L298n Motor Driver

IC

5

3. NRF24L01 wireless

module

1

4. PMDC Motor 1

5. Robosoft Compact

Chassis

1

6. 1602 LCD Display 1

7. Connecting wires As

required

Microcontroller The microcontroller employed for the robot is

AtMega328 microcontroller. It processes the control integer

received from the glove and determines which action is to be

executed. It then controls 2 PMDC motors accordingly to get the

desired action. In the current work we have used 4 actions which

are Forward, Backward, Right and Left. A 16x2 LCD is used to

display the current action being executed simultaneously.

Actuator The actuators are two PMDC motors which are

controlled using L298n dual-h bridge motor driver module. With

the help of this module we are able to provide 9V supply to the

motors to get 60 rpm rotation to drive the robot. The motors used

are high torque motors to provide high torque in order to drive

even a heavy robot if required. The dual-h bridge L298n module

can be used to control the speed of the motors as well using

PWM. We have used it control the direction of both motors in

same direction or opposite directions to execute forward,

backward, left or right action accordingly.

The above discussed hardware components are arranged on a

robosoft compact robot chassis which helps to make the robot

more compact and is hence able to move more freely even in an

obstacle plain surface.

4. Software Design And Implementation

In the present work, the programming of AtMega328

microcontroller and AtMega2560 has been done in Embedded C

language. Fig.4 and Fig.5 shows the program structure for the

glove and robot unit respectively. The figure 4 describes the

algorithm for working of glove. The gesture sensing glove

receives analog input from the the three sensors. The gesture is


74

sensed using those input values and then a control integer is

selected for the gesture. This control integer(i.e. x) is then

transmitted to the robot unit.

Fig. 4 Flow chart for Gesture Sensing Glove.

The figure 5 explains the algorithm for working of the robot. It

receives the control integer from the gesture sensing glove and

then uses that integer to decide which action is to be

implemented. Each integer value has been assigned a particular

action which is to be performed by the robot

Fig. 5 Flowchart for program structure of Robot.


75

5. Applications and Future Scope

The present work really shines in it‗s applications. It has a wide

range of applications as it can be used for control of various

electronic devices. In the present work, it has been used to

control a robot. However this technique is not bounded by the

given application. Some of the other applications of the gesture

sensing technique are:

It can be used in medical field for disabled patients. As it

can be paired with a person‘s cellphone and can be used by

the patient to send messages like ―I need water‖ or ―I need

my pills‖. By doing a subtle gesture.

It can be used to control all the electronic devices such as

LEDs, fans, TV, Air conditioners etc. in our home to

practically implement the concept of ―smart home‖ and

IOET(internet of everything).

It can be used for making games more interactive by

providing a much natural way to input data for RPGs etc.

It can help in giving feedback in Augmented reality

devices such as Google glass.

It can be paired with a smart phone to provide a more

interactive and fun way to control and interact with it.

It can be used by physically disabled people who can‘t

speak to sense the hand signs as gestures and then use a

display or speakers to convey that message.

It can be used to control a modern electric vehicle via

pairing it to the master control of that vehicle using

Human-Machine-Interface technology and hidden Markov

method.

It can be used as an interactive input device for computers.

6. Conclusion

This paper explains the working of hand gesture sensing

technique using three sensors i.e. accelerometer, flex sensors and

finger contact sensors and use it to drive a two wheel drive robot

in wireless mode using radio frequency. This can be applied in

distress situations for search and rescue or can be used for bomb

detection and dismantling if a robot arm is attached which again

can be controlled using the glove. Some of the important features


76

of the glove are: it can be paired with most of the electronic

devices, can be used for wide range of applications from medical

care to leisure, precise sensing of various gestures etc. The

gesture sensing technique can also be used for other applications

which include making an easy way for disabled patients to

communicate with their caretaker, making an interactive way for

deaf people to interact by giving the hand gestures, Controlling

the electronic devices in a house using different gestures with the

concept of IOT or to provide an interactive platform for

controlling a smart phone. Table 3 provides the gestures

recognized by the glove and their application in the present work.

Table 3 Gesture and the related application

S.No. Gesture Application

1. Leaning the hand forward and

clenching the wrist.

Move the Robot forward.

2. Leaning the hand backward

and releasing the clench.

Move the Robot backward.

3. Tilting the hand to the left and

clenching the wrist.

Move the Robot to the left.

4. Tilting the hand to the right

and releasing the clench.

Move the Robot to the

right.

7. References

[1] Karam, M. ―A framework for research and design of

gesture-based human computer interactions‖, Thesis.

Southampton: University of Southampton, (2006)

[2] Adnan Ibraheem and Rafiqu Izaman Khan, ―Survey on

Various Gesture Recognition Technologies and

Techniques‖, International Journal of Computer

Applications (0975 –8887) Volume 50 – No.7, July 2012.

[3] Billinghurst, J. ―Gesture Based Interaction‖, In Haptic

Input, Chapter 14. .: Buxton, (2002).

[4] C. Lee and Y. Xu, ―Online, Interactive Learning of

Gestures for Human/Robot Interfaces‖, In IEEE

International Conference on Robotics and Automation,

(1996), pp. 2982-2987.


77

[5] F. Parada-Loira, E. González-Agulla, and J. L. Alba-

Castro, ―Hand gestures to control infotainment equipment

in cars ,‖ in Proc.IEEE Intell.Veh. Symp., 2014, pp. 1–6.

[6] R.-D. Vatavu, ―User-defined gestures for free-hand tv

control,‖ in Proc. Eur. Conf. Interact. TV Video, 2012, pp.

45–48.

[7] Zu Zhang, Xiang Chen, Wen hui Wang, ei Hai Yang,

Vuokko Lantz, ―Hand gesture recognition and virtual

game control based on 3D accelerometer and EMG sensors

―, IUI‘09 Proceedings of the 14th international conference

on Intelligent user interfaces, February 08 - 11, 2009 .

[8] T. Shiratori and J. K. Hodgins, ―Accelerometer-based user

interfaces for the control of a physically simulated

character‖, ACM Transactions on Graphics, vol. 27, no. 5,

(2008), pp. 1–9.

[9] Ukita, N., Kono, Y., Kidode, M. ―Wearable Vision

Interfaces: Towards Wearable Information Playing in

Daily Life‖, Paper. Japan: Nara Institute of Science and

Technology, (2002)

[10] Luigi Lamberti & Francesco Camastra, ―Real-Time Hand

Gesture Recognition Using a Color Glove‖, 16th

international conference on Image analysis and processing:

Part I (ICIAP), pp. 365-373, 2011

[11] Ankita Saxena, Deepak Kumar Jain, Ananya

Singhal,―Hand Gesture Recognition Using an Android

Device‖, International Conference on Communication

Systems and Network Technologies , 2014.

[12] C. Helen, H. Brian, and B. Richard, ―Sign language

recognition,‖ in Visual Analysis of Humans. London, U.K.:

Springer-Verlag, 2011, pp. 539–562.

[13] yal, P. K. Bora, ―A study on static hand gesture

recognition using moments‖, In Proceedings of

International Conference on Signal Processing and

Communications (SPCOM), (2010), pp. 1-5.

[14] Starner, T., Pentland, A. ―Visual Recognition of

American Sign Language Using Hidden Markov Models‖,

Paper. Cambridge, MA Massachusetts: Massachusetts

Institute of Technology, (1995).


78

design of hand glove for wireless gesture control of robot · design of hand glove for wireless...

Documents