hand motion-based remote control interface with...

International Journal of Advanced Robotic Systems

Hand Motion-Based Remote Control Interface with Vibrotactile Feedback for Home Robots Regular Paper

Juan Wu1, Guifang Qiao1, Jun Zhang1, Ying Zhang1 and Guangming Song1,* 1 School of Instrument Science and Engineering, Southeast University, Nanjing, China * Corresponding author E-mail: [email protected] Received 9May 2012; Accepted 3May 2013 DOI: 10.5772/56617 © 2013 Wu et al.; licensee InTech. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract This paper presents the design and implementation of a hand-held interface system for the locomotion control of home robots. A handheld controller is proposed to implement hand motion recognition and hand motion-based robot control. The handheld controller can provide a ‘connect-and-play’ service for the users to control the home robot with visual and vibrotactile feedback. Six natural hand gestures are defined for navigating the home robots. A three-axis accelerometer is used to detect the hand motions of the user. The recorded acceleration data are analysed and classified to corresponding control commands according to their characteristic curves. A vibration motor is used to provide vibrotactile feedback to the user when an improper operation is performed. The performances of the proposed hand motion-based interface and the traditional keyboard and mouse interface have been compared in robot navigation experiments. The experimental results of home robot navigation show that the success rate of the handheld controller is 13.33% higher than the PC based controller. The precision of the handheld controller is 15.4% more than that of the PC and the execution time is 24.7% less than the PC based controller. This means that the

proposed hand motion-based interface is more efficient and flexible. Keywords Hand Motion Recognition, Home Robot, Control Interface, Handheld Controller

1. Introduction

In recent years, more and more mobile robots have moved away from industry to enter home environments. As the size and the cost have decreased significantly, the home robot is now available for use as one of the most popular consumer electronic products [1]. More and more home robots are now working around us and they help us a lot in our daily lives. A wide variety of home robots have been proposed to do housework such as cooking, cleaning, houseplant watering and pet feeding. They are also being widely used in home security, entertainment, rehabilitation training and home care for the elderly [2-5]. As the home robots get closer in our daily lives, the question arises: How to interact with them? Complicated control interfaces designed for skilled

1Juan Wu, Guifang Qiao, Jun Zhang, Ying Zhang and Guangming Song: Hand Motion-Based Remote Control Interface with Vibrotactile Feedback for Home Robots

www.intechopen.com

ARTICLE

www.intechopen.com Int J Adv Robotic Sy, 2013, Vol. 10, 270:2013

workers and experts are not suitable for ordinary home users. They prefer simple and natural interaction with home robots through voices and gestures. This requires a user-friendly interface that allows the robot to understand voice and gesture commands. The voice interface is suitable for a simple call-and-come service for home robots [6] but it is not suitable for continuous remote control and usually it cannot work normally due to the interference of ambient noise. Therefore, hand gesture or hand motion-based interfaces are more suitable for control of home robots. Several methods have been proposed for hand gesture recognition, such as marker-based gesture recognition, vision-based motion recognition, haptic-based motion recognition and EMG-based hand motion recognition [7]. In [8], a real-time hand gesture recognition system based on difference image entropy using a stereo camera is introduced. The proposed method shows an average recognition rate of 85%. Other hand gesture recognition methods use wearable sensors, accelerometers, angular rate sensors and data gloves to detect hand gestures. In [9], a data glove is used for 3D hand motion tracking and gesture recognition. In [10], the authors proposed a set of recognition algorithms: TC, FAcaGMM and FEC. They evaluated the algorithms on a data glove with 13 different types of grasps and ten in-hand manipulations. In [11-12], the authors used a wearable sensor to recognize hand gestures and daily activities in a smart assisted living system to help elderly people, patients and the disabled. In [13], a wearable wristwatch-type controller is introduced to offer a unified way to control various devices. The controller uses simple and effective hand motion gestures for controlling devices. In [14], an accelerometer combined with a visual tracker is used to detect hand movements in a system for human computer interaction. As compared to the wearable sensor-based control interface, the handheld interface is more suitable for controlling home robots. Since people are used to this kind of control mode they can learn to use it quickly and easily. Wearable sensors fixed on the body of a person are not convenient and flexible to use when he or she wants to control a robot. In [15], the authors introduce a handheld interface system for 3D interaction with digital media contents. The system can track the full six degrees-of-freedom position and orientation of a handheld controller. The gesture recognition depends on acceleration and position measurements. A hand-gesture-based control interface for navigating a car robot is introduced in [16]. A three-axis accelerometer is adopted to record the hand trajectories of a user. In [17-18], the authors proposed a handheld system to recognize hand motions. The system contains three MEMS accelerometers and a Bluetooth wireless module.

Hand gestures usually can be described by the tilt angles of the hand. So tilt sensors can be used to detect the angles of a hand gesture. In [19], a tilt sensor was designed using standard accelerometers. The accuracy of the tilt sensor is 0.3° over the full measurement range of pitch and roll. In [20], the authors used a Kalman filter to estimate inclination from the signals of a three-axis accelerometer for measuring inclination of body segments and activity of daily living (ADL). This method is nearly twice as accurate as the methods based on low-pass filtering of accelerometer signals. In [21], a three-axis accelerometer and a dual-axis angular rate sensor are utilized for orientation sensing in mobile virtual environments. It presents a technical and theoretical description for implementing an orientation-aware device, which is used for navigating large images spread out on a virtual hemispheric space in front of the user through a mobile display. In [22], a novel approach for hand gesture recognition is introduced. In [23], a hand gesture recognition system is implemented to detect hand gestures in any orientation. The system is integrated on an interactive robot, allowing for real-time hand gesture interaction with the robot. Gestures are translated into goals for the robot, telling him where to go. Vibration motors are usually used in handheld interfaces to provide vibrotactile feedback. In [15], a vibration motor and a voice-coil actuator are adopted to achieve vibrotactile feedback. In [24], vibrotactile actuators are used in a handheld input device for providing spatial and directional information. In [25], a vibrotactile feedback approach for posture guidance is introduced and in [26], multi-day training with vibrotactile feedback for virtual object manipulation is introduced. Experimental results show that participants are able to utilize the vibrotactile feedback to improve the performance of virtual object manipulation. In this paper, we present a hand motion-based remote control interface with vibrotactile feedback for home robots. A handheld controller is proposed to implement hand motion recognition and hand motion-based robot control. The handheld controller can provide a ‘connect-and-play’ service for the users to control the home robot. Meanwhile, it implements the function of visual and vibrotactile feedback. Six simple hand gestures are defined for locomotion control of the robot. They are easy to use for untrained people. A three-axis accelerometer is used to detect the hand motions of the user. The recorded acceleration data are analysed and classified to corresponding control commands according to their characteristic curves. A vibration motor is used to provide vibrotactile feedback to the user when an improper operation is performed. The remainder of this paper is organized as follows. Section 2 introduces the overall system architecture that

2 Int J Adv Robotic Sy, 2013, Vol. 10, 270:2013 www.intechopen.com

includes the hardware dein Section 3. Thand motion4. The experprototype syremarks are g

2. System des

The conceptucontrol system

Figure 1.Concrobot by rotatirecognized by commands.

The system csurveillance axis accelerointeraction. Imotions of thto robot conthe robot applications. establishing handheld conand TCP/IP. system is shrobot with applications. been present

Figure 2.The h

The softwareFigure 3. Thcontroller inc

handheld conesign of the haThe methods

n-based robot rimental resuystem are givgiven in Sectio

scription

ual architectum is shown in

ceptual system ing the handhel the handheld c

consists of a hrobot. The haometer and It can sense he user and co

ntrol commandto impleme

There are wireless comntroller and t The home s

hown in Figuran on-board

The detaileded in [5].

home surveillan

e structure of he embeddedclude a hand

ntroller and thandheld controf hand motiocontrol are pr

ults on the peven in Sectioon 4.

ure of the han Figure 1.

architecture. Thld controller. Thcontroller and c

handheld contandheld contr

a vibration the accelerat

onvert those ads. The comment various three alterna

mmunication lthe robot, i.e.,surveillance rre 2. It is a pd camera fo

d design work

ce robot.

the control syd programs motion recog

e home robot.roller is preseon recognitionresented in Secerformance ofon 5. Conclu

and motion b

he user controlhe hand motionconverted to m

troller and a hroller has a th

motor for ions of the h

acceleration vamands are sen

motion contive schemas

links between, Wi-Fi, Bluetobot used in

palm-sized moor home secuk of this robot

ystem is showin the hand

gnizer, a graph

. The ented n and ction f the

uding

based

ls the ns are

motion

home hree-user

hand alues nt to ntrol s for n the tooth

n our obile urity t has

wn in dheld hical

usergeneimplevedockcurrmou

Figusyste

3. H

A p5. Tits portby micWi-vibr

Figu

ThethatneedaccegoocoreIntecont‘con

r interface erator. The e

plement basic el behaviours king and rechrent status of unted or ceilin

ure 3.Software stem.

Handheld cont

rototype of thThe size of the

weight is abtable equipme

hand motiorocontroller uFi adapter, aration motor, a

ure 4.Hardware

core board it provides mded here. Aeleration acqud real-time p

e board throuernet via thtrolling. Ther

nnect-and-play

(GUI) and embedded prlocomotion bsuch as vid

arging. The obthe robot are

ng-mounted ca

tructure of the h

roller

he handheld co controller is

bout 250g. Thent designed fons. It incl

unit (MCU), a a touch screeas shown in Fi

structure of the

is a small emmost of the A standaloneisition and pr

performance. ugh a serial phe Wi-Fi adrefore, the coy’ service fo

a vibrotactrogram in th

behaviours andeo transmissbstacle inform also useful wameras are av

hand motion-ba

ontroller is sho137mm×90mm

he handheld for controllingludes a corthree-axis acc

en, two USB igure 4.

e handheld cont

mbedded comcommunicati

e MCU is rocessing in orIt communicport. Users ca

dapter for rontroller can

or the users.

tile feedbackhe robot can

nd other high-ion, cruising,

mation and thewhen no wall-ailable.

ased control

own in Figurem×31mm and

controller isg home robotsre board, acelerometer, a

ports and a

troller.

mputer systemon interfacesadopted for

rder to ensureates with thean access the

remote robotprovide theAnd it can

k n -, e -

e d s s a a a

m s r e e e t e n


www.intechopen.com

provide sufftouchscreen Lfeedback foralso allows configure the

Figure 5.Implesize of the han(L×W×H). Its w

4. Hand moti

4.1 Coordinate

The first stepby the handhframes. Thesystem of thThe three-axiorigin of thcoincide withdirection of vertically oudirection of right side wscreen. Thenthe front dire

Figure 6.3-dimcontroller.

ficient bandwiLCD module

r the users totouch input

e control syste

emented prototyndheld controlleweight is 252g.

ion-based con

e System

p to implemeheld controllee three-dime

he handheld cis acceleromet

he coordinate h the axes depthe Z-axis is t from the scrthe X-axis is

when a user hn, the positiveection in which

mensional coord

idth for videois used to giv

o implement rfor the users

em.

ype of the handr is 137mm×90m

ntrol

enting hand mer is the definensional Cartcontroller is shter is assumed

system and picted in Figudefined as th

reen of the dedefined as th

holds the dev direction of h the device is

inate system of

o transmissiove necessary vremote contros to manage

held controller.mm×31mm

motion recognnition of refertesian coordihown in Figud to be fixed to

its sensing ure 6. The poshe direction gevice. The poshe direction tovice and seesthe Y-axis wis pointing.

the handheld

on. A visual ol. It

and

. The

nition rence inate

ure 6. o the axes

sitive going sitive o the s the ill be

4.2 H

Therotaaviayawoftecomfirstrotarotacan rota In ocontone of mfor perfcomfor tmeathe

Figucontreprusercomtilts com180°user

We robo

Hand Motion R

handheld coated about thation terminol

w, pitch and rn represente

mposed of thot rotation aboation about thation about th

be achievedations.

our work, alltrol are simplof the three

more than onethis kind o

formance is mplicated gestu

the user to leaasured accelerhandheld con

ure 7.Hand mottroller is in the esents the com

r tilts the contrmand to move

the controllemand to make to the right. S

r tilts the control

have designeot navigation

Recognition

ontroller is a 3he three ortho

ogy, these rotoll. The orien

ed by usingose three elemout the Z-axihe X-axis of ishe Y-axis is cad by composi

of the plannele gestures, eaelemental rota

e elemental roof application

not accepures are used.arn and use. Sration values

ntroller.

tion set for conright hand of tmand to turn toller to the rigthe robot forwar forward (baa U-turn whe

S represents theller 180° to the l

ed six simpleand control,

3D rigid bodogonal axes. tations will bentation of the g Euler angmental rotationis is called ys called pitchalled roll. Aning those thr

ed hand motiach of which ations. Gestur

otation are toon. The real-

ptable when . It will also b

So we directlyto describe th

ntrolling the hothe user in this the robot right ght (left). F (B) ard (backward)ackward). U

en the user tiltse command to left.

e hand motioas shown in

y that can beAccording toreferred to asrigid body isles that aren angles. The

yaw, the nexth and the lastny orientationree elemental

ions for robotcontains onlyres composedo complicated-time control

too manybe too difficulty use the threehe rotation of

ome robot. Theexample. R (L)(left) when therepresents the

) when the userrepresents thes the controllerstop when the

ons for homeFigure 7. The

e o s s e e t t

n l

t y d d l

y t e f

e ) e e r e r e

e e


hand motionA user can easpecial trainiany two of thhandheld codifferent hanthese hand commands. Tensure that asystem is imp R (L) represewhen the userepresents th(backward) w(backward). Uwhen the useback of thecommand to the left with t Group X-

axis 1

X-axis × ×

2 Y-axis

↗ ↘

↗, ↘ ↘,↗

Table 1. Handrobot. ‘↗’ reprincreases mmonotonicallydecreasing late

The handhelhand motionintegrated acaxes change sfor any propaxis will chanchange signibecause all theither a singBased on tacceleration therefore ide

Hand motion He

R 0xL 0xF 0xB 0xU 0xS 0x

Table 2.The mcommands.

n patterns are asily repeat aling. There is ahe hand motioontroller to nd motion pa

motions to This kind of usa user-indepenplemented.

ents the commer tilts the cohe commandwhen the usU represents ter tilts the cone controller stop when ththe back of the

Y-axis

HandMotion

↗ F ↘ B × R × L × U × S

d motion chararesents that the

monotonically. y. ‘↗, ↘’ reper. ‘↗, ↘’ is oppo

ld controller cns by analysinccelerometer. simultaneouslposed hand mnge little whilificantly as cohe proposed hgle roll rotatithis principlcharacteristicntify different

Rader Length M

xAA 0x07 xAA 0x07 xAA 0x07 xAA 0x07 xAA 0x07 xAA 0x07

mapping between

all commonlyl these actionsa significant don patterns, saccurately diatterns. Then

correspondinser interface dndent and use

mand to turn thntroller to the

d to move ther tilts the cthe command

ntroller 180° tofacing up.

he user tilts thee controller fa

d n

Position oMAX

FinishingBeginninFinishingBeginnin

Middle×

acteristics for coe N-axis (N = X

‘↘’ repreresents increa

osite to ‘↘,↗’

continuously ng the accelera

The acceleraly during the h

motion, the acle that in the oompared to t

hand motions ion or a single, we can cs of every ht hand motion

Robot CommanMotion D/A S

‘R’ * ‘L’ * ‘F’ * ‘B’ * ‘U’ * ‘S’ *

n hand motion a

y used in dailys well withoutdifference betwo it is easy foistinguish amwe can tran

ng robot condesign allows uer-friendly con

he robot right e right (left). Fhe robot forwcontroller forwd to make a U-o the right withS represents e controller 18acing up.

of Position MIN

g Beginninng Finishing Beginnin

ng Finishin ×

Middle

ontrolling the hX or Y) acceleresents decreasing at first

recognizes thation values otions in the thand motionscceleration in

other two axesthe previous are designed t

gle pitch rotaget the un

hand motion ns.

nd Speed Tail C

* 0x55* 0x55* 0x55* 0x55* 0x55* 0x55

and robot contr

y life. t any ween r the

mong nslate ntrol us to ntrol

(left) F (B) ward ward -turn h the

the 80° to

of

ng ng ng ng

e

home ration easing

and

he six of the three s, but n one s will

one, to be

ation. nique

and

CRC* * * * * *

rol

The7, hdatadiffacceshowmotacce

Figu

Figu

Figu

e six hand mohave been cara to assess erent hand eleration chawn in Figurtions are reelerations in t

ure 8.Acceleratio

ure 9. Accelerati

ure 10.Accelerat

otion trials, wrried out to g

the accelermotions.

anges duringres 8-13. Theeflected by the three axes

on characteristic

on characteristi

ion characterist

which are shogather typicalration charaExample pl

g the hand e directions the variatio

s.

cs of the R moti

ics of the L moti

tics of the F mot

wn in Figurel accelerationacteristics oflots of themotions areof the handn trends of

on.

ion.

tion.

e n f e e d f


www.intechopen.com

Figure 11.Acce

Figure 12.Acce

Figure 13.Acce

The hand mogroup is wheX-axis, whilerevolved arorotation axis[-2m/s2, +2mmotion recoghandheld conin Figure 8 a

eleration charac

eleration charac

eleration charac

otions can beere the handhee the other oound the Y-s almost rem

m/s2]. Thereforgnition is to fntroller is rev

and Figure 9,

cteristics of the B

cteristics of the U

cteristics of the S

divided intoeld controller one is where-axis. The acains within are, the first sfind the axis avolved. For exthe Y-axis acc

B motion.

U motion.

S motion.

two groups. is revolved bythe controlle

cceleration of a threshold rstep of the haround whichxample, as shceleration rem

One y the er is

f the ange hand h the hown mains

almX-axsign TheotheTheTabis pof ehana 15Thethe smoSamare obtaFinaspeerobo

FiguMoticontwhic

4.3 R

Oncidencontcontin Ffeeddetehanrecocontupdthe

ost unchangexis and the Znificantly.

second step ier two axes to variation trenle 1. The algoresented in Fiach axis are sd motion is de50 acceleration data from thmoving-avera

oothly. By comple_NEW and

their positionained and thally, we can oed and target ot control com

ure 14. The proion represents trol the robot sch the robot nee

Robot Control C

ce one of thentified, it will trol commandtroller. This pFigure15. If thdback from thected. Then thd motion is

ognized corretrol command

dated on the Gvibration mot

d during the Z-axis accelera

is to compare o determine wnds of the six hrithm of the vigure 14. The ampled with etected with an data sequene accelerationage filter to momparing thed Sample_OLDns in the sequhe variation obtain the stydistance/angle

mmands.

ocess of the varthe hand motio

speed. D/A repeds to adjust.

Commands

e hand motiobe converted

d by the controrocess of the c

he controller he robot, a nhe controller correct or noctly, it will b

d. All operatinGUI. The users tor.

R/L roll motations increas

e the variationwhich the hand

hand motionsvariation trend

values of thea fixed 10ms

a 1.5s time spance for each h

n sensor firstlymake the data e two continD, MAX(i) anuence respecttrends can b

yles of hand me, which are n

riation trends ron type. The Sppresents the dis

ons shown into the corresp

ol algorithm rcontrol algoridoes not recenew hand m

will identifyot. If the hanbe mapped inng status mes

s also can get f

tion while thee or decrease

n trends of thed motions are.s are shown inds recognitione accelerationsinterval. Each

an. We can gethand motion.

y need to passcurve change

nuous values,d MIN(j) (i, jtively) can bebe examined.motion, robotneeded for the

ecognized. Thepeed is used tostance or angle

n Figure 7 isponding robotrunning in thethm is showneive the errorotion will be

y whether thend motion isnto the robotssages will befeedback from

e e

e .

n n s h t . s e , j e . t e

e o e

s t e n r e e s t e

m


Figure 15. Con

The seven-byLength, Motfields. The HThe Motion fDistance/Anghandheld convelocity of thwhether the details of the

4.4 Graphical

A graphical hand motionlocally and srobot is in a with only hisbe designed users to imhandheld coshaped virtuthe handheimmediately For example,perform a sttrack will liggenerated. If light, it indidefault levegenerated. Hmethod to shard to reach

ntrol algorithm o

yte commandtion, Distance

Header, Lengthfield is consistgle field is ntroller. The She hand motio

commands ae mapping rela

User Interface

user interfacn based controstays in sightremote site a

s or her nakedto provide ne

mplement remontroller is shal track is use

eld controlleknow what

, if the user titandard R m

ght up to remif the circle in ticates that theel posture wHowever, westop the roboth and hold by

of the handheld

d packet conte/Angle, Speeh and Tail fieltent with the hdecided by

Speed field is dons. The CRCare error decoation are listed

ce (GUI) is nool system whet. However, iand cannot bed eyes. Therefecessary visua

mote control. hown in Fig

ed to display tr. Therefore command is

ilts the contromotion, the rig

nd the user ththe centre of te handheld cwith no moe do not sut since this lethe user.

d controller.

tains the Heaed, Tail and ds are predefihand motions.tilt angle ofdetermined by

C field will enoding or not. d in Table 2.

ot needed foren the robot win most casese seen by the fore, a GUI ha

al feedback forThe GUI of

gure 16. A crthe current po

the user sent to the roller to the righ

ght branch ofhat an R motiothe track givescontroller is inotion comm

uggest using evel of postu

ader, CRC ined. . The f the y the

nsure The

r the works s the user as to r the

f the ross-

ose of will

obot. ght to f the on is s out n its ands

this ure is

Figu

4.5 V

A vvibrperfopeThevibrfeedcounaltermotshow

Figu

ure 16.Graphic u

Vibrotactile Fee

vibration morotactile feedbformed. As srating statuse vibrotactile

ration signals dback varies nts. The frequred by changtor. The charawn in Figure 1

ure 17.Character

user interface of

edback Generato

otor is used back when ashown in Fig

es that need tofeedback genbased on the

in vibration uency of its sging the voltacteristics of v17.

ristics of vibratio

f the handheld c

or

to alert than improper gure 15, theo be fed back

nerator will de statuses. Thfrequencies a

sinusoidal vibtage level apvibration mot

on motor outpu

controller.

he user withoperation is

re are manyk to the users.determine thehe vibrotactileand vibration

bration can bepplied to theor output are

ut.

h s y . e e n e e e


www.intechopen.com

Recognized

CouRL F BUS

No DeteSingle m

recognitiAver

recogniti

Table 3.Recogn

5. Experimen

Some experimperformance system. The motion recogthe performinterface in tinvestigated.

5.1 Robot Com

The control based controresponse seinterface genthe robot tooand execute will be discato be unsmogenerates antoo slowly, and delay. unacceptable The responseits commandwhen we adjthe control ina CER of 10characterizatenable a CERbeen done. 3each test. Thclear that wincreases witCER is 97%. almost unchcontrol, the C

d Motion Runt 10R 9

40

0U 0

1ection 0

motion ion rate

95

age ion rate

nition results of

nts

ments have bof the proporobot comm

gnition rate omance of the

the robot nav

mmands Execut

flexibility ofol interface ensitivity of nerates and seo fast, the robevery comm

arded, which ooth and unsnd sends the c

the user wilThe contro

e.

e sensitivity ofd execution rajust the commnterface. For m00% is neededtion tests to dR of 100%. Fiv30 control comhe test results

when the CDIth the CDI. WIf the CDI inc

hanged. With CDI of100ms is

ActualR L F 00 100 100 95 15 4 4 81 0 0 0 89 0 1 6 0 0 0 1 0 0 0 3 1 5% 81% 89%

88

f the hand motio

been carried osed hand mot

mand executioof the handhe

hand motiovigation applic

tion

f the proposeis directly in

the robot. ends the contbot will not b

mand in time. will cause th

stable. If the control comml become awol efficiency

f the robot canate (CER). Themand detectionmost of the cod. Therefore,determine thee tests with di

mmands are ses are shown iI is less than

When the CDI creases again,consideration

s chosen for o

l Motion B U

100 100 11 2 1 0 0 0 92 0 3 90 3 8 0 0

91% 90% 8

8.5%

ons.

ut to evaluatetion-based con

on rate, the held controller on based concations have b

ed hand motnfluenced by

If the control commandbe able to proSome comma

he control procontrol inter

mands to the rware of the py will then

n be quantifiee CER will chn interval (CDntrol applicat, we have care shortest CDifferent CDIs hent to the robin Figure 18. 100ms, the equals 100ms, the CER remn of the real-ur control sys

S 100

0 8 0 0 6 85 1

85%

e the ntrol hand

and ntrol been

tion-y the ntrol ds to ocess ands ocess rface robot ause

n be

ed by hange DI) of tions, rried

DI to have ot in It is CER

s, the mains -time stem.

Figucom

5.2 H

Severecofreetrajemotindisamspee Thehanbefohanfor mot3. Baverreco

5.3 P

In contwasFigutestbcomin thuserto rhancontallomouconthomrout

ure 18.Commanmand detection

Hand Motions

eral factors cognition. First space so thectory for anytion will be ividual user d

me hand motioed each time.

refore we seld motion rec

orehand. Eachd motion 20 100 times. Fi

tions were recBoth the singlerage recognitognition rate o

Performance of

order to testroller in the s built in our lure 19. The sibed surface i

mplete the navhe figure, whirs are asked tremotely condheld controltrol program. ws the usersuse interface, trol the robot.

me robot connter through W

d execution raten interval (CDI).

Recognition

can affect thely, all the 3D

hat it is impy motion type

made differifferences. Eve

on will be mad

ected five subcognition testh person was times, so eachinally, 600 teorded in the de hand motiotion rate areof six hand mo

f Handheld Con

st the perforrobot navigataboratory. Thide length of s 30cm. The

vigation task bich is from poo use the han

ntrol the robller and the PBut the contr

s to use the which only hThe handheld

nect to a wiWi-Fi.

e (CER) changes.

e results of hand motion

possible to dee. Secondly, thrently each

ven for the samde with differe

bjects to partts without sp

asked to reph hand motio

est results of database, as shon recognitione calculated. otions is 88.5%

ntroller

rmance of ttion applicatio

he testbed setueach white shome robot

by following tosition A to pondheld controbot respectivePC run the samrol program on

traditional khas several bad controller, thireless local a

s with the

hand motions are made inefine a fixedhe same handtime due to

me person, theent range and

ticipate in thepecial trainingpeat the sameon was testedthe six hand

hown in Tablen rate and the

The average%.

he handheldons, a testbed

up is shown insquare on theis ordered tothe pink linesosition B. The

oller and a PCely. Both theme high-leveln the PC only

keyboard andsic buttons tohe PC and thearea network

n n d d o e d

e g e d d e e e

d d n e o s e C e l

y d o e k


Figure 19.Test

The navigatiocontrol interfthe deviationdestination isthe successfuattempt by while the navThe success controller is 9control is ocontroller is 1times of the average timehandheld con

Figure 20.Dest

Figure 21.Exec

bed setup for th

on test is repeface. The test n between the s less than 10cm

ul tests are shothe handheldvigation attemrate of naviga93.33%. Meanwonly 80%. Th15.4% more thnavigation tese to completntroller is 24.7%

tination deviatio

cution times of t

he robot naviga

eated 15 timeis considered actual destinam. The destin

own in Figure d controller sumpt by the PC

ation attemptswhile, the suche precision han that of the sts are shownte the naviga% less than tha

ons of the navig

the navigation t

ation tests.

s for each typto be successf

ation and the iation deviation20. The navigaucceeds 14 tisucceeds 12 ti

s by the handcess rate of thof the handPC. The execu

n in Figure 21.ation task byat by the PC.

gation tests.

tests.

pe of ful if ideal ns of ation imes, imes. dheld he PC dheld ution . The

y the

6. C

We hancontaxisimpbasethe homvisuhanTheDiffcomaccehanproprecohomhancont15.424.7efficbase In twilland

7. A

TheRobScieJian ThisFouNatGraNewGra

8. R

[1] C

[2] R

[3] G

Conclusion

have presentd motion-batrol of home

s accelerometeplement hand ed robot contr‘connect-and-

me robot. Meaual and vibrotd gestures arey are easily u

ferent hand mmparing the receleration chard motion reposed handhognition rate me robot navig

dheld controltroller. The p

4% more than7% less than thciency and fleed control inte

the future, wel enable the u

d improve the h

Acknowledgme

research repobotic Sensor aence and Engingsu, China.

s work was suundation of Ch

ural Science nt BK2009103

w Century Exnt NCET-10-0

References

C. D. Nugent,E. Prassler, independent vol. 5, no. 1, p

R.C. Luo, T.developmentIEEE Int. Co422-427, 2005

G. Song, Y. Zharchitecture

ted the designsed interfacerobots. A han

er and a vibramotion recog

rol. The handh-play’ service anwhile, it imtactile feedbace defined for nused by peoplmotions are idcorded accelerracteristics of ecognition tesheld controllof 88.5%. Th

gation show tller is 13.33% precision of tn that by the he PC based cexibility of therface.

e plan to enriusers to contrhand motion r

ents

orted in this pand Control Lineering, Sout

upported in phina under GraFoundation o

3 and BK2011xcellent Talen

0330.

, D. D. Finlay,“Home autliving”, IEEE

pp. 1-9, 2008. Y. Hsu, T.Y.t of intelligennf. on Mechatr

5. hou, Z. Wei anfor adding m

n and impleme system forndheld controation motor isgnition and hheld controllefor the users

mplements thck. Six simple

navigating thele without spdentified by aration data wi

every hand st results shler achieved he experimenthat the succehigher than

the handheldPC. The execcontroller. Th

he proposed h

ich the hand rol the robot recognition ra

paper was carrLab, School otheast Univer

part by the Naant 60875070 aof Jiangsu Pr1254 and the nts in Unive

, P. Fiorini, Y. tomation as E Trans. Auto

. Lin and Kent home sectronics, Taipei,

nd A. Song, “mobility to w

mentation of ar locomotion

oller with a 3-s proposed tohand motion-er can provideto control the

he function ofe and natural

e home robots.ecial training.

analysing andith the unique

motion. Thehow that the

an averagental results ofess rate of thethe PC based

controller iscution time is

his verifies thehand motion-

motions thatmore flexibly

ate.

ried out at theof Instrumentrsity, Nanjing,

atural Scienceand 60905045,ovince underProgram for

rsities, under

Tsumaki anda means of

om. Sci. Eng.,

K.L. Su, “Thecurity robot”,, Taiwan, pp.

A smart nodeireless sensor

a n -o -e e f l . .

d e e e e f e d s s e -

t y

e t ,

e , r r r

d f ,

e , .

e r


www.intechopen.com

networks”, Sens Actuators A Phys, vol. 147, no. 1, pp. 216–221, 2008.

[4] G. Song, K. Yin, Y. Zhou and X. Cheng, “A surveillance robot with hopping capabilities for home security”, IEEE Trans Consum Electron, vol. 55, no. 4, pp. 2034-2039, 2009.

[5] G. Song, H. Wang, J. Zhang and T. Meng, “Automatic docking system for recharging home surveillance robots”, IEEE Trans Consum Electron, vol. 57, no. 2, pp. 428-435, 2011.

[6] Y. Oh, J. Yoon, J. Park, M. Kim and H. Kim, “A name recognition based call-and-come service for home robots”, IEEE Trans Consum Electron, vol. 54, no. 2, pp. 247-253, 2008.

[7] H. Liu, “Exploring Human Hand Capabilities into Embedded Multifingered Object Manipulation”, IEEE Transactions on Industrial Informatics, vol. 7, no. 3, pp. 389-398, 2011.

[8] D. Lee and K. Hong, “Game interface using hand gesture recognition”, Proc. - Int. Conf. Comput. Sci. Convergence Inf. Technol., (ICCIT), Seoul, Korea, pp. 1092-1097, 2010.

[9] J. Kim, N. D. Thang and T. Kim, “3D hand motion tracking and gesture recognition using a data glove”, IEEE Intl Symp on Industrial Electronics, Seoul, pp. 1013-1018, 2009.

[10] Z. Ju, H. Liu, “A Unified Fuzzy Framework for Human-Hand Motion Recognition”, IEEE Transaction on Fuzzy Systems, vol. 19, no. 5, pp. 901-913. 2011.

[11] C. Zhu, W. Sun and W. Sheng, “Wearable sensors based human intention recognition in smart assisted living systems”, IEEE Intl Conf on Information and Automation, Zhangjiajie, China, pp. 954-959, 2008.

[12] C. Zhu and W. Sheng, “Wearable sensor-based hand gesture and daily activity recognition for robot-assisted living”, IEEE Transactions on Systems, Man, and Cybernetics-Part A: Systems and Humans, vol. 41, no. 3, pp. 569-573, 2011.

[13] D. Lee, J. Lim, J. Sunwoo, I. Cho and C. Lee, “Actual remote control: a universal remote control using hand motions on a virtual menu”, IEEE Trans Consum Electron, vol. 55, no. 3, pp. 1439-1446, 2009.

[14] V. A. Prisacariu and I. Reid, “Robust 3d hand tracking for human computer interaction”, IEEE Intl Conf on Automatic Face & Gesture Recognition and Workshops, pp. 368-375, 2011.

[15] S. Kim, G. Park, S. Yim, S. Choi and S. Choi, “Gesture-Recognizing hand-held interface with vibrotactile feedback for 3d interaction”, IEEE Trans Consum Electron, vol. 55, no. 3, pp. 1169-1177, 2009.

[16] X. Wu, M. Su and P. Wang, “A hand-gesture-based control interface for a car-robot”, IEEE/RSJ Intl Conf on Intelligent Robots and Systems, Taipei, Taiwan, pp. 4644-4648, 2010.

[17] R. Xu, S. Zhou and W.J. Li, “MEMS Accelerometer Based Nonspecific-User Hand Gesture Recognition”, IEEE Sensors Journal, vol.12, no. 5, pp. 1166-1173, 2012.

[18] S. Zhou, Q. Shan, F. Fei, W.J. Li, C. Kwong, P. Wu, B. Meng, C. Chan and J. Liou, “Gesrure Recognition for Interactive Controllers Using MEMS Motion Sensors”, IEEE International Conf on Nano/Micro Engineered and Molecular Systems, Shenzhen, China, pp.935-940, 2009.

[19] S. Luczak, W. Oleksiuk and M. Bodnicki, “Sensing tilt with MEMS accelerometers”, IEEE Sensors Journal, vol. 6, no. 6, pp. 1669-1675, 2006.

[20] H. J. Luinge and P. H. Veltink, “Inclination measurement of human movement using a 3D accelerometer with autocalibration”, IEEE Trans. Neural Syst. Rehabil. Eng., vol. 12, no. 1, pp. 112-121, 2004.

[21] B. Lee, W. Bang, J. D. K. Kim and C. Y. Kim, “Orientation estimation in mobile virtual environments with inertial sensors”, IEEE Trans Consum Electron, vol. 57, no. 2, pp. 802-810, 2011.

[22] M. Sigalas, H. Baltzakis and P. Trahanias, “Gesture recognition based on arm tracking for human-robot interaction”, IEEE/RSJ Intl Conf on Intelligent Robots and Systems, Taipei, Taiwan, pp. 5424-5429, 2010.

[23] M. Bergh, D. Carton, R. Nijs, N. Mitsou, C. Landsiedel, K. Kuehnlenz, D. Wollherr, L. Gool and M. Buss, “Real-Time 3D hand gesture interaction with a robot for understanding directions from humans”, in IEEE Intl Symp on Robot and Human Interactive Communication, Atlanta, GA, USA, pp. 357-362, 2011.

[24] G. Yang, D. Ryu and Sungchul Kang, “Vibrotactile display for hand-held input device providing spatial and directional information”, Third Joint EuroHaptics conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, Salt Lake City, UT, USA, pp. 79-84, 2009

[25] Y. Zheng and J.B. Morrell, “A vibrotactile feedback approach to posture guidance”, IEEE Haptics Symp, Waltham, MA, USA, 2010, pp. 351-358.

[26] Q. An, Y. Matsuoka and C.E. Stepp, “Multi-day training with vibrotactile feedback for virtual object manipulation”, IEEE Intl Conf on Rehabilitation Robotics, Zurich, Switzerland, pp. 1-5, 2011.


hand motion-based remote control interface with...

Documents