design and implementation of gesture controlled …robotics, to create robots which are...

1.INTRODUCTIONRobotics is a new emerging technology in the field of

science.Manyresearchersacrosstheglobearecomingupwithnewproductstohelphumans.Robotscanbecontrolledwirelessly or by usingwired controllers. Inmany of theindustriesrobotshavetakentheplaceofhumansbutstilltheyneedtobecontrolledbythemonly.Earlierrobotswerecontrolledusingphysicaldevicesbutnowadaystheycanbecontrolled via various methods. Hand gesture-controlledrobotsweredevelopedinordertohelphumanstodothingswhicharedifficult foranyhuman likedisarmingabomb,handling nuclear materials or any day to day activity. Aroboticarmconstructedwithdiverse joints that replaceshumanarm joint.Theentire joint ismotordrivenwithacontrollermadeupofjoysticktooperateeachjointoftheroboticarm.Themainaim is todesigna roboticarmwhich is

implementedonroboticcarasshowninfigure1,sothatitcanbeusedinplaceswherehumanbeingscan’treach.Itcanbeusedasafirefightingrobot,canbeusedinconstruction,medicalandmilitaryfields.

Figure1-Prototype

2.RelatedWork

Several researches have been made in the field ofrobotics, to create robots which are user-friendly,implementing user interfaces such as 3D joystick, touchscreens, etc. But these techniques do not give accurateresults and provide slow response. A gesture controlledroboticcaranda3Dprintedarmcontrolledwithajoystickare usually two different things and integrating themtogetherissomethingnew.

Ahandgesture-controlledrobotusingaccelerometer,flex sensors and Zigbee which uses the Sixth Sensetechnology was implemented in [1]. Some of the basichardware materials were used along with the Zigbeemodules and simple assembly languagewas used. It hadseveralapplicationsinthemachineindustryforpickingandplacing.

Anothergesture-controlledrobotusedforresearchanddevelopmentwasimplementedusing3Dmotiontrackingwirelessly with the help of Bluetooth in [2]. This waspredominantlydevelopmentfortheoperatorwhodoesnotneedspecialcompetencestocontrolthemanipulatordevice.Onthecontrary,hecanpayhisfullattentionattheprocessorobjectwhichisofmostinteresttoanobserver.

International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056

Volume: 06 Issue: 09 | Sep 2019 www.irjet.net p-ISSN: 2395-0072

ChinmayPatil1*,ShivaniSharma2,SnehaSingh3

1,2,3Student of B.Tech Electronics & Telecommunication, NMIMS University’s MPSTME, Mumbai, India 400056.

DesignandImplementationofGestureControlledRobotwitha

RoboticArm

© 2019, IRJET | Impact Factor value: 7.34 | ISO 9001:2008 Certified Journal | 351

---------------------------------------------------------------------***---------------------------------------------------------------------Abstract - This paper puts forward the design andimplementationofGestureControlledRobotwithRoboticCarinaverysimpleandefficientway.ARobot isamechanicaldevice that can perform physical tasks. Robotic arm is amechanical system which is used in manipulating themovementoflifting,moving,andmanipulatingtheworkpieceto lighten theworkofman.The3Dprinted roboticarm isoperated&controlledwithajoystickwhiletheroboticcariswirelessly gesture controlled with hand gesture whichtransmits signals totheprototype through the transmitterfixed on the gloves put on hands, instead of conventionalmethodsused.Anaccelerometerandmicrocontrollerisusedtoconvertthehandgesturesintosignalforcontrollingofrobot.KeyWords:Accelerometer,GestureControlledRobot,Joystick,3Dprintedroboticarm.

ItmadetheuseofpythoncodingalongwiththelynxmotionAL5Amodule.

Anaccelerometer-basedgesture-controlledarmforanROV(RemotelyOperatedVehicle)wasdevelopedusingasimplegyroscope and joystick and discussed in [3]. Hence thisrequirednoprevious training for theoperatoraswellasprovideforasmoothandaccuratemovement.

A wireless robotic arm that uses GUI for wirelesscommunication was evolved in [4]. It had five separatemovements controlled by five servo motors. Going towirelessfromwiredismucheasierwithlesshassleandGUIhelpstocommunicatebetweensystemsinaneasierway.

Asimplehandgestureroboticarmusingaccelerometerwasput intoeffect.Analgorithm forgesturesensinghasbeendevelopedin[5]toreplacetheapproachoftraditionalcontrolling mechanism of robots, by an innovative handgesture-basedcontrolling.ItmadeuseofWINAVRSTUDIO4.0, PROTEUS for coding and had a large range ofcommunication.

The light weight robotic arm using a joystick wasimplementedin[6].Thearmisconstructedalmostentirelyofplasticelements: inflatable links,airbagactuators, andacrylonitrilebutadienestyrene(ABS)joints.Anewmethodof controlwas used and thiswas especially designed formedicalandhealthcarepurposes.

Ajoystick-controlledarmwithauditoryfeedbackwasdeveloped in [7]. Joystick-based operation is a popularmethod for operating various types of robots, such asexcavators,cranes,andspace tele robotics.Thegoal istocreateefficientmethodsforcontrollingsuchdevices.

A gesture-controlled car using simple methods wasimplementedin[8].Itshowsaroboticsystemthatallowsanyone, especially normal people with no technicalbackground,toinstructarobotjustshowingitwhatitshoulddo.

Asolelymilitarypurposedgesture-controlledrobotwasdeveloped in [9]. There are numerous robots usingcommandsfromuserorself-controlled,therequirementforgesture controlled robots are on the rise for militarypurposes,whichiscalledasUnmannedgroundvehicles(UGVs). The main advantage of this is that it is gesturefunctioningwithoutbasestationassistance.AHandMotionControlledRoboticVehiclewithobstacledetectionwhichidentifiestrendsintechnologyapplicationsandusabilityarediscussedin[10].Anapproachispresentedthat is based on detection ofmotion of handwhichwillcontrolvehiclemovementandrefrainsmovementofvehicleifanobstacleisdetectedinpath.

3.SystemDescription

Figure2showstheblockdiagramofTransmittersectionofGestureControlledRobot.Here,theaccelerometerinitiatesthe process by point data to the ATMEGA 328microcontroller(8-bit)basedonthegestureperformedbythe user. This data is processed by the microcontrollertowardstheRFtransmittermodule(433.92MHzfrequencyrange).

Figure2-BlockschematicofTransmittersectionofGesture

controlledrobot

Figure3showstheBlockschematicofReceiversectionofGesturecontrolledrobot.ThereceiverRFmodulereceives(433MHzfrequencyrange)thedatasentbythetransmitterand processes it further to the microcontroller at thereceiverend.Andthus,themicrocontroller(8-bit)directsthegivensignaltotheL293Dmotordriverwhicheventuallyperformstherequiredcorrespondingphysicaltasks.

Figure3-BlockschematicofReceiversectionofGesture

controlledrobot

Figure4showstheblockdiagramofTransmittersectionofGestureControlledRobot.TheUSBjoystick(analog)isusedbytheuseratthetransmitterend.Oncetheuserperformsaspecificgestureonjoystick,basedontheassignedthresholdvaluesthemessageistransmittedtoRS232whichprocessesthe corresponding message and sends it towards thereceiverend.



© 2019, IRJET | Impact Factor value: 7.34 | ISO 9001:2008 Certified Journal | Page 1352

Figure4-BlockschematicofimplementedRoboticarm

Atthereceiverendthemessageisreceivedanddecodedbythe SCB (Servo Controller Board) and responsible forrequiredmomentoftheroboticarm.

4.SystemMethodologyFigure5showstheprocessofthetransmittersection,theprocessisinitiatedbysettingtheXandYpinofthejoystick.FurthertheXandYpinsarereadandthethresholdvaluesaresetforadjustingthesensitivity.

Figure5-FlowchartofTransmittersection

A particular threshold is assigned for every particulargesture in a loop. If X < 340 is true, a forward move isdirectedelseitmovestothenextinputthatisifX>400istrue,abackwardmoveisdirected.Similarly, IfY<340istrue,arightmoveisdirected,elseitmovestothenextinputthatisifY>400istrue,aleftmoveisdirected.Thisstepterminatesandstopsuntilthewholemessageisgoneandwaitsforthenextmessagetocome.Inasimilarway,theXandYpinsofthejoystickareset.IfX-pos<300istrue,anuparmmoveisdirectedelseitmovestothenextinput that is if X-pos > 700 is true, a down armmove is

directed. Similarly, If Y-pos < 300 is true, a leftmove isdirected,elseitmovestothenextinputthatisifY>700istrue,aleftmoveisdirected.Again,thisstepterminatesandstopsuntilthewholemessageisgoneandwaitsforthenewmessage to come. The followingmessage determines theopeningandclosingoftheroboticarm.ThevaluesofthiscorrespondingmessagearereadandifVal<300istruethenthearmclosesandifVal<300isfalsethenthearmopens.Aftertheprocessingofthismessagetheentirecycleofoneparticulargestureisterminatedandthesystemwaitsforanewmessagesignal.Figure6showstheflowofthereceiversection,thatonceitreceivessignalsfromthetransmitter,itinitializesQ1,Q2,Q3andQ4servopins.Afterthisthereceiver’sPLLisstartedandtheoutputforthemotorcontrol isset.Nextarethenon-blockingsignalswhicharedeterminedbyusingloopsinthereceiver’scode.Ifthestatementistruethenitgoestothenextblockthatisinti=0,whichdirectlyfollowsthenextiterationthatisi++.Andiftheconditionisfalseitgoestothenextblockinti=0followedbyi<bufferandifthisistrueitrefreshes,otherwisechecksthefirstdatareceived.

Figure6-FlowchartofReceiversection

Nowifthedatareceivedischar(buff[i])==‘s’(‘s’representsStop)andistruethenitstopsandiffalsethenitmovestothe next statement char (buff[i])== ‘f’ (‘f’ representsForward)andwhen this statement is true itprocessesaforward move signal otherwise it moves to the nextstatement char (buff[i])== ‘b’ (‘b’ represents Backward)which when true processes a backward move signalotherwisemovestothenextstatementchar(buff[i])==‘l’(‘l’represents Left) whichwhen true processes a left movesignal otherwise moves to the next statement char(buff[i])== ‘r’ (‘r’ represents Right) which when trueprocessesarightmovesignalotherwisemovestothenextstatementchar(buff[i])==‘u’(‘u’representsUp)whichwhen




true processes an upward movement signal otherwisemoves to the next statement char (buff[i])== ‘d’ (‘d’representsDown)whichwhentrueprocessesadownwardmovesignal.InaSimilarmannerwithcontinuationthemovementofarmto open and close are also described as follows: If theconditionof thedownwardmovement iswrong then thecodemovestothenextstatementwhichischar(buff[i])==‘g’(‘g’representsRightarm)whichwhentrueprocessesarightmovesignalforthearmotherwisemovestothenextstatementchar(buff[i])==‘h’(‘h’representsLeftarm)whichwhentrueprocessesaleftmovesignalforthearmotherwisemoves to the next statement char (buff[i])== ‘t’ (‘t’representsStoparm)whichwhentrueprocessesastopsignalforthearmotherwisemovestothenextandthefinalstatementchar(buff[i])==‘c’(‘c’representsClose)whichwhentrueprocessesaclosemovesignalforthearm.Amongstalltheconditionsifany1conditionistruethenitdirectlyprogressestonextiterationi++whichisfollowedbythe conditions of the robotic car and conditions of thejoystickcontrolledroboticarm.Andthisisfurtherfollowedbytheterminationstepthatis‘Stop’.

5.HardwareandSoftwareModules

L293DMotorDriver

- It drives the DC motors and is used to driveinduction loads. L293Dis a 16-pin ICwhich cancontrola setof twoDCmotorssimultaneously inany direction. Itmeans that you can control twoDCmotorswith a singleL293DIC. Dual H-bridgeMotorDriverintegratedcircuit(IC).

Carchassis

- ProvidesroomfortheDCmotorsaswellasotherModules.

RFModule(TransmitterandReceiver)

- RFmodule(radiofrequencymodule)isa(usually)small electronic device used to transmit and/orreceiveradiosignals between two devices. In anembedded system it is often desirable tocommunicatewithanotherdevicewirelessly.

Wheels

- Theyareattached to thechassiswith thehelpofscrewsand thenconnected to theDCmotors formovement.

Microcontroller(ATMega328)

- Asingle-chipmicrocontrollerhavinga8-bitAVRandisusedinmanyautonomoussystems.

Accelerometer(ADXL337)

- Itisadevicethatmeasuresproperacceleration.AnalogJoystick




- Itisusedtocontroltheroboticarm.6.ImplementationIn thisprototype, thegesture-controlledcarisdevelopedusinganaccelerometer,RFmoduleandiscontrolledusingmicrocontroller ATMega328. The accelerometer and thetransmittermodulealongwiththemicrocontrollercompriseof the transmitter section. The data is transmitted usingradio frequency via the transmitter’s end. The receiversection consists of the receiver module and themicrocontrollerwhichinturncommandsthemotordrivers.HencethedataisreceivedbythereceiverRFmoduleandthecommandsaregiventothemotorsviathemotordriver.Theweightof thecar ispreciselyevaluatedandcalculated inordertobalanceouttheweightoftheroboticarmwhichismountedonthecarandthusallowsthearmtomovefreelywithoutanydominance.Specificsetofthresholdvaluesareassignedforrespectivehandgesturemomentssuchasforward,backward,leftandright. On the transmitter end the accelerometer will bemounted on a human hand glove which will be used toperformhand gestureswhichwill be responsible for themomentoftheroboticcar.Whileonthereceiverend,thereis a motor driver which takes the message sent by thetransmitterandreceivedbythereceiverandconvertsthesesignalsintomechanicalmomentsviathemotordriver.Theroboticarmwillbea3Dprintedprototypeinordertokeepitlightweightandforeasymountingontheroboticcar.Thedesignofthearmiskeptsimpletoimplementandeasytohandle.Thejoystickisusedtocontrolthearmforsmoothmovement. With the help of the microcontroller thecommandsofthejoystickaretransferredtothearmandthemovementsaredone.Commandslikeforward,backward,left,rightandstoparebeingused.Similartothatoftheroboticcar,eventhejoystickcomprisesofatransmitterandareceiverend.Andinasimilarway,specificsetofthresholdvaluesareassignedforrespectivejoystickcommandswhichincludeuparm,downarm, leftarm,rightarm,armcloseandarmopen.

Figure7-Variousgesturesformovementofvehicle

7.ResultandConclusion:

Figure8,9showthefinalhardwaremodelofthejoystickcontrolledroboticarmandgesturecontrolledroboticcarrespectively.

Figure8-RoboticArm

Figure9-GesturecontrolledRoboticcar

Theobjectiveofthisproject,ofdevelopingthehardwareandsoftwareforagesturecontrolledroboticcarandajoystick-controlled arm has been accomplished. From theobservationsitisclearthatthemovementofanarmandcaris precise, easy to control and user friendly. This robotcontrolmethodisexpectedtoovercometheproblem.Itcanbeusedasafirefightingrobottohelpthepeoplefromthefireaccident.Thesecanbeusedinconstructionfieldandalsoinindustriestocontroltrolleyandlift.Theycanalsobeusedinmilitaryandmedicalapplications.




8.References:

[1] Raheja J.L, Shyam R, Kumar U, Prasad P.B : “Robot

Controlled Using Hand Motion Recognition”, IEEESecondInternationalConferenceonMachineLearningandComputing,2010.

[2] Karolis Root, Renaldas Urniezius, “Research anddevelopmentofagesture-controlledrobotmanipulatorsystem”,IEEEInternationalConferenceonMultisensorFusion and Integration for Intelligent Systems(MFI),2016.

[3] Ronny Mardiyanto, MochamadFajar Rinaldi Utomo,Djoko Purwanto, HeriSuryoatmojo, “Development ofhandgesturerecognitionsensorbasedonaccelerometerand gyroscope for controlling arm of underwaterremotelyoperatedrobot”,IEEEInternationalSeminaronIntelligentTechnologyandItsApplications(ISITIA),2017.

[4] MuhammedM.B.,Mohd.MujiS.Z.B,ZakariaS.R,binMohd.JenuM.Z,“MR999-EWirelessRoboticArm”,IEEEAnnual India Conference (INDICON), Banglore, India,December,2016.

[5] SetiaArchika ,Mittal Surbhi ,NigamPadmini , SinghShalini,GangwarSurendra,“HandGestureRecognitionBasedRobotUsingAccelerometerSensor”,InternationalJournalofAdvancedResearchinElectrical,ElectronicsandInstrumentationEngineering

[6] Hye-JongKim,YutoTanaka,AkihiroKawamura,SadaoKawamura, Yasutaka Nishioka, “Development of aninflatableroboticarmsystemcontrolledbyajoystick,''24th IEEE International Symposium on Robot andHumanInteractiveCommunication(RO-MAN),2015.

[7] Nikolaos Mavridis, Georgios Pierris, Paolo Gallina,ZacharoulPapamitsiou,UmairSaad,“OnthesubjectivedifficultyofJoystick-basedrobotarmteleoperationwithauditory feedback”, IEEE 8th GCC Conference &Exhibition,2015.

[8] Ms.Asmita Jadhav,Ms.DeepikaPawar,Ms.KashmiraPathare,Ms.PrachiSale,Prof.Thakare.R,“HandGestureControlledRobotUsingArduino”,InternationalJournalfor Research in Applied Science & EngineeringTechnology(IJRASET).

[9] SathyanarayananM,AzharuddinS,KumarS,KhanG,“Gesture Controlled Robot for Military Purpose”,International Journal For Technological Research InEngineeringVolume1,Issue11,July-2014.

[10] Dang Ankur, Chandorkar Vibhum, Pawar Aishwarya,Pawar Dhairya, “Hand Motion Controlled RoboticVehicle With Obstacle Detection”, InternationalResearchJournalofEngineeringandTechnology(IRJET)Volume:04Issue:09,Sep-2017.




design and implementation of gesture controlled …robotics, to create robots which are...

Documents