gesture based wireless shadow robot !
TRANSCRIPT
GESTURE BASED WIRELESS SHADOW ROBOTGroup# 40
A Project SubmittedBy
1. Ratul, Ahasan Ulla ID: 13-23070-12. Sumnoon, Sharif Md. ID: 11-19449-23. Kabir, Sharif Raihan ID: 12-21365-2
4. Islam, Md. Nazrul ID: 12-21730-2
Under the Supervision ofDr. M. Tanseer AliAssistant Professor
American International University - Bangladesh
INTRODUCTION In the field of robotics, gesture controlling is becoming
popular lately. People need more user friendly and interactive robots for their work. Gesture detection makes thing easy for the users.
The project proposes a robotic body which mimics the action of human body
PROJECT GOALS
Develop the shadow code to read the sensor readings. Design and develop a skeleton structure for robot frame Implementation of control mechanism or Actuators and Drive
Systems. Using of Kinect Sensor data for the real time movement of
upper portion of body. Basic non-feedback walking sequence for the forward walking
of the Shadow Robot. Create save and replay mode the shadow commands . Implement a feedback IP camera at robot head.
PROJECT BLOCK DIAGRAM
THE KINECT SENSOR
1. Infrared (IR) projector.2. IR depth sensor.3. RGB camera.4. Group of tilt motors.5. Three-axis accelerometer.6. Microphone array.
KINECT SDK
• The Kinect SDK is synthesized set of libraries of software’s and the tools that is helpful for us for using the Kinect-based natural input.
COLOR STREAM Color image
formatResolution FPS Data
InfraredResol
uzion640x480
Fps30
640 x 480 30 Pixel format
is gray16
RawBayerRes
oluzion1280x9
60Fps12
1280 x 960 12 Bayer data
RawBayerRes
oluzion640x48
0Fps30
640 x 480 30 Bayer data
RawYuvResol
uzion640x480
Fps15
640 x 480 15 Raw YUV
RgbResoluzio
n1280x960Fps
12
1280 x 960 12 RGB
(X8R8G8B8)
RgbResoluzio
n640x480Fps1
5
640 x 480 15 Raw YUV
DEPTH STREAM
Depth image format Resolution Frame rate
Resoluzion640x480Fps30 640 x 480 30 FPS
Resoluzion320x240Fps30 320 x 240 30 FPS
Resolution80x60Fps 80 x 60 30 FPS
DEPTH IMAGE
THE BODY POSITIONING
The body positioning system incorporates depth mapping and decision tree analysis.
SKELETON IMAGE
JOINT ANGLES CALCULATION FROM SKELETON IMAGE
Skeleton joints tracking from Skeleton Image. Skeleton frame data to skeleton Arrays. Angle Calculation (ex: Inverse Kinematic
approach) using X,Y,Z co-ordinates of the joints that is saved in the arrays.
INVERSE KINEMATIC ANGLE CALCULATION
ARDUINO AND SERVO MOTORS
The Arduino Mega is used in our project for two purposes.
To generate servo signals from the received strings.
To calibrate the servo motors for initial position making and for creating walking sequence
GENERATING SERVO SIGNAL USING ARDUINO MEGA
END
START
If string Available
Read HC-05
If the slave String is entirely Complete
Break the string until “&” sign
Convert the rest of the piece into servo angle
Compare servo Id if it matches with saved Ids
Break it further until “:” for servo ID
NOYES
YES
NO
NO YES
CALIBRATION OF JOINT SERVOS USING ARDUINO
Arduino can be used to control up to 12 servos with minimum jitter. Two major parts are there in the application
The first part is the firmware, that needs to upload in the arduino. The second part is to install a simple software named “Serial servo
controller”.
COMMUNICATION BETWEEN MASTER AND SLAVE BLUETOOTH
To enable Wireless connectivity that enables windows pc bluetooth , C# language is used. Bluetooth can be programmed by C# in two ways.
Using Windows Sockets. That is programmed under windows sockets class.
Using Computer’s Serial Comport.
SENDING STRING DATA FROM PC BLUETOOTH
if (serial.IsOpen){try {String hexstring = ("1" + ":" + LShoulder + "&" + "2" + ":" + LElbow + "&" + "3" + ":" + RShoulder + "&" + "4" + ":" + RElbow);;serial.Writeln(hexstring);Thread.Sleep(1); }catch (Exception ex){para.Inlines.Add("Failed to SEND" + data + "\n" + ex + "\n"); }}
NON-FEEDBACK WALKING OF BIPED
The walking of the Shadow Robot follows 8 repetitive steps.
I. Stand still II. Lean Left III. Right Step forward
IV. Stand stillV. Lean RightVI. Left step forward
DESIGN PROTOTYPE AND SIMULATION
DESIGN PROTOTYPE AND SIMULATION
DESIGN PROTOTYPE AND SIMULATION
APPLICATIONS OF THE PROJECT
Artificial Intelligence. Industrial use. Simulation. Training & Education. Assistive living.Entertainment.
FUTURE WORK
Design A Dynamic Walk sequence with feedback from Gyro and accelerometer.
Finger and Face tracking. Making an Trained and Intelligent
Humanoid .
REFERENCES
[1] Jungong Han; Ling Shao; Dong Xu; Shotton, J., "Enhanced Computer Vision With Microsoft Kinect Sensor: A Review," in Cybernetics, IEEE Transactions on, vol.43, no.5, pp.1318-1334, Oct. 2013 doi: 10.1109/TCYB.2013.2265378.
[2] P. Baerlocher and R. Boilic, Inverse Kinematics Techniques for the Interactive Posture Control of Articulated Figures, PhD thesis, EcolePolytechniqueFederale de Lausanne, 2001.
[3] Machida, E.; Meifen Cao; Murao, T.; Hashimoto, H., "Human motion tracking of mobile robot with Kinect 3D sensor," in SICE Annual Conference (SICE), 2012 Proceedings of , vol., no.,pp.2207-2211,20-23Aug.2012.
[4] Xu, D. and Acosta, A. (2005) An Analysis of the Inverse Kinematics for a 5-DOF Manipulator. International Journal of Automation and Computing, 2, 114-124. http://dx.doi.org/10.1007/s11633-005-0114-1.
[5] Stephan Waldherr; Roseli Romero and Sebastian Thrun, A Gesture Based Interface for Human-Robot Interaction. , Autonomous Robots, September 2000, Volume 9, Issue 2, pp 151-173.
[6] W. Xu and E. J. Lee, “Continuous gesture recognition system using improved HMM algorithm based on 2D and 3D space”, International Journal of Multimedia and Ubiquitous Engineering, vol. 7, no. 2, (2012), pp. 335-340.
[7] Bhuiyan , M. & Picking, R. (2009). Gesture-controlled user interfaces, what have we done and what's next?, Proceedings of the Fifth Collaborative Research Symposium on Security, E-Learning, Internet and Networking (SEIN 2009), Darmstadt, Germany, 26-27 November 2009, pp59-60.
ANY QUARIES
For further information email: [email protected]
Thank You !