VISION ASSISTED PICK AND PLACE ROBOTIC ARM GUIDED BY IMAGE

PROCESSING CONCEPTS FOR OBJECT SORTING SYSTEM

BY, ANJANA ANILKUMARS7 CSECOLLEGE OF ENGINEERING MUNNAR

OVERVIEW

Introduction Concepts Machine vision Working demonstration Advantages and

Disadvantages Applications Conclusion and

Discussions References

INTRODUCTION

• George Charles Devol -Inventor of the first industrial robot Unimate and this was a turning point

• Pick and place robotic arms thus came into existence followed by a change in industrial field by the application of robotics

• A robotic arm is a robotic manipulator, usually programmable, with similar functions to A robotic arm is a robotic manipulator, usually programmable, with similar functions to a human arm.a human arm.

CONCEPTS

• Block diagram

• Image Acquisition

• Image Processing

• Data bus tools

• Object identification using signature method

• The process behind the operation

Getting in to the concepts

• Main function of a robotic manipulator is to pick some object from a fixed position and to place it in some other fixed area

• Basic model of a robot arm consists of

• The servomotor is controlled by a pulse width modulated signal (control signal) from the controller unit through its control wire

• We can either use a SG-5010 servo motor or Tower pro SG-90 servo motor according to the specifications needed

• Block diagram

MACHINE VISION

The process behind

• Usually a high quality webcam is used for image acquisition• The image captured will then converted into grey scale image• Crop the image • Adjust Contrast• Gray scale image is converted to binary image• Sobel Edge Detection Algorithm• Image Dilation• Fill closed spaces (create blobs)• Find bounding box of each object

Acquired image Grey scale image Crop the image and contrast adjustment

IMAGE PROCESSING

IMAGE PROCESSING

Convert to Binary Image Edge detection Image dilation

IMAGE PROCESSING

Fill Holes

IMAGE PROCESSING

Find Bounding Box of each Object

SOBEL EDGE DETECTION

• Performs a 2-D spatial gradient measurement on an image.

• Uses a pair of 3x3 convolution masks, one estimating the gradient in the x-direction (columns) and the other estimating the gradient in the y-direction (rows).

• A convolution mask is usually much smaller than the actual image and is slid over the image, manipulating a square of pixels at a time.

SOBEL MASKS

SOBEL PROCEDURE

THE PROCESS

ALGORITHM FOR THE OPERATION

• This method is used to convert colour image to greyscale image using Median filter to remove noise from the image and then convert greyscale image to binary image.

• The binary image is then processed to find object.• .As we need to find centroid of object, binary image is then, processed with image negative

algorithm.• After finding centroid of object, signature of object is generated. Signature is a 1-dimensional

plot of distance from centre to boundary of object at regular increment in angle from 0 to 360.

OBJREC ALGORITHM FOR OPERATIONAL USE

• Objrec algorithm is developed in MATLAB to recognize the colour and send command to the controller using serial communication

• Various steps involved are

Objrec algorithm

TRANSFER OF CONTROL SIGNALS

• Object coordinates computed in the previous step are used to develop a program to generate and send digital signals to robot controller via the parallel port

• Interfacing circuit contains isolation circuit with optocoupler. • Control signal send from PC to Optocoupler via parallel port.

• Optocoupler is necessary to prevent system from damage

WORKING MODEL

ADVANTAGES

• Consistency of performance.• 24/7 continuous working.• Reduced amount of operator errors.• It can move from one location to another

location.• Robotic workers never get tired.• Do not need to be paid.• Can be made to perform even the most

dangerous tasks without concern.

DISADVANTAGES

• High standard of maintenance required

• Precise programming needed (time, training, specialist knowledge)

• when computer systems failure will cause breakdown

• New products require complete reprogramming• Certain processes still need a skilled operator• Complex and expensive equipment to buy and

install

APPLICATIONS

Industries for assembly,process automation,welding

Medical applications Hazardous enviornments PCB manufacturing units. Space exploration Furnace manufacturing units. Oil refineries. Notable robotic arms and low cost robotic arms Pick and place robotic arms can be used as a

good teaching aid

Enhancements that can be applied

• Range Estimation for Robot arm Applications using Image Segmentation and Curve Fitting Tool

• Control Design and Implementation of Intelligent Vehicle with Robot Arm and Computer Vision

• Automated Library System Using Robotic Arm Robot arm Applications using Image Segmentation and Curve Fitting Tool

• Wireless Control of Pick and Place Robotic Arm Using an Android Application

• Control Design and Implementation of Intelligent Vehicle with Robot Arm and Computer Vision

• Object Locator and Collector Robotic Arm Using Artificial Neural Network

References

• INTERNATIONAL JOURNAL OF SCIENTIFIC & TECHNOLOGY RESEARCH VOLUME 5, ISSUE 06, JUNE 2016 Position Control Method For Pick And Place Robot Arm For Object Sorting System -Khin Moe Myint, Zaw Min Min Htun, Hla Myo Tun

• International Journal of Advanced Computational Engineering and Networking, -, Sept-2013 Object Detection Using Image Processing For An Industrial Robot -OBJECT DETECTION USING IMAGE PROCESSING FOR AN

• INDUSTRIAL ROBOT P.B.VIJAYALAXMI, 2ROHAN PUTTA, 3GAYATRI SHINDE, 4PUNIT LOHANI,Assistant Professor, Department of Computer Engineering, Fr. C. Rodrigues Institute of Technology, Vashi, India,Department of Computer Engineering, Fr. C. Rodrigues Institute of Technology, Vashi, India

• Paul RP, Shimano B, Mayer GE. Kinematic control equation for simple manipulators. IEEE Trans. Syst. Man.Cybern. SMC-11. 198; 449-455.

• International Journal of Robotics and Automation (IJRA)-Vol. 2, No. 3, September 2013-End-Effector Position Analysis Using Forward Kinematics for 5 DOF Pravak Robot Arm-Jolly Shah1, S.S.Rattan1, B.C.Nakra21ME Department, NIT, Kurukshetra, 136118, India

• International Journal of Mechanical & Mechatronics Engineering IJMME-IJENS , October 2013 - Development of a Prototype Robot Manipulator for Industrial Pick-and-Place Operations

• 1.Ayokunle A. Awelewa;• 2.Kenechukwu C. Mbanisi;• 3.Samuel O.Majekodunmi;• 4.Ishioma A. Odigwe; • 5.Ayoade F. Agbetuyi; • 6.Isaac A. Samuel • -Department of Electrical and Information Engineering, College of Science & Technology, Covenant University, Ota, Ogun

State, Nigeria.• Determining the Benefit of Human Input in Human-in-the-Loop Robotic• Systems,Christine Bringes, Yun Lin, Yu Sun, Redwan Alqasemi-2013 IEEE RO-MAN: The 22nd IEEE International

Symposium on Robot and Human Interactive Communication Gyeongju, Korea• Project on― Pick n Place Robot‖, made by Bharat Jain and Dinesh Rajput, under guidance of Prof. KaviArya, IIT

Mumbai,2010.

Conclusion and discussion

• The analysed robots for pick and place application using image processing are able to detect the fully desired object and to place it at the desired location.

• The colour detection capability is enhanced by using high speed processers and low cost cameras that use little power.

• MATLAB is used for GUI and for processing the image acquired. A new software

• RoboCIM has been used which has inbuilt program which controls the robot.

• The co-ordinates of the object are acquired through the camera and edited in the program.

• Then this software can send signals to he robot and the robot then moves toward the object,picks it and places it in the desired location

THANK YOU