research article an instantaneous kinematics method … i/jers vol...r. senthilnathan 1, r....
Post on 08-Oct-2020
0 Views
Preview:
TRANSCRIPT
Journal of Engineering Research and Studies E-ISSN 0976-7916
JERS/Vol.I/ Issue II/Oct.-Dec.,2010/69-78
Research Article
AN INSTANTANEOUS KINEMATICS METHOD FOR ROBOT
CONTROL IN VISION BASED SORTING
APPLICATION R. Senthilnathan
1, R. Sivaramakrishnan
1, A. Jothilingam
1 and C. Senthil Vel
1
Address for Correspondence
1 Department of Production Technology, MIT Campus, Anna University Chennai
Madras Institute of Technology, Chennai 600044 E mail: sen2dext@gmail.com
ABSTRACT
Vision guided robotics for manufacturing is an area which has diversified applications and geometry based issues to be
addressed. The paper is an attempt to solve a typical problem that could be faced by an bearing manufacturing industry
incorporating vision assisted robot based sorting of annular metallic ball bearings. The robots incorporating magnetic
grippers are the ones which are more prone to the problem of picking up closely placed components where in picking of
one component would disturb the position of the other. It is obvious that in an electromagnetic gripper this task could be
achieved by controlling the current in the windings of the gripper from which the magnetic field could be altered to apply
only the required gripping force. Though online cameras could be a better choice for such application whereby new
position of the dislocated neighboring component could be easily found, this paper is just an attempt to avoid the
neighbouring parts being disturbed from their actual positions. The component’s priority for being picked and placed in the
respective containers might have to be varied dynamically based on the locations of the object of interest and the
neighboring object. Generally this is a particular issue required to be addressed during the picking and placing operation
involving ferromagnetic parts handled by an electromagnetic gripper. The main objective of the paper is to develop an
instantaneous inverse kinematics scheme for the typical cases of closed located bearings which has to be taken care for
change in priority of picking and spatial positioning of the electromagnetic gripper. The instantaneous kinematics is used to
compute the new coordinate of the end-effector which might be suitably offset from the centers of the bearings; that is the
actual coordinate of the end-effector position given by the original kinematic equations of the robot considered.
KEYWORDS Instantaneous kinematics, Sorting, Electromagnetic Gripper, Articulated Robot Control.
I. INTRODUCTION
ROBOTS allow easy reprogramming capability to
adapt to varying task requirement and hence have
found extensive use in manufacturing automation
in factories [2]. Computer Vision methods are
gradually being incorporated in robotics work-cell
in the modern factories to provide additional
flexibility to part handling in pick and place.
Vision is essential for robots working in an
unstructured environment [5]. Vision based sorting
of manufactured components using a robot has
been in place for quite some time. Though the
robots are programmed to do the assigned tasks,
the decision making capability of the robot has to
be fully contextual. This is possible with sensors
and vision is the best option in terms of accuracy
and volume [3]. The paper targets the problem that
arises in the handling of circular ferromagnetic
materials from vision data. Circular metallic
bearings are considered for the experimentation.
Generally ferromagnetic components can be easily
handled using a magnetic gripper. Permanent
magnet grippers are generally avoided since they
would require special mechanisms to place the
part. Electromagnetic grippers are the best choice
due to ease of control. It is very intuitive that
magnetic grippers pose the problem of excess field
that might disturb other components placed in the
vicinity of the object of interest. Though online
cameras have been in place in manufacturing
industries for over a long time where in frames of a
scene can be obtained for every fraction of a
Journal of Engineering Research and Studies E-ISSN 0976-7916
JERS/Vol.I/ Issue II/Oct.-Dec.,2010/69-78
second, the paper attempts to sort the bearings just
with a single image of the scene. The paper
proposes a technique for strict control of robot
such that no neighboring components are disturbed
which could be achieved by two means. One by
controlling the gripping force which is decided
based on the volume and weight of the part
handled. Secondly an instantaneous kinematics
algorithm is developed to position the robot end
effector at the maximum possible distance such
that the neighboring components are not
influencing the magnetic field of the
electromagnetic gripper.
II. MACHINE VISION
Vision guided robots is a combination of a number
of fields such computer vision, machine vision,
image processing, control theory and robotics.
Computer vision gives the theoretical background
while machine vision describes the selection of
practical parameters such as camera, lighting and
so on. Both the above mentioned fields deal with
formation of an image.
A. Camera Model
The camera used for imaging the bearings is an
IDS U-EYE 2 mega-pixels CCD camera with USB
interface. Though perspective camera model is the
most widely used non-linear model of a camera,
the paper models the camera used as a scaled
orthographic projection which is a special case of
affine projection. This approximation for camera
calibration is valid since the relative depth of
points in the scene (the height of the bearings) is
small compared to the distance of the camera from
the scene, which is approximately 1 foot. From the
scaled orthographic projection the image
coordinates for a point cP = [x, y, z]
T whose
coordinates are expressed with respect to the
camera coordinate frame, c, will project on to the
image plane with coordinates p = [u, v]T.
= s (1)
where s is a fixed scale factor.
B. Lighting
Since it is only desired to measure the size of the
circular components (bearings) for the purpose of
sorting, the surface features of the bearings are of
least importance. The inner and outer diameters of
the bearings are to be measured for which only the
inner and outer profile information are required.
Hence bottom lighting is chosen for the purpose.
The light source is formed by two incandescent
bulbs with 25W each. The luminous intensity is
measured using a lux meter and the value is set at
an optimum value 200 lux. In bottom lighting over
illumination will introduce surface features and
reduces the corner features. One of the important
issue witnessed in back lighting is the lighting in
the inner periphery of the bearings which would
prevent from obtaining perfect circles from which
the centers and the diameter are found. This
problem is tackled using a filtering operation. The
conceptual diagram of the bottom lighting setup is
shown in Fig 1.
Fig. 1 Back Lighting
Journal of Engineering Research and Studies E-ISSN 0976-7916
JERS/Vol.I/ Issue II/Oct.-Dec.,2010/69-78
III. IMAGE ANALYSIS
An image processing algorithm is developed for
the three main image processing objectives viz.
determination of the coordinates of the centers of
the bearings, the diameters of the bearings and
correlating the information derived in pixels to
actual world dimensions. Image Processing
Toolbox which comes as a part of the MATLAB
software is utilized for all the computations. The
biggest advantage of bottom lighting is the
reduction of the color level to approximately two.
The Fig. 1 shows a sample image of the bearings.
All the image analysis to extract the desired
information are done in binary level image to be
more precise the method to find the diameters
starts after obtaining the edges in the image. Before
locating the centre coordinates there are two
nontrivial tasks, one is finding the total number of
bearings present in the image and the other is
tracing the circles one by one to find the centre and
diameter. It is to be noted that both these tasks
require tracing boundaries. For the task of
boundary tracing the image should be a binary
image, the objects should have intensity value ‘1’
and the background should be ‘0’. The convention
adopted in boundary tracing is illustrated in Fig. 2.
Fig. 2 Sample Image
For the boundary tracing operation the first white
pixel which actually would correspond to a edge
pixel of a circle is to be found and the tracing
direction generally is to be known (though it does
not hav any significance in the case considered
since complete circles are traced).
Fig. 3 Convention for Boundary Tracing
A. Wiener Filtering
The image of the bearings taken under bottom
lighting has the inherent problem of the inner
periphery being lit. This is a noise which is
removed using Weiner filter. The filter is a square
mask of size ten, which is operated on the binary
image.
Fig. 4 Example of Lighting Defect
Wiener filter estimates the local mean and variance
around each pixel. Filtering operation removes
lone black pixels and a clean connected binary
image of the bearings is obtained. A sample image
of a bearing before and after applying Wiener filter
is shown in Fig 5 and Fig 6 respectively.
Fig. 5 Image before Filtering
Journal of Engineering Research and Studies E-ISSN 0976-7916
JERS/Vol.I/ Issue II/Oct.-Dec.,2010/69-78
Fig. 6 Image after Applying Wiener Filter
B. Edge Detection
Once the filtered image is obtained the next part
would be measure the inner and outer diameters of
the bearings. The paper proposes a circle based
diameter detection scheme for the sorting of the
bearings, though there are so many techniques such
as the blob analysis and so on. It is very evident
that every bearing has two edges. The edges are
simple circle whose diameter is actually the
diameter of the bearings. Running an edge detector
algorithm on the filtered image would extract two
clear circles for each bearing from which the
diameters can be found. The lighting condition and
image demands the edge detector to be so robust
particularly in the context of discontinuities along
the edge. If f(x,y) denotes the image and G(x,y)
denote the Guassian function given by Equation 2
with which the input image is smoothed [4].
G(x,y) = e^{-(x2+y2)/2σ2} (2)
The smoothed image fs(x,y) is formed by
convolving G and f. This operation is followed by
computing the gradient magnitude and direction
(angle) which is common in almost all edge
detection processes. Once it is found nonmaxima
suppression is applied to the gradient magnitude
image. One of the main step that distinguishes the
Canny edge detector from other edge detecting
algorithms is that it uses a double thresholding and
connectivity analysis to detect and link edges [4].
The Canny Edge Detector is chosen after careful
analysis of other existing edge detectors. The
Sobel, Robert and Perwitt edge detectors were
tried, but none of them gave perfectly connected
edge. Hence Canny edge detector was chosen for
edge detection. The comparison between Canny
edge detctor and other edge detectors is shown in
Fig. 7. The discontinuities are clearly visible in the
edges found by all the operators except for the
Canny edge detector.
Fig 7. Performance of Different Edge Detectors
Sobel Perwitt
Roberts Canny
Journal of Engineering Research and Studies E-ISSN 0976-7916
JERS/Vol.I/ Issue II/Oct.-Dec.,2010/69-78
A. Circle Detection
Once the coordinates of the edge pixels are
obtained, the centre of the circle can be found from
the equation of circle given by Equation 3.
(x-xc)2 + (y-yc)
2 = R
2 (3)
(xc,yc) is the center of the circle and R is the
radius of the circle. In terms of parameters a, b, c
the equation of the circle can be written as in
Equation 4.
x2 + y
2 + a*x + b*y + c = 0 (4)
where
a = -2*xc (5)
b = -2*yc (6)
c = xc2 + yc
2 – R
2 (7)
This way the radius (diameter) and the centers of
the bearings are found. The algorithm developed
for finding the radius and centre of the circles
utilizes a erase after finding approach. The first
ever white pixel forming the edge of the circle is
found and then the entire region is traced which
actually forms a complete outer circle. From the
first white pixel on the outer cicle the the first
white pixel on the inner circle is found and again
the same procedure is followed.
Once the radius and centre is found the circle is
erased from the image by converting the white
pixels to black.This way the centre and diameters
of all the bearings are found. Once the diameters
are found the bearings can be sorted based on
either the inner or outer diameter depending the
demand of the application. The resolution of the
image processing algorithm to distinguish a
bearing is found to be 0.5mm which is more than
sufficient for the bearings since as found in the
bearing catalog, the minimum difference between
any two standard bearings will not exceed 1mm.
The results viz. the centre coordinates and the
measured diameter of the bearings in Fig 1 is
shown in Table I. The centre coordinates shown in
the table are with respect to the world reference
frame, which means the camera calibration is made
and all the measurements made in the pixel
dimensions are converted to world dimensions
(mm). The paper assumes the robot's centre to be
the origin of the world reference frame. The details
of the robot calbration are explained in the robot
control section.
ROBOT CONFIGURATION
The robot manipulator is an articulated robot
configuration with the end effector being an
electromagnetic gripper. The robot is actuated by
stepper motors for all the three links. The base has
a twisting joint to which the stepper motor is
directly coupled to the link, the other two links
utilizes a worm and pinion gear mechanism for
increasing the resolution and also to ensure motion
only in one direction. The gear ratios are 48:1 and
56:1 for the link 1 and link 2 respectively. The
robot configuration used and the corresponding
coordinate reference frames are shown in Fig. 8.
Journal of Engineering Research and Studies E-ISSN 0976-7916
JERS/Vol.I/ Issue II/Oct.-Dec.,2010/69-78
TABLE I: IMAGE ANALYSIS RESULTS
Actual
inner dia.
in mm
Actual Outer
Dia.
in mm
Centre
Coordinates
in mm
Estimated inner dia.
in mm
Estimated outer dia.
in mm
9.0 22 (56.00, 180.84) 8.5414 21.6315
6.0 17 (22.00, 162.84) 6.4485 16.6487
Fig. 8 Robot Configuration
A. Electromagnetic Gripper
Electromagnetic Grippers are easier to control
which requires a source of DC power and an
appropriate controller unit. When the part is to be
released the controller unit reverses the polarity at
a reduced power level before the switching off the
electromagnet. This procedure act to cancel the
residual magnetism in the work piece and ensure
the positive release of the object. The
electromagnetic gripper used in the robot is in Fig.
9 with its winding exposed. The windings are
electrically insulated both from the ferrite core and
from the atmosphere. The electromagnetic gripper
is fabricated by winding copper coils on a ferrite
core (an engine valve is chosen as the core) which
offers a better permeability [1]. A 1mm copper
wire is used for the coil. Effort is not made to
calculate a relation between the gripping force and
the induced magnetic field, since the maximum
payload considered is only a bearing with outer
diameter of 22mm which would weigh not more
than about 20 grams. A 12V DC potential
difference across the windings is used for the
excitation.
Journal of Engineering Research and Studies E-ISSN 0976-7916
JERS/Vol.I/ Issue II/Oct.-Dec.,2010/69-78
Fig. 9 Electromagnetic Gripper
I. ROBOT CONTROL
Controlling the robot involves positioning the
electromagnetic gripper on the desired bearing
based on the ascending or descending order of
sorting as demanded by the application. The sub-
tasks involved are applying a suitable gripping
force in the form of magnetic field which can be
controlled by using a Pulse Width Modulation
(PWM) technique which is explained in the
Transitory Kinematics Method section of the
paper. Also the robot control is basically an inverse
kinematics problem. For doing this the joint angles
has to be found. The tasks of image processing,
image analysis and other inverse kinematic
computations are done the MATLAB environment.
An 8-bit microcontroller (AT89C51) is used as part
of the hardware. The main use of the
microcontroller is to control the stepper motors and
the electromagnetic gripper and to establish a serial
communication with MATLAB software
environment which would send all the computed
details required for positioning the robot.
A. Inverse Kinematics
Calculating the position and orientation of the hand
of the robot is called forward kinematics. The
forward kinematics maps the value of the joint
vector to the transformation matrix relating the
gripper’s frame to the robot’s world reference
frame. The task involved here is to position the
manipulator's end effector to a known point in
space which means the point's pose is a known
quantity. The known point is nothing but the
coordinates of the centers of the bearings which is
defined with respect to the world reference frame
(robot's centre).
With inverse kinematics, it is possible to determine
the value of each joint in order to place the arm at a
desired position and orientation (orientation
problem is negligible since ball bearings can be
approximated to be a 2D component). The inverse
solution is generally more difficult than the
forward solution (for serial manipulators). The
equation generated are non linear and may not
posses obvious solution. The inverse kinematic
equations for the manipulator configuration
considered are as follows.
=tan-1( ) (8)
= (9)
= (10)
θ1 is angle of twisting joint, θ2 is angle of first
rotational joint and θ3 is angle of second rotational
joint. L1 and L2 are the link lengths.
Journal of Engineering Research and Studies E-ISSN 0976-7916
JERS/Vol.I/ Issue II/Oct.-Dec.,2010/69-78
Redundancy in inverse kinematics is a major area
to be addressed in any robotics application. The
multiple solutions of inverse kinematics are
observed in the manipulator configuration chosen
but the solving the redundancy is beyond the scope
of the paper. Hence the solutions of the Equations
8 through 10 are taken as the only inverse
kinematic solution.
B. Robot Calibration
Robot calibration is a term applied to the procedure
used in determining actual vales which describes
the geometrical dimensions and mechanical
characteristics of a robot structure. These values
can usually be classified as kinematic and dynamic
parameters kinematics parameter primarily
describe a robot arm length and relative joint axis
orientation while the dynamic parameters describe
arm and joint masses and internal friction. The
determination of these parameters is most
important for improving and maintaining the
accuracy at which robot position and motion are
controlled. A calibrated robot has a higher absolute
positioning accuracy than an uncalibrated one, i.e.,
the real position of the robot end effector
corresponds better to the position calculated from
the mathematical model of the robot. The robot
calibration here also involves compensation for the
offset of the lighting table from the robot (Region
of Interest in the Image and the physical offset) as
shown the Fig 10. The base of the robot used in
here is directly coupled to the stepper motor.
Hence the angle made by the twisting joint is equal
to the angle made by the stepper motor. Here the
base motor has step angle of 1.8 degrees.
Fig. 10 Experimental Setup
The other two revolute joint is connected to the
stepper motor with the help of worm and gear
mechanism. Here the stepper motor used has step
angle of 1.8 degrees which is a pretty good
resolution. Now as the joints are not connected
directly to the motors it has to be calibrated. That
is for one revolution of motor what is the angle
made by the link connected to the joint. For link2 it
was found out that for one complete rotation of
motor the gear lifted the link by 6.5 degrees. The
motor required 200 pulses for one complete
rotation. For link3 it was found out that for one
complete rotation of motor the gear lifted the link
by 8.0 degrees. The motor required 200 pulses for
one complete rotation.
Let the new angles be θ1’, θ2
’ and θ3’. It is to be
noted that θ1 is equal to θ1’ since the twist joint is
directly coupled to the motor without any gears.
Journal of Engineering Research and Studies E-ISSN 0976-7916
JERS/Vol.I/ Issue II/Oct.-Dec.,2010/69-78
C. Transitory Kinematics Method
The main aim of the paper is to demonstrate a
transitory kinematics algorithm for the case of
closely spaced components. If a component (here
bearing) is closely placed with another component
and if the robot's end effector is an magnetic
gripper, the chances for the position of the
neighboring component being disturbed are more.
The method suggested here would be of use even if
the camera used is online such that frames can be
taken after the robot completes one part handling
cycle. If a component’s position in Cartesian space
is less than or equal a value say ‘m’ then the
instantaneous scheme applies. If the difference
(xi~xn) or (yi~yn) is less than or equal to a threshold
'm' similarly then the actual coordinates are
changes as follows.
xi=xi+f and yi=yi+f (11)
xi and yi are the coordinates of desired bearing and
xn and yn are the coordinates of the neighboring
component and f is the offset of the end effector
from the centre of the desired component so that
the field strength do not disturb the neighboring
component. This is done by altering the joint
angles of the robot. The inverse kinematics
equation of the robot which generally gives the
modified joint angles θ1’, θ2
’ and θ3
’ has to be
changed to new angles α, β and γ respectively. The
amount of gripping force to be applied in order to
pick a component must be carefully estimated
based on the knowledge of the approximate weight
of the component. The static force acting on the
component (the mass of the component) is alone
considered, though forces acting on the robot's end
effector (on the part in turn) is of good relevance to
be addressed but is beyond the scope of the paper.
Separate gripper force analysis in the context of
slip is to be made for accurate force calculations
and for reducing the chances of slip.
The variable gripping force depending on the size
of the component (actually the weight of the
component) is achieved by using a PWM
technique. The electromagnetic gripper is operated
by using a solid state relay and controlled by the
microcontroller. The 16-bit timer in the
microcontroller is used for generating a square
wave of suitable duty cycle. The duty cycle is
computed in the MATLAB platform itself as a
linear function of the outer diameter. This linear
model is not much accurate since the depth (height
of the bearing) detail is not available from the
image, for which 3D vision techniques has to be
adopted which is again beyond the scope of the
paper. Also to be noted that the linearity holds
(approximately) only for the circular metallic ball
bearings considered, for instance it might not work
for circular metallic washers which would exactly
appear the same as bearings.
II. CONCLUSION
Thus the instantaneous (transitory) kinematics
equations do not change the actual robot
kinematics equation of the robot but momentarily
alters the inverse kinematics equations as the
situation demands. The algorithm developed for
sorting includes a switching provision between the
sorting based on inner and outer diameters. The
instantaneous change also contributes for priority
change of the bearings. If the bearings are sorted
based on the descending order of either the inner or
outer diameter of the circular components, the
Journal of Engineering Research and Studies E-ISSN 0976-7916
JERS/Vol.I/ Issue II/Oct.-Dec.,2010/69-78
priority might have to be changed for closely
spaced components since smaller components can
be picked up with a small gripping force (less
magnetic field intensity) compared to larger
components. Although circular bearings alone are
considered for experimentation, the variable
inverse kinematics scheme can be used for any
kind of part handling where electromagnetic
grippers are used. The dynamic forces are
compensated by simply applying extra gripping
force though it is not the apt procedure. The
dynamic force analysis in the context of slip and
consideration of the height of the component
(validity of the orthographic camera model) is part
of the extension of the work carried out so far.
REFERENCES
[1] Di Xiao, Bijoy K. Ghosh, Ning Xi, and T. J. Tarn
(2000) ‘Sensor- Based Hybrid Position/Force
Control of a Robot Manipulator in an Uncalibrated
Environment’-IEEE, pp 635-645
[2] Mikell P. Groover, Mitchell Weiss, Roger N.
Nagel, Nicholas G. Odrey (1986) ‘Industrial
Robotics Technology, Programming and
Application’, McGraw-Hill, 1986, pp 22-28,118-
120,181-183
[3] Hodges S.E. and Richards R.J. (1995) ‘Visual
Robot Control for Cheap Flexible Assembly’,
Image Processing and its Applications, Fifth
International Conference, Jul 1995, pp 707 - 711.
[4] Rafael C. Gonzalez, Richard E. Woods (2008)
‘Digital Image Processing’, fourth edition, Pearson
Education, pp 1 - 31.
[5] H. Pikkarainen, T. Heimonen, H. Juvalainen, M.
Manninen (2001) ‘Experiments to Achieve
Accurate Vision Guided Robot Movements’,
Proceedings of the 4th IEEE International
Symposium on Assembly and Task planning , pp
351-355
[6] Antonio J. Sanchez and Jose M. Martinez (2000)
‘Robot Arm Pick and Place Behavior Programming
System Using Visual Perception’ – IEEE,pp 507-
510
[7] Liangyu Lei (2004), A Machine Vision System for
Inspecting Bearing- Diameter, IEEE world congress
on intelligent control and automation, vol 5, pp
3904-3906.
[8] Han Jian-hai, Zhao Shu-Shang and Sun Wei (2007),
Research on Sub pixel detecting on-line System
Based on Machine Vision for Inner Diameter of
Bearings, vol 7,pp 2049-2052.
[9] Aravind, Rajparthiban and Tiffany (2009),
Development of Semi-Automatic Pick and Place
Robot for Material Handling Systems, vol 56, pp
172-175.
top related