research article an instantaneous kinematics method … i/jers vol...r. senthilnathan 1, r....

Journal of Engineering Research and Studies E-ISSN 0976-7916

JERS/Vol.I/ Issue II/Oct.-Dec.,2010/69-78

Research Article

AN INSTANTANEOUS KINEMATICS METHOD FOR ROBOT

CONTROL IN VISION BASED SORTING

APPLICATION R. Senthilnathan

1, R. Sivaramakrishnan

1, A. Jothilingam

1 and C. Senthil Vel

1

Address for Correspondence

1 Department of Production Technology, MIT Campus, Anna University Chennai

Madras Institute of Technology, Chennai 600044 E mail: [email protected]

ABSTRACT

Vision guided robotics for manufacturing is an area which has diversified applications and geometry based issues to be

addressed. The paper is an attempt to solve a typical problem that could be faced by an bearing manufacturing industry

incorporating vision assisted robot based sorting of annular metallic ball bearings. The robots incorporating magnetic

grippers are the ones which are more prone to the problem of picking up closely placed components where in picking of

one component would disturb the position of the other. It is obvious that in an electromagnetic gripper this task could be

achieved by controlling the current in the windings of the gripper from which the magnetic field could be altered to apply

only the required gripping force. Though online cameras could be a better choice for such application whereby new

position of the dislocated neighboring component could be easily found, this paper is just an attempt to avoid the

neighbouring parts being disturbed from their actual positions. The component’s priority for being picked and placed in the

respective containers might have to be varied dynamically based on the locations of the object of interest and the

neighboring object. Generally this is a particular issue required to be addressed during the picking and placing operation

involving ferromagnetic parts handled by an electromagnetic gripper. The main objective of the paper is to develop an

instantaneous inverse kinematics scheme for the typical cases of closed located bearings which has to be taken care for

change in priority of picking and spatial positioning of the electromagnetic gripper. The instantaneous kinematics is used to

compute the new coordinate of the end-effector which might be suitably offset from the centers of the bearings; that is the

actual coordinate of the end-effector position given by the original kinematic equations of the robot considered.

KEYWORDS Instantaneous kinematics, Sorting, Electromagnetic Gripper, Articulated Robot Control.

I. INTRODUCTION

ROBOTS allow easy reprogramming capability to

adapt to varying task requirement and hence have

found extensive use in manufacturing automation

in factories [2]. Computer Vision methods are

gradually being incorporated in robotics work-cell

in the modern factories to provide additional

flexibility to part handling in pick and place.

Vision is essential for robots working in an

unstructured environment [5]. Vision based sorting

of manufactured components using a robot has

been in place for quite some time. Though the

robots are programmed to do the assigned tasks,

the decision making capability of the robot has to

be fully contextual. This is possible with sensors

and vision is the best option in terms of accuracy

and volume [3]. The paper targets the problem that

arises in the handling of circular ferromagnetic

materials from vision data. Circular metallic

bearings are considered for the experimentation.

Generally ferromagnetic components can be easily

handled using a magnetic gripper. Permanent

magnet grippers are generally avoided since they

would require special mechanisms to place the

part. Electromagnetic grippers are the best choice

due to ease of control. It is very intuitive that

magnetic grippers pose the problem of excess field

that might disturb other components placed in the

vicinity of the object of interest. Though online

cameras have been in place in manufacturing

industries for over a long time where in frames of a

scene can be obtained for every fraction of a



second, the paper attempts to sort the bearings just

with a single image of the scene. The paper

proposes a technique for strict control of robot

such that no neighboring components are disturbed

which could be achieved by two means. One by

controlling the gripping force which is decided

based on the volume and weight of the part

handled. Secondly an instantaneous kinematics

algorithm is developed to position the robot end

effector at the maximum possible distance such

that the neighboring components are not

influencing the magnetic field of the

electromagnetic gripper.

II. MACHINE VISION

Vision guided robots is a combination of a number

of fields such computer vision, machine vision,

image processing, control theory and robotics.

Computer vision gives the theoretical background

while machine vision describes the selection of

practical parameters such as camera, lighting and

so on. Both the above mentioned fields deal with

formation of an image.

A. Camera Model

The camera used for imaging the bearings is an

IDS U-EYE 2 mega-pixels CCD camera with USB

interface. Though perspective camera model is the

most widely used non-linear model of a camera,

the paper models the camera used as a scaled

orthographic projection which is a special case of

affine projection. This approximation for camera

calibration is valid since the relative depth of

points in the scene (the height of the bearings) is

small compared to the distance of the camera from

the scene, which is approximately 1 foot. From the

scaled orthographic projection the image

coordinates for a point cP = [x, y, z]

T whose

coordinates are expressed with respect to the

camera coordinate frame, c, will project on to the

image plane with coordinates p = [u, v]T.

= s (1)

where s is a fixed scale factor.

B. Lighting

Since it is only desired to measure the size of the

circular components (bearings) for the purpose of

sorting, the surface features of the bearings are of

least importance. The inner and outer diameters of

the bearings are to be measured for which only the

inner and outer profile information are required.

Hence bottom lighting is chosen for the purpose.

The light source is formed by two incandescent

bulbs with 25W each. The luminous intensity is

measured using a lux meter and the value is set at

an optimum value 200 lux. In bottom lighting over

illumination will introduce surface features and

reduces the corner features. One of the important

issue witnessed in back lighting is the lighting in

the inner periphery of the bearings which would

prevent from obtaining perfect circles from which

the centers and the diameter are found. This

problem is tackled using a filtering operation. The

conceptual diagram of the bottom lighting setup is

shown in Fig 1.

Fig. 1 Back Lighting



III. IMAGE ANALYSIS

An image processing algorithm is developed for

the three main image processing objectives viz.

determination of the coordinates of the centers of

the bearings, the diameters of the bearings and

correlating the information derived in pixels to

actual world dimensions. Image Processing

Toolbox which comes as a part of the MATLAB

software is utilized for all the computations. The

biggest advantage of bottom lighting is the

reduction of the color level to approximately two.

The Fig. 1 shows a sample image of the bearings.

All the image analysis to extract the desired

information are done in binary level image to be

more precise the method to find the diameters

starts after obtaining the edges in the image. Before

locating the centre coordinates there are two

nontrivial tasks, one is finding the total number of

bearings present in the image and the other is

tracing the circles one by one to find the centre and

diameter. It is to be noted that both these tasks

require tracing boundaries. For the task of

boundary tracing the image should be a binary

image, the objects should have intensity value ‘1’

and the background should be ‘0’. The convention

adopted in boundary tracing is illustrated in Fig. 2.

Fig. 2 Sample Image

For the boundary tracing operation the first white

pixel which actually would correspond to a edge

pixel of a circle is to be found and the tracing

direction generally is to be known (though it does

not hav any significance in the case considered

since complete circles are traced).

Fig. 3 Convention for Boundary Tracing

A. Wiener Filtering

The image of the bearings taken under bottom

lighting has the inherent problem of the inner

periphery being lit. This is a noise which is

removed using Weiner filter. The filter is a square

mask of size ten, which is operated on the binary

image.

Fig. 4 Example of Lighting Defect

Wiener filter estimates the local mean and variance

around each pixel. Filtering operation removes

lone black pixels and a clean connected binary

image of the bearings is obtained. A sample image

of a bearing before and after applying Wiener filter

is shown in Fig 5 and Fig 6 respectively.

Fig. 5 Image before Filtering



Fig. 6 Image after Applying Wiener Filter

B. Edge Detection

Once the filtered image is obtained the next part

would be measure the inner and outer diameters of

the bearings. The paper proposes a circle based

diameter detection scheme for the sorting of the

bearings, though there are so many techniques such

as the blob analysis and so on. It is very evident

that every bearing has two edges. The edges are

simple circle whose diameter is actually the

diameter of the bearings. Running an edge detector

algorithm on the filtered image would extract two

clear circles for each bearing from which the

diameters can be found. The lighting condition and

image demands the edge detector to be so robust

particularly in the context of discontinuities along

the edge. If f(x,y) denotes the image and G(x,y)

denote the Guassian function given by Equation 2

with which the input image is smoothed [4].

G(x,y) = e^{-(x2+y2)/2σ2} (2)

The smoothed image fs(x,y) is formed by

convolving G and f. This operation is followed by

computing the gradient magnitude and direction

(angle) which is common in almost all edge

detection processes. Once it is found nonmaxima

suppression is applied to the gradient magnitude

image. One of the main step that distinguishes the

Canny edge detector from other edge detecting

algorithms is that it uses a double thresholding and

connectivity analysis to detect and link edges [4].

The Canny Edge Detector is chosen after careful

analysis of other existing edge detectors. The

Sobel, Robert and Perwitt edge detectors were

tried, but none of them gave perfectly connected

edge. Hence Canny edge detector was chosen for

edge detection. The comparison between Canny

edge detctor and other edge detectors is shown in

Fig. 7. The discontinuities are clearly visible in the

edges found by all the operators except for the

Canny edge detector.

Fig 7. Performance of Different Edge Detectors

Sobel Perwitt

Roberts Canny



A. Circle Detection

Once the coordinates of the edge pixels are

obtained, the centre of the circle can be found from

the equation of circle given by Equation 3.

(x-xc)2 + (y-yc)

2 = R

2 (3)

(xc,yc) is the center of the circle and R is the

radius of the circle. In terms of parameters a, b, c

the equation of the circle can be written as in

Equation 4.

x2 + y

2 + a*x + b*y + c = 0 (4)

where

a = -2*xc (5)

b = -2*yc (6)

c = xc2 + yc

2 – R

2 (7)

This way the radius (diameter) and the centers of

the bearings are found. The algorithm developed

for finding the radius and centre of the circles

utilizes a erase after finding approach. The first

ever white pixel forming the edge of the circle is

found and then the entire region is traced which

actually forms a complete outer circle. From the

first white pixel on the outer cicle the the first

white pixel on the inner circle is found and again

the same procedure is followed.

Once the radius and centre is found the circle is

erased from the image by converting the white

pixels to black.This way the centre and diameters

of all the bearings are found. Once the diameters

are found the bearings can be sorted based on

either the inner or outer diameter depending the

demand of the application. The resolution of the

image processing algorithm to distinguish a

bearing is found to be 0.5mm which is more than

sufficient for the bearings since as found in the

bearing catalog, the minimum difference between

any two standard bearings will not exceed 1mm.

The results viz. the centre coordinates and the

measured diameter of the bearings in Fig 1 is

shown in Table I. The centre coordinates shown in

the table are with respect to the world reference

frame, which means the camera calibration is made

and all the measurements made in the pixel

dimensions are converted to world dimensions

(mm). The paper assumes the robot's centre to be

the origin of the world reference frame. The details

of the robot calbration are explained in the robot

control section.

ROBOT CONFIGURATION

The robot manipulator is an articulated robot

configuration with the end effector being an

electromagnetic gripper. The robot is actuated by

stepper motors for all the three links. The base has

a twisting joint to which the stepper motor is

directly coupled to the link, the other two links

utilizes a worm and pinion gear mechanism for

increasing the resolution and also to ensure motion

only in one direction. The gear ratios are 48:1 and

56:1 for the link 1 and link 2 respectively. The

robot configuration used and the corresponding

coordinate reference frames are shown in Fig. 8.



TABLE I: IMAGE ANALYSIS RESULTS

Actual

inner dia.

in mm

Actual Outer

Dia.

in mm

Centre

Coordinates

in mm

Estimated inner dia.

in mm

Estimated outer dia.

in mm

9.0 22 (56.00, 180.84) 8.5414 21.6315

6.0 17 (22.00, 162.84) 6.4485 16.6487

Fig. 8 Robot Configuration

A. Electromagnetic Gripper

Electromagnetic Grippers are easier to control

which requires a source of DC power and an

appropriate controller unit. When the part is to be

released the controller unit reverses the polarity at

a reduced power level before the switching off the

electromagnet. This procedure act to cancel the

residual magnetism in the work piece and ensure

the positive release of the object. The

electromagnetic gripper used in the robot is in Fig.

9 with its winding exposed. The windings are

electrically insulated both from the ferrite core and

from the atmosphere. The electromagnetic gripper

is fabricated by winding copper coils on a ferrite

core (an engine valve is chosen as the core) which

offers a better permeability [1]. A 1mm copper

wire is used for the coil. Effort is not made to

calculate a relation between the gripping force and

the induced magnetic field, since the maximum

payload considered is only a bearing with outer

diameter of 22mm which would weigh not more

than about 20 grams. A 12V DC potential

difference across the windings is used for the

excitation.



Fig. 9 Electromagnetic Gripper

I. ROBOT CONTROL

Controlling the robot involves positioning the

electromagnetic gripper on the desired bearing

based on the ascending or descending order of

sorting as demanded by the application. The sub-

tasks involved are applying a suitable gripping

force in the form of magnetic field which can be

controlled by using a Pulse Width Modulation

(PWM) technique which is explained in the

Transitory Kinematics Method section of the

paper. Also the robot control is basically an inverse

kinematics problem. For doing this the joint angles

has to be found. The tasks of image processing,

image analysis and other inverse kinematic

computations are done the MATLAB environment.

An 8-bit microcontroller (AT89C51) is used as part

of the hardware. The main use of the

microcontroller is to control the stepper motors and

the electromagnetic gripper and to establish a serial

communication with MATLAB software

environment which would send all the computed

details required for positioning the robot.

A. Inverse Kinematics

Calculating the position and orientation of the hand

of the robot is called forward kinematics. The

forward kinematics maps the value of the joint

vector to the transformation matrix relating the

gripper’s frame to the robot’s world reference

frame. The task involved here is to position the

manipulator's end effector to a known point in

space which means the point's pose is a known

quantity. The known point is nothing but the

coordinates of the centers of the bearings which is

defined with respect to the world reference frame

(robot's centre).

With inverse kinematics, it is possible to determine

the value of each joint in order to place the arm at a

desired position and orientation (orientation

problem is negligible since ball bearings can be

approximated to be a 2D component). The inverse

solution is generally more difficult than the

forward solution (for serial manipulators). The

equation generated are non linear and may not

posses obvious solution. The inverse kinematic

equations for the manipulator configuration

considered are as follows.

=tan-1( ) (8)

= (9)

= (10)

θ1 is angle of twisting joint, θ2 is angle of first

rotational joint and θ3 is angle of second rotational

joint. L1 and L2 are the link lengths.



Redundancy in inverse kinematics is a major area

to be addressed in any robotics application. The

multiple solutions of inverse kinematics are

observed in the manipulator configuration chosen

but the solving the redundancy is beyond the scope

of the paper. Hence the solutions of the Equations

8 through 10 are taken as the only inverse

kinematic solution.

B. Robot Calibration

Robot calibration is a term applied to the procedure

used in determining actual vales which describes

the geometrical dimensions and mechanical

characteristics of a robot structure. These values

can usually be classified as kinematic and dynamic

parameters kinematics parameter primarily

describe a robot arm length and relative joint axis

orientation while the dynamic parameters describe

arm and joint masses and internal friction. The

determination of these parameters is most

important for improving and maintaining the

accuracy at which robot position and motion are

controlled. A calibrated robot has a higher absolute

positioning accuracy than an uncalibrated one, i.e.,

the real position of the robot end effector

corresponds better to the position calculated from

the mathematical model of the robot. The robot

calibration here also involves compensation for the

offset of the lighting table from the robot (Region

of Interest in the Image and the physical offset) as

shown the Fig 10. The base of the robot used in

here is directly coupled to the stepper motor.

Hence the angle made by the twisting joint is equal

to the angle made by the stepper motor. Here the

base motor has step angle of 1.8 degrees.

Fig. 10 Experimental Setup

The other two revolute joint is connected to the

stepper motor with the help of worm and gear

mechanism. Here the stepper motor used has step

angle of 1.8 degrees which is a pretty good

resolution. Now as the joints are not connected

directly to the motors it has to be calibrated. That

is for one revolution of motor what is the angle

made by the link connected to the joint. For link2 it

was found out that for one complete rotation of

motor the gear lifted the link by 6.5 degrees. The

motor required 200 pulses for one complete

rotation. For link3 it was found out that for one

complete rotation of motor the gear lifted the link

by 8.0 degrees. The motor required 200 pulses for

one complete rotation.

Let the new angles be θ1’, θ2

’ and θ3’. It is to be

noted that θ1 is equal to θ1’ since the twist joint is

directly coupled to the motor without any gears.



C. Transitory Kinematics Method

The main aim of the paper is to demonstrate a

transitory kinematics algorithm for the case of

closely spaced components. If a component (here

bearing) is closely placed with another component

and if the robot's end effector is an magnetic

gripper, the chances for the position of the

neighboring component being disturbed are more.

The method suggested here would be of use even if

the camera used is online such that frames can be

taken after the robot completes one part handling

cycle. If a component’s position in Cartesian space

is less than or equal a value say ‘m’ then the

instantaneous scheme applies. If the difference

(xi~xn) or (yi~yn) is less than or equal to a threshold

'm' similarly then the actual coordinates are

changes as follows.

xi=xi+f and yi=yi+f (11)

xi and yi are the coordinates of desired bearing and

xn and yn are the coordinates of the neighboring

component and f is the offset of the end effector

from the centre of the desired component so that

the field strength do not disturb the neighboring

component. This is done by altering the joint

angles of the robot. The inverse kinematics

equation of the robot which generally gives the

modified joint angles θ1’, θ2

’ and θ3

’ has to be

changed to new angles α, β and γ respectively. The

amount of gripping force to be applied in order to

pick a component must be carefully estimated

based on the knowledge of the approximate weight

of the component. The static force acting on the

component (the mass of the component) is alone

considered, though forces acting on the robot's end

effector (on the part in turn) is of good relevance to

be addressed but is beyond the scope of the paper.

Separate gripper force analysis in the context of

slip is to be made for accurate force calculations

and for reducing the chances of slip.

The variable gripping force depending on the size

of the component (actually the weight of the

component) is achieved by using a PWM

technique. The electromagnetic gripper is operated

by using a solid state relay and controlled by the

microcontroller. The 16-bit timer in the

microcontroller is used for generating a square

wave of suitable duty cycle. The duty cycle is

computed in the MATLAB platform itself as a

linear function of the outer diameter. This linear

model is not much accurate since the depth (height

of the bearing) detail is not available from the

image, for which 3D vision techniques has to be

adopted which is again beyond the scope of the

paper. Also to be noted that the linearity holds

(approximately) only for the circular metallic ball

bearings considered, for instance it might not work

for circular metallic washers which would exactly

appear the same as bearings.

II. CONCLUSION

Thus the instantaneous (transitory) kinematics

equations do not change the actual robot

kinematics equation of the robot but momentarily

alters the inverse kinematics equations as the

situation demands. The algorithm developed for

sorting includes a switching provision between the

sorting based on inner and outer diameters. The

instantaneous change also contributes for priority

change of the bearings. If the bearings are sorted

based on the descending order of either the inner or

outer diameter of the circular components, the



priority might have to be changed for closely

spaced components since smaller components can

be picked up with a small gripping force (less

magnetic field intensity) compared to larger

components. Although circular bearings alone are

considered for experimentation, the variable

inverse kinematics scheme can be used for any

kind of part handling where electromagnetic

grippers are used. The dynamic forces are

compensated by simply applying extra gripping

force though it is not the apt procedure. The

dynamic force analysis in the context of slip and

consideration of the height of the component

(validity of the orthographic camera model) is part

of the extension of the work carried out so far.

REFERENCES

[1] Di Xiao, Bijoy K. Ghosh, Ning Xi, and T. J. Tarn

(2000) ‘Sensor- Based Hybrid Position/Force

Control of a Robot Manipulator in an Uncalibrated

Environment’-IEEE, pp 635-645

[2] Mikell P. Groover, Mitchell Weiss, Roger N.

Nagel, Nicholas G. Odrey (1986) ‘Industrial

Robotics Technology, Programming and

Application’, McGraw-Hill, 1986, pp 22-28,118-

120,181-183

[3] Hodges S.E. and Richards R.J. (1995) ‘Visual

Robot Control for Cheap Flexible Assembly’,

Image Processing and its Applications, Fifth

International Conference, Jul 1995, pp 707 - 711.

[4] Rafael C. Gonzalez, Richard E. Woods (2008)

‘Digital Image Processing’, fourth edition, Pearson

Education, pp 1 - 31.

[5] H. Pikkarainen, T. Heimonen, H. Juvalainen, M.

Manninen (2001) ‘Experiments to Achieve

Accurate Vision Guided Robot Movements’,

Proceedings of the 4th IEEE International

Symposium on Assembly and Task planning , pp

351-355

[6] Antonio J. Sanchez and Jose M. Martinez (2000)

‘Robot Arm Pick and Place Behavior Programming

System Using Visual Perception’ – IEEE,pp 507-

510

[7] Liangyu Lei (2004), A Machine Vision System for

Inspecting Bearing- Diameter, IEEE world congress

on intelligent control and automation, vol 5, pp

3904-3906.

[8] Han Jian-hai, Zhao Shu-Shang and Sun Wei (2007),

Research on Sub pixel detecting on-line System

Based on Machine Vision for Inner Diameter of

Bearings, vol 7,pp 2049-2052.

[9] Aravind, Rajparthiban and Tiffany (2009),

Development of Semi-Automatic Pick and Place

Robot for Material Handling Systems, vol 56, pp

172-175.

research article an instantaneous kinematics method … i/jers vol...r. senthilnathan 1, r....

Documents