Using Physics Based Models in Virtual Reality for Dynamic Emulation ofRobotic Systems

Y. F. Li, J. G. Wang and J. K. L. HoDepartment of Manufacturing Engineering and Engineering Management

City University of Hong Kong, Kowloon, Hong KongTel: (852) 2788 8410, Fax: (852) 2788 8423, Email: meyfli@cityu.edu.hk

Abstract

This paper describes the work of an on-going projectinvestigating the incorporation of physics-basedmodeling for simulating dynamic interactions in roboticaided manipulations. The research aims at achievingbetter simulation of robot behaviors in real situations,making it a useful tool for off-line programming. Byallowing manipulation strategy simulation andperformance visualization of a designed operation to becarried out within a virtual environment, the system willbe a valuable tool for handling strategy design, testing,modijcation, and optimization without fear of damagingthe physrcal sys tems. As an environment fordemonstrating tasks to robots, the systems will also be auseful platform for robot teaching and training. Thesystem has particular promtse in flexible assemblywhere it is more economical to perform thereconfiguration of a work cell and to generate or modibhandling strategies for new tasks without havrng to shutdown the real Jystem. This paper reports the initial workcarried out with the preliminary results given.

1. Introduction

In many robotic tasks such as assembly, the dynamicinteractions between the robot and the environment orwork piece (such as in peg-in-hole and grasping) play avital role in the success or failure of an operation [ 1,2]especially when the object or work cell configurationchanges which is typical in flexibleassembly/manufacturing [2]. Computer simulation isuseful for studying this effect and attempts have beenmade in using the concept of “physical model” for thepurpose of producing realistic animation of a sceneinvolving rigid [3] or deformable [6] objects. However,most classical simulation packages use geometry-basedCAD models in which no dynamic interaction can besupported. Even with some advanced VR packages,simulated sensing functions especially contact sensingessential for studying the interactions within thesimulation environment are still serious lacking [4]. Theeffects of dynamic interactions and therefore theperformance of a handling strategy in assembly dependmainly on the physical properties of the objects involved

such as the mass, elasticity, friction and collision forces.Thus, it is of practical interests to incorporate thephysical properties in modeling virtual objects so as tovisualize the behavior of a robotic operation and evaluatethe performance of the designed manipulation strategy[7]. The VR system enhanced in this way will facilitateiterative tests and modifications of assembly strategiesbefore implementation. An application scenario of theVR system for food product handling is illustrated inFig. 1. An example of the use of the VR system lies instudying the effects of application of force/torque ingrasping, the force/torque in handling the object can bevisualized and permanent deformation or damage to theproduct can be avoided.Operator :

Robot & tool

Human-computerInterface

1

Modelsand database

1Physics based simulation

TModlficat~on

VIrtUlObJeCt modelmg e”YIrO”ment

_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ___--.__-_.____-.___.

Robot

Fig. 1 The application of virtual reality system inrobotic aided manipulation

2. Physics based modeling

The geometrically based modeling approach used mcurrent VR systems presents an inherent flaw for

388o-8186-8445-3198 $10.00 0 1998 IEEE

simulating robotic operations since the interactionsbetween the robot and the object which are governed bythe law of physics cannot be reflected. Physically correctemulation is desired to gain insights into the dynamicbehaviors of the objects to be handled and the robots [5].In our system, we incorporated dynamics in modelingthe virtual objects and the robot. In particular, objectdeformations are supported via the physics basedmodeling. A flexible object under consideration ismodeled as a set of particles of masses interconnected byspring and damper systems as adopted in [5] except thatwe deal with 3D structure here. The dynamic behaviorsof the virtual objects including their deformations andmotions are determined by the motions of these particles,which is effectively dictated by the Newtonian law ofmechanics [6] via the following equation:

[M]P + [C]i + [K]x = F (2-l)

where [M] is the mass matrix, [C] is the damping

matrix, [K] is the stiffness matrix, F is the external

force applied to the particles. X, i, if are thedisplacement, velocity and acceleration vectors of theparticles respectively.

The objects to be handled and the handling tools (e.g.gripper) are modeled as flexible objects using the abovephysics based model. Other objects that do not exhibitsignificant compliance, such as the links of a robot, arestill modeled as rigid bodies but with their dynamicproperties (e.g. mass/inertia) incorporated. The physicalquantities such as contact force, velocity, acceleration,kinetic and potential energy are used to simulate thebehaviors of the objects and their interactions. Byautomatically formulating the dynamic deferentialequations, the models are made easily expendable. Theequations of motion are on-line formulated according tothe change in contact between virtual objects. A contactstate register detects the events of contact and break ateach time step, which is used in determining theequations to be employed The solution of the aboveequation was achieved by numetical integrationiteratively at each time step within which linear changesin the acceleration was assumed. Using Wilson 8method as an improvement for the linear accelerationmethod, unconditional stability could be achieved when8 2 1.3 7 With the acceleration varying linearly from tto t + @I with 8 2 1, the motion of the particles att + At is calculated from the knowledge of the motionat t In the our implementation, we chose 8 = 1.4 Thecurrent implementation was conducted on a PC-basedVR platform under a Superscape VRT environment. Theprograms were developed using Watcom C and downloaded to the VR environment for execution,

3. Dynamic measurements and interactionsin virtual environment

It has been experimentally noted that it isdifftcult to manipulate objects in VR withoutinformation feedback of the interaction forces [7].This information can readily be provided in thephysics based modeling adopted here since theforces take an explicit part in the formulation of theobject motion equations. To provide the informationon the interactions between virtual objects, a forcesensor model was developed and incorporated inthe simulation. Based on our previous work onmodal analysis of force sensing elements, asimplified first order model of a force sensor wasdeveloped [8]. Using this model, a six-axisforce/torque sensor model can be derived asillustrated in Fig 2.

Fig. 2 The dynamic model of the six-axis forcefor measuringF(FI,Fyr~)andM(MI,M,,M,)

For small deformations in the sensing elements, it hasbeen shown that the outputs of the sensor areproportional to the displacements [8] which can be usedto measure the forces and torques. The outputs of theforce sensor are related to the deformations of thesensing elements via the following equations:

(F, >, = 4K(z - 20 1 (3-l)

(MXj5 = 2KQ2(B, - %I) (3-2)

(MY >, = 2Ka2 (0, - e,, 1 (3-3)

(F, 1, = =(x - xg ) (3-4)

(F,), = 2KO - ~0) (3-5)

00, = 4Ka2(8, - e,,) (3-6)where (F,, F,, F,), and (M,, A4,, M,), are the force

and torque variables of the sensor, K is the stiffness ofthe sensing elements, x,, , y,, , zo, OX0 , 0 , and

8, o are the initial values of displacement variab;ls. The

interaction force information is provided to the user viavisual and audio cues as well as numerical outputs. Fig.3 shows the case when a ball falls to a plate mounted onthe force sensor. The interaction forces and torques aredisplayed via the bars by the side of the sensor apartfrom the numerical outputs.

389

Fig. 3 A test of the force sensor

When a ball is dropped, it falls and collides with therigid plate attached to the force sensor, bounces back,and falls again a couple of times before its potentialenergy is dissipated. The collision forces measured bythe sensor is plotted in Fig 4.

Time(ms)

Fig. 4 The measurement of collision force

The above sensor model suffers from the limit of thegraphics resolution. To solve this problem, a motionprediction approach is taken. At any time step, we usethe current state of the systems to predict the motions ofthe objects in a small time interval which can be afraction of the simulation step. Using the predictedobject motions, a more precise “measurement” in theforce/torque sensmg can be achieved. In ourimplementation, a better than 2% precision in the forcemeasurement was found achievable when singleprecision numbers were used. Other types of sensorswere also implemented. An example is the proximityrange sensor which is useful in detecting impendingcontacts between a robot end effector and object. Eachtime the sensor is invoked, a bullet is sent out. Onhitting a target, the distance traveled is returned as therange measured. This information is needed formaintaining stability in controlling the transition fromfree motion to contact. We have incorporated a rangesensor in each of the fingers of an simulated grippers.

4. Discussions and further work

The aim of the work is to develop a virtual realityplatform that can support the emulation of the dynamicbehaviors of robots and their interactions with objects.The virtual environment is enhanced by the use of

390

physics-based modeling. The platform being developedwill allow contact sensing, modeltng of dynamicinteractions as enhanced by visual and auditoryrendering. A basic robot work cell has been built in VRincluding robots, deformable objects. conveyers. andgrippers. Dynamic properties have been incorporatedinto the models of the robots and objects. The physicsbased model was applied to the flexible object modelingand force sensing. Interaction rendering was studiedwith some via visual and auditory cues developed. It isfound that physical properties of virtual objects areimportant for simulating and visualizing the dynamicbehaviors of robotic aided manipulations. Visual andauditory cues are useful in helping the operator respondmore quickly to happenings in the absence of otherfeedback devices. Computational cost has been the mainobstacle to the implementatton of physics basedmodeling in VR for robotic applications to achieve realtime performance. Therefore, attention needs to be paidto the efftciency issue. Our future work will include thestudy of more complicated models and development ofmore efficient methods for simulation and visualization.The modeling of the virtual tools with their attributesdeserves special attention. The preliminary results showthe promise of implementing physics based models inrobotic applications and assembly planning. With theincreasing computer power. faster simulation andvisualization of dynamic interactions among objectsusing more complex physics based models willeventually become feasible.

Acknowledgments

This work is supported by the Research GrantsCouncil of Hong Kong through grant number 9040309.

References

C. Chen, and M. Trivedi. Simulation and Animation ofsensor-driven robots. IEEE Trans. On Robotrcs andAutomatzon, Vol. 10, No. 5, pages 684-704, Oct. 1994.H. Najjari and S. J. Sterner, Integrated sensor-basedcontrol system for a flexible assembly cell, Mechatronics,vol. 7, No. 3, pp. 231-262, 1997.D. Baraff, Coping with friction for non-penetrating rigidbody simulation, Computer Graphics, Vol. 25, No. 4, pp.3140,1991.N. Tamoff, A. Jacoff, and R. Lumta, Graphical simulationfor sensor based robot programmmg, Journal of Intelligentand Robotic Systems, vol. 5, pp. 49-62, 1992.A. Joukhadar, C. Bard and C. Laugier, Combininggeometric and physical models: the case of a dexteroushand, Proc. IEEE IntemationaI Conference on IntelligentRobots and Systems, pp.374-380, 1995.D. Terzopoulos, Elasttcally deformable models, ComputerGraphics, Vol. 21, No. 4, pp. 205-214, July, 1987.R. Gupta, T. Shendan, and D. Whiteney, Experimentsusing multimodal virtual envuonments in design forassembly, Presence, Vol. 6, No. 3, pp. 318-338, June 1997.X. B. Chen, Y F. Li, On the dynamtc behavior of wnstforce sensor for robots, Proc. IEEE InternationalConference on Instrumentation a n d M e a s u r e m e n tTechnology, Ottawa, Canada, May 19-21, pp. 963-968,1997.