Design and implementation of Haptic based Artificial Hand Mode (AHM) based on

Virtual Reality Interface of DHM System

1Noaman M. Noaman, 2Abbas Kader Abbas, 3Rana Mazin Kasim 1Department of Computer Engineering /Computer Man College Sudan, 2Medical

Engineering Dept Medical Information Technology, RWTH –Germany, 3Biomedical

Engineering Dept.; RWTH Aachen –Germany

Abstract. .Dexterous Hand Master (DHM) is one of exoskeletal controlling devices used to measure the positions of an operator’s hand, and utilize these positions to control the motion of a remote manipulator or slave. Applications motivating development of such device have ranged from the control of a remote dexterous robot hand in physical applications, to manipulate motions of a virtual hand. A simplified kinematics model of human hand was developed to meet ability of a proposed DHM glove. In this study the design of the integrated Haptic Hand have been developed based artificial hand model for assistive therapy of partial paralyzed and unilateral paraplegic subjects for upper extremities. As many spasmodic extensor-flexor muscles of the human upper limbs, this model are based on DHM like system utilized the phalangeal biomechanical model to simulate the virtual reality system for arthrokinematic scheme of designed system.

Keywords. Haptic, DHM, Virtual Reality, Arthrokinematics, properioceptive mode

A Myocybernetics MCT hand and DHM are by definition connected an EMG-driven interface to human residual muscles in upper limb and thus makes it possible to exploit sensorimotor mechanisms for controlling hand actions and complex movement. While the ultimate goal of the myocybernetic prosthetic hand presented in this paper (myocybernetics) is to allow human amputees dexterous sensorimotor control (Fig.1)via a neural interface that provides efferent commands to control the hand and sensory feedback from artificial sensors—this paper focuses on the bioinspired design of this hand. Within the foreseeable future, neural interfaces will allow only a limited number of channels for exchanging efferent and afferent signals with the central nervous system (CNS) of a human. The cybernetic hand presented in this paper overcomes this limitation by its mechanical design that allows hand preshaping and specific grasping forces on the basis of only a few efferent control signals. Moreover, the integrated design makes it possible to provide task-specific feedback by utilizing a few sensory channels.

I. INTRODUCTION

Challenges for Assistive TechnologyG. Eizmendi et al. (Eds.)IOS Press, 2007© 2007 The authors and IOS Press. All rights reserved.

543

II. DESIGN ASPECTS OF DHM VR-BASED MODEL

The problem of functional replacement of an upper limb is an ancient problem: historically humans have replaced a hand lost in war or accidents with prosthesis for cosmetic, vocational, or personal autonomy reasons. The interest of the user community is primarily task-oriented, that is, patients express their need to replace the missing limbs to be able to autonomously perform their own activities of daily living (ADLs) [7]. For both practical and technological reasons the engineering design of biomechatronic haptic hands requires a restricted requirement list. Compare, for instance, the task of grasping an avocado with that of determining if it is ripe. It is thus necessary to define priorities among the different requirements and to address separately those that are considered the most important ones. Yet, to create a universal priority list to generalize design rules is challenging because individual subjects may have very personal preferences and expectations for cosmetic appearance, functionality, or reliability depending on their psychological, cultural, and geographical background [4].

aspects of haptic inference for the amputated patient .The main factors for this are low functionality, poor cosmetic appearance, and low controllability [4]. In short, in addition to cosmetics, many subjects find it impossible to perform many grasping tasks and the control system is unnatural, making the hand an external device that is not part of the subject’s body. The most known and worldwide implanted haptic prosthetic hand is the Otto Bock SensorHand [6]. The SensorHand, however, is unable to match even a small fraction of the capabilities of the human hand or offer a grip that adapts to the form of objects [5]. The hand mechanism does not allow adequate wrapping of objects, and its low compliance leads to instability of the grasped object in the presence of external disturbances. These results in a negative loop in the mechanism design because to provide sufficient grasp stability, it is necessary to use higher force; consequently high-power [4]. VR-mode that is used in construction of DHM-plant in virtual environment in order to simulate the actual mechanical interaction of DHM-based system with external effects of properioceptive/haptic inferences, the design paradigm was based on the approximated geometry of the anatomical part of the hand

Figure.1 (a) illustration of dexterous manipulation of DHM Cyber glove-system mechanical FBD with object in sensing, (b) Kinematics free body diagram for the DHM linkage of haptic interface in manipulation

and grasping force approach [3].

Manipulation

2F1F

ginprasgObject

underp

Finger tip

Finger ti

As in Fig.1 illustrated the main aspect of tactile haptic dynamics for grasping and manipulation [2]. in which any design of AHS system should adapted to achieve thhos

N.M. Noaman et al. / Design and Implementation of Haptic Based Artificial Hand Mode (AHM)544

III. MATHEMATICAL MODELING OF HAPTIC HAND VR-BASED

DYNAMICAL SYSTEM The first step in developing the kinematics of the DM Problem is to calculate the required fingertip forces from a desired force/torque wrench zn the object. The basis for this calculation is the grasp Jacobian relationship [4].

tipobj f.f G …………….. (1)

The grasp Jacobian (or grasp map), G, can be obtained by resolving each fingertip force to a common coordinate frame embedded in the object. For each fingertip i, this force resolution results in the mapping matrix G,

itipobj ff iG ……………. (2)

In Equation 2, the force vectors f are generalized vectors: they may include both forces and torques. The individual mapping matrices Gi are concatenated to form the grasp map G, and the fingertip force vectors are also grouped into one vector.

m

1

tip

tip2

tip

21obj

f

...

f

f

......f mGGG ...……….. (3)

Note that Eq. 3 is a simplified treatment of the problem. Typically the fingertip forces are represented in a coordinate frame at the contact point on the surface of the object. Then, knowing each contact type [6], the number of allowable force directions at each contact is reduced [9]. Minimizing the dimension of the problem illustrated in [7, 10]. The grasp Jacobian developed above allows us to calculate the required contact forces from the desired force on the object. In order to produce these forces at the fingertips, we now develop a hand Jacobian, which will allow us to calculate the joint torques from the contact forces [19]. The hand Jacobian, Jh, is based on the standard Jacobian, which relates end effector forces to individual joint torques for a robotic manipulator (in this case one for each finger).

itipfTii J …………….. (4)

In the DM problem, these individual Jacobians are brought together to form the hand Jacobian.

mtip

tip2

1tip

2

1

2

1

f

...

f

f

0...0

0.........

......0

0...0

...Tm

T

T

m J

J

J

……………… (5)

A nice conceptual picture of the roles of the grasp Jacobian and the hand Jacobian is shown in Figure.2 [6]. Given a set of contact forces, the individual joint torques can be obtained by multiplying by the transpose of the hand Jacobian, Jh, and the forces on the object can be obtained by multiplying by the grasp Jacobian, G [10].

cf.Th

J ……………. (6)

cobj ff G ………………. (7)

N.M. Noaman et al. / Design and Implementation of Haptic Based Artificial Hand Mode (AHM) 545

Figure 2 The role of the hand and grasp Jacobians. For the fingers, is the vector of joint torques and is the vector of joint velocities. For the contacts, fc, the vector of contact forces and xc is the vector of contact

point velocities. For the object, the resultant force vector and the vector of object velocities are shown [6].

Fig.4 (a) the CAD-design system for the Myocybertonics model based on VR-system translational kernel, (b) VR-Simulation mechanical design.

Alternatively, from the kinematics point of view, the z contact point velocities can be obtained from the finger joint velocities by multiplying by the hand Jacobian or by multiplying the object velocity by the grasp Jacobian:

objT

ch vGxJ ……… (8)

IV. RESULTS

Simulation result of the MCT-Model with virtual reality system used in the operating environment, VRML® (Mathworks, MA, USA), based platform used in the simulation design, the complete prototyping design of the haptic prosthetic hand system is illustrated in fig (4a,4b) and fig (5a,5b) where basic kinematics implemented in the VR-actuating path, the method of Jacobian grasping techniques is demonstrated in different anatomical and functional position of natural hand, adapting a virtual tactile sensing for integrating this mode in new AHS-system , local kinematics enhancement was observer in model , with some incompetence in repeated tactile feedback raised during simulation , nonlinearities appear when implementing driver-based linkage ,although this overcome by virtual adapting gain elements which were inserted in the path of haptic DHM-model , another important point in simulation result is joint-to joint motion plane adaptation in which 84%-87% of arthrokinematics compatibility was achieved for haptic DHM-system

Relative velocities

of contact points on

each of the two objects

Velocities of contact

frames on objects

(1 and 2) and spin

Figure 3 Contact variables and contact frames of rolling and contact system variables [7].

1u

2u

vyx

y

z

Contact equation

1u

2u

vyx

y

z

Contact equation

vx

1u

2u

vyx

y

z

Contact equation

Relative velocities

of contact points on

each of the two objects

Velocities of contact

frames on objects

(1 and 2) and spin

vx

1u

2u

vyx

y

z

Contact equation

vx

Fig (3) illustrate the main interaction parameter of the contact equation in Jacobian space which resembles the virtual vectors of motion and force in DHM model.

N.M. Noaman et al. / Design and Implementation of Haptic Based Artificial Hand Mode (AHM)546

V. CONCLUSIONS

The general accepted definition of a prosthetic upper-limb device is that it provides functional replacement of a lost organ without restoring normal control modality. The aim of the Myocybernetics hand unit that is based on DHM-Concept is to investigate methodologies that go beyond this straightforward approach of functional replacement and to ultimately connect the hand with the human brain. The modular design of the DHM-hand makes it possible not only to work in incremental steps toward this goal, a goal that can easily be motivated by the limited success of existing prosthetic technologies but also to exploit its advanced technologies to critically test VR- hypotheses regarding tactile sensorimotor control and the functional roles of the biological sensory systems during manipulation tasks, DHM-Hand provides a framework for clinical assessments and comparisons between different VR interfaces and their integration with the local controller of the hand. But the ultimate success of the DHM-Hand does not depend on whether VR-succession interfaces are found to be advantageous or not, because it's modular design allows practically any kind of user interface. Moreover, its modular control architecture allows for testing of the usefulness of various sets of grasps in both daily and vocational activities.

REFERENCES

[1]. Stellin G, Cappiello G, Roccella S, Becchi F, Metta G, Carrozza MC, Dario P, Sandini G (2006) Preliminary design of an anthropomorphic dexterous hand for a 2-years-old humanoid: towards cognition. In: Proceedings of the 1st IEEE/RASEMBS international conference on biomedical robotics and biomechatronic, Pisa, Italy, pp 290–295.

[2]. Nader M (1990) The artificial substitution of missing hands with myoelectrical prostheses. Clin Orth Rela Res 258:9–17

[3]. Tegin J, Wikander J (2005) Tactile sensing in intelligent robotic manipulation: a review. Ind robot Int J 32(1):64–70

[4].Nielsen KD, Cabrera AF, Nascimento OFD (2006) EEG based BCI-towards a better control. IEEE Trans Neural Syst Rehabil Eng 14(2):202–204

[5]. Pfurtscheller G, Muller-Putz GR, Schlogl A, Graimann B, Scherer R, Leeb R, Brunner C, Keinrath C, Lee F, Townsend G, Vidaurre C, Neuper C (2006) 15 years of BCI research at Graz University of Technology: current projects. IEEE Trans Neural Syst Rehabil Eng 14(2):205–210

[6]- http://www.ottobockus.com ; Otto Bock Healthcare, Minneapolis, MN [7].Pylatiuk C, Mounier S, Kargov A, Schulz S, Bretthauer G (2004a) Progress in the development of a

multifunctional hand prosthesis. In: Proceedings of the 26th IEEE annual conference of the Engineering in Medicine and Biology Society, pp 4260–4263

[8]. Zecca M, Cappiello G, Sebastiani F, Roccella S,Vecchi F,Carrozza MC, Dario P (2004) Experimental analysis of the properioceptive and exteroceptive sensors of an under actuated prosthetic hand. Adv Rehab. Robot 306:233–242

[9]. Riso RR, Ignagni RA, Keith MW (1991) Cognitive feedback for use with FES upper extremity neuroprostheses. IEEE Trans Biomed Eng 38(1):29–38.

[10].Sasaki Y, Nakayama Y, Yoshida M (2002) Sensory feedback system using interferential current for EMG prosthetic hand. In: Proceedings of the 2nd joint EMBS international conference, Houston, TX

Fig.5-a- VR-based DHM model using VRML®

Builder platform in MATLAB-environment

Fig.5-b- the haptic sensor integrated in the MCT-proposed model for three metacarpi - links with 6 DOF

N.M. Noaman et al. / Design and Implementation of Haptic Based Artificial Hand Mode (AHM) 547

Modelling and Simulation of 6DOF Grasping Myoelectric Artificial Hand

Mode (AHM) based on Anatomical model of Phalanx-model

1Noaman M. Noaman, 2Abbas Kader Abbas, 3Rasha Bassam Abdul Jabbar 1Department of Computer Engineering /Computer Man College Sudan, 2Medical Engineering Dept. medical Information technology RWTH-Aachen, 3Biomedical

engineering Dept, FACH-Juelich-Germany

Abstract. The concept of the bioprosthesis of AHM modeling system implementation in the simulated hardware module is presented, modeling of the phalangeal kinematics system and analysis of AHS was achieved for building functional prototype plant of prosthetic hand. The experiment based on simulation of proposed system using parallel computing experiments show that the proposed solution provides the desired dexterity and agility of the artificial hand model in comparison to DHM-design strategy.

Recent development in rehabilitation engineering field is in artificial hand technology and design of a highly performance and reliable with high degree of ergonomics, the first thing to be discussed and illustrated inn the introduction of the AHS–system of the upper extremity and it's prosthetic device which has a characteristics of the of a combination for robotic-gripper control module and dynamic mechanical circuit. That implemented in the terminal device of a prosthetic hand design and the implementation of the electronic processing interface unit for introducing a new field of Biocybernetics that is used a moderate and advanced electronic network for optimistic and linear operation of the prosthetic device and reduction the instability of the system [1]. The artificial hand system is consisting of the following units that are implemented as integrating prosthetic device which are the following:

The dynamic circuit unit which is the actuating and initiating the motion of the mechanical linkages.

The electronic circuit unit that controls the electrical signal fed into actuating

element (DC–servomotor) of dynamic circuit for the finger. In this study development

of biomechanical based design of AHS is proposed through initiation of 6 DOF dynamic linkages in which the prosthesis design was based on biomechanics aspects of 2-linkage pendulum kinematics [2].

I. INTRODUCTION

Challenges for Assistive TechnologyG. Eizmendi et al. (Eds.)

IOS Press, 2007© 2007 The authors and IOS Press. All rights reserved.

548

II. DESIGN ASPECTS OF THE AHM MECHANICAL-LINKAGE

In general, the proposed AHM consists of three main mechanical articulated linkage which simulate biomechanical prospects of the anatomical parts of the hand and taking into account the anthropomorphic characteristics of AHS, basically the linkage inertia and dynamic joints articulation properties was setting through, FBD, using tabulated biomorphic system for the anatomical phalanx which is consider here as a bar with inertial properties and damping coefficient which resembles the joint angular friction and associated with elasticity constant for the spring force in the joint under actuation , figure (1) illustrates the DOF –distribution for AHM system model , that used 2 DOF for single finger compartment and single DOF for thumb articulation [3].

The simulation of the model can be with virtual reality demonstrations for the AHM system, the use of SimMechanics® platform for simulation with wide range of flexibility in the design of the torque manipulation units, basically the single finger is considered as grasping force prospects and it was synthesized by inverse kinematics equation for parallel planar model that is used in the AHS system [7], [8].

III. MATHEMATICAL MODELING OF ARTIFICIAL HAND DYNAMICAL LINKAGE

A model of the palm-finger system based on the anatomical one and studies the motion paradigm of this system. The model consists in a kinematics chain containing dual bodies connected through joints S1 and S2 (Figure.2). The first body is the palm and links together the wrist and the proximal phalange of the finger, which is the second body of the system. The wrist allows the rotation of the hand with respect to the arm, meaning three degree of freedom (DoF). Metacarpophalangeal (MCP) joint allows two kinds of motions (two DoFs) to the proximal phalange of the finger:

• Adduction/abduction (in the palm plane). [2] • Flexion and extension (with respect to the palm). [3]

The third body is the middle phalange and is linked to the proximal phalange through distal interphalangeal (DIP) joint. The last body of the studied system is the distal

FIG .1 a- Proposed Model of AHS-System

Based On the Anthropomorphic Approximation FIG. 1 b- the anatomical illustration of human hand

with illustration of inter phalangeal joint in point of articulation [3].

N.M. Noaman et al. / Modelling and Simulation of 6DOF Grasping Myoelectric AHM 549

phalange, linked to the middle phalange through the proximal interphalangeal (PIP) joint. The last two joints have only one DoF each, so the whole system has seven DoFs. Each joint is characterized by a specific geometry and by a minimum and maximum angle which constrain the motion of the system [2]. Another constraint is introduced by the naturalness of the hand motion [4]. For example, to flex or to extend the finger, all the phalanges are moving in the same time and to catch an object each phalange is moving separately (the first one which moves is the proximal phalange, followed by the middle and then the distal phalanges until the object is fully controlled), figure (3) illustrate mechanical design of articulated phalanx system [9].

.

IV. DESIGN CONSTRAINS IN THE AHS MODEL With respect to the modular design of AHM system some constrains in implementation of such system were noticed, one of the main problem is the definition of CG for dynamic linkage through multiple operation of grasping, picking, holding deformable shapes and complex kinematics through actuating plane, precision of movement which is synchronized with external input; e.g. EMG-ENG signals and its

FIG. 2 Mechanical model of single phalanx joint system for MCT-plant system Approximation

FIG. 3 Mechanical design of the articulated phalanx system units for palmar

0 10 20 30 40 50 60 70 80 90 100-0.005

0

0.005

0.01

0.015

0.02

0.025

Time (sec)

Am

plit

ude in r

adia

ns (

rad)

Step response for the 3 diffreent PIIP implemented in

AHS-module

PIP1

PIP2

PIP3

Step Response

Time (sec)

Am

plit

ude

Impulse Response

Time (sec)

Am

plit

ud

e

0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 50

0.01

0.02

0.03

0.04

0.05

0.06

System: dc_ahs1_feedback

Time (sec): 1.04

Amplitude: 0.0565

System: dc_ahs1_feedback

Rise Time (sec): 0.593

0 0.5 1 1.50

0.02

0.04

0.06

0.08

0.1

0.12

0.14

System: dc_ahs1_feedback

Peak amplitude: 0.128

At time (sec): 0.165

System: dc_ahs1_feedback

Settling Time (sec): 1.22

Figure .4 (Right)- unit step response for AHS-system, (Left) impulse response for AHS –system [12].

S1 S2

N.M. Noaman et al. / Modelling and Simulation of 6DOF Grasping Myoelectric AHM550

correlated Biopotenial signals that is used as actuating functions for the AHS system [5],[7]. below are the crucial constrains that adopted form the simulation of AHS model with different stiffness and damping coefficient that is derived through design and analysis of the kinematics model for plant system[9] , (1) Minimization of the spring-stiffness- constant it shows transient effect on step response and this will cause non-compensated vibration of the inter-phalangeal joint[6]. (2) Discretization of the plant adding some dead-time effect to the plant response for shortest time-constants of the PID-controller [6]. ,(3) Effect of the dc-motor inductor value must be lie with appropriate range from (0.351 mH- 20 mH) in order to achieve stale and smooth operating point ,(4) Selection of the PID-controller gains for proportional action and derivative action and Integral action should obey the following rule for dynamic range of Op-Amp units , (5) Geometrical properties of the phalangeal lever-system should based on optimal inertial matrix (Jn) for these links the designer should take into consideration the ratio of rotational torque (Tr)-to-Inertial of the dynamic link(JL) which must be less than or equal (3.62) for robust-response in time domain-system , this add a high

tolerant effect on the inertial properties of AHS , the phalangeal and interphalangeal design must simulate the real and actual dynamic space of joint kinematics [8].

V. RESULTS Simulation results for the step response and dynamic performance of the AHS model is show in figure(4) , in this paradigm reduction of rise time (tr) and increasing of Mp peak amplitude for step repose of AHS was recognized as one aim for the design , different gain settings was achieved in order to tune the dynamics of the feedback module for perceptional sensing, the effect of different flexion-extension angle in a varieties of subject under test for anthropomorphic AHS system design in order to establish a high performance criteria in different anatomical and kinematical operational conditions , a variation of the distal-phalangeal joint and carpal-phalangeal joint where main concentration of the AHS design pay attention on these two joints in order to set a good controlling approach with actuating driving mode from DC-stepper motor system , fig(5.b) showing comparison between three mode of prosthetic hand system , AHS ,DHM , AHS-MCT units ,

0 20 40 60 80 100 120 140 160-0.025

-0.02

-0.015

-0.01

-0.005

0

0.005

0.01

0.015

0.02

0.025

Simulated comand step-response for AHS

MCT-concept unit [4].

Time (second)

Am

plit

ude in R

adia

ns (

Rad)

PIP1 AHS

PIP1 DHM

PIP-AHS-MCT

FIG.5-b simulated step command response for the three modes and comparison of both of system

FIG. 5-a Developed Simulink Model for

Artificial Hand System that is based on anthropomorphic system approximation of PIP and DIP- Simulation with SISO-tool/ MATLAB

N.M. Noaman et al. / Modelling and Simulation of 6DOF Grasping Myoelectric AHM 551

VI. CONCLUSIONS The study of the human hand kinematics can offer interesting solutions for the human hand prosthesis development or in fields like computer animation or gesture recognition that add a greatest value for the assistive medical device and upper limb rehabilitation engineering techniques. The model which is presented in this paper is an anatomical one which based on anthropomorphic data, having the same elements and approximated motions. It can be successfully used to develop a functional prototype capable to copy as much as possible the natural model. The developer will have to decide what components should be used in order to obtain a cost-efficient and useful prosthesis so the patients will be eager to wear it.

REFERENCES

[1] Fuentes, O. and Nelson, R.C., “Learning Dexterous Manipulation Skills Using the Evolution Strategy”, in Proc. IEEE International Conference of Robotics and Automation (ICRA’97), Albuquerque, New Mexico, April 1997.

[2] N. Dechev, W. L. Cleghorn, S. Naumann. Multi-Segmented Finger Design of an Experimental Prosthetic Hand, Proc. of the 6th National Applied Mechanisms & Robotics conference, December, 199

[3] Dorf, R. C. and Bishop, R. H., “Modern Control Systems”, 9 Edition, Prentice-Hall, N. J., 2001 [4] J. Lin, Z. Wu, T. S. Huang. Modeling the Constraints of Human Hand Motion, Proc. of 5th Annual Federated Laboratory Symposium, Maryland, 2001

[4] Light, C.M., Chappell, P.H., Hudgins, B., Englehart, K.: Intelligent multifunction myoelectric control of hand prostheses, Journal of Medical Engineering & Technology,

Volume 26, Number 4, (July/August 2002), 139– 146. [5] Bouzit, M., Burdea, G., Popeseu, G. and Boian, R., “The Rutgers Master II-New Design

Force-Feedback Glove”, IEEE trans. Mechatronics, vol. 7, no. 2, pp. 256-263, June 2002.[6] Loredana Ungureanu, Doina Drãgulescu Modeling of Human Hand finger as Automatic system ", IEEE, ICRA '05 proceeding, Vienna

[6] Richard, C, Dorf, "Mechatronics System design", 3ed Edition, CRC-press, New York, 2004 [7] Mayer Kutz, Standard handbook of Biomedical Engineering and Design, McGraw Hill,

2004, 1st edition [8] De Luca, C.J.: The use of Surface Electromyography in Biomechanics, Journal of Applied

Biomechanics, Vol. 13, No. 2, May 1997. [9] Wolczowski, A.: Smart Hand: The Concept of Sensor based Control, Proc. of 7th IEEE Int. Symposium on Methods and Models in Automation and Robotics, Miedzyzdroje, 2001.

Author: Noaman M. Noaman (Assist.Prof. Control System engineering) Address: Department of computer Engineering and informatics- Computer Man College-Al Khartoum - Republic of Sudan Tel: + E-mail: noaman961@yahoo.com

Author: Abbas Kader Abbas (M.Sc. BME, Ph.D. student Biomedical Informatics) address: Kuhlwetterstr.8 –Zi 305 Aachen /ZIP-Post Code :D 52075-Germany Tel: +491 7621977243 E-mail: abbas.khudair@gmail.com

Author: Rasha B. Abduljabaar (M.Sc. BME) Address: Kaiserstr 23 Juelich-Germany Tel: + 491 7621 E-mail: rashabassamss@yahoo.com

N.M. Noaman et al. / Modelling and Simulation of 6DOF Grasping Myoelectric AHM552