multi-.ngered actile feedback from virtual and remote...

Multi-fingered Tactile Feedback from Virtual and Remote Environments

Alexander Kron Gunther SchmidtInstituted of Automatic Control Engineering

Technische Universitat MunchenD-80290 Munchen, [email protected]

AbstractThis paper outlines a novel approach of displaying multi-

fingered tactile feedback information from remote and vir-tual environments. Miniature tactile fingertip modules aredeveloped allowing multi-fingered integration into an al-ready existing hand force exoskeleton for display of com-bined tactile and kinesthetic feedback. The tactile feedbackcomprises generation of both, vibrotactile and thermal stim-uli directly at the operator’s fingertip where a multitude ofhuman tactile receptors are located. The proposed parallelconfiguration of integrated vibration and heat display withcombined kinesthetic feedback actuators considers a novelapproach towards the generation of holistic haptic sensa-tions. Corresponding tactile rendering algorithms are pre-sented computing realistic tactile stimuli of object proper-ties from either remote real or virtual environments. More-over, the article discusses the design of a developed ther-mal and texture sensor with corresponding data processingalgorithms for online identification of tactile object prop-erties in a remote environment. The new quality of gen-erated multi-fingered tactile feedback was evaluated in sev-eral experiments enabling a human operator performing ex-ploratory tasks.

1 Introduction

The design of advanced interactive systems enabling ahuman operator (HO) executing exploratory and manipu-latory tasks in virtual (VE) and remote (RE) environmentsrequires multimodal feedback [6, 11]. Typical applicationsare tele-surgery, tele-assembly, simulation and training sys-tems. Today, the interaction in an environment is performedover a human-system interface (HSI) equipped with inte-grated multimodal feedback actuators (see Fig. 1), each re-lated to one of the human senses: vision, audition, kinesthe-sia, touch, smell and taste. While highly developed 3D visu-alization techniques and auditory feedback systems are al-ready commercially available, the design of advanced hap-tic feedback systems is still in a rather early stage of devel-opment.

Typically humans are accustomed to execute exploratoryor manipulatory tasks by using their hands and fingers [11].Hand and finger kinematics comprise many degrees of free-dom ensuring a high level of skillfulness and dexterity. Thesensitiveness of the human hand is based on a multitude ofneural receptors embedded inside the fingertips. Humansare experienced in using this permanently available tactileand kinesthetic information in an intuitive manner. The as-sumption underlying our work is that intuitive and moreproper task execution in VEs or REs requires similar hap-tic feedback. In this context, the design of multi-fingeredhaptic feedback displays play a key role.

Figure 1. Telediagnosis: an application for anadvanced interactive system with multimodalfeedback.

In spite of a considerable progress in the design ofexoskeleton-based multi-fingered kinesthetic feedback dis-plays [2, 14], there is still a lack of integrated tactile feed-back components. However, additional tactile feedbackwould augment various applications - e.g. palpation of avirtual patient in a medical training system or palpationof a remotely located patient via a telediagnosis system(see. Fig. 1). A physician diagnosing for example skindiseases will require beside visual and auditory feedbackcombined kinesthetic and tactile information of several hap-tic submodalities - e.g. texture, thermal and stiffness feed-back. State-of-the-art tactile feedback displays already en-sure high feedback quality, but often only for a single sub-

Proceedings of the11th Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems (HAPTICS’03) 0-7695-1890-7/03 $17.00 © 2003 IEEE

modality and single-fingered display [5, 8]. Correspondingbulky hardware set-ups prohibit multi-fingered extensionas well as the desired system integration into hand kines-thetic feedback devices. Concerning heat and vibrationfeedback several single-fingered displays have been devel-oped [9, 13] as well as first multi-fingered tactile feedbackgloves with either embedded vibrotactile or thermal actu-ators [4, 7]. An overall parallel configuration of a multi-fingered kinesthetic feedback display combined with inte-grated vibrotactile and heat fingertip display has not beenrealized up to now.

This paper outlines a design for this type of multi-fingered haptic feedback display, based on integrat-ing miniature modular tactile fingertip devices into anexoskeleton-based kinesthetic feedback glove. The tac-tile feedback enables generation of vibrotactile and ther-mal stimuli directly at the operator’s fingertip. The pa-per presents the design of the developed tactile fingertipfeedback modules and describes corresponding algorithmsfor tactile feedback computation. Contrary to experiencingVEs, teletaction in REs requires in addition online identifi-cation of remote tactile object properties. The article out-lines novel designs of tactile sensors and implemented dataprocessing algorithms for remote data acquisition and iden-tification of texture and thermal properties. The proposedmethods allow the generation of high fidelity tactile feed-back interacting with both VEs as well as REs. The newquality of displayed multi-fingered feedback was evaluatedin several experiments.

The paper is organized as follows: Section 2 outlinesthe hardware set-up of recently developed tactile fingertipmodules. Section 3 reports on tactile feedback computationalgorithms. Section 4 describes sensor design and methodsfor tactile data acquisition in REs. Experimental results ofthe multi-fingered tactile feedback generation from REs andVEs are finally discussed in Section 5.

2 Tactile Fingertip Feedback Modules

Multi-fingered tactile sensation is achieved by the designof tactile fingertip feedback modules - called TactTip mod-ules (see Fig. 2a). The midget fingertip devices producevibrotactile and thermal stimuli [10] at an operator’s finger-tip.

Vibrotactile feedback is generated by means of a minia-turized DC-motor with a freewheeling out of balance massat top of the motor shaft (see Fig. 2b). With a maxi-mum speed of 10.000 RPM the vibration motor producesvibratory frequencies up to about 166 Hz with ampli-tudes > 3 µm. This type of vibrotactile feedback stimulatesPacini as well as Meissner’s corpuscles embedded in thehuman fingertip [3] and ensures noticeable vibrotactile sen-sation. The vibration motor is controlled by an adjustablemotor current as input. Amplitude and frequency of the vi-bratory output are coupled by a fixed relation.

Figure 2. Tactile fingertip feedback module.

The additional temperature feedback display comprises 4serially connected Peltier elements producing cold as wellas hot temperatures. Fig. 2c illustrates the tiny elementswith a size of 4x4x2.3 mm3. The display achieves heatingand cooling rates of +4.0 K/s and -2.5 K/s, respectively, andgenerates temperatures over the range from 8◦C to 80◦C.For safety reasons the highest output temperature is limitedto 42◦C. The Peltier elements are mounted on a small-sizedaluminium plate (10x15x0.5 mm3) together with a standardNTC resistor measuring the displayed temperature for heatcontrol. This plate ensures optimal thermal contact with thehuman fingertip. The overall temperature display is clippedonto a specially designed vibration case, embedding thefreewheeling vibration motor. Since Peltier elements areonly capable of producing temperature differences betweenboth sides of an element, generation of a low temperature atthe displaying side is directly accompanied by heat develop-ment on the opposite side. To avoid such heating a miniatur-ized water cooling circuit is integrated into the module (seeFig. 2a). The miniature size of the presented TactTip mod-ules ensures the multi-fingered system integration into anexoskeleton-based kinesthetic feedback display as demon-strated by Fig. 3 [10].

Figure 3. Integration of TactTip modules in ahand force feedback exoskeleton.

3 Tactile Feedback Computation

Beside the physical generation of tactile stimuli, render-ing algorithms are required for computing stimuli whichensure realistic vibrotactile and thermal sensations. Im-plemented feedback computation algorithms applied to theTactTip modules are described next.


3.1 Computation of Vibrotactile Stimuli

Display of vibrotactile stimuli ensures perception of tex-ture and object surface properties [9, 12]. Typical vibratoryactuator components - e.g. piezoceramics, voice coil speak-ers - are controlled by the common vibratory parameters:amplitude, frequency, and damping [12]. With respect toa miniature actuator design (< 10x15x3 mm3) these actu-ators produce vibratory stimuli too weak to be perceivedin addition to strong finger forces up to 10 N. In our ap-proach midget vibration motor ensures generation of no-ticeable stimuli. For this purpose the motor only provides2 vibratory control parameters: amplitude and damping ofmotor rotation speed, while amplitude and frequency of thevibratory output are coupled by a fixed relation. However,experiments with the vibration motor have led to the fol-lowing hypothesis: “A vibration motor controlled by meansof individual rendering algorithms can provide the HO withvibrotactile feedback comprising expected information con-tent”. This type of feedback can be fused by the HO to re-alistic sensations ensuring differentiation of various surfacetextures.

The implemented vibrotactile rendering algorithms takeinto account a bias component as well as additional dampedimpulses of motor rotation speed for producing the desiredvibration feedback. As typical exploratory procedures [11]the algorithms distinguish tapping at and stroking over anobject surface. Tapping means that either the fingertip col-lides with an object surface for a short time or it remainsfixed on the surface after a tap. At any rate the algorithmdisplays a damped vibration impulse for a constant timeinterval Tt (see Fig. 4). The generated tapping impulse isdamped by an exponential decay of the corresponding mo-tor rotation speed At(t). First At(t) is set to a fixed attackamplitude Ao, which is made proportional to the HO’s tap-ping velocity vn normal to the object surface multiplied byan object specific constant value Kv,t, Eq.(1). The corre-sponding damping coefficient Bt also depends on the objectmaterial.

Ao(v) = Kv,t · vn (1)

Stroking means, that the HO’s fingertip slides with thevelocity vs(t) and the contact force Fn(t) over an objectsurface. Typically vibration stimuli are perceived strongerby increasing vs(t) or Fn(t) [12]. During stroking the am-plitude As(t) of the motor speed is set proportional to bothvs(t) and Fn(t) with material specific coefficients Kv,s andKF,s, i.e.

As(v(t), F (t)) = Kv,s · vs(t) + KF,s · Fn(t) (2)

Eq.(2) shows that stroking over object surfaces with ho-mogenously distributed texture is displayed by a bias com-ponent of the motor speed, depending on vs(t) or Fn(t)(see Fig. 4). Vibrotactile perception of patterned objectsis realized by adding damped impulses J(t) to As(t). Suchan added impulse has a constant offset amplitude AJ withagain an object specific damping coefficient BJ .

The proposed vibrotactile feedback computation algo-rithms are applied for interaction in VEs as well as REs.Results of experiments comparing displayed vibration feed-back from VEs with stimuli sensed by using the humanhand in a real physical environment are presented in Sec-tion 5.1.

Figure 4. Motor rotation speed while tappingand stroking over an object surface.

3.2 Computation of Thermal Stimuli

For display of realistic thermal stimuli heat transmissionphenomena need to be considered [1]. Note, that the HO’stemperature perception differs while exploring varying ma-terials with equal surface temperature as caused by objectspecific heat conductivities. The human temperature per-ception is mainly affected by a dynamic temperature dis-tribution inside the fingertip where human thermoreceptorsare embedded. The proposed thermal rendering algorithmsuse a corresponding dynamical model determining the tem-perature distribution inside the fingertip during two signifi-cant exploratory modes:

• free fingertip motion in 3D space without object con-tact,

• fingertip contact with objects.

In the model the human fingertip tissue is decomposedinto 3 cutaneous layers: epidermis, dermis, and endodermis.For each layer the gain in heat accumulation dQ/dt per cm2

is given bydQ

dt= ∆xρc

dT

dt(3)

where ∆x is the thickness of a layer, ρ is the density ofthe cutaneous layer, c is the specific heat and dT/dt is thetemperature change rate. Heat exchange between adjacentlayers is modelled by

φ(t) =K(n+1,n)

∆X(n+1,n)[Tn+1 − Tn] (4)

where K(n+1,n) is the thermal conductivity between thecenter of both layers n+1 and n, ∆X(n+1,n) is the distancebetween these centers, and Tn is the individual temperature


of a layer 1. The model assumes heat flow through all lay-ers to be in normal direction and neglects lateral heat flow.The heat flow between endodermis and dermis is affected byadditional heat flows from fat tissue and blood vessels. Dy-namical parameters like metabolism and the arterial bloodtemperature are here assumed to be constant for feedbackcomputation.

Fig. 5 illustrates the overall scheme of the implementedthermal feedback computation. In a first stage the temper-ature distribution inside the fingertip for an individual con-tact in an environment is computed by ordinary differentialequations, leading to a three-layered ODE-model. Theseequations require initial temperature values of the tissue lay-ers T 0 as well as parameter vectors θi, comprising specificthermal parameters of the tissue layers and virtual objects.The ODE-model provides T s

1 , representing the temperatureof the first tissue layer. Note, that we assume human skinreceptors for thermal perception to be located in this layer.Fig. 6 depicts the simulated temperature distribution causedby heat flow through all three layers for fingertip contact atobjects out of iron and wood (surface temperature 35◦C) af-ter free fingertip motion (environmental temperature 25◦C).

Figure 5. Scheme of thermal feedback com-putation.

A second three-layered ODE-model simulates the fin-gertip contact with the real aluminium plate of the TactTipmodule. T s

1 from the first ODE-model enables the computa-tion of a solid display temperature T s

d , whose display leadsto the desired T s

1 inside the physical HO’s fingertip. A tem-perature control loop controls the thermal feedback device.Experimental results, reported in Section 5.1, demonstratedthat the proposed rendering algorithms ensure the sensationof realistic thermal object properties during exploration inVEs or REs.

4 Tactile Data Acquisition

The proposed rendering algorithms for tactile feedbackgeneration with the TactTip modules are applied to task ex-ecution in VEs as well as in REs. Interaction with a VEonly requires specifying individual rendering parameters,

1Specific thermal characteristics of these 3 cutaneous layers are pre-sented in [10].

Figure 6. Simulated temperature distributionfor fingertip contact at iron and wood.

like e.g. surface roughness, heat conductivity, etc. On theother hand object interaction in a RE requires online tac-tile data acquisition by means of tactile sensors. Data pro-cessing algorithms must compute corresponding texture andthermal rendering parameters by identifying remote objectproperties from measured data. Developed sensor designsas well as appropriate data processing algorithms are de-scribed next.

4.1 Texture Measurement

Measurement of texture information in a RE is achievedby means of a developed miniaturized texture sensor basedon the piezoelectric effect (see Fig. 7). The sensor devicecomprises 2 small-sized piezoceramics (∅ 10 mm) pastedtogether with a cone-shaped tip made out of steel. Duringobject surface exploration the miniature tip of the sensortransmits pressure signals to the piezoceramics for transfor-mation into voltage signals. Theses signals are integrated,low pass filtered and amplified. Pressure inputs up to 10 Ngenerates output signals up to 10 V. The 12 bit A/D con-verted pressure signals are employed for identification ofthe object surface roughness.

Figure 7. Sensor design for texture measure-ment.

Section 3.1 has already demonstrated that the proposedcomputation of vibrotactile stimuli requires a multitude ofobject and task specific parameters. The correspondingparameter identification is achieved by fusing sensor dataavailable from the remote telemanipulation system. Fig. 8outlines an overall scheme of sensor data fusion and dataprocessing.


Figure 8. Scheme of vibrotactile parameter identification in RE.Telemanipulation systems are typically equipped with

position and force sensors. Corresponding available sen-sor data as well as measured texture data are inputs for thevibrotactile parameter identification algorithm. A measuredcontact force normal to an object surface characterizes con-tact with the object. In addition the operator’s intention ofexploring the object’s surface needs to be determined. Atany rate the first contact with an object surface results indisplaying a damped impulse characterized by the tappingmode. If the HO rests after a tap with the endeffector ofthe telemanipulator on the surface, no vibration feedback isexpected. The quiescent mode is identified if stroke veloci-ties are less than a threshold value or the HO did not movea certain minimum distance from the tapping point. If theoperator motion exceeds the determined distance by a min-imum stroke velocity, the algorithm identifies the strokingmode. The mode identification returns to quiescent mode,if the HO looses object contact or rests again on the surface.

During the short time period of a tap an identificationof an object material is not feasible. In this case, requiredobject specific parameters Bt and Kv,t are determined byheuristics. They are considered constant for each displayeddamped impulse. Thus vibratory stimuli displaying a taponly vary because of different normal contact velocities.

While stroking over a surface, the stroke velocity as wellas the contact force are derived from position and force sen-sor data. The parametersKv,s andKf,s are identified by ap-proximating the actual roughness of an object surface usingthe time discrete pressure signal pk, which is measured bythe texture sensor presented above. The implemented dataprocessing algorithm computes absolute values pk, Eq.(5)and an average value pk, Eq.(6) at each time-step k.

pk = |pk| (5)

Note, that pk is always evaluated as an average valueover all k = 1..i discrete time-steps belonging to an actualstroking procedure.

pk =∑i

k=1 pk

i(6)

Kv,s and Kf,s are defined by classification of computedpk. If pk shows a value inside a heuristically chosen in-terval, the required parameters are determined. The corre-sponding intervals for the output parameter values Kv,s andKf,s are also chosen heuristically.

Stroking over patterned structures, e.g. grooves or edges,at a discrete time-step k = j is identified by a value pj

much greater than actual computed pj . In this case thealgorithm sets an impulse flag. High pressure signals pk

(k > j) caused by stroking over the patterned structureinfluence computation of the next time-discrete pk values.However, the algorithm requires pk values as reasonable es-timates of the common surface roughness neglecting spo-radic patterns. Thus the algorithm computes next pk values(k > j) using the stored pj value instead of actual pk val-ues until stroking over the patterned structure is finished.If actual pk falls below a threshold close to pj , the end ofthe patterned structure is detected. From here pk values areagain used for computing pk at succeeding time-steps. Thepresented algorithm is appropriate for generating a reason-able estimate for the required object surface roughness.

Section 5.2 outlines results demonstrating that measuredand computed vibrotactile parameters generate vibratorystimuli enabling the HO to distinguish between varying ob-ject surfaces in a RE.

4.2 Thermal Parameter Identification

The challenge of identifying thermal parameters in a REis attached by determining the current surface temperatureas well as material specific parameters, e.g the heat conduc-tivity. While miniature temperature sensors measuring theobject surface temperature with short time delays are com-mercially available, tiny sensors determining heat conduc-tivity of a material are still lacking. However, this parameteris required for the previously presented thermal renderingalgorithms. We designed a novel miniature thermal sensorbased on the thermal contact model depicted in Fig. 9.

Let us assume a thermal sensor component with an em-bedded heat source which is coming into contact with aremote object as a heat sink. Neglecting all lateral heat flow,


the remaining heat flow in normal direction is computed by

φ =A(Ts − To)

∆xs/λs + ∆xo/λo(7)

with the contact area A, the temperatures Ts, To, the con-ductivities λs, λo, and the thicknesses ∆xs,∆xo. Foronline thermal parameter identification the parametersA, Ts, λs,∆xs are known. Since φ cannot be measured,we introduce C as a virtual heat capacity for describing thedispensed sensor heat Q by

φ = CdTs

dt=

dQ

dt(8)

Inserting Eq.( 7) into Eq.( 8) leads to the ODE:dTs

dt= α

A

c(Ts − To) (9)

withα =

∆xs

λs+

∆xo

λo(10)

Its solution is given by

Ts(t) = (T 0s − T 0

o )e−(α/C·At) + T 0o (11)

with the initial temperatures T 0s , T 0

o at the beginning of athermal contact.

Figure 9. Contact model for heat flow in RE.

If the design of a device producing the proposed type ofmodelled heat flow during object contact is possible, extrap-olation of the measured temperature data allows to predictthe exponential response of Ts(t), see Eq.(11). A corre-sponding prediction algorithm determines the virtual heatconductivity term α/C as well as the steady state tempera-ture Ts(t → ∞), which is equal to the actual object surfacetemperature T 0

o . However, because of several simplifyingassumptions concerning the modelled heat flow, the pre-dicted α/C is not the exact material specific heat conduc-tivity. It represents a virtual parameter ensuring a classifica-tion of known remote object materials. Thus, the proposedalgorithm allows an online material identification. Subse-quently, demanded object specific thermal parameters aredetermined using a look-up table.

Fig. 10 illustrates the developed thermal sensor design.It comprises a small-sized case (14x14x8 mm3) made outof PVC with two separated chambers. In the first cham-ber a standard silizium spread resistor (SSR) is embeddedfor temperature measurement. The second chamber is filledby a miniaturized Peltier element (4x4x2.3 mm3). Both ac-tuator and sensor components are coupled with a thin alu-minium plate as top of the sensor.

The integrated Peltier element enables heating of the alu-minium plate. If no contact between telemanipulator andobjects is detected, the temperature of the aluminium plateis closed loop controlled to a defined offset temperaturecompared with environmental temperature2. If an objectcontact is detected, the controlled sensor heating is switchedoff and the embedded SSR measures Ts(t) required for theproposed prediction algorithm. 5-8 seconds after a contactthe implemented algorithm is capable of generating reason-able values for the actual object surface temperature as wellas the virtual heat conductivity used for material identifica-tion. Results of this thermal online parameter identificationprocedure are presented in Section 5.2

Figure 10. Sensor design for temperature andheat conductivity measurement in RE.

5 Experimental Results

The following two subsections outline experimental re-sults of generated thermal and vibrotactile feedback fromVEs and REs.

5.1 Object Identification ExperimentsThe quality of displayed tactile feedback was evaluated

in object identification experiments. Virtual scenarios weredeveloped for testing texture identification by vibrotactilefeedback as well as the capability of generating realisticthermal sensations. In these experiments additional kines-thetic feedback on fingers and wrist was applied. The cor-responding test objects were both implemented in a VE andwere also available as real components.

For a texture exploration experiment three object sur-faces - sandpaper, foam and plain PVC - were selected pro-viding homogeneously distributed surface textures. Othertest objects with patterned textures were grooved PVCblocks (see Fig. 11a). Virtual objects were visualized withthe same optical appearance (see Fig. 11b), but they dif-fered with respect to surface properties. The subjects wereassigned the task to match the virtual objects with their realequivalent. Additionally, each of the subjects rated the qual-ity of perceived haptic stimuli from 1 (unrealistic) up to5 (realistic). For this rating the virtual objects were visu-alized with the textured optical appearance as depicted inFig. 11c. One result of the experiment (see Table 1) wasthat surfaces with homogenous texture were more difficultto identify than grooved objects.

2The environmental temperature in RE is measured by a separate tem-perature sensor.


Figure 11. Real and virtual scenarios.Note, that we implemented all objects with same stiff-

ness for kinesthetic feedback display. This caused difficul-ties for the subjects to distinguish between the foam andthe plain PVC object. This experiment proved that sub-jects were capable to identify varying surface textures bythe displayed vibratory stimuli. Moreover, the experimentdemonstrated that subjects were capable of fusing paralleldisplayed tactile and kinesthetic stimuli to overall hapticperceptions. However, if haptic stimuli were not consistent- like e.g. displaying foam with too high stiffness - fusionof perceived stimuli was disturbed.

surface matching rating[%] x σ

foam 60 2.45 0.9945sandpaper 90 2.7 0.9579plain PVC 70 3.6 0.9894

grooved PVCclosely 90 3.55 0.6863

arbitrarily 85 3.65 0.8751widely 85 3.85 0.7452

Table 1. Results of 20 novice haptic feedbackusers matching real and virtual objects.

For thermal exploration experiments objects made outof aluminium (a), ceramic (c), PVC (p) and wood (w) wereagain available as real components and were implementedas VE with same surface temperatures. Subjects were as-signed to explore respectively two different virtual as wellas physical object materials (see Fig. 11d), based on theirthermal perception. The results demonstrated a satisfac-tory accordance between the exploration of objects in theVE and in the real world as depicted in Table 2. In bothcases, materials with relatively similar heat conductivitysuch as PVC and wood turned out to be harder to distin-guish, whereas significant differences in thermal conductiv-ity were perceived reliably. The experiment substantiatedthe conclusion still achieved from the texture explorationscenario, that subjects were able to fuse the simultaneouslydisplayed tactile and kinesthetic stimuli.

5.2 Remote Data AcquisitionThe efficiency of the earlier presented texture sensor and

the capability of implemented data processing algorithms

obj. real [%] virtual [%]1-2 1 < 2 1 = 2 1 > 2 1 < 2 1 = 2 1 > 2a-p 100 0 0 100 0 0c-w 91.7 8.3 0 83.3 16.7 0a-c 91.7 0 8.3 91.7 0 8.3p-w 41.7 58.3 0 66.7 33.3 0

Table 2. Results of 12 novice haptic feed-back users exploring two objects with differ-ent thermal properties.

to identify online required parameters for vibrotactile feed-back computation were tested in a single-fingered remotescenario (see Fig. 12). Fig. 12a illustrates a RE with varyingtextured objects as well as the telemanipulator carrying theendeffector equipped with the texture sensor (see Fig. 12b).

Figure 12. Texture sensing in RE.At the operator site subjects were enabled to explore re-

mote objects by means of a hand force exoskeleton com-bined with one TactTip module located at the index fin-gertip. The subject’s task was to identify different objectsurfaces by performing a stroking procedure. The resultsproved that the texture sensor and implemented data pro-cessing algorithms allowed the identification of homoge-nously textured surfaces with differing roughness. Sub-jects were also able to assert the sensation of patterns, e.g.grooves and edges, on corresponding surfaces. The onlineidentified vibratory parameters led to a satisfactory qual-ity of vibrotactile feedback which the HO fused to realistichaptic sensations.

The capability to identify thermal parameters in REswere tested using the presented thermal sensor for touch-ing varying object materials with similar surface tempera-tures. Fig. 13 illustrates predicted results of the object sur-face temperature as well as of the virtual heat conductivitytouching an aluminium object. The time responses showthat the prediction algorithm generates 7 seconds after con-tact satisfactory predicted values for both thermal parame-ters T 0

s , T 0o . The implemented algorithm allows identifica-

tion of the object material by means of classification usingthe evaluated virtual heat conductivity. The subsequent pa-rameter determination using a look-up-table of known ma-terials in RE ensures transmission of reliable thermal pa-rameters to the operator site a short time period after a sur-face contact. With these parameters the computation of re-alistic thermal feedback for display with the TactTip module


is made possible. Subjects asserted the sensation of differ-ent thermal sensations touching objects with differing heatconductivities but similar surface temperatures in the RE.

Figure 13. Results of thermal parameter iden-tification in RE.

6 Conclusions

This paper outlined a methodology and experimental re-sults for display of multi-fingered haptic feedback fromVEsand REs taking into account the combined display of heat,vibration and kinesthetic feedback. The proposed hardwareset-up is based on a hand force exoskeleton equipped withminiature tactile fingertip devices. The article presented thedesign of novel TactTip modules generating desired vibro-tactile as well as thermal stimuli. Moreover, we reported ontactile rendering algorithms for generation of vibrotactileand thermal stimuli with the TactTip module ensuring real-istic sensation of thermal object properties and surface tex-tures. For the interaction with REs novel sensor designs anddata processing algorithms were developed. The efficiencyof the proposed methods for displaying multi-fingered tac-tile feedback from REs and VEs was demonstrated in sev-eral exploratory experiments. The results proved that oper-ators were capable of fusing parallel haptic stimuli of vari-ous submodalities into overall haptic perceptions. We con-sider the proposed extended haptic glove system a novelapproach towards the generation of holistic haptic sensa-tions. We are convinced that this type of combined tactileand kinesthetic feedback display will be key for task exe-cution in advanced VE and RE applications, as for examplethe above mentioned palpation scenario of virtual or remotepatients. We expect that this new quality of haptic sensa-tions will increase the operator’s degree of immersion intocorresponding environments and will support a proper andmore intuitive operator’s task execution.

AcknowledgmentsThis work was supported in part by the German Research

Foundation (DFG) within the Collaborative Research Cen-tre SFB 453 on “High-Fidelity Telepresence and Teleac-tion”. Furthermore we would like to thank various mem-bers of our laboratory, especially the technicians, for theirinvaluable support in the design of the experiments.

References

[1] M. Bergamasco, A. A. Alessi, and M. Calcara. ThermalFeedback in Virtual Environments. Presence, 6(6):617–629,December 1997.

[2] M. Bouzit. Design, Implementation and Testing of a DataGlove with Force Feedback for Virtual and Real Objects Tele-manipulation. PhD thesis, Laboratoire de Robotique de Paris,1996.

[3] G. Bruggencate. Allgemeine Physiologie sensorischer Sys-teme. In P. Deetjen and E. J. Speckmann, editors, Physiolo-gie, pages 43–55. Schwarzenberg, Oldenburg, 1994.

[4] D. Caldwell, N. Tsagarakis, and A. Wardle. MechanoThermo and Proprioceptor Feedback for Integrated HapticFeedback. In Proc. of IEEE Int. Conf. on Robotics and Au-tomation, pages 2491–2496, 1997.

[5] V. Hayward and J.-M. Cruz-Hernandez. Tactile Display De-vice using Distributed Lateral Skin Stretch. In Proc. of theNinth Annual Symposium on Haptic Interfaces for Virtual En-vironment and Teleoperator Systems, pages 1309–1314, Or-lando, FL, 2000.

[6] J. Hollerbach. Some Current Issues in Haptics Research. InProc. of the IEEE Int. Conf. on Robotics and Automation,pages 757–762, 2000.

[7] Immersion, Corp. Tactile feedback man-machine inter-face device. An United States Patent, no.: 6,088,017http://www.immersion.com, 2000.

[8] P. Kammermeier, A. Kron, and G. Schmidt. Towards intu-itive multi-fingered haptic exploration and manipulation. InProc. of the Workshop on Advances in Interactive MultimodalTelepresence Systems, pages 57–70, Munchen, March 2001.

[9] D. A. Kontarinis and R. D. Howe. Tactile Display of Vibra-tory Information in Teleoperation and Virtual Environments.Presence, 4(4):387–402, 1995.

[10] A. Kron and G. Schmidt. Multi-fingered Haptic Feedbackfrom Virtual Environments by means of Midget Tactile Fin-gertip Modules. In Proc. of the 8th Mechatronics ForumInternational Conference, pages 1191–1200, University ofTwente, Netherlands, 2002.

[11] S. Lederman and R. Klatzky. Designing Haptic and Mul-timodal Interfaces: A Cognitive Scientist’s Perspective. InProc. of the Workshop on Advances in Interactive MultimodalTelepresence Systems, pages 71–80, Munchen, March 2001.

[12] A. M. Okamura, J. T. Dennerlein, and R. D. Howe. VibrationFeedback Models for Virtual Environments. In Proc. of theIEEE International Conference on Robotics and Automation,pages 674–679, Leuven, Belgium, 1998.

[13] M. P. Ottensmeyer and J. K. Salisbury. Hot and Cold Run-ning VR: adding thermal stimuli to the haptic experience. InProc. of the 2nd PHANDToM User’s Group Workshop, AILab Technical Report 1617, pages 34–37, 1997.

[14] M. Turner, D. Gomez, M. Tremblay, and M. Cutkovsky. Pre-liminary Tests of an Arm-Grounded Haptic Feedback Devicein Telemanipulation. In Proc. of the Winter Annual Meetingof ASME’98, volume 64, pages 145 – 149, Nov. 1998.


multi-.ngered actile feedback from virtual and remote...

Documents