Shaping Event-Based Haptic TransientsVia an Improved Understanding of Real Contact Dynamics

Jonathan P. Fiene Katherine J. Kuchenbeckerjfiene@stanfordalumni.org katherin@stanfordalumni.org

Telerobotics Lab, Stanford University, USA

Abstract

Haptic interactions with stiff virtual surfaces feel morerealistic when a short-duration transient is added to thespring force at contact. But how should this event-basedtransient be shaped? To answer this question, we present atargeted user study on virtual surface realism that demon-strates the importance of scaling transients correctly andhints at the complexity of this dynamic relationship. We thenpresent a detailed examination of the dynamics of tappingon a rigid surface with a hand-held probe; theoretical mod-eling is combined with empirical data to determine the influ-ence of impact velocity, impact acceleration, and user gripforce on the resulting transient surface force. The derivedmathematical relationships provide a formula for generat-ing open-loop, event-based force transients upon impactwith a virtual surface. By incorporating an understandingof the dynamics of real interactions into the re-creation ofvirtual contact, these findings promise to improve the per-formance and realism of a wide range of haptic simulations.

1 Introduction

Realistic excitation of the user’s sense of touch duringvirtual interactions has been shown to decrease task comple-tion time, error incidence, and cognitive load [2, 14]. Nu-merous applications exist wherein realistic virtual simula-tions could have significant potential, ranging from medicaltraining to mechanical design. For instance, a sufficientlyrealistic simulator could be used to immerse surgeons incomplex operative tasks without endangering live patients,thereby providing a safe, repeatable environment for learn-ing and practicing new procedures. Such simulators canbe used for both traditional and minimally-invasive surgery(MIS), with particular application in the emergent field ofrobotic MIS. To be a viable alternative to traditional train-ing, systems must faithfully reproduce the haptic cues that

surgeons experience during live procedures, a design goalthat requires careful attention to the dynamics of the corre-sponding real interactions.

While today’s impedance-type haptic simulators canadeptly portray interactions with soft environments, render-ing realistic rigid contact has been a significantly more chal-lenging objective. A large body of research over the pastdecade has sought to improve the feel of virtual hard con-tact, and one of the most promising advances has been rec-ognizing the value of transient contact forces. An overviewof this field is presented in Section 2, and Section 3 presentsa human subject study that examines how transient ampli-tude affects surface realism. In Section 4, we introduce aset of dynamic models based on first principles to revealthe relationship between the mechanical parameters of thecontacting objects (stylus and hand mass, surface stiffness,and surface damping), the user-controlled impact param-eters (incoming velocity, incoming acceleration, and gripforce), and the shape of the resulting force transient. Un-derstanding the dynamics of contact between a hand-heldstylus and a physical hard object will facilitate the processof virtual transient selection and enable more realistic vir-tual environments.

2 Background

While both slow pressing and discrete taps yield usefulinformation about an unknown material touched through astylus, LaMotte showed that active tapping best enables hu-mans to make accurate material property judgments, as thequickly changing contact force provides salient cues aboutsurface compliance [10]. During interactions with rigidsurfaces, haptic feedback necessarily becomes the primarysensory channel because the tool’s penetration into the sur-face is too small to be detected via vision or propriocep-tion. As shown in Fig. 1, tapping a stylus on a firm surfaceproduces a high-frequency transient acceleration, which isknown to excite the Pacinian corpuscles that lay deep withinthe glabrous skin of the hand [1, 17]. The central nervous

Time (ms)

Acce

lera

tion

(m/s

2 )

0 50 100

0

40

80

−40

Figure 1: The acceleration of a hand-held probewhen tapped on a firm surface.

system processes this sensory response and generates a hap-tic impression of the event, which depends primarily on thematerial and geometry of the object and is naturally con-trolled for changes in contact velocity and grip force [10].

Contact acceleration transients like the one shown inFig. 1 are often modeled with an exponentially decayingsinusoid, where the amplitude is linearly proportional tothe impact velocity and the frequency and duration dependempirically on the material properties of the surface andtool [12, 16]. Unfortunately, the low closed-loop bandwidthof most existing impedance-type haptic interfaces restrictsthe maximum virtual stiffness to that of soft foam [3, 8].Simulations that attempt to portray hard materials withclosed-loop position feedback thus feel far too soft and lackthe high-frequency accelerations that users expect when in-teracting with hard objects. While efforts have been madeto overcome these limitations, the required 1000-fold im-provement in renderable stiffness remains elusive.

One promising alternative for rendering stiff contact isthe paradigm of event-based haptics, wherein traditionalposition-based feedback is augmented with the open-loopdisplay of high-frequency force transients. Wellman andHowe first implemented such an algorithm on a hapticdevice, providing vibratory feedback via a secondary ac-tuator mounted near the user’s fingertips [16]. Shortlythereafter, Salcudean and Vlaar showed that an open-looppulse at contact improved the perceived stiffness of vir-tual walls without causing instability [13], and Okamura etal. demonstrated that overlaying proportional feedback withpsychophysically-tuned decaying sinusoids improved mate-rial discrimination [11]. Hwang et al. continued this line ofresearch through the use of short-duration force pulses tobring the stylus to rest with minimal penetration [7].

Kuchenbecker et al. then developed a deterministic ap-proach to transform real recorded accelerations into open-loop motor current transients to be played at contact, us-ing a full dynamic model of the haptic system to ensureaccurate matching of the virtual and real accelerations [8].A complementary user study of blind tapping on real andvirtual surfaces showed that the inclusion of fixed-widthpulse, decaying sinusoid, or acceleration matched transientssignificantly improved the perceived realism of virtual sur-faces over proportional feedback. Further work by Fiene et

al. demonstrated that a grip-force sensor could be used toestimate changing hand dynamics, which could then be ac-counted for in real time by adjusting the system’s dynamicmodel [4]. Such a system was shown to be capable of pro-ducing event-based virtual surfaces that accurately recre-ated recorded transient accelerations over a wide range ofgrip force values and incoming velocities.

3 Perception of Transient Amplitude

Prior work has shown that event-based transients gen-erally improve the realism of virtual hard contact, but lit-tle is known about the relationship between transient shapeand surface feel. We hypothesize that the perceived real-ism of a virtually rendered surface is strongly dependent onhow the amplitude of the acceleration transient comparesto that produced during interactions with the correspondingreal surface. A brief user study was conducted to test thishypothesis, as presented below.

3.1 Experimental Design

For direct user comparison between real and virtual sur-faces, the workspace of a desktop Phantom was divided lat-erally between a virtual rendering and a real object (a layerof wood on a foam substrate, similar to that used in [8]). Astylus was attached to the distal link of the Phantom to al-low the user to tap on both surfaces. An Analog DevicesADXL321 ±18 g accelerometer was affixed near the tipof the stylus, and a Flexi-force A201-1 force-sensitive re-sistor was located under the user’s index finger to measuregrip force. A PC running RTAI Linux was used to renderthe virtual surface and sample the sensors at 5 kHz using aNational Instruments PCI-1200 card, providing resolutionsof 0.172 m/s2 and 0.009 N respectively. The user’s hapticsense was isolated through the use of a visual barrier andheadphones playing cacophonous noise.

The virtual surface consisted of a 50 ms long, 50 Hzdecaying-sinusoid transient overlaid on a 900 N/m spring.For each tap on the virtual surface, the 32 Ns/m transientamplitude was scaled by the incoming velocity and multi-plied by a random value between 0 and 2 to introduce widevariations in contact response. Users were instructed to al-ternate tap on the real and virtual surfaces. After each pairof taps, users were asked to rate the feel of the virtual sur-face on a seven-point bipolar rating scale, where a rating of0 corresponds to a realistic virtual rendering, +3 is far toostrong and -3 is far too weak.

3.2 Results

Three users participated in the study, each rating 62 vir-tual surfaces. To understand the significance of these rat-

−3 −2 −1 0 +1 +2 +30.0

0.5

1.0

1.5

Ampl

itude

Rat

io

Too StrongToo Weak User Rating

Figure 2: Relationship between realism ratingsand the virtual-to-real transient amplitude ratio.

ings, we quantitatively compared each virtual contact to theresponse of the real surface, as follows. First, a linear fit be-tween incoming velocity and peak contact acceleration fortaps on the real surface was determined for each user. Thenthe amplitude ratio of each virtual tap was calculated bydividing the measured peak acceleration by that predictedusing the linear scaling model. Figure 2 shows the meanand standard deviation of these ratios for each realism ratingaveraged across subjects; as hypothesized, amplitude ratiocorrelates well with rated realism. Interestingly, the bestrealism rating (0) corresponds to a mean amplitude ratioof approximately 0.86, echoing the anecdotal observationof [8] that users prefer virtual transients that are slightlyless powerful than real surfaces. Additionally, the widestandard deviations of Fig. 2 show that an amplitude ratiothat is corrected for velocity alone does not fully character-ize the intricacies of user perception, though it captures astrong trend. As such, the remainder of this work will ex-amine the dynamics of real interactions to more thoroughlyunderstand the factors controlling transient amplitude.

4 Modeling and Analysis of Real Tapping

Event-based haptic algorithms commonly scale or indexthe amplitude of the contact transient by the velocity withwhich the user impacts the virtual surface [8, 11, 16]. Whilethis relationship significantly improves the feel of hard vir-tual surfaces, it does not provide an obvious way to incorpo-rate the effects of other varying parameters such as incom-ing acceleration and changes in hand dynamics. To under-stand how these other parameters affect the resulting surfaceforce, an ATI Mini-40 force sensor was placed beneath asample object, and the Phantom setup described above wasused to measure position, acceleration, grip force, and sur-face force for a series of 300 real taps spanning a range ofincoming velocities, incoming accelerations, and grip forcelevels.

To elucidate the relationships between the user-controlled parameters and the resulting surface force, this

section introduces three successive dynamic models of thestylus/hand system at contact and tests them against thisrecorded tap data. In all models, hand motion (position, ve-locity, and acceleration) and surface force are defined pos-itive away from the surface, such that negative incomingvelocities cause positive surface forces.

4.1 Momentum Approach

The first explored model treats the stylus/hand systemas a lumped mass, m, with a constant pre-impact velocityof vin. Under conservation of momentum, perfectly plasticimpact with a stationary surface will produce zero velocityafter time t1, and the change in linear momentum, L, of themass must equal the integral of the surface force, as follows

L1 − L0 = 0−m vin =∫ t1

0

Fs(t) dt (1)

Assuming that the contact acceleration follows an exponen-tially decaying sinusoid, the surface force will take the form

Fs(t) = β sin(ωt) e−αt (2)

where the frequency, ω, and decay rate, α, are empiricallydetermined from sample contacts. Substituting (2) into (1),solving for β, and rewriting (2) yields

Fs(t) = −m(α2 + ω2)ω

vin sin(ωt)e−αt (3)

which provides the previously observed linear correlationbetween incoming velocity and transient force amplitude.The peak of the decaying sinusoid surface force,

F ∗s = −m vin

√α2 + ω2 e−(α/ω) tan−1(ω/α) (4)

is therefore also proportional to the incoming velocity.Least squares regression between the incoming velocity

and the peak of the surface force for the recorded data yieldsthe linear fit shown in Fig. 3, which has an RMS error of0.205 N. While this model captures the primary trend, thescatter suggests that other factors contribute to the dynamicsof a tap, echoing the results of the user study in Sec. 3.

F ∗s = −32.34 vin

-0.1Peak

Sur

face

For

ce (N

) 76

5

4

3

Incoming Velocity (m/s)-0.12 -0.14 -0.16 -0.18 -0.2

Figure 3: Linear fit between stylus incoming ve-locity and peak surface force.

4.2 Adding a Constant Force

Recognizing that the user’s hand velocity may not beconstant before contact, we can use a measurement of thepre-impact acceleration, ain, to augment the model derivedfrom the momentum approach. This acceleration stemsfrom the user applying a force, Fin =m ain, which we cantreat as constant over the short duration of the impact. Thesurface force after contact must oppose this additional handforce, resulting in

Fs(t) = −m(α2 + ω2)ω

vin sin(ωt) e−αt − Fin (5)

F ∗s = −m vin

√α2 + ω2 e−(α/ω) tan−1(ω/α) −m ain (6)

Introducing an acceleration-dependent term into the least-squares fitting process reduces the RMS error to 0.148 N.Figure 4 presents these results graphically, wherein each pa-rameter is isolated by subtracting off the estimated contri-bution of the other parameters. The empirical data showsthe predicted linear effect of incoming acceleration, thoughquantization of the acceleration signal produces significantstriping. The resulting vertical spread in the data would col-lapse somewhat with a higher-resolution acceleration mea-surement.

Analyzing the result of the linear fit in connectionwith (6), we see that the acceleration scaling term shouldprovide a direct estimate of the impacting system mass.Considering the addition of the stylus and device, the least-squares result of 0.335 kg is reasonable when comparedwith other hand mass estimates [15].

F ∗s = −33.66 vin − 0.335 ain

Incoming Acceleration (m/s2)

PSF

Com

pone

nt (N

)

2 1.5 1 0.5 0 -0.5 -1

0.40.2

0-0.2-0.4-0.6-0.8

-0.1

PSF

Com

pone

nt (N

) 7

6

5

4

3

Incoming Velocity (m/s)-0.12 -0.14 -0.16 -0.18 -0.2

Figure 4: Linear fits between incoming velocity, in-coming acceleration, and peak surface force.

bs

ks

m

b k

xd

xm

Figure 5: Dynamic Impact Model.

4.3 Hand and Surface Dynamics

It is widely recognized that the dynamics of the handchange with configuration and muscle contraction. By ana-lyzing the surface force recorded from real taps with a stylusheld in a two-finger grasp, Fiene et al. found an approxi-mately linear relationship between grip force and the effec-tive stiffness, k, and damping, b, of the user’s hand [4], aresult that matches well with other research [5, 6, 9].

To account for changes in user impedance, we introducethe model shown in Fig. 5, which combines a second-orderuser model with a spring-damper representation of the sur-face. We define the state vector, x = [xm xm xd]>, theinput vector u = [xd], and the full state-space dynamics

x = Ax + Bu y = Cx + Du (7)

x =

0 1 0−(k+ks)

m−(b+bs)

mkm

0 0 0

xm

xm

xd

+

0bm1

[xd] (8)

y =[−ks −bs 0

] xm

xm

xd

+ [0] [xd] (9)

where the output y is the surface force, Fs. Assuming theuser has a constant desired acceleration, we know the inputxd(t)=vin + aint, and we can include an initial user force,applied via the spring k, by choosing the following initialstate vector

x0 =

xm,0

xm,0

xd,0

=

0vin

m ain/k

(10)

The Laplace transform of the system output can then bederived with the equation

Y(s) = C(sI−A)−1BU(s) + C(sI−A)−1x0 (11)

where U(s) = [vin/s + ain/s2], providing

Fs(s)=− (vins + ain)(bss + ks)(ms2 + bs + k)s3(ms2 + (b + bs)s + (k + ks))

(12)

Solving for the inverse Laplace transform of Fs(s) resultsin a time-domain solution of the form

Fs(t) = −β sin(ωt + φ)e−αt − c2t2 − c1t− c0 (13)

supporting the use of a decaying sinusoid transient in theprevious two models. After comparing the terms in (12)and (13), we can determine that the parameters governingthe decaying sinusoid are

ω =

√k + ks

m−

(b + bs

m

)2

α =b + bs

m(14)

β =

√c3

(v2

in −b + bs

k + ksvinain +

m

k + ksa2

in

)(15)

where

c3 =4(bbsks −mk2

s − kb2s)

2

(k + ks)2((b + bs)2 − 4m(k + ks))(16)

and the phase shift, φ, which stems from the addition of adecaying cosinusoid term, has the form

tanφ = p1p2ain − p3vin

p4ain + p5vin(17)

where the pn terms are various combinations of the system’smechanical parameters. Simulation has shown minimal ef-fect from the phase shift, which is on the order of 0.02 radi-ans for the parameters of our system. The remaining threeterms in (13) are

c2 = keain c1 = kevin + k2e d ain (18)

c0 = k2e d vin +

mk2s + ke(2bbs − d(k + ks))

(k + ks)2ain (19)

where the series spring stiffness, ke, and the d term are:

ke =kks

k + ksand d =

b

k2+

bs

k2s

(20)

Examining the system for the simplified case of zero in-coming acceleration, the time-domain response reduces to

Fs(t) = vin√

c3 sin(ωt + φ)e−αt + kevin(t + ked) (21)

which combines a velocity-scaled decaying sinusoid withramp and step functions resulting from movement of thedesired position within the surface.

To visualize how changes in hand parameters affect thepeak surface force, a simulation of the system was createdusing nominal values for the surface parameters and thehand mass. Fig. 6(a) shows the resulting change in peaksurface force as a function of hand stiffness and dampingfor zero incoming acceleration. The solid line, which is also

6 8 10 120

200

5.2

5.3

5.4

5.5

Damping, b (Ns/m)Stiffness, k (N/m)

Peak

Sur

face

For

ce (N

)

4 8 12Grip Force (N)

5.2

5.3

5.4

5.5

Peak

Sur

face

For

ce (N

)

0

(a) (b)

Figure 6: Simulation of the change in peak surfaceforce as a function of (a) the hand stiffness anddamping, and (b) the user-specific grip-force.

F ∗s = −32.92 vin − 0.35 ain − 0.022 gin

PSF

Com

pone

nt (N

)

2 1.5 1 0.5 0 -0.5 -1

0.40.2

0-0.2-0.4-0.6-0.8

0.40.2

0-0.2

-0.1

PSF

Com

pone

nt (N

) 7

6

5

4

3

Incoming Acceleration (m/s2)

Incoming Velocity (m/s)-0.12 -0.14 -0.16 -0.18 -0.2

0

PSF

Com

pone

nt (N

)

-0.4Grip Force (N)

5 10 15

0.6

Figure 7: Linear fits between incoming velocity, in-coming acceleration, grip force, and peak surfaceforce.

shown in Fig. 6(b), represents the user-dependent relation-ship between hand parameters and grip force from [4]. Theflatness of the surface in Fig. 6(a) suggests that user-to-userdifferences in the grip-force-to-hand-parameter relationshipshould not significantly affect the overall correlation to peaksurface force.

Introducing a linear dependence on grip force into theleast-squares parameter fitting routine reduces the RMS er-ror to 0.081 N, suggesting that the inclusion of both incom-ing acceleration and grip force most accurately accounts forthe amplitude of the transient response. The componentsof the peak surface force can be visualized along with theleast-squares results in Fig. 7.

5 Conclusion

This work provides an improved understanding of thedynamics of real contact and will enable more realistic vir-tual rendering of hard objects. The user study presentedin Section 3 shows that user realism ratings correlate withthe amplitude of event-based virtual surface transients. In-terpreting the results of Section 4, we see that momentumanalysis provides a justification for the commonly observedapproximate linear relationship between incoming velocityand transient force amplitude. The addition of a constantuser-applied force accounts for the more subtle effect ofincoming acceleration. The state-space model, which in-cludes both hand and surface dynamics, provides a theo-retical basis for the commonly assumed decaying sinusoidresponse. It also yields parametric relationships for the si-nusoid frequency and exponential decay rate based on mea-surable mechanical properties of the surface and the user.Finally, this treatment provides a formula for the surfaceforce as a function of time after contact.

Future work will continue to explore the structure andimplications of these models to understand how they canbest be employed in the real-time rendering of virtual hardcontact, including extension to more complex interactionssuch as three-dimensional contact and multi-body impacts.To ensure smooth transitions after event-based output, wewill develop methods for combining the calculated open-loop transients, which do not necessarily decay to zero,with traditional proportional feedback. We will also per-form more extensive user testing to determine the full effectof transient shape on perception.

Acknowledgments

The authors thank Professor Gunter Niemeyer of theStanford Telerobotics Lab for his encouragement and sup-port of this work.

References

[1] K. R. Boff and J. E. Lincoln. Engineering DataCompendium: Human Perception and Performance, vol-ume 3. Armstrong Aerospace Research Laboratory, Wright-Patterson AFB, 1988.

[2] G. C. Burdea. Force and Touch Feedback for Virtual Reality.John Wiley and Sons, Inc., 1996.

[3] N. Diolaiti, G. Niemeyer, F. Barbagli, and J. K. Salisbury.Stability of haptic rendering: Discretization, quantization,time delay, and Coulomb effects. IEEE Transactions onRobotics, 22(2):256–268, Apr. 2006.

[4] J. Fiene, K. J. Kuchenbecker, and G. Niemeyer. Event-basedhaptic tapping with grip force compensation. In Proc. IEEEHaptics Symposium, pages 117–123, Mar. 2006.

[5] A. Z. Hajian and R. D. Howe. Identification of the mechan-ical impedance at the human finger tip. J. BiomechanicalEngineering, 119(1):109–114, Feb. 1997.

[6] C. J. Hasser and M. R. Cutkosky. System identification ofthe human hand grasping a haptic knob. In Proc. IEEE Hap-tics Symposium, pages 171–180, Mar. 2002.

[7] J. D. Hwang, M. D. Williams, and G. Niemeyer. Towardevent-based haptics: Rendering contact using open-loopforce pulses. In Proc. IEEE Haptics Symposium, pages 24–31, Mar. 2004.

[8] K. J. Kuchenbecker, J. Fiene, and G. Niemeyer. Improvingcontact realism through event-based haptic feedback. IEEETrans on Visualization and Computer Graphics, 12(2):219–230, March/April 2006.

[9] K. J. Kuchenbecker, J. G. Park, and G. Niemeyer. Char-acterizing the human wrist for improved haptic interaction.In Proc. ASME Int. Mechanical Engineering Congress andExposition, volume 2, paper 42017, Nov. 2003.

[10] R. H. LaMotte. Softness discrimination with a tool. J. Neu-rophysiology, 83:1777–1786, 2000.

[11] A. M. Okamura, M. R. Cutkosky, and J. T. Dennerlein.Reality-based models for vibration feedback in virtual envi-ronments. IEEE/ASME Trans. on Mechatronics, 6(3):245–252, Sept. 2001.

[12] A. M. Okamura, J. T. Dennerlein, and R. D. Howe. Vibrationfeedback models for virtual environments. In Proc. IEEEInt. Conf. on Robotics and Automation, volume 3, pages674–679, May 1998.

[13] S. Salcudean and T. Vlaar. On the emulation of stiff wallsand static friction with a magnetically levitated input-outputdevice. ASME J. Dynamic Systems, Measurement and Con-trol, 119:127–132, 1997.

[14] M. Srinivasan, C. Basdogan, and C.-H. Ho. Haptic interac-tions in virtual worlds: Progress and prospects. In Proc. Int.Conf. on Smart Materials, Structures, and Systems, 1999.

[15] C. R. Wagner and R. D. Howe. Mechanisms of performanceenhancement with force feedback. In Proc. IEEE WorldHaptics Conf., Mar. 2005.

[16] P. Wellman and R. D. Howe. Towards Realistic Display inVirtual Environments. In Proc. ASME Dynamic Systems andControl Division, volume 57, pages 713–718, 1995.

[17] G. Westling and R. S. Johansson. Responses in glabrous skinmechanoreceptors during precision grip in humans. Experi-mental Brain Research, 66(1):128–140, 1987.