haptic interface technologies
TRANSCRIPT
02/14/2010 EECE596: Copyright since 1999, Sidney Fels01/09/2019 CPEN541: Copyright since 1999, Sidney Fels
Haptic Interface Technologies
01/09/2019 CPEN541: Copyright since 1999, Sidney Fels
Haptics: Overview
• Haptics: Introduction – tactile – proprioception
• Tactile Interface Technologies • Force Feedback Technologies
01/09/2019 CPEN541: Copyright since 1999, Sidney Fels
Haptics: Introduction
• Sense of touch and kinesthesia – tactile sensing – proprioception – Somatosensory system
• Bidirectional – sense environment
• temperature, vibration, weight, etc. – manipulate environment
• push, pull, pinch, hit, rotate, etc.
01/09/2019 CPEN541: Copyright since 1999, Sidney Fels
Haptics: Tactile Sensing• Can sense:
– texture/vibration – temperature of object or environment – slip detection – surface compliance – elasticity – viscosity – electrical/thermal conductivity – vibration (other than for texture) – initial contact detection – gauging force required for manipulation
01/09/2019 CPEN541: Copyright since 1999, Sidney Fels
Haptics: Tactile I/F
• Critical for performance in many tasks – Massimino and Sheridan, 1993 – tracking (Patrick, Sheridan and Massimino, 1990) – target pointing (Akamatsu, 1994) – degraded visual condition (Massimino and Sheridan,
1993) • See Lederman’s work for many experiments
looking at attributes of tactile sensing • Considered to be critical for virtual environments
– needed for immersion and sense of reality
01/09/2019 CPEN541: Copyright since 1999, Sidney Fels
Haptics: Tactile Sensing• Four main sense organs
– Meissner’s Corpuscles • surface curvature, velocity, local shape, flutter, slip • poor spatial resoln’ • 43%
– Pacinian Corpuscles • vibration, slip, acceleration • 70-1000Hz response frequency range • 13%
– Markel’s Disks • skin curvature, local shape, pressure • 25%
– Ruffini Endings • skin stretch, local force • 19%
01/09/2019 CPEN541: Copyright since 1999, Sidney Fels
Haptics: Tactile Sensing• Skin properties:
– finger pad (Reynier and Hayward, 1993) • spatial resolution, 0.15mm; two-point discrimination 1-3mm • smooth glass feature, 2µm for single dot, 0.06µm for grating,
0.85µm for straight lines • line orientation detection, 8.7mm separation (sequential);
13.1mm separation (simultaneous) • 5.5msec separation of two 1msec stimulti
– increase stimuli duration -> reduce threshold
– Other results available for different properties • strain, texture properties, temperature, etc. • check Burdea and Coiffet (1994)
01/09/2019 CPEN541: Copyright since 1999, Sidney Fels
Haptics: Tactile Actuators
01/09/2019 CPEN541: Copyright since 1999, Sidney Fels
Haptics: Tactile Displays (commercial)• CyberTouch (Virtual Technolgies)
– little vibrators on fingers • 0-200Hz
– $14,000 US • TouchMaster (Exos)
– voice coils on fingers – 210-240Hz
• Tactool System (Xtensory) - sold as kits – pins driven by shape memory alloys
• impulse sensation (30g) • vibration (20Hz) • $1,500
• Displace Temp. Sensing System (CM Research) – thimble – thermoelectric heat pump (Peltier effect) – $10,000
01/09/2019 CPEN541: Copyright since 1999, Sidney Fels
Haptics: Peltier Effect
P - Bismuth N - Telluride
01/09/2019 CPEN541: Copyright since 1999, Sidney Fels
Haptics: CyberTouch
01/09/2019 CPEN541: Copyright since 1999, Sidney Fels
Haptics: Tactile Displays (commercial)
• Intelligent Systems Solutions – air bladders + pneumatics – not available anymore
• Braille Displays – Freedom of Speech (85-8 dot cells)
• Games – Playstation controller – fishing, driving, baseball
01/09/2019 CPEN541: Copyright since 1999, Sidney Fels
Haptics: Tactile Displays (research)
• Reading aids – Sherrick, 1984; Shimizu, 1986; Barfield and Furness,
1995 • tools for studying haptics
– Cholewiak and Sherrick, 1981; Schneider, 1988, Lederman, 1999
• tools for rehabilitation – Wise et al., 1990
• tools for teleoperation
01/09/2019 CPEN541: Copyright since 1999, Sidney Fels
Haptics: Tactile Displays (research)• Disney - TeslaTouch
– http://www.ivanpoupyrev.com/projects/teslatouch.php
– https://www.youtube.com/watch?v=zo1n5CyCKr0
01/09/2019 CPEN541: Copyright since 1999, Sidney Fels
Haptics: Tactile Displays (research)
• Armstrong Laboratory – 5x6 array tactile stimulator, SMA wires – solenoid attached to Phantom
• Harvard (Howe) – 4x6 array of pins driven by SMA actuators
• 2.1mm spacing, 3mm height, 62msec rise/fall time, BW 6-7Hz • studied palpation,
– shape display (single row) + force feedback display – 3 subjects, 60 trials, tumor finding task, 4mm cylinder inside
foam rubber • error <1mm in 50%; <3mm in 95%; no shape info > 13mm
01/09/2019 CPEN541: Copyright since 1999, Sidney Fels
Haptics: Tactile Displays (research)
– voice coils + force feedback device (teleoperation) • investigated 3 types of tasks
– detection of vibration is fundamental goal • worn ball bearing • vibration feedback helps discrimination
– vibrations indicate state of task • piercing plastic membrane • vibration feedback reduces force exerted and increases response
time – vibration not important to task
• peg-in-hole task • no effect, but subjective improvement
01/09/2019 CPEN541: Copyright since 1999, Sidney Fels
Haptics: Tactile Displays (research)
• Hokkaido (Ino) – displays for shearing, pressure and temperature – Shear display
• table mounted pneumatic driver – Temp. display
01/09/2019 CPEN541: Copyright since 1999, Sidney Fels
Haptics: Tactile Displays (research)
• MIT - Touch Lab (Srinivasan) – linear and planar graspers
• used for psychophysical experiments • use with Phantom
– libraries available for • compliance, viscosity, mass, shape, texture, friction, walls and
corners – created “force shading” for creating feel of smoothly
curved surfaces – what about contact sounds?
01/09/2019 CPEN541: Copyright since 1999, Sidney Fels
Haptics: Tactile Displays (research)
• Karlsruhe Research Centre (Germany) – three 24 needle print heads – needles vibrated at 600Hz to simulate contact pressure – actuated by sensors attached to forceps
• Many others: – Queens, (Lederman) - spinning disks – Sandia Labs, 2x3 array of electromagnetic actuators – TiNi Alloy Comp., 5x6 array of tactor pins – U of Salford, UK, vibration with piezo-electric actuator, Peltier
effect heat-pump for temp., pneumatic bladders for contact force, glove mounted
01/09/2019 CPEN541: Copyright since 1999, Sidney Fels
Haptics: Kinesthetic Interfaces
• Awareness of position and movement and forces on body parts
• Force feedback has been shown to be important – Teleoperation
• Hill and Salisbury, 1976; Hanaford, 1989 and Howe, 1992 – Molecular docking
• Ouh-Young, Beard and Brooks, 1989 – Grasping tasks
• Gomez, Burdea and Langrana, 1995 (reduced error and learning time by 50%)
01/09/2019 CPEN541: Copyright since 1999, Sidney Fels
Haptics: Kinesthetic Interfaces• Where is it useful?
– Virtual reality/Augmented reality – medicine
• surgery • diagnosis
– scientific visualization – data manipulation – interactive art – situations where auditory and visual feedback are
limited • aids to disabled • peripheral tasks • 3D manipulation
01/09/2019 CPEN541: Copyright since 1999, Sidney Fels
Haptics: Kinesthetic Interfaces
• Basic idea: 1. Measure movement and forces exerted by user (fingers,
hand, arm, body) 2. Calculate effect of forces on manipulated objects and
resulting forces on user • virtual or real objects
3. Present forces to the user’s fingers, wrist, arms etc. as appropriate
01/09/2019 CPEN541: Copyright since 1999, Sidney Fels
Haptics: Kinesthetic Interfaces
• Devices are either – earth grounded – off-the-body – exoskeletons
01/09/2019 CPEN541: Copyright since 1999, Sidney Fels
Haptics: Kinesthetic Technologies
01/09/2019 CPEN541: Copyright since 1999, Sidney Fels
Haptics: Kinesthetic Technologies
• Other technologies – magnetic levitation – tendons – ???
01/09/2019 CPEN541: Copyright since 1999, Sidney Fels
Haptics: Human Kinesthetic Properties• In addition to cells mentioned we have:
– Golgi endings in joint ligaments • joint torque
– Ruffini type endings in joint capsules • capsule stretch
– Golgi tendon organs • muscle tension
– muscle spindle organs • muscle stretch and rate of change • probably most important for kinesthetic sense
– nothing known for weight or effort • inside central nervous system
01/09/2019 CPEN541: Copyright since 1999, Sidney Fels
Haptics: Human Kinesthetic Properties• Somatosensory system is not symmetric
– force control and perceptual bandwidths differ (Brooks, 1990)
• human output: deliver forces at 5-10Hz • human input: perceived up to 1kHz
• JND for force sensing is about 7% • rigid body - 153 to 415 N/cm • force production: 16.5N distal finger to 192.3N
shoulder joint • output resoln’: high at finger tip, low at shoulder
01/09/2019 CPEN541: Copyright since 1999, Sidney Fels
Haptics: Human Grasp Properties• Some force outputs for different types of grasps
– Jacobus et al, 1992
01/09/2019 CPEN541: Copyright since 1999, Sidney Fels
Haptics: Human Grasp Properties
• Current technologies generally change way they work based on grasp types
• several ways to categorize grasps – Schlesinger (1919)
• cylindrical, finger-tip, hook, palmar, spherical, lateral – Napier (1956)
• distinguish between power and precision grasp – MacKenzie (1990s)
• prehensile vs non-prehensile
01/09/2019 CPEN541: Copyright since 1999, Sidney Fels
Haptics: Human Grasp Properties
Cutkosky and Howe, 1990
01/09/2019 CPEN541: Copyright since 1999, Sidney Fels
Haptics: Force-feedback Issues
• Should keep grasping/reflected forces to less than 15% of max (Wiker, 1989) – comfort and fatigue issues – I.e., index finger 7N, middle 6N, ring 4.5N
01/09/2019 CPEN541: Copyright since 1999, Sidney Fels
Haptics: Issues for Creating FF/B Devices
1. Sampling data from user – lag, update rates etc.
2. Computing forces – S/W libraries needed like graphics libraries
3. Presenting forces – calculating control parameters in real-time
• 1000Hz update • Impedance control (force) vs. Compliance control (position)
– overcome inertia
01/09/2019 CPEN541: Copyright since 1999, Sidney Fels
Haptics: Force-F/B Devices (comm)
• Phantom (SensAble Devices, Massey) – thimble on finger
• single point force feedback only – motors and cables and good engineering – can use more than one at a time (if you have $$$)
01/09/2019 CPEN541: Copyright since 1999, Sidney Fels
Haptics: Force-F/B Devices (comm)• Impulse Engine (Immersion Corp.)
• 5 dof – 2 dof pivot – insertion translation – rotation of tool – open-close motion of instrument
• servo-motor actuators • laproscopic and endoscopic simulation
01/09/2019 CPEN541: Copyright since 1999, Sidney Fels
Haptics: Force-F/B Devices (comm)
• Laproscopic Impulse Engine
01/09/2019 CPEN541: Copyright since 1999, Sidney Fels
Haptics: Force-F/B Devices (comm)
• CyberGrasp (Virtual Technologies) – motor actuated tendons attached to fingers
• 1 dof/finger • used to be flappers on end for contact force
– mounted on CyberGlove
01/09/2019 CPEN541: Copyright since 1999, Sidney Fels
Cyberforce• Exoskelton
– arm forces • Tendons
– grasp forces
https://www.youtube.com/watch?v=4aMCJDOEi0k
01/09/2019 CPEN541: Copyright since 1999, Sidney Fels
Haptics: Force-F/B Devices (comm)
• Exos – 4 dof Force Feedback Master
• surgical simulator • hand and arm
– Force Exoskeleton
01/09/2019 CPEN541: Copyright since 1999, Sidney Fels
Haptics: Force-F/B Devices (comm)
• SaFiRE (Exos) – exoskeleton for hand
• applies forces to thumb, index finger and wrist • 8 dof: 3 dof to thumb, 3 dof to index, 2 dof to wrist • links grounded to forearm supply 3D Cartesian forces to
fingertips and palm • DC motors used with wires and gears
01/09/2019 CPEN541: Copyright since 1999, Sidney Fels
Haptics: Force-F/B Devices (comm)• Hand Exoskeleton Haptic Display (Exos)
– based on SaFiRE hand exoskeleton • 1 dof to thumb, 2 dof to index • slip display on thumb and index • boom mounted; 2 dof position sensing, 1 dof vertical FF/B
01/09/2019 CPEN541: Copyright since 1999, Sidney Fels
Haptics: Force-F/B Devices (comm)• Per-Force Handcontroller and Finger Forcer
Option (Cybernet Sys. Corp) – 6 dof force feedback joystick – finger force feedback added to top of joystick
• thimbles attached to finger tips
01/09/2019 CPEN541: Copyright since 1999, Sidney Fels
Haptics: Force-F/B Devices (comm)
• Haptic Master (Iwata in Tsukuba) – 3D force and torque – three sets of pantograph links
01/09/2019 CPEN541: Copyright since 1999, Sidney Fels
• Interactor (Aura)
Haptics: Force-F/B Devices (comm)
01/09/2019 CPEN541: Copyright since 1999, Sidney Fels
Haptics: Force-F/B Devices (research)• Many issues remain to be solved for force
feedback devices – commercial FF/B displays only scratch the surface
• Extensive research on how to use these displays – not a lot of practical uses yet
• Most research effort to develop FF/B displays has come from teleoperation – applies to virtual environments – other effort from psychophysical testing labs – effort to explore application of FF/B displays done in
well funded labs
01/09/2019 CPEN541: Copyright since 1999, Sidney Fels
Haptics: Force-F/B Devices (research)• Ino, Hokkaido
– elbow joint force feedback – metal hydride actuator
• temperature changes in alloy controls pressure of hydrogen gas • pressure converted into propulsion • 300g, cylinder is 20.62 mm, 6g of hydride • lift 10kg to 50mm with vel. of 9mm/sec • noise free, no sudden impact force • similar compliance to elbow joint
– report good results (obviously) • probably not good for hand • probably work for knee and other large joints
01/09/2019 CPEN541: Copyright since 1999, Sidney Fels
Haptics: Force-F/B Devices (research)• MIT (Salisbury)
– continuing with Phantom • changing tip
– creating software • haptic rendering - calculating and generating forces • still point interaction
– contact forces – contact persistence – impedance – some curvature, texture and friction
• want general purpose s/w libraries to allow for haptic rendering
– Interval was working on some of this as well
01/09/2019 CPEN541: Copyright since 1999, Sidney Fels
Haptics: Force-F/B Devices (research)
• Institut des Systemes Intelligents et de Robotique, Université Pierre et Marie Curie (Hayward previously McGill) – 9 dof Stylus - tendon driven single point – Pantograph
• 2 dof translation - 10cm X 10cm • FF/B mouse • found acceleration best for shock and hard contacts • for effective performance need
– wide frequency response – high accuracy in presenting forces – mechanical impedance controllable over wide range – dynamic response up to 50-100Hz
01/09/2019 CPEN541: Copyright since 1999, Sidney Fels
Haptics: Force-F/B Devices (research)
Pantograph
Stylus
01/09/2019 CPEN541: Copyright since 1999, Sidney Fels
Haptics: Force-F/B Devices (research)• Northwestern University (Colgate)
– 4 dof Force Reflecting Manipulandum • translation + rotation in horizontal plane • high impedance control
– stiff springs and hard walls • no singularities in the work space
– study physics based simulation for haptic interfaces
01/09/2019 CPEN541: Copyright since 1999, Sidney Fels
Haptics: Force-F/B Devices (research)• Rutgers (Burdea)
– Rutgers Master • uses pneumatic microcylinders positioned in palm • ends attached to tips of 3 fingers and thumb
– Experiment 1: • manipulate ball without deforming it • compared
– visual f/b • (ball, bar graph and mono/stereo)
– audio f/b – force f/b
• Results – FF/B most significant non-redundant modality – FF/B + audio best redundant combo.
01/09/2019 CPEN541: Copyright since 1999, Sidney Fels
Haptics: Force-F/B Devices (research)
• U. of North Carolina (Brooks) – Argonne Remote Manipulator (ARM) – 6 dof large workspace device
• hand grip display, joint action at shoulder and outward – used mostly for molecular docking
01/09/2019 CPEN541: Copyright since 1999, Sidney Fels
Haptics: Force-F/B Devices (research)
• Others: – Suzuki Motor Corp: SPICE
• 6 dof, mechanical + motors
– Tokyo Inst. Of Tech • single point FF/B • strings and motors
01/09/2019 CPEN541: Copyright since 1999, Sidney Fels
Haptics: Force-F/B Devices (research)
• CCRMA (Gillespie) – piano/keyboard key feedback
• simulate touch of different keyboards • 7 keys, DC motor driven, dynamic simulation of key action
• UBC (Salcudean) – maglev joystick force feedback – motor controlled pantograph with additional dof
• UBC (Maclean) – 1 dof rotary devices
• Cobots (Peshkin and Colgate) – COBOTS
• use mechanical constraints to set up forces
01/09/2019 CPEN541: Copyright since 1999, Sidney Fels
Haptics: Cobots
01/09/2019 CPEN541: Copyright since 1999, Sidney Fels
Haptics: Summary• Need good haptic interfaces for:
– immersive VR – 3D direct manipulation i/f – effective teleoperation – mediated communication
• Very active area of research
01/09/2019 CPEN541: Copyright since 1999, Sidney Fels
Haptics: Summary• Current status of Tactile displays
– limited area (finger tip) – don’t scale up – large physical dimensions - not practical – no software models + psychophysical info available + devices capable of meeting resolution and B/W reqd. in
specific domains + >20-30Hz BW + pin displays are close to human resolution + vibration available at wide range of frequencies
– not high enough for skillful manipulation + temperature displays have sufficient resolution
01/09/2019 CPEN541: Copyright since 1999, Sidney Fels
Haptics: Summary• Force feedback displays
– generally application specific – generally small workspace – encumbering – most use servo motor actuation
• stability problems • backdrivability • friction
– Specs for good force feedback • force resolution: 12 bits, • position resolution: 0.001 in • passive friction less than 1% of max. force • minimum sampling rate: 2000Hz • latency <1msec
– Phantom is closest to meeting all of these for single point
01/09/2019 CPEN541: Copyright since 1999, Sidney Fels
Haptics: Summary– Still can’t represent solid, immovable wall or rigid
objects • instability at solid edge
– add viscosity to whole environment – reduce stiffness - everything is a sponge
• More problems – sensor accuracy – latency of computer – actuator performance – difference between actuator and sensor placement – mechanical transmission difficulties – non-linear system due to dynamics of user characteristics – safety – software models lacking – more psychophysical testing needed
01/09/2019 CPEN541: Copyright since 1999, Sidney Fels
Haptics: Summary
• Currently – no general purpose haptic interfaces – no practical applications in common use
• Future: – robotic graphics? – New actuators? – Non-linear, deformable manipulators