vr06 nonvisual displays - tu-freiberg.de · help us determine firmness, approximate shape, and...

Virtuelle Realität

1

Teil 6:Aural and Haptic Displays

Virtuelle RealitätWintersemester 2006/07

Prof. Bernhard Jung

Prof. B. Jung Virtuelle Realität, WS 2006/07

Overview

Aural DisplaysHaptic Displays

Further information:

Sherman & Craik, 2002, chapter 4

The Haptics Community Web Site: http://haptic.mech.northwestern.edu/

Virtuelle Realität

2


Aural Display Paradigms

stationary displays -- speakersdisplays that move with the head (head-coupled) -- headphones


Properties of Aural Display Paradigms

Aural presentation qualitiesNumber of Display Channels Sound Stage Localization Masking Amplification

Virtuelle Realität

3


Aural Qualities

number of display channelstwo sensory inputs (ears) different sounds to each ear provide cues about from where a sound is coming

can display same signal to both ears (monophonic) can display different signal to each ear (stereophonic) can display different signal to multiple speakers (e.g. quadrophonc, octaphonic, surround)

multiple speaker displays rely on ears to naturally localize sounds


Aural Qualities

Sound stagethe source from which a sound appears to emanate head-referenced vs. world-referenced

head-referenced sound stage moves with the head world-referenced sound stage remains fixed with the world world-referenced is how we naturally experience sound

in immersive environments, sound should be world-referenced

when headphones are used, head tracking is needed to reproduce a world-referenced sound stage

Virtuelle Realität

4


Aural Qualities

localization / spatializationlocalization is our brain's ability to determine the location from which a sound is emanating

we localize sounds using cues such as the time and volume differences of when a sound reaches each ear, and the differences in the actual sounds caused by the shape of our bodies, especially the outer ear

spatialization is our technology's ability to make a sound appear to come from particular points in space

spatialization is easier with headphones because we can directly control what is heard by each ear


Aural Qualities

Maskingloud sounds mask softer sounds physical objects can mask a sound (often without completely occluding it) speaker systems cannot effectively mask real-world sounds closed headphones are best for experiences where the participant is only supposed to hear sounds from the virtual world

Amplificationneed to boost the sounds to hearable levels must take care not to amplify sounds too much, especially with headphones

Virtuelle Realität

5


Properties of Aural Display Paradigms

Logistic qualitiesNoise Pollution User mobilityInterface with tracker methodsEnvironment requirementsAssociability with other sense displaysPortabilityThroughputEncumbranceSafetyCost


Logistic Qualities of Aural Displays

Noise pollution (unwanted sounds)goes both ways sounds speakers from speakers may be objectionable to others not involvedspeaker displays can be disturbed by surfaces that echo

User mobilityWired headphones limit mobility Speakers and wireless-headphones are okay If visual display requires wires to the head, wires of headphone can be incorporated

Interface with tracker methodsmagnets in the speakers/headphones headphone magnets smaller, but closer to the tracker receiver (problem for electromagnetic trackers) loud sounds from speakers may interfere with ultrasonic trackers

• the audio display system only cares about tracking information when adjusting output to user movement (ie. a world referenced sound stage)

Virtuelle Realität

6



Environment requirementssquare room (like a CAVE) can be a problem for speaker displays, e.g. echoes hard-surfaced square room even worse

Associability with other sense displaystypically, stationary with stationary & head-based with head-based

headphones easily incorporated with the visual head-based display for spatialized audio, headphones best choice even in stationary visual display

Portabilityheadphones obviously more portable than speakers

not only smaller, but don't require additional amplification



Throughputas with visual head-based displays, headphones require time to put on / take offheadphones require a separate pair for all immersed participantsspeakers work much better for larger groups

Encumbrancespeakers generally more comfortable for lengthy periods of time

Safetyhearing damage is possible through both speakers and headphones headphones closer to the ear though, so may be more of a concernwires to headphones are a potential tripping hazard hygiene

Costhigh-quality headphones more expensive than high-quality speakers but cost per listener may be less with speakers amplification system for speakers adds to the cost

Virtuelle Realität

7


Aural Displays - Summary

Benefits of stationary displays (speakers)Works well with stationary visual displays Does not require sound processing to create a world-referenced sound stage (one that remains stable to the virtual world) More user mobility (less cables) Less encumbrance (none in fact) Faster throughput

Benefits of Head-coupled displays (headphones)Works well with head-coupled visual displays Easier to implement spatialized (head-referenced) sound fields Easier to reject unwanted room noise More portable Private


Haptic Displays –Feel and Touch the Virtual Environment

Two forms of haptic perception: Kinesthetic (proprioceptic, force) feedback

e.g. joints angles, muscle length & tensionTactile feedback

nerve sensors at skin surface (pressure, temperature, …)

Virtuelle Realität

8



Haptic Displays

Primary forms of haptic displays used in VRTactile displays End-effector displays Robotically operated shape displays

Virtuelle Realität

9


Tactile Haptic displays

Sensed by the skin -- the largest single organ of the human body. Actuators mounted generally on the fingers and hand. Generally no need for world grounding. Bladder actuators Vibrator actuators Pin actuators Thermo-actuators

Teletact Glove (30 bladders) Cricket prop (vibrating) CyberTouch (vibro-tactile)


Tactile Haptic displays

SmartTouchelectro-tactile display

University of TokioUniversity of Karlsruhe

Virtuelle Realität

10


Tactile Haptic Displays

Painstation

- Pong arcade game

- PEU Pain Execution Unit - electro shocks- heat - lashes

painstation.de


End-Effector Displays

a mechanical device that provides a force to the participant's extremities generally linked to mechanical tracking sensors generally world grounded often operate with respect to a single point in the virtual world

Sensable Phantom Omni

Virtuelle Realität

11



2 Phantom force feedback devices



Rutgers Dexterous Master

Virtuelle Realität

12



Immersion CyberGrasp with force feedback via exoskeleton

http://easylink.playstream.com/immersion/CyberGrasp.rm

Immersion CyberForceCyberGrasp +

6DOF mechanical tracking



Immersion Haptic Workstation

http://easylink.playstream.com/immersion/HWS-3min-LQ.wvx

Virtuelle Realität

13


End-Effector Displays - Teleoperation of Robots

Sarcos Dextrous Arm

www.sarcos.com/teleop_videos.html

Utah-MIT Dextrous Hand



Sarcos Uniport specialized device for simulation of laparoscopic surgery

Virtuelle Realität

14


Robotically Operated Shape Displays

Only few research prototypesUse robots to place a (phyiscal) representation of the virtual world where the user is reaching. May be generic (corners and edges) May be specific (e.g. selection of switches) Usually uses a finger surrogate (stick-like object) for fast tracking, and safety.


Properties of Haptic Display Paradigms

Haptic presentation qualitieskinesthetic cues tactile cues grounding amount of contact number of display channels degrees of freedom form fidelity resolution latency crucialityframe rate size

Virtuelle Realität

15


Haptic presentation qualities

Kinesthetic cuesnerve inputs that sense angles of joints, muscle length, tension, and resistance to muscle effort within the body. help us determine firmness, approximate shape, and physical forces 75 joints in the body (44 in the hands), so very difficult to display them all to the participant

Tactile cuesthe sensory receptors at the skin mechanoreceptors -- shape and surface texture thermoreceptors -- heat transfer electroreceptors -- electric current flow nociceptors -- pain from tissue damage



Groundingforce & resistance displays require an anchor self-grounded

motion limited between two parts of the same thing self-grounded systems more portable but limited in the types of forces that can be displayedbody-mounted haptic displays, e.g. tactile CyberGlove, exoskeletons

world-groundedmotion limited by something attached to the world e.g. Phantom, steering wheels, joystick

Number of display channelshow many points of contact with the body e.g. phantom has one point where the user can influence the virtual world phantom in each hand -- two channels

Virtuelle Realität

16



Degrees of freedom6-DOF in unconstrained movement 1-DOF display: how far down a tube can you insert a laparoscope camera 2-DOF display: how far down a tube, plus twist 3-DOF display: location of the finger tip or stylus 6-DOF display: location and orientation of a finger tip or stylus



Formthe shape of what the participant interacts with generic: sphere, stick, plane specific object (prop): handgun, steering wheel amorphous: able to change shape to multiple specific representations

e.g. glove, pin display

Fidelitysafety may require low-fidelity in many circumstances how rapidly can the system change to the proper display (force, temperature, etc.) force displays can be rated by a maximum stiffness measurement

20 Newton/cm will generally be perceived as a solid, immovable wall. 40 Newton/cm is the maximum force that a human finger can exert 10 Newton/cm is the highest force used when doing fine manipulation

Virtuelle Realität

17



Spatial resolutiondiffers by body region finger tips can sense differences 2mm apart 30mm on the forearm 70mm on the back Therefore higher resolution required at the fingertip

Temporal resolutionlow frame rate in a force display causes the object to be perceived as mushy or shakey1000 Hz is a good minimum

sensory homunculus:mapping the human

somatosensory cortex



Latency toleranceAs with frame rate, a low latency display is crucial especially for force displays Even more important when two people are trying to manipulate a single object (e.g. try to pick up an object together)Because force displays are connected by armatures, low-latency mechanical tracking is easy to integrate

Sizelarger displays allow broader range of motion, but are generallyhigher-power devices (and therefore less safe). smaller displays work well for tasks such as surgery where the operators hands do not cover large spaces

Virtuelle Realität

18


Properties of Haptic Display Paradigms

Logistic qualitiesuser mobility interface with tracker methods environment requirements associability with other sense displays portability throughput encumbrance safety cost


Logistic Qualities of Haptic Displays

User mobilityWorld-grounded displays require the user to be near the device. For applications that don't require the user to cover much territory, this is not a problem – e.g. surgery (real or simulated). Self-grounded displays can move with the user, though cables might be a concern.

Interface with tracker methodsA responsive & accurate tracking system is required. Mechanical tracking often works for force displays because of the linkages already present to render the force.

Virtuelle Realität

19



Environment requirementsLarge force displays often require special rooms equipped to handle hydraulic or pneumatic pressure pumps. Smaller force or tactile displays can work on the desktop, at a kiosk, or be held in the hand.

Associability with other sense displaysBecause the user must come in physical contact with a haptic display to get the sensation, it is difficult to hide them. Occlusive head-based displays thus often are used in conjunction with haptic displays to overcome this. Another solution is to wear the force display (e.g. a force system worn on the shoulder & arm).



PortabilitySmall tactile and small force displays can likely be easily transported. Displays requiring a hydraulic/pneumatic system often require equipment that is hard to move -- even if the device itself is self-grounded. Of course, world-grounded system that are physically mounted to the ceiling, floor or wall tend to be hard to move.

ThroughputWearable devices such as tactile gloves and self-grounded force displays usually take time to put on / take off.Force displays that interact with a stylus or other finger surrogate that the user can simply grab or release allow users to switch rapidly.

EncumbranceSelf-grounded, exoskeleton-style devices are generally fairly encumbering. Small force displays and glove devices are less so. Most haptic displays still involve a fair amount of wires though, so even the smaller devices must contend with them.

Virtuelle Realität

20



SafetySafety is a significant concern with many haptic displays, especially large force displays

e.g. large robots, large exoskeletonsTemperature and other tactile displays may also be hazardous. Often displays that can produce forces or other potentially harmful results are equipped with a "drop-dead" switch.

CostMost haptics displays are costly. They are still not very widely used, and thus have not reached mass market prices. Exception: force feedback steering wheels, joysticks, and vibrators in game controllers.

• .


Benefits of Tactile displayshelps in the fine manipulation of virtual objects (some) can be added to some end-effector displays can be world or body grounded (usually body-grounded) body-grounded method is mobile generally less-expensive generally portable

Benefits of End-effector displayscan be world or body grounded (exoskeletal is body-grounded) exoskeletal method is mobile world-grounded method is not very encumbering fast and accurate tracking is usually built into display

Benefits of Robotically operated shape displayscan provide a very realistic haptic display world-grounded display fast and accurate tracking is usually built into display works primarily with head-based visual displays

Virtuelle Realität

21


Other Senses: Vestibular (Balance) Display


Other Senses

Food Simulatordisplays biting force, integrated

auditory and chemical sensations

University of Tsukuba, Japan

vr06 nonvisual displays - tu-freiberg.de · help us determine firmness, approximate shape, and...

Documents