Cosc 6326/Psych6750X

Movement, Tracking

Moving about virtual and real environments

• Simulating motion through the environment– visual (vection)– vestibular stimulation– auditory surround motion/vibration– tactile surround motion/vibration– cues associate with active and passive limb

motion (proprioception and efference)

Motion Perception

• Human beings are highly sensitive to retinal motion– hyperacuity

– relative motion more salient than absolute

– need to account for eye movements and distance to obtain linear from angular speed

• autokinetic effect

• induced motion

– motion can define form

• apparent motion– sense of motion from sequential presentation of

discrete images– 15 frames per second normally considered

marginally acceptable– want for 30 to 60+ frames per second, ideally at

refresh rate

• patterns of motion give information about depth and 3D motion

Depth from motion

Figure ground segregation

• Motion can define form

• Boundaries of foreground image defined by accretion deletion regions

Motion in depth• Various cues: looming, changing disparity

• Time to contact/passage can be judged independent of speed

• Direction of motion in depth is also important

Motion Parallax

• translation of observer w.r.t. to scene (or of scene w.r.t. observer)– angular speed of objects across optic array

varies with distance

Motion Parallax

• if eye fixates– objects nearer than fixation point move with

observer, further objects move against observer– retinal speed varies with distance from point of

fixation

• motion parallax provides information about spatial layout of the environment and relative motion relative to the environment

• many similarities to depth from stereopsis

Self motion

Optic flow

• Flow patterns in velocity field can be analysed

• could provide direction of heading (e.g. from focus of expansion)

• perception of self-motion/motion in depth (vection)

• layout and depth of environment

• traditionally optic flow has been considered to be essential for control of locomotion

• note:– optic flow refers to velocity patterns in ‘optic arrays’

not the retina

– need to account for eye movements to extract optic flow from retinal flow

– real flow fields are complex

• Optic flow is a key component in– perception of self-motion– postural reflexes/body sway– eye movements– cybersickness

Visual direction based locomotion

• Recently the idea that locomotion relies directly on the processing of optic flow has been challenged

• Alternative is locomotion is based upon egocentric direction (see seminar)

Vection

• Normally world is stable and stationary

• A sense of self-motion can be elicited by visual motion (optic flow)

• Circular vection – illusory rotation from optic flow

• Linear vection – illusory translation from optic flow

• vection has a latency– depends on stimulus– reported as 5-30 s– believed to be due to conflict with vestibular

system, may benefit from vestibular priming

• may be partial or saturated, can drop out

• vection is– strengthened by large FOV– increased with increased optical

velocity/spatio-temporal frequency– sensitive to foreground background

relationships (driven by background)

• integrated with vestibular and motor copy (efferent) signals centrally

• Vection– can provide compelling self motion cues– enhances presence and realism

• but – is linked to cyber sickness and other negative

aftereffects

• main issue is mismatch between visual and other self motion cues (to be discussed later).

• as well as vection, vision can provide important static cues to orientation (as we’ll see in the tumbling room demo)

http://www.medicine.mcgill.ca/physio/cullenlab/img/ear.gif

Vestibular Inputs

• organs of balance and equilibrium in inner ear

• two distinct sub-systems– semicircular canals: angular

acceleration

– otoliths:linear acceleration

• self-motion perception, eye movements, postural control, orientation wrt gravity

• canal response– inertial sensors detect angular acceleration (three degrees

of freedom)– high-pass frequency response

• insensitive to low-frequency signals

• codes head velocity over relevant frequency range

– vision and other senses contribute as well but vestibular system most reliable at high frequency

– drives very low latency compensatory eye movements (vestibulo-ocular reflex VOR)

• otolith response– sensitive to three-dimensional linear

acceleration– also sensitive to gravity (head tilt or pitch)– linear acceleration due to gravity can be

confused with linear acceleration due to motion• possible to trade tilt for acceleration

Other non-visual cues to motion

• proprioceptive and somato-sensory cues can contribute to effective perception of self motion, e.g.– wind on face or body– weight on limbs, soles of feet, seat etc. can indicate

direction of gravity/accelerations– active and passive limb motions associated with

motion, efference copy– visceral receptors

Motion Displays

Actual Motion Displays

mozu.mes.titech.ac.jp/research/ walk/PW/gif/Stewart.gif

NASA space shuttle simulator

Flogiston chair Spaceball

Other issues

• active motion?

• range of possible motion limited

• requirement to stay centered in range of motion

Washout filter• Vestibular system is most sensitive to changes in

velocity (accelerometer)– provides an onset cue for self motion

– extended motion simulated with vision and other cues

• some problems with a motion base– simulating extended motion with limited travel

(isomorphism)?

– need to stay near centre of range to prepare for future motion

• After onset cue, desirable to ‘home’ to a neutral position.

• Do this gradually and ideally imperceptibly – ‘washout’

• Washout filters are based on highpass filters to give washout and a fast transient response

• Wearable computers allow long range natural (physical) motion

• Many devices have been proposed to introduce somatosensory and proprioceptive cues– treadmills– bicycles– hang gliders (skis, motorcycles, …)– importance of ‘feel for vehicle controls (haptic

feedback)

Treadmills

• Simulating locomotion may require displays for:– forward motion

– turning

– slope

– terrain roughness, solidity …

– other postures (e.g. crawling)

• http://intron.kz.tsukuba.ac.jp/vrlab_web/gaitmaster/gaitmaster_2.mpg

movie

Tracking

• critical component for immersive displays

• supports – structure from motion perception– maneuvering– locomotion– active perception– situational awareness

…

• Field of Regard– amount of space (in terms of visual angle) filled

by the virtual world– an effective increased effective FOV beyond

the instantaneous FOV that is provided by tracking head motion

• Tracker issues– resolution, smoothness, degrees of freedom– display lag is a major problem in head-slaved

displays (more in sensory conflict lecture):• sensor transduction, data transmission, input handling,

simulation, rendering, wait for frame refresh• when head moves or vibrates display is not updated

immediately• prediction can help

• Consequences:– fatigue, nausea (simulator sickness), poor

performance– poor fidelity and distortions of virtual space – image slip, blur or instability

• See Foxlin handout to be discussed

Navigation

• Two components to navigating– way finding– travel

Wayfinding

• Refers to determination of current location and path to desired location

• Need to maneuver through environment and avoid obstacles when wayfinding (or wandering)

• need to provide cognitive information to support location determination and wayfinding behaviour (navigational awareness)

• people believed to form cognitive maps or models of the environment

• In VR we need to be able to navigate about the VE.

• A typical goal of wearable computers is to provide navigational awareness.

• Aids to wayfinding– landmarks (natural or artificial)– path following/leaving trails– maps– memorable place names– compass/instruments– exocentric views– coordinate display, grid structure– constrained travel

Travel

• Current VR systems allow small tracked work spaces

• To move to other places in the virtual environment (or in the real world) need to support travel

• Physical travel fits real world– Locomotion on foot is natural form of travel– Vehicular travel is also familiar

• direct or indirect control and locomotion

• Travel through computer based worlds using mice and joystick is familiar to many (computer games etc)

• Travel through time is also useful

Travel Methods and Metaphors

• Real locomotion– modest distances, limited by tracking

capabilities– some tricks possible (seminar later)– typically some real locomotion supported in

addition to other techniques

• Ride along a preplanned route

• ‘Tow rope’ or river metaphor

• Fly mode– most common, lots of freedom– walk through is fly mode constrained to follow

the terrain– direction and speed control– pointer (gaze, torso, gesture, wand/joystick, …)– can drive vehicles using virtual or physical

controls, often with increased control over speed, acceleration …

• moving /scaling the world– often used in visualization

• ‘put me there’ method– jump to a defined location– ask to go to named location– define location in a exocentric, map or world-in-

miniature display– ‘stepping’ into a picture or aperture into a different

world

• all methods except real locomotion dissociate real from virtual travel

• Problems with this dissociation– conflict between real and physical motion– unnatural, may interfere with wayfinding– not suitable for AR or wearable applications

• Real motion is most natural but not always desirable in virtual environments (for example travelling large distances or through outer space etc)

• Interest in extending the range of real physical motion – Hi-ball and other long range trackers– vision based AR trackers & wearable systems– GPS, landmark based– York Trike project