dr. gallimore10/18/20151 cognitive issues in vr chapter 13 wickens & baker

Dr. Gallimore 04/20/23 1

Cognitive Issues in VRCognitive Issues in VRChapter 13Chapter 13

Wickens & BakerWickens & Baker


Cognitive IssuesCognitive Issues

Perceiving Perceiving UnderstandingUnderstanding Short termShort term

– Where am I , what do I see, where do I go, how?Where am I , what do I see, where do I go, how?

Long termLong term– What can I learnWhat can I learn


5 Features of VE5 Features of VE

3-D vs 2-D viewing3-D vs 2-D viewing– 3D more realistic and similar to how we view the real 3D more realistic and similar to how we view the real

world.world.

Dynamic vs static displayDynamic vs static display– E.G.. Movie may provide better understanding than E.G.. Movie may provide better understanding than

static pictures of same information.static pictures of same information.

Closed-loop vs open-loop interactionClosed-loop vs open-loop interaction– Closed-loop person has control over the aspects of the Closed-loop person has control over the aspects of the

world. Active vs passiveworld. Active vs passive



Ego-referenced vs world referenceEgo-referenced vs world reference– Ego vs exocentric view points.Ego vs exocentric view points.

– Frame of reference.Frame of reference.

– Ego reference considered immersed in environment.Ego reference considered immersed in environment.

Multimodel interactionMultimodel interaction– Our interactions and learning in the real world is based Our interactions and learning in the real world is based

on information input into multiple channels.on information input into multiple channels.» Vision, auditory, touch, kinesthetic feedback, gesture, etc...Vision, auditory, touch, kinesthetic feedback, gesture, etc...



Five elements are not independentFive elements are not independent We don’t know what must be includedWe don’t know what must be included Research focuses on subsets of VE features Research focuses on subsets of VE features

because of difficulty of incorporating everythingbecause of difficulty of incorporating everything What you need will depend on the purpose of the What you need will depend on the purpose of the

environmentenvironment


Uses of VEUses of VE

According to Wickens & Baker, there are four According to Wickens & Baker, there are four categories of tasks that VE systems are designed categories of tasks that VE systems are designed to serveto serve

1. On-line performance - manipulation in a remote 1. On-line performance - manipulation in a remote environment. Issues include:environment. Issues include:– Closed loop performance Closed loop performance

– Situation awarenessSituation awareness

– Workload or cognitive effortWorkload or cognitive effort



2. Off-line training and rehearsal - practice in 2. Off-line training and rehearsal - practice in forgiving environment. Issues include:forgiving environment. Issues include:– Transfer of training Transfer of training – How realistic must it be?How realistic must it be?

3. On line comprehension - obtaining 3. On line comprehension - obtaining understanding, comprehending, insight of understanding, comprehending, insight of information. Issues include:information. Issues include:– How should the information be modeled?How should the information be modeled?– Will modeling choices affect understanding incorrectly?Will modeling choices affect understanding incorrectly?



Off-line learning and knowledge acquisition- Off-line learning and knowledge acquisition- transfer knowledge to be employed later is transfer knowledge to be employed later is different form (e.g. Education)different form (e.g. Education)– Does it improve understanding more than other Does it improve understanding more than other

methods.methods.

– How can deeper knowledge be taught in this How can deeper knowledge be taught in this environment?environment?


Tasks in Ve’sTasks in Ve’s

Human factors principles should be applied in the Human factors principles should be applied in the development of VE. Including analysis of user development of VE. Including analysis of user needs and tasks.needs and tasks.

Wickens & baker discuss 6 tasks that may be Wickens & baker discuss 6 tasks that may be performed in a VE. performed in a VE. SearchSearch navigationnavigation

Visual-motor couplingVisual-motor coupling manipulationmanipulation

Perception and inspectionPerception and inspection learninglearning


Tasks in Ve’s: SearchTasks in Ve’s: Search

Search - looking for information in a 3D world, Search - looking for information in a 3D world, database, image, etc.database, image, etc.– MapsMaps

– Different frames of referenceDifferent frames of reference

– Aids (cues) to help with searchAids (cues) to help with search


Tasks in Ve’s: NavigationTasks in Ve’s: Navigation

NavigationNavigation– Where am I, where do I go?Where am I, where do I go?

– Point to point vs smooth navigationPoint to point vs smooth navigation

– Travel metaphor (flight, walking, logarithmic)Travel metaphor (flight, walking, logarithmic)

– Situation awarenessSituation awareness

– Control devices and ability to navigateControl devices and ability to navigate» Stability of perceptual motor loop Stability of perceptual motor loop

– Perceptual biasesPerceptual biases


Perceptual BiasPerceptual Bias

Humans use cues to distinguish locations and Humans use cues to distinguish locations and orientations of self and objects in their environments orientations of self and objects in their environments such as:such as:– Regularly spaced texture, size, binocular cues, gradients, Regularly spaced texture, size, binocular cues, gradients,

slant, optic flow, etc.. (Depth perception cues). slant, optic flow, etc.. (Depth perception cues). – These cues need to be included in the environment, but These cues need to be included in the environment, but

designed cues may not provide accurate perception of object designed cues may not provide accurate perception of object location. location.

– Examples of two perceptual biases: Examples of two perceptual biases: » Virtual space effectVirtual space effect

» 2D-3D effect2D-3D effect


Perceptual Biases Cont.Perceptual Biases Cont.

Virtual space effect - bias in perceived location of Virtual space effect - bias in perceived location of objects because of display magnification ( wide objects because of display magnification ( wide field of view) or minification (narrow field of field of view) or minification (narrow field of view).view).

2D-3D effect - perceived rotation of vectors 2D-3D effect - perceived rotation of vectors towards a plane that is more closely parallel to the towards a plane that is more closely parallel to the viewing plane. (Under estimation of slant of a viewing plane. (Under estimation of slant of a surface, figure 13-1).surface, figure 13-1).


Perceptual Biases Cont.Perceptual Biases Cont.

Research needsResearch needs– What are the perceptual biases?What are the perceptual biases?

– What designs or aids can we include to help users?What designs or aids can we include to help users?

– What tasks are affected?What tasks are affected?


Tasks in VE:Tasks in VE:Visual Motor CouplingVisual Motor Coupling

The VE system can introduce artifacts that interrupt The VE system can introduce artifacts that interrupt visual-motor coupling (e.G..Reaching, turning the visual-motor coupling (e.G..Reaching, turning the head and seeing update of visual scene). Five issues head and seeing update of visual scene). Five issues are:are:– Gain Gain

– Time delayTime delay

– OrderOrder

– Travel-view decoupling: direction of motion decoupled Travel-view decoupling: direction of motion decoupled from direction of gazefrom direction of gaze

– Field-of-viewField-of-view


Visual Motor Coupling: GainVisual Motor Coupling: Gain

Gain setting on control input. Gain setting on control input. – Low gain, need to move the input a great deal before Low gain, need to move the input a great deal before

change occurs.change occurs.

– High gain, small input, large change or displacement. High gain, small input, large change or displacement. (Can lead to instability of control).(Can lead to instability of control).

– Appropriate gain settings will depend on input device Appropriate gain settings will depend on input device and task. and task.

» Example space ball rotation vs translationExample space ball rotation vs translation


Visual Motor Coupling: Time DelayVisual Motor Coupling: Time Delay

Time delays or lags in updating the VE.Time delays or lags in updating the VE.– Flight simulation shows that 50 ms can impact Flight simulation shows that 50 ms can impact

performance.performance.

– May also effect spatial sensory reflexes (VOR).May also effect spatial sensory reflexes (VOR).

– To reduce time delays consider:To reduce time delays consider:» processing required due to complexity of imagesprocessing required due to complexity of images

» adaptive or progressive imagery (simplified views)adaptive or progressive imagery (simplified views)

» predictive filtering of user inputpredictive filtering of user input

» research is still neededresearch is still needed


Visual Motor Coupling: Control OrderVisual Motor Coupling: Control Order

System state - Task DependentSystem state - Task Dependent– zero order - position controlzero order - position control

– First-order - constant rate of changeFirst-order - constant rate of change» use when navigation for movement to locationsuse when navigation for movement to locations


Visual Motor Coupling:Travel View DecouplingVisual Motor Coupling:Travel View Decoupling

Direction of movement of travel, decoupled from Direction of movement of travel, decoupled from direction of gaze.direction of gaze.– Difficult to accomplish, especially with system lagsDifficult to accomplish, especially with system lags

– May cause disorientationMay cause disorientation

– Needs researchNeeds research


Visual Motor Coupling: Field of ViewVisual Motor Coupling: Field of View

Field of view Vs viewing angleField of view Vs viewing angle– Field of view - how much of the field we can see at Field of view - how much of the field we can see at

once (Instantaneous , without moving head).once (Instantaneous , without moving head).

– Viewing angle is width subtended by a display.Viewing angle is width subtended by a display.

– Field of regard - with limited field of view can rotate Field of regard - with limited field of view can rotate our head and get larger field of regard.our head and get larger field of regard.

– Real world: viewing angle = field of viewReal world: viewing angle = field of view

– VE can increase field of view as viewing angle is held VE can increase field of view as viewing angle is held constant (wide field of view, minification) or decrease constant (wide field of view, minification) or decrease field of view (magnification)field of view (magnification)


Visual Motor Coupling: Field of ViewVisual Motor Coupling: Field of View

If use wider field of view:If use wider field of view:– see more information in the scene at once, may help see more information in the scene at once, may help

with situation awarenesswith situation awareness

– distort perceived location of objectsdistort perceived location of objects

– produced compelling sense of forward motionproduced compelling sense of forward motion

– may result in greater mismatch between motion and may result in greater mismatch between motion and vestibular inputsvestibular inputs


Tasks in VE: ManipulationTasks in VE: Manipulation

Issues for manipulating objects are similar to those Issues for manipulating objects are similar to those for Navigation and Visual Motor Couplingfor Navigation and Visual Motor Coupling– gain, time lag, control order, depiction of location and gain, time lag, control order, depiction of location and

orientation, frame of referenceorientation, frame of reference

– also consider feedback (force or auditory)also consider feedback (force or auditory)


Tasks in VE : Tasks in VE : Perception and InspectionPerception and Inspection

Features need for this task may be different than Features need for this task may be different than those needed for Navigation. those needed for Navigation. – What is needed is task dependentWhat is needed is task dependent

» May not need large amount of visual realism May not need large amount of visual realism


Tasks in VE:LearningTasks in VE:Learning Procedural trainingProcedural training

– how much realism do we need? how much realism do we need? – Active participation more important than realismActive participation more important than realism

Perceptual motor skill learningPerceptual motor skill learning– active controlactive control– sensory feedbacksensory feedback

Spatial learning and navigational rehearsalSpatial learning and navigational rehearsal– ability to build mental mapsability to build mental maps

Conceptual learning (understanding phenomenon)Conceptual learning (understanding phenomenon)– active explorationactive exploration– alternative knowledge representationsalternative knowledge representations


Effort and WorkloadEffort and Workload

The VE could increase rather than workload. The VE could increase rather than workload. Need to evaluate the design. If interacting in the Need to evaluate the design. If interacting in the world becomes the primary task that diminishes world becomes the primary task that diminishes the purpose, it may not be worth it.the purpose, it may not be worth it.

Forcing more effort to understanding relationships Forcing more effort to understanding relationships can at times be beneficial to learning.can at times be beneficial to learning.

dr. gallimore10/18/20151 cognitive issues in vr chapter 13 wickens & baker

Documents