visual attention€¦ · ”unexpected visitor” eye movement experimental ”the subject is asked...
TRANSCRIPT
![Page 1: Visual Attention€¦ · ”unexpected visitor” Eye Movement Experimental ”The subject is asked to ... ”unexpected visitor” ” to have a free examination....” ” to estimate](https://reader036.vdocuments.mx/reader036/viewer/2022071005/5fc216fa19a7127842001b9a/html5/thumbnails/1.jpg)
Visual Attention
![Page 2: Visual Attention€¦ · ”unexpected visitor” Eye Movement Experimental ”The subject is asked to ... ”unexpected visitor” ” to have a free examination....” ” to estimate](https://reader036.vdocuments.mx/reader036/viewer/2022071005/5fc216fa19a7127842001b9a/html5/thumbnails/2.jpg)
![Page 3: Visual Attention€¦ · ”unexpected visitor” Eye Movement Experimental ”The subject is asked to ... ”unexpected visitor” ” to have a free examination....” ” to estimate](https://reader036.vdocuments.mx/reader036/viewer/2022071005/5fc216fa19a7127842001b9a/html5/thumbnails/3.jpg)
![Page 4: Visual Attention€¦ · ”unexpected visitor” Eye Movement Experimental ”The subject is asked to ... ”unexpected visitor” ” to have a free examination....” ” to estimate](https://reader036.vdocuments.mx/reader036/viewer/2022071005/5fc216fa19a7127842001b9a/html5/thumbnails/4.jpg)
Attention selects one modality over another
Cell phone use while driving
![Page 5: Visual Attention€¦ · ”unexpected visitor” Eye Movement Experimental ”The subject is asked to ... ”unexpected visitor” ” to have a free examination....” ” to estimate](https://reader036.vdocuments.mx/reader036/viewer/2022071005/5fc216fa19a7127842001b9a/html5/thumbnails/5.jpg)
Where to look?
• Many visual processes are expensive
• Humans don’t process the whole visual field
• How do we decide what to process?
• How can we use insights about this to make machine vision more efficient?
![Page 6: Visual Attention€¦ · ”unexpected visitor” Eye Movement Experimental ”The subject is asked to ... ”unexpected visitor” ” to have a free examination....” ” to estimate](https://reader036.vdocuments.mx/reader036/viewer/2022071005/5fc216fa19a7127842001b9a/html5/thumbnails/6.jpg)
Selective Attention• Perception and awareness are necessarily selective
(mobile phone while driving)• Perception is not (usually) passive, autonomous,
feedfoward and stimulus-driven--it is active, involves feedback, and goal-driven
• Behavioral goals determine states of perceptual selectivity, which in turn can modulate sensory input via cortical feedback
• Attention is the set of cognitive mechanisms that implement the selection of relevant perceptual input and the rejection of irrelevant input
![Page 7: Visual Attention€¦ · ”unexpected visitor” Eye Movement Experimental ”The subject is asked to ... ”unexpected visitor” ” to have a free examination....” ” to estimate](https://reader036.vdocuments.mx/reader036/viewer/2022071005/5fc216fa19a7127842001b9a/html5/thumbnails/7.jpg)
1. At a computational level, the brain has a limited capacity to process information (limited, for example, in storage capacity and in the number of computational units). The goal of perception—to accurately recognize or categorize an almost infinite variety of inputs very rapidly—requires suitable constraints to be tractable. Attention limitswhich inputs the system attempts to recognize or categorize.
Why can’t we see and experience everything at once? That is, why is selective attention necessary? Here are two candidate answers
2. Action—for example, locomotion, reaching movements, and eye movements—are also limited by biomechanical constraints to only one or a few at a time, in an appropriate sequence (cf. Karl Lashley, “The Problem of Serial Order in Behavior”), and because there is a need for perception-action correspondence, the scope of perception is limited in order to guide desired actions.
![Page 8: Visual Attention€¦ · ”unexpected visitor” Eye Movement Experimental ”The subject is asked to ... ”unexpected visitor” ” to have a free examination....” ” to estimate](https://reader036.vdocuments.mx/reader036/viewer/2022071005/5fc216fa19a7127842001b9a/html5/thumbnails/8.jpg)
Three Modes of Attentional Control
• Stimulus-driven, bottom-up, involuntary, automatic
• Goal-driven, top-down, voluntary, deliberate
• Hybrid: a combination of top-down and bottom-up control.
![Page 9: Visual Attention€¦ · ”unexpected visitor” Eye Movement Experimental ”The subject is asked to ... ”unexpected visitor” ” to have a free examination....” ” to estimate](https://reader036.vdocuments.mx/reader036/viewer/2022071005/5fc216fa19a7127842001b9a/html5/thumbnails/9.jpg)
![Page 10: Visual Attention€¦ · ”unexpected visitor” Eye Movement Experimental ”The subject is asked to ... ”unexpected visitor” ” to have a free examination....” ” to estimate](https://reader036.vdocuments.mx/reader036/viewer/2022071005/5fc216fa19a7127842001b9a/html5/thumbnails/10.jpg)
![Page 11: Visual Attention€¦ · ”unexpected visitor” Eye Movement Experimental ”The subject is asked to ... ”unexpected visitor” ” to have a free examination....” ” to estimate](https://reader036.vdocuments.mx/reader036/viewer/2022071005/5fc216fa19a7127842001b9a/html5/thumbnails/11.jpg)
Four Domains ofPerceptual Selectivity
• Space: the “spotlight of attention”
• Features: color, motion, orientation, etc.
• Objects: overlap, occlusion, transparency, grouping, segmentation, etc.
• Sensory Modalities:-- Vision– Audition– Touch
![Page 12: Visual Attention€¦ · ”unexpected visitor” Eye Movement Experimental ”The subject is asked to ... ”unexpected visitor” ” to have a free examination....” ” to estimate](https://reader036.vdocuments.mx/reader036/viewer/2022071005/5fc216fa19a7127842001b9a/html5/thumbnails/12.jpg)
Guided SearchWolfe (1994)
![Page 13: Visual Attention€¦ · ”unexpected visitor” Eye Movement Experimental ”The subject is asked to ... ”unexpected visitor” ” to have a free examination....” ” to estimate](https://reader036.vdocuments.mx/reader036/viewer/2022071005/5fc216fa19a7127842001b9a/html5/thumbnails/13.jpg)
Guided Search
• Observed that some conjuntionsearches can be efficient.
• Suggested that these searches are not random, but can be guided by – salience as defined by local feature
contrast– top-down goals as specified by the current
search set
![Page 14: Visual Attention€¦ · ”unexpected visitor” Eye Movement Experimental ”The subject is asked to ... ”unexpected visitor” ” to have a free examination....” ” to estimate](https://reader036.vdocuments.mx/reader036/viewer/2022071005/5fc216fa19a7127842001b9a/html5/thumbnails/14.jpg)
Visual salience
• Salience ~ visual prominence
• Must be cheap to calculate
• Related to features that we collect from very early stages of visual processing
• Colour, orientation, intensity change and motion are all important indicators of salience
![Page 15: Visual Attention€¦ · ”unexpected visitor” Eye Movement Experimental ”The subject is asked to ... ”unexpected visitor” ” to have a free examination....” ” to estimate](https://reader036.vdocuments.mx/reader036/viewer/2022071005/5fc216fa19a7127842001b9a/html5/thumbnails/15.jpg)
������ ������ ��� ���� ��������� ��������
![Page 16: Visual Attention€¦ · ”unexpected visitor” Eye Movement Experimental ”The subject is asked to ... ”unexpected visitor” ” to have a free examination....” ” to estimate](https://reader036.vdocuments.mx/reader036/viewer/2022071005/5fc216fa19a7127842001b9a/html5/thumbnails/16.jpg)
Alfred L. Yarbus was a Russian psychologist (known there as AlfredLukjanovic Jarbus) who studied eye movements in the 1950s and 1960s.
Eye Movement Experimental
”unexpected visitor”
![Page 17: Visual Attention€¦ · ”unexpected visitor” Eye Movement Experimental ”The subject is asked to ... ”unexpected visitor” ” to have a free examination....” ” to estimate](https://reader036.vdocuments.mx/reader036/viewer/2022071005/5fc216fa19a7127842001b9a/html5/thumbnails/17.jpg)
Eye Movement Experimental
”The subject is asked to ...”unexpected visitor”
![Page 18: Visual Attention€¦ · ”unexpected visitor” Eye Movement Experimental ”The subject is asked to ... ”unexpected visitor” ” to have a free examination....” ” to estimate](https://reader036.vdocuments.mx/reader036/viewer/2022071005/5fc216fa19a7127842001b9a/html5/thumbnails/18.jpg)
” to have a free examination....”
![Page 19: Visual Attention€¦ · ”unexpected visitor” Eye Movement Experimental ”The subject is asked to ... ”unexpected visitor” ” to have a free examination....” ” to estimate](https://reader036.vdocuments.mx/reader036/viewer/2022071005/5fc216fa19a7127842001b9a/html5/thumbnails/19.jpg)
” to estimate the wealth of the family... ”
![Page 20: Visual Attention€¦ · ”unexpected visitor” Eye Movement Experimental ”The subject is asked to ... ”unexpected visitor” ” to have a free examination....” ” to estimate](https://reader036.vdocuments.mx/reader036/viewer/2022071005/5fc216fa19a7127842001b9a/html5/thumbnails/20.jpg)
” to guess what the family had been doing beforethe arrival of the 'unexpected visitor’.....”
![Page 21: Visual Attention€¦ · ”unexpected visitor” Eye Movement Experimental ”The subject is asked to ... ”unexpected visitor” ” to have a free examination....” ” to estimate](https://reader036.vdocuments.mx/reader036/viewer/2022071005/5fc216fa19a7127842001b9a/html5/thumbnails/21.jpg)
” to memorize the location of the people and objects in the painting...
![Page 22: Visual Attention€¦ · ”unexpected visitor” Eye Movement Experimental ”The subject is asked to ... ”unexpected visitor” ” to have a free examination....” ” to estimate](https://reader036.vdocuments.mx/reader036/viewer/2022071005/5fc216fa19a7127842001b9a/html5/thumbnails/22.jpg)
” to memorize the clothes worn by the position of the objects and people in the room.....
![Page 23: Visual Attention€¦ · ”unexpected visitor” Eye Movement Experimental ”The subject is asked to ... ”unexpected visitor” ” to have a free examination....” ” to estimate](https://reader036.vdocuments.mx/reader036/viewer/2022071005/5fc216fa19a7127842001b9a/html5/thumbnails/23.jpg)
” to give the ages of the people ...”
![Page 24: Visual Attention€¦ · ”unexpected visitor” Eye Movement Experimental ”The subject is asked to ... ”unexpected visitor” ” to have a free examination....” ” to estimate](https://reader036.vdocuments.mx/reader036/viewer/2022071005/5fc216fa19a7127842001b9a/html5/thumbnails/24.jpg)
” to estimate how long the 'unexpected visitor" had been away."
![Page 25: Visual Attention€¦ · ”unexpected visitor” Eye Movement Experimental ”The subject is asked to ... ”unexpected visitor” ” to have a free examination....” ” to estimate](https://reader036.vdocuments.mx/reader036/viewer/2022071005/5fc216fa19a7127842001b9a/html5/thumbnails/25.jpg)
The General Vision Problem (oversimplified)
Images
Features/Saliency
Context
Attention/Selection
Recognition/InferenceCan all vision problems be described by this diagram ?
![Page 26: Visual Attention€¦ · ”unexpected visitor” Eye Movement Experimental ”The subject is asked to ... ”unexpected visitor” ” to have a free examination....” ” to estimate](https://reader036.vdocuments.mx/reader036/viewer/2022071005/5fc216fa19a7127842001b9a/html5/thumbnails/26.jpg)
![Page 27: Visual Attention€¦ · ”unexpected visitor” Eye Movement Experimental ”The subject is asked to ... ”unexpected visitor” ” to have a free examination....” ” to estimate](https://reader036.vdocuments.mx/reader036/viewer/2022071005/5fc216fa19a7127842001b9a/html5/thumbnails/27.jpg)
Web usability
![Page 28: Visual Attention€¦ · ”unexpected visitor” Eye Movement Experimental ”The subject is asked to ... ”unexpected visitor” ” to have a free examination....” ” to estimate](https://reader036.vdocuments.mx/reader036/viewer/2022071005/5fc216fa19a7127842001b9a/html5/thumbnails/28.jpg)
Direct AttentionMost higher animals in the world have an ability to sense danger by spotting anomalies in their environment and surviving by taking appropriate evasive action. Those organisms that have the benefit of vision are able to direct attention rapidly towards the unusual without any prior knowledge of the environment.
![Page 29: Visual Attention€¦ · ”unexpected visitor” Eye Movement Experimental ”The subject is asked to ... ”unexpected visitor” ” to have a free examination....” ” to estimate](https://reader036.vdocuments.mx/reader036/viewer/2022071005/5fc216fa19a7127842001b9a/html5/thumbnails/29.jpg)
People and animals are able to spot anomalies in a scene no part of which they have seen before and attention is drawn in general to the anomalous Object in a scene, not to the common-or-garden or the familiar. It seems unlikely that such novelty can be sharply identified by processes which are totally dependent upon comparisons with a growing dictionary of independent recognition codes.
![Page 30: Visual Attention€¦ · ”unexpected visitor” Eye Movement Experimental ”The subject is asked to ... ”unexpected visitor” ” to have a free examination....” ” to estimate](https://reader036.vdocuments.mx/reader036/viewer/2022071005/5fc216fa19a7127842001b9a/html5/thumbnails/30.jpg)
Why difference of Gaussians?
![Page 31: Visual Attention€¦ · ”unexpected visitor” Eye Movement Experimental ”The subject is asked to ... ”unexpected visitor” ” to have a free examination....” ” to estimate](https://reader036.vdocuments.mx/reader036/viewer/2022071005/5fc216fa19a7127842001b9a/html5/thumbnails/31.jpg)
A Visual Attention Model
Visual attention is driven by the salience. Salience is normally strongest in a region if small neighborhoods in that region are dissimilar to neighbourhoods elsewhere in the image.
![Page 32: Visual Attention€¦ · ”unexpected visitor” Eye Movement Experimental ”The subject is asked to ... ”unexpected visitor” ” to have a free examination....” ” to estimate](https://reader036.vdocuments.mx/reader036/viewer/2022071005/5fc216fa19a7127842001b9a/html5/thumbnails/32.jpg)
Algorithm
Neighbor at x mismatching at y
![Page 33: Visual Attention€¦ · ”unexpected visitor” Eye Movement Experimental ”The subject is asked to ... ”unexpected visitor” ” to have a free examination....” ” to estimate](https://reader036.vdocuments.mx/reader036/viewer/2022071005/5fc216fa19a7127842001b9a/html5/thumbnails/33.jpg)
Visual Attention
![Page 34: Visual Attention€¦ · ”unexpected visitor” Eye Movement Experimental ”The subject is asked to ... ”unexpected visitor” ” to have a free examination....” ” to estimate](https://reader036.vdocuments.mx/reader036/viewer/2022071005/5fc216fa19a7127842001b9a/html5/thumbnails/34.jpg)
Visual Attention
![Page 35: Visual Attention€¦ · ”unexpected visitor” Eye Movement Experimental ”The subject is asked to ... ”unexpected visitor” ” to have a free examination....” ” to estimate](https://reader036.vdocuments.mx/reader036/viewer/2022071005/5fc216fa19a7127842001b9a/html5/thumbnails/35.jpg)