introduction to cognitive science lecture 2: vision in humans and machines 1 vision in humans and...
Post on 20-Dec-2015
217 views
TRANSCRIPT
![Page 1: Introduction to Cognitive Science Lecture 2: Vision in Humans and Machines 1 Vision in Humans and Machines September 10, 2009](https://reader030.vdocuments.mx/reader030/viewer/2022032611/56649d415503460f94a1ba85/html5/thumbnails/1.jpg)
Introduction to Cognitive Science Lecture 2: Vision in Humans
and Machines
1
Vision in Humans Vision in Humans and Machinesand Machines
September 10, 2009
![Page 2: Introduction to Cognitive Science Lecture 2: Vision in Humans and Machines 1 Vision in Humans and Machines September 10, 2009](https://reader030.vdocuments.mx/reader030/viewer/2022032611/56649d415503460f94a1ba85/html5/thumbnails/2.jpg)
Visible light is just a part of the Visible light is just a part of the electromagnetic spectrumelectromagnetic spectrum
September 10, 2009 Introduction to Cognitive Science Lecture 2: Vision in Humans and Machines
2
![Page 3: Introduction to Cognitive Science Lecture 2: Vision in Humans and Machines 1 Vision in Humans and Machines September 10, 2009](https://reader030.vdocuments.mx/reader030/viewer/2022032611/56649d415503460f94a1ba85/html5/thumbnails/3.jpg)
Cross Section of the Human EyeCross Section of the Human Eye
September 10, 2009 Introduction to Cognitive Science Lecture 2: Vision in Humans and Machines
3
![Page 4: Introduction to Cognitive Science Lecture 2: Vision in Humans and Machines 1 Vision in Humans and Machines September 10, 2009](https://reader030.vdocuments.mx/reader030/viewer/2022032611/56649d415503460f94a1ba85/html5/thumbnails/4.jpg)
Anatomy of the Visual SystemAnatomy of the Visual System The EyesThe Eyes
Cornea:Cornea: Transparent outer covering of the eye that Transparent outer covering of the eye that
admits lightadmits light
Pupil:Pupil: Adjustable opening in the iris that regulates Adjustable opening in the iris that regulates
the amount of light that enters the eyethe amount of light that enters the eye
Iris:Iris: Pigmented ring of muscles situated behind Pigmented ring of muscles situated behind
the corneathe cornea
September 10, 2009 Introduction to Cognitive Science Lecture 2: Vision in Humans and Machines
4
![Page 5: Introduction to Cognitive Science Lecture 2: Vision in Humans and Machines 1 Vision in Humans and Machines September 10, 2009](https://reader030.vdocuments.mx/reader030/viewer/2022032611/56649d415503460f94a1ba85/html5/thumbnails/5.jpg)
Anatomy of the Visual SystemAnatomy of the Visual System PhotoreceptorsPhotoreceptors
Retina:Retina: The neural tissue and photoreceptive cellsThe neural tissue and photoreceptive cellslocated on the inner surface of the posteriorlocated on the inner surface of the posteriorportion of the eye.portion of the eye.
Rod:Rod: Photoreceptor cells of the retina, sensitive to Photoreceptor cells of the retina, sensitive to
light of low intensity.light of low intensity.
Cone:Cone: Photoreceptor cells of the retina; maximally Photoreceptor cells of the retina; maximally
sensitive to one of three different wavelengths sensitive to one of three different wavelengths of light and hence encodes color vision.of light and hence encodes color vision.
![Page 6: Introduction to Cognitive Science Lecture 2: Vision in Humans and Machines 1 Vision in Humans and Machines September 10, 2009](https://reader030.vdocuments.mx/reader030/viewer/2022032611/56649d415503460f94a1ba85/html5/thumbnails/6.jpg)
6
![Page 7: Introduction to Cognitive Science Lecture 2: Vision in Humans and Machines 1 Vision in Humans and Machines September 10, 2009](https://reader030.vdocuments.mx/reader030/viewer/2022032611/56649d415503460f94a1ba85/html5/thumbnails/7.jpg)
7
Anatomy of the Visual SystemAnatomy of the Visual System The EyesThe Eyes
Lens:Lens:Consists of a series of transparent, onion-like Consists of a series of transparent, onion-like
layers. Its shape can be changed by layers. Its shape can be changed by contraction of ciliary muscles.contraction of ciliary muscles.
Accommodation:Accommodation: Changes in the thickness of the lens, Changes in the thickness of the lens,
accomplished by the ciliary muscles, that accomplished by the ciliary muscles, that focus images of near or distant objects on the focus images of near or distant objects on the retinaretina
![Page 8: Introduction to Cognitive Science Lecture 2: Vision in Humans and Machines 1 Vision in Humans and Machines September 10, 2009](https://reader030.vdocuments.mx/reader030/viewer/2022032611/56649d415503460f94a1ba85/html5/thumbnails/8.jpg)
8
![Page 9: Introduction to Cognitive Science Lecture 2: Vision in Humans and Machines 1 Vision in Humans and Machines September 10, 2009](https://reader030.vdocuments.mx/reader030/viewer/2022032611/56649d415503460f94a1ba85/html5/thumbnails/9.jpg)
9
Anatomy of the Visual SystemAnatomy of the Visual System The EyesThe Eyes
Fovea:Fovea: Area of retina that mediates the most acute Area of retina that mediates the most acute
vision. Contains only color-sensitive cones.vision. Contains only color-sensitive cones.
Optic Disk:Optic Disk: Location on retina where fibers of ganglion Location on retina where fibers of ganglion
cells exit the eye. Responsible for the blind cells exit the eye. Responsible for the blind spot.spot.
![Page 10: Introduction to Cognitive Science Lecture 2: Vision in Humans and Machines 1 Vision in Humans and Machines September 10, 2009](https://reader030.vdocuments.mx/reader030/viewer/2022032611/56649d415503460f94a1ba85/html5/thumbnails/10.jpg)
10
Coding of Visual Information in the RetinaCoding of Visual Information in the Retina
Coding of Light and DarkCoding of Light and Dark
Receptive field:Receptive field: That portion of the visual field in which the That portion of the visual field in which the
presentation of visual stimuli will produce an presentation of visual stimuli will produce an alteration in the firing rate of a particular alteration in the firing rate of a particular neuron.neuron.
![Page 11: Introduction to Cognitive Science Lecture 2: Vision in Humans and Machines 1 Vision in Humans and Machines September 10, 2009](https://reader030.vdocuments.mx/reader030/viewer/2022032611/56649d415503460f94a1ba85/html5/thumbnails/11.jpg)
11
PhotoreceptorPhotoreceptor
BipolarBipolar
GanglionGanglion
![Page 12: Introduction to Cognitive Science Lecture 2: Vision in Humans and Machines 1 Vision in Humans and Machines September 10, 2009](https://reader030.vdocuments.mx/reader030/viewer/2022032611/56649d415503460f94a1ba85/html5/thumbnails/12.jpg)
12
Major cell types of the retinaMajor cell types of the retina
![Page 13: Introduction to Cognitive Science Lecture 2: Vision in Humans and Machines 1 Vision in Humans and Machines September 10, 2009](https://reader030.vdocuments.mx/reader030/viewer/2022032611/56649d415503460f94a1ba85/html5/thumbnails/13.jpg)
13
Receptive fieldsReceptive fields
![Page 14: Introduction to Cognitive Science Lecture 2: Vision in Humans and Machines 1 Vision in Humans and Machines September 10, 2009](https://reader030.vdocuments.mx/reader030/viewer/2022032611/56649d415503460f94a1ba85/html5/thumbnails/14.jpg)
14
Color MixingColor Mixing
![Page 15: Introduction to Cognitive Science Lecture 2: Vision in Humans and Machines 1 Vision in Humans and Machines September 10, 2009](https://reader030.vdocuments.mx/reader030/viewer/2022032611/56649d415503460f94a1ba85/html5/thumbnails/15.jpg)
15
Coding of Visual Information in the RetinaCoding of Visual Information in the Retina Photoreceptors: Trichromatic CodingPhotoreceptors: Trichromatic Coding
Peak Peak wavelengthwavelength sensitivities of the three cones: sensitivities of the three cones:Blue cone:Blue cone: Short-Short- Blue-violet (420 nm) Blue-violet (420 nm) Green cone:Green cone: Medium-Medium- Green (530 nm)Green (530 nm)Red Cone:Red Cone: Long-Long- Yellow-green (560nm)Yellow-green (560nm)
![Page 16: Introduction to Cognitive Science Lecture 2: Vision in Humans and Machines 1 Vision in Humans and Machines September 10, 2009](https://reader030.vdocuments.mx/reader030/viewer/2022032611/56649d415503460f94a1ba85/html5/thumbnails/16.jpg)
16
![Page 17: Introduction to Cognitive Science Lecture 2: Vision in Humans and Machines 1 Vision in Humans and Machines September 10, 2009](https://reader030.vdocuments.mx/reader030/viewer/2022032611/56649d415503460f94a1ba85/html5/thumbnails/17.jpg)
17
Coding of Visual Information in the RetinaCoding of Visual Information in the Retina Retinal Ganglion Cells:Retinal Ganglion Cells:
Opponent-Process CodingOpponent-Process Coding
Negative afterimage:Negative afterimage: The image seen after a portion of the retina is exposed to an The image seen after a portion of the retina is exposed to an
intense visual stimulus; consists of colors complimentary to intense visual stimulus; consists of colors complimentary to those of the physical stimulus.those of the physical stimulus.
Complimentary colors:Complimentary colors: Colors that make white or gray when mixed together.Colors that make white or gray when mixed together.
![Page 18: Introduction to Cognitive Science Lecture 2: Vision in Humans and Machines 1 Vision in Humans and Machines September 10, 2009](https://reader030.vdocuments.mx/reader030/viewer/2022032611/56649d415503460f94a1ba85/html5/thumbnails/18.jpg)
18
![Page 19: Introduction to Cognitive Science Lecture 2: Vision in Humans and Machines 1 Vision in Humans and Machines September 10, 2009](https://reader030.vdocuments.mx/reader030/viewer/2022032611/56649d415503460f94a1ba85/html5/thumbnails/19.jpg)
19
Analysis of Visual InformationAnalysis of Visual Information
Anatomy of the Striate cortexAnatomy of the Striate cortex
David Hubel and Torsten WieselDavid Hubel and Torsten Wiesel
1960’s at Harvard University1960’s at Harvard University
Discovered that neurons in the visual cortex did Discovered that neurons in the visual cortex did not simply respond to light; they selectively not simply respond to light; they selectively responded to specific features of the visual world.responded to specific features of the visual world.
![Page 20: Introduction to Cognitive Science Lecture 2: Vision in Humans and Machines 1 Vision in Humans and Machines September 10, 2009](https://reader030.vdocuments.mx/reader030/viewer/2022032611/56649d415503460f94a1ba85/html5/thumbnails/20.jpg)
20
![Page 21: Introduction to Cognitive Science Lecture 2: Vision in Humans and Machines 1 Vision in Humans and Machines September 10, 2009](https://reader030.vdocuments.mx/reader030/viewer/2022032611/56649d415503460f94a1ba85/html5/thumbnails/21.jpg)
21
![Page 22: Introduction to Cognitive Science Lecture 2: Vision in Humans and Machines 1 Vision in Humans and Machines September 10, 2009](https://reader030.vdocuments.mx/reader030/viewer/2022032611/56649d415503460f94a1ba85/html5/thumbnails/22.jpg)
22
Stimuli in Stimuli in receptive receptive field of field of neuronneuron
![Page 23: Introduction to Cognitive Science Lecture 2: Vision in Humans and Machines 1 Vision in Humans and Machines September 10, 2009](https://reader030.vdocuments.mx/reader030/viewer/2022032611/56649d415503460f94a1ba85/html5/thumbnails/23.jpg)
23
Cat V1 (striate Cat V1 (striate cortex)cortex)
Orientation Orientation preference preference mapmap
Ocular Ocular dominance dominance mapmap
![Page 24: Introduction to Cognitive Science Lecture 2: Vision in Humans and Machines 1 Vision in Humans and Machines September 10, 2009](https://reader030.vdocuments.mx/reader030/viewer/2022032611/56649d415503460f94a1ba85/html5/thumbnails/24.jpg)
24
![Page 25: Introduction to Cognitive Science Lecture 2: Vision in Humans and Machines 1 Vision in Humans and Machines September 10, 2009](https://reader030.vdocuments.mx/reader030/viewer/2022032611/56649d415503460f94a1ba85/html5/thumbnails/25.jpg)
September 10, 2009 Introduction to Cognitive Science Lecture 2: Vision in Humans and Machines
25
“Data Flow Diagram”of Visual Areas inMacaque Brain
Blue:motion perception pathway
Green:object recognition pathway
![Page 26: Introduction to Cognitive Science Lecture 2: Vision in Humans and Machines 1 Vision in Humans and Machines September 10, 2009](https://reader030.vdocuments.mx/reader030/viewer/2022032611/56649d415503460f94a1ba85/html5/thumbnails/26.jpg)
September 10, 2009 Introduction to Cognitive Science Lecture 2: Vision in Humans and Machines
26
Computer VisionComputer VisionA typical computer vision applications are complex and consist of different levels of processing, from the low-level pixel-by-pixel analysis to the high-level creation of scene descriptions.
Generally, computer vision systems consist of an image processing stage, followed by a scene analysis stage.
The following slide outlines the structure of a computer vision system.
![Page 27: Introduction to Cognitive Science Lecture 2: Vision in Humans and Machines 1 Vision in Humans and Machines September 10, 2009](https://reader030.vdocuments.mx/reader030/viewer/2022032611/56649d415503460f94a1ba85/html5/thumbnails/27.jpg)
September 10, 2009 Introduction to Cognitive Science Lecture 2: Vision in Humans and Machines
27
Computer VisionComputer Vision
A simple two-stage model of computer vision:
Image processing
Sceneanalysis
Bitmap image
Scene description
feedback (tuning)
Prepare image for scene analysis
Build an iconic model of the world
![Page 28: Introduction to Cognitive Science Lecture 2: Vision in Humans and Machines 1 Vision in Humans and Machines September 10, 2009](https://reader030.vdocuments.mx/reader030/viewer/2022032611/56649d415503460f94a1ba85/html5/thumbnails/28.jpg)
September 10, 2009 Introduction to Cognitive Science Lecture 2: Vision in Humans and Machines
28
Computer VisionComputer VisionThe image processing stage prepares the input image for the subsequent scene analysis.
Usually, image processing results in one or more new images that contain specific information on relevant features of the input image.
The information in the output images is arranged in the same way as in the input image. For example, in the upper left corner in the output images we find information about the upper left corner in the input image.
![Page 29: Introduction to Cognitive Science Lecture 2: Vision in Humans and Machines 1 Vision in Humans and Machines September 10, 2009](https://reader030.vdocuments.mx/reader030/viewer/2022032611/56649d415503460f94a1ba85/html5/thumbnails/29.jpg)
September 10, 2009 Introduction to Cognitive Science Lecture 2: Vision in Humans and Machines
29
Computer VisionComputer VisionThe scene analysis stage interprets the results from the image processing stage.
Its output completely depends on the problem that the computer vision system is supposed to solve.
For example, it could be the number of bacteria in a microscopic image, or the identity of a person whose retinal scan was input to the system.
![Page 30: Introduction to Cognitive Science Lecture 2: Vision in Humans and Machines 1 Vision in Humans and Machines September 10, 2009](https://reader030.vdocuments.mx/reader030/viewer/2022032611/56649d415503460f94a1ba85/html5/thumbnails/30.jpg)
September 10, 2009 Introduction to Cognitive Science Lecture 2: Vision in Humans and Machines
30
Digitizing Visual ScenesDigitizing Visual ScenesWith regard to spatial resolution, we will map the intensity in our image onto a two-dimensional finite array:
[0, 0] [0, 1] [0, 2] [0, 3]
[1, 0] [1, 1] [1, 2] [1, 3]
[2, 0] [2, 1] [2, 2] [2, 3]
y’
x’
![Page 31: Introduction to Cognitive Science Lecture 2: Vision in Humans and Machines 1 Vision in Humans and Machines September 10, 2009](https://reader030.vdocuments.mx/reader030/viewer/2022032611/56649d415503460f94a1ba85/html5/thumbnails/31.jpg)
September 10, 2009 Introduction to Cognitive Science Lecture 2: Vision in Humans and Machines
31
ThresholdingThresholding
Here, the right image is created from the left image by Here, the right image is created from the left image by thresholding, assuming that object pixels are darker thresholding, assuming that object pixels are darker than background pixels.than background pixels.
As you can see, the result is slightly imperfect (dark As you can see, the result is slightly imperfect (dark background pixels).background pixels).
![Page 32: Introduction to Cognitive Science Lecture 2: Vision in Humans and Machines 1 Vision in Humans and Machines September 10, 2009](https://reader030.vdocuments.mx/reader030/viewer/2022032611/56649d415503460f94a1ba85/html5/thumbnails/32.jpg)
September 4, 2007 Computer VisionLecture 1: Digital Images/Binary Image Processing
32
Geometric PropertiesGeometric Properties
![Page 33: Introduction to Cognitive Science Lecture 2: Vision in Humans and Machines 1 Vision in Humans and Machines September 10, 2009](https://reader030.vdocuments.mx/reader030/viewer/2022032611/56649d415503460f94a1ba85/html5/thumbnails/33.jpg)
September 10, 2009 Introduction to Cognitive Science Lecture 2: Vision in Humans and Machines
33
Geometric PropertiesGeometric PropertiesWe could teach our program what the objects look We could teach our program what the objects look like at different sizes and orientations, and let the like at different sizes and orientations, and let the program search all possible positions in the input.program search all possible positions in the input.
However, that would be a very inefficient and However, that would be a very inefficient and inflexible approach.inflexible approach.
Instead, it is much simpler and more efficient to Instead, it is much simpler and more efficient to standardizestandardize the input before performing object the input before performing object recognition.recognition.
We can We can scalescale the input object to a given size, the input object to a given size, centercenter it in the image, and it in the image, and rotaterotate it towards a specific it towards a specific orientation.orientation.
![Page 34: Introduction to Cognitive Science Lecture 2: Vision in Humans and Machines 1 Vision in Humans and Machines September 10, 2009](https://reader030.vdocuments.mx/reader030/viewer/2022032611/56649d415503460f94a1ba85/html5/thumbnails/34.jpg)
September 10, 2009 Introduction to Cognitive Science Lecture 2: Vision in Humans and Machines
34
Noise ReductionNoise Reduction
Here, a size filter perfectly removes all noise in the Here, a size filter perfectly removes all noise in the input image.input image.
![Page 35: Introduction to Cognitive Science Lecture 2: Vision in Humans and Machines 1 Vision in Humans and Machines September 10, 2009](https://reader030.vdocuments.mx/reader030/viewer/2022032611/56649d415503460f94a1ba85/html5/thumbnails/35.jpg)
September 10, 2009 Introduction to Cognitive Science Lecture 2: Vision in Humans and Machines
35
Noise ReductionNoise Reduction
However, if our threshold is too high, “accidents” may However, if our threshold is too high, “accidents” may happen.happen.
![Page 36: Introduction to Cognitive Science Lecture 2: Vision in Humans and Machines 1 Vision in Humans and Machines September 10, 2009](https://reader030.vdocuments.mx/reader030/viewer/2022032611/56649d415503460f94a1ba85/html5/thumbnails/36.jpg)
September 10, 2009 Introduction to Cognitive Science Lecture 2: Vision in Humans and Machines
36
Edge DetectionEdge Detection
Calculating the magnitude of the brightness gradient Calculating the magnitude of the brightness gradient with a Sobel filter. Left: original image; right: filtered with a Sobel filter. Left: original image; right: filtered image.image.
![Page 37: Introduction to Cognitive Science Lecture 2: Vision in Humans and Machines 1 Vision in Humans and Machines September 10, 2009](https://reader030.vdocuments.mx/reader030/viewer/2022032611/56649d415503460f94a1ba85/html5/thumbnails/37.jpg)
September 10, 2009 Introduction to Cognitive Science Lecture 2: Vision in Humans and Machines
37
TextureTexture
![Page 38: Introduction to Cognitive Science Lecture 2: Vision in Humans and Machines 1 Vision in Humans and Machines September 10, 2009](https://reader030.vdocuments.mx/reader030/viewer/2022032611/56649d415503460f94a1ba85/html5/thumbnails/38.jpg)
September 10, 2009 Introduction to Cognitive Science Lecture 2: Vision in Humans and Machines
38
TextureTextureTexture Texture is an important cue for biological vision is an important cue for biological vision systems to estimate the boundaries of objects.systems to estimate the boundaries of objects.
Also, Also, texture gradienttexture gradient is used to estimate the is used to estimate the orientation of surfaces.orientation of surfaces.
For example, on a perfect lawn the grass texture is For example, on a perfect lawn the grass texture is the same everywhere.the same everywhere.
However, the further away we look, the finer this However, the further away we look, the finer this texture becomes – this change is called texture texture becomes – this change is called texture gradient.gradient.
For the same reasons, texture is also a useful feature For the same reasons, texture is also a useful feature for for computer vision systemscomputer vision systems..
![Page 39: Introduction to Cognitive Science Lecture 2: Vision in Humans and Machines 1 Vision in Humans and Machines September 10, 2009](https://reader030.vdocuments.mx/reader030/viewer/2022032611/56649d415503460f94a1ba85/html5/thumbnails/39.jpg)
September 10, 2009 Introduction to Cognitive Science Lecture 2: Vision in Humans and Machines
39
Texture GradientTexture Gradient
![Page 40: Introduction to Cognitive Science Lecture 2: Vision in Humans and Machines 1 Vision in Humans and Machines September 10, 2009](https://reader030.vdocuments.mx/reader030/viewer/2022032611/56649d415503460f94a1ba85/html5/thumbnails/40.jpg)
September 10, 2009 Introduction to Cognitive Science Lecture 2: Vision in Humans and Machines
40
TextureTextureThe most fundamental question is: How can we The most fundamental question is: How can we “measure”“measure” texture, i.e., how can we quantitatively texture, i.e., how can we quantitatively distinguish between different textures?distinguish between different textures?
Of course it is not enough to look at the intensity of Of course it is not enough to look at the intensity of individual individual pixels.pixels.
Since the repetitive local arrangement of intensity Since the repetitive local arrangement of intensity determines the texture, we have to analyze determines the texture, we have to analyze neighborhoods neighborhoods of pixels to measure texture of pixels to measure texture properties.properties.
![Page 41: Introduction to Cognitive Science Lecture 2: Vision in Humans and Machines 1 Vision in Humans and Machines September 10, 2009](https://reader030.vdocuments.mx/reader030/viewer/2022032611/56649d415503460f94a1ba85/html5/thumbnails/41.jpg)
September 10, 2009 Introduction to Cognitive Science Lecture 2: Vision in Humans and Machines
41
Stereo VisionStereo Vision
Geometry of binocular stereo visionGeometry of binocular stereo vision
![Page 42: Introduction to Cognitive Science Lecture 2: Vision in Humans and Machines 1 Vision in Humans and Machines September 10, 2009](https://reader030.vdocuments.mx/reader030/viewer/2022032611/56649d415503460f94a1ba85/html5/thumbnails/42.jpg)
September 10, 2009 Introduction to Cognitive Science Lecture 2: Vision in Humans and Machines
42
Statistical Pattern RecognitionStatistical Pattern Recognition
![Page 43: Introduction to Cognitive Science Lecture 2: Vision in Humans and Machines 1 Vision in Humans and Machines September 10, 2009](https://reader030.vdocuments.mx/reader030/viewer/2022032611/56649d415503460f94a1ba85/html5/thumbnails/43.jpg)
September 10, 2009 Introduction to Cognitive Science Lecture 2: Vision in Humans and Machines
43
Object RecognitionObject RecognitionThis algorithm learns to recognize 25 different chairs:
It is shown each chair from 25 different viewing angles.
![Page 44: Introduction to Cognitive Science Lecture 2: Vision in Humans and Machines 1 Vision in Humans and Machines September 10, 2009](https://reader030.vdocuments.mx/reader030/viewer/2022032611/56649d415503460f94a1ba85/html5/thumbnails/44.jpg)
September 10, 2009 Introduction to Cognitive Science Lecture 2: Vision in Humans and Machines
44
The AlgorithmThe Algorithm