the brain from retina to extrastriate cortex. neural processing responsible for vision...

The Brain

from retina to extrastriate cortex

Neural processing responsible for vision

• photoreceptors• retina

– bipolar and horizontal cells– ganglion cells (optic nerve)

• optic nerves• optic chiasma (X)• lateral geniculate body• striate cortex• extrastriate cortex

Photoreceptors

Ganglion cells

Light

Lateral inhibition

• Edge detection and contrast enhancement

• Bipolar, Horizontal and Ganglion cells

1000 0

100

Lateral inhibition

• If no activity in neighboring photoreceptors,

output = output of photoreceptor

100100 100

0

Lateral inhibition

• if activity in neighboring photoreceptors,– output is decreased, possibly absent

100100 100

0

(-.5) (-.5)

-50 -50

+

(1.0)

100

200200 200

0

(-.5) (-.5)

-100 -100

+

(1.0)

200

Lateral inhibition via addition and negative weights

cornea

crystallinelens

retina: photoreceptors = rods + cones

opticnerve

Optic nerve

• axons of the ganglion cells– 1 million optic nerves– 120 million photoreceptors

From light to vision

Lateral Geniculate Nucleus (LGN)

StriateCortexGeniculo-Striate Pathway

(LGN)

StriateCortex

Striate cortex(primary visual centre)

• Neurons are edge detectorsfires when an edge of a particular orientation is present

(LGN)

StriateCortex

Striate cortex(primary visual centre)

• Neurons are edge detectorsfires when an edge of a particular orientation is present

frequent output

vertical bar

(LGN)

StriateCortex

Striate cortex(primary visual centre)

• Neurons are edge detectorsfires when an edge of a particular orientation is present

infrequent output

diagonal bar

Edge detection

• each cell “tuned” to particular orientation– vertical– horizontal– diagonal

• cats: only horizontal and vertical• humans: 10 degree steps• edges at particular orientations and positions

Extrastriate cortex(beyond the striate cortex)

V1

Extrastriate cortex

• Each area handles separate aspect of visual analysis– “V1-V2 complex”: Map for edges– V3: Map for form and local movement– V4: Map for colour– V5: Map for global motion

• Each is a visual module– connects to other areas– operates largely independently

Douglas A. Lyon, Ph.D.Chair, Computer Engineering Dept. Fairfield

University, CT, USA

Lyon@DocJava.com, http://www.DocJava.com

Copyright 2002 © DocJava, Inc.

Background

• It is easy to find a bad edge!• We know a good edge when we see it!

The Problem

• Given an expert + an image.

• The expert places markers on a good edge.

• Find a way to connect the markers.

Motivation

• Experts find/define good edges

• Knowledge is hard to encode.

Approach

• Mark an important edge

• Pixels=graph nodes

• Search in pixel space using a heuristic

• A* is faster than DP

Assumptions

• User is a domain expert

• Knowledge rep=heuristics+markers

Applications

• Photo interpretation

• Path planning (source+destination)

• Medical imaging

Photo Interpretation

Echocardiogram

•3D-multi-scale analysis

Path Plans, the idea

Path Planning-pre proc.•hist+thresh

•Dil+Skel

Path Planning - Search

The Idea

• The mind selects from filter banks

• The mind tunes the filters

Gabor filter w/threshold

• The Strong edge is the wrong edge!

Canny Edge Detector

Mehrotra and Zhang

Sub bands for 3x3 Robinson

Sub Bands 7x7 Gabor

Gabor-biologically motivated

Log filters=prefilter+laplacian

2 1

2 2 ex 2 y 2

2 2

1

4 1 x2 y2

2 2

ex 2 y 2

2 2

2 f (x, y) 2 f

x2 2 f

y2

Mexican Hat (LoG Kernel)

The Log kernel

Oriented Filters are steerable

Changing Scale at 0 Degrees

Changing Phase at 0 degrees

Summary

• Heuristics+markers= knowledge• Low-level image processing still needed• Global optimization is not appropriate for

all applications• Sometimes we only want a single, good

edge