1

Computational Vision

CSCI 363, Fall 2012Lecture 18Stereopsis III

2

Summary of Marr-Poggio

Horizontal lines: constant Right eye position.

Vertical lines: Constant left eye position.

Diagonal lines: Constant disparity.

3

Limitations of the Marr-Poggio Algorithm

Problems with Marr-Poggio algorithm:• Does not specify what features are being matched.• Does not make use of zero crossings or multiple spatial scales.• Does not make use of vergence eye movements.

Importance of Vergence Eye Movements:•Humans can only fuse images with disparities of about +/- 10 min of arc. This range is called Panum's fusional area.•If we need to fuse the images of an object that is nearer or farther than this, we make vergence eye movements to change the plane of fixation.•Humans make many vergence eye movements as we scan the environment.

4

Marr-Poggio-GrimsonBasic idea:

Use zero crossings after filtering with

Large leads to widely spread zero crossings that are easier to match but may give less accurate estimates of disparity.

Small leads to more finely tuned disparities.

Start with large filters, and find matches for the zero crossings.We can use the estimates of disparity with the large filters to guide vergence eye movements to align features for comparing finer disparities with the smaller filters.

€

∇2G

€

∇2G

€

∇2G

5

Zero crossings for different filter sizes

6

Range of disparities to searchSuppose we filter an image with a operator for which the width of the inner portion is w.

Given a zero crossing at some position in the filtered image,the probability of another zero crossing within +/- w/2 of the first one is 5%.

Therefore, if there is a zero crossing at a given position in the left image and a zero crossing in the right image within the range of +/- w/2 of the one in the left image, the probability that these two are a correct match is 95%.

€

∇2G

+w/2-w/2

L

R

d

7

Determining correctnessIf the correct disparity is in the range of +/- w/2, then we will find matches for nearly all the zero crossings in the left image (95%).

If the correct disparity is not in the range, then the probability of a match is 40%.

Therefore, if we are only finding matches for 40% of the zero crossings, we are not looking at the correct range of disparities.

We can correct this by making vergence eye movements.

8

Increasing the range of disparities examined

The actual algorithm uses a range of +/- w. This leads to more false matches.

For a range of +/- w, the likelihood of a false match is at most 50%. So 50% of the time our matches will be correct.

If we have a false match, we usually have one crossed disparity and one uncrossed disparity matches.

We can use neighboring points to disambiguate the false matches.

9

Disambiguating false matchesTo determine the correct matches, we match both from the left image to the right and from the right image to the left.

L1 L2

R1 R2

L2 can only match R2

R1 can only match L1

These determine the matches uniquely.

It can be shown that if more than 70% of the zero crossings have a match within +/- w, then the disparity range is correct.

If not, use a vergence eye movement and try again.

10

The full algorithmA. To determine initial correspondence:1. Find zero crossings using a operator with a central width, w.2. For each horizontal slice:

2.1 Find the nearest neighbors in the right image for each zero-crossing in the left image.2.2 Find the nearest neighbors in the left image for each zero-crossing in the right image.2.3 For each pair of zero-crossings that are closest neighbors of one another, find the distance, initial, of the right fragment from the left. If initial is less than the matching tolerance, m, consider the zero crossings matched with disparity initial.

€

∇2G

11

Finishing the algorithmB. To determine final correspondence:1. Find the zero-crossings using a operator of width w/2.2. For each horizontal slice:

2.1 For each zero crossing in the left image:2.1.1 Find the nearest zero crossing in the left image that matched when

the operator was width w.2.1.2 Offset the zero crossing by a distance of initial, the disparity of the

nearest matching zero crossing found at the lower resolution with operator width w. 2.2 Find the nearest neighbors in the right image for the zero crossing in the left image2.3 Find the nearest neighbors in the left image for the zero crossing in the right image.2.4 For each pair of zero crossings that are closest neighbors to one another, find the distance, new, that the right is separated from the left. If new is less than the matching tolerance, m/2, then consider the zero crossings matched with a total disparity of final = new + initial.

€

∇2G

12

Do Humans Use the Same Constraints as Marr-Poggio?

1. Similarity: We probably use this one.Humans cannot fuse a white dot with a black dot.

2. Epipolar: The brain probably uses some version of this.If one image is shifted upward (or downward), people cannot

fuse the two images.

3. Uniqueness: We probably don't rely on this.There are examples of images where can fuse two features in

one image with a single feature in the other.

13

Violations of Uniqueness constraint

Panum's limiting case:Matching one line with two

Braddick's demonstration:Matching one point with tworepeatedly

Stereo algorithms can deal with Braddick's demonstration with slight modifications.

14

The Continuity ConstraintThe brain probably uses some form of continuity constraint.

Evidence:There is a limit to how quickly disparity can change from one location to the next, and still produce stereo fusion.

For a plane that is steeply slanted in depth, people lose the ability to see the slant and just see the step edge.

15

Does the Brain use Zero Crossings?

•Many machine vision algorithms extract edges first (e.g. with zero crossings) and then compute the disparities of matched edges.

•They use edges for matching because they correspond to important physical features (e.g. object boundaries).

•We also know that people can localize the positions of edges very accurately. This accurate localization is required for stereo vision.

•However, it is not clear what primitives the human brain matches when computing stereo disparity.

•The information is combined in V1, so it is after the center-surround operators (like the laplacian operators). Zero crossing information would be available.

16

Zero crossings are not enoughStereo Pairs

Luminance Profile

Convolved images

Left Right

17

Perception vs. ComputationPerceived Depth Computed depth of zero

crossings

Positions of peaks and troughs

Computed depth using peaks and troughs

18

Some V1 cells are tuned to disparity

Tuned Excitatory Tuned Inhibitory

Some cells are narrowly tuned for disparity. Most prefer a disparity near zero.

19

Near and Far cells

Some cells are broadly tuned for disparity, preferring either near objects or far objects.