hebrew university image processing - 2006 why mosaic · 2006-12-18 · 2 7 hebrew university image...

1

1

Hebrew University Image Processing - 2006

Targil 9 :

Panoramas - Stitching and BlendingMin-Cut Stitching

Many slides from Alexei Efros2


Why Mosaic ?Are you getting the whole picture?

• Compact Camera FOV = 50 x 35°

3



• Compact Camera FOV = 50 x 35°• Human FOV = 200 x 135°

4



• Compact Camera FOV = 50 x 35°• Human FOV = 200 x 135°

• Panoramic Mosaic = 360 x 180°

5


Stages in building panoramas

• Find alignment between overlapping images• Choose motion transformation between images

(translation, translation + rotation, affine, homography)

• Choose compositing surface for warping• Assign pixels in the panorama to source images• Seamlessly blend images

6


Stitching the images together

Why is this a challenge ?• Exposure differences• Scene illumination• Blurring (miss-registration)• Ghosting (moving objects)

Goal – invisible seams between images • Minimal amount of seams artifacts : edges that did

not appear in the original images

2

7


ApproachesAssuming that the images have already been aligned

• Simple seam location + smooth the transition between the images (blend)• Feathering (Alpha blending)• Pyramid blending• Gradient domain blending

– Less suitable when there is miss-alignment or moving objects

• Search for optimal seam• Dynamic programming• Graph cuts

– Less suitable for thin strips– Less suitable when global differences (intensity) are presence

8


Stitching The Panorama - Simple seam location

t t+1 t+2


Cut & Paste using Center Strips (Voronoi)

1 23

Strip taken from “2”

Stitching The Panorama - Simple seam location

Voronoi diarams

10


Image Blending

11


Feathering

01

01

+

=Encoding transparency

I(x,y) = (αR, αG, αB, α)

Iblend= Ileft + Iright

12


Affect of Window Size

0

1 left

right0

1

3

13


Affect of Window Size

0

1

0

1

14


Good Window Size

0

1

“Optimal” Window: smooth but not ghosted

15


Band-pass filtering

Laplacian Pyramid (subband images)Created from Gaussian pyramid by subtraction

Gaussian Pyramid (low-pass images)

16


Laplacian Pyramid

Enables toBlend low frequenciesOver a large spatial rangeAnd high frequenciesover a short range

17


Pyramid Blending

0

1

0

1

0

1

Left pyramid Right pyramidblend18


Gradient-domain Image Stitching• mosaic should be as similar as possible to the input

images, both geometrically and photometrically.

• The seam between the stitched images should be invisible.

How this can be achieved ?

• Compute the derivatives of the input images (Ix1, Iy1, Ix2 Iy2)

• Stitch the derivative images to form a field F = (Fx; Fy).• Find the mosaic image I’ whose gradients are closest to F.

4

19


Gradient-domain Image Stitching(Levin , Zomet, Peleg, Weiss, 2004)

20


Poison Image Editing (Perez , Gangnet. Blake 2003)

21


What about moving objects ?

22


Don’t blend, CUT! (Search for optimal seam)

So far we only tried to blend between two images. What about finding an optimal seam?

Moving objects become ghosts

23


Where should the cut pass ?

24


Davis, 1998Segment the mosaic

• Single source image per segment• Avoid artifacts along boundries

– Dijkstra’s algorithm

5

25


Input texture

B1 B2

Random placement

of blocks

block

B1 B2

Neighboring blocks

constrained by overlap

B1 B2

Minimal error

boundary cut

Another application for this approach:Texture synthesis

(Efros & Freeman, 2001)

26


min. error boundary

Minimal error boundary

overlapping blocks vertical boundary

_ =

2

overlap error

27


28


29


Maximum flow problem

Max flow problem:• Each edge is a “pipe”• Find the largest flow F of

“water” that can be sent from the “source” to the “sink” along the pipes

• Edge weights give the pipe’s capacity

“source”

A graph with two terminals

S T

“sink”

a flow F

30


Minimum cut problem

Min cut problem:• Find the cheapest way to cut

the edges so that the “source” is completely separated from the “sink”

• Edge weights now represent cutting “costs”

a cut C

“source”


S T

“sink”

6

31


Max flow/Min cut theorem

Max Flow = Min Cut:• Maximum flow saturates the edges

along the minimum cut.• Ford and Fulkerson, 1962• Problem reduction!

Ford and Fulkerson gave first polynomial time algorithm for globally optimal solution

“source”


S T

“sink”

32


How does min-cut relates to our problem ?

Define the problem of finding the seam as a min-cut problem :

the selected path will run between pairs of pixels.

33


What are the nodes of the graphlocation of pixels p=(x,y).

What are the edges of the grapheach pixel (node) has 4 neighbors pixelThe weight of the edge (flow capacity) is the color difference between pairs of pixels that the edge connectsW(p1, p2,A,B) = ||A( p1)−B(p1)||+||A( p2)−B(p2)||• p1,p2 are two adjacent pixel • A(p1) and B(p1) be the pixel colors at the location p1 in image A

and B, respectively

What are the Source and Target of the graphPixels we want to define explicitly from which image to take

What is the meaning of the resulting cutThe cut location will determin where do we want to have the seam between the images

How does min-cut relates to our problem ?

34


Img B

Edge WeightsIf the edge connects pixels from Image B that has same

(or very close) colors of image A in both sides of the edges:

• Weight is very small -> we want the cut this edge

If the edge connects pixels from Image B that has different colors from image A in any side of the cut

• Weight is bigger (as the difference is bigger)->we don’t want the cut this edge

Img A

(x,y) (x+1,y)

Img A

s(x,y) t(x+1,y)

Img B

W(p1, p2,A,B) = ||A(p1)−B(p1)||+||A(p2)−B(p2)||

35


Applying Min cuts on images (simple example à la Boykov&Jolly, ICCV’01)

n-links

s

t a cuthard

constraint

hard

constraint

Minimum cost cut can be computed in polynomial time

(max-flow/min-cut algorithms)

the selected path will run between pairs of pixels.

Stitch from im1

Stitch from im2

36


Putting it all togetherCompositing images/mosaics

• Have a clever blending function– Feathering– Center-weighted– blend different frequencies differently– Gradient based blending

• Choose the right pixels from each image– Dynamic programming – optimal seams– Graph-cuts

Now, let’s put it all together:• Interactive Digital Photomontage, 2004 (video)

7

37


38


Stitching Example

39


Why do we need the min-cut ?

Simple Stitching min-cutSimple Stitching

min-cut

40


Mosaic Stitching Examples

hebrew university image processing - 2006 why mosaic · 2006-12-18 · 2 7 hebrew university image...

Documents