![Page 1: Generating Classic Mosaics with Graph Cuts Y. Liu, O. Veksler and O. Juan University of Western Ontario, Canada Ecole Centrale de Paris, France](https://reader036.vdocuments.mx/reader036/viewer/2022062511/5517883f55034645368b53a1/html5/thumbnails/1.jpg)
Generating Classic Mosaics with Graph Cuts
Y. Liu, O. Veksler and O. JuanUniversity of Western Ontario, Canada
Ecole Centrale de Paris, France
![Page 2: Generating Classic Mosaics with Graph Cuts Y. Liu, O. Veksler and O. Juan University of Western Ontario, Canada Ecole Centrale de Paris, France](https://reader036.vdocuments.mx/reader036/viewer/2022062511/5517883f55034645368b53a1/html5/thumbnails/2.jpg)
Different Types of Mosaics
A. Hausner [SIGGRAPH 01]
K. Smith et. al [SIGGRAPH 05]
J. Orchard et. al [NPAR 08]
Jesus at Basilica of Sant' Apollinare Nuovo
![Page 3: Generating Classic Mosaics with Graph Cuts Y. Liu, O. Veksler and O. Juan University of Western Ontario, Canada Ecole Centrale de Paris, France](https://reader036.vdocuments.mx/reader036/viewer/2022062511/5517883f55034645368b53a1/html5/thumbnails/3.jpg)
Classic Mosaic: Main constraints• Tile orientation should emphasize important
scene edges• Tiles should be packed tightly while preserving
their completeness
Bad TilingGood Tiling
![Page 4: Generating Classic Mosaics with Graph Cuts Y. Liu, O. Veksler and O. Juan University of Western Ontario, Canada Ecole Centrale de Paris, France](https://reader036.vdocuments.mx/reader036/viewer/2022062511/5517883f55034645368b53a1/html5/thumbnails/4.jpg)
Previous Work• User Interaction– Hausner [SIGGRAPH2001]– Elber and Wolberg[Visual Computer 2003]
• Explicit Edge detection– Di Blasi et. al [Visual Computer 2005]– Battiato et. al [WSCG 2006]
• Implicit Methods– Battiato et. al [EG 2008]– Liu et. al [EMMCVPR 2007]
![Page 5: Generating Classic Mosaics with Graph Cuts Y. Liu, O. Veksler and O. Juan University of Western Ontario, Canada Ecole Centrale de Paris, France](https://reader036.vdocuments.mx/reader036/viewer/2022062511/5517883f55034645368b53a1/html5/thumbnails/5.jpg)
Previous Work Cont.
• Requires user interaction• Discontinuities in tile
orientation filed• No control over tile overlap
• Local optimization for tile orientation
• Heuristics for tile packing
Hausner [SIGGRAPH 2001] Battiato et. Al [EG 2008]
![Page 6: Generating Classic Mosaics with Graph Cuts Y. Liu, O. Veksler and O. Juan University of Western Ontario, Canada Ecole Centrale de Paris, France](https://reader036.vdocuments.mx/reader036/viewer/2022062511/5517883f55034645368b53a1/html5/thumbnails/6.jpg)
Our Method: Overview• We address the problem in a global
optimization framework– pixel labelling problem
• Formulate energy which encodes constraints for a good mosaic– Encourage tiles to align to image edges– Smoothly varying tile orientations– Tight tile packing– Overlapping tiles prohibited
High Energy
Low Energy
![Page 7: Generating Classic Mosaics with Graph Cuts Y. Liu, O. Veksler and O. Juan University of Western Ontario, Canada Ecole Centrale de Paris, France](https://reader036.vdocuments.mx/reader036/viewer/2022062511/5517883f55034645368b53a1/html5/thumbnails/7.jpg)
Mosaic as Pixel Labelling • Each pixel p is assigned a pairwise label
(vp , ϕp)
• ϕp is tile orientation– if tile centered at p is visible,
orientation is ϕp• vp is visibility– 1 means tile is present– 0 means tile is hidden
0
0
1
1 ϕp
![Page 8: Generating Classic Mosaics with Graph Cuts Y. Liu, O. Veksler and O. Juan University of Western Ontario, Canada Ecole Centrale de Paris, France](https://reader036.vdocuments.mx/reader036/viewer/2022062511/5517883f55034645368b53a1/html5/thumbnails/8.jpg)
Energy Function( , )E v (1 )p
p P
v
( )p p pp P
v D
{ , }
( , , , )pq p q p qp q N
V v v
encourages dense packing
encourages tiles to align to image edges
encourages smooth tile orientations and prohibits tile overlap
high cost
low cost
high cost
high cost
low cost
high cost
low cost
![Page 9: Generating Classic Mosaics with Graph Cuts Y. Liu, O. Veksler and O. Juan University of Western Ontario, Canada Ecole Centrale de Paris, France](https://reader036.vdocuments.mx/reader036/viewer/2022062511/5517883f55034645368b53a1/html5/thumbnails/9.jpg)
Optimization with Graph cut • NP-hard to optimize all variables
simultaneously • We use step-wise optimization– First, ignore the visibility labels and optimize the
orientation variables– Then, fix the orientation labels and optimize the
visibility variables
![Page 10: Generating Classic Mosaics with Graph Cuts Y. Liu, O. Veksler and O. Juan University of Western Ontario, Canada Ecole Centrale de Paris, France](https://reader036.vdocuments.mx/reader036/viewer/2022062511/5517883f55034645368b53a1/html5/thumbnails/10.jpg)
Optimizing Orientation Variables
• Assign an orientation label ϕp to each pixel– don’t know where the tiles
will be placed, so optimize for all possible locations
• approximately optimized by alpha-expansion [Boykov et.al. PAMI01]
( )E ( )p pp P
D
{ , }
( , )pq p qp q N
V
![Page 11: Generating Classic Mosaics with Graph Cuts Y. Liu, O. Veksler and O. Juan University of Western Ontario, Canada Ecole Centrale de Paris, France](https://reader036.vdocuments.mx/reader036/viewer/2022062511/5517883f55034645368b53a1/html5/thumbnails/11.jpg)
Optimizing the visibility variables• Heuristically build many mosaic layers
according to the orientation labels– Each layer consists of non-overlapping tiles– Each layer is as dense as possible
![Page 12: Generating Classic Mosaics with Graph Cuts Y. Liu, O. Veksler and O. Juan University of Western Ontario, Canada Ecole Centrale de Paris, France](https://reader036.vdocuments.mx/reader036/viewer/2022062511/5517883f55034645368b53a1/html5/thumbnails/12.jpg)
Maintain current solution Randomly pick mosaic
layer Find the smallest energy
stitching
1
10
0 110
1 01
101
1
formulated as binary labelling problem energy roughly corresponds to the area of the
gap space, therefore encouraging reduction in gaps
tiles will not overlap since energy would be infinite
Submodular energy, optimized with a graph cut [Kolmogorov et. al. PAMI2004]
Stitching Layers
![Page 13: Generating Classic Mosaics with Graph Cuts Y. Liu, O. Veksler and O. Juan University of Western Ontario, Canada Ecole Centrale de Paris, France](https://reader036.vdocuments.mx/reader036/viewer/2022062511/5517883f55034645368b53a1/html5/thumbnails/13.jpg)
Layer Stitching Illustration• Tiles may cross the image edges and create a
blurry effect
Current Solution Random Layer
![Page 14: Generating Classic Mosaics with Graph Cuts Y. Liu, O. Veksler and O. Juan University of Western Ontario, Canada Ecole Centrale de Paris, France](https://reader036.vdocuments.mx/reader036/viewer/2022062511/5517883f55034645368b53a1/html5/thumbnails/14.jpg)
Layer Stitching illustration• Tiles overlapping intensity edges are removed
Good stitching
Bad stitching
![Page 15: Generating Classic Mosaics with Graph Cuts Y. Liu, O. Veksler and O. Juan University of Western Ontario, Canada Ecole Centrale de Paris, France](https://reader036.vdocuments.mx/reader036/viewer/2022062511/5517883f55034645368b53a1/html5/thumbnails/15.jpg)
Overview of Layer Stitching• Start from a random layer• In each iteration, stitch the current solution
and a candidate mosaic layer together• Repeat until the result is visually pleasing
![Page 16: Generating Classic Mosaics with Graph Cuts Y. Liu, O. Veksler and O. Juan University of Western Ontario, Canada Ecole Centrale de Paris, France](https://reader036.vdocuments.mx/reader036/viewer/2022062511/5517883f55034645368b53a1/html5/thumbnails/16.jpg)
Experimental Results
Our Approach
Battiato et. Al [EG 2008]
![Page 17: Generating Classic Mosaics with Graph Cuts Y. Liu, O. Veksler and O. Juan University of Western Ontario, Canada Ecole Centrale de Paris, France](https://reader036.vdocuments.mx/reader036/viewer/2022062511/5517883f55034645368b53a1/html5/thumbnails/17.jpg)
Our Approach
Battiato et. Al [EG 2008]
Experimental Results
![Page 18: Generating Classic Mosaics with Graph Cuts Y. Liu, O. Veksler and O. Juan University of Western Ontario, Canada Ecole Centrale de Paris, France](https://reader036.vdocuments.mx/reader036/viewer/2022062511/5517883f55034645368b53a1/html5/thumbnails/18.jpg)
Experimental Results
![Page 19: Generating Classic Mosaics with Graph Cuts Y. Liu, O. Veksler and O. Juan University of Western Ontario, Canada Ecole Centrale de Paris, France](https://reader036.vdocuments.mx/reader036/viewer/2022062511/5517883f55034645368b53a1/html5/thumbnails/19.jpg)
Experimental Results
![Page 20: Generating Classic Mosaics with Graph Cuts Y. Liu, O. Veksler and O. Juan University of Western Ontario, Canada Ecole Centrale de Paris, France](https://reader036.vdocuments.mx/reader036/viewer/2022062511/5517883f55034645368b53a1/html5/thumbnails/20.jpg)
Mosaic with Different Tile Size
Input Image 10 by 10 20 by 20
![Page 21: Generating Classic Mosaics with Graph Cuts Y. Liu, O. Veksler and O. Juan University of Western Ontario, Canada Ecole Centrale de Paris, France](https://reader036.vdocuments.mx/reader036/viewer/2022062511/5517883f55034645368b53a1/html5/thumbnails/21.jpg)
Number of Iterations in Stitching
Starting Layer 27 iteration 156 iteration
![Page 22: Generating Classic Mosaics with Graph Cuts Y. Liu, O. Veksler and O. Juan University of Western Ontario, Canada Ecole Centrale de Paris, France](https://reader036.vdocuments.mx/reader036/viewer/2022062511/5517883f55034645368b53a1/html5/thumbnails/22.jpg)
Edge Avoidance
Small Weight Large Weight
![Page 23: Generating Classic Mosaics with Graph Cuts Y. Liu, O. Veksler and O. Juan University of Western Ontario, Canada Ecole Centrale de Paris, France](https://reader036.vdocuments.mx/reader036/viewer/2022062511/5517883f55034645368b53a1/html5/thumbnails/23.jpg)
Conclusion and future work• We address classic mosaic problem in an energy
optimization framework• Our energy function encodes desired mosaic properties
• Tiles align to image edges, minimize gap space, prohibit tile overlap
• We eliminate monotonous user interaction of drawing feature edges
• Take advantage of powerful discrete optimization methods based on graph cuts
• Results are visually pleasing• Extensions:
• tiles of different shapes and sizes• video mosaics