graphcut textures image and video synthesis using graph cuts vivek kwatravivek kwatra, arno schödl,...
TRANSCRIPT
![Page 1: Graphcut Textures Image and Video Synthesis Using Graph Cuts Vivek KwatraVivek Kwatra, Arno Schödl, Irfan Essa, Greg Turk and Aaron BobickArno SchödlIrfan](https://reader036.vdocuments.mx/reader036/viewer/2022062805/5697bfe31a28abf838cb4bc5/html5/thumbnails/1.jpg)
Graphcut Textures Image and Video Synthesis Using Graph CutsVivek Kwatra, Arno Schödl, Irfan Essa, Greg Turk and Aaron Bobick
![Page 2: Graphcut Textures Image and Video Synthesis Using Graph Cuts Vivek KwatraVivek Kwatra, Arno Schödl, Irfan Essa, Greg Turk and Aaron BobickArno SchödlIrfan](https://reader036.vdocuments.mx/reader036/viewer/2022062805/5697bfe31a28abf838cb4bc5/html5/thumbnails/2.jpg)
Overview
• Brief Introduction to Texture Synthesis
• Related Work• The Graph Cut Technique• Patch Placement• Image Synthesis Results• Video Synthesis• Conclusion
![Page 3: Graphcut Textures Image and Video Synthesis Using Graph Cuts Vivek KwatraVivek Kwatra, Arno Schödl, Irfan Essa, Greg Turk and Aaron BobickArno SchödlIrfan](https://reader036.vdocuments.mx/reader036/viewer/2022062805/5697bfe31a28abf838cb4bc5/html5/thumbnails/3.jpg)
Texture Synthesis
• The ability to generate a reasonable amount of texture from a sample input texture or from some training data
![Page 4: Graphcut Textures Image and Video Synthesis Using Graph Cuts Vivek KwatraVivek Kwatra, Arno Schödl, Irfan Essa, Greg Turk and Aaron BobickArno SchödlIrfan](https://reader036.vdocuments.mx/reader036/viewer/2022062805/5697bfe31a28abf838cb4bc5/html5/thumbnails/4.jpg)
Related Work
• Three Classes of Algorithms– Pixel Based
•Parametric Model– Use a fixed number of parameters
•Non-parametric Model– Use a collection of exemplars to model
the texture
– Patch Based•Uses patches of texture from the input
texture
![Page 5: Graphcut Textures Image and Video Synthesis Using Graph Cuts Vivek KwatraVivek Kwatra, Arno Schödl, Irfan Essa, Greg Turk and Aaron BobickArno SchödlIrfan](https://reader036.vdocuments.mx/reader036/viewer/2022062805/5697bfe31a28abf838cb4bc5/html5/thumbnails/5.jpg)
Image Quilting
• Patches of texture overlap, a dynamic programming algorithm is used to find the minimum error boundary between the patches
min. error boundary
![Page 6: Graphcut Textures Image and Video Synthesis Using Graph Cuts Vivek KwatraVivek Kwatra, Arno Schödl, Irfan Essa, Greg Turk and Aaron BobickArno SchödlIrfan](https://reader036.vdocuments.mx/reader036/viewer/2022062805/5697bfe31a28abf838cb4bc5/html5/thumbnails/6.jpg)
Max Flow / Min Cut Problem
• Problem: Find the maximum amount that can flow from s to t
• Theorem: The maximal amount of a flow is equal to the capacity of a minimal cut
![Page 7: Graphcut Textures Image and Video Synthesis Using Graph Cuts Vivek KwatraVivek Kwatra, Arno Schödl, Irfan Essa, Greg Turk and Aaron BobickArno SchödlIrfan](https://reader036.vdocuments.mx/reader036/viewer/2022062805/5697bfe31a28abf838cb4bc5/html5/thumbnails/7.jpg)
A new way to find the cut
• Model overlapping patches as directed graphs• Assign sources and sinks, and set the edges of
adjacent pixel equal color difference• M(1, 4, A, B) = ||A(1) - B(1)|| + ||A(4) - B(4)||• run the max flow algorithm to find optimal seam
![Page 8: Graphcut Textures Image and Video Synthesis Using Graph Cuts Vivek KwatraVivek Kwatra, Arno Schödl, Irfan Essa, Greg Turk and Aaron BobickArno SchödlIrfan](https://reader036.vdocuments.mx/reader036/viewer/2022062805/5697bfe31a28abf838cb4bc5/html5/thumbnails/8.jpg)
Accounting for Old Seams
• We can incorporate old seam costs into the problem, and thus determine which pixels (if any) from the new patch should cover over some of the old seams.
![Page 9: Graphcut Textures Image and Video Synthesis Using Graph Cuts Vivek KwatraVivek Kwatra, Arno Schödl, Irfan Essa, Greg Turk and Aaron BobickArno SchödlIrfan](https://reader036.vdocuments.mx/reader036/viewer/2022062805/5697bfe31a28abf838cb4bc5/html5/thumbnails/9.jpg)
Surrounded Regions
• Sometimes we may want to cover old seams
![Page 10: Graphcut Textures Image and Video Synthesis Using Graph Cuts Vivek KwatraVivek Kwatra, Arno Schödl, Irfan Essa, Greg Turk and Aaron BobickArno SchödlIrfan](https://reader036.vdocuments.mx/reader036/viewer/2022062805/5697bfe31a28abf838cb4bc5/html5/thumbnails/10.jpg)
Patch Placement
• We’ve talked about how to slice and dice, but how do we lay down the texture patches?– Random Placement: Fast and works well
for random textures– Entire Patch Matching: Normalize the sum-
of-squared-differences and divide by area of patch. Pick a random, but good match.
– Sub-Patch Matching: Pick a place in the output texture to place a patch, then search the input texture for the best match
![Page 11: Graphcut Textures Image and Video Synthesis Using Graph Cuts Vivek KwatraVivek Kwatra, Arno Schödl, Irfan Essa, Greg Turk and Aaron BobickArno SchödlIrfan](https://reader036.vdocuments.mx/reader036/viewer/2022062805/5697bfe31a28abf838cb4bc5/html5/thumbnails/11.jpg)
Results
![Page 12: Graphcut Textures Image and Video Synthesis Using Graph Cuts Vivek KwatraVivek Kwatra, Arno Schödl, Irfan Essa, Greg Turk and Aaron BobickArno SchödlIrfan](https://reader036.vdocuments.mx/reader036/viewer/2022062805/5697bfe31a28abf838cb4bc5/html5/thumbnails/12.jpg)
Results
![Page 13: Graphcut Textures Image and Video Synthesis Using Graph Cuts Vivek KwatraVivek Kwatra, Arno Schödl, Irfan Essa, Greg Turk and Aaron BobickArno SchödlIrfan](https://reader036.vdocuments.mx/reader036/viewer/2022062805/5697bfe31a28abf838cb4bc5/html5/thumbnails/13.jpg)
Results
![Page 14: Graphcut Textures Image and Video Synthesis Using Graph Cuts Vivek KwatraVivek Kwatra, Arno Schödl, Irfan Essa, Greg Turk and Aaron BobickArno SchödlIrfan](https://reader036.vdocuments.mx/reader036/viewer/2022062805/5697bfe31a28abf838cb4bc5/html5/thumbnails/14.jpg)
Extensions and Refinements
• Adapting the Cost Function: Pay attention to frequency content present in the image or video
• Feathering and multi-resolution splining: Reduces ability to notice obvious seams
• FFT-Based Acceleration: For patch matching. SSD can be expensive. One example had a reduction of 10 minutes to 5 seconds after switching to the FFT method.
![Page 15: Graphcut Textures Image and Video Synthesis Using Graph Cuts Vivek KwatraVivek Kwatra, Arno Schödl, Irfan Essa, Greg Turk and Aaron BobickArno SchödlIrfan](https://reader036.vdocuments.mx/reader036/viewer/2022062805/5697bfe31a28abf838cb4bc5/html5/thumbnails/15.jpg)
Additional Transformations
• Input image can be rotated, mirrored and scaled to produce interesting results
![Page 16: Graphcut Textures Image and Video Synthesis Using Graph Cuts Vivek KwatraVivek Kwatra, Arno Schödl, Irfan Essa, Greg Turk and Aaron BobickArno SchödlIrfan](https://reader036.vdocuments.mx/reader036/viewer/2022062805/5697bfe31a28abf838cb4bc5/html5/thumbnails/16.jpg)
Additional Transformations Continued
![Page 17: Graphcut Textures Image and Video Synthesis Using Graph Cuts Vivek KwatraVivek Kwatra, Arno Schödl, Irfan Essa, Greg Turk and Aaron BobickArno SchödlIrfan](https://reader036.vdocuments.mx/reader036/viewer/2022062805/5697bfe31a28abf838cb4bc5/html5/thumbnails/17.jpg)
Rotation Example
• Image Quilting vs Graph Cut
Input Graph Cut Result Image Quilting Result
![Page 18: Graphcut Textures Image and Video Synthesis Using Graph Cuts Vivek KwatraVivek Kwatra, Arno Schödl, Irfan Essa, Greg Turk and Aaron BobickArno SchödlIrfan](https://reader036.vdocuments.mx/reader036/viewer/2022062805/5697bfe31a28abf838cb4bc5/html5/thumbnails/18.jpg)
Interactive Graph Cuts
• User has a hand in placing image, graph cut algorithm then finds the best cut
![Page 19: Graphcut Textures Image and Video Synthesis Using Graph Cuts Vivek KwatraVivek Kwatra, Arno Schödl, Irfan Essa, Greg Turk and Aaron BobickArno SchödlIrfan](https://reader036.vdocuments.mx/reader036/viewer/2022062805/5697bfe31a28abf838cb4bc5/html5/thumbnails/19.jpg)
Interactive Graph Cuts Continued
![Page 20: Graphcut Textures Image and Video Synthesis Using Graph Cuts Vivek KwatraVivek Kwatra, Arno Schödl, Irfan Essa, Greg Turk and Aaron BobickArno SchödlIrfan](https://reader036.vdocuments.mx/reader036/viewer/2022062805/5697bfe31a28abf838cb4bc5/html5/thumbnails/20.jpg)
Video Synthesis
• Video textures turn existing video into an infinitely playing form by finding smooth transitions from one part of the video to another
• Patches in the case of video are the whole 3D space-time blocks of video
![Page 21: Graphcut Textures Image and Video Synthesis Using Graph Cuts Vivek KwatraVivek Kwatra, Arno Schödl, Irfan Essa, Greg Turk and Aaron BobickArno SchödlIrfan](https://reader036.vdocuments.mx/reader036/viewer/2022062805/5697bfe31a28abf838cb4bc5/html5/thumbnails/21.jpg)
Video Synthesis Continued
![Page 22: Graphcut Textures Image and Video Synthesis Using Graph Cuts Vivek KwatraVivek Kwatra, Arno Schödl, Irfan Essa, Greg Turk and Aaron BobickArno SchödlIrfan](https://reader036.vdocuments.mx/reader036/viewer/2022062805/5697bfe31a28abf838cb4bc5/html5/thumbnails/22.jpg)
Conclusion
• A new texture synthesis algorithm was introduced that works not only in 2D, but in 3D
• It’s has advantages over previous patch based methods
• It’s very fast, taking between 5 seconds and 5 minutes to generate results
![Page 23: Graphcut Textures Image and Video Synthesis Using Graph Cuts Vivek KwatraVivek Kwatra, Arno Schödl, Irfan Essa, Greg Turk and Aaron BobickArno SchödlIrfan](https://reader036.vdocuments.mx/reader036/viewer/2022062805/5697bfe31a28abf838cb4bc5/html5/thumbnails/23.jpg)
Questions
• Any questions?