syde 575: introduction to image processing image restoration: digital inpainting (no textbook...
TRANSCRIPT
![Page 1: SYDE 575: Introduction to Image Processing Image Restoration: Digital Inpainting (no textbook readings)](https://reader035.vdocuments.mx/reader035/viewer/2022081501/5697bfab1a28abf838c9ac6a/html5/thumbnails/1.jpg)
SYDE 575: Introduction to Image Processing
Image Restoration:Digital Inpainting
(no textbook readings)
![Page 2: SYDE 575: Introduction to Image Processing Image Restoration: Digital Inpainting (no textbook readings)](https://reader035.vdocuments.mx/reader035/viewer/2022081501/5697bfab1a28abf838c9ac6a/html5/thumbnails/2.jpg)
What is Inpainting?
Term “inpainting” originated from art restorers, who manually fills in parts of a painting that has cracked off over time
In digital image processing, inpainting refers to the process of automatically restoring missing information in images and videos
![Page 3: SYDE 575: Introduction to Image Processing Image Restoration: Digital Inpainting (no textbook readings)](https://reader035.vdocuments.mx/reader035/viewer/2022081501/5697bfab1a28abf838c9ac6a/html5/thumbnails/3.jpg)
Why inpainting?
Remove physical deterioration
Cracks Scratches Dust
Source: Oliveira et al. 2001
![Page 4: SYDE 575: Introduction to Image Processing Image Restoration: Digital Inpainting (no textbook readings)](https://reader035.vdocuments.mx/reader035/viewer/2022081501/5697bfab1a28abf838c9ac6a/html5/thumbnails/4.jpg)
Why inpainting?
Recover lost blocks in transmission of images and videos
Source: Liu et al. 2007
![Page 5: SYDE 575: Introduction to Image Processing Image Restoration: Digital Inpainting (no textbook readings)](https://reader035.vdocuments.mx/reader035/viewer/2022081501/5697bfab1a28abf838c9ac6a/html5/thumbnails/5.jpg)
Why inpainting?
Remove unwanted image content
Power-lines Birds People Text
Source: Oliveira et al. 2001
![Page 6: SYDE 575: Introduction to Image Processing Image Restoration: Digital Inpainting (no textbook readings)](https://reader035.vdocuments.mx/reader035/viewer/2022081501/5697bfab1a28abf838c9ac6a/html5/thumbnails/6.jpg)
Problem Formulation
Fill in target region using information from source region Q
W
![Page 7: SYDE 575: Introduction to Image Processing Image Restoration: Digital Inpainting (no textbook readings)](https://reader035.vdocuments.mx/reader035/viewer/2022081501/5697bfab1a28abf838c9ac6a/html5/thumbnails/7.jpg)
Inpainting Algorithms
Digital inpainting algorithms generally categorized into two main groups: Diffusion-based methods Exemplar-based methods
![Page 8: SYDE 575: Introduction to Image Processing Image Restoration: Digital Inpainting (no textbook readings)](https://reader035.vdocuments.mx/reader035/viewer/2022081501/5697bfab1a28abf838c9ac6a/html5/thumbnails/8.jpg)
Diffusion-based Methods
Inspired by the physical diffusion process, where molecules spread from areas of high concentration to areas of low concentration to fill a volume
For digital inpainting, information from source region is “diffused” into the target region to fill in missing information
![Page 9: SYDE 575: Introduction to Image Processing Image Restoration: Digital Inpainting (no textbook readings)](https://reader035.vdocuments.mx/reader035/viewer/2022081501/5697bfab1a28abf838c9ac6a/html5/thumbnails/9.jpg)
Diffusion-based Methods
Diffusion in digital images is analogous to repeatedly smoothing (convolution)
Intuitively, diffusion-based methods repeatedly smooth image content from the source region to the target region until the target region is filled
![Page 10: SYDE 575: Introduction to Image Processing Image Restoration: Digital Inpainting (no textbook readings)](https://reader035.vdocuments.mx/reader035/viewer/2022081501/5697bfab1a28abf838c9ac6a/html5/thumbnails/10.jpg)
Simple Diffusion-based Algorithm
Let be the target region, be the source region Define boundary in target region Convolve with isotropic smoothing
kernel (e.g., Gaussian) for a number of iterations
Define new boundary in new smaller target region
Repeat process until the entire target region is filled in
W Q
Q1¶Q
1¶Q
2¶Q
![Page 11: SYDE 575: Introduction to Image Processing Image Restoration: Digital Inpainting (no textbook readings)](https://reader035.vdocuments.mx/reader035/viewer/2022081501/5697bfab1a28abf838c9ac6a/html5/thumbnails/11.jpg)
Results
Source: Oliveira et al. 2001
![Page 12: SYDE 575: Introduction to Image Processing Image Restoration: Digital Inpainting (no textbook readings)](https://reader035.vdocuments.mx/reader035/viewer/2022081501/5697bfab1a28abf838c9ac6a/html5/thumbnails/12.jpg)
Disadvantages
Appearance of blurring Very noticeable for large regions and
structures Difficult to fill in large regions properly
Why?Restricted to using local informationMany situations where local information does not characterize the missing information
![Page 13: SYDE 575: Introduction to Image Processing Image Restoration: Digital Inpainting (no textbook readings)](https://reader035.vdocuments.mx/reader035/viewer/2022081501/5697bfab1a28abf838c9ac6a/html5/thumbnails/13.jpg)
Illustration of Exemplar-based Inpainting
![Page 14: SYDE 575: Introduction to Image Processing Image Restoration: Digital Inpainting (no textbook readings)](https://reader035.vdocuments.mx/reader035/viewer/2022081501/5697bfab1a28abf838c9ac6a/html5/thumbnails/14.jpg)
Simple Exemplar-based Algorithm
Let be the target region, be the source region Define boundary in target region Find patches with the best match for
patches around as exemplarsSimilarity between patches can be determined using measures such as mean square error (MSE)
W Q
Q1¶Q
1¶Q
![Page 15: SYDE 575: Introduction to Image Processing Image Restoration: Digital Inpainting (no textbook readings)](https://reader035.vdocuments.mx/reader035/viewer/2022081501/5697bfab1a28abf838c9ac6a/html5/thumbnails/15.jpg)
Simple Exemplar-based Algorithm (Cont'd)
Fill patches around with the exemplars
Define new boundary in new smaller target region
Repeat process until the entire target region is filled in
1¶Q
2¶Q
![Page 16: SYDE 575: Introduction to Image Processing Image Restoration: Digital Inpainting (no textbook readings)](https://reader035.vdocuments.mx/reader035/viewer/2022081501/5697bfab1a28abf838c9ac6a/html5/thumbnails/16.jpg)
Results
Source: Criminisi et al. 2004