image pyramids - department of computer science ...fidler/slides/2019/csc420/lecture5.pdf · idea:...
TRANSCRIPT
![Page 1: Image Pyramids - Department of Computer Science ...fidler/slides/2019/CSC420/lecture5.pdf · Idea: Re-size Image Re-scale the image multiple times! Do correlation on every size! template](https://reader036.vdocuments.mx/reader036/viewer/2022062605/5fceaa5ca9f0ab064c082813/html5/thumbnails/1.jpg)
Image Pyramids
Sanja Fidler CSC420: Intro to Image Understanding 1 / 35
![Page 2: Image Pyramids - Department of Computer Science ...fidler/slides/2019/CSC420/lecture5.pdf · Idea: Re-size Image Re-scale the image multiple times! Do correlation on every size! template](https://reader036.vdocuments.mx/reader036/viewer/2022062605/5fceaa5ca9f0ab064c082813/html5/thumbnails/2.jpg)
Finding Waldo
Let’s revisit the problem of finding Waldo
This time he is on the road
image
template (filter)
Sanja Fidler CSC420: Intro to Image Understanding 2 / 35
![Page 3: Image Pyramids - Department of Computer Science ...fidler/slides/2019/CSC420/lecture5.pdf · Idea: Re-size Image Re-scale the image multiple times! Do correlation on every size! template](https://reader036.vdocuments.mx/reader036/viewer/2022062605/5fceaa5ca9f0ab064c082813/html5/thumbnails/3.jpg)
Finding Waldo
He comes closer but our filter doesn’t know that
How can we find Waldo?
image
template (filter)
Sanja Fidler CSC420: Intro to Image Understanding 3 / 35
![Page 4: Image Pyramids - Department of Computer Science ...fidler/slides/2019/CSC420/lecture5.pdf · Idea: Re-size Image Re-scale the image multiple times! Do correlation on every size! template](https://reader036.vdocuments.mx/reader036/viewer/2022062605/5fceaa5ca9f0ab064c082813/html5/thumbnails/4.jpg)
Idea: Re-size Image
Re-scale the image multiple times! Do correlation on every size!
template (filter)
Sanja Fidler CSC420: Intro to Image Understanding 4 / 35
![Page 5: Image Pyramids - Department of Computer Science ...fidler/slides/2019/CSC420/lecture5.pdf · Idea: Re-size Image Re-scale the image multiple times! Do correlation on every size! template](https://reader036.vdocuments.mx/reader036/viewer/2022062605/5fceaa5ca9f0ab064c082813/html5/thumbnails/5.jpg)
This image is huge. How can we make it smaller?
Sanja Fidler CSC420: Intro to Image Understanding 5 / 35
![Page 6: Image Pyramids - Department of Computer Science ...fidler/slides/2019/CSC420/lecture5.pdf · Idea: Re-size Image Re-scale the image multiple times! Do correlation on every size! template](https://reader036.vdocuments.mx/reader036/viewer/2022062605/5fceaa5ca9f0ab064c082813/html5/thumbnails/6.jpg)
Image Sub-Sampling
Idea: Throw away every other row and column to create a 1/2 size image
1/4
1/8
[Source: S. Seitz]Sanja Fidler CSC420: Intro to Image Understanding 6 / 35
![Page 7: Image Pyramids - Department of Computer Science ...fidler/slides/2019/CSC420/lecture5.pdf · Idea: Re-size Image Re-scale the image multiple times! Do correlation on every size! template](https://reader036.vdocuments.mx/reader036/viewer/2022062605/5fceaa5ca9f0ab064c082813/html5/thumbnails/7.jpg)
Image Sub-Sampling
Why does this look so crufty?
!"#$$%&'$())*+$ !",$$%#'$())*+$!"&$
[Source: S. Seitz]
Sanja Fidler CSC420: Intro to Image Understanding 7 / 35
![Page 8: Image Pyramids - Department of Computer Science ...fidler/slides/2019/CSC420/lecture5.pdf · Idea: Re-size Image Re-scale the image multiple times! Do correlation on every size! template](https://reader036.vdocuments.mx/reader036/viewer/2022062605/5fceaa5ca9f0ab064c082813/html5/thumbnails/8.jpg)
Even worse for synthetic images
I want to resize my image by factor 2
And I take every other column and every other row (1st, 3rd, 5th, etc)
Figure: Dashed line denotes the border of the image (it’s not part of the image)
Sanja Fidler CSC420: Intro to Image Understanding 8 / 35
![Page 9: Image Pyramids - Department of Computer Science ...fidler/slides/2019/CSC420/lecture5.pdf · Idea: Re-size Image Re-scale the image multiple times! Do correlation on every size! template](https://reader036.vdocuments.mx/reader036/viewer/2022062605/5fceaa5ca9f0ab064c082813/html5/thumbnails/9.jpg)
Even worse for synthetic images
I want to resize my image by factor 2
And I take every other column and every other row (1st, 3rd, 5th, etc)
Where is the rectangle!
Figure: Dashed line denotes the border of the image (it’s not part of the image)
Sanja Fidler CSC420: Intro to Image Understanding 8 / 35
![Page 10: Image Pyramids - Department of Computer Science ...fidler/slides/2019/CSC420/lecture5.pdf · Idea: Re-size Image Re-scale the image multiple times! Do correlation on every size! template](https://reader036.vdocuments.mx/reader036/viewer/2022062605/5fceaa5ca9f0ab064c082813/html5/thumbnails/10.jpg)
Even worse for synthetic images
What’s in the image?
Now I want to resize my image by half in the width direction
And I take every other column (1st, 3rd, 5th, etc)
Sanja Fidler CSC420: Intro to Image Understanding 9 / 35
![Page 11: Image Pyramids - Department of Computer Science ...fidler/slides/2019/CSC420/lecture5.pdf · Idea: Re-size Image Re-scale the image multiple times! Do correlation on every size! template](https://reader036.vdocuments.mx/reader036/viewer/2022062605/5fceaa5ca9f0ab064c082813/html5/thumbnails/11.jpg)
Even worse for synthetic images
What’s in the image?
Now I want to resize my image by half in the width direction
And I take every other column (1st, 3rd, 5th, etc)
Sanja Fidler CSC420: Intro to Image Understanding 9 / 35
![Page 12: Image Pyramids - Department of Computer Science ...fidler/slides/2019/CSC420/lecture5.pdf · Idea: Re-size Image Re-scale the image multiple times! Do correlation on every size! template](https://reader036.vdocuments.mx/reader036/viewer/2022062605/5fceaa5ca9f0ab064c082813/html5/thumbnails/12.jpg)
Even worse for synthetic images
What’s in the image?
Now I want to resize my image by half in the width direction
And I take every other column (1st, 3rd, 5th, etc)
Where is the chicken!
Sanja Fidler CSC420: Intro to Image Understanding 9 / 35
![Page 13: Image Pyramids - Department of Computer Science ...fidler/slides/2019/CSC420/lecture5.pdf · Idea: Re-size Image Re-scale the image multiple times! Do correlation on every size! template](https://reader036.vdocuments.mx/reader036/viewer/2022062605/5fceaa5ca9f0ab064c082813/html5/thumbnails/13.jpg)
Image Sub-Sampling
[Source: F. Durand]Sanja Fidler CSC420: Intro to Image Understanding 10 / 35
![Page 14: Image Pyramids - Department of Computer Science ...fidler/slides/2019/CSC420/lecture5.pdf · Idea: Re-size Image Re-scale the image multiple times! Do correlation on every size! template](https://reader036.vdocuments.mx/reader036/viewer/2022062605/5fceaa5ca9f0ab064c082813/html5/thumbnails/14.jpg)
Even worse for synthetic images
What’s happening?
[Source: L. Zhang]Sanja Fidler CSC420: Intro to Image Understanding 11 / 35
![Page 15: Image Pyramids - Department of Computer Science ...fidler/slides/2019/CSC420/lecture5.pdf · Idea: Re-size Image Re-scale the image multiple times! Do correlation on every size! template](https://reader036.vdocuments.mx/reader036/viewer/2022062605/5fceaa5ca9f0ab064c082813/html5/thumbnails/15.jpg)
Aliasing
Occurs when your sampling rate is not high enough to capture the amountof detail in your image
To do sampling right, need to understand the structure of your signal/image
The minimum sampling rate is called the Nyquist rate
[Source: R. Urtasun]
Sanja Fidler CSC420: Intro to Image Understanding 12 / 35
![Page 16: Image Pyramids - Department of Computer Science ...fidler/slides/2019/CSC420/lecture5.pdf · Idea: Re-size Image Re-scale the image multiple times! Do correlation on every size! template](https://reader036.vdocuments.mx/reader036/viewer/2022062605/5fceaa5ca9f0ab064c082813/html5/thumbnails/16.jpg)
Aliasing
Occurs when your sampling rate is not high enough to capture the amountof detail in your image
To do sampling right, need to understand the structure of your signal/image
The minimum sampling rate is called the Nyquist rate
[Source: R. Urtasun]
Sanja Fidler CSC420: Intro to Image Understanding 12 / 35
![Page 17: Image Pyramids - Department of Computer Science ...fidler/slides/2019/CSC420/lecture5.pdf · Idea: Re-size Image Re-scale the image multiple times! Do correlation on every size! template](https://reader036.vdocuments.mx/reader036/viewer/2022062605/5fceaa5ca9f0ab064c082813/html5/thumbnails/17.jpg)
Aliasing
Occurs when your sampling rate is not high enough to capture the amountof detail in your image
To do sampling right, need to understand the structure of your signal/image
The minimum sampling rate is called the Nyquist rate
[Source: R. Urtasun]
Sanja Fidler CSC420: Intro to Image Understanding 12 / 35
![Page 18: Image Pyramids - Department of Computer Science ...fidler/slides/2019/CSC420/lecture5.pdf · Idea: Re-size Image Re-scale the image multiple times! Do correlation on every size! template](https://reader036.vdocuments.mx/reader036/viewer/2022062605/5fceaa5ca9f0ab064c082813/html5/thumbnails/18.jpg)
Mr. Nyquist
Harry Nyquist says that one should look at the frequencies of the signal.
One should find the highest frequency (via Fourier Transform)
To sample properly you need to sample with at least twice that frequency
For those interested: http://en.wikipedia.org/wiki/Nyquist%E2%80%
93Shannon_sampling_theorem
He looks like a smart guy, we’ll justbelieve him
Sanja Fidler CSC420: Intro to Image Understanding 13 / 35
![Page 19: Image Pyramids - Department of Computer Science ...fidler/slides/2019/CSC420/lecture5.pdf · Idea: Re-size Image Re-scale the image multiple times! Do correlation on every size! template](https://reader036.vdocuments.mx/reader036/viewer/2022062605/5fceaa5ca9f0ab064c082813/html5/thumbnails/19.jpg)
2D example
Good sampling
Bad sampling
[Source: N. Snavely]Sanja Fidler CSC420: Intro to Image Understanding 14 / 35
![Page 20: Image Pyramids - Department of Computer Science ...fidler/slides/2019/CSC420/lecture5.pdf · Idea: Re-size Image Re-scale the image multiple times! Do correlation on every size! template](https://reader036.vdocuments.mx/reader036/viewer/2022062605/5fceaa5ca9f0ab064c082813/html5/thumbnails/20.jpg)
Going back to Downsampling ...
When downsampling by a factor of two, the original image has frequenciesthat are too high
High frequencies are caused by sharp edges
How can we fix this?
[Adopted from: R. Urtasun]
Sanja Fidler CSC420: Intro to Image Understanding 15 / 35
![Page 21: Image Pyramids - Department of Computer Science ...fidler/slides/2019/CSC420/lecture5.pdf · Idea: Re-size Image Re-scale the image multiple times! Do correlation on every size! template](https://reader036.vdocuments.mx/reader036/viewer/2022062605/5fceaa5ca9f0ab064c082813/html5/thumbnails/21.jpg)
Going back to Downsampling ...
When downsampling by a factor of two, the original image has frequenciesthat are too high
High frequencies are caused by sharp edges
How can we fix this?
[Adopted from: R. Urtasun]
Sanja Fidler CSC420: Intro to Image Understanding 15 / 35
![Page 22: Image Pyramids - Department of Computer Science ...fidler/slides/2019/CSC420/lecture5.pdf · Idea: Re-size Image Re-scale the image multiple times! Do correlation on every size! template](https://reader036.vdocuments.mx/reader036/viewer/2022062605/5fceaa5ca9f0ab064c082813/html5/thumbnails/22.jpg)
Gaussian pre-filtering
Solution: Blur the image via Gaussian, then subsample. Very simple!
!"#$%
!"###$#&%
'#!'()*"+% !"#$% '#!'()*"+% ,%!%#
!%###$#&%
!&#!"#
[Source: N. Snavely]Sanja Fidler CSC420: Intro to Image Understanding 16 / 35
![Page 23: Image Pyramids - Department of Computer Science ...fidler/slides/2019/CSC420/lecture5.pdf · Idea: Re-size Image Re-scale the image multiple times! Do correlation on every size! template](https://reader036.vdocuments.mx/reader036/viewer/2022062605/5fceaa5ca9f0ab064c082813/html5/thumbnails/23.jpg)
Subsampling with Gaussian pre-filtering
G 1/4 G 1/8 Gaussian 1/2
[Source: S. Seitz]
Sanja Fidler CSC420: Intro to Image Understanding 17 / 35
![Page 24: Image Pyramids - Department of Computer Science ...fidler/slides/2019/CSC420/lecture5.pdf · Idea: Re-size Image Re-scale the image multiple times! Do correlation on every size! template](https://reader036.vdocuments.mx/reader036/viewer/2022062605/5fceaa5ca9f0ab064c082813/html5/thumbnails/24.jpg)
Compare with ...
1/4 (2x zoom) 1/8 (4x zoom) 1/2
[Source: S. Seitz]
Sanja Fidler CSC420: Intro to Image Understanding 18 / 35
![Page 25: Image Pyramids - Department of Computer Science ...fidler/slides/2019/CSC420/lecture5.pdf · Idea: Re-size Image Re-scale the image multiple times! Do correlation on every size! template](https://reader036.vdocuments.mx/reader036/viewer/2022062605/5fceaa5ca9f0ab064c082813/html5/thumbnails/25.jpg)
Where is the Rectangle?
My image
Figure: Dashed line denotes the border of the image (it’s not part of the image)
Sanja Fidler CSC420: Intro to Image Understanding 19 / 35
![Page 26: Image Pyramids - Department of Computer Science ...fidler/slides/2019/CSC420/lecture5.pdf · Idea: Re-size Image Re-scale the image multiple times! Do correlation on every size! template](https://reader036.vdocuments.mx/reader036/viewer/2022062605/5fceaa5ca9f0ab064c082813/html5/thumbnails/26.jpg)
Where is the Rectangle?
My image
Let’s blur
Figure: Dashed line denotes the border of the image (it’s not part of the image)
Sanja Fidler CSC420: Intro to Image Understanding 19 / 35
![Page 27: Image Pyramids - Department of Computer Science ...fidler/slides/2019/CSC420/lecture5.pdf · Idea: Re-size Image Re-scale the image multiple times! Do correlation on every size! template](https://reader036.vdocuments.mx/reader036/viewer/2022062605/5fceaa5ca9f0ab064c082813/html5/thumbnails/27.jpg)
Where is the Rectangle?
My image
Let’s blur
And now take every other row and column
Figure: Dashed line denotes the border of the image (it’s not part of the image)
Sanja Fidler CSC420: Intro to Image Understanding 19 / 35
![Page 28: Image Pyramids - Department of Computer Science ...fidler/slides/2019/CSC420/lecture5.pdf · Idea: Re-size Image Re-scale the image multiple times! Do correlation on every size! template](https://reader036.vdocuments.mx/reader036/viewer/2022062605/5fceaa5ca9f0ab064c082813/html5/thumbnails/28.jpg)
Where is the Chicken?
My image
Sanja Fidler CSC420: Intro to Image Understanding 20 / 35
![Page 29: Image Pyramids - Department of Computer Science ...fidler/slides/2019/CSC420/lecture5.pdf · Idea: Re-size Image Re-scale the image multiple times! Do correlation on every size! template](https://reader036.vdocuments.mx/reader036/viewer/2022062605/5fceaa5ca9f0ab064c082813/html5/thumbnails/29.jpg)
Where is the Chicken?
My image
Let’s blur
Sanja Fidler CSC420: Intro to Image Understanding 20 / 35
![Page 30: Image Pyramids - Department of Computer Science ...fidler/slides/2019/CSC420/lecture5.pdf · Idea: Re-size Image Re-scale the image multiple times! Do correlation on every size! template](https://reader036.vdocuments.mx/reader036/viewer/2022062605/5fceaa5ca9f0ab064c082813/html5/thumbnails/30.jpg)
Where is the Chicken?
My image
Let’s blur
And now take every other column
Sanja Fidler CSC420: Intro to Image Understanding 20 / 35
![Page 31: Image Pyramids - Department of Computer Science ...fidler/slides/2019/CSC420/lecture5.pdf · Idea: Re-size Image Re-scale the image multiple times! Do correlation on every size! template](https://reader036.vdocuments.mx/reader036/viewer/2022062605/5fceaa5ca9f0ab064c082813/html5/thumbnails/31.jpg)
Gaussian Pyramids [Burt and Adelson, 1983]
A sequence of images created with Gaussian blurring and downsampling iscalled a Gaussian Pyramid
In computer graphics, a mip map [Williams, 1983]
How much space does a Gaussian pyramid take compared to original image?
[Source: S. Seitz]Sanja Fidler CSC420: Intro to Image Understanding 21 / 35
![Page 32: Image Pyramids - Department of Computer Science ...fidler/slides/2019/CSC420/lecture5.pdf · Idea: Re-size Image Re-scale the image multiple times! Do correlation on every size! template](https://reader036.vdocuments.mx/reader036/viewer/2022062605/5fceaa5ca9f0ab064c082813/html5/thumbnails/32.jpg)
Gaussian Pyramids [Burt and Adelson, 1983]
A sequence of images created with Gaussian blurring and downsampling iscalled a Gaussian Pyramid
In computer graphics, a mip map [Williams, 1983]
How much space does a Gaussian pyramid take compared to original image?
[Source: S. Seitz]Sanja Fidler CSC420: Intro to Image Understanding 21 / 35
![Page 33: Image Pyramids - Department of Computer Science ...fidler/slides/2019/CSC420/lecture5.pdf · Idea: Re-size Image Re-scale the image multiple times! Do correlation on every size! template](https://reader036.vdocuments.mx/reader036/viewer/2022062605/5fceaa5ca9f0ab064c082813/html5/thumbnails/33.jpg)
Example of Gaussian Pyramid
[Source: N. Snavely]
Sanja Fidler CSC420: Intro to Image Understanding 22 / 35
![Page 34: Image Pyramids - Department of Computer Science ...fidler/slides/2019/CSC420/lecture5.pdf · Idea: Re-size Image Re-scale the image multiple times! Do correlation on every size! template](https://reader036.vdocuments.mx/reader036/viewer/2022062605/5fceaa5ca9f0ab064c082813/html5/thumbnails/34.jpg)
Image Up-Sampling
This image is too small, how can we make it 10 times as big?
[Source: N. Snavely, R. Urtasun]
Sanja Fidler CSC420: Intro to Image Understanding 23 / 35
![Page 35: Image Pyramids - Department of Computer Science ...fidler/slides/2019/CSC420/lecture5.pdf · Idea: Re-size Image Re-scale the image multiple times! Do correlation on every size! template](https://reader036.vdocuments.mx/reader036/viewer/2022062605/5fceaa5ca9f0ab064c082813/html5/thumbnails/35.jpg)
Image Up-Sampling
This image is too small, how can we make it 10 times as big?
Simplest approach: repeat each row and column 10 times
[Source: N. Snavely, R. Urtasun]
Sanja Fidler CSC420: Intro to Image Understanding 23 / 35
![Page 36: Image Pyramids - Department of Computer Science ...fidler/slides/2019/CSC420/lecture5.pdf · Idea: Re-size Image Re-scale the image multiple times! Do correlation on every size! template](https://reader036.vdocuments.mx/reader036/viewer/2022062605/5fceaa5ca9f0ab064c082813/html5/thumbnails/36.jpg)
Interpolation
!" #" $" %" &"
d = 1 in this example
Recall how a digital image is formed
F [x , y ] = quantize{f (xd , yd)}
It is a discrete point-sampling of a continuous function
If we could somehow reconstruct the original function, any new image couldbe generated, at any resolution and scale
[Source: N. Snavely, S. Seitz]Sanja Fidler CSC420: Intro to Image Understanding 24 / 35
![Page 37: Image Pyramids - Department of Computer Science ...fidler/slides/2019/CSC420/lecture5.pdf · Idea: Re-size Image Re-scale the image multiple times! Do correlation on every size! template](https://reader036.vdocuments.mx/reader036/viewer/2022062605/5fceaa5ca9f0ab064c082813/html5/thumbnails/37.jpg)
Interpolation
!" #" $" %" &"
d = 1 in this example
Recall how a digital image is formed
F [x , y ] = quantize{f (xd , yd)}
It is a discrete point-sampling of a continuous function
If we could somehow reconstruct the original function, any new image couldbe generated, at any resolution and scale
[Source: N. Snavely, S. Seitz]Sanja Fidler CSC420: Intro to Image Understanding 24 / 35
![Page 38: Image Pyramids - Department of Computer Science ...fidler/slides/2019/CSC420/lecture5.pdf · Idea: Re-size Image Re-scale the image multiple times! Do correlation on every size! template](https://reader036.vdocuments.mx/reader036/viewer/2022062605/5fceaa5ca9f0ab064c082813/html5/thumbnails/38.jpg)
Interpolation
!" #" $" %" &"
d = 1 in this example
Recall how a digital image is formed
F [x , y ] = quantize{f (xd , yd)}
It is a discrete point-sampling of a continuous function
If we could somehow reconstruct the original function, any new image couldbe generated, at any resolution and scale
[Source: N. Snavely, S. Seitz]Sanja Fidler CSC420: Intro to Image Understanding 24 / 35
![Page 39: Image Pyramids - Department of Computer Science ...fidler/slides/2019/CSC420/lecture5.pdf · Idea: Re-size Image Re-scale the image multiple times! Do correlation on every size! template](https://reader036.vdocuments.mx/reader036/viewer/2022062605/5fceaa5ca9f0ab064c082813/html5/thumbnails/39.jpg)
Interpolation
What if we don’t know f ?
[Source: N. Snavely, S. Seitz]
Sanja Fidler CSC420: Intro to Image Understanding 25 / 35
![Page 40: Image Pyramids - Department of Computer Science ...fidler/slides/2019/CSC420/lecture5.pdf · Idea: Re-size Image Re-scale the image multiple times! Do correlation on every size! template](https://reader036.vdocuments.mx/reader036/viewer/2022062605/5fceaa5ca9f0ab064c082813/html5/thumbnails/40.jpg)
Interpolation
What if we don’t know f ?
Guess an approximation: for example nearest-neighbor
[Source: N. Snavely, S. Seitz]
Sanja Fidler CSC420: Intro to Image Understanding 25 / 35
![Page 41: Image Pyramids - Department of Computer Science ...fidler/slides/2019/CSC420/lecture5.pdf · Idea: Re-size Image Re-scale the image multiple times! Do correlation on every size! template](https://reader036.vdocuments.mx/reader036/viewer/2022062605/5fceaa5ca9f0ab064c082813/html5/thumbnails/41.jpg)
Interpolation
What if we don’t know f ?
Guess an approximation: for example nearest-neighbor
Guess an approximation: for example linear
[Source: N. Snavely, S. Seitz]
Sanja Fidler CSC420: Intro to Image Understanding 25 / 35
![Page 42: Image Pyramids - Department of Computer Science ...fidler/slides/2019/CSC420/lecture5.pdf · Idea: Re-size Image Re-scale the image multiple times! Do correlation on every size! template](https://reader036.vdocuments.mx/reader036/viewer/2022062605/5fceaa5ca9f0ab064c082813/html5/thumbnails/42.jpg)
Interpolation
What if we don’t know f ?
Guess an approximation: for example nearest-neighbor
Guess an approximation: for example linear
More complex approximations: cubic, B-splines
[Source: N. Snavely, S. Seitz]
Sanja Fidler CSC420: Intro to Image Understanding 25 / 35
![Page 43: Image Pyramids - Department of Computer Science ...fidler/slides/2019/CSC420/lecture5.pdf · Idea: Re-size Image Re-scale the image multiple times! Do correlation on every size! template](https://reader036.vdocuments.mx/reader036/viewer/2022062605/5fceaa5ca9f0ab064c082813/html5/thumbnails/43.jpg)
Linear Interpolation
Linear interpolation:
G (x) =x2 − x
x2 − x1F (x1) +
x − x1x2 − x1
F (x2)
Sanja Fidler CSC420: Intro to Image Understanding 26 / 35
![Page 44: Image Pyramids - Department of Computer Science ...fidler/slides/2019/CSC420/lecture5.pdf · Idea: Re-size Image Re-scale the image multiple times! Do correlation on every size! template](https://reader036.vdocuments.mx/reader036/viewer/2022062605/5fceaa5ca9f0ab064c082813/html5/thumbnails/44.jpg)
Interpolation: 1D Example
Let’s make this signal triple length
Sanja Fidler CSC420: Intro to Image Understanding 27 / 35
![Page 45: Image Pyramids - Department of Computer Science ...fidler/slides/2019/CSC420/lecture5.pdf · Idea: Re-size Image Re-scale the image multiple times! Do correlation on every size! template](https://reader036.vdocuments.mx/reader036/viewer/2022062605/5fceaa5ca9f0ab064c082813/html5/thumbnails/45.jpg)
Interpolation: 1D Example
Let’s make this signal triple length (d = 3)
Sanja Fidler CSC420: Intro to Image Understanding 27 / 35
![Page 46: Image Pyramids - Department of Computer Science ...fidler/slides/2019/CSC420/lecture5.pdf · Idea: Re-size Image Re-scale the image multiple times! Do correlation on every size! template](https://reader036.vdocuments.mx/reader036/viewer/2022062605/5fceaa5ca9f0ab064c082813/html5/thumbnails/46.jpg)
Interpolation: 1D Example
Let’s make this signal triple length (d = 3)
If i/d is an integer, just copy from the signal
Sanja Fidler CSC420: Intro to Image Understanding 27 / 35
![Page 47: Image Pyramids - Department of Computer Science ...fidler/slides/2019/CSC420/lecture5.pdf · Idea: Re-size Image Re-scale the image multiple times! Do correlation on every size! template](https://reader036.vdocuments.mx/reader036/viewer/2022062605/5fceaa5ca9f0ab064c082813/html5/thumbnails/47.jpg)
Interpolation: 1D Example
Let’s make this signal triple length (d = 3)
If i/d is an integer, just copy from the signal
Otherwise use the interpolation formula
Sanja Fidler CSC420: Intro to Image Understanding 27 / 35
![Page 48: Image Pyramids - Department of Computer Science ...fidler/slides/2019/CSC420/lecture5.pdf · Idea: Re-size Image Re-scale the image multiple times! Do correlation on every size! template](https://reader036.vdocuments.mx/reader036/viewer/2022062605/5fceaa5ca9f0ab064c082813/html5/thumbnails/48.jpg)
Linear Interpolation via Convolution
Linear interpolation:
G (x) =x2 − x
x2 − x1F (x1) +
x − x1x2 − x1
F (x2)
With t = x − x1 and d = x2 − x1 we can get:
G (x) =d − t
dF (x − t) +
t
dF (x + d − t)
( Kind of looks like convolution: G (x) =∑
t h(t)F (x − t) ) )
Sanja Fidler CSC420: Intro to Image Understanding 28 / 35
![Page 49: Image Pyramids - Department of Computer Science ...fidler/slides/2019/CSC420/lecture5.pdf · Idea: Re-size Image Re-scale the image multiple times! Do correlation on every size! template](https://reader036.vdocuments.mx/reader036/viewer/2022062605/5fceaa5ca9f0ab064c082813/html5/thumbnails/49.jpg)
Linear Interpolation via Convolution
Linear interpolation:
G (x) =x2 − x
x2 − x1F (x1) +
x − x1x2 − x1
F (x2)
With t = x − x1 and d = x2 − x1 we can get:
G (x) =d − t
dF (x − t) +
t
dF (x + d − t)
( Kind of looks like convolution: G (x) =∑
t h(t)F (x − t) ) )
Sanja Fidler CSC420: Intro to Image Understanding 28 / 35
![Page 50: Image Pyramids - Department of Computer Science ...fidler/slides/2019/CSC420/lecture5.pdf · Idea: Re-size Image Re-scale the image multiple times! Do correlation on every size! template](https://reader036.vdocuments.mx/reader036/viewer/2022062605/5fceaa5ca9f0ab064c082813/html5/thumbnails/50.jpg)
Interpolation via Convolution: 1D Example
Let’s make this signal triple length
Sanja Fidler CSC420: Intro to Image Understanding 29 / 35
![Page 51: Image Pyramids - Department of Computer Science ...fidler/slides/2019/CSC420/lecture5.pdf · Idea: Re-size Image Re-scale the image multiple times! Do correlation on every size! template](https://reader036.vdocuments.mx/reader036/viewer/2022062605/5fceaa5ca9f0ab064c082813/html5/thumbnails/51.jpg)
Interpolation via Convolution: 1D Example
Let’s make this signal triple length (d = 3)
Sanja Fidler CSC420: Intro to Image Understanding 29 / 35
![Page 52: Image Pyramids - Department of Computer Science ...fidler/slides/2019/CSC420/lecture5.pdf · Idea: Re-size Image Re-scale the image multiple times! Do correlation on every size! template](https://reader036.vdocuments.mx/reader036/viewer/2022062605/5fceaa5ca9f0ab064c082813/html5/thumbnails/52.jpg)
Interpolation via Convolution: 1D Example
Let’s make this signal triple length (d = 3)
What should be my “reconstruction” filter h (such that G = h ∗ G ′)?
Sanja Fidler CSC420: Intro to Image Understanding 29 / 35
![Page 53: Image Pyramids - Department of Computer Science ...fidler/slides/2019/CSC420/lecture5.pdf · Idea: Re-size Image Re-scale the image multiple times! Do correlation on every size! template](https://reader036.vdocuments.mx/reader036/viewer/2022062605/5fceaa5ca9f0ab064c082813/html5/thumbnails/53.jpg)
Interpolation via Convolution: 1D Example
Let’s make this signal triple length (d = 3)
What should be my “reconstruction” filter h (such that G = h ∗ G ′)?
h = [0, 1d , . . . ,
d−1d , 1, d−1d , . . . , 1
d , 0], where d my upsampling factor
Sanja Fidler CSC420: Intro to Image Understanding 29 / 35
![Page 54: Image Pyramids - Department of Computer Science ...fidler/slides/2019/CSC420/lecture5.pdf · Idea: Re-size Image Re-scale the image multiple times! Do correlation on every size! template](https://reader036.vdocuments.mx/reader036/viewer/2022062605/5fceaa5ca9f0ab064c082813/html5/thumbnails/54.jpg)
Interpolation via Convolution: 1D Example
Let’s make this signal triple length (d = 3)
What should be my “reconstruction” filter h (such that G = h ∗ G ′)?
h = [0, 1d , . . . ,
d−1d , 1, d−1d , . . . , 1
d , 0], where d my upsampling factor
Sanja Fidler CSC420: Intro to Image Understanding 29 / 35
![Page 55: Image Pyramids - Department of Computer Science ...fidler/slides/2019/CSC420/lecture5.pdf · Idea: Re-size Image Re-scale the image multiple times! Do correlation on every size! template](https://reader036.vdocuments.mx/reader036/viewer/2022062605/5fceaa5ca9f0ab064c082813/html5/thumbnails/55.jpg)
Interpolation via Convolution: 1D Example
Let’s make this signal triple length (d = 3)
What should be my “reconstruction” filter h (such that G = h ∗ G ′)?
h = [0, 1d , . . . ,
d−1d , 1, d−1d , . . . , 1
d , 0], where d my upsampling factor
Sanja Fidler CSC420: Intro to Image Understanding 29 / 35
![Page 56: Image Pyramids - Department of Computer Science ...fidler/slides/2019/CSC420/lecture5.pdf · Idea: Re-size Image Re-scale the image multiple times! Do correlation on every size! template](https://reader036.vdocuments.mx/reader036/viewer/2022062605/5fceaa5ca9f0ab064c082813/html5/thumbnails/56.jpg)
Interpolation via Convolution: 1D Example
Let’s make this signal triple length (d = 3)
What should be my “reconstruction” filter h (such that G = h ∗ G ′)?
h = [0, 1d , . . . ,
d−1d , 1, d−1d , . . . , 1
d , 0], where d my upsampling factor
Sanja Fidler CSC420: Intro to Image Understanding 29 / 35
![Page 57: Image Pyramids - Department of Computer Science ...fidler/slides/2019/CSC420/lecture5.pdf · Idea: Re-size Image Re-scale the image multiple times! Do correlation on every size! template](https://reader036.vdocuments.mx/reader036/viewer/2022062605/5fceaa5ca9f0ab064c082813/html5/thumbnails/57.jpg)
Interpolation via Convolution (1D)
!"#$%&'()$*+,-.)/*0+,(
1$%)$-.2,$3456+)(3,.$)7+&%0+,(
83,$%)(3,.$)7+&%0+,(
9%/--3%,()$*+,-.)/*0+,(
:+/)*$;(<=(>/)&$--(
Sanja Fidler CSC420: Intro to Image Understanding 30 / 35
![Page 58: Image Pyramids - Department of Computer Science ...fidler/slides/2019/CSC420/lecture5.pdf · Idea: Re-size Image Re-scale the image multiple times! Do correlation on every size! template](https://reader036.vdocuments.mx/reader036/viewer/2022062605/5fceaa5ca9f0ab064c082813/html5/thumbnails/58.jpg)
Image Interpolation (2D)
Let’s make this image triple size
Copy image in every third pixel. What about the remaining pixels in G?
Sanja Fidler CSC420: Intro to Image Understanding 31 / 35
![Page 59: Image Pyramids - Department of Computer Science ...fidler/slides/2019/CSC420/lecture5.pdf · Idea: Re-size Image Re-scale the image multiple times! Do correlation on every size! template](https://reader036.vdocuments.mx/reader036/viewer/2022062605/5fceaa5ca9f0ab064c082813/html5/thumbnails/59.jpg)
Image Interpolation (2D)
Let’s make this image triple size
Copy image in every third pixel. What about the remaining pixels in G?
How shall we compute this value?
Sanja Fidler CSC420: Intro to Image Understanding 31 / 35
![Page 60: Image Pyramids - Department of Computer Science ...fidler/slides/2019/CSC420/lecture5.pdf · Idea: Re-size Image Re-scale the image multiple times! Do correlation on every size! template](https://reader036.vdocuments.mx/reader036/viewer/2022062605/5fceaa5ca9f0ab064c082813/html5/thumbnails/60.jpg)
Image Interpolation (2D)
Let’s make this image triple size
Copy image in every third pixel. What about the remaining pixels in G?
One possible way: nearest neighbor interpolation
Sanja Fidler CSC420: Intro to Image Understanding 31 / 35
![Page 61: Image Pyramids - Department of Computer Science ...fidler/slides/2019/CSC420/lecture5.pdf · Idea: Re-size Image Re-scale the image multiple times! Do correlation on every size! template](https://reader036.vdocuments.mx/reader036/viewer/2022062605/5fceaa5ca9f0ab064c082813/html5/thumbnails/61.jpg)
Image Interpolation (2D)
Let’s make this image triple size
Copy image in every third pixel. What about the remaining pixels in G?
Better: bilinear interpolation (check out details:http://en.wikipedia.org/wiki/Bilinear_interpolation)
Sanja Fidler CSC420: Intro to Image Understanding 31 / 35
![Page 62: Image Pyramids - Department of Computer Science ...fidler/slides/2019/CSC420/lecture5.pdf · Idea: Re-size Image Re-scale the image multiple times! Do correlation on every size! template](https://reader036.vdocuments.mx/reader036/viewer/2022062605/5fceaa5ca9f0ab064c082813/html5/thumbnails/62.jpg)
Reconstruction Filters
What does the 2D version of this hat function look like?
!"#$%#&'(()*+",#(*+-"#!%),.%+(
/-"+-($0+1.%+2(!"#$%#&'((!"#"$%&'("$)%'*+#&,+$(
Sanja Fidler CSC420: Intro to Image Understanding 32 / 35
![Page 63: Image Pyramids - Department of Computer Science ...fidler/slides/2019/CSC420/lecture5.pdf · Idea: Re-size Image Re-scale the image multiple times! Do correlation on every size! template](https://reader036.vdocuments.mx/reader036/viewer/2022062605/5fceaa5ca9f0ab064c082813/html5/thumbnails/63.jpg)
Reconstruction Filters
What does the 2D version of this hat function look like?
!"#$%#&'(()*+",#(*+-"#!%),.%+(
/-"+-($0+1.%+2(!"#$%#&'((!"#"$%&'("$)%'*+#&,+$(
And filter for nearest neighbor interpolation?
Sanja Fidler CSC420: Intro to Image Understanding 32 / 35
![Page 64: Image Pyramids - Department of Computer Science ...fidler/slides/2019/CSC420/lecture5.pdf · Idea: Re-size Image Re-scale the image multiple times! Do correlation on every size! template](https://reader036.vdocuments.mx/reader036/viewer/2022062605/5fceaa5ca9f0ab064c082813/html5/thumbnails/64.jpg)
Reconstruction Filters
What does the 2D version of this hat function look like?
!"#$%#&'(()*+",#(*+-"#!%),.%+(
/-"+-($0+1.%+2(!"#$%#&'((!"#"$%&'("$)%'*+#&,+$(
And filter for nearest neighbor interpolation?
Sanja Fidler CSC420: Intro to Image Understanding 32 / 35
![Page 65: Image Pyramids - Department of Computer Science ...fidler/slides/2019/CSC420/lecture5.pdf · Idea: Re-size Image Re-scale the image multiple times! Do correlation on every size! template](https://reader036.vdocuments.mx/reader036/viewer/2022062605/5fceaa5ca9f0ab064c082813/html5/thumbnails/65.jpg)
Reconstruction Filters
What does the 2D version of this hat function look like?
!"#$%#&'(()*+",#(*+-"#!%),.%+(
/-"+-($0+1.%+2(!"#$%#&'((!"#"$%&'("$)%'*+#&,+$(
Better filters give better resampled images: Bicubic is a common choice
Sanja Fidler CSC420: Intro to Image Understanding 32 / 35
![Page 66: Image Pyramids - Department of Computer Science ...fidler/slides/2019/CSC420/lecture5.pdf · Idea: Re-size Image Re-scale the image multiple times! Do correlation on every size! template](https://reader036.vdocuments.mx/reader036/viewer/2022062605/5fceaa5ca9f0ab064c082813/html5/thumbnails/66.jpg)
Image Interpolation via Convolution (2D)
Let’s make this image triple size: copy image values in every third pixel,place zeros everywhere else
Sanja Fidler CSC420: Intro to Image Understanding 33 / 35
![Page 67: Image Pyramids - Department of Computer Science ...fidler/slides/2019/CSC420/lecture5.pdf · Idea: Re-size Image Re-scale the image multiple times! Do correlation on every size! template](https://reader036.vdocuments.mx/reader036/viewer/2022062605/5fceaa5ca9f0ab064c082813/html5/thumbnails/67.jpg)
Image Interpolation via Convolution (2D)
Let’s make this image triple size: copy image values in every third pixel,place zeros everywhere else
Convolution with a reconstruction filter (e.g., bilinear) and you get theinterpolated image
Sanja Fidler CSC420: Intro to Image Understanding 33 / 35
![Page 68: Image Pyramids - Department of Computer Science ...fidler/slides/2019/CSC420/lecture5.pdf · Idea: Re-size Image Re-scale the image multiple times! Do correlation on every size! template](https://reader036.vdocuments.mx/reader036/viewer/2022062605/5fceaa5ca9f0ab064c082813/html5/thumbnails/68.jpg)
Image Interpolation
Original image
Interpolation results
!"#$"%&'(")*+,-$.)(&"$/-0#1-(. 2)0)("#$.)(&"$/-0#1-(. 2)34,)3.)(&"$/-0#1-(.
[Source: N. Snavely]
Sanja Fidler CSC420: Intro to Image Understanding 34 / 35
![Page 69: Image Pyramids - Department of Computer Science ...fidler/slides/2019/CSC420/lecture5.pdf · Idea: Re-size Image Re-scale the image multiple times! Do correlation on every size! template](https://reader036.vdocuments.mx/reader036/viewer/2022062605/5fceaa5ca9f0ab064c082813/html5/thumbnails/69.jpg)
Summary – Stuff You Should Know
To down-scale an image: blur it with a small Gaussian (e.g., σ = 1.4) anddownsample
To up-scale an image: interpolation (nearest neighbor, bilinear, bicubic, etc)
Gaussian pyramid: Blur with Gaussian filter, downsample result by factor 2,blur it with the Gaussian, downsample by 2...
Matlab functions:
imresize(image, scale, method): Matlab’s function for resizingthe image, where method=“nearest”, “bilinear”, “bicubic” (worksfor downsampling and upsampling)
skimage.transform.resize andskimage.transform.rescale: Python’s function for resizing,where Order is in the range 0-5 with the following semantics: 0:Nearest-neighbor 1: Bi-linear (default) 2: Bi-quadratic 3: Bi-cubic
Sanja Fidler CSC420: Intro to Image Understanding 35 / 35