qi shan, brian curless, and todayoshi kohno university of ...€¦ · qi shan, brian curless, and...
TRANSCRIPT
![Page 1: Qi Shan, Brian Curless, and Todayoshi Kohno University of ...€¦ · Qi Shan, Brian Curless, and Todayoshi Kohno University of Washington . What is obscure glass? Obscure glass:](https://reader034.vdocuments.mx/reader034/viewer/2022050503/5f958258291b6a7c735b45b2/html5/thumbnails/1.jpg)
Seeing through Obscure
Glass
Qi Shan, Brian Curless, and Todayoshi Kohno
University of Washington
![Page 2: Qi Shan, Brian Curless, and Todayoshi Kohno University of ...€¦ · Qi Shan, Brian Curless, and Todayoshi Kohno University of Washington . What is obscure glass? Obscure glass:](https://reader034.vdocuments.mx/reader034/viewer/2022050503/5f958258291b6a7c735b45b2/html5/thumbnails/2.jpg)
What is obscure glass?
Obscure glass:
glass with texture relief
Refraction distorts and blurs
the background.
![Page 3: Qi Shan, Brian Curless, and Todayoshi Kohno University of ...€¦ · Qi Shan, Brian Curless, and Todayoshi Kohno University of Washington . What is obscure glass? Obscure glass:](https://reader034.vdocuments.mx/reader034/viewer/2022050503/5f958258291b6a7c735b45b2/html5/thumbnails/3.jpg)
Examples of obscure glass
![Page 4: Qi Shan, Brian Curless, and Todayoshi Kohno University of ...€¦ · Qi Shan, Brian Curless, and Todayoshi Kohno University of Washington . What is obscure glass? Obscure glass:](https://reader034.vdocuments.mx/reader034/viewer/2022050503/5f958258291b6a7c735b45b2/html5/thumbnails/4.jpg)
Examples of obscure glass
![Page 5: Qi Shan, Brian Curless, and Todayoshi Kohno University of ...€¦ · Qi Shan, Brian Curless, and Todayoshi Kohno University of Washington . What is obscure glass? Obscure glass:](https://reader034.vdocuments.mx/reader034/viewer/2022050503/5f958258291b6a7c735b45b2/html5/thumbnails/5.jpg)
Examples of obscure glass
![Page 6: Qi Shan, Brian Curless, and Todayoshi Kohno University of ...€¦ · Qi Shan, Brian Curless, and Todayoshi Kohno University of Washington . What is obscure glass? Obscure glass:](https://reader034.vdocuments.mx/reader034/viewer/2022050503/5f958258291b6a7c735b45b2/html5/thumbnails/6.jpg)
Examples of obscure glass
![Page 7: Qi Shan, Brian Curless, and Todayoshi Kohno University of ...€¦ · Qi Shan, Brian Curless, and Todayoshi Kohno University of Washington . What is obscure glass? Obscure glass:](https://reader034.vdocuments.mx/reader034/viewer/2022050503/5f958258291b6a7c735b45b2/html5/thumbnails/7.jpg)
Camera
![Page 8: Qi Shan, Brian Curless, and Todayoshi Kohno University of ...€¦ · Qi Shan, Brian Curless, and Todayoshi Kohno University of Washington . What is obscure glass? Obscure glass:](https://reader034.vdocuments.mx/reader034/viewer/2022050503/5f958258291b6a7c735b45b2/html5/thumbnails/8.jpg)
Camera
![Page 9: Qi Shan, Brian Curless, and Todayoshi Kohno University of ...€¦ · Qi Shan, Brian Curless, and Todayoshi Kohno University of Washington . What is obscure glass? Obscure glass:](https://reader034.vdocuments.mx/reader034/viewer/2022050503/5f958258291b6a7c735b45b2/html5/thumbnails/9.jpg)
The goal of the project
* =
* = -1
Inverse
![Page 10: Qi Shan, Brian Curless, and Todayoshi Kohno University of ...€¦ · Qi Shan, Brian Curless, and Todayoshi Kohno University of Washington . What is obscure glass? Obscure glass:](https://reader034.vdocuments.mx/reader034/viewer/2022050503/5f958258291b6a7c735b45b2/html5/thumbnails/10.jpg)
Digital approach
Optical approach
-1
. ( )
![Page 11: Qi Shan, Brian Curless, and Todayoshi Kohno University of ...€¦ · Qi Shan, Brian Curless, and Todayoshi Kohno University of Washington . What is obscure glass? Obscure glass:](https://reader034.vdocuments.mx/reader034/viewer/2022050503/5f958258291b6a7c735b45b2/html5/thumbnails/11.jpg)
A linear form
I FL
. =
I F
Rewriting the formation process in a matrix-vector form
Lknown unknown unknown
= one pixel
.
![Page 12: Qi Shan, Brian Curless, and Todayoshi Kohno University of ...€¦ · Qi Shan, Brian Curless, and Todayoshi Kohno University of Washington . What is obscure glass? Obscure glass:](https://reader034.vdocuments.mx/reader034/viewer/2022050503/5f958258291b6a7c735b45b2/html5/thumbnails/12.jpg)
Digital approach
Calibration
Reconstruction
. =
-1
. ( )
Recover from a
single input?
known unknown known
known unknown known
known unknown unknown
![Page 13: Qi Shan, Brian Curless, and Todayoshi Kohno University of ...€¦ · Qi Shan, Brian Curless, and Todayoshi Kohno University of Washington . What is obscure glass? Obscure glass:](https://reader034.vdocuments.mx/reader034/viewer/2022050503/5f958258291b6a7c735b45b2/html5/thumbnails/13.jpg)
Calibration (obtaining F)
Assumption: have access to both sides of the glass.
Calibration process:
1. Place a set of known patterns at the back of the obscure glass;
2. Capture the images through the obscure glass
![Page 14: Qi Shan, Brian Curless, and Todayoshi Kohno University of ...€¦ · Qi Shan, Brian Curless, and Todayoshi Kohno University of Washington . What is obscure glass? Obscure glass:](https://reader034.vdocuments.mx/reader034/viewer/2022050503/5f958258291b6a7c735b45b2/html5/thumbnails/14.jpg)
The number of calibration patterns
For 50x50 kernels, a naïve algorithms needs more than
2,500 image pairs
We assume sparseness of F combined with a multi-
scale optimization
The number of required patterns
◦ 2,500 -> ~100
![Page 15: Qi Shan, Brian Curless, and Todayoshi Kohno University of ...€¦ · Qi Shan, Brian Curless, and Todayoshi Kohno University of Washington . What is obscure glass? Obscure glass:](https://reader034.vdocuments.mx/reader034/viewer/2022050503/5f958258291b6a7c735b45b2/html5/thumbnails/15.jpg)
Experimental procedure:
1) Position camera close to glass.
2) Capture calibration patterns through glass;
estimate F.
3) Remove, then approximately replace camera.
4) Take multiple shots from nearby viewpoints.
5) Compute a result image for each shot
6) Keep best image (nearest to original
viewpoint).
![Page 16: Qi Shan, Brian Curless, and Todayoshi Kohno University of ...€¦ · Qi Shan, Brian Curless, and Todayoshi Kohno University of Washington . What is obscure glass? Obscure glass:](https://reader034.vdocuments.mx/reader034/viewer/2022050503/5f958258291b6a7c735b45b2/html5/thumbnails/16.jpg)
Results
![Page 17: Qi Shan, Brian Curless, and Todayoshi Kohno University of ...€¦ · Qi Shan, Brian Curless, and Todayoshi Kohno University of Washington . What is obscure glass? Obscure glass:](https://reader034.vdocuments.mx/reader034/viewer/2022050503/5f958258291b6a7c735b45b2/html5/thumbnails/17.jpg)
Increase allowed size of kernels
![Page 18: Qi Shan, Brian Curless, and Todayoshi Kohno University of ...€¦ · Qi Shan, Brian Curless, and Todayoshi Kohno University of Washington . What is obscure glass? Obscure glass:](https://reader034.vdocuments.mx/reader034/viewer/2022050503/5f958258291b6a7c735b45b2/html5/thumbnails/18.jpg)
Digital approach
Optical approach
-1
. ( )
![Page 19: Qi Shan, Brian Curless, and Todayoshi Kohno University of ...€¦ · Qi Shan, Brian Curless, and Todayoshi Kohno University of Washington . What is obscure glass? Obscure glass:](https://reader034.vdocuments.mx/reader034/viewer/2022050503/5f958258291b6a7c735b45b2/html5/thumbnails/19.jpg)
Optical approach
![Page 20: Qi Shan, Brian Curless, and Todayoshi Kohno University of ...€¦ · Qi Shan, Brian Curless, and Todayoshi Kohno University of Washington . What is obscure glass? Obscure glass:](https://reader034.vdocuments.mx/reader034/viewer/2022050503/5f958258291b6a7c735b45b2/html5/thumbnails/20.jpg)
Optical approach
![Page 21: Qi Shan, Brian Curless, and Todayoshi Kohno University of ...€¦ · Qi Shan, Brian Curless, and Todayoshi Kohno University of Washington . What is obscure glass? Obscure glass:](https://reader034.vdocuments.mx/reader034/viewer/2022050503/5f958258291b6a7c735b45b2/html5/thumbnails/21.jpg)
![Page 22: Qi Shan, Brian Curless, and Todayoshi Kohno University of ...€¦ · Qi Shan, Brian Curless, and Todayoshi Kohno University of Washington . What is obscure glass? Obscure glass:](https://reader034.vdocuments.mx/reader034/viewer/2022050503/5f958258291b6a7c735b45b2/html5/thumbnails/22.jpg)
Snell’s Law
sin
sin
in out
out in
n
n
in
in
out
out
![Page 23: Qi Shan, Brian Curless, and Todayoshi Kohno University of ...€¦ · Qi Shan, Brian Curless, and Todayoshi Kohno University of Washington . What is obscure glass? Obscure glass:](https://reader034.vdocuments.mx/reader034/viewer/2022050503/5f958258291b6a7c735b45b2/html5/thumbnails/23.jpg)
Snell’s Law
sin,
sin
in outout in
out in
in out
nn n
n
in
out
in
out
![Page 24: Qi Shan, Brian Curless, and Todayoshi Kohno University of ...€¦ · Qi Shan, Brian Curless, and Todayoshi Kohno University of Washington . What is obscure glass? Obscure glass:](https://reader034.vdocuments.mx/reader034/viewer/2022050503/5f958258291b6a7c735b45b2/html5/thumbnails/24.jpg)
![Page 25: Qi Shan, Brian Curless, and Todayoshi Kohno University of ...€¦ · Qi Shan, Brian Curless, and Todayoshi Kohno University of Washington . What is obscure glass? Obscure glass:](https://reader034.vdocuments.mx/reader034/viewer/2022050503/5f958258291b6a7c735b45b2/html5/thumbnails/25.jpg)
![Page 26: Qi Shan, Brian Curless, and Todayoshi Kohno University of ...€¦ · Qi Shan, Brian Curless, and Todayoshi Kohno University of Washington . What is obscure glass? Obscure glass:](https://reader034.vdocuments.mx/reader034/viewer/2022050503/5f958258291b6a7c735b45b2/html5/thumbnails/26.jpg)
Limitations
Digital approach requires:
◦ sparse-kernel glass;
◦ access to both sides for a period of time;
◦ ability to re-position camera reasonably well
Applied substance requires access to the textured
side(s)
Placing substance and/or camera on glass reveals
the observer
![Page 27: Qi Shan, Brian Curless, and Todayoshi Kohno University of ...€¦ · Qi Shan, Brian Curless, and Todayoshi Kohno University of Washington . What is obscure glass? Obscure glass:](https://reader034.vdocuments.mx/reader034/viewer/2022050503/5f958258291b6a7c735b45b2/html5/thumbnails/27.jpg)
Digital approach
Optical approach
![Page 28: Qi Shan, Brian Curless, and Todayoshi Kohno University of ...€¦ · Qi Shan, Brian Curless, and Todayoshi Kohno University of Washington . What is obscure glass? Obscure glass:](https://reader034.vdocuments.mx/reader034/viewer/2022050503/5f958258291b6a7c735b45b2/html5/thumbnails/28.jpg)
Backup slides
![Page 29: Qi Shan, Brian Curless, and Todayoshi Kohno University of ...€¦ · Qi Shan, Brian Curless, and Todayoshi Kohno University of Washington . What is obscure glass? Obscure glass:](https://reader034.vdocuments.mx/reader034/viewer/2022050503/5f958258291b6a7c735b45b2/html5/thumbnails/29.jpg)
Natural image statistics
…
![Page 30: Qi Shan, Brian Curless, and Todayoshi Kohno University of ...€¦ · Qi Shan, Brian Curless, and Todayoshi Kohno University of Washington . What is obscure glass? Obscure glass:](https://reader034.vdocuments.mx/reader034/viewer/2022050503/5f958258291b6a7c735b45b2/html5/thumbnails/30.jpg)
Natural image statistics
…