renng thesis
TRANSCRIPT
-
7/30/2019 Renng Thesis
1/203
D I G I T A L L I G H T F I E L D P H O T O G R A P H Y
a dissertation
submitted to the department of computer science
and the committee on graduate studies
of stanford university
in partial fulfillment of the requirementsfor the degree of
doctor of philosophy
R N
J
-
7/30/2019 Renng Thesis
2/203
C R N
A R R
-
7/30/2019 Renng Thesis
3/203
I I , ,
D
P.
P H P A
I I , , D
P.
M L
I I , ,
D
P.
M H
A U C G S.
-
7/30/2019 Renng Thesis
4/203
-
7/30/2019 Renng Thesis
5/203
Acknowledgments
I P H, M
L M H , I
. P ,
, I
. I . M L
, ,
. M H
.
I .
I B W D N -
. D S.
I
. M B
, . G D
,
. A A M Y -
. K F
,
.
A . S
K W K I S S -
, T J C M, S P A R
A O A, M H Z I.
-
7/30/2019 Renng Thesis
6/203
I , I H G A G S-
G L - , J G .
T P C, B C, J F, K F, A E G,
J G, B H, B O, D O
.
A M R F . T
,
. A S B G
. I
S U S C
.
I S G L.
I K F,
. M K -
, I . J K
. I
: E C, M H, G H, B
M, K P, R R, P S R W. S
J O, M P B W
.
I , ,
, M B, S C, A C,
P F, N H, L H, J J, S M, E M, M
P, J R, M R, D W M W. S
N , ,
.
F, I , Y F, B L C K N,
. M ,
. T .
-
7/30/2019 Renng Thesis
7/203
'PS.BNBBOE1BQB
-
7/30/2019 Renng Thesis
8/203
-
7/30/2019 Renng Thesis
9/203
-
7/30/2019 Renng Thesis
10/203
contents
4 Digital Refocusing 49
. P W . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . I S A . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. T R P . . . . . . . . . . . . . . . . . . . . . . .
. T N P . . . . . . . . . . . . . . . . . . . . . . . . . .
. E P . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. S . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. P A . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5 Signal Processing Framework 79
. P W . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. O . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. P I F D . . . . . . . . . . . . . . . . .
.. G F S T . . . . . . . . . . . . . .
.. F S P T . . . . . . . . . . . . . . . . . . .
.. P E F L F . . . . . . . . . . . .
. B-L A R P . . . . . . . . . . . . . .
. F S D R . . . . . . . . . . . . . . . . . . . . . . . . .
. L F . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6 Selectable Refocusing Power 113
. S P G L F C . . . . . . . . . . .
. O F P L . . . . . . . . . . . . . . . . . . .
. E P C . . . . . . . . . . . . . . . . . . . . . . .
. E R- S . . . . . . . . . . . . . . . . . . . . .
7 Digital Correction of Lens Aberrations 131
. P W . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. N . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. V A R L F . . . . . . . . . . . . . . . .
. R O C . . . . . . . . . . . . . . . . . . . .
. D C A . . . . . . . . . . . . . . . . . . . . . . . . . .
-
7/30/2019 Renng Thesis
11/203
contents
. C R A P-C L . . . . . . . . . .
. S C P . . . . . . . . . . . . . . . . . . . . . . . . .. M I Q M . . . . . . . . . . . . . . . . . .
.. C A: C L . . . . . . . . . . . . . . . . . . .
.. C P A D L . . . . . . . . .
8 Conclusion 167
A Proofs 171
a. G F S T . . . . . . . . . . . . . . . . . . . . . . . .
a. F L F I T . . . . . . . . . . . . . . . . . . . . . . a. P F-D S L F . . . . . . . . . . . . . .
Bibliography 177
-
7/30/2019 Renng Thesis
12/203
-
7/30/2019 Renng Thesis
13/203
List of Figures
1 Introduction 1
. C . . . . . . . . . . . . . . .
. D . . . . . . . . . . . . . . . . . . . . . . . . .
. R . . . . . . . . . . .
. D . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2 Light Fields and Photographs 11
. P . . . . . . . . . .
. T . . . . . . . . . . . . . . . . . . . .
. P . . . . . . . . . . . . . . . . . P . . . . . . . . . . .
. - . . . . . . . . . . . . . . . . . . . . . . .
3 Recording a Photographs Light Field 23
. S . . . .
. O . . . . . . . . . . . . . . . . .
. R . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. C . . . .
. S- . . . . . . . . . . . . . . .
. E . . . . . . . . . . . . . . . . . .
. M . . . . . . . . . . . .
. G : - . . . . . . . . . . . . . . .
. G : . . . . . . . . . . . . . . . . .
-
7/30/2019 Renng Thesis
14/203
list of figures
. P . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. M . . . . . . . . . . . . . . . . . . . . . . . . S . . . . . . . . . . . . . . .
4 Digital Refocusing 49
. E . . . . . . . . . . . . . .
. S-- . . . . . . . . . . . . . . . . . . . . . . . . .
. A - -- . . . . . . . . . .
. C - . .
. I . . . . . . .
. E : . . . . . . .
. E : . . . . . .
. E . . . . . . . . . .
. R . . . . . . . . . . . . . . . . . . .
. L . . . . . . . . . . . . .
. F - . . . . . . . . . . . . . . . . . . . . . . . . . . .
. M . . . . . . . .
. T . . . . . . . . . . . . . . . . . . .
. H- . . . . . . . . . . . . . . . . . . . . . . . .
. E . . . . . . . . . . . . .
. D . . . . . . . . . . . . . . . . . . . .
. M . . . . . . . . . . . . . . . . . .
5 Signal Processing Framework 79
. F- . . . . . .
. F- . . . . . .
. R F . . . . . . . . . . . . . . . . . . . . P I O . . . . . . . . . . . . . . . . . . . . . . . . .
. C F S T . . . . . . . . . . . . . . . . . . . . . . . . . .
. G F S T . . . . . . . . . . . . . . . . . . . . . . . .
. F S P T . . . . . . . . . . . . . . . . . . . . . . . .
. F L F P T . . . . . . . . . . . . . . . . . . .
-
7/30/2019 Renng Thesis
15/203
-
7/30/2019 Renng Thesis
16/203
list of figures
. C . . . .
. C . . . . . . . . . . psf rms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. E . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. A - - C . . . . . . . . . . .
. S . . . . . . . . . . . . . . . . . . . . . . . . . . .
. H - psf . . . . . . . . . . . .
. mtf ( ) . . . . . . . .
. mtf ( ) . . . . . . . .
. R- . . . . . . . . . . . . . .
. D . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. P . . . . . . . . . . . . . . .
8 Conclusion 167
A Proofs 171
-
7/30/2019 Renng Thesis
17/203
1Introduction
T , -
. T -
. I
. I , -
. T sum total
. - ,
,
.
I , . W -
,
. I , -
. T .
H, I
- . I , ,
.
O
. T . T -
,
.
-
7/30/2019 Renng Thesis
18/203
chapter . introduction
1.1 The Focus Problem in Photography
F . I , P -
Charivari D -
[N ].
Y . Y
, .
Y ; ,
,
, .
F , -
. I ,
.
T
. A ,
. A
. A - , -
. T
.
T . I
depth o feld
. A ,
. T
: I ?O ,
F .a-c,
. O , ,
F .a-c.
-
7/30/2019 Renng Thesis
19/203
.. the focus problem in photography
(a): W /, /
(b): M /, /
(c): N /, /
F .: C . A /n /n .
- - ,
. H,
,
.
T
. T . A
, .T ( )
. I
,
.
-
7/30/2019 Renng Thesis
20/203
-
7/30/2019 Renng Thesis
21/203
.. trends in digital photography
. F ,
: , .
D , -
(mp) [A ].
G , . T
-
. F ,
, mp [K ].
W -, . S
mp . F ,
- (. )
- ( ) [A ], mp. T
. F, -
. S,
, -- (snr). T -
- , -
. T
-
. H, -
. W
.
A , , , . D
. T
-
. A rgb
(, ) ,
-
7/30/2019 Renng Thesis
22/203
chapter . introduction
(a) (b) (c)
F .: D .
F .a. I ,
, . D [R-
. ]
rgb , F .b c. T
-, - . O
:
[N M ] [D
M ]; [Y . ];
- -
[B . ]; - ;
. C
.
I , - -
. T ,
.
1.3 Digital Light Field Photography
M
. T
-
7/30/2019 Renng Thesis
23/203
-
7/30/2019 Renng Thesis
24/203
chapter . introduction
.
, .
T
. I -
. S,
, . F
-
, .
1.4 Dissertation Overview
Organizational Themes
T -
: -
. T
.
System Design: Optics and Algorithms T - . T
.
Mathematical Analysis o Perormance T -
. T
. T C - -
. T
F , - . T
.
Computer-Simulated Validation S - -
- . F, -
, -
-
7/30/2019 Renng Thesis
25/203
.. dissertation overview
.
A Prototype Camera and Experimental Validation T - , -
. T - -
. T
-- . I , I
-
.
Dissertation Roadmap
$IBQUFSPUBUJPO
$IBQUFSJHIUJFMEBNFSB
$IBQUFS
FGPDVTJOH
$IBQUFS
PVSJFSOBMZTJT
$IBQUFS
FTPMVUJPOSBEFPT$IBQUFS
FOTPSSFDUJPO
F .: R.
I
-
, -
, ,
.
F .
, -
. P
S . ., -
. C -
,
. C . T -
,
.
Chapter Descriptions
C .
C , ,
-
7/30/2019 Renng Thesis
26/203
chapter . introduction
. I .
C , . I -
.
C F . T
F- -.
C -,
.
C ,
.
C .
-
7/30/2019 Renng Thesis
27/203
2Light Fields and Photographs
P . F
, ,
. I
.
T
, light feld ,
ray-space. B
-, ,
. T
.
2.1 Previous Work
T .
A B [] L V. L
H []
G [] M,
M S [] A.
I ,
-
7/30/2019 Renng Thesis
28/203
chapter . light fields and photographs
, -,
. A B - ,
plenoptic unction d d
d . T
[A B ]. T - -
,
. T ra-
diance [P ],
( ).
T d light feld,
, L H [] G
. [] ( Lumigraph). T d d
-
, , , . I ,
, .
T epipolar volumes B, B
M [] .
O . F -
,
d , -
. T -
C W [],
MM B []. F , image-based ren-
dering ,
, .
S K [] ,
siggraph . T
F L plenus , .Light feld G [] R, M .
-
7/30/2019 Renng Thesis
29/203
.. the light field flowing into the camera
,
F .W , -
. T [L H ; G . ]
,
. O ,
[C . ] -
[M . ; W . ; C . ]. A
-
, [S .
; B . ; M . ]. T -
, . P-
-
.
2.2 The Light Field Flowing into the Camera
O insidethe camera body. W ,
. T
. I ,
. N, I
,
.
I F .,
:
. T -
.
I ray diagram F .,
u , x
-
7/30/2019 Renng Thesis
30/203
chapter . light fields and photographs
F .: P .
. L u , u
. I , x
. O d
(u, v) (x,y) . L
L(x,y, u, v), L(x, u) d .
T Cartesian ray-space diagram F . -
- . T (x, u) C -. I
- , F-
.. T - d . A
B [A B ] -
. L H []
-
7/30/2019 Renng Thesis
31/203
.. photograph formation
F .: T .
, .
2.3 Photograph Formation
I ,
. T -
, , ccd cmos -
,
. E
.
I ,
. F , ,
-
7/30/2019 Renng Thesis
32/203
chapter . light fields and photographs
F .: T .
,
. A cmos
[C W ],
. N,
,
F . .
F . .
T ( d) -
x , u .
O x ,
-.
I , - F . ,
-
7/30/2019 Renng Thesis
33/203
.. photograph formation
. T -
- . T x ,
u .
T , ,
x . L
,
-
F ..
F .
. I , x
. C
. F ,
. I
F .a, ,
. T
-. I , F .b
, .
T -
x . A
u , x . I -
x , u
x (F .a). T
x (F .b). T
, -
, x .
T
x , . I, F .
x ,
T , . I , - -.
-
7/30/2019 Renng Thesis
34/203
chapter . light fields and photographs
(a) (b)
F .: T x .
-
7/30/2019 Renng Thesis
35/203
.. imaging equations
. T :
, .
2.4 Imaging Equations
T d - ,
. H,
d .
,
. A
C ,
.
T , F ., -
. C
[S . ]:
EF(x,y) =1
F2
LF(x,y, u, v) cos
4 dudv, (.)
F , EF(x,y)
(x,y), LF
F, (x,y, u, v) . T cos4
- optical vignetting. I
. H, E .
cmos .
F , E . uv xy ,
L .
, cos4
, - L(x,y, u, v) := L(x,y, u, v) cos4.
-
7/30/2019 Renng Thesis
36/203
chapter . light fields and photographs
-FOTQMBOF
'JMNQMBOF
0
0
0
0
0
0
F .:
- .
T - -
. F - -
-
. S,
,
.
L
x
. A F ., -
,
-
-. I
F,
-
LF (x, u) LF(x, u)
E ..
T
x
u LF - x u LF.
B , u x,
x u+ (x u) FF . A d x u, y v . A ,
= F/F ,
LF(x,y, u, v) = LFu+ x u
, v+ y v
, u, v
= LF
u
1 1
+x
, v
1 1
+y
, u, v
. (.)
T d
. A
-
7/30/2019 Renng Thesis
37/203
.. imaging equations
, -
. I I . [] ,
[C . ; S
. ; D . ; V . ; N ]. T
[G B
].
C E . . (x,y)
F = F :
E(F)(x,y) = 12F2 LF
u
1 1
+ x
, v
1 1
+ y
, u, vdudv. (.)
T d
, d.
-
7/30/2019 Renng Thesis
38/203
-
7/30/2019 Renng Thesis
39/203
3Recording a Photographs Light Field
T
. A
, . T
. E ,
(, , F .).
T integral photography L [] I [; ].
. P
, ,
.
I ,
A W [] plenoptic camera. T
. S .
generalized light feld camera .
A - .
S . .
T ,
-
.
-
7/30/2019 Renng Thesis
40/203
chapter . recording a photographs light field
3.1 A Plenoptic Camera Records the Light Field
T , ,
, F .. T ,
. I ,
. I ,
,
.
A - F .. T - ,
. ,
. T
(x, u)
- . A , -
, . T
. C C . T
x .
E -
. F , F . , -
. A -
,
, F .. I -
F . b,
, .
T
( F .). I ,
,
. I
-
7/30/2019 Renng Thesis
41/203
.. a plenoptic camera records the light field
F .: S .
. I
, , . T
,
. I ,
u x ,
.
T - . F , ,
. O , -
,
. A ,
. F, S .
-
7/30/2019 Renng Thesis
42/203
chapter . recording a photographs light field
.
3.2 Computing Photographs from the Light Field
T -
. T -
. F , F .a
. A C , -, F . b.
, .
T -
. S, ,
,
, . T -
F .a. F . b -
. E - F . b. W
F . b. T
b . T
- .
T
, . H, -
. F ,
,
S . C ,
, C .
A
. T
, , C . H,
-
7/30/2019 Renng Thesis
43/203
-
7/30/2019 Renng Thesis
44/203
chapter . recording a photographs light field
(z) (z)
F .: R . T - .
-
7/30/2019 Renng Thesis
45/203
.. three views of the recorded light field
. A ,
. S ,
. T
.
T
. E d d
. M ,
.
Raw Light Field Photograph
T
, F .. M,
,
F .: C -
F ..
, -
. L , -
,
-. E -
. I
,
(x,y) ,
(u, v) .
T F .
:
(I z),
(I z). L I z,
. T
-
7/30/2019 Renng Thesis
46/203
chapter . recording a photographs light field
(z) (z)
F .: S- F .. I z z - , .
-
7/30/2019 Renng Thesis
47/203
-
7/30/2019 Renng Thesis
48/203
chapter . recording a photographs light field
(z) (z)
F .: E F .. I z z - , .
-
7/30/2019 Renng Thesis
49/203
-
7/30/2019 Renng Thesis
50/203
chapter . recording a photographs light field
3.4 Resolution Limits of the Plenoptic Camera
A ,
. T
, d ,
- .
Microlens Optics
T u . I ,
.
M
. T
. H,
(
),
. A ,
. T ,
. H,
, S ..
T -
, . W .
T
. T
- , F ..
P .
-
7/30/2019 Renng Thesis
51/203
-
7/30/2019 Renng Thesis
52/203
-
7/30/2019 Renng Thesis
53/203
.. generalizing the plenoptic camera
( . mp)
. A, , , . mp , .
3.5 Generalizing the Plenoptic Camera
A ,
,
. T
F . . T -
. T
.
O , , ,
-
. I ,
[A . ]. T ( ,
). I
.
N,
. F .
, -.
F .a ,
. T -
u x -.
F .b . T
- ,
. T
. I C ,
-
7/30/2019 Renng Thesis
54/203
chapter . recording a photographs light field
(a) (b) (c)
F .: G : .
. H -
,
. F , F .a
. T -
,
F .b c.
F .c -
. I
-
7/30/2019 Renng Thesis
55/203
-
7/30/2019 Renng Thesis
56/203
chapter . recording a photographs light field
,
, , , , .
O --
. T . I
. A
. M
. A ,
.
T --
, , . T ,
(.. mp) . I ,
(.. mp)
, -
. A ,
vlsi ,
.
Components
T -
. M
. T - , , ,
F .b.
O M fb. T K
kaf-ce , . . F - C . W
-
7/30/2019 Renng Thesis
57/203
-
7/30/2019 Renng Thesis
58/203
chapter . recording a photographs light field
. O A O
A ( -.-s). T - . T
.
Assembly
T
, .
T -
: . T . W -
,
.
, F .b. T
. ,
. W
.F .
. W
, ,
. A
. T F .
- .
W
. W . T
- ,
. T
[S ] (. )
- (/).
-
7/30/2019 Renng Thesis
59/203
.. prototype light field camera
-ICROLENSARRAY
,ENSHOLDER
!DJUSTMENT
SCREWS
"ASEPLATE
3EPARATIONSPRINGS
0HOTOSENSOR
$IGITALBACK#HIPPACKAGE
MMMICROLENSES
PIXELS
MM
-ICROLENSARRAYACTIVESURFACE
F .: Le: S . E , - . Right: P .
,
- . T (
) . T
. T
-.
W -
. T
/, . T
.
Operation
U , M -
F . I, F ieee-,
-
7/30/2019 Renng Thesis
60/203
chapter . recording a photographs light field
. W -
, , .
-
- . T
. -
C ,
.
T
. I ,
, . T
- - /,
. T
. F , - /
( ). T
-
( ).
3.7 Related Work and Further Reading
Integral Photography, Plenoptic Cameras and Shack-Hartmann Sensors
O , [L-
; I ] , . I
d , [O ; J
O ]. I ,
d - [D . ; O . ; N . ; Y N
]. T , - -
d , d .
G . [] -
d . T
-
7/30/2019 Renng Thesis
61/203
.. related work and further reading
, -
F .. A , .
A
. A W ,
[A W ]. I , -
,
. A ,
, (-)
. T -
:
, .
A S-H [S P
; P S ]. I -
.
I , S-H ,
-. S
[B ]
[H . ]), -
[ ]. A -
, [R W ].
I , S-H
[L . ; H ]. C -
S-H
-
.
Other Related Optical Systems
, -
[L H ;
-
7/30/2019 Renng Thesis
62/203
-
7/30/2019 Renng Thesis
63/203
-
7/30/2019 Renng Thesis
64/203
-
7/30/2019 Renng Thesis
65/203
4Digital Refocusing
T :
. I F ., a-a
S .. T ,
/ . T
,
. I a,
, I I
. I .
P
digital reocusing . I a-a I
I . T
. I ,
,
. T ,
, ,
.
F .b , digitally extended
depth o feld. I , . T
-
7/30/2019 Renng Thesis
66/203
chapter . digital refocusing
(a) (a) (a)
(a) (a) (b)
F .: E (aa) (b).
.
I b a-a [A .
],
.
A . T
,
(snr). S ..
,
u .
-
7/30/2019 Renng Thesis
67/203
-
7/30/2019 Renng Thesis
68/203
chapter . digital refocusing
Focus and Depth from Defocus and Auto-Focus
O
[S . ; K A ]. T
depth rom deocus,
[K
; P ; S ]. D
- [N . ; P . ].
H . A
, .
T - -
- . B
.
Extended Depth of Field
L , Waveront Coding D J [] -
. I -
. D . I ,
, ,
,
, - . D
J ,
.
T
.
T , O . [], H-
[], E K [] A . [] ( I ).
-
7/30/2019 Renng Thesis
69/203
.. image synthesis algorithms
M
[ . ].
4.2 Image Synthesis Algorithms
T
C . T
.
Refocusing
T
. C
E .,
(x,y):
E(F)(x,y) =1
2F2
LF
u(11/)+ x/, v(11/)+ y/, u, v
dudv. (.)
R LF xy F
uv , F, E(F)
( F)O ,
u v . T -
S . -
u v. (u(11/)+ x/, v(11/)+ y/, u, v)
. T , LF -
.
H,
. E E .
-
-
7/30/2019 Renng Thesis
70/203
-
7/30/2019 Renng Thesis
71/203
.. image synthesis algorithms
(a): N (b): R (c): R
F .: S-- , - .
- (u, v),
, , . F .
, -, x
y, . U - --
- ,
. F .a , = 1, 0 -
. T
.
T -- - - F .. F ,
. H,
-
7/30/2019 Renng Thesis
72/203
chapter . digital refocusing
(a): U
(b): S- (c): U, (c): A
F .: A - -- .
--
. T F .-c, -
- . T -
- -- . W -
,
. F , ,
F .-a - F .-b.
T - (u, v)
E .. T
- , -
.
A - , -
. T - . T - F . c.
T - .
-
7/30/2019 Renng Thesis
73/203
.. image synthesis algorithms
Q d . T
- E ..A
. Vignetting F . c. T
- (
F .b c). A
E . . T
, ,
LF .
A
. F ,
. D
. F . c
, F . c.
Extending the Depth of Field
T . P-
- . T digitally stopping down the lens,
. S
. T
, ,
. T snr ( F .b).
I . L
,
. T , . A -
, ,.
I -
. T ,
, . B
-
7/30/2019 Renng Thesis
74/203
chapter . digital refocusing
(a): U (b): S- (c): E dof
F .: C - - .
E ., snr , . T , F .c,
- , . I -
, digitally extending the depth o feld
snr .
T F . z .
O -
, C . T F .a . I
,
. F .b F . z
.
T
. F ,
-
7/30/2019 Renng Thesis
75/203
.. theoretical refocusing performance
F . . C
, S ., I digital photomontage A . []
.
D
. I
. I
graph-cut optimization [B . ] ,
gradient-domain diusion [P . ] . F
,
. B ,
, .
4.3 Theoretical Refocusing Performance
T -
. T -
- - . A
N , - N
, N .
C F . T -
- d . B-
,
. T -
C ,
.
F,
N - . A
N= . I
,
-
7/30/2019 Renng Thesis
76/203
chapter . digital refocusing
. I
, , .A
. S
- N,
N
. I , / N= ,
/ . P
,
.F . . I I b,
, . I
, - :
. T, ,
. I I c,
. O
I a. T
. I, ,
, .
D .
T eective depth o ocus
. T
. I
- . I
- . I
, eective
, -
. F
-
7/30/2019 Renng Thesis
77/203
.. theoretical noise performance
-FOT%FTJSFE
GPDBMQMBOF%FQUIPGGPDVTPGBDPOWFOUJPOBMDBNFSB
&FDUJWFEFQUIPGGPDVTPGUIFMJHIUmFMEDBNFSB
F .: I . T , .
(..
, N ). T -
N. T N -
.
T N ,
F .. T - N
,
. T : -
- .
4.4 Theoretical Noise Performance
A -
. F ,
(.. -
). H,
-
7/30/2019 Renng Thesis
78/203
chapter . digital refocusing
NN (..
N2
).L -
/A /(A N), /A . A ,
-
. F,
/(A N) .H, N2
. I 1/N2
- . A, ,
N2 .
A , snr -
? T , N2
? T ,
, - ,
. T
P [G],
. F , , ,
. G , -
P ,
. U - , snr
N , N2 , P
N .
T , . L ,
, NN . I , -
N . T
-
7/30/2019 Renng Thesis
79/203
.. experimental performance
. T
. T ,
. I , N2
, 1/N2 1/N2
. T snr N. O
- -
, .
4.5 Experimental Performance
T
. I ,
,
.
Experimental Method
T ,
. , I
away . I ,
.
I I
- . T, ,
?
A -
, - . I
.
S -
-
7/30/2019 Renng Thesis
80/203
chapter . digital refocusing
,
- .A
- -
. T
,
. I I ,
,
.
A
. N, , P
. T snr
.
Refocusing Results
I / , -
/ /. T - . T -
. T
-.
F .
. N / -
/ . T
/ , /. I
/ .F . ,
(mtf) . I mtf
F [W ].
F , I ,
, F .
-
7/30/2019 Renng Thesis
81/203
.. experimental performance
/ R L F / C
/ C / C / C
F .: E : .
-
7/30/2019 Renng Thesis
82/203
chapter . digital refocusing
$POWFOUJPOBM
3FGPDVTFE-JHIU'JFME
F .: E : mtf . H 0 N .
F . mtf . T
mtf . T
- , ., , , , . A
- , ,
mtf. T , /
/ . T mtf
/ ,
F ..
A , , -
, / . I ,
/ , -
.
T . A F .,
,
. A, -
, S .. W
/ ,
-
7/30/2019 Renng Thesis
83/203
.. experimental performance
/ , -
.
Noise Results
I . A -
, I snr /
/ . T -
snr ., (
. / ). T
snr P , S-
.. I snr
- , I -
. T snr --
- .
/ C / C / R L F
F .: E .
-
7/30/2019 Renng Thesis
84/203
chapter . digital refocusing
(a): C
(b): U (b): R (b): E dof
F .: R .
F . . T - F .,
,
. I F ., / -
- . T /
, . N /
-
7/30/2019 Renng Thesis
85/203
.. technical summary
(a): /, / L F, E dof
(b): /, / C P
(c): /, / C P
F .: C .
/ . T / ,
. N , / . T
snr . / /
.
4.6 Technical Summary
T
- .
F . ,
. N
I b ,
/
-
7/30/2019 Renng Thesis
86/203
chapter . digital refocusing
- / . H,
/ . I I b, . N, ,
. W .
I b
.
F .
. T
F . F .a. I I b,
/ , I ( /)
. A ,
. I
I a,
P . I , I c, I / ,
- /
I a. T -
. T
. H, I c
. F .
-
.
I ,
. T
. T
.
4.7 Photographic Applications
,
. T -
.
-
7/30/2019 Renng Thesis
87/203
.. photographic applications
Portraiture
G
. I , -
. F
, -- ,
,
. I , .
I , . O
, .
U . H, ,
. W
,
, . F .a ,
- ,
. S ,
, F .c.
(a): M- (b): R (c): R -
F .: F -
A ,
. I ,
-
7/30/2019 Renng Thesis
88/203
-
7/30/2019 Renng Thesis
89/203
.. photographic applications
F .: T .
Action Photography
A
. L ,
. T -
, ,
.
F . . T
a-a b-b
. T / .
I c-c .
T ,
/ .
Landscape Photography
W , . T
,
. I
.
-
7/30/2019 Renng Thesis
90/203
chapter . digital refocusing
(a) (a) (a)
(b) (b) (b)
(c) (c) (c)
F .: D - .
-
7/30/2019 Renng Thesis
91/203
.. photographic applications
A ,
, . S . C ,
. F , -
,
.
T , -
,
- -
.
(a): R (a): R (a): E dof
(b): R (b): R (b): E dof
F .: E .
F . . I ,
-
7/30/2019 Renng Thesis
92/203
chapter . digital refocusing
(a) (a) (a)
(b) (b) (b)
F .: D .
, . T
- ,
.
Macro photography
T ,
, ,
. , : , -
,
-
7/30/2019 Renng Thesis
93/203
.. photographic applications
(a): H (b): L
(c): F (d): C
F .: M
.
F , ,
.
B ,
. M - . I , ,
. T ,
. A , -
, ,
-
7/30/2019 Renng Thesis
94/203
-
7/30/2019 Renng Thesis
95/203
5Signal Processing Framework
A , F -
-
. T F
. F ,
. I
,
C .
S . , F
. T ,
F . L ,
F
, B [].
5.1 Previous Work
T F C . [].
T , , - -
. T
- ,
.
-
7/30/2019 Renng Thesis
96/203
chapter . signal processing framework
I I . []. T
F , . T -
, -
.
M , S . []
- - . T -
-- . I ,
- . A
, .
F, D . [] F
,
, . W
,
.
5.2 Overview
T F -
. W
(C ) ,
. I , F
d d .
T , F d
d F . T F S P T,
S ., .
F . -
, F . T
, ,
-
7/30/2019 Renng Thesis
97/203
.. overview
N R R
F .: T ( ) F ( ).
-
7/30/2019 Renng Thesis
98/203
chapter . signal processing framework
-. T -
F . F . T - F ,
(mr) . I , kx ku
F- x u ( ky
kv). T F-
F . T . T
,
, .
T F - - R B [] F
S T ( C S T F P-S
T). B -
. H,
,
(ct) (pet) [M
], (mr) [N ].
T d d , -
d -
d . N,
, - -
. F , mr
S .
F-.
P F S P T-
-
. S . -
, . I ,
S .
-.
F .c -:
-
7/30/2019 Renng Thesis
99/203
.. overview
$POUJOVPVTMJHIUFME 1FSGFDUQIPUPHSBQI 3FDPSEFEMJHIUFME
1IPUPHSBQIDPNQVUFEXJUIPVUSFGPDVTJOH
3FGPDVTFEQIPUPHSBQIXJUIGVMMTQBUJBMSFTPMVUJPO
3FGPDVTFEUPPGBSXJUIMPTTJOTQBUJBMSFTPMVUJPO
3FEVDFEQIPUPCBOEXJEUI
F .: F- .
,
. T F-
. . I , , -
. W -
, -
,
,
.
W - , -
. F . , d
-
7/30/2019 Renng Thesis
100/203
chapter . signal processing framework
. S
-, - (F .df).
T , ,
(F .b). T -
- . B -
- ,
.
T S ., F .e f
. T ,
F .: T
F
.
,
- . U ,
-,
(
) . H, -
,
(-
). T
F .. A
. S-
. , C
.
S . F S P T
F S D R . T -
d F -
d F . T
O(n2 log n), n . T
-
7/30/2019 Renng Thesis
101/203
.. photographic imaging in the fourier domain
O(n4) , -
d .
5.3 Photographic Imaging in the Fourier Domain
P>1P
-
7/30/2019 Renng Thesis
102/203
chapter . signal processing framework
A ,
F S T. T F S T [D ] d d F F -
d . T
, F .. C,
dc ( ) ,
. F ,
. I
,
.
T ,
, d
d F . W , S .. ..
,
E . ..
5.3.1 Generalization of the Fourier Slice Theorem
L -, d . A
R [L M ], -
N M . T
-
change o basis (.. N- ). T
.
I Pj L INM N- M-
N M : INM [f] (x1, . . . , xM) =f(x1, . . . , xN) dxM+1 . . . dxN.
S L SNM N- M - NM -: SNM [f] (x1, . . . , xM) = f(x1, . . . , xM, 0 , . . . , 0).
-
7/30/2019 Renng Thesis
103/203
.. photographic imaging in the fourier domain
C B L B
N- . I B N- NN, B[f] (x) = f(B1x), x N- , B1 B.
S L a
, a[f](x) = a f(x).
F T L FN N- F -, FN . FN [f] (u) =
f(x) exp (2i (x u)) dx,
x u N- ,
N- .
W , :
generalized fourier slice theorem Let fbe an N-dimensional unction. I we
change the basis of, integral-project it down toM o its dimensions, and Fourier transorm the
resulting unction, the result is equivalent to Fourier transorming f, changing the basis with
the normalized inverse transpose o the original basis, and slicing it down to M dimensions.
Compactly in terms o operators, the theorem says:
FM INM B= SNM BT|BT| F
N, (.)
where the transpose o the inverse oBis denoted byBT, andBT is its scalar determinant.A A a..
F . N-
, M- , F . O ( F .
.) N= , M = d (B= R). I , (R = RT), RT
1. A , F .
-
7/30/2019 Renng Thesis
104/203
-
7/30/2019 Renng Thesis
105/203
.. photographic imaging in the fourier domain
, . R
A a.. A , () -
.
T . F -
,
N- , GN, F , GN. F
, M ,
GM. A F ., ( -
) O(n
N
) , n N. A GN ( ) -
F ( ), O(nM log n)
F F .
F ,
. F ,
ct , - -
. I ,
F F . W
.
5.3.2 Fourier Slice Photograph Theorem
T F S P T,
. T IO G F S T.
T I O (E .) -
():
P [LF] = 12F2
I42 B [LF] , (.)
-
7/30/2019 Renng Thesis
106/203
chapter . signal processing framework
:
I C B B d :
B =
0 1 00 0 1 0 0 1 0
0 0 0 1
B1 =
1/ 0 1 1/ 00 1/ 0 1 1/0 0 1 0
0 0 0 1
N B B1 , B,
generic G F S T. D I42 E . E ..
W G F S T (E .) -
E . F- . F, - E .
P = 12F2
F2 (F2 I42 B).
I , G F S T (E .)
,
P = 12F2
F2 S42 B
TBT F
4
.
F, BT
= B1 = 1/2, :
P = 1F2
F2 S42 BT F4, (.)
(P) d F : d (S42 BT), d (F2), (1/F2).
B ,
F :
-
7/30/2019 Renng Thesis
107/203
.. photographic imaging in the fourier domain
F P I O
P =1F2
S42 BT. (.)
I P ,
S42 B. T :
P[G](kx, ky) =1
F2G( kx, ky, (1 ) kx, (1 ) ky). (.)
A E . E . , , :
fourier slice photograph theorem
P = F2 P F4. (.)
A photograph is the inverse d Fourier transorm o a dilatedd slice in the d Fourier trans-
orm o the light feld.
F .
. T P F- P. T , F .
I ,P F F
F . I
,
.
T P, ,
P, . T
P (E .) P (E .). B F- -,
. I , F S P T C-
T, .
F ,
. T
-
7/30/2019 Renng Thesis
108/203
chapter . signal processing framework
2P
LF .F
EF 'F
O(n4)
O(n4 logn)
O(n2)
O(n2 logn)
4QBUJBMEPNBJO*NBHJOH
F4
F2
'PVSJFS5SBOTGPSN
'PVSJFS5SBOTGPSN
'PVSJFSEPNBJO*NBHJOH
F .: F S P T. d LF, - d EF, F ,LF EF.
F ., n d .
T P (O(n2))
P (O(n4)). T S ..
5.3.3 Photographic Effect of Filtering the Light Field
A E .. I
, ,
, ? T -
.
filtered light field imaging theorem A d convolution o a light feld results
in a d convolution o each photograph. Te d flter kernel is simply the photograph o the dflter kernel ocused at the same depth. Compactly in terms o operators,
P C4h = C2P[h] P, (.)
where we have expressed convolution with the ollowing operator:
-
7/30/2019 Renng Thesis
109/203
.. photographic imaging in the fourier domain
Cv CNh N-
h, CN
h [F](x
) =F(x
u
) h(u
) du
x
u
N- , F h N- .
F . . O , LF d
, LF d d h. EF EF
, . T EF d
EF, d d , h.
I , ,
. A a. . A ,
F S P T C T
F . I ,
, .
I
P
LF
EF
$POWPMVUJPO
$POWPMVUJPO
P
LF
EF
C4h
'JMUFSLFSOFM
C2P [h]
'JMUFSLFSOFM
P
F .: F L F P T. d LF, , LF, EF EF.
-
7/30/2019 Renng Thesis
110/203
chapter . signal processing framework
. A ,
. I d,
(x,y, u, v) = (x) (y) (u) (v), d :
(x) =
1 |x| < 120
.
W (x,y, u, v) ,
P [LF ] = (P C4) [LF]
= (C2P[] P) [LF]= P [] P [LF] .
F , , fg d d f g, a b d - d a b.
T
. T (P [LF]) d P []. T d d . W d ? T
d .
T
-
7/30/2019 Renng Thesis
111/203
.. photographic imaging in the fourier domain
. T d , - ,
, .
B d , d d
, - . E , , d .
T F L F I T
. F ,
, ,
d , h(x,y, u, v). A ,
, (..
) h(x,y, u, v). G ,
P[h]. C d : . A
.
O , -
. F, ,
F -
. T
(C ), , - . A
-, -
. I , -
,
.
-
7/30/2019 Renng Thesis
112/203
chapter . signal processing framework
5.4 Band-Limited Analysis of Refocusing Performance
T F L F I T
, . W -
? H ,
-
?
T , S .,
- . S . -
-. T ,
F L F I T.
T - F
d , (x,y, u, v). B C T, F
F .
I F - ,
. L - , sinc(x,y, u, v) = sinc(x) sinc(y) sinc(u) sinc(v),
sinc(x) =sin x
x.
I , - , LF,
exact , LF, d sinc:
LF =C
4lowpass [LF] , where
lowpass(x,y, u, v) =1
(xu)2 sinc
xx
,y
x,u
u,v
u
.
I , x u
, . T 1/(xu)2 - -
sinc.
-
7/30/2019 Renng Thesis
113/203
.. band-limited analysis of refocusing performance
Analytic Form for Refocused Photographs
O , EF ,
- , LF. T F L F P
T. L = F/F,
EF = PLF
= P
C4lowpass [LF]
= C2P[lowpass] [P [LF]] = C2P[lowpass] [EF ] ,
EF exact F. T -
d- . T d d sinc , P [lowpass].
I d sinc d sinc :
P [lowpass] = P
1
(xu)2 sinc
xx
,y
x,u
u,v
u
=1
D2x sinc
x
Dx,y
Dx
, (.)
N d sinc , :
Dx = max(x, |1 |u). (.)
T , F S P
T , ( A a.).
T d sinc, -
- . T
d (E .)
.
Interpretation of Refocusing Performance
R x u. L
Wx,
-
7/30/2019 Renng Thesis
114/203
chapter . signal processing framework
Wu. W , Nx = Wx/x
Nu =Wu/
u.
S = (F/F) u =Wu/Nu,
|x| |(1 )u|
|F F| xNuF
Wu. (.)
T , F,
, .. . W
N- ,
- .
P, E . E .,
(x). N,
, Wx, (Wx). T
( E .). I > 1, ,
, S ..
T, N
Wxx
=Wxx
. (.)
T ,
. T
E .. N
, Nu, S ..
I , ..
|F F| > xNuF
Wu,
- , EF , |1 |u,
-
7/30/2019 Renng Thesis
115/203
.. fourier slice digital refocusing
x ( E .). T ,
Wx|1 |u ,
,Wx/x. I ,
, N- .
R- -
. S = F/F u =Wu/Nu,
|1
|
Wxu
=Wx
Wu
NuF |F
F|
. (.)
B NuFWu
f- Nu
, WuNuF |F F| - -
|F F|.I , ,
, Nu , S .. N
, Nu. D ,
.
I , - -
-
C .
5.5 Fourier Slice Digital Refocusing
T F S P T ,
. T
: - (
). T , ..
-
7/30/2019 Renng Thesis
116/203
-
7/30/2019 Renng Thesis
117/203
.. fourier slice digital refocusing
wwww
F
wwww2>
wwww2=
wwww2>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>:
8>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>:
1
3
-JHIU'JFME'PVSJFS4QFDUSVN
'PVSJFS4MJDF&YUSBDUJPO
'PVSJFS4MJDFT
*OWFSTF'PVSJFS5SBOTGPSN
3FGPDVTFE1IPUPHSBQIT
-JHIU'JFME
'PVSJFS5SBOTGPSN
F .: F S D R .
-
7/30/2019 Renng Thesis
118/203
-
7/30/2019 Renng Thesis
119/203
-
7/30/2019 Renng Thesis
120/203
chapter . signal processing framework
(a): W - (b): W -
F .: R .
F [M ], -
d F .
T d F -
. F . - K-
B , .
U, -
,
.
Suppressing Aliasing Artifacts
T , -
-. O
, - d F (PF F4) [LF]. I- F
. T - .
E
F . S,
. H,
,
-
7/30/2019 Renng Thesis
121/203
.. fourier slice digital refocusing
. A ,
.
(a): (F ) (b): I F (a)
(c): N (d): C (b)
F .: R F .
F . . I a F . I b
. S
, I d. F , c
. T
.
-
7/30/2019 Renng Thesis
122/203
chapter . signal processing framework
O , -
. T , -
.
T : .
A , sinc -
. O - -
. T -
- ,
J . [] , M [] F .
(a): Q ( )
(b): K-B ( .)
(c): K-B ( .)
F .: A . R .
J . , -, K-
B . T K-B
. F .
K-B . S, K-
B . .
A , bokeh
. U
-
7/30/2019 Renng Thesis
123/203
.. fourier slice digital refocusing
(a): W (b): W
F .: A . K-B - .
LF(x,y, u, v) A(u, v) -
. F , A(u, v)
(u, v) . I F ,
F
A(u, v). T F [L ].
T -
- F [L ;
M ]. T ,
- .. I F .,
I a -
I b -.
Implementation Summary
I S ., . I
- ,
. P (..) .
. P- F
.
-
7/30/2019 Renng Thesis
124/203
chapter . signal processing framework
F
S
F .: C F .
I , d F ,
. U - K-B . A .
. F , .
( ) .
. O d F . A F , -
.
Performance Summary
T - F S
- C .
D , F . .
-
7/30/2019 Renng Thesis
125/203
.. fourier slice digital refocusing
F
S
F .: C F II.
F . T
( = ). F .
F ,
- . R
,
, .
N, F- .. A , -
F- .
I , cpu F
. W , F [N ]. T F
-
7/30/2019 Renng Thesis
126/203
chapter . signal processing framework
S uv ,
, F .
5.6 Light Field Tomography
T -
F . T
. T light feld tomography ct -
.
L d d . W
, F d -,
, . A
, /2 /2. H, d d ,
F . A
d R [D ] . T d
F S T d R
.
A
,
F . I ,
d .
I , d
. I F S P T ( E-
. ) - d
d d F :
( kx, ky, (1 ) kx, (1 ) ky)where [0,), and kx, ky R
. (.)
-
7/30/2019 Renng Thesis
127/203
.. light field tomography
I , d
d . I d F .O d
. D d
. O d slit light feld d
. T
d d (
). B ,
d d
.
-
7/30/2019 Renng Thesis
128/203
-
7/30/2019 Renng Thesis
129/203
6Selectable Refocusing Power
T -
,
. T
. C
-,
. T generalized light feld camera.
O - -
. T
,
. H, ,
, - -
. F .
. N
,
.
T - F .
-
7/30/2019 Renng Thesis
130/203
-
7/30/2019 Renng Thesis
131/203
.. sampling pattern of the generalized light field camera
(a): = 1 (b): = . (c): = . (d): = 0
F .: D .
= 0. A S .,
beyond .
F . -. T
F .. N
F . . - , -. A
F . , -
. T
( x ), ( u ). A
-
-
7/30/2019 Renng Thesis
132/203
chapter . selectable refocusing power
, . H, -
.
R ,
. H,
. A = 0
, .
T ,
. I ,
MM M
M,
MM,
M = max((1 )M, M). (.)
Deriving the Sampling Pattern
F . - , -
- . T
.A F .
. E -
. D
-,
C . T
-, , -
.F .a , , i,
. F .b -
, . L
- :
, xi, , si, , wi. I
-
7/30/2019 Renng Thesis
133/203
-
7/30/2019 Renng Thesis
134/203
-
7/30/2019 Renng Thesis
135/203
.. optimal focusing of the photographic lens
.
N -. F .e
, . A
, F .f
u
.
A , ,
- - . T
F .f
F . . T
, .
The Problem with Focusing the Microlenses Too Close
T
( < ). A -
closer ,
urther ( > ). T - . T
, . B
() , - -
,
-. F ,
.
6.2 Optimal Focusing of the Photographic Lens
A -
. I
. I , -
-
7/30/2019 Renng Thesis
136/203
-
7/30/2019 Renng Thesis
137/203
-
7/30/2019 Renng Thesis
138/203
chapter . selectable refocusing power
, E . (1 ),
. H (1 ). T , . E
(1 ).I , , -
F, Fopt. D -
F
max((1 )M,M).
6.3 Experiments with Prototype Camera
T ., -
. I
. T -
S .: I -,
. I , I
.F . - : -
( ), -
( ). T -
/ iso
. W .
. O -
.
N = .
, -
. I , = .,
- . H, -,
-
7/30/2019 Renng Thesis
139/203
.. experiments with prototype camera
= .
= .
W-
0
F .: D .
. I, - . .
( ) .
T
. T
, . F . . E . . ,
. , -
, F ..
A , 0 -
- [M N ] . T
,
- . N
-
-
7/30/2019 Renng Thesis
140/203
chapter . selectable refocusing power
C R- C R-
F .: C
-, = . ( ), = 1.
, - -
- .
6.4 Experiments with Ray-Trace Simulator
,
P H - [] . F .
-. T
- F ., , -
. T F . = ., = .. T
. T
-
7/30/2019 Renng Thesis
141/203
-
7/30/2019 Renng Thesis
142/203
chapter . selectable refocusing power
T -
. T - iso . T
-. A -
, ,
.
MTF Analysis
T - mtf ,
- . T .
A
. B -
,
.
T mtf
. F ,
, - -. T
. T
F mtf -
-.
T mtf
. T
mtf . A
mtf, , mtf .
F .a
. T - -
, .
-
7/30/2019 Renng Thesis
143/203
.. experiments with ray-trace simulator
. N .
F .a . F, ( ) , . T -
- = ( = 0, ., ., .
), , -
(1 ) . T , -. T
. T
, E-
.. A , (
). T ,
.
F .b - E-
., ,
F .c. T
,
E .. T F .c .
F . - F .b F .c -. T ,
. T
. T F .a
= .
.
T E ..
T F .b,
.c .d. R
E ..
T
,
. I F .b, .c .d, -
,
-
7/30/2019 Renng Thesis
144/203
chapter . selectable refocusing power
(FOFSBMJ[FEMJHIU
FMEDBNFSBDPOHVSBUJPOT
$FOUFSFEWFSTJPO
$VTUPNJ[FEQMFOPQUJDDBNFSBT
3FMBUJWFPQUJDBMNJTGPDVTJNBHFTJEF
4QBUJBMGSFRVFODZGPS
DVUPDZDMFTNN
NJDSPONJDSPONJDSPO
NJDSPONJDSPO
F .: mtf .
( ). I
, between ,
. T , , 1
, F .e.
-
7/30/2019 Renng Thesis
145/203
.. experiments with ray-trace simulator
NJDSPO
NJDSPO
NJDSPO
NJDSPO
NJDSPO
NJDSPO
NJDSPO
NJDSPO
3FMBUJWFPQUJDBMNJTGPDVTJNBHFTJEF
4QBUJBM
GSFRVFODZGPSDVUPD
ZDMFTNN
F .: mtf II.
-
7/30/2019 Renng Thesis
146/203
chapter . selectable refocusing power
Summary
T
-
. T
, - -
. A
. T ,
,
.F , -
. A
. T
-
,
.
T ,
. T
-
. F ,
,
.
T - :
. B ,
.
-
7/30/2019 Renng Thesis
147/203
-
7/30/2019 Renng Thesis
148/203
chapter . digital correction of lens aberrations
. T -
, W [K ]. I -
. I , C
. F, ,
D , .
-
. Z
F .: S
.
. Z -
,
. H, -
-
,
[M ]. A ,
,
[D L ]! ,
- [S ],
.
O
[K ].
T -
. T -
. T :
, -
. D
.
F ,
-
7/30/2019 Renng Thesis
149/203
.. previous work
( = 1) . I ,
, . T - , -
. T ,
, - .
7.1 Previous Work
R- . P
- . I S E P,
C L H [ A ] -
[K ]. A - ,
, /., . T
, ,
- ,
[N ].
K -
[K ]. T
. O
- - lasl
L A [B ].
I , -
- [P
C ; C . ], - - [K . ]. T C .
P C M-
C [K ]. I
, - ,
- [P H ] I
-
7/30/2019 Renng Thesis
150/203
chapter . digital correction of lens aberrations
. O
- - .I , -
[M . ]. T
.
7.2 Terminology and Notation
T -
. T two - :
ideal-, , aberrated-
, . I
,
. T ,
-. L
(x,y, u, v)
(x,y, u, v).
T - . A -
. I ,
. I
, G ,
[S ]. T G -
, -
. T
G .
T , C, -
:
-
7/30/2019 Renng Thesis
151/203
-
7/30/2019 Renng Thesis
152/203
-
7/30/2019 Renng Thesis
153/203
.. visualizing aberrations in recorded light fields
F .: A - - .
-
7/30/2019 Renng Thesis
154/203
chapter . digital correction of lens aberrations
F .d.
F .b :
. T . A
- . F, S .,
.
T - ,
.
E - F .. T . F .
- ,
. I
,
.
7.4 Review of Optical Correction Techniques
T -
.
, .
F . . T - ,
. T -
ray-intercept curves
,
, - . T F . . T
( u -
), . S
, x . O
, .
-
7/30/2019 Renng Thesis
155/203
-
7/30/2019 Renng Thesis
156/203
chapter . digital correction of lens aberrations
I ,
. T , , F . .
(F .).
7.5 Digital Correction Algorithms
A ,
: . -
,
- . I ,
,
.
F . -,
. F .a
, - . T . F .b
(x, u) - , (x, u) -
. E
. T F .a.
F .c
. T
. I , F .d
. N
, , - F .c.
F .ac : pixel-order
, . A -
ray-order, .
-
7/30/2019 Renng Thesis
157/203
.. digital correction algorithms
(a) (b)
(c) (d)
F .: R- .
T -
. T :
-
. T . T
- . T -
.
Pixel-Order Image Synthesis
T -
. I
.
. S . A M-C
-
7/30/2019 Renng Thesis
158/203
chapter . digital correction of lens aberrations
.
. C - .
. R -
,
.
. E
. T
d . L- -
. S, .
. A .
Ray-Order Re-Projection of the Light Field
T - -
. I
.
. S
. A M-C
,
.
. , , -
.
. R
.
. I . A ,
.
A , ,
.
-
7/30/2019 Renng Thesis
159/203
.. digital correction algorithms
Confidence Weighting for Increased Contrast Enhancement
T -
. W
. F, - F .,
. S, -
,
, . F , F .d
.
T -
, . T
-
. T
weighted - .
I , I ,
x y .
F , .
w(x,y) = h(x) h(y), where
h(x) =
1, x 1exp
(1x)2
22
, x > 1
. (.)
I , G -
. T x y
, . I
= G -. C
x
y, (x,y, u, v), .
F . . E
. T
,
. A
-
7/30/2019 Renng Thesis
160/203
chapter . digital correction of lens aberrations
(z) (z) (z)
F .: W ( ) . E .
-
7/30/2019 Renng Thesis
161/203
.. digital correction algorithms
- , (z z).
C , .
R -
. E . ,
. C
. T -, ,
( 1) -
. F ,
F . , -
. H,
- . W
. A -
, .
Computing the Projected 2D Size of Light Field Samples
C x y E .
d . I ,
,
C, . I , x
- :
x 1x
Cxx x +
Cxy y +
Cxu u +
Cxv v
, (.)
I C :
Cx,y, u, v
=
Cx(x
,y, u, v),
Cy(x,y, u, v),
Cu(x,y, u, v),
Cv(x,y, u, v)
= (x,y, u, v) . (.)
-
7/30/2019 Renng Thesis
162/203
-
7/30/2019 Renng Thesis
163/203
.. correcting recorded aberrations in a plano-convex lens
(a) (b)
F .: S- - .
F .a. T
- .
I -
- ( ). A
, / . I
-
, F .,
.
I M-C -. T
,
. T
-. A -, I
.
-
7/30/2019 Renng Thesis
164/203
-
7/30/2019 Renng Thesis
165/203
-
7/30/2019 Renng Thesis
166/203
-
7/30/2019 Renng Thesis
167/203
.. simulated correction performance
7.7 Simulated Correction Performance
I ,
. T , [S ]
- (psf)
. T psf
. I -
, . O
. I
- ?
,
.
7.7.1 Methods and Image Quality Metrics
T . A
, -
. T , ,
(. mp). A NN , N= (), N= . S
, N . N= , . -
, [A
]. . [M ],
, N= ,
. .
T psf
. C psf ,
. I ,
. F psf -
, - ,
-
7/30/2019 Renng Thesis
168/203
chapter . digital correction of lens aberrations
psf .
A psf , -. T
(a): S
(b): C
F .: psf rms
.
psf d
, . F ,
psf
. A ,
( -
). F -, psf .
I , , I
psf.
T
(rms) ,
. T rms
psf
. A ,
rms - . O
,
:
. F .a
psf , rms .
O , rms ,
. F
, F .b psf coma.
T - psf
psf . C
[S ]. T rms
, .
T rms , d
-
7/30/2019 Renng Thesis
169/203
.. simulated correction performance
. T rms
d , - -. A , -
,
.
A , rms eective res-
olution. T . I
F .: E
.
, ,
rms (F .). T . I
,
(
2R)2,
R rms ( ),
2R
. T
-
psf
. E psf-
.
T mtf. T mtf
,
. I
F . I, I
mtf : psf
F . A
mtf [P . ].
T mtf N
.
S mtf F psf, - psf.
-
7/30/2019 Renng Thesis
170/203
chapter . digital correction of lens aberrations
T mtf,
. I ,
mtf. A mtf
.
7.7.2 Case Analysis: Cooke Triplet Lens
L psf . T /.
C [ E ]. F . psf. C a,
b c : ,
- , .
T
. ,
. T -
. T
. I , . F , D a
. I , D c
, .
T F . , -.
T , x, , , u,
. T N= . T
.
I D c,
. I ,
. I ,
D c ,
- , .
T -
-
7/30/2019 Renng Thesis
171/203
.. simulated correction performance
(a)
(a)
(a)
(b)
(b)
(b)
(c)
(c)
(c)
F .: A - - C .
. I , -
. C, -
. T
, ,
-
7/30/2019 Renng Thesis
172/203
chapter . digital correction of lens aberrations
MICRONS
(a): C
NJDSPOT
(b): M
NJDSPOT
(c): E
U C psf.
MICRONS
(d): C
NJDSPOT
(e): M
NJDSPOT
(f): E
F .: D C psf.
, . T
: -
,
.
F . psf
F .. T rms . W
(), rms , . mp .
F .
. I psf
, .
F .
-
7/30/2019 Renng Thesis
173/203
.. simulated correction performance
TQPUSBEJVTQJ
YFMT /
/
'SBDUJPOPGFMEPGWJFX
/
/
/
F .: H - psf .
(NN) N= ( ) . T N rms
. T . F -
, N= , rms
. T psf -
( ) . , rms psf.
T rms
. T
, -
. T . mp
. mp N= . T ,
. O -
-, . S,
. I ,
,
.
T -
, . F .
, mtf . W ,
-
7/30/2019 Renng Thesis
174/203
chapter . digital correction of lens aberrations
%JTUBODFGSPNPQUJDBMBYJTNN
6ODPSSFDUFENN6ODPSSFDUFENN6ODPSSFDUFENN$PSSFDUFENN$PSSFDUFENN$PSSFDUFENN
F .: mtf ( ).
%JTUBODFGSPNPQUJDBMBYJTNN
6ODPSSFDUFENN6ODPSSFDUFENN6ODPSSFDUFENN$PSSFDUFENN$PSSFDUFENN$PSSFDUFENN
F .: mtf ( ).
N . T mtf
: /, / / N . T / N
, . C
. T
. G mtf
,
.
F . , ,
, .
S, , , .
-
7/30/2019 Renng Thesis
175/203
.. simulated correction performance
D , -
.
F . ,
: (.. ). T
, . T
- , M
. T
( ), . T
.
T mtf F .
. T extends the useul ocal range
. H, ,
. T -. F,
. F . -
F .. F
,
.
U,
. T F .
d-f a-c. T
. A ,
,
.
T ,
-
. H
.
-
7/30/2019 Renng Thesis
176/203
chapter . digital correction of lens aberrations
(a): C (b): M (c): E
R- , .
(d): C (e): M (f): E
F .: R- .
7.7.3 Correction Performance Across a Database of Lenses
T -
- . I -
-
.
,
. T
-- Z [Z ]
.
T Z . F,
-
7/30/2019 Renng Thesis
177/203
.. simulated correction performance
, -
- . S,
. T
. F ,
,
. F . ,
.
T , , G
, - . T
-. M
, . N, ,
. T
,
.
I , - . T
- .
F . . E ,
. F , -
. T
NN, N= 1 ( ), - N= , . N
.
T
. E ,
-. I
, -
.
A -
. I ,
-
7/30/2019 Renng Thesis
178/203
chapter . digital correction of lens aberrations
L F D G F A R
e- C /. e- /. e- usp ,, /.
e- C usp ,, /.
f- W usp ,, /. f- W- /.
g- /. g- slr /. i- P /. l- D G /.
l- D G
/.
l- P usp ,, /. l- P usp ,, /.
l- L usp ,, /.
l- M G usp ,, /. l- L usp ,, /.
l- L. . usp ,, /. l- G usp ,, /. l- D usp ,, /.
l- - G usp ,, /. l- M usp ,, /.
l- D G /.
I F ..
F .: D .
-
. F , , vlsi ,
. , ( ). ,
. F -
. T ,
.
-
7/30/2019 Renng Thesis
179/203
.. simulated correction performance
,FZ
&FDUJWF3FTPMVUJPO
F .: I .
-
7/30/2019 Renng Thesis
180/203
chapter . digital correction of lens aberrations
A (
- ) . T . F
. mp .
A ,
( ) ,
-, . I
,
[A ]. O
, . T
-
.
In Closing
, -
, . O
: . A
. I , -
(.. )
.
I ,
. A -
, F . ,
. F
-,
u, . O
,
-
7/30/2019 Renng Thesis
181/203
.. simulated correction performance
. O ,
, .
T
: ? ,
- S .
. W
. M
. T -
F .d . R
, , . W
, . A
- -
F .. I ,
. I , ,
.
T - -
. C
,
.
-
7/30/2019 Renng Thesis
182/203
-
7/30/2019 Renng Thesis
183/203
8Conclusion
T , H C-B
( ) , , -
Decisive Moment .
T , --,
.
A L, ,
, , C-B
.
B . I , I -
. T
-. O
. F D , - ,
-, A S ,
,
. B - ,
.
-
7/30/2019 Renng Thesis
184/203
-
7/30/2019 Renng Thesis
185/203
. H
.T -
. A d -
[L . ], -
. -
. W -
- [ ],
? I ,
.
T . W
: ,
. I I
, I
.
B , . P
,
. ,
is
. R
,
. I