renng thesis

Upload: dewddude

Post on 04-Apr-2018

236 views

Category:

Documents


0 download

TRANSCRIPT

  • 7/30/2019 Renng Thesis

    1/203

    D I G I T A L L I G H T F I E L D P H O T O G R A P H Y

    a dissertation

    submitted to the department of computer science

    and the committee on graduate studies

    of stanford university

    in partial fulfillment of the requirementsfor the degree of

    doctor of philosophy

    R N

    J

  • 7/30/2019 Renng Thesis

    2/203

    C R N

    A R R

  • 7/30/2019 Renng Thesis

    3/203

    I I , ,

    D

    P.

    P H P A

    I I , , D

    P.

    M L

    I I , ,

    D

    P.

    M H

    A U C G S.

  • 7/30/2019 Renng Thesis

    4/203

  • 7/30/2019 Renng Thesis

    5/203

    Acknowledgments

    I P H, M

    L M H , I

    . P ,

    , I

    . I . M L

    , ,

    . M H

    .

    I .

    I B W D N -

    . D S.

    I

    . M B

    , . G D

    ,

    . A A M Y -

    . K F

    ,

    .

    A . S

    K W K I S S -

    , T J C M, S P A R

    A O A, M H Z I.

  • 7/30/2019 Renng Thesis

    6/203

    I , I H G A G S-

    G L - , J G .

    T P C, B C, J F, K F, A E G,

    J G, B H, B O, D O

    .

    A M R F . T

    ,

    . A S B G

    . I

    S U S C

    .

    I S G L.

    I K F,

    . M K -

    , I . J K

    . I

    : E C, M H, G H, B

    M, K P, R R, P S R W. S

    J O, M P B W

    .

    I , ,

    , M B, S C, A C,

    P F, N H, L H, J J, S M, E M, M

    P, J R, M R, D W M W. S

    N , ,

    .

    F, I , Y F, B L C K N,

    . M ,

    . T .

  • 7/30/2019 Renng Thesis

    7/203

    'PS.BNBBOE1BQB

  • 7/30/2019 Renng Thesis

    8/203

  • 7/30/2019 Renng Thesis

    9/203

  • 7/30/2019 Renng Thesis

    10/203

    contents

    4 Digital Refocusing 49

    . P W . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . I S A . . . . . . . . . . . . . . . . . . . . . . . . . . . .

    . T R P . . . . . . . . . . . . . . . . . . . . . . .

    . T N P . . . . . . . . . . . . . . . . . . . . . . . . . .

    . E P . . . . . . . . . . . . . . . . . . . . . . . . . . . .

    . S . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

    . P A . . . . . . . . . . . . . . . . . . . . . . . . . . . .

    5 Signal Processing Framework 79

    . P W . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

    . O . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

    . P I F D . . . . . . . . . . . . . . . . .

    .. G F S T . . . . . . . . . . . . . .

    .. F S P T . . . . . . . . . . . . . . . . . . .

    .. P E F L F . . . . . . . . . . . .

    . B-L A R P . . . . . . . . . . . . . .

    . F S D R . . . . . . . . . . . . . . . . . . . . . . . . .

    . L F . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

    6 Selectable Refocusing Power 113

    . S P G L F C . . . . . . . . . . .

    . O F P L . . . . . . . . . . . . . . . . . . .

    . E P C . . . . . . . . . . . . . . . . . . . . . . .

    . E R- S . . . . . . . . . . . . . . . . . . . . .

    7 Digital Correction of Lens Aberrations 131

    . P W . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

    . N . . . . . . . . . . . . . . . . . . . . . . . . . . . .

    . V A R L F . . . . . . . . . . . . . . . .

    . R O C . . . . . . . . . . . . . . . . . . . .

    . D C A . . . . . . . . . . . . . . . . . . . . . . . . . .

  • 7/30/2019 Renng Thesis

    11/203

    contents

    . C R A P-C L . . . . . . . . . .

    . S C P . . . . . . . . . . . . . . . . . . . . . . . . .. M I Q M . . . . . . . . . . . . . . . . . .

    .. C A: C L . . . . . . . . . . . . . . . . . . .

    .. C P A D L . . . . . . . . .

    8 Conclusion 167

    A Proofs 171

    a. G F S T . . . . . . . . . . . . . . . . . . . . . . . .

    a. F L F I T . . . . . . . . . . . . . . . . . . . . . . a. P F-D S L F . . . . . . . . . . . . . .

    Bibliography 177

  • 7/30/2019 Renng Thesis

    12/203

  • 7/30/2019 Renng Thesis

    13/203

    List of Figures

    1 Introduction 1

    . C . . . . . . . . . . . . . . .

    . D . . . . . . . . . . . . . . . . . . . . . . . . .

    . R . . . . . . . . . . .

    . D . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

    2 Light Fields and Photographs 11

    . P . . . . . . . . . .

    . T . . . . . . . . . . . . . . . . . . . .

    . P . . . . . . . . . . . . . . . . . P . . . . . . . . . . .

    . - . . . . . . . . . . . . . . . . . . . . . . .

    3 Recording a Photographs Light Field 23

    . S . . . .

    . O . . . . . . . . . . . . . . . . .

    . R . . . . . . . . . . . . . . . . . . . . . . . . . . . .

    . C . . . .

    . S- . . . . . . . . . . . . . . .

    . E . . . . . . . . . . . . . . . . . .

    . M . . . . . . . . . . . .

    . G : - . . . . . . . . . . . . . . .

    . G : . . . . . . . . . . . . . . . . .

  • 7/30/2019 Renng Thesis

    14/203

    list of figures

    . P . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

    . M . . . . . . . . . . . . . . . . . . . . . . . . S . . . . . . . . . . . . . . .

    4 Digital Refocusing 49

    . E . . . . . . . . . . . . . .

    . S-- . . . . . . . . . . . . . . . . . . . . . . . . .

    . A - -- . . . . . . . . . .

    . C - . .

    . I . . . . . . .

    . E : . . . . . . .

    . E : . . . . . .

    . E . . . . . . . . . .

    . R . . . . . . . . . . . . . . . . . . .

    . L . . . . . . . . . . . . .

    . F - . . . . . . . . . . . . . . . . . . . . . . . . . . .

    . M . . . . . . . .

    . T . . . . . . . . . . . . . . . . . . .

    . H- . . . . . . . . . . . . . . . . . . . . . . . .

    . E . . . . . . . . . . . . .

    . D . . . . . . . . . . . . . . . . . . . .

    . M . . . . . . . . . . . . . . . . . .

    5 Signal Processing Framework 79

    . F- . . . . . .

    . F- . . . . . .

    . R F . . . . . . . . . . . . . . . . . . . . P I O . . . . . . . . . . . . . . . . . . . . . . . . .

    . C F S T . . . . . . . . . . . . . . . . . . . . . . . . . .

    . G F S T . . . . . . . . . . . . . . . . . . . . . . . .

    . F S P T . . . . . . . . . . . . . . . . . . . . . . . .

    . F L F P T . . . . . . . . . . . . . . . . . . .

  • 7/30/2019 Renng Thesis

    15/203

  • 7/30/2019 Renng Thesis

    16/203

    list of figures

    . C . . . .

    . C . . . . . . . . . . psf rms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

    . E . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

    . A - - C . . . . . . . . . . .

    . S . . . . . . . . . . . . . . . . . . . . . . . . . . .

    . H - psf . . . . . . . . . . . .

    . mtf ( ) . . . . . . . .

    . mtf ( ) . . . . . . . .

    . R- . . . . . . . . . . . . . .

    . D . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

    . P . . . . . . . . . . . . . . .

    8 Conclusion 167

    A Proofs 171

  • 7/30/2019 Renng Thesis

    17/203

    1Introduction

    T , -

    . T -

    . I

    . I , -

    . T sum total

    . - ,

    ,

    .

    I , . W -

    ,

    . I , -

    . T .

    H, I

    - . I , ,

    .

    O

    . T . T -

    ,

    .

  • 7/30/2019 Renng Thesis

    18/203

    chapter . introduction

    1.1 The Focus Problem in Photography

    F . I , P -

    Charivari D -

    [N ].

    Y . Y

    , .

    Y ; ,

    ,

    , .

    F , -

    . I ,

    .

    T

    . A ,

    . A

    . A - , -

    . T

    .

    T . I

    depth o feld

    . A ,

    . T

    : I ?O ,

    F .a-c,

    . O , ,

    F .a-c.

  • 7/30/2019 Renng Thesis

    19/203

    .. the focus problem in photography

    (a): W /, /

    (b): M /, /

    (c): N /, /

    F .: C . A /n /n .

    - - ,

    . H,

    ,

    .

    T

    . T . A

    , .T ( )

    . I

    ,

    .

  • 7/30/2019 Renng Thesis

    20/203

  • 7/30/2019 Renng Thesis

    21/203

    .. trends in digital photography

    . F ,

    : , .

    D , -

    (mp) [A ].

    G , . T

    -

    . F ,

    , mp [K ].

    W -, . S

    mp . F ,

    - (. )

    - ( ) [A ], mp. T

    . F, -

    . S,

    , -- (snr). T -

    - , -

    . T

    -

    . H, -

    . W

    .

    A , , , . D

    . T

    -

    . A rgb

    (, ) ,

  • 7/30/2019 Renng Thesis

    22/203

    chapter . introduction

    (a) (b) (c)

    F .: D .

    F .a. I ,

    , . D [R-

    . ]

    rgb , F .b c. T

    -, - . O

    :

    [N M ] [D

    M ]; [Y . ];

    - -

    [B . ]; - ;

    . C

    .

    I , - -

    . T ,

    .

    1.3 Digital Light Field Photography

    M

    . T

  • 7/30/2019 Renng Thesis

    23/203

  • 7/30/2019 Renng Thesis

    24/203

    chapter . introduction

    .

    , .

    T

    . I -

    . S,

    , . F

    -

    , .

    1.4 Dissertation Overview

    Organizational Themes

    T -

    : -

    . T

    .

    System Design: Optics and Algorithms T - . T

    .

    Mathematical Analysis o Perormance T -

    . T

    . T C - -

    . T

    F , - . T

    .

    Computer-Simulated Validation S - -

    - . F, -

    , -

  • 7/30/2019 Renng Thesis

    25/203

    .. dissertation overview

    .

    A Prototype Camera and Experimental Validation T - , -

    . T - -

    . T

    -- . I , I

    -

    .

    Dissertation Roadmap

    $IBQUFSPUBUJPO

    $IBQUFSJHIUJFMEBNFSB

    $IBQUFS

    FGPDVTJOH

    $IBQUFS

    PVSJFSOBMZTJT

    $IBQUFS

    FTPMVUJPOSBEFPT$IBQUFS

    FOTPSSFDUJPO

    F .: R.

    I

    -

    , -

    , ,

    .

    F .

    , -

    . P

    S . ., -

    . C -

    ,

    . C . T -

    ,

    .

    Chapter Descriptions

    C .

    C , ,

  • 7/30/2019 Renng Thesis

    26/203

    chapter . introduction

    . I .

    C , . I -

    .

    C F . T

    F- -.

    C -,

    .

    C ,

    .

    C .

  • 7/30/2019 Renng Thesis

    27/203

    2Light Fields and Photographs

    P . F

    , ,

    . I

    .

    T

    , light feld ,

    ray-space. B

    -, ,

    . T

    .

    2.1 Previous Work

    T .

    A B [] L V. L

    H []

    G [] M,

    M S [] A.

    I ,

  • 7/30/2019 Renng Thesis

    28/203

    chapter . light fields and photographs

    , -,

    . A B - ,

    plenoptic unction d d

    d . T

    [A B ]. T - -

    ,

    . T ra-

    diance [P ],

    ( ).

    T d light feld,

    , L H [] G

    . [] ( Lumigraph). T d d

    -

    , , , . I ,

    , .

    T epipolar volumes B, B

    M [] .

    O . F -

    ,

    d , -

    . T -

    C W [],

    MM B []. F , image-based ren-

    dering ,

    , .

    S K [] ,

    siggraph . T

    F L plenus , .Light feld G [] R, M .

  • 7/30/2019 Renng Thesis

    29/203

    .. the light field flowing into the camera

    ,

    F .W , -

    . T [L H ; G . ]

    ,

    . O ,

    [C . ] -

    [M . ; W . ; C . ]. A

    -

    , [S .

    ; B . ; M . ]. T -

    , . P-

    -

    .

    2.2 The Light Field Flowing into the Camera

    O insidethe camera body. W ,

    . T

    . I ,

    . N, I

    ,

    .

    I F .,

    :

    . T -

    .

    I ray diagram F .,

    u , x

  • 7/30/2019 Renng Thesis

    30/203

    chapter . light fields and photographs

    F .: P .

    . L u , u

    . I , x

    . O d

    (u, v) (x,y) . L

    L(x,y, u, v), L(x, u) d .

    T Cartesian ray-space diagram F . -

    - . T (x, u) C -. I

    - , F-

    .. T - d . A

    B [A B ] -

    . L H []

  • 7/30/2019 Renng Thesis

    31/203

    .. photograph formation

    F .: T .

    , .

    2.3 Photograph Formation

    I ,

    . T -

    , , ccd cmos -

    ,

    . E

    .

    I ,

    . F , ,

  • 7/30/2019 Renng Thesis

    32/203

    chapter . light fields and photographs

    F .: T .

    ,

    . A cmos

    [C W ],

    . N,

    ,

    F . .

    F . .

    T ( d) -

    x , u .

    O x ,

    -.

    I , - F . ,

  • 7/30/2019 Renng Thesis

    33/203

    .. photograph formation

    . T -

    - . T x ,

    u .

    T , ,

    x . L

    ,

    -

    F ..

    F .

    . I , x

    . C

    . F ,

    . I

    F .a, ,

    . T

    -. I , F .b

    , .

    T -

    x . A

    u , x . I -

    x , u

    x (F .a). T

    x (F .b). T

    , -

    , x .

    T

    x , . I, F .

    x ,

    T , . I , - -.

  • 7/30/2019 Renng Thesis

    34/203

    chapter . light fields and photographs

    (a) (b)

    F .: T x .

  • 7/30/2019 Renng Thesis

    35/203

    .. imaging equations

    . T :

    , .

    2.4 Imaging Equations

    T d - ,

    . H,

    d .

    ,

    . A

    C ,

    .

    T , F ., -

    . C

    [S . ]:

    EF(x,y) =1

    F2

    LF(x,y, u, v) cos

    4 dudv, (.)

    F , EF(x,y)

    (x,y), LF

    F, (x,y, u, v) . T cos4

    - optical vignetting. I

    . H, E .

    cmos .

    F , E . uv xy ,

    L .

    , cos4

    , - L(x,y, u, v) := L(x,y, u, v) cos4.

  • 7/30/2019 Renng Thesis

    36/203

    chapter . light fields and photographs

    -FOTQMBOF

    'JMNQMBOF

    0

    0

    0

    0

    0

    0

    F .:

    - .

    T - -

    . F - -

    -

    . S,

    ,

    .

    L

    x

    . A F ., -

    ,

    -

    -. I

    F,

    -

    LF (x, u) LF(x, u)

    E ..

    T

    x

    u LF - x u LF.

    B , u x,

    x u+ (x u) FF . A d x u, y v . A ,

    = F/F ,

    LF(x,y, u, v) = LFu+ x u

    , v+ y v

    , u, v

    = LF

    u

    1 1

    +x

    , v

    1 1

    +y

    , u, v

    . (.)

    T d

    . A

  • 7/30/2019 Renng Thesis

    37/203

    .. imaging equations

    , -

    . I I . [] ,

    [C . ; S

    . ; D . ; V . ; N ]. T

    [G B

    ].

    C E . . (x,y)

    F = F :

    E(F)(x,y) = 12F2 LF

    u

    1 1

    + x

    , v

    1 1

    + y

    , u, vdudv. (.)

    T d

    , d.

  • 7/30/2019 Renng Thesis

    38/203

  • 7/30/2019 Renng Thesis

    39/203

    3Recording a Photographs Light Field

    T

    . A

    , . T

    . E ,

    (, , F .).

    T integral photography L [] I [; ].

    . P

    , ,

    .

    I ,

    A W [] plenoptic camera. T

    . S .

    generalized light feld camera .

    A - .

    S . .

    T ,

    -

    .

  • 7/30/2019 Renng Thesis

    40/203

    chapter . recording a photographs light field

    3.1 A Plenoptic Camera Records the Light Field

    T , ,

    , F .. T ,

    . I ,

    . I ,

    ,

    .

    A - F .. T - ,

    . ,

    . T

    (x, u)

    - . A , -

    , . T

    . C C . T

    x .

    E -

    . F , F . , -

    . A -

    ,

    , F .. I -

    F . b,

    , .

    T

    ( F .). I ,

    ,

    . I

  • 7/30/2019 Renng Thesis

    41/203

    .. a plenoptic camera records the light field

    F .: S .

    . I

    , , . T

    ,

    . I ,

    u x ,

    .

    T - . F , ,

    . O , -

    ,

    . A ,

    . F, S .

  • 7/30/2019 Renng Thesis

    42/203

    chapter . recording a photographs light field

    .

    3.2 Computing Photographs from the Light Field

    T -

    . T -

    . F , F .a

    . A C , -, F . b.

    , .

    T -

    . S, ,

    ,

    , . T -

    F .a. F . b -

    . E - F . b. W

    F . b. T

    b . T

    - .

    T

    , . H, -

    . F ,

    ,

    S . C ,

    , C .

    A

    . T

    , , C . H,

  • 7/30/2019 Renng Thesis

    43/203

  • 7/30/2019 Renng Thesis

    44/203

    chapter . recording a photographs light field

    (z) (z)

    F .: R . T - .

  • 7/30/2019 Renng Thesis

    45/203

    .. three views of the recorded light field

    . A ,

    . S ,

    . T

    .

    T

    . E d d

    . M ,

    .

    Raw Light Field Photograph

    T

    , F .. M,

    ,

    F .: C -

    F ..

    , -

    . L , -

    ,

    -. E -

    . I

    ,

    (x,y) ,

    (u, v) .

    T F .

    :

    (I z),

    (I z). L I z,

    . T

  • 7/30/2019 Renng Thesis

    46/203

    chapter . recording a photographs light field

    (z) (z)

    F .: S- F .. I z z - , .

  • 7/30/2019 Renng Thesis

    47/203

  • 7/30/2019 Renng Thesis

    48/203

    chapter . recording a photographs light field

    (z) (z)

    F .: E F .. I z z - , .

  • 7/30/2019 Renng Thesis

    49/203

  • 7/30/2019 Renng Thesis

    50/203

    chapter . recording a photographs light field

    3.4 Resolution Limits of the Plenoptic Camera

    A ,

    . T

    , d ,

    - .

    Microlens Optics

    T u . I ,

    .

    M

    . T

    . H,

    (

    ),

    . A ,

    . T ,

    . H,

    , S ..

    T -

    , . W .

    T

    . T

    - , F ..

    P .

  • 7/30/2019 Renng Thesis

    51/203

  • 7/30/2019 Renng Thesis

    52/203

  • 7/30/2019 Renng Thesis

    53/203

    .. generalizing the plenoptic camera

    ( . mp)

    . A, , , . mp , .

    3.5 Generalizing the Plenoptic Camera

    A ,

    ,

    . T

    F . . T -

    . T

    .

    O , , ,

    -

    . I ,

    [A . ]. T ( ,

    ). I

    .

    N,

    . F .

    , -.

    F .a ,

    . T -

    u x -.

    F .b . T

    - ,

    . T

    . I C ,

  • 7/30/2019 Renng Thesis

    54/203

    chapter . recording a photographs light field

    (a) (b) (c)

    F .: G : .

    . H -

    ,

    . F , F .a

    . T -

    ,

    F .b c.

    F .c -

    . I

  • 7/30/2019 Renng Thesis

    55/203

  • 7/30/2019 Renng Thesis

    56/203

    chapter . recording a photographs light field

    ,

    , , , , .

    O --

    . T . I

    . A

    . M

    . A ,

    .

    T --

    , , . T ,

    (.. mp) . I ,

    (.. mp)

    , -

    . A ,

    vlsi ,

    .

    Components

    T -

    . M

    . T - , , ,

    F .b.

    O M fb. T K

    kaf-ce , . . F - C . W

  • 7/30/2019 Renng Thesis

    57/203

  • 7/30/2019 Renng Thesis

    58/203

    chapter . recording a photographs light field

    . O A O

    A ( -.-s). T - . T

    .

    Assembly

    T

    , .

    T -

    : . T . W -

    ,

    .

    , F .b. T

    . ,

    . W

    .F .

    . W

    , ,

    . A

    . T F .

    - .

    W

    . W . T

    - ,

    . T

    [S ] (. )

    - (/).

  • 7/30/2019 Renng Thesis

    59/203

    .. prototype light field camera

    -ICROLENSARRAY

    ,ENSHOLDER

    !DJUSTMENT

    SCREWS

    "ASEPLATE

    3EPARATIONSPRINGS

    0HOTOSENSOR

    $IGITALBACK#HIPPACKAGE

    MMMICROLENSES

    PIXELS

    MM

    -ICROLENSARRAYACTIVESURFACE

    F .: Le: S . E , - . Right: P .

    ,

    - . T (

    ) . T

    . T

    -.

    W -

    . T

    /, . T

    .

    Operation

    U , M -

    F . I, F ieee-,

  • 7/30/2019 Renng Thesis

    60/203

    chapter . recording a photographs light field

    . W -

    , , .

    -

    - . T

    . -

    C ,

    .

    T

    . I ,

    , . T

    - - /,

    . T

    . F , - /

    ( ). T

    -

    ( ).

    3.7 Related Work and Further Reading

    Integral Photography, Plenoptic Cameras and Shack-Hartmann Sensors

    O , [L-

    ; I ] , . I

    d , [O ; J

    O ]. I ,

    d - [D . ; O . ; N . ; Y N

    ]. T , - -

    d , d .

    G . [] -

    d . T

  • 7/30/2019 Renng Thesis

    61/203

    .. related work and further reading

    , -

    F .. A , .

    A

    . A W ,

    [A W ]. I , -

    ,

    . A ,

    , (-)

    . T -

    :

    , .

    A S-H [S P

    ; P S ]. I -

    .

    I , S-H ,

    -. S

    [B ]

    [H . ]), -

    [ ]. A -

    , [R W ].

    I , S-H

    [L . ; H ]. C -

    S-H

    -

    .

    Other Related Optical Systems

    , -

    [L H ;

  • 7/30/2019 Renng Thesis

    62/203

  • 7/30/2019 Renng Thesis

    63/203

  • 7/30/2019 Renng Thesis

    64/203

  • 7/30/2019 Renng Thesis

    65/203

    4Digital Refocusing

    T :

    . I F ., a-a

    S .. T ,

    / . T

    ,

    . I a,

    , I I

    . I .

    P

    digital reocusing . I a-a I

    I . T

    . I ,

    ,

    . T ,

    , ,

    .

    F .b , digitally extended

    depth o feld. I , . T

  • 7/30/2019 Renng Thesis

    66/203

    chapter . digital refocusing

    (a) (a) (a)

    (a) (a) (b)

    F .: E (aa) (b).

    .

    I b a-a [A .

    ],

    .

    A . T

    ,

    (snr). S ..

    ,

    u .

  • 7/30/2019 Renng Thesis

    67/203

  • 7/30/2019 Renng Thesis

    68/203

    chapter . digital refocusing

    Focus and Depth from Defocus and Auto-Focus

    O

    [S . ; K A ]. T

    depth rom deocus,

    [K

    ; P ; S ]. D

    - [N . ; P . ].

    H . A

    , .

    T - -

    - . B

    .

    Extended Depth of Field

    L , Waveront Coding D J [] -

    . I -

    . D . I ,

    , ,

    ,

    , - . D

    J ,

    .

    T

    .

    T , O . [], H-

    [], E K [] A . [] ( I ).

  • 7/30/2019 Renng Thesis

    69/203

    .. image synthesis algorithms

    M

    [ . ].

    4.2 Image Synthesis Algorithms

    T

    C . T

    .

    Refocusing

    T

    . C

    E .,

    (x,y):

    E(F)(x,y) =1

    2F2

    LF

    u(11/)+ x/, v(11/)+ y/, u, v

    dudv. (.)

    R LF xy F

    uv , F, E(F)

    ( F)O ,

    u v . T -

    S . -

    u v. (u(11/)+ x/, v(11/)+ y/, u, v)

    . T , LF -

    .

    H,

    . E E .

    -

  • 7/30/2019 Renng Thesis

    70/203

  • 7/30/2019 Renng Thesis

    71/203

    .. image synthesis algorithms

    (a): N (b): R (c): R

    F .: S-- , - .

    - (u, v),

    , , . F .

    , -, x

    y, . U - --

    - ,

    . F .a , = 1, 0 -

    . T

    .

    T -- - - F .. F ,

    . H,

  • 7/30/2019 Renng Thesis

    72/203

    chapter . digital refocusing

    (a): U

    (b): S- (c): U, (c): A

    F .: A - -- .

    --

    . T F .-c, -

    - . T -

    - -- . W -

    ,

    . F , ,

    F .-a - F .-b.

    T - (u, v)

    E .. T

    - , -

    .

    A - , -

    . T - . T - F . c.

    T - .

  • 7/30/2019 Renng Thesis

    73/203

    .. image synthesis algorithms

    Q d . T

    - E ..A

    . Vignetting F . c. T

    - (

    F .b c). A

    E . . T

    , ,

    LF .

    A

    . F ,

    . D

    . F . c

    , F . c.

    Extending the Depth of Field

    T . P-

    - . T digitally stopping down the lens,

    . S

    . T

    , ,

    . T snr ( F .b).

    I . L

    ,

    . T , . A -

    , ,.

    I -

    . T ,

    , . B

  • 7/30/2019 Renng Thesis

    74/203

    chapter . digital refocusing

    (a): U (b): S- (c): E dof

    F .: C - - .

    E ., snr , . T , F .c,

    - , . I -

    , digitally extending the depth o feld

    snr .

    T F . z .

    O -

    , C . T F .a . I

    ,

    . F .b F . z

    .

    T

    . F ,

  • 7/30/2019 Renng Thesis

    75/203

    .. theoretical refocusing performance

    F . . C

    , S ., I digital photomontage A . []

    .

    D

    . I

    . I

    graph-cut optimization [B . ] ,

    gradient-domain diusion [P . ] . F

    ,

    . B ,

    , .

    4.3 Theoretical Refocusing Performance

    T -

    . T -

    - - . A

    N , - N

    , N .

    C F . T -

    - d . B-

    ,

    . T -

    C ,

    .

    F,

    N - . A

    N= . I

    ,

  • 7/30/2019 Renng Thesis

    76/203

    chapter . digital refocusing

    . I

    , , .A

    . S

    - N,

    N

    . I , / N= ,

    / . P

    ,

    .F . . I I b,

    , . I

    , - :

    . T, ,

    . I I c,

    . O

    I a. T

    . I, ,

    , .

    D .

    T eective depth o ocus

    . T

    . I

    - . I

    - . I

    , eective

    , -

    . F

  • 7/30/2019 Renng Thesis

    77/203

    .. theoretical noise performance

    -FOT%FTJSFE

    GPDBMQMBOF%FQUIPGGPDVTPGBDPOWFOUJPOBMDBNFSB

    &FDUJWFEFQUIPGGPDVTPGUIFMJHIUmFMEDBNFSB

    F .: I . T , .

    (..

    , N ). T -

    N. T N -

    .

    T N ,

    F .. T - N

    ,

    . T : -

    - .

    4.4 Theoretical Noise Performance

    A -

    . F ,

    (.. -

    ). H,

  • 7/30/2019 Renng Thesis

    78/203

    chapter . digital refocusing

    NN (..

    N2

    ).L -

    /A /(A N), /A . A ,

    -

    . F,

    /(A N) .H, N2

    . I 1/N2

    - . A, ,

    N2 .

    A , snr -

    ? T , N2

    ? T ,

    , - ,

    . T

    P [G],

    . F , , ,

    . G , -

    P ,

    . U - , snr

    N , N2 , P

    N .

    T , . L ,

    , NN . I , -

    N . T

  • 7/30/2019 Renng Thesis

    79/203

    .. experimental performance

    . T

    . T ,

    . I , N2

    , 1/N2 1/N2

    . T snr N. O

    - -

    , .

    4.5 Experimental Performance

    T

    . I ,

    ,

    .

    Experimental Method

    T ,

    . , I

    away . I ,

    .

    I I

    - . T, ,

    ?

    A -

    , - . I

    .

    S -

  • 7/30/2019 Renng Thesis

    80/203

    chapter . digital refocusing

    ,

    - .A

    - -

    . T

    ,

    . I I ,

    ,

    .

    A

    . N, , P

    . T snr

    .

    Refocusing Results

    I / , -

    / /. T - . T -

    . T

    -.

    F .

    . N / -

    / . T

    / , /. I

    / .F . ,

    (mtf) . I mtf

    F [W ].

    F , I ,

    , F .

  • 7/30/2019 Renng Thesis

    81/203

    .. experimental performance

    / R L F / C

    / C / C / C

    F .: E : .

  • 7/30/2019 Renng Thesis

    82/203

    chapter . digital refocusing

    $POWFOUJPOBM

    3FGPDVTFE-JHIU'JFME

    F .: E : mtf . H 0 N .

    F . mtf . T

    mtf . T

    - , ., , , , . A

    - , ,

    mtf. T , /

    / . T mtf

    / ,

    F ..

    A , , -

    , / . I ,

    / , -

    .

    T . A F .,

    ,

    . A, -

    , S .. W

    / ,

  • 7/30/2019 Renng Thesis

    83/203

    .. experimental performance

    / , -

    .

    Noise Results

    I . A -

    , I snr /

    / . T -

    snr ., (

    . / ). T

    snr P , S-

    .. I snr

    - , I -

    . T snr --

    - .

    / C / C / R L F

    F .: E .

  • 7/30/2019 Renng Thesis

    84/203

    chapter . digital refocusing

    (a): C

    (b): U (b): R (b): E dof

    F .: R .

    F . . T - F .,

    ,

    . I F ., / -

    - . T /

    , . N /

  • 7/30/2019 Renng Thesis

    85/203

    .. technical summary

    (a): /, / L F, E dof

    (b): /, / C P

    (c): /, / C P

    F .: C .

    / . T / ,

    . N , / . T

    snr . / /

    .

    4.6 Technical Summary

    T

    - .

    F . ,

    . N

    I b ,

    /

  • 7/30/2019 Renng Thesis

    86/203

    chapter . digital refocusing

    - / . H,

    / . I I b, . N, ,

    . W .

    I b

    .

    F .

    . T

    F . F .a. I I b,

    / , I ( /)

    . A ,

    . I

    I a,

    P . I , I c, I / ,

    - /

    I a. T -

    . T

    . H, I c

    . F .

    -

    .

    I ,

    . T

    . T

    .

    4.7 Photographic Applications

    ,

    . T -

    .

  • 7/30/2019 Renng Thesis

    87/203

    .. photographic applications

    Portraiture

    G

    . I , -

    . F

    , -- ,

    ,

    . I , .

    I , . O

    , .

    U . H, ,

    . W

    ,

    , . F .a ,

    - ,

    . S ,

    , F .c.

    (a): M- (b): R (c): R -

    F .: F -

    A ,

    . I ,

  • 7/30/2019 Renng Thesis

    88/203

  • 7/30/2019 Renng Thesis

    89/203

    .. photographic applications

    F .: T .

    Action Photography

    A

    . L ,

    . T -

    , ,

    .

    F . . T

    a-a b-b

    . T / .

    I c-c .

    T ,

    / .

    Landscape Photography

    W , . T

    ,

    . I

    .

  • 7/30/2019 Renng Thesis

    90/203

    chapter . digital refocusing

    (a) (a) (a)

    (b) (b) (b)

    (c) (c) (c)

    F .: D - .

  • 7/30/2019 Renng Thesis

    91/203

    .. photographic applications

    A ,

    , . S . C ,

    . F , -

    ,

    .

    T , -

    ,

    - -

    .

    (a): R (a): R (a): E dof

    (b): R (b): R (b): E dof

    F .: E .

    F . . I ,

  • 7/30/2019 Renng Thesis

    92/203

    chapter . digital refocusing

    (a) (a) (a)

    (b) (b) (b)

    F .: D .

    , . T

    - ,

    .

    Macro photography

    T ,

    , ,

    . , : , -

    ,

  • 7/30/2019 Renng Thesis

    93/203

    .. photographic applications

    (a): H (b): L

    (c): F (d): C

    F .: M

    .

    F , ,

    .

    B ,

    . M - . I , ,

    . T ,

    . A , -

    , ,

  • 7/30/2019 Renng Thesis

    94/203

  • 7/30/2019 Renng Thesis

    95/203

    5Signal Processing Framework

    A , F -

    -

    . T F

    . F ,

    . I

    ,

    C .

    S . , F

    . T ,

    F . L ,

    F

    , B [].

    5.1 Previous Work

    T F C . [].

    T , , - -

    . T

    - ,

    .

  • 7/30/2019 Renng Thesis

    96/203

    chapter . signal processing framework

    I I . []. T

    F , . T -

    , -

    .

    M , S . []

    - - . T -

    -- . I ,

    - . A

    , .

    F, D . [] F

    ,

    , . W

    ,

    .

    5.2 Overview

    T F -

    . W

    (C ) ,

    . I , F

    d d .

    T , F d

    d F . T F S P T,

    S ., .

    F . -

    , F . T

    , ,

  • 7/30/2019 Renng Thesis

    97/203

    .. overview

    N R R

    F .: T ( ) F ( ).

  • 7/30/2019 Renng Thesis

    98/203

    chapter . signal processing framework

    -. T -

    F . F . T - F ,

    (mr) . I , kx ku

    F- x u ( ky

    kv). T F-

    F . T . T

    ,

    , .

    T F - - R B [] F

    S T ( C S T F P-S

    T). B -

    . H,

    ,

    (ct) (pet) [M

    ], (mr) [N ].

    T d d , -

    d -

    d . N,

    , - -

    . F , mr

    S .

    F-.

    P F S P T-

    -

    . S . -

    , . I ,

    S .

    -.

    F .c -:

  • 7/30/2019 Renng Thesis

    99/203

    .. overview

    $POUJOVPVTMJHIUFME 1FSGFDUQIPUPHSBQI 3FDPSEFEMJHIUFME

    1IPUPHSBQIDPNQVUFEXJUIPVUSFGPDVTJOH

    3FGPDVTFEQIPUPHSBQIXJUIGVMMTQBUJBMSFTPMVUJPO

    3FGPDVTFEUPPGBSXJUIMPTTJOTQBUJBMSFTPMVUJPO

    3FEVDFEQIPUPCBOEXJEUI

    F .: F- .

    ,

    . T F-

    . . I , , -

    . W -

    , -

    ,

    ,

    .

    W - , -

    . F . , d

  • 7/30/2019 Renng Thesis

    100/203

    chapter . signal processing framework

    . S

    -, - (F .df).

    T , ,

    (F .b). T -

    - . B -

    - ,

    .

    T S ., F .e f

    . T ,

    F .: T

    F

    .

    ,

    - . U ,

    -,

    (

    ) . H, -

    ,

    (-

    ). T

    F .. A

    . S-

    . , C

    .

    S . F S P T

    F S D R . T -

    d F -

    d F . T

    O(n2 log n), n . T

  • 7/30/2019 Renng Thesis

    101/203

    .. photographic imaging in the fourier domain

    O(n4) , -

    d .

    5.3 Photographic Imaging in the Fourier Domain

    P>1P

  • 7/30/2019 Renng Thesis

    102/203

    chapter . signal processing framework

    A ,

    F S T. T F S T [D ] d d F F -

    d . T

    , F .. C,

    dc ( ) ,

    . F ,

    . I

    ,

    .

    T ,

    , d

    d F . W , S .. ..

    ,

    E . ..

    5.3.1 Generalization of the Fourier Slice Theorem

    L -, d . A

    R [L M ], -

    N M . T

    -

    change o basis (.. N- ). T

    .

    I Pj L INM N- M-

    N M : INM [f] (x1, . . . , xM) =f(x1, . . . , xN) dxM+1 . . . dxN.

    S L SNM N- M - NM -: SNM [f] (x1, . . . , xM) = f(x1, . . . , xM, 0 , . . . , 0).

  • 7/30/2019 Renng Thesis

    103/203

    .. photographic imaging in the fourier domain

    C B L B

    N- . I B N- NN, B[f] (x) = f(B1x), x N- , B1 B.

    S L a

    , a[f](x) = a f(x).

    F T L FN N- F -, FN . FN [f] (u) =

    f(x) exp (2i (x u)) dx,

    x u N- ,

    N- .

    W , :

    generalized fourier slice theorem Let fbe an N-dimensional unction. I we

    change the basis of, integral-project it down toM o its dimensions, and Fourier transorm the

    resulting unction, the result is equivalent to Fourier transorming f, changing the basis with

    the normalized inverse transpose o the original basis, and slicing it down to M dimensions.

    Compactly in terms o operators, the theorem says:

    FM INM B= SNM BT|BT| F

    N, (.)

    where the transpose o the inverse oBis denoted byBT, andBT is its scalar determinant.A A a..

    F . N-

    , M- , F . O ( F .

    .) N= , M = d (B= R). I , (R = RT), RT

    1. A , F .

  • 7/30/2019 Renng Thesis

    104/203

  • 7/30/2019 Renng Thesis

    105/203

    .. photographic imaging in the fourier domain

    , . R

    A a.. A , () -

    .

    T . F -

    ,

    N- , GN, F , GN. F

    , M ,

    GM. A F ., ( -

    ) O(n

    N

    ) , n N. A GN ( ) -

    F ( ), O(nM log n)

    F F .

    F ,

    . F ,

    ct , - -

    . I ,

    F F . W

    .

    5.3.2 Fourier Slice Photograph Theorem

    T F S P T,

    . T IO G F S T.

    T I O (E .) -

    ():

    P [LF] = 12F2

    I42 B [LF] , (.)

  • 7/30/2019 Renng Thesis

    106/203

    chapter . signal processing framework

    :

    I C B B d :

    B =

    0 1 00 0 1 0 0 1 0

    0 0 0 1

    B1 =

    1/ 0 1 1/ 00 1/ 0 1 1/0 0 1 0

    0 0 0 1

    N B B1 , B,

    generic G F S T. D I42 E . E ..

    W G F S T (E .) -

    E . F- . F, - E .

    P = 12F2

    F2 (F2 I42 B).

    I , G F S T (E .)

    ,

    P = 12F2

    F2 S42 B

    TBT F

    4

    .

    F, BT

    = B1 = 1/2, :

    P = 1F2

    F2 S42 BT F4, (.)

    (P) d F : d (S42 BT), d (F2), (1/F2).

    B ,

    F :

  • 7/30/2019 Renng Thesis

    107/203

    .. photographic imaging in the fourier domain

    F P I O

    P =1F2

    S42 BT. (.)

    I P ,

    S42 B. T :

    P[G](kx, ky) =1

    F2G( kx, ky, (1 ) kx, (1 ) ky). (.)

    A E . E . , , :

    fourier slice photograph theorem

    P = F2 P F4. (.)

    A photograph is the inverse d Fourier transorm o a dilatedd slice in the d Fourier trans-

    orm o the light feld.

    F .

    . T P F- P. T , F .

    I ,P F F

    F . I

    ,

    .

    T P, ,

    P, . T

    P (E .) P (E .). B F- -,

    . I , F S P T C-

    T, .

    F ,

    . T

  • 7/30/2019 Renng Thesis

    108/203

    chapter . signal processing framework

    2P

    LF .F

    EF 'F

    O(n4)

    O(n4 logn)

    O(n2)

    O(n2 logn)

    4QBUJBMEPNBJO*NBHJOH

    F4

    F2

    'PVSJFS5SBOTGPSN

    'PVSJFS5SBOTGPSN

    'PVSJFSEPNBJO*NBHJOH

    F .: F S P T. d LF, - d EF, F ,LF EF.

    F ., n d .

    T P (O(n2))

    P (O(n4)). T S ..

    5.3.3 Photographic Effect of Filtering the Light Field

    A E .. I

    , ,

    , ? T -

    .

    filtered light field imaging theorem A d convolution o a light feld results

    in a d convolution o each photograph. Te d flter kernel is simply the photograph o the dflter kernel ocused at the same depth. Compactly in terms o operators,

    P C4h = C2P[h] P, (.)

    where we have expressed convolution with the ollowing operator:

  • 7/30/2019 Renng Thesis

    109/203

    .. photographic imaging in the fourier domain

    Cv CNh N-

    h, CN

    h [F](x

    ) =F(x

    u

    ) h(u

    ) du

    x

    u

    N- , F h N- .

    F . . O , LF d

    , LF d d h. EF EF

    , . T EF d

    EF, d d , h.

    I , ,

    . A a. . A ,

    F S P T C T

    F . I ,

    , .

    I

    P

    LF

    EF

    $POWPMVUJPO

    $POWPMVUJPO

    P

    LF

    EF

    C4h

    'JMUFSLFSOFM

    C2P [h]

    'JMUFSLFSOFM

    P

    F .: F L F P T. d LF, , LF, EF EF.

  • 7/30/2019 Renng Thesis

    110/203

    chapter . signal processing framework

    . A ,

    . I d,

    (x,y, u, v) = (x) (y) (u) (v), d :

    (x) =

    1 |x| < 120

    .

    W (x,y, u, v) ,

    P [LF ] = (P C4) [LF]

    = (C2P[] P) [LF]= P [] P [LF] .

    F , , fg d d f g, a b d - d a b.

    T

    . T (P [LF]) d P []. T d d . W d ? T

    d .

    T

  • 7/30/2019 Renng Thesis

    111/203

    .. photographic imaging in the fourier domain

    . T d , - ,

    , .

    B d , d d

    , - . E , , d .

    T F L F I T

    . F ,

    , ,

    d , h(x,y, u, v). A ,

    , (..

    ) h(x,y, u, v). G ,

    P[h]. C d : . A

    .

    O , -

    . F, ,

    F -

    . T

    (C ), , - . A

    -, -

    . I , -

    ,

    .

  • 7/30/2019 Renng Thesis

    112/203

    chapter . signal processing framework

    5.4 Band-Limited Analysis of Refocusing Performance

    T F L F I T

    , . W -

    ? H ,

    -

    ?

    T , S .,

    - . S . -

    -. T ,

    F L F I T.

    T - F

    d , (x,y, u, v). B C T, F

    F .

    I F - ,

    . L - , sinc(x,y, u, v) = sinc(x) sinc(y) sinc(u) sinc(v),

    sinc(x) =sin x

    x.

    I , - , LF,

    exact , LF, d sinc:

    LF =C

    4lowpass [LF] , where

    lowpass(x,y, u, v) =1

    (xu)2 sinc

    xx

    ,y

    x,u

    u,v

    u

    .

    I , x u

    , . T 1/(xu)2 - -

    sinc.

  • 7/30/2019 Renng Thesis

    113/203

    .. band-limited analysis of refocusing performance

    Analytic Form for Refocused Photographs

    O , EF ,

    - , LF. T F L F P

    T. L = F/F,

    EF = PLF

    = P

    C4lowpass [LF]

    = C2P[lowpass] [P [LF]] = C2P[lowpass] [EF ] ,

    EF exact F. T -

    d- . T d d sinc , P [lowpass].

    I d sinc d sinc :

    P [lowpass] = P

    1

    (xu)2 sinc

    xx

    ,y

    x,u

    u,v

    u

    =1

    D2x sinc

    x

    Dx,y

    Dx

    , (.)

    N d sinc , :

    Dx = max(x, |1 |u). (.)

    T , F S P

    T , ( A a.).

    T d sinc, -

    - . T

    d (E .)

    .

    Interpretation of Refocusing Performance

    R x u. L

    Wx,

  • 7/30/2019 Renng Thesis

    114/203

    chapter . signal processing framework

    Wu. W , Nx = Wx/x

    Nu =Wu/

    u.

    S = (F/F) u =Wu/Nu,

    |x| |(1 )u|

    |F F| xNuF

    Wu. (.)

    T , F,

    , .. . W

    N- ,

    - .

    P, E . E .,

    (x). N,

    , Wx, (Wx). T

    ( E .). I > 1, ,

    , S ..

    T, N

    Wxx

    =Wxx

    . (.)

    T ,

    . T

    E .. N

    , Nu, S ..

    I , ..

    |F F| > xNuF

    Wu,

    - , EF , |1 |u,

  • 7/30/2019 Renng Thesis

    115/203

    .. fourier slice digital refocusing

    x ( E .). T ,

    Wx|1 |u ,

    ,Wx/x. I ,

    , N- .

    R- -

    . S = F/F u =Wu/Nu,

    |1

    |

    Wxu

    =Wx

    Wu

    NuF |F

    F|

    . (.)

    B NuFWu

    f- Nu

    , WuNuF |F F| - -

    |F F|.I , ,

    , Nu , S .. N

    , Nu. D ,

    .

    I , - -

    -

    C .

    5.5 Fourier Slice Digital Refocusing

    T F S P T ,

    . T

    : - (

    ). T , ..

  • 7/30/2019 Renng Thesis

    116/203

  • 7/30/2019 Renng Thesis

    117/203

    .. fourier slice digital refocusing

    wwww

    F

    wwww2>

    wwww2=

    wwww2>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>:

    8>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>:

    1

    3

    -JHIU'JFME'PVSJFS4QFDUSVN

    'PVSJFS4MJDF&YUSBDUJPO

    'PVSJFS4MJDFT

    *OWFSTF'PVSJFS5SBOTGPSN

    3FGPDVTFE1IPUPHSBQIT

    -JHIU'JFME

    'PVSJFS5SBOTGPSN

    F .: F S D R .

  • 7/30/2019 Renng Thesis

    118/203

  • 7/30/2019 Renng Thesis

    119/203

  • 7/30/2019 Renng Thesis

    120/203

    chapter . signal processing framework

    (a): W - (b): W -

    F .: R .

    F [M ], -

    d F .

    T d F -

    . F . - K-

    B , .

    U, -

    ,

    .

    Suppressing Aliasing Artifacts

    T , -

    -. O

    , - d F (PF F4) [LF]. I- F

    . T - .

    E

    F . S,

    . H,

    ,

  • 7/30/2019 Renng Thesis

    121/203

    .. fourier slice digital refocusing

    . A ,

    .

    (a): (F ) (b): I F (a)

    (c): N (d): C (b)

    F .: R F .

    F . . I a F . I b

    . S

    , I d. F , c

    . T

    .

  • 7/30/2019 Renng Thesis

    122/203

    chapter . signal processing framework

    O , -

    . T , -

    .

    T : .

    A , sinc -

    . O - -

    . T -

    - ,

    J . [] , M [] F .

    (a): Q ( )

    (b): K-B ( .)

    (c): K-B ( .)

    F .: A . R .

    J . , -, K-

    B . T K-B

    . F .

    K-B . S, K-

    B . .

    A , bokeh

    . U

  • 7/30/2019 Renng Thesis

    123/203

    .. fourier slice digital refocusing

    (a): W (b): W

    F .: A . K-B - .

    LF(x,y, u, v) A(u, v) -

    . F , A(u, v)

    (u, v) . I F ,

    F

    A(u, v). T F [L ].

    T -

    - F [L ;

    M ]. T ,

    - .. I F .,

    I a -

    I b -.

    Implementation Summary

    I S ., . I

    - ,

    . P (..) .

    . P- F

    .

  • 7/30/2019 Renng Thesis

    124/203

    chapter . signal processing framework

    F

    S

    F .: C F .

    I , d F ,

    . U - K-B . A .

    . F , .

    ( ) .

    . O d F . A F , -

    .

    Performance Summary

    T - F S

    - C .

    D , F . .

  • 7/30/2019 Renng Thesis

    125/203

    .. fourier slice digital refocusing

    F

    S

    F .: C F II.

    F . T

    ( = ). F .

    F ,

    - . R

    ,

    , .

    N, F- .. A , -

    F- .

    I , cpu F

    . W , F [N ]. T F

  • 7/30/2019 Renng Thesis

    126/203

    chapter . signal processing framework

    S uv ,

    , F .

    5.6 Light Field Tomography

    T -

    F . T

    . T light feld tomography ct -

    .

    L d d . W

    , F d -,

    , . A

    , /2 /2. H, d d ,

    F . A

    d R [D ] . T d

    F S T d R

    .

    A

    ,

    F . I ,

    d .

    I , d

    . I F S P T ( E-

    . ) - d

    d d F :

    ( kx, ky, (1 ) kx, (1 ) ky)where [0,), and kx, ky R

    . (.)

  • 7/30/2019 Renng Thesis

    127/203

    .. light field tomography

    I , d

    d . I d F .O d

    . D d

    . O d slit light feld d

    . T

    d d (

    ). B ,

    d d

    .

  • 7/30/2019 Renng Thesis

    128/203

  • 7/30/2019 Renng Thesis

    129/203

    6Selectable Refocusing Power

    T -

    ,

    . T

    . C

    -,

    . T generalized light feld camera.

    O - -

    . T

    ,

    . H, ,

    , - -

    . F .

    . N

    ,

    .

    T - F .

  • 7/30/2019 Renng Thesis

    130/203

  • 7/30/2019 Renng Thesis

    131/203

    .. sampling pattern of the generalized light field camera

    (a): = 1 (b): = . (c): = . (d): = 0

    F .: D .

    = 0. A S .,

    beyond .

    F . -. T

    F .. N

    F . . - , -. A

    F . , -

    . T

    ( x ), ( u ). A

    -

  • 7/30/2019 Renng Thesis

    132/203

    chapter . selectable refocusing power

    , . H, -

    .

    R ,

    . H,

    . A = 0

    , .

    T ,

    . I ,

    MM M

    M,

    MM,

    M = max((1 )M, M). (.)

    Deriving the Sampling Pattern

    F . - , -

    - . T

    .A F .

    . E -

    . D

    -,

    C . T

    -, , -

    .F .a , , i,

    . F .b -

    , . L

    - :

    , xi, , si, , wi. I

  • 7/30/2019 Renng Thesis

    133/203

  • 7/30/2019 Renng Thesis

    134/203

  • 7/30/2019 Renng Thesis

    135/203

    .. optimal focusing of the photographic lens

    .

    N -. F .e

    , . A

    , F .f

    u

    .

    A , ,

    - - . T

    F .f

    F . . T

    , .

    The Problem with Focusing the Microlenses Too Close

    T

    ( < ). A -

    closer ,

    urther ( > ). T - . T

    , . B

    () , - -

    ,

    -. F ,

    .

    6.2 Optimal Focusing of the Photographic Lens

    A -

    . I

    . I , -

  • 7/30/2019 Renng Thesis

    136/203

  • 7/30/2019 Renng Thesis

    137/203

  • 7/30/2019 Renng Thesis

    138/203

    chapter . selectable refocusing power

    , E . (1 ),

    . H (1 ). T , . E

    (1 ).I , , -

    F, Fopt. D -

    F

    max((1 )M,M).

    6.3 Experiments with Prototype Camera

    T ., -

    . I

    . T -

    S .: I -,

    . I , I

    .F . - : -

    ( ), -

    ( ). T -

    / iso

    . W .

    . O -

    .

    N = .

    , -

    . I , = .,

    - . H, -,

  • 7/30/2019 Renng Thesis

    139/203

    .. experiments with prototype camera

    = .

    = .

    W-

    0

    F .: D .

    . I, - . .

    ( ) .

    T

    . T

    , . F . . E . . ,

    . , -

    , F ..

    A , 0 -

    - [M N ] . T

    ,

    - . N

    -

  • 7/30/2019 Renng Thesis

    140/203

    chapter . selectable refocusing power

    C R- C R-

    F .: C

    -, = . ( ), = 1.

    , - -

    - .

    6.4 Experiments with Ray-Trace Simulator

    ,

    P H - [] . F .

    -. T

    - F ., , -

    . T F . = ., = .. T

    . T

  • 7/30/2019 Renng Thesis

    141/203

  • 7/30/2019 Renng Thesis

    142/203

    chapter . selectable refocusing power

    T -

    . T - iso . T

    -. A -

    , ,

    .

    MTF Analysis

    T - mtf ,

    - . T .

    A

    . B -

    ,

    .

    T mtf

    . F ,

    , - -. T

    . T

    F mtf -

    -.

    T mtf

    . T

    mtf . A

    mtf, , mtf .

    F .a

    . T - -

    , .

  • 7/30/2019 Renng Thesis

    143/203

    .. experiments with ray-trace simulator

    . N .

    F .a . F, ( ) , . T -

    - = ( = 0, ., ., .

    ), , -

    (1 ) . T , -. T

    . T

    , E-

    .. A , (

    ). T ,

    .

    F .b - E-

    ., ,

    F .c. T

    ,

    E .. T F .c .

    F . - F .b F .c -. T ,

    . T

    . T F .a

    = .

    .

    T E ..

    T F .b,

    .c .d. R

    E ..

    T

    ,

    . I F .b, .c .d, -

    ,

  • 7/30/2019 Renng Thesis

    144/203

    chapter . selectable refocusing power

    (FOFSBMJ[FEMJHIU

    FMEDBNFSBDPOHVSBUJPOT

    $FOUFSFEWFSTJPO

    $VTUPNJ[FEQMFOPQUJDDBNFSBT

    3FMBUJWFPQUJDBMNJTGPDVTJNBHFTJEF

    4QBUJBMGSFRVFODZGPS

    DVUPDZDMFTNN

    NJDSPONJDSPONJDSPO

    NJDSPONJDSPO

    F .: mtf .

    ( ). I

    , between ,

    . T , , 1

    , F .e.

  • 7/30/2019 Renng Thesis

    145/203

    .. experiments with ray-trace simulator

    NJDSPO

    NJDSPO

    NJDSPO

    NJDSPO

    NJDSPO

    NJDSPO

    NJDSPO

    NJDSPO

    3FMBUJWFPQUJDBMNJTGPDVTJNBHFTJEF

    4QBUJBM

    GSFRVFODZGPSDVUPD

    ZDMFTNN

    F .: mtf II.

  • 7/30/2019 Renng Thesis

    146/203

    chapter . selectable refocusing power

    Summary

    T

    -

    . T

    , - -

    . A

    . T ,

    ,

    .F , -

    . A

    . T

    -

    ,

    .

    T ,

    . T

    -

    . F ,

    ,

    .

    T - :

    . B ,

    .

  • 7/30/2019 Renng Thesis

    147/203

  • 7/30/2019 Renng Thesis

    148/203

    chapter . digital correction of lens aberrations

    . T -

    , W [K ]. I -

    . I , C

    . F, ,

    D , .

    -

    . Z

    F .: S

    .

    . Z -

    ,

    . H, -

    -

    ,

    [M ]. A ,

    ,

    [D L ]! ,

    - [S ],

    .

    O

    [K ].

    T -

    . T -

    . T :

    , -

    . D

    .

    F ,

  • 7/30/2019 Renng Thesis

    149/203

    .. previous work

    ( = 1) . I ,

    , . T - , -

    . T ,

    , - .

    7.1 Previous Work

    R- . P

    - . I S E P,

    C L H [ A ] -

    [K ]. A - ,

    , /., . T

    , ,

    - ,

    [N ].

    K -

    [K ]. T

    . O

    - - lasl

    L A [B ].

    I , -

    - [P

    C ; C . ], - - [K . ]. T C .

    P C M-

    C [K ]. I

    , - ,

    - [P H ] I

  • 7/30/2019 Renng Thesis

    150/203

    chapter . digital correction of lens aberrations

    . O

    - - .I , -

    [M . ]. T

    .

    7.2 Terminology and Notation

    T -

    . T two - :

    ideal-, , aberrated-

    , . I

    ,

    . T ,

    -. L

    (x,y, u, v)

    (x,y, u, v).

    T - . A -

    . I ,

    . I

    , G ,

    [S ]. T G -

    , -

    . T

    G .

    T , C, -

    :

  • 7/30/2019 Renng Thesis

    151/203

  • 7/30/2019 Renng Thesis

    152/203

  • 7/30/2019 Renng Thesis

    153/203

    .. visualizing aberrations in recorded light fields

    F .: A - - .

  • 7/30/2019 Renng Thesis

    154/203

    chapter . digital correction of lens aberrations

    F .d.

    F .b :

    . T . A

    - . F, S .,

    .

    T - ,

    .

    E - F .. T . F .

    - ,

    . I

    ,

    .

    7.4 Review of Optical Correction Techniques

    T -

    .

    , .

    F . . T - ,

    . T -

    ray-intercept curves

    ,

    , - . T F . . T

    ( u -

    ), . S

    , x . O

    , .

  • 7/30/2019 Renng Thesis

    155/203

  • 7/30/2019 Renng Thesis

    156/203

    chapter . digital correction of lens aberrations

    I ,

    . T , , F . .

    (F .).

    7.5 Digital Correction Algorithms

    A ,

    : . -

    ,

    - . I ,

    ,

    .

    F . -,

    . F .a

    , - . T . F .b

    (x, u) - , (x, u) -

    . E

    . T F .a.

    F .c

    . T

    . I , F .d

    . N

    , , - F .c.

    F .ac : pixel-order

    , . A -

    ray-order, .

  • 7/30/2019 Renng Thesis

    157/203

    .. digital correction algorithms

    (a) (b)

    (c) (d)

    F .: R- .

    T -

    . T :

    -

    . T . T

    - . T -

    .

    Pixel-Order Image Synthesis

    T -

    . I

    .

    . S . A M-C

  • 7/30/2019 Renng Thesis

    158/203

    chapter . digital correction of lens aberrations

    .

    . C - .

    . R -

    ,

    .

    . E

    . T

    d . L- -

    . S, .

    . A .

    Ray-Order Re-Projection of the Light Field

    T - -

    . I

    .

    . S

    . A M-C

    ,

    .

    . , , -

    .

    . R

    .

    . I . A ,

    .

    A , ,

    .

  • 7/30/2019 Renng Thesis

    159/203

    .. digital correction algorithms

    Confidence Weighting for Increased Contrast Enhancement

    T -

    . W

    . F, - F .,

    . S, -

    ,

    , . F , F .d

    .

    T -

    , . T

    -

    . T

    weighted - .

    I , I ,

    x y .

    F , .

    w(x,y) = h(x) h(y), where

    h(x) =

    1, x 1exp

    (1x)2

    22

    , x > 1

    . (.)

    I , G -

    . T x y

    , . I

    = G -. C

    x

    y, (x,y, u, v), .

    F . . E

    . T

    ,

    . A

  • 7/30/2019 Renng Thesis

    160/203

    chapter . digital correction of lens aberrations

    (z) (z) (z)

    F .: W ( ) . E .

  • 7/30/2019 Renng Thesis

    161/203

    .. digital correction algorithms

    - , (z z).

    C , .

    R -

    . E . ,

    . C

    . T -, ,

    ( 1) -

    . F ,

    F . , -

    . H,

    - . W

    . A -

    , .

    Computing the Projected 2D Size of Light Field Samples

    C x y E .

    d . I ,

    ,

    C, . I , x

    - :

    x 1x

    Cxx x +

    Cxy y +

    Cxu u +

    Cxv v

    , (.)

    I C :

    Cx,y, u, v

    =

    Cx(x

    ,y, u, v),

    Cy(x,y, u, v),

    Cu(x,y, u, v),

    Cv(x,y, u, v)

    = (x,y, u, v) . (.)

  • 7/30/2019 Renng Thesis

    162/203

  • 7/30/2019 Renng Thesis

    163/203

    .. correcting recorded aberrations in a plano-convex lens

    (a) (b)

    F .: S- - .

    F .a. T

    - .

    I -

    - ( ). A

    , / . I

    -

    , F .,

    .

    I M-C -. T

    ,

    . T

    -. A -, I

    .

  • 7/30/2019 Renng Thesis

    164/203

  • 7/30/2019 Renng Thesis

    165/203

  • 7/30/2019 Renng Thesis

    166/203

  • 7/30/2019 Renng Thesis

    167/203

    .. simulated correction performance

    7.7 Simulated Correction Performance

    I ,

    . T , [S ]

    - (psf)

    . T psf

    . I -

    , . O

    . I

    - ?

    ,

    .

    7.7.1 Methods and Image Quality Metrics

    T . A

    , -

    . T , ,

    (. mp). A NN , N= (), N= . S

    , N . N= , . -

    , [A

    ]. . [M ],

    , N= ,

    . .

    T psf

    . C psf ,

    . I ,

    . F psf -

    , - ,

  • 7/30/2019 Renng Thesis

    168/203

    chapter . digital correction of lens aberrations

    psf .

    A psf , -. T

    (a): S

    (b): C

    F .: psf rms

    .

    psf d

    , . F ,

    psf

    . A ,

    ( -

    ). F -, psf .

    I , , I

    psf.

    T

    (rms) ,

    . T rms

    psf

    . A ,

    rms - . O

    ,

    :

    . F .a

    psf , rms .

    O , rms ,

    . F

    , F .b psf coma.

    T - psf

    psf . C

    [S ]. T rms

    , .

    T rms , d

  • 7/30/2019 Renng Thesis

    169/203

    .. simulated correction performance

    . T rms

    d , - -. A , -

    ,

    .

    A , rms eective res-

    olution. T . I

    F .: E

    .

    , ,

    rms (F .). T . I

    ,

    (

    2R)2,

    R rms ( ),

    2R

    . T

    -

    psf

    . E psf-

    .

    T mtf. T mtf

    ,

    . I

    F . I, I

    mtf : psf

    F . A

    mtf [P . ].

    T mtf N

    .

    S mtf F psf, - psf.

  • 7/30/2019 Renng Thesis

    170/203

    chapter . digital correction of lens aberrations

    T mtf,

    . I ,

    mtf. A mtf

    .

    7.7.2 Case Analysis: Cooke Triplet Lens

    L psf . T /.

    C [ E ]. F . psf. C a,

    b c : ,

    - , .

    T

    . ,

    . T -

    . T

    . I , . F , D a

    . I , D c

    , .

    T F . , -.

    T , x, , , u,

    . T N= . T

    .

    I D c,

    . I ,

    . I ,

    D c ,

    - , .

    T -

  • 7/30/2019 Renng Thesis

    171/203

    .. simulated correction performance

    (a)

    (a)

    (a)

    (b)

    (b)

    (b)

    (c)

    (c)

    (c)

    F .: A - - C .

    . I , -

    . C, -

    . T

    , ,

  • 7/30/2019 Renng Thesis

    172/203

    chapter . digital correction of lens aberrations

    MICRONS

    (a): C

    NJDSPOT

    (b): M

    NJDSPOT

    (c): E

    U C psf.

    MICRONS

    (d): C

    NJDSPOT

    (e): M

    NJDSPOT

    (f): E

    F .: D C psf.

    , . T

    : -

    ,

    .

    F . psf

    F .. T rms . W

    (), rms , . mp .

    F .

    . I psf

    , .

    F .

  • 7/30/2019 Renng Thesis

    173/203

    .. simulated correction performance

    TQPUSBEJVTQJ

    YFMT /

    /

    'SBDUJPOPGFMEPGWJFX

    /

    /

    /

    F .: H - psf .

    (NN) N= ( ) . T N rms

    . T . F -

    , N= , rms

    . T psf -

    ( ) . , rms psf.

    T rms

    . T

    , -

    . T . mp

    . mp N= . T ,

    . O -

    -, . S,

    . I ,

    ,

    .

    T -

    , . F .

    , mtf . W ,

  • 7/30/2019 Renng Thesis

    174/203

    chapter . digital correction of lens aberrations

    %JTUBODFGSPNPQUJDBMBYJTNN

    6ODPSSFDUFENN6ODPSSFDUFENN6ODPSSFDUFENN$PSSFDUFENN$PSSFDUFENN$PSSFDUFENN

    F .: mtf ( ).

    %JTUBODFGSPNPQUJDBMBYJTNN

    6ODPSSFDUFENN6ODPSSFDUFENN6ODPSSFDUFENN$PSSFDUFENN$PSSFDUFENN$PSSFDUFENN

    F .: mtf ( ).

    N . T mtf

    : /, / / N . T / N

    , . C

    . T

    . G mtf

    ,

    .

    F . , ,

    , .

    S, , , .

  • 7/30/2019 Renng Thesis

    175/203

    .. simulated correction performance

    D , -

    .

    F . ,

    : (.. ). T

    , . T

    - , M

    . T

    ( ), . T

    .

    T mtf F .

    . T extends the useul ocal range

    . H, ,

    . T -. F,

    . F . -

    F .. F

    ,

    .

    U,

    . T F .

    d-f a-c. T

    . A ,

    ,

    .

    T ,

    -

    . H

    .

  • 7/30/2019 Renng Thesis

    176/203

    chapter . digital correction of lens aberrations

    (a): C (b): M (c): E

    R- , .

    (d): C (e): M (f): E

    F .: R- .

    7.7.3 Correction Performance Across a Database of Lenses

    T -

    - . I -

    -

    .

    ,

    . T

    -- Z [Z ]

    .

    T Z . F,

  • 7/30/2019 Renng Thesis

    177/203

    .. simulated correction performance

    , -

    - . S,

    . T

    . F ,

    ,

    . F . ,

    .

    T , , G

    , - . T

    -. M

    , . N, ,

    . T

    ,

    .

    I , - . T

    - .

    F . . E ,

    . F , -

    . T

    NN, N= 1 ( ), - N= , . N

    .

    T

    . E ,

    -. I

    , -

    .

    A -

    . I ,

  • 7/30/2019 Renng Thesis

    178/203

    chapter . digital correction of lens aberrations

    L F D G F A R

    e- C /. e- /. e- usp ,, /.

    e- C usp ,, /.

    f- W usp ,, /. f- W- /.

    g- /. g- slr /. i- P /. l- D G /.

    l- D G

    /.

    l- P usp ,, /. l- P usp ,, /.

    l- L usp ,, /.

    l- M G usp ,, /. l- L usp ,, /.

    l- L. . usp ,, /. l- G usp ,, /. l- D usp ,, /.

    l- - G usp ,, /. l- M usp ,, /.

    l- D G /.

    I F ..

    F .: D .

    -

    . F , , vlsi ,

    . , ( ). ,

    . F -

    . T ,

    .

  • 7/30/2019 Renng Thesis

    179/203

    .. simulated correction performance

    ,FZ

    &FDUJWF3FTPMVUJPO

    F .: I .

  • 7/30/2019 Renng Thesis

    180/203

    chapter . digital correction of lens aberrations

    A (

    - ) . T . F

    . mp .

    A ,

    ( ) ,

    -, . I

    ,

    [A ]. O

    , . T

    -

    .

    In Closing

    , -

    , . O

    : . A

    . I , -

    (.. )

    .

    I ,

    . A -

    , F . ,

    . F

    -,

    u, . O

    ,

  • 7/30/2019 Renng Thesis

    181/203

    .. simulated correction performance

    . O ,

    , .

    T

    : ? ,

    - S .

    . W

    . M

    . T -

    F .d . R

    , , . W

    , . A

    - -

    F .. I ,

    . I , ,

    .

    T - -

    . C

    ,

    .

  • 7/30/2019 Renng Thesis

    182/203

  • 7/30/2019 Renng Thesis

    183/203

    8Conclusion

    T , H C-B

    ( ) , , -

    Decisive Moment .

    T , --,

    .

    A L, ,

    , , C-B

    .

    B . I , I -

    . T

    -. O

    . F D , - ,

    -, A S ,

    ,

    . B - ,

    .

  • 7/30/2019 Renng Thesis

    184/203

  • 7/30/2019 Renng Thesis

    185/203

    . H

    .T -

    . A d -

    [L . ], -

    . -

    . W -

    - [ ],

    ? I ,

    .

    T . W

    : ,

    . I I

    , I

    .

    B , . P

    ,

    . ,

    is

    . R

    ,

    . I