axial light field for curved mirrors: reflect your perspective, widen your view
DESCRIPTION
Axial Light Field for Curved Mirrors: Reflect Your Perspective, Widen Your View. Yuichi Taguchi Amit Agrawal Srikumar Ramalingam Ashok Veeraraghavan Mitsubishi Electric Research Labs (MERL). Non-Single Viewpoint Image (Spherical Mirror + Perspective Camera). - PowerPoint PPT PresentationTRANSCRIPT
Mitsubishi Electric Research Labs (MERL) Axial Light Field Taguchi, Agrawal, Ramalingam & Veeraraghavan
Axial Light Field for Curved Mirrors:Axial Light Field for Curved Mirrors:Reflect Your Perspective, Widen Your ViewReflect Your Perspective, Widen Your View
Yuichi TaguchiAmit Agrawal
Srikumar RamalingamAshok Veeraraghavan
Mitsubishi Electric Research Labs (MERL)
Mitsubishi Electric Research Labs (MERL) Axial Light Field Taguchi, Agrawal, Ramalingam & Veeraraghavan
Non-Single Viewpoint Image(Spherical Mirror + Perspective Camera)
Mitsubishi Electric Research Labs (MERL) Axial Light Field Taguchi, Agrawal, Ramalingam & Veeraraghavan
Single-Viewpoint Image (Cube Map, FOV 140)
Mitsubishi Electric Research Labs (MERL) Axial Light Field Taguchi, Agrawal, Ramalingam & Veeraraghavan
No Approximation, No Knowledge of Scene Geometry
Mitsubishi Electric Research Labs (MERL) Axial Light Field Taguchi, Agrawal, Ramalingam & Veeraraghavan
Single/Non-Single ViewpointSingle/Non-Single Viewpoint
Perspective Camera Catadioptric System (Mirror + Camera)
Single Viewpoint Single Viewpoint Non-Single Viewpoint
Virtual Viewpoint
Hyperbolic Mirror Spherical
MirrorLocus of
Viewpoint (Caustic)
Perspective Camera
Perspective Camera
Mitsubishi Electric Research Labs (MERL) Axial Light Field Taguchi, Agrawal, Ramalingam & Veeraraghavan
Single-Viewpoint Catadioptric SystemsSingle-Viewpoint Catadioptric Systems
• Only a few single-viewpoint configurations
• Other common configurations lead to non-single viewpoint– Spherical mirror– Parabolic mirror with perspective camera– Hyperbolic/elliptic mirror with perspective camera not on foci
[Baker & Nayar 99]
Hyperbola/Ellipse Parabola
Foci
Orthographic CameraPerspective Camera
Virtual Viewpoint
Mitsubishi Electric Research Labs (MERL) Axial Light Field Taguchi, Agrawal, Ramalingam & Veeraraghavan
Generating Single-Viewpoint Image fromGenerating Single-Viewpoint Image fromNon-Single Viewpoint ImageNon-Single Viewpoint Image
• Distortion Correction Approaches– Use one image– Single-viewpoint approximation – Use scene prior [Swaminathan et al. 03]
– Generating exact perspective views is impossible without knowing scene geometry
• Our Approach– Use multiple images– Capture all the rays required to generate an exact single-
viewpoint image– Does not require scene prior and 3D reconstruction
Scene Prior (e.g. Plane)
Mitsubishi Electric Research Labs (MERL) Axial Light Field Taguchi, Agrawal, Ramalingam & Veeraraghavan
• If we capture all the rays that pass through the virtual viewpoint, we can generate a single-viewpoint image– What is the best possible sampling?
Light FieldLight Field
Mirror Surface
Virtual Viewpoint
Light Field Plane
[Levoy & Hanrahan 96, Gortler et al. 96]
Mitsubishi Electric Research Labs (MERL) Axial Light Field Taguchi, Agrawal, Ramalingam & Veeraraghavan
Input Images (Axial Light Field)
Mitsubishi Electric Research Labs (MERL) Axial Light Field Taguchi, Agrawal, Ramalingam & Veeraraghavan
Input Output
Copy Circles
Mitsubishi Electric Research Labs (MERL) Axial Light Field Taguchi, Agrawal, Ramalingam & Veeraraghavan
Single-Viewpoint Image (Cube Map, FOV 140)
Mitsubishi Electric Research Labs (MERL) Axial Light Field Taguchi, Agrawal, Ramalingam & Veeraraghavan
Key IdeaKey Idea
• Rotationally symmetric mirrors• Capture axial light field
– Move the camera along the mirror axis– Exact single-viewpoint image generation
without scene prior and 3D reconstruction– Better sampling than typical planar light field
Planar Light Field
AxialLight Field
Symmetry Axis
Mitsubishi Electric Research Labs (MERL) Axial Light Field Taguchi, Agrawal, Ramalingam & Veeraraghavan
Geometric InterpretationGeometric Interpretation
A cone of rays in virtual camera
(Angle )
A cone of raysin real camera
(Distance d, Angle )
d
Axial camera [Ramalingam et al. 06](All the rays pass through the axis)
Virtual Viewpoint
Rotationally Symmetric
Mirror
Mitsubishi Electric Research Labs (MERL) Axial Light Field Taguchi, Agrawal, Ramalingam & Veeraraghavan
Light Fields for Symmetric Mirrors in 3DLight Fields for Symmetric Mirrors in 3D
x
y
u
v
x-u-v Slice
x
u
v
Mitsubishi Electric Research Labs (MERL) Axial Light Field Taguchi, Agrawal, Ramalingam & Veeraraghavan
Light Fields for Symmetric Mirrors in 3DLight Fields for Symmetric Mirrors in 3D
x
Virtual Viewpoint
Spherical Mirror
y
Planar Light Field Sampling
x-u-v Slice
x
u
v
y=0
Mitsubishi Electric Research Labs (MERL) Axial Light Field Taguchi, Agrawal, Ramalingam & Veeraraghavan
Light Fields for Symmetric Mirrors in 3DLight Fields for Symmetric Mirrors in 3D
x
Virtual Viewpoint
Spherical Mirror
y
Planar Light Field Sampling
x
u
v
x-u-v Slice y0
Mitsubishi Electric Research Labs (MERL) Axial Light Field Taguchi, Agrawal, Ramalingam & Veeraraghavan
Light Fields for Symmetric Mirrors in 3DLight Fields for Symmetric Mirrors in 3D
x
Virtual Viewpoint
Spherical Mirror
y
x
u
v
Axial Light Field Sampling
x-u-v Slice (x,y)=(uz,vz)
Mitsubishi Electric Research Labs (MERL) Axial Light Field Taguchi, Agrawal, Ramalingam & Veeraraghavan
Light Fields for Symmetric Mirrors in 3DLight Fields for Symmetric Mirrors in 3D
x
Virtual Viewpoint
Spherical Mirror
y
u
v
Axial Light Field Sampling
Image Plane
u-v Slice (x,y)=(uz,vz)
Mitsubishi Electric Research Labs (MERL) Axial Light Field Taguchi, Agrawal, Ramalingam & Veeraraghavan
Planar LF vs. Axial LFPlanar LF vs. Axial LF
• Planar Light Field– Sample a 4D subset of rays– The camera captures a cone of rays
only when placed on the mirror axis• Other cameras capture only a few rays
• Axial Light Field– Sample a 3D subset of rays– Every camera captures a cone of rays
• Light rays required to generate a single-viewpoint image are concentrated in this 3D subset
Mitsubishi Electric Research Labs (MERL) Axial Light Field Taguchi, Agrawal, Ramalingam & Veeraraghavan
A cone of rays in virtual camera
(Angle )
A cone of raysin real camera
(Distance d, Angle )
d
Virtual Camera
Real Camera
Axial LF Sampling ParametersAxial LF Sampling Parameters
Mitsubishi Electric Research Labs (MERL) Axial Light Field Taguchi, Agrawal, Ramalingam & Veeraraghavan
Axial LF Sampling ParametersAxial LF Sampling Parameters
Ray angle in virtual camera () []
Location of real camera (d)
Ray angle in virtual camera () []
Ray angle in real camera () []
Real Camera FOV
Cone
Concave Sphere
Parabola
SphereMirror Shapes
Virtual camera
Resolution Reduction
Mitsubishi Electric Research Labs (MERL) Axial Light Field Taguchi, Agrawal, Ramalingam & Veeraraghavan
Single-Viewpoint Image GenerationSingle-Viewpoint Image Generation
Input Images
Virtual Perspective Image
Copy Resized Circles
A cone of rays in virtual camera
(Angle )
A cone of raysin real camera
(Angle , Distance d)
Mitsubishi Electric Research Labs (MERL) Axial Light Field Taguchi, Agrawal, Ramalingam & Veeraraghavan
Simulation Results for Different Mirror ShapesSimulation Results for Different Mirror Shapes
Input Far
Input Near
Output
ConeConcave SphereParabolaSphere
Limited Input FOV
InvalidZ positionFOV 24
FOV 140
Mitsubishi Electric Research Labs (MERL) Axial Light Field Taguchi, Agrawal, Ramalingam & Veeraraghavan
Changing Resolution PropertyChanging Resolution Property
• Change focal length (zoom) for each capture position– Change resolution property without changing mirror shapes
0 50 100 150 200 250 3000
50
100
150
200
250
300
Circle radius in virtual image [pixels]
Circle radius in real image [pixels]
0 50 100 150
1
2
3
4
5
Mag. factor of focal length
Location of real camera
Spherical Mirror
Constant focal length
Variable focal length
To achieve resolution propertysimilar to a perspective camera
Mitsubishi Electric Research Labs (MERL) Axial Light Field Taguchi, Agrawal, Ramalingam & Veeraraghavan
ComparisonComparison
Planar Light Field Axial Light Field
Same Number (25) of Input Images (FOV 32 x 24)
Setup
Mirror Ball
Robot Arm
Mitsubishi Electric Research Labs (MERL) Axial Light Field Taguchi, Agrawal, Ramalingam & Veeraraghavan
Output Cube Maps (FOV 140)
Aliasing /Ghosting
Planar Light Field Axial Light Field
ComparisonComparison
Mitsubishi Electric Research Labs (MERL) Axial Light Field Taguchi, Agrawal, Ramalingam & Veeraraghavan
AdvantagesAdvantages
• Axial light field based catadioptric imaging
– Exact single-viewpoint image generation in wide angle• Without any scene prior, 3D reconstruction
– For any rotationally symmetric mirror
– Offer changing resolution property• Without changing mirror shape
– Virtual viewpoints can be varied on the mirror axis
Mitsubishi Electric Research Labs (MERL) Axial Light Field Taguchi, Agrawal, Ramalingam & Veeraraghavan
LimitationsLimitations
• Static scene
• Precise camera motion along the mirror axis
• Implementation using multiple cameras would be difficult
Mitsubishi Electric Research Labs (MERL) Axial Light Field Taguchi, Agrawal, Ramalingam & Veeraraghavan
SummarySummary
• Analysis of light rays for rotationally symmetric mirrors• Axial light field sampling• Exact single-viewpoint image generation