advanced computer graphics (fall 2010) cs 283, lecture 17: frequency analysis and signal processing...
Post on 20-Dec-2015
217 views
TRANSCRIPT
Advanced Computer Graphics Advanced Computer Graphics (Fall 2010)(Fall 2010)
CS 283, Lecture 17: Frequency Analysis and Signal Processing for Rendering
Ravi Ramamoorthi
http://inst.eecs.berkeley.edu/~cs283/fa10
MotivationMotivation
Signal-processing provides new understanding
Methods based on (Spherical) Fourier analysis
Allows understanding of sampling rates (in IBR)
Frequency-domain algorithms like convolution
This lecture high-level, mostly conceptual ideas. Follow original papers for details, applications
Plenoptic SamplingPlenoptic Sampling
Plenoptic Sampling. Chai, Tong, Chan, Shum 00
Signal-processing on light field
Minimal sampling rate for antialiased rendering
Relates to depth range, Fourier analysis
Fourer spectra derived for 2D light fields for simplicity. Same ideas extend to 4D
Siggraph’2000, July 27, 2000
A Geometrical Intuition
Zmin
Zopt
Camera i Camera i+1
Disparity Error <
1 Pixel
Rendering Camera
Siggraph’2000, July 27, 2000
Minimum Sampling Curve
Joint Image and
Geometry Space
Joint Image and
Geometry Space
Minimum Sampling Curve
Minimum Sampling Curve
Number of Depth Layers
1 2 3 6 12 Accurate Depth
Number of Images
2x2
8x8
4x4
16x16
32x32
Frequency Analysis and Sheared Reconstruction for Rendering Motion Blur
Frequency Analysis and Sheared Reconstruction for Rendering Motion Blur
Kevin Egan
Yu-Ting Tseng
Nicolas Holzschuch
Frédo Durand
Ravi Ramamoorthi
Columbia University
Columbia University
INRIA -- LJK
MIT CSAIL
University of California, Berkeley
Shear in Space-TimeShear in Space-Time
x
y t
x
f(x, t)
• Object moving with low velocity
f(x, y)
shear
Basic Example – Fourier DomainBasic Example – Fourier Domain
• Fourier spectrum, zero velocity
t
x
f(x, t) F(Ωx, Ωt)texture
bandwidth
Ωt
Ωx
Basic Example – Fourier DomainBasic Example – Fourier Domain
• Low velocity, small shear in both domains
f(x, t) F(Ωx, Ωt)
t
x
slope = -speed
Ωt
Ωx
Basic Example – Fourier DomainBasic Example – Fourier Domain
• Large shear
f(x, t) F(Ωx, Ωt)
t
x Ωt
Ωx
Basic Example – Fourier DomainBasic Example – Fourier Domain
• Non-linear motion, wedge shaped spectra
f(x, t)
Ωt
Ωx
F(Ωx, Ωt)
t
x
shutter bandlimits in
time
-min speed
-max speed
shutter applies blur across time
indirectly bandlimits in
space
Sampling in Fourier DomainSampling in Fourier Domain
Ωt
Ωxt
x+
• Sampling produces replicas in Fourier domain
• Sparse sampling produces dense replicas
Fourier DomainPrimal Domain
Standard Reconstruction FilteringStandard Reconstruction Filtering
• Standard filter, dense sampling (slow)
Ωt
no aliasing
Ωx
Fourier Domainreplicas
Standard Reconstruction FilterStandard Reconstruction Filter
• Standard filter, sparse sampling (fast)
Ωt
Fourier Domain
aliasing
Ωx
Sheared Reconstruction FilterSheared Reconstruction Filter
• Our sheared filter, sparse sampling (fast)
Ωt
Ωx
No aliasing!
Fourier Domain
Sheared Reconstruction FilterSheared Reconstruction Filter
• Compact shape in Fourier = wide in primal
t
x
Primal Domain
Ωt
Ωx
Fourier Domain
Reflection as ConvolutionReflection as Convolution
My PhD thesis (A signal-processing framework for forward and inverse rendering Stanford 2002)
Rewrite reflection equation on curved surfaces as a convolution with frequency-space product form
Theoretical underpinning for much work on relighting (next lecture), limits of inverse problems
Low-dimensional lighting models for Lambertian
AssumptionsAssumptions
Known geometry
Convex curved surfaces: no shadows, interreflection
Distant illumination
Homogeneous isotropic materials
Later precomputed methods: relax many assumptions
Reflection as Convolution (2D)Reflection as Convolution (2D)
2
2
id( )iL Lighting
( )oB Reflected Light Field
( , )i o BRDF
Li o B
i
B
L L
o
Reflection as Convolution (2D)Reflection as Convolution (2D)
2
2
id( )iL Lighting
( )oB Reflected Light Field
( , )i o BRDF
( )iL ( , )i o id2
2
( , )oB
Li o B
L
i
Bo
Reflection as Convolution (2D)Reflection as Convolution (2D)
( )iL ( , )i o id2
2
( , )oB
Li o B
L
i
Bo
ConvolutionConvolution
x
Signal f(x) Filter g(x)
Output h(u)
u
( ) ( ) ( )h u g x u f x dx
h f g g f Fourier analysis
h f g
Reflection as Convolution (2D)Reflection as Convolution (2D)
B L
Frequency: product
Spatial: integral
( )iL ( , )i o id2
2
( , )oB
, ,2l p l l pB L Fourier analysis
R. Ramamoorthi and P. Hanrahan “Analysis of Planar Light Fields from Homogeneous Convex Curved Surfaces under Distant Illumination” SPIE Photonics West 2001: Human Vision and Electronic Imaging VI pp 195-208
L
i
Bo
Li o B
Spherical HarmonicsSpherical Harmonics
-1-2 0 1 2
0
1
2
.
.
.
( , )lmY
xy z
xy yz 23 1z zx 2 2x y
l
m
1
Spherical Harmonic AnalysisSpherical Harmonic Analysis
, ,2l p l l pB L
2D:
22
0 0 ,( , , , ) ( [ , ]) ( , , , )o o i i i i o o i iB L R d d
3D:
( )iL ( , )i o id2
2
( , )oB
, ,lm pq l lm lq pqB L
Insights: Signal ProcessingInsights: Signal Processing
Signal processing framework for reflection Light is the signal BRDF is the filter Reflection on a curved surface is convolution
Insights: Signal ProcessingInsights: Signal Processing
Signal processing framework for reflection Light is the signal BRDF is the filter Reflection on a curved surface is convolution
Filter is Delta function : Output = Signal
Mirror BRDF : Image = Lighting [Miller and Hoffman 84]
Image courtesy Paul Debevec
Insights: Signal ProcessingInsights: Signal Processing
Signal processing framework for reflection Light is the signal BRDF is the filter Reflection on a curved surface is convolution
Signal is Delta function : Output = Filter
Point Light Source : Images = BRDF [Marschner et al. 00]
Am
plit
ude
Frequency
Roughness
Phong, Microfacet ModelsPhong, Microfacet Models
Mirror
Illumination estimationill-posed for rough surfaces
Analytic formulae in R. Ramamoorthi and P. Hanrahan“A Signal-Processing Framework for Inverse Rendering” SIGGRAPH 2001 pp 117-128
LambertianLambertian
Incident radiance (mirror sphere)
Irradiance (Lambertian)
N
l
2 / 3
/ 4
0
l0 1 2
R. Ramamoorthi and P. Hanrahan “On the Relationship between Radiance and Irradiance: Determining the Illumination from Images of a Convex Lambertian Object” Journal of the Optical Society of America A 18(10) Oct 2001 pp 2448-2459
R. Basri and D. Jacobs “Lambertian Reflectance and Linear Subspaces” ICCV 2001 pp 383-390
2 1
2
2
( 1) !2
( 2)( 1) 2 !
l
l l l
lA l even
l l
9 Parameter Approximation9 Parameter Approximation
-1-2 0 1 2
0
1
2
( , )lmY
xy z
xy yz 23 1z zx 2 2x y
l
m
Exact imageOrder 29 terms
RMS Error = 1%
For any illumination, average error < 3% [Basri Jacobs 01]
Ramamoorthi and Hanrahan 01b
Convolution for general materialsConvolution for general materials
Ramamoorthi and Hanrahan 01
( , ) ( ) ,B N V L R N l l V dl
B L
Frequency: product
Spatial: integral
ij i ijB L
Spherical harmonic analysis
Real-Time RenderingReal-Time Rendering
Motivation: Interactive rendering with natural illumination and realistic, measured materials
Ramamoorthi and Hanrahan 02
Normal Map Filtering and RenderingNormal Map Filtering and Rendering
Han, Sun, Ramamoorthi, Grinspun 07
More Frequency AnalysisMore Frequency Analysis
Many other papers Shadows: Soler and Sillion 98, Ramamoorthi et al. 04 Gradients: Ward, Heckbert, Arvo, Igehy, Holzschuch,
Chen, Ramamoorthi, … Wavelets: Gortler et al. 93, …, Ng et al. 03
Full frequency analysis of light transport Durand et al. 05 Space and Angle, generalizes previous work
Many recent papers in computational imaging [Levin et al. 08, 09]