11
Efficient Isotropic Efficient Isotropic BRDF MeasurementBRDF Measurement
Matthew BrandMatthew [email protected]@merl.com
TheThe UNIVERSITY UNIVERSITY ofof NORTH CAROLINA NORTH CAROLINA atat CHAPEL HILL CHAPEL HILL
Wojciech MatusikWojciech [email protected]@lcs.mit.edu
Hanspeter PfisterHanspeter [email protected]@merl.com
Leonard McMillanLeonard [email protected]@cs.unc.edu
22
Our Goal: Data-Driven Our Goal: Data-Driven Reflectance ModelingReflectance Modeling
• Given a set of precise reflectance Given a set of precise reflectance measurements from real surfaces is it measurements from real surfaces is it possible to interpolate other plausible possible to interpolate other plausible surface models?surface models?
• Bidirectional Reflectance Distribution Bidirectional Reflectance Distribution FunctionFunction– Assumes all interaction occurs at a pointAssumes all interaction occurs at a point– Generally, a 4D functionGenerally, a 4D function
ir
i r
33
ir
d
Our Goal: Data-Driven Our Goal: Data-Driven Reflectance ModelingReflectance Modeling
• Given a set of precise reflectance Given a set of precise reflectance measurements from real surfaces is it measurements from real surfaces is it possible to interpolate other plausible possible to interpolate other plausible surface models?surface models?
• Bidirectional Reflectance Distribution Bidirectional Reflectance Distribution FunctionFunction– Assumes all interaction occurs at a pointAssumes all interaction occurs at a point– Generally, a 4D functionGenerally, a 4D function– Isotropic simplificationIsotropic simplification
(3D, independent of (3D, independent of ii))
44
Compact yet Plausible Era: Models which could be fit to measured data
Physically Motivated Era: Model parameters which “could” be measured
Classical Analytical BRDF Classical Analytical BRDF Reflectance modelsReflectance models
• Blinn-Torrance-Sparrow (1977)Blinn-Torrance-Sparrow (1977)
• Cook-Torrance (1982)Cook-Torrance (1982)
• He-Torrance-Sillion-Greenberg (1991)He-Torrance-Sillion-Greenberg (1991)
• Ward (1992)Ward (1992)
• Koenderink-van Doorn-Stavridi (1996)Koenderink-van Doorn-Stavridi (1996)
• Lafortune-Foo-Torrance-Greenberg (1997)Lafortune-Foo-Torrance-Greenberg (1997)
55
BRDF Data AcquisitionBRDF Data Acquisition
• BRDF capture device inspired by Steve BRDF capture device inspired by Steve Marschner’s designMarschner’s design
• A homogeneous sphere of material A homogeneous sphere of material illuminated by a point light sourceilluminated by a point light source
• Each image contains thousands of Each image contains thousands of BRDF measurementsBRDF measurements
• Megapixel+ digital camera Megapixel+ digital camera (QImaging Retiga 1300 – 1300 x 1030)(QImaging Retiga 1300 – 1300 x 1030)
• HDR Images HDR Images (18 10-bit exposures from 40 (18 10-bit exposures from 40 s to 20 s to 20 s)s)
• Stable wide spectrum light sourceStable wide spectrum light source((Hamamatsu SQ Xenon lamp)
• Precision turntable (Kaidan) and contact digitizer (Faro)
66
Data Binning and Data Binning and ParameterizationParameterization
• 20-80 million reflectance 20-80 million reflectance measurements per measurements per materialmaterial
• Each BRDF entailsEach BRDF entails90 x 90 x 360 x 3 90 x 90 x 360 x 3 “degree-sized” “degree-sized” measurement binsmeasurement bins
• Enforcing reciprocityEnforcing reciprocity
• Bilateral symmetryBilateral symmetry
• Outlier removal to reduce Outlier removal to reduce effects of noise and non-effects of noise and non-homogeneityhomogeneity
• ValidationValidation “Real” “Tabulated”
StandardParameterization
Rusinkiewicz’sParameterization
77
BRDF Raw NumbersBRDF Raw Numbers
• Each Image: 62500 - 250000 BRDF measurementsEach Image: 62500 - 250000 BRDF measurements(projected sphere radii between 200 – 400 pixels (projected sphere radii between 200 – 400 pixels w/approx half in shadow)w/approx half in shadow)
• 330 HDR images for light positions covering a 177º arc330 HDR images for light positions covering a 177º arc(20.625 – 82.5 M measurements)(20.625 – 82.5 M measurements)
• Measurements distributed into 3x90x90x180 = 4.374 M Measurements distributed into 3x90x90x180 = 4.374 M binsbins
• Average bin has ~10 measurementsAverage bin has ~10 measurements(~20 with Bilateral symmetry)(~20 with Bilateral symmetry)
• Estimates derived from ~5 to ~10 samples/binEstimates derived from ~5 to ~10 samples/bin(with lower and upper quartiles removed)(with lower and upper quartiles removed)
Resulting Model: 17 MB/BRDFResulting Model: 17 MB/BRDF
88
Rendering from Tabulated Rendering from Tabulated BRDFsBRDFs
• Even without further analysis, our BRDFs Even without further analysis, our BRDFs are immediately useful.are immediately useful.
• Renderings made with Wann Jensen’s Dali Renderings made with Wann Jensen’s Dali renderer with a custom isotropic BRDF renderer with a custom isotropic BRDF shader.shader.
Nickel Hematite Gold Paint Pink Felt
99
IT WAS PAINFUL!!!IT WAS PAINFUL!!!
• Each ball took over 12 Each ball took over 12 hourshours
• In 2 years we measured In 2 years we measured slightly more than 100 slightly more than 100 balls balls
• We repeated the whole We repeated the whole process two times, using process two times, using different light sources, and different light sources, and photometric calibrationsphotometric calibrations
• Can we apply what Can we apply what we’ve learned thus far we’ve learned thus far to speed up the process to speed up the process in the future?in the future?
1010
Our Pipedream, In a Our Pipedream, In a NutshellNutshell
1. Measure BRDFs
2. Reduce Model’s Size(Compression)
3. Optimize Sampling
3. Find Compact Basis
1111
Principle Component Principle Component Analysis of BRDFsAnalysis of BRDFs
• PCA applied across the entire PCA applied across the entire corpus of acquired BRDFscorpus of acquired BRDFs
• Compact RepresentationCompact Representation
• PCA Basis Vectors arePCA Basis Vectors arenon-intuitivenon-intuitive– Global supportGlobal support– Negative valuesNegative values– Input sensitiveInput sensitive
• More details in upcoming More details in upcoming SIGGRAPH paper “Data-Driven BRDF Modeling”SIGGRAPH paper “Data-Driven BRDF Modeling”
mean 5 10 20 30 45 60 all
1212
Wavelet Analysis of BRDFsWavelet Analysis of BRDFs
• Signal Localization – Typical BRDFs exhibit high Signal Localization – Typical BRDFs exhibit high frequencies in only very specific regions of their frequencies in only very specific regions of their parameter spaceparameter space
• Wavelets – Spatially compact basis functions of Wavelets – Spatially compact basis functions of varying scalevarying scale
• Used before (Schröder & Sweldens, 95) (Lalonde & Used before (Schröder & Sweldens, 95) (Lalonde & Fournier, 97)Fournier, 97)
1313
Wavelet Analysis of BRDFsWavelet Analysis of BRDFs
• For a given BRDF, GFor a given BRDF, Gii, 97% of the information is in only , 97% of the information is in only a few wavelet coefficients (~50000)a few wavelet coefficients (~50000)
• This gives a set of coefficients CThis gives a set of coefficients Cii
1414
Wavelet Analysis of BRDFsWavelet Analysis of BRDFs
• For a given BRDF, GFor a given BRDF, Gii, 97% of the information is in only , 97% of the information is in only a few wavelet coefficients (~50000)a few wavelet coefficients (~50000)
• This gives a set of coefficients CThis gives a set of coefficients Cii
• Repeat for all BRDFs, and compute their unionRepeat for all BRDFs, and compute their union
CWB = CWB = UU C Cii (~69000) Common Wavelet (~69000) Common Wavelet Basis (CWB)Basis (CWB)
i=1
N
1515
BRDF CompressionBRDF Compression
• 69,000 CWB approximately 4.7% the size of the raw 69,000 CWB approximately 4.7% the size of the raw datadata (to better (to better than 3%)than 3%)
jji HG
1616
CWB Representations of CWB Representations of BRDFsBRDFs
CWB reconstructions on left, dense samples on right
Dark-redpaint
Goldpaint
Orangeplastic
Aluminumbronze
0.7%
0.9%
2.1%
1.2%
1717
BRDF EstimationBRDF Estimation
• In order to optimize sampling we’d like In order to optimize sampling we’d like bases with local support (not necessarily bases with local support (not necessarily the case with PCA)the case with PCA)
• The problem of approximating a new The problem of approximating a new BRDFBRDF Hcg new
4M x 69Kwavelet basis
functions
69K x 1unknowns
4M x 1BRDF
We have 4M equations, which do We have 4M equations, which do we use?we use?
Large, but relatively sparse,thanks to wavelets
(~ 40 non-zero elements on average)
1818
BRDF Equation SelectionBRDF Equation Selection
• Find a set of 69K equations Find a set of 69K equations (making sure selected rows are linearly (making sure selected rows are linearly independent)independent)
• Does not robustly compute wavelet Does not robustly compute wavelet coefficients from low levels in the tree coefficients from low levels in the tree (remedy described in paper)(remedy described in paper)
cgH new
1 '
Large (69K x 69K), but relatively sparse, thanks to wavelets
(~ 40 non-zero elements/row on average)
1919
Sampling Implications of Sampling Implications of CWBCWB
• We know 69K specific places to measure each We know 69K specific places to measure each BRDFBRDF– Each “row” is a (Each “row” is a (hhdddd) specification) specification– Measure and then solve for Measure and then solve for cc– Still needs HDR and averaging to reduce noiseStill needs HDR and averaging to reduce noise– Down from 82MDown from 82M
• To reconstruct BRDF set coefficients not in CWB to To reconstruct BRDF set coefficients not in CWB to zero and invert waveletzero and invert wavelet
2020
BRDFs Reconstructed from BRDFs Reconstructed from CWB samplesCWB samples
BRDFs based on 69K samples on left, dense samples on right
Dark-redpaint
Goldpaint
Orangeplastic
Aluminumbronze
1.0%
1.3%
3.2%
1.2%
2121
Pull-Push Reconstruction of Pull-Push Reconstruction of BRDFsBRDFs
• Alternatively, use Pull-Push to reconstruct Alternatively, use Pull-Push to reconstruct (Gortler, et al 96)(Gortler, et al 96)
• Treat as scattered data interpolation Treat as scattered data interpolation problemproblem
2222
Push-Pull ReconstructionsPush-Pull Reconstructions
Dark-redpaint
Goldpaint
Orangeplastic
Aluminumbronze
0.6%
0.9%
2.5%
1.1%
2323
Linear Combinations of Linear Combinations of BRDFs (LCB)BRDFs (LCB)
• Can we find a linear combination of our Can we find a linear combination of our existing BRDFs that match any new one?existing BRDFs that match any new one?
• Our PCA analysis tells us we can do thisOur PCA analysis tells us we can do this
• It is even more compact (100 coeff/BRDF It is even more compact (100 coeff/BRDF vs. 65K)vs. 65K)
N65
4321
2424
Finding a Linear Finding a Linear Combination of BRDFsCombination of BRDFs
• Set up the following linear systemSet up the following linear system
• Each column of P is a BRDFEach column of P is a BRDF
• Each row of P is a (Each row of P is a (hhdddd) BRDF ) BRDF measurementmeasurement
• Once again more equations than unknownsOnce again more equations than unknowns
Pab new
4M x 100BRDFs
100 x 1unknowns
4M x 1BRDF
2525
LCB Solution MethodLCB Solution Method
• Randomly select a set of N equations, Randomly select a set of N equations, XX
• Find the ratio of the largest and smallest Find the ratio of the largest and smallest eigenvalues of eigenvalues of XXTTXX (N x N SVD) (N x N SVD)
• Randomly replace one row in Randomly replace one row in XX with an with an unused row from unused row from PP
• Find ratio of the largest and smallest Find ratio of the largest and smallest eigenvalues of new eigenvalues of new XXTTXX, if smaller accept , if smaller accept new row, else rejectnew row, else reject
• Repeat process until the set becomes stableRepeat process until the set becomes stable
• Try for various values of NTry for various values of N
2626
Linear Combinations of Linear Combinations of BRDFsBRDFs
BRDFs based on 800 samples
Dark-redpaint
Goldpaint
Orangeplastic
Aluminumbronze
1.8%
1.8%
4.3%
2.5%
2727
Advantages and Advantages and Disadvantages of MethodsDisadvantages of Methods
• CWB and Push-Pull require 69K CWB and Push-Pull require 69K measurements, but they don’t require our measurements, but they don’t require our BRDF database for reconstructionBRDF database for reconstruction
• Linear Combination of BRDFs requires Linear Combination of BRDFs requires fewer overall measurements (800), but it fewer overall measurements (800), but it relies on the availability of our BRDF relies on the availability of our BRDF database database
2828
Conclusions & Future WorkConclusions & Future Work
• Presented 2 novel methods for representing Presented 2 novel methods for representing and sampling BRDFsand sampling BRDFs
• We’ve built a system for the rapid and We’ve built a system for the rapid and accurate acquisition of BRDFs, in theoryaccurate acquisition of BRDFs, in theory
• Better wavelets, Anisotropic BRDFsBetter wavelets, Anisotropic BRDFs
• Not all linear combinations of BRDFs are validNot all linear combinations of BRDFs are valid
• Not all convex combinations of BRDFs are Not all convex combinations of BRDFs are validvalid
• What combinations are valid?What combinations are valid?
• Intuitive linear parameters for BRDF designIntuitive linear parameters for BRDF design
2929
AcknowledgementsAcknowledgements
• Henrik Wann Jensen for assistance with Henrik Wann Jensen for assistance with DaliDali
• Paul Lalonde for his wavelet shaderPaul Lalonde for his wavelet shader
• Paul Debevec for his environment mapsPaul Debevec for his environment maps
• Markus Gross, Steven Gortler for their Markus Gross, Steven Gortler for their helpful insights helpful insights
Thank you!Thank you!