11

Efficient Isotropic Efficient Isotropic BRDF MeasurementBRDF Measurement

Matthew BrandMatthew Brandbrand@merl.combrand@merl.com

TheThe UNIVERSITY UNIVERSITY ofof NORTH CAROLINA NORTH CAROLINA atat CHAPEL HILL CHAPEL HILL

Wojciech MatusikWojciech Matusikmatusik@lcs.mit.edumatusik@lcs.mit.edu

Hanspeter PfisterHanspeter Pfisterpfister@merl.compfister@merl.com

Leonard McMillanLeonard McMillanmcmillan@cs.unc.edumcmillan@cs.unc.edu

22

Our Goal: Data-Driven Our Goal: Data-Driven Reflectance ModelingReflectance Modeling

• Given a set of precise reflectance Given a set of precise reflectance measurements from real surfaces is it measurements from real surfaces is it possible to interpolate other plausible possible to interpolate other plausible surface models?surface models?

• Bidirectional Reflectance Distribution Bidirectional Reflectance Distribution FunctionFunction– Assumes all interaction occurs at a pointAssumes all interaction occurs at a point– Generally, a 4D functionGenerally, a 4D function

ir

i r

33

ir

d

Our Goal: Data-Driven Our Goal: Data-Driven Reflectance ModelingReflectance Modeling

• Given a set of precise reflectance Given a set of precise reflectance measurements from real surfaces is it measurements from real surfaces is it possible to interpolate other plausible possible to interpolate other plausible surface models?surface models?

• Bidirectional Reflectance Distribution Bidirectional Reflectance Distribution FunctionFunction– Assumes all interaction occurs at a pointAssumes all interaction occurs at a point– Generally, a 4D functionGenerally, a 4D function– Isotropic simplificationIsotropic simplification

(3D, independent of (3D, independent of ii))

44

Compact yet Plausible Era: Models which could be fit to measured data

Physically Motivated Era: Model parameters which “could” be measured

Classical Analytical BRDF Classical Analytical BRDF Reflectance modelsReflectance models

• Blinn-Torrance-Sparrow (1977)Blinn-Torrance-Sparrow (1977)

• Cook-Torrance (1982)Cook-Torrance (1982)

• He-Torrance-Sillion-Greenberg (1991)He-Torrance-Sillion-Greenberg (1991)

• Ward (1992)Ward (1992)

• Koenderink-van Doorn-Stavridi (1996)Koenderink-van Doorn-Stavridi (1996)

• Lafortune-Foo-Torrance-Greenberg (1997)Lafortune-Foo-Torrance-Greenberg (1997)

55

BRDF Data AcquisitionBRDF Data Acquisition

• BRDF capture device inspired by Steve BRDF capture device inspired by Steve Marschner’s designMarschner’s design

• A homogeneous sphere of material A homogeneous sphere of material illuminated by a point light sourceilluminated by a point light source

• Each image contains thousands of Each image contains thousands of BRDF measurementsBRDF measurements

• Megapixel+ digital camera Megapixel+ digital camera (QImaging Retiga 1300 – 1300 x 1030)(QImaging Retiga 1300 – 1300 x 1030)

• HDR Images HDR Images (18 10-bit exposures from 40 (18 10-bit exposures from 40 s to 20 s to 20 s)s)

• Stable wide spectrum light sourceStable wide spectrum light source((Hamamatsu SQ Xenon lamp)

• Precision turntable (Kaidan) and contact digitizer (Faro)

66

Data Binning and Data Binning and ParameterizationParameterization

• 20-80 million reflectance 20-80 million reflectance measurements per measurements per materialmaterial

• Each BRDF entailsEach BRDF entails90 x 90 x 360 x 3 90 x 90 x 360 x 3 “degree-sized” “degree-sized” measurement binsmeasurement bins

• Enforcing reciprocityEnforcing reciprocity

• Bilateral symmetryBilateral symmetry

• Outlier removal to reduce Outlier removal to reduce effects of noise and non-effects of noise and non-homogeneityhomogeneity

• ValidationValidation “Real” “Tabulated”

StandardParameterization

Rusinkiewicz’sParameterization

77

BRDF Raw NumbersBRDF Raw Numbers

• Each Image: 62500 - 250000 BRDF measurementsEach Image: 62500 - 250000 BRDF measurements(projected sphere radii between 200 – 400 pixels (projected sphere radii between 200 – 400 pixels w/approx half in shadow)w/approx half in shadow)

• 330 HDR images for light positions covering a 177º arc330 HDR images for light positions covering a 177º arc(20.625 – 82.5 M measurements)(20.625 – 82.5 M measurements)

• Measurements distributed into 3x90x90x180 = 4.374 M Measurements distributed into 3x90x90x180 = 4.374 M binsbins

• Average bin has ~10 measurementsAverage bin has ~10 measurements(~20 with Bilateral symmetry)(~20 with Bilateral symmetry)

• Estimates derived from ~5 to ~10 samples/binEstimates derived from ~5 to ~10 samples/bin(with lower and upper quartiles removed)(with lower and upper quartiles removed)

Resulting Model: 17 MB/BRDFResulting Model: 17 MB/BRDF

88

Rendering from Tabulated Rendering from Tabulated BRDFsBRDFs

• Even without further analysis, our BRDFs Even without further analysis, our BRDFs are immediately useful.are immediately useful.

• Renderings made with Wann Jensen’s Dali Renderings made with Wann Jensen’s Dali renderer with a custom isotropic BRDF renderer with a custom isotropic BRDF shader.shader.

Nickel Hematite Gold Paint Pink Felt

99

IT WAS PAINFUL!!!IT WAS PAINFUL!!!

• Each ball took over 12 Each ball took over 12 hourshours

• In 2 years we measured In 2 years we measured slightly more than 100 slightly more than 100 balls balls

• We repeated the whole We repeated the whole process two times, using process two times, using different light sources, and different light sources, and photometric calibrationsphotometric calibrations

• Can we apply what Can we apply what we’ve learned thus far we’ve learned thus far to speed up the process to speed up the process in the future?in the future?

1010

Our Pipedream, In a Our Pipedream, In a NutshellNutshell

1. Measure BRDFs

2. Reduce Model’s Size(Compression)

3. Optimize Sampling

3. Find Compact Basis

1111

Principle Component Principle Component Analysis of BRDFsAnalysis of BRDFs

• PCA applied across the entire PCA applied across the entire corpus of acquired BRDFscorpus of acquired BRDFs

• Compact RepresentationCompact Representation

• PCA Basis Vectors arePCA Basis Vectors arenon-intuitivenon-intuitive– Global supportGlobal support– Negative valuesNegative values– Input sensitiveInput sensitive

• More details in upcoming More details in upcoming SIGGRAPH paper “Data-Driven BRDF Modeling”SIGGRAPH paper “Data-Driven BRDF Modeling”

mean 5 10 20 30 45 60 all

1212

Wavelet Analysis of BRDFsWavelet Analysis of BRDFs

• Signal Localization – Typical BRDFs exhibit high Signal Localization – Typical BRDFs exhibit high frequencies in only very specific regions of their frequencies in only very specific regions of their parameter spaceparameter space

• Wavelets – Spatially compact basis functions of Wavelets – Spatially compact basis functions of varying scalevarying scale

• Used before (Schröder & Sweldens, 95) (Lalonde & Used before (Schröder & Sweldens, 95) (Lalonde & Fournier, 97)Fournier, 97)

1313

Wavelet Analysis of BRDFsWavelet Analysis of BRDFs

• For a given BRDF, GFor a given BRDF, Gii, 97% of the information is in only , 97% of the information is in only a few wavelet coefficients (~50000)a few wavelet coefficients (~50000)

• This gives a set of coefficients CThis gives a set of coefficients Cii

1414

Wavelet Analysis of BRDFsWavelet Analysis of BRDFs

• For a given BRDF, GFor a given BRDF, Gii, 97% of the information is in only , 97% of the information is in only a few wavelet coefficients (~50000)a few wavelet coefficients (~50000)

• This gives a set of coefficients CThis gives a set of coefficients Cii

• Repeat for all BRDFs, and compute their unionRepeat for all BRDFs, and compute their union

CWB = CWB = UU C Cii (~69000) Common Wavelet (~69000) Common Wavelet Basis (CWB)Basis (CWB)

i=1

N

1515

BRDF CompressionBRDF Compression

• 69,000 CWB approximately 4.7% the size of the raw 69,000 CWB approximately 4.7% the size of the raw datadata (to better (to better than 3%)than 3%)

jji HG

1616

CWB Representations of CWB Representations of BRDFsBRDFs

CWB reconstructions on left, dense samples on right

Dark-redpaint

Goldpaint

Orangeplastic

Aluminumbronze

0.7%

0.9%

2.1%

1.2%

1717

BRDF EstimationBRDF Estimation

• In order to optimize sampling we’d like In order to optimize sampling we’d like bases with local support (not necessarily bases with local support (not necessarily the case with PCA)the case with PCA)

• The problem of approximating a new The problem of approximating a new BRDFBRDF Hcg new

4M x 69Kwavelet basis

functions

69K x 1unknowns

4M x 1BRDF

We have 4M equations, which do We have 4M equations, which do we use?we use?

Large, but relatively sparse,thanks to wavelets

(~ 40 non-zero elements on average)

1818

BRDF Equation SelectionBRDF Equation Selection

• Find a set of 69K equations Find a set of 69K equations (making sure selected rows are linearly (making sure selected rows are linearly independent)independent)

• Does not robustly compute wavelet Does not robustly compute wavelet coefficients from low levels in the tree coefficients from low levels in the tree (remedy described in paper)(remedy described in paper)

cgH new

1 '

Large (69K x 69K), but relatively sparse, thanks to wavelets

(~ 40 non-zero elements/row on average)

1919

Sampling Implications of Sampling Implications of CWBCWB

• We know 69K specific places to measure each We know 69K specific places to measure each BRDFBRDF– Each “row” is a (Each “row” is a (hhdddd) specification) specification– Measure and then solve for Measure and then solve for cc– Still needs HDR and averaging to reduce noiseStill needs HDR and averaging to reduce noise– Down from 82MDown from 82M

• To reconstruct BRDF set coefficients not in CWB to To reconstruct BRDF set coefficients not in CWB to zero and invert waveletzero and invert wavelet

2020

BRDFs Reconstructed from BRDFs Reconstructed from CWB samplesCWB samples

BRDFs based on 69K samples on left, dense samples on right

Dark-redpaint

Goldpaint

Orangeplastic

Aluminumbronze

1.0%

1.3%

3.2%

1.2%

2121

Pull-Push Reconstruction of Pull-Push Reconstruction of BRDFsBRDFs

• Alternatively, use Pull-Push to reconstruct Alternatively, use Pull-Push to reconstruct (Gortler, et al 96)(Gortler, et al 96)

• Treat as scattered data interpolation Treat as scattered data interpolation problemproblem

2222

Push-Pull ReconstructionsPush-Pull Reconstructions

Dark-redpaint

Goldpaint

Orangeplastic

Aluminumbronze

0.6%

0.9%

2.5%

1.1%

2323

Linear Combinations of Linear Combinations of BRDFs (LCB)BRDFs (LCB)

• Can we find a linear combination of our Can we find a linear combination of our existing BRDFs that match any new one?existing BRDFs that match any new one?

• Our PCA analysis tells us we can do thisOur PCA analysis tells us we can do this

• It is even more compact (100 coeff/BRDF It is even more compact (100 coeff/BRDF vs. 65K)vs. 65K)

N65

4321

2424

Finding a Linear Finding a Linear Combination of BRDFsCombination of BRDFs

• Set up the following linear systemSet up the following linear system

• Each column of P is a BRDFEach column of P is a BRDF

• Each row of P is a (Each row of P is a (hhdddd) BRDF ) BRDF measurementmeasurement

• Once again more equations than unknownsOnce again more equations than unknowns

Pab new

4M x 100BRDFs

100 x 1unknowns

4M x 1BRDF

2525

LCB Solution MethodLCB Solution Method

• Randomly select a set of N equations, Randomly select a set of N equations, XX

• Find the ratio of the largest and smallest Find the ratio of the largest and smallest eigenvalues of eigenvalues of XXTTXX (N x N SVD) (N x N SVD)

• Randomly replace one row in Randomly replace one row in XX with an with an unused row from unused row from PP

• Find ratio of the largest and smallest Find ratio of the largest and smallest eigenvalues of new eigenvalues of new XXTTXX, if smaller accept , if smaller accept new row, else rejectnew row, else reject

• Repeat process until the set becomes stableRepeat process until the set becomes stable

• Try for various values of NTry for various values of N

2626

Linear Combinations of Linear Combinations of BRDFsBRDFs

BRDFs based on 800 samples

Dark-redpaint

Goldpaint

Orangeplastic

Aluminumbronze

1.8%

1.8%

4.3%

2.5%

2727

Advantages and Advantages and Disadvantages of MethodsDisadvantages of Methods

• CWB and Push-Pull require 69K CWB and Push-Pull require 69K measurements, but they don’t require our measurements, but they don’t require our BRDF database for reconstructionBRDF database for reconstruction

• Linear Combination of BRDFs requires Linear Combination of BRDFs requires fewer overall measurements (800), but it fewer overall measurements (800), but it relies on the availability of our BRDF relies on the availability of our BRDF database database

2828

Conclusions & Future WorkConclusions & Future Work

• Presented 2 novel methods for representing Presented 2 novel methods for representing and sampling BRDFsand sampling BRDFs

• We’ve built a system for the rapid and We’ve built a system for the rapid and accurate acquisition of BRDFs, in theoryaccurate acquisition of BRDFs, in theory

• Better wavelets, Anisotropic BRDFsBetter wavelets, Anisotropic BRDFs

• Not all linear combinations of BRDFs are validNot all linear combinations of BRDFs are valid

• Not all convex combinations of BRDFs are Not all convex combinations of BRDFs are validvalid

• What combinations are valid?What combinations are valid?

• Intuitive linear parameters for BRDF designIntuitive linear parameters for BRDF design

2929

AcknowledgementsAcknowledgements

• Henrik Wann Jensen for assistance with Henrik Wann Jensen for assistance with DaliDali

• Paul Lalonde for his wavelet shaderPaul Lalonde for his wavelet shader

• Paul Debevec for his environment mapsPaul Debevec for his environment maps

• Markus Gross, Steven Gortler for their Markus Gross, Steven Gortler for their helpful insights helpful insights

Thank you!Thank you!