CS 395/495-25: Spring 2004CS 395/495-25: Spring 2004

IBMR: IBMR:

Measuring Lights, Materials, Measuring Lights, Materials, Lenses and moreLenses and more

Jack TumblinJack Tumblinjet@cs.northwestern.edujet@cs.northwestern.edu

Recall: An Image Is…Recall: An Image Is…

2D Image:2D Image:A map of light A map of light

intensities intensities

Light + 3D Scene:Light + 3D Scene:Illumination, Illumination,

shape, movement, shape, movement, surface BRDF,… surface BRDF,…

Pos

ition

(x,y

)P

ositi

on(x

,y)A ‘Camera’:

?What are ALL the possibilities?

An Planar Projection Image Is…An Planar Projection Image Is…

Image PlaneImage PlaneI(x,y)I(x,y)

Ang

le(

Ang

le(

,, ))

Pos

ition

(x,y

)P

ositi

on(x

,y)

2D Image:2D Image:Collection of rays Collection of rays

through a point through a point

Light + 3D Scene:Light + 3D Scene:Illumination, Illumination,

shape, movement, shape, movement, surface BRDF,… surface BRDF,…

‘‘Center of Center of Projection’Projection’

(P(P33 or P or P22 Origin) Origin)

Image Making: Pinhole Image Making: Pinhole Thin Lens Thin Lens

• Interactive Thin Lens Demo Interactive Thin Lens Demo (search ‘physlet optical bench’)(search ‘physlet optical bench’)

http://www.swgc.mun.ca/physics/physlets/opticalbench.htmlhttp://www.swgc.mun.ca/physics/physlets/opticalbench.html

• From this geometry (for next time)From this geometry (for next time)Can you derive Thin Lens Law?Can you derive Thin Lens Law?

Incident Light MeasurementIncident Light Measurement• Flux Flux WW = power, Watts, # photons/sec = power, Watts, # photons/sec

• Uniform, point-source light: Uniform, point-source light:

flux on a patch of surface falls with distanceflux on a patch of surface falls with distance22 E = Watts/rE = Watts/r22

rr

Light MeasurementLight Measurement• Flux Flux WW = power, Watts, # photons/sec = power, Watts, # photons/sec

• Irradiance Irradiance EE: flux arriving : flux arriving per unit areaper unit area,,

(regardless of direction)(regardless of direction)E = Watts/area = dW/dAE = Watts/area = dW/dA

But direction makes a But direction makes a big difference when big difference when

computing E...computing E...

Foreshortening Effect: cos(Foreshortening Effect: cos())• Larger Incident angle Larger Incident angle ii

spreads same flux over larger areaspreads same flux over larger area• flux per unit area becomes flux per unit area becomes W cos( W cos( ii) / ) / areaarea

• Foreshortening geometry imposes Foreshortening geometry imposes an angular term cos(an angular term cos(ii) on energy transfer) on energy transfer

circular ‘bundle’ circular ‘bundle’ of incident rays, of incident rays,

flux Wflux W

WW ii

Irradiance EIrradiance E• To find irradiance at a point on a surface,To find irradiance at a point on a surface,• Find flux from each (point?) light source,Find flux from each (point?) light source,• Weight flux by its direction:Weight flux by its direction: cos( cos(ii) ) • Add all light sources: or more precisely, Add all light sources: or more precisely,

integrate over entire hemisphere integrate over entire hemisphere Defines Radiance L:Defines Radiance L:

L = (watts / area) / srL = (watts / area) / sr(sr = steradians; solid angle; (sr = steradians; solid angle;

= surface area on unit sphere)= surface area on unit sphere)

Radiance LRadiance L• But for distributed (non-point) light sources? But for distributed (non-point) light sources?

integrate flux over the entire hemisphere integrate flux over the entire hemisphere ..But what are the units of what we integrate?But what are the units of what we integrate?

Radiance LRadiance L

L = (watts / area) / srL = (watts / area) / sr(sr = steradians; solid angle; (sr = steradians; solid angle;

= surface area on unit sphere)= surface area on unit sphere)

Lighting InvariantsLighting InvariantsWhy doesn’t surface intensity change with distance?Why doesn’t surface intensity change with distance?

• We know point source flux drops with distance: 1/rWe know point source flux drops with distance: 1/r22 • We know surface is made of infinitesimal point sources...We know surface is made of infinitesimal point sources...

CamCam‘‘intensity’: 1/rintensity’: 1/r22

‘‘intensity’: constant (?!?!) intensity’: constant (?!?!)

Lighting InvariantsLighting InvariantsWhy doesn’t surface intensity change with distance?Why doesn’t surface intensity change with distance?

Because camera pixels measure Because camera pixels measure RadianceRadiance, not flux!, not flux!– pixel value pixel value flux *cos( flux *cos() / sr) / sr– ‘‘good lens’ design: cos(good lens’ design: cos() term vanishes. Vignetting=residual error.) term vanishes. Vignetting=residual error.

• Pixel’s size in sr fixed:Pixel’s size in sr fixed:– Point source fits in one pixel: 1/rPoint source fits in one pixel: 1/r22

– Viewed surface area grows by rViewed surface area grows by r22, , cancels 1/rcancels 1/r22 flux falloff flux falloff

CamCamLight ‘intensity’: 1/rLight ‘intensity’: 1/r22

Surface ‘intensity’: constant (?!?!) Surface ‘intensity’: constant (?!?!)

Lighting InvariantsLighting InvariantsRadiance Images are LINEAR: Radiance Images are LINEAR:

··(Radiance caused by (Light 1)) +(Radiance caused by (Light 1)) +

··(Radiance caused by (Light 2))(Radiance caused by (Light 2))

= Radiance caused by (= Radiance caused by (·· Light 1 + Light 1 + ··Light 2)Light 2)

http://www.sgi.com/grafica/synth/index.htmlhttp://www.sgi.com/grafica/synth/index.html

++ ==

Lighting InvariantsLighting InvariantsLight is Linear: Light is Linear:

··(Radiance caused by (Light 1)) +(Radiance caused by (Light 1)) +

··(Radiance caused by (Light 2))(Radiance caused by (Light 2))

= Radiance caused by (= Radiance caused by (·· Light 1 + Light 1 + ··Light 2)Light 2)

http://www.sgi.com/grafica/synth/index.htmlhttp://www.sgi.com/grafica/synth/index.html

-- ==

Allows ‘negative’ light!Allows ‘negative’ light!

Point-wise Light ReflectionPoint-wise Light Reflection• Given:Given:

– Infinitesimal surface patch Infinitesimal surface patch dAdA, , – illuminated by irradiance amount illuminated by irradiance amount EE – from just one direction from just one direction ((ii,,ii))

• How should we measure the returned light?How should we measure the returned light?• Ans: Ans: by emittedby emitted

RADIANCERADIANCEmeasured for allmeasured for alloutgoing directions:outgoing directions:(measured on surface of (measured on surface of ))

dAdAii

ii

Point-wise Light Reflection: BRDFPoint-wise Light Reflection: BRDFBBidirectional idirectional RReflectance eflectance DDistribution istribution FFunction unction

FFrr((ii,,II,,ee,,ee)) = = LLee((ee,,ee) / E) / Eii((ii,,ii))

• Still a ratio (outgoing/incoming) light, butStill a ratio (outgoing/incoming) light, but• BRDF: BRDF: Ratio of Ratio of

outgoing outgoing RADIANCERADIANCE in one direction: in one direction: LLee((ee,,ee))that results from that results from incoming incoming IRRADIANCEIRRADIANCE in one direction: in one direction: EEii((ii,,ii))

• Units are tricky:Units are tricky:

BRDF = FBRDF = Frr = L = Le e // EEii

dAdAii

ii

LLeeEEii

Point-wise Light Reflection: BRDFPoint-wise Light Reflection: BRDFBBidirectional idirectional RReflectance eflectance DDistribution istribution FFunction unction

FFrr((ii,,II,,ee,,ee)) = = LLee((ee,,ee) / E) / Eii((ii,,ii))

• Still a ratio (outgoing/incoming) light, butStill a ratio (outgoing/incoming) light, but• BRDF: Ratio of BRDF: Ratio of

outgoing RADIANCE in one direction: Loutgoing RADIANCE in one direction: Lee((ee,,ee))that results from that results from incoming IRRADIANCE in one direction: Eincoming IRRADIANCE in one direction: E ii((ii,,ii))

• Units are tricky:Units are tricky:

BRDF = FBRDF = Frr = L = Le e // EEii = = ( Watts/area/sr) /(Watts/area)( Watts/area/sr) /(Watts/area)

dAdAii

ii

LLeeEEii

Point-wise Light Reflection: BRDFPoint-wise Light Reflection: BRDFBBidirectional idirectional RReflectance eflectance DDistribution istribution FFunction unction

FFrr((ii,,II,,ee,,ee)) = = LLee((ee,,ee) / E) / Eii((ii,,ii))

• Still a ratio (outgoing/incoming) light, butStill a ratio (outgoing/incoming) light, but• BRDF: Ratio of BRDF: Ratio of

outgoing RADIANCE in one direction: Loutgoing RADIANCE in one direction: Lee((ee,,ee))that results from that results from incoming IRRADIANCE in one direction: Eincoming IRRADIANCE in one direction: E ii((ii,,ii))

• Units are tricky:Units are tricky:

BRDF = FBRDF = Frr = L = Le e // EEii = ( Watts/area/sr) / = 1/sr = ( Watts/area/sr) / = 1/sr(Watts/area)(Watts/area)

Point-wise Light Reflection: BRDFPoint-wise Light Reflection: BRDFBBidirectional idirectional RReflectance eflectance DDistribution istribution FFunction unction

FFrr((ii,,II,,ee,,ee)) = = LLee((ee,,ee) / E) / Eii((ii,,ii), and (1/sr)units), and (1/sr)units

• ‘‘Bidirectional’ because value is SAME if we Bidirectional’ because value is SAME if we swap in,out directions: swap in,out directions: ((ee,,ee)) ((ii,,ii))

Important Property! aka ‘Helmholtz Reciprocity’Important Property! aka ‘Helmholtz Reciprocity’• BRDF Results from surface’sBRDF Results from surface’s

microscopic structure...microscopic structure...• Still only an approximation:Still only an approximation:

ignores subsurface scattering... ignores subsurface scattering...

dAdAii

ii

LLeeEEii

Scattering Difficulties:Scattering Difficulties:For many surfaces, single-point BRDFs do not existFor many surfaces, single-point BRDFs do not exist

Angles Depend on Angles Depend on refractive index, refractive index, scattering, cell wallscattering, cell wallstructures, etc.structures, etc.

Depends on total areaDepends on total areaof cell wall interfacesof cell wall interfaces

Example:Example: Leaf Structure Leaf Structure

dAdAii

ii

LLeeEEii

Subsurface Scattering ModelsSubsurface Scattering ModelsClassical: Kubelka-Monk(1930s, for paint; many proprietary variants), Classical: Kubelka-Monk(1930s, for paint; many proprietary variants), CG approach: Hanrahan & Krueger(1990s)CG approach: Hanrahan & Krueger(1990s)More Recent: ‘dipole model’ (2001, Jensen) More Recent: ‘dipole model’ (2001, Jensen)

Marble BSSRDFMarble BSSRDFMarble BRDFMarble BRDF

Subsurface Scattering ModelsSubsurface Scattering ModelsClassical: Kubelka-Monk(1930s, for paint; many proprietary variants), Classical: Kubelka-Monk(1930s, for paint; many proprietary variants), CG approach: Hanrahan & Krueger(1990s)CG approach: Hanrahan & Krueger(1990s)More Recent: ‘dipole model’ (2001, Jensen) More Recent: ‘dipole model’ (2001, Jensen)

Skin BSSRDF (approximated)Skin BSSRDF (approximated)Skin BRDF (measured)Skin BRDF (measured)

BSSRDF ModelBSSRDF ModelApproximates scattering result as Approximates scattering result as

embedded point sources below a BRDF surface:embedded point sources below a BRDF surface:

BSSRDF:BSSRDF: “A Practical Model for Subsurface Light Transport” Henrik “A Practical Model for Subsurface Light Transport” Henrik Wann Jensen, Steve Marschner, Marc Levoy, Pat Hanrahan, Wann Jensen, Steve Marschner, Marc Levoy, Pat Hanrahan, SIGGRAPH’01 (SIGGRAPH’01 (onlineonline))

BSSRDF ModelBSSRDF Model• Embedded point sources Embedded point sources

below a BRDF surfacebelow a BRDF surface

• Ray-based, tested, Ray-based, tested, Physically-Measurable Physically-Measurable Model Model

• ?Useful as a ?Useful as a predictive model for predictive model for IBMR data?IBMR data?

Wann Jensen et al., 2001Wann Jensen et al., 2001

Summary: Light MeasurementSummary: Light Measurement• Flux Flux WW = power, Watts, # photons/sec = power, Watts, # photons/sec• Irradiance Irradiance E = Watts/area = dW/dAE = Watts/area = dW/dA• Radiance Radiance L = (Watts/area)/sr = (dW/dA)/srL = (Watts/area)/sr = (dW/dA)/sr• BRDF: BRDF: Measure EMITTED radiance that Measure EMITTED radiance that

results from INCOMING irradiance from just results from INCOMING irradiance from just one direction:one direction:BRDF = FBRDF = Frr = L = Le / e / EEii = (Watts/area) / = (Watts/area) /

(Watts/area (Watts/areasr)sr)

IBMR ToolsIBMR Tools• Digital Light Input:Digital Light Input:

– Light meterLight meter: measure visible irradiance E : measure visible irradiance E (some have plastic ‘dome’ to ensure accurate foreshortening)

– CameraCamera: pixels measure Radiance L: pixels measure Radiance L i i ; flux arriving at lens ; flux arriving at lens from one (narrow solid) anglefrom one (narrow solid) angle

• Digital Light Output:Digital Light Output:– LuminairesLuminaires: point lights, extended(area) sources: point lights, extended(area) sources– Emissive SurfacesEmissive Surfaces: CRT, LCD surface: CRT, LCD surface– ProjectorsProjectors: laser dot,stripe,scan; video display: laser dot,stripe,scan; video display

• Light Modifiers (Digital?):Light Modifiers (Digital?):– Calibration objectsCalibration objects, shadow sources, etc., shadow sources, etc.– LensesLenses,,diffusersdiffusers, filters, reflectors, collimators..., filters, reflectors, collimators...– ?Where are the BRDF displays / printers??Where are the BRDF displays / printers?

Two Big Missing PiecesTwo Big Missing Pieces• Computer controlled BRDF. Computer controlled BRDF.

– Can we really do without it? Can we really do without it? – are cameras and projectors enough to are cameras and projectors enough to

‘import the visible world’ into our computers?‘import the visible world’ into our computers?

• BRDF is not enough:BRDF is not enough:– Subsurface scattering is Subsurface scattering is

crucial aspect of photographed imagescrucial aspect of photographed images– ? how can we model it? measure it? use it?? how can we model it? measure it? use it?

More help:More help:• GREAT explanation of BRDF: GREAT explanation of BRDF: • www.cs.huji.ac.il/~danix/advanced/RenderingEq.pdfwww.cs.huji.ac.il/~danix/advanced/RenderingEq.pdf

• Some questions about measuring light:Some questions about measuring light:

ENDEND

ProProjjectsects??(due Tues May 25!)(due Tues May 25!)

Let’s discuss them…Let’s discuss them…

IBMR---May 13,2004:IBMR---May 13,2004:

Summary: Light MeasurementSummary: Light Measurement• Flux Flux WW = power, Watts, # photons/sec = power, Watts, # photons/sec• Irradiance Irradiance E = Watts/area = dW/dAE = Watts/area = dW/dA• Radiance Radiance L = (Watts/area)/sr = (dW/dA)/srL = (Watts/area)/sr = (dW/dA)/sr• BRDF: BRDF: Measure EMITTED radiance that Measure EMITTED radiance that

results from INCOMING irradiance from just results from INCOMING irradiance from just one direction:one direction:BRDF = FBRDF = Frr = L = Le / e / EEii = (Watts/area) / = (Watts/area) /

(Watts/area (Watts/areasr)sr)

IBMR: Measure,Create, Modify LightIBMR: Measure,Create, Modify LightHow can we measure How can we measure ‘rays’ ‘rays’ of light? Light Sources? Scattered rays? etc.of light? Light Sources? Scattered rays? etc.

Shape, Shape, Position,Position,Movement,Movement,

BRDF,BRDF,Texture,Texture,ScatteringScattering

Emitted Emitted LightLight

Reflected,Reflected,Scattered,Scattered,Light …Light …

Cameras capture Cameras capture subset of these subset of these rays.rays.

Digital light Digital light sources sources (Projectors) can (Projectors) can produce a subset produce a subset of these rays.of these rays.

‘‘Scene’ modifies Set of Light RaysScene’ modifies Set of Light RaysWhat measures light rays in, out of scene? What measures light rays in, out of scene?

Measure Light LEAVING a Scene?Measure Light LEAVING a Scene?Towards a camera?...Towards a camera?...

Measure Light LEAVING a Scene?Measure Light LEAVING a Scene?Towards a camera: Towards a camera: Radiance. Radiance.

Light Field ImagesLight Field Imagesmeasure Radiancemeasure Radiance L(x,y)L(x,y)

Measure light ENTERING a Scene?Measure light ENTERING a Scene?from a (collection of) point sources at infinity?from a (collection of) point sources at infinity?

Measure light ENTERING a Scene?Measure light ENTERING a Scene?from a (collection of) point sources at infinity?from a (collection of) point sources at infinity?

‘‘Light Map’ Images Light Map’ Images (texture map light source)(texture map light source)

describes Irradiancedescribes Irradiance E(x,y)E(x,y)

Measure light ENTERING a Scene?Measure light ENTERING a Scene?leaving a video projector lens?leaving a video projector lens?

RadianceRadiance LL‘‘Reversed’ Camera:Reversed’ Camera:emits Radianceemits Radiance L(x,y)L(x,y)

Measure light ENTERING a Scene?Measure light ENTERING a Scene?from a video projector?—Leaving Lens: from a video projector?—Leaving Lens:

IrradianceIrradiance EE

RadianceRadiance LL

• Cleaner Formulation:Cleaner Formulation:– Orthographic camera,Orthographic camera,– positioned on sphere positioned on sphere

around object/scenearound object/scene– Orthographic projector,Orthographic projector,– positioned on spherepositioned on sphere

around object/scenearound object/scene– (and wavelength and time)(and wavelength and time)

F(F(xxcc,y,ycc,,cc,,cc,,xxll,y,yll ll,,ll,, , t, t))

‘‘Full 8-D Light Field’ Full 8-D Light Field’ (10-D, actually: time, (10-D, actually: time, ))

cameracamera

projectorprojector

Summary: Light MeasurementSummary: Light Measurement• Flux Flux WW = power, Watts, # photons/sec = power, Watts, # photons/sec• Irradiance Irradiance E = Watts/area = dW/dAE = Watts/area = dW/dA• Radiance Radiance L = (Watts/area)/sr = (dW/dA)/srL = (Watts/area)/sr = (dW/dA)/sr• BRDF: BRDF: Measure EMITTED radiance that results Measure EMITTED radiance that results

from INCOMING irradiance from just one direction:from INCOMING irradiance from just one direction:BRDF = FBRDF = Frr = L = Le / e / EEii = (Watts/area) / = (Watts/area) /

(Watts/area (Watts/areasr)sr)

Lenses map radiance to the image plane (x,y):Lenses map radiance to the image plane (x,y):THUS: Pixel x,y must measureTHUS: Pixel x,y must measure RadianceRadiance L at x,y. L at x,y.

well, not exactly; there are distortions!well, not exactly; there are distortions!……

What do Photos Measure?What do Photos Measure?

What We WantWhat We Want What We GetWhat We Get

Film Response:Film Response:(digital cameras, video cards too!)(digital cameras, video cards too!)

approximately linear, approximately linear, but ONLY on log-log axes.but ONLY on log-log axes.

Two Key parameters:Two Key parameters:m == scale == exposurem == scale == exposure

== gamma == ‘contrastyness’== gamma == ‘contrastyness’

???? ????0 255 0 255

Domain of Human Vision:Domain of Human Vision:from ~10-6 to ~10+8 cd/m2

Range of Typical Displays:Range of Typical Displays:from ~1 to ~100 cd/m2

starlightstarlight moonlightmoonlight office lightoffice light daylightdaylight flashbulbflashbulb

1010-6-6 1010-2-2 11 1010 100100 1010+4+4 1010+8+8

Problem:Map Scene to Display

High-Contrast Image Capture?High-Contrast Image Capture?• An open problem! (esp. for video...)An open problem! (esp. for video...)• Direct (expensive) solution:Direct (expensive) solution:

– Flying Spot Radiometer: Flying Spot Radiometer: brute force instrument, costly, slow, delicate brute force instrument, costly, slow, delicate

– Novel Image Sensors:Novel Image Sensors:line-scan cameras, logarithmic CMOS line-scan cameras, logarithmic CMOS

circuits, circuits, cooled detectors, rate-based cooled detectors, rate-based detectors...detectors...

• Most widely used idea: multiple exposuresMost widely used idea: multiple exposures• Elegant paper (Elegant paper (Debevec1996Debevec1996) describes how:) describes how:(On class website)(On class website)

starlight moonlight office light daylight flashbulb

Use Overlapped Exposure ValuesUse Overlapped Exposure Values

Use Overlapped Exposure ValuesUse Overlapped Exposure Values

starlight moonlight office light daylight flashbulb

Use Overlapped Exposure ValuesUse Overlapped Exposure Values

starlight moonlight office light daylight flashbulb

Use Overlapped Exposure ValuesUse Overlapped Exposure Values

starlight moonlight office light daylight flashbulb

What is the camera response curve?What is the camera response curve?And what are the pixel radiances?And what are the pixel radiances?(See Debevec SIGGRAPH 1997:)(See Debevec SIGGRAPH 1997:)

?? f(logL)f(logL)

Debevec’97 MethodDebevec’97 Method

j=0j=0j=1j=1

i=2i=2j=3j=3

j=4j=4j=5j=5

j=6j=6

STEP 1: STEP 1: --number the images ‘i’,--number the images ‘i’,--pick fixed spots (x--pick fixed spots (xjj,y,yjj) ) that sample scene’s that sample scene’s radiance values logLradiance values logLii well: well:

j=0j=0 11 22 33 44 55 66

??logLlogLii

Pix

el V

alue

ZP

ixel

Val

ue Z

f(logL)f(logL)

Debevec’97 MethodDebevec’97 Method

j=0j=0j=1j=1

i=2i=2j=3j=3

j=4j=4j=5j=5

j=6j=6

STEP 2: STEP 2: --Collect pixel values Z--Collect pixel values Zijij (from image i, location j)

--(All of them sample the --(All of them sample the response curve f(logL)…)response curve f(logL)…)

logLlogLii

Pix

el V

alue

ZP

ixel

Val

ue Z

j=0j=0 11 22 33 44 55 66

?? f(logL)f(logL)

??

logLlogLii

Pix

el V

alue

ZP

ixel

Val

ue Z

j=0j=0 11 22 33 44 55 66ZZijij

• In image IIn image Ij j , `exposure’ changed by log(2, `exposure’ changed by log(2jj) = j * log(2) ) = j * log(2)

• In image IIn image Ij j ,pixel at ,pixel at (x(xii,y,yii)) has has knownknown pixel value pixel value ZZijij and and unknownunknown radiance radiance logLlogLii

• Film response curve: f(logL) = Z; we know severalFilm response curve: f(logL) = Z; we know several logL logLii:: f(f(log(Llog(Ljj * 2 * 2jj))) ) = = ZZijij , or more simply: , or more simply: f(f(logLlogLii + j*C + j*C)) = = ZZijij

Pix

el V

alue

ZP

ixel

Val

ue Z

F(logL) ? F(logL) ?

logLlogL

Debevec’97 MethodDebevec’97 Method

Debevec’97 Method: Debevec’97 Method: It’s another Null-Space Problem…It’s another Null-Space Problem…

??

f(logLf(logLii – (j*C)) – (j*C))logLlogLii

Pix

el V

alue

ZP

ixel

Val

ue Z

j=0j=0 11 22 33 44 55 66ZZijij

• In image IIn image Ij j ,pixel at (x,pixel at (xii,y,yii) has ) has knownknown pixel value pixel value ZZijij and and unknownunknown radiance radiance logLlogLii

• f(logLf(logLii + j*C + j*C)) = = ZZij ij How do we find How do we find f() f() and and logLlogLii??

• TRICK: Use TRICK: Use f()f() as a scale factor for each pixel value as a scale factor for each pixel value ZZijij

ffijij ** ( (logLlogLjj + j*C)+ j*C) – – ZZijij = 0= 0

Pix

el V

alue

ZP

ixel

Val

ue Z

ffijij

Debevec’97 Method: Debevec’97 Method: It’s another Null-Space Problem…It’s another Null-Space Problem…

??

f(logLf(logLii – (j*C)) – (j*C))logLlogLii

Pix

el V

alue

ZP

ixel

Val

ue Z

j=0j=0 11 22 33 44 55 66ZZijij

ffijij ** ( (logLlogLjj + j*C)+ j*C) – – ZZijij = 0= 0

Pix

el V

alue

ZP

ixel

Val

ue Z

ffijij

Wait,wait, wait. We have TWO unknowns? Wait,wait, wait. We have TWO unknowns?

NEXT WEEK: Read Debevec’97NEXT WEEK: Read Debevec’97Explain how we solve this!Explain how we solve this!

CameraCamera Abilities / Limitations Abilities / Limitations• Nonlinear Intensity Response: S-shaped Nonlinear Intensity Response: S-shaped

(on log-log axes)(on log-log axes)

• Low-Contrast Devices: Noise limited Low-Contrast Devices: Noise limited (~500:1)(~500:1)

• Varied Spectral Response: RGBVaried Spectral Response: RGB11 != RGB != RGB22......• Color Sensing Strategies:Color Sensing Strategies:

– 3-chip cameras: best, but expensive!3-chip cameras: best, but expensive!– Mosaic sensor: trades resolution for colorMosaic sensor: trades resolution for color

• Nonuniform sensitivity & geometryNonuniform sensitivity & geometry– Lens limitations (vignetting, radial distortion, Lens limitations (vignetting, radial distortion,

bloom/scatter, uneven focus, ...)bloom/scatter, uneven focus, ...)– CCD Sensor geometry: VERY exact, repeatableCCD Sensor geometry: VERY exact, repeatable

DisplayDisplay Abilities / Limitations Abilities / Limitations• Nonlinear Intensity Response: S-shapedNonlinear Intensity Response: S-shaped• Low-Contrast Devices Low-Contrast Devices

– scattering usually sets upper boundsscattering usually sets upper bounds– Best Contrast: laser projectors, some DLP Best Contrast: laser projectors, some DLP

devices, specialized devices...)devices, specialized devices...)• Varied Spectral Response: RGBVaried Spectral Response: RGB11 != RGB != RGB22......• Color Reproducing Strategies: varied...Color Reproducing Strategies: varied...• Nonuniform sensitivity & geometry:Nonuniform sensitivity & geometry:

– CRTs: e-beam CRTs: e-beam cos(cos(), ), distortion, focus, distortion, focus, convergence...convergence...

– LCDs, DLPs: VERY exact, (but pixels die, etc.)LCDs, DLPs: VERY exact, (but pixels die, etc.)

Light Modifiers? Light Modifiers? Discuss! Discuss!• Low-Contrast BRDF ‘Devices’ to measure light?Low-Contrast BRDF ‘Devices’ to measure light?

– ‘‘Light Probe’ mirror sphere BRDF = ?Light Probe’ mirror sphere BRDF = ?– Diffuse reflectances limited to about 0.02Diffuse reflectances limited to about 0.020.950.95– Diffractive materials: complex BRDF may be useful...Diffractive materials: complex BRDF may be useful...– (Transmissive LCDs?) (Transmissive LCDs?) ?Can you name more??Can you name more?

• PRECISELY Linear ‘Response’ to light... PRECISELY Linear ‘Response’ to light... BRDFs are fixed ratios; no intensity dependence!BRDFs are fixed ratios; no intensity dependence!

• Smudge, nick may modify BRDF drasticallySmudge, nick may modify BRDF drastically• Shadows? Precision? Inter-reflections?Shadows? Precision? Inter-reflections?• PRECISE input/output symmetryPRECISE input/output symmetry--BUT----BUT--• Scattering WITHIN material can be trouble...Scattering WITHIN material can be trouble...

What is the complete IBMR toolset?What is the complete IBMR toolset?• Camera(s) + light probe, etc:Camera(s) + light probe, etc:

arbitrary Radiance meter. arbitrary Radiance meter.

• Sphere of Projectors/CRTs:Sphere of Projectors/CRTs: arbitrary Irradiance source. arbitrary Irradiance source.

• Some (as yet unknown) device:Some (as yet unknown) device: arbitrary BRDF / light ray modifier arbitrary BRDF / light ray modifier

Is our toolset complete complete?Is our toolset complete complete?have we spanned the IBMR problem? ...have we spanned the IBMR problem? ...

Missing the most important tool…Missing the most important tool…• Human Visual System.Human Visual System.

– the receiver/user for MOST IBMR data.the receiver/user for MOST IBMR data.– Eye is a very poor light meter, but very good at Eye is a very poor light meter, but very good at

sensing BRDF and (some) shape.sensing BRDF and (some) shape.– Eye senses change;Eye senses change;

integration used to estimate the integration used to estimate the worldworld

– Eye permits tradeoffs of Eye permits tradeoffs of geometry vs. surface appearancegeometry vs. surface appearance

– Eye permits selective radiance distortions, Eye permits selective radiance distortions, especially to illumination:especially to illumination:

Details Everywhere;segmented partial -ordering of intensities.Local changes matter.Absolute intensities don’t matter much, but boundaries, shading, & CHANGES do.

---WANTED:---visually important informationin machine-readable form.

Picture: Copy Appearance

Visible Light MeasurementVisible Light Measurement• ‘‘Visible Light’ = what our eyes can perceive;Visible Light’ = what our eyes can perceive;

– narrow-band electromagnetic energy:narrow-band electromagnetic energy: 400-700 nm400-700 nm (nm = 10 (nm = 10-9 -9 meter)meter)<1 octave; (honey bees: 3-4 ‘octaves’ ?chords?)<1 octave; (honey bees: 3-4 ‘octaves’ ?chords?)

• Not uniformly visible vs. wavelength Not uniformly visible vs. wavelength : : – Equiluminant CurveEquiluminant Curve

defines ‘luminance’ defines ‘luminance’ vs. wavelengthvs. wavelength

– eyes sense spectraleyes sense spectralCHANGES well, butCHANGES well, butnot wavelengthnot wavelength

– MetamerismMetamerism

http://www.yorku.ca/eye/photopik.htmhttp://www.yorku.ca/eye/photopik.htm

Visual Appearance MeasurementVisual Appearance Measurement• Measurement of Light—easy. Perception?—hardMeasurement of Light—easy. Perception?—hard..

– ‘‘Color’ ==crudely perceived wavelength spectrumColor’ ==crudely perceived wavelength spectrum– 3 sensed dimensions from spectra.3 sensed dimensions from spectra.– CIE-standard X,Y,Z color spectra: linear coord. system CIE-standard X,Y,Z color spectra: linear coord. system

for spectra that spans all perceivable colors X,Y,Zfor spectra that spans all perceivable colors X,Y,Z– Projective! Projective! luminance = Z luminance = Z

chromaticity = (x,y) = (X/Z, Y/Z)chromaticity = (x,y) = (X/Z, Y/Z)– NOT perceptually uniform... (MacAdam’s ellipses...)NOT perceptually uniform... (MacAdam’s ellipses...)

• Many Standard Texts, tutorials on colorMany Standard Texts, tutorials on color– Good: Good: http://www.colourware.co.uk/cpfaq.htmhttp://www.colourware.co.uk/cpfaq.htm– Good: Good: http://www.yorku.ca/eye/toc.htmhttp://www.yorku.ca/eye/toc.htm – Watt & Watt pg 277-281Watt & Watt pg 277-281

ENDEND