CS 395/495-25: Spring 2003CS 395/495-25: Spring 2003

IBMR: Week 10A IBMR: Week 10A

Dimensionality ofDimensionality ofIBMR MethodsIBMR Methods

Jack TumblinJack Tumblin

jet@cs.northwestern.edujet@cs.northwestern.edu

RemindersReminders

• ProjA graded: Good Job! 90,95, 110ProjA graded: Good Job! 90,95, 110

• ProjB graded: Good! minor H confusions.ProjB graded: Good! minor H confusions.

• MidTerm graded: novel solutions MidTerm graded: novel solutions encouraged.encouraged.

• ProjC due Friday, May 16: graded.ProjC due Friday, May 16: graded.

• ProjD posted, due Friday May 30 ProjD posted, due Friday May 30 too many unfinished—final is too many unfinished—final is

coming!coming!

• Take-Home Final Exam: Take-Home Final Exam: Assign on Thurs June 5, due June 11Assign on Thurs June 5, due June 11

Handouts today:Handouts today:

• Levoy/Marschner BibliographyLevoy/Marschner Bibliography

• Masselus et. al, 2003: Masselus et. al, 2003: Relighting with 4D incident Light FieldsRelighting with 4D incident Light Fields((on the websiteon the website))

‘‘Rendering’ from a camera?Rendering’ from a camera?

Many choices of dimensions for IBMR:Many choices of dimensions for IBMR:

Shape, Shape, Position,Position,Movement,Movement,

BRDF,BRDF,Texture,Texture,ScatteringScattering

Emitted Emitted LightLight

Reflected,Reflected,Scattered,Scattered,Light …Light …

yycc

zzcc xxcc

CameraCamera

1-D? Moving Line-scan Cameras1-D? Moving Line-scan Cameras

Multiple-Center-of-Projection ImagesMultiple-Center-of-Projection Images

Rademacher, Pl, Bishop, G., SIGGRAPH '98Rademacher, Pl, Bishop, G., SIGGRAPH '98

1-D? Moving Line-scan Cameras1-D? Moving Line-scan Cameras

Rademacher, Pl, Bishop, G., Multiple-Center-of-Projection Images SIGGRAPH '98.Rademacher, Pl, Bishop, G., Multiple-Center-of-Projection Images SIGGRAPH '98. Rademacher, Pl, Bishop, G., Multiple-Center-of-Projection Images SIGGRAPH '98.Rademacher, Pl, Bishop, G., Multiple-Center-of-Projection Images SIGGRAPH '98.

1-D? Moving Line-scan Cameras1-D? Moving Line-scan Cameras

Shum, Concentric Image Mosaics SIGGRAPHShum, Concentric Image Mosaics SIGGRAPH

• Andrew Davidhazy, RITAndrew Davidhazy, RIT: : http://www.rit.edu/~andpph/http://www.rit.edu/~andpph/

1-D? Moving Line-scan Cameras1-D? Moving Line-scan Cameras

? What other unusual mappings of position vs. ? What other unusual mappings of position vs. 4-D ray direction might help in IBMR?4-D ray direction might help in IBMR?– Example: [Alison Ortony] torus map: Example: [Alison Ortony] torus map:

--pre-aligned epipolar geometry, --pre-aligned epipolar geometry, --no ‘north/south’ pole problems--no ‘north/south’ pole problems

2-D: Image-Space Manipulations2-D: Image-Space Manipulations

• Morphs Morphs – Beier92, Beier92, – Seitz96, Seitz96, – LightField Morphs 2002 ...LightField Morphs 2002 ...

• deep pixelsdeep pixels– PTMs, PTMs, – Environment Matte...Environment Matte...

Impostors/Billboards:Impostors/Billboards:

Malzbender, HPlabs 2001

A A MostlyMostly 2-D Method2-D Method

Polynomial Texture MapsPolynomial Texture Maps

Store just Store just 66 coefficientscoefficients

at each pixel, at each pixel, get Interactive get Interactive

re-lighting...re-lighting...

‘‘2.5’-D: Images + a little more...2.5’-D: Images + a little more...

• 2.5D: Depth textures2.5D: Depth textures

• Billboards/Impostors, LDIs: Billboards/Impostors, LDIs: Multiple pixels along line-of sight... Jonathan Shade, Steven J. Gortler, Li-wei Multiple pixels along line-of sight... Jonathan Shade, Steven J. Gortler, Li-wei He, Richard Szeliski. He, Richard Szeliski. SIGGRAPH 98.SIGGRAPH 98. http://grail.cs.washington.edu/projects/ldi/http://grail.cs.washington.edu/projects/ldi/

Generalization: LDI Trees, SIGG99Generalization: LDI Trees, SIGG99 http://www.cs.unc.edu/~ibr/pubs/chang-sg99/abstract.htmlhttp://www.cs.unc.edu/~ibr/pubs/chang-sg99/abstract.html

‘‘2.5’-D: Images + a little more...2.5’-D: Images + a little more...

2.5D: Images plus different lighting2.5D: Images plus different lighting

• Relighting by linear image combos: demoRelighting by linear image combos: demohttp://www.sgi.com/grafica/synth/index.htmlhttp://www.sgi.com/grafica/synth/index.html

• Non-Physical Combinations:Non-Physical Combinations:– Negative Light Negative Light – Generalizations—Malzbender2001 and others,Generalizations—Malzbender2001 and others,– Debevec: sphere of point sources Debevec: sphere of point sources

polarizer separates diffuse, specular terms polarizer separates diffuse, specular terms (simplified BRDF)(simplified BRDF)

– Factor images for BRDF samples: Factor images for BRDF samples: McCool, RamamoorthyMcCool, Ramamoorthy

Debevec ‘Light Stage’Debevec ‘Light Stage’

Image-Based Actual Re-lightingImage-Based Actual Re-lighting

Film the background in Milan,Film the background in Milan,Measure incoming light,Measure incoming light,

Light the actress in Los AngelesLight the actress in Los Angeles

Matte the backgroundMatte the background

Matched LA and Milan lighting.Matched LA and Milan lighting.

Debevec et al., SIGG2001

Williams 1998: ‘Inflated Silhouettes’Williams 1998: ‘Inflated Silhouettes’• http://graphics.stanford.edu/workshops/ibr98/#Schedule%20of%20sessionshttp://graphics.stanford.edu/workshops/ibr98/#Schedule%20of%20sessions

Williams 1998: ‘Inflated Silhouettes’Williams 1998: ‘Inflated Silhouettes’

• Symmetric, ‘Natural’ objectsSymmetric, ‘Natural’ objects• Reliable Silhouette extractionReliable Silhouette extraction• Implicit Function for Depth: Implicit Function for Depth:

rounded, diffused, relaxation...rounded, diffused, relaxation...• Refine with ?Refine with ? texture/shading?texture/shading?

• manual ‘paintbox’ for depth?manual ‘paintbox’ for depth?

• Simple, quick, Simple, quick, fairly good results...fairly good results...

Williams 1998: ‘Inflated Silhouettes’Williams 1998: ‘Inflated Silhouettes’• http://graphics.stanford.edu/workshops/ibr98/#Schedule%20of%20sessionshttp://graphics.stanford.edu/workshops/ibr98/#Schedule%20of%20sessions

Not bad! Can we do better?Not bad! Can we do better?

3-D Methods3-D Methods

• Fans, silhouettes:Fans, silhouettes:– Ramesh Raskar and Michael Cohen. Ramesh Raskar and Michael Cohen. Image Precision Silhouette Edges.Image Precision Silhouette Edges. I3D 99 I3D 99..

• Voxel CarvingVoxel Carving(Seitz, others)(Seitz, others)

Seitz,1998Seitz,1998

3D: Try image + other dimensions3D: Try image + other dimensions

• Halle: Multiple Viewpoint Rendering (SIGG98)Halle: Multiple Viewpoint Rendering (SIGG98)http://web.media.mit.edu/~halazar/sig98/halle98.http://web.media.mit.edu/~halazar/sig98/halle98.

pdfpdf

Oh et. al, 2001: 2DOh et. al, 2001: 2D3D3D

• 3D ‘Power tools’ for editing 2D images3D ‘Power tools’ for editing 2D imageshttp://graphics.lcs.mit.edu/ibedit/ibedit_s2001_cameraReady.pdfhttp://graphics.lcs.mit.edu/ibedit/ibedit_s2001_cameraReady.pdf

4-D Methods4-D Methods

Incoming Rays: 4DIncoming Rays: 4D

Outgoing Rays: 4DOutgoing Rays: 4D

• Light fieldsLight fields

• Illum FieldsIllum Fields

• Shape/Light field refinementShape/Light field refinement

4D Methods4D Methods

Light-source Light Fields Light-source Light Fields Accurate Light Source Acquisition and RenderingAccurate Light Source Acquisition and Rendering

M. Goesele, X. Granier, W. Heidrich, H.-P. Seidel, SIGG2003M. Goesele, X. Granier, W. Heidrich, H.-P. Seidel, SIGG2003

Image-Based Synthetic Re-lightingImage-Based Synthetic Re-lighting

Masselus et al., 2002Masselus et al., 2002

Basis imagesIlluminant direction

estimationTake pictures and move hand-held light source

Incidentlight mapCreate weighted sum

…

· W1 + · W2 +

+ · Wn

Relit object

Masselus et al., 2002,Masselus et al., 2002, “Free-Form Light Stage” “Free-Form Light Stage”

Matusik 2002

Image-Based Shape ApproximationImage-Based Shape Approximation

Matusik 2002

Matusik 2002

Image-Based Shape ApproximationImage-Based Shape Approximation

Rushmeier, 2001

Fine geometric Details Fine Texture/Normal Details

Image-Based Shape RefinementImage-Based Shape Refinement

5-D Methods and beyond...5-D Methods and beyond...

surface light fields,surface light fields,(Chen et al, SIGG02)(Chen et al, SIGG02)

Synthetic+ realSynthetic+ real

5-D Methods and beyond...5-D Methods and beyond...

Relighting with 4D Incident Light FieldsRelighting with 4D Incident Light Fields Vincent Masselus, Vincent Masselus, Pieter Peers, Philip Dutre and Yves D. Willems SIGG2003 Pieter Peers, Philip Dutre and Yves D. Willems SIGG2003

Rushmeier, 1996

Interreflections?Interreflections?

ENDEND