realtime caustics using distributed photon mapping

Realtime CausticsRealtime Causticsusingusing

Distributed Photon MappingDistributed Photon Mapping

Johannes Günther Ingo WaldJohannes Günther Ingo Wald** Philipp Slusallek Philipp Slusallek

Computer Graphics GroupComputer Graphics GroupSaarland UniversitySaarland University

((**now at MPII Saarbrücken)now at MPII Saarbrücken)

June 21, 2004 EGSR 2004, Norrköping, Sweden 2

IntroductionIntroduction

• What is wrong with this picture?What is wrong with this picture?


IntroductionIntroduction

• With caustic illuminationWith caustic illumination


OutlineOutline

• Introduction & MotivationIntroduction & Motivation• Previous WorkPrevious Work• Analysis of Photon MappingAnalysis of Photon Mapping• Faster Photon MappingFaster Photon Mapping• Parallelization IssuesParallelization Issues• Results & ExamplesResults & Examples


Previous WorkPrevious WorkCausticsCaustics

• (Bidirectional) path tracing [Lafortune, Veach](Bidirectional) path tracing [Lafortune, Veach]– Need many samples Need many samples too slow / too noisy too slow / too noisy

• Density estimation [Shirley, Walter]Density estimation [Shirley, Walter]– Use of pre-computed, view-independent illumination Use of pre-computed, view-independent illumination

representation representation Not interactive Not interactive

• Particles in hierarchical radiosity system [Granier]Particles in hierarchical radiosity system [Granier]– Particle tracing accelerated by exploiting HR structureParticle tracing accelerated by exploiting HR structure– Still too slow for interactive useStill too slow for interactive use

• Hardware based projection [Wand]Hardware based projection [Wand]– Implemented via texture lookups on GFXImplemented via texture lookups on GFX– Limited to far lights and no occluders consideredLimited to far lights and no occluders considered– Only reflective caustic generatorsOnly reflective caustic generators


Previous WorkPrevious Work Photon Mapping Photon Mapping

• Photon map [Jensen]Photon map [Jensen]– Preferred tool for causticsPreferred tool for caustics– Independent of scene geometryIndependent of scene geometry– But problematic for interactive use (later)But problematic for interactive use (later)

• Hashed photon mapping [Wald]Hashed photon mapping [Wald]– Approximation Approximation lower quality lower quality

• On programmable graphics hardware [Purcell]On programmable graphics hardware [Purcell]– Also approximativeAlso approximative– Limited performance (interactive only at low Limited performance (interactive only at low

resolution)resolution)– No caustics during interactionNo caustics during interaction– Simple scenesSimple scenes


OutlineOutline



Analysis – OverviewAnalysis – Overview

Photon Map:Photon Map:

• PreprocessingPreprocessing– Photon generationPhoton generation– Construction of Construction of kdkd-tree-tree

• Queries during renderingQueries during rendering– Search Search kk-nearest photons -nearest photons Density estimation Density estimation


AnalysisAnalysis

• Interactivity: now ‘pre’processing every frameInteractivity: now ‘pre’processing every frame• High quality caustic: > 100K photonsHigh quality caustic: > 100K photons

• No problem with realtime ray tracing No problem with realtime ray tracing (>4Mrays/s)(>4Mrays/s)??

• Low caustic photon yieldLow caustic photon yield– Several rays per photonSeveral rays per photon– Only few photons contribute to causticsOnly few photons contribute to caustics

(in many cases below 10%)(in many cases below 10%)

• Slow photon shootingSlow photon shooting– Incoherent rays Incoherent rays slow ray tracing, no SSE slow ray tracing, no SSE– Costly light source and BRDF samplingCostly light source and BRDF sampling

(up to 60% of total shooting time)(up to 60% of total shooting time) 4-15 times slower4-15 times slower


Analysis

• Building kd-tree– Serious problem with many photons

complexity O(n log n)– Hard to parallelize– Small changes in map complete rebuild

• Queries are costly– Traversal of kd-tree– Density estimation About 10 times as expensive as tracing a ray

Have to address all of these problems


OutlineOutline



Selective Photon Tracing Selective Photon Tracing [Dmitriev][Dmitriev]

• Adaptive sampling in path spaceAdaptive sampling in path space

• Exploiting similarity property of Halton numbersExploiting similarity property of Halton numbers• Indirect caustics capturedIndirect caustics captured• Improve yield by factor 4Improve yield by factor 4


Improve Improve kdkd-tree-tree

• Balancing Balancing kdkd-tree requires costly median -tree requires costly median findingfinding

• Instead: “split in middle”Instead: “split in middle”– Spatial center of current voxelSpatial center of current voxel– Now unbalancedNow unbalanced

Construction 1.5-4 times fasterConstruction 1.5-4 times faster


Improve Improve kdkd-tree-tree

• Balancing Balancing kdkd-tree requires costly median -tree requires costly median findingfinding

• Instead: “split in middle”Instead: “split in middle”– Spatial center of current voxelSpatial center of current voxel– Now unbalancedNow unbalanced

Construction 1.5-4 times fasterConstruction 1.5-4 times faster

• Not intuitiveNot intuitive: also traversal faster [Wald ’04]: also traversal faster [Wald ’04]– Better adaptation to photon densityBetter adaptation to photon density

Query 1.5-2.5 times fasterQuery 1.5-2.5 times faster


Accumulation of PhotonsAccumulation of Photons

• Performance still not enough for high qualityPerformance still not enough for high quality

Improve quality by accumulation across Improve quality by accumulation across framesframes

• kdkd-tree with “split in middle” allows -tree with “split in middle” allows augmentationaugmentation

• Accumulated result is Accumulated result is view-independentview-independent!! High quality walkthroughHigh quality walkthrough


OutlineOutline



Interleaved Sampling & Interleaved Sampling & Filtering [Keller]Filtering [Keller]

5x5 interleaving5x5 interleaving27k photons27k photons filteredfiltered

• Quality comparable to 25 x 27k = 680k Quality comparable to 25 x 27k = 680k photonsphotons


FilteringFilteringCaustic IlluminationCaustic Illumination

• Originally: restrict filtering to smooth regionsOriginally: restrict filtering to smooth regions– Continuity tests on server limits to 5fps (IGI [Wald ’02])Continuity tests on server limits to 5fps (IGI [Wald ’02])

But:But:• Only filter caustic illuminationOnly filter caustic illumination

– Potential artifacts less visiblePotential artifacts less visible

No need for costly discontinuity checksNo need for costly discontinuity checks

New:New:• Simple box filter Simple box filter efficient implementation efficient implementation

– SSE, cache friendly, independent of filter sizeSSE, cache friendly, independent of filter size

• Performance: CPU 300 fps, network now ~22 fpsPerformance: CPU 300 fps, network now ~22 fps


OutlineOutline



PerformancePerformance

• Scalability: roughly linearScalability: roughly linear• Upper limit: about 22 fps Upper limit: about 22 fps (network bounded)(network bounded)


Comparison to IGI Comparison to IGI [Wald ’02][Wald ’02]

• Same #clients (9) and frame rate (4.8 fps)Same #clients (9) and frame rate (4.8 fps)

2600 phs2600 phs

2500 phs2500 phs 16000 phs16000 phs

8000 phs8000 phs

IGI systemIGI system new systemnew system


ExamplesExamples

• Indirect causticsIndirect caustics• No performance dropNo performance drop


ExamplesExamples

• Headlight simulationHeadlight simulation• Complex modelComplex model• Many layers of glassMany layers of glass• Pointlights inside bulbsPointlights inside bulbs• Using 18 Dual clientsUsing 18 Dual clients

250k phs / 3 fps250k phs / 3 fps 25M phs / 11 fps25M phs / 11 fps


ExamplesExamples

• Headlight simulationHeadlight simulation• Complex modelComplex model• Many layers of glassMany layers of glass• Pointlights inside bulbsPointlights inside bulbs• Using 18 Dual clientsUsing 18 Dual clients

250k phs / 3 fps250k phs / 3 fps 25M phs / 11 fps25M phs / 11 fps photophoto


VideoVideo


Conclusion Conclusion

• Improvements on all aspects of the photon mapImprovements on all aspects of the photon map– Improved yield by Selective Photon TracingImproved yield by Selective Photon Tracing– Optimized Optimized kdkd-tree (construction and traversal)-tree (construction and traversal)

• Efficient parallelizationEfficient parallelization– Interleaved SamplingInterleaved Sampling– Faster filteringFaster filtering

Realtime caustic simulation High quality walkthrough Linear scalability, up to 22 fps


Questions ?Questions ?

Questions ?Questions ?


No discontinuity No discontinuity buffer?buffer?

• 5x5 filter, blurring over edges hardly visible5x5 filter, blurring over edges hardly visible


Photon MapPhoton Map

Two pass approach:Two pass approach:• PreprocessingPreprocessing

– GenerationGeneration• Emit photons into sceneEmit photons into scene• Trace path until absorptionTrace path until absorption

– Construction of Construction of kdkd-tree-tree• For fast retrieval of hit pointsFor fast retrieval of hit points

• During rendering (ray tracing)During rendering (ray tracing)– At each point to shade: do queryAt each point to shade: do query

• Search Search kk-nearest photons-nearest photons• Accumulate power & divide by occupied areaAccumulate power & divide by occupied area

Density estimationDensity estimation

Concentrate on causticsConcentrate on caustics


Client-Server Client-Server ApproachApproach

• High computational cost High computational cost need parallelization need parallelization– even more necessary with photon mappingeven more necessary with photon mapping

• Commodity PCsCommodity PCs– 24 Dual-AthlonMP 1800+24 Dual-AthlonMP 1800+

• For ray tracingFor ray tracing• ““Thin” clients: 100 Mbit Ethernet, no graphics card, ...Thin” clients: 100 Mbit Ethernet, no graphics card, ...

– 1 server node1 server node• Runs application, controls clients, etc.Runs application, controls clients, etc.

• Fully switched 100MBit networkFully switched 100MBit network– Gigabit uplink to serverGigabit uplink to server


OutlookOutlook

• Combination with Instant Radiosity / IGI2 to Combination with Instant Radiosity / IGI2 to add indirect illuminationadd indirect illumination

• Further improve sampling and caustic photon Further improve sampling and caustic photon yieldyield

• Still need a clusterStill need a cluster

Special Hardware (e.g. SaarCOR)Special Hardware (e.g. SaarCOR)


GoalGoal

• Generate and visualize caustic light effectsGenerate and visualize caustic light effects• Interactive frame ratesInteractive frame rates

– At video resolution 640x480At video resolution 640x480

• Support complex scenesSupport complex scenes• Preview quality acceptable during scene Preview quality acceptable during scene

modificationsmodifications• High quality walkthroughsHigh quality walkthroughs


Improve YieldImprove Yieldof Caustic Photonsof Caustic Photons

• Cannot help on #rays per photon pathCannot help on #rays per photon path• But: improve efficiency of samplingBut: improve efficiency of sampling

• Jensen:Jensen:– Project caustic generators onto (hemisphere of) light Project caustic generators onto (hemisphere of) light

sources, sample only occupied solid anglesources, sample only occupied solid angle– Captures only direct causticsCaptures only direct caustics– Mesh dependent, not in realtimeMesh dependent, not in realtime

• Wald:Wald:– Project only bounding volumeProject only bounding volume– Geometry independentGeometry independent– Bounding volume often too coarse approximationBounding volume often too coarse approximation– Requires manual user interaction / scene preparationRequires manual user interaction / scene preparation


Distribution OptionsDistribution Options

• Generate same photon map on each clientGenerate same photon map on each client– Inefficient, only rendering/queries in parallelInefficient, only rendering/queries in parallel– Not scalableNot scalable

• Generate different parts of photon map on Generate different parts of photon map on clientsclients– Need exchange of photonsNeed exchange of photons– But: network bandwidth too smallBut: network bandwidth too small– Rendering can only start after receiving all photonsRendering can only start after receiving all photons

latency latency

• Generate different photon maps on clientsGenerate different photon maps on clients– Combine Combine illuminationillumination on server by interleaving pixels on server by interleaving pixels– Reduce noise by filtering in image spaceReduce noise by filtering in image space

Interleaved sampling & discontinuity buffering [Keller]Interleaved sampling & discontinuity buffering [Keller]

realtime caustics using distributed photon mapping

Documents

costlytraversal of kd

realtime ray tracing

times sloweregsr

times fasteregsr

interactive usehardware

low resolutionno caustics

photons complexity

times fasternot intuitive