state of the art in realtime ray tracing & interactive global illumination

State of the Art inState of the Art in

Realtime Ray TracingRealtime Ray Tracing&&

Interactive Global IlluminationInteractive Global Illumination

Ingo WaldIngo Wald

Carsten Benthin Joerg Schmittler Philipp SlusallekCarsten Benthin Joerg Schmittler Philipp Slusallek

Saarland UniversitySaarland University

Timothy J. PurcellTimothy J. Purcell

Stanford UniversityStanford University

August 9th, 2003 Realtime Ray Tracing & Interactive Global Illumination

2

AgendaAgenda

• Introduction & Motivation [Wald]Introduction & Motivation [Wald]• Part I – Interactive Ray Tracing Architectures [Both]Part I – Interactive Ray Tracing Architectures [Both]

– Ray tracing in SW: The RTRT/OpenRT engine [Wald]Ray tracing in SW: The RTRT/OpenRT engine [Wald]

– Ray tracing on GPUs [Purcell]Ray tracing on GPUs [Purcell]

– Special purpose ray tracing hardware: SaarCOR [Purcell]Special purpose ray tracing hardware: SaarCOR [Purcell]

• Part II – Advanced Ray Tracing Issues [Wald]Part II – Advanced Ray Tracing Issues [Wald]– Handling dynamic scenesHandling dynamic scenes

– The OpenRT interactive ray tracing APIThe OpenRT interactive ray tracing API

– Practical applicationsPractical applications

• Part III – Interactive Global Illumination [Wald]Part III – Interactive Global Illumination [Wald]– Instant Global IlluminationInstant Global Illumination

• Conclusion & Outlook into the Future [Both]Conclusion & Outlook into the Future [Both]


3

AgendaAgenda











4

Introduction & MotivationIntroduction & Motivation

Computer Graphics Today:Computer Graphics Today:• Interactive Graphics (Games etc): Interactive Graphics (Games etc):

Hardware Accelerated “Triangle Rasterization” Hardware Accelerated “Triangle Rasterization”– ““paint each triangles onto the screen”paint each triangles onto the screen”

– High performance hardwareHigh performance hardware

– But: Only simple models, approximations, only local effects, etc.But: Only simple models, approximations, only local effects, etc.

• High-Quality Graphics (Movies, Design, Simulation etc):High-Quality Graphics (Movies, Design, Simulation etc): Software (offline) “Ray Tracing” Software (offline) “Ray Tracing”– ““trace rays from eye backwards into the scene”trace rays from eye backwards into the scene”

– Reverse physical light transport: Can model all effectsReverse physical light transport: Can model all effects

– All effects, complex scenes, physically correct, etc.All effects, complex scenes, physically correct, etc.

– But: Historically too costly for interactivityBut: Historically too costly for interactivity


5

Ray TracingRay Tracing

Simple AlgorithmSimple Algorithm

1.) Create ray from eye through pixel1.) Create ray from eye through pixel


6




2.) Trace ray into scene2.) Trace ray into scene


7




2.) Trace ray into scene2.) Trace ray into scene– Find objects nearby ray (traverse special data structures)Find objects nearby ray (traverse special data structures)

– Compute ray-object intersection testsCompute ray-object intersection tests

– Determine closest hitpointDetermine closest hitpoint

Grid (2D) Octree (2D)


8




3.) Compute color of ray (“shade” the ray)3.) Compute color of ray (“shade” the ray)



9




3.) Compute color of ray (“shade” the ray)3.) Compute color of ray (“shade” the ray)– Can recursively shoot new rays (e.g. reflections, shadow tests, etc)Can recursively shoot new rays (e.g. reflections, shadow tests, etc)



10




3.) Compute color of ray (“shade” the ray)3.) Compute color of ray (“shade” the ray)

4.) Display final image4.) Display final image



11


Advantages:Advantages:• Complexity is logarithmic in scene sizeComplexity is logarithmic in scene size

Ideally suited for complex modelsIdeally suited for complex models

• Allows plug-n-play shadingAllows plug-n-play shading Simple to use and extendSimple to use and extend

• Allows for Allows for exact exact visibility queriesvisibility queries– Physically correct shadows, reflections, global effects, ...Physically correct shadows, reflections, global effects, ...

Very beneficial to have at interactive rates !Very beneficial to have at interactive rates !


12


Advantages:Advantages:• Complexity is logarithmic in scene sizeComplexity is logarithmic in scene size

Ideally suited for complex modelsIdeally suited for complex models

• Allows plug-n-play shadingAllows plug-n-play shading Simple to use and extendSimple to use and extend

• Allows for Allows for exact exact visibility queriesvisibility queries– Physically correct shadows, reflections, global effects, ...Physically correct shadows, reflections, global effects, ...

Very beneficial to have at interactive rates !Very beneficial to have at interactive rates !

Part IPart I

Interactive Ray TracingInteractive Ray Tracing


15

Realizing Realtime Ray TracingRealizing Realtime Ray Tracing

Realtime Ray Tracing offers many advantages !Realtime Ray Tracing offers many advantages !

Question: How to realize Realtime Ray Tracing ?Question: How to realize Realtime Ray Tracing ?

Today: Three options...Today: Three options...• Faster software ray tracingFaster software ray tracing

– Supercomputers: Muuss et al, Utah [Parker et al.] Supercomputers: Muuss et al, Utah [Parker et al.]

– Clusters: Saarland OpenRT [Wald et al.], Utah [deMarle et al.]Clusters: Saarland OpenRT [Wald et al.], Utah [deMarle et al.]

• Ray Tracing on programmable GPUsRay Tracing on programmable GPUs– Carr et al., Purcell et al.Carr et al., Purcell et al.

• Building special purpose hardwareBuilding special purpose hardware– Schmittler et al.Schmittler et al.


16

AgendaAgenda










Software Ray TracingSoftware Ray Tracing

The RTRT/OpenRT Interactive Ray The RTRT/OpenRT Interactive Ray Tracing SystemTracing System


18

The OpenRT Interactive Ray The OpenRT Interactive Ray Tracing EngineTracing Engine

The OpenRT Interactive Ray Tracing Engine:The OpenRT Interactive Ray Tracing Engine:• Won’t go into technical detailsWon’t go into technical details

– No details on any kernel algorithmsNo details on any kernel algorithms

– See the STAR...See the STAR...

• More important: Summarize “State of the Art”More important: Summarize “State of the Art”– State of the Art of OpenRT FeaturesState of the Art of OpenRT Features

– State of the Art Performance DataState of the Art Performance Data

– Today’s possibilities, problems and challengesToday’s possibilities, problems and challenges


19

The OpenRT Interactive Ray The OpenRT Interactive Ray Tracing EngineTracing Engine

Features of OpenRT:Features of OpenRT:• Highly efficient RT kernels [EG’01,...]Highly efficient RT kernels [EG’01,...]

– Specially optimized for modern CPUs (caches, SSE, ...)Specially optimized for modern CPUs (caches, SSE, ...)– Recently much faster than EG’01 results...Recently much faster than EG’01 results...

• Parallelization on PC cluster [EuroPar’03]Parallelization on PC cluster [EuroPar’03]– Up to 24 dual-Athlon MP 1800+ Up to 24 dual-Athlon MP 1800+ – Usually linear scalability up to 25 fps@640x480Usually linear scalability up to 25 fps@640x480

• Can handle dynamic scenes [Part II]Can handle dynamic scenes [Part II]• OpenGL-like API [Part II]OpenGL-like API [Part II]

– Supports arbitrarily programmable ShadersSupports arbitrarily programmable Shaders

• Supports interactive global illumination [Part III]Supports interactive global illumination [Part III]


20

RTRT/OpenRT RTRT/OpenRT State of the Art PerformanceState of the Art Performance

Important: Much higher performance than EG’01:Important: Much higher performance than EG’01:• Obviously: CPUs got faster....Obviously: CPUs got faster....

– 2.5GHz Pentium IV vs. 866MHz Pentium III: 3x faster2.5GHz Pentium IV vs. 866MHz Pentium III: 3x faster

• Plus: Better compiler supportPlus: Better compiler support– Intel C Compiler: ~2x faster than gccIntel C Compiler: ~2x faster than gcc– ICC intrinsics: much better than hand-coded assemblerICC intrinsics: much better than hand-coded assembler

• Plus: Better kd tree construction algorithmsPlus: Better kd tree construction algorithms– Modified “surface area heuristic” algorithm [Havran]Modified “surface area heuristic” algorithm [Havran]Roughly 2x faster than originally used kd-treesRoughly 2x faster than originally used kd-trees

• Plus: Faster implementation...Plus: Faster implementation... Taken together: Taken together: Much Much faster then EG’01faster then EG’01


21

RTRT/OpenRT RTRT/OpenRT State of the Art Performance DataState of the Art Performance Data

State of the art performance dataState of the art performance data• On single PentiumIV 2.5GHzOn single PentiumIV 2.5GHz• In Frames per second at 1024x1024 pixelsIn Frames per second at 1024x1024 pixels

Up to 7 million rays per second on 2.5GHz laptop !Up to 7 million rays per second on 2.5GHz laptop !

RT&ShadingRT&Shading

SceneScene

SSESSE

no shd.no shd.

SSESSE

simple shd.simple shd.

No SSENo SSE

simple shd.simple shd.

ERW6 (static)ERW6 (static) 7.17.1 2.32.3 1.371.37

ERW6 (dynamic)ERW6 (dynamic) 4.84.8 1.971.97 1.061.06

Conf (static)Conf (static) 4.554.55 1.931.93 1.21.2

Conf (dynamic)Conf (dynamic) 2.942.94 1.61.6 0.820.82

Soda HallSoda Hall 4.124.12 1.81.8 1.0551.055


22

RTRT/OpenRT PerformanceRTRT/OpenRT PerformanceImportant LessonsImportant Lessons

• Raw ray tracing performance quite highRaw ray tracing performance quite high– Up to 7M rays/sec on 2.5 GHz notebookUp to 7M rays/sec on 2.5 GHz notebook

• Corresponds to 7fps@1024x1024Corresponds to 7fps@1024x1024

– Still up to 4 million rays/sec on 1.5Mtri modelStill up to 4 million rays/sec on 1.5Mtri model

• But: Hard to get peak performanceBut: Hard to get peak performance– Shading cost, coherent rays, ...Shading cost, coherent rays, ...

– But even ‘average’ performance quite high !But even ‘average’ performance quite high !

• Important: Today’s biggest bottleneck is Important: Today’s biggest bottleneck is shadingshading ! !– Even simple shading in ERW6: 7.1Mrps Even simple shading in ERW6: 7.1Mrps 2.3 Mrps 2.3 Mrps

– Future research: Have to look at faster shading !Future research: Have to look at faster shading !• SSE, Packet shading, ...SSE, Packet shading, ...


23

AgendaAgenda










Ray Tracing on GPUsRay Tracing on GPUs&&

Special Purpose Ray Tracing Hardware Special Purpose Ray Tracing Hardware – The SaarCOR Architecture -– The SaarCOR Architecture -


25

AgendaAgenda










Part IIPart II

Advanced IssuesAdvanced Issues


27

Advanced Ray Tracing IssuesAdvanced Ray Tracing Issues

Part I: Different ways to realize realtime ray tracingPart I: Different ways to realize realtime ray tracing• Can’t (yet) say which is best...Can’t (yet) say which is best...• But can say: “It But can say: “It isis coming” coming”

– Actually, it’s available already today...Actually, it’s available already today...


28

Advanced Ray Tracing IssuesAdvanced Ray Tracing Issues

Ray Tracing is coming Ray Tracing is coming New issues need to be solved: New issues need to be solved:• How to handle dynamic scenes ?How to handle dynamic scenes ?

– Need new algorithmsNeed new algorithms

• How can I use it ?How can I use it ? API !API !

• What new things can I do with it ?What new things can I do with it ?– Build and explore new applicationsBuild and explore new applications

• Important: Those issues have to be solved anyway...Important: Those issues have to be solved anyway...– No matter wether it’s GPU’s, CPU’s, or SaarCOR, ...No matter wether it’s GPU’s, CPU’s, or SaarCOR, ...

– All these issues are equally valid for all architectures!All these issues are equally valid for all architectures!

Advanced Ray Tracing Issues:Advanced Ray Tracing Issues:

Dynamic ScenesDynamic Scenes


30

Handling Dynamic ScenesHandling Dynamic Scenes

• Ray tracing already beneficial in static scenesRay tracing already beneficial in static scenes– Visualizing complex modelsVisualizing complex models

– Complex shadingComplex shading

– High quality walkthroughs (e.g. w/ global illumination)High quality walkthroughs (e.g. w/ global illumination)

– ........

But: “Static only” is a severe limitationBut: “Static only” is a severe limitation

Cannot really interact with the model !Cannot really interact with the model !

NeedNeed support for dynamically changing scenes… support for dynamically changing scenes…


31


• Fact: Ray Tracing Fact: Ray Tracing needs needs acceleration structureacceleration structure– Too costly to rebuild each frameToo costly to rebuild each frame

• Problem: Few research on this topic so far…Problem: Few research on this topic so far…– Just wasn’t interesting before interactive ray tracing…Just wasn’t interesting before interactive ray tracing…

– Previous work: Usually on special casesPrevious work: Usually on special cases• Utah ‘Hack’: Keep dynamic objects out of accel structure…Utah ‘Hack’: Keep dynamic objects out of accel structure…

• [Reinhard RW2001]: Incremental updates of Uniform Grid[Reinhard RW2001]: Incremental updates of Uniform Grid– Costly, not hierarchicalCostly, not hierarchical

• [Moeller, EG2001]: Only rigid-body animation[Moeller, EG2001]: Only rigid-body animation


32


• Different kinds of dynamic behavior Different kinds of dynamic behavior – Hierarchical, rigid-body motion vs. unstructured Hierarchical, rigid-body motion vs. unstructured

motionmotion

– Constrained unstructured motion (e.g. maximum Constrained unstructured motion (e.g. maximum displacement)displacement)

– All triangles animated vs. few triangles animatedAll triangles animated vs. few triangles animated

– Amortized over many or over few raysAmortized over many or over few rays

– ……

• Actually different problemsActually different problemsNeed specialized solutionsNeed specialized solutions


33


Our approach:Our approach:• Offer suite of different techniquesOffer suite of different techniques

– Hierarchical animation of whole objectsHierarchical animation of whole objects

– Fast reconstruction of objects for unstructured motion Fast reconstruction of objects for unstructured motion (with sacrifices in traversal speed)(with sacrifices in traversal speed)

– High-quality BSPs for often-used static objects High-quality BSPs for often-used static objects (with relatively long rebuild time)(with relatively long rebuild time)

• Allow to combine them in a hierarchical wayAllow to combine them in a hierarchical way API !API !


34


• Application groups geometry into ‘objects’Application groups geometry into ‘objects’– Similar to building display lists (Similar to building display lists (API)API)

– Each object has its own acceleration structureEach object has its own acceleration structure

• Support Unstructured Motion Support Unstructured Motion – Use different BSPs for different objectsUse different BSPs for different objects

• Cheap BSPs for unstructured motionCheap BSPs for unstructured motion

• High-quality BSPs for objects w/ rigid-body motionHigh-quality BSPs for objects w/ rigid-body motion

– Allow localized, fast rebuild of unstructured motionAllow localized, fast rebuild of unstructured motion

• But: Unstructured motion still costlyBut: Unstructured motion still costly– Too much data to send over network !!!Too much data to send over network !!!

– Tolerable for moderately complex objects (16k-64ktri)Tolerable for moderately complex objects (16k-64ktri)


35


• Objects are ‘instantiated’ into the sceneObjects are ‘instantiated’ into the scene– Just like ‘calling’ a display list (Just like ‘calling’ a display list ( API) API)– Each instance has a transformation attached to itEach instance has a transformation attached to it

• Hierarchical Anim: Inversely transform rays instead of objectsHierarchical Anim: Inversely transform rays instead of objectsChanging this transformation transforms whole objectChanging this transformation transforms whole object

– Instances are organized in additional hierarchy levelInstances are organized in additional hierarchy level• With its own acceleration structureWith its own acceleration structure• Only thisOnly this has to be rebuilt every frame has to be rebuilt every frame

• Side Effect: Instantiation is for free Side Effect: Instantiation is for free – Sunflowers: 26k instances of 36k-sunflower Sunflowers: 26k instances of 36k-sunflower

1 1 billionbillion triangles triangles


36

Handling Dynamic ScenesHandling Dynamic Scenes- Results- Results


37

Handling Dynamic ScenesHandling Dynamic Scenes- Results- Results

• TopLevel BSP reconstruction tolerableTopLevel BSP reconstruction tolerable– Some milliseconds even for a few thousand objectsSome milliseconds even for a few thousand objects– But definitely is not for free...But definitely is not for free...

• Hierarchical animation is cheapHierarchical animation is cheap• Unstructured motion still a problemUnstructured motion still a problem• But: Sufficient for most applications so farBut: Sufficient for most applications so far

– Used exclusively in all RTRT/OpenRT applicationsUsed exclusively in all RTRT/OpenRT applications

• Future work:Future work:– Better support for “immediate mode” renderingBetter support for “immediate mode” rendering– Port algorithms to SaarCOR architecturePort algorithms to SaarCOR architecture


The OpenRT APIThe OpenRT API


39

The need for an APIThe need for an API

• So far: All technical problems have been addressedSo far: All technical problems have been addressed– Fast ray tracing implementationsFast ray tracing implementations

– Shading, massive models, etc, are easy...Shading, massive models, etc, are easy...

– Dynamic scenesDynamic scenes

““Theoretically” we’re done...Theoretically” we’re done...

• Questions: How can I use it ?Questions: How can I use it ?Need an API !Need an API !


40

The need for an APIThe need for an API

• So far: All technical problems have been addressedSo far: All technical problems have been addressed– Fast ray tracing implementationsFast ray tracing implementations

– Shading, massive models, etc, is easy...Shading, massive models, etc, is easy...

– Dynamic scenesDynamic scenes

““Theoretically” we’re done...Theoretically” we’re done...

• Questions: How can I use it ?Questions: How can I use it ?Need an API !Need an API !

• Problem: No APIs for Problem: No APIs for interactive interactive RT availableRT available– OpenGL not suitable for ray tracingOpenGL not suitable for ray tracing

– Renderman/Rayshade/Povray Renderman/Rayshade/Povray inherentlyinherently offline … offline …


41

The OpenRT Interactive Ray The OpenRT Interactive Ray Tracing APITracing API

Goals for an Interactive Ray Tracing API:Goals for an Interactive Ray Tracing API:• As easy to learn and use as (basic) OpenGLAs easy to learn and use as (basic) OpenGL

– Leverage existing OpenGL experienceLeverage existing OpenGL experience

• As powerful in shading as RenderManAs powerful in shading as RenderMan

OpenRT: Combine the best of bothOpenRT: Combine the best of both• Application API much like OpenGL/GLUTApplication API much like OpenGL/GLUT

– Plus necessary extensions for Ray Tracing Plus necessary extensions for Ray Tracing • Shaders, Objects, ...Shaders, Objects, ...

• Shader API like RenderManShader API like RenderMan


42


• Application API very OpenGL-likeApplication API very OpenGL-like– Geometry/Transformations just like OpenGLGeometry/Transformations just like OpenGL


43



rtMatrixMode(RT_MODELVIEW);rtMatrixMode(RT_MODELVIEW);

rtPushMatrix();rtPushMatrix();

rtRotatef(1,0,0,M_PI/2);rtRotatef(1,0,0,M_PI/2);

rtScalef(1,2,1);rtScalef(1,2,1);

rtBegin(RT_TRIANGLES);rtBegin(RT_TRIANGLES);

rtColor3f(0,1,0)rtColor3f(0,1,0)

rtNormal3f(1,1,1);rtNormal3f(1,1,1);

rtVertex3f(0,0,0);rtVertex3f(0,0,0);

......

rtEnd();rtEnd();

rtPopMatrix();rtPopMatrix();


44



– Geometry ObjectsGeometry Objects• Just like Display Lists (except: no side effects)Just like Display Lists (except: no side effects)

• rtNewObjects(), rtBegin/EndObject(), rtInstantiate(),…rtNewObjects(), rtBegin/EndObject(), rtInstantiate(),…

– Support fully programmable shader objectsSupport fully programmable shader objects• Surface, Light, and Pixel Shaders, ‘Renderer Object’Surface, Light, and Pixel Shaders, ‘Renderer Object’

• Difference to GL: Different Rendering SemanticsDifference to GL: Different Rendering Semantics– ““Retained-Mode” vs. “Immediate Mode”Retained-Mode” vs. “Immediate Mode”

• Changes take effect only at end of frame !Changes take effect only at end of frame !

• No “state machine” modelNo “state machine” model


45


Shader APIShader API• Shaders loaded from DLL’s/.so’sShaders loaded from DLL’s/.so’s• Declare Shader Declare Shader

– E.g. shader parameters etcE.g. shader parameters etc

• Write callback-functionsWrite callback-functions– ‘‘Shade()’ (for surfaces), ‘Illuminate()’ (for lights), …Shade()’ (for surfaces), ‘Illuminate()’ (for lights), …

• Access scene data with RenderMan like APIAccess scene data with RenderMan like API– Geometry: rtsShadingNormal(), …Geometry: rtsShadingNormal(), …– Lights: rtsIlluminate(), rtsOccluded(), ...Lights: rtsIlluminate(), rtsOccluded(), ...

• Can shoot arbitrary secondary raysCan shoot arbitrary secondary rays– rtsTrace(…)rtsTrace(…)


46


• OpenRT SummaryOpenRT Summary– Abstracts from underlying engineAbstracts from underlying engine

• Completely hides kernel implementationCompletely hides kernel implementation• Hides distribution infrastructureHides distribution infrastructure

– Low-level API rather than scene graph APILow-level API rather than scene graph API– OpenGL like application interfaceOpenGL like application interface– RenderMan like flexible shader APIRenderMan like flexible shader API

• State of the ArtState of the Art– OpenRT V1.0 is out...OpenRT V1.0 is out...– Used exclusively to drive Saarland RTRT/OpenRTUsed exclusively to drive Saarland RTRT/OpenRT

• Used for all shown applications (except GPUs and SaarCOR)Used for all shown applications (except GPUs and SaarCOR)

– Ongoing work: Ongoing work: SaarCOR@OpenRTSaarCOR@OpenRT

mailto:SaarCOR@OpenRT


ApplicationsApplications


48

Advanced Ray Tracing Issues: Advanced Ray Tracing Issues: ApplicationsApplications

• So far: All ingredients for a complete rendering So far: All ingredients for a complete rendering engine are available !engine are available !– Technology (Fast ray tracing, dynamic scenes)Technology (Fast ray tracing, dynamic scenes)

– API to use itAPI to use it

• Availability of a new technology opens up potential Availability of a new technology opens up potential for new applicationsfor new applications– Have to explore what’s possible and what not...Have to explore what’s possible and what not...

– Many applications might not even have been thought of !Many applications might not even have been thought of !

• Here: Only brief overview of today’s possibilitiesHere: Only brief overview of today’s possibilities


49

Application I: Application I: Classical Ray TracingClassical Ray Tracing

• Classical Ray Tracing with Complex Shading EffectsClassical Ray Tracing with Complex Shading Effects– Local: Volume object, Lightfield, bumpmapping, procedural shaders....Local: Volume object, Lightfield, bumpmapping, procedural shaders....

– Global Effects: Shadows, Reflections, ...Global Effects: Shadows, Reflections, ... All effects work together (e.g. shadows from lightfield & volume obj.)All effects work together (e.g. shadows from lightfield & volume obj.)

““Office” w/ complex shading effectsOffice” w/ complex shading effects


50

Application I: Application I: Classical Ray TracingClassical Ray Tracing

• Classical Ray Tracing for physically correct simulation Classical Ray Tracing for physically correct simulation – Industrial project w/ Hella corpIndustrial project w/ Hella corp

– Simulate reflection&refraction behavior of car headlightSimulate reflection&refraction behavior of car headlight

– Up to 25 levels of reflection/refraction, complex glass effectsUp to 25 levels of reflection/refraction, complex glass effects

““Car Headlight” – Physically correct glass simulationCar Headlight” – Physically correct glass simulation


51

Application II: Application II: Visualization of Complex ModelsVisualization of Complex Models

• Interactive Visualization of Massively Complex ModelsInteractive Visualization of Massively Complex Models– ““Power Plant”: 3x12.5million individual trianglesPower Plant”: 3x12.5million individual triangles

– Simple shading: up to 25 fps ...Simple shading: up to 25 fps ...

– ... on only 2 or 3 clients... on only 2 or 3 clients

– Highly complex geometryHighly complex geometry


52

Application II: Application II: Visualization of Complex ModelsVisualization of Complex Models

• Oliver Deussen’s “Sunflowers” SceneOliver Deussen’s “Sunflowers” Scene– Including shadows from sun and transparency through leavesIncluding shadows from sun and transparency through leaves

– One One billionbillion triangles triangles

– 7-8 fps @ 24 nodes7-8 fps @ 24 nodes


53

Applications III:Applications III:Interactive Lighting SimulationInteractive Lighting Simulation

• Lighting SimulationLighting Simulation– Physically correct simulation of global light propagationPhysically correct simulation of global light propagation

• Including indirect lighting effectsIncluding indirect lighting effects

– Fully recomputed each frame Fully recomputed each frame • Can move objects, change materials ...Can move objects, change materials ...

– No details here [Part III]No details here [Part III]


54

AgendaAgenda










Part IIIPart III

Using Realtime Ray Tracing Using Realtime Ray Tracing for for

Interactive Global IlluminationInteractive Global Illumination


56

Using Realtime Ray Tracing Using Realtime Ray Tracing for Interactive G.I.for Interactive G.I.

• Fast Global Illumination is an old problem...Fast Global Illumination is an old problem...– Lots of previous workLots of previous work

• Radiosity+OpenGL, Render Cache, Shading Cache, etc.Radiosity+OpenGL, Render Cache, Shading Cache, etc.

• Usually limited due to lacking performance to (re)compute all Usually limited due to lacking performance to (re)compute all effects every frame...effects every frame...

– Need to trace Need to trace manymany rays rays

– Ray tracing historically costlyRay tracing historically costlyUsually offlineUsually offline

• Realtime ray tracing: Tracing rays now much faster ...Realtime ray tracing: Tracing rays now much faster ... Should theoretically allow for interactive global illuminationShould theoretically allow for interactive global illumination


57

Interactive Global Interactive Global IlluminationIllumination

Requirements for Interactive GIRequirements for Interactive GI• Low communication overheadLow communication overhead

– Low bandwidth between clientsLow bandwidth between clients

– High latency due to commodity networkHigh latency due to commodity network

– No shared memoryNo shared memoryComplicates/Forbids use of global data structuresComplicates/Forbids use of global data structures

• Low sampling rate for interactive useLow sampling rate for interactive use– Limited to 40-70 rays (not paths !) per pixelLimited to 40-70 rays (not paths !) per pixel

• New: Avoid temporal noise and artifactsNew: Avoid temporal noise and artifacts– Previously not an issue with off-line methodsPreviously not an issue with off-line methods


58

Interactive GlobalInteractive GlobalIlluminationIllumination

• Traditional algorithms: do not scaleTraditional algorithms: do not scale– RadiosityRadiosity

• Global data structure for radiosity valuesGlobal data structure for radiosity values

– Photon MapPhoton Map• Global kd-treeGlobal kd-tree

• Reshooting photons & rebuilding treeReshooting photons & rebuilding tree

Don’t easily map to a RTRT architecture... Don’t easily map to a RTRT architecture...

• MC: would scale but is too noisyMC: would scale but is too noisy– Especially for animations and interactive use !Especially for animations and interactive use !


59

““Instant Global Instant Global Illumination” (IGI)Illumination” (IGI)

Our Approach (w/ Keller and Kollig, Kaiserslautern)Our Approach (w/ Keller and Kollig, Kaiserslautern)• Instant Radiosity TechniqueInstant Radiosity Technique

– Smooth lighting approximation using “virtual point lights”Smooth lighting approximation using “virtual point lights”

• Computed with Ray TracingComputed with Ray Tracing– Adds specular effects (reflections/refraction)Adds specular effects (reflections/refraction)– Many coherent shadow rays Many coherent shadow rays Very efficient ! Very efficient !

• Simplified Photon-Map for CausticsSimplified Photon-Map for Caustics• Interleaved Sampling (ILS)Interleaved Sampling (ILS)

– Improves overall quality and visual appearanceImproves overall quality and visual appearance

• Discontinuity BufferDiscontinuity Buffer– Removes ILS artifacts in image spaceRemoves ILS artifacts in image space


60

IGI:IGI:Instant RadiosityInstant Radiosity

• Step 1: Trace few particles from lightsStep 1: Trace few particles from lights– Place ‘virtual point lights’ (VPLs) at hit pointsPlace ‘virtual point lights’ (VPLs) at hit points

– RQMC techniques (RQMC techniques ( see Keller’s QMC tutorial notes !!!) see Keller’s QMC tutorial notes !!!)

• Step 2: Illuminate scene from VPLsStep 2: Illuminate scene from VPLs– Per-pixel computationPer-pixel computation

– Supports glossy effects Supports glossy effects

• Converges quickly Converges quickly for many VPLsfor many VPLs

• Hard shadow artifacts Hard shadow artifacts for few VPLsfor few VPLs


61

IGI:IGI:Interleaved Sampling Interleaved Sampling

• Use multiple VPL sets per imageUse multiple VPL sets per image– Different VPLs for each pixel in a 3x3 gridDifferent VPLs for each pixel in a 3x3 grid

– Breaks up hard shadow artifactsBreaks up hard shadow artifacts

– Increased number of VPLs per imageIncreased number of VPLs per image

• Still same costStill same cost• Still high coherenceStill high coherence

– Ray bundle to Ray bundle to same set of VPLssame set of VPLs

• Strong AliasingStrong Aliasing– ILS-Grid is clearly visibleILS-Grid is clearly visible


62

IGI:IGI:Discontinuity BufferDiscontinuity Buffer

• Filter Irradiance valuesFilter Irradiance values– Combine irradiance from neighboring pixelsCombine irradiance from neighboring pixels

– 3x3 grid: using all VPLs for each pixel3x3 grid: using all VPLs for each pixel

– Smoothing of ILS-artifacts in image space Smoothing of ILS-artifacts in image space

• Adaptive filterAdaptive filter– Avoids filtering acrossAvoids filtering across

discontinuities (baseddiscontinuities (based

on normals&distance)on normals&distance)

– Some Filter artifacts Some Filter artifacts


63

IGI:IGI:SummarySummary

• Base ingredientBase ingredient– Instant radiosity + fast RT Instant radiosity + fast RT

• Interleaved samplingInterleaved sampling– Better lighting, coherenceBetter lighting, coherence

• Filtering: Remove artifactsFiltering: Remove artifacts– Smoothing in image spaceSmoothing in image space

• Important: All images same #VPLSImportant: All images same #VPLS– ~Same performance~Same performance

• Results: Very good qualityResults: Very good quality– Starting at ~20 rays/pixel !Starting at ~20 rays/pixel ! sam

e #V

PL

s =

~sa

me

fram

e ra

te !

sam

e #V

PL

s =

~sa

me

fram

e ra

te !


64

““IGI2” (2003)IGI2” (2003)Scalable ComputationScalable Computation

• Original implementation [RW’02] had several problemsOriginal implementation [RW’02] had several problems– Scalability problems (server bottleneck,...)Scalability problems (server bottleneck,...)

– Performance problems Performance problems (at best 5fps, slow execution per CPU,...)(at best 5fps, slow execution per CPU,...)

– Quality problems (fixed lighting model, no antialiasing, ...)Quality problems (fixed lighting model, no antialiasing, ...)

““IGI2” : New, scalable implementationIGI2” : New, scalable implementation• Still same basic ideas (IR+RT+ILS+DiscoBuffer)Still same basic ideas (IR+RT+ILS+DiscoBuffer)• But: New and Improved IdeasBut: New and Improved Ideas

– No details: Go see the talk tomorrow !No details: Go see the talk tomorrow !


65

““IGI2” (2003)IGI2” (2003)Scalable ComputationScalable Computation

New and Improved Ideas [Benthin EG’03, New and Improved Ideas [Benthin EG’03, this Friday !this Friday !]]• Better Image QualityBetter Image Quality

– Efficient anti-aliasingEfficient anti-aliasing• 4x-anti-aliasing for 10-20% overhead4x-anti-aliasing for 10-20% overhead

– Programmable BRDF-shadersProgrammable BRDF-shaders

• Much faster Much faster – Complete re-implementationComplete re-implementation– SSE packet tracing of both primary and shadow raysSSE packet tracing of both primary and shadow rays– Parallel shader interface supporting SSEParallel shader interface supporting SSE

• Removed scalability bottleneckRemoved scalability bottleneck– Discontinuity filtering on the clientsDiscontinuity filtering on the clients


66

IGI2 – Scalability GraphIGI2 – Scalability Graph

Scalabilitygraph ondual processorAthlon 1800+clients(up to 48 CPUs)


67

IGI2 Results:IGI2 Results:Efficient Anti-AliasingEfficient Anti-Aliasing

Single sample per pixel at 4 fps(detail view of the conference scene)

Four samples per pixel at 3.6 fps Cost: Only 10% overhead!


68

IGI2 Results:IGI2 Results:Performance & Image QualityPerformance & Image Quality

Old Algorithm New Scalable Algorithm

Conference Scene on 8 clients at same performance levelThe new system would render at 12 fps at the samequality as the old system


69

IGI2 Results:IGI2 Results:Performance & Image QualityPerformance & Image Quality

Old Algorithm New Scalable Algorithm

Conference Scene on 8 clients at same performance levelThe new system would render at 12 fps at the samequality as the old system


70

IGI2 – State of the ArtIGI2 – State of the Art


71



72



73

Instant Global IlluminationInstant Global IlluminationSummarySummary

• Instant Global IlluminationInstant Global Illumination– Instant RadiosityInstant Radiosity– Interleaved SamplingInterleaved Sampling– Discontinuity BufferingDiscontinuity Buffering– All realized with fast ray tracingAll realized with fast ray tracing

• Improved Implementation [EG’03 – This Friday !]Improved Implementation [EG’03 – This Friday !]– Efficient antialiasingEfficient antialiasing– Programmable BRDF shadersProgrammable BRDF shaders– Improved scalability and performanceImproved scalability and performance

High performance interactive global illuminationHigh performance interactive global illumination– Can compute most important effects at up to 25fpsCan compute most important effects at up to 25fps


74

IGI - VideoIGI - Video

IGI - VideoIGI - Video

For more information, also seeFor more information, also see

http://www.OpenRT.dehttp://www.OpenRT.de


75

IGI – Remaining IssuesIGI – Remaining Issues

Remaining ProblemsRemaining Problems• Many light sourcesMany light sources

– Problem: Too few VPLs for many different lightsProblem: Too few VPLs for many different lights

– Preliminary approach sketched at EGSR’03Preliminary approach sketched at EGSR’03• Combine contribution estimation and importance samplingCombine contribution estimation and importance sampling

• Temporal aliasingTemporal aliasing– Especially during changes to the scene...Especially during changes to the scene...

• Glossy reflectionsGlossy reflections• CausticsCaustics


76

AgendaAgenda











77

State of the Art in RTRT&IGIState of the Art in RTRT&IGISummarySummary

• Today: Many approaches to realize RTRTToday: Many approaches to realize RTRT– Software systems, GPUs, Special-purpose HWSoftware systems, GPUs, Special-purpose HW

• Talked about Advanced IssuesTalked about Advanced Issues– Dynamic Scenes, API IssuesDynamic Scenes, API Issues

• New applicationsNew applications– Complex Models, high-quality rendering, Interactive Complex Models, high-quality rendering, Interactive

Global Illumination, ...Global Illumination, ...

Interactive RT has progressed to an advanced stateInteractive RT has progressed to an advanced state• But: Still lots of potential for future researchBut: Still lots of potential for future research

– Better support for dynamic scenesBetter support for dynamic scenes– Better & faster shadingBetter & faster shading


78

State of the Art in RTRT&IGIState of the Art in RTRT&IGIConclusionConclusion

• Realtime Ray Tracing Realtime Ray Tracing is is comingcoming– It will soon be realtime on the desktop...It will soon be realtime on the desktop...

• faster CPUs/GPUs, faster CPUs/GPUs,

• better implementationsbetter implementations

• It It will will change computer graphicschange computer graphics– By enabling completely new applicationsBy enabling completely new applications

• Actually, it is already available todayActually, it is already available today– At least at limited resolutions and frame rate...At least at limited resolutions and frame rate...

• Go explore your own applications...Go explore your own applications...– Go see what RTRT can do for you...Go see what RTRT can do for you...


79

State of the Art in RTRT&IGIState of the Art in RTRT&IGIQuestions ?Questions ?

Questions ?Questions ?

state of the art in realtime ray tracing & interactive global illumination

Documents

compute color of ray

global effects

interactive graphics

saarcor purcellpart

local effects

effectsall effects

highquality graphics

new rays