Anton KaplanyanAnton Kaplanyan11 Carsten DachsbacherCarsten Dachsbacher22

11Crytek GmbH Crytek GmbH 22VISUS / University StuttgartVISUS / University Stuttgart

ACM SIGGRAPH Symposium on Interactive 3D Graphics and GamesACM SIGGRAPH Symposium on Interactive 3D Graphics and Games21 February, 2010, 21 February, 2010, WashingtonWashington, USA, USA

Motivation

ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games 2010, Washington 2

Previous work

ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games 2010, Washington 3

Irradiance volumes Greger et al. 1997

SH Irradiance VolumesTatarchuk 2004

PRT: Spherical HarmonicsSloan et al. 2004

Spherical proxies with SH ExponentiationZhong et al. 2007

Image-Space Photon MappingMcGuire and Luebke 2009

Multi-resolution Splatting Nichols and Wyman 2009

Previous work, continued

ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games 2010, Washington 4

Instant radiosityKeller 1997

Many-lights approachWalter et al. 2005Hasan et al. 2007Chevlak-Postavak et al. 2008

VPL visibilityLaine et al. 2007Ritschel et al. 2008

Previous work, continued

ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games 2010, Washington 5

Disk-based Color BleedingBunell 2005Christensen 2008

Finite Element: AntiradianceDachsbacher et al. 2007

MicrorenderingRitschel et al. 2010

All techniques above have one or more of the following limitations:•Precomputed or redundant data (problems with dynamic and/or editable scenes)•Not suitable for game production performance-wiseMost of dynamic techniques are without indirect visibility

Previous work, lattice methods

ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games 2010, Washington 6

Light Propagation Maps Fattal 2009

Lattice-Boltzmann Lighting Geist et al. 2004

Lattice-Based Volumetric GlobalIlluminationQiu et al. 2007

Basic idea

ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games 2010, Washington 7

Basic idea

ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games 2010, Washington 8

Basic idea

ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games 2010, Washington 9

Propagation demo

Overview

ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games 2010, Washington 11

Light Propagation Volumes

• Use many-lights approach to capture sources of indirect lighting

• Sample directly lit surfaces and initialize 3D grid

• Represent directional distribution with Spherical Harmonics– Inspired by SH Irradiance Volumes [Tatarchuk04]

• Iterative, local propagation: cell-to-cell

ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games 2010, Washington 12

Secondary Light Sources

ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games 2010, Washington 13

Secondary Light Sources

ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games 2010, Washington 14

Reflective shadow mapsFlux NormalDepth

Injection

ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games 2010, Washington 15

Pipeline

ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games 2010, Washington 16

Reflective shadow maps Radiance volume gathering

VPL

VPL

VPL

Discretize initial VPL distribution by the regular grid and SH

A set of regularly sampled VPLs of the scene from light position

?

Light injection into the volume• Every element of

Reflective Shadow Map is a secondary lights

• Render as a point primitive into 3D grid– Represent flux in Spherical

Harmonics• Accumulate all VPLs into the grid• The 3D grid is initialized with

initial reflected light in the endACM SIGGRAPH Symposium on Interactive 3D Graphics and Games 2010, Washington 17

n

Light Propagation

ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games 2010, Washington 18

Pipeline

ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games 2010, Washington 19

Reflective shadow maps Radiance volume gathering

VPL

VPL

VPL

Discretize initial VPL distribution by the regular grid and SH

Iterative propagation

Propagate light iteratively going from one cell to another

A set of regularly sampled VPLs of the scene from light position

Iterative Light Propagation• Local cell-to-cell propagation

across the 3D grid– Iterate till the light travels through

the entire volume– Similar to SH Discrete Ordinate

Method (used for participating media illumination)

– Number of iterations depend on the resolution of the grid

ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games 2010, Washington 20

The propagation iteration• 6 axial directions of propagation• Use contour faces as a

propagation wave front• Integrate source

intensity by the solid angle to get incoming flux for the face f

The propagation iteration• Use more than 6

directions– Only 6 direct neighbors– Compute light

propagation to eachface of neighbors’ cells

– 30 virtual directions – SHDOM: 27 neighbor

cells = 27 directions– good trade-off of

memory bandwidth vs “ray effect”• “Ray effect” - light propagates in a set of fictitious directions

4 directions of propagation

8 directions of propagation

Reprojection• Acquire the incident flux through

the receiving face• Create a new point light in the

center of receiving cell– Oriented towards the face– Causing exactly the same flux as the face received

• Generate clamped cosine lobe in SH basis similar to injection stage

• Accumulate the resulting SH coefficients into the destination cell for next iteration

Scene rendering

ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games 2010, Washington 24

Rendering

• Look-up grid with trilinear interpolation• Evaluate the irradiance with cosine lobe of

surface’s normal• Apply dampening factor – Compute directional derivative towards normal– Dampen based on derivative deviation from the

intensity distribution direction

ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games 2010, Washington 25

Results of indirect illumination

Cascaded Light Propagation Volumes

ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games 2010, Washington 27

• Motivation: memory and bandwidth cost is o(N^3) for increase of LPV grid– Impossible to support large scenes

• Idea: use multiple nested grids to refine resolution hierarchically– Do not consider small objects for

large sparse grids

• Transfer propagated lighting fromnested grid to the parent grid

• Illuminate scene similarly tocascaded shadow maps

• Reduces the number of iterations sufficient per cascade

Cascaded Indirect Illumination

ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games 2010, Washington 28

1 cascade1 cascade

3 cascades3 cascades

Fuzzy Secondary Occlusion

• Introduce a “fuzzyblocking” between cells

• Use another grid for blocking• Occlusion is view-dependent• Projected size of an occluder

is a cosine lobe

ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games 2010, Washington 29

blocker

Fuzzy Secondary Occlusion

• Represent it as SH • Store into occlusion grid• Sample surfaces using

rasterization – Possibly multiple views

• Very similar to light injection

• Interpolate blockinglinearly in between cells

ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games 2010, Washington 30

Camera view

Light view

Scene

Fuzzy Secondary Occlusion

ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games 2010, Washington

W/o secondary occlusionW/o secondary occlusion

With secondary occlusionWith secondary occlusion

Multiple Bounces• Idea: use information

from occlusion gridto compute multipleindirect reflections

• Reflect light duringeach propagationiteration

• Avoid self-illumination by injecting reflectedlight at safety-distance

ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games 2010, Washington 32

Glossy Reflections

ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games 2010, Washington 33

• Idea: Compute incident light from reflection direction by marching through LPV grid

• Go few steps back in propagation time to reduce light smearing

• 4 cells is sufficient for moderately glossy objects

• Lookups into multiple cells prevent discontinuitiesin glossy reflections

Indirect lighting in isotropic participating media

• Ray march through the LPV • Accumulate inscattered light• Limited to single-scattering • Step through the whole

grid along viewdirection – Back to front

accumulation

ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games 2010, Washington 34

Timings (Crytek Sponza)

ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games 2010, Washington 35

Depends on scene complexity

Depends on image size (1280x720)

8 iterations

32^3 grid size

Results

ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games 2010, Washington 36

Reference, 42 minReference, 42 min LPV, 78 fps @GTX285LPV, 78 fps @GTX285

Reference PBRT, 45 minReference PBRT, 45 min LPV, 60 fps @GTX285LPV, 60 fps @GTX285

Limitations of the method• Only diffuse inter-reflections• Sparse spatial and

low-frequency angular approximations– Light diffusion: light transport

smears in all directions– Spatial discretization: visible

for occlusion and very coarsegrids

• Incomplete information for secondary occlusionACM SIGGRAPH Symposium on Interactive 3D Graphics and Games 2010, Washington 37

Conclusion• Full-dynamic: scene, view, lighting changes• Real-time: GPU- and consoles- friendly• Production-eligible (simple tweaking)• Highly scalable– proportionally to quality

• Stable, flicker-free• Supports complex geometry (e.g. foliage)

ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games 2010, Washington 38

Video

ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games 2010, Washington 39

THANK YOU FOR YOUR ATTENTION

See the paper for more details

ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games 2010, Washington 40

We’d like to thank: Crytek and especially the CEO Cevat Yerli for giving us an opportunity to make this researchThe whole Crytek R&D department and artists for help providedMany people across the industry and research community for interesting discussions and provided feedbacks

Backup slide: Small details

• Stability of the solution– RSM one-texel snapping– One-cell snapping for LPVs– Temporal SSAA with reprojection for RSM injection

• Self-illumination and light bleeding– Half-cell VPL shifting to normal direction during

RSM injection– Directional derivative in normal direction to

compute a dampening factor

ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games 2010, Washington 41

Backup slide: Console optimizations

• For both consoles– Store everything in signed QUVW8 format, [-1;1] with

scaling factor– Use h/w 3D textures and trilinear filtering

• Xbox 360– Unwrap RT vertically to avoid bank conflicts during

injection – Use API bug work-around to resolve into a 3D slice

• PlayStation 3– Use memory aliasing for render into 3D texture– Use 2x MSAA aliasing to reduce pixel work twice

ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games 2010, Washington 42

Backup slide: Console optimizations II

• Render Reflective Shadow Map Usually 128 x 128 is ok

• Inject each pixel into unwrapped LPV with a swarm of points 16384 points in one DIP Use vertex texture fetch on X360 Use R2VB on PlayStation 3

• Multi-layered unwrapping to avoid bank conflicts during RSM injection on Xbox 360

• All together: 3,0 ms on X360/PS3

Backup slide: Massive LightingRender sliced unwrapped light box

into LPV (spatial overdraw vs screen-space, maximum 1024x32 pixels)

Convert light’s radiant intensity into SHShadows are not supported

Coverage in unwrapped render target

Light in the Light Propagation Volume