high-quality depth of field using line samples
TRANSCRIPT
![Page 1: High-Quality Depth of Field using Line Samples](https://reader033.vdocuments.mx/reader033/viewer/2022051522/5896eda11a28ab95028bbfa3/html5/thumbnails/1.jpg)
High-Quality Parallel Depth-of-Field Using Line Samples
Stanley Tzeng, Anjul Patney, Andrew Davidson, Mohamed S. Ebeida, Scott A. Mitchell, John D. Owens
University of California, Davis Sandia National Laboratories
1
![Page 2: High-Quality Depth of Field using Line Samples](https://reader033.vdocuments.mx/reader033/viewer/2022051522/5896eda11a28ab95028bbfa3/html5/thumbnails/2.jpg)
Depth of Field is Beautiful
Pixar’s Toy Story 3
![Page 3: High-Quality Depth of Field using Line Samples](https://reader033.vdocuments.mx/reader033/viewer/2022051522/5896eda11a28ab95028bbfa3/html5/thumbnails/3.jpg)
The Problem
4
Good noise-reducing Depth-of-Field effects
takes a lot of point samples!
![Page 4: High-Quality Depth of Field using Line Samples](https://reader033.vdocuments.mx/reader033/viewer/2022051522/5896eda11a28ab95028bbfa3/html5/thumbnails/4.jpg)
The Problem
5
Good noise-reducing Depth-of-Field effects
takes a lot of point samples!
![Page 5: High-Quality Depth of Field using Line Samples](https://reader033.vdocuments.mx/reader033/viewer/2022051522/5896eda11a28ab95028bbfa3/html5/thumbnails/5.jpg)
The Problem
6
Good noise-reducing Depth-of-Field effects
takes a lot of point samples!
![Page 6: High-Quality Depth of Field using Line Samples](https://reader033.vdocuments.mx/reader033/viewer/2022051522/5896eda11a28ab95028bbfa3/html5/thumbnails/6.jpg)
The Problem
7
Good noise-reducing Depth-of-Field effects
takes a lot of point samples!
![Page 7: High-Quality Depth of Field using Line Samples](https://reader033.vdocuments.mx/reader033/viewer/2022051522/5896eda11a28ab95028bbfa3/html5/thumbnails/7.jpg)
This Talk
• Instead of point samples, use line samples!
– Heavier compute per sample, but need fewer samples for good results
• A tiled line sampling renderer for graphics hardware
– How to make line sampling work within tight memory constraints?
![Page 8: High-Quality Depth of Field using Line Samples](https://reader033.vdocuments.mx/reader033/viewer/2022051522/5896eda11a28ab95028bbfa3/html5/thumbnails/8.jpg)
Line Samples
A dimensional extension of point samples
Gribel et al. : High-Quality Spatio-Temporal Rendering using Semi-Analytical Visibility
Jones and Perry: Antialiasing with Line Samples
9
![Page 9: High-Quality Depth of Field using Line Samples](https://reader033.vdocuments.mx/reader033/viewer/2022051522/5896eda11a28ab95028bbfa3/html5/thumbnails/9.jpg)
Point Samples in a pixel
10
Scene primitive
Pixel
![Page 10: High-Quality Depth of Field using Line Samples](https://reader033.vdocuments.mx/reader033/viewer/2022051522/5896eda11a28ab95028bbfa3/html5/thumbnails/10.jpg)
Point Samples in a pixel
11
Pixel
Shoot samples
![Page 11: High-Quality Depth of Field using Line Samples](https://reader033.vdocuments.mx/reader033/viewer/2022051522/5896eda11a28ab95028bbfa3/html5/thumbnails/11.jpg)
Point Samples in a pixel
12
Pixel
Intersect &
Generate colored samples
![Page 12: High-Quality Depth of Field using Line Samples](https://reader033.vdocuments.mx/reader033/viewer/2022051522/5896eda11a28ab95028bbfa3/html5/thumbnails/12.jpg)
Point Samples in a pixel
13
Pixel
Composite
![Page 13: High-Quality Depth of Field using Line Samples](https://reader033.vdocuments.mx/reader033/viewer/2022051522/5896eda11a28ab95028bbfa3/html5/thumbnails/13.jpg)
Line Samples in a pixel
14
Pixel
![Page 14: High-Quality Depth of Field using Line Samples](https://reader033.vdocuments.mx/reader033/viewer/2022051522/5896eda11a28ab95028bbfa3/html5/thumbnails/14.jpg)
Line Samples in a pixel
15
Pixel
Shoot samples
![Page 15: High-Quality Depth of Field using Line Samples](https://reader033.vdocuments.mx/reader033/viewer/2022051522/5896eda11a28ab95028bbfa3/html5/thumbnails/15.jpg)
Line Samples in a pixel
16
Pixel
Intersect &
Generate colored line segments
![Page 16: High-Quality Depth of Field using Line Samples](https://reader033.vdocuments.mx/reader033/viewer/2022051522/5896eda11a28ab95028bbfa3/html5/thumbnails/16.jpg)
Line Samples in a pixel
17
Pixel
Composite
![Page 17: High-Quality Depth of Field using Line Samples](https://reader033.vdocuments.mx/reader033/viewer/2022051522/5896eda11a28ab95028bbfa3/html5/thumbnails/17.jpg)
Side Note
18
Pixel
Samples can be in any arbitrary orientation.
![Page 18: High-Quality Depth of Field using Line Samples](https://reader033.vdocuments.mx/reader033/viewer/2022051522/5896eda11a28ab95028bbfa3/html5/thumbnails/18.jpg)
Depth of Field Sampling
• Point Sampling: Generate 4D point samples in (x,y,u,v) space.
– Point on screen (x,y) +
– Point on lens (u,v)
• Line Sampling: Fix a point in screen space (x,y), sample along the lens in (u,v) space.
![Page 19: High-Quality Depth of Field using Line Samples](https://reader033.vdocuments.mx/reader033/viewer/2022051522/5896eda11a28ab95028bbfa3/html5/thumbnails/19.jpg)
How to Sample Along the Lens?
• Good area coverage • Uneven line lengths
Grid Pinwheel
• Even Line Lengths • Bias towards center
![Page 20: High-Quality Depth of Field using Line Samples](https://reader033.vdocuments.mx/reader033/viewer/2022051522/5896eda11a28ab95028bbfa3/html5/thumbnails/20.jpg)
How to Sample Along the Lens? Pinwheel
• Even Line Lengths • Bias towards center
Reweigh the samples obtained
![Page 21: High-Quality Depth of Field using Line Samples](https://reader033.vdocuments.mx/reader033/viewer/2022051522/5896eda11a28ab95028bbfa3/html5/thumbnails/21.jpg)
Line Samples for DoF
u
v
24
Step 1: Project scene geometry from
screen space to lens space.
y
x
![Page 22: High-Quality Depth of Field using Line Samples](https://reader033.vdocuments.mx/reader033/viewer/2022051522/5896eda11a28ab95028bbfa3/html5/thumbnails/22.jpg)
Line Samples for DoF
u
v
25
Step 2: Sample the lens using a
wagonwheel pattern
y
x
![Page 23: High-Quality Depth of Field using Line Samples](https://reader033.vdocuments.mx/reader033/viewer/2022051522/5896eda11a28ab95028bbfa3/html5/thumbnails/23.jpg)
Line Samples for DoF
u
v
26
Step 3: Record intersected segments
y
x
![Page 24: High-Quality Depth of Field using Line Samples](https://reader033.vdocuments.mx/reader033/viewer/2022051522/5896eda11a28ab95028bbfa3/html5/thumbnails/24.jpg)
Line Samples for DoF
u
v
27
Step 4: Composite segments to pixel
color
y
x
![Page 25: High-Quality Depth of Field using Line Samples](https://reader033.vdocuments.mx/reader033/viewer/2022051522/5896eda11a28ab95028bbfa3/html5/thumbnails/25.jpg)
Line Samples using DoF
28
![Page 26: High-Quality Depth of Field using Line Samples](https://reader033.vdocuments.mx/reader033/viewer/2022051522/5896eda11a28ab95028bbfa3/html5/thumbnails/26.jpg)
Line Samples using DoF
29
![Page 27: High-Quality Depth of Field using Line Samples](https://reader033.vdocuments.mx/reader033/viewer/2022051522/5896eda11a28ab95028bbfa3/html5/thumbnails/27.jpg)
Line Samples using DoF
30
![Page 28: High-Quality Depth of Field using Line Samples](https://reader033.vdocuments.mx/reader033/viewer/2022051522/5896eda11a28ab95028bbfa3/html5/thumbnails/28.jpg)
Line Samples using DoF
31
![Page 29: High-Quality Depth of Field using Line Samples](https://reader033.vdocuments.mx/reader033/viewer/2022051522/5896eda11a28ab95028bbfa3/html5/thumbnails/29.jpg)
Comparison
Point Samples Line Samples
32
![Page 30: High-Quality Depth of Field using Line Samples](https://reader033.vdocuments.mx/reader033/viewer/2022051522/5896eda11a28ab95028bbfa3/html5/thumbnails/30.jpg)
Comparison
Point Samples Line Samples
33
Metric of Comparison!
![Page 31: High-Quality Depth of Field using Line Samples](https://reader033.vdocuments.mx/reader033/viewer/2022051522/5896eda11a28ab95028bbfa3/html5/thumbnails/31.jpg)
Line Sampling on the GPU
What’s really important to us:
• Easy parallel processing
• Bounded memory requirements
– Shared memory is limited
– Games require fixed storage
![Page 32: High-Quality Depth of Field using Line Samples](https://reader033.vdocuments.mx/reader033/viewer/2022051522/5896eda11a28ab95028bbfa3/html5/thumbnails/32.jpg)
Parallel Implementation: 2 Kernels
35
Intersect line samples with scene geometry, & generate
line segments
Global Line Segment Buffer
Composite and filter segments into final pixels
![Page 33: High-Quality Depth of Field using Line Samples](https://reader033.vdocuments.mx/reader033/viewer/2022051522/5896eda11a28ab95028bbfa3/html5/thumbnails/33.jpg)
Parallel Implementation: 2 Kernels
36
Intersect line samples with scene geometry, & generate
line segments
Global Line Segment Buffer
Composite and filter segments into final pixels
Problem Occurs Here!
![Page 34: High-Quality Depth of Field using Line Samples](https://reader033.vdocuments.mx/reader033/viewer/2022051522/5896eda11a28ab95028bbfa3/html5/thumbnails/34.jpg)
Problem
800 x 600 x 16 x 16 x 24 width height # line
samples # line
segments Bytes per segment
37
![Page 35: High-Quality Depth of Field using Line Samples](https://reader033.vdocuments.mx/reader033/viewer/2022051522/5896eda11a28ab95028bbfa3/html5/thumbnails/35.jpg)
Binning • Split screen into tiles – use shared memory
instead of global memory
• Combine the two kernels into one
38
Global Memory
Shared Memory
![Page 36: High-Quality Depth of Field using Line Samples](https://reader033.vdocuments.mx/reader033/viewer/2022051522/5896eda11a28ab95028bbfa3/html5/thumbnails/36.jpg)
New Problem
• New problem: Shared memory is small.
?
Line Segment Buffer
Line Segments
39
![Page 37: High-Quality Depth of Field using Line Samples](https://reader033.vdocuments.mx/reader033/viewer/2022051522/5896eda11a28ab95028bbfa3/html5/thumbnails/37.jpg)
Line Segments
Solution: Keep only the important line segments!
Line Segments
40
![Page 38: High-Quality Depth of Field using Line Samples](https://reader033.vdocuments.mx/reader033/viewer/2022051522/5896eda11a28ab95028bbfa3/html5/thumbnails/38.jpg)
Heuristics
Only keep segments that contribute the most to the color of the sample
Green Segment Too Small
Green Segment Occluded
41
![Page 39: High-Quality Depth of Field using Line Samples](https://reader033.vdocuments.mx/reader033/viewer/2022051522/5896eda11a28ab95028bbfa3/html5/thumbnails/39.jpg)
Heuristics
Only keep segments that contribute the most to the color of the sample
Green Segment Too Small
Green Segment Occluded
42
Blue Segment Merged
![Page 40: High-Quality Depth of Field using Line Samples](https://reader033.vdocuments.mx/reader033/viewer/2022051522/5896eda11a28ab95028bbfa3/html5/thumbnails/40.jpg)
Heuristics
Rules:
• Discard if segment is too short
• Discard if segment is occluded
• Merge if segments are similar
A small amount of shared memory = small tiles Another Problem!
43
![Page 41: High-Quality Depth of Field using Line Samples](https://reader033.vdocuments.mx/reader033/viewer/2022051522/5896eda11a28ab95028bbfa3/html5/thumbnails/41.jpg)
Bins Overflow
4x4 bins, 256 max triangles 45
![Page 42: High-Quality Depth of Field using Line Samples](https://reader033.vdocuments.mx/reader033/viewer/2022051522/5896eda11a28ab95028bbfa3/html5/thumbnails/42.jpg)
Bins Overflow
4x4 bins, 512 max triangles 46
![Page 43: High-Quality Depth of Field using Line Samples](https://reader033.vdocuments.mx/reader033/viewer/2022051522/5896eda11a28ab95028bbfa3/html5/thumbnails/43.jpg)
Bins Overflow
4x4 bins, 1024 max triangles 47
![Page 44: High-Quality Depth of Field using Line Samples](https://reader033.vdocuments.mx/reader033/viewer/2022051522/5896eda11a28ab95028bbfa3/html5/thumbnails/44.jpg)
Bins Overflow
48
Overflow because of depth
Overflow because of large CoC.
![Page 45: High-Quality Depth of Field using Line Samples](https://reader033.vdocuments.mx/reader033/viewer/2022051522/5896eda11a28ab95028bbfa3/html5/thumbnails/45.jpg)
Bins Overflow
49
Overflow because of depth
Overflow because of large CoC.
![Page 46: High-Quality Depth of Field using Line Samples](https://reader033.vdocuments.mx/reader033/viewer/2022051522/5896eda11a28ab95028bbfa3/html5/thumbnails/46.jpg)
Bins Overflow
50
Overflow because of depth
Overflow because of large CoC.
![Page 47: High-Quality Depth of Field using Line Samples](https://reader033.vdocuments.mx/reader033/viewer/2022051522/5896eda11a28ab95028bbfa3/html5/thumbnails/47.jpg)
Bins Overflow
51
Overflow because of depth
Overflow because of large CoC.
![Page 48: High-Quality Depth of Field using Line Samples](https://reader033.vdocuments.mx/reader033/viewer/2022051522/5896eda11a28ab95028bbfa3/html5/thumbnails/48.jpg)
Level of Detail
52
Close and blurred Low detail
Close and sharp High detail
Far and blurred Low detail
![Page 49: High-Quality Depth of Field using Line Samples](https://reader033.vdocuments.mx/reader033/viewer/2022051522/5896eda11a28ab95028bbfa3/html5/thumbnails/49.jpg)
Level of Detail c / a says, in additional to a traditional LoD scheme: • If object is close but very blurred – choose lower LoD • If object is far but still sharp – choose higher LoD
No LoD LoD Ground Truth
1/3 as many triangles, and 1.6 x improvement in speed
![Page 50: High-Quality Depth of Field using Line Samples](https://reader033.vdocuments.mx/reader033/viewer/2022051522/5896eda11a28ab95028bbfa3/html5/thumbnails/50.jpg)
Final System Structure
54
![Page 51: High-Quality Depth of Field using Line Samples](https://reader033.vdocuments.mx/reader033/viewer/2022051522/5896eda11a28ab95028bbfa3/html5/thumbnails/51.jpg)
Final System Structure
55 Stage 1: Vertex Transform and CoC computation.
![Page 52: High-Quality Depth of Field using Line Samples](https://reader033.vdocuments.mx/reader033/viewer/2022051522/5896eda11a28ab95028bbfa3/html5/thumbnails/52.jpg)
Final System Structure
56 Stage 2: Triangle Binning into Tiles
![Page 53: High-Quality Depth of Field using Line Samples](https://reader033.vdocuments.mx/reader033/viewer/2022051522/5896eda11a28ab95028bbfa3/html5/thumbnails/53.jpg)
Final System Structure
57 Stage 3: Tile Sample, Composite, and Filter
![Page 54: High-Quality Depth of Field using Line Samples](https://reader033.vdocuments.mx/reader033/viewer/2022051522/5896eda11a28ab95028bbfa3/html5/thumbnails/54.jpg)
Results
256 Point Samples 0.49 fps
16 Line Samples 2.04 fps
58
![Page 55: High-Quality Depth of Field using Line Samples](https://reader033.vdocuments.mx/reader033/viewer/2022051522/5896eda11a28ab95028bbfa3/html5/thumbnails/55.jpg)
Results
59
256 Point Samples 0.64 fps
16 Line Samples 2.86 fps
![Page 56: High-Quality Depth of Field using Line Samples](https://reader033.vdocuments.mx/reader033/viewer/2022051522/5896eda11a28ab95028bbfa3/html5/thumbnails/56.jpg)
Drawbacks
Several Problems to the current model
• Heavily dependent on shared memory.
• Not completely accurate – sometimes significant segments may be discarded.
• Shading is done once per line segment.
– Can cause oversmoothing.
60
![Page 57: High-Quality Depth of Field using Line Samples](https://reader033.vdocuments.mx/reader033/viewer/2022051522/5896eda11a28ab95028bbfa3/html5/thumbnails/57.jpg)
Artifacts Point Samples
• Too few point samples causes noise
Line Samples
• Too few line samples causes ghosting
![Page 58: High-Quality Depth of Field using Line Samples](https://reader033.vdocuments.mx/reader033/viewer/2022051522/5896eda11a28ab95028bbfa3/html5/thumbnails/58.jpg)
Convergence Point Samples
• Takes many samples to converge
Line Samples
• Converges much faster than point samples
![Page 59: High-Quality Depth of Field using Line Samples](https://reader033.vdocuments.mx/reader033/viewer/2022051522/5896eda11a28ab95028bbfa3/html5/thumbnails/59.jpg)
Contributions
• Depth-of-field using line sampling techniques
• Implementing tiled line sampler on GPUs
• Blur dependent level of detail
63
![Page 60: High-Quality Depth of Field using Line Samples](https://reader033.vdocuments.mx/reader033/viewer/2022051522/5896eda11a28ab95028bbfa3/html5/thumbnails/60.jpg)
Future Work
• Line sampling in more aggressive stochastic rendering systems
– Area lighting / Soft shadows
– Global Illumination
• Better memory management
– Better heuristics?
64
![Page 61: High-Quality Depth of Field using Line Samples](https://reader033.vdocuments.mx/reader033/viewer/2022051522/5896eda11a28ab95028bbfa3/html5/thumbnails/61.jpg)
Thank you!
65
Special thanks to Intel’s ISTC-VC and NSF for funding!
Alduin from TES V: Skyrim