interactive level-set surface deformation on the gpu aaron lefohn university of california, davis
TRANSCRIPT
![Page 1: Interactive Level-Set Surface Deformation on the GPU Aaron Lefohn University of California, Davis](https://reader036.vdocuments.mx/reader036/viewer/2022062423/56649ea35503460f94ba7ca7/html5/thumbnails/1.jpg)
Interactive Level-Set Surface Interactive Level-Set Surface Deformation on the GPUDeformation on the GPU
Aaron Lefohn University of California, Davis
![Page 2: Interactive Level-Set Surface Deformation on the GPU Aaron Lefohn University of California, Davis](https://reader036.vdocuments.mx/reader036/viewer/2022062423/56649ea35503460f94ba7ca7/html5/thumbnails/2.jpg)
Problem StatementProblem Statement• Goal
– Interactive system for deformable surface manipulation– Level-sets
• Challenges– Deformation is slow– Deformation is hard to control
• Solution– Accelerate level-set computation with GPU– Visualize computation in real-time
![Page 3: Interactive Level-Set Surface Deformation on the GPU Aaron Lefohn University of California, Davis](https://reader036.vdocuments.mx/reader036/viewer/2022062423/56649ea35503460f94ba7ca7/html5/thumbnails/3.jpg)
CollaboratorsCollaborators
University of Utah
Joe Kniss
Joshua Cates
Charles Hansen
Ross Whitaker
![Page 4: Interactive Level-Set Surface Deformation on the GPU Aaron Lefohn University of California, Davis](https://reader036.vdocuments.mx/reader036/viewer/2022062423/56649ea35503460f94ba7ca7/html5/thumbnails/4.jpg)
Overview Overview • Why?
– Motivation and previous work
• How?
– Streaming level-set algorithm– Real-time visualization– Segmentation application
• Does it work?
– Demonstration– User study
![Page 5: Interactive Level-Set Surface Deformation on the GPU Aaron Lefohn University of California, Davis](https://reader036.vdocuments.mx/reader036/viewer/2022062423/56649ea35503460f94ba7ca7/html5/thumbnails/5.jpg)
Deformable SurfacesDeformable Surfaces
• Applications of Level-Sets– Fluid simulation– Surface reconstruction for 3D scanning– Surface processing– Image / Volume segmentation
Introduction
![Page 6: Interactive Level-Set Surface Deformation on the GPU Aaron Lefohn University of California, Davis](https://reader036.vdocuments.mx/reader036/viewer/2022062423/56649ea35503460f94ba7ca7/html5/thumbnails/6.jpg)
Fluid SimulationFluid Simulation
Premoze et al., Eurographics, 2003
Introduction
![Page 7: Interactive Level-Set Surface Deformation on the GPU Aaron Lefohn University of California, Davis](https://reader036.vdocuments.mx/reader036/viewer/2022062423/56649ea35503460f94ba7ca7/html5/thumbnails/7.jpg)
Surface ProcessingSurface Processing
Tasdizen et al., IEEE Visualization, 2002
Introduction
![Page 8: Interactive Level-Set Surface Deformation on the GPU Aaron Lefohn University of California, Davis](https://reader036.vdocuments.mx/reader036/viewer/2022062423/56649ea35503460f94ba7ca7/html5/thumbnails/8.jpg)
Volume SegmentationVolume Segmentation
Lefohn et al., IEEE Visualization, 2003
Introduction
![Page 9: Interactive Level-Set Surface Deformation on the GPU Aaron Lefohn University of California, Davis](https://reader036.vdocuments.mx/reader036/viewer/2022062423/56649ea35503460f94ba7ca7/html5/thumbnails/9.jpg)
• Implicit surface–
• Distance transform
– denotes inside/outside
• Surface motion
– – F = Signed speed in direction of normal
Level-Set MethodLevel-Set Method
Introduction
![Page 10: Interactive Level-Set Surface Deformation on the GPU Aaron Lefohn University of California, Davis](https://reader036.vdocuments.mx/reader036/viewer/2022062423/56649ea35503460f94ba7ca7/html5/thumbnails/10.jpg)
CPU Level-Set AccelerationCPU Level-Set Acceleration
InitializeDomain
ComputeUpdateDomain
• Narrow-Band/Sparse-Grid– Compute PDE only near the surface
• Adalsteinson et al. 1995• Whitaker et al. 1998• Peng et al. 1999
• Time-dependent, sparse-grid solver
Introduction
![Page 11: Interactive Level-Set Surface Deformation on the GPU Aaron Lefohn University of California, Davis](https://reader036.vdocuments.mx/reader036/viewer/2022062423/56649ea35503460f94ba7ca7/html5/thumbnails/11.jpg)
GPU Level-Set AccelerationGPU Level-Set Acceleration
• Strzodka et al. 2001– 2D level-set solver on NVIDIA GeForce 2– No narrow-band optimization
• Lefohn et al. 2002– Brute force 3D implementation on ATI Radeon 8500– 1x – 2x faster than CPU, but 10x more computations– No narrow-band optimization
Introduction
![Page 12: Interactive Level-Set Surface Deformation on the GPU Aaron Lefohn University of California, Davis](https://reader036.vdocuments.mx/reader036/viewer/2022062423/56649ea35503460f94ba7ca7/html5/thumbnails/12.jpg)
Overview Overview • Why?
– Motivation and previous work
• How?
– Streaming level-set algorithm– Real-time visualization– Segmentation application
• Does it work?
– Demonstration– User study
![Page 13: Interactive Level-Set Surface Deformation on the GPU Aaron Lefohn University of California, Davis](https://reader036.vdocuments.mx/reader036/viewer/2022062423/56649ea35503460f94ba7ca7/html5/thumbnails/13.jpg)
• Sparse Volume Computation– CPU algorithm: Traverse linked list of active voxels– GPU algorithm: Compute all active voxels in parallel
• Data structures change after each PDE time step
GPU Narrow-Band SolverGPU Narrow-Band Solver
InitializeDomain
ComputeUpdateDomain
Algorithm
![Page 14: Interactive Level-Set Surface Deformation on the GPU Aaron Lefohn University of California, Davis](https://reader036.vdocuments.mx/reader036/viewer/2022062423/56649ea35503460f94ba7ca7/html5/thumbnails/14.jpg)
GPU-Based Narrow-Band Level-Set Solver
• Goals1 Leverage GPU parallelism2 Minimize unnecessary computation 2 Minimize GPU memory usage3 Fast update of sparse data structures4 Interactive visualization
Algorithm GoalsAlgorithm Goals
Algorithm
![Page 15: Interactive Level-Set Surface Deformation on the GPU Aaron Lefohn University of California, Davis](https://reader036.vdocuments.mx/reader036/viewer/2022062423/56649ea35503460f94ba7ca7/html5/thumbnails/15.jpg)
Algorithm SolutionsAlgorithm Solutions
• Pack Active Voxels Into 2D Texture– Increase parallelism, reduce computation and memory use
• Efficient GPU-to-CPU Message Passing– Fast update of packed data structure
• PDE-Based Distance Transform– Increase parallelism
• On-The-Fly Decompression Volume Rendering– Interactive visualization without increasing memory use
Algorithm
![Page 16: Interactive Level-Set Surface Deformation on the GPU Aaron Lefohn University of California, Davis](https://reader036.vdocuments.mx/reader036/viewer/2022062423/56649ea35503460f94ba7ca7/html5/thumbnails/16.jpg)
• Multi-Dimensional Virtual Memory– 3D virtual memory– 2D physical memory– 16 x 16 pixel pages
A Dynamic, SparseGPU Data StructureA Dynamic, SparseGPU Data Structure
Algorithm
Virtual Memory Space Physical Memory Space
Unused Pages
Outside Active PagesInside
![Page 17: Interactive Level-Set Surface Deformation on the GPU Aaron Lefohn University of California, Davis](https://reader036.vdocuments.mx/reader036/viewer/2022062423/56649ea35503460f94ba7ca7/html5/thumbnails/17.jpg)
A Dynamic, SparseGPU Data StructureA Dynamic, SparseGPU Data Structure
• GPU: Does the Real Work– Level-set computation (2D physical memory)– Issues memory requests
• CPU: Manages memory– Memory manager– Page table (3D virtual memory)
Algorithm
CPU GPU
Physical Addresses for Active Memory Pages
Memory Requests
PDE Computation
15-250 passes
![Page 18: Interactive Level-Set Surface Deformation on the GPU Aaron Lefohn University of California, Davis](https://reader036.vdocuments.mx/reader036/viewer/2022062423/56649ea35503460f94ba7ca7/html5/thumbnails/18.jpg)
A Dynamic, SparseGPU Data StructureA Dynamic, SparseGPU Data Structure
• Problem– Neighbor lookups across page boundaries– Branching slow on GPU
• Solution– Substreams
• Create homogeneous data streams• Resolve conditionals with geometry• Goodnight 2003, Harris 2003, Lefohn 2003• Optimizes cache and pre-fetch performance
– Kapasi et al., Micro 33, 2000
Algorithm
![Page 19: Interactive Level-Set Surface Deformation on the GPU Aaron Lefohn University of California, Davis](https://reader036.vdocuments.mx/reader036/viewer/2022062423/56649ea35503460f94ba7ca7/html5/thumbnails/19.jpg)
GPU-to-CPUMessage Passing
GPU-to-CPUMessage Passing
• Problem: Active Voxel Set is Time-Dependent– GPU memory request mechanism– Low bandwidth GPU-to-CPU communication
• Solution– Compress GPU memory request– Use GPU computation to save GPU-to-CPU bandwidth
Algorithm
Mipmapping
s +x -x +y -y +z -z
![Page 20: Interactive Level-Set Surface Deformation on the GPU Aaron Lefohn University of California, Davis](https://reader036.vdocuments.mx/reader036/viewer/2022062423/56649ea35503460f94ba7ca7/html5/thumbnails/20.jpg)
Algorithm SolutionsAlgorithm Solutions
• Pack Active Voxels Into 2D Texture– Increase parallelism, reduce computation and memory use
• Efficient GPU-to-CPU Message Passing– Fast update of packed data structure
• PDE-Based Distance Transform– Increase parallelism
• On-The-Fly Decompression Volume Rendering– Interactive visualization without increasing memory use
Algorithm
![Page 21: Interactive Level-Set Surface Deformation on the GPU Aaron Lefohn University of California, Davis](https://reader036.vdocuments.mx/reader036/viewer/2022062423/56649ea35503460f94ba7ca7/html5/thumbnails/21.jpg)
Distance Transform on GPUDistance Transform on GPU
• Problem: Where is the narrow band?– Use voxel values to indicate distance from surface– Traditional methods not data-parallel
• Solution– Include distance transform in level-set PDE– Similar to Fedkiw et al. ,J. Comp. Phys., 1999
Algorithm
![Page 22: Interactive Level-Set Surface Deformation on the GPU Aaron Lefohn University of California, Davis](https://reader036.vdocuments.mx/reader036/viewer/2022062423/56649ea35503460f94ba7ca7/html5/thumbnails/22.jpg)
Overview Overview
• Why?
– Motivation and previous work
• How?
– Streaming level-set algorithm– Real-time visualization (with Joe Kniss)– Segmentation application
• Does it work?
– Live demonstration– User study
![Page 23: Interactive Level-Set Surface Deformation on the GPU Aaron Lefohn University of California, Davis](https://reader036.vdocuments.mx/reader036/viewer/2022062423/56649ea35503460f94ba7ca7/html5/thumbnails/23.jpg)
Direct Volume Rendering of Level Set
Direct Volume Rendering of Level Set
• Reconstruct 2D slice of virtual memory space– On-the-fly decompression on GPU– Use 2D geometry and texture coordinates
Visualization
![Page 24: Interactive Level-Set Surface Deformation on the GPU Aaron Lefohn University of California, Davis](https://reader036.vdocuments.mx/reader036/viewer/2022062423/56649ea35503460f94ba7ca7/html5/thumbnails/24.jpg)
Direct Volume Rendering of Level Set
Direct Volume Rendering of Level Set
• Fully general volume rendering of compressed data– 2D slice-based rendering: No data duplication– Tri-linear interpolation– Full transfer function and lighting capabilities
Visualization
![Page 25: Interactive Level-Set Surface Deformation on the GPU Aaron Lefohn University of California, Davis](https://reader036.vdocuments.mx/reader036/viewer/2022062423/56649ea35503460f94ba7ca7/html5/thumbnails/25.jpg)
Overview Overview
• Why?
– Motivation and previous work
• How?
– Streaming level-set algorithm– Real-time visualization– Segmentation application
• Does it work?
– Demonstration– User study
![Page 26: Interactive Level-Set Surface Deformation on the GPU Aaron Lefohn University of California, Davis](https://reader036.vdocuments.mx/reader036/viewer/2022062423/56649ea35503460f94ba7ca7/html5/thumbnails/26.jpg)
Level-Set Segmentation Application
Level-Set Segmentation Application
• Idea: Segment surface from 3D image– Begin with “seed” surface – Deform surface into target segmentation
Application
![Page 27: Interactive Level-Set Surface Deformation on the GPU Aaron Lefohn University of California, Davis](https://reader036.vdocuments.mx/reader036/viewer/2022062423/56649ea35503460f94ba7ca7/html5/thumbnails/27.jpg)
• Velocity attracts level set to desired feature• Segmentation Parameters
1. Intensity value of interest (center)2. Width of intensity interval (variance)3. Percentage of data vs. smoothing
Segmentation ApplicationSegmentation Application
Application
![Page 28: Interactive Level-Set Surface Deformation on the GPU Aaron Lefohn University of California, Davis](https://reader036.vdocuments.mx/reader036/viewer/2022062423/56649ea35503460f94ba7ca7/html5/thumbnails/28.jpg)
Threshold Speed TermThreshold Speed Term
• Attract level set to range of voxel intensities
Width (Variance) Center (Mean)
Application
![Page 29: Interactive Level-Set Surface Deformation on the GPU Aaron Lefohn University of California, Davis](https://reader036.vdocuments.mx/reader036/viewer/2022062423/56649ea35503460f94ba7ca7/html5/thumbnails/29.jpg)
Curvature Speed TermCurvature Speed Term
• Enforce surface smoothness– Prevent segmentation “leaks”– Smooth noisy solution
Seed Surface No Curvature With Curvature
Application
![Page 30: Interactive Level-Set Surface Deformation on the GPU Aaron Lefohn University of California, Davis](https://reader036.vdocuments.mx/reader036/viewer/2022062423/56649ea35503460f94ba7ca7/html5/thumbnails/30.jpg)
Overview Overview
• Why?
– Motivation and previous work
• How?
– Streaming level-set algorithm– Real-time visualization– Segmentation application
• Does it work?
– Demonstration– User study
![Page 31: Interactive Level-Set Surface Deformation on the GPU Aaron Lefohn University of California, Davis](https://reader036.vdocuments.mx/reader036/viewer/2022062423/56649ea35503460f94ba7ca7/html5/thumbnails/31.jpg)
GPU Narrow-Band Performance
GPU Narrow-Band Performance
• Performance– 10x – 15x faster than optimized CPU version (Insight Toolkit)– Linear dependence on size of narrow band
• Bottlenecks– Fragment processor– Conservative time step
• Need for global accumulation register (min, max, sum, etc.)
Results
![Page 32: Interactive Level-Set Surface Deformation on the GPU Aaron Lefohn University of California, Davis](https://reader036.vdocuments.mx/reader036/viewer/2022062423/56649ea35503460f94ba7ca7/html5/thumbnails/32.jpg)
2D Narrow-Band Example2D Narrow-Band Example
Results
![Page 33: Interactive Level-Set Surface Deformation on the GPU Aaron Lefohn University of California, Davis](https://reader036.vdocuments.mx/reader036/viewer/2022062423/56649ea35503460f94ba7ca7/html5/thumbnails/33.jpg)
DemoDemo
• Segmentation of MRI volumes– 1283 scalar volume
• Hardware Details– ATI Radeon 9800 Pro– 2.4 GHz Intel Xeon– 1 GB of RAM
Results
![Page 34: Interactive Level-Set Surface Deformation on the GPU Aaron Lefohn University of California, Davis](https://reader036.vdocuments.mx/reader036/viewer/2022062423/56649ea35503460f94ba7ca7/html5/thumbnails/34.jpg)
Region-of-Interest Volume RenderingRegion-of-Interest Volume Rendering
• Limit extent of volume rendering– Use level-set segmentation to specify region– Add level-set value to transfer function
Center for In Vivo Microscopy
Duke University
Results
![Page 35: Interactive Level-Set Surface Deformation on the GPU Aaron Lefohn University of California, Davis](https://reader036.vdocuments.mx/reader036/viewer/2022062423/56649ea35503460f94ba7ca7/html5/thumbnails/35.jpg)
Overview Overview
• Why?
– Motivation and previous work
• How?
– Streaming level-set algorithm– Real-time visualization– Segmentation application
• Does it work?
– Demonstration– User study (with Josh Cates)
![Page 36: Interactive Level-Set Surface Deformation on the GPU Aaron Lefohn University of California, Davis](https://reader036.vdocuments.mx/reader036/viewer/2022062423/56649ea35503460f94ba7ca7/html5/thumbnails/36.jpg)
Evaluation User StudyEvaluation User Study
• Goal– Can a user quickly find parameter settings to create an
accurate, precise 3D segmentation?
– Relative to hand contouring
Evaluation
![Page 37: Interactive Level-Set Surface Deformation on the GPU Aaron Lefohn University of California, Davis](https://reader036.vdocuments.mx/reader036/viewer/2022062423/56649ea35503460f94ba7ca7/html5/thumbnails/37.jpg)
User Study MethodologyUser Study Methodology
• Six users and nine data sets– Harvard Brigham and Women’s Hospital Brain Tumor
Database– 256 x 256 x 124 MRI
• No pre-processing of data & no hidden parameters
• Ground truth – Expert hand contouring– STAPLE method (Warfield et al. MICCAI 2002)
Evaluation
![Page 38: Interactive Level-Set Surface Deformation on the GPU Aaron Lefohn University of California, Davis](https://reader036.vdocuments.mx/reader036/viewer/2022062423/56649ea35503460f94ba7ca7/html5/thumbnails/38.jpg)
User Study ResultsUser Study Results
• Efficiency– 6 ± 3 minutes per segmentation (vs multiple hours)– Solver idle 90% - 95% of time
• Precision– Intersubject similarity significantly better– 94.04% ± 0.04% vs. 82.65% ± 0.07%
• Accuracy– Within error bounds of expert hand segmentations – Bias towards smaller segmentations– Compares well with other semi-automatic techniques
– Kaus et al., Radiology, 2001
Evaluation
![Page 39: Interactive Level-Set Surface Deformation on the GPU Aaron Lefohn University of California, Davis](https://reader036.vdocuments.mx/reader036/viewer/2022062423/56649ea35503460f94ba7ca7/html5/thumbnails/39.jpg)
SummarySummary
• Interactive Level-Set System– 10x – 15x speedup over optimized CPU implementation– Segmentation application– Intuitive parameter tuning– User study
• Algorithm Developments– Multi-dimensional virtual memory– Substreams– PDE-based distance transform– Message passing– Volume rendering packed data
Conclusions
![Page 40: Interactive Level-Set Surface Deformation on the GPU Aaron Lefohn University of California, Davis](https://reader036.vdocuments.mx/reader036/viewer/2022062423/56649ea35503460f94ba7ca7/html5/thumbnails/40.jpg)
Future DirectionsFuture Directions
• Other Level-Set Applications– Surface processing, surface reconstruction, physical simulation
• Integrate GPGPU Code Into Open Source Software– The Insight Toolkit (www.itk.org)?
• “Interactive Visulation”– GPGPU allows for simultaneous visualization and simulation– What problems can be solved with “interactive visulation?”– What is the user interface for a visulation?
Conclusions
![Page 41: Interactive Level-Set Surface Deformation on the GPU Aaron Lefohn University of California, Davis](https://reader036.vdocuments.mx/reader036/viewer/2022062423/56649ea35503460f94ba7ca7/html5/thumbnails/41.jpg)
AcknowledgementsAcknowledgements• Joe Kniss – Volume rendering
• Josh Cates – Tumor user study
• Gordon Kindlmann – “Teem” raster-data toolkit
• Milan Ikits – “Glew” OpenGL extension wrangler
• Ross Whitaker, Charles Hansen, Steven Parker, and John Owens
• Utah CS, SCI, and CIPIC faculty, students, and staff
• ATI: Evan Hart, Mark Segal, Jeff Royle, and Jason Mitchell
• Brigham and Women’s Hospital, Duke University, UCSD
• National Science Foundation Graduate Fellowship
• Office of Naval Research grant #N000140110033
• National Science Foundation grant #ACI008915 and #CCR0092065
![Page 42: Interactive Level-Set Surface Deformation on the GPU Aaron Lefohn University of California, Davis](https://reader036.vdocuments.mx/reader036/viewer/2022062423/56649ea35503460f94ba7ca7/html5/thumbnails/42.jpg)
Questions?For More Information
http://graphics.cs.ucdavis.edu/~lefohn/
Google “Lefohn level set”
Journal Papers Based on this Work
Lefohn, Kniss, Hansen, Whitaker, “A Streaming Narrow Band Algorithm: Interactive Computation and Visualization of Level Sets,” IEEE Transactions on Visualization and Computer Graphics, to appear 2004.
Cates, Lefohn, Whitaker, “GIST: An Interactive, GPU-Based Level-Set Segmentation Tool for 3D Medical Images,” Medical Image Analysis, to appear 2004