importance driven volume rendering authors: i. viola, a. kanitsar, m. gr ö ler
DESCRIPTION
Importance Driven Volume Rendering Authors: I. Viola, A. Kanitsar, M. Gr ö ler. Visualization II Instructor: Jessica Crouch. Problem. Volume visualization has inherent difficulties with occlusion Exterior surfaces occlude interior surfaces - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Importance Driven Volume Rendering Authors: I. Viola, A. Kanitsar, M. Gr ö ler](https://reader035.vdocuments.mx/reader035/viewer/2022070404/56813b85550346895da4aee4/html5/thumbnails/1.jpg)
Importance Driven Volume Rendering
Authors: I. Viola, A. Kanitsar, M. Gröler
Visualization II
Instructor: Jessica Crouch
![Page 2: Importance Driven Volume Rendering Authors: I. Viola, A. Kanitsar, M. Gr ö ler](https://reader035.vdocuments.mx/reader035/viewer/2022070404/56813b85550346895da4aee4/html5/thumbnails/2.jpg)
Problem
• Volume visualization has inherent difficulties with occlusion– Exterior surfaces occlude interior surfaces– Interior surfaces might be more interesting or
important
• How do you render a volume so that you see the most important parts of the data?
![Page 3: Importance Driven Volume Rendering Authors: I. Viola, A. Kanitsar, M. Gr ö ler](https://reader035.vdocuments.mx/reader035/viewer/2022070404/56813b85550346895da4aee4/html5/thumbnails/3.jpg)
Motivation
• 3D medical image visualization in general
• Specific example: liver tumor visualization– For radiotherapy planning– For surgery planning
• Need to clearly see:– Tumor location, size, shape– Blood vessel tree– Parenchyma
(functional tissue, as opposed to connective tissue)http://www.vislab.uq.edu.au/research/liver/images/3D_liver_model.jpg
![Page 4: Importance Driven Volume Rendering Authors: I. Viola, A. Kanitsar, M. Gr ö ler](https://reader035.vdocuments.mx/reader035/viewer/2022070404/56813b85550346895da4aee4/html5/thumbnails/4.jpg)
http://www.anatomyatlases.org/atlasofanatomy/plate33/images/33-4_static.jpg http://mbi.dkfz-heidelberg.de/projects/liver/images/vessels.jpg
![Page 5: Importance Driven Volume Rendering Authors: I. Viola, A. Kanitsar, M. Gr ö ler](https://reader035.vdocuments.mx/reader035/viewer/2022070404/56813b85550346895da4aee4/html5/thumbnails/5.jpg)
Methods
• Approaches add visual emphasis to important portions of the scene
– How do you define importance?
– How do you add visual emphasis?
![Page 6: Importance Driven Volume Rendering Authors: I. Viola, A. Kanitsar, M. Gr ö ler](https://reader035.vdocuments.mx/reader035/viewer/2022070404/56813b85550346895da4aee4/html5/thumbnails/6.jpg)
Previous Work: Transfer Functions
• Specify a function (probably non-linear) that maps scalar voxel data to color/opacity– Simplest example: threshold function
– Others?
• Map data through transfer function, then use a standard volume rendering algorithm– MIP (Maximum Intensity Projection)
– Ray casting
– Volume Splatting
![Page 7: Importance Driven Volume Rendering Authors: I. Viola, A. Kanitsar, M. Gr ö ler](https://reader035.vdocuments.mx/reader035/viewer/2022070404/56813b85550346895da4aee4/html5/thumbnails/7.jpg)
Previous Work: Transfer Functions
• Assumes the important parts of the data can be specified using ranges of voxel intensities
• How could this scheme work for the liver?
![Page 8: Importance Driven Volume Rendering Authors: I. Viola, A. Kanitsar, M. Gr ö ler](https://reader035.vdocuments.mx/reader035/viewer/2022070404/56813b85550346895da4aee4/html5/thumbnails/8.jpg)
Previous Work: Focus + Context Rendering• Focus:
– To which part of the scene is the viewer paying attention?
• Eye tracking
– Render that part with more detail, with magnification, or other enhancement
– Let focus define importance
• Context: – The relationship (distance) between a part of the scene
and the focus (focal) point– Detail or magnification gradually fades as distance
increases
![Page 9: Importance Driven Volume Rendering Authors: I. Viola, A. Kanitsar, M. Gr ö ler](https://reader035.vdocuments.mx/reader035/viewer/2022070404/56813b85550346895da4aee4/html5/thumbnails/9.jpg)
Previous Work: Focus + Context Rendering• Has the unique property that the viewer
determines what part of the visualization is emphasized– When is this appropriate?
– How would this work for the liver application?
![Page 10: Importance Driven Volume Rendering Authors: I. Viola, A. Kanitsar, M. Gr ö ler](https://reader035.vdocuments.mx/reader035/viewer/2022070404/56813b85550346895da4aee4/html5/thumbnails/10.jpg)
Previous Work: Methods for Adding Visual Emphasis
• Sparse Representation:– Just render important parts of the data
• Iso-surfaces• Contour lines / curves (approximation of
wireframe)
• Cut-Away Views:– Traditional artist rendering of anatomy,
layered structures– Clipping plane(s) or curved clipping
surface(s)
![Page 11: Importance Driven Volume Rendering Authors: I. Viola, A. Kanitsar, M. Gr ö ler](https://reader035.vdocuments.mx/reader035/viewer/2022070404/56813b85550346895da4aee4/html5/thumbnails/11.jpg)
Importance Driven Rendering
• Basic idea: – Wherever a less important object occludes a
more important object, render the less important object sparsely so that the more important object is mostly visible.
• Assumes importance is user-defined & static• View dependent approach
– Not focus dependent
![Page 12: Importance Driven Volume Rendering Authors: I. Viola, A. Kanitsar, M. Gr ö ler](https://reader035.vdocuments.mx/reader035/viewer/2022070404/56813b85550346895da4aee4/html5/thumbnails/12.jpg)
Necessary Model Data
• Each voxel is assigned to an object– Segmentation, classification
• Each object is assigned a level of importance– In addition to color, opacity
![Page 13: Importance Driven Volume Rendering Authors: I. Viola, A. Kanitsar, M. Gr ö ler](https://reader035.vdocuments.mx/reader035/viewer/2022070404/56813b85550346895da4aee4/html5/thumbnails/13.jpg)
Rendering Logic
• Sparseness level of a surface patch indicates what percentage of its footprint on the image plane it colors
• Sparseness level is assigned to a surface patch based on whether it is more or less important than other surfaces it occludes (or is occluded by)
![Page 14: Importance Driven Volume Rendering Authors: I. Viola, A. Kanitsar, M. Gr ö ler](https://reader035.vdocuments.mx/reader035/viewer/2022070404/56813b85550346895da4aee4/html5/thumbnails/14.jpg)
![Page 15: Importance Driven Volume Rendering Authors: I. Viola, A. Kanitsar, M. Gr ö ler](https://reader035.vdocuments.mx/reader035/viewer/2022070404/56813b85550346895da4aee4/html5/thumbnails/15.jpg)
Importance Compositing
• Option 1: Maximum Importance Projection– For each pixel, render the most important voxel
that projects onto the pixel– Assign sparseness level of 0 or 1 to each point
1
2 3
![Page 16: Importance Driven Volume Rendering Authors: I. Viola, A. Kanitsar, M. Gr ö ler](https://reader035.vdocuments.mx/reader035/viewer/2022070404/56813b85550346895da4aee4/html5/thumbnails/16.jpg)
Importance Compositing: MImP
• Efficient, fast?
• How depth perception work?– Occlusion is an important depth cue
• Naïve method of MImP:
![Page 17: Importance Driven Volume Rendering Authors: I. Viola, A. Kanitsar, M. Gr ö ler](https://reader035.vdocuments.mx/reader035/viewer/2022070404/56813b85550346895da4aee4/html5/thumbnails/17.jpg)
MImP Improved
• Apply a conical rather than cylindrical cut-out to the objects surrounding an important object– Gradual change in rendering depth– Allows you to see cut away layers of surrounding
objects, rather than a deep “hole”
![Page 18: Importance Driven Volume Rendering Authors: I. Viola, A. Kanitsar, M. Gr ö ler](https://reader035.vdocuments.mx/reader035/viewer/2022070404/56813b85550346895da4aee4/html5/thumbnails/18.jpg)
MImP Conical Cut-Out Implementation
1. Find the important object’s footprint on the image plane.
2. For each pixel in the footprint, find the maximum depth of all the object’s voxels that project to the pixel
- Gives objects “back wall”
3. Create a conical clipping surface by projecting the back wall toward the image plane
- Assuming a ray casting approach, just start tracing clipped rays at the clipping surface rather than at the image plane
![Page 19: Importance Driven Volume Rendering Authors: I. Viola, A. Kanitsar, M. Gr ö ler](https://reader035.vdocuments.mx/reader035/viewer/2022070404/56813b85550346895da4aee4/html5/thumbnails/19.jpg)
• Option 2: Average Importance Compositing– Compute sum of object importances along a ray– Each surface point intersected by the ray is
assigned a sparseness level based on the ratio of its importance to the sum of all intersected surface importances
Importance Compositing
![Page 20: Importance Driven Volume Rendering Authors: I. Viola, A. Kanitsar, M. Gr ö ler](https://reader035.vdocuments.mx/reader035/viewer/2022070404/56813b85550346895da4aee4/html5/thumbnails/20.jpg)
Average Importance Compositing
• Efficiency of implementation?– Use binary object footprints
• Artifacts: jaggy transitions– Enlarge and smooth footprints
![Page 21: Importance Driven Volume Rendering Authors: I. Viola, A. Kanitsar, M. Gr ö ler](https://reader035.vdocuments.mx/reader035/viewer/2022070404/56813b85550346895da4aee4/html5/thumbnails/21.jpg)
![Page 22: Importance Driven Volume Rendering Authors: I. Viola, A. Kanitsar, M. Gr ö ler](https://reader035.vdocuments.mx/reader035/viewer/2022070404/56813b85550346895da4aee4/html5/thumbnails/22.jpg)
Levels of Sparseness
• Given a voxel’s level of sparseness, how should it be rendered?– With MImP, render as solid or not at all– With Avg. Importance Compositing, more
choices• Color and Opacity Modulation
• Screen-Door Transparency
• Volume Thinning
![Page 23: Importance Driven Volume Rendering Authors: I. Viola, A. Kanitsar, M. Gr ö ler](https://reader035.vdocuments.mx/reader035/viewer/2022070404/56813b85550346895da4aee4/html5/thumbnails/23.jpg)
• Increase the opacity and possibly saturation of a voxel with increasing importance– Is playing with the color confusing?
Color and Opacity Modulation
![Page 24: Importance Driven Volume Rendering Authors: I. Viola, A. Kanitsar, M. Gr ö ler](https://reader035.vdocuments.mx/reader035/viewer/2022070404/56813b85550346895da4aee4/html5/thumbnails/24.jpg)
Screen-Door Transparency
• Sparse surfaces are rendered as the fibers in a quadrilateral mesh– Higher sparseness level results in larger
(coarser) mesh weave
• Advantages/Disadvantages?– Depth perception– Distraction
![Page 25: Importance Driven Volume Rendering Authors: I. Viola, A. Kanitsar, M. Gr ö ler](https://reader035.vdocuments.mx/reader035/viewer/2022070404/56813b85550346895da4aee4/html5/thumbnails/25.jpg)
Volume Thinning
• Render objects using a set of iso-surfaces
• The number of iso-surface voxels to render per object depends on the sparsity level of the object
![Page 26: Importance Driven Volume Rendering Authors: I. Viola, A. Kanitsar, M. Gr ö ler](https://reader035.vdocuments.mx/reader035/viewer/2022070404/56813b85550346895da4aee4/html5/thumbnails/26.jpg)
Evaluation
• Show rendered results for the Gecko and Monster datasets
• All rendering off-line
• No quantitative evaluation
• No demonstration for the motivating problem: liver
![Page 27: Importance Driven Volume Rendering Authors: I. Viola, A. Kanitsar, M. Gr ö ler](https://reader035.vdocuments.mx/reader035/viewer/2022070404/56813b85550346895da4aee4/html5/thumbnails/27.jpg)
Evaluation• Several methods were presented, it would
be nice to know which work the best• Would require human perception
experiments– Task: plan a cutting path for removal of a liver
tumor that avoids major vessels• Try it using MImP, Opacity Modulation, Screen
Door, and Volume Thinning• Do people perform faster or more reliably using one
of the types? Your prediction?
![Page 28: Importance Driven Volume Rendering Authors: I. Viola, A. Kanitsar, M. Gr ö ler](https://reader035.vdocuments.mx/reader035/viewer/2022070404/56813b85550346895da4aee4/html5/thumbnails/28.jpg)
Evaluation
• Second consideration: – Path planning is an interactive task– Would like to rotate camera & change viewpoint
• Which methods might be implemented in real-time?
![Page 29: Importance Driven Volume Rendering Authors: I. Viola, A. Kanitsar, M. Gr ö ler](https://reader035.vdocuments.mx/reader035/viewer/2022070404/56813b85550346895da4aee4/html5/thumbnails/29.jpg)
Conclusion
• Have not addressed half the problem: how to assign importance to objects– Application dependent, but need automated
methods
![Page 30: Importance Driven Volume Rendering Authors: I. Viola, A. Kanitsar, M. Gr ö ler](https://reader035.vdocuments.mx/reader035/viewer/2022070404/56813b85550346895da4aee4/html5/thumbnails/30.jpg)
Conclusion• Interesting, but incomplete work
– Should be evaluated, validated
• Most interesting idea is MImP with conical cut-out– Don’t have to use ray casting, although the authors do
– Consider volume splatting
• Avg. Importance Compositing methods likely to remain off-line rendering options for the near future
![Page 31: Importance Driven Volume Rendering Authors: I. Viola, A. Kanitsar, M. Gr ö ler](https://reader035.vdocuments.mx/reader035/viewer/2022070404/56813b85550346895da4aee4/html5/thumbnails/31.jpg)
Discussion Questions
Consider applying this to visualization of a building volume (3D blueprints).
1. How might you assign importance?
2. Efficiency considerations aside, which of the rendering methods would be most visually effective?
![Page 32: Importance Driven Volume Rendering Authors: I. Viola, A. Kanitsar, M. Gr ö ler](https://reader035.vdocuments.mx/reader035/viewer/2022070404/56813b85550346895da4aee4/html5/thumbnails/32.jpg)
Discussion Questions
What if you had a dataset that included air flow velocity for the volume, and you were interested in understanding the operation of the circulation system?
1. How might you assign importance?2. Efficiency considerations aside, which of the rendering methods would be most visually effective?
![Page 33: Importance Driven Volume Rendering Authors: I. Viola, A. Kanitsar, M. Gr ö ler](https://reader035.vdocuments.mx/reader035/viewer/2022070404/56813b85550346895da4aee4/html5/thumbnails/33.jpg)
Discussion Questions
• Implementation:– Could any of the methods described be
implemented using the traditional forward-rendering graphics pipeline?
• How, or why not?
– What about implementation with a shading language?
• A Z-Buffer type of approach for importance sorting?