PowerPoint Presentation

Hierarchical Exploration of Volumes Using Multilevel Segmentation of the Intensity-Gradient HistogramsCheuk Yiu IpAmitabh VarshneyJoseph JaJa

Volume Exploration ChallengeRaw Volume Data CubeMeaningful Visualization[Voreen CG&A 09]

Similar to other papers in this sessionGiven a volumetric data cube on the left. We want to generate a meaningful visualization on the right.2Transfer Function EvolutionHow do we find the right transfer function?

RGBAIntensityTraditionally, we use a transfer function to control the appearance of a visualization, the transfer function may look pretty complex.Finding the right transfer function has always been a challenge in scientific visualization

3Histogram Helps Transfer Function Design

[Drebin et al. SIGGRAPH 88]To assist transfer function design, one of the first volume rendering papers shows a histogram may be helpful.4Histogram Helps Transfer Function Design

[Drebin et al. SIGGRAPH 88]The peaks of the histogram are composed by the distributions of different constituents5Histogram Helps Transfer Function Design

[Drebin et al. SIGGRAPH 88]By assigning different colors and opacities to these components. we can visualize different materials6Histogram Helps Transfer Function Design

For example, by following the peak in the histogram we can obtain this visualization of the cadaver head dataset7Histograms May Not Always Help

However, the separation of the distributions may not be always clear8Volume ExplorationExhaustively explore the dataset

In this case, we have to exhaustively explore the dataset by modifying the transfer function 9

Volume ExplorationSeek for salient featuresWe seek for the salient features one by one10

Volume ExplorationSeek for salient features11

Volume ExplorationSeek for salient features12

Volume ExplorationFeature locations can be arbitraryThe meaningful feature locations can be pretty arbitrary132D Transfer FunctionsGradient ( f(x) ) captures boundariesHistogram shapes Volume segments

IntensityGradient[Kindlmann & Durkin VolVis98,Kniss et al. TVCG 02]To assist the search for meaningful and salient features, the transfer function has been extended to 2D by incorporating gradient.We plot a histogram in 2D with the intensity on the x axis, gradient on the y axis, the darkness of the shade represents the frequency

The gradient helps capturing material boundaries.

The amazing insight here is shapes on the 2D intensity-gradient histogram correspond to meaningful volumetric segments.This has been implemented in popular visualization packages such as voreen Imagevis3d and VisIt14Advances on New AttributesHigher Derivatives: f(x) [Kindlmann & Durkin VolVis98]Specific features:Size [Correa and Ma TVCG 08]LH-transform [Sereda et al. TVCG 06],Domain specific semantic attributes [Salama et al. TVCG 06]Select good views:Visibility [Correa and Ma TVCG 10], Information divergence [Ruiz et al. TVCG 11]Many subsequent work has introduced new attributes such as higher derivatives, size, LH Transform, and even domain specific semantic attributes

Visibility and information divergence have been used to select good views.

For this work we will go back to intensity and gradient15

Challenges of the 2D Transfer FunctionSearch for separated meaningful features1 Region 1 MinuteLets take a look at the use of a 2D transfer functionUsers draw shapes on the 2D histogram to extract meaningful componentsThis video shows how a user works with the histogram in ImageVis3D

The arc structures can clearly be seen on the intensity gradient histogramThe user is trying to use a polygon widget to highlight the feature that corresponds to the top arc.

As we can see, manipulating the widget to separate the crown from the root takes quite some effort.The challenge here is the histogram does not show where do the good features separate.

Extracting this crown takes almost 1 min16Histogram and Volume Features

Lets take a look at the vismale dataset17

Histogram and Volume FeaturesSkin

Histogram and Volume FeaturesFlesh

Histogram and Volume FeaturesSkull

Histogram and Volume FeaturesSinus

Histogram and Volume FeaturesTeeth

We can see that meaningful components do correspond to histogram segments.In this talk, my goal is to show you how to discover these regions in a systematic manner.22Approximate Histogram Transfer Functions

[Wang et al. TVCG 2011]Existing approaches directly or indirectly fit the histogram

User Specified [Kniss et al. TVCG 02,Fogal et al. 2010]Existing approaches directly or indirectly fit the 2D histogram to assist transfer function designThese primitives may not cover the whole histogram completely to guarantee exhaustive exploration23Reduce Search to ClassificationRecursive histogram classificationTight coverage with a few segmentsExhaustive exploration

In this work, we attempt to reduce this search to classification by recursively segmenting the histogram.We introduce a visual-inspired approach that tightly fits the histogram with a few segments.We will show how to interactively and exhaustively explore the whole dataset24OverviewSegment the histogram statisticsBuild an exhaustive multilevel hierarchyUser interactive exploration

Given a volume dataset, we mimic the user behavior by computationally segmenting the histogram statistics of the volume, we build an exhaustive multilevel hierarchy and use it to provide interactive exploration.

Our approach provides a few advantagesVisual segmentation matches user intuition, we guarantee tight and complete coverage, Our approach augments the intensity-gradient feature space without requiring the users to learn any new features25OverviewVisual segmentation matches user intuitionComplete coverageUsers stay in a familiar feature space

Given a volume dataset, we mimic the user behavior by computationally segmenting the histogram statistics of the volume, we build an exhaustive multilevel hierarchy and use it to provide interactive exploration.

Our approach provides a few advantagesVisual segmentation matches user intuition, we guarantee tight and complete coverage, Our approach augments the intensity-gradient feature space without requiring the users to learn any new features26Intensity-gradient Histogram Users implicitly recognize shapes from the histogramSegment this histogram as an image

IntensityGradient

The intensity gradient histogram is very effective becauseHuman users can extract volume segments by recognizing the arcs and blobsGiven these histograms, can we extract these shapes automatically?

We mimic this user behavior by applying computer vision techniques to segment the histogram as an image in a top-down fashion 27Normalized-cut Image SegmentationNormalized-cut (ncut) image segmentation [Shi & Malik PAMI 98]Min ncut produces balanced segments Eigenanalysis approximates the min ncut for k segmentsB

[Wang et al. PATTERN RECOGN LETT 06]Instead of directly segmenting the volume data.We segment the histogram by using normalized-cut to find shapesThis example shows the normalized cut provides intuitive segments on natural images. Normalize cut models the image as a graph, with the similarly of pixels as edge weightsThe weight of a cut in the graph is the total weights along the separating edges.shi and malik normalize this cut by the weights of the associated segments in the graph.The minimum normalize cut favors balanced segmentsWe find the top k eigenvectors to approximate this minimum cut for k different segments28Normalized-cut on Intensity-gradient Histogram

We apply normalized-cut on 2562 8-bit histogramsk=2 separates the tooth from the volume boxk=10 shows segments of the tooth crown and rootk=20 shows different material boundariesWe apply normalized cut to segment 256 sq 8 bit histogram images.Eigenanalysis on these images only takes a few seconds When k = 2 the tooth is separated from the volume box k=10 shows segments of the tooth crown and rootk = 20 shows different material boundaries. These segments tightly cover the whole histogram.

So how many segments should we choose to compose our visualization?29Which k should we pick ?Iteratively picking k is tediousIncreasing k may not subdivide region of user interest

Try k = 2, 3, 4,

Iteratively test and pick k is time consuming and yet increasing k may not subdivide regions of interest.For example when we increase k from 4 to 5 , the box is subdivided. We believe users would prefer subdividing the tooth.30Replace k with User-driven Exploration

Multilevel Segmentation Hierarchy: Apply normalized-cut recursivelywe need to allow users to select the appropriate segments instead of choosing a specific k, In order to accomplish this, we build a hierarchy by recursively applying the normalized cut.31

Multilevel Segmentation HierarchySelectively inspect segments of choiceThis hierarchy leads the users to traverse the dataset, selectively subdivide and inspect segments of their choice32

Multilevel Segmentation Hierarchy[Xia & Varshney Vis 96, Hoppe SIGGRAPH 97, Luebke & Erikson SIGGRAPH 97]Any cut guarantees complete coverageView-dependent LoD hierarchiesAny cut along this hierarchy guarantees complete coverage.

This is similar to view dependent level of details hierarchy33Multilevel Segmentation Hierarchy

Having this flexibility, the next question is which segment should we subdivide?34Information Guided TraversalSegment entropy

High entropy Complex segment

Entropy2.86.8We evaluate information content of the segments to provide some guidance

We use red to show which segment has a higher entropy

for example the entropy of the tooth is higher than the volume box, suggesting the tooth is more complex, we subdivide the tooth.

35

What if the Entropies are SimilarThe segment entropies can be similar

Which segment should we divide next?Use Information Gain

EntropyBut, It is possible that the segment entropies are similar. For example there is no difference for the 2 segments in the right arc.

Which segment should we divide next ?We use info gain to answer this question

36Information-Gain Guided Traversal

Information Gain = Entropy reduction after a subdivision

High Information Gain Structural separation

InformationGain

0.110.01we alternatively evaluate the information gain of a subdivisionThe information gain is defined as the entropy reduction of a subdivisionSubdivisions with high information gain suggest separations of structures.

We use blue to represent the information gain, for example splitting the lower segment with high information gain separates the surfaces of the tooth crown and dentine37

Interactive ExplorationExplore the segmentation hierarchySelective expansionInteractive visualizationsExhaustively explore the tooth in 1 minute

Here we show how a user can interactively explore the segmentation hierarchy, the user selectively click on the histogram to expand the segments and interactively compose visualizationsNotice that the tooth crown in the upper arc can be identified in only 3 expansions.Users iteratively expand the regions of interest and hide uninteresting regions.A visualization of the volume dataset is composed along this hierarchy traversalIn addition to identifying different meaningful segments, users can also segment and hide the undesired noise

In contrast with extracting a single region. We can now exhaustively explore the whole tooth in 1 min.38ExamplesEngine blockVisible Human Male Head

Lets take a look at some results

Internal componentsEngine block and surface

Skull, flesh, skin, highlight sinus and teeth39ExamplesTomatoHurricane Isabel

TomatoSkin, Chamber, Center Column

Isabel pressure, with no distinct material boundariesEye, Eye Wall, Rain band, and further segment the eye

40ConclusionsComputational segmentation mimics user interactionsIntuitive volumetric classification Exhaustive multilevel hierarchyInformation guided traversalInteractive exploration

In conclusion, we have shown the idea of using computational visual segmentation to effectively mimic user interaction.It classifies intuitive volumetric regions.We have built an exhaustive multilevel hierarchy from these segmentsand We have provided information content to guide the hierarchy traversalUsers can interactively explore and visualize with these ingredients.41Future WorkImprove the information content measuresAutomatic color assignment for segmentsSegment histograms with different attributesTime varying datasetsAcknowledgementsNational Science Foundation: CCF 05-41120, CMMI 08-35572, CNS 09-59979NVIDIA CUDA Center of Excellence ProgramDerek Juba, Sujal Bista, Yang Yang, M. Adil Yalcin, and the reviewers for improving this paper and presentationThe SciVis Best Paper Award committeeThank you!

Questions ?Source code for building the hierarchy:www.cs.umd.edu/~ipcy/software/volsegtree/

Papers and videos:Cheuk Yiu Ipwww.cs.umd.edu/~ipcy/ GVIL Research Highlightswww.cs.umd.edu/gvil/

44